* 4.6:
CLOUDSTACK-9075 - Uses the same vlan since it should have been already released
CLOUDSTACK-9075 - Adds VPC static routes test
CLOUDSTACK-9075 - Covers Private GW ACL with Redundant VPCs
CLOUDSTACK-9075 - Add method to get list of Physical Networks per zone
CLOUDSTACK-6276 Removing unused parameter in integration test for projects
CLOUDSTACK-6276 Removing unused parameter in integration test
CLOUDSTACK-6276 Fixing affinity groups for projects
We need to send an Unsolicited ARP to the gateway, instead of our own address. We now encounter problems when people deploy/destroy/deploy and get the same public ip.
CLOUDSTACK-9062: Improve S3 implementation.The S3 implementation is far from finished, this commit focuses on the bases.
- Upgrade AWS SDK to latest version.
- Rewrite S3 Template downloader.
- Rewrite S3Utils utility class.
- Improve addImageStoreS3 API command.
- Split various classes for convenience.
- Various minor improvements and code optimizations.
A side effect of the new AWS SDK is that it, by default, uses the V4 signature. Therefore I added an option to specify the Signer, so it stays compatible with previous versions.
Please review thoroughly, both code inspection and (automated) integration tests. Currently no integration tests are available specifically for S3. Therefore the implementation is needed to be tested manually, for now...
What I tested:
- Greenfield install -> will download latest systemvm template automatically to S3.
- Upload a template/iso
- Download a template/iso
- Restart of management server -> list available templates -> doesn't download them again if available.
* pr/1083:
CLOUDSTACK-9062: Improve S3 implementation.
Signed-off-by: Remi Bergsma <github@remi.nl>
* 4.6:
CLOUDSTACK-9015 - Delete public IP in order to get both IP and NAT rule removed.
CLOUDSTACK-9015 - Add test to cover the rVPC routers stop/start/reboot scenario
CLOUDSTACK-9015 - Make sure the Backup router can talk to the Master router after a stop/start/reboot
CLOUDSTACK-9067 - As I developer I want to remove all the unused router-shell scripts from ACSThis PR removes the unused shell scripts that were present in the ACS project. Those script were replaced by the.
Some of the scripts are used by the HyperV Resource, which were hardcoded. I took the opportunity to use the Java constants over there as well, so the next one touching the code will know they exist and won't hardcode anything.
The following task were applied:
* Remove the shell files and the Java constants that were mapping them;
* Apply the use of the Java constants to the HyperV Resource class;
* Wrap the String.format() method in the StringUtils so we can test the changes in the HyperV Resource class.
The last point was added because I do not have a HyperV test environment. Hence, I wanted to make sure the tiny code I changed is covered at least by unit tests.
* pr/1084:
CLOUDSTACK-9067 - Replaces hardcoded paths with the VRScripts constants.
CLOUDSTACK-9067 - Fomatting the code of HypervDirectConnectResource class
CLOUDSTACK-9067 - Remove old script file from the project
Signed-off-by: Remi Bergsma <github@remi.nl>
[4.6.1] CLOUDSTACK-9015 - Redundant VPC Virtual Router's state is BACKUP & BACKUP or MASTER & MASTERThis PR closes#1064
All the details can be found in the original PR, which won't be merged because it was created agains master. Once this PR is closed, the original one will be also closed.
* pr/1070:
CLOUDSTACK-9015 - Delete public IP in order to get both IP and NAT rule removed.
CLOUDSTACK-9015 - Add test to cover the rVPC routers stop/start/reboot scenario
CLOUDSTACK-9015 - Make sure the Backup router can talk to the Master router after a stop/start/reboot
Signed-off-by: Remi Bergsma <github@remi.nl>
The S3 implementation is far from finished, this commit focusses on the bases.
- Upgrade AWS SDK to latest version.
- Rewrite S3 Template downloader.
- Rewrite S3Utils utility class.
- Improve addImageStoreS3 API command.
- Split various classes for convenience.
- Various minor improvements and code optimalisations.
A side effect of the new AWS SDK is that it, by default, uses the V4 signature. Therefore I added an option to specify the Signer, so it stays compatible with previous versions.
CLOUDSTACK-9058 - Respond with "saved_password" if no password is to be issued.The password server on the virtual router should respond with "saved_password" if no password is to be issued. This allows for backwards compatibility with Windows Guest VMs which require the "saved_password" response.
* pr/1079:
CLOUDSTACK-9058
Signed-off-by: Remi Bergsma <github@remi.nl>
- Stop KeepaliveD/ConntrackD if the eth2 (guest) interface is not configured and UP
- Only setup the redundancy after all the router configuration is done
- Open the FW for the VRRP communitation
- 224.0.0.18 and 225.0.0.50
- Set keepalived.conf.templ by default to use interface eth2 (guest)
- It will be reconfigured anyway, but having eth2 there is more clear
CLOUDSTACK-8993: DHCP fails with "no address available" when an IP is reused
Repopulate /etc/dhcphosts.txt to remove old entries with the same IP address.
* pr/981:
CLOUDSTACK-8993: DHCP fails with "no address available" when an IP is reused
Signed-off-by: Remi Bergsma <github@remi.nl>
- If we stop/start a router, the state in the file will still say MASTER, when it is actually not
- Checking the state based on the interface (eth1) state
- Once master.py is called by keepalived, save the state in the json file to BACKUP just to make sure it's also written there
- Do not use the API call because it will read what is in the database, that might not have been updated yet
* Check the status in the router directly instead
- Remove all the sleeps
- It was working before because the Routers were restarting about 10 times for each operation
e.g. adding a VM to a network ot acquiring a new IP.
- Adding stat_rules of internal LB to iptables
We needed one extra rule in the INPUT chain
- With the keepalived fixed they should not be needed anymore. So first reducing them drasticaly
- I am now making a backup of the template file, write to the template file and compare it with the existing configuration
- The template file is recovered afer the process
- I also check if the process is running
- I fixed a bug in the compare method
- I am now updating the configuration variable once the file content is flushed to disk
- The default is Accept and will be changed based on the configuration of the offering.
CLOUDSTACK-8934 - The default egress is set as Deny in the router.
- We had to change it on the Java side in order to make the apply it once the default is defined as allowed on the net offering
- That's not the place to fix the default routes for redundant VPC,
- Adding tests to cover PF and FW in isolated networks
* Will still add some tests for egress as well
- The cidr was replaced by the single IP, which broke the feature.
- Wait during transition from master to backup otherwise the test fails due to wronge state
- Instead of changing the router type in a local variable, lets have a dedicated file for the dhcpsrvr routers
- The file is called iptables-dhcpsrvr, just like we have iptables-vpcrouter and iptables-router
CLOUDSTACK-8843: Fixed issue in default iptables rules on shared network VROn basic zone share network VR default iptables rules are not applied correctly. Due to this ssh to VR got failed.
In shared network the VR type is 'dhcpsrvr' not router. So corrected it in the ''del_standard' method to select the correct type.
Testing:
1. VR is deployed correctly.
2. Tested restart, stop, start VR.
3. New VM deployment is success.
4. ssh to VR from the host is successful.
5. iptables rules on the VR came up correctly.
below is the output from the VR:
iptables -L INPUT -nv
Chain INPUT (policy DROP 16 packets, 1056 bytes)
pkts bytes target prot opt in out source destination
0 0 ACCEPT all -- * * 0.0.0.0/0 224.0.0.18
0 0 ACCEPT all -- * * 0.0.0.0/0 225.0.0.50
104 9800 ACCEPT all -- eth0 * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
281 36500 ACCEPT all -- eth1 * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
0 0 ACCEPT all -- eth2 * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
6 504 ACCEPT icmp -- * * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0
2 656 ACCEPT udp -- eth0 * 0.0.0.0/0 0.0.0.0/0 udp dpt:67
13 780 ACCEPT tcp -- eth1 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:3922 state NEW,ESTABLISHED
0 0 ACCEPT tcp -- eth0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:80 state NEW
0 0 ACCEPT tcp -- eth0 * 10.147.40.0/23 0.0.0.0/0 state NEW tcp dpt:8080
* pr/842:
CLOUDSTACK-8843: Fixed issue in default iptables rules on shared network VR
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8881: Fixed Static and PF configuration issue1. For static nat filter rules are not configured in VR.
2. Corrected vm ip in PF rule.
* pr/882:
CLOUDSTACK-8881: Fixed Static and PF configuration issue
Signed-off-by: Remi Bergsma <github@remi.nl>
Configured dnsmasq to listen on all interfaces so that vpn client gets dns1. Dnsmasq is not listening on the ppp+ interfaces due to this remote access vpn clients dns requests are dropped.
2. Configured the dnsmasq to listen on all the interfaces except public. There is firewall to allow only specific cidr to allow the dns requests.
Tested from windows client nslookup.
* pr/870:
Configured dnsmasq to listen on all interfaces so that vpn client gets dns
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8891: Fixed default iptables rules on VR for guest trafficVR default iptables rules in INPUT chain are configured partially.
In CsAddress.py rules are configured while configuring public interface, guest interface post configuration is missed. Fixed to configure guest post configuration so that iptables rules are configured.
Testing:
1. Deployed vm in the network.
2.iptables rules on the VR configured correctly.
3.VM got the dhcp ip address from the VR.
* pr/867:
CLOUDSTACK-8891: Fixed default iptables rules on VR for guest traffic
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8798 Fixed the vrrp virtual ip config in case of rvr enablFixed the vrrp virtual ip config in case of rvr enabled isolated networks.
changed the CsRedundant.py to bring down the public interface when rvr changes state to
backup. Also fixed vrrp authentication for isolated networks.
This fix dose not effect the vpc networks. it is only meant for rvr isolated networks.
manullay deployed a vm in rvr enabled isolated network and ran the tests below.
nosetests --with-marvin --marvin-config=/marvin-config test/integration/component/test_redundant_router_services.py
Test redundant router internals ... === TestName: test_enableVPNOverRvR | Status : SUCCESS ===
----------------------------------------------------------------------
Ran 1 test in 633.336s
nosetests --with-marvin --marvin-config=/marvin-config test/integration/component/test_redundant_router_cleanups.py
Test network garbage collection with RVR ... === TestName: test_network_gc | Status : SUCCESS ===
ok
Test restarting RvR network without cleanup ... === TestName: test_restart_ntwk_no_cleanup | Status : SUCCESS ===
ok
Test restart RvR network with cleanup ... === TestName: test_restart_ntwk_with_cleanup | Status : SUCCESS ===
----------------------------------------------------------------------
Ran 3 tests in 2120.263s
* pr/800:
CLOUDSTACK-8798 Fixed the vrrp virtual ip config in case of rvr enabled isolated networks. changed the CsRedundant.py to bring down the public interface when rvr changes state to backup. Also fixed vrrp authentication for isolated networks.
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
CLOUDSTACK-8688 - default policies for INPUT and FORWARD should be set to DROP instead of ACCEPT
- In order to be able to access the routers via the link local interface, we have to add a rules with NEW and ESTABLISHED state
* pr/765:
CLOUDSTACK-8688 - Adding Marvin tests in order to cover the fixes applied
CLOUDSTACK-8688 - default policies for INPUT and FORWARD should be set to DROP instead of ACCEPT
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
CLOUDSTACK-8710: Fixed applying iptables rules for s2s vpn
@remibergsma @wilderrodrigues
Moved applying iptables rules apply after vpn configuration so that vpn specific rules also get applied
* pr/690:
CLOUDSTACK-8710: Fixed applying iptables rules for s2s vpn
This closes#690
Signed-off-by: Remi Bergsma <github@remi.nl>
Logging before:
2015-08-12 16:30:07,126 Searching for 192.168.23.6 and replacing with 192.168.23.6 192.168.23.5: PSK "preSharedKey"
Logging after:
2015-08-12 16:30:07,126 Searching for 192.168.23.6 and replacing with 192.168.23.6 192.168.23.5: PSK "****"
- For package installation/update please refer to: cloud-tools/appliance/definitions/systemvmtemplate/install_systemvm_packages.sh
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
This closes#587
- preempt delay reverted on version 1.2.13 - from the backports
- vrrp : Revert "Honor preempt_delay setting on startup.".
- See changelog: http://www.keepalived.org/changelog.html
- Refactoring some variable names to avoid misunderstanding
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
- We use no preempt mode with state set as EQUAL to both nodes, no need to have Priotities setup
- Do not add IPs as comments to the configuration. If a new guest interface is added, the file will change anyway.
- This was used in the past when keepalived would restart for each new interface added
- Removed the long sleep form the tests: we now sleep 5 seconds per PF rule added
CLOUDSTACK-8616 - Fix keepalived.ts/2 files comparison
- Add call to set_fault() in case of router transits to that state
- Removing commented out code
CLOUDSTACK-8616 - Fixing check_heartbeat.sh.templ
CLOUDSTACK-8616 - Call set_fault from the check_heartbeat.sh script
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
- After configuration save the ipdated in files
* /etc/iptables/router_rules.v4 and /etc/iptables/router_rules.v6
* Reload the configuration on reboot via the /etc/rc.local using iptables-restore
In 6ac06e5e5e logrotate was changed to run hourly.
Some logrotate configs still have set `daily` only which results in logs not
rotated hourly. The only way to ensure the log is rotated is to use size.
This closes#162
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Pub IP port forwarding and static NAT fixed for single VPCs
- Pub IP port forwarding fixed for redundant VPCs
[wip] fix static NAT for redundant VPCs
This closes#150
- With the changes added by the rVPC work, the bump priority became deprecated.
This commit includes a refactor to get it removed from the following resources:
* Java classes
* domain_router table - removing the is_priority_bumpup column
* Fixing unit tests
All changes were tested with:
XenServer 6.2 running under our VMWare zone
CloudStack Management Server running on MacBook Pro
MySql running on MackBook Pro
Storage Type: Local
This is a known issue in irqbalance 1.0.3 and was partially fixed in 1.0.4.
Using the package 1.0.6 from wheezy backports helped on many VRs balancing the
interrupts, but not on all.
It seems only 1.0.7 fixes all issues regarding this, also see thread in user ML
http://mail-archives.apache.org/mod_mbox/cloudstack-users/201503.mbox/%3C5508540E.4090302%40renemoser.net%3E
This closes#119
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit a45f0e29e2)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The logic is same as passwd_server_ip script which runs password server on all
IPs on eth0 interface.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 294503a025)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Fixed failure on using eth10 (which CS now supports)
Refactored CSForward to us CsInterface object instead of teh way it was kludged together
Fixed hex conversion for device numbers and iptables marks
- If for some reason the cmdLine json doesn't contain the password key, which is almost impossible to happen,
we generate a password based on other unique data per VPC
Customise the messages showed during VPC restart based on redundancy
Do not show "Make Redundant" option if VPC is already redundant
Show on VPC details if a VPC is redundant
De Morgans laws chaps
The negation of a conjunction is the disjunction of the negations.
The negation of a disjunction is the conjunction of the negations.
Remi Bergsma
Wilder Rodrigues
Boris Schrijver
dsclose
Wei Zhou
Joakim Sernbrant
Bharat Kumar
Jayapal
SudharmaJain
Jayapal
Michael Andersen
Kishan Kavala
Ian Southam
David Bierce
Abhinandan Prateek
Rajani Karuturi
Rohit Yadav
Rene Moser
Rene Moser