CLOUDSTACK-8925 - Default allow for Egress rules is not being configured properly in VR iptables rulesThis PR fixes the router default policy for egress. When the default is DENY, the router still allows outgoing connections.
The test component/test_routers_network_ops.py was improved to cover that case as well. The results were:
Test redundant router internals ... === TestName: test_01_isolate_network_FW_PF_default_routes_egress_true | Status : SUCCESS ===
ok
Test redundant router internals ... === TestName: test_02_isolate_network_FW_PF_default_routes_egress_false | Status : SUCCESS ===
ok
Test redundant router internals ... === TestName: test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true | Status : SUCCESS ===
ok
Test redundant router internals ... === TestName: test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 4 tests in 3636.656s
OK
/tmp//MarvinLogs/test_routers_network_ops_QDL429/results.txt (END)
* pr/1023:
CLOUDSTACK-8925 - Implement the default egress DENY/ALLOW properly
CLOUDSTACK-8925 - Improve the default egress tests in order to cover newly entered rules
CLOUDSTACK-8925 - Add egress dataset to test_data.py
CLOUDSTACK-8925 - Drop the traffic when default egress is set to false
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9007 - Write test to check that the /etc/dhcphosts.txt doesn't contain duplicate IPsThis PR contains a test that will cover the fix on PR #981
The tests does the following:
* Creates account, service offering, network offering, network
* Deploys two virtual machines
- Each machine with a pre-assigned IP
* Creates two FW and PF rules
* Checks that SSH into the VMs works
* Checks default routes from both VMs
* Checks that the /etc/dhcphosts.txt contains 1 entry per VM IP
* Destroys/Expunges 1 VM
* Creates a new VM with the same IP as the destroyed one
* Checks that the /etc/dhcphosts.txt contains 1 entry per VM IP
* pr/1002:
CLOUDSTACK-9007 - Add test check that /etc/dhcphosts.txt doesn't contain duplicate IPs
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8991 - IP address is not removed from VR even after disabling static NATThis PR fixes the Public IP removal form the virtual routers. It also improves the existing test_network.py.
* pr/989:
CLOUDSTACK-8991 - Process the IPs that have been removed
CLOUDSTACK-8991 - Remove public IP form interface in case add = false
CLOUDSTACK-8991 - Make sure the public IP is removed form the router before checking
Signed-off-by: Remi Bergsma <github@remi.nl>
smoke/test_internal_lb.py: Fix template not ready errorAdd wait for template download
Refactored template section of services
Added some extra logging in the setup phase
* pr/971:
Add wait for template download Refactored template section of services
Signed-off-by: Remi Bergsma <github@remi.nl>
Fix error message in test_isolate_network_FW_PF_default_routesWhile running test_isolate_network_FW_PF_default_routes it is expected
that SSH'ing into a VM does not work immediately. However, when it fails
(as expected) witht he follwoing error
====Trying SSH Connection: Host:192.168.23.12 Uer:root
Port:22 RetryCnt:1===
Traceback (most recent call last):
File "/home/jenkins/workspace/mccloud/mct-run-marvin-tests@2/venv/lib/python2.7/site-packages/marvin/sshClient.py", line 121, in createConnection timeout=self.timeout)
File "/home/jenkins/workspace/mccloud/mct-run-marvin-tests@2/venv/lib/python2.7/site-packages/paramiko/client.py", line 251, in connect retry_on_signal(lambda: sock.connect(addr))
File /home/jenkins/workspace/mccloud/mct-run-marvin-tests@2/venv/lib/python2.7/site-packages/paramiko/util.py", line 270, in retry_on_signal return function()
File /home/jenkins/workspace/mccloud/mct-run-marvin-tests@2/venv/lib/python2.7/site-packages/paramiko/client.py", line 251, in <lambda> retry_on_signal(lambda: sock.connect(addr))
File "/usr/lib64/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args)
error: [Errno 113] No route to host
it would try to print a message that generates a actual error:
```
DEBUG: ====Trying SSH Connection: Host:192.168.23.12 User:root
Port:22 RetryCnt:0===
test_isolate_network_FW_PF_default_routes
(integration.component.test_routers_network_ops.TestIsolatedNetworks):
CRITICAL: EXCEPTION: test_isolate_network_FW_PF_default_routes:
Traceback (most recent call last):,
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()',
File "/home/jenkins/workspace/mccloud/mct-run-marvin-tests@2/test/integration/component/test_routers_network_ops.py", line 448, in test_isolate_network_FW_PF_default_routes self.fail("Failed to SSH into VM - %s" % (public_ip.ipaddress.ipaddress)),
"AttributeError: 'unicode' object has no attribute 'ipaddress'"
```
* pr/972:
Fix error message in test_isolate_network_FW_PF_default_routes
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8935 - Cannot remove [r]VPC networks due to RTNETLINK errorThis PR fixes the "sequence item 0: expected string, NoneType found" error found in the CsDhcp.py file when attempting to remo a network from a VPC.
* pr/967:
CLOUDSTACK-8935 - Filter the DNS list because it might contain 1 None entry which breaks the code.
CLOUDSTACK-8935 - Clean up network resources in the right order
CLOUDSTACK-8935 - Do not retry 60 times when we expect the SSH to fail
CLOUDSTACK-8935 - Check if the key is available in the dictionary
CLOUDSTACK-8935 - Add a check to avoid exception related to None value
Signed-off-by: Remi Bergsma <github@remi.nl>
While running test_isolate_network_FW_PF_default_routes it is expected
that SSH'ing into a VM does not work immediately. However, when it fails
(as expected) witht he follwoing error
====Trying SSH Connection: Host:192.168.23.12 Uer:root
Port:22 RetryCnt:1===
Traceback (most recent call last):
File "/home/jenkins/workspace/mccloud/mct-run-marvin-tests@2/venv/lib/python2.7/site-packages/marvin/sshClient.py", line 121, in createConnection timeout=self.timeout)
File "/home/jenkins/workspace/mccloud/mct-run-marvin-tests@2/venv/lib/python2.7/site-packages/paramiko/client.py", line 251, in connect retry_on_signal(lambda: sock.connect(addr))
File /home/jenkins/workspace/mccloud/mct-run-marvin-tests@2/venv/lib/python2.7/site-packages/paramiko/util.py", line 270, in retry_on_signal return function()
File /home/jenkins/workspace/mccloud/mct-run-marvin-tests@2/venv/lib/python2.7/site-packages/paramiko/client.py", line 251, in <lambda> retry_on_signal(lambda: sock.connect(addr))
File "/usr/lib64/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args)
error: [Errno 113] No route to host
it would try to print a message that generates a actual error:
DEBUG: ====Trying SSH Connection: Host:192.168.23.12 User:root
Port:22 RetryCnt:0===
test_isolate_network_FW_PF_default_routes
(integration.component.test_routers_network_ops.TestIsolatedNetworks):
CRITICAL: EXCEPTION: test_isolate_network_FW_PF_default_routes:
Traceback (most recent call last):,
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()',
File "/home/jenkins/workspace/mccloud/mct-run-marvin-tests@2/test/integration/component/test_routers_network_ops.py", line 448, in test_isolate_network_FW_PF_default_routes self.fail("Failed to SSH into VM - %s" % (public_ip.ipaddress.ipaddress)),
"AttributeError: 'unicode' object has no attribute 'ipaddress'"
CLOUDSTACK-8933 SSVm and CPVM do not survive a reboot from APIThis closes PR #930 as well.
I Rebased @bvbharat's PR with latest Master and tested the SSVM/CPVM and the routers: rVPC; VPC; VR; and RVR.
* pr/959:
CLOUDSTACK-8933 - Improves the the test internals of the SSVM/CPVM
CLOUDSTACK-8933 - Replace infinite loop by a for loop
CLOUDSTACK-8933 SSVm and CPVM do not survive a reboot from API
This closes#930
Signed-off-by: Remi Bergsma <github@remi.nl>
Add optional fields: iprange and fordisplay to Marvin base.py class method Vpn.create
Add optional field: passive to Marvin base.py class method Vpn.createVpnConnection
- Do not use the API call because it will read what is in the database, that might not have been updated yet
* Check the status in the router directly instead
- Remove all the sleeps
- It was working before because the Routers were restarting about 10 times for each operation
e.g. adding a VM to a network ot acquiring a new IP.
- Adding stat_rules of internal LB to iptables
We needed one extra rule in the INPUT chain
- With the keepalived fixed they should not be needed anymore. So first reducing them drasticaly
- I am now making a backup of the template file, write to the template file and compare it with the existing configuration
- The template file is recovered afer the process
- I also check if the process is running
- I fixed a bug in the compare method
- I am now updating the configuration variable once the file content is flushed to disk
- Due to an issue with VPC routers (CLOUDSTACK-8935) we are not able to destroy networks before destroying the routers
- Added a forcestop/destroy routers inside the tearDown to make sure it passes. The issue will be addressed in a separate PR
- Make sure the routers list is cleaned after destroy_routers() is called
- Populate routers list after the router is recreated
- Add egress tests in order to check if VMs can reach the outside world
- Increase the wait when testing redundant routers: they fight to become master
- Make sure the clean up is done properly
- That's not the place to fix the default routes for redundant VPC,
- Adding tests to cover PF and FW in isolated networks
* Will still add some tests for egress as well
- The cidr was replaced by the single IP, which broke the feature.
- Wait during transition from master to backup otherwise the test fails due to wronge state
CLOUDSTACK-8726 : Automation for Quickly attaching multiple data disks to a new VMAttach multiple Volumes simultaneously to a Running VM ... === TestName: test_attach_multiple_volumes | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 1 test in 196.931s
OK
* pr/683:
changed the testcase skip code into setup method
Imparting changes mentioned by nitt10prashant
Automation for multiple disk attachments to instance
Signed-off-by: sanjeev <sanjeev@apache.org>
CLOUDSTACK-8756:Incorrect guest os mapping in CCP 4.2.1-6 for CentOS 5.9Check the bug 8756 for more details
* pr/728:
CLOUDSTACK-8756:Incorrect guest os mapping in CCP 4.2.1-6 for CentOS 5.9
Signed-off-by: Rajani Karuturi <rajani.karuturi@citrix.com>
CLOUDSTACK-8688 - default policies for INPUT and FORWARD should be set to DROP instead of ACCEPT
- In order to be able to access the routers via the link local interface, we have to add a rules with NEW and ESTABLISHED state
* pr/765:
CLOUDSTACK-8688 - Adding Marvin tests in order to cover the fixes applied
CLOUDSTACK-8688 - default policies for INPUT and FORWARD should be set to DROP instead of ACCEPT
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
- Changing refactored the utils.get_process_status() function
- Adding 2 tests: test_01_single_VPC_iptables_policies and test_02_routervm_iptables_policies
CLOUDSTACK-8759 - Destroying VPC router results in a new unusable VPC routerHow to reproduce the problem:
1. Stop/Destroy the VPC router
2. Add a virtual machine to one of the VPC tier - it will trigger a VPC router creation
3. Router is created, but the NICs are not configured
How to recover without this fix:
1. Stop/destroy the VPC router and restart the VPC
Side effects: private gateways could be misconfigured.
Root cause:
In the VpcNetworkHelperImpl.configureDefaultNics() method, the guest network nic was added in the map prior to the control and public NICs. The order in the map should not matter, however in the LibvirtComputingResource.createVifs() method, there is a logic that relies on the device index - the array index - in order to create the control nic. I advise a refactor on the data model in order to be able to identify the NIC type instead of relying in the array index.
An integration test was added to cover the fix:
* test_vpc_router_nics.py
Environment:
Management Server running on CentOS 7.1
KVM host running on CentOS 7.1
CloudStack Agent/Common 4.6.0-SNAPSHOT
Executing the test:
```
nosetests --with-marvin --marvin-config=/data/shared/marvin/mct-zone2-kvm2-ISOLATED.cfg -s -a tags=advanced,required_hardware=true component/test_vpc_router_nics.py
```
Remark: during the SSH there might be stack traces on the console due to the connection retry. It takes some time to get the PF rules in place and reach the VMs. So, just let the test run until the end.
```
Test results:
Create a vpc with two networks with two vms in each network ... === TestName: test_01_VPC_nics_after_destroy | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 1 test in 774.020s
OK
/tmp//MarvinLogs/test_vpc_router_nics_VH6E9S/results.txt (END)
```
* pr/773:
CLOUDSTACK-8759 - Fix guets nic allocation
CLOUDSTACK-8759 - Adding a marvin test in order to cover the fix
CLOUDSTACK-8759 - The guest nic has to be added after the control nic
Signed-off-by: Remi Bergsma <github@remi.nl>
- The test will create a VPC, add 2 tier, 2 VMs, ACL, PF and SSH into the VM
- Then it will stop the router, destroy the router, add another VM to 1 tier and check that we can reach all the VMs
Made following fixes in simulator
- Support for ScaleVmCommand/NetworkRulesVmSecondaryIpCommand in resource layer
- Added support for scaling up a running VM in simulator
- Fixed some method names not following convention
In order to test PR #725 using simulator some of these changes are needed.
Based on the way HV check is present in the scale VM API, had to explicitly put simulator related check to allow support. The ideal way would be to remove all these HV specific check from code and made them some configuration (by putting them in hypervisor_capabilities table in DB). But that will be a bigger effort outside the scope of this PR.
Signed-off-by: Koushik Das <koushik@apache.org>
CLOUDSTACK-8716: Verify creation of snapshot from volume when the task is performed repeatedly in zone wide primary StorageOn VMWare with a Zone wide primary storage and more than two clusters verify successful creation of snapshot multiple times.
* pr/665:
CLOUDSTACK-8716: Verify creation of snapshot from volume when the task is performed repeatedly in zone wide primary Storage
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-8723: Verify API call "listUsageRecords" returns usage of new volume created after restore VMAfter restoring a running VM current ROOT disk gets destroyed and new ROOT disk gets created.
This testcase is to check if volume usage of this newly created volume is listed in listUsageRecords API.
* pr/675:
CLOUDSTACK-8723: Verify API call "listUsageRecords" returns usage of new volume created after restore VM
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-8757:FTP modules are not loaded in VRcheck bug 8757 for more details .
* pr/729:
CLOUDSTACK-8757:FTP modules are not loaded in VR
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-8716: Verify creation of snapshot from volume when the task is performed repeatedly in zone wide primary Storage
-Removing redundent code
-Added validate list function for list snapshot operation
CLOUDSTACK-8716: Verify creation of snapshot from volume when the task is performed repeatedly in zone wide primary Storage
Removed duplicate test data related to vm properties.Modified tests dependent on it
Removed duplicte service offerings from test data and modified tests dependent on it
Bug-Id: CLOUDSTACK-8617
This closes#644
CLOUDSTACK-8689-Verify-effect-of-changing-value-of-XenServer-Max-guest-limitis-on-previously-added-hosts -Addning check for empty list and increamenting maxguestlimit accordingly
CLOUDSTACK-8689: Verify effect of changing value of XenServer Maxguestlimits on previously added hosts -As testcase is changing maxguestlimits global setting it will affect on other testcases also hence moving it to component/maint folder
This closes#638
As part of volume sync, that runs during of SSVM start-up, the volume_store_ref entry was getting deleted. Volume GC relies on this entry to move volume to destroyed state.
Since the entry was getting deleted, GC thread never moved the volume from UploadError/UploadAbandoned to Destroyed. Fix is to not remove the volume_store_ref entry as part
of volume sync and let GC thread handle the clean up.
This closes#611
- We use no preempt mode with state set as EQUAL to both nodes, no need to have Priotities setup
- Do not add IPs as comments to the configuration. If a new guest interface is added, the file will change anyway.
- This was used in the past when keepalived would restart for each new interface added
- Removed the long sleep form the tests: we now sleep 5 seconds per PF rule added
CLOUDSTACK-8616 - Fix keepalived.ts/2 files comparison
- Add call to set_fault() in case of router transits to that state
- Removing commented out code
CLOUDSTACK-8616 - Fixing check_heartbeat.sh.templ
CLOUDSTACK-8616 - Call set_fault from the check_heartbeat.sh script
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
listing the hosts for migration for instance with tag
Added marvin test for this issue.
Steps
1. Create a Compute service offering with the tag.
2. Create a Guest VM with the compute service offering
created above.
3. find hosts to migrate the vm crated above
Validations
1. Ensure that the offering is created with the
tag.The listServiceOffering API should list show tag
2. findHostsForMigration cmd should list both suitable
and not-suitable hosts
Priti Sarap
Remi Bergsma
Wilder Rodrigues
sadhu suresh
Michael Andersen
sanjeev
shweta agarwal
Nick Livens
Boris Schrijver
Miguel Ferreira
Rajani Karuturi
Koushik Das
manasaVeloori
Rohit Yadav
radu-stefanache
Sowmya Krishnan
pavan018
Daan Hoogland
nitt10prashant
Gaurav Aradhye
Daan Hoogland
Jayapal
Ian Southam
Rajani Karuturi
Sudhansu
Rafael da Fonseca
Sanjeev Neelarapu
Sowmya Krishnan
Abhinav Roy