Cloudstack-8885 added blocked connection listener for rabbitmqeventbus When rabbitmq connections are blocked(for example when rabbitmq is
server is out of space), all the cloudstack threads which does any
action and publishes to rabbitmq(for example login, launch vm etc.) are
all blocked.
Added a blocked connection listener to handle this and unblock the
parent thread.
also updated rabbitmq amqp client to 3.5.4 from 3.4.2
Testing:
I didnt find a way to write tests for this change as of now.
manually tested that login and other cloudstack apis work when the configured rabbitmq server goes out of space and publish actions are blocked.
* pr/857:
CLOUDSTACK-8885: added blocked connection listener for rabbitmqeventbus
updating rabbitmq amqp client to 3.5.4 from 3.4.2
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8860: improve error messages in VM deployment code path.improved the error messages in vm deployment code path. added some more data to the error messages and also fixed some errors using internal ids to use uuids.
* pr/864:
CLOUDSTACK-8860: improve error messages in VM deployment code path.
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9195: Cancelled/failed async jobs not getting cleaned up from DB
Setting 'job_complete_msid' field in async_job table when marking job as cancelled during MS restart.
This will ensure that the job is cleaned up as part of GC thread.
* pr/1272:
CLOUDSTACK-9195: Cancelled/failed async jobs not getting cleaned up from DB Setting 'job_complete_msid' field in async_job table when marking job as cancelled during MS restart. This will ensure that the job is cleaned up as part of GC thread.
Signed-off-by: Remi Bergsma <github@remi.nl>
* 4.7:
Refactor public ip retrieval into method
CLOUDSTACK-9244 Fix setting up RFC1918 routes
CLOUDSTACK-9239 throw exception on deprecated command
Enhance VR performance by selectively executing tasks instead of brute-forcing
CLOUDSTACK-9236: Load Balancing Health Check button displayed when non-NetScaler offering is used
CLOUDSTACK-9239 throw exception on deprecated commandan API call to LockAccount fails silently with a 200 return and no contents. client are responsible to check and will find that no locking has actually occured. This way they may be lured to a sense of false security.
Throwing an exption makes the failure explicit.
cloudmonkey lock account name=<name> domainid=<id>
now errors out with an http result of 530
* pr/1349:
CLOUDSTACK-9239 throw exception on deprecated command
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9244 Fix setting up RFC1918 routesPublic ip var was empty so routes were always set. Corrected this.
* pr/1352:
Refactor public ip retrieval into method
CLOUDSTACK-9244 Fix setting up RFC1918 routes
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9236: Load Balancing Health Check button displayed when non-NetScaler offering is usedLoad balancing health check option / button should only be displayed when a NetScaler based networking offering is being used.
If you try to use the health check option when NetScaler is not being used as the load balancing technology the UI throws an error stating that the health check option is not supported.
This button / option should be removed from the UI if NetScaler is not included in the network offering as customers will be confused.
Fix:
===
This button will be shown only when the load balancer is NetScaler.
Otherwise it is hidden.
* pr/1338:
CLOUDSTACK-9236: Load Balancing Health Check button displayed when non-NetScaler offering is used
Signed-off-by: Remi Bergsma <github@remi.nl>
test: Fix Libvirt test so that it works on WindowsThis test failed on Windows, using the File.separator it should run fine on Windows.
* pr/1242:
test: Fix Libvirt test so that it works on Windows
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9132: API createVolume takes empty string for name parameterSteps to Reproduce:
================
Create a volume using createVolume API where parameter name is empty.
It creates a volume with empty name.
But the name parameter is mandatory.(Issue)
Expected Behaviour:
================
It shouldn't create a volume with an empty name. Error should be returned.
Solution:
=======
Added a condition to check in case of empty string. If the name is an empty string, it generates a random name for the volume. Made the name field optional in UI as well as in API.
* pr/1319:
CLOUDSTACK-9132: API createVolume takes empty string for name parameter
Signed-off-by: Remi Bergsma <github@remi.nl>
* 4.7:
Fix execution counter to support separate counts per thread
Add test to check that each thread has it's own execution counter
CLOUDSTACK-9231: Root volume migration from one primary to another primary storage within the same cluster is failing
CLOUDSTACK-9231: Root volume migration from one primary to another primary storage within the same cluster is failingEXPECTED BEHAVIOUR:
====================
Root Volume migration within cluster should work.
ACTUAL BEHAVIOUR:
==================
Root volume migration within cluster failed.
This situation arises when there are two management server accessing the same database.
When the migration request comes the command is forwarded from one management server to another because the host is owned by the second management server. So, serialisation of map from one to another fails.
Fix:
===
This is fixed by converting the maps to lists.
* pr/1336:
CLOUDSTACK-9231: Root volume migration from one primary to another primary storage within the same cluster is failing
Signed-off-by: Remi Bergsma <github@remi.nl>
- Implement Search boxes in all Metrics views
- Fix threshold calculations for host and storage pool metrics view
- Consider overcommit ratios for calculation allocated thresholds
- Save/pass context while navigating across resources and metrics view
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Implement a NSX API request execution counter per threadThe NSX plugin has a execution counter to prevent infinite recursion (and as a result a stack overflow exception). However, the thread safeness of this counter are not as desired. The counter was implemented with an AtomicInteger which make it safe for multiple threads to update and read it. The desired property would be to have a counter per thread.
This PR addresses that issue.
* pr/1294:
Fix execution counter to support separate counts per thread
Add test to check that each thread has it's own execution counter
Signed-off-by: Remi Bergsma <github@remi.nl>
resolving a conflict due to reformatting
* pr/1344:
CLOUDSTACK-9154 - Sets the pub interface down when all guest nets are gone
CLOUDSTACK-9187 - Makes code ready for more something like ethXXXX, if we ever get that far
CLOUDSTACK-9188 - Reads network GC interval and wait from configDao
CLOUDSTACK-9187 - Fixes interface allocation to VRRP instances
CLOUDSTACK-9187 - Adds test to cover multiple nics and nic removal
CLOUDSTACK-9154 - Adds test to cover nics state after GC
CLOUDSTACK-9154 - Returns the guest iterface that is marked as added
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9230: Remove unnecessary return statement from cloudStack.jsRemoved the unnecessary return statement.
The statement is never reached.
* pr/1335:
CLOUDSTACK-9230: Remove unnecessary return statement from cloudStack.js
Signed-off-by: Remi Bergsma <github@remi.nl>
* 4.7:
CLOUDSTACK-9154 - Sets the pub interface down when all guest nets are gone
CLOUDSTACK-9187 - Makes code ready for more something like ethXXXX, if we ever get that far
CLOUDSTACK-9188 - Reads network GC interval and wait from configDao
CLOUDSTACK-9187 - Fixes interface allocation to VRRP instances
CLOUDSTACK-9187 - Adds test to cover multiple nics and nic removal
CLOUDSTACK-9154 - Adds test to cover nics state after GC
CLOUDSTACK-9154 - Returns the guest iterface that is marked as added
Conflicts:
engine/orchestration/src/org/apache/cloudstack/engine/orchestration/NetworkOrchestrator.java
[4.7] Critical VPCVR issues fixed: CLOUDSTACK-9154; CLOUDSTACK-9187; and CLOUDSTACK-9188This PR applies the same fixes as in the PR #1259, but against branch 4.7.
Please refer to PR #1259 for the tests results and all the comments already made there.
Issues fixed are:
* CLOUDSTACK-9154: rVPC doesn't recover from cleaning up of network garbage collector
* CLOUDSTACK-9187: rVPC routers in Master/Master due to concurrency problem when writing the keepalivd.conf
* CLOUDSTACK-9188: NetworkGarbageCollector is not using gc.interval and gc.wait from settings
Those changes have been covered by 2 new tests added to ```smoke/test_vpc_redundant.py```:
* test_04_rvpc_network_garbage_collector_nics
* test_05_rvpc_multi_tiers
The test ```test_04_rvpc_network_garbage_collector_nics``` depends on the global settings for the network.gc.interval and gc.wait. If one wants the test to run quicker, please change the settings (default is 600 seconds for each) and restart the Management Server before running the tests. I would suggest to set it to 60 seconds.
In addition, the NetworkGarbageCollector was redefining the settings above mentioned and not reading their values through ConfigDao. Due to that, the settings were not being applied properly and the test was waiting to long to check the VPC routers.
* pr/1277:
CLOUDSTACK-9154 - Sets the pub interface down when all guest nets are gone
CLOUDSTACK-9187 - Makes code ready for more something like ethXXXX, if we ever get that far
CLOUDSTACK-9188 - Reads network GC interval and wait from configDao
CLOUDSTACK-9187 - Fixes interface allocation to VRRP instances
CLOUDSTACK-9187 - Adds test to cover multiple nics and nic removal
CLOUDSTACK-9154 - Adds test to cover nics state after GC
CLOUDSTACK-9154 - Returns the guest iterface that is marked as added
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9001: Modifying snapshot results validation Currently snapshots results validation is based on length of snapshot result but if snapshot creation fails then None type object will not have "len" attribute hence modifying the validation.
* pr/994:
CLOUDSTACK-9001: Modifying snapshot results validation in testpath_uuid_event testpath
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9005: Modifying tearDown functionModifying tearDown function to check if data volume is in detached state before deleting the volume
* pr/1000:
CLOUDSTACK-9005: Modifying tearDown function
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9041: Modifying template creation from snapshot function In create_from_snapshot function of Template class there is no parameter to accept if the template is public hence default it is creating private templates
Hence adding this parameter.
* pr/1041:
CLOUDSTACK-9041: Modifying template creation from snapshot function in base.py
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9240 remove 40GB filesize limit from SSVM scriptsBoth createvolume.sh and createtmplt.sh have a 40GB hardcoded limit for the size of the template that gets created. I could not find any justification of that. I am just removing them as they caused us a huge headache when we tried to create bigger templates and they failed without any good error.
This closes#1223 (This PR was against master, made a new one against 4.7)
Thanks Syed <syed1.mushtaq@gmail.com>
* pr/1343:
CLOUDSTACK-9240 remove 40GB filesize limit from SSVM scripts
Signed-off-by: Remi Bergsma <github@remi.nl>
Removed cloud-cli folder and contents, as it is not maintained or used anymore.Remove legacy code.
Compiles ok:
```
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7:16.245s
[INFO] Finished at: Sat Dec 12 15:06:24 CET 2015
[INFO] Final Memory: 101M/821M
[INFO] ------------------------------------------------------------------------
```
Running integration tests now! Ping @wido @wilderrodrigues @remibergsma
* pr/1228:
Removed cloud-cli folder and contents, as it is not maintained or used anymore.
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9210: Pass secondary IPs to default_network_rules() functionThis is a mandatory argument but it was NOT passed which caused the
re-programming of security groups to fail.
Simple fix to just add the argument since the variable is available
there.
* pr/1309:
CLOUDSTACK-9210: Pass secondary IPs to default_network_rules() function
Signed-off-by: Remi Bergsma <github@remi.nl>
* 4.7:
Implement CheckHealthCommand for NSX controllers
Fix log message that refers to agent, not host
Prevent NullPointerException when host does not belong to a pod
Add Health Check Command to NSX pluginThe NSX plugin does not support the HeathCheckCommand. Instead it fakes a PingCommand as a call tot he control cluster status API.
However, we have seen in production that the management server will sometimes find the NSX controller to be behind on ping and that will trigger a HealthCheckCommand which will return with an unsupported command answer.
Once this happens the controller is put into Alert state and will not recover until the management sever is restarted.
In addition, during the investigation, there will be a null pointer exception due tot he fact that the NSX controllers do not live in a pod.
This PR tries to address those two issues.
* pr/1293:
Implement CheckHealthCommand for NSX controllers
Fix log message that refers to agent, not host
Prevent NullPointerException when host does not belong to a pod
Signed-off-by: Remi Bergsma <github@remi.nl>
* 4.7:
CLOUDSTACK-9222 Prevent cloud.log.1 filling up the disk
Add integration test for restartVPC with cleanup, and Private Gateway enabled.
Nullpointer Exception in NicProfileHelperImpl
NicProfileHelperImpl NullpointerException when ipVO is nullWhen a VPC has a private gateway, and one would like to restart the VPC with **cleanup** it would fail.
This PR adds a NullPointer check and verifies it with an integration test.
```
test_01_vpc_privategw_acl (integration.smoke.test_privategw_acl.TestPrivateGwACL) ... === TestName: test_01_vpc_privategw_acl | Status : SUCCESS ===
ok
test_02_vpc_privategw_static_routes (integration.smoke.test_privategw_acl.TestPrivateGwACL) ... === TestName: test_02_vpc_privategw_static_routes | Status : SUCCESS ===
ok
test_03_vpc_privategw_restart_vpc_cleanup (integration.smoke.test_privategw_acl.TestPrivateGwACL) ... === TestName: test_03_vpc_privategw_restart_vpc_cleanup | Status : SUCCESS ===
ok
test_04_rvpc_privategw_static_routes (integration.smoke.test_privategw_acl.TestPrivateGwACL) ... === TestName: test_04_rvpc_privategw_static_routes | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 4 tests in 2945.055s
OK
```
* pr/1328:
Add integration test for restartVPC with cleanup, and Private Gateway enabled.
Nullpointer Exception in NicProfileHelperImpl
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9222 Prevent cloud.log.1 filling up the diskDelay Compress results in more space usage than needed. Since we have copy truncate we don't need it.
* pr/1329:
CLOUDSTACK-9222 Prevent cloud.log.1 filling up the disk
Signed-off-by: Remi Bergsma <github@remi.nl>
* 4.7:
Fix unable to setup more than one Site2Site VPN Connection
FIX S2S VPN rVPC: Check only redundant routers in state MASTER
PEP8 of integration/smoke/test_vpc_vpn
Add S2S VPN test for Redundant VPC
Make integration/smoke/test_vpc_vpn Hypervisor independant
FIX VPN: non-working ipsec commands
[UI] MADNESS
[DB] Add force_encap field to s2s_customer_gateway table
[ROUTER] Add forceencaps field to python router ipsec config method
[TEST] unittest needs rework
[MARVIN] Add forceencap field to VpnCustomerGateway class in marvin base
[CORE] Add Force UDP Encapsulation option to Site2Site VPN
CLOUDSTACK-9186: Root admin cannot see VPC created by Domain admin user
CLOUDSTACK-9192: UpdateVpnCustomerGateway is failing
CLOUDSTACK-6485 prevent ip asignment of private gw iface
CLOUDSTACK-9204 Do not error when staticroute is already gone
make both check lines consistent
CLOUDSTACK-9181 Prevent syntax error in checkrouter.sh
CLOUDSTACK-9202 Bump ssh timeout
[4.7] FIX Site2SiteVPN on redundant VPCThis PR:
- fixes the inability to setup more than one Site2Site VPN connection from a VPC
- fixes starting of Site2Site VPN on redundant VPC
- fixes Site2Site VPN state checking on redundant VPC
- improves the vpc_vpn test to allow multple hypervisors
- adds an integration test for Site2Site VPN on redundant VPC
Tested it on 4.7 single Xen server zone:
command:
```
nosetests --with-marvin --marvin-config=/data/shared/marvin/mct-zone1-xen1.cfg -a tags=advanced,required_hardware=true /tmp/test_vpc_vpn.py
```
results:
```
Test Site 2 Site VPN Across redundant VPCs ... === TestName: test_01_redundant_vpc_site2site_vpn | Status : SUCCESS ===
ok
Test Remote Access VPN in VPC ... === TestName: test_01_vpc_remote_access_vpn | Status : SUCCESS ===
ok
Test Site 2 Site VPN Across VPCs ... === TestName: test_01_vpc_site2site_vpn | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 3 tests in 1490.076s
OK
```
also performed numerous manual inspections of state of VPN connections and connectivity between VPC's
* pr/1276:
Fix unable to setup more than one Site2Site VPN Connection
FIX S2S VPN rVPC: Check only redundant routers in state MASTER
PEP8 of integration/smoke/test_vpc_vpn
Add S2S VPN test for Redundant VPC
Make integration/smoke/test_vpc_vpn Hypervisor independant
FIX VPN: non-working ipsec commands
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9202 Bump ssh timeout for VR commandsIt seems the VR needs more time for some of its commands. Until we figured out the root cause, this allows the VRs to start again.
Error seen:
```
2015-12-28 14:35:18,201 ERROR [c.c.n.r.VirtualNetworkApplianceManagerImpl] (Work-Job-Executor-1:ctx-34ff7f80 job-39723/job-39726 ctx-d63de41b) Timed out in waiting SSH execution result
2015-12-28 14:35:18,201 WARN [c.c.n.r.VirtualNetworkApplianceManagerImpl] (Work-Job-Executor-1:ctx-34ff7f80 job-39723/job-39726 ctx-d63de41b) Command: com.cloud.agent.api.Command failed while starting virtua
l router
2015-12-28 14:35:18,201 INFO [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-1:ctx-34ff7f80 job-39723/job-39726 ctx-d63de41b) The guru did not like the answers so stopping VM[DomainRouter|r-1534-VM]
.Answer":{"result":true,"wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Timed out in waiting SSH execution result","wait":0}}] }
```
* pr/1291:
CLOUDSTACK-9202 Bump ssh timeout
Signed-off-by: Remi Bergsma <github@remi.nl>
nvazquez
Remi Bergsma
Rene Moser
Daan Hoogland
Rohit Yadav
Boris Schrijver
Miguel Ferreira
Abhinandan Prateek
