cloudstack/engine
Vishesh c3b77cb7b8
Fix host stuck in connecting state (#8502)
There are a lot of test failures due to test_vm_life_cycle.py in multiple PRs due to host not available for migration of VMs.
#8438 (comment)
#8433 (comment)
#7344 (comment)

While debugging I noticed that the hosts get stuck in Connecting state because MS is waiting for a response of the ReadyCommand from the agent. Since we take a lock on connection and disconnection, restarting the agent doesn't work. To fix this, we have to restart the MS or wait for ~1 hour (default timeout).

On the agent side, it gets stuck waiting for a response from the Script execution.

To reproduce, run smoke/test_vm_life_cycle.py (TestSecuredVmMigration test class to be specific). Once the tests are complete, you will notice that some hosts are stuck in Connecting state. And restarting the agent fails due to the named lock. Locks on DB can be checked using the below query.

SELECT *
FROM performance_schema.metadata_locks
INNER JOIN performance_schema.threads ON THREAD_ID = OWNER_THREAD_ID
WHERE PROCESSLIST_ID <> CONNECTION_ID() \G;

This PR adds a wait for the ready command and a timeout to the Script execution to ensure that the thread doesn't get stuck and the named lock from database is released.
2024-01-15 13:56:34 +05:30
..
api vm-import: kvm import and fix volume size when lesser than 1GiB (#8500) 2024-01-12 13:32:02 +01:00
components-api New Feature: Import VMware VMs into KVM (#7881) 2023-12-07 12:59:56 +05:30
orchestration Fix host stuck in connecting state (#8502) 2024-01-15 13:56:34 +05:30
schema Fix reorder/list pools when cluster details are not set, while deploying vm / attaching volume (#8373) 2024-01-10 18:13:32 +05:30
service pre-commit: standardize more line endings to `lf` (#7804) 2023-10-18 11:25:00 +02:00
storage Fix host stuck in connecting state (#8502) 2024-01-15 13:56:34 +05:30
userdata mvn: fix userdata dependency version (#8017) 2023-09-28 14:37:49 +05:30
pom.xml ObjectStore Framework with MinIO and Simulator plugins (#7752) 2023-12-01 17:51:00 +05:30