`reboot.host.and.alert.management.on.heartbeat.timeout` has to be
disabled. Even the high availability isn't enabled when there is an
issue with a storage CloudStack will reboot the host
* NAS backup: resume paused VM on backup failure and fix missing exit
When a NAS backup job fails (e.g. due to backup storage being full or
I/O errors), the VM may remain indefinitely paused because:
1. The cleanup() function never checks or resumes the VM's paused state
that was set by virsh backup-begin during the push backup operation.
2. The 'Failed' case in the backup job monitoring loop calls cleanup()
but lacks an 'exit' statement, causing an infinite loop where the
script repeatedly detects the failed job and calls cleanup().
3. Similarly, backup_stopped_vm() calls cleanup() on qemu-img convert
failure but does not exit, allowing the loop to continue with
subsequent disks despite the failure.
This fix:
- Adds VM state detection and resume to cleanup(), ensuring the VM is
always resumed if found in a paused state during error handling
- Adds missing 'exit 1' after cleanup() in the Failed backup job case
to prevent the infinite monitoring loop
- Adds missing 'exit 1' after cleanup() in backup_stopped_vm() on
qemu-img convert failure
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* ci: retrigger workflow (flaky/stale shards)
* nasbackup.sh: keep cleanup() best-effort if domstate fails
With set -eo pipefail, a non-zero virsh domstate (libvirt unavailable or the
domain gone) in the vm_state assignment would abort cleanup() before the
rm/umount/rmdir. Append '|| true' so cleanup always runs to completion.
Addresses the review suggestion (Copilot, endorsed by @abh1sar and @weizhouapache).
Signed-off-by: James Peru <jmsperu@gmail.com>
---------
Signed-off-by: James Peru <jmsperu@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: jmsperu <jmsperu@users.noreply.github.com>
PR #13254 added an 'allocated' case to Status.vue getText() that calls
$t('label.allocated'), but did not update the test mock data, causing
two Allocated badge tests to fail in CI.
* Async event bus publishing in AsyncJobManagerImpl to reduce API thread contention
publishOnEventBus() was calling _messageBus.publish() synchronously on the
request thread, which blocks on MessageBusBase$Gate (an exclusive mutex).
JFR analysis showed this causing up to 107ms waits on Jetty request threads,
contributing to 502 errors from the upstream load balancer.
Move event bus publishing to a dedicated single-threaded executor so API
request threads are no longer blocked by Gate contention. Event ordering
is preserved by the single-threaded executor.
* Wrap async event bus publish in ManagedContextRunnable for DB connection safety
The sole subscriber (ApiServer.handleAsyncJobPublishEvent) performs DAO reads
(getUserIncludingRemoved, getAccount, findById) inside its callback.
Without ManagedContextRunnable, the EventBus thread would not have proper
TransactionLegacy lifecycle management, risking DB connection leaks.
* logger fix
* review comments
* review changes
---------
Co-authored-by: Aaron Chung <aaron_chung@apple.com>
* listHosts: add 'core' details
* Include msid response and integrate core details into the base response method
Co-authored-by: abennatan <a_bennatan@apple.com>
* kvm: Add a configuration setting to switch between multicast and evpn VXLAN modes
Using the 'network.vxlan.mode' setting you can switch between the multicast (default) and evpn VXLAN modes on a KVM Agent.
When nothing is configured CloudStack will default to multicast by using the modifyvxlan.sh script in the background.
If this setting is set to 'evpn' the KVM Agent will execute the 'modifyvxlan-evpn.sh' script which will configure the VXLAN
devices for EVPN (usually with FRRouting with BGP) mode.
This removes the need to manually replace a shell script on the hypervisor to switch modes.
Existing environments are not touched by this and it is safe to add this setting a an environment already using EVPN for the
VXLAN deployment.
* Add network.vxlan.mode to agent.properties
Make sure there is an example in the agent.properties file so people
can easily discover this configuration setting exists
* linstor: update java-linstor to 0.7.0 to support auth token api
With Linstor 1.34.0 a new authentication mode is supported:
* Bearer token
To support that it had to be implemented in the java-linstor library
and we need to store the auth token per storage pool.
Also per default with this auth mode Linstor will run with
HTTPS enabled, so we also have to support that.
* ui: seed default-on toggles in zone wizard while hidden
StaticInputsForm.fillValue() only seeded defaults for currently-displayed
fields, so a display-gated switch with checked:true bound to an undefined
value and rendered as off once revealed. Seed checked switch/checkbox
fields even while hidden, so the Linstor 'Allow self-signed certificate'
toggle defaults on in the zone creation wizard.
Adds two new operator-level configuration keys to control the number of secondary storage copies made for public and private templates, decoupling replica count from template visibility.
- secstorage.public.template.copy.max (default: 0 = all stores, preserving existing behavior)
- secstorage.private.template.copy.max (default: 1, preserving existing behavior)
* refactor vmschedule to make it more generic and add new commands for it
* fixup
* update marvin tests to use the new APIs
* Update UI to use new APIs
* minor fixups
* Add support scheduling min & max for autoscaling groups
* Add marvin test for ASG
* fixup
* fixup
* fixup
* Fixup
* pre-allocate a second empty cdrom slot at boot (hardcoded)
* drive cdrom slot count via vm.cdrom.max.count ConfigKey
* add vm_iso_map table + VO/DAO
* persist multi-ISO state via vm_iso_map
* carry target cdrom slot through AttachCommand to KVM agent
* enforce per-VM cdrom cap, clamp to hypervisor max
* make detachIso accepts an ISO id
* expose attached ISOs as isos[] in listVirtualMachines response
* extract CDROM_PRIMARY_DEVICE_SEQ constant
* unit tests for cdrom slot allocation logic
* implement multi-ISO attachment and detachment for VMs with enhanced validation
* implement multi-ISO display in InstanceTab with computed property for attached ISOs
* add warning alert for max CDROM selections and enhance global capacity fetching
* enhance ISO attachment validation to handle multiple ISOs and prevent duplicates
* refactor ISO attachment logic for detachment and validation
* add unit tests for ISO detachment resolution and validation logic
* add mock for VmIsoMapDao in UserVmJoinDaoImplTest and set lenient behavior for listByVmId
* refactor ISO attachment logic and enhance UI for multi-CDROM management
* refactor ISO attachment methods to use VM ID and improve parameter handling
* remove unnecessary mock for VM ISO mapping in TemplateManagerImplTest
* add 'since' attribute to ISO detach command parameter description
* scope vm.cdrom.max.count to cluster
* add support for configurable CD-ROM count per VM and improve handling in TemplateManager
* add HostDetailsDao mock to UserVmJoinDaoImplTest
* fix: handle null poolId when loading attached ISO slots in prepareIsoForVmProfile
* implement listByIsoId method in VmIsoMapDao and update TemplateManagerImpl for ISO deletion checks
* improve logging messages for ISO deletion checks
* add unit tests for CD-ROM handling and enforce limits in TemplateManager
* refactor: update configuration value handling and improve notification logic
* refactor: rename CD-ROM references to ISO and update related logic
* refactor: enhance effective CD-ROM max count logic to handle missing host IDs and improve cluster ID retrieval
* refactor: enhance effective CD-ROM max count logic to handle misconfigurations during VM boot
* refactor: enhance effective CD-ROM max count logic to retrieve host ID from candidates based on hypervisor type
* refactor: enhance host ID retrieval logic for VMs based on hypervisor type
* feat: add bootable ISO flag to AttachedIsoResponse and update UI to display it
* refactor: simplify effectiveMaxCdroms method and improve logging for CD-ROM capacity
* test: update AttachedIsoResponseTest to include bootable flag in constructor tests
* feat: include bootable flag in AttachedIsoResponse for user VMs
* feat: enhance CD-ROM management by defining empty slots for user VMs