Commit Graph

170 Commits

Author SHA1 Message Date
Suresh Kumar Anaparti a4a52c9665
Merge branch '4.22' 2026-05-08 20:57:36 +05:30
Suresh Kumar Anaparti 4359198904
KVM Host HA improvements - Fix to not cancel VM HA items when Host HA inspection in progress, and some code improvements (#13088)
* Host HA code improvements

* Fix to not cancel VM HA items when Host HA is enabled & inspection in progress, and some code improvements

- When Host HA inspection in progress, the investigor returns the Host Status as Up which cancels the VM HA items
- Don't cancel the VM HA items, instead reschedule them to try again later

* Changes to consider Recovered/Available Host HA state along with the agent connection status to determine the Host HA inspection in progress or not, and some code improvements
2026-05-08 19:50:50 +05:30
Rene Peinthor 5b9a3d7d32
linstor: Fix a file handle resource leak opening template.properties (#13091) 2026-05-04 14:43:06 +05:30
Sergiy Kukunin 089eb36e47
Linstor: fix create volume from snapshot on primary storage (#13043)
* Linstor: fix create volume from snapshot on primary storage

When creating a volume from a snapshot on Linstor primary storage
(with lin.backup.snapshots=false), the operation fails with:
"Only the following image types are currently supported: VHD, OVA,
QCOW2, RAW (for PowerFlex and FiberChannel)"

Root cause: the Linstor driver does not handle SNAPSHOT -> VOLUME in
its canCopy()/copyAsync() methods. This causes DataMotionServiceImpl
to fall through to StorageSystemDataMotionStrategy (selected because
Linstor advertises STORAGE_SYSTEM_SNAPSHOT=true). That strategy's
verifyFormatWithPoolType() rejects RAW format for Linstor pools,
since RAW is only allowed for PowerFlex and FiberChannel.

Additionally, VolumeOrchestrator.createVolumeFromSnapshot() attempts
to back up the snapshot to secondary storage when the storage plugin
does not advertise CAN_CREATE_TEMPLATE_FROM_SNAPSHOT. This backup
fails because the snapshot only exists on Linstor primary storage.

Fix:
- Add CAN_CREATE_TEMPLATE_FROM_SNAPSHOT capability so the
  orchestrator skips the backup-to-secondary path
- Add canCopySnapshotToVolumeCond() to match SNAPSHOT -> VOLUME
  when both are on the same Linstor primary store
- Wire it into canCopy() to intercept at DataMotionServiceImpl
  before strategy selection, bypassing StorageSystemDataMotionStrategy
- Implement copySnapshotToVolume() which delegates to the existing
  createResourceFromSnapshot() for native Linstor snapshot restore

This follows the same pattern used by the StorPool plugin, which
handles SNAPSHOT -> VOLUME directly in its driver rather than going
through StorageSystemDataMotionStrategy.

Tested on CloudStack 4.22 with Linstor LVM_THIN storage, creating
a volume from a 1TB CNPG Postgres database snapshot. Volume creates
successfully with correct path and deletes cleanly.

* Let CloudRuntimeException propagate from copySnapshotToVolume

Remove try/catch in copySnapshotToVolume so that CloudRuntimeException
from createResourceFromSnapshot propagates to the caller, ensuring
CloudStack properly notices and reports the failure.

* Fix CAN_CREATE_TEMPLATE_FROM_SNAPSHOT breaking template creation

Setting CAN_CREATE_TEMPLATE_FROM_SNAPSHOT unconditionally to true
caused createTemplate from snapshot to take the StorPool-specific
code path in TemplateManagerImpl, which sends a CopyCommand to a
system VM that Linstor cannot handle.

Fix: make CAN_CREATE_TEMPLATE_FROM_SNAPSHOT conditional on the same
flag as STORAGE_SYSTEM_SNAPSHOT (!BackupSnapshots). When snapshots
are backed up to secondary (the default), the old template creation
flow works. When snapshots stay on primary, the direct path is used.

Also fix checkstyle: remove unused DataObject import in test.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-29 11:23:08 +05:30
Suresh Kumar Anaparti ffb8b91ff5
Merge branch '4.22' 2026-01-29 19:42:07 +05:30
Suresh Kumar Anaparti f5778eaa32
Merge branch '4.20' into 4.22 2026-01-29 19:41:29 +05:30
Rene Peinthor 6ba5e08221
Linstor: support live migration from other primary storage (#12532)
* Linstor: Refactor resource creation methods to LinstorUtil

Move reusable methods from LinstorPrimaryDataStoreDriverImpl to LinstorUtil
to enable sharing with other components:
- logLinstorAnswer, logLinstorAnswers, checkLinstorAnswersThrow
- getRscGrp, getEncryptedLayerList, applyQoSSettings
- createResourceBase, createResource, spawnResource
- canShareTemplateForResourceGroup, foundShareableTemplate

Add LIN_PROP_DRBDOPT_EXACT_SIZE constant and exactSize parameter support
for DRBD exact-size property handling during resource creation.

* Linstor: Add LinstorDataMotionStrategy for VM live migration

Implement DataMotionStrategy for live migration of VMs with volumes on
Linstor or other primary storage.

Key features:
- Support live migration with storage from other primary storages
- Preserve DRBD exact-size property during migration
2026-01-29 14:38:12 +05:30
Abhishek Kumar 538578366a Merge remote-tracking branch 'apache/4.22' 2026-01-13 11:49:07 +05:30
Abhishek Kumar 031fbf43d4 Merge remote-tracking branch 'apache/4.20' into 4.22 2026-01-13 11:48:05 +05:30
Rene Peinthor 8dcfc7c767
Linstor fix host picking (#12047) 2026-01-12 11:29:31 +01:00
Suresh Kumar Anaparti 4628385051
Merge branch '4.22' 2026-01-05 16:17:50 +05:30
Suresh Kumar Anaparti 959c5fff68
Merge branch '4.20' into 4.22 2026-01-05 15:57:26 +05:30
Rene Peinthor 10dcbd76f0
linstor: Provide /dev/drbd/by-res/ resource paths to CloudStack (#12300) 2025-12-31 10:55:54 +01:00
dahn c81295439f
removed code in comments (#11145) 2025-12-08 16:31:48 +01:00
Harikrishna Patnala dbda673e1f Updating pom.xml version numbers for release 4.23.0.0-SNAPSHOT
Signed-off-by: Harikrishna Patnala <harikrishna.patnala@gmail.com>
2025-11-05 16:54:39 +05:30
Harikrishna Patnala d160731b9f Updating pom.xml version numbers for release 4.22.1.0-SNAPSHOT
Signed-off-by: Harikrishna Patnala <harikrishna.patnala@gmail.com>
2025-11-05 16:07:07 +05:30
Harikrishna Patnala 71f47d6130 Updating pom.xml version numbers for release 4.22.0.0
Signed-off-by: Harikrishna Patnala <harikrishna.patnala@gmail.com>
2025-10-30 19:23:56 +05:30
John Bampton e7b8326058
pre-commit auto add license for all Markdown files (#11870) 2025-10-24 16:04:22 +05:30
Wei Zhou e333ce9782
Updating pom.xml version numbers for release 4.20.3.0-SNAPSHOT 2025-10-24 09:13:19 +02:00
Wei Zhou 4dc3931233
Updating pom.xml version numbers for release 4.20.2.0
Signed-off-by: Wei Zhou <weizhou@apache.org>
2025-10-16 11:42:56 +02:00
Manoj Kumar 9e535e35d2
Support xz format for template registration (#11786) 2025-10-14 09:13:12 +02:00
Suresh Kumar Anaparti f67b738eb3
Migrate volume improvements, to bypass secondary storage when copy volume between pools is allowed directly (#11625)
* Migrate volume improvements, to bypass secondary storage when copy volume between pools is allowed directly

* Bypass secondary storage for copy volume between zone-wide pools and
- local storage on host in the same zone
- cluser-wide pools in the same zone

* Bypass secondary storage for volumes on ceph/rdb pool when the scope permits

* Fix dest disk format while migrating volume from ceph/rbd to nfs, and some code improvements

* unit tests

* Update suitable disk offering(s) for volume(s) after migrate VM with volumes when change in pool type (shared or local)

Currently, Migrate VM with volume(s) bypasses the service and disk offerings of the volumes, as the target pools for migration are specified,
which ignores the offerings. Offering change is required when pool type (shared or local) is changed, mainly
- when volume on shared pool is migrated to local pool
- when volume on local pool is migrated to shared pool

* Update with proper message while migrate volume when target pool and offering type mismatches (both are not shared/local)

* Consider host scope first during endpoint selection while copying between primary storages

* Update disk offering count (for listDiskOfferings api) while removing offerings with tags mismatch with storage tags
2025-10-09 16:00:46 +05:30
Daan Hoogland 5f63e8493c Merge branch '4.20' 2025-10-07 15:12:53 +02:00
Daan Hoogland aca8235960 Merge branch '4.19' into 4.20 2025-10-07 14:28:47 +02:00
Rene Peinthor a208db54ea
linstor: use sparse/discard qemu-img convert on thin devices (#11787) 2025-10-06 09:10:53 +02:00
Suresh Kumar Anaparti 1033be4b31
Updating pom.xml version numbers for release 4.22.0.0-SNAPSHOT
Signed-off-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-08-28 12:00:42 +05:30
Suresh Kumar Anaparti f9513b47bf
Updating pom.xml version numbers for release 4.21.0.0
Signed-off-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-08-22 11:42:37 +05:30
Suresh Kumar Anaparti 2c34f5e495
Merge branch '4.20' 2025-08-15 19:54:41 +05:30
Suresh Kumar Anaparti 9111bbd8da
Merge branch '4.19' into 4.20 2025-08-15 19:49:59 +05:30
Rene Peinthor 25f93b1d6b
linstor: fix getVolumeStats if multiple Linstor primary storages are used (#11397)
We didn't account for caching the volume stats for each used Linstor
cluster, so the first asked Linstor cluster would prevent caching
for all the others and so null was returned.

Now we have invalidate counters for each Linstor cluster and
also store the cache result with the Linstor cluster address prefixed.
2025-08-15 19:20:39 +05:30
Daan Hoogland 0b3959221b Merge branch '4.20' 2025-07-29 16:50:55 +02:00
Daan Hoogland 609efcc231 Merge branch '4.19' into 4.20 2025-07-25 22:01:17 +02:00
ghernadi a4263da8ae
linstor: Use template's uuid if pool's downloadPath is null as resource-name (#11053)
Also added an integration test for templates from snapshots
2025-07-25 07:51:11 -04:00
Pearl Dsilva 379ee07d88 Updating pom.xml version numbers for release 4.19.4.0-SNAPSHOT
Signed-off-by: Pearl Dsilva <pearl1594@gmail.com>
2025-06-06 18:00:09 +05:30
Pearl Dsilva b5e2c181f9 Updating pom.xml version numbers for release 4.20.2.0-SNAPSHOT
Signed-off-by: Pearl Dsilva <pearl1594@gmail.com>
2025-06-06 15:38:12 +05:30
Pearl Dsilva c61a5eb430 Updating pom.xml version numbers for release 4.20.1.0
Signed-off-by: Pearl Dsilva <pearl1594@gmail.com>
2025-05-30 12:43:00 +05:30
Daan Hoogland 0c7d47138d Updating pom.xml version numbers for release 4.19.3.0
Signed-off-by: Daan Hoogland <daan@onecht.net>
2025-05-30 09:08:58 +02:00
Wei Zhou 842b2f8c24
Merge remote-tracking branch 'apache/4.20' 2025-05-19 21:25:37 +02:00
Harikrishna b17808bfba
Introducing Storage Access Groups for better management for host and storage connections (#10381)
* Introducing Storage Access Groups to define the host and storage pool connections

In CloudStack, when a primary storage is added at the Zone or Cluster scope, it is by default connected to all hosts within that scope. This default behavior can be refined using storage access groups, which allow operators to control and limit which hosts can access specific storage pools.

Storage access groups can be assigned to hosts, clusters, pods, zones, and primary storage pools. When a storage access group is set on a cluster/pod/zone, all hosts within that scope inherit the group. Connectivity between a host and a storage pool is then governed by whether they share the same storage access group.

A storage pool with a storage access group will connect only to hosts that have the same storage access group. A storage pool without a storage access group will connect to all hosts, including those with or without a storage access group.
2025-05-19 11:33:29 +05:30
Rene Peinthor 4259e0b51b
linstor: fix host connect recursion regression (#10878) 2025-05-16 12:37:37 +02:00
Daan Hoogland 64828f66e8 Merge branch '4.20' 2025-05-13 13:34:23 +02:00
Daan Hoogland dd84c74e82 Merge branch '4.19' into 4.20 2025-05-13 11:41:36 +02:00
Rene Peinthor 88ce639255
Linstor: implement volume and storage stats (#10850) 2025-05-13 10:06:35 +02:00
Daan Hoogland 8af021c6f6 Merge branch '4.20' 2025-03-27 17:03:13 +01:00
Daan Hoogland 5f93ce71bb Merge branch '4.19' into 4.20 2025-03-27 16:44:42 +01:00
Rene Peinthor f4a7c8ab89
linstor: implement missing deleteDatastore (#10561)
Somehow deleteDatastore was never implemented, that meant:
templates haven't been cleaned up on datastore delete and
also agents have never been informed about storage pool removal.
2025-03-18 08:50:19 -04:00
Daan Hoogland 9c6f2a9e14 Merge release branch 4.20 to main
* 4.20:
  Fix Stats Collector to not divide by zero (#10492)
  linstor: try to delete -rst resource before snapshot backup (#10443)
2025-03-12 11:31:56 +01:00
Daan Hoogland f8adedc280 Merge release branch 4.19 to 4.20
* 4.19:
  linstor: try to delete -rst resource before snapshot backup (#10443)
2025-03-12 11:31:16 +01:00
Rene Peinthor 95c24810ab
linstor: try to delete -rst resource before snapshot backup (#10443)
If a -rst resource wasn't deleted because of a failed copy,
a reoccurring snapshot attempt couldn't be done, because there
was still the "old" -rst resource. To prevent this always
try to remove the -rst resource before, if it doesn't exist it is a noop.
2025-03-10 16:23:01 +01:00
Daan Hoogland 4a3686297d Updating pom.xml version numbers for release 4.19.3.0-SNAPSHOT
Signed-off-by: Daan Hoogland <daan@onecht.net>
2025-02-25 10:43:11 +01:00