Commit Graph

104 Commits

Author SHA1 Message Date
Manoj Kumar 9e535e35d2
Support xz format for template registration (#11786) 2025-10-14 09:13:12 +02:00
Suresh Kumar Anaparti f67b738eb3
Migrate volume improvements, to bypass secondary storage when copy volume between pools is allowed directly (#11625)
* Migrate volume improvements, to bypass secondary storage when copy volume between pools is allowed directly

* Bypass secondary storage for copy volume between zone-wide pools and
- local storage on host in the same zone
- cluser-wide pools in the same zone

* Bypass secondary storage for volumes on ceph/rdb pool when the scope permits

* Fix dest disk format while migrating volume from ceph/rbd to nfs, and some code improvements

* unit tests

* Update suitable disk offering(s) for volume(s) after migrate VM with volumes when change in pool type (shared or local)

Currently, Migrate VM with volume(s) bypasses the service and disk offerings of the volumes, as the target pools for migration are specified,
which ignores the offerings. Offering change is required when pool type (shared or local) is changed, mainly
- when volume on shared pool is migrated to local pool
- when volume on local pool is migrated to shared pool

* Update with proper message while migrate volume when target pool and offering type mismatches (both are not shared/local)

* Consider host scope first during endpoint selection while copying between primary storages

* Update disk offering count (for listDiskOfferings api) while removing offerings with tags mismatch with storage tags
2025-10-09 16:00:46 +05:30
Daan Hoogland 5f63e8493c Merge branch '4.20' 2025-10-07 15:12:53 +02:00
Daan Hoogland aca8235960 Merge branch '4.19' into 4.20 2025-10-07 14:28:47 +02:00
Rene Peinthor a208db54ea
linstor: use sparse/discard qemu-img convert on thin devices (#11787) 2025-10-06 09:10:53 +02:00
Suresh Kumar Anaparti 2c34f5e495
Merge branch '4.20' 2025-08-15 19:54:41 +05:30
Suresh Kumar Anaparti 9111bbd8da
Merge branch '4.19' into 4.20 2025-08-15 19:49:59 +05:30
Rene Peinthor 25f93b1d6b
linstor: fix getVolumeStats if multiple Linstor primary storages are used (#11397)
We didn't account for caching the volume stats for each used Linstor
cluster, so the first asked Linstor cluster would prevent caching
for all the others and so null was returned.

Now we have invalidate counters for each Linstor cluster and
also store the cache result with the Linstor cluster address prefixed.
2025-08-15 19:20:39 +05:30
Daan Hoogland 0b3959221b Merge branch '4.20' 2025-07-29 16:50:55 +02:00
Daan Hoogland 609efcc231 Merge branch '4.19' into 4.20 2025-07-25 22:01:17 +02:00
ghernadi a4263da8ae
linstor: Use template's uuid if pool's downloadPath is null as resource-name (#11053)
Also added an integration test for templates from snapshots
2025-07-25 07:51:11 -04:00
Wei Zhou 842b2f8c24
Merge remote-tracking branch 'apache/4.20' 2025-05-19 21:25:37 +02:00
Harikrishna b17808bfba
Introducing Storage Access Groups for better management for host and storage connections (#10381)
* Introducing Storage Access Groups to define the host and storage pool connections

In CloudStack, when a primary storage is added at the Zone or Cluster scope, it is by default connected to all hosts within that scope. This default behavior can be refined using storage access groups, which allow operators to control and limit which hosts can access specific storage pools.

Storage access groups can be assigned to hosts, clusters, pods, zones, and primary storage pools. When a storage access group is set on a cluster/pod/zone, all hosts within that scope inherit the group. Connectivity between a host and a storage pool is then governed by whether they share the same storage access group.

A storage pool with a storage access group will connect only to hosts that have the same storage access group. A storage pool without a storage access group will connect to all hosts, including those with or without a storage access group.
2025-05-19 11:33:29 +05:30
Rene Peinthor 4259e0b51b
linstor: fix host connect recursion regression (#10878) 2025-05-16 12:37:37 +02:00
Daan Hoogland dd84c74e82 Merge branch '4.19' into 4.20 2025-05-13 11:41:36 +02:00
Rene Peinthor 88ce639255
Linstor: implement volume and storage stats (#10850) 2025-05-13 10:06:35 +02:00
Daan Hoogland 5f93ce71bb Merge branch '4.19' into 4.20 2025-03-27 16:44:42 +01:00
Rene Peinthor f4a7c8ab89
linstor: implement missing deleteDatastore (#10561)
Somehow deleteDatastore was never implemented, that meant:
templates haven't been cleaned up on datastore delete and
also agents have never been informed about storage pool removal.
2025-03-18 08:50:19 -04:00
Daan Hoogland f8adedc280 Merge release branch 4.19 to 4.20
* 4.19:
  linstor: try to delete -rst resource before snapshot backup (#10443)
2025-03-12 11:31:16 +01:00
Rene Peinthor 95c24810ab
linstor: try to delete -rst resource before snapshot backup (#10443)
If a -rst resource wasn't deleted because of a failed copy,
a reoccurring snapshot attempt couldn't be done, because there
was still the "old" -rst resource. To prevent this always
try to remove the -rst resource before, if it doesn't exist it is a noop.
2025-03-10 16:23:01 +01:00
Wei Zhou 42a77c7646
LinstorStorageAdaptor: fix lint error (#10378)
This is found in some PRs

plugins/storage/volume/linstor/src/main/java/com/cloud/hypervisor/kvm/storage/LinstorStorageAdaptor.java:510: poperties ==> properties
2025-02-13 09:12:05 +01:00
Daan Hoogland 4f3e8e8c5a Merge branch '4.19' into 4.20 2025-02-12 15:00:51 +01:00
Rene Peinthor df99a29483
linstor: Fix using multiple primary storage with same linstor-controller (#10280) 2025-02-06 10:18:04 +01:00
Rene Peinthor 55e8eaab89
Linstor: encryption support (#10126)
This introduces a new encryption mode, instead of a simple bool.
Now also storage driver can just provide encrypted volumes to CloudStack.
2025-02-04 15:18:49 +01:00
Daan Hoogland 5167c3b613 Merge branch '4.19' into 4.20 2025-01-21 11:59:43 +01:00
Rene Peinthor 1ff68cf9b1
linstor: Fix ZFS snapshot backup (#10219)
Linstor plugin used the wrong zfs dataset path to hide/unhide
the snapshot device.
Also don't use the full path to the zfs binary.
2025-01-21 15:40:17 +05:30
Vishesh a4224e58cc
Improve logging to include more identifiable information (#9873)
* Improve logging to include more identifiable information for kvm plugin

* Update logging for scaleio plugin

* Improve logging to include more identifiable information for default volume storage plugin

* Improve logging to include more identifiable information for agent managers

* Improve logging to include more identifiable information for Listeners

* Replace ids with objects or uuids


* Improve logging to include more identifiable information for engine

* Improve logging to include more identifiable information for server

* Fixups in engine

* Improve logging to include more identifiable information for plugins

* Improve logging to include more identifiable information for Cmd classes

* Fix toString method for StorageFilterTO.java
2025-01-06 16:42:37 +05:30
Daan Hoogland b7f0aac519 Merge branch '4.19' into 4.20 2024-12-20 14:34:39 +01:00
Rene Peinthor a9587bfd2e
kvm-storage: provide isVMMigrate information to storage plugins (#10093)
Particular Linstor needs can use this information to only allow
dual volume access for live migration and not enable it in general,
which can and will lead to data corruption if for some reason
2 VMs get started on 2 different hosts.
2024-12-18 09:13:41 +01:00
Rene Peinthor a2f2e87c12
linstor: improve heartbeat check with also asking linstor (#10105)
If a node doesn't have a DRBD connection to another node,
additionally ask Linstor-Controller if the node is alive.
Otherwise we would have simply said no and the node might still be alive.
This is always the case in a non hyperconverged setup.
2024-12-16 09:59:57 +01:00
Daan Hoogland da54234585 Merge branch '4.19' into 4.20.merge 2024-12-03 16:32:15 +01:00
Rene Peinthor d54b105a03
Linstor: add support for ISO block devices and direct download (#9792) 2024-11-28 17:47:47 +01:00
João Jandre b38ee63c48 Merge branch '4.19' 2024-11-13 10:47:24 -03:00
Rene Peinthor dfe4a67859
kvm: ref-count secondary storage pool usage (#9498)
If a secondary storage pool is used by e.g.
2 concurrent snapshot->template actions,
if the first action finished it removed the netfs mount
point for the other action.
Now the storage pools are usage ref-counted and will only
deleted if there are no more users.
2024-11-13 10:32:46 -03:00
Daan Hoogland f7b7339bcf Merge branch '4.19' 2024-11-07 11:33:50 +01:00
Rohit Yadav 5b7c86aa45 Merge remote-tracking branch 'origin/4.18' into 4.19
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-11-07 14:51:43 +05:30
Rene Peinthor 371e244375
linstor: fix live migrate on non-hyperconverged setups (#9832)
In non-hyperconverged setups, diskless nodes don't have a connection
to each other, so setting properties there had no effect.
Now it is checked if a connection exists,
between the live migration nodes and if not,
it will set the allow-two-primaries on resource-definition level.
2024-11-07 14:46:32 +05:30
Daan Hoogland b068c68bff merge conflict (in loggers) 2024-09-09 15:51:43 +02:00
Daan Hoogland abfa92928c merge conflicts 4.19 -> main 2024-09-09 14:48:20 +02:00
Rene Peinthor 3f5a77ef58
Linstor: Fix migrate primary storage (#9528) 2024-09-09 10:01:41 +02:00
Rohit Yadav 60c9c9adb6 Merge remote-tracking branch 'origin/4.19'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-09-04 13:20:42 +05:30
Rohit Yadav a851ee375c Merge remote-tracking branch 'origin/4.18' into 4.19
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-09-04 12:30:00 +05:30
Rene Peinthor 537c0a1e8d
linstor: set/unset allow-two-primaries and protocol on rc level (#9560) 2024-09-03 16:31:07 +05:30
Nicolas Vazquez 296a44e932
Fix main branch issues (#9553) 2024-08-20 15:26:41 +02:00
Daan Hoogland 5bf81cf002 Merge release branch 4.19 to main
* 4.19:
  linstor: Improve copyPhysicalDisk performance (#9417)
2024-08-20 11:20:39 +02:00
Daan Hoogland 3d8d4875fe Merge release branch 4.18 to 4.19
* 4.18:
  linstor: Improve copyPhysicalDisk performance (#9417)
2024-08-20 11:19:44 +02:00
Rene Peinthor 27f23f4f75
linstor: Improve copyPhysicalDisk performance (#9417)
Tell qemu-img that we don't want to use a write cache (we are a block device)
and also specify that we have zeroed devices in most cases.
2024-08-20 11:17:57 +02:00
Abhishek Kumar 3c9bc07d06 Merge remote-tracking branch 'apache/4.19' 2024-07-06 13:01:50 +05:30
Vishesh ba85b4d4ff
Merge release branch 4.18 to 4.19
* 4.18:
  linstor: deleteAsync fallback to volume UUID if path not set yet (#9325)
  linstor: do not use the same static ApiClient for all calls (#9326)
2024-07-05 12:45:27 +05:30
Rene Peinthor eb1a0ef7f2
linstor: deleteAsync fallback to volume UUID if path not set yet (#9325) 2024-07-04 17:35:01 +05:30