Apply the review comments from the first round on #13061:
* FlashArrayAdapter.snapshot() and both getSnapshot() entry points now
wrap the returned FlashArrayVolume in withAddressType(). Without this,
snapshots taken against an NVMe-TCP pool had the constructor-default
AddressType.FIBERWWN and ProviderSnapshot.getAddress() emitted an FC
style WWN instead of the NVMe EUI-128, which the adaptive driver then
persisted as the snapshot path. Verified end-to-end against Purity 6.7.7:
a fresh NVMe-TCP snapshot now lands with install_path starting 006c... ,
matching the source volume's EUI (previously it was 6-24a9370...).
* FlashArrayAdapter.attach() - retry path after 'Connection already
exists' no longer requires a hostgroup-scoped match for NVMe-TCP. If
hostgroup is not configured, or the existing connection is host-scoped,
fall back to matching by host name, same as the Fibre Channel branch.
Also normalize the 'volume lun is not found' message when no
connection list is returned.
* FlashArrayAdapter.attach() - initial 'Volume attach did not return lun
information' exception message now mentions both lun (FC) and nsid
(NVMe-TCP) so the error is not misleading on NVMe deployments.
* FlashArrayAdapter.getVolumeByAddress() - validate the EUI-128 length
before slicing. A short/malformed address used to throw
StringIndexOutOfBoundsException deep inside getFlashArrayItem and be
swallowed as 'not found'; now a clear RuntimeException is raised with
the expected vs actual length.
* FlashArrayVolume.getAddress() - same defensive check when building an
EUI-128 from the FlashArray volume serial; if the serial is shorter
than 24 hex chars, fail with a clear message instead of SIOOBE.
* MultipathNVMeOFAdapterBase.connectPhysicalDisk() - Integer.parseInt of
the STORAGE_POOL_DISK_WAIT detail is now guarded; a non-numeric value
falls back to the default rather than aborting the connect.
* MultipathNVMeOFAdapterBase.rescanAllControllers() - honour the boolean
return from Process.waitFor(). If an nvme ns-rescan invocation does
not complete in NS_RESCAN_TIMEOUT_SECS we destroyForcibly() it, so
hung nvme-cli processes do not accumulate while the namespace poll
loop retries.
* NVMeTCPAdapter - rename LOGGER_NVMETCP to LOGGER to match the naming
convention used in the other KVM adapters.
Signed-off-by: Eugenio Grosso <eugenio.grosso@gmail.com>
The adaptive storage framework hard-coded FiberChannel as the KVM-side
pool type for every provider it fronts. With a separate NVMeTCP pool
type now available (and a dedicated NVMe-oF adapter on the KVM side),
teach the lifecycle to route a pool to the right adapter based on a
transport= URL parameter:
https://user:pass@host/api?...&transport=nvme-tcp
-> StoragePoolType.NVMeTCP -> NVMeTCPAdapter on the KVM host
When the query parameter is absent the default stays FiberChannel, so
existing FC deployments on Primera or FlashArray continue to work
unchanged.
The choice is made in the shared AdaptiveDataStoreLifeCycleImpl rather
than inside each vendor plugin so every adaptive provider (FlashArray,
Primera, any future one) speaks the same configuration vocabulary.
Teach FlashArrayAdapter to talk to a pool over NVMe over TCP instead of
Fibre Channel.
The transport is selected from a new transport= option on the storage
pool URL (or the equivalent storage_pool_details entry), e.g.
https://user:pass@fa:443/api?pod=cs&transport=nvme-tcp&hostgroup=cluster1
Defaults remain Fibre Channel / WWN addressing when transport is absent
or anything other than nvme-tcp, so existing FC pools are unaffected.
Beyond the transport parsing itself the adapter now:
* Tracks a per-pool volumeAddressType (AddressType.NVMETCP or
FIBERWWN) and stamps every volume it hands back to the framework
with it (withAddressType), so the adaptive driver path stores the
correct type=... field in the CloudStack volume path (used later
by the KVM driver to locate the device).
* Attaches pod-backed NVMe-TCP volumes at the host-group level
(POST /connections?host_group_names=...) instead of per-host, so
the array assigns a consistent NSID to every member host; falls
back to per-host attach for FC or when no hostgroup is configured.
* Tolerates a missing nsid in the FlashArray connections response
for NVMe-TCP - Purity does not return one for host-group NVMe
connections; the namespace is identified on the host by EUI-128
from FlashArrayVolume.getAddress(), so a placeholder value is
returned to the caller purely for informational tracking.
* Resolves NVMETCP addresses back to volumes in getVolumeByAddress
by reversing the EUI-128 layout (strip optional eui. prefix, drop
leading 00 and the embedded Pure OUI).
* Indexes NVMe connections in getConnectionIdMap by host name (the
array returns one entry per host inside a host-group connection),
so connid.<hostname> tokens in the path still match in
parseAndValidatePath on the KVM side.
Followed by a matching adaptive/KVM driver change (separate commit).
Preparatory data-model changes for NVMe-TCP support on the adaptive
storage framework. No behaviour change for existing Fibre Channel
users - the extra enum value, field, and getter/setter are only
exercised by callers that explicitly use them.
ProviderVolume.AddressType gains a NVMETCP value alongside FIBERWWN,
so adapters can declare that a volume is addressed by an NVMe EUI-128
(NGUID) rather than a SCSI WWN.
FlashArrayVolume.getAddress() produces the NGUID layout expected by
the Linux kernel for a FlashArray NVMe namespace:
00 + serial[0:14] + 24a937 (Pure 6-hex OUI) + serial[14:24]
which matches the /dev/disk/by-id/nvme-eui.<id> symlink emitted by
udev. Fibre Channel callers (addressType != NVMETCP) still get the
existing 6 + 24a9370 + serial form.
FlashArrayConnection gains a nsid field to carry the namespace id the
FlashArray REST API attaches to host-group-scoped NVMe connections,
when it is present.
* PowerFlex/ScaleIO client initialization, authentication and command execution improvements
* Migrate VM with volume not supported yet for PowerFlex/ScaleIO
* review changes
* Linstor: Refactor resource creation methods to LinstorUtil
Move reusable methods from LinstorPrimaryDataStoreDriverImpl to LinstorUtil
to enable sharing with other components:
- logLinstorAnswer, logLinstorAnswers, checkLinstorAnswersThrow
- getRscGrp, getEncryptedLayerList, applyQoSSettings
- createResourceBase, createResource, spawnResource
- canShareTemplateForResourceGroup, foundShareableTemplate
Add LIN_PROP_DRBDOPT_EXACT_SIZE constant and exactSize parameter support
for DRBD exact-size property handling during resource creation.
* Linstor: Add LinstorDataMotionStrategy for VM live migration
Implement DataMotionStrategy for live migration of VMs with volumes on
Linstor or other primary storage.
Key features:
- Support live migration with storage from other primary storages
- Preserve DRBD exact-size property during migration
* 4.22:
fix install path for systemvm templates when introducing new sec storage (#11605)
fix Sensitive Data Exposure Through Exception Logging in OVM Hypervis… (#12032)
Fix snapshot physical size after migration (#12166)
ConfigDrive: use file absolute path instead of canonical path to create ISO (#11623)
Add log for null templateVO (#12406)
snapshot: fix listSnapshots for volume which got delete and whose storage pool got deleted (#12433)
Notify user if template upgrade is not required (#12483)
Fix: proper permissions for systemvm template registrations on hardened systems (#12098)
Allow modification of user vm details if user.vm.readonly.details is empty (#10456)
NPE fix while deleting storage pool when pool has detached volumes (#12451)
* NPE fix while deleting storage pool when pool has detached volumes
* review
* unit tests
* Added log for volumes not attached to any VMs
* update filter, log and test
* updated volume dao method names returning non destroyed volumes
* build fix
---------
Co-authored-by: dahn <daan@onecht.net>
This PR aligns the use of terminology, renaming VM / virtual machine references to 'Instance' and also capitalising the terms Templates, Network, Snapshot, User, Account in CloudStack APIs, error and log messages, events, tooltips, etc. Many typos, grammar and spelling mistakes were fixed, also terms like IPv4, VPN, VPC, etc. were properly capitalised. Some error messages were cleaned for better readability. The test cases, expecting some exception strings were adjusted accordingly.
Here is the wiki page, describing the changes in details:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Object+Naming+and+Title+Case+Convention
---------
Co-authored-by: Manoj Kumar <manojkr.itbhu@gmail.com>
Co-authored-by: Harikrishna <harikrishna.patnala@gmail.com>
* Migrate volume improvements, to bypass secondary storage when copy volume between pools is allowed directly
* Bypass secondary storage for copy volume between zone-wide pools and
- local storage on host in the same zone
- cluser-wide pools in the same zone
* Bypass secondary storage for volumes on ceph/rdb pool when the scope permits
* Fix dest disk format while migrating volume from ceph/rbd to nfs, and some code improvements
* unit tests
* Update suitable disk offering(s) for volume(s) after migrate VM with volumes when change in pool type (shared or local)
Currently, Migrate VM with volume(s) bypasses the service and disk offerings of the volumes, as the target pools for migration are specified,
which ignores the offerings. Offering change is required when pool type (shared or local) is changed, mainly
- when volume on shared pool is migrated to local pool
- when volume on local pool is migrated to shared pool
* Update with proper message while migrate volume when target pool and offering type mismatches (both are not shared/local)
* Consider host scope first during endpoint selection while copying between primary storages
* Update disk offering count (for listDiskOfferings api) while removing offerings with tags mismatch with storage tags
* storage: change storage pool to Up state when cancel storage migration
* Update 11773: connect host to shared pool after cancelling storage migration
* Update 11773: update db only
* Update 11773: skip capacity update for storpool
Eugenio Grosso
Eugenio Grosso
Daan Hoogland
sandeeplocharla
poddm
Wei Zhou
Suresh Kumar Anaparti
owsferraro
Klaus de Freitas Dornsbach
Abhisar Sinha
Rene Peinthor
Abhishek Kumar
vladimirpetrov
dahn
Harikrishna Patnala
John Bampton
Manoj Kumar
Henrique Sato