Commit Graph

36813 Commits

Author SHA1 Message Date
dahn 6b25ed7a02
prevent an NPE on an uninitialised TemplateObject (#8898)
* prevent an NPE on an uninitialised TemplateObject

* move npe handler up-stack

* Update engine/storage/image/src/main/java/org/apache/cloudstack/storage/image/store/TemplateObject.java

* catch yet one level up

* Update engine/orchestration/src/main/java/org/apache/cloudstack/engine/orchestration/VolumeOrchestrator.java

* Update engine/storage/image/src/main/java/org/apache/cloudstack/storage/image/store/TemplateObject.java

* extra guard

* Revert "prevent an NPE on an uninitialised TemplateObject"

This reverts commit e602a65ea62e4707828483a4ddea288d81ff06f5.
2024-06-26 21:02:08 +05:30
Wei Zhou 22cd00ffb1
veeam: fix issues with PreSetup and DVS and Solidfire (#9256)
* Veeam: find storage pool by path for PreSetup and VMFS

* Veeam: support VMware distributed virtual switch

* Veeam: sync volumes on Solidfire after backup restoration

user faced the issue that backup is restored but the DATA disk is gone (ROOT disk is ok)
```
2024-05-03 12:00:32,868 ERROR [o.a.c.b.BackupManagerImpl] (API-Job-Executor-13:ctx-aa8a1d85 job-149661 ctx-73328567) (logid:6510cf06) Failed to import VM [vmInternalName: i-169-9679-VM] from backup restoration [{"backupType":"Full","externalId":"821ca400-a5da-4282-bf3f-7c7e38a6cdb4","id":257,"uuid":"69399101-5cbd-461c-8a48-f0c70eac0b24","vmId":9679}] with hypervisor [type: VMware] due to: [Couldn't find storage pool -iqn.2010-01.com.solidfire:3p53.data-9679.221-0].
```

On managed storage, the datastore name of DATA disk is determined by the iscsi_name of the volume.

* Veeam: set correct path for DATA disks on solidfire
2024-06-26 18:02:25 +05:30
Abhishek Kumar b22315db85
server: event for HA vm start (#9202) 2024-06-26 15:38:47 +05:30
Abhishek Kumar 7a8066d3ca
server: fix resource count updated unmanage vm (#8169)
Do not decrement resource count for a running VM twice while unmanaging a VM
2024-06-26 14:34:32 +05:30
Wei Zhou 52247c57bb
agent: reconnect after waiting 5 seconds (#9258)
* agent: reconnect after waiting 5 seconds

* Update agent/src/main/java/com/cloud/agent/Agent.java

* Update agent/src/main/java/com/cloud/agent/Agent.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-06-26 14:33:26 +05:30
slavkap f7f7013ed0
Added ostypeid in the snapshot response (#9150) 2024-06-26 14:18:38 +05:30
slavkap 6c06e85c80
Temporarily backup StorPool volume before expunge (#8843)
* Temporarily backup StorPool volume before expunge

Sometimes the users delete the volumes by mistake. This enhancment
provides a solution to backup the volume before it's deleted. The user
will be able to see the snapshot in CloudStack UI/CLI and create only a
volume from it.
A task will check (by default on every 5mins) if the snapshots are
deleted from StorPool

Global settings to enable the delay delete option:

`storpool.delete.after.interval` - The interval (in seconds) after the StorPool snapshot will be deleted
`storpool.list.snapshots.delete.after.interval` - The interval (in seconds) to fetch the StorPool snapshots with deleteAfter flag

Minor fix when deleting snapshots

* added Apache licence

* addressed comments
2024-06-26 13:58:04 +05:30
Wei Zhou 121839277b
UI: ignore error host is already in the database when create a zone (#9231) 2024-06-26 13:29:35 +05:30
Wei Zhou ae4b6d3b6c
CKS/calico: set arp_ignore and arp_announce to 0 in k8s controller/nodes (#9186) 2024-06-26 12:18:50 +05:30
Vishesh 4f85c3fd23
Honour listall param for list templates api (#9300)
* Honour listall param for list templates api

* Don't show templates from another project inside a project view
2024-06-26 12:09:56 +05:30
Vishesh dc74d5ba88
Let network guru decide if ipv6 cidr size can't be equal to 64 (#9289) 2024-06-26 02:43:26 +05:30
Vishesh 2bf36ef935
DRS: Ensure the destination host is part of the same cluster (#9245)
* DRS: Ensure the destination host is part of the same cluster

* Add and fix unit tests
2024-06-26 02:09:26 +05:30
dahn 6d7c042bc1
Accept a role ID on linking an account to LDAP (#8236)
* accept role on link account to ldap

* reformat tests

* validation

* Update plugins/user-authenticators/ldap/src/main/java/org/apache/cloudstack/api/command/LinkAccountToLdapCmd.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-06-26 01:26:28 +05:30
SadiJr 7f0d9a0304
[Veeam] Check for failures in the restore process (#7224)
* Validate failure state in Veeam restore process

* Address Daan review, and properly call method

* Address bryan's reviews

* remove return

Co-authored-by: SadiJr <sadi@scclouds.com.br>
Co-authored-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-06-26 00:41:38 +05:30
Abhisar Sinha 4eb43651e2
Ability to specify NFS mount options while adding a primary storage and modify them on a pre-existing primary storage (#8947)
* Ability to specify NFS mount options while adding a primary storage and modify it later

* Pull 8947: Rename all occurrence of nfsopt to nfsMountOpt and added nfsMountOpts to ApiConstants

* Pull 8947: Refactor code - move into separate methods

* Pull 8947: CollectionsUtils.isNotEmpty and switch statement in LibvirtStoragePoolDef.java

* Pull 8947: UI - cancel maintainenace will remount the storage pool and apply the options

* Pull 8947: UI - moved edit NFS mount options to edit Primary Storage form

* Pull 8947: UI - moved 'NFS Mount Options' to below 'Type' in dataview

* Pull 8947: Fixed message in AddPrimaryStorage.vue

* Pull 8947: Convert _nfsmountOpts to Set in libvirtStoragePoolDef

* Pull 8947: Throw exception and log error if mount fails due to incorrect mount option

* Pull 8947: Added UT and moved integration test to component/maint

* Pull 8947: Review comments

* Pull 8947: Removed password from integration test

* Pull 8947: move details allocation to inside the if loop in getStoragePoolNFSMountOpts

* Pull 8947: Fixed a bug in AddPrimaryStorage.vue

* Pull 8947: Pool should remain in maintenance mode if mount fails

* Pull 8947: Removed password from integration test

* Pull 8947: Added UT

* Pull 8875: Fixed a bug in CloudStackPrimaryDataStoreLifeCycleImplTest

* Pull 8875: Fixed a bug in LibvirtStoragePoolDefTest

* Pull 8947: minor code restructuring

* Pull 8947 : added some ut for coverage

* Fix LibvirtStorageAdapterTest UT
2024-06-25 23:45:35 +05:30
Suresh Kumar Anaparti 620ed164d8
VMware: Improve error messaging / logs when starting non-user VMs, and secondary storage not available or doesn't have enough capacity (#9207) 2024-06-25 12:25:42 +05:30
Rene Glover 6ee6603359
Updates to HPE-Primera and Pure FlashArray Drivers to use Host-based VLUN Assignments (#8889)
* Updates to change PUre and Primera to host-centric vlun assignments; various small bug fixes

* update to add timestamp when deleting pure volumes to avoid future conflicts

* update to migrate to properly check disk offering is valid for the target storage pool

* Updates to change PUre and Primera to host-centric vlun assignments; various small bug fixes

* update to add timestamp when deleting pure volumes to avoid future conflicts

* update to migrate to properly check disk offering is valid for the target storage pool

* improve error handling when copying volumes to add precision to which step failed

* rename pure volume before delete to avoid conflicts if the same name is used before its expunged on the array

* remove dead code in AdaptiveDataStoreLifeCycleImpl.java

* Fix issues found in PR checks

* fix session refresh TTL logic

* updates from PR comments

* logic to delete by path ONLY on supported OUI

* fix to StorageSystemDataMotionStrategy compile error

* change noisy debug message to trace message

* fix double callback call in handleVolumeMigrationFromNonManagedStorageToManagedStorage

* fix for flash array delete error

* fix typo in StorageSystemDataMotionStrategy

* change copyVolume to use writeback to speed up copy ops

* remove returning PrimaryStorageDownloadAnswer when connectPhysicalDisk returns false during KVMStorageProcessor template copy

* remove change to only set UUID on snapshot if it is a vmSnapshot

* reverting change to UserVmManagerImpl.configureCustomRootDiskSize

* add error checking/simplification per comments from @slavkap

* Update engine/storage/datamotion/src/main/java/org/apache/cloudstack/storage/motion/StorageSystemDataMotionStrategy.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* address PR comments from @sureshanaparti

---------

Co-authored-by: GLOVER RENE <rg9975@cs419-mgmtserver.rg9975nprd.app.ecp.att.com>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-06-25 10:35:39 +05:30
Vishesh 351de5fabd
engine/orchestration: Update overcommit ratio during live VM migration (#9178)
During live migration of a VM from between hosts having different cgroup versions (cgroupv2 & cgroup), overcommit ratio is ignored.

This PR fixes the above issue.
2024-06-24 20:45:31 +05:30
Rohit Yadav 42e71175d7 Merge remote-tracking branch 'origin/4.18' into 4.19 2024-06-24 20:39:55 +05:30
Abhishek Kumar 4d6df586da
engine-schema: fix options for host.allocators.order config (#8733)
With d8c7e34b38 options were added to the host.allocators.order config. Currently, it allows adding only FirstFitRouting as the value. This PR fixes the behaviour and allows other host allocators to be added.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-24 15:55:48 +05:30
Abhishek Kumar de266a4cb8
api,server: allow updating hypervisor capabilities with hypervisor and version (#8475)
* api,server: allow updating hypervisor capabilities with hypervisor and
version

hypervisor and hypervisorversion parameter added to the
updateHypervisorCapabilities API.

* param description

* Update server/src/main/java/com/cloud/server/ManagementServerImpl.java

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: Henrique Sato <henriquesato2003@gmail.com>
Co-authored-by: Vishesh <vishesh92@gmail.com>
2024-06-24 14:41:01 +05:30
Wei Zhou fcfa371337
kvm: support Rocky/RHEL/OL/Alma in the same cluster (#8641)
* kvm: support Rocky/RHEL/OL/Alma in the same cluster

* Update PR#8641: add unit tests
2024-06-24 13:10:41 +05:30
slavkap 8b07b66f14
Fix volume snapshot of encrypted NFS/StorPool volume (#8873)
* Fix volume snapshot of encrypted NFS/StorPool volume

* remove comments

* removed invoking the real qemu convert command

* fix UnsatisfiedLink error in unit tests

* addressed comments

extracted method
2024-06-24 13:09:21 +05:30
João Jandre 3e30283500
Fix migration from local storage to NFS in KVM (#8909)
* fix migration from local to nfs

* remove unused imports

* remove dead code
2024-06-24 13:02:48 +05:30
Wei Zhou f944d4c61d
debian: add cpu-checker to debian/control (#9263) 2024-06-24 13:00:40 +05:30
Pearl Dsilva f792684b9c
Support migration of VM imported from a remote host (#9259) 2024-06-24 12:46:21 +05:30
Suresh Kumar Anaparti c17aa0d9ad
Import Remote KVM VM logging improvements (#9284) 2024-06-24 11:34:37 +05:30
Vishesh 6a518e29b7
Allow deletion of external managed cks nodes (#9183)
* Allow deleteion of external managed cks nodes

* Fix unit tests

* Update plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/KubernetesClusterHelperImpl.java

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-23 22:08:13 +05:30
Rene Peinthor f4612c51ec
libvirtstorage: Make sure netfs storage was really mounted (#8887) 2024-06-23 19:41:02 +05:30
dahn 7e00678691
set isSystem for SVM IPs (#9281)
* set isSystem for SSVM IPs

* Revert "set isSystem for SSVM IPs"

This reverts commit 4ba71b3d6b4ea377f6778d166ff9da7d41e5d007.

* set isSystem flag for SSVMs
2024-06-23 19:36:45 +05:30
Vishesh 59e9ab9efe
Fix volume response for service offering with disk offering (#9273) 2024-06-22 11:17:11 +05:30
Suresh Kumar Anaparti 9055610034
Remove duplicate network state checks before shutdown network (#8462) 2024-06-21 14:12:07 +02:00
Wei Zhou 313a165e62
server: add global setting consoleproxy.sslEnabled (#8809) 2024-06-21 15:56:20 +05:30
Vishesh 674495b162
Fixup startVM on simulator (#9199) 2024-06-21 15:53:45 +05:30
Wei Zhou 0c422aca76
server: fix additional zones cannot be removed (#9261)
Got an exception when delete a zone

```
com.cloud.utils.exception.CloudRuntimeException: The zone cannot be deleted because there are Secondary storages in this zone
```
2024-06-21 15:44:30 +05:30
dahn 60f234c682
remove Project Template Permissions inhibition (#9196) 2024-06-21 15:20:49 +05:30
Gabriel Pordeus Santos 083ac069ca
fix assignvm template permission check (#8886) 2024-06-21 10:58:46 +02:00
Wei Zhou ded7b4dbe5
test: fix test failure on ubuntu 24.04: "top: unknown option 'n'" (#9262) 2024-06-21 10:46:06 +05:30
Suresh Kumar Anaparti 5ab23cd9c9
Timeout config to copy the disks of remote KVM instance while importing the instance from an external host (#9213)
* Added timeout config to copy the disks of remote KVM instance while importing the instance from an external host

* Updated copy config units to mins

* Cleanup remote converted file and local file when copy failed
2024-06-21 10:28:18 +05:30
Abhishek Kumar 097359bef9
plugins/shutdown: fix triggerShutdown scheduling and response (#9276)
Earlier the triggerShutdown API would immediately shutdown the MS and if
it is the same MS on which API is called it would lead to error in the
API call. This change adds a delay to the process so the MS would be
able to send response to the API.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-21 10:12:16 +05:30
João Jandre ae3fa5d0de
Add configuration to limit the number of rows deleted from vm_stats (#8740)
Co-authored-by: João Jandre <joao@scclouds.com.br>
2024-06-20 14:26:36 +02:00
Michael Wodniok 7dce3d87d4
[linstor] Fix revertSnapshot (#9271)
Signed-off-by: Michael Wodniok (WorNet AG) <michael.wodniok@wor.net>
Co-authored-by: Michael Wodniok <michael.wodniok@wor.net>
2024-06-20 10:52:49 +02:00
dahn 84b91cc9a3
assertion improvement on cluster DRS (#8508)
* PR template enhanced

* assertion improvement on cluster DRS

* Update test/integration/smoke/test_cluster_drs.py

* Fix host id check for balanced drs e2e test

* Update test/integration/smoke/test_cluster_drs.py

Co-authored-by: Vishesh <vishesh92@gmail.com>
2024-06-20 10:51:30 +05:30
Daan Hoogland 3997e59678 Merge release branch 4.18 to 4.19
* 4.18:
  Update extraconfig for platform param in xen/xcpng (#9248)
2024-06-19 18:55:29 +02:00
Wei Zhou 81269ad62a
VNF: add API listVnfAppliances (#8994) 2024-06-19 19:49:21 +05:30
Harikrishna 2315a73a20
User friendly name of Downloaded Templates Volumes and ISOs (#9252) 2024-06-19 12:47:43 +02:00
Abhishek Kumar 7a0cd55e98
ui: fix user create featured template from volume (#9253)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-19 08:52:12 +02:00
Wei Zhou 227c15624d
vxlan: do not create duplicated network for private gateway (#9232) 2024-06-19 09:44:49 +03:00
Suresh Kumar Anaparti cc52b38e54
Update extraconfig for platform param in xen/xcpng (#9248)
* Update extraconfig for platform param in xen/xcpng

* Fix map param key, not to replace '-' with '_' (replace only applicable to param / map-param)

* Added unit tests

* Add license for tests file
2024-06-18 23:39:50 +05:30
Suresh Kumar Anaparti 8b02624e61
User data content size validation, and related code improvements (#8418)
Co-authored-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-06-18 20:07:13 +02:00