Commit Graph

36033 Commits

Author SHA1 Message Date
Marcus Sorensen f49265c14c
Fix missing code from backport of 4.16 version of dom0 CPU reserve (#374)
Co-authored-by: Marcus Sorensen <mls@apple.com>
2024-02-05 11:53:45 +05:30
Suresh Kumar Anaparti b44710c8a9
Pass StoragePoolType object for poolType dao attribute - fixes conversion to DB column (#371) 2024-02-02 14:10:02 +05:30
Marcus Sorensen e610d2c54c
Fix libvirt domain event listener by properly processing events (#364)
Co-authored-by: Marcus Sorensen <mls@apple.com>
2024-01-29 14:46:20 +05:30
Suresh Kumar Anaparti 0201e0af95
Allocate new ROOT volume (on restore virtual machine operation) only when resource count increment succeeds (#367)
* Allocate new volume on restore virtual machine operation when resource count increment succeeds
- keep them in transaction, and fail operation if resource count increment fails

* Added some (negative) unit tests for restore vm
2024-01-29 14:43:24 +05:30
Marcus Sorensen 40dd867198
Apple base418 storagepooltype as class (#351)
* StoragePoolType as a class

* Fix agent side StoragePoolType enum to class

* Handle StoragePoolType for StoragePoolJoinVO

* Since StoragePoolType is a class, it cannot be converted by @Enumerated annotation.
Implemented conveter class and logic to utilize @Convert annotation.

* Fix UserVMJoinVO for StoragePoolType

* fixed missing imports

* Since StoragePoolType is a class, it cannot be converted by @Enumerated annotation.
Implemented conveter class and logic to utilize @Convert annotation.

* Fixed equals for the enum.

* removed not needed try/catch for prepareAttribute

* Added license to the file.

* Implemented "supportsPhysicalDiskCopy" for storage adaptor. (#352)

Co-authored-by: mprokopchuk <mprokopchuk@apple.com>

* Add javadoc to StoragePoolType class

* Add unit test for StoragePoolType comparisons

* StoragePoolType "==" and ".equals()" fix.

* Fix for abstract storage adaptor set up issue

* review comments

---------

Co-authored-by: Marcus Sorensen <mls@apple.com>
Co-authored-by: mprokopchuk <mprokopchuk@apple.com>
Co-authored-by: mprokopchuk <mprokopchuk@gmail.com>
Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2024-01-25 14:58:44 +05:30
Vishesh 47e53eceed
E2e test resource listing (#363)
* Use dualzones for ci github actions

* Update advdualzone.cfg to be similar to advanced.cfg & fixup test_metrics_api.py

* Add e2e tests for listing of accounts, disk_offerings, domains, hosts, service_offerings, storage_pools, volumes

* Add test for listing volumes with tags filter

* Add check for existing volumes in test_list_volumes

* Wait for volumes to be deleted on cleanup

* Filter out volumes in Destroy state before checking the count of volumes
2024-01-25 14:52:06 +05:30
Suresh Kumar Anaparti 7fef155621
Remove sensitive params (VmPassword, etc) from VMWork log (#369)
* Remove sensitive params (VmPassword, etc) from VMWork log

* Added unit tests

* review comments
2024-01-24 17:49:20 +05:30
Suresh Kumar Anaparti e704b6e492
Fix reorder/list pools when cluster details are not set (#358)
* Fix reorder/list pools when cluster details are not set

* minor code improvements

* added unit tests
2024-01-18 15:29:00 +05:30
kishankavala 99939d22a7
CleanUp Async Jobs after mgmt server maintenance (#356)
* Cleanup Volume AsyncJob after mgmt server stop

* Clean Up Vm async job resources during mggmt server stop

* Use State.isTransitional method to identify trnsition states

* Add cleanup for Network Async Job

* Add license

* Added RevertSnapshotting to volume transition state. Fixed spacing code style

* Added transitional flag in Volume state

* Updated network event for failed job, (re)added cleanup for volumes created from snapshots, and some code improvements

* Added java doc for volume state constructor

* Fixed cleanup SNAPSHOT_ID entry in volume details for failed volumes created from snapshots

---------

Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2024-01-09 17:54:26 +05:30
Suresh Kumar Anaparti 6f4cf60fab
Updated jetty maxFormContentSize value to 1048576 bytes (default is 200000 bytes), to support user data upto 1048576 bytes (#360)
* Updated jetty maxFormContentSize value to 1048576 bytes (default is 200000 bytes)

* Updated content size config using 'max.form.content.size' parameter in server.properties

* Updated content size config parameter (in server.properties) to 'request.content.size'
2024-01-03 18:04:11 +05:30
Harikrishna 28be74e0b9
Add lock mechanism considering template id, pool id, host id (#345)
* Add lock mechanism considering template id, pool id, host id

* Added missing lock
2023-12-06 13:51:25 +05:30
anniejili af4e657aee
Clear pool id if volume in allocated state (#341)
Co-authored-by: Annie Li <ji_li@apple.com>
2023-11-21 15:42:43 +05:30
Vishesh 63a4efa4c9
Use UserVmDao for listVirtualMachines API to increase performance (#343) 2023-11-10 13:08:30 +05:30
Wei Zhou c32d2fa990 CKS: fix wrong format of cluster size on UI (#8182)
(cherry picked from commit e6f048bc2e)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-11-07 21:00:23 +05:30
Rohit Yadav 7260204447 ui: Admin, account and project dashboard improvements
This backports only the dashboard changes from
https://github.com/apache/cloudstack/pull/7956 and
5d9ae31f1b

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 3376f94886)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-11-07 20:59:10 +05:30
Vishesh b9c3752ce0
Fix: Select another pod if all hosts in the pod becomes unavailable (#339) 2023-11-07 15:11:21 +01:00
Vishesh a7c7a33131
Apple base418 agent lock during reconnect (#340)
Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-11-03 16:56:15 +01:00
Abhishek Kumar 42131fdd16 ui: fix bulk delete template from zones (#8118)
Fixes #8083

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
(cherry picked from commit e199678101)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-26 12:21:07 +05:30
Abhishek Kumar bf3dff2f57 marvin,test: fix directdownload template checksum test (#8096)
* marvin,test: fix directdownload template checksum test

During failure while deploying a VM with wrong checksum template, VM may be left in Error state. This PR adds code to delete such VM.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove unnecessary logs

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
(cherry picked from commit a2ec1f3777)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-26 12:21:02 +05:30
Abhishek Kumar 0c96202c7d ui: correctly show volume physical size (#8119)
Fixes #8073

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
(cherry picked from commit f62b634033)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-26 12:20:57 +05:30
Peinthor Rene 8875a94242 linstor: fix template copy on non hyperconverged setups (#8114)
Making a diskful resource was meant as an optimization,
but cannot work on non hyperconverged setups,
as the storage nodes (diskful) are not part of the cloudstack cluster.

(cherry picked from commit 67cb9b9e40)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-26 12:20:51 +05:30
Abhishek Kumar f4b9e6c988 test: add test for standalone snapshot (#8104)
Fixes #8034

Adds the following test for a backed-up snapshot (original template and VM deleted beforehand):
- Create volume from snapshot
- Create a template from the snapshot and deploy a VM using it

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
(cherry picked from commit 540c7b802f)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-26 12:20:43 +05:30
Harikrishna 198e48c7c5 Fix VM snapshot size during storage capacity check (#8101)
(cherry picked from commit 0183e25279)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-26 12:20:37 +05:30
Harikrishna 4c7a81bd82 Fix UUID for child datastores in all cases (#8057)
(cherry picked from commit 76ab621a5a)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-26 12:20:32 +05:30
Peinthor Rene d49b0c4253 linstor: Fix template volume missing on copy node (#8082)
A TODO was overseen and never implemented,
which could trigger the following bug:

If Linstor didn't create a resource (diskless or diskfull) on
the cloudstack choosen node, it would not be able to copy the
template data there, it even seems no error was
triggered and the new template file silently just became
empty/corrupt.

(cherry picked from commit 4a86a0d233)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-26 12:20:28 +05:30
Abhishek Kumar 03fa5799e6 test,refactor: fix test_project_resources cleanup (#8097)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
(cherry picked from commit 065abe2a3b)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-26 12:20:20 +05:30
Abhishek Kumar 6f925f0022 kvm: fix direct download template size (#8093)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
(cherry picked from commit ba24a18f27)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-26 12:20:11 +05:30
Harikrishna d1849a4033 Fix NPE if global setting implicit.host.tags is set to null (#8066)
(cherry picked from commit fb3a2ecb57)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-26 12:19:21 +05:30
slavkap a768c96a6d Create snapshot from VM snapshot without memory for NFS/Local storage (#8117)
(cherry picked from commit 6ae3b73ca2)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-26 12:18:56 +05:30
Nicolas Vazquez 3e2717424d Address review comments (#338)
This adds the missing commit to the fix #335 from the upstream PR:
apache/cloudstack#7977

(cherry picked from commit b5f77f9c3b53af7e8b05730da9807a2c9eb017a5)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-12 14:20:00 +05:30
Harikrishna 84fee7b896 ui: Fix non admin logouts (#8065)
If a user (non-admin) logs out from a session, then login page is not loading completely. Few starter APIs like listIds are failing and showing unauthorised access notification in Login Page. Also if SAML is enabled, it is not getting enabled since the corresponding API are failed. User needs to refresh the browser to get it back.

(cherry picked from commit 8b281284a2)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-11 21:03:09 +05:30
Wei Zhou f570934482 .github: run Sonar Check only on PRs from apache/cloudstack branches (#8058)
This PR fixes #8050

(cherry picked from commit 864a195868)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-11 21:02:57 +05:30
Harikrishna db54a09860 Default value of force should be false for template delete operation (#7731)
* default value of force should be false

* Added force flag in tests

(cherry picked from commit a9f3af85cb)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-11 21:02:48 +05:30
Wei Zhou 846cc2f26c systemvm: remove config in /etc/pam.d/systemd-user to fix user@0.service (#8048)
the service `user@0.service` fails in system vms and virtual routers

This PR removes a change to fix memory leak of SSH connections in the systemvm templates with old linux kernel.

(cherry picked from commit e290ac5451)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-11 21:02:44 +05:30
Rohit Yadav 18e7276df0 storage: allow VM snapshots without memory for KVM when global setting allows (#8062)
This removes the conditional logic where comment notest to remove it
after PR #5297 is merged that is applicable for ACS 4.18+. Only when the
global setting is enabled and memory isn't selected, VM snapshot could
be allowed for VMs on KVM that have qemu-guest-agent running.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 8350ce5aa4)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-11 21:01:31 +05:30
Nicolas Vazquez fe3cf81739 Fix #303: Run recalculation recurrent task only in the longest running management server (#335)
* Run recalculation recurrent task only in the longest running management server

* Fix timeout and recalculation when no child domains

(cherry picked from commit 3f35e3a6ef24017d2f63613ca4362521b1ee78b6)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-11 20:52:57 +05:30
Rohit Yadav a54f37890c
Merge pull request #322 from shapeblue/4.18-migration-issue320
Issue-320: Backport and changes to help with 4.18 migration
2023-10-10 13:15:33 +05:30
dahn 81f9da037b ldap trust map cleanup on domain delete (#7915)
Co-authored-by: Wei Zhou <weizhou@apache.org>
(cherry picked from commit 09ae0499b2)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-09 19:15:47 +05:30
Pearl Dsilva 73c86b8a30 VR live patching: Allow live patch of VPC VRs even if networks are in allocated / shutdown state (#7958)
(cherry picked from commit 951ba04cf0)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-09 19:15:32 +05:30
Fabricio Duarte 4f86c452de Fix 2FA becoming enabled when the user inputs the wrong code during setup (#7972)
(cherry picked from commit ef742210b5)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-09 19:15:25 +05:30
Peinthor Rene 1fdaec5c90 linstor: resize root disk on offerings with different size (#7952)
(cherry picked from commit 96205a51ef)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-09 19:14:28 +05:30
Wei Zhou 24346b5d65 server: fix mysql error when list Shared templates for project (#8020)
(cherry picked from commit 3d8cc63bc3)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-09 19:14:20 +05:30
Wei Zhou 4498c7807b systemvm: check if patchfile exists and is valid tarball before patching (#7953)
(cherry picked from commit 51add0a066)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-09 19:14:14 +05:30
Daniel Augusto Veronezi Salvador 5b7837eae1 Fix: Convert volume to another directory instead of copying it while taking volume snapshots on KVM (#8041)
(cherry picked from commit 9b8eaeea78)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-09 19:12:40 +05:30
Oscar Sandoval 832995abd6 Improve template download error message (#8029)
(cherry picked from commit c0128e2f9a)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-09 19:12:12 +05:30
Wei Zhou 1b36cd9210 test: fix test_vm_autoscaling.py which does not work due to userdata improvement (#7921)
(cherry picked from commit 78411fd405)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-29 14:32:29 +05:30
Rohit Yadav 1b35a5aa1c Fix build, missing imports
Note: these are however redundant, both on main branch and the private
branch across two classes.

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-28 14:31:24 +05:30
Abhishek Kumar 160a13a029 userdata: fix append scenarios (#7741)
Fixes case of appending userdata when both template and vm data are either shellscript or cloudconfig
Fixes error when appending gzip userdata
Fixes case when userdata manual text from VM is not getting decoded-encoded correctly.
Fixes case of appending multipart data when both template and vm data contain same format types.
Refactor - moved validateUserData method to UserDataManager class
Refactor userdata test to check resultant multipart userdata thoroughly

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
(cherry picked from commit 729e6d1446)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-28 12:48:46 +05:30
Marcus Sorensen c01ed90569 Trigger out of band VM state update via libvirt event when VM stops (#7963)
* Trigger out of band VM state update via libvirt event when VM stops

* Add License headers, refactor nested try

---------

Co-authored-by: Marcus Sorensen <mls@apple.com>
(cherry picked from commit 3694667f50)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-28 12:22:15 +05:30
Rohit Yadav e77107819b kvm: skip KVMHostInfoTest when build OS isn't Linux
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-27 17:47:10 +05:30