Commit Graph

36003 Commits

Author SHA1 Message Date
Harikrishna 84fee7b896 ui: Fix non admin logouts (#8065)
If a user (non-admin) logs out from a session, then login page is not loading completely. Few starter APIs like listIds are failing and showing unauthorised access notification in Login Page. Also if SAML is enabled, it is not getting enabled since the corresponding API are failed. User needs to refresh the browser to get it back.

(cherry picked from commit 8b281284a2)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-11 21:03:09 +05:30
Wei Zhou f570934482 .github: run Sonar Check only on PRs from apache/cloudstack branches (#8058)
This PR fixes #8050

(cherry picked from commit 864a195868)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-11 21:02:57 +05:30
Harikrishna db54a09860 Default value of force should be false for template delete operation (#7731)
* default value of force should be false

* Added force flag in tests

(cherry picked from commit a9f3af85cb)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-11 21:02:48 +05:30
Wei Zhou 846cc2f26c systemvm: remove config in /etc/pam.d/systemd-user to fix user@0.service (#8048)
the service `user@0.service` fails in system vms and virtual routers

This PR removes a change to fix memory leak of SSH connections in the systemvm templates with old linux kernel.

(cherry picked from commit e290ac5451)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-11 21:02:44 +05:30
Rohit Yadav 18e7276df0 storage: allow VM snapshots without memory for KVM when global setting allows (#8062)
This removes the conditional logic where comment notest to remove it
after PR #5297 is merged that is applicable for ACS 4.18+. Only when the
global setting is enabled and memory isn't selected, VM snapshot could
be allowed for VMs on KVM that have qemu-guest-agent running.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 8350ce5aa4)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-11 21:01:31 +05:30
Nicolas Vazquez fe3cf81739 Fix #303: Run recalculation recurrent task only in the longest running management server (#335)
* Run recalculation recurrent task only in the longest running management server

* Fix timeout and recalculation when no child domains

(cherry picked from commit 3f35e3a6ef24017d2f63613ca4362521b1ee78b6)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-11 20:52:57 +05:30
Rohit Yadav a54f37890c
Merge pull request #322 from shapeblue/4.18-migration-issue320
Issue-320: Backport and changes to help with 4.18 migration
2023-10-10 13:15:33 +05:30
dahn 81f9da037b ldap trust map cleanup on domain delete (#7915)
Co-authored-by: Wei Zhou <weizhou@apache.org>
(cherry picked from commit 09ae0499b2)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-09 19:15:47 +05:30
Pearl Dsilva 73c86b8a30 VR live patching: Allow live patch of VPC VRs even if networks are in allocated / shutdown state (#7958)
(cherry picked from commit 951ba04cf0)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-09 19:15:32 +05:30
Fabricio Duarte 4f86c452de Fix 2FA becoming enabled when the user inputs the wrong code during setup (#7972)
(cherry picked from commit ef742210b5)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-09 19:15:25 +05:30
Peinthor Rene 1fdaec5c90 linstor: resize root disk on offerings with different size (#7952)
(cherry picked from commit 96205a51ef)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-09 19:14:28 +05:30
Wei Zhou 24346b5d65 server: fix mysql error when list Shared templates for project (#8020)
(cherry picked from commit 3d8cc63bc3)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-09 19:14:20 +05:30
Wei Zhou 4498c7807b systemvm: check if patchfile exists and is valid tarball before patching (#7953)
(cherry picked from commit 51add0a066)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-09 19:14:14 +05:30
Daniel Augusto Veronezi Salvador 5b7837eae1 Fix: Convert volume to another directory instead of copying it while taking volume snapshots on KVM (#8041)
(cherry picked from commit 9b8eaeea78)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-09 19:12:40 +05:30
Oscar Sandoval 832995abd6 Improve template download error message (#8029)
(cherry picked from commit c0128e2f9a)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-09 19:12:12 +05:30
Wei Zhou 1b36cd9210 test: fix test_vm_autoscaling.py which does not work due to userdata improvement (#7921)
(cherry picked from commit 78411fd405)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-29 14:32:29 +05:30
Rohit Yadav 1b35a5aa1c Fix build, missing imports
Note: these are however redundant, both on main branch and the private
branch across two classes.

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-28 14:31:24 +05:30
Abhishek Kumar 160a13a029 userdata: fix append scenarios (#7741)
Fixes case of appending userdata when both template and vm data are either shellscript or cloudconfig
Fixes error when appending gzip userdata
Fixes case when userdata manual text from VM is not getting decoded-encoded correctly.
Fixes case of appending multipart data when both template and vm data contain same format types.
Refactor - moved validateUserData method to UserDataManager class
Refactor userdata test to check resultant multipart userdata thoroughly

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
(cherry picked from commit 729e6d1446)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-28 12:48:46 +05:30
Marcus Sorensen c01ed90569 Trigger out of band VM state update via libvirt event when VM stops (#7963)
* Trigger out of band VM state update via libvirt event when VM stops

* Add License headers, refactor nested try

---------

Co-authored-by: Marcus Sorensen <mls@apple.com>
(cherry picked from commit 3694667f50)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-28 12:22:15 +05:30
Rohit Yadav e77107819b kvm: skip KVMHostInfoTest when build OS isn't Linux
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-27 17:47:10 +05:30
Marcus Sorensen 33a5c159a3 KVM Agent config to reserve dom0 CPUs (#326)
Cherry-picked from 1a722edfce6234319c5eaecf3cd0076fa9690267

Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-09-27 17:15:40 +05:30
Marcus Sorensen d083df504d LibvirtServerDiscoverer should only process added KVM hosts (#324)
Don't use LibvirtServerDiscoverer's processHostAdded() in CustomServerDiscoverer

(cherry picked from commit ba0dfea6fd469063d6a154d2056f250feed84f10)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-27 17:07:16 +05:30
Marcus Sorensen 7ed872c1d6 Use direct download timeout configs for URL check (#316)
(cherry picked from commit 289fe4bd5e8fad79fc416a26c3345ff93efb466a)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-27 17:06:20 +05:30
Nicolas Vazquez 62fce86234 Generate cloud-init multipart user data for template append policy (#7643)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
(cherry picked from commit b1fc279872)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-27 16:50:41 +05:30
Nicolas Vazquez c6c32776b5 UI: Fix user role login due to missing API access on custom hypervisor name (#7939)
* UI: Fix user role login due to missing API access on custom hypervisor name

* Refactor to include the custom HW display name as part of the response of listCapabilities API

* Add since parameter

(cherry picked from commit 940733cedf)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-27 16:19:51 +05:30
Nicolas Vazquez 4dfa38aae9 plugins: Add Custom hypervisor minimal changes (#7692)
Design document: https://cwiki.apache.org/confluence/display/CLOUDSTACK/%5BDRAFT%5D+Minimal+changes+to+allow+new+dynamic+hypervisor+type%3A+Custom+Hypervisor

This PR introduces the minimal changes to add a new hypervisor type (internally named Custom in the codebase, and configurable display name), allowing to write an external hypervisor plugin as a Custom Hypervisor to CloudStack

The custom hypervisor name is set by the setting: 'hypervisor.custom.display.name'. The new hypervisor type does not affect the behaviour of any CloudStack operation, it simply introduces a new hypervisor type into the system.

CloudStack does not have any means to dynamically add new hypervisor types. The hypervisor types are internally preset by an enum defined within the CloudStack codebase and unless a new version supports a new hypervisor it is not possible to add a host of a hypervisor that is not in part of the enum. It is possible to implement minimal changes in CloudStack to support a new hypervisor plugin that may be developed privately

This PR is an initial work on allowing new dynamic hypervisor types (adds a new element to the HypervisorType enum, but allows variable display name for the hypervisor)

Replace the HypervisorType from a fixed enum to an extensible registry mechanism, registered from the hypervisor plugin

- The new hypervisor type is internally named 'Custom' to the CloudStack services (management server and agent services, database records).
- A new global setting ‘hypervisor.custom.display.name’ allows administrators to set the display name of the hypervisor type. The display name will be shown in the CloudStack UI and API.
   - In case the ‘hypervisor.list’ setting contains the display name of the new hypervisor type, the setting value is automatically updated after the ‘hypervisor.custom.display.name’ setting is updated.
- The new Custom hypervisor type supports:
   - Direct downloads (the ability to download templates into primary storage from the hypervisor hosts without using secondary storage)
   - Local storage (use hypervisor hosts local storage as primary storage)
   - Template format: RAW format (the templates to be registered on the new hypervisor type must be in RAW format)
- The UI is also extended to display the new hypervisor type and the supported features listed above.
- The above are the minimal changes for CloudStack to support the new hypervisor type, which can be tested by integrating the plugin codebase with this feature.

This PR allows the cloud administrators to test custom hypervisor plugins implementations in CloudStack and easily integrate it into CloudStack as a new hypervisor type ("Custom"), reducing the implementation to only the hypervisor supported specific storage/networking and the hypervisor resource to communicate with the management server.

- CloudStack admin should be able to create a zone for the new custom hypervisor and add clusters, hosts into the zone with normal operations
- CloudStack users should be able to execute normal VMs/volumes/network/storage operations on VMs/volumes running on the custom hypervisor hosts

(cherry picked from commit 8b5ba13b81)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-27 16:19:42 +05:30
Marcus Sorensen fe50018fba Allow configkey to set 'cloud-name' cloud-init metadata (#7964)
* Allow configkey to set 'cloud-name' cloud-init metadata

* Update engine/api/src/main/java/com/cloud/vm/VirtualMachineManager.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update server/src/main/java/com/cloud/network/NetworkModelImpl.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update server/src/main/java/com/cloud/network/router/CommandSetupHelper.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Revert "Update server/src/main/java/com/cloud/network/router/CommandSetupHelper.java"

This reverts commit 8abc3e38c4.

* Revert "Update server/src/main/java/com/cloud/network/NetworkModelImpl.java"

This reverts commit 7f239be919.

* Rework/Fix review code suggestions

---------

Co-authored-by: Marcus Sorensen <mls@apple.com>
Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
(cherry picked from commit 155a30748c)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-27 15:47:58 +05:30
Oscar Sandoval be2bea64a9 fix units label (#325)
(cherry picked from commit d708603b6346033f0d023edfebe85113fcc928eb)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-27 15:45:45 +05:30
Marcus Sorensen a571cde7a9 Increase reserve on ScaleIO disk formatting for fragmentation (#317)
Signed-off-by: Marcus Sorensen <mls@apple.com>
Co-authored-by: Marcus Sorensen <mls@apple.com>
(cherry picked from commit 59217e3fe19fc01863d678a34a618914afaf901b)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-27 15:43:14 +05:30
Marcus Sorensen d8db09ac58 Publish event for VM.STOP when out of band stop is detected
Signed-off-by: Marcus Sorensen <mls@apple.com>
(cherry picked from commit 72a3d54db0a0bb0afbcbee35abc1e98ef102aa49)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-27 15:41:47 +05:30
Abhishek Kumar 0d9aa70edd test: enable and fix scaleio unit tests (#289)
Fixes #275

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
(cherry picked from commit c82bc16a20527d5d07c323d5323bccd56fc604e0)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-27 15:32:07 +05:30
Marcus Sorensen 5604638b84 Apple base416 passphrase enc (#240)
* Move PassphraseVO to use String instead of byte[] to support Encrypt annotation

* Check for unencrypted passphrases before migrating passphrase table

---------

Co-authored-by: Marcus Sorensen <mls@apple.com>

Fixes #239

This PR moves PassphraseVO passphrase to String type. Since the
GenericDaoBase manipulates encrypted fields as Strings we don't improve
anything by handling as byte arrays. We still use byte arrays to pass
these values down to the agents and we can get some security gains
there.

This PR also handles cases where the passphrase field may be previously
unencrypted, and upgrades them to encrypted fields using the old
encryption during cloudstack-migrate-databases. Then the process can
upgrade to new encryption normally.
2023-09-27 13:24:13 +05:30
Marcus Sorensen a9fb36174d Skip volume resize during service offering change when no size exists (#200) 2023-09-27 13:24:08 +05:30
Nicolas Vazquez 20952b4842 Auto Enable/Disable KVM hosts (#7170)
* Auto Enable Disable KVM hosts

* Improve health check result

* Fix corner cases

* Script path refactor

* Fix sonar cloud reports

* Fix last code smells

* Add marvin tests

* Fix new line on agent.properties to prevent host add failures

* Send alert on auto-enable-disable and add annotations when the setting is enabled

* Address reviews

* Add a reason for enabling or disabling a host when the automatic feature is enabled

* Fix comment on the marvin test description

* Fix for disabling the feature if the admin has manually updated the host resource state before any health check result

(cherry picked from commit be66eb2a35)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-15 13:15:59 +05:30
Marcus Sorensen 8597758ed8 Support Jetty's live cert reload on HTTPS frontend (#7355)
* Support Jetty's live cert reload

Signed-off-by: Marcus Sorensen <mls@apple.com>

* Update ServerDaemon.java

---------

Signed-off-by: Marcus Sorensen <mls@apple.com>
Co-authored-by: Marcus Sorensen <mls@apple.com>
(cherry picked from commit 9ca5f287eb)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-15 13:15:59 +05:30
Rohit Yadav c8f02a8b34 engine/orchestartion: fix/refactor to Account.Type.ADMIN
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-15 13:15:59 +05:30
Marcus Sorensen 0c192f3609 When VM start fails at host for admin, report error (#208)
* When VM start fails at host for admin, report error

Signed-off-by: Marcus Sorensen <mls@apple.com>

* Report ResourceUnavailableExceptions that result in InsufficientCapacityException to admin

* Update error message to be more straightforward

Signed-off-by: Marcus Sorensen <mls@apple.com>
Co-authored-by: Marcus Sorensen <mls@apple.com>
(cherry picked from commit 8885d252f0)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-14 20:00:46 +05:30
Wei Zhou 4bdff06acd Updating pom.xml version numbers for release 4.18.1.0
Signed-off-by: Wei Zhou <weizhou@apache.org>
2023-09-07 08:50:50 +02:00
Abhishek Kumar f049f5409e
server: fix dualstack ipv6 networks for vxlan (#7933)
Fixes #7926

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-09-07 08:46:45 +02:00
Wei Zhou 126dd5fa4c
kvm: fix live vm migration between local storage pools (#7945) 2023-09-07 08:22:37 +05:30
Rohit Yadav 0bb05f7871
ui: don't skip if dataview has multiple items in response (#7947)
This would fix the case of multiple items return in API response for a
resource such as a template or ISO in case of multi-zone env.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-09-06 16:41:41 +02:00
Nicolas Vazquez 57c61fb33c
Fix direct download https compressed qcow2 template checker (#7932)
This PR fixes an issue on direct download while registering HTTPS compressed files
Fixes: #7929
2023-09-01 08:16:03 +02:00
Marcus Sorensen 89e0a4ca25
PowerFlex: Handle missing volumes gracefully during delete volume (#7924) 2023-08-31 21:43:45 +02:00
Marcus Sorensen 2cccd8f754
db: Add index on cluster_details.name for FirstFitPlanner speedup (#7922) 2023-08-31 21:43:23 +02:00
Wei Zhou 439d70fd2b
xen/xcpng: set vmr.VCPUsMax to minimum of global setting and host cpu cores (#7887)
Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-08-28 14:26:20 +02:00
Wei Zhou 26581b7741
server: throw new exception when rootdisksize is required but not set (#7913)
* server: throw new exception when rootdisksize is required but not set

* PR7913: fix an issue with PR6441
2023-08-28 08:57:40 +02:00
Wei Zhou 0dd6bb7c67
systemvm: update size and state of systemvm template during registration (#7874) 2023-08-25 16:46:32 +02:00
Fabricio Duarte 57d4d0d94a
UI: Infer template settings in the deploy VM wizard (#7867) 2023-08-25 16:36:35 +02:00
Rohit Yadav 93bd5b7a69
ui: speed up compute instance listing (#7911)
* ui: speed up compute instance listing

The default listVirtualMachinesMetrics APIs assumings details=all which
isn't really used in the list view. By changing this to a subset of
details, we can see gains upto 10x in listing speed in the UI. When
moving to the resource/detail view of the UI, don't pass state or
details so `all` the details of the instance are returned.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* Update AutogenView.vue

---------

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-08-25 13:57:44 +02:00
Rohit Yadav 6f7725ab35
ui: Fix refresh and re-route behaviour (#7846)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-08-25 13:45:35 +02:00