Commit Graph

11583 Commits

Author SHA1 Message Date
Daan Hoogland 02be2cdf67 Merge release branch 4.18 to main
* 4.18:
  cleanup removal of physical network (#7630)
  console proxy: use AeadBase64Encryptor instead of AES/CBC/PKCS5Padding (#7237)
2023-07-05 11:52:23 +02:00
dahn 8e171785d1
cleanup removal of physical network (#7630)
Co-authored-by: Bryan Lima <42067040+BryanMLima@users.noreply.github.com>
2023-07-05 11:12:46 +02:00
Wei Zhou 3e04779f60
console proxy: use AeadBase64Encryptor instead of AES/CBC/PKCS5Padding (#7237) 2023-07-05 11:01:32 +02:00
Daan Hoogland fbb9dec6e5 Merge release branch 4.18 to main
* 4.18:
  server: do not check zone imagestores for directdownload template delete (#7607)
  UI: Zone wizard fix (#7588)
2023-07-04 10:17:43 +02:00
Abhishek Kumar 5e5d194d77
server: do not check zone imagestores for directdownload template delete (#7607)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-04 09:37:06 +02:00
mprokopchuk 70d5470f48
If ResourceCountCheckTask throws an exception the scheduled task is not going to run again until the management servers are restarted. (#7670)
Co-authored-by: Maxim Prokopchuk <mprokopchuk@apple.com>
2023-07-04 08:45:15 +02:00
dahn 1275db4081
UI: Zone wizard fix (#7588)
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-03 11:56:52 +02:00
Rohit Yadav 3b054b2665 Merge remote-tracking branch 'origin/4.18' 2023-07-03 13:17:25 +05:30
Abhishek Kumar de6ce503dc
api: correct error on resize volume resource allocation failure (#7687)
This PR resource throws exception with the correct error code and logs the error message when a resource allocation failure is encountered during resize volume operation.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-03 12:57:14 +05:30
Daniel Augusto Veronezi Salvador c6237c48ac
Retrieve only resource count on user dashboard (#7617)
* Allow retrieving only the count of resources on APIs listPublicIpAddresses, listNetworks, listVirtualMachines and listVolumes

* Use parameter to retrieve only the count of resources in the dashboard

* Create abstract class
2023-06-30 11:09:55 -03:00
Wei Zhou 41403c9cf9 Merge remote-tracking branch 'apache/4.18' 2023-06-27 11:52:37 +02:00
dahn 06caf32bc8
Guest os mappings improvements (#6979)
* Guest OS mapping improvements
- Checks the OS mapping name in hypervisor (VMware, XenServer)
- Displays guest OS mappings in UI

* Added API getHypervisorGuestOsNames to list the guest OS names in the hypervisor, and code improvements

* Some static analysis fixes

* Removed commented code in listview

* Guest OS list

* UI changes for adding guest os and mappings

* Added guest os mappings in guest os form

* Added new filter to guest os mapping

* Name and description changes

* VMWare Host and cluster MO unit tests

* CheckGuestOsMapping command and answer unit tests

* GetHypervisorGuestOsNames command and answer unit tests

* VmwareResource unitests

* GuestOsMapper unittests

* icon changes

* Addressed review comments

* Renaming fixes

* Removed comments

* marvin tests for guest os operations

* Added marvin tests for OS mappings

* Document links and UI improvements

* Added deduplication for the list guest OS API

* Fixed linter failure

* Few bug fixes and UI changes

* Few improvements

* Addressed code smells

* Fixed UI issues after rebase

---------

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
Co-authored-by: Harikrishna Patnala <harikrishna.patnala@gmail.com>
2023-06-27 13:49:20 +05:30
slavkap faaf72b1a4
Volume encryption support for StorPool plug-in (#7539)
Supported Virtual machine operations:
- live migration of VM to another host
- virtual machine snapshots (group snapshot without memory)
- revert VM snapshot
- delete VM snapshot
Supported Volume operations:
- attach/detach volume
- live migrate volume between two StorPool primary storages
- volume snapshot
- delete snapshot
- revert snapshot
2023-06-26 11:24:51 +02:00
Vishesh fa3f2a75eb
feature: VM Scheduler (#7397)
This PR adds a feature to setup schedules to stop/start/restart your VMs.

Documentation PR: apache/cloudstack-documentation#313
Related issue: #3387
2023-06-26 13:36:46 +05:30
Nicolas Vazquez c809201247
Fix: Volumes on lost local storage cannot be removed (#7594) 2023-06-23 12:22:15 +02:00
Vishesh 0acc66f51d
server: Add check on host's status while deleting config drive on host cache (#7584)
This PR adds a check on host's status. Without this if the agent is not in Up or Connecting state, expunging of a VM fails.

Steps to reproduce:
- Enable vm.configdrive.force.host.cache.use in Global Configuration.
- Create a L2 network with config drive
- Deploy a vm with the L2 network created in previous step
- Stop the vm and destroy vm (not expunge it)
- Stop the cloudstack-agent on the VM's host
- Expunge the vm

Fixes: #7428
2023-06-23 13:46:22 +05:30
Abhishek Kumar 0466bf9d47
server,api,ui: host auto-select for migrateVirtualMachineWithVolume (#7554)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-22 14:53:51 +02:00
Wei Zhou 09a4a252d7 Merge remote-tracking branch 'apache/4.18' into HEAD 2023-06-21 15:08:56 +02:00
Harikrishna 40cc10a73d
Allow volume migrations in ScaleIO within and across ScaleIO storage clusters (#7408)
* Live storage migration of volume in scaleIO within same storage scaleio cluster

* Added migrate command

* Recent changes of migration across clusters

* Fixed uuid

* recent changes

* Pivot changes

* working blockcopy api in libvirt

* Checking block copy status

* Formatting code

* Fixed failures

* code refactoring and some changes

* Removed unused methods

* removed unused imports

* Unit tests to check if volume belongs to same or different storage scaleio cluster

* Unit tests for volume livemigration in ScaleIOPrimaryDataStoreDriver

* Fixed offline volume migration case and allowed encrypted volume migration

* Added more integration tests

* Support for migration of encrypted volumes across different scaleio clusters

* Fix UI notifications for migrate volume

* Data volume offline migration: save encryption details to destination volume entry

* Offline storage migration for scaleio encrypted volumes

* Allow multiple Volumes to be migrated with migrateVirtualMachineWithVolume API

* Removed unused unittests

* Removed duplicate keys in migrate volume vue file

* Fix Unit tests

* Add volume secrets if does not exists during volume migrations. secrets are getting cleared on package upgrades.

* Fix secret UUID for encrypted volume migration

* Added a null check for secret before removing

* Added more unit tests

* Fixed passphrase check

* Add image options to the encypted volume conversion
2023-06-21 11:57:05 +05:30
dahn 1aa4f80741
accept first word from host os string for backwards compatibility (#7620) 2023-06-15 10:09:31 +03:00
dahn ae10263b3b
Possibility to choose the source NAT IP address on a isolated network or VPC (#6442)
Co-authored-by: NuxRo <nux@li.nux.ro>
Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
2023-06-09 14:51:53 +02:00
Abhishek Kumar 41e8ad7487
api,server,ui: vr,systemvm in public ip response (#7403)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Wei Zhou <weizhou@apache.org>
2023-06-08 15:05:59 +02:00
Abhishek Kumar c944727d33 server: fix apache/4.18 merge failure
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-06 13:26:06 +05:30
Abhishek Kumar 6ea5f56e62 Merge remote-tracking branch 'apache/4.18' into main 2023-06-06 13:12:55 +05:30
Abhishek Kumar 2d6a069812
server: improve storage GC to skip expunging possible duplicate volumes (#7313)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-05 10:03:24 +02:00
Abhishek Kumar f905f5f39a Merge remote-tracking branch 'apache/4.18' into main 2023-06-01 16:50:24 +05:30
Abhishek Kumar 7319debc87
cks: k8s cluster on vpc tier (#7479)
* cks: cluster on vpc tier

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wip

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* python fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix trailing space

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-01 15:29:06 +05:30
Abhishek Kumar 0c13024329
refactor: remove duplicate inject (#7548)
VolumeDao has been injected twice. This PR removes and refactors injection.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-26 07:19:54 +05:30
Rohit Yadav ce44ff4148 Merge remote-tracking branch 'origin/4.18' 2023-05-25 22:16:16 +05:30
Abhishek Kumar 8849e0f464
server: fix volume detach operation when no vm host (#7526)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-25 15:48:27 +02:00
SadiJr 43a5d6260f
Enable CPU cap in VRs (#7234)
Co-authored-by: SadiJr <sadi@scclouds.com.br>
2023-05-23 22:39:42 +02:00
John Bampton 11d45654a6
misc: fix spelling (#7206)
This PR fixes spellings
2023-05-23 11:06:16 +05:30
Rohit Yadav 2f309b573b Merge remote-tracking branch 'origin/4.18' 2023-05-12 12:58:49 +05:30
Oscar Sandoval b6443a2b1f
increase log detail for limit checking, fix getDomainReservation() (#7506)
In troubleshooting ops issues we see logs like:

Maximum domain resource limits of Type 'user_vm' for Domain Id = 763 is exceeded: Domain Resource Limit = (1 bytes) 1, Current Domain Resource Amount = (0 bytes) 0, Requested Resource Amount = (1 bytes) 1."

However there is one missing value (currentResourceReservation) that is used in the calculation of limit check but it is not logged, which leads to confusion. Above we see we are using “0” and requested 1, with our limit being 1, but was rejected. Without logging all the values used in the calculation we don’t understand why it failed.

Additionally, if we had this log above it would be clearer that a second bug is occurring. When we query for domain level resource reservations in “getDomainReservation” the actual SearchBuilder is the listAccountAndTypeSearch, not the listDomainAndTypeSearch. As a result, when we call getDomainReservation the query returns any outstanding domain reservation for any account, as domain ID is not a valid filter for the account search.

This PR:

Increases detailed information in log for checking resource limit to include reservations information for functions: checkDomainResourceLimit() and checkAccountResourceLimit

Fixes getDomainReservation() to use listDomainAndTypeSearch instead of listAccountAndTypeSearch

Co-authored-by: Oscar Sandoval <osandovalocana@apple.com>
2023-05-12 12:53:18 +05:30
Rohit Yadav 62fa45aa81 Merge remote-tracking branch 'origin/4.18' 2023-05-11 23:40:53 +05:30
Abhishek Kumar a0eb0aa5f7
api,server: fix VM.CREATE events on vm deploy without start (#7421)
Fixes #6697

Allows the server to generate started and completed events for VM.CREATE event type when VM is deployed with startvm=false.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-11 23:23:00 +05:30
Rohit Yadav 232c86a715 Merge remote-tracking branch 'origin/4.18' 2023-05-11 23:19:43 +05:30
Abhishek Kumar d9e0a3e6c3
server: add config SSVM capacity scan (#7448)
Fixes #7438

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-11 12:58:10 +05:30
Wei Zhou 9d46df57f2
kvm: add vm setting for nic multiqueue number and packed virtqueues (#7333)
This PR adds two vm setting for user vms on KVM

- nic multiqueue number
- packed virtqueues enabled . optional are true and false (false by default). It requires qemu>=4.2.0 and libvirt >=6.3.0

Tested ok on ubuntu 22 and rocky 8.4
2023-05-09 15:19:26 +05:30
Rohit Yadav a2561df25b Merge remote-tracking branch 'origin/4.18' 2023-05-08 12:57:38 +05:30
Marcus Sorensen 8604cb5328
server: Fix DirectDownload certificate check initial delay (#7494)
This PR adjusts the DirectDownload certificate check initial delay. Since the time unit is in hours, I think it was a mistake to schedule the initial check to be in 60 hours after management servers start - the intention was likely 60 seconds. We had turned this feature on to run hourly, not realizing we would have to wait 2.5 days to see it first run!

Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-05-08 12:09:50 +05:30
Abhishek Kumar e234c3ccdc
server: guard vm start inter-cluster migration with config (#7401)
During the start of a stopped VM when there is not enough capacity in the current cluster CloudStack can migrate it to a new cluster. This can be an expensive operation when Cluster scope storage is used as migration can be carried out using SSVM and secondary storage.
This PR allows controlling this behaviour with the existing global config - `migrate.vm.across.clusters`

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-08 12:08:57 +05:30
Marcus Sorensen 3cb4c801fb
server: fix null pointer on powerflex attach volume edge case (#7498)
This PR fixes a null pointer edge case where a PowerFlex volume is attached to a VM.

In this edge case, a VM has been created, started, stopped, and then reimaged. VM has a last host ID but the newly attached volume does not yet have a storage pool assigned, it will be assigned on the VM start. However, we assume that if the VM's host is not null, we need to try to revoke access to the volume for the host. Since there is no storage pool yet for the volume, we hit a null pointer when checking to see if the volume's pool is PowerFlex.

This was affecting all storage types, I could reproduce it with local storage, since the null pointer is at the check for pool's type.

Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-05-08 12:07:41 +05:30
GaOrtiga 8b5bfb145e
create parameter to determine whether roles are public or private (#6960)
Co-authored-by: Gabriel Ortiga Fernandes <gabriel.fernandes@scclouds.com.br>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-05-01 15:26:10 +02:00
Abhishek Kumar b84744d9a5
server: validate ip address value on update config (#7415)
Fixes #6958

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-04-28 16:11:33 +05:30
Rahul Agarwal 0ed4950896
server: allow updating project name (#7149)
This PR adds name in updateProject API to allow renaming 'name' field with description from both API and UI level.

Fixes: #7107

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Rahul Agarwal <rahul.agarwal@shapeblue.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-04-25 15:29:13 +05:30
Abhishek Kumar e035d73641 Merge remote-tracking branch 'apache/4.18' into main 2023-04-19 12:57:01 +05:30
Abhishek Kumar 41bbedb530 Merge remote-tracking branch 'apache/4.17' into 4.18 2023-04-19 12:36:10 +05:30
Vishesh 79eae89a87
ui: Add filtering by state in account, systemvms, router and storagepool (#7368)
This PR allows admin to filter resources by state for systemvms, router & storagepool. This is part of #7366 .

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-04-19 12:22:24 +05:30
Wei Zhou fb51504dff
server: dedicate vxlan range to account (#7215)
This fixes #6871
2023-04-19 12:14:53 +05:30