* Allow copy of templates from secondary storages of other zone when adding a new secondary storage
* Add API param and UI changes on add secondary storage page
* Make copy template across zones non blocking
* Code fixes
* unused imports
* Add copy template flag in zone wizard and remove NFS checks
* Fix UI
* Label fixes
* code optimizations
* code refactoring
* missing changes
* Combine template copy and download into a single asynchronous operation
* unused import and fixed conflicts
* unused code
* update config message
* Fix configuration setting value on add secondary storage page
* Removed unused code
* Update unit tests
The secondary storage selectors allow operators to specify, for instance, that volumes should go to a specific secondary storage A. Thus, when uploading a volume, it will always be downloaded to secondary storage A.
The cold volume migration moves volumes to a secondary storage before moving them to the destination primary storage. This process does not consider the secondary storage selectors. However, some companies want to dedicate specific secondary storages for cold migration.
To address this, this PR makes the cold volume migration process consider the secondary storage selectors.
* 4.22:
fix install path for systemvm templates when introducing new sec storage (#11605)
fix Sensitive Data Exposure Through Exception Logging in OVM Hypervis… (#12032)
Fix snapshot physical size after migration (#12166)
ConfigDrive: use file absolute path instead of canonical path to create ISO (#11623)
Add log for null templateVO (#12406)
snapshot: fix listSnapshots for volume which got delete and whose storage pool got deleted (#12433)
Notify user if template upgrade is not required (#12483)
Fix: proper permissions for systemvm template registrations on hardened systems (#12098)
Allow modification of user vm details if user.vm.readonly.details is empty (#10456)
NPE fix while deleting storage pool when pool has detached volumes (#12451)
This fixes the case when the storage pool is removed as well the KVM
host and the subsequent volumes on the host. When that happened, listing
snapshots (for recovery purposes) cause NPE as the pool_id was null, but
last_pool_id for the related destroyed volume wasn't null. This adds a
fallback logic.
Signed-off-by: Rohit Yadav <rohit@yadav.cloud>
* NPE fix while deleting storage pool when pool has detached volumes
* review
* unit tests
* Added log for volumes not attached to any VMs
* update filter, log and test
* updated volume dao method names returning non destroyed volumes
* build fix
---------
Co-authored-by: dahn <daan@onecht.net>
* Add support for dedicating backup offerings to domains
* Add tests and UI support and update response params
* add license header
* exclude backupofferingdetailsvo from sonar
* fix pre-commit checks - missing / extra EOF line
* add test
* EOF
* filter backup offerings by domain id
* add unit tests
* add more unit tests and remove response file from code coverage check
* update checks
* address review comments: extract common code, fix tests
* added bean definition
* address comments
* add unit tests to increase coverage
* pre-commit check failure fix
* address merge issue
* allow updating backup offering when only domain id is modified
This PR fixes#11502
- Prevent service offering update to specific domains if any instance for the offering are outside of those
- Removal of offerings is skipped if it is in use by any Instance.
Bundling all hypervisor SystemVM templates in release packages simplifies installs but inflates build time and artifact size. This change enables downloading templates on demand when they’re not found after package installation. The download path is wired into both cloud-setup-management and the existing SystemVM template registration flow.
For connected or mirrored environments, a repository URL prefix can be provided to support air-gapped setups: pass --systemvm-templates-repository <URL-prefix> to cloud-setup-management, or set system.vm.templates.download.repository=<URL-prefix> in server.properties for post-setup registration.
If templates are already present (bundled or preseeded), behavior is unchanged and no download is attempted.
---------
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
* Cleanup userconcentratedpod_random and userconcentratedpod_firstfit allocation algorithm
* use firstfit instead of random for userconcentratedpod_firstfit
This PR aligns the use of terminology, renaming VM / virtual machine references to 'Instance' and also capitalising the terms Templates, Network, Snapshot, User, Account in CloudStack APIs, error and log messages, events, tooltips, etc. Many typos, grammar and spelling mistakes were fixed, also terms like IPv4, VPN, VPC, etc. were properly capitalised. Some error messages were cleaned for better readability. The test cases, expecting some exception strings were adjusted accordingly.
Here is the wiki page, describing the changes in details:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Object+Naming+and+Title+Case+Convention
---------
Co-authored-by: Manoj Kumar <manojkr.itbhu@gmail.com>
Co-authored-by: Harikrishna <harikrishna.patnala@gmail.com>
* 4.22:
Update templateConfig.sh to not break with directorys with space on t… (#10898)
Fix VM and volume metrics listing regressions (#12284)
packaging: use latest cmk release link directly (#11429)
api:rename RegisterCmd.java => RegisterUserKeyCmd.java (#12259)
Prioritize copying templates from other secondary storages instead of downloading them (#10363)
Show time correctly in the backup schedule UI (#12012)
kvm: use preallocation option for fat disk resize (#11986)
Python exception processing static routes fixed (#11967)
KVM memballooning requires free page reporting and autodeflate (#11932)
api: create/register/upload template with empty template tag (#12234)
* Fix import VM tasks pagination
* Fix UI for pagination and proper listing
* Fixes and improvements
* Polish UI
* Restore config.json
* Fix state on parameter description
* Support creation of PV(persistent volumes) in CloudStack projects
* add support for snapshot APIs for project role
* Add support to setup csi driver on k8s cluster creation
* fix deploy script
* update response
* fix table name
* fix linter
* show if csi driver is setup in cluster
* delete pvs whose reclaim policy is delete when cluster is destroyed
* update ref
* move changes to 4.22
* fix variables
* fix eof
* add createCrossZoneInstnaceEnabled to BackupOfferingResponse
* show use IP Address from Backup button when orignal instance is expunged
* Fix NPE in takeBackup if the vm template is deleted.
* Add since to Cross zone instance creation in BackupOfferingResponse.java
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
* Store and show Guest os type in the backup metadata
* show warning in create instance from backup form if guest os type is different
* show warning in create instance from backup form if guest os type is different
* backupvmexpunged -> isbackupvmexpunged
* review comments
* fix npe
* improve err msg
* err msg
---------
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
* Add source VM name on virt-v2v migration log entries
* Improve the feedback by displaying the running importing tasks
* Add source VM name prefix on more conversion logs
* Improve listing and also list completed tasks
* Pass extra parameters to virt-v2v if administrator allows via global setting
* Add Force converting directly to storage pool option
* Refactor based on review comments
* Add properties for env vars for the instance conversion
* Add separate component for Import VM Tasks
* applying copilot suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Fix importing unmanaged instances due to incorrect internal name
* Add VM prefix on each log operation for conversion
* Log the original VM name instead of the cloned VM in case of cloning
* Allow searching storage pool by UUID after conversion to support SharedMountPoint
* Fix search pools logic
* Improve UI and add checks for force convert to pool parameter
* Support Local storage when forceconverttopool is set to true
* Add config key to for allowed extra params and add validation
* Fix params lists
* Fix compile error
* Remove extra stubbings
* Fix extra params execution
---------
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
* API: Add support to list all snapshot policies & backup schedules
* Add support for backup policy listing without tying it to the vmid
* add tests for snapshot policy listing
* update tests for listbackupschedules
* remove trailing spaces and fix lint failure
* Add upgrade test
* remove unused import
* add create policy - snap/backup in the list view with resource (volume/vm) selection
* add translations
* refresh parent list
* remove unnecessary alert info
* fix checks for UI backup schedule list view
* fix checks for UI backup schedule list view
* add back access checks
* add since param
* fix failing test
* update snapshot policy and backup schedule ownership when VM is moved
* fix issue with showing vm selection
* fix unit test failure
* Update list snappolicy & backup schedule logic to list only those that belong to a proj or for root admin those that belong to it, unless listall & projid is passed
* fix test
* support snap / backup policy search using keyword
* fix tests
* Migrate volume improvements, to bypass secondary storage when copy volume between pools is allowed directly
* Bypass secondary storage for copy volume between zone-wide pools and
- local storage on host in the same zone
- cluser-wide pools in the same zone
* Bypass secondary storage for volumes on ceph/rdb pool when the scope permits
* Fix dest disk format while migrating volume from ceph/rbd to nfs, and some code improvements
* unit tests
* Update suitable disk offering(s) for volume(s) after migrate VM with volumes when change in pool type (shared or local)
Currently, Migrate VM with volume(s) bypasses the service and disk offerings of the volumes, as the target pools for migration are specified,
which ignores the offerings. Offering change is required when pool type (shared or local) is changed, mainly
- when volume on shared pool is migrated to local pool
- when volume on local pool is migrated to shared pool
* Update with proper message while migrate volume when target pool and offering type mismatches (both are not shared/local)
* Consider host scope first during endpoint selection while copying between primary storages
* Update disk offering count (for listDiskOfferings api) while removing offerings with tags mismatch with storage tags
* storage: change storage pool to Up state when cancel storage migration
* Update 11773: connect host to shared pool after cancelling storage migration
* Update 11773: update db only
* Update 11773: skip capacity update for storpool
* Return details of the storage pool in the response including url, and update capacityBytes and capacityIops if applicable while creating storage pool
* Added capacitybytes parameter to the storage pool response in sync with the capacityiops response parameter and createStoragePool cmd request parameter (existing disksizetotal parameter in the storage pool response can be deprecated)
* Don't keep url in details
* Persist the capacityBytes and capacityIops in the storage_pool_details table while creating storage pool as well, for consistency - as these are updated with during update storage pool
* rebase with main fixes
This PR adds support for specifying user data (cloud-init) for system VMs via Zone Scoped global settings. This allows the operators to customize the System VMs and setup monitoring, logging or execute any custom commands.
We set the user data from the global setting in /var/cache/cloud/cmdline, and use the NoCloud datasource to process user data. cloud-init service is still disabled in the system VMs and it's executed as part of the cloud-postinit service which executes the postinit.sh script.
Added global settings:
systemvm.userdata.enabled - Disabled by default. Needs to be enabled to utilize the feature.
console.proxy.vm.userdata - UUID of the User data to be used for Console Proxy
secstorage.vm.userdata - UUID of the User data to be used for Secondary Storage VM
virtual.router.userdata - UUID of the User data to be used for Virtual Routers
Suresh Kumar Anaparti
Abhisar Sinha
Davi Torres
Abhishek Kumar
Harikrishna
Fabricio Duarte
Daan Hoogland
Nicolas Vazquez
GaOrtiga
Bernardo De Marco Gonçalves
Henrique Sato
Wei Zhou
Manoj Kumar
Rohit Yadav
Vishesh
Pearl Dsilva
YoulongChen
vladimirpetrov
Vitor Hugo Homem Marzarotto
dahn
João Jandre
John Bampton
julien-vaz