This PR enhances the existing CLVM implementation which was based on the deprecated CLVM technology which was based on corosync/pacemaker. With RHEL 7 having reached EOL, CLVM seems to be broken. CLVM supports RAW volumes on LVM , where as CLVM_NG support QCOW2 on LVM.
Further details: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modernized+CLVM%3A+Enhancements+and+CLVM_NG+support
NOTE: On testing - it was identified that incremental snapshots for clvm-ng do not work as expected. As of now it's been removed from scope. So, CLVM and CLVM_NG would only support full snapshots.
* add support for proper cleanup of snapshots and prevent vol snapshot of running vm
* remove snap vol restriction for sunning vms
* refactor clvm code
* add support for live migration
* add support for migrating lvm lock
* clvm deletion called explicitly
* made necessary changes to allow migration of lock and deletion of detached volumes
* fix create vol from snap and attach
* add support to revert snapshot for clvm
* add support to revert snapshot for clvm
* make zero fill configurable
* make setting non-dynamic & fix test
* fix locking at vol/vm creation
* fix revert snapshot format type and handle revert snapshot functionality for clvm
* 1. Create clvmlockmanager and move common code \n
2. handle attaching volumes to stopped VMs \n
3. Handle lock transfer when VM is started on another host
* add license
* remove command/answer classes from sonar coverage check
* add support for new gen clvm with template (qcow2) backing
* Add support for clvm_ng - which allows qcow2 on block storage , linked clones, etc
* fix test and use physical size + 50% of virtual size for backing file, while virtual size + pe for disk
* migrate clvm volumes as full clone and allow migration from clvm to nfs
* fix clvm_ng to nfs migration, and handle overhead calc
* support live migration from clvm_ng to nfs and vice-versa
* add support to migrate to and from clvm to nfs
* fix creation of volume on destination host during migration to clvm/clvm-ng
* support live vm migration between clvm -> clvm-ng (vice-versa), nfs -> clvm (vice-versa) and nfs->clvm-ng (vice-versa)
* add unit tests for clvm/clvm_ng operations
* Add support for incremental volume snapshots for clvm_ng
* prevent snapshot backup for incremental clvm_ng snaps, fix build failure, add unit tests
* fix lockhost on creation of volumes from snap and fix bitmap issue when migrating a vol with incremental snap
* restrict pre and post migration commands to only kvm hosts where vm has CLVM/CLVM-NG volumes
* evist lock tracking - use lvs command to get lock host than DB
* add test for pre/post migration
* Create a CLVM storage adaptor
* update existing clvm get stats method
* fix precommit check failure
* Apply suggestions from code review
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
* Apply suggestions from code review
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
* improve lock host retrieval logic and quicker retrival using db host as first check point and then fanning out
* add proper support for resizing of clvm_ng which calculated PE correctly for qcow2 metadata
* fallback to full snapshots for clvm-ng - incremental not supported in 4.23
* expunge volume detail of lock host on vm expunge
* if vmmigration with volume is done to the same clvm volume group, then dont do data transfer, just lock transfer and vm
* add clvm pools with deterministic uuid , so as to prevent adding the same pool twic
* added a small improvement to factor in a senario when lv is inactive on all hosts, could happen in storage outage issue
* address comment - extract common code for endpoint identification if clvm pool type
* Address comments - add early return guard to reduce indentation
* minor improvement - when migrating vm with volumes, if there's a failures, change the clvm vols to exclusive on source from shared, and on success, change dest vol to exclusive only for cross-pool migration
* cleanup unused code and tests for incremental snaps for clvmng and other cleanups
* allow storage browser to list lv in clvm, fix clvm shrink, overprovisioning factor isnt used for clvm pools - so set it to 1 and prevented display of provisioning type for clvm
* no need to have locktransfercommand to execute in sequence
* increase lv cmd timeouts to consider cluster load
---------
Co-authored-by: Pearl Dsilva <pearl1954@gmail.com>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
This PR introduces the initial implementation of Veeam integration support for KVM in CloudStack by adding a UHAPI-compatible server and image server components.
Veeam Backup & Replication interacts with virtualization platforms using its Universal Hypervisor API (UHAPI). To enable backup and restore workflows for CloudStack-managed KVM environments, this change introduces a UHAPI server that exposes CloudStack resources through a UHAPI-compatible interface.
In addition to the control plane APIs, an image server component is introduced to handle the data transfer operations required during backup and restore workflows.
The integration consists of two main components:
1. UHAPI Server (Control Plane) named CloudStack Veeam Control Service
A lightweight UHAPI server runs inside the CloudStack management server and exposes endpoints under:
/ovirt-engine
- /api - For APIs
- /sso - For authentication
- /services/pki-resource - For certificates
This server provides inventory discovery APIs required by Veeam and translates CloudStack resources into the structures expected by UHAPI.
The server:
- exposes infrastructure inventory
- handles authentication and session tokens
- maps CloudStack resources to UHAPI-compatible representations
2. Image Server (Data Plane) named CloudStack Image Service
A separate image server component is introduced to handle backup and restore data transfer operations.
This component:
- serves disk image data during backup
- receives image data during restore operations
- exposes endpoints used by Veeam worker components
- integrates with CloudStack storage to read and write VM disk data
The separation between both these components server ensures that:
- metadata APIs and control operations remain lightweight
- bulk image transfer operations are handled independently
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Abhisar Sinha <63767682+abh1sar@users.noreply.github.com>
Co-authored-by: abh1sar <abhisar.sinha@gmail.com>
Co-authored-by: Wei Zhou <weizhou@apache.org>
* Host HA code improvements
* Fix to not cancel VM HA items when Host HA is enabled & inspection in progress, and some code improvements
- When Host HA inspection in progress, the investigor returns the Host Status as Up which cancels the VM HA items
- Don't cancel the VM HA items, instead reschedule them to try again later
* Changes to consider Recovered/Available Host HA state along with the agent connection status to determine the Host HA inspection in progress or not, and some code improvements
* Fix domain parsing for GPU
* Add Display controller to GPU class check
this adds support for the amd instinct mi2xx accelorator crards in the discovery script.
Co-authored-by: Piet Braat <piet@phiea.nl>
cks: harden ISO build script with pipefail, --fail, and strict mode:
- Replace 'set -e' with 'set -o errexit', 'set -o nounset', 'set -o pipefail';
- Add TRACE env var support for debug with 'set -o xtrace';
- Add --fail (-f) flag to curl commands in pipelines to prevent silent;
- Add --fail and tar.gz integrity validation for etcd download;
- Fix ARCH validation error message to list all accepted values
(x86_64, amd64, aarch64, arm64).
cks: fix semver comparison using sort -V instead of awk:
- Replace awk numeric comparison with a sort -V based version_lt()
function. The previous approach treated '1.9.0' as 1.9 and '1.18.0'
as 1.18, making 1.9 > 1.18 and selecting the wrong source for
kubelet.service and 10-kubeadm.conf.
cks: vendor kubelet.service and 10-kubeadm.conf to reduce supply-chain risk:
- Vendor kubelet.service and 10-kubeadm.conf from shapeblue/cloudstack-nonoss
into the repository instead of fetching them at build time from a mutable
remote branch. These files are executed with root privileges as systemd
units, and pinning to a mutable branch without checksum verification
posed a supply-chain risk.
cks: only add Cilium Helm repo if not already configured:
- Replace 'helm repo add ... || true' with an explicit check via
'helm repo list'. The previous approach suppressed all failures
(including network/DNS errors), which could cause a later, less-clear
failure during 'helm template'.
cks: move Cilium ISO script to scripts/util/cks/ and add download progress:
- Move create-kubernetes-binaries-iso-with-cilium.sh to scripts/util/cks/
alongside vendored kubelet.service and 10-kubeadm.conf;
- Download kubeadm, kubelet, kubectl individually with --progress-bar
for visibility during long downloads.
* etcd download: use arch variable instead of hard-coded amd64
Replace hard-coded linux-amd64 in etcd tarball download URL and output filename with linux- to support both amd64 and arm64 architectures.
* container-selinux: install from distro repos instead of CentOS 7 mirror
Remove hard-coded CentOS 7 x86_64 RPM URL for container-selinux and install directly from the system's configured repositories, which handles architecture and version automatically.
* build_name: fix dead-code fallback default
Check whether parameter is empty before constructing build_name, so the fallback default setup--.iso is actually reachable.
* CNI download: fail on HTTP errors for both URL attempts
Use curl -f on both primary and legacy CNI download URLs and exit with a clear error if neither succeeds, instead of only checking for 404 on the first attempt.
* temp directory: use mktemp and trap for cleanup
Replace fixed /tmp/iso with mktemp -d to avoid collisions between concurrent runs, and register a trap EXIT to ensure cleanup on failure or early exit.
* extension/proxmox: improve host vm power reporting
Add `statuses` action in extensions to report VM power states
This PR introduces support for retrieving the power state of all VMs on a host directly from an extension using the new `statuses` action.
When available, this provides a single aggregated response, reducing the need for multiple calls.
If the extension does not implement `statuses`, the server will gracefully fall back to querying individual VMs using the existing `status` action.
This helps with updating the host in CloudStack after out-of-band migrations for the VM.
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
* address review
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
---------
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
* XenServer 8.4/XCP-ng 8.3: Support vTPM
* fix issue
* add log for windows 11 or other such guests OSs that require vtpm
* remove secure bootmode requirement
* Fix uefi setting on host for xenserver 8.4
* 4.22:
fix install path for systemvm templates when introducing new sec storage (#11605)
fix Sensitive Data Exposure Through Exception Logging in OVM Hypervis… (#12032)
Fix snapshot physical size after migration (#12166)
ConfigDrive: use file absolute path instead of canonical path to create ISO (#11623)
Add log for null templateVO (#12406)
snapshot: fix listSnapshots for volume which got delete and whose storage pool got deleted (#12433)
Notify user if template upgrade is not required (#12483)
Fix: proper permissions for systemvm template registrations on hardened systems (#12098)
Allow modification of user vm details if user.vm.readonly.details is empty (#10456)
NPE fix while deleting storage pool when pool has detached volumes (#12451)
Related to https://github.com/apache/cloudstack/issues/10029#issuecomment-2531599607
We have umask 0077, so cloud-install-sys-tmplt is creating by default paths like below
```
$ ls -l /mnt/secondary/template/tmpl/
total 16
drwx------. 3 root root 4096 Nov 19 13:58 1
drwxrwxrwx. 7 root root 4096 Oct 31 09:42 2
drwxrwxrwx. 3 root root 4096 Oct 30 15:59 4
drwxr-xr-x. 2 root root 4096 Oct 31 10:21 5
$ ls -l /mnt/secondary/template/tmpl/1/
total 4
drwx------. 2 root root 4096 Nov 19 13:59 3
$ ls -l /mnt/secondary/template/tmpl/1/3/
total 549848
-rw-------. 1 root root 563032576 Nov 19 13:59 d23a1e19-c563-4f69-85ca-8721cf02082c.qcow2
-rw-------. 1 root root 287 Nov 19 13:59 template.properties
```
This results to the permissions problems later on, when trying to access the image
Signed-off-by: Artem Sidorenko <artem.sidorenko@telekom.de>
This PR aligns the use of terminology, renaming VM / virtual machine references to 'Instance' and also capitalising the terms Templates, Network, Snapshot, User, Account in CloudStack APIs, error and log messages, events, tooltips, etc. Many typos, grammar and spelling mistakes were fixed, also terms like IPv4, VPN, VPC, etc. were properly capitalised. Some error messages were cleaned for better readability. The test cases, expecting some exception strings were adjusted accordingly.
Here is the wiki page, describing the changes in details:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Object+Naming+and+Title+Case+Convention
---------
Co-authored-by: Manoj Kumar <manojkr.itbhu@gmail.com>
Co-authored-by: Harikrishna <harikrishna.patnala@gmail.com>
* Support creation of PV(persistent volumes) in CloudStack projects
* add support for snapshot APIs for project role
* Add support to setup csi driver on k8s cluster creation
* fix deploy script
* update response
* fix table name
* fix linter
* show if csi driver is setup in cluster
* delete pvs whose reclaim policy is delete when cluster is destroyed
* update ref
* move changes to 4.22
* fix variables
* fix eof
This PR introduces console access support for instances deployed using Orchestrator Extensions, available via either VNC or a direct URL.
- CloudStack queries the extension using the getconsole action.
- For VNC-based access, the extension must return host/port/ticket details. CloudStack then forwards these to the Console Proxy VM (CPVM) in the instance’s zone. It is assumed that the CPVM can reach the specified host and port.
- For direct URL access, the extension returns a console URL with the protocol set to `direct`. The URL is then provided directly to the user.
- The built-in Proxmox Orchestrator Extension now supports console access via VNC. The extension calls the Proxmox API to fetch console details and returns them in the required format.
Also, adds changes to send caller details to the extension payload.
```
# cat /var/lib/cloudstack/management/extensions/Proxmox/02b650f6-bb98-49cb-8cac-82b7a78f43a2.json | jq
{
"caller": {
"roleid": "6b86674b-7e61-11f0-ba77-1e00c8000158",
"rolename": "Root Admin",
"name": "admin",
"roletype": "Admin",
"id": "93567ed9-7e61-11f0-ba77-1e00c8000158",
"type": "ADMIN"
},
"virtualmachineid": "126f4562-1f0f-4313-875e-6150cabeb72f",
...
```
Documentation PR: https://github.com/apache/cloudstack-documentation/pull/560
---------
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
* scripts: fix external provision to use correct power state
The valid states are poweron and poweroff.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* strip string while processing powerstate for HyperV
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* ignore warning that spills over to exten output string
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
---------
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This feature adds the ability to create a new instance from a VM backup for dummy, NAS and Veeam backup providers. It works even if the original instance used to create the backup was expunged or unmanaged. There are two parts to this functionality:
Saving all configuration details that the VM had at the time of taking the backup. And using them to create an instance from backup.
Enabling a user to expunge/unmanage an instance that has backups.
This PR allows attaching of GPU devices via PCI, mdev or VF to an Instance for KVM.
It allows the operator to discover the GPU devices on the KVM host and create a Compute Offering with GPU support based on the available GPU devices on the host. Once the operator has created the Compute offering, it can be used by users to launch Instances with GPU devices.
The Extensions Framework in Apache CloudStack is designed to provide a flexible and standardised mechanism for integrating external systems and custom workflows into CloudStack’s orchestration process. By defining structured hook points during key operations—such as virtual machine deployment, resource preparation, and lifecycle events—the framework allows administrators and developers to extend CloudStack’s behaviour without modifying its core codebase.
* [CKS] Create Kubernetes ISO support for ARCH optional parameter
* Include arch on the build name
* Refactor
* Fix ISO naming
* Apply suggestions from code review
Co-authored-by: Wei Zhou <weizhou@apache.org>
* Reorder parameters
---------
Co-authored-by: Wei Zhou <weizhou@apache.org>
Ewerton Silva
Pearl Dsilva
Vishesh
Abhishek Kumar
Suresh Kumar Anaparti
Daan Hoogland
Wei Zhou
Abhisar Sinha
Artem Sidorenko
argusb
vladimirpetrov
Phsm Qwerty
dahn
John Bampton
Manoj Kumar
dk-blackfuel
vishesh92
StepBee
Orsiris de Jong
Rohit Yadav
Harikrishna
Nicolas Vazquez