Commit Graph

4712 Commits

Author SHA1 Message Date
Abhisar Sinha 70af55e848
UI support for extraconfig in deploy and update instance (#11719) 2025-09-30 09:20:44 +02:00
Abhishek Kumar 2c1aad4db8
Merge remote-tracking branch 'apache/4.20' 2025-09-27 09:00:45 +05:30
Abhishek Kumar 928972f767
extension/proxmox: add console access for instances (#11601)
This PR introduces console access support for instances deployed using Orchestrator Extensions, available via either VNC or a direct URL.

- CloudStack queries the extension using the getconsole action.
- For VNC-based access, the extension must return host/port/ticket details. CloudStack then forwards these to the Console Proxy VM (CPVM) in the instance’s zone. It is assumed that the CPVM can reach the specified host and port.
- For direct URL access, the extension returns a console URL with the protocol set to `direct`. The URL is then provided directly to the user.
- The built-in Proxmox Orchestrator Extension now supports console access via VNC. The extension calls the Proxmox API to fetch console details and returns them in the required format.

Also, adds changes to send caller details to the extension payload.
```
# cat /var/lib/cloudstack/management/extensions/Proxmox/02b650f6-bb98-49cb-8cac-82b7a78f43a2.json | jq
{
  "caller": {
    "roleid": "6b86674b-7e61-11f0-ba77-1e00c8000158",
    "rolename": "Root Admin",
    "name": "admin",
    "roletype": "Admin",
    "id": "93567ed9-7e61-11f0-ba77-1e00c8000158",
    "type": "ADMIN"
  },
  "virtualmachineid": "126f4562-1f0f-4313-875e-6150cabeb72f",
  ...
```

Documentation PR: https://github.com/apache/cloudstack-documentation/pull/560

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-09-27 08:54:27 +05:30
Abhisar Sinha 23c9e83047
Create Instance from backup on another Zone (DRaaS use case) (#11560)
* draas initial changes

* Added option to enable disaster recovery on a backup respository. Added UpdateBackupRepositoryCmd api.

* Added timeout for mount operation in backup restore configurable via global setting

* Addressed review comments

* fix for simulator test failures

* Added UT for coverage

* Fix create instance from backup ui for other providers

* Added events to add/update backup repository

* Fix race in fetchZones

* One more fix in fetchZones in DeployVMFromBackup.vue

* Fix zone selection in createNetwork via Create Instance from backup form.

* Allow template/iso selection in create instance from backup ui

* rename draasenabled to crosszoneinstancecreation

* Added Cross-zone instance creation in test_backup_recovery_nas.py

* Added UT in BackupManagerTest and UserVmManagerImplTest

* Integration test added for Cross-zone instance creation in test_backup_recovery_nas.py
2025-09-25 13:28:29 +05:30
Manoj Kumar 96992d3d64
server: Fix vpclimit count for listAcccount API response (#11686) 2025-09-24 10:28:24 +02:00
Wei Zhou e78b5cc3cc
Merge remote-tracking branch 'apache/4.20' 2025-09-24 09:27:08 +02:00
Nicolas Vazquez 1a223fd2ba
server: Fix VM import DB sequence issue on import failure (#11659)
* Fix VM import DB sequence issue on import failure

* Remove ununsed imports

* Refactor to avoid duplicating the next ID for VM sequence
2025-09-23 08:48:18 +02:00
Suresh Kumar Anaparti 40dec99659
server: Cleanup allocated snapshots / vm snapshots, and update pending ones to Error on MS start (#8452)
* Remove allocated snapshots / vm snapshots on start

* Check and Cleanup snapshots / vm snapshots on MS start

* rebase fixes

* Update volume state (from Snapshotting) on MS start when its snapshot job not finished and snapshot in Creating state
2025-09-23 08:37:10 +02:00
Wei Zhou 69998f7c40
api,server: support templatetype when upload template from local (#11682) 2025-09-22 10:46:43 +02:00
dahn aca8732102
[router] make a distinction between fatal errors, warnings and unknown as healthcheck result (#10710)
* [routers] distiction between fatal failure and warning or unknown on healthchecks

* UI status for router health checks

* status from scripts varied

* automation signalled errors

* revert removal of update sql

* upgradeversion

* move config item and further cleanup

* handling services better

* backwards compatible response

---------

Co-authored-by: Daan Hoogland <dahn@apache.org>
2025-09-22 11:39:05 +05:30
Harikrishna 3ef2556900
Fix scaleKubernetesCluster API (#11652)
* Fix scaleKubernetesCluster

* Added more state transitions
2025-09-20 17:57:45 +05:30
CodeBleu 273ae03274
Allow updating of Load Balancer source CIDR list (#11568) 2025-09-17 13:49:46 +02:00
Nicolas Vazquez 96cabf7e43
[KVM] Allow passing the OS type machine for KVM XML domains through VM setting (#11569)
* [KVM] Allow passing OS type machine to XML domains
2025-09-16 15:49:20 +05:30
Pearl Dsilva 23a53e213e
Add LB service to Custom Netris VPC/Network offerings (#11410)
* Add LB service to Custom VPC/Network offerings

* fix offering

* externalize common checks to a utility file

* remove trailing spaces
2025-09-16 09:40:33 +05:30
Wei Zhou 70a4503ea1
Merge remote-tracking branch 'apache/4.20' 2025-09-11 14:04:52 +02:00
Wei Zhou b46e29dc67
Improvement: SSL offloading with Virtual Router (#11468)
* SSL offloading with Virtual Router

* PR11468: fix pre-commit errors

* PR11468: api->getAPI/postAPI in UI

* SSL: add smoke tests for VPC in user project

* PR11468: address Daan's comments

* Fix test/integration/smoke/test_ssl_offloading.py

* SSL: remove ssl certificates when clean up account

* SSL offloading: add unit tests

* SSL offloading: UI fixes part 1

* SSL offloading: UI changes part 2

* SSL offloading: add more unit tests

* SSL offloading: more unit tests 3

* SSL offloading: wrong check

* SSL offloading: more and more unit tests

* SSL offloading: add testUpdateLoadBalancerRule5
2025-09-11 16:37:18 +05:30
Abhishek Kumar e3681a04bb
server: allow migration of vm with snapshots for vmware (#9305)
Fixes #9061

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-09-10 12:25:16 +02:00
Wei Zhou 8089d32740
Merge remote-tracking branch 'apache/4.20' 2025-09-09 19:50:22 +02:00
Wei Zhou f99942cf9a
Merge remote-tracking branch 'apache/4.19' into 4.20 2025-09-08 10:00:01 +02:00
Harikrishna fd46e61032
Added events for snapshots, vmsnapshots, internalLB operations (#11230)
* Added events for snapshots, vmsnapshots, internalLB operations

* Update server/src/main/java/com/cloud/resource/ResourceManagerImpl.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* small fix

* Unit test - ArgumentMatchers change

Co-authored-by: dahn <daan.hoogland@gmail.com>

* Unit test - ArgumentMatchers change

Co-authored-by: dahn <daan.hoogland@gmail.com>

---------

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2025-09-02 18:02:30 +05:30
Wei Zhou 3c4f458819
Merge remote-tracking branch 'apache/4.20' 2025-08-30 18:42:10 +02:00
Wei Zhou 5837c4f29e
Merge remote-tracking branch 'apache/4.19' into 4.20 2025-08-30 18:41:25 +02:00
Wei Zhou d161dc7669
api: use single quote instead of double quote in StatsResponse (#11537) 2025-08-29 18:36:30 +02:00
Suresh Kumar Anaparti 1033be4b31
Updating pom.xml version numbers for release 4.22.0.0-SNAPSHOT
Signed-off-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-08-28 12:00:42 +05:30
Suresh Kumar Anaparti f9513b47bf
Updating pom.xml version numbers for release 4.21.0.0
Signed-off-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-08-22 11:42:37 +05:30
Suresh Kumar Anaparti 99919fda7f
Merge branch '4.20' 2025-08-11 16:44:27 +05:30
Suresh Kumar Anaparti bd252b5983
Merge branch '4.19' into 4.20 2025-08-11 16:43:33 +05:30
Wei Zhou 78e146278b
api: fix scale or upgrade systemvm (#11062) 2025-08-08 16:06:18 +05:30
Suresh Kumar Anaparti ca8ce8c885
Merge branch '4.20' 2025-08-04 16:44:40 +05:30
slavkap e5f61164b3
Support of snapshot copy to primary storage in different zones. (#9478)
* Support of snapshot copy to different StorPool primary storage between zones
2025-08-04 16:35:16 +05:30
Abhisar Sinha 626f3de69a
Handle project delete in detailsview. (#11197) 2025-08-04 16:26:37 +05:30
Pearl Dsilva db63d2428b
Netris: Fix Netris provider parameter name and response (#11377)
* Fix Netris url param name and allow cidr size for routed mode vpcs

* rename response parameter name
2025-08-02 22:33:43 +05:30
Bernardo De Marco Gonçalves 9de77e1cc1
API to list console sessions (#11016)
* create API to list console sessions
2025-08-01 20:28:33 +05:30
Abhishek Kumar a60c8cab14
api,server,ui: allow listing events by state (#11355)
* api,server,ui: allow listing events by state

This change allows listing events by a particular state - Created, Scheduled, Started, Completed.
A new parameter - state has been added to the listEvents API and corresponding changes have been added in the UI.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Update api/src/main/java/org/apache/cloudstack/api/command/user/event/ListEventsCmd.java

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-07-31 19:22:15 +05:30
Abhisar Sinha e805e45342
Show chain size in snapshot response for incremental snapshots (#11313) 2025-07-31 14:52:48 +02:00
Abhisar Sinha a87c5c2b3a
Create new Instance from VM backup (#10140)
This feature adds the ability to create a new instance from a VM backup for dummy, NAS and Veeam backup providers. It works even if the original instance used to create the backup was expunged or unmanaged. There are two parts to this functionality:
Saving all configuration details that the VM had at the time of taking the backup. And using them to create an instance from backup.
Enabling a user to expunge/unmanage an instance that has backups.
2025-07-31 15:47:22 +05:30
Suresh Kumar Anaparti b9febe4b4f
Merge branch '4.20' 2025-07-31 15:23:45 +05:30
Pearl Dsilva 2944bd1eda
API: Set Object name when expunging VM (#11352) 2025-07-31 11:11:40 +05:30
Suresh Kumar Anaparti 1f1e38f3a8
Support to list templates in ready state (new API parameter 'isready', similar to list ISOs) (#11343)
* Support to list templates in ready state (new API parameter 'isready', similar to list ISOs), and UI to display Templates/ISOs in ready state wherever applicable
2025-07-31 10:08:35 +05:30
Suresh Kumar Anaparti 70468a6258
Merge branch '4.20' 2025-07-30 18:30:07 +05:30
Suresh Kumar Anaparti 7e0414354a
Merge branch '4.19' into 4.20 2025-07-30 18:14:58 +05:30
Bernardo De Marco Gonçalves f73cb5621d
Refactoring retention of backup schedules (#11223)
* refactor backup schedule retention workflows

Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>
2025-07-30 18:12:04 +05:30
Abhisar Sinha 294aef5ecf
Fix listCapacity sort by usage (#11316) 2025-07-30 14:05:03 +02:00
Abhisar Sinha da3ba17884
Fix build and ui build errors in main (#11332)
* fix build error in registeruserdatacmd

* remove unused imports

* fix ui build errors
2025-07-29 17:02:16 -03:00
nvazquez 2bffabba7b
Fix unit tests after merge forward 2025-07-29 13:41:40 -03:00
Daan Hoogland 8429223782 missing imports from merge forward 2025-07-29 17:04:06 +02:00
Daan Hoogland 0b3959221b Merge branch '4.20' 2025-07-29 16:50:55 +02:00
Nicolas Vazquez 8756be5c18
[Multi-Arch] Select Template Arch when creating template from volume (#11068)
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
Co-authored-by: Vishesh <vishesh92@gmail.com>
2025-07-29 11:41:43 +02:00
Vishesh f6ad184ea2
Feature: Add support for GPU with KVM hosts (#11143)
This PR allows attaching of GPU devices via PCI, mdev or VF to an Instance for KVM.

It allows the operator to discover the GPU devices on the KVM host and create a Compute Offering with GPU support based on the available GPU devices on the host. Once the operator has created the Compute offering, it can be used by users to launch Instances with GPU devices.
2025-07-29 13:46:24 +05:30
Abhishek Kumar 4aed972e78
api,server,extensions: allow updating extension resource map details (#11303)
* api,server,extensions: allow updating extension resource map details

This PR makes changes for allowing updating details for an extension resource mapping.
Currently, extensions only support Cluster to be registered therefore changes has been added to updateCluster functionality.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-07-29 10:25:52 +05:30
Harikrishna cca8b2fef9
Extensions Framework & Orchestrate Anything (#9752)
The Extensions Framework in Apache CloudStack is designed to provide a flexible and standardised mechanism for integrating external systems and custom workflows into CloudStack’s orchestration process. By defining structured hook points during key operations—such as virtual machine deployment, resource preparation, and lifecycle events—the framework allows administrators and developers to extend CloudStack’s behaviour without modifying its core codebase.
2025-07-28 10:41:17 +05:30
Daan Hoogland 609efcc231 Merge branch '4.19' into 4.20 2025-07-25 22:01:17 +02:00
Pearl Dsilva 0d4147f3f6
Netris Network Plugin Integration with CloudStack (#10458)
The Netris Plugin introduces Netris as a network service provider in CloudStack to be able to create and manage Virtual Private Clouds (VPCs) in CloudStack, being able to orchestrate the following network functionalities:

- Network segmentation with Netris-VXLAN isolation method
- Routing between "public" IP and network segments with an ACS ROUTED mode offering
- SourceNAT, DNAT, 1:1 NAT between "public" IP and network segments with an ACS NATTED mode offering
- Routing between VPC network segments (tiers in ACS nomenclature)
- Access Lists (ACLs) between VPC tiers and "public" network (TCP, UDP, ICMP) both as global egress rules and "public" IP specific ingress rules.
- ACLs between VPC network tiers (TCP, UDP, ICMP)
- External load balancing – between VPC network tiers and "public" IP
- Internal load balancing – between VPC network tiers
- CloudStack Virtual Router services (DHCP, DNS, UserData, Password Injection, etc…)
2025-07-25 15:26:42 +05:30
Abhisar Sinha eeb4d55e89
polish: Fix some inconsistencies in object names and messages (#10856)
* Replace ACL list with ACL and related changes to the json files in public/locales

* Replace ACL list with ACL and related changes to network.js, VpcTab.vue and AclRulesTab.vue

* Replace Export ACLs with Export ACL rules

* standardize the term user data everywhere. fix the placeholder in register user data form.

* Convert resource names in main menu and action buttons to Title case

* Use special icon for sharedfs instance and prefix for sharedfs volumes

* Give custom icon precedence over shared fs icon

* Fixed some issues with public/locale files

* Revert sharedfsvm changes

* Added label.add.acl.name to en.json which was incorrectly removed

* replace all *userdata* labels to *user.data* in public json files.

* remove redundant labels label.user.data.l2 and label.replace.acl.list

* Update ui/src/views/offering/AddNetworkOffering.vue

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* user data -> User Data in cmd and response

* fix more Title case on action buttons and labels.

---------

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-07-24 22:18:35 +05:30
dahn 4111061d29
list only own zones for resource admin (#11087)
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-07-24 09:26:57 +02:00
Manoj Kumar 264e404108
Fix for dynamic scaling toggle for instance (#11086)
* Fix for dynamic scaling toggle for instance

* Update api/src/main/java/org/apache/cloudstack/api/response/CapabilitiesResponse.java

---------

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-07-24 09:30:20 +05:30
dahn 6d5cefdfe9
add since parameter to BackupScheduleResponse (#11265) 2025-07-23 16:58:45 +05:30
Bernardo De Marco Gonçalves 9d049707d5
Fix deletion of backup schedules (#11222) 2025-07-23 09:31:17 +02:00
Abhisar Sinha e7172161dd
Guard OS type update for iso/template with existing vms (#11215)
* Guard OS type update for iso/template with existing vms

* fix identation

* rename vm -> instance

* force update iso/template as true by default via api

* add missing message.success.update.iso label
2025-07-22 18:42:24 +05:30
João Jandre 53eb2c5b9b
File-based disk-only VM snapshot with KVM as hypervisor (#10632)
Co-authored-by: João Jandre <joao@scclouds.com.br>
Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>
2025-07-16 08:54:07 +02:00
Manoj Kumar e8ab0ae70a
CPU to Memory weight based algorithm to order cluster (#10997)
* CPU to Memory weight based algorithm to order cluster
host.capacityType.to.order.clusters config will support new algorithm: COMBINED
which will work with host.capacityType.to.order.clusters.cputomemoryweight and capacity will be
computed based on CPU and memory both and using weight factor

* minor changes

* add unit tests

* update desc and add validation

* handle copilot review comments

* add log indicating chosen capacityType for ordering

---------

Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2025-07-15 16:40:53 +05:30
Bryan Lima fb6adacc51
GUI whitelabel runtime system (#8942)
* Add first version

* Add guithemedetails join

* Update since and remove extra line

* Limit information on API response for non admin users

* Add base files for preset themes

* Add miising license

* Revert cookie check

* Fix imports

* Fix pre-commit

* Address log4j2 string to format review and add license to css files

* Fix infinite loading

* Move event details to service implementation

* Move view to a specific view file

* Refactoring gui theme classes

* Normalize package name

* Address Henrique review

* Fix create table SQL

* Add interface for Dao classes

* Remove extra tabs

* Address unauthorized call when 2FA is enabled

* Remove trailing whitespaces

* Apply suggestions from code review

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

---------

Co-authored-by: Henrique Sato <henriquesato2003@gmail.com>
Co-authored-by: Bernardo De Marco Gonçalves <bernardomg2004@gmail.com>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-07-15 13:29:09 +05:30
slavkap 54b44cc316
KVM: Option to deploy a VM with existing volume/snapshot (#10503)
* Option to deploy a VM with existing volume/snapshot

* smoke test changes

check if the hypervisor is KVM
check if the primary storage's scope is ZONE wide

* skip all tests if the storage isn't Zone-Wide and the hypervisor isn't KVM

* support StorPool tags

add StorPool tags to a volume created from snapshot or to a volume which
will be attached as a ROOT to a new VM

* Add StorPool tags on the new ROOT volume

* Add the StorPool's tags when volume is created from a snapshot or a
volume is attached as a ROOT to a VM

* Addressed review
2025-07-14 15:10:45 +05:30
Daan Hoogland 0dbd761fbb Merge branch '4.20' 2025-07-14 11:17:08 +02:00
Suresh Kumar Anaparti 749ddb975f
Support ApiServer to enforce POST requests for state changing APIs and requests with timestamps (#10899)
Co-authored-by: Kevin Li <kli74@apple.com>
2025-07-05 15:25:48 +02:00
Nicole Schmidt d5fd3ec36e
Add API command remove management server (#10325)
* Add API command remove management server

* Apply suggestions from code review

Co-authored-by: Bernardo De Marco Gonçalves <bernardomg2004@gmail.com>

* Apply sugestions from code review

* Update log message with current management server state

Co-authored-by: Bernardo De Marco Gonçalves <bernardomg2004@gmail.com>

* Apply suggestions from code review

Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>

* Update api/src/main/java/org/apache/cloudstack/api/command/admin/management/RemoveManagementServerCmd.java

Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>

* Remove unused imports

---------

Co-authored-by: Bernardo De Marco Gonçalves <bernardomg2004@gmail.com>
Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>
2025-07-04 14:49:28 -03:00
Suresh Kumar Anaparti 80f46ad55d
[VMware to KVM Migration] Fix for converted instance npe issue when source vmware instance ovf is exported from management server (#11003) 2025-07-04 13:54:54 +05:30
Daan Hoogland 3e3a0c0678 Merge branch '4.20' 2025-07-03 15:29:05 +02:00
Suresh Kumar Anaparti be22bfe2c9
Management Server - Prepare for Maintenance and Cancel Maintenance improvements (#10995)
* Management Server - Prepare for Maintenance and Cancel Maintenance improvements:
- Added new setting 'management.server.maintenance.ignore.maintenance.hosts' to ignore hosts in maintenance states  while preparing management server for maintenance. This skips agent transfer and agents count check for hosts in maintenance.
- Rebalance indirect agents after cancel maintenance, using rebalance parameter in cancelMaintenance API
- Force maintenance after maintenance window timeout, using forced parameter in prepareForMaintenance API.
- Propagate 'indirect.agent.lb.check.interval' setting change to the host agents.

* rebases fixes

* code improvements, cleanup

* [UI] Set rebalance true by default in cancel maintenance dialog

* Update MS state after executing cluster cmd in the target MS, and some code improvements

* code improvements

* Ensure the host lb algorithm 'shuffle' is applied once before disabling the indirect agent lb check background task
2025-07-03 12:17:04 +05:30
Nicolas Vazquez 75147b7811
[Vmware to KVM Migration] Display virt-v2v and ovftool versions for supported hosts for migration (#11019)
* [Vmware to KVM Migration] Display virt-v2v and ovftool versions for supported hosts for migration

* Fix UI display

* Address review comments

* Fix ovftool and version display - also display versions on host details view
2025-06-23 12:49:51 +02:00
Nicolas Vazquez 6adfda2818
CKS Enhancements (#9102)
CKS Enhancements:

* Ability to specify different compute or service offerings for different types of CKS cluster nodes – worker, master or etcd

* Ability to use CKS ready custom templates for CKS cluster nodes

* Add and Remove external nodes to and from a kubernetes cluster

Co-authored-by: nvazquez <nicovazquez90@gmail.com>

* Update remove node timeout global setting

* CKS/NSX : Missing variables in worker nodes

* CKS: Fix ISO attach logic

* CKS: Fix ISO attach logic

* address comment

* Fix Port - Node mapping when cluster is scaled in the presence of external node(s)

* CKS: Externalize control and worker node setup wait time and installation attempts

* Fix logger

* Add missing headers and fix end of line on files

* CKS Mark Nodes for Manual Upgrade and Filter Nodes to add to CKS cluster from the same network

* Add support to deploy CKS cluster nodes on hosts dedicated to a domain

---------

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>

* Support unstacked ETCD

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>

* Fix CKS cluster scaling and minor UI improvement

* Reuse k8s cluster public IP for etcd nodes and rename etcd nodes

* Fix DNS resolver issue

* Update UDP active monitor to ICMP

* Add hypervisor type to CKS cluster creation to fix CKS cluster creation when External hosts added

* Fix build

* Fix logger

* Modify hypervisor param description in the create CKS cluster API

* CKS delete fails when external nodes are present

* CKS delete fails when external nodes are present

* address comment

* Improve network rules cleanup on failure adding external nodes to CKS cluster

* UI: Fix etcd template was not honoured

* UI: Fix etcd template was not honoured

* Refactor

* CKS: Exclude etcd nodes when calculating port numbers

* Fix network cleanup in case of CKS cluster failure

* Externalize retries and inverval for NSX segment deletion

* Fix CKS scaling when external node(s) present in the cluster

* CKS: Fix port numbers displayed against ETCD nodes

* Add node version details to every node of k8s cluster - as we now support manual upgrade

* Add node version details to every node of k8s cluster - as we now support manual upgrade

* update column name

* CKS: Exclude etcd nodes when calculating port numbers

* update param name

* update param

* UI: Fix CKS cluster creation templates listing for non admins

* CKS: Prevent etcd node start port number to coincide with k8s cluster start port numbers

* CKS: Set default kubernetes cluster node version to the kubernetes cluster version on upgrade

* CKS: Set default kubernetes cluster node version to the kubernetes cluster version on upgrade

* consolidate query

* Fix upgrade logic

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>

* Fix CKS cluster version upgrade

* CKS: Fix etcd port numbers being skipped

* Fix CKS cluster with etcd nodes on VPC

* Move schema and upgrade for 4.20

* Fix logger

* Fix after rebasing

* Add support for using different CNI plugins with CKS

* Add support for using different CNI plugins with CKS

* remove unused import

* Add UI support and list cni config API

* necessary UI changes

* add license

* changes to support external cni

* UI changes

* Fix NPE on restarting VPC with additional public IPs

* fix merge conflict

* add asnumber to create k8s svc layer

* support cni framework to use as-numbers

* update code

* condition to ignore undefined jinja template variables

* CKS: Do not pass AS number when network ID is passed

* Fix deletion of Userdata / CNI Configuration in projects

* CKS: Add CNI configuration details to the response and UI

* Explicit events for registering cni configuration

* Add Delete cni configuration API

* Fix CKS deployment when using VPC tiers with custom ACLs

* Fix DNS list on VR

* CKS: Use Network offering of the network passed during CKS cluster creation to get the AS number

* CKS cluster with guest IP

* Fix: Use control node guest IP as join IP for external nodes addition

* Fix DNS resolver issue

* Improve etcd indexing - start from 1

* CKS: Add external node to a CKS cluster deployed with etcd node(s) successfully

* CKS: Add external node to a CKS cluster deployed with etcd node(s) successfully

* simplify logic

* Tweak setup-kube-system script for baremetal external nodes

* Consider cordoned nodes while getting ready nodes

* Fix CKS cluster scale calculations

* Set token TTL to 0 (no expire) for external etcd

* Fix missing quotes

* Fix build

* Revert PR 9133

* Add calico commands for ens35 interface

* Address review comments: plan CKS cluster deployment based on the node type

* Add qemu-guest-agent dependency for kvm based templates

* Add marvin test for CKS clusters with different offerings per node type

* Remove test tag

* Add marvin test and fix update template for cks and since annotations

* Fix marvin test for adding and removing external nodes

* Fix since version on API params

* Address review comments

* Fix unit test

* Address review comments

* UI: Make CKS public templates visible to non-admins on CKS cluster creation

* Fix linter

* Fix merge error

* Fix positional parameters on the create kubernetes ISO script and make the ETCD version optional

* fix etcd port displayed

* Further improvements to CKS  (#118)

* Multiple nics support on Ubuntu template

* Multiple nics support on Ubuntu template

* supports allocating IP to the nic when VM is added to another network - no delay

* Add option to select DNS or VR IP as resolver on VPC creation

* Add API param and UI to select option

* Add column on vpc and pass the value on the databags for CsDhcp.py to fix accordingly

* Externalize the CKS Configuration, so that end users can tweak the configuration before deploying the cluster

* Add new directory to c8 packaging for CKS config

* Remove k8s configuration from resources and make it configurable

* Revert "Remove k8s configuration from resources and make it configurable"

This reverts commit d5997033ebe4ba559e6478a64578b894f8e7d3db.

* copy conf to mgmt server and consume them from there

* Remove node from cluster

* Add missing /opt/bin directory requrired by external nodes

* Login to a specific Project view

* add indents

* Fix CKS HA clusters

* Fix build

---------

Co-authored-by: Nicolas Vazquez <nicovazquez90@gmail.com>

* Add missing headers

* Fix linter

* Address more review comments

* Fix unit test

* Fix scaling case for the same offering

* Revert "Login to a specific Project view"

This reverts commit 95e37563f4.

* Revert "Fix CKS HA clusters" (#120)

This reverts commit 8dac16aa35.

* Apply suggestions from code review about user data

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* Update api/src/main/java/org/apache/cloudstack/api/command/user/userdata/BaseRegisterUserDataCmd.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* Refactor column names and schema path

* Fix scaling for non existing previous offering per node type

* Update node offering entry if there was an existing offering but a global service offering has been provided on scale

---------

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
Co-authored-by: Daan Hoogland <daan@onecht.net>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-06-19 11:00:42 +05:30
Felipe f8c4121ea3
Add access modifiers to `VirtualMachineTO` (#9277)
Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>
2025-06-17 08:35:04 +02:00
Abhishek Kumar bce17b627d
ui,api,server: template categorization based on os (#10773)
Adds new interface for image selection (template/iso) for an instance in UI.
Old interface can still be used and it can be configured using UI configuration (config.json)

OS categories/Guest OS categories have been improved with ability to create new categories, delete an existing category, and marking a category as featured to allow it to show up in the UI in the image selection interface.

New APIs added:
- addOsCategory
- deleteOsCategory
- updateOsCategory

APIs updated:
- updateOsType
- listTemplates
- listOsCategories

Several improvements in UI especially related to forms - DeloyVM, ReinstallVM, CreateVnfAppliance, AddAutoscaleGroup.

DeployVM form can now be opened from template/ISO details view with query params.

Reorganized (removed and added some) OS categories to the following (in the same order):
```
1. Ubuntu
2. Debian
3. Fedora
4. CentOS
5. Rocky Linux
6. Alma Linux
7. Oracle
8. RedHat
9. SUSE
10. Windows
11. Other
```

Documentation PR: https://github.com/apache/cloudstack-documentation/pull/500

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-06-10 15:25:51 +05:30
Pearl Dsilva 379ee07d88 Updating pom.xml version numbers for release 4.19.4.0-SNAPSHOT
Signed-off-by: Pearl Dsilva <pearl1594@gmail.com>
2025-06-06 18:00:09 +05:30
Pearl Dsilva b5e2c181f9 Updating pom.xml version numbers for release 4.20.2.0-SNAPSHOT
Signed-off-by: Pearl Dsilva <pearl1594@gmail.com>
2025-06-06 15:38:12 +05:30
Pearl Dsilva c61a5eb430 Updating pom.xml version numbers for release 4.20.1.0
Signed-off-by: Pearl Dsilva <pearl1594@gmail.com>
2025-05-30 12:43:00 +05:30
Daan Hoogland 0c7d47138d Updating pom.xml version numbers for release 4.19.3.0
Signed-off-by: Daan Hoogland <daan@onecht.net>
2025-05-30 09:08:58 +02:00
Manoj Kumar 7632814cd2
Instance lease: Allow deployment of instances with lease duration and leaseexpiry action (#10560)
* FR-248: Instance lease, WIP commit

* insert lease expiry into db and use that to filter exiring vms, add asyncjobmanager

* Add leaseDuration and leaseExpiryAction in Service offering create flow

* Update listVM cmd to allow listing only leased instances

* Add methods to fetch instances for which lease is expiring in next days

* Changes included:
config key setup and configured for alert email
lease options in create and update vm screen
handle delete protection, edit vm, create vm
validated stop and detroy, delete protection

* Update UI screens for leased properties coming from config and service offering

* use global lock before running scheduler

* Unit tests

* Flow changes done in UI based on discussion

* Include view changes in schema upgrade files and use feature in various UI elements

* Added integration test for vm deployment, UI enhancements for user persona, bug fixes

* validate integration tests, minor ui changes and log messages

* fix build: moving configkey from setup to test itself

* Disable testAlert to unblock build and trim whitespaces in integration tests

* Address review comments

* Minor changes in EditVM screen

* Use ExecutorService instead of Timer and TimerTask

* Additional review comments

* Incorporate following changes:
1. Execute lease action once on the instance
2. Cancel lease on instance when feature is disabled
3. Relevant events when lease gets disabled, cancelled, executed
4. Disable associating lease after deployment
5. UI elements and flow changes
6. Changes based on feedback from demo

* Handle pr review comments

* address review comments

* move instance.lease.enabled config to VMLeaseManager interface

* bug fix in edit instance flow and reject api request for invalid values

* max allowed lease is for 100 years

* log instance ids for expired instance

* Fix config validation for value range and code coverage improvement

* fix lease expiry request failures in async

* dont use forced: true for StopVmCmd

* Update server/src/main/java/org/apache/cloudstack/vm/lease/VMLeaseManager.java

Co-authored-by: Vishesh <vishesh92@gmail.com>

* handle review comments

---------

Co-authored-by: Rohit Yadav <rohityadav89@gmail.com>
Co-authored-by: Vishesh <vishesh92@gmail.com>
2025-05-28 17:40:09 +05:30
Daan Hoogland 650b5ec3da Merge branch '4.20' 2025-05-27 18:18:39 +02:00
Nicolas Vazquez d5fbd07b9f Adding privilege checks on user and account operations
Co-authored-by: Harikrishna <harikrishna.patnala@gmail.com>
2025-05-27 20:08:16 +05:30
Abhishek Kumar 19d6b979af cks: create separate service account in project
A separate service account will be created and added in the project, if
not exist already, when a Kubernetes cluster is deployed in a project.
This account will have a role with limited API access.

Cleanup clusters on owner account cleanup, delete service account
if needed

When the owner account of k8s clusters is deleted, while its node VMs
get expunged, the cluster entry in DB remain present. This fixes the
issue by cleaning up all clusters for the account deleted.

Project k8s service account will be deleted on account cleanup or when
there is no active k8s cluster remaining

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-05-27 20:08:12 +05:30
nvazquez a0080a04fe Adding privilege checks on user and account operations
Co-authored-by: Harikrishna <harikrishna.patnala@gmail.com>
2025-05-27 16:24:29 +02:00
Abhishek Kumar e2f187912c cks: create separate service account in project
A separate service account will be created and added in the project, if
not exist already, when a Kubernetes cluster is deployed in a project.
This account will have a role with limited API access.

Cleanup clusters on owner account cleanup, delete service account
if needed

When the owner account of k8s clusters is deleted, while its node VMs
get expunged, the cluster entry in DB remain present. This fixes the
issue by cleaning up all clusters for the account deleted.

Project k8s service account will be deleted on account cleanup or when
there is no active k8s cluster remaining

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-05-27 16:24:29 +02:00
Pearl Dsilva f496ed6eaf Merge branch '4.20' of https://github.com/apache/cloudstack 2025-05-22 12:05:47 +05:30
Wei Zhou d0dc6d8093
mgmt: add back serviceip in ManagementServerResponse (#10891)
* mgmt: add back serviceip in ManagementServerResponse

* api: add Deprecated annotation to serviceip
2025-05-21 09:55:52 +05:30
Wei Zhou 842b2f8c24
Merge remote-tracking branch 'apache/4.20' 2025-05-19 21:25:37 +02:00
Harikrishna b17808bfba
Introducing Storage Access Groups for better management for host and storage connections (#10381)
* Introducing Storage Access Groups to define the host and storage pool connections

In CloudStack, when a primary storage is added at the Zone or Cluster scope, it is by default connected to all hosts within that scope. This default behavior can be refined using storage access groups, which allow operators to control and limit which hosts can access specific storage pools.

Storage access groups can be assigned to hosts, clusters, pods, zones, and primary storage pools. When a storage access group is set on a cluster/pod/zone, all hosts within that scope inherit the group. Connectivity between a host and a storage pool is then governed by whether they share the same storage access group.

A storage pool with a storage access group will connect only to hosts that have the same storage access group. A storage pool without a storage access group will connect to all hosts, including those with or without a storage access group.
2025-05-19 11:33:29 +05:30
Daan Hoogland 8f8c685d17 Merge branch '4.19' into 4.20 2025-05-16 15:51:37 +02:00
Manoj Kumar d5ba23c848
Introduce volume allocation algorithm global configuration (#10696) 2025-05-16 14:06:42 +02:00
Suresh Kumar Anaparti 13ab8a04d1
Fix for Vlan doesn't match issue while adding IP range for the shared network without any IP range (#10837) 2025-05-16 09:24:55 +02:00
Nicolas Vazquez 8d3ae3e057
[Vmware] Improve listing of Vmware Datacenter VMs for migration to KVM (#10770)
Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-05-14 08:12:27 +02:00
João Jandre 6fdaf51ddc
KVM incremental snapshot feature (#9270)
* KVM incremental snapshot feature

* fix log

* fix merge issues

* fix creation of folder

* fix snapshot update

* Check for hypervisor type during parent search

* fix some small bugs

* fix tests

* Address reviews

* do not remove storPool snapshots

* add support for downloading diff snaps

* Add multiple zones support

* make copied snapshots have normal names

* address reviews

* Fix in progress

* continue fix

* Fix bulk delete

* change log to trace

* Start fix on multiple secondary storages for a single zone

* Fix multiple secondary storages for a single zone

* Fix tests

* fix log

* remove bitmaps when deleting snapshots

* minor fixes

* update sql to new file

* Fix merge issues

* Create new snap chain when changing configuration

* add verification

* Fix snapshot operation selector

* fix bitmap removal

* fix chain on different storages

* address reviews

* fix small issue

* fix test

---------

Co-authored-by: João Jandre <joao@scclouds.com.br>
2025-05-12 10:50:30 -03:00
Wei Zhou fd74895ad0
New feature: Reconcile commands (CopyCommand, MigrateCommand, MigrateVolumeCommand) (#10514) 2025-05-02 09:15:03 +02:00
Pearl Dsilva 2df1ac5106 Merge branch '4.20' of https://github.com/apache/cloudstack 2025-04-28 12:15:48 +05:30
Pearl Dsilva 0785ba046e Merge branch '4.19' of https://github.com/apache/cloudstack into 4.20 2025-04-28 11:10:08 +05:30
Wei Zhou 5d28e66b52
vTPM: support KVM and VMware (#10543)
* KVM: add Virtual TPM model and version

* KVM: add admin-only VM setting GUEST.CPU.MODE and GUEST.CPU.MODEL

* VMware: add vTPM

* vTPM: do not set Key due to 'Cannot add multiple devices using the same device key..'

* vTPM: add unit test testTpmModel

* engine/schema: remove user vm details for guest CPU mode/model

* vTPM: extra methods as Daan's requests

* vTPM: add unit tests in VmwareResourceTest

* vTPM: update unit tests in VmwareResourceTest

* vTPM: add unit test in LibvirtComputingResourceTest

* vTPM: use the default TPM version if an invalid version is passed

* vTPM: requires UEFI on vmware and do nothing if it is not enabled/disabled

* vTPM: let uses to add UEFI on vmware

* Update plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/VmwareResource.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* Update plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/VmwareResource.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* vTPM: remove template details for guest CPU mode/model

* UI: boot vm from ISO into UEFI/SECURE mode

---------

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-04-26 14:35:36 +05:30
Abhishek Kumar 12c077d704
api,ui: multi arch improvements (#10289) 2025-04-25 11:02:27 +02:00
dahn 5a5c865133
Revert "Add the option to filter by host when retrieving of unregistered VMs (#9925)" (#10647)
This reverts commit aa6c581e40.
2025-04-24 09:06:57 +02:00
Daan Hoogland 3c75d9363b Merge branch '4.20' 2025-04-17 15:59:41 +02:00
Eric Kalendra 55c811547a
Update Mockito to 5.16.1 (#10686)
Dependency name change mockito-inline to mockito-core. Inline is now the default and the last version of mockito-inline released is 5.2.0.

assertj-core in user-authenticators/saml2 pulls in an incompatible version of byte-buddy and required an exclusion. Updating the version of assertj is left for a future PR.
The upgrade requires Java 11+, dropping support for Java 8. CloudStack documentation already says to use Java 11 and does not indicate that java 8 is supported.

Test classes using @RunWith(MockitoJUnitRunner.class) now get run in strict mode. Changes were made to tests where the stubbing intention was clear. In ManagementServerMaintenanceManagerImplTest there are 5 tests where the intention of the test is unclear. Each of the statements now use Mockito.lenient() to avoid the exception. Other cases in the tests follow a similar pattern.
Minor clean up.

Both @Spy and Mockito.spy( should not be used. Favored the annotation.
Both @RunWith(MockitoJUnitRunner.class) and MockitoAnnotations.openMocks(this); should not be used. Favored the annotation.
Unnecessary extends TestCase removed.
@InjectMocks and new in statement unnecessary. Removed new when issue presented.
Some of the Cmd classes like UpdateNetworkCmd have a type tree that includes fields of type Object. This appears to cause issues with injection, requiring that @Mock fields be available. This is where the following fields were added in multiple places:
Object job;
ResponseGenerator _responseGenerator;
Wrong number of parameters for Mockito.when in LibvirtRevertSnapshotCommandWrapperTest.java
2025-04-16 18:10:28 +05:30