Commit Graph

4614 Commits

Author SHA1 Message Date
Pearl Dsilva 138ed60076
Further improvements to CKS (#118)
* Multiple nics support on Ubuntu template

* Multiple nics support on Ubuntu template

* supports allocating IP to the nic when VM is added to another network - no delay

* Add option to select DNS or VR IP as resolver on VPC creation

* Add API param and UI to select option

* Add column on vpc and pass the value on the databags for CsDhcp.py to fix accordingly

* Externalize the CKS Configuration, so that end users can tweak the configuration before deploying the cluster

* Add new directory to c8 packaging for CKS config

* Remove k8s configuration from resources and make it configurable

* Revert "Remove k8s configuration from resources and make it configurable"

This reverts commit d5997033ebe4ba559e6478a64578b894f8e7d3db.

* copy conf to mgmt server and consume them from there

* Remove node from cluster

* Add missing /opt/bin directory requrired by external nodes

* Login to a specific Project view

* add indents

* Fix CKS HA clusters

* Fix build

---------

Co-authored-by: Nicolas Vazquez <nicovazquez90@gmail.com>
2025-05-27 21:33:20 -03:00
nvazquez f0130d2ebd
Merge branch 'main' into cks-enhancements-upstream 2025-05-27 21:21:35 -03:00
Daan Hoogland 650b5ec3da Merge branch '4.20' 2025-05-27 18:18:39 +02:00
Nicolas Vazquez d5fbd07b9f Adding privilege checks on user and account operations
Co-authored-by: Harikrishna <harikrishna.patnala@gmail.com>
2025-05-27 20:08:16 +05:30
Abhishek Kumar 19d6b979af cks: create separate service account in project
A separate service account will be created and added in the project, if
not exist already, when a Kubernetes cluster is deployed in a project.
This account will have a role with limited API access.

Cleanup clusters on owner account cleanup, delete service account
if needed

When the owner account of k8s clusters is deleted, while its node VMs
get expunged, the cluster entry in DB remain present. This fixes the
issue by cleaning up all clusters for the account deleted.

Project k8s service account will be deleted on account cleanup or when
there is no active k8s cluster remaining

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-05-27 20:08:12 +05:30
Pearl Dsilva 6804d62f9f Merge branch 'main' of https://github.com/apache/cloudstack into cks-enhancements-upstream 2025-05-22 18:23:17 +05:30
Pearl Dsilva f496ed6eaf Merge branch '4.20' of https://github.com/apache/cloudstack 2025-05-22 12:05:47 +05:30
Wei Zhou d0dc6d8093
mgmt: add back serviceip in ManagementServerResponse (#10891)
* mgmt: add back serviceip in ManagementServerResponse

* api: add Deprecated annotation to serviceip
2025-05-21 09:55:52 +05:30
nvazquez 5e7f86b84f
Merge branch 'main' into cks-enhancements-upstream 2025-05-19 21:04:56 -03:00
Wei Zhou 842b2f8c24
Merge remote-tracking branch 'apache/4.20' 2025-05-19 21:25:37 +02:00
Harikrishna b17808bfba
Introducing Storage Access Groups for better management for host and storage connections (#10381)
* Introducing Storage Access Groups to define the host and storage pool connections

In CloudStack, when a primary storage is added at the Zone or Cluster scope, it is by default connected to all hosts within that scope. This default behavior can be refined using storage access groups, which allow operators to control and limit which hosts can access specific storage pools.

Storage access groups can be assigned to hosts, clusters, pods, zones, and primary storage pools. When a storage access group is set on a cluster/pod/zone, all hosts within that scope inherit the group. Connectivity between a host and a storage pool is then governed by whether they share the same storage access group.

A storage pool with a storage access group will connect only to hosts that have the same storage access group. A storage pool without a storage access group will connect to all hosts, including those with or without a storage access group.
2025-05-19 11:33:29 +05:30
nvazquez 0f79583466
Merge branch 'main' into cks-enhancements-upstream 2025-05-16 13:36:49 -03:00
Daan Hoogland 8f8c685d17 Merge branch '4.19' into 4.20 2025-05-16 15:51:37 +02:00
Manoj Kumar d5ba23c848
Introduce volume allocation algorithm global configuration (#10696) 2025-05-16 14:06:42 +02:00
Suresh Kumar Anaparti 13ab8a04d1
Fix for Vlan doesn't match issue while adding IP range for the shared network without any IP range (#10837) 2025-05-16 09:24:55 +02:00
Nicolas Vazquez 8d3ae3e057
[Vmware] Improve listing of Vmware Datacenter VMs for migration to KVM (#10770)
Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-05-14 08:12:27 +02:00
João Jandre 6fdaf51ddc
KVM incremental snapshot feature (#9270)
* KVM incremental snapshot feature

* fix log

* fix merge issues

* fix creation of folder

* fix snapshot update

* Check for hypervisor type during parent search

* fix some small bugs

* fix tests

* Address reviews

* do not remove storPool snapshots

* add support for downloading diff snaps

* Add multiple zones support

* make copied snapshots have normal names

* address reviews

* Fix in progress

* continue fix

* Fix bulk delete

* change log to trace

* Start fix on multiple secondary storages for a single zone

* Fix multiple secondary storages for a single zone

* Fix tests

* fix log

* remove bitmaps when deleting snapshots

* minor fixes

* update sql to new file

* Fix merge issues

* Create new snap chain when changing configuration

* add verification

* Fix snapshot operation selector

* fix bitmap removal

* fix chain on different storages

* address reviews

* fix small issue

* fix test

---------

Co-authored-by: João Jandre <joao@scclouds.com.br>
2025-05-12 10:50:30 -03:00
nvazquez bfa6f52766
Merge branch 'main' into cks-enhancements-upstream 2025-05-11 17:38:08 -03:00
Wei Zhou fd74895ad0
New feature: Reconcile commands (CopyCommand, MigrateCommand, MigrateVolumeCommand) (#10514) 2025-05-02 09:15:03 +02:00
Pearl Dsilva 2df1ac5106 Merge branch '4.20' of https://github.com/apache/cloudstack 2025-04-28 12:15:48 +05:30
Pearl Dsilva 0785ba046e Merge branch '4.19' of https://github.com/apache/cloudstack into 4.20 2025-04-28 11:10:08 +05:30
Wei Zhou 5d28e66b52
vTPM: support KVM and VMware (#10543)
* KVM: add Virtual TPM model and version

* KVM: add admin-only VM setting GUEST.CPU.MODE and GUEST.CPU.MODEL

* VMware: add vTPM

* vTPM: do not set Key due to 'Cannot add multiple devices using the same device key..'

* vTPM: add unit test testTpmModel

* engine/schema: remove user vm details for guest CPU mode/model

* vTPM: extra methods as Daan's requests

* vTPM: add unit tests in VmwareResourceTest

* vTPM: update unit tests in VmwareResourceTest

* vTPM: add unit test in LibvirtComputingResourceTest

* vTPM: use the default TPM version if an invalid version is passed

* vTPM: requires UEFI on vmware and do nothing if it is not enabled/disabled

* vTPM: let uses to add UEFI on vmware

* Update plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/VmwareResource.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* Update plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/VmwareResource.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* vTPM: remove template details for guest CPU mode/model

* UI: boot vm from ISO into UEFI/SECURE mode

---------

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-04-26 14:35:36 +05:30
nvazquez a9bc998d1a
Merge branch 'main' into cks-enhancements-upstream 2025-04-25 07:42:45 -03:00
Abhishek Kumar 12c077d704
api,ui: multi arch improvements (#10289) 2025-04-25 11:02:27 +02:00
dahn 5a5c865133
Revert "Add the option to filter by host when retrieving of unregistered VMs (#9925)" (#10647)
This reverts commit aa6c581e40.
2025-04-24 09:06:57 +02:00
Daan Hoogland 3c75d9363b Merge branch '4.20' 2025-04-17 15:59:41 +02:00
Eric Kalendra 55c811547a
Update Mockito to 5.16.1 (#10686)
Dependency name change mockito-inline to mockito-core. Inline is now the default and the last version of mockito-inline released is 5.2.0.

assertj-core in user-authenticators/saml2 pulls in an incompatible version of byte-buddy and required an exclusion. Updating the version of assertj is left for a future PR.
The upgrade requires Java 11+, dropping support for Java 8. CloudStack documentation already says to use Java 11 and does not indicate that java 8 is supported.

Test classes using @RunWith(MockitoJUnitRunner.class) now get run in strict mode. Changes were made to tests where the stubbing intention was clear. In ManagementServerMaintenanceManagerImplTest there are 5 tests where the intention of the test is unclear. Each of the statements now use Mockito.lenient() to avoid the exception. Other cases in the tests follow a similar pattern.
Minor clean up.

Both @Spy and Mockito.spy( should not be used. Favored the annotation.
Both @RunWith(MockitoJUnitRunner.class) and MockitoAnnotations.openMocks(this); should not be used. Favored the annotation.
Unnecessary extends TestCase removed.
@InjectMocks and new in statement unnecessary. Removed new when issue presented.
Some of the Cmd classes like UpdateNetworkCmd have a type tree that includes fields of type Object. This appears to cause issues with injection, requiring that @Mock fields be available. This is where the following fields were added in multiple places:
Object job;
ResponseGenerator _responseGenerator;
Wrong number of parameters for Mockito.when in LibvirtRevertSnapshotCommandWrapperTest.java
2025-04-16 18:10:28 +05:30
Fabricio Duarte ac6b1b382c
Migrate public templates that have URLs on data migration across secondary storages (#10364)
Co-authored-by: Fabricio Duarte <fabricio.duarte@scclouds.com.br>
2025-04-15 13:48:45 +02:00
Bernardo De Marco Gonçalves 6de084ca97
Add download link of volumes, templates and ISOs to the download event details (#10564) 2025-04-15 10:24:28 +02:00
nvazquez 94449f4e20
Merge branch 'main' into cks-enhancements-upstream 2025-04-14 08:21:33 -03:00
Bryan Lima cb4848bc1a
Add support to RBD erasure code pools (#9808)
* Readd filename string on qemuimg create

* Remove empty object on the data pool details of storage pools with no data pool

* Only use the method createPhysicalDiskByLibVirt with RBD when the pool is of erasure code type. Also added javadoc for createPhysicalDisk method

* Change literal '/' string to File.separator

* Add support for erasure code pools

* Fix null on putAll
2025-04-02 08:19:00 -03:00
Daan Hoogland 8af021c6f6 Merge branch '4.20' 2025-03-27 17:03:13 +01:00
Daan Hoogland 5f93ce71bb Merge branch '4.19' into 4.20 2025-03-27 16:44:42 +01:00
Suresh Kumar Anaparti 9dceae4614
MS maintenance improvements (#10417)
* Update last agents during ms maintenance, and some code improvements

* Send 503 (Service Unavailable) response status when maintenance or shutdown is initiated
[Any load balancer in the clustered environment can avoid routing requests to this MS node]

* Migrate systemvm agents before routing host agents, and some code improvements

* Added events for ms maintenance and shutdown operations

* Added the following ms maintenance and shutdown improvements

- block new agent connections during prepare for maintenance of ms

- maintain avoids ms list

- propagate updated management servers list and lb algorithm in host and indirect.agent.lb.algorithm settings respectively, to systemvm (non-routing) agents

- updated setup ms list and migrate agent connections to executor service

- migrate agent connection through executor, and send the answer to the ms host that initiated the migration

- re-initialize ssl handshake executor if it is shutdown

- don't allow prepare for maintenance or shutdown when other management server nodes are in preparing states

- don't allow trigger shutdown when management server is up and other management server nodes are in preparing states

- stop agent connections monitor on ms maintenance

- update avoid ms list in ready command

- updated connected host from the client connection

- update last agents in ms metrics from the database

- updated some agent config descriptions

- update last management server in the hosts during shutdown

- added agents and lastagents in management server response

- updated management server maintenance & shutdown unit tests

- some code improvements

* refactored code / addressed comments

* removed shutdown testcase (maybe, calling System.exit)

* Revert "removed shutdown testcase (maybe, calling System.exit)"

This reverts commit e14b071715.

* avoid system.exit during shutdown test

* code improvements

* testcase fix

* Fix cutoff time in agent connections monitor thread
2025-03-19 14:18:05 +05:30
Wei Zhou 7978141464
api: fix EntityReference in NetworkResponse.java (#10563) 2025-03-18 09:20:50 +01:00
Pearl Dsilva ea36568e47 Merge branch '4.20' of https://github.com/apache/cloudstack 2025-03-17 17:17:43 -04:00
Pearl Dsilva 704d7a9787 Merge branch '4.19' of https://github.com/apache/cloudstack into 4.20 2025-03-17 17:17:21 -04:00
Pearl Dsilva 02d0dca24b
List only VMs associated to a userdata (#10569)
* List only VMs associated to a userdata

* add since param
2025-03-17 17:14:57 -04:00
nvazquez e904b1b60c
Merge branch 'main' into cks-enhancements-upstream 2025-03-17 09:23:12 -03:00
Abhishek Kumar 1c1dad977e Merge remote-tracking branch 'apache/4.20' 2025-03-06 09:55:27 +05:30
Pearl Dsilva 7f4e6a9d51
NAS B&R Plugin enhancements (#9666)
* NAS B&R Plugin enhancements

* Prevent printing mount opts which may include password by removing from response

* revert marvin change

* add sanity checks to validate minimum qemu and libvirt versions

* check is user running script is part of libvirt group

* revert changes of retore expunged VM

* add code coverage ignore file

* remove check

* issue with listing schedules and add defensive checks

* redirect logs to agent log file

* add some more debugging

* remove test file

* prevent deletion of cks cluster when vms associated to a backup offering

* delete all snapshot policies when bkp offering is disassociated from a VM

* Fix `updateTemplatePermission` when the UI is set to a language other than English (#9766)

* Fix updateTemplatePermission UI in non-english language

* Improve fix

---------

* Add nobrl in the mountopts for cifs file system

* Fix restoration of VM / volumes with cifs

* add cifs utils for el8

* add cifs-utils for ubuntu cloudstack-agent

* syntax error

* remove required constraint on both vmid and id params for the delete bkp schedule command
2025-03-04 11:32:09 -05:00
Pearl Dsilva 3a28a87483 Merge branch '4.20' of https://github.com/apache/cloudstack 2025-02-27 11:20:25 -05:00
Daan Hoogland 88916dcf2b Merge branch '4.19' into 4.20 2025-02-27 17:10:54 +01:00
Wei Zhou f992ebb52a
fix volume migration across cluster-scope pools (#10266) 2025-02-27 08:50:22 -05:00
nvazquez fbe0496b77
Merge branch 'main' into cks-enhancements-upstream 2025-02-26 10:23:02 -03:00
dahn 48f890a693
resolve merge problems in the backup framework (#10457)
Co-authored-by: Wei Zhou <weizhou@apache.org>
2025-02-26 14:12:40 +01:00
nvazquez ddaa65f4f3
Address review comments 2025-02-25 10:35:12 -03:00
Daan Hoogland 4a3686297d Updating pom.xml version numbers for release 4.19.3.0-SNAPSHOT
Signed-off-by: Daan Hoogland <daan@onecht.net>
2025-02-25 10:43:11 +01:00
Daan Hoogland 24b7c66251 Merge branch '4.20' 2025-02-24 14:33:12 +01:00
Daan Hoogland 08ad1c70ba Merge branch '4.19' into 4.20 2025-02-24 14:21:14 +01:00