Commit Graph

36700 Commits

Author SHA1 Message Date
rRajivramachandran 6afffe1e17
Fix JSON string comparision fin tests (#8183) 2023-11-03 18:12:33 +05:30
John Bampton a7c6599ae2
Bump the `actions/checkout` action to `v4` (#8133)
https://github.com/actions/checkout
2023-11-03 18:10:49 +05:30
Codegass b2938c0528
Refactor testCRUDAcl into Separate Test Cases (#7705)
- Extracted shared ACL setup logic into a private helper method, setupAcl().
- Split original testCRUDAcl into two separate tests: testCRUDAclReadAll and testCRUDAclReadOne.
- Each test case now represents a unique scenario for better readability and maintainability.
- Replaced assertTrue(false) with fail() in catch blocks for better test failure indication.

These changes aim to enhance the clarity and maintainability of the test suite, and ensure each test case checks only one scenario.
2023-11-03 18:08:15 +05:30
Abhishek Kumar 256892b3f8 build: fix failure after forward merge
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-11-03 17:28:20 +05:30
Stephan Krug 8fa8cefb5b
Add and improve logs in ACS bootstrap process (#6890)
* Add and improve logs

* Fix unnecessary calls

---------

Co-authored-by: Stephan Krug <stephan.krug@scclouds.com.br>
2023-11-03 17:25:38 +05:30
gzhao9 9e8f591ace
Refactoring org.apache.cloudstack.network.tungsten.service (#8098)
* Refactoring reduces mock cloning of TungstenAnswer

* Apply suggestions from code review

Great suggestions, thanks a lot!

Co-authored-by: dahn <daan.hoogland@gmail.com>

* Rename CreateMockTungstenAnswer  to MockTungstenAnswerFactory

* Updated parameter to camel case.

* Revised in accordance with the latest update

* Replace all `\r` with `\n`.

* Replace all \r with \n.

* temp for re-uploading

* reupdate

* update line ending

* update ling ending

* Add static methods to avoid duplicate creation of new

---------

Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-11-03 17:19:59 +05:30
gzhao9 2f97e3bd83
refactor MockNetworkVO (#8137)
* refactor MockNetworkVO

* Apply suggestions from code review

Co-authored-by: dahn <daan.hoogland@gmail.com>

* adding static

adding a static method to the MockNetworkVO class that generates a MockNetworkVO rather than using new everytime.

---------

Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-11-03 17:19:32 +05:30
gzhao9 39c0706f1f
Update XenServer610WrapperTest.java (#8139)
* Update XenServer610WrapperTest.java

* Apply suggestions from code review

Co-authored-by: Rohit Yadav <rohityadav89@gmail.com>

---------

Co-authored-by: Rohit Yadav <rohityadav89@gmail.com>
2023-11-03 17:18:48 +05:30
Fabricio Duarte ec3d0f54fe
Standardize StringUtils usages in NetworlModelImpl (#7980)
The CloudStack coding conventions specify org.apache.commons.lang3.StringUtils as the default alternative for string operations, and com.cloud.utils.StringUtils for operations not covered by the former.

Since org.apache.commons.lang3.StringUtils is seen as the default alternative and is used more often in NetworkModelImpl.java, this PR inverts how these two classes are referenced in this file in order to standardize it.
2023-11-03 16:48:36 +05:30
Daan Hoogland a15cb81c85 Merge remote-tracking branch 'apache/4.18' into main 2023-11-03 11:55:26 +01:00
Wei Zhou e6f048bc2e
CKS: fix wrong format of cluster size on UI (#8182) 2023-11-03 10:01:12 +01:00
Harikrishna 1e133d05c7
kvm: Handle the failures when setting up memory balloon stats period for KVM VMs (#8049) 2023-11-03 09:07:11 +01:00
Rohit Yadav 78213da828
appliance: use Debian 11.8.0 and systemvm version to 4.19.0 (#8084)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-11-02 16:50:03 +01:00
Pierre-Luc Dion a3565fc8db
Update GPU support for V100,T4,A5500. (#6639) 2023-11-02 10:46:10 +01:00
João Jandre e1b6093c64
add 'management.network.cidr' ip to mgmt certificate (#7728) 2023-11-02 09:32:27 +01:00
John Bampton f090c77f41
misc: fix spelling (#7549)
Co-authored-by: Stephan Krug <stekrug@icloud.com>
2023-11-02 09:23:53 +01:00
sato03 9c4d18f9b1
role ispublic param on UI (#7973)
Co-authored-by: Henrique Sato <henrique.sato@scclouds.com.br>
2023-11-02 09:22:18 +01:00
John Bampton b2b707b90e
Add pre-commit hooks: check-symlinks and destroyed-symlinks (#8071)
https://github.com/pre-commit/pre-commit-hooks#check-symlinks

https://github.com/pre-commit/pre-commit-hooks#destroyed-symlinks
2023-11-02 09:07:50 +01:00
Abhishek Kumar 2dac6cb674
ui: fix snapshot copy actions for non-rootadmins (#8164)
Fixes Snapshot copy, zone delete option not showing in UI for non-rootadmin roles

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-11-01 17:32:09 +05:30
Vishesh 5362bad442
Storage Management (#7949) 2023-11-01 10:46:22 +01:00
Stephan Krug 288f066d9f
Add logs to script execution (#6874)
Co-authored-by: Stephan Krug <stephan.krug@scclouds.com.br>
2023-10-31 15:06:00 +01:00
Harikrishna 235e4fe190
Oauth2 integration with CloudStack (#7996)
OAuth2, the industry-standard authorization or authentication framework, simplifies the process of
granting access to resources. CloudStack supports OAuth2 authentication wherein users can login into
CloudStack without using a username and password. Support for Google and Github providers has been added.
Other OAuth2 providers can be easily integrated with CloudStack using its plugin framework.

The login page will show provider options when the OAuth2 is enabled and corresponding providers are configured.

"OAuth configuration" sub-section is present under "Configuration" where admins can register the corresponding
OAuth providers.
2023-10-31 13:25:28 +05:30
Harikrishna 67eddd7c0e
Fix allignment on layout on right panel (#8060) 2023-10-30 12:50:17 +01:00
Wei Zhou bd52fa8a12
New feature: VNF templates and appliances integration (#8022) 2023-10-27 10:23:00 +02:00
Vishesh 3b11663d87
Fix failure on agent reconnection (#8089) 2023-10-26 16:54:36 +02:00
Vishesh a06f8a8763
Fixup updateSnapshotInfo failure due to forward merging (#8150)
Fixes build failures
2023-10-26 17:14:11 +05:30
Daan Hoogland 587d1d7dba Merge remote-tracking branch 'apache/4.18' into main 2023-10-26 09:37:38 +02:00
slavkap 6ae3b73ca2
Create snapshot from VM snapshot without memory for NFS/Local storage (#8117) 2023-10-26 08:46:14 +02:00
Marcus Sorensen 4ff592ac2c
Use UserVmDao for listVirtualMachines API to increase performance (#8012)
Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-10-26 08:44:09 +02:00
Vishesh ea90848429
Feature: Add support for DRS in a Cluster (#7723)
This pull request (PR) implements a Distributed Resource Scheduler (DRS) for a CloudStack cluster. The primary objective of this feature is to enable automatic resource optimization and workload balancing within the cluster by live migrating the VMs as per configuration.
Administrators can also execute DRS manually for a cluster, using the UI or the API.
Adds support for two algorithms - condensed & balanced. Algorithms are pluggable allowing ACS Administrators to have customized control over scheduling.

Implementation
There are three top level components:

    Scheduler
    A timer task which:

    Generate DRS plan for clusters
    Process DRS plan
    Remove old DRS plan records

    DRS Execution
    We go through each VM in the cluster and use the specified algorithm to check if DRS is required and to calculate cost, benefit & improvement of migrating that VM to another host in the cluster. On the basis of cost, benefit & improvement, the best migration is selected for the current iteration and the VM is migrated. The maximum number of iterations (live migrations) possible on the cluster is defined by drs.iterations which is defined as a percentage (as a value between 0 and 1) of total number of workloads.

    Algorithm
    Every algorithms implements two methods:
        needsDrs - to check if drs is required for cluster
        getMetrics - to calculate cost, benefit & improvement of a migrating a VM to another host.

Algorithms

    Condensed - Packs all the VMs on minimum number of hosts in the cluster.
    Balanced - Distributes the VMs evenly across hosts in the cluster.
    Algorithms use drs.level to decide the amount of imbalance to allow in the cluster.

APIs Added

listClusterDrsPlan

    id - ID of the DRS plan to list
    clusterid - to list plans for a cluster id

generateClusterDrsPlan

    id - cluster id
    iterations - The maximum number of iterations in a DRS job defined as a percentage (as a value between 0 and 1) of total number of workloads. Defaults to value of cluster's drs.iterations setting.

executeClusterDrsPlan

    id - ID of the cluster for which DRS plan is to be executed.
    migrateto - This parameter specifies the mapping between a vm and a host to migrate that VM. Format of this parameter: migrateto[vm-index].vm=<uuid>&migrateto[vm-index].host=<uuid>.

Config Keys Added

    ClusterDrsPlanExpireInterval
    Key drs.plan.expire.interval
    Scope Global
    Default Value 30 days
    Description The interval in days after which old DRS records will be cleaned up.

    ClusterDrsEnabled
    Key drs.automatic.enable
    Scope Cluster
    Default Value false
    Description Enable/disable automatic DRS on a cluster.

    ClusterDrsInterval
    Key drs.automatic.interval
    Scope Cluster
    Default Value 60 minutes
    Description The interval in minutes after which a periodic background thread will schedule DRS for a cluster.

    ClusterDrsIterations
    Key drs.max.migrations
    Scope Cluster
    Default Value 50
    Description Maximum number of live migrations in a DRS execution.

    ClusterDrsAlgorithm
    Key drs.algorithm
    Scope Cluster
    Default Value condensed
    Description DRS algorithm to execute on the cluster. This PR implements two algorithms - balanced & condensed.

    ClusterDrsLevel
    Key drs.imbalance
    Scope Cluster
    Default Value 0.5
    Description Percentage (as a value between 0.0 and 1.0) of imbalance allowed in the cluster. 1.0 means no imbalance
    is allowed and 0.0 means imbalance is allowed.

    ClusterDrsMetric
    Key drs.imbalance.metric
    Scope Cluster
    Default Value memory
    Description The cluster imbalance metric to use when checking the drs.imbalance.threshold. Possible values are memory and cpu.
2023-10-26 11:48:18 +05:30
Abhishek Kumar 68e1d46698
reduce collaborators count to 10 (#8149)
Maximum 10 collaborators are supported by ASF infra.

soreana is already a committer now and can be removed from this list.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-10-26 11:17:52 +05:30
Fabricio Duarte fb76fd10f0
Add winterhazel to .asf.yaml collaborators (#8147)
Added winterhazel as a collaborator in .asf.yaml to be able to report issues with tags, and be assigned to issues and PRs.
2023-10-26 10:36:35 +05:30
Vishesh 27ae7d8bc4
Fixup: register vmscheduler configkey (#8116)
This PR registers the global setting VMScheduledJobExpireInterval.
2023-10-26 00:57:10 +05:30
Rohit Yadav ebf1409622
dashboard: on admin dashboard use red-dot icon when hosts are in alert (#8144)
This improves the function to not show the alert status on host icon
when there are no hosts in alert state. Instead a colour/theme matching
exclaimation is show next to the host icon, otherwise show green tick.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-25 22:04:30 +05:30
Aakash Sinha 8fe13f91a1
OSType response: isuserdefined as string not bool #8127 (#8128)
Co-authored-by: Aakash Sinha <mailtomeaakash@gmail.com>
2023-10-25 11:02:57 +02:00
Abhishek Kumar 39d6c20fb3 Merge remote-tracking branch 'apache/4.18' into main 2023-10-25 11:08:06 +05:30
Harikrishna ae724b9d4c
Fix EULA section while parsing OVF file (#8081)
* Fix EULA section while parsing OVF file
* Updated unit test to check both the cases
* Added separate tests

Fixes #8039
2023-10-25 11:01:01 +05:30
Nicolas Vazquez b09acb8fd0
UI: Fix deploy VM wizard vApp properties (#8072)
This PR fixes the vApp properties values sent as part of the deployVirtualMachine API through the UI

Fixes: #7998
2023-10-25 10:28:42 +05:30
Abhishek Kumar 543c54c718
api,server,ui: snapshot copy, multi-zone replica (#7873)
This PR adds new functionality to copy snapshots across zones and take snapshots for multiple zones.

Copy functionality is similar to template copy. The source zone acts as the web server from where the destination zone(s) can download the snapshot files. For this purpose, a new API - `copySnapshot` has been added. The response for copySnapshot will be returning zone and download details from the first destination zone of the request. This behaviour is similar to the `copyTemplate` API.

In a similar manner, multiple zones can be selected while taking the snapshots or creating snapshot policies. For this snapshot will be taken in the base zone(in which volume is present) and then copied to the additional zones. A new parameter - `zoneids` has been added to `createSnapshot` and `createSnapshotPolicy` APIs.

As snapshots can be present on multiple zones (secondary stores), a new parameter `zoneid` has been added to delete the snapshot copy on a specific zone.

`listSnapshots` API has been updated to allow listing snapshot entries for different zones/datastores. New parameters - `showUnique`, `locationType` have been added.

Events generated during snapshot operations will now be linked to the snapshot itself rather than the volume of the snapshot.

`listSnapshotPolicies` and `createSnapshotPolicy` APIs will return zone details of the zones in which backup will be scheduled for the policy.

----
New API added
`copySnapshot`

Request and response params updated for APIs
```
- listSnapshots
- deleteSnapshot
- createTemplate
- listZones
- listSnapshotPolicies
- createSnapshotPolicy
```
UI updated for
- Snapshot detail view
- Create snapshot form
- Create snapshot policy form
- Create volume (from snapshot) form
- Create template (from snapshot) form

Doc PR: https://github.com/apache/cloudstack-documentation/pull/344
PR: https://github.com/apache/cloudstack/pull/7873
2023-10-23 09:01:58 +02:00
Abhishek Kumar 99ded8169b Merge remote-tracking branch 'apache/4.18' into main 2023-10-20 17:40:19 +05:30
sato03 a8700bff7f
server: set Default NIC when VM has no default NIC (#7859)
Co-authored-by: Henrique Sato <henrique.sato@scclouds.com.br>
2023-10-20 11:40:10 +02:00
Fabricio Duarte d3d3027ba6
Fix ISOs and templates listing pagination (#7974) 2023-10-20 11:37:04 +02:00
Abhishek Kumar 1bb6130795 Merge remote-tracking branch 'apache/4.18' into main 2023-10-20 11:00:22 +05:30
Abhishek Kumar e199678101
ui: fix bulk delete template from zones (#8118)
Fixes #8083

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-10-20 10:36:20 +05:30
Abhishek Kumar a2ec1f3777
marvin,test: fix directdownload template checksum test (#8096)
* marvin,test: fix directdownload template checksum test

During failure while deploying a VM with wrong checksum template, VM may be left in Error state. This PR adds code to delete such VM.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove unnecessary logs

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-10-19 19:40:03 +02:00
Abhishek Kumar f62b634033
ui: correctly show volume physical size (#8119)
Fixes #8073

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-10-19 15:58:58 +05:30
Abhishek Kumar fcbf540369 Merge remote-tracking branch 'apache/4.18' into main 2023-10-19 10:48:00 +05:30
Peinthor Rene 67cb9b9e40
linstor: fix template copy on non hyperconverged setups (#8114)
Making a diskful resource was meant as an optimization,
but cannot work on non hyperconverged setups,
as the storage nodes (diskful) are not part of the cloudstack cluster.
2023-10-19 10:46:20 +05:30
Abhishek Kumar 20046ffe61 Merge remote-tracking branch 'apache/4.18' into main 2023-10-19 10:39:40 +05:30
Abhishek Kumar 540c7b802f
test: add test for standalone snapshot (#8104)
Fixes #8034

Adds the following test for a backed-up snapshot (original template and VM deleted beforehand):
- Create volume from snapshot
- Create a template from the snapshot and deploy a VM using it

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-10-19 09:48:16 +05:30