Commit Graph

36390 Commits

Author SHA1 Message Date
nvazquez ab994dbfcf
Merge branch 'main' into nsx-integration 2023-11-06 08:15:33 -03:00
Nicolas Vazquez ecd32d5632
[NSX] Fix update source NAT IP (#8176)
* [NSX] Fix update source NAT IP

* Fix startup

* Fix API result
2023-11-06 08:15:00 -03:00
rRajivramachandran e9b24b6c32
Make authentication request parameter order to be deterministic (#8185) 2023-11-06 09:53:49 +01:00
slavkap 2bb182c3e1
KVM Host HA enhancement for StorPool storage (#8045)
Extending the current functionality of KVM Host HA for the StorPool storage plugin and the option for easy integration for the rest of the storage plugins to support Host HA

This extension works like the current NFS storage implementation. It allows it to be used simultaneously with NFS and StorPool storage or only with StorPool primary storage.

If it is used with different primary storages like NFS and StorPool, and one of the health checks fails for storage, there is an option to report the failure to the management with the global config kvm.ha.fence.on.storage.heartbeat.failure. By default this option is disabled when enabled the Host HA service will continue with the checks on the host and eventually will fence the host
2023-11-04 12:35:37 +05:30
dahn 0caf18bc1a
ui: indent issue in list view (#8184)
Fixes list view issues in UI after forward merge
2023-11-04 12:33:29 +05:30
nvazquez 42300aa80e
Merge branch 'main' into nsx-integration 2023-11-03 12:08:39 -03:00
vladimirpetrov 121531e2c3
Polish UI (#7855)
This PR aligns the use of terminology, renaming VM / virtual machine references to 'Instance' and also capitalising the terms Templates, Network, Snapshot, User, Account in CloudStack UI. “VM snapshots” moved under the “Compute” main menu, and renamed to “Instance Snapshots”. “Snapshots” (under Storage section) renamed to “Volume Snapshots”.
2023-11-03 18:18:40 +05:30
rRajivramachandran 6afffe1e17
Fix JSON string comparision fin tests (#8183) 2023-11-03 18:12:33 +05:30
John Bampton a7c6599ae2
Bump the `actions/checkout` action to `v4` (#8133)
https://github.com/actions/checkout
2023-11-03 18:10:49 +05:30
Codegass b2938c0528
Refactor testCRUDAcl into Separate Test Cases (#7705)
- Extracted shared ACL setup logic into a private helper method, setupAcl().
- Split original testCRUDAcl into two separate tests: testCRUDAclReadAll and testCRUDAclReadOne.
- Each test case now represents a unique scenario for better readability and maintainability.
- Replaced assertTrue(false) with fail() in catch blocks for better test failure indication.

These changes aim to enhance the clarity and maintainability of the test suite, and ensure each test case checks only one scenario.
2023-11-03 18:08:15 +05:30
Abhishek Kumar 256892b3f8 build: fix failure after forward merge
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-11-03 17:28:20 +05:30
Stephan Krug 8fa8cefb5b
Add and improve logs in ACS bootstrap process (#6890)
* Add and improve logs

* Fix unnecessary calls

---------

Co-authored-by: Stephan Krug <stephan.krug@scclouds.com.br>
2023-11-03 17:25:38 +05:30
gzhao9 9e8f591ace
Refactoring org.apache.cloudstack.network.tungsten.service (#8098)
* Refactoring reduces mock cloning of TungstenAnswer

* Apply suggestions from code review

Great suggestions, thanks a lot!

Co-authored-by: dahn <daan.hoogland@gmail.com>

* Rename CreateMockTungstenAnswer  to MockTungstenAnswerFactory

* Updated parameter to camel case.

* Revised in accordance with the latest update

* Replace all `\r` with `\n`.

* Replace all \r with \n.

* temp for re-uploading

* reupdate

* update line ending

* update ling ending

* Add static methods to avoid duplicate creation of new

---------

Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-11-03 17:19:59 +05:30
gzhao9 2f97e3bd83
refactor MockNetworkVO (#8137)
* refactor MockNetworkVO

* Apply suggestions from code review

Co-authored-by: dahn <daan.hoogland@gmail.com>

* adding static

adding a static method to the MockNetworkVO class that generates a MockNetworkVO rather than using new everytime.

---------

Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-11-03 17:19:32 +05:30
gzhao9 39c0706f1f
Update XenServer610WrapperTest.java (#8139)
* Update XenServer610WrapperTest.java

* Apply suggestions from code review

Co-authored-by: Rohit Yadav <rohityadav89@gmail.com>

---------

Co-authored-by: Rohit Yadav <rohityadav89@gmail.com>
2023-11-03 17:18:48 +05:30
Fabricio Duarte ec3d0f54fe
Standardize StringUtils usages in NetworlModelImpl (#7980)
The CloudStack coding conventions specify org.apache.commons.lang3.StringUtils as the default alternative for string operations, and com.cloud.utils.StringUtils for operations not covered by the former.

Since org.apache.commons.lang3.StringUtils is seen as the default alternative and is used more often in NetworkModelImpl.java, this PR inverts how these two classes are referenced in this file in order to standardize it.
2023-11-03 16:48:36 +05:30
Daan Hoogland a15cb81c85 Merge remote-tracking branch 'apache/4.18' into main 2023-11-03 11:55:26 +01:00
Wei Zhou e6f048bc2e
CKS: fix wrong format of cluster size on UI (#8182) 2023-11-03 10:01:12 +01:00
Harikrishna 1e133d05c7
kvm: Handle the failures when setting up memory balloon stats period for KVM VMs (#8049) 2023-11-03 09:07:11 +01:00
Rohit Yadav 78213da828
appliance: use Debian 11.8.0 and systemvm version to 4.19.0 (#8084)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-11-02 16:50:03 +01:00
Pierre-Luc Dion a3565fc8db
Update GPU support for V100,T4,A5500. (#6639) 2023-11-02 10:46:10 +01:00
João Jandre e1b6093c64
add 'management.network.cidr' ip to mgmt certificate (#7728) 2023-11-02 09:32:27 +01:00
John Bampton f090c77f41
misc: fix spelling (#7549)
Co-authored-by: Stephan Krug <stekrug@icloud.com>
2023-11-02 09:23:53 +01:00
sato03 9c4d18f9b1
role ispublic param on UI (#7973)
Co-authored-by: Henrique Sato <henrique.sato@scclouds.com.br>
2023-11-02 09:22:18 +01:00
John Bampton b2b707b90e
Add pre-commit hooks: check-symlinks and destroyed-symlinks (#8071)
https://github.com/pre-commit/pre-commit-hooks#check-symlinks

https://github.com/pre-commit/pre-commit-hooks#destroyed-symlinks
2023-11-02 09:07:50 +01:00
Pearl Dsilva 9ec7dccbd4 test failure fix 2023-11-01 21:56:13 -04:00
Pearl Dsilva cf1597a192 fix test failure 2023-11-01 21:42:07 -04:00
Pearl Dsilva 548f0408c5 Merge branch 'main' of https://github.com/apache/cloudstack into nsx-integration 2023-11-01 15:23:15 -04:00
Pearl Dsilva a36f355877
NSX: Router Public nic to get IP from systemVM Ip range (#8172)
* NSX: Router Public nic to get IP from systemVM Ip range

* Fix VR IP address and setSourceNatIp command

* NSX: hide systemVM reserved IP range SourceNAT

* fix test

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>
2023-11-01 16:14:33 -03:00
Abhishek Kumar 2dac6cb674
ui: fix snapshot copy actions for non-rootadmins (#8164)
Fixes Snapshot copy, zone delete option not showing in UI for non-rootadmin roles

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-11-01 17:32:09 +05:30
Vishesh 5362bad442
Storage Management (#7949) 2023-11-01 10:46:22 +01:00
nvazquez e3a2762235
Fix public NIC NPE on broadcast URI 2023-11-01 00:34:42 -03:00
Pearl Dsilva dc90d0b9dd
NSX: fix LB member addition and deletion and add defensive checks (#8167) 2023-10-31 21:35:02 -03:00
Nicolas Vazquez d0fe7b269f
NSX: Fix VR public NIC allocation (#8166) 2023-10-31 20:34:56 -03:00
Stephan Krug 288f066d9f
Add logs to script execution (#6874)
Co-authored-by: Stephan Krug <stephan.krug@scclouds.com.br>
2023-10-31 15:06:00 +01:00
Pearl Dsilva 20cb9f56f3 Merge branch 'main' of https://github.com/apache/cloudstack into nsx-integration 2023-10-31 06:10:52 -04:00
Harikrishna 235e4fe190
Oauth2 integration with CloudStack (#7996)
OAuth2, the industry-standard authorization or authentication framework, simplifies the process of
granting access to resources. CloudStack supports OAuth2 authentication wherein users can login into
CloudStack without using a username and password. Support for Google and Github providers has been added.
Other OAuth2 providers can be easily integrated with CloudStack using its plugin framework.

The login page will show provider options when the OAuth2 is enabled and corresponding providers are configured.

"OAuth configuration" sub-section is present under "Configuration" where admins can register the corresponding
OAuth providers.
2023-10-31 13:25:28 +05:30
Nicolas Vazquez a4be3eb380
[NSX] Add SNAT support (#8100)
* In progress add source NAT

* Fix after merge

* Fix tests

* Fix NPE on isolated network deletion

* Reserve source NAT IP when its not passed for NSX VPC

* Create source NAT rule on VR NIC allocation

* Fix update VPC and remove VPC to update and remove SNAT rule

* Fix packaging

* Address review comment

* Fix build

* fix build - unused import

* Add defensive checks

* Add missing design to NSX public guru

---------

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
2023-10-31 00:54:27 -03:00
Pearl Dsilva 72bdc38381 build failure fix 2023-10-30 20:41:44 -04:00
Pearl Dsilva c6d6463e20 Merge branch 'main' of https://github.com/apache/cloudstack into nsx-integration 2023-10-30 20:14:21 -04:00
Pearl Dsilva ff85d3df3b
Nsx add lb rule (#8161)
* NSX: Create and delete NSX Static Nat rules

* fix issues with static nat

* add static nat

* Support to add and delete Port forward rules

* add license

* fix adding multiple pf rules

* cleanup

* NSX: Add support to create and delete Load balancer rules

* fix deletion of lb rules

* add header file and update protocol detail
2023-10-30 19:55:03 -04:00
Pearl Dsilva ce1659e8fc
NSX: Create and Delete static NAT & Port forward rules (#8131)
* NSX: Create and delete NSX Static Nat rules

* fix issues with static nat

* add static nat

* Support to add and delete Port forward rules

* add license

* fix adding multiple pf rules

* cleanup

* fix lint check

* fix smoke tests

* fix smoke tests
2023-10-30 19:37:12 -04:00
Harikrishna 67eddd7c0e
Fix allignment on layout on right panel (#8060) 2023-10-30 12:50:17 +01:00
Wei Zhou bd52fa8a12
New feature: VNF templates and appliances integration (#8022) 2023-10-27 10:23:00 +02:00
Vishesh 3b11663d87
Fix failure on agent reconnection (#8089) 2023-10-26 16:54:36 +02:00
Vishesh a06f8a8763
Fixup updateSnapshotInfo failure due to forward merging (#8150)
Fixes build failures
2023-10-26 17:14:11 +05:30
Daan Hoogland 587d1d7dba Merge remote-tracking branch 'apache/4.18' into main 2023-10-26 09:37:38 +02:00
slavkap 6ae3b73ca2
Create snapshot from VM snapshot without memory for NFS/Local storage (#8117) 2023-10-26 08:46:14 +02:00
Marcus Sorensen 4ff592ac2c
Use UserVmDao for listVirtualMachines API to increase performance (#8012)
Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-10-26 08:44:09 +02:00
Vishesh ea90848429
Feature: Add support for DRS in a Cluster (#7723)
This pull request (PR) implements a Distributed Resource Scheduler (DRS) for a CloudStack cluster. The primary objective of this feature is to enable automatic resource optimization and workload balancing within the cluster by live migrating the VMs as per configuration.
Administrators can also execute DRS manually for a cluster, using the UI or the API.
Adds support for two algorithms - condensed & balanced. Algorithms are pluggable allowing ACS Administrators to have customized control over scheduling.

Implementation
There are three top level components:

    Scheduler
    A timer task which:

    Generate DRS plan for clusters
    Process DRS plan
    Remove old DRS plan records

    DRS Execution
    We go through each VM in the cluster and use the specified algorithm to check if DRS is required and to calculate cost, benefit & improvement of migrating that VM to another host in the cluster. On the basis of cost, benefit & improvement, the best migration is selected for the current iteration and the VM is migrated. The maximum number of iterations (live migrations) possible on the cluster is defined by drs.iterations which is defined as a percentage (as a value between 0 and 1) of total number of workloads.

    Algorithm
    Every algorithms implements two methods:
        needsDrs - to check if drs is required for cluster
        getMetrics - to calculate cost, benefit & improvement of a migrating a VM to another host.

Algorithms

    Condensed - Packs all the VMs on minimum number of hosts in the cluster.
    Balanced - Distributes the VMs evenly across hosts in the cluster.
    Algorithms use drs.level to decide the amount of imbalance to allow in the cluster.

APIs Added

listClusterDrsPlan

    id - ID of the DRS plan to list
    clusterid - to list plans for a cluster id

generateClusterDrsPlan

    id - cluster id
    iterations - The maximum number of iterations in a DRS job defined as a percentage (as a value between 0 and 1) of total number of workloads. Defaults to value of cluster's drs.iterations setting.

executeClusterDrsPlan

    id - ID of the cluster for which DRS plan is to be executed.
    migrateto - This parameter specifies the mapping between a vm and a host to migrate that VM. Format of this parameter: migrateto[vm-index].vm=<uuid>&migrateto[vm-index].host=<uuid>.

Config Keys Added

    ClusterDrsPlanExpireInterval
    Key drs.plan.expire.interval
    Scope Global
    Default Value 30 days
    Description The interval in days after which old DRS records will be cleaned up.

    ClusterDrsEnabled
    Key drs.automatic.enable
    Scope Cluster
    Default Value false
    Description Enable/disable automatic DRS on a cluster.

    ClusterDrsInterval
    Key drs.automatic.interval
    Scope Cluster
    Default Value 60 minutes
    Description The interval in minutes after which a periodic background thread will schedule DRS for a cluster.

    ClusterDrsIterations
    Key drs.max.migrations
    Scope Cluster
    Default Value 50
    Description Maximum number of live migrations in a DRS execution.

    ClusterDrsAlgorithm
    Key drs.algorithm
    Scope Cluster
    Default Value condensed
    Description DRS algorithm to execute on the cluster. This PR implements two algorithms - balanced & condensed.

    ClusterDrsLevel
    Key drs.imbalance
    Scope Cluster
    Default Value 0.5
    Description Percentage (as a value between 0.0 and 1.0) of imbalance allowed in the cluster. 1.0 means no imbalance
    is allowed and 0.0 means imbalance is allowed.

    ClusterDrsMetric
    Key drs.imbalance.metric
    Scope Cluster
    Default Value memory
    Description The cluster imbalance metric to use when checking the drs.imbalance.threshold. Possible values are memory and cpu.
2023-10-26 11:48:18 +05:30