Commit Graph

4448 Commits

Author SHA1 Message Date
Pearl Dsilva 6fcdf2092f build failure - fix unit test 2023-11-14 22:43:25 -05:00
Pearl Dsilva b0257dfaba Add Firewall rules 2023-11-14 22:24:44 -05:00
Pearl Dsilva 96de7e3fcc Add support for routed NSX Isolated networks \n and non RFC 1918 compliant IPs 2023-11-14 11:02:52 -05:00
Pearl Dsilva 770a96ae40 Add support for routed NSX Isolated networks \n and non RFC 1918 compliant IPs 2023-11-14 10:39:31 -05:00
Pearl Dsilva e7f5c59b27 Merge branch 'nsx-acl-types' of https://github.com/shapeblue/cloudstack into nsx-cks-support 2023-11-14 10:08:14 -05:00
nvazquez d72829c602
Fix policy rule ID and add more unit tests 2023-11-13 22:59:21 -03:00
nvazquez 1a24ba6949
Remove distributed firewall policy on segment deletion 2023-11-13 16:44:22 -03:00
nvazquez c6acd8fb61
Create Distributed Firewall rules 2023-11-13 16:20:57 -03:00
nvazquez bd9be1239c
Remove group for segment before removing segment 2023-11-12 22:55:17 -03:00
nvazquez 28b4fee6c2
Add unit tests 2023-11-12 22:33:01 -03:00
nvazquez e5965a152f
NSX: Create segment group on segment creation 2023-11-12 21:07:51 -03:00
nvazquez 053521077c
Merge branch 'main' into nsx-integration 2023-11-10 13:50:26 -03:00
Peinthor Rene 68e504aff9
Linstor backup snaphots (#8067)
This PR adds an config option for the Linstor primary storage driver, that allows you to automatically backup
volume snapshots to the secondary storage.
Additionally it will not mangle the need java-linstor dependency into the client.jar, but instead just copy
the java-linstor.jar into lib.

Config option is called: lin.backup.snapshots and is default false

The scope of this change should be limited, as it only touches the Linstor driver and a part of copyAsync
was implemented with 2 new Linstor specific commands.
2023-11-09 09:38:10 +05:30
Wei Zhou 861107fa5b
CKS: make clustertype optional to keep backwards compatibility (#8180)
This PR fixes the issue that 4.18 cmk/api to create CKS cluster does not work in 4.19
2023-11-08 00:31:38 +05:30
Pearl Dsilva 88a07f479f
NSX - add LB route Advertizement (#8192) 2023-11-07 11:05:02 -03:00
Pearl Dsilva daea80d12b Merge branch 'nsx-add-lb-route-adv' of https://github.com/apache/cloudstack into nsx-cks-support 2023-11-07 08:43:29 -05:00
Pearl Dsilva c70abf00d3 NSX: Cks Support 2023-11-06 16:04:34 -05:00
nvazquez ab994dbfcf
Merge branch 'main' into nsx-integration 2023-11-06 08:15:33 -03:00
Nicolas Vazquez ecd32d5632
[NSX] Fix update source NAT IP (#8176)
* [NSX] Fix update source NAT IP

* Fix startup

* Fix API result
2023-11-06 08:15:00 -03:00
rRajivramachandran e9b24b6c32
Make authentication request parameter order to be deterministic (#8185) 2023-11-06 09:53:49 +01:00
slavkap 2bb182c3e1
KVM Host HA enhancement for StorPool storage (#8045)
Extending the current functionality of KVM Host HA for the StorPool storage plugin and the option for easy integration for the rest of the storage plugins to support Host HA

This extension works like the current NFS storage implementation. It allows it to be used simultaneously with NFS and StorPool storage or only with StorPool primary storage.

If it is used with different primary storages like NFS and StorPool, and one of the health checks fails for storage, there is an option to report the failure to the management with the global config kvm.ha.fence.on.storage.heartbeat.failure. By default this option is disabled when enabled the Host HA service will continue with the checks on the host and eventually will fence the host
2023-11-04 12:35:37 +05:30
nvazquez 42300aa80e
Merge branch 'main' into nsx-integration 2023-11-03 12:08:39 -03:00
Codegass b2938c0528
Refactor testCRUDAcl into Separate Test Cases (#7705)
- Extracted shared ACL setup logic into a private helper method, setupAcl().
- Split original testCRUDAcl into two separate tests: testCRUDAclReadAll and testCRUDAclReadOne.
- Each test case now represents a unique scenario for better readability and maintainability.
- Replaced assertTrue(false) with fail() in catch blocks for better test failure indication.

These changes aim to enhance the clarity and maintainability of the test suite, and ensure each test case checks only one scenario.
2023-11-03 18:08:15 +05:30
gzhao9 9e8f591ace
Refactoring org.apache.cloudstack.network.tungsten.service (#8098)
* Refactoring reduces mock cloning of TungstenAnswer

* Apply suggestions from code review

Great suggestions, thanks a lot!

Co-authored-by: dahn <daan.hoogland@gmail.com>

* Rename CreateMockTungstenAnswer  to MockTungstenAnswerFactory

* Updated parameter to camel case.

* Revised in accordance with the latest update

* Replace all `\r` with `\n`.

* Replace all \r with \n.

* temp for re-uploading

* reupdate

* update line ending

* update ling ending

* Add static methods to avoid duplicate creation of new

---------

Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-11-03 17:19:59 +05:30
gzhao9 2f97e3bd83
refactor MockNetworkVO (#8137)
* refactor MockNetworkVO

* Apply suggestions from code review

Co-authored-by: dahn <daan.hoogland@gmail.com>

* adding static

adding a static method to the MockNetworkVO class that generates a MockNetworkVO rather than using new everytime.

---------

Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-11-03 17:19:32 +05:30
gzhao9 39c0706f1f
Update XenServer610WrapperTest.java (#8139)
* Update XenServer610WrapperTest.java

* Apply suggestions from code review

Co-authored-by: Rohit Yadav <rohityadav89@gmail.com>

---------

Co-authored-by: Rohit Yadav <rohityadav89@gmail.com>
2023-11-03 17:18:48 +05:30
Daan Hoogland a15cb81c85 Merge remote-tracking branch 'apache/4.18' into main 2023-11-03 11:55:26 +01:00
Harikrishna 1e133d05c7
kvm: Handle the failures when setting up memory balloon stats period for KVM VMs (#8049) 2023-11-03 09:07:11 +01:00
Pearl Dsilva 01384c6b48 NSX: Add ALL LB IP to the list of route advertisements in tier1 2023-11-02 10:36:32 -04:00
João Jandre e1b6093c64
add 'management.network.cidr' ip to mgmt certificate (#7728) 2023-11-02 09:32:27 +01:00
John Bampton f090c77f41
misc: fix spelling (#7549)
Co-authored-by: Stephan Krug <stekrug@icloud.com>
2023-11-02 09:23:53 +01:00
Pearl Dsilva 548f0408c5 Merge branch 'main' of https://github.com/apache/cloudstack into nsx-integration 2023-11-01 15:23:15 -04:00
Pearl Dsilva a36f355877
NSX: Router Public nic to get IP from systemVM Ip range (#8172)
* NSX: Router Public nic to get IP from systemVM Ip range

* Fix VR IP address and setSourceNatIp command

* NSX: hide systemVM reserved IP range SourceNAT

* fix test

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>
2023-11-01 16:14:33 -03:00
Vishesh 5362bad442
Storage Management (#7949) 2023-11-01 10:46:22 +01:00
Pearl Dsilva dc90d0b9dd
NSX: fix LB member addition and deletion and add defensive checks (#8167) 2023-10-31 21:35:02 -03:00
Nicolas Vazquez d0fe7b269f
NSX: Fix VR public NIC allocation (#8166) 2023-10-31 20:34:56 -03:00
Pearl Dsilva 20cb9f56f3 Merge branch 'main' of https://github.com/apache/cloudstack into nsx-integration 2023-10-31 06:10:52 -04:00
Harikrishna 235e4fe190
Oauth2 integration with CloudStack (#7996)
OAuth2, the industry-standard authorization or authentication framework, simplifies the process of
granting access to resources. CloudStack supports OAuth2 authentication wherein users can login into
CloudStack without using a username and password. Support for Google and Github providers has been added.
Other OAuth2 providers can be easily integrated with CloudStack using its plugin framework.

The login page will show provider options when the OAuth2 is enabled and corresponding providers are configured.

"OAuth configuration" sub-section is present under "Configuration" where admins can register the corresponding
OAuth providers.
2023-10-31 13:25:28 +05:30
Nicolas Vazquez a4be3eb380
[NSX] Add SNAT support (#8100)
* In progress add source NAT

* Fix after merge

* Fix tests

* Fix NPE on isolated network deletion

* Reserve source NAT IP when its not passed for NSX VPC

* Create source NAT rule on VR NIC allocation

* Fix update VPC and remove VPC to update and remove SNAT rule

* Fix packaging

* Address review comment

* Fix build

* fix build - unused import

* Add defensive checks

* Add missing design to NSX public guru

---------

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
2023-10-31 00:54:27 -03:00
Pearl Dsilva 72bdc38381 build failure fix 2023-10-30 20:41:44 -04:00
Pearl Dsilva c6d6463e20 Merge branch 'main' of https://github.com/apache/cloudstack into nsx-integration 2023-10-30 20:14:21 -04:00
Pearl Dsilva ff85d3df3b
Nsx add lb rule (#8161)
* NSX: Create and delete NSX Static Nat rules

* fix issues with static nat

* add static nat

* Support to add and delete Port forward rules

* add license

* fix adding multiple pf rules

* cleanup

* NSX: Add support to create and delete Load balancer rules

* fix deletion of lb rules

* add header file and update protocol detail
2023-10-30 19:55:03 -04:00
Pearl Dsilva ce1659e8fc
NSX: Create and Delete static NAT & Port forward rules (#8131)
* NSX: Create and delete NSX Static Nat rules

* fix issues with static nat

* add static nat

* Support to add and delete Port forward rules

* add license

* fix adding multiple pf rules

* cleanup

* fix lint check

* fix smoke tests

* fix smoke tests
2023-10-30 19:37:12 -04:00
Daan Hoogland 587d1d7dba Merge remote-tracking branch 'apache/4.18' into main 2023-10-26 09:37:38 +02:00
slavkap 6ae3b73ca2
Create snapshot from VM snapshot without memory for NFS/Local storage (#8117) 2023-10-26 08:46:14 +02:00
Vishesh ea90848429
Feature: Add support for DRS in a Cluster (#7723)
This pull request (PR) implements a Distributed Resource Scheduler (DRS) for a CloudStack cluster. The primary objective of this feature is to enable automatic resource optimization and workload balancing within the cluster by live migrating the VMs as per configuration.
Administrators can also execute DRS manually for a cluster, using the UI or the API.
Adds support for two algorithms - condensed & balanced. Algorithms are pluggable allowing ACS Administrators to have customized control over scheduling.

Implementation
There are three top level components:

    Scheduler
    A timer task which:

    Generate DRS plan for clusters
    Process DRS plan
    Remove old DRS plan records

    DRS Execution
    We go through each VM in the cluster and use the specified algorithm to check if DRS is required and to calculate cost, benefit & improvement of migrating that VM to another host in the cluster. On the basis of cost, benefit & improvement, the best migration is selected for the current iteration and the VM is migrated. The maximum number of iterations (live migrations) possible on the cluster is defined by drs.iterations which is defined as a percentage (as a value between 0 and 1) of total number of workloads.

    Algorithm
    Every algorithms implements two methods:
        needsDrs - to check if drs is required for cluster
        getMetrics - to calculate cost, benefit & improvement of a migrating a VM to another host.

Algorithms

    Condensed - Packs all the VMs on minimum number of hosts in the cluster.
    Balanced - Distributes the VMs evenly across hosts in the cluster.
    Algorithms use drs.level to decide the amount of imbalance to allow in the cluster.

APIs Added

listClusterDrsPlan

    id - ID of the DRS plan to list
    clusterid - to list plans for a cluster id

generateClusterDrsPlan

    id - cluster id
    iterations - The maximum number of iterations in a DRS job defined as a percentage (as a value between 0 and 1) of total number of workloads. Defaults to value of cluster's drs.iterations setting.

executeClusterDrsPlan

    id - ID of the cluster for which DRS plan is to be executed.
    migrateto - This parameter specifies the mapping between a vm and a host to migrate that VM. Format of this parameter: migrateto[vm-index].vm=<uuid>&migrateto[vm-index].host=<uuid>.

Config Keys Added

    ClusterDrsPlanExpireInterval
    Key drs.plan.expire.interval
    Scope Global
    Default Value 30 days
    Description The interval in days after which old DRS records will be cleaned up.

    ClusterDrsEnabled
    Key drs.automatic.enable
    Scope Cluster
    Default Value false
    Description Enable/disable automatic DRS on a cluster.

    ClusterDrsInterval
    Key drs.automatic.interval
    Scope Cluster
    Default Value 60 minutes
    Description The interval in minutes after which a periodic background thread will schedule DRS for a cluster.

    ClusterDrsIterations
    Key drs.max.migrations
    Scope Cluster
    Default Value 50
    Description Maximum number of live migrations in a DRS execution.

    ClusterDrsAlgorithm
    Key drs.algorithm
    Scope Cluster
    Default Value condensed
    Description DRS algorithm to execute on the cluster. This PR implements two algorithms - balanced & condensed.

    ClusterDrsLevel
    Key drs.imbalance
    Scope Cluster
    Default Value 0.5
    Description Percentage (as a value between 0.0 and 1.0) of imbalance allowed in the cluster. 1.0 means no imbalance
    is allowed and 0.0 means imbalance is allowed.

    ClusterDrsMetric
    Key drs.imbalance.metric
    Scope Cluster
    Default Value memory
    Description The cluster imbalance metric to use when checking the drs.imbalance.threshold. Possible values are memory and cpu.
2023-10-26 11:48:18 +05:30
Nicolas Vazquez f46b7883a8
[NSX] Allow NSX isolated networks (#8132)
* Add network offerings for NSX on isolated networks

* Fix offerings creation

* In progress NSX isolated network

* Fixes

* Fix NIC allocation to router
2023-10-24 15:00:41 -03:00
Pearl Dsilva 2fe8e9cd96 Merge branch 'nsx-integration' of https://github.com/apache/cloudstack into nsx-integration 2023-10-23 11:53:01 -04:00
Pearl Dsilva be2b8edd20
NSX: Refactor Network & VPC offering (#8110)
* [NSX] Refactor API wrapper operations

* Network offering changes for NSX

* fix services and provider combination

* address comments: rename param

* update nsx_mode parameter

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>
2023-10-23 12:50:13 -03:00
Pearl Dsilva 69ded800e9
modify NSX resource naming convention (#8095)
* modify NSX resource naming convention

* remove unused imports

* add a setup phase between desgin and implementation of a network for intermediary steps

* add method to all classes
2023-10-23 12:31:30 -03:00