Commit Graph

170 Commits

Author SHA1 Message Date
Pearl Dsilva 93c181dfd5 Merge branch 'main' of https://github.com/apache/cloudstack into nsx-integration 2024-02-08 10:04:57 -05:00
João Jandre 49cecaed06
Normalize loggers and upgrade log4j 1.2 to log4j 2.19 (#7131)
* Normalize logs

All classes that could have their loggers inherited from their fathers had their own loggers deleted;
Most loggers didn't have to be static, so most of them were normalized so that they wouldn't be;
All loggers are protected now;
Static logger's name are now 'LOGGER';
Non-static logger's name are now 'logger';
New class DbUpgradeAbstractImpl created so that all Upgraders extend it and inherit its logger

* Upgrade log4j

* fix errors caused by the merge

* Refactor cglibThrowableRenderer functionality to log4j2 and upgrade the last configuration files

* fix sonarcloud bug

* Fix errors caused by merge, remove some unused loggers, and rename a variable that was mistakenly renamed on the normalization commit

* Readd snmpTrapAppender, remove TestAppender

* Regenerate changes

* regenerate changes

* refactor last custom appender

* fix systemvm configuration xml

* Regenerate changes

* Regenerate changes

* regenerate changes

* Regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* Fix utils pom

* fix some tests

* regenerate changes

* Fix jar being printed on exception

* fix logging in system VMs, fix commands not having log4j2 classpath.

* regenerate changes

* Fix some unwanted renomeations

* fix end of file

* regenerate changes

* regenerate changes

* fix merge error

* regenerate changes

* fix tests

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* readd reload4j to tungsten as juniper depends on it

* Regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* re-add reload4j dependency to network-contrail, as juniper depends on it

* regenerate changes

* regenerate changes

* regenerate changes

* fix typo

* regenerate changes

* regenerate changes

* Fix end of files

* regenerate changes

* add logj42 to cloud-utils-SHADED.jar

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* Regenerate changes

* Regenerate changes

* Regenerate changes

* regenerate changes

* Regenerate changes

* regenerate changes

* Regenerate changes

* Regenerate changes

* Regenerate changes

* regenerate changes

* Regenerate changes

* Regenerate changes

* fix some tests

* Regenerate changes

* Regenerate changes

* fix test

* Regenerate changes

* Regenerate changes
2024-02-08 09:55:41 -03:00
Rohit Yadav 2bfa9de282 Merge remote-tracking branch 'origin/4.19' 2024-02-07 14:21:05 +05:30
Rohit Yadav 0d36098c76 Merge remote-tracking branch 'origin/4.18' into 4.19 2024-02-07 14:20:39 +05:30
Vishesh 399bd0a067
Upgrade to mockito 4 and handle Mockito deprecations (#8427) 2024-02-06 14:20:37 +01:00
Wei Zhou 69e8ebc03f
CKS: retry if unable to drain node or unable to upgrade k8s node (#8402)
* CKS: retry if unable to drain node or unable to upgrade k8s node

I tried CKS upgrade 16 times, 11 of 16 upgrades succeeded.

2 of 16 upgrades failed due to
```
error: unable to drain node "testcluster-of7974-node-18c8c33c2c3" due to error:[error when evicting pods/"cloud-controller-manager-5b8fc87665-5nwlh" -n "kube-system": Post "https://10.0.66.18:6443/api/v1/namespaces/kube-system/pods/cloud-controller-manager-5b8fc87665-5nwlh/eviction": unexpected EOF, error when evicting pods/"coredns-5d78c9869d-h5nkz" -n "kube-system": Post "https://10.0.66.18:6443/api/v1/namespaces/kube-system/pods/coredns-5d78c9869d-h5nkz/eviction": unexpected EOF], continuing command...
```

3 of 16 upgrades failed due to
```
Error from server: error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=roles", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=Role"
Name: "kubernetes-dashboard", Namespace: "kubernetes-dashboard"
from server for: "/mnt/k8sdisk//dashboard.yaml": etcdserver: leader changed
```

* CKS: remove tests of creating/deleting HA clusters as they are covered by the upgrade test

* Update PR 8402 as suggested

* test: remove CKS cluster if fail to create or verify
2024-02-06 11:14:10 +01:00
nvazquez d5efb869fd
Merge branch 'main' into nsx-integration 2024-02-03 17:37:25 -03:00
Abhishek Kumar 7dffbc6e47 Updating pom.xml version numbers for release 4.20.0.0-SNAPSHOT
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-02-02 18:16:37 +05:30
Abhishek Kumar a7b97ff3b0 Updating pom.xml version numbers for release 4.19.1.0-SNAPSHOT
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-02-02 18:06:04 +05:30
Abhishek Kumar 2746225b99 Updating pom.xml version numbers for release 4.19.0.0
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-01-29 10:21:52 +05:30
Pearl Dsilva 080f171c6d
NSX: Cleanup NSX resources during k8s cluster cleanup (#8528) 2024-01-19 12:48:08 -03:00
nvazquez 2b05dd93a1
Merge branch 'main' into nsx-integration 2024-01-12 14:13:53 -03:00
Abhishek Kumar 2253a33c1e Merge remote-tracking branch 'apache/4.18' 2023-12-20 08:58:30 +05:30
Wei Zhou ab70108f15
CKS: create Security Groups for CKS clusters of each account (#8316)
This PR fixes #7684

The security groups contain the same rules for port 22 and 6443, no need to recreate for each CKS cluster.
2023-12-20 08:57:27 +05:30
nvazquez 7814829a48
Merge branch 'main' into nsx-integration 2023-12-14 09:37:14 -03:00
Abhishek Kumar 080a5aee00 Merge remote-tracking branch 'apache/4.18' 2023-12-12 17:01:52 +05:30
Harikrishna 3ce7c39bef
cks: handle errors while scaling cluster (#8107)
This PR fixes the issue #7920
2023-12-12 16:57:28 +05:30
Abhishek Kumar 4bdf35b7b0 Merge remote-tracking branch 'apache/4.18' 2023-12-09 12:04:21 +05:30
Wei Zhou fc44df7c95
CKS: create HA cluster with 3 control VMs instead 2 (#8297)
This PR fixes the test failures with CKS HA-cluster upgrade.
In production, the CKS HA cluster should have at least 3 control VMs as well.
The etcd cluster requires 3 members to achieve reliable HA. The etcd daemon in control VMs uses RAFT protocol to determine the roles of nodes. During upgrade of CKS with HA, the etcd become unreliable if there are only 2 control VMs.
2023-12-09 11:33:05 +05:30
Sina Kashipazha 2993c99363
Add missing hosts info to the prometheus exporter output. (#8328)
Sometimes the hostStats object of the agents becomes null in the management server. It is a rare situation, and we haven't found the root cause yet, but it occurs occasionally in our CloudStack deployments with many hosts.

The hostStat is null, even though the agent is UP and hosting multiple VMs. It is possible to access the VM consoles and execute tasks on them.

This pull request doesn't address the issue directly; rather it displays those hosts in Prometheus so we can restart the agent and get the necessary information.
2023-12-08 19:51:06 +05:30
Pearl Dsilva 389dbe895a
NSX: Add CKS Support & Firewall rules for Isolated Networks (#8189)
* NSX: Add ALL LB IP to the list of route advertisements in tier1

* NSX: Support Source NAT on NSX Isolated networks

* NSX: Cks Support

* NSX: Create segment group on segment creation

* Add unit tests

* Remove group for segment before removing segment

* Create Distributed Firewall rules

* Remove distributed firewall policy on segment deletion

* Fix policy rule ID and add more unit tests

* Add support for routed NSX Isolated networks \n and non RFC 1918 compliant IPs

* Add support for routed NSX Isolated networks \n and non RFC 1918 compliant IPs

* Add Firewall rules

* build failure - fix unit test

* fix npes

* Add support to delete firewall rules

* update nsx cks offering

* add license

* update order of ports in PF & FW rules

* fix filter for getting transport zones

* CKS support changed - MTU updated, etc

* add LB for CKS on VPC

* address comments

* adapt upstream cks logic for vpc

* rever mtu hack

* update UI changes as per upstream fix

* change display test for CKS n/w offerings for isolated and VPC tiers

* add extra line for linter

* address comment

* revert list change

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>
2023-12-07 16:43:56 -03:00
João Jandre 26b01f6f3b
Flexible tags for hosts and storage pools (#7489)
Co-authored-by: João Jandre <joao@scclouds.com.br>
2023-11-30 09:36:47 +01:00
Daan Hoogland 98d643efe6 Merge release branch 4.18 to main
* 4.18:
  Fixed spelling and added missing states to response (#8248)
  Let Prometheus exporter plugin support utf8 characters (#8228)
2023-11-18 18:41:31 +01:00
dahn 1a2dbebe48
Let Prometheus exporter plugin support utf8 characters (#8228) 2023-11-15 09:48:11 +01:00
Daan Hoogland 05b9b6e2e7 Merge branch '4.18' into main 2023-11-13 11:36:51 +01:00
Abhishek Kumar d0f3233fda
edge-zone,kvm,iso,cks: allow k8s deployment with direct-download iso (#8142)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-11-10 13:56:05 +01:00
Wei Zhou 861107fa5b
CKS: make clustertype optional to keep backwards compatibility (#8180)
This PR fixes the issue that 4.18 cmk/api to create CKS cluster does not work in 4.19
2023-11-08 00:31:38 +05:30
Daan Hoogland 72cf9740f9 Merge branch '4.18' 2023-10-06 13:50:29 +02:00
Ben a20ab40b67
Ensure getCapacityState() is not called for hosts in maintenance (#8025) 2023-10-06 09:49:57 +02:00
John Bampton 4eb110af73
Remove unneeded duplicate words (#7850) 2023-09-18 13:16:33 +02:00
Wei Zhou 246bb24b0f Updating pom.xml version numbers for release 4.18.2.0-SNAPSHOT
Signed-off-by: Wei Zhou <weizhou@apache.org>
2023-09-12 17:26:53 +02:00
Wei Zhou 4bdff06acd Updating pom.xml version numbers for release 4.18.1.0
Signed-off-by: Wei Zhou <weizhou@apache.org>
2023-09-07 08:50:50 +02:00
Daan Hoogland 5559668f12 Merge branch '4.18' 2023-08-15 09:15:17 +02:00
Wei Zhou bf9c71430b
CKS: check access when get kubernetescluster config (#7854) 2023-08-14 10:02:20 +02:00
Rohit Yadav ac882f3d07 Merge remote-tracking branch 'origin/4.18' 2023-08-08 15:56:19 +05:30
Sina Kashipazha 9df23f951b
Prometheus exporter fix cpu/memory usage labels (#7629) 2023-08-07 20:47:03 +02:00
Vishesh 665fb83e73
Remove powermock from kubernetes plugin (#7788) 2023-07-28 13:23:26 +02:00
Vishesh 594c70dde0
Sync precommit config from main (#7732)
Co-authored-by: John Bampton <jbampton@users.noreply.github.com>
Co-authored-by: dahn <daan@onecht.net>
2023-07-07 11:18:16 +02:00
Vishesh 2fcbe6241f
cks: Add unmanaged kubernetes cluster (#7515)
There are tools like cluster-api which create and manage kubernetes cluster on CloudStack. This PR adds the option to add unmanaged kubernetes cluster which are not managed by CKS plugin. This helps provide a consolidated view of unmanaged clusters on CloudStack. The changes done make sure that operations for managed clusters are not executed for unmanaged clusters.

Two new APIs have also been added:

1. addVirtualMachinesToKubernetesCluster - to add VMs to unmanaged clusters.
2. removeVirtualMachinesFromKubernetesCluster - to remove VMs to unmanaged clusters.

Two APIs have been updated:

1. createKubernetesCluster - made KUBERNETES_VERSION_ID, SERVICE_OFFERING_ID, SIZE as not required for unmanaged clusters. Add an additional parameter, managed, which is true by default.
2. listKubernetesClusters - Add a parameter managed to filter on managed field.

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-07-03 13:07:33 +05:30
Abhishek Kumar 5fa9b8f683
cks: enable by default (#7509)
This change will allow CKS to be enabled by default on new installs. It would not affect server or performance but would help highlighting k8s support in CloudStack.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-12 14:11:02 +02:00
Abhishek Kumar f905f5f39a Merge remote-tracking branch 'apache/4.18' into main 2023-06-01 16:50:24 +05:30
Abhishek Kumar 7319debc87
cks: k8s cluster on vpc tier (#7479)
* cks: cluster on vpc tier

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wip

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* python fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix trailing space

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-01 15:29:06 +05:30
Abhishek Kumar f636580195
cks,ui: allow changing stopped cluster offering, improvements (#7475)
* cks,ui: allow changing stopped cluster offering, improvements

Fixes #7454

- Allows changing compute offering for a stopped cluster
- Allows compute offering change when the cluster has autoscaling enabled

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-29 15:58:06 +05:30
Abhishek Kumar 8ff0071bec
ui,api: make description option for createKubernetesCluster (#7541)
Make the description for createKubernetesCLuster API optional. Similar to #7180

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-25 15:22:58 +05:30
Ben 3e8c0684ed
Prometheus: Ensure tagged hosts maintenance status is reported consistently (#7471)
When a host is not tagged, its maintenance status is reported in the
cloudstack_hosts_total metric: maintenance_enabled is OFFLINE,
maintenance_disabled is ONLINE.

When a host is tagged, its maintenance status is now also verified to
ensure consistent behaviour.

In prometheus exporter, maintenance status for cloudstack_hosts_total_by_tag is not checked. While it is checked for cloudstack_hosts_total metric.
Classified by_tag or not, metrics should be the same.

Fixes: #7470
2023-05-23 11:14:43 +05:30
John Bampton c2e17310d6
Add three more `pre-commit` checks (#7083)
Co-authored-by: dahn <daan@onecht.net>
2023-03-27 13:28:55 +02:00
Daan Hoogland fb4f6a334d Updating pom.xml version numbers for release 4.19.0.0-SNAPSHOT
Signed-off-by: Daan Hoogland <daan@onecht.net>
2023-03-15 19:46:01 +01:00
Daan Hoogland 05cda2729f Updating pom.xml version numbers for release 4.18.1.0-SNAPSHOT
Signed-off-by: Daan Hoogland <daan@onecht.net>
2023-03-15 19:38:14 +01:00
Daan Hoogland 0574087284 Updating pom.xml version numbers for release 4.18.0.0
Signed-off-by: Daan Hoogland <daan@onecht.net>
2023-03-11 09:35:41 +01:00
David Jumani 2d914edff0
cks: Fix provider deployment when cluster is in a project (#7147) 2023-02-01 11:14:29 +01:00