Commit Graph

4657 Commits

Author SHA1 Message Date
Wido den Hollander c3f0d14d31
storage/object: Add support for Ceph RGW Object Store (#8389)
This feature adds support for Ceph's RADOS Gateway (RGW) support for the
Object Store feature of CloudStack.

The RGW of Ceph is Amazon S3 compliant and is therefor an easy and straigforward
implementation of basic S3 features.

Existing Ceph environments can have the RGW added as an additional feature to a
cluster already providing RBD (Block Device) to a CloudStack environment.

Introduce the BucketTO to pass to the drivers. This replaces just passing the bucket's name.

Some upcoming drivers require more information then just the bucket name to perform their actions,
for example they require the access and secret key which belong to the account of this bucket.

This is leftover code from a long time ago and this validation test has nu influence
on the end result on how a URL will be used afterwards.

We should support hosts pointing to an IPv6(-only) address out of the box.

For the code it does not matter if it's IPv4 or IPv6. This is the admin's choice.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-09-05 17:58:39 +05:30
Abhisar Sinha 605534b417
feature: Shared Storage Filesystem as a First Class Feature (#9208)
This PR implements Storage filesystem as a first class feature.
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Filesystem+as+a+First+Class+Feature

Documentation PR: apache/cloudstack-documentation#420

Co-authored-by: Wei Zhou <weizhou@apache.org>
2024-09-05 17:22:32 +05:30
Vishesh 72d0546d8b
Shared Network Firewall (Security groups) in Advanced zone without security groups (#9415) 2024-09-05 10:35:05 +02:00
João Jandre 6a559f46e3
Go back to default timestamp on logging (#8925) 2024-09-05 09:13:04 +02:00
Abhishek Kumar 7e085d5e1d
framework/db: use HikariCP as default and improvements (#9518)
Per docs, if the mysql connector is JDBC2 compliant then it should use
the Connection.isValid API to test a connection.
(https://docs.oracle.com/javase/8/docs/api/java/sql/Connection.html#isValid-int-)

This would significantly reduce query lags and API throughput, as for
every SQL query one or two SELECT 1 are performed everytime a Connection
is given to application logic.

This should only be accepted when the driver is JDBC4 complaint.

As per the docs, the connector-j can use /* ping */ before calling
SELECT 1 to have light weight application pings to the server:
https://dev.mysql.com/doc/connector-j/en/connector-j-usagenotes-j2ee-concepts-connection-pooling.html

Replaces dbcp2 connection pool library with more performant HikariCP.
With this unit tests are failing but build is passing.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Rohit Yadav <rohityadav89@gmail.com>
2024-09-05 09:36:58 +05:30
Rohit Yadav 60c9c9adb6 Merge remote-tracking branch 'origin/4.19'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-09-04 13:20:42 +05:30
Rohit Yadav a851ee375c Merge remote-tracking branch 'origin/4.18' into 4.19
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-09-04 12:30:00 +05:30
Wido den Hollander 39152323e3
kvm: Present the UUID of the VM as serial through smbios information (#9236)
This PR makes sure a KVM VM gets the UUID of the VM as a static serialnumber through smbios.

Some applications on primarily Windows servers require a stable serial number for licensing purposes. By providing this serial number we can make sure these applications can have a license configured.

More information: https://libvirt.org/formatdomain.html#smbios-system-information
2024-09-04 12:14:17 +05:30
Abhishek Kumar 0692a296ce
engine-orchestration: fix issue for empty product in vm metadata (#9610)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-09-03 14:03:57 +02:00
Rene Peinthor 537c0a1e8d
linstor: set/unset allow-two-primaries and protocol on rc level (#9560) 2024-09-03 16:31:07 +05:30
slavkap 12d9c26747
Added support for storpool_qos service (#8755) 2024-08-29 09:23:25 +02:00
Abhishek Kumar 5a496e725b Merge remote-tracking branch 'apache/4.19' 2024-08-28 16:07:31 +05:30
Suresh Kumar Anaparti c05edc40f4
Updated invalid parameter/value to com.cloud.exception.InvalidParameterValueException (#9455) 2024-08-28 08:54:43 +02:00
Wei Zhou d7ca05e5ba
Build: drop EL7 support, support JRE17 for packages and sonar check (#8609)
This PR fixes the issue with sonar check

```
Error:  Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.9.1.2184:sonar (default-cli) on project cloudstack:
Error:
Error:  The version of Java (11.0.22) used to run this analysis is deprecated, and SonarCloud no longer supports it. Please upgrade to Java 17 or later.
Error:  You can find more information here: https://docs.sonarsource.com/sonarcloud/appendices/scanner-environment/
```

main changes
- Support build/packaging using JDK17
- Still supports JDK11 for building
- Support JRE17 for use in production installation
- Drop EL7 support

The community packages will be still packaged using JDK11. 
If uses want, they can build by JDK17 as well.

Signed-off-by: Wei Zhou <wei.zhou@shapeblue.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-08-27 16:09:10 +05:30
Suresh Kumar Anaparti 6c0492366c
[VMware] Disconnect/Detach config drive ISO (if exists) on stop VM (#9468) 2024-08-27 11:11:01 +02:00
Wei Zhou b61c3b8102
add domainpath to WebhookResponse 2024-08-26 18:10:32 +02:00
Wei Zhou f5efe56fb5
Merge remote-tracking branch 'apache/4.19' 2024-08-26 17:34:15 +02:00
Harikrishna de3537c98a
Added domain path to all entities (#9540) 2024-08-26 11:04:00 +02:00
Abhishek Kumar 1e12a80210
orchestration,hypervisor: allow custom manufacture, product for vm (#9163)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-08-22 17:19:04 +02:00
Wei Zhou b215abc30a
server: do not check affinity groups if no vm group mappings (#9542)
this fixes the exception in smoke test test_affinity_groups

```
2024-08-19T08:34:15,132 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-87:[ctx-f7804a8e, job-9232]) (logid:b71ddec8) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.DeployVMCmdByAdmin com.cloud.utils.exception.CloudRuntimeException: Unable to find on DB, due to: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')  FOR UPDATE' at line 1
    at com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:441)
    at com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:368)
    at com.cloud.utils.db.GenericDaoBase.search(GenericDaoBase.java:357)
    at com.cloud.utils.db.GenericDaoBase.lockRows(GenericDaoBase.java:343)
    at org.apache.cloudstack.affinity.dao.AffinityGroupDaoImpl.listByIds(AffinityGroupDaoImpl.java:171)
```
2024-08-21 10:27:33 +02:00
Nicolas Vazquez 296a44e932
Fix main branch issues (#9553) 2024-08-20 15:26:41 +02:00
Gabriel Pordeus Santos f84e04372c
Fix being able to expunge a VM through destroyVirtualMachine even when role rule does not allow (#8689) 2024-08-20 12:02:04 +02:00
Daan Hoogland 5bf81cf002 Merge release branch 4.19 to main
* 4.19:
  linstor: Improve copyPhysicalDisk performance (#9417)
2024-08-20 11:20:39 +02:00
Daan Hoogland 3d8d4875fe Merge release branch 4.18 to 4.19
* 4.18:
  linstor: Improve copyPhysicalDisk performance (#9417)
2024-08-20 11:19:44 +02:00
Rene Peinthor 27f23f4f75
linstor: Improve copyPhysicalDisk performance (#9417)
Tell qemu-img that we don't want to use a write cache (we are a block device)
and also specify that we have zeroed devices in most cases.
2024-08-20 11:17:57 +02:00
Bernardo De Marco Gonçalves 01c721fcda
Improvements to quota tariffs APIs and UI (#9225)
* reface quotaTariffList process and add listOnlyRemoved parameter

* add unit tests for createQuotaTariffResponse and isUserAllowedToSeeActivationRules methods

* update QuotaTariffListCmdTest

* refactor quota tariffs creation

* refactor quota tariffs update

* fix unit test in JsInterpreter

* remove unused import

* refactor quota listing and add quota deletion

* add functionality to create tariff from UI, not working when specifying dates

* fix date parsing

* add labels

* fix details view of tariffs

* new update tariff view

* fix filter placeholder

* remove debug html

* add labels

* make value field to be required when updating a tariff

* add labels

* add portuguese labels

* remove unused label

* fix updating tariff when there was no enddate specified

* refactor dates

* refactor dates

* clear code

* update disabled dates in date picker

* clear ListView component

* fix unnecessary updates when the new end date was equal to the exising end date

* fix when today was selected to start date

* add keyword to filter

* change usage type response

* add keyword and usagetype filter on UI

* fix disabled end dates in date picker

* modify datepickers to use datetime

* small fixes

* make value an unrequired field on update form

* remove duplicate import

* remove unused css classes

* add UI support for position parameter

* resize input fields to fill all available horizontal space

* remove console.log()

* remove unnecessary fully qualified names

* replace `usagetypeid` property name to `id` on `listUsageTypes` API call

* replace `usagetypeid` property name to `id` on `listUsageTypes` API call
2024-08-15 14:16:44 -03:00
Daan Hoogland 287a47266b Merge branch '4.19' 2024-08-14 12:33:47 +02:00
Vishesh c98f1b8b3c
Ensure affinity groups are honored when VMs are deployed in parallel (#9201) 2024-08-12 14:02:16 +02:00
Rohit Yadav cc1dcf51b7 Merge remote-tracking branch 'origin/4.19'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-08-08 11:53:55 +05:30
Rene Peinthor 21f3fde7b4
libvirtstorageadaptor: better handle failed libvirt storagepool destroy (#9390)
If the libvirt mount point is still busy and can't be unmounted
right now, it was waited 5 seconds and an plain unmount was tried,
without cleaning up the libvirt storagepool.
This kept libvirt thinking the storagepool
is active and mounted (which it wasn't).

Now after the plain unmount call, also
the libvirt storagepool will be destroyed.
2024-08-07 09:53:35 +02:00
nvazquez b73f634ea6
Merge branch '4.19' 2024-08-06 12:39:13 -03:00
Vishesh 0be87ac654
Fixup CKS UI for external managed clusters (#9434) 2024-07-29 10:48:58 +05:30
Abhishek Kumar e41add31e7 saml: signature check improvements
Adminstrators should ensure that IDP configuration has a signing certificate for the actual signature check to be performed. In addition to this, this change introduces a new global setting saml2.check.signature, with the default value of true, which can deliberately fail a SAML login attempt when the SAML response has a missing signature.
Purges the SAML token upon handling the first SAML response.

Authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-07-19 14:24:43 +05:30
Suresh Kumar Anaparti 3faf7cd2f1
Updating pom.xml version numbers for release 4.19.2.0-SNAPSHOT
Signed-off-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2024-07-19 10:29:26 +05:30
Vishesh 35fd17c62e
Merge branch '4.19' 2024-07-17 17:43:40 +05:30
Suresh Kumar Anaparti 9f4c895974
Updating pom.xml version numbers for release 4.19.1.0
Signed-off-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2024-07-15 17:19:29 +05:30
Rohit Yadav 2cfb541a1d
saml: purge token after first response and improve setting description (#9377)
* saml: purge token after first response and improve setting description

This improves the description of a saml signature checking global
setting, and purges the SAML token upon handling the first SAML
response.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* fix failing unit test

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

---------

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-07-15 09:45:28 +05:30
Suresh Kumar Anaparti 7f8f113709
Merge branch '4.18' into 4.19 2024-07-15 00:59:06 +05:30
Suresh Kumar Anaparti 877c4d9a64
LibvirtOvsFetchInterfaceCommandWrapperTest fix (test fails in mac) - skip it if no interfaces with eth and wl (#9382)
* test fix (fails in mac) - skip it if no interfaces with eth and wl

* review comments
2024-07-15 00:56:23 +05:30
João Jandre b9c7275c25
Quota tariff order (#8347) 2024-07-14 16:05:15 +02:00
Felipe a87778be9a
Refactor KubernetesClusterResourceModifierActionWorker (#8801)
Co-authored-by: dahn <daan.hoogland@gmail.com>
2024-07-14 16:02:04 +02:00
John Bampton c923e673cf
pre-commit: add `XML` files to the `trailing-whitespace` check (#9131) 2024-07-12 09:42:54 +02:00
John Bampton 806692bcea
pre-commit: enable Python flake8 rule `E272` (#9361)
There should be only one space before a keyword.
2024-07-10 09:17:25 +02:00
Vishesh 0af88353fb
Merge branch '4.19' 2024-07-09 10:40:21 +05:30
Rohit Yadav 7977d1475e
plugins: make default signature check mandatory (#9357)
This improves upon #9219, to make the signature checks mandatory by
default but allows for users to relax the setting if they really must.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-07-09 09:49:27 +05:30
John Bampton b69cc0272d
pre-commit: enable Python Flake8 rule `E713` (#9016)
Co-authored-by: Vishesh <vishesh92@gmail.com>
2024-07-08 09:00:18 +02:00
Abhishek Kumar 3c9bc07d06 Merge remote-tracking branch 'apache/4.19' 2024-07-06 13:01:50 +05:30
Abhishek Kumar 2cf838cc1d framework/cluster: improve cluster service, integration API server
- mTLS implementation for cluster service communication
- Listen only on the specified cluster node IP address instead of all interfaces
- Validate incoming cluster service requests are from peer management servers based on the server's certificate dns name which can be through global config - ca.framework.cert.management.custom.san
- Hardening of KVM command wrapper script exeicution
- Improve API server integration port check
- cloudstack-management.default: don't have JMX configuration if not needed. JMX is used for instrumentation; users who need to use it should enable it explicitly

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Wei Zhou <weizhou@apache.org>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-07-06 11:29:26 +05:30
Abhishek Kumar beebeed5e2 framework/cluster: improve cluster service and integration API service
- mTLS implementation for cluster service communication
- Listen only on the specified cluster node IP address instead of all interfaces
- Validate incoming cluster service requests are from peer management servers based on the server's certificate dns name which can be through global config - ca.framework.cert.management.custom.san
- Hardening of KVM command wrapper script execution
- Improve API server integration port check
- cloudstack-management.default: don't have JMX configuration if not needed. JMX is used for instrumentation; users who need to use it should enable it explicitly

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Wei Zhou <weizhou@apache.org>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-07-05 19:05:06 +05:30
Vishesh ba85b4d4ff
Merge release branch 4.18 to 4.19
* 4.18:
  linstor: deleteAsync fallback to volume UUID if path not set yet (#9325)
  linstor: do not use the same static ApiClient for all calls (#9326)
2024-07-05 12:45:27 +05:30