Commit Graph

2057 Commits

Author SHA1 Message Date
Vishesh 8be18e587f
FR75 Enforce strict host tag checking (#421)
* Enforce strict host tag checking

* Add e2e tests

* Add more information to error log

* Fix e2e test

* Update global settings descrption

* fixup

* Fix e2e test teardown
2024-06-25 14:38:59 +05:30
Abhishek Kumar 8f88103a29
FR72 - api,server: purge expunged resources (#405)
This PR introduces the functionality of purging removed DB entries for CloudStack entities (currently only for VirtualMachine).
There would be three mechanisms for purging removed resources:
- Background task - CloudStack will run a background task which runs at a defined interval. Other parameters for this task can be controlled with new global settings.
- API - New API `purgeExpungedResources`. It will allow passing the following parameters - resourcetype, batchsize, startdate, enddate
- Config for service offering. Service offerings can be created with purgeresources parameter which would allow purging resources immediately on expunge.

Following new global settings have been added:
- `expunged.resources.purge.enabled`: Default: false. Whether to run a background task to purge the DB records of the expunged resources.
- `expunged.resources.purge.resources`: Default: (empty). A comma-separated list of resource types that will be considered by the background task to purge the DB records of the expunged resources. Currently only VirtualMachine is supported. An empty value will result in considering all resource types for purging.
- `expunged.resources.purge.interval`: Default: 86400. Interval (in seconds) for the background task to purge the DB records of the expunged resources.
- `expunged.resources.purge.delay`: Default: 300. Initial delay (in seconds) to start the background task to purge the DB records of the expunged resources task.
- `expunged.resources.purge.batch.size`: Default: 50. Batch size to be used during purging of the DB records of the expunged resources.
- `expunged.resources.purge.start.time`: Default: (empty). Start time to be used by the background task to purge the DB records of the expunged resources. Use format `yyyy-MM-dd` or `yyyy-MM-dd HH:mm:ss`.
- `expunged.resources.purge.keep.past.days`: Default: 30. The number of days in the past from the execution time of the background task to purge the DB records of the expunged resources for which the expunged resources must not be purged. To enable purging DB records of the expunged resource till the execution of the background task, set the value to zero.
- `expunged.resource.purge.job.delay`: Default: 180. Delay (in seconds) to execute the purging of the DB records of an expunged resource initiated by the configuration in the offering. Minimum value should be 180 seconds and if a lower value is set then the minimum value will be used.

Upstream PRs:
https://github.com/apache/cloudstack/pull/8999
https://github.com/apache/cloudstack-documentation/pull/397

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2024-06-19 12:59:50 +05:30
Vishesh c3eba5e213
Fix exceeding of resource limits with powerflex (#443)
* Fix exceeding of resource limits with powerflex

* Fix for volume prepare during VM start

* resolve comments

* Add e2e tests

* Fixup

* Update e2e tests

* minor refactoring

* refactoring

* fixup

---------

Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2024-05-08 20:54:54 +05:30
Marcus Sorensen f896586925
Update version to 4.18.1.1 (#417)
* Update version to 4.18.1.1

* Update changelog

* Update changelog

* Update changelog

---------

Co-authored-by: Marcus Sorensen <mls@apple.com>
2024-04-08 09:27:57 -06:00
Vishesh 47e53eceed
E2e test resource listing (#363)
* Use dualzones for ci github actions

* Update advdualzone.cfg to be similar to advanced.cfg & fixup test_metrics_api.py

* Add e2e tests for listing of accounts, disk_offerings, domains, hosts, service_offerings, storage_pools, volumes

* Add test for listing volumes with tags filter

* Add check for existing volumes in test_list_volumes

* Wait for volumes to be deleted on cleanup

* Filter out volumes in Destroy state before checking the count of volumes
2024-01-25 14:52:06 +05:30
Abhishek Kumar bf3dff2f57 marvin,test: fix directdownload template checksum test (#8096)
* marvin,test: fix directdownload template checksum test

During failure while deploying a VM with wrong checksum template, VM may be left in Error state. This PR adds code to delete such VM.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove unnecessary logs

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
(cherry picked from commit a2ec1f3777)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-10-26 12:21:02 +05:30
Wei Zhou 4bdff06acd Updating pom.xml version numbers for release 4.18.1.0
Signed-off-by: Wei Zhou <weizhou@apache.org>
2023-09-07 08:50:50 +02:00
Marcus Sorensen 6cded7e010
Support GNU sed apidoc for MacOS build (#7888)
Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-08-22 14:31:20 +05:30
Rohit Yadav 29c7b3167e
cloudstack: bump to use 4.18.1.0 systemvmtemplate (#7828)
* cloudstack: bump to use 4.18.1.0 systemvmtemplate

Use new 4.18.1.0 systemvmtemplate

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* fix cd ISO url

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

---------

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-08-11 07:38:47 +02:00
Wei Zhou b32ba953cf
test: upgrade kubernetes versions to 1.25.0/1.26.0 (#7685)
* test: upgrade versions to 1.25.0/1.26.0

* PR7685: move settings to test_data.py
2023-08-10 09:48:08 +02:00
Vishesh 594c70dde0
Sync precommit config from main (#7732)
Co-authored-by: John Bampton <jbampton@users.noreply.github.com>
Co-authored-by: dahn <daan@onecht.net>
2023-07-07 11:18:16 +02:00
Wei Zhou f3ad9e6743
marvin: fix smoke test failures if mgmt server is Ubuntu (#7583) 2023-06-27 11:57:02 +02:00
Abhishek Kumar 03910c2783 Merge remote-tracking branch 'apache/4.17' into 4.18 2023-06-08 15:34:03 +05:30
Abhishek Kumar 20865901d1 Merge remote-tracking branch 'apache/4.16' into 4.17 2023-06-08 15:33:20 +05:30
Abhishek Kumar 24da2bcf78
marvin: fix test_data for parsing failure (#7602)
PR #6512 seems to have broken the test_data.py. This PR fixes test_data.
2023-06-08 15:32:04 +05:30
Abhishek Kumar b030ea19a4 Merge remote-tracking branch 'apache/4.17' into 4.18 2023-06-06 13:12:12 +05:30
Abhishek Kumar f298008215 Merge remote-tracking branch 'apache/4.16' into 4.17 2023-06-06 13:11:24 +05:30
Pearl Dsilva bd5d478a3a
Refactor test and change IP range (#6512)
* Refactor test and change IP range

* countdown from 254 to allow multiple pseudo public nets counting up from 0

* cleanup

* location of asserts improved

---------

Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: Daan Hoogland <daan@onecht.net>
2023-06-05 16:39:33 +02:00
Rohit Yadav 4138ca7c5d systemvmtemplate: fix Debian 11.7.0 ISO URL
Fixes systemvmtemplate build issue

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-05-25 22:15:28 +05:30
Vishesh 83fd75f707
Marvin: Fix deployDataCenter's cleanup (#7521) 2023-05-12 10:46:52 +02:00
Wei Zhou ea2b537985
fix tools/build/setnextversion.sh: do not update tools/apidoc/pom.xml (#7163)
Steps to reproduce the issue

(1) on ubuntu server, run "git checkout 4.17.2.0"
(2) packaging
cd packaging/
./build-deb.sh -b test1

expected result: succeed
actual result:

[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR]   The project org.apache.cloudstack:cloud-apidoc:4.17.2.0-test1-test1 (/data/cloudstack/tools/apidoc/pom.xml) has 1 error
[ERROR]     Non-resolvable parent POM for org.apache.cloudstack:cloud-apidoc:4.17.2.0-test1-test1: Could not find artifact org.apache.cloudstack:cloud-tools:pom:4.17.2.0-test1-test1 in central (https://repo.maven.apache.org/maven2) and 'parent.relativePath' points at wrong local POM @ line 25, column 13 -> [Help 2]
2023-05-08 13:00:36 +02:00
dahn 897dad619b
marvin: replace encodestring for encodebytes (#7027)
Fixes: #6962
2023-05-08 12:51:12 +05:30
Rohit Yadav 74a414e76f Fix build failure and 4.18.0.0->4.18.1.0 upgrade path
Unblocks smoketests/health check runs on 4.18 branch.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-03-16 13:18:26 +05:30
Daan Hoogland 05cda2729f Updating pom.xml version numbers for release 4.18.1.0-SNAPSHOT
Signed-off-by: Daan Hoogland <daan@onecht.net>
2023-03-15 19:38:14 +01:00
Daan Hoogland 0574087284 Updating pom.xml version numbers for release 4.18.0.0
Signed-off-by: Daan Hoogland <daan@onecht.net>
2023-03-11 09:35:41 +01:00
dahn 597a8030b7
integration test fixes for 4.18 (#7190)
* cleanup in resource housekeeping

* cleanup in resource housekeeping

* Update test/integration/smoke/test_deploy_vm_extra_config_data.py

Co-authored-by: dahn <daan.hoogland@gmail.com>

* revert most cleanup code

* Revert "cleanup in resource housekeeping"

This reverts commit 1d140bd215.

* add extra config to VirtualMachine.create(..)

---------

Co-authored-by: Wei Zhou <weizhou@apache.org>
2023-02-14 08:20:11 +01:00
kiranchavala 23d89813b7
Added steps to get the offical docker image for cloudstack simulator form Apache (#7192) 2023-02-10 11:32:17 +01:00
Nicolas Vazquez c44ba8ff3c
UI: Integrate Tungsten Fabric (#7166)
Co-authored-by: utchoang <hoangnm@unitech.vn>
Co-authored-by: davidjumani <dj.davidjumani1994@gmail.com>
Co-authored-by: dahn <daan@onecht.net>
2023-02-10 11:00:42 +01:00
Wei Zhou 62e342c1bc
utils,framework/db: Introduce new database encryption cipher based on AesGcmJce (#7003) 2023-02-02 16:25:49 +01:00
David Jumani c774b865c9
Tungsten integration (#7065)
Co-authored-by: rtodirica <rtodirica@ena.com>
Co-authored-by: Huy Le <huylm@unitech.vn>
Co-authored-by: radu-todirica <Radu.Todirica@ness.com>
Co-authored-by: Huy Le <minh.le@ext.ewerk.com>
Co-authored-by: Simon Weller <siweller77@gmail.com>
Co-authored-by: dahn <daan@onecht.net>
2023-02-01 09:19:53 +01:00
Suresh Kumar Anaparti d8c7e34b38
Improve global settings UI to be more intuitive/logical (#5797)
Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
Co-authored-by: davidjumani <dj.davidjumani1994@gmail.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: dahn <daan@onecht.net>
2023-01-31 11:23:43 +01:00
Daan Hoogland a7d2d8d750 Merge release branch 4.17 to main
* 4.17:
  marvin: newer python setuptools doesn't like -SNAPSHOT in marvin version (#7120)
  VR: fix warning Expected X answers while executing SetXXXCommand but Y (#7050)
2023-01-24 16:08:32 +01:00
Rohit Yadav abef381aa6
marvin: newer python setuptools doesn't like -SNAPSHOT in marvin version (#7120) 2023-01-24 15:28:07 +01:00
Rohit Yadav e7b7efe117
marvin: install mysql-connector-python version 8.0.31 (#7111) 2023-01-24 09:54:02 +01:00
John Bampton d74f64a2e1
Use lowercase HTTP header field names so we are compatible with HTTP/2 (#7006) 2023-01-23 11:17:54 +01:00
Wei Zhou 4418574478
simulator: use ubuntu 22.04 in Dockerfile (#6978) 2023-01-12 12:54:36 +01:00
dahn df96af3de4
delete F5 and SRX plugins (#7023) 2023-01-11 12:07:44 +01:00
slavkap b392084950
Enable live volume migration for StorPool and small fixes (#6661) 2023-01-10 18:21:39 +01:00
dahn 6790371fe2
fix merge diff display (#7047) 2023-01-09 11:43:32 +01:00
Sina Kashipazha 20306d6129
Allow creating atmost 1 physical network with null tag (#6781) 2023-01-05 21:36:31 +01:00
Pearl Dsilva 3044d63a8b
Configurable MTU for VR (#6426)
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-01-04 09:42:24 +01:00
Rohit Yadav d52e4d72b6
systemvmtemplate: update Debian ISO URL and checksum (#7013)
This fixes packer systemvmtemplate build

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2022-12-23 11:26:38 +05:30
John Bampton d0902893b8
Fix spelling (#7002)
Co-authored-by: Stephan Krug <stekrug@icloud.com>
2022-12-22 11:47:35 +01:00
Rohit Yadav 2dc9f1e32c
.github: Migrate Travis to Github Actions and Simulator based CI (#6956)
Runs on default spec:
2-core CPU (x86_64)
7 GB of RAM
14 GB of SSD space

https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2022-12-21 16:36:52 +05:30
Rohit Yadav 458883575a Updating pom.xml version numbers for release 4.17.3.0-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2022-12-16 15:25:16 +00:00
Rohit Yadav 5b9a989ab0 Updating pom.xml version numbers for release 4.17.2.0
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2022-12-14 05:22:52 +00:00
John Bampton def7ce655d
Fix spelling (#6898)
Co-authored-by: davidjumani <dj.davidjumani1994@gmail.com>
2022-12-13 14:58:14 +01:00
dahn 3106dba28c
git-pr py3 compatible (#6961)
* py3 print statement

* python3
2022-12-13 15:45:13 +05:30
Rohit Yadav 01b79e797c Merge remote-tracking branch 'origin/4.17' into main
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2022-12-12 14:16:31 +00:00
Abhishek Kumar 5516f74fce
systemvm: fix management route (#6921)
Fixes #6873
When the management server is in a different subnet than the pod in which a VPC VR is deployed, an ip route must be added just after creation of eth0 to allow the connectivity between the VR and the management server.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2022-12-12 17:41:54 +05:30