Commit Graph

104 Commits

Author SHA1 Message Date
PeterBackroll cfad83413d
Merge branch 'apache:main' into Backroll 2025-02-21 09:33:56 +01:00
Abhisar Sinha a7beaaf73b
Add Resource Limits to Backups and Object Storage (#10017)
Doc PR : https://github.com/apache/cloudstack-documentation/pull/461
This PR fixes https://github.com/apache/cloudstack/issues/8638

== Description

Four new Resource Types have been added. Admin can configure corresponding resource limits for the tenants at different levels (domain, account, project) 
User dashboard's Storage section will show the new resources, their limits and current usage.

1. backup - No. of backups used by the account
2. backup_storage - Backup storage allocated for the account
3. bucket - No. of buckets used by the accounts
4. object_storage - Object storage allocated for the account.

Some other related changes done to BnR framework:

1. Maximum number of Backups to retain can be specified while creating Backup schedules, similar to Scheduled snapshots.
2. Oldest Scheduled backup of the same interval type will be deleted once the number reaches the configured max Backups value.
3. Code refactor: Moved syncBackups method from BackupProvider to the framework BackupManagerImpl, as it is a common functionality and all providers were using duplicated code.

Changes done to the Object Storage Framework

1. Quota parameter is made mandatory while creating a bucket. Bucket quota is considered to be the allocated space and will be used to enforce Resource limits.

== Schema Changes:

1. New Column `max_backups` added to `backup_schedule` table
4. New Column `backup_interval_type` added to `backups` table

== Api Changes:

1. createBackup: new Parameter `scheduleid`. It should be specified whenever a scheduled backup is created. This will translate to the `backup_interval_type` in the `backups` table.
3. createBackupScheduke: new Parameter `max_backups`. To specify maximum number of backups to retain for the given schedule.

== Configurations:

|Setting |Scope |Default Value |Description|
|-------|--------|--------------|-----------|
|backup.max.hourly |Global |8 |Maximum recurring hourly backups to be retained for an instance|
|backup.max.daily |Global |8 |Maximum recurring daily backups to be retained for an instance|
|backup.max.weekly |Global |8 |Maximum recurring weekly backups to be retained for an instance|
|backup.max.monthly |Global |8 |Maximum recurring monthly backups to be retained for an instance|
|max.account.backups| Global| 20 | The default maximum number of backups that can be created for an account|
|max.account.backup.storage| Global| 400 | The default maximum backup storage space (in GiB) that can be used for an account|
|max.domain.backups| Global| 40 | The default maximum number of backups that can be created for an domain|
|max.domain.backup.storage| Global| 800 | The default maximum backup storage space (in GiB) that can be used for an domain|
|max.project.backups| Global| 20 | The default maximum number of backups that can be created for an project|
|max.project.backup.storage| Global| 400 | The default maximum backup storage space (in GiB) that can be used for an project|

|Setting |Scope |Default Value |Description|
|-------|--------|--------------|-----------|
|max.account.buckets| Global| 20 | The default maximum number of buckets that can be created for an account|
|max.account.object.storage| Global| 400 | The default maximum object storage space (in GiB) that can be used for an account|
|max.domain.buckets| Global| 40 | The default maximum number of buckets that can be created for an domain|
|max.domain.object.storage| Global| 800 | The default maximum object storage space (in GiB) that can be used for an domain|
|max.project.buckets| Global| 20 | The default maximum number of buckets that can be created for an project|
|max.project.object.storage| Global| 400 | The default maximum object storage space (in GiB) that can be used for an project|


Co-authored-by: Daan Hoogland <daan@onecht.net>
Co-authored-by: Lucas Martins <56271185+lucas-a-martins@users.noreply.github.com>
Co-authored-by: Lucas Martins <lucas.martins@scclouds.com.br>
Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2025-02-07 16:56:20 +05:30
Pierre Charton 010728ead8 End-of-file 2025-01-10 17:23:53 +01:00
Matthias 9e33b091ae
Merge branch 'apache:main' into Backroll 2025-01-10 10:57:09 +01:00
Pierre Charton 0dfeccbbdc Remove unsed import from test file 2025-01-10 10:34:39 +01:00
Daan Hoogland fadb39ece7 Merge release branch 4.20 to main
* 4.20:
  merge errors fixed
  Restrict the migration of volumes attached to VMs in Starting state (#9725)
  server, plugin: enhance storage stats for IOPS (#10034)
  Introducing granular command timeouts global setting (#9659)
  Improve logging to include more identifiable information (#9873)
2025-01-08 14:01:19 +01:00
Vishesh a4224e58cc
Improve logging to include more identifiable information (#9873)
* Improve logging to include more identifiable information for kvm plugin

* Update logging for scaleio plugin

* Improve logging to include more identifiable information for default volume storage plugin

* Improve logging to include more identifiable information for agent managers

* Improve logging to include more identifiable information for Listeners

* Replace ids with objects or uuids


* Improve logging to include more identifiable information for engine

* Improve logging to include more identifiable information for server

* Fixups in engine

* Improve logging to include more identifiable information for plugins

* Improve logging to include more identifiable information for Cmd classes

* Fix toString method for StorageFilterTO.java
2025-01-06 16:42:37 +05:30
PeterBackroll fabd4aeea1
Update plugins/backup/backroll/pom.xml
Co-authored-by: dahn <daan.hoogland@gmail.com>
2024-12-23 11:42:36 +01:00
Pierre Charton 0a4c33ddb3 UT 2024-12-21 00:02:59 +01:00
Pierre Charton 81d9669603 UT + unnessary login remove 2024-12-20 23:44:44 +01:00
Pierre Charton a2dcaf251c Error unused import 2024-12-20 21:33:03 +01:00
Pierre Charton eb46332fe0 UT + refactoring to test 2024-12-20 17:40:06 +01:00
Pierre Charton a4d8809ad6 UT on BackrollBackUpProvider 2024-12-20 12:33:00 +01:00
Pierre Charton 0ea2ce5b52 UT + correction bug 2024-12-20 11:03:33 +01:00
Pierre Charton eb400da8ad Merge branch 'Backroll' into Backroll-dev 2024-12-16 17:02:11 +01:00
Pierre Charton ab25490807 Merge with Backroll + bug backup 2024-12-16 16:55:51 +01:00
Pierre Charton 4b05c8c523 Lint fix 2024-12-16 10:29:09 +01:00
Pierre Charton 31c3be44bc Lint fix 2024-12-16 09:32:28 +01:00
Pierre Charton 78878587e5 Merge branch 'Backroll' into Backroll-dev 2024-12-13 15:58:40 +01:00
Pierre Charton cbe491a37d Unit Test 2024-12-13 15:57:50 +01:00
Pierre Charton abc719197c Merge branch 'Backroll' into Backroll-dev + Test unitaite 2024-12-13 15:57:25 +01:00
Pierre Charton fe180a1bb6 fix license 2024-12-13 10:44:05 +01:00
Pierre Charton 112ea11d61 Fix lint errors and log 2024-12-13 10:35:48 +01:00
Pierre Charton 6ed21e2770 Fix Version Pom 2024-12-13 09:38:45 +01:00
Pierre Charton e05c6f7e6a Merge branch 'Backroll-dev' of https://github.com/DIMSI-IS/cloudstack into Backroll-dev 2024-12-13 09:35:55 +01:00
Pierre Charton 3543554371 Sync Backroll dev before merge backroll 2024-12-13 09:35:37 +01:00
Pierre Charton da787f0a2c Merge branch 'Backroll' of https://github.com/DIMSI-IS/cloudstack into Backroll 2024-12-13 09:34:26 +01:00
Pierre Charton 5729e84dd0 General refactoring 2024-12-13 09:33:48 +01:00
PeterBackroll 61cafe2f9c
Merge branch 'main' into Backroll 2024-12-13 08:37:04 +01:00
João Jandre d9774a8462 Updating pom.xml version numbers for release 4.21.0.0-SNAPSHOT
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-11-27 11:47:06 -03:00
João Jandre c63c7ee63e Updating pom.xml version numbers for release 4.20.1.0-SNAPSHOT
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-11-27 11:40:45 -03:00
João Jandre 2fe3fcef7c Updating pom.xml version numbers for release 4.20.0.0
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-11-19 08:54:07 -03:00
Pierre Charton bd0bc0f39d Don't delete backup when we dissociate backup offering 2024-11-15 11:13:26 +01:00
Pierre Charton 655080678d Bug on list of vm on the instances 2024-11-07 10:09:57 +01:00
Pierre Charton 82b503e9f6 Fix deserialization + fix connection stability 2024-11-05 21:09:50 +01:00
Pierre Charton 3e90f8e460 WIP 2024-10-28 20:38:20 +01:00
Pierre Charton 6a690b7431 Revert "WIP error handling"
This reverts commit 01c2238e45.
2024-10-21 17:33:48 +02:00
Pierre Charton e9cbe0ed6a Revert "Refactoring error handling"
This reverts commit d37a35b489.
2024-10-21 17:33:21 +02:00
Matthias Dhellin d37a35b489 Refactoring error handling 2024-10-15 22:53:13 +02:00
Matthias Dhellin 01c2238e45 WIP error handling 2024-10-15 18:52:29 +02:00
Matthias Dhellin 8e3c8f7be5 Naming 2024-10-07 18:02:56 +02:00
Matthias Dhellin 561b9b6a1f Refactored some null checks 2024-10-07 17:32:12 +02:00
Matthias Dhellin e4b212f30e Extracted string constants 2024-10-07 17:17:17 +02:00
Matthias Dhellin 63bdc0ffa2 Factorized duplicated code 2024-10-07 16:57:27 +02:00
Matthias Dhellin 50a993c486 Factorized duplicated code 2024-10-07 12:25:42 +02:00
Matthias Dhellin 2d580ff462 Checked license 2024-10-01 23:49:37 +02:00
Matthias Dhellin 9c83c2bf1b Reorganized imports 2024-10-01 19:38:31 +02:00
Matthias Dhellin d21cd1273e Reorganized imports 2024-10-01 19:28:53 +02:00
Pierre Charton bf78e4ccf2 fix: spelling 2024-10-01 19:01:09 +02:00
Pierre Charton 56ab08f85a Fix pom and restoreBackedUpVolume 2024-09-11 18:01:05 +02:00