Commit Graph

12038 Commits

Author SHA1 Message Date
Vishesh dc74d5ba88
Let network guru decide if ipv6 cidr size can't be equal to 64 (#9289) 2024-06-26 02:43:26 +05:30
Vishesh 2bf36ef935
DRS: Ensure the destination host is part of the same cluster (#9245)
* DRS: Ensure the destination host is part of the same cluster

* Add and fix unit tests
2024-06-26 02:09:26 +05:30
Abhisar Sinha 4eb43651e2
Ability to specify NFS mount options while adding a primary storage and modify them on a pre-existing primary storage (#8947)
* Ability to specify NFS mount options while adding a primary storage and modify it later

* Pull 8947: Rename all occurrence of nfsopt to nfsMountOpt and added nfsMountOpts to ApiConstants

* Pull 8947: Refactor code - move into separate methods

* Pull 8947: CollectionsUtils.isNotEmpty and switch statement in LibvirtStoragePoolDef.java

* Pull 8947: UI - cancel maintainenace will remount the storage pool and apply the options

* Pull 8947: UI - moved edit NFS mount options to edit Primary Storage form

* Pull 8947: UI - moved 'NFS Mount Options' to below 'Type' in dataview

* Pull 8947: Fixed message in AddPrimaryStorage.vue

* Pull 8947: Convert _nfsmountOpts to Set in libvirtStoragePoolDef

* Pull 8947: Throw exception and log error if mount fails due to incorrect mount option

* Pull 8947: Added UT and moved integration test to component/maint

* Pull 8947: Review comments

* Pull 8947: Removed password from integration test

* Pull 8947: move details allocation to inside the if loop in getStoragePoolNFSMountOpts

* Pull 8947: Fixed a bug in AddPrimaryStorage.vue

* Pull 8947: Pool should remain in maintenance mode if mount fails

* Pull 8947: Removed password from integration test

* Pull 8947: Added UT

* Pull 8875: Fixed a bug in CloudStackPrimaryDataStoreLifeCycleImplTest

* Pull 8875: Fixed a bug in LibvirtStoragePoolDefTest

* Pull 8947: minor code restructuring

* Pull 8947 : added some ut for coverage

* Fix LibvirtStorageAdapterTest UT
2024-06-25 23:45:35 +05:30
Vishesh 3923f80c22
Merge branch '4.19' 2024-06-25 18:53:57 +05:30
Vishesh f6ceeab3b3
server: Enforce strict host tag check (#9017)
Documentation PR: https://github.com/apache/cloudstack-documentation/pull/398

Currently, an administrator can break host tag compatibility for a VM administrator by certain operations:
* deploy/start VM on a specific host
* migrate VM
* restore VM
* scale VM

This PR allows the user to specify tags which must be checked during these operations.

Global Settings
1. `vm.strict.host.tags` - A comma-separated list of tags for strict host check (Default - empty)
2. `vm.strict.resource.limit.host.tag.check` - Determines whether the resource limits tags are considered strict or not (Default - true)

During the above operations, we now check and throw an error if host tags compatibility is being broken for tags specified in `vm.strict.host.tags`. If `vm.strict.resource.limit.host.tag.check` is set to `true`, tags set in `resource.limit.host.tags` are also checked during these operations.
2024-06-25 14:42:17 +05:30
Rene Glover 6ee6603359
Updates to HPE-Primera and Pure FlashArray Drivers to use Host-based VLUN Assignments (#8889)
* Updates to change PUre and Primera to host-centric vlun assignments; various small bug fixes

* update to add timestamp when deleting pure volumes to avoid future conflicts

* update to migrate to properly check disk offering is valid for the target storage pool

* Updates to change PUre and Primera to host-centric vlun assignments; various small bug fixes

* update to add timestamp when deleting pure volumes to avoid future conflicts

* update to migrate to properly check disk offering is valid for the target storage pool

* improve error handling when copying volumes to add precision to which step failed

* rename pure volume before delete to avoid conflicts if the same name is used before its expunged on the array

* remove dead code in AdaptiveDataStoreLifeCycleImpl.java

* Fix issues found in PR checks

* fix session refresh TTL logic

* updates from PR comments

* logic to delete by path ONLY on supported OUI

* fix to StorageSystemDataMotionStrategy compile error

* change noisy debug message to trace message

* fix double callback call in handleVolumeMigrationFromNonManagedStorageToManagedStorage

* fix for flash array delete error

* fix typo in StorageSystemDataMotionStrategy

* change copyVolume to use writeback to speed up copy ops

* remove returning PrimaryStorageDownloadAnswer when connectPhysicalDisk returns false during KVMStorageProcessor template copy

* remove change to only set UUID on snapshot if it is a vmSnapshot

* reverting change to UserVmManagerImpl.configureCustomRootDiskSize

* add error checking/simplification per comments from @slavkap

* Update engine/storage/datamotion/src/main/java/org/apache/cloudstack/storage/motion/StorageSystemDataMotionStrategy.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* address PR comments from @sureshanaparti

---------

Co-authored-by: GLOVER RENE <rg9975@cs419-mgmtserver.rg9975nprd.app.ecp.att.com>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-06-25 10:35:39 +05:30
Rohit Yadav 42e71175d7 Merge remote-tracking branch 'origin/4.18' into 4.19 2024-06-24 20:39:55 +05:30
Abhishek Kumar de266a4cb8
api,server: allow updating hypervisor capabilities with hypervisor and version (#8475)
* api,server: allow updating hypervisor capabilities with hypervisor and
version

hypervisor and hypervisorversion parameter added to the
updateHypervisorCapabilities API.

* param description

* Update server/src/main/java/com/cloud/server/ManagementServerImpl.java

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: Henrique Sato <henriquesato2003@gmail.com>
Co-authored-by: Vishesh <vishesh92@gmail.com>
2024-06-24 14:41:01 +05:30
Wei Zhou fcfa371337
kvm: support Rocky/RHEL/OL/Alma in the same cluster (#8641)
* kvm: support Rocky/RHEL/OL/Alma in the same cluster

* Update PR#8641: add unit tests
2024-06-24 13:10:41 +05:30
Suresh Kumar Anaparti c17aa0d9ad
Import Remote KVM VM logging improvements (#9284) 2024-06-24 11:34:37 +05:30
dahn 7e00678691
set isSystem for SVM IPs (#9281)
* set isSystem for SSVM IPs

* Revert "set isSystem for SSVM IPs"

This reverts commit 4ba71b3d6b4ea377f6778d166ff9da7d41e5d007.

* set isSystem flag for SSVMs
2024-06-23 19:36:45 +05:30
Vishesh 59e9ab9efe
Fix volume response for service offering with disk offering (#9273) 2024-06-22 11:17:11 +05:30
Wei Zhou 313a165e62
server: add global setting consoleproxy.sslEnabled (#8809) 2024-06-21 15:56:20 +05:30
dahn 60f234c682
remove Project Template Permissions inhibition (#9196) 2024-06-21 15:20:49 +05:30
Gabriel Pordeus Santos 083ac069ca
fix assignvm template permission check (#8886) 2024-06-21 10:58:46 +02:00
Suresh Kumar Anaparti 5ab23cd9c9
Timeout config to copy the disks of remote KVM instance while importing the instance from an external host (#9213)
* Added timeout config to copy the disks of remote KVM instance while importing the instance from an external host

* Updated copy config units to mins

* Cleanup remote converted file and local file when copy failed
2024-06-21 10:28:18 +05:30
João Jandre ae3fa5d0de
Add configuration to limit the number of rows deleted from vm_stats (#8740)
Co-authored-by: João Jandre <joao@scclouds.com.br>
2024-06-20 14:26:36 +02:00
Wei Zhou d81ffd2d86
Merge remote-tracking branch 'apache/4.19' 2024-06-20 08:57:17 +02:00
Abhishek Kumar 3e6900ac1a
api,server: purge expunged resources (#8999)
This PR introduces the functionality of purging removed DB entries for CloudStack entities (currently only for VirtualMachine). There would be three mechanisms for purging removed resources:

    Background task - CloudStack will run a background task which runs at a defined interval. Other parameters for this task can be controlled with new global settings.
    API - New admin-only API purgeExpungedResources. It will allow passing the following parameters - resourcetype, batchsize, startdate, enddate. Currently, API is not supported in the UI.
    Config for service offering. Service offerings can be created with purgeresources parameter which would allow purging resources immediately on expunge.

Following new global settings have been added:

    expunged.resources.purge.enabled: Default: false. Whether to run a background task to purge the expunged resources
    expunged.resources.purge.resources: Default: (empty). A comma-separated list of resource types that will be considered by the background task to purge the expunged resources. Currently only VirtualMachine is supported. An empty "value will result in considering all resource types for purging
    expunged.resources.purge.interval: Default: 86400. Interval (in seconds) for the background task to purge the expunged resources
    expunged.resources.purge.delay: Default: 300. Initial delay (in seconds) to start the background task to purge the expunged resources task.
    expunged.resources.purge.batch.size: Default: 50. Batch size to be used during expunged resources purging.
    expunged.resources.purge.start.time: Default: (empty). Start time to be used by the background task to purge the expunged resources. Use format yyyy-MM-dd or yyyy-MM-dd HH:mm:ss.
    expunged.resources.purge.keep.past.days: Default: 30. The number of days in the past from the execution time of the background task to purge the expunged resources for which the expunged resources must not be purged. To enable purging expunged resource till the execution of the background task, set the value to zero.
    expunged.resource.purge.job.delay: Default: 180. Delay (in seconds) to execute the purging of an expunged resource initiated by the configuration in the offering. Minimum value should be 180 seconds and if a lower value is set then the minimum value will be used.

Documentation PR: apache/cloudstack-documentation#397

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Wei Zhou <weizhou@apache.org>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-06-20 11:34:44 +05:30
Wei Zhou 81269ad62a
VNF: add API listVnfAppliances (#8994) 2024-06-19 19:49:21 +05:30
Harikrishna 2315a73a20
User friendly name of Downloaded Templates Volumes and ISOs (#9252) 2024-06-19 12:47:43 +02:00
Suresh Kumar Anaparti 8b02624e61
User data content size validation, and related code improvements (#8418)
Co-authored-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-06-18 20:07:13 +02:00
Daan Hoogland 373f017002 Merge branch '4.19' 2024-06-18 19:58:43 +02:00
Daan Hoogland 050ee44137 Merge branch '4.18' into 4.19 2024-06-18 16:05:45 +02:00
Vishesh 6c7426e3a7
Prevent network disruption on adding a VPC tier for redundant VRs (#9251) 2024-06-18 13:47:45 +02:00
dahn 7c5b7ca077
Extra parameter for UpdateImageStore (#8941)
* Extra parameter for UpdateImageStore

* add name parameter

* ui

* cleanup

* update DB from storage stats results
2024-06-18 12:31:17 +05:30
Harikrishna bb0c1f93af
Add volume encryption checks during the disk offering change (#9209) 2024-06-17 10:36:47 +02:00
Wei Zhou aab1361949
server: fix mgmt server 503 error by renaming kubernetesClusterHelper to kubernetesServiceHelper 2024-06-15 08:35:36 +02:00
Bryan Lima 00fe25ab01
Fix allocation of VMs with multiple clusters (#8611)
* Fix allocation of VMs with multiple clusters

* Readd debug guard
2024-06-14 13:54:01 +03:00
dahn ed86dc973b
protect against missing service offering (#9235)
* protect agains missing service offering

* search removed before assuming none

* import

* javadoc
2024-06-14 15:25:46 +05:30
Daan Hoogland cb9b3134f7 Merge branch '4.19' 2024-06-14 10:30:10 +02:00
Abhishek Kumar ce9b2c52f3
cks: fix events (#9070)
Fixes #8043

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-14 12:22:39 +05:30
Abhishek Kumar df5c546559
server: remove username required condition for listVmsForImport (#9124)
* server: remove username required condition for listVmsForImport

Fixes #8414

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* revert

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-14 12:14:59 +05:30
Rohit Yadav 2ca0857bd5
api: listVM API improvement followup, change returning of stats detail (#9177)
- Changes behaviour of details param handling via global setting:
  - listVirtualMachines API: when the details param is not provided, it returns whether stats are returned controlled by a new global setting `list.vm.default.details.stats`
  - listVirtualMachinesMetrics API: when the details param is not provided, it uses `all` details including `stats`
- Users who are affected slow performance of the listVirtualMachines API response time can set `list.vm.default.details.stats` to `false`
- Remove ConfigKey vm.stats.increment.metrics.in.memory which was renamed to `vm.stats.increment.metrics` in #5984 and also remove unused/unnecessary global settings via upgrade path
- Changes default value of VM stats accumulation setting `vm.stats.increment.metrics` to false until a better solution emerges. Since #5984, this is true and during the execution of listVM APIs the stats are clubbed/calculated which can immensely slow down list VM API calls. Any costly operations such as summing of stats shouldn't be done during the course of a synchronous API, such as the list VM API.
- Fix UI that uses listVirtualMachinesMetrics to not call `stats` detail when in list view without metrics selected.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-06-14 11:03:14 +05:30
Harikrishna 517cddcb15
Fix error message if specific host does not have capacity (#9218) 2024-06-13 12:38:51 +05:30
Vishesh 6fe835e118
Merge branch '4.19' 2024-06-13 11:57:55 +05:30
Suresh Kumar Anaparti 6fda757936
While starting VM with 'considerlasthost' enabled, don't load host tags/details for the last host when it doesn't exist [main] (#9063) 2024-06-12 17:03:18 +02:00
Vishesh faf94387a6
Disable logs to stdout and capture remaining stdout logs to a file (#8880)
* Log stdout to a file

* Add logrotation
2024-06-12 13:46:31 +03:00
Vishesh c756e4370a
List volumes by service offering id (#9211)
* Allow listing of volumes by service offering id

* Address comments
2024-06-12 16:13:56 +05:30
dahn 4de975ff25
list by isEncrypted (#8643)
* list by isEncrypted

* use filter on VO and cleanup

* add encryption type to volume response

* Update api/src/main/java/org/apache/cloudstack/api/command/user/volume/ListVolumesCmd.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-06-12 15:11:00 +05:30
Suresh Kumar Anaparti 503ae64a3d
Upload volume format check with the supported hypervisor, fail if supported hypervisor not found on zone (#9224) 2024-06-12 11:40:29 +02:00
Pearl Dsilva 0f770194df
Configure config drive ISO with password file on reinstallation of VM with password-enabled template (#9113)
* Configure config drive ISO with password file on reinstallation of VM with password-enabled template

* address comment
2024-06-12 12:00:08 +03:00
Abhishek Kumar 1577218999
server,event: fix events for storage maintenance (#9056)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-12 09:43:13 +02:00
Abhishek Kumar ca59720015
vpc,event: fix events for createVpc (#9055)
Fixes #8496

* vpc,event: fix events for createVpc
2024-06-12 12:56:08 +05:30
Suresh Kumar Anaparti 4e7c6682fd
While starting VM with considerlasthost enabled, don't load host tags/details for the last host when it doesn't exist (#9037) 2024-06-12 07:49:03 +02:00
Wei Zhou edf7394919
server: search template name and display text if keyword is passed (#9215) 2024-06-11 19:35:43 +05:30
Suresh Kumar Anaparti 2e3f76ec03
Improve error messaging / logs when listing VMs on the remote KVM host (for import) (#9204) 2024-06-11 14:48:21 +02:00
Harikrishna 6edcf32c5e
Fix vsphere storage policy when compute offering uses the disk offering attached to it (#9203) 2024-06-11 14:49:58 +03:00
Bernardo De Marco Gonçalves 1383625c93
Fix `*.smtp.useAuth`, `quota.usage.smtp.useStartTLS` and `*.smtp.enabledSecurityProtocols` settings definitions (#9031)
* change configs definitions

* add normalization query

* add ui support

* add labels

* add end of line to SQL script
2024-06-11 10:54:03 +03:00
Abhishek Kumar fcca3e8f39
api,server: encryptformat in volume response (#9106)
Fixes #9014

* api,server: encryptformat in volume response

* Apply suggestions from code review

* remove passphrase db change
2024-06-11 11:20:50 +05:30
Vishesh cc8dc84f64
server: fix resource reservation leakage (#9169)
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-10 12:30:06 +05:30
Abhishek Kumar be552fdce9
feature: webhooks (#8674)
* api,server,ui: weebhoks feature

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* registry of message busses

* test bus

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix and refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for webhook dispatch history

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes, initial ui

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* improvements

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for account webhook cleanup

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix remaining event bus usage

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for testing webhook dispatch

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wip

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* make element

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* missing

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* buid fix

* fix lint

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for project delete check

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add collapse in create

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui fix and refactor for eventditributor publish

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* update org.json and add json validation

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* schema fixes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wordings

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: improve progress button

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui improvements

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove unrelated change

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* search and count

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add payloadurl in info

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* positive progress

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix hmac key

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* create webhook form fixes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor, address feedback

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* indentation

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix filters

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove test eventbus

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* default scope be Local

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add lifecycle smoke test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add test for webhook deliveries

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix lint

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor - losgs and others

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* unit tests

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix lint

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* build fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* smoke test fix, log refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* get bean from all components

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: missing label

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* address review comments

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add some more tests

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* lint

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* rename setting

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* upgrade: move 4.19.0->4.20.0 to 4.19.1->4.20.0

* fix test delivery layout

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix webhook secret display

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add http to payloadurl when no scheme

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* allow removing secretkey for webhook

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix update sslverification

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* disallow same payload url for same account

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix delivery with url w/o scheme

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* api: listApis should return params based on caller

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wip changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Update engine/schema/src/main/resources/META-INF/db/schema-41900to42000.sql

* remove unique constraint for now

Constraint is present in Java code validations

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fixes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: add option to delete multiple deliveries

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add filter for deliveries, delete api start/endtime support

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* do not throw error when no deliveries removed

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: fix deliveries table column sorting, time filter cancel

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove isDebugEnabled wrapping

* merge fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Daan Hoogland <daan@onecht.net>
Co-authored-by: Wei Zhou <weizhou@apache.org>
2024-06-10 10:40:12 +05:30
Daan Hoogland b29391c329 Merge LTS branch '4.19' into main 2024-06-07 09:53:57 +02:00
Abhishek Kumar 91c7bc722f
server,cks: check if vm is cks node during vm destroy (#9057)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-06 10:54:02 +02:00
João Jandre 631d6ad09b
Do not retrieve VM's stats on normal VM listing (#8782)
* Do not retrieve VM's stats on normal VM listing

* Add config to control the behavior

* address reviews
2024-06-05 17:45:28 +05:30
Wei Zhou 875cd14919
server: fix fail to list public ips of shared networks sometimes (#8624) 2024-06-04 09:28:39 +02:00
Wei Zhou 5433e775e5
New feature: Implicit host tags (#8929)
* Merge two HostTagVO and HostTagDaoImpl

* Implicit host tags

* PR8929: add since

* Update variable names

* Update 8929: add unit test in LibvirtComputingResourceTest

* Update 8929: add explicithosttags in response

* Update 8929 UI: Update explicit host tags

* Update 8929: remove host tags and change labels on UI

* Update 8929: update host_view to use explicit_host_tags.is_tag_a_rule

* Update: ui polish for host tags

* Update 8929: fix UI error if no host tags
2024-05-30 17:21:13 +05:30
Vishesh 87b55af197
Fixup response code on incorrect credentials (#8671) 2024-05-30 08:48:53 +02:00
Daan Hoogland f1c3d2c4be Merge release branch 4.19 to main
* 4.19:
  engine-schema: add removed columne for cloud.user_data, fix delete op (#9120)
  ui: fix create menu item access (#9104)
  server: honor listall param for listiso api (#9064)
  ui: add move to top/bottom for acl rules list (#9119)
  ui: fix columns for exportacls csv (#9118)
  ui: fix create network access in deploy vm wizard (#9117)
  UI: Add search filters (#9068)
2024-05-28 09:44:05 +02:00
Abhishek Kumar 261530c92a
server: honor listall param for listiso api (#9064)
Fixes #8315

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-05-28 12:43:43 +05:30
John Bampton 40c5d35377
Fix spelling in docs, logs, exception messages etc (#9076) 2024-05-28 07:41:20 +02:00
Fabricio Duarte 371ce12abe
Normalize dates in Usage and Quota APIs (#8243)
* Normalize dates in Usage and Quota APIs

* Apply Daan's sugestions

Co-authored-by: dahn <daan.hoogland@gmail.com>

* Restore removed sinces

* Add missing space

* Change param descriptions for quotaBalance and quotaStatement

* Apply Daniel's suggestions

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

---------

Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
2024-05-27 12:06:52 -03:00
Abhishek Kumar acce88ff39 Merge remote-tracking branch 'apache/4.19' 2024-05-27 15:14:29 +05:30
Abhishek Kumar 57e67afdf0
api,server: list autoscalevmgroups with keyword (#9046)
Fixes #9042

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-05-27 13:47:44 +05:30
Hans Rakers e817e04343
Fix typo keyparis -> keypairs in InvalidParameterValueException (#9100) 2024-05-24 17:41:02 +02:00
Abhishek Kumar daf6b9d103
api,ui: vm template format, fix vm info link (#9094) 2024-05-24 17:39:52 +02:00
Daan Hoogland b38f1364e6 Merge branch '4.19' 2024-05-23 12:34:04 +02:00
Abhishek Kumar 33659fdf06
server,test: fix resourceid for VOLUME.DETROY in restore VM (#9032)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-05-22 11:02:14 +02:00
Vishesh 21af134087
Fix exceeding of resource limits with powerflex (#9008)
* Fix exceeding of resource limits with powerflex

* Add e2e tests

* Update server/src/main/java/com/cloud/vm/UserVmManagerImpl.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* fixup

---------

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-05-08 20:55:19 +05:30
Vishesh 6b4955affe
Fix message publish in transaction (#8980)
* Fix message publish in transaction

* Resolve comments
2024-05-07 13:27:31 +05:30
Henrique Sato 0d1bc7dfd0
Limit `listRoles` API visibility (#8639)
Co-authored-by: Henrique Sato <henrique.sato@scclouds.com.br>
2024-05-07 09:12:49 +02:00
Daan Hoogland d5241d319c Merge release branch 4.19 to main
* 4.19:
  Fix: Update rootdisksize detail on restore VM (#8981)
  UI: hide Cleanup/Expunge when delete a CloudManaged CKS cluster (#8593)
2024-04-30 11:35:29 +02:00
Vishesh 027149487b
Fix: Update rootdisksize detail on restore VM (#8981)
* Fix: Update rootdisksize detail on restore VM

* minor fixup
2024-04-29 22:15:06 +05:30
Daan Hoogland e61f3bae4d Merge branch '4.19' 2024-04-29 11:37:40 +02:00
Rohit Yadav 3de1f8b4ba Merge remote-tracking branch 'origin/4.18' into 4.19
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-04-29 13:44:34 +05:30
dahn 54926c324f
sanatise log in case multiple passwordsa have been passed (#8891) 2024-04-29 09:37:36 +02:00
Vishesh 08132acaa2
Fix restore VM with allocated root disk (#8977)
* Fix restore VM with allocated root disk

* Add e2e test for restore vm

* Add more checks for e2e test
2024-04-29 12:19:05 +05:30
Vishesh 80a8b80a9d
Update volume's passphrase to null if diskOffering doesn't support encryption (#8904) 2024-04-29 12:18:09 +05:30
dahn eead2710f8
explanatory error message on delete attempt of default system offering (#8883)
Co-authored-by: Gabriel Pordeus Santos <gabrielpordeus@gmail.com>
2024-04-25 16:45:42 +02:00
Vishesh cfdb33a052
Fixup resource limit checks (#8935) 2024-04-25 12:59:35 +02:00
Vishesh e409c6d870
Fixup listing of serivce offering & storagepools with tags (#8937) 2024-04-25 12:58:52 +02:00
Marcus Sorensen 27124c1031
Add ability to set cpu.threadspercore similar to existing cpu.corespersocket (#8850)
* Add ability to set cpu.threadspercore similar to existing cpu.corespersocket

* add cpu.threadspercore to VM and template detail options

* Update plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* add vm detail for KVM

---------

Co-authored-by: Marcus Sorensen <mls@apple.com>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-04-24 18:01:21 +05:30
Daan Hoogland 0af923e618 Merge branch '4.19' 2024-04-24 14:01:02 +02:00
dahn a6f3560152
fix log4j to log4j2 issue (#8966) 2024-04-24 10:08:15 +02:00
Wei Zhou 4ffeec6ba4
vpc: create vpc tiers with dns server of vpc instead of zone (#8621)
* vpc: create vpc tiers with dns server of vpc instead of zone

with this change
```
root@r-575-VM:~# cat /etc/dnsmasq.d/cloud.conf
dhcp-hostsfile=/etc/dhcphosts.txt
listen-address=127.0.0.1,172.17.1.1,172.17.2.1,172.17.3.1
dhcp-range=set:interface-eth2-0,172.17.1.1,static
dhcp-option=tag:interface-eth2-0,15,cs2cloud.internal
dhcp-option=tag:interface-eth2-0,6,172.17.1.1,10.0.32.1,8.8.8.8
dhcp-option=tag:interface-eth2-0,3,172.17.1.1
dhcp-option=eth2,26,1500
dhcp-option=tag:interface-eth2-0,1,255.255.255.0
dhcp-range=set:interface-eth3-1,172.17.2.1,static
dhcp-option=tag:interface-eth3-1,15,cs2cloud.internal
dhcp-option=tag:interface-eth3-1,6,172.17.2.1,8.8.8.8,1.1.1.1
dhcp-option=tag:interface-eth3-1,3,172.17.2.1
dhcp-option=eth3,26,1500
dhcp-option=tag:interface-eth3-1,1,255.255.255.0
dhcp-range=set:interface-eth4-2,172.17.3.1,static
dhcp-option=tag:interface-eth4-2,15,cs2cloud.internal
dhcp-option=tag:interface-eth4-2,6,172.17.3.1,8.8.8.8,1.1.1.1
dhcp-option=tag:interface-eth4-2,3,172.17.3.1
dhcp-option=eth4,26,1500
dhcp-option=tag:interface-eth4-2,1,255.255.255.0
root@r-575-VM:~#
```

* Update PR8621 test:ArgumentMatchers instead of Matchers in test
2024-04-24 09:10:52 +02:00
SadiJr 96ae479000
[Usage] Create network billing (#7236)
Co-authored-by: Bryan Lima <bryan.lima@hotmail.com>
Co-authored-by: SadiJr <sadi@scclouds.com.br>
Co-authored-by: Bryan Lima <42067040+BryanMLima@users.noreply.github.com>
Co-authored-by: Henrique Sato <henriquesato2003@gmail.com>
2024-04-24 08:52:49 +02:00
Daan Hoogland a358c9a410 Merge release branch 4.19 to main
* 4.19:
  New feature: Import/Unamange DATA volume from storage pool (#8808)
2024-04-23 16:13:16 +02:00
Wei Zhou 0b857def68
New feature: Import/Unamange DATA volume from storage pool (#8808) 2024-04-23 16:05:59 +02:00
Daan Hoogland 66c10bf42f logger issues 2024-04-23 12:55:17 +02:00
Daan Hoogland c31aebc188 Merge release branch 4.19 to main
* 4.19:
  server: fix duplicated records for templates if there are multiple zones (#8933)
2024-04-23 12:22:53 +02:00
Wei Zhou f6802e9198
server: fix duplicated records for templates if there are multiple zones (#8933) 2024-04-23 12:06:45 +02:00
Rohit Yadav 93d4816ee8 Merge remote-tracking branch 'origin/4.19'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

 Conflicts:
	plugins/storage/volume/linstor/src/main/java/com/cloud/hypervisor/kvm/storage/LinstorStorageAdaptor.java
2024-04-23 15:24:19 +05:30
Rohit Yadav 0fa71f5696 Merge remote-tracking branch 'origin/4.18' into 4.19 2024-04-23 15:21:44 +05:30
Wei Zhou 7b57480d60
Merge remote-tracking branch 'apache/4.19' into main 2024-04-23 10:06:16 +02:00
Vishesh c081f60427
server: Fix null pointer exception in restore VM (#8930) 2024-04-22 19:21:16 +02:00
dahn 0577b0ac8e
server: add logs to public ip allocation attempt (#8239) 2024-04-22 10:31:48 +02:00
SadiJr 49d244f251
[Usage] Create VPC billing (#7235)
Co-authored-by: SadiJr <sadi@scclouds.com.br>
Co-authored-by: Bryan Lima <bryan.lima@hotmail.com>
2024-04-22 09:42:24 +02:00
Wei Zhou 8ff2c018cc
test: fix test failures in ActionEventInterceptorTest (#8938)
* test: fix test failures in ActionEventInterceptorTest

```
Error:  Failures:
Error:    ActionEventInterceptorTest.testInterceptComplete:247
Error:    ActionEventInterceptorTest.testInterceptException:261
Error:    ActionEventInterceptorTest.testInterceptStartAsync:234 expected:<Starting VM[.  Id: 0 Name: null]> but was:<Starting VM[]>
```

* Update 8938: move CallContext.unregister as well
2024-04-19 15:10:38 +02:00
Daan Hoogland 7de8a6d082 Merge LTS branch '4.19' into main 2024-04-18 08:54:58 +02:00
João Jandre 8a101fbbc1 Updating pom.xml version numbers for release 4.18.3.0-SNAPSHOT
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-04-17 11:11:57 -03:00
Vishesh 8511014707
Fix error message for checkVolume command (#8842) 2024-04-17 17:27:27 +05:30
Vishesh 63a0797b18
Introduce scheduled executor wrapper with dynamic interval (#8916)
* Introduce scheduled executor wrapper with dynamic interval

* Add validation for configkey
2024-04-17 15:15:37 +05:30
Vishesh ebaf5a47b9
Speedup resource count calculation (#8903)
* Speed up resource count calculation

* Refactor resource count calculation

* Start transaction for updateCountByDeltaForIds
2024-04-17 14:21:30 +05:30
Henrique Sato 090c1e38a6
Improve vm.allocation.algorithm global setting description (#8823)
Co-authored-by: Henrique Sato <henrique.sato@scclouds.com.br>
2024-04-16 16:55:20 +02:00
Henrique Sato 075b413041
Fix VR inheriting boot settings from user's VM (#8894)
Co-authored-by: Henrique Sato <henrique.sato@scclouds.com.br>
2024-04-16 16:54:16 +02:00
Wei Zhou 45daa1ce59
Merge remote-tracking branch 'apache/4.19' 2024-04-12 16:40:07 +02:00
Vishesh b998e7dbb6
Allow overriding root disk offering & size, and expunge old root disk while restoring a VM (#8800)
* Allow overriding root diskoffering id & size while restoring VM

* UI changes

* Allow expunging of old disk while restoring a VM

* Resolve comments

* Address comments

* Duplicate volume's details while duplicating volume

* Allow setting IOPS for the new volume

* minor cleanup

* fixup

* Add checks for template size

* Replace strings for IOPS with constants

* Fix saveVolumeDetails method

* Fixup

* Fixup UI styling
2024-04-12 17:47:52 +05:30
João Jandre 154566f914 Updating pom.xml version numbers for release 4.18.2.0
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-04-12 08:25:04 -03:00
Vishesh 19f79b1d94
Merge branch '4.19' 2024-04-12 16:31:40 +05:30
Vishesh 730cc5d5b8
Change iops on offering change (#8872)
* Change IOPS on disk offering change

* Remove iops & bandwidth limits before copying template

* minor refactor

* Handle diskOfferingDetails

* Fixup
2024-04-11 17:01:55 +05:30
Vishesh a44f28bb31
Merge branch '4.19' 2024-04-10 17:30:37 +05:30
Vishesh c24c1a5c00
Fix VM import & VM delete with custom offering (#8813) 2024-04-10 11:29:29 +02:00
Wei Zhou 3fd5680d9a
Merge remote-tracking branch 'apache/4.19' 2024-04-08 14:16:33 +02:00
Wei Zhou c7626ebfd6
Merge remote-tracking branch 'apache/4.18' into 4.19 2024-04-08 14:14:04 +02:00
Wei Zhou a5508acc54
server: fix haproxy misconfiguration after VPC VR start (#8881) 2024-04-05 15:01:08 -03:00
Abhishek Kumar 55b28d645f Merge remote-tracking branch 'apache/4.19' 2024-04-04 17:54:42 +05:30
Abhishek Kumar 02305fbc5f Merge remote-tracking branch 'apache/4.19' 2024-04-04 17:36:05 +05:30
Vishesh c25264bada
UI Fixes: allow filtering on alertype in UI & add events tab (#8630) 2024-04-04 13:51:56 +02:00
Vishesh b8f8321f0f
Allow listing of inactive offerings (#8821) 2024-04-04 13:46:44 +02:00
Abhishek Kumar ff3e9bd821 engine-storage: control download redirection
Add a global setting to control whether redirection is allowed while
downloading templates and volumes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-04-04 14:11:05 +05:30
Wei Zhou 939d0b9011 engine-storage: control download redirection
Add a global setting to control whether redirection is allowed while
downloading templates and volumes

core: some changes on SimpleHttpMultiFileDownloader
similar as HttpTemplateDownloader

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
(cherry picked from commit b1642bc3bf)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-04-04 11:19:20 +05:30
dahn 67e2061f4b api: client verification in servlet
This introduces new global settings to handle how client address checks
are handled by the API layer:

proxy.header.verify: enables/disables checking of ipaddresses from a
                     proxy set header
proxy.header.names: a list of names to check for allowed ipaddresses
                    from a proxy set header.
proxy.cidr: a list of cidrs for which \"proxy.header.names\" are
            honoured if the \"Remote_Addr\" is in this list.

(cherry picked from commit b65546636d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit b1e0bf9dbd)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-04-04 11:19:15 +05:30
Wei Zhou fc3c625beb server: fix security issues caused by extraconfig on KVM
- Move allow.additional.vm.configuration.list.kvm from Global to Account setting
- Disallow VM details start with "extraconfig" when deploy VMs
- Skip changes on VM details start with "extraconfig" when update VM settings
- Allow only extraconfig for DPDK in service offering details
- Check if extraconfig values in vm details are supported when start VMs
- Check if extraconfig values in service offering details are supported when start VMs
- Disallow add/edit/update VM setting for extraconfig on UI

(cherry picked from commit e6e4fe16fb)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 7aea9db1c8)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-04-04 11:19:08 +05:30
dahn 00f687db1b api: client verification in servlet
This introduces new global settings to handle how client address checks
are handled by the API layer:

proxy.header.verify: enables/disables checking of ipaddresses from a
                     proxy set header
proxy.header.names: a list of names to check for allowed ipaddresses
                    from a proxy set header.
proxy.cidr: a list of cidrs for which \"proxy.header.names\" are
            honoured if the \"Remote_Addr\" is in this list.

(cherry picked from commit b65546636d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-04-04 11:00:33 +05:30
Wei Zhou 72b2eb0087 server: fix security issues caused by extraconfig on KVM
- Move allow.additional.vm.configuration.list.kvm from Global to Account setting
- Disallow VM details start with "extraconfig" when deploy VMs
- Skip changes on VM details start with "extraconfig" when update VM settings
- Allow only extraconfig for DPDK in service offering details
- Check if extraconfig values in vm details are supported when start VMs
- Check if extraconfig values in service offering details are supported when start VMs
- Disallow add/edit/update VM setting for extraconfig on UI

(cherry picked from commit e6e4fe16fb)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-04-04 11:00:10 +05:30
Vishesh c4036800de
Fix: check root disk offering tagged limits during VM deploy (#8871) 2024-04-03 19:42:43 +05:30
Vishesh 93f31828f5
Hypervisor as class (#8798)
* HypervisorType as a class

* Fixup

* fixup

* Add missing annotation

* Resolve comments

* Handle parallels typo

* Add missing @Convert annotation
2024-04-02 17:36:15 +05:30
Wei Zhou 08d9d06d45
api,server,ui: add project ID and name to UserDataResponse (#8656)
* api,server,ui: add project ID and name to UserDataResponse

* Update: add since
2024-03-25 19:15:34 +05:30
John Bampton e44c17e077
Fix spelling in docs and code comments only (#8774) 2024-03-25 14:38:01 +01:00
Wei Zhou 12f65fbcc0
build/packaging: build nsx only if noredist is passed (#8772) 2024-03-21 17:00:31 +01:00
Rohit Yadav 01d44347e7 Merge remote-tracking branch 'origin/4.19'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

 Conflicts:
	plugins/network-elements/juniper-contrail/pom.xml
	pom.xml
	tools/marvin/setup.py

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-03-21 18:38:01 +05:30
Marcus Sorensen 2e88eb45a3
Update mysql-connector version (#8753)
Co-authored-by: Marcus Sorensen <mls@apple.com>
2024-03-21 18:09:06 +05:30
Wei Zhou 502a633ddd build: fix build failure after merge-forward 2024-03-20 14:07:42 +01:00
Rohit Yadav 0e32380e0a Merge remote-tracking branch 'origin/4.19' 2024-03-20 15:50:01 +05:30
Rohit Yadav 308ed1399e Merge remote-tracking branch 'origin/4.18' into 4.19 2024-03-20 15:49:21 +05:30
Rohit Yadav 720407ba73
snapshot: don't schedule next snapshot job for a removed volume (#8735)
* snapshot: don't schedule next snapshot job for a removed volume

When management server starts, it starts the snapshot scheduler. In case
there is a volume snapshot policy which exists for a volume which does
not exist, it can cause SQL constraint issue and cause the management
server to break from starting its various components and cause HTTP 503
error.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* remove schedule on missing volume

---------

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-03-19 09:01:58 -03:00
Vishesh 33dc7465c2
Merge remote-tracking branch 'origin/4.19' 2024-03-19 15:18:02 +05:30
Vishesh fe5d741236
Merge branch '4.18' into '4.19' 2024-03-19 13:59:29 +05:30
Vishesh 0043540fa3
Use join instead of views (#8321) 2024-03-18 18:08:19 +01:00
Abhishek Kumar ffd59720dd
storage,plugins: delegate allow zone-wide volume migration check and access grant check to storage drivers (#8762)
* storage,plugins: delegate allow zone-wide volume migration check and access grant to storage drivers

Following checks have been delegated to storage drivers,
- For volumes on zone-wide storage, whether they need storage migration when VM is migrated
- Whther volume required grant access

Apply fixes in resolving PrimaryDataStore

* add tests

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* unused import

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Update engine/orchestration/src/test/java/org/apache/cloudstack/engine/orchestration/VolumeOrchestratorTest.java

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-03-18 17:28:14 +05:30
Wei Zhou 9d748f0837
server: update default value of setting 'expose.dns.externally' to 'false' (#8776) 2024-03-15 10:43:16 -03:00
Bernardo De Marco Gonçalves e0731c5098
Fix the removal of network offering tags (#8758) 2024-03-15 13:59:19 +01:00
nvazquez db564b12b6
Merge branch '4.19' 2024-03-14 22:46:48 -03:00
Nicolas Vazquez f25d35c5d6
VM Import: Use datastore name when datastore path is not set to search for pools on DB (#8759) 2024-03-14 22:07:23 -03:00
Vishesh e87c6cfcb1
Fix resource count discrepancies (#8302)
* Fix resource count discrepancies

* Fixup while removing vm

* Fix discrepancies when starting VMs

* Fixup tests

* Fix failing tests

* Don't take lock when amount is negative

---------

Co-authored-by: dahn <daan@onecht.net>
2024-03-13 18:22:44 +05:30
Pearl Dsilva 6dc3d06037
NSX integration (#7919)
* NSX integration - skeletal code

* Fix module not loading on startup

* add upgrade path and daos
\n add nsx controller command

* add support for adding and listing nsx provider to a zone

* add license

* add default VPC offering and update upgrade path

* add global setting to enable nsx plugin

* add delete nsx controller operation

* add nsxresource

* add NSX resource , api client, create tier1 gw

* update db

* update response and add license

* Add support to create and delete nsx tier-1 gateway

* add license

* cleanup and add skeletal code for network creation

* add create/delete segment and UI integration

* add license

* address code smells - part 1

* fix test / build failure

* NSX integration - skeletal code

* Fix module not loading on startup

* add upgrade path and daos
\n add nsx controller command

* add support for adding and listing nsx provider to a zone

* add license

* add default VPC offering and update upgrade path

* add global setting to enable nsx plugin

* add delete nsx controller operation

* add nsxresource

* add NSX resource , api client, create tier1 gw

* update db

* update response and add license

* Add support to create and delete nsx tier-1 gateway

* add license

* cleanup and add skeletal code for network creation

* add create/delete segment and UI integration

* add license

* address code smells - part 1

* fix test / build failure

* add ui changes + update nsx_provider table transport zones + use NSX broadcast domain for add nics to router

* ui: fix password field, and backend changes

* add route advertisement

* update offering

* update offering

* add sleep before deletion of vpc / tier g/w for ports to be removed

* move creation of segments to design phase

* change provider to VPC router for Dhcp & dns service in an nsx offering

* Add public nic for NSX

* reserve first IP (after g/w) of subnet for router nic - NSX

* revert reserving 1st IP in vpc segments

* [NSX] Create a DHCP relay and add it to a VPC tier segment (#107)

* Create DHCP relay command and execute request

* In progress integrate with networking

* Create DHCP relay config on the network VR allocation

* Revert domain router dao changes

* Create DHCP relay con VR nic plug to NSX network

* Link DHCP relay config to segment after creation

* [NSX] Cleanup DHCP Relay config on segment deletion (#108)

* Cleanup DHCP Relay config on segment deletion

* update segment & relay name generators and call delete dhcprelay after deletion of segment

* address comment

* [NSX] Fix DHCP relay config deletion was missing zone name (#8068)

* [NSX] Refactor API wrapper operations (#8059)

* [NSX] Refactor API wrapper operations

* Big refactor

* Address review comment

* change network cidr to cidr to prevent NPE

* add domain and zone names to the various networks - vpc & tier

---------

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>

* Nsx unit tests (#8090)

* Add tests

* add test for NsxGuestNetworkGuru

* add unit tests for NsxResource

* add unti tests for NsxElement

* cleanup

* [NSX] Refactor API wrapper operations

* update tests

* update tests - add nsxProviderServiceImpl test

* add unit test - NsxServiceImpl

* add license

* Big refactor

* Address review comment

* change network cidr to cidr to prevent NPE

* add domain and zone names to the various networks - vpc & tier

* fix tests

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>

* modify NSX resource naming convention (#8095)

* modify NSX resource naming convention

* remove unused imports

* add a setup phase between desgin and implementation of a network for intermediary steps

* add method to all classes

* NSX: Refactor Network & VPC offering (#8110)

* [NSX] Refactor API wrapper operations

* Network offering changes for NSX

* fix services and provider combination

* address comments: rename param

* update nsx_mode parameter

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>

* fix test

* [NSX] Allow NSX isolated networks (#8132)

* Add network offerings for NSX on isolated networks

* Fix offerings creation

* In progress NSX isolated network

* Fixes

* Fix NIC allocation to router

* NSX: Add Step for Adding Public traffic network for NSX During zone creation (#8126)

* NSX: Add Step for Adding Public traffic network for NSX

* address comments and cleanup

* address comment

* remove indent

* NSX: Create and Delete static NAT & Port forward  rules (#8131)

* NSX: Create and delete NSX Static Nat rules

* fix issues with static nat

* add static nat

* Support to add and delete Port forward rules

* add license

* fix adding multiple pf rules

* cleanup

* fix lint check

* fix smoke tests

* fix smoke tests

* Nsx add lb rule (#8161)

* NSX: Create and delete NSX Static Nat rules

* fix issues with static nat

* add static nat

* Support to add and delete Port forward rules

* add license

* fix adding multiple pf rules

* cleanup

* NSX: Add support to create and delete Load balancer rules

* fix deletion of lb rules

* add header file and update protocol detail

* build failure fix

* [NSX] Add SNAT support (#8100)

* In progress add source NAT

* Fix after merge

* Fix tests

* Fix NPE on isolated network deletion

* Reserve source NAT IP when its not passed for NSX VPC

* Create source NAT rule on VR NIC allocation

* Fix update VPC and remove VPC to update and remove SNAT rule

* Fix packaging

* Address review comment

* Fix build

* fix build - unused import

* Add defensive checks

* Add missing design to NSX public guru

---------

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>

* NSX: Fix VR public NIC allocation (#8166)

* NSX: fix LB member addition and deletion and add defensive checks (#8167)

* Fix public NIC NPE on broadcast URI

* NSX: Router Public nic to get IP from systemVM Ip range (#8172)

* NSX: Router Public nic to get IP from systemVM Ip range

* Fix VR IP address and setSourceNatIp command

* NSX: hide systemVM reserved IP range SourceNAT

* fix test

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>

* fix test failure

* test failure fix

* [NSX] Fix update source NAT IP (#8176)

* [NSX] Fix update source NAT IP

* Fix startup

* Fix API result

* NSX - add LB route Advertizement (#8192)

* [NSX] Add ACL types support (#8224)

* NSX: Create segment group on segment creation

* Add unit tests

* Remove group for segment before removing segment

* Create Distributed Firewall rules

* Remove distributed firewall policy on segment deletion

* Fix policy rule ID and add more unit tests

* Fix DROP action rules and transform tests

* Add new ACL rules

* Fixes

* associate security policies with groups and not to DFW and add deletion of rules

* Fix name convention

---------

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>

* NSX: Fix creation of VPCs (#8320)

* Fix ACL rules creation (#8323)

* [NSX] Fix database views (#8325)

* NSX: Add CKS Support & Firewall rules for Isolated Networks (#8189)

* NSX: Add ALL LB IP to the list of route advertisements in tier1

* NSX: Support Source NAT on NSX Isolated networks

* NSX: Cks Support

* NSX: Create segment group on segment creation

* Add unit tests

* Remove group for segment before removing segment

* Create Distributed Firewall rules

* Remove distributed firewall policy on segment deletion

* Fix policy rule ID and add more unit tests

* Add support for routed NSX Isolated networks \n and non RFC 1918 compliant IPs

* Add support for routed NSX Isolated networks \n and non RFC 1918 compliant IPs

* Add Firewall rules

* build failure - fix unit test

* fix npes

* Add support to delete firewall rules

* update nsx cks offering

* add license

* update order of ports in PF & FW rules

* fix filter for getting transport zones

* CKS support changed - MTU updated, etc

* add LB for CKS on VPC

* address comments

* adapt upstream cks logic for vpc

* rever mtu hack

* update UI changes as per upstream fix

* change display test for CKS n/w offerings for isolated and VPC tiers

* add extra line for linter

* address comment

* revert list change

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>

* fix ui build failure

* [NSX] Address SonarCloud Bugs (#8341)

* [NSX] Address SonarCloud Bugs

* Fix NSX API connection issues

* NSX: Add unit tests to increase coverage (#8355)

* NSX: Add unit tests

* cleanup unused imports

* add more unit tests

* add tests for publicnsxnetworkguru

* add license

* fix build failures

* address sonar comment

* fix security hotspots

* NSX: Add more unit tests (#8381)

* NSX : Unit tests

* remove unused imports

* remove unused import causing build failure

* fix build failures due to unused imports

* fix build failure

* fix test assertion

* remove unused imports

* remove unused import

* Nsx UI zone bug (#8398)

* NSX: Attempt to fix NSX Zone creation bug for public networks

* fix zone wizard public traffic issue

* add proper filtering of offerings based on VPC nsx mode

* clean up console logs

* NSX: Fix code smells and reported bugs (#8409)

* NSX: Fix code smells and reported bugs

* fox override issue

* remove unused imports

* fix test

* refactor code to reduce complexity

* add lisence

* cleanup

* fix build failure

* fix build failure

* address comments

* test - add config to ignore certain files from test coverage

* test exclusion of classes from test cov

* rever pom changes

* [NSX] Add more unit tests (#8431)

* [NSX] Add more unit tests

* More tests

* Fix build errors

* NSX: Prevent creation of L2 and Shared networks for NSX (#8463)

* NSX: Prevent creation of L2 and Shared networks for NSX

* add checks to backend to prevent creation of l2 and shared networks in nsx zones and filter only nsx offerings when creating isolated networks

* cleanup

* NSX: Fix code smells (#8436)

* NSX: Fix code smells

* Add changes to service creation logic

* CKS: Add action to during firewall rule creation (#8498)

* NSX,UI: Deduplicate network list when creating kubernetes clusters (#8513)

* NSX: Make LB service selectable in network offering (#8512)

* NSX: Make LB service selectable in network offering

* fix label

* address comments

* address comments

* NSX: Add appropriate error message when icmp type is set to -1 for NSX (#8504)

* NSX: Add appropriate error message when icmp type is set to -1 for NSX

* address comments

* update text

* fix test

* fix test - build failure

* fix test - build failure

* NSX: Cleanup NSX resources during k8s cluster cleanup (#8528)

* fix test failure

* NSX: Improve segment deletion process (#8538)

* NSX: Add passive monitor for NSX LB to test whether a server is available (#8533)

* NSX: Add passive monitor for NSX LB to test whether a server is available

* Add active monitors too

* fix build failure

* NSX: Add check for ICMP code / type for NSX zones (#8542)

* NSX: Fix Routed Mode for Isolated and VPC networks (#8534)

* NSX: Fix Routed Mode for Isolated and VPC networks

* NSX: Fix Routed mode - add checks for ports added for FW rules

* clean up code

* fix build failure

* NSX: Add retry logic with sleep to delete segments (#8554)

* NSX: Add retry logic with sleep to delete segments

* add logs

* NSX: Fix custom ACL check (#2)

* NSX: Fix custom ACL check

* NSX: Fix custom ACL check

* Nsx vpc routed mode (#5)

* NSX: Fix VPC routed mode

* NSX: VPC route mode

* remove unnecessary changes

* Nsx: Support internal LB (#4)

* NSX: Support internal LB service in NSX

* add lb removal logic

* Fix UI issue hiding internal LB tab

* Refactor method name

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>

* NSX: Improve NSX resource cleanup process (#3)

* Fix unit test

* NSX: Add SourceNAT service to the default Routed offering for VPC (#13)

* Fix VPC restart with cleanup (#12)

* NSX: Fix ACL rule removal on replacement and fix rule order (#11)

* NSX: fix smoke test failure for ACLs (#9)

* Fix unit tests

* Fix NSX plugin pom XML

* NSX: Add support to re-order ACL rules (NSX FW rules) (#14)

* [WIP] NSX: Add support to re-order ACL rules (NSX FW rules)

* fix reordering of acl rules on all networks that it is associated to

* clean up and attempt test fix

* Fix tests

* Remove unused import

* tweak reorder logic

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>

* Fix zone creation issue for internal load balancer

* Fix

* Fix unit test

* fix logger

* fix logger

* fix logger

* NSX: Fix VPC form to ignore source NAT IP when creating VPCs and fix label

* Move SQL changes to the newest schema file

* NSX: Last Fixes

* Fix build

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>
2024-03-07 23:34:10 -03:00
Fabricio Duarte 9a73a2f9e1
Display dates in Quota and Usage messages according to the timezone configurations (#8230)
Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
2024-03-04 12:44:03 +01:00
Wei Zhou 240db19567 Merge remote-tracking branch 'apache/4.19' 2024-03-04 09:07:19 +01:00
Wei Zhou e86363be8e Merge remote-tracking branch 'apache/4.18' into 4.19 2024-03-04 09:06:53 +01:00
Wei Zhou 10f72a17f6
server: use ecdsa instead of ed25519 when generate public/private keys (#8719)
PR #8549 replaced RSA with ed25519. unfornately, ed25519 is unsupported in FIPS mode

```
$  ssh-keygen -t ed25519 -m PEM -N '' -f key1
ED25519 keys are not allowed in FIPS mode

$  ssh-keygen -t ecdsa -m PEM -N '' -f key1
Generating public/private ecdsa key pair.
Your identification has been saved in key1
Your public key has been saved in key1.pub
The key fingerprint is:
.........

```
2024-03-03 15:56:13 +01:00