etan39
/
cloudstack
mirror of https://github.com/apache/cloudstack.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
36,853 Commits 332 Branches 325 Tags 799 MiB
Commit Graph

1740 Commits

Author SHA1 Message Date
Daan Hoogland cb9b3134f7 Merge branch '4.19' 2024-06-14 10:30:10 +02:00
Rohit Yadav 2ca0857bd5
api: listVM API improvement followup, change returning of stats detail (#9177) 
- Changes behaviour of details param handling via global setting:
  - listVirtualMachines API: when the details param is not provided, it returns whether stats are returned controlled by a new global setting `list.vm.default.details.stats`
  - listVirtualMachinesMetrics API: when the details param is not provided, it uses `all` details including `stats`
- Users who are affected slow performance of the listVirtualMachines API response time can set `list.vm.default.details.stats` to `false`
- Remove ConfigKey vm.stats.increment.metrics.in.memory which was renamed to `vm.stats.increment.metrics` in #5984 and also remove unused/unnecessary global settings via upgrade path
- Changes default value of VM stats accumulation setting `vm.stats.increment.metrics` to false until a better solution emerges. Since #5984, this is true and during the execution of listVM APIs the stats are clubbed/calculated which can immensely slow down list VM API calls. Any costly operations such as summing of stats shouldn't be done during the course of a synchronous API, such as the list VM API.
- Fix UI that uses listVirtualMachinesMetrics to not call `stats` detail when in list view without metrics selected.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
 2024-06-14 11:03:14 +05:30
Rohit Yadav 0f8a839a1d
engine/schema: force index in user_vm_view to speed up VM instance listing (#9198) 
The user_vm_view can end up not picking the right index to join against
the user_ip_address table causing full table scan on the user_ip_address
table. This could be related to a MySQL bug
https://bugs.mysql.com/bug.php?id=41220

In a test environment with 20k shared networks and over 20M IPs, the
listVirtualMachines API was found to take over 17s to return list of
just 10 VMs. However, with this fix it would now take under 200ms to
return the list.

MySQL slow query logging showed ~nearly 20M table scans of the IP
address table:
```
# User@Host: cloud[cloud] @ localhost [127.0.0.1]  Id:    39
# Query_time: 8.227541  Lock_time: 0.000014 Rows_sent: 12  Rows_examined: 19,667,235
SET timestamp=1715410270;
SELECT user_vm_view.id, user_vm_view.name /*snipped*/ FROM user_vm_view
WHERE user_vm_view.id IN (4,6,7,8,9,10,11,12,13,14,15,16);
```

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
 2024-06-13 12:29:11 +05:30
Vishesh 6fe835e118
Merge branch '4.19' 2024-06-13 11:57:55 +05:30
Wei Zhou b2ef53b8a2
kvm: replace ISO path in vm XML configuration during vm migration (#9212) 
* kvm: replace ISO path in vm XML configuration during vm migration

* Update 9212: address comments

* kvm: fix vm migration if there are multiple image stores
 2024-06-12 16:01:23 +02:00
dahn 4de975ff25
list by isEncrypted (#8643) 
* list by isEncrypted

* use filter on VO and cleanup

* add encryption type to volume response

* Update api/src/main/java/org/apache/cloudstack/api/command/user/volume/ListVolumesCmd.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
 2024-06-12 15:11:00 +05:30
Bernardo De Marco Gonçalves 1383625c93
Fix `*.smtp.useAuth`, `quota.usage.smtp.useStartTLS` and `*.smtp.enabledSecurityProtocols` settings definitions (#9031) 
* change configs definitions

* add normalization query

* add ui support

* add labels

* add end of line to SQL script
 2024-06-11 10:54:03 +03:00
Abhishek Kumar fcca3e8f39
api,server: encryptformat in volume response (#9106) 
Fixes #9014

* api,server: encryptformat in volume response

* Apply suggestions from code review

* remove passphrase db change
 2024-06-11 11:20:50 +05:30
Abhishek Kumar 21d07132ad
engine-schema: make adding/dropping indexes idempotent (#9171) 
* engine-schema: make adding/dropping indexes idempotent

Fixes #9170

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
 2024-06-10 16:32:30 +05:30
Vishesh cc8dc84f64
server: fix resource reservation leakage (#9169) 
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
 2024-06-10 12:30:06 +05:30
Abhishek Kumar be552fdce9
feature: webhooks (#8674) 
* api,server,ui: weebhoks feature

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* registry of message busses

* test bus

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix and refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for webhook dispatch history

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes, initial ui

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* improvements

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for account webhook cleanup

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix remaining event bus usage

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for testing webhook dispatch

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wip

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* make element

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* missing

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* buid fix

* fix lint

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for project delete check

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add collapse in create

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui fix and refactor for eventditributor publish

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* update org.json and add json validation

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* schema fixes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wordings

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: improve progress button

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui improvements

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove unrelated change

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* search and count

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add payloadurl in info

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* positive progress

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix hmac key

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* create webhook form fixes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor, address feedback

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* indentation

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix filters

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove test eventbus

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* default scope be Local

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add lifecycle smoke test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add test for webhook deliveries

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix lint

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor - losgs and others

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* unit tests

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix lint

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* build fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* smoke test fix, log refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* get bean from all components

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: missing label

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* address review comments

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add some more tests

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* lint

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* rename setting

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* upgrade: move 4.19.0->4.20.0 to 4.19.1->4.20.0

* fix test delivery layout

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix webhook secret display

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add http to payloadurl when no scheme

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* allow removing secretkey for webhook

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix update sslverification

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* disallow same payload url for same account

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix delivery with url w/o scheme

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* api: listApis should return params based on caller

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wip changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Update engine/schema/src/main/resources/META-INF/db/schema-41900to42000.sql

* remove unique constraint for now

Constraint is present in Java code validations

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fixes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: add option to delete multiple deliveries

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add filter for deliveries, delete api start/endtime support

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* do not throw error when no deliveries removed

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: fix deliveries table column sorting, time filter cancel

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove isDebugEnabled wrapping

* merge fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Daan Hoogland <daan@onecht.net>
Co-authored-by: Wei Zhou <weizhou@apache.org>
 2024-06-10 10:40:12 +05:30
Daan Hoogland b29391c329 Merge LTS branch '4.19' into main 2024-06-07 09:53:57 +02:00
Abhishek Kumar 91c7bc722f
server,cks: check if vm is cks node during vm destroy (#9057) 
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
 2024-06-06 10:54:02 +02:00
Wei Zhou 9b3c318db6
upgrade: move 4.19.0->4.20.0 to 4.19.1->4.20.0 (#8893) 
This PR move upgrade files for 4.19.0->4.20.0 to 4.19.1->4.20.0
 2024-06-03 11:42:28 +05:30
Wei Zhou 5433e775e5
New feature: Implicit host tags (#8929) 
* Merge two HostTagVO and HostTagDaoImpl

* Implicit host tags

* PR8929: add since

* Update variable names

* Update 8929: add unit test in LibvirtComputingResourceTest

* Update 8929: add explicithosttags in response

* Update 8929 UI: Update explicit host tags

* Update 8929: remove host tags and change labels on UI

* Update 8929: update host_view to use explicit_host_tags.is_tag_a_rule

* Update: ui polish for host tags

* Update 8929: fix UI error if no host tags
 2024-05-30 17:21:13 +05:30
Daan Hoogland f1c3d2c4be Merge release branch 4.19 to main 
* 4.19:
  engine-schema: add removed columne for cloud.user_data, fix delete op (#9120)
  ui: fix create menu item access (#9104)
  server: honor listall param for listiso api (#9064)
  ui: add move to top/bottom for acl rules list (#9119)
  ui: fix columns for exportacls csv (#9118)
  ui: fix create network access in deploy vm wizard (#9117)
  UI: Add search filters (#9068)
 2024-05-28 09:44:05 +02:00
Abhishek Kumar c970141e25
engine-schema: add removed columne for cloud.user_data, fix delete op (#9120) 
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
 2024-05-28 09:43:14 +02:00
Abhishek Kumar acce88ff39 Merge remote-tracking branch 'apache/4.19' 2024-05-27 15:14:29 +05:30
Abhishek Kumar daf6b9d103
api,ui: vm template format, fix vm info link (#9094) 2024-05-24 17:39:52 +02:00
Daan Hoogland e61f3bae4d Merge branch '4.19' 2024-04-29 11:37:40 +02:00
Rohit Yadav 3de1f8b4ba Merge remote-tracking branch 'origin/4.18' into 4.19 
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
 2024-04-29 13:44:34 +05:30
Wei Zhou 0e08a126df
systemvm: add template_zone_ref record when add a new zone with same hypervisor type (#8395) 2024-04-25 08:41:59 +02:00
Daan Hoogland 0af923e618 Merge branch '4.19' 2024-04-24 14:01:02 +02:00
SadiJr 96ae479000
[Usage] Create network billing (#7236) 
Co-authored-by: Bryan Lima <bryan.lima@hotmail.com>
Co-authored-by: SadiJr <sadi@scclouds.com.br>
Co-authored-by: Bryan Lima <42067040+BryanMLima@users.noreply.github.com>
Co-authored-by: Henrique Sato <henriquesato2003@gmail.com>
 2024-04-24 08:52:49 +02:00
Wei Zhou bf9fdaddbd
Fix build errors due to log4j 2.x changes 2024-04-23 11:43:08 +02:00
Wei Zhou 7b57480d60
Merge remote-tracking branch 'apache/4.19' into main 2024-04-23 10:06:16 +02:00
dahn 6502dde8c3
field enlarged and db upgrade (#8675) 
* 4.19 -> 4.19.1 and enlarge url field
 2024-04-23 13:01:08 +05:30
SadiJr 49d244f251
[Usage] Create VPC billing (#7235) 
Co-authored-by: SadiJr <sadi@scclouds.com.br>
Co-authored-by: Bryan Lima <bryan.lima@hotmail.com>
 2024-04-22 09:42:24 +02:00
João Jandre 8a101fbbc1 Updating pom.xml version numbers for release 4.18.3.0-SNAPSHOT 
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
 2024-04-17 11:11:57 -03:00
Vishesh ebaf5a47b9
Speedup resource count calculation (#8903) 
* Speed up resource count calculation

* Refactor resource count calculation

* Start transaction for updateCountByDeltaForIds
 2024-04-17 14:21:30 +05:30
João Jandre 154566f914 Updating pom.xml version numbers for release 4.18.2.0 
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
 2024-04-12 08:25:04 -03:00
Vishesh 19f79b1d94
Merge branch '4.19' 2024-04-12 16:31:40 +05:30
Vishesh 730cc5d5b8
Change iops on offering change (#8872) 
* Change IOPS on disk offering change

* Remove iops & bandwidth limits before copying template

* minor refactor

* Handle diskOfferingDetails

* Fixup
 2024-04-11 17:01:55 +05:30
Abhishek Kumar 55b28d645f Merge remote-tracking branch 'apache/4.19' 2024-04-04 17:54:42 +05:30
Abhishek Kumar 02305fbc5f Merge remote-tracking branch 'apache/4.19' 2024-04-04 17:36:05 +05:30
Vishesh b8f8321f0f
Allow listing of inactive offerings (#8821) 2024-04-04 13:46:44 +02:00
Rohit Yadav f2da882d68 Merge remote-tracking branch 'origin/4.18' into 4.19 2024-04-04 14:58:50 +05:30
Wei Zhou 7a9985b42b
upgrade: add unit tests from/to a security release (#8870) 
* upgrade: add unit tests from/to a security release

* upgrade: add unit test from a old security release to a new security release
 2024-04-04 14:55:29 +05:30
Daniel Augusto Veronezi Salvador 9072f8ee1c
Add ReservationDao bean to the Usage Server (#8868) 2024-04-04 09:59:48 +02:00
Vishesh 93f31828f5
Hypervisor as class (#8798) 
* HypervisorType as a class

* Fixup

* fixup

* Add missing annotation

* Resolve comments

* Handle parallels typo

* Add missing @Convert annotation
 2024-04-02 17:36:15 +05:30
Wei Zhou 4b4dfef96f
fix simulator CI failure test_04_list_storage_pools_keyword_filter (#8814) 2024-03-21 22:09:53 +01:00
Rohit Yadav 01d44347e7 Merge remote-tracking branch 'origin/4.19' 
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

 Conflicts:
	plugins/network-elements/juniper-contrail/pom.xml
	pom.xml
	tools/marvin/setup.py

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
 2024-03-21 18:38:01 +05:30
Marcus Sorensen 2e88eb45a3
Update mysql-connector version (#8753) 
Co-authored-by: Marcus Sorensen <mls@apple.com>
 2024-03-21 18:09:06 +05:30
Vishesh 24d5c45f7f
Add indexes for vm_stats (#8737) 
* Add indexes for vm_stats

* Remove index on timestamp

* Chnage index from vm_id,timestamp to vm_id
 2024-03-21 14:05:47 +05:30
Vishesh 33dc7465c2
Merge remote-tracking branch 'origin/4.19' 2024-03-19 15:18:02 +05:30
Vishesh 0043540fa3
Use join instead of views (#8321) 2024-03-18 18:08:19 +01:00
João Jandre f76c6f3ea9
Quota email configuration (#8307) 
* Quota email configuration feature
 2024-03-18 08:26:41 -03:00
nvazquez db564b12b6
Merge branch '4.19' 2024-03-14 22:46:48 -03:00
Vishesh e87c6cfcb1
Fix resource count discrepancies (#8302) 
* Fix resource count discrepancies

* Fixup while removing vm

* Fix discrepancies when starting VMs

* Fixup tests

* Fix failing tests

* Don't take lock when amount is negative

---------

Co-authored-by: dahn <daan@onecht.net>
 2024-03-13 18:22:44 +05:30
Daan Hoogland d99b1b9c2d Merge branch '4.18' into 4.19 2024-03-08 08:19:49 +01:00