Commit Graph

139 Commits

Author SHA1 Message Date
Daan Hoogland 0af923e618 Merge branch '4.19' 2024-04-24 14:01:02 +02:00
SadiJr 96ae479000
[Usage] Create network billing (#7236)
Co-authored-by: Bryan Lima <bryan.lima@hotmail.com>
Co-authored-by: SadiJr <sadi@scclouds.com.br>
Co-authored-by: Bryan Lima <42067040+BryanMLima@users.noreply.github.com>
Co-authored-by: Henrique Sato <henriquesato2003@gmail.com>
2024-04-24 08:52:49 +02:00
Wei Zhou bf9fdaddbd
Fix build errors due to log4j 2.x changes 2024-04-23 11:43:08 +02:00
Wei Zhou 7b57480d60
Merge remote-tracking branch 'apache/4.19' into main 2024-04-23 10:06:16 +02:00
SadiJr 49d244f251
[Usage] Create VPC billing (#7235)
Co-authored-by: SadiJr <sadi@scclouds.com.br>
Co-authored-by: Bryan Lima <bryan.lima@hotmail.com>
2024-04-22 09:42:24 +02:00
dahn e1922da249
merge forward error in logger name (#8949) 2024-04-19 14:21:18 +02:00
Daan Hoogland 3cabe69cd5 Merge release branch 4.19 to main
* 4.19:
  protect against null-path (#8915)
  UI: Fix missing locale strings for Status widget (#8792)
  Add a shutdownhook to remove jobs owned by the process (#8896)
2024-04-19 12:45:30 +02:00
Daan Hoogland 78e07cff62 Merge release branch 4.18 to 4.19
* 4.18:
  protect against null-path (#8915)
  UI: Fix missing locale strings for Status widget (#8792)
  Add a shutdownhook to remove jobs owned by the process (#8896)
2024-04-19 12:43:34 +02:00
dahn 5f8450f28f
Add a shutdownhook to remove jobs owned by the process (#8896)
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-04-19 09:18:41 +02:00
Rohit Yadav 01d44347e7 Merge remote-tracking branch 'origin/4.19'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

 Conflicts:
	plugins/network-elements/juniper-contrail/pom.xml
	pom.xml
	tools/marvin/setup.py

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-03-21 18:38:01 +05:30
Marcus Sorensen 2e88eb45a3
Update mysql-connector version (#8753)
Co-authored-by: Marcus Sorensen <mls@apple.com>
2024-03-21 18:09:06 +05:30
Abhishek Kumar bb244842fa Merge remote-tracking branch 'apache/4.19' 2024-03-15 15:18:54 +05:30
Abhishek Kumar 0780f0e5c6 Merge remote-tracking branch 'apache/4.18' into 4.19 2024-03-15 14:54:07 +05:30
Vishesh f8fd22c2d6
Usage: Fixup entity exists exception (#8726) 2024-03-12 09:08:41 +01:00
Fabricio Duarte 9a73a2f9e1
Display dates in Quota and Usage messages according to the timezone configurations (#8230)
Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
2024-03-04 12:44:03 +01:00
Fabricio Duarte d171582163
Fix false positives in Usage sanity checks of templates and network offerings (#8136)
Co-authored-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-02-26 21:14:24 +01:00
João Jandre 49cecaed06
Normalize loggers and upgrade log4j 1.2 to log4j 2.19 (#7131)
* Normalize logs

All classes that could have their loggers inherited from their fathers had their own loggers deleted;
Most loggers didn't have to be static, so most of them were normalized so that they wouldn't be;
All loggers are protected now;
Static logger's name are now 'LOGGER';
Non-static logger's name are now 'logger';
New class DbUpgradeAbstractImpl created so that all Upgraders extend it and inherit its logger

* Upgrade log4j

* fix errors caused by the merge

* Refactor cglibThrowableRenderer functionality to log4j2 and upgrade the last configuration files

* fix sonarcloud bug

* Fix errors caused by merge, remove some unused loggers, and rename a variable that was mistakenly renamed on the normalization commit

* Readd snmpTrapAppender, remove TestAppender

* Regenerate changes

* regenerate changes

* refactor last custom appender

* fix systemvm configuration xml

* Regenerate changes

* Regenerate changes

* regenerate changes

* Regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* Fix utils pom

* fix some tests

* regenerate changes

* Fix jar being printed on exception

* fix logging in system VMs, fix commands not having log4j2 classpath.

* regenerate changes

* Fix some unwanted renomeations

* fix end of file

* regenerate changes

* regenerate changes

* fix merge error

* regenerate changes

* fix tests

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* readd reload4j to tungsten as juniper depends on it

* Regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* re-add reload4j dependency to network-contrail, as juniper depends on it

* regenerate changes

* regenerate changes

* regenerate changes

* fix typo

* regenerate changes

* regenerate changes

* Fix end of files

* regenerate changes

* add logj42 to cloud-utils-SHADED.jar

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* Regenerate changes

* Regenerate changes

* Regenerate changes

* regenerate changes

* Regenerate changes

* regenerate changes

* Regenerate changes

* Regenerate changes

* Regenerate changes

* regenerate changes

* Regenerate changes

* Regenerate changes

* fix some tests

* Regenerate changes

* Regenerate changes

* fix test

* Regenerate changes

* Regenerate changes
2024-02-08 09:55:41 -03:00
Rakesh 0ba26912df
Fix typo in alert name (#8579)
Co-authored-by: Rakesh Venkatesh <rakeshv@apache.org>
2024-02-05 14:19:30 +05:30
John Bampton dda672503f
Remove unneeded duplicate words (#8358)
This PR removes some unneeded duplicate words.
2023-12-15 17:13:32 +05:30
kishankavala 5651eab49c
ObjectStore Framework with MinIO and Simulator plugins (#7752)
This PR adds Object Storage feature to CloudStack.

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/%5BDRAFT%5D+CloudStack+Object+Store
2023-12-01 17:51:00 +05:30
John Bampton f090c77f41
misc: fix spelling (#7549)
Co-authored-by: Stephan Krug <stekrug@icloud.com>
2023-11-02 09:23:53 +01:00
Vishesh 594c70dde0
Sync precommit config from main (#7732)
Co-authored-by: John Bampton <jbampton@users.noreply.github.com>
Co-authored-by: dahn <daan@onecht.net>
2023-07-07 11:18:16 +02:00
John Bampton c2e17310d6
Add three more `pre-commit` checks (#7083)
Co-authored-by: dahn <daan@onecht.net>
2023-03-27 13:28:55 +02:00
SadiJr ad64ee3090
[Usage] Fix wrong usage_type (#7229) 2023-03-03 13:10:38 +01:00
Wei Zhou 62e342c1bc
utils,framework/db: Introduce new database encryption cipher based on AesGcmJce (#7003) 2023-02-02 16:25:49 +01:00
Suresh Kumar Anaparti d8c7e34b38
Improve global settings UI to be more intuitive/logical (#5797)
Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
Co-authored-by: davidjumani <dj.davidjumani1994@gmail.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: dahn <daan@onecht.net>
2023-01-31 11:23:43 +01:00
dahn cdaad257ea
resolve sanity check last id file acces problems (#6825)
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2022-12-21 17:10:14 +01:00
Daniel Augusto Veronezi Salvador 2ca164ac96
Quota custom tariffs (#5909)
Co-authored-by: GutoVeronezi <daniel@scclouds.com.br>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2022-10-17 10:03:50 +02:00
dahn 53b658636f
fix usage charging for deleted/resized volumes (#6737) 2022-09-29 11:07:54 +02:00
Suresh Kumar Anaparti 571339f872
Merge branch '4.16' into main 2022-01-31 00:10:03 +05:30
Nicolas Vazquez 1fc7d70da2
Add idempotent primary keys on tables missing them (#5785)
* In progress primary keys

* Refactor in progress to idempotent way

* Finish SQL changes

* Add java code to match new columns

* Fix imports

* Fix tests

* Remove comments

* Fix index name on vmsnapshot

* Fix parse from correct column on usage storage

* Fix parser columns

* Fix NPE

* Fix NPE for the rest of the occurrences

* Further fix for similar issue
2022-01-30 23:31:07 +05:30
Suresh Kumar Anaparti 982eef202f
Merge branch '4.16' into main 2022-01-26 12:21:24 +05:30
sureshanaparti 06b5e84182
Removed redundant parsing of VMSnapshot usage record. (#5872) 2022-01-24 18:06:02 +01:00
José Flauzino 80cb361b97
Handle NullPointerException when sending email alerts (#5649)
* Handle NullPointerException when sending email

* Improve the log message related to the recipient list is empty

Co-authored-by: José Flauzino <jose@scclouds.com.br>
2021-11-23 16:53:28 +05:30
Daniel Augusto Veronezi Salvador b4aabadc4d
Replace string libraries with org.apache.commons.lang3.StringUtils (#5386)
* Replace google lib for lang3 and adjust methods calls

* Replace string libs by lang3

* Prohibit others string libs

Co-authored-by: GutoVeronezi <daniel@scclouds.com.br>
2021-11-18 13:41:48 +05:30
Wei Zhou a1942b4e8e
usage: updateNewMaxId after sanity check (#5646) 2021-11-01 23:05:03 -03:00
Rakesh 3c2360c9d2
VM has wrong network statistics with multiple nics in shared networks (#4741)
If vm has multiple nics belonging to different shared networks then
wrong statistics will be collected since network id is not considred
as primary key. Make the change so that primary key contains network
id so that traffic belonging to that corresponding network is shown

If network id is not added to primary key then all the traffic of all
shared networks will show up in one nic.

Co-authored-by: Rakesh Venkatesh <rakeshv@apache.org>
2021-09-14 23:54:34 -03:00
davidjumani a000361238
Merge remote-tracking branch 'apache/4.15' into main 2021-09-03 16:55:10 +05:30
Wei Zhou 7a7e449af6
usage: fix backup usage (#5259)
when creating usage record for backup usages in a period, we need to check if vm offering is changed. if vm offering is changed, create a new cloud usage record, otherwise update existing usage record.

This PR fixes #4982
2021-09-03 13:29:56 +05:30
Daniel Augusto Veronezi Salvador e962f0f271
Unify SMTP mail sending (#4954)
* Add mail dependencies

* Create util to send SMTP mail

* Add unit tests to SMTP mail sender

* Use SMTP mail util on quota alert

* Use SMTP mail util on alert

* Use SMTP mail util on project

* Use SMTP mail util on usage alert

* Remove copyright line in license header

Co-authored-by: Gabriel Beims Bräscher <gabrascher@gmail.com>

* Remove copyright line in license header

Co-authored-by: Gabriel Beims Bräscher <gabrascher@gmail.com>

* Remove copyright line in license header

Co-authored-by: Gabriel Beims Bräscher <gabrascher@gmail.com>

* Remove copyright line in license header

Co-authored-by: Gabriel Beims Bräscher <gabrascher@gmail.com>

* Remove copyright line in license header

Co-authored-by: Gabriel Beims Bräscher <gabrascher@gmail.com>

Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
Co-authored-by: Gabriel Beims Bräscher <gabrascher@gmail.com>
2021-06-15 23:18:26 +02:00
Olivier Lemasle 5f8289ffe9
Re-enable IP address usage hiding (#4327) 2020-11-07 10:42:44 +01:00
div8cn f66ae34ea7
Fix Usage failed to get pid (#4144)
* Update UsageManagerImpl.java

When System.getProperty Failure to get the PID will cause the service to stay in the config phase. The startup cannot continue and there will be no log output
Use managementfactory to get the PID and throw an exception when the PID cannot be obtained.

* Update usage/src/main/java/com/cloud/usage/UsageManagerImpl.java

Co-authored-by: dahn <daan.hoogland@gmail.com>

* Update UsageManagerImpl.java

* Update UsageManagerImpl.java

* Update UsageManagerImpl.java

Co-authored-by: dahn <daan.hoogland@gmail.com>
2020-10-28 15:49:53 +05:30
Rodrigo D. Lopez d676ffa0a0
Handle with VM snapshot events (#4251)
Co-authored-by: Rodrigo <rodrigo@scclouds.com.br>
2020-10-16 09:19:44 +02:00
Spaceman1984 b586eb22f1
Human readable sizes in logs (#4207)
This PR adds outputting human readable byte sizes in the management server logs, agent logs, and usage records. A non-dynamic global variable is added (display.human.readable.sizes) to control switching this feature on and off. This setting is sent to the agent on connection and is only read from the database when the management server is started up. The setting is kept in memory by the use of a static field on the NumbersUtil class and is available throughout the codebase.

Instead of seeing things like:
2020-07-23 15:31:58,593 DEBUG [c.c.a.t.Request] (AgentManager-Handler-12:null) (logid:) Seq 8-1863645820801253428: Processing: { Ans: , MgmtId: 52238089807, via: 8, Ver: v1, Flags: 10, [{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-224-VM","bytesSent":"106496","bytesReceived":"0","result":"true","details":"","wait":"0",}}] }

The KB MB and GB values will be printed out:

2020-07-23 15:31:58,593 DEBUG [c.c.a.t.Request] (AgentManager-Handler-12:null) (logid:) Seq 8-1863645820801253428: Processing: { Ans: , MgmtId: 52238089807, via: 8, Ver: v1, Flags: 10, [{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-224-VM","bytesSent":"(104.00 KB) 106496","bytesReceived":"(0 bytes) 0","result":"true","details":"","wait":"0",}}] }

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Human+Readable+Byte+sizes
2020-08-13 15:55:16 +05:30
Rodrigo D. Lopez 7b5700a040
usage: refactor on UsageManagerImpl.createVPNUserEvent (#4085)
Refactor on UsageManagerImpl.createVPNUserEvent. Currently, the present method creates and removes a VPN user event. So this PR abstract this method into an handle, and creates methods to create and delete VPN user events.

Additionally, the new created method prevents duplicated entries for same user, with same userId, domainId and zoneId
2020-06-18 09:11:51 +05:30
Rohit Yadav b54d19b3b9
server: fix TransactionLegacy DB connection leaks due to DB switching by B&R thread (#4121)
BackupSync task would switch between databases to update backup usage
metrics in the cloud_usage.usage_backup table. The current framework
and the usage in ManagedContext causes database connection
(LegacyTransaction) leaks. When the thread runs faster, the issue is
easily reproducible and checking via heap dump analysis or using JMX
MBeans. This fixes by moving the task of backup data updation for
usage data to the usage server by publishing usage events instead of
switching between databases in a local thread while in a
ManagedContextRunnable.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-06-16 13:30:31 +05:30
Rohit Yadav 318924d801
CloudStack Backup & Recovery Framework (#3553) 2020-03-03 13:27:58 +01:00
Rohit Yadav d90341ebf1
cloudstack: add JDK11 support (#3601)
This adds support for JDK11 in CloudStack 4.14+:

- Fixes code to build against JDK11
- Bump to Debian 9 systemvmtemplate with openjdk-11
- Fix Travis to run smoketests against openjdk-11
- Use maven provided jdk11 compatible mysql-connector-java
- Remove old agent init.d scripts

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-02-12 12:58:25 +05:30
Nicolas Vazquez bfc08715cc Display VM snapshot tags on usage records (#3560)
* Refactor usage helper tables to include VM snapshot id

* Fix resource type and resource id while listing usage records

* Add defensive checks
2019-08-20 14:20:23 +01:00
Rohit Yadav 202777e52a
server: publish volume resize event for volumes (#3219)
Problem: The disk offering change is not reflected in cloud_usage database table.
Root Cause: The resizeVolume API does not publish the volume disk offering change event to the
cloud_usage database table.
Solution: This issue has been fixed by refactoring the resizeVolume API to publish this disk offering change for volumes that either in Allocated or Ready state.

Moves the method that published events for volumes in Ready state from
the VolumeStateListener class to the orchestrateResizeVolume method in
the VolumeApiService.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-21 13:28:08 +05:30