Commit Graph

529 Commits

Author SHA1 Message Date
dahn 58466c8954 api: add command to list management servers (#3150)
* api: add command to list management servers

* api: add number of mangement servers in listInfrastructure command

* ui: add block for mangement servers on infra page

* api name resolution method cleanup
2019-02-01 16:23:39 -02:00
dahn b363fd49f7 Vmware offline migration (#2848)
* - Offline VM and Volume migration on Vmware hypervisor hosts
- Also add VM disk consolidation call on successful VM migrations

* Fix indentation of marvin test file and reformat against PEP8

* * Fix few comment typos
* Refactor debug messages to use String.format() when debug log level is enabled.

* Send list of commands returned by hypervisor Guru instead of explicitly selecting the first one

* Fix unhandled NPE during VM migration

* Revert back to distinct event descriptions for VM to host or storage pool migration

* Reformat test_primary_storage file against PEP-8 and Remove unused imports

* Revert back the deprecation messages in the custom StringUtils class to favour the use of the ApacheUtils
2019-01-25 10:05:13 -02:00
GabrielBrascher e6149f6996 Fix null pointer exception when using ConfigKey.Scope.ManagementServer 2018-11-27 14:28:56 -02:00
Kui LIU d53fc94485 CLOUDSTACK-10365: Change the "getXXX" boolean method names to "isXXX" (#2847)
These boolean-return methods are named as "getXXX".
Other boolean-return methods are named as "isXXX".
Considering there methods will return boolean values, it should be more clear and consistent to rename them as "isXXX".
(rebase #2602 and #2816)
2018-09-22 17:20:48 +02:00
Khosrow Moossavi 7c6630bca7 Cleanup POMs (#2613)
* Cleaup and code-formatting POM files

* Remove obsolete mycila license-maven-plugin

* Remove obsolete console-proxy/plugin project

* Move console-proxy-rdbconsole under console-proxy parent

* Use correct parent path for rdpconsole

* Order alphabetally items in setnextversion.sh

* Unifiy License header in POMs

* Alphabetic order of modules definition

* Extract all defined versions into parent pom

* Remove obsolete files: version-info.in, configure-info.in

* Remove redundant defaultGoal

* Remove useless checkstyle plugin from checkstyle project

* Order alphabetally items in pom.xml

* Add aditional SPACEs to fix debian build

* Don't execute checkstyle on parent projects

* Use UTF-8 encoding in building checkstyle project

* Extract plugin versions into properties

* Execute PMD plugin on all the projects with -Penablefindbugs

* Upgrade maven plugins to latest version

* Make sure to always look for apache parent pom from repository

* Fix incorrect version grep in debian packaging

* Fix rebase conflicts

* Fix rebase conflicts

* Remove PMD for now to be fixed on another PR
2018-07-25 14:39:37 -03:00
ernjvr 542d4da16c asyncjobs: add endtime to async jobs (#2739)
There is currently no functional mechanism that captures or persists the end time of when an asynchronous job has finished. As a result, users are not able to do any reporting about the duration of various asynchronous jobs in Cloudstack.
Link to FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Add+End+Time+To+Asynchronous+Jobs
2018-07-25 14:48:01 +05:30
Nicolas Vazquez 9fb36e7c73 db: Update Apache DBCP version (#2718)
* Update Apache DBCP version

* Fix DB connection

* Prevent hang on the db creation
2018-07-10 11:53:59 +05:30
Kui LIU 2c681b8c5f CLOUDSTACK-10363: refactor the "getXXX" and "listXXX" method names to "findXXX" (#2599)
These three methods are not direct getter or list.
They try to find the target objects with the related arguments.
So that, renaming them as "findXXX" should be more intuitive.
2018-05-01 20:44:43 +05:30
Kui LIU 93509a431c CLOUDSTACK-10360: Change the method name. (#2598)
The method is named as "scoped" that seems to whether the variable config is scoped in _scopedStorages or not. Actually, the method tries to find a storage of which scope equals to the scope of config. So that, the method name "findStorage" should be more clear than "scoped".
2018-04-30 09:24:47 +02:00
Marc-Aurèle Brothier ca1760a46b CLOUDSTACK-10320 - Invalid pair for response object breaking response parsing (#2481) 2018-03-23 16:55:22 +00:00
nvazquez 1c99fd7388 Merge branch '4.11' 2018-03-21 08:12:59 -03:00
Nicolas Vazquez 6a75423779 CLOUDSTACK-10231: Asserted fixes for Direct Download on KVM (#2408)
Several fixes addressed:

- Dettach ISO fails when trying to detach a direct download ISO
- Fix for metalink support on SSVM agents (this closes CLOUDSTACK-10238)
- Reinstall VM from bypassed registered template (this closes CLOUDSTACK-10250)
- Fix upload certificate error message even though operation was successful
- Fix metalink download, checksum retry logic and metalink SSVM downloader
2018-03-20 19:24:46 +05:30
Rafael Weingärtner 8a3943b763 CLOUDSTACK-10132: Fix POM of "agent-lb" project (#2491)
Post fwd merging, this fixes build issues.
2018-03-16 08:04:22 +05:30
Rohit Yadav 8ef131745a Merge branch '4.11' 2018-03-15 16:46:50 +05:30
Rohit Yadav 30175d6879
CLOUDSTACK-10132: Extend support for management servers LB for agents (#2469)
The new CA framework introduced basic support for comma-separated
list of management servers for agent, which makes an external LB
unnecessary.

This extends that feature to implement LB sorting algorithms that
sorts the management server list before they are sent to the agents.
This adds a central intelligence in the management server and adds
additional enhancements to Agent class to be algorithm aware and
have a background mechanism to check/fallback to preferred management
server (assumed as the first in the list). This is support for any
indirect agent such as the KVM, CPVM and SSVM agent, and would
provide support for management server host migration during upgrade
(when instead of in-place, new hosts are used to setup new mgmt server).

This FR introduces two new global settings:

- `indirect.agent.lb.algorithm`: The algorithm for the indirect agent LB.
- `indirect.agent.lb.check.interval`: The preferred host check interval
  for the agent's background task that checks and switches to agent's
  preferred host.

The indirect.agent.lb.algorithm supports following algorithm options:

- static: use the list as provided.
- roundrobin: evenly spreads hosts across management servers based on
  host's id.
- shuffle: (pseudo) randomly sorts the list (not recommended for production).

Any changes to the global settings - `indirect.agent.lb.algorithm` and
`host` does not require restarting of the mangement server(s) and the
agents. A message bus based system dynamically reacts to change in these
global settings and propagates them to all connected agents.

Comma-separated management server list is propagated to agents on
following cases:
- Addition of a host (including ssvm, cpvm systevms).
- Connection or reconnection by the agents to a management server.
- After admin changes the 'host' and/or the
  'indirect.agent.lb.algorithm' global settings.

On the agent side, the 'host' setting is saved in its properties file as:
`host=<comma separated addresses>@<algorithm name>`.

First the agent connects to the management server and sends its current
management server list, which is compared by the management server and
in case of failure a new/update list is sent for the agent to persist.

From the agent's perspective, the first address in the propagated list
will be considered the preferred host. A new background task can be
activated by configuring the `indirect.agent.lb.check.interval` which is
a cluster level global setting from CloudStack and admins can also
override this by configuring the 'host.lb.check.interval' in the
`agent.properties` file.

Every time agent gets a ms-host list and the algorithm, the host specific
background check interval is also sent and it dynamically reconfigures
the background task without need to restart agents.

Note: The 'static' and 'roundrobin' algorithms, strictly checks for the
order as expected by them, however, the 'shuffle' algorithm just checks
for content and not the order of the comma separate ms host addresses.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-15 16:34:03 +05:30
Rohit Yadav 0ece15f86e Updating pom.xml version numbers for release 4.11.1.0-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-02-26 16:57:48 +01:00
Rohit Yadav 6ffbce6159 Updating pom.xml version numbers for release 4.11.0.1-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-02-05 11:13:50 +01:00
Rohit Yadav 5dada1f7ed Updating pom.xml version numbers for release 4.11.0.0
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-01-26 13:13:37 +01:00
Marc-Aurèle Brothier 893a88d225 CLOUDSTACK-10105: Use maven standard project structure in all projects (#2283)
Remove maven standard module (which only a few were using) and get ride of maven customization for the projects structure.

- moved all directories to src/main/java, src/main/resources, src/main/scripts, src/test/java, src/test/resources
- grep scan to search for src/com and src/org left over
- grep for <project>/scripts to fix pom.xml configuration
- remove custom <build> configuration in pom.xml

Signed-off-by: Marc-Aurèle Brothier <m@brothier.org>
2018-01-20 03:19:27 +05:30
Rohit Yadav 072dbc0720 Updating pom.xml version numbers for master to 4.12.0.0-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-01-15 17:43:45 +05:30
Abhinandan Prateek 64832fd70a CLOUDSTACK-4757: Support OVA files with multiple disks for templates (#2146)
CloudStack volumes and templates are one single virtual disk in case of XenServer/XCP and KVM hypervisors since the files used for templates and volumes are virtual disks (VHD, QCOW2). However, VMware volumes and templates are in OVA format, which are archives that can contain a complete VM including multiple VMDKs and other files such as ISOs. And currently, Cloudstack only supports Template creation based on OVA files containing a single disk. If a user creates a template from a OVA file containing more than 1 disk and launches an instance using this template, only the first disk is attached to the new instance and other disks are ignored.
Similarly with uploaded volumes, attaching an uploaded volume that contains multiple disks to a VM will result in only one VMDK to being attached to the VM.

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Support+OVA+files+containing+multiple+disks

This behavior needs to be improved in VMWare to support OVA files with multiple disks for both uploaded volumes and templates. i.e. If a user creates a template from a OVA file containing more than 1 disk and launches an instance using this template, the first disk should be attached to the new instance as the ROOT disk and volumes should be created based on other VMDK disks in the OVA file and should be attached to the instance.

Signed-off-by: Abhinandan Prateek <abhinandan.prateek@shapeblue.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-01-10 22:10:41 +05:30
Nicolas Vazquez e86bb41e0e CLOUDSTACK-10146: Bypass Secondary Storage for KVM templates (#2379)
This feature allows using templates and ISOs avoiding secondary storage as intermediate cache on KVM. The virtual machine deployment process is enhanced to supported bypassed registered templates and ISOs, delegating the work of downloading them to primary storage to the KVM agent instead of the SSVM agent.

Template and ISO registration:
- When hypervisor is KVM, a checkbox is displayed with 'Direct Download' label.
- API methods registerTemplate and registerISO are both extended with this new parameter directdownload.
- On template or ISO registration, no download job is sent to SSVM agent, CloudStack would only persist an entry on template_store_ref indicating that template or ISO has been marked as 'Direct Download' (bypassing Secondary Storage). These entries are persisted as:
template_id = Template or ISO id on vm_template table
store_id NULL
download_state = BYPASSED
state = Ready
(Note: these entries allow users to deploy virtual machine from registered templates or ISOs)
- An URL validation command is sent to a random KVM host to check if template/ISO location can be reached. Metalink are also supported by this feature. In case of a metalink, it is fetched and URL check is performed on each of its URLs.
- Checksum should be provided as indicated on #2246: {ALGORITHM}CHKSUMHASH
- After template or ISO is registered, it would be displayed in the UI

Virtual machine deployment:
When a 'Direct Download' template is selected for deployment, CloudStack would delegate template downloading to destination storage pool via destination host by a new pluggable download manager.
Download manager would handle template downloading depending on URL protocol. In case of HTTP, request headers can be set by the user via vm_template_details. Those details should be persisted as:
Key: HTTP_HEADER
Value: HEADERNAME:HEADERVALUE

In case of HTTPS, a new API method is added uploadTemplateDirectDownloadCertificate to allow user importing a client certificate into all KVM hosts' keystore before deployment.
After template or ISO is downloaded to primary storage, usual entry would be persisted on template_spool_ref indicating the mapping between template/ISO and storage pool.
2018-01-09 12:22:18 +05:30
dahn 45df928e04 CLOUDSTACK-10117: Account ldap binding (#2381)
Map an ldap group to an account. Ldap related settings on a domain level.
2018-01-06 22:58:28 +05:30
PranaliM 7253969088 CLOUDSTACK-10065: Optimize SQL queries in listTemplate API to improve performance (#2260)
The db queries in listTemplateAPI could be optimized to get unique results from the database which could help in reducing the listTemplate API response time.
2018-01-05 11:27:41 +05:30
Abhinandan Prateek 391952da5b CLOUDSTACK-9867: VM snapshot on primary storage usage metrics (#2035)
VM snapshot on primary storage usage metrics.
2017-12-28 14:57:10 +05:30
Rohit Yadav facc5945f0 CLOUDSTACK-10193: Fix smoke tests failures with new systemvmtemplate
- Several systemvmtemplate optimizations
- Uses new macchinina template for running smoke tests
- Switch to latest Debian 9.3.0 release for systemvmtemplate
- Introduce a new `get_test_template` that uses tiny test template
  such as macchinina as defined test_data.py
- rVR related fixes and improvements

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-12-23 09:22:44 +05:30
Rafael Weingärtner 9179bd54d4 CLOUDSTACK-10195: CloudStack MySQL HA problem - No database selected (#2364)
When using CloudStack with database HA configuration user receives warnings constantly saying “No database selected”.

This problem happens at com.cloud.cluster.ClusterManagerImpl.getHeartbeatTask().new ManagedContextRunnable(){...}.runInContext(), line 550. The scheme of the database is not properly set in the transaction when database HA options are enabled.
2017-12-19 17:55:13 +05:30
Rafael Weingärtner 3c6df7c970 CLOUDSTACK-10196: Remove ejb-api 3.0 dependency (#2348)
Fixed QuotaAlertManagerImplTest, which was injecting mock objects manually.
2017-12-18 23:59:06 +05:30
Marc-Aurèle Brothier f0eabad16d Fix GroupBy (+ having) condition and tests (#1723)
The GroupBy + having isn't used currently in the code but was not clean.
It removes unused arguments and variables and adds a test based on a DAO
to show a full example on how to use it.

Signed-off-by: Marc-Aurèle Brothier <m@brothier.org>
2017-11-09 21:01:12 +05:30
Rohit Yadav eda3b35bfa CLOUDSTACK-10012: Migrate to Embedded Jetty
- Migrate to embedded Jetty server.
- Improve ServerDaemon implementation.
- Introduce a new server.properties file for easier configuration.
- Have a single /etc/default/cloudstack-management to configure env.
- Reduce shaded jar file, removing unnecessary dependencies.
- Upgrade to Spring 5.x, upgrade several jar dependencies.
- Does not shade and include mysql-connector, used from classpath instead.
- Upgrade and use bountcastle as a separate un-shaded jar dependency.
- Remove tomcat related configuration and files.
- Have both embedded UI assets in uber jar and separate webapp directory.
- Refactor systemd and init scripts, cleanup packaging.
- Made cloudstack-setup-databases faster, using `urandom`.
- Remove unmaintained distro packagings.
- Moves creation and usage of server keystore in CA manager, this
  deprecates the need to create/store cloud.jks in conf folder and
  the db.cloud.keyStorePassphrase in db.properties file. This also
  remove the need of the --keystore-passphrase in the
  cloudstack-setup-encryption script.
- GZip contents dynamically in embedded Jetty

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-11-03 23:57:25 +05:30
Harika Punna 6bb0ca2f85 This feature separates the snapshot creation on primary and its backing up on secondary.
As part of this, a new parameter, which is optional, is added to CreateSnapshotCmd, which seperates the creation and backup.

More details in the FS-
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Separate+creation+and+backup+operations+for+a+volume+snapshot
2017-10-04 14:39:03 +05:30
Nitin Kumar Maharana a52ce3628c CLOUDSTACK-10058: Error while opening the Settings tab in Secondary storage (#2254)
Root Cause:
Some global parameters contains NULL value, where the code doesn't handle NULL check.
So it fails with an exception. Hence nothing appears on the field(ERROR).

Solution:
Added required NULL check.
2017-09-01 13:13:39 +02:00
Rohit Yadav 7ce54bf7a8 CLOUDSTACK-9993: Securing Agents Communications (#2239)
This introduces a new certificate authority framework that allows
pluggable CA provider implementations to handle certificate operations
around issuance, revocation and propagation. The framework injects
itself to `NioServer` to handle agent connections securely. The
framework adds assumptions in `NioClient` that a keystore if available
with known name `cloud.jks` will be used for SSL negotiations and
handshake.

This includes a default 'root' CA provider plugin which creates its own
self-signed root certificate authority on first run and uses it for
issuance and provisioning of certificate to CloudStack agents such as
the KVM, CPVM and SSVM agents and also for the management server for
peer clustering.

Additional changes and notes:
- Comma separate list of management server IPs can be set to the 'host'
  global setting. Newly provisioned agents (KVM/CPVM/SSVM etc) will get
  radomized comma separated list to which they will attempt connection
  or reconnection in provided order. This removes need of a TCP LB on
  port 8250 (default) of the management server(s).
- All fresh deployment will enforce two-way SSL authentication where
  connecting agents will be required to present certificates issued
  by the 'root' CA plugin.
- Existing environment on upgrade will continue to use one-way SSL
  authentication and connecting agents will not be required to present
  certificates.
- A script `keystore-setup` is responsible for initial keystore setup
  and CSR generation on the agent/hosts.
- A script `keystore-cert-import` is responsible for import provided
  certificate payload to the java keystore file.
- Agent security (keystore, certificates etc) are setup initially using
  SSH, and later provisioning is handled via an existing agent connection
  using command-answers. The supported clients and agents are limited to
  CPVM, SSVM, and KVM agents, and clustered management server (peering).
- Certificate revocation does not revoke an existing agent-mgmt server
  connection, however rejects a revoked certificate used during SSL
  handshake.
- Older `cloudstackmanagement.keystore` is deprecated and will no longer
  be used by mgmt server(s) for SSL negotiations and handshake. New
  keystores will be named `cloud.jks`, any additional SSL certificates
  should not be imported in it for use with tomcat etc. The `cloud.jks`
  keystore is stricly used for agent-server communications.
- Management server keystore are validated and renewed on start up only,
  the validity of them are same as the CA certificates.

New APIs:
- listCaProviders: lists all available CA provider plugins
- listCaCertificate: lists the CA certificate(s)
- issueCertificate: issues X509 client certificate with/without a CSR
- provisionCertificate: provisions certificate to a host
- revokeCertificate: revokes a client certificate using its serial

Global settings for the CA framework:
- ca.framework.provider.plugin: The configured CA provider plugin
- ca.framework.cert.keysize: The key size for certificate generation
- ca.framework.cert.signature.algorithm: The certificate signature algorithm
- ca.framework.cert.validity.period: Certificate validity in days
- ca.framework.cert.automatic.renewal: Certificate auto-renewal setting
- ca.framework.background.task.delay: CA background task delay/interval
- ca.framework.cert.expiry.alert.period: Days to check and alert expiring certificates

Global settings for the default 'root' CA provider:
- ca.plugin.root.private.key: (hidden/encrypted) CA private key
- ca.plugin.root.public.key: (hidden/encrypted) CA public key
- ca.plugin.root.ca.certificate: (hidden/encrypted) CA certificate
- ca.plugin.root.issuer.dn: The CA issue distinguished name
- ca.plugin.root.auth.strictness: Are clients required to present certificates
- ca.plugin.root.allow.expired.cert: Are clients with expired certificates allowed

UI changes:
- Button to download/save the CA certificates.

Misc changes:
- Upgrades bountycastle version and uses newer classes
- Refactors SAMLUtil to use new CertUtils

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-08-28 12:15:11 +02:00
pavanaravapalli a40e0f0097 CLOUDSTACK-9939: Volumes stuck in Creating state while restarting the Management Server when the volume creation is in progress (#2163) 2017-08-19 21:10:43 +02:00
Gabriel Beims Bräscher e0b6dcbe4a CLOUDSTACK-9925: Quota memory tariff value is for 1MB of RAM used per month (not hour) (#2119)
The quota memory tariff description in the CloudStack UI is wrong when defines that the value is for "using 1MB or RAM for 1 hour". The quota currency values reflect the value of a resource used per month, not an hour.

Quota divides the tariff value by the number of hours a month has (30 days - 720 hours); then it calculates the credits used by a client based on the amount of resources used per hour. The memory tariff description in the interface is wrong and can guide users to configure values for an hour.
2017-08-03 00:47:33 +02:00
Rohit Yadav be8ae9822d Merge branch '4.10' 2017-07-26 15:19:13 +02:00
Rohit Yadav c1f30d954b Merge branch '4.9' into 4.10 2017-07-26 15:18:42 +02:00
Wei Zhou 03aca885ae CLOUDSTACK-8186: make setRemoved to null possible 2017-07-26 14:57:23 +02:00
Rohit Yadav 01cbfe5d42 Merge remote-tracking branch 'origin/4.10' 2017-07-22 11:46:16 +02:00
Rohit Yadav 2b06f1ddaf Merge remote-tracking branch 'origin/4.9' into 4.10 2017-07-22 11:45:30 +02:00
Nathan Johnson 047bb1c74a CLOUDSTACK-9830 Fix DST bug in QuotaAlertManagerTest
Replace the use of joda with Calendar, set explicitly to UTC when
doing date manipulation.
2017-07-22 11:36:23 +02:00
Rajani Karuturi 4bc7c270fa Updating pom.xml version numbers for release 4.11.0.0-SNAPSHOT
Signed-off-by: Rajani Karuturi <rajanikaruturi@gmail.com>
2017-07-12 12:09:38 +05:30
Rajani Karuturi 4f229d19ad Updating pom.xml version numbers for release 4.10.1.0-SNAPSHOT
Signed-off-by: Rajani Karuturi <rajanikaruturi@gmail.com>
2017-07-10 10:36:24 +05:30
Rajani Karuturi 9d2893d44a Updating pom.xml version numbers for release 4.10.0.0
Signed-off-by: Rajani Karuturi <rajanikaruturi@gmail.com>
2017-07-03 10:06:43 +05:30
Rajani Karuturi 511ebe6dd4 Merge pull request #927 from karuturi/secure-configs
CLOUDSTACK-9901 secure and hidden config values are returned as plaintext string
2017-06-06 15:37:44 +05:30
Rajani Karuturi 1bd66cb03e Merge pull request #2072 from Accelerite/CLOUDSTACK-9895_ParallelVolumes
CLOUDSTACK-9895 : Added support for parallel volume(s) creation from a volume snapshot
2017-05-31 14:05:05 +05:30
Pavan Kumar Aravapalli 502f813370 CLOUDSTACK-9895 : Added support for parallel volume(s) creation from a volume snapshot 2017-05-31 11:27:30 +05:30
Rajani Karuturi fca41148b7 CLOUDSTACK-9901 secure and hidden config values are returned as plaintext string
secure and hidden config values are first unencrypted before returning
them in the api. This is not desired as they are secure configs
returning encrypted strings for secure and hidden configs if encryption
is enabled.
2017-05-03 14:58:02 +05:30
Daan Hoogland 61189db1d9 CE-113 configure workervm gc based on job expiry 2017-04-18 18:11:00 +02:00
Rohit Yadav 8b6e96bca9 Updating pom.xml version numbers for release 4.9.3.0-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-01-06 10:40:15 +05:30
Rohit Yadav dfc39c1f08 Updating pom.xml version numbers for release 4.9.2.0
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-01-03 12:28:47 +05:30
Rohit Yadav 0dce1c50c1 CLOUDSTACK-9456: Update Spring version in maven poms
- Bump spring-framework version to 4.x and Jetty to version that runs with JDK8
- Bump servet dependency version
- Migrate spring xmls to version 4, fixes schema locations that are 3.0
  dependent in various xmls.
- Fix failing tests due to spring upgrade
  (Thanks @marcaurele Marc-Aurèle Brothier for fixing them)
    * Fix test DeploymentPlanningManagerImplTest
    * Fix GloboDNS test

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-16 21:21:20 +05:30
Rohit Yadav 5e19e64f2f Updating pom.xml version numbers for release 4.9.2.0-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-16 20:48:16 +05:30
Rohit Yadav af2679959b Updating pom.xml version numbers for release 4.9.1.0
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-10 08:38:03 +05:30
Rohit Yadav 70c79ad13c
Merge branch '4.9' 2016-12-07 01:35:56 +05:30
Rohit Yadav abfcd5b95f CLOUDSTACK-9632: Upgrade bouncy castle to version 1.55
- Upgrades Maven dependency version to v1.55
- Fixes bountycastle usages and issues
- Adds timeout to jetty/annotation scanning
- Fixes servlet issue, uses servlet 3.1.0
- Downgrade javassist used by reflections to fix annotation process errors
- Make console-proxy-rdp bc dependency same as rest of the codebase
- Picks up PR #1510 by Daan

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-06 17:24:24 +05:30
Rohit Yadav 7a5e88ccb4
Merge branch '4.9' 2016-11-23 13:29:39 +05:30
Rohit Yadav 072af7a677 Merge pull request #1684 from shapeblue/cloudstack-9489
CLOUDSTACK-9489: the new config vars that are added do not goto DB if values are set to NULL, removing this check so the entries in DB are made with NULL values

* pr/1684:
  CLOUDSTACK-9489: the new config vars that are added do not goto DB if values are set to NULL, removing this check so the entries in DB are made with NULL values

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-11-23 13:18:55 +05:30
John Burwell 4c15cfce07 Merge release branch 4.9 to master
* 4.9:
  CLOUDSTACK-9460: For long running transactions, if the connection is timed out by the mysql server then refresh it
2016-11-17 00:55:05 -05:00
John Burwell 22d074607c Merge release branch 4.8 to 4.9
* 4.8:
  CLOUDSTACK-9460: For long running transactions, if the connection is timed out by the mysql server then refresh it
2016-11-17 00:54:38 -05:00
nvazquez 2e77496601 CLOUDSTACK-9438: Fix for CLOUDSTACK-9252 - Make NFS version changeable in UI 2016-09-28 08:51:37 -07:00
Abhinandan Prateek a39cc61fc6 CLOUDSTACK-9489: the new config vars that are added do not goto DB if values are set to NULL, removing this check so the entries in DB are made with NULL values 2016-09-23 10:58:11 +05:30
Abhinandan Prateek a813baed49 CLOUDSTACK-9460: For long running transactions, if the connection is
timed out by the mysql server then refresh it
2016-09-19 14:08:26 +05:30
John Burwell 8d11511b1f Adds support for four position versions and optional db upgrades
Often, patch and security releases do not require schema migrations or
data migrations.  However, if an empty upgrade class and associated
scripts are not defined, the upgrade process will break.  With this
change, if a release does not have an upgrade, a noop DbUpgrade is added
to the upgrade path.  This approach allows the upgrade to proceed and
for the database to properly reflect the installed version.  This change
should make the release process simpler as RMs no longer need to
rememeber to create this boilerplate code when starting a new release.

Beginning with the 4.8.2.0 and 4.9.1.0 releases, the project will
formally adopt a four (4) position release number to properly accomodate
rekeases that contain only CVE fixes.  The DatabaseUpgradeChecker and
Version classes made assumptions that they would always parse and
compare three (3) position version numbers.  This change adds the
CloudStackVersion value object that supports both three (3) and four (4)
version numbers.   It encapsulates version comparsion logic, as well as,
the rules to allow three (3) and four (4) to interoperate.

  * Modifies DatabaseUpgradeChecker to handle derive an upgrade path for
  a version that was not explicitly specified.  It determines the
  releases the first release before it with database migrations and uses
  that list as the basis for the list for version being calculated.  A
  noop upgrade is then added to the list which causes no schema changes
  or data migrations, but will update the database to the version.
  * Adds unit tests for the upgrade path calculation logic in
  DatabaseUpgradeChecker
  * Removes dummy upgrade logic for the 4.8.2.0 introduced in previous
  versions of this patch
  * Introduces the CloudStackVersion value object which parses and
  compares three (3) and four (4) position version numbers.  This class
  is intended to replace com.cloud.maint.Version.
  * Adds the junit-dataprovider dependency -- allowing test data to be
  concisely generated separately from the execution of a test case.
  Used extensively in the CloudStackVersionTest.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-30 13:32:32 +05:30
Rohit Yadav 9555492b4d Merge branch '4.9' 2016-08-23 14:16:53 +05:30
Rohit Yadav f13c224da1 Updating pom.xml version numbers for release 4.9.1.0-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-19 13:53:39 +05:30
Will Stevens 740bd45be6 Updating pom.xml version numbers for release 4.8.2-SNAPSHOT
Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-08-15 10:31:09 -04:00
Will Stevens a63db21d16 Updating pom.xml version numbers for release 4.8.1
Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-08-08 10:06:21 -04:00
Will Stevens 62aa3b2bfa Updating pom.xml version numbers for release 4.10.0-SNAPSHOT
Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-07-29 10:11:34 -04:00
Will Stevens 227ff3884d Updating pom.xml version numbers for release 4.9.0
Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-07-25 16:56:04 -04:00
Nick Livens c9567495ab Make sure that the DB drivers are loaded before initiating connections 2016-05-31 14:39:43 +02:00
Nick Livens 02c5d44f0b Dynamic loading of DB driver + support for other DB providers 2016-05-23 15:50:28 +02:00
Will Stevens 06c6b367e7 Merge pull request #1532 from exoscale/fix/db-cache-miss
DAO: Hit the cache for entity flagged as removed tooI came along this part of the code and I don't see any reason why the cache should not be used when fetching with the "removed" ones. It will help decrease the number of DB queries.

*It can be merged in many CS versions*

* pr/1532:
  DAO: Rewrite change for method findByIdIncludingRemoved(ID id)
  dao: Hit the cache for entity flagged as removed too since they are put in cache afterwards.

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-13 17:04:57 -04:00
Marc-Aurèle Brothier 39aa0e4f35 DAO: Rewrite change for method findByIdIncludingRemoved(ID id)
Also change the sibling method findById(final ID id)
2016-05-13 11:01:39 +02:00
Will Stevens 20cf8b23c0 Merge pull request #1453 from GabrielBrascher/brascher-removeUnusedClasses2
Remove classes with no referencesI used UCDetector (http://www.ucdetector.org/) as a plugin for Eclipse.  With this tool, I discovered a lot of code without any reference (variables, methods and classes).

Following the work that was done at [https://github.com/apache/cloudstack/pull/1448]; this pull request had the goal of removing some of these classes. To check if I wasn't missing anything I searched for any file that could reference some of those classes. As I haven't found any way of these classes being used, they were removed. Note that some of them I found other references, but references such as commented lines or tests, nothing that could indicate their use (as XML files configuring beans or another class instantiating an object with "new").

Waiting for tests. Please tell me if I am missing something.

Removed Classes:
- org.apache.cloudstack.framework.jobs.JobCancellationException (**Note:** removed
variable JobCancellationException in com.cloud.utils.SerialVersionUID)
- org.apache.cloudstack.ldap.NoSuchLdapUserException (**Note:** removed test file
/cloud-plugin-user-authenticator-ldap/test/groovy/org/apache/cloudstack/ldap/NoSuchLdapUserExceptionSpec.groovy)
- com.cloud.agent.api.storage.CreateVolumeOVAAnswer
- com.cloud.exception.MissingParameterValueException
- org.apache.cloudstack.api.response.StatusResponse
- org.apache.cloudstack.api.response.VolumeDetailResponse
- org.apache.cloudstack.api.response.UpgradeVmResponse
- org.apache.cloudstack.api.response.AddIpToVmNicResponse
- org.apache.cloudstack.api.response.TemplateZoneResponse (**Note:** at
org.apache.cloudstack.api.response.TemplateResponse, there is this
comment "To avoid breaking backwards compatibility, we still treat a
template at different zones as different templates, so not embedding
template_zone information in this TemplateZoneResponse set. `private
Set<TemplateZoneResponse> zones;`" but right now it is not used)
- org.apache.cloudstack.api.response.NicDetailResponse

* pr/1453:
  Removed classes with no reference

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-12 16:48:03 -04:00
Will Stevens 688522ecd4 Merge pull request #1385 from DaanHoogland/CLOUDSTACK-9265
CLOUDSTACK-9265 cleanup around httpclient versionssome cleanup done
- replaced HttpStatus from org.apache.commons.httpclient with that from org.apache.http
- removed unthrown HttpException
- left auto reformat in place

* pr/1385:
  CLOUDSTACK-9265 cleanup around httpclient versions

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-12 11:11:28 -04:00
Will Stevens baecaa8c8e Merge pull request #1440 from rodrigo93/Removed-Unused-Void-class
Removed Unused Void ClassRemoved the class _Void.java_ from _/cloud-framework-ipc/src/org/apache/cloudstack/framework/async/Void.java_ since the method _public void AsyncMethod(Object realParam,
AsyncCompletionCallback<Void> callback)_ does not exist and it does not seem to be used in another place.
Also, removed an unnecessary comment in
_/cloudstack-framework/ipc/test/org/apache/cloudstack/framework/codestyle/AsyncSampleListenerStyleCaller.java_

* pr/1440:
  Removed Unused Void Class

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-10 23:50:57 -04:00
Will Stevens ccb4fe0e70 Merge pull request #1350 from shapeblue/master-quotalock
Quota: consolidated lockable account check to a method. Added unit tests to check lockablity of various accounts.

Currently normal user and domain admin accounts are eligible for locking.

* pr/1350:
  Quota: consolidated lockable account check to a method. Added unit tests to check lockablity of various accounts

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-04 10:27:52 -04:00
Marc-Aurèle Brothier 696440a675 dao: Hit the cache for entity flagged as removed too since they are put
in cache afterwards.
2016-05-04 14:38:11 +02:00
Daan Hoogland cb85b7e8be CLOUDSTACK-9265 cleanup around httpclient versions 2016-05-02 16:25:01 +02:00
Daan Hoogland 101668994d further maven dependency updates from Daan
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-04-22 16:36:37 +05:30
Rohit Yadav 770aa0133e framework/quota: fix checkstyle issue
Fixes enum name to supress checkstyle failure due to the latest checkstyle
version

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-04-22 16:20:34 +05:30
Rohit Yadav bb29b1d063 maven: Upgrade dependency versions
Updated most dependencies to latest minor releases, EXCEPT:

- Gson 2.x
- Major spring framework version
- Servlet version
- Embedded jetty version
- Mockito version (beta)
- Mysql lib minor version upgrade (breaks mysql-ha plugin)

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-04-22 16:20:16 +05:30
weingartner 4179606c0b Removed classes with no reference
- org.apache.cloudstack.framework.jobs.JobCancellationException (removed
variable JobCancellationException in com.cloud.utils.SerialVersionUID)
- org.apache.cloudstack.ldap.NoSuchLdapUserException (removed test file
/cloud-plugin-user-authenticator-ldap/test/groovy/org/apache/cloudstack/ldap/NoSuchLdapUserExceptionSpec.groovy)
- com.cloud.agent.api.storage.CreateVolumeOVAAnswer
- com.cloud.exception.MissingParameterValueException
- org.apache.cloudstack.api.response.StatusResponse
- org.apache.cloudstack.api.response.VolumeDetailResponse
- org.apache.cloudstack.api.response.UpgradeVmResponse
- org.apache.cloudstack.api.response.AddIpToVmNicResponse
- org.apache.cloudstack.api.response.TemplateZoneResponse (at
org.apache.cloudstack.api.response.TemplateResponse, there is this
comment "To avoid breaking backwards compatibility, we still treat a
template at different zones as different templates, so not embedding
template_zone information in this TemplateZoneResponse set. `private
Set<TemplateZoneResponse> zones;`" but right now it is not used)
- org.apache.cloudstack.api.response.NicDetailResponse
2016-04-07 21:03:05 -03:00
Will Stevens e72a69a8a2 Merge pull request #1254 from shapeblue/master-9174
CLOUDSTACK-9174: A deleted account results in NPEWhen an account is deleted from cloudstack for which quota is still
being calculated and if the quota reaches minimum threshold then
quota service will try to alert the user. This results in NPE and is
fixed by excluding such accounts from alerting and other quota related
mechanisms.

* pr/1254:
  CLOUDSTACK-9174: A deleted account results in NPE

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-07 11:39:22 -04:00
Rodrigo 01034ea8d6 Removed Unused Void Class
Removed the class Void.java from
/cloud-framework-ipc/src/org/apache/cloudstack/framework/async/Void.java
since the method public void AsyncMethod(Object realParam,
AsyncCompletionCallback<Void> callback) does not 
exist and it does not seemed to be used in another place.
Also, removed an unnecessary comment in
/cloudstack-framework/ipc/test/org/apache/cloudstack/framework/codestyle/AsyncSampleListenerStyleCaller.java
2016-03-13 14:35:51 -03:00
nvazquez 9857f41d9a CLOUDSTACK-9298: Add @MappedSuperClass support for persistence inheritance 2016-03-10 06:07:29 -08:00
Abhinandan Prateek 04ca7e57d5 Quota: consolidated lockable account check to a method. Added unit tests
to check lockablity of various accounts
2016-01-28 17:53:54 +05:30
Rohit Yadav 06fe7844db Merge pull request #1240 from shapeblue/master-9161
CLOUDSTACK-9161: fix the quota marvin test  1. Create a dummy user, as existing user may already have stale quota
  data
  2. fix the tests to use the dummy user
  3. a boundary condition was revealed and fixed for a new user where
  quota service has never run and created bootstrap entries

* pr/1240:
  CLOUDSTACK-9161: fix the quota marvin test

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-27 12:03:28 +01:00
Abhinandan Prateek ee2bc828a9 Merge pull request #1289 from shapeblue/master-quotafb
Quota: findbug fixesFindbug fixes for cloud-framework-quota and cloud-plugin-database-quota.

* pr/1289:
  Quota: findbug fixes

Signed-off-by: Abhinandan Prateek <abhinandan.prateek@shapeblue.com>
2016-01-27 15:41:23 +05:30
Remi Bergsma 43ab98d823 Updating pom.xml version numbers for release 4.9.0-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2016-01-26 15:12:20 +01:00
Remi Bergsma 32fcc47117 Updating pom.xml version numbers for release 4.8.1-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2016-01-26 09:39:00 +01:00
Remi Bergsma 62f218b7bd Updating pom.xml version numbers for release 4.8.0
Signed-off-by: Remi Bergsma <github@remi.nl>
2016-01-20 23:43:35 +01:00
Remi Bergsma 1bfba70b62 Merge pull request #1272 from koushik-das/CLOUDSTACK-9195
CLOUDSTACK-9195: Cancelled/failed async jobs not getting cleaned up from DB

Setting 'job_complete_msid' field in async_job table when marking job as cancelled during MS restart.
This will ensure that the job is cleaned up as part of GC thread.

* pr/1272:
  CLOUDSTACK-9195: Cancelled/failed async jobs not getting cleaned up from DB Setting 'job_complete_msid' field in async_job table when marking job as cancelled during MS restart. This will ensure that the job is cleaned up as part of GC thread.

Signed-off-by: Remi Bergsma <github@remi.nl>
2016-01-20 14:37:37 +01:00
Abhinandan Prateek 1304406fd1 Quota: findbug fixes
Findbug fixes for cloud-framework-quota and cloud-plugin-database-quota.
2016-01-18 09:17:30 +05:30
Remi Bergsma ee2ccc4d41 Merge release branch 4.7 to master
* 4.7:
  QuotaStatementTest: fix the method invocation verification
2016-01-05 15:27:06 +01:00
Abhinandan Prateek 0eac900fc4 QuotaStatementTest: fix the method invocation verification 2016-01-05 11:04:33 +05:30
Koushik Das a6f9181ee1 CLOUDSTACK-9195: Cancelled/failed async jobs not getting cleaned up from DB
Setting 'job_complete_msid' field in async_job table when marking job as cancelled during MS restart.
This will ensure that the job is cleaned up as part of GC thread.
2015-12-22 13:53:29 +05:30
Remi Bergsma 8f5a2920e8 Updating pom.xml version numbers for release 4.8.0-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-21 22:09:31 +01:00
Remi Bergsma 5b112408c7 Updating pom.xml version numbers for release 4.7.1-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-21 19:55:04 +01:00