CLOUDSTACK-9456: Migrate master to Spring 4.xThis changes makes CloudStack use spring 4:
```
- Bump spring-framework version to 4.x and Jetty to version that runs with JDK7
- Bump servet dependency version
- Migrates various xmls to use version independent schema uris
```
Outstanding issue:
- Testing of various non-standard plugins such as network and storage plugins etc.
Since, this is a big change pinging for review -- @jburwell @karuturi @wido @murali-reddy @abhinandanprateek @DaanHoogland @GaborApatiNagy @JayapalUradi @kishankavala @K0zka @nvazquez @rafaelweingartner @pyr and others
@blueorangutan package
* pr/1638:
CLOUDSTACK-9456: Update Spring version in maven poms
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
XenServer 7 SupportThis PR adds support for XenServer 7. I have manually done the following tests
- Create a new cluster with XenServer7
- Add Primary storage: Should create an SR on XS7
- Add another XS7 host to the Pool
- Add host2 to Cloudstack
- Create VM1 from template
- Create VM2 from template
- Ping/SSH VM1 to VM2 and vice-versa
- Stop/Delete/Expunge VM2
- Create Data disk
- Attach it to VM1
- Create VM snaphsot of VM1
- Restore VM snapshot of VM1
- Delete VM snapshot of VM1
- Create Volume snapshot of Datadisk
- Create volume snapshot of Root disk
- Create new template from snapshot of root disk
- Create volume from snapshot of datadisk
- Detach datadisk volume
- Delete datadisk volume
- Aquire a public IP
- Create a static nat to VM1
- Live migrate VM1 while traffic on VM
- Delete VM1
* pr/1711:
[CLOUDSTACK-9662] Add support for XenServer 7
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Bump spring-framework version to 4.x and Jetty to version that runs with JDK8
- Bump servet dependency version
- Migrate spring xmls to version 4, fixes schema locations that are 3.0
dependent in various xmls.
- Fix failing tests due to spring upgrade
(Thanks @marcaurele Marc-Aurèle Brothier for fixing them)
* Fix test DeploymentPlanningManagerImplTest
* Fix GloboDNS test
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Cloudstack 9586: When using local storage with Xenserver prepareTemplate does not work with multiple primary storeThe race condition will happen whenever there are multiple primary storages and the CS tries to mount the secondary store to xenserver host simultaneously.
Due to synchronised block one mount will be successful and other thread will get the already mounted SR. Without the fix the two thread will try to mount it parallely and one will fail on Xenserver.
* pr/1765:
Cloudstack 9586: When using local storage with Xenserver prepareTemplate does not work with multiple primary store The race condition will happen whenever there are multiple primary storages and the CS tries to mount the secondary store to xenserver host simultaneously. Due to synchronised block one mount will be successful and other thread will get the already mounted SR. Without the fix the two thread will try to mount it parallely and one will fail on Xenserver.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The race condition will happen whenever there are multiple primary storages and the CS tries to mount the secondary store to xenserver host simultaneously.
Due to synchronised block one mount will be successful and other thread will get the already mounted SR. Without the fix the two thread will try to mount it parallely and one will fail on Xenserver.
CLOUDSTACK-9503: Increased the VR script timeout. Most of the changes are about converting int/long time values to joda Duration.
* pr/1745:
CLOUDSTACK-9503: Increased the VR script timeout. Most of the changes are about converting int/long time values to joda Duration.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This PR adds an ability to Pass a new parameter, locationType,
to the “createSnapshot” API command. Depending on the locationType,
we decide where the snapshot should go in case of managed storage.
There are two possible values for the locationType param
1) `Standard`: The standard operation for managed storage is to
keep the snapshot on the device. For non-managed storage, this will
be to upload it to secondary storage. This option will be the
default.
2) `Archive`: Applicable only to managed storage. This will
keep the snapshot on the secondary storage. For non-managed
storage, this will result in an error.
The reason for implementing this feature is to avoid a single
point of failure for primary storage. Right now in case of managed
storage, if the primary storage goes down, there is no easy way
to recover data as all snapshots are also stored on the primary.
This features allows us to mitigate that risk.
Often, patch and security releases do not require schema migrations or
data migrations. However, if an empty upgrade class and associated
scripts are not defined, the upgrade process will break. With this
change, if a release does not have an upgrade, a noop DbUpgrade is added
to the upgrade path. This approach allows the upgrade to proceed and
for the database to properly reflect the installed version. This change
should make the release process simpler as RMs no longer need to
rememeber to create this boilerplate code when starting a new release.
Beginning with the 4.8.2.0 and 4.9.1.0 releases, the project will
formally adopt a four (4) position release number to properly accomodate
rekeases that contain only CVE fixes. The DatabaseUpgradeChecker and
Version classes made assumptions that they would always parse and
compare three (3) position version numbers. This change adds the
CloudStackVersion value object that supports both three (3) and four (4)
version numbers. It encapsulates version comparsion logic, as well as,
the rules to allow three (3) and four (4) to interoperate.
* Modifies DatabaseUpgradeChecker to handle derive an upgrade path for
a version that was not explicitly specified. It determines the
releases the first release before it with database migrations and uses
that list as the basis for the list for version being calculated. A
noop upgrade is then added to the list which causes no schema changes
or data migrations, but will update the database to the version.
* Adds unit tests for the upgrade path calculation logic in
DatabaseUpgradeChecker
* Removes dummy upgrade logic for the 4.8.2.0 introduced in previous
versions of this patch
* Introduces the CloudStackVersion value object which parses and
compares three (3) and four (4) position version numbers. This class
is intended to replace com.cloud.maint.Version.
* Adds the junit-dataprovider dependency -- allowing test data to be
concisely generated separately from the execution of a test case.
Used extensively in the CloudStackVersionTest.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* 4.8:
CLOUDSTACK-9353: [XenServer] Fixed VM migration with storage
Added ASF license to unit test file
Added unit test to verify ordering
Fixed ordering of network ACL rules being sent to the VR. The comparator was inverted
* 4.7:
CLOUDSTACK-9353: [XenServer] Fixed VM migration with storage
Added ASF license to unit test file
Added unit test to verify ordering
Fixed ordering of network ACL rules being sent to the VR. The comparator was inverted
CLOUDSTACK-8800 : Improved the listVirtualMachines API call to include memory utilization information for a VMThis PR introduces the changes proposed in PR #780 with some work to make the code null safe.
During this PR, I have also removed some unused code.
* pr/1444:
Removed unnecessary check when creating the “userVmResponse” object.
Fixed issues from CLOUDSTACK-8800 that were introduced in PR 780
CLOUDSTACK-8800 : Improved the listVirtualMachines API call to include memory utilization information for a VM for xenserver,kvm and for vmware.
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CID-1338387: Deletion of method endPointSelector.selectHypervisorHostFollowing the discussions and analysis presented on PR #1056 create by @DaanHoogland
This PR is intended to push those changes that were discussed there regarding the of endPointSelector.selectHypervisorHost method.
* pr/1124:
Deletion of method endPointSelector.selectHypervisorHost
Signed-off-by: Will Stevens <williamstevens@gmail.com>
It was worked around some possible runtime exceptions introduced by the
changes that were added by the PR 780. Basically, the points in which a
null pointer exception could happen, we added safety checks to avoid
them. It was create a specific method do that, all together test cases
were created for this newly method that was added.
CLOUDSTACK-9130: Make RebootCommand similar to start/stop/migrate agent commands w.r.t. "execute in sequence" flag
RebootCommand now behaves in the same way as start/stop/migrate agent commands w.r.t. to sequential/parallel execution.
* pr/1200:
CLOUDSTACK-9130: Make RebootCommand similar to start/stop/migrate agent commands w.r.t. "execute in sequence" flag RebootCommand now behaves in the same way as start/stop/migrate agent commands w.r.t. to sequential/parallel execution.
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Removed unnecessary code from getGuestOsType in CitrixResourceBaseConsidering that all mapping of Guest OS Names to their respective hypervisor compatible types is made thorugh accessing a database, we've decided to remove a bit of code in the XcpOssResource class which was doing that same thing for 2 different OS's (both of which ARE in the database). That has led us to a bunch of unused parameters in the getGuestOsType method from its superclass, which we've also decided to remove. Test cases were added for four different possibilities for the platformEmulator String: one for a null String, one for a blank String, one for an empy String and one for a random case with a valid String.
* pr/1262:
Remove test cases duplicated code.
Removed unnecessary code from getGuestOsType in CitrixResourceBase
Signed-off-by: Will Stevens <williamstevens@gmail.com>
SecurityGroupRulesCmd code cleanupWrote a test and cleaned some duplicate code with the objective to evaluate the jenkins pull request process at builds.a.o
worthwhile to keep, IMHO.
* pr/1287:
SecurityGroupRulesCmd code cleanup review comments handled
deal with PMD warnings
code cleanup
security rules test
remove autogenerated pydev files
Signed-off-by: Koushik Das <koushik@apache.org>
Fixed return type Void to void in DataMotionStrategy.The main changes are:
- Changing methods Void to void.
- Removal of the method Void copyAsync(DataObject srcData, DataObject
destData, AsyncCompletionCallback<CopyCommandResult> callback) that was
never used.
We noticed that methods form that class are using the return type Void
with capital V. This way that method has to return a null value at the
end.
Removed trim lines from XenServerStorageMotionStrategy.
The trim lines were removed from XenServerStorageMotionStrategy.
* pr/969:
Changed return of methods from DataMotionStrategy, Void to void
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* 4.7:
Fix execution counter to support separate counts per thread
Add test to check that each thread has it's own execution counter
CLOUDSTACK-9231: Root volume migration from one primary to another primary storage within the same cluster is failing
This situation arises when there are two management server accessing the same database.
When the migration request comes the command is forwarded from one management server to another because
the host is owned by the second management server. So, serialization of map from one to another fails.
This is fixed by converting the maps to lists.
Fix also reverts below commit as below solution making assumption about hypervisor which are not applicable
in case of XenServer and VmWare
Revert "CLOUDSTACK-8964: Can't create template or volume from snapshot"
This reverts commit ccf5d75cfb.
Create test cases to getPatchFilePath method and class names changed In this commit we created tests cases for the respective classes in package com.cloud.hypervisor.xenserver.resource.
We added test cases to check the implementation of com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getPatchFiles. Therefore, we test in a more comprehensive way the tests that already exist to check the code of com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getPatchFilePath.
We added a new abstract class, called com.cloud.hypervisor.xenserver.resource.CitrixResourceBaseTest.java
This class has two tests methods:
* com.cloud.hypervisor.xenserver.resource.CitrixResourceBaseTest.testGetPathFilesExeption(CitrixResourceBase), this method tests if the getPatchFilePath() method throws the com.cloud.utils.exception.CloudRuntimeException.CloudRuntimeException when the com.cloud.utils.script.Script.findScript(String, String) return a null value;
* com.cloud.hypervisor.xenserver.resource.CitrixResourceBaseTest.testGetPathFilesListReturned(CitrixResourceBase), this method tests the correct return value from getPatchFilePath() method, basically, verify if the returned list contain the file with the same absolute path that was retrieved from the findScript method.
We also changed the name of those test classes, the change was basically remove the Path word from the name of classes.
* pr/944:
created tests cases for method "citrixResourceBase.getPatchFiles"
Signed-off-by: Remi Bergsma <github@remi.nl>
we also changed the name of those test classes, the change was basically
remove the “Path” word from the name of classes.
change the private method
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getPatchFiles
to protected
added new class in test cases
The main changes are:
• Changing methods “Void” to “void”.
• Removal of the method “Void copyAsync(DataObject srcData, DataObject
destData, AsyncCompletionCallback callback)” that was never used. We
noticed that methods form that class are using the return type Void with
capital V. This way that method has to return a null value at the end.
Storage motion of vm across clusters/xenserver-pools fails in a
clustered management server setup. In xen storage motion we have to send
a migrate_receive command to the destination host, followed by
migrate_send command to the source host. The sr and network detials of
the destination host have to be passed in migrate_send command on the
source. While migrating across clusters the source and destination
resources are seperate objects. To pass this information across
resources we have to send seperate migrate with storage receive and send
commands to the resource. In a clustered ms setup these commands may
have to be forwarded to another ms as the resource may be owned by it.
The serilization of the command and answer objects fails in such case as
it doesn't understand the xapi sr and network objects. Made a change to
serialize these objects in the resource layer and pass the around as
strings in the command and answer objects. Reviewed-By: Likitha
The S3 implementation is far from finished, this commit focusses on the bases.
- Upgrade AWS SDK to latest version.
- Rewrite S3 Template downloader.
- Rewrite S3Utils utility class.
- Improve addImageStoreS3 API command.
- Split various classes for convenience.
- Various minor improvements and code optimalisations.
A side effect of the new AWS SDK is that it, by default, uses the V4 signature. Therefore I added an option to specify the Signer, so it stays compatible with previous versions.
If device id passed as part of API and available then use it otherwise fallback on XS to automatically assign one.
For ISO device id used is 3 and it is processed before any other entry to avoid conflict.
This reverts commit cd7218e241, reversing
changes made to f5a7395cc2.
Reason for Revert:
noredist build failed with the below error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project cloud-plugin-hypervisor-vmware: Compilation failure
[ERROR] /home/jenkins/acs/workspace/build-master-noredist/plugins/hypervisors/vmware/src/com/cloud/hypervisor/guru/VMwareGuru.java:[484,12] error: non-static variable logger cannot be referenced from a static context
[ERROR] -> [Help 1]
even the normal build is broken as reported by @koushik-das on dev list
http://markmail.org/message/nngimssuzkj5gpbz
@cristofolini comments:
Removed unnecessary duplicated code by having the body of the
getPatchFiles method only in the
CitrixResourceBase superclass. Given that all of its implementations
consisted of the same code
except for the path which contains the necessary file for that
implementation. An abstract method
getPatchFilePath was created in the CitrixResourceBase superclass so
that each implementation may
return the path containing the specific file needed by that
implementation.
Test cases were created for each implementation, simple as they may be.
One assert is made to verify
that the path returned by each implementation corresponds to the path
that was previously specified
on each getPatchFiles implementation.
removed trailing whitespace
- Adding more unit tests as suggested by Daan and Rajani.
- Mocking SshHelper and the Script classes, used by Citrix and Libvirt resources respectively.
- Changed location of the update_host_passwd script
- Updated the patch files for XenServer
- Updated the script path on LibvirtComputing class
- Removed the hostIP from the LibvirtUpdateHostPasswordCommandWrapper execute() method
- Adding update_host_passwd to VRScripts
- Add accessor method to host password on CitrixResourceBase
- Add implementation to CitrixUpdateHostPasswordCommandWrapper
- Improve testUpdateHostPasswordCommand() unit test on CitrixRequestWrapperTest
- Add line to patch files on xenserver directory
Concerning the LibVirt change:
- I forgot to assing the return of the getDefaultHypervisorScriptsDir() method to the hypervisorScriptsDir variable
Rohit Yadav
Syed
Anshul Gangwar
Mike Tutkowski
Abhinandan Prateek
Koushik Das
subhash yedugundla
John Burwell
Will Stevens
weingartner
Maneesha.P
Remi Bergsma
Nitin Kumar Maharana
Daan Hoogland
cirstofolini
Boris Schrijver
SudharmaJain
Anshul Gangwar
pedro-martins
Devdeep Singh
Funs Kessen
Rajani Karuturi
Miguel Ferreira
Funs
Jayapal
wilderrodrigues
Sanjay Tripathi
Rafael da Fonseca