Commit Graph

79 Commits

Author SHA1 Message Date
John Burwell 8d11511b1f Adds support for four position versions and optional db upgrades
Often, patch and security releases do not require schema migrations or
data migrations.  However, if an empty upgrade class and associated
scripts are not defined, the upgrade process will break.  With this
change, if a release does not have an upgrade, a noop DbUpgrade is added
to the upgrade path.  This approach allows the upgrade to proceed and
for the database to properly reflect the installed version.  This change
should make the release process simpler as RMs no longer need to
rememeber to create this boilerplate code when starting a new release.

Beginning with the 4.8.2.0 and 4.9.1.0 releases, the project will
formally adopt a four (4) position release number to properly accomodate
rekeases that contain only CVE fixes.  The DatabaseUpgradeChecker and
Version classes made assumptions that they would always parse and
compare three (3) position version numbers.  This change adds the
CloudStackVersion value object that supports both three (3) and four (4)
version numbers.   It encapsulates version comparsion logic, as well as,
the rules to allow three (3) and four (4) to interoperate.

  * Modifies DatabaseUpgradeChecker to handle derive an upgrade path for
  a version that was not explicitly specified.  It determines the
  releases the first release before it with database migrations and uses
  that list as the basis for the list for version being calculated.  A
  noop upgrade is then added to the list which causes no schema changes
  or data migrations, but will update the database to the version.
  * Adds unit tests for the upgrade path calculation logic in
  DatabaseUpgradeChecker
  * Removes dummy upgrade logic for the 4.8.2.0 introduced in previous
  versions of this patch
  * Introduces the CloudStackVersion value object which parses and
  compares three (3) and four (4) position version numbers.  This class
  is intended to replace com.cloud.maint.Version.
  * Adds the junit-dataprovider dependency -- allowing test data to be
  concisely generated separately from the execution of a test case.
  Used extensively in the CloudStackVersionTest.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-30 13:32:32 +05:30
Will Stevens 740bd45be6 Updating pom.xml version numbers for release 4.8.2-SNAPSHOT
Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-08-15 10:31:09 -04:00
Will Stevens a63db21d16 Updating pom.xml version numbers for release 4.8.1
Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-08-08 10:06:21 -04:00
Will Stevens 8f330b0b92 Merge release branch 4.7 to 4.8
* 4.7:
  Fix Sync of template.properties in Swift
  Configure rVPC for router.redundant.vrrp.interval advert_int setting
  Have rVPCs use the router.redundant.vrrp.interval setting
  Resolve conflict as forceencap is already in master
  Split the cidr lists so we won't hit the iptables-resture limits
  Check the existence of 'forceencap' parameter before use
  Do not load previous firewall rules as we replace everyhing anyway
  Wait for dnsmasq to finish restart
  Remove duplicate spaces, and thus duplicate rules.
  Restore iptables at once using iptables-restore instead of calling iptables numerous times
  Add iptables copnversion script.
2016-05-18 15:54:32 -04:00
Syed f5ac8ddded Fix Sync of template.properties in Swift 2016-05-13 22:08:16 +00:00
Remi Bergsma 32fcc47117 Updating pom.xml version numbers for release 4.8.1-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2016-01-26 09:39:00 +01:00
Remi Bergsma a4ac9d3fff Updating pom.xml version numbers for release 4.7.2-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2016-01-26 09:33:56 +01:00
Remi Bergsma 62f218b7bd Updating pom.xml version numbers for release 4.8.0
Signed-off-by: Remi Bergsma <github@remi.nl>
2016-01-20 23:43:35 +01:00
Remi Bergsma 8f5a2920e8 Updating pom.xml version numbers for release 4.8.0-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-21 22:09:31 +01:00
Remi Bergsma 5b112408c7 Updating pom.xml version numbers for release 4.7.1-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-21 19:55:04 +01:00
Remi Bergsma 2f26a859a9 Updating pom.xml version numbers for release 4.7.0
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-13 21:09:53 +01:00
Remi Bergsma 4ecfc29267 Merge release branch 4.6 to master
* 4.6:
  Use version for RC branch name instead of branch
  make sure all files are updates with new version
  Update L10N resource files with 4.6 strings from Transifex (20151129)
  Fix secondary storage not working with swift
  CLOUDSTACK-9083: Add disk serial to kvm virt xml
2015-11-30 22:30:52 +01:00
Syed 1e67a5d2c8 Fix secondary storage not working with swift 2015-11-27 13:48:18 -05:00
Daan Hoogland 329b3a5611 CID-1339965 handle retrun value on failure 2015-11-27 10:30:11 +01:00
cirstofolini 1a64c247ad Removed unnecessary @Local annotations and their respective imports from the ComponentLifecycleBase class and its subclasses. 2015-11-21 18:31:11 -02:00
Boris Schrijver 5c0366c99e CLOUDSTACK-9062: Improve S3 implementation.
The S3 implementation is far from finished, this commit focusses on the bases.

 - Upgrade AWS SDK to latest version.
 - Rewrite S3 Template downloader.
 - Rewrite S3Utils utility class.
 - Improve addImageStoreS3 API command.
 - Split various classes for convenience.
 - Various minor improvements and code optimalisations.

A side effect of the new AWS SDK is that it, by default, uses the V4 signature. Therefore I added an option to specify the Signer, so it stays compatible with previous versions.
2015-11-19 15:29:10 +01:00
Remi Bergsma 1f53f2a93e Updating pom.xml version numbers for release 4.7.0-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-15 18:54:13 +01:00
Remi Bergsma b38c3bed0c Updating pom.xml version numbers for release 4.6.1-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-13 21:27:57 +01:00
Remi Bergsma e31ade03c6 Updating pom.xml version numbers for release 4.6.0
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-10 15:45:34 +01:00
Rajani Karuturi 1056171aca CLOUDSTACK-8808: Successfully registered VHD template is downloaded
again due to missing virtualsize property in template.properties

We have multiple file processors to process different types of image
formats. The processor interface has two methods getVirtualSize() and
process().

    1. getVirtualSize() as the name says, returns the virtual size of
the file and is used at get the size while copying files from NFS to s3
    2. process() returns FormatInfo struct which has fileType, size,
virutalSize, filename.  on successfully downloading a template, each
file is passed to all the processors.process() and whichever returns a
FormatInfo, that will be used to create template.properties file.  If
process() throws an InternalErrorException, template installation fails.
But, if process() returns null, template registration is successful with
template.properties missing some attributes like virtualSize, file
format etc. which results in this bug on restart of ssvm/cloud
service/management server.

failing the template download if virutalsize or some other properties
cannot be determined.

The following changes are done:
getVirtualSize() to always return size(if it can calculate, get virtual
size else return file size). This would mean the following changes

    1. QCOW2Processor.getVirtualSize() to return file size if virtual
size calculation fails
    2. VHDProcessor.getVirtualSize() to return file size if virtual size
calculation fails

process() to throw InternalErrorException if virtual size calculation
fails or any other exceptions occur. This would mean the following
changes

    1. OVAProcessor to throw InternalErrorException if untar fails
    2. QCOW2Processor to throw InternalErrorException if virtual size
calculation fails
    3. VHDProcessor to throw InternalErrorException if virtual size
calculation fails
2015-09-30 15:43:51 +05:30
Boris Schrijver 4a770fc63a Fixed Findbugs issue introduced by 1c6378ec00 PR #795. 2015-09-14 10:56:11 +02:00
Boris Schrijver 1971614e31 Added QCOW2 virtual size checking for S3.
- Cleaned up S3TemplateDownloader
- Created static QCOW2 utils class.
- Reformatted some parts of DownloadManagerImpl
2015-09-10 16:45:48 +02:00
Rajani Karuturi 8bc0294014 Revert "Merge pull request #714 from rafaelweingartner/master-lrg-cs-hackday-003"
This reverts commit cd7218e241, reversing
changes made to f5a7395cc2.

Reason for Revert:

noredist build failed with the below error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project cloud-plugin-hypervisor-vmware: Compilation failure
[ERROR] /home/jenkins/acs/workspace/build-master-noredist/plugins/hypervisors/vmware/src/com/cloud/hypervisor/guru/VMwareGuru.java:[484,12] error: non-static variable logger cannot be referenced from a static context
[ERROR] -> [Help 1]

even the normal build is broken as reported by @koushik-das on dev list
http://markmail.org/message/nngimssuzkj5gpbz
2015-08-31 11:27:57 +05:30
Rafael Weingartner 3818257a68 Solved jira ticket: CLOUDSTACK-8750 2015-08-28 22:35:08 -03:00
Wido den Hollander 9dbc99c608 CLOUDSTACK-8648: Pass the proper storage layer when configuring
Signed-off-by: Pierre-Luc Dion <pdion891@apache.org>
2015-07-26 10:25:06 -04:00
Wido den Hollander 554f5f9fbb CLOUDSTACK-8648: Do not configure the ImageFormat Processor when fetching filesize
It will throw an exception and that's needed.

Also, make the log show about which file we are talking about

Signed-off-by: Wido den Hollander <wido@widodh.nl>
Signed-off-by: Pierre-Luc Dion <pdion891@apache.org>
2015-07-26 10:25:05 -04:00
Daan Hoogland bb2c02d144 coverity 1116693: properties template file creation unlikely but if repetitive, serious.
Signed-off-by: Daan Hoogland <daan@onecht.net>
2015-07-09 21:30:24 +02:00
Rajani Karuturi d39b993512 Fixed a few findbugs issues after the merge 0b83559
HttpUploadServerHandler.java:142, DM_BOXED_PRIMITIVE_FOR_PARSING
NfsSecondaryStorageResource.java:2630, DM_BOXED_PRIMITIVE_FOR_PARSING
NfsSecondaryStorageResource.java:2775, DM_DEFAULT_ENCODING
EncryptionUtil.java:59, DM_DEFAULT_ENCODING
2015-04-30 13:56:44 +05:30
Rajani Karuturi 0b8355920e Merge branch 'volume-upload' into master
This closes #206
2015-04-29 11:12:53 +05:30
Rajani Karuturi 75ae90b073 volume-upload: added account level secondary storage resource limit checks 2015-04-24 15:48:44 +05:30
Rajani Karuturi 0525e4763b volume-upload: refactored some error messages 2015-04-24 15:45:29 +05:30
Rajani Karuturi a7e511c1a3 Added file exists check for onetime post url
also, fixed an issue where in the upload was going to error state in
case of parallel call to the same post url
2015-04-17 18:08:23 +05:30
Rajani Karuturi ce823a3a26 Returning template/volume download percent in UploadStatusAnswer
Also updating the store ref entries on receiving the answer.
2015-04-09 15:38:58 +05:30
Rajani Karuturi 878b43bc64 volume-upload: removed hardcoded vhd for Image format check. 2015-03-30 17:56:59 +05:30
Rajani Karuturi 018023c1ef volume upload: added validation for file formats
merged TemplateUtils and ImageStoreUtil to a singe ImageStoreUtil
also added a unittest for ImageStoreUtil
2015-03-20 16:25:13 +05:30
Rajani Karuturi d5dffb5dc9 volume upload: added md5 checksum validation
also fixed the issue wherein the successful uploads where also moving to
error state as the channelinactive is called after the end of successful
upload as well.
added a fileuploaded boolean to check when the channel is inactive.
2015-03-18 12:20:51 +05:30
Rajani Karuturi 6b8b4b92e6 handling tcp close event in netty server
while the file is in uploading state and connection is reset by peer,
volume upload request posted again shows the upload is IN_Progress
state. marking them as errored
2015-03-17 15:21:25 +05:30
Rajani Karuturi 3de5d9db5f volume upload: Restart of MS leads to loss of browser uploaded templates
on restart of management server, template sync runs. It checks for
templates in ssvm using the uniquename. If it doesnt find any, cleans
the directory. In case of uploaded templates, these are getting saved
using name instead on uniquename and hence template sync cant find them
and does cleanup. Using uniquename in template.properties now.
2015-03-16 12:45:24 +05:30
Rajani Karuturi b16520bcec volume upload: added max size check for temaplte/volume post upload
used the existing configuration variables max.template.iso.size and
storage.max.volume.upload.size for templates and volumes respectively.
2015-03-03 17:27:40 +05:30
Rajani Karuturi 58f2fb1bb1 volume upload: post response http error code is 200 even for error case
Returned appropriate error code(bad request, internal server error, ok)
2015-02-17 12:31:38 +05:30
Rajani Karuturi 643165a07e volume upload: used netty server to do the file upload 2015-02-17 12:31:27 +05:30
Rajani Karuturi 01c41d09db volume upload: returning the install path and size in UploadStatusCommand 2015-02-17 11:28:32 +05:30
Rajani Karuturi 7231daa71d volume upload: fixed the post body parsing logic
as long as the boundary is completely in the byte buffer it works.
2015-01-29 16:55:29 +05:30
Rajani Karuturi 92669e314e volume upload: modified parsing logic
vhd file download works fine. some open issues with parsing params
2015-01-29 16:55:28 +05:30
Bharat Kumar 2a2d4364e1 made changes to move a file to the specified localtion
Signed-off-by: Rajani Karuturi <rajanikaruturi@gmail.com>
2015-01-20 16:02:33 +05:30
Rajani Karuturi a31e1bff33 volume upload: fixed build error from the commit b0a96c4c85 2015-01-20 12:44:05 +05:30
Rajani Karuturi b0a96c4c85 volume upload: fixed nio server blocking thread issue 2015-01-20 12:27:26 +05:30
Rajani Karuturi 15ae33a5b6 volume upload: refactored post upload command and used it during template registration 2015-01-20 09:24:14 +05:30
Rajani Karuturi ad6b7b30cb Volume upload: sending the nfsstore url in the metadata
decoding the metadata and deserialising it on receiving it
2015-01-19 18:04:14 +05:30
Rajani Karuturi ca0f9de6c7 volume upload: parsing post body to get all the params 2015-01-19 12:04:30 +05:30