Commit Graph

746 Commits

Author SHA1 Message Date
Likitha Shetty 6c649ce3ae CLOUDSTACK-8411. Unable to delete an uploaded volume after CCP fails to attach the volume to a VM.
Correctly update the status of an uploaded volume upon failure to attach it to a VM.

(cherry picked from commit 10a106f5d8)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-04-29 16:50:40 +02:00
Rajani Karuturi 0b8355920e Merge branch 'volume-upload' into master
This closes #206
2015-04-29 11:12:53 +05:30
Likitha Shetty 10a106f5d8 CLOUDSTACK-8411. Unable to delete an uploaded volume after CCP fails to attach the volume to a VM.
Correctly update the status of an uploaded volume upon failure to attach it to a VM.
2015-04-28 09:52:06 +05:30
Santhosh Edukulla 86943da26e Fixed few coverity issues
Signed-off-by: Santhosh Edukulla <santhosh.edukulla@gmail.com>
(cherry picked from commit 0a9742f914)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-04-27 14:43:48 +02:00
Koushik Das bc399b981f volume-upload: If SSVM is destroyed and started, then partially uploaded volumes/templates remain in
inconsistent state (NotUploaded or UploadInProgress) and doesn't transition to any terminal
state (Uploaded, Uploaded). As a result these volumes/templates cannot be removed. Fix is to handle such
volume/template entries correctly.
2015-04-22 19:28:51 +05:30
Likitha Shetty 326bb3e0a4 CLOUDSTACK-8320. Upon a failed migration, a dummy volume is created which remains in 'Expunging' state.
Set destination volume path as NULL while duplicating volume during migration.
If migration fails, destination volume will be marked as removed. And if migration succeeds, volume path will be rightly updated.

(cherry picked from commit d30d5644bb)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-04-17 15:42:02 +02:00
Likitha Shetty 55f2e45d14 CLOUDSTACK-8319. For both 'MigrateVolume' and 'MigrateVMWithVolumes, ensure VM's vconfiguration files are migrated along with VM's root volume.
(cherry picked from commit bdd28a45ed)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-04-17 15:42:02 +02:00
Sanjay Tripathi 30604e973a CLOUDSTACK-8386: CS is returning sec. storage even if it is full and there are other sec. storage with free space. 2015-04-14 16:10:40 +05:30
Rohit Yadav 694b723899 CLOUDSTACK-8312: Fix NPE regression, copy template can have NULL volume size
The copy command reply can have null size returned, so check and set values

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 53ca0b1861)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-03-20 17:27:14 +05:30
Rohit Yadav 53ca0b1861 CLOUDSTACK-8312: Fix NPE regression, copy template can have NULL volume size
The copy command reply can have null size returned, so check and set values

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-03-20 17:25:52 +05:30
Rohit Yadav c198dfdb7a Update pom and version usage to 4.5.1-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-03-12 12:13:18 +05:30
Likitha Shetty d30d5644bb CLOUDSTACK-8320. Upon a failed migration, a dummy volume is created which remains in 'Expunging' state.
Set destination volume path as NULL while duplicating volume during migration.
If migration fails, destination volume will be marked as removed. And if migration succeeds, volume path will be rightly updated.
2015-03-12 11:57:02 +05:30
Likitha Shetty bdd28a45ed CLOUDSTACK-8319. For both 'MigrateVolume' and 'MigrateVMWithVolumes, ensure VM's vconfiguration files are migrated along with VM's root volume. 2015-03-12 11:57:02 +05:30
Sanjay Tripathi 5aedeb74f9 CLOUDSTACK-8312: The storage usage of "Primary Storage Allocated" is incorrect.
(cherry picked from commit 520d96724a)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-03-11 13:07:47 +05:30
Sanjay Tripathi 520d96724a CLOUDSTACK-8312: The storage usage of "Primary Storage Allocated" is incorrect. 2015-03-10 16:52:25 +05:30
Rohit Yadav ef99f01e59 Merge remote-tracking branch 'anshul/CLOUDSTACK-6900'
This closes #68

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-03-02 16:40:44 +05:30
Rohit Yadav 8cfd374f04 CLOUDSTACK-8224: Don't try to unlock if template is not locked
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-02-06 17:43:12 +05:30
Likitha Shetty 7de7885a54 CLOUDSTACK-8125. VM fails to start on the first attempt after a cold migration.
Update volume chain_info to NULL during cold migration.
Otherwise during VM start, CCP will configure and try to power-on the VM with wrong disk information.

(cherry picked from commit 7b32b8a268)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-02-05 15:30:49 +05:30
Damodar 01cc1b816d CLOUDSTACK-7792: Usage Events to be captured based on Volume State Machine
(cherry picked from commit 781648fb10)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Conflicts:
	engine/orchestration/src/org/apache/cloudstack/engine/orchestration/VolumeOrchestrator.java
	engine/storage/volume/src/org/apache/cloudstack/storage/volume/VolumeServiceImpl.java
2015-02-05 15:22:21 +05:30
Koushik Das 6ff8752fd0 volume-upload: gc thread to clean partially uploaded files
Fixed multiple issues in the template code path
2015-01-31 17:25:19 +05:30
Koushik Das ae21f44c7b volume-upload: Volume uploaded using new POST based upload goes to inconsistent state on destroying SSVM
Added a check not to cleanup volume related entries when SSVM is destroyed for volumes upload using the POST based upload feature
2015-01-31 17:19:58 +05:30
Koushik Das 6598a65dfb volume-upload: gc thread to clean partially uploaded files
Fixed the following:
- Destroying volume in 'UploadAbandoned' state resulted in NPE
- Existing upload volume functionality interfered with this, added proper checks to prevent that
2015-01-31 17:13:22 +05:30
Mike Tutkowski bd6a89939d Adding/updating comments around the volume's hypervisor snapshot reserve field 2015-01-29 15:50:42 -07:00
Mike Tutkowski 1b51bbbf74 Update hypervisor snapshot reserve for the root volume earlier than when it is currently being set 2015-01-29 13:01:51 -07:00
Rajani Karuturi a710743871 volume upload: persisting into volume store ref only when SSVM is found 2015-01-29 16:55:24 +05:30
Rajani Karuturi b343fb79e3 volume upload: fixed NPE if SSVM is not up 2015-01-29 16:55:16 +05:30
Mike Tutkowski c2330f48dd Replace synchronized with the use of database locking 2015-01-27 17:36:21 -07:00
Mike Tutkowski c847831662 Do not allow deletion of the snapshot if the snapshot is being copied 2015-01-26 14:57:28 -07:00
Nitin Mehta abf4e5c646 CLOUDSTACK-8181: Introducing a new allocator called firstfitleastconsumed. The purpose of this allocator is to find hosts/pools with least capacity usage (in terms of percentage) within the cluster and use those resources first before others. This allocator can be used changing vm.allocation.algorithm. For hosts it would decide the least consumed host through the setting host.capacityType.to.order.clusters to base the usage on cpu or ram.
Reviewed-by: Prachi
2015-01-26 10:48:06 -08:00
Mike Tutkowski 8204a44890 When creating a template from a managed-storage snapshot, put the snapshot in the Copying state before performing the copy (don't want to do anything with the snapshot until this process has completed) 2015-01-25 23:29:27 -07:00
Mike Tutkowski 2094286c8a Adding support for creating a volume from a snapshot when the snapshot is on managed storage 2015-01-21 13:06:05 -07:00
Mike Tutkowski 0f84e042b9 Adding support for creating a volume from a snapshot when the snapshot is on managed storage 2015-01-20 15:24:33 -07:00
Rohit Yadav 4e01d61f87 engine: remove redundant import
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-01-20 11:45:34 +05:30
Min Chen ff70ef797c CLOUDSTACK-8101: volume sync not working as expected - MS restart during upload volume leaves volume in hung state.
(cherry picked from commit e559b15b6a)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Conflicts:
	engine/storage/volume/src/org/apache/cloudstack/storage/volume/VolumeServiceImpl.java
2015-01-20 11:43:36 +05:30
Nitin Mehta 1aeaf4dcc9 CLOUDSTACK-7920: NPE in the payload was causing the ssvm agent to not connect, fix it and also make sure that template/volume sync are robust that exceptions do not cause ssvm agent disconnect issues.
(cherry picked from commit 5213401ace)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Conflicts:
	engine/storage/volume/src/org/apache/cloudstack/storage/volume/VolumeServiceImpl.java
2015-01-20 11:39:34 +05:30
Rajani Karuturi ad6b7b30cb Volume upload: sending the nfsstore url in the metadata
decoding the metadata and deserialising it on receiving it
2015-01-19 18:04:14 +05:30
Abhinandan Prateek d8def3dc0b CLOUDSTACK-8164: Look for all host to avoid NPE
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit a1a601be5a)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-01-19 11:30:49 +05:30
Abhinandan Prateek a1a601be5a CLOUDSTACK-8164: Look for all host to avoid NPE
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-01-19 11:29:51 +05:30
Sanjay Tripathi 466e6d15a4 CLOUDSTACK-8097: Failed to create volume snapshot after vm live migration across clusters.
(cherry picked from commit 0c4128e024)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-01-18 17:59:31 +05:30
Likitha Shetty d807bf0a4e CLOUDSTACK-8107. Failed to create snapshot from volume when the task is performed repeatedly in zone wide primary Storage.
While taking a snapshot of a volume, CS chooses the endpoint to perform backup snapshot operation by selecting any host that has the storage containing the volume mounted on it.
Instead, if the volume is attached to a VM, the endpoint chosen by CS should be the host that contains the VM.

(cherry picked from commit a75a431373)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-01-18 17:51:19 +05:30
Sanjay Tripathi 8676ff26e0 CLOUDSTACK-8066: There is not way to know the size of the snapshot created.
(cherry picked from commit 9153b8bede)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-01-18 17:25:41 +05:30
Rohit Yadav d9bd235bf6 engine/storage: add NPE check, break if parent snapshot is null
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit c06ca09b54)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-01-18 17:20:56 +05:30
Rohit Yadav c06ca09b54 engine/storage: add NPE check, break if parent snapshot is null
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-01-18 17:20:09 +05:30
Koushik Das 053c160b0f volume upload: management server polling and upload status from agent
MS polling logic to query status for templates that are uploaded
2015-01-16 18:03:36 +05:30
Anshul Gangwar 91e1881be8 CLOUDSTACK-6900: This fixes volume migration failing after that volume is downloaded.
When we download volume then we create entry in volume_store_ref table.
We mark the volume entry to ready state once download_url gets generated.
When we migrate that volume, then again one more entry is created with same volume id.
Its state is marked as allocated. Later we try to list only one dataobject in datastore
for state transition during volume migration. If the listed volume's state is allocated
then migration passes otherwise it fails.

Below fix will remove the randomness and give priority to volume entry which is made for
migration (download_url/extracturl will be null in case of migration). Giving priority to
download volume case is not needed as there will be only one entry in that case so no randomness.
2015-01-16 15:15:14 +05:30
Koushik Das 627f5a62dc volume upload: management server polling and upload status from agent
MS polling logic to query status for volumes that are uploaded
2015-01-14 10:19:12 +05:30
Daan Hoogland 5251fb2dc3 CID-1261670 added null check 2015-01-05 16:01:13 +01:00
Likitha Shetty 7b32b8a268 CLOUDSTACK-8125. VM fails to start on the first attempt after a cold migration.
Update volume chain_info to NULL during cold migration.
Otherwise during VM start, CCP will configure and try to power-on the VM with wrong disk information.
2014-12-24 14:33:51 +05:30
Likitha Shetty a75a431373 CLOUDSTACK-8107. Failed to create snapshot from volume when the task is performed repeatedly in zone wide primary Storage.
While taking a snapshot of a volume, CS chooses the endpoint to perform backup snapshot operation by selecting any host that has the storage containing the volume mounted on it.
Instead, if the volume is attached to a VM, the endpoint chosen by CS should be the host that contains the VM.
2014-12-23 09:48:35 +05:30
Min Chen e559b15b6a CLOUDSTACK-8101: volume sync not working as expected - MS restart during upload volume leaves volume in hung state. 2014-12-19 16:51:41 -08:00
Min Chen c5d3a73c67 CLOUDSTACK-8101: volume sync not working as expected - MS restart during
upload volume leaves volume in hung state.
2014-12-19 16:43:02 -08:00
Min Chen ea634550fc CLOUDSTACK-8101: volume sync not working as expected - MS restart during upload volume leaves volume in hung state. 2014-12-19 15:51:52 -08:00
Sanjay Tripathi 0c4128e024 CLOUDSTACK-8097: Failed to create volume snapshot after vm live migration across clusters. 2014-12-19 14:26:48 +05:30
Rajani Karuturi 9bb6cf8452 volume upload: persisting the volume metadata
on calling GetUploadParamsForVolume, persisting the metadata to db
    validating the account limits and incrementing the appropriate limits
    encoded the metadata on management server using preshared key
2014-12-18 17:50:51 +05:30
Sanjay Tripathi 9153b8bede CLOUDSTACK-8066: There is not way to know the size of the snapshot created. 2014-12-17 16:00:19 +05:30
Hiroki Ohashi 9d31e59d5b CLOUDSTACK-7412: Can't create proper template from VM on S3 secondary storage environment
Signed-off-by: Rajani Karuturi <rajanikaruturi@gmail.com>
2014-11-27 17:07:27 +05:30
Hiroki Ohashi 0f8f6eff2c CLOUDSTACK-7412: Can't create proper template from VM on S3 secondary storage environment
Signed-off-by: Rajani Karuturi <rajanikaruturi@gmail.com>
2014-11-27 17:06:08 +05:30
Laszlo Hornyak 3577423da9 removed executable flags from java classes
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
2014-11-23 19:49:01 +01:00
Min Chen b85183ede7 CLOUDSTACK-7678:volumes are getting uploaded successfully with wrong
url.
2014-11-21 15:18:47 -08:00
Min Chen 9ac93d3e43 CLOUDSTACK-7678:volumes are getting uploaded successfully with wrong url. 2014-11-21 14:36:36 -08:00
Edison Su 4140811549 CLOUDSTACK-7947:
double check if parent snapshot is removed or not, when creating new
snapshot

Reviewed-by: Min
2014-11-19 16:13:05 -08:00
Edison Su bd79965329 CLOUDSTACK-7947:
double check if parent snapshot is removed or not, when creating new
snapshot

Reviewed-by: Min
2014-11-19 16:12:28 -08:00
Edison Su d856a2acad CLOUDSTACK-7946:
remove leftover state in volume and snapshot table in case of mgt server
shutdown during storage operation.
Reviewed-by: Min
2014-11-19 16:08:27 -08:00
Edison Su 67113ff0b2 CLOUDSTACK-7946:
remove leftover state in volume and snapshot table in case of mgt server
shutdown during storage operation.
Reviewed-by: Min
2014-11-19 16:06:37 -08:00
Edison Su c05cda0d28 CLOUDSTACK-7945:
set removed field in snapshots table in case of snaphsot failure.

Reviewed-by: Frank
2014-11-19 15:58:47 -08:00
Edison Su 5018db8b62 CLOUDSTACK-7945:
set removed field in snapshots table in case of snaphsot failure.

Reviewed-by: Frank
2014-11-19 15:58:07 -08:00
Prachi Damle 968ca060ee CLOUDSTACK-5853 cannot deploy vm with differing service storage tag and data disk storage tag
Changes:
- Reverting Marcus's fix since this issue has already fixed by https://issues.apache.org/jira/browse/CLOUDSTACK-5426
2014-11-18 14:07:51 -08:00
Prachi Damle c5399cdef0 CLOUDSTACK-5853 cannot deploy vm with differing service storage tag and data disk storage tag
Changes:
- Reverting Marcus's fix since this issue has already fixed by https://issues.apache.org/jira/browse/CLOUDSTACK-5426
2014-11-18 09:47:23 -08:00
Nitin Mehta 5213401ace CLOUDSTACK-7920: NPE in the payload was causing the ssvm agent to not connect, fix it and also make sure that template/volume sync are robust that exceptions do not cause ssvm agent disconnect issues. 2014-11-14 16:19:10 -08:00
Mike Tutkowski 2042660a68 Added a "long getUsedIops(StoragePool)" method to the PrimaryDataStoreDriver interface 2014-11-12 13:38:58 -07:00
Mike Tutkowski 18091e9642 Adding support for backend snapshots to Xenserver625StorageProcessor 2014-11-10 22:41:00 -07:00
Damodar 781648fb10 CLOUDSTACK-7792: Usage Events to be captured based on Volume State Machine 2014-11-10 15:19:02 +05:30
Devdeep Singh cf27ecd91d CLOUDSTACK-7721. Runtime exception while trying to release lock on template_spool_ref.
When a template a copied from a secondary to primary, we were trying to release a lock
twice, once in the create/copy base image function and in the create/copy base image
complete callback routine. This caused the exception as reported in the bug. Fixed by
updating the code make sure we release the lock in copy base image function only as
this is the place we took acquired the lock.
2014-11-10 13:22:39 +05:30
Mike Tutkowski ebb96f5e2f Handle the case where the volume a snapshot was created from is removed from the database 2014-11-06 19:44:46 -07:00
Edison Su 885bba926c should not duplicate vm instance id, during the volume migration
Conflicts:
	engine/storage/volume/src/org/apache/cloudstack/storage/volume/VolumeServiceImpl.java
2014-11-06 15:36:58 -08:00
Mike Tutkowski 92d9ef9c17 Handle the case where a snapshot is associated with a deleted volume 2014-11-06 15:34:39 -07:00
Mike Tutkowski 455ffc3342 Create a template from a SAN snapshot 2014-11-06 10:33:23 -07:00
Sanjay Tripathi 5a67fe7369 CLOUDSTACK-7842: wrong size column is getting updated with snapshot physical size in snapshot_store_ref table.
Also fixed the issue that snapshot size with hypervisor XS >= 6.2.5 is not getting updated in snapshot_store_ref table.
2014-11-05 16:44:54 +05:30
Sanjay Tripathi b844155240 CLOUDSTACK-7842: Wrong size column is getting updated with snapshot physical size in snapshot_store_ref table.
Also fixed the issue that snapshot size with hypervisor XS >= 6.2.5 is not getting updated in snapshot_store_ref table.
2014-11-05 16:42:28 +05:30
Sanjay Tripathi a53d39c1b6 CLOUDSTACK-7835: Deleted volumes with null UUID and no removed timestamp in database still appear.
Also removed CREATING -> DESTROY via DESTROYREQUESTED, which was causing the volume to get stuck in expunging
state.
2014-11-04 17:57:31 +05:30
Sanjay Tripathi 1d503bb853 CLOUDSTACK-7835: Deleted volumes with null UUID and no removed timestamp in database still appear.
Also removed CREATING -> DESTROY via DESTROYREQUESTED, which was causing the volume to get stuck in expunging
state.
2014-11-04 17:02:50 +05:30
Santhosh Edukulla 1aef5cba66 Fixed Coverity Issues 2014-11-03 16:45:30 +05:30
Edison Su 6b5c874fe9 should not duplicate vm instance id, during the volume migration 2014-10-30 21:27:51 -07:00
Hugo Trippaers 4ebaf0a583 Bump master version to 4.6.0-SNAPSHOT after branching 4.5.0-SNAPSHOT 2014-10-29 14:54:23 +01:00
Mike Tutkowski c7aa9b4439 Reorganized the snapshot logic a bit 2014-10-21 16:01:15 -06:00
Mike Tutkowski 3690574a23 Reply that reverting is not supported 2014-10-21 16:01:15 -06:00
Mike Tutkowski 8766c8ad43 Improve error handling for a host-side error 2014-10-21 16:01:14 -06:00
Mike Tutkowski 1b5bb7d8c6 Invoke grantAccess and revokeAccess one time each using a SnapshotInfo (where previously a VolumeInfo was being passed in)
Handle the case where a SnapshotInfo is passed into the grantAccess and revokeAccess methods
2014-10-21 16:01:14 -06:00
Mike Tutkowski 57dacf99a2 Changed "boolean connectVolumeToHost(VolumeInfo, Host, DataStore)" to "boolean grantAccess(DataObject, Host, DataStore)"
Changed "void disconnectVolumeFromHost(VolumeInfo, Host, DataStore)" to "void revokeAccess(DataObject, Host, DataStore)"
2014-10-21 16:01:14 -06:00
Mike Tutkowski b29265f159 Logic around granting and revoking access to the volume that backs the snapshot 2014-10-21 16:01:14 -06:00
Mike Tutkowski 0cea0346ae Support for back-end snapshots on primary storage 2014-10-21 16:01:14 -06:00
Mike Tutkowski 06f6b00cd1 Snapshot support 2014-10-21 16:01:13 -06:00
Mike Tutkowski e70dde09c9 Implementing snapshot-strategy support for reverting a volume to a snapshot 2014-10-14 22:04:18 -06:00
Mike Tutkowski 35a8b70799 Adding a snapshot strategy for systems that want to take snapshots that reside on their own system (as opposed to the default approach of taking a hypervisor snapshot and then copying it to secondary storage) 2014-10-14 12:26:40 -06:00
Santhosh Edukulla 0a9742f914 Fixed few coverity issues
Signed-off-by: Santhosh Edukulla <santhosh.edukulla@gmail.com>
2014-10-14 17:16:50 +05:30
Min Chen 0db0e5e3cc CLOUDSTACK-7700: Volume Snapshot Async Job returns Success for a failed
operation.
2014-10-13 15:03:59 -07:00
Min Chen 1e20600eca CLOUDSTACK-7700: Volume Snapshot Async Job returns Success for a failed
operation.
2014-10-13 14:26:27 -07:00
Nitin Mehta 50ee9810a8 CLOUDSTACK-6826: Improving the download url expiration where the expiration work would be handled by the ssvm that has the corresponding symlink created on it. In case it doesnt exist, then would be any one of the ssvm in the zone.
Also when the ssvm is destroyed all the download urls are expired to be cleaned up in the next run by the new ssvm.

(cherry picked from commit ce90837357)
2014-10-13 00:40:03 -04:00
Min Chen 93b9ef8d43 CLOUDSTACK-6478:Fix a typo in RemoteHostEndPoint.setId().
(cherry picked from commit aed36c2776)
2014-10-13 00:29:17 -04:00
Hugo Trippaers 160aa846a5 CLOUDSTACK-7665 File.separator shouldn't be used in this case, the
separator is fixed just like the rest of the path
(cherry picked from commit 47ac3e4fd1)
2014-10-13 00:19:44 -04:00