Commit Graph

1569 Commits

Author SHA1 Message Date
Rajani Karuturi 29dfeac857 CLOUDSTACK-8889: delete volume doesnt decrement primary store resource count
Primary Storage count for an account does not decrease when a Data Disk
is deleted belonging to the account unless the VM to which volume
belonged is destroyed

The resource counts are updated even before the disk is actually deleted
resulting in the same value.
Moved the resource counts updation to after the expunge operation as
thats when the disk is actually deleted.

all the tests in test/integration/component/test_ps_limits.py now pass
2015-10-26 17:40:27 +05:30
Rajani Karuturi ac80a2df50 Merge pull request #808 from karuturi/CLOUDSTACK-8835
CLOUDSTACK-8835: Added alerts incase of template download failureAuthored-By: @sanjaytripathi
Reviewed-By: @devdeep

* pr/808:
  CLOUDSTACK-8835: Added alerts incase of template download failure

Signed-off-by: Rajani Karuturi <rajani.karuturi@citrix.com>
2015-09-14 09:47:46 +05:30
Rajani Karuturi f888e93e44 Merge pull request #782 from karuturi/CLOUDSTACK-8816
Cloudstack 8816 entityuuid missing in some of the eventsIn some of the events generated, entity uuid was missing making it difficult to find the entity. Fixed the same.

Tested it on rabbitmq instance.
There are the events before after the fix:

Before
--------------------------------------------------------------------------------

routing_key: management-server.ActionEvent.ACCOUNT-DELETE.Account.*
exchange: cloudstack-events
message_count: 2
payload:
{"eventDateTime":"2015-09-04 17:59:24 +0530","status":"Scheduled","description":"deleting User test4 (id: 28) and accountId \u003d 28","event":"ACCOUNT.DELETE","Account":"c09e2e81-8edc-4c27-b072-25005b522b63","account":"bd73dc2e-35c0-11e5-b094-d4ae52cb9af0","user":"bd7ea748-35c0-11e5-b094-d4ae52cb9af0"}

payload_bytes: 304
payload_encoding: string
redelivered: False

--------------------------------------------------------------------------------

routing_key: management-server.AsyncJobEvent.complete.Account.*
exchange: cloudstack-events
message_count: 0
payload: {"cmdInfo":"{\"id\":\"9dd3abc2-3f8b-4852-aa60-a74b234acb13\",\"response\":\"json\",\"sessionkey\":\"5ig1ItP2_5v-mgY4cVJbJN5hw_w\",\"ctxDetails\":\"
{\\\"interface com.cloud.user.Account\\\":\\\"9dd3abc2-3f8b-4852-aa60-a74b234acb13\\\"}

\",\"cmdEventType\":\"ACCOUNT.DELETE\",\"expires\":\"2015-09-07T11:11:56+0000\",\"ctxUserId\":\"2\",\"signatureversion\":\"3\",\"httpmethod\":\"GET\",\"uuid\":\"9dd3abc2-3f8b-4852-aa60-a74b234acb13\",\"ctxAccountId\":\"2\",\"ctxStartEventId\":\"447\"}","instanceType":"Account","jobId":"5004989d-0cde-4922-8afa-66bf38b75ea7","status":"SUCCEEDED","processStatus":"0","commandEventType":"ACCOUNT.DELETE","resultCode":"0","command":"org.apache.cloudstack.api.command.admin.account.DeleteAccountCmd","jobResult":"org.apache.cloudstack.api.response.SuccessResponse/null/
{\"success\":true}

","account":"bd73dc2e-35c0-11e5-b094-d4ae52cb9af0","user":"bd7ea748-35c0-11e5-b094-d4ae52cb9af0"}
payload_bytes: 914
payload_encoding: string
redelivered: False

--------------------------------------------------------------------------------

After
--------------------------------------------------------------------------------

 routing_key: management-server.ActionEvent.ACCOUNT-DELETE.Account.e5e2db91-414d-484c-99d5-c4e265c14ad8
exchange: cloudstack-events
message_count: 13
payload: {"eventDateTime":"2015-09-07 17:32:26 +0530","status":"Completed","description":"Successfully completed deleting account. Account Id: 45","event":"ACCOUNT.DELETE","entityuuid":"e5e2db91-414d-484c-99d5-c4e265c14ad8","entity":"com.cloud.user.Account","account":"bd73dc2e-35c0-11e5-b094-d4ae52cb9af0","user":"bd7ea748-35c0-11e5-b094-d4ae52cb9af0"}
payload_bytes: 344
payload_encoding: string
redelivered: True

--------------------------------------------------------------------------------

routing_key: management-server.AsyncJobEvent.complete.Account.e5e2db91-414d-484c-99d5-c4e265c14ad8
exchange: cloudstack-events
message_count: 12
payload: {"cmdInfo":"{\"id\":\"e5e2db91-414d-484c-99d5-c4e265c14ad8\",\"response\":\"json\",\"sessionkey\":\"8AJVbn8HIpg5LZ_VaVfSPs_QN2k\",\"ctxDetails\":\"{\\\"interface com.cloud.user.Account\\\":\\\"e5e2db91-414d-484c-99d5-c4e265c14ad8\\\"}\",\"cmdEventType\":\"ACCOUNT.DELETE\",\"expires\":\"2015-09-07T12:17:42+0000\",\"ctxUserId\":\"2\",\"signatureversion\":\"3\",\"httpmethod\":\"GET\",\"uuid\":\"e5e2db91-414d-484c-99d5-c4e265c14ad8\",\"ctxAccountId\":\"2\",\"ctxStartEventId\":\"465\"}","instanceType":"Account","instanceUuid":"e5e2db91-414d-484c-99d5-c4e265c14ad8","jobId":"0bb08486-6d9f-4e9f-bfef-b7463c42e71b","status":"SUCCEEDED","processStatus":"0","commandEventType":"ACCOUNT.DELETE","resultCode":"0","command":"org.apache.cloudstack.api.command.admin.account.DeleteAccountCmd","jobResult":"org.apache.cloudstack.api.response.SuccessResponse/null/{\"success\":true}","account":"bd73dc2e-35c0-11e5-b094-d4ae52cb9af0","user":"bd7ea748-35c0-11e5-b094-d4ae52cb9af0"}
payload_bytes: 968
payload_encoding: string
redelivered: True

--------------------------------------------------------------------------------

* pr/782:
  CLOUDSTACK-8816 Systemvm reboot event doesnt have uuids. Fixed the same
  CLOUDSTACK-8816: Project UUID is not showing for some of operations in RabbitMQ.
  CLOUDSTACK-8816: entity uuid missing in create network event
  CLOUDSTACK-8816: instance uuid is missing in events for delete account
  CLOUDSTACK-8816 Fixed entityUuid missing in some cases is events

Signed-off-by: Rajani Karuturi <rajani.karuturi@citrix.com>
2015-09-14 09:42:44 +05:30
Sanjay Tripathi 9a35f87d37 CLOUDSTACK-8835: Added alerts incase of template download failure
Reviewed-By: Devdeep
2015-09-11 16:07:35 +05:30
Wei Zhou 007f5b842d CLOUDSTACK-5863: Add unit tests for create/delete/revert volume snapshot 2015-09-10 08:01:30 +02:00
Rajani Karuturi 37c3451aae CLOUDSTACK-8816 Fixed entityUuid missing in some cases is events
context parameters is Map<object,Object>. This has been used
interchangeably with String and class object resulting the param value
not being available in some cases if its put with object and get with
class.forName()

made the put and get consistent by using Object as key everywhere.
2015-09-09 14:23:27 +05:30
Wei Zhou 312cb877b1 CLOUDSTACK-5863: change response from SuccessResponse to SnapshotResponse 2015-08-28 08:54:11 +02:00
Wei Zhou 92344c006d CLOUDSTACK-5863: revert volume snapshot for KVM/QCOW2 2015-08-24 11:01:50 +02:00
Milamber 72e55029ef Improve debug message when the scheduler prepare to snapshot (volume 'id' and add volume name)
Signed-off-by: Milamber <milamber@apache.org>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

This closes #676

(cherry picked from commit 606afe341d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-14 14:12:46 +05:30
Mike Tutkowski 8b0266d12e Merge branch 'pr/547'
* pr/547:
  CLOUDSTACK-8601. VMFS storage added as local storage can be re-added as shared storage. Fail addition of a VMFS shared storage pool in case it has already been added as local storage in CS.

Signed-off-by: Mike Tutkowski <mike.tutkowski@solidfire.com>
2015-08-10 19:00:53 -06:00
Mike Tutkowski b1905c2040 CLOUDSTACK-8698: Retrieve a new device ID, if needed 2015-08-05 13:58:22 -06:00
Daan Hoogland e2b6237464 CLOUDSTACK-8656: debug messages on interupted exceptions 2015-07-30 16:03:10 +02:00
Likitha Shetty 13a98dd196 CLOUDSTACK-8601. VMFS storage added as local storage can be re-added as shared storage.
Fail addition of a VMFS shared storage pool in case it has already been added as local storage in CS.
2015-07-01 10:47:36 +05:30
Daan Hoogland bb613baa2b findbugs: repeated condition seems c&p error the tested states sugest that EXPUNGED should be the last one
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

This closes #491
2015-06-19 15:26:53 +03:00
Koushik Das 3f7e31ed05 CLOUDSTACK-8301: Enable configuring local storage use for system VMs at zone level
Made system.vm.use.local.storage a zone level configuration.
2015-05-19 21:32:14 +05:30
Devdeep Singh a99c9d0e68 Implementation for the ability to disable a storage pool for provisioning
... of new volumes. Following changes are implemented 1. Disable or enable a pool with the
updateStoragePool api. A new 'enabled' parameter added for the same. 2. When a
pool is disabled the state of the pool is updated to 'Disabled' in the db. On
enabling it is updated back to 'Up'. Alert is raised when a pool is disabled or
enabled. 3. Updated other storage providers to also honour the disabled state.
4. A disabled pool is skipped by allocators for provisioing of new volumes. 5.
Since the allocators skip a disabled pool for provisioning of volumes, the
volumes are also not listed as a destination for volume migration.

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Disabling+Storage+Pool+for+Provisioning

This closes #257

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-05-19 11:16:49 +01:00
Rajani Karuturi bd71fcb650 Fixed two coverity reported issues
Dereference after null check
Dm: Dubious method used

This closes #219
2015-05-01 16:08:26 +05:30
Rajani Karuturi 0b8355920e Merge branch 'volume-upload' into master
This closes #206
2015-04-29 11:12:53 +05:30
Rajani Karuturi 75ae90b073 volume-upload: added account level secondary storage resource limit checks 2015-04-24 15:48:44 +05:30
Koushik Das bc399b981f volume-upload: If SSVM is destroyed and started, then partially uploaded volumes/templates remain in
inconsistent state (NotUploaded or UploadInProgress) and doesn't transition to any terminal
state (Uploaded, Uploaded). As a result these volumes/templates cannot be removed. Fix is to handle such
volume/template entries correctly.
2015-04-22 19:28:51 +05:30
Rajani Karuturi f172fbef8a Bug-Id: CS-39350 Able to upload Volume greater than the Resource limit defined for Primary Storage.
Reviewed-By: Harikrishna Patnala <harikrishna.patnala@citrix.com>
2015-04-20 13:01:19 +05:30
Rajani Karuturi 4f35d3611a volume upload: fixed the error message incase of wrong image format 2015-04-16 17:43:46 +05:30
Rohit Yadav 06e353e5c7 vmware: support vmfs as local storage pool type
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 752d784d92)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-04-16 17:36:52 +05:30
Wido den Hollander 9836ccdcf3 CLOUDSTACK-8313: Allow overprovisioning of local storage pools 2015-04-14 17:30:27 +02:00
Koushik Das 4338bea070 volume upload: cannot delete a volume in NotUploaded state
Volume deletion is not allowed in NotUploaded or UploadInProgress states, added validation in delete volume API for same
2015-04-14 16:08:54 +05:30
Rohit Yadav a07497b373 server: fix NPE case in VolumeApiServiceImpl
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit df934c9541)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-04-13 15:47:51 +05:30
Rajani Karuturi ce823a3a26 Returning template/volume download percent in UploadStatusAnswer
Also updating the store ref entries on receiving the answer.
2015-04-09 15:38:58 +05:30
Sanjay Tripathi 5f59091418 CLOUDSTACK-8350: Volume attachment failed when user stops the VM during attaching DISK. 2015-03-27 15:26:32 +05:30
Koushik Das c62ce3eecb volume-upload: Templates in uploaded error state are also accounted in resource limits which leads to failure during template upload even though there aren't any active ones
Fixed the storage GC thread to clean up templates in error/abandoned states.
2015-03-24 09:21:42 +05:30
Damodar 27fa8a9453 CLOUDSTACK-7948: Two VOLUME.DELETE Events are being registered instead of one - On Destroying a User VM belonging to a Project 2015-03-23 16:27:21 +05:30
Rajani Karuturi 018023c1ef volume upload: added validation for file formats
merged TemplateUtils and ImageStoreUtil to a singe ImageStoreUtil
also added a unittest for ImageStoreUtil
2015-03-20 16:25:13 +05:30
Rajani Karuturi d5dffb5dc9 volume upload: added md5 checksum validation
also fixed the issue wherein the successful uploads where also moving to
error state as the channelinactive is called after the end of successful
upload as well.
added a fileuploaded boolean to check when the channel is inactive.
2015-03-18 12:20:51 +05:30
Funs c27c69438b hypervisors: add OVM3 plugin that supports OVM 3.2.1/3.3.x
This is a plugin that puts in ovm3 support ranging from 3.3.1 to 3.3.2. Basic
functionality is in here, advanced networking etc..

Snapshots only work when a VM is stopped now due to the semantics of OVM's raw
image implementation (so snapshots should work on a storage level underneath the
hypervisor shrug)

This closes #113

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-03-12 11:33:42 +05:30
Koushik Das 47a22e0594 volume-upload: Unable to download template which has been uploaded via POST URL
During upload, POST url is saved in template_store_ref DB table. Now during download, same url is incorrectly returned back.
Fixed the code to cleanup POST url from DB on successful template upload.
2015-03-11 17:55:04 +05:30
Koushik Das 1dae3a4a13 volume-upload: Volume resource count is incremented even for failed and abandoned volumes
Added logic to clean up abandoned and failed volume uploads. This is done as part of storage GC.
2015-03-11 16:24:03 +05:30
Rajani Karuturi b16520bcec volume upload: added max size check for temaplte/volume post upload
used the existing configuration variables max.template.iso.size and
storage.max.volume.upload.size for templates and volumes respectively.
2015-03-03 17:27:40 +05:30
Rajani Karuturi dd1a8da977 volume upload: Volume entry created in DB even though GetUpload* API fails with "No ssvm present" error
Signed-off-by: Rajani Karuturi <rajanikaruturi@gmail.com>
2015-02-27 17:42:32 +05:30
Rohit Yadav e1827fc696 CLOUDSTACK-8285: check and update IP capacity states based on allocation state
- Backend should update if state was diabled and now has changed
- UI's fetch latest does not actually fetch latest

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 985a61652e)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-02-25 16:47:40 +05:30
Koushik Das dc870b5114 volume upload: Size is listed as zero for uploaded volumes
Updating the virtual size for template/volume when upload is completed.
2015-02-19 17:13:47 +05:30
Koushik Das 075c841052 volume upload: cannot delete a volume in NotUploaded state Added exception handler in the upload status monitor thread. 2015-02-17 12:31:41 +05:30
Koushik Das d19ea5226a volume upload: Use volume/template UUID instead of ID in UploadStatusCommand 2015-02-17 12:31:40 +05:30
Rajani Karuturi b616894c21 volume upload: fixed issues during request validation
adding the encoded metadata to the signature
2015-02-17 11:50:58 +05:30
Rajani Karuturi 5c4fd12a9b volume upload: using the timeout from configuration
removed the hardcoded 60 min
2015-02-17 11:33:29 +05:30
Rajani Karuturi 3814677977 volume upload: getUploadParamsForVolume url has IP address instead of hostname, SSL fails 2015-02-17 11:31:44 +05:30
Koushik Das 121ff19cba volume-upload: Attaching uploaded volume to an instance failing while copying from secondary to primary storage
The volume/template size and install path was not getting updated properly.
The upload status response should return these properties on completion of upload.
2015-01-31 17:27:46 +05:30
Koushik Das c9908d2678 volume-upload: management server polling and upload status from agent
Some more fixes to the polling logic
2015-01-31 17:10:46 +05:30
Koushik Das 3e857bde08 volume-upload: management server polling and upload status from agent
Fixed some issues with the polling logic to handle scenarios when SSVM is destroyed/down.
Also changed the status of the volume_store_ref entries so that subsequent operations work fine.
2015-01-31 17:02:18 +05:30
Mike Tutkowski 5f16bf746b When canceling maintenance mode, the logic was looking at the id field of the work object instead of the id field of the VM object. 2015-01-29 23:11:29 -07:00
Mike Tutkowski 31f67c2b3c If the HypervisorType of a storage pool is Any, we need to retrieve hosts in the given zone for each HypervisorType. 2015-01-29 19:18:04 -07:00
Mike Tutkowski bd6a89939d Adding/updating comments around the volume's hypervisor snapshot reserve field 2015-01-29 15:50:42 -07:00