Commit Graph

389 Commits

Author SHA1 Message Date
Alena Prokharchyk d66e9c7b51 CLOUDSTACK-2843: Provideded an ability to turn off the synchronization being done on the CS agent side,
for number of commands participating in Vm deployment process, as parallel deployment is supported on the hypervisor side.
The behavior is controlled by global config varirables:

"execute.in.sequence.hypervisor.commands" (false by default) sets/resets the synchronization for commands:
=========================
StartCommand
StopCommand
CreateCommand
CopyVolumeCommand

"execute.in.sequence.network.element.commands" (false by default) sets/resets the synchronization for commands:
==========================
DhcpEntryCommand
SavePasswordCommand
UserDataCommand
VmDataCommand

As a part of the fix, increased the global lock timeout to 30 mins in several VR scripts:
===========================
edithosts.sh
savepassword.sh
userdata.sh

to support situations when multiple concurrent calls to the script are being made.
2013-06-19 10:12:13 -07:00
Dave Brosius 3a02942123 set rpcProvider field correctly
Signed-off-by: Dave Brosius <dbrosius@mebigfatguy.com>
Signed-off-by: Prasanna Santhanam <tsp@apache.org>
2013-06-17 19:12:02 +05:30
Edison Su c2da4eac89 fix NPE for cache ref cnt 2013-06-14 17:24:31 -07:00
Edison Su a715eb8121 clean up storage related code, and add lru replacement algorithm for cache storage 2013-06-14 15:55:59 -07:00
Min Chen bb85a564da Remove state from vm_template table, we should use template_store_ref
state column to represent the state of a template in each zone.
2013-06-14 15:14:09 -07:00
Min Chen 7a3e7d1891 Send CopyCommand to Host instead of randomly picking host and ssvm in
backup snapshot. For VMware, it will delegate to ssvm in VMwareGuru.
2013-06-13 11:21:25 -07:00
Min Chen 18aeef3ef1 Merge branch 'master' (up to commit
c30d9be3ce) into object_store.
2013-06-12 11:20:32 -07:00
Nitin Mehta 07e5cbe813 CLOUDSTACK-2932
Allow deleting of snapshots that have errored out. Simply mark the removed column as there is no physiocal clean up required
2013-06-11 16:23:21 +05:30
Min Chen 3bf15bd1ff Code cleanup. 2013-06-10 18:03:15 -07:00
Min Chen cc0de88088 Remove old DeleteTemplateCommand, DeleteVolumeCommand,
DeleteSnapshotBackupCommand, replaced by DeleteCommand.
2013-06-10 17:55:30 -07:00
Min Chen 04b5f53392 Removed unused code handling for DeleteSnapshotBackupCommand since we
don't send that command anymore.
2013-06-07 17:55:05 -07:00
Sanjay Tripathi da5c4619c3 CLOUDSTACK-2286: Volume created from snapshot state is in allocated state instead of Ready state which is letting Primary storage not to increment the resources. 2013-06-07 17:17:54 +05:30
Min Chen 03f4c60360 Add missing new file. 2013-06-06 18:01:25 -07:00
Min Chen 66e702222d Simplify various DeleteTemplateCommnad, DeleteVolumeCommand and
DeleteSnapshotBackupCommand to use one DeleteCommand, also provide
BaseImageStoreDriverImpl class for plugin to inherit to avoid code
duplication.
2013-06-06 18:00:38 -07:00
Min Chen ef03d5a122 Move data store specific extract template/iso logic from
TemplateManager to data store driver.
2013-06-04 11:50:43 -07:00
Sateesh Chodapuneedi 7998413f48 CLOUDSTACK-2029 zone wide primary storage support for cloudstack over vmware deployments
Added hypervisor type to CreateStoragePoolCmd & Storage pool responses.
DatastoreLifeCycle would consider hypervisor type while attaching datastore to zone.
ZoneWideStoragePoolAllocator would filter zone wide primary storage pools by hypervisor type along with tags in disk profile.
hypervisor type is mandatory parameter if scope is specified as ZONE while creating primary storage pool.
As of now KVM, VMware are allowed to use ZoneWideStoragePoolAllocator.

Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
2013-06-01 00:05:27 +05:30
Min Chen eb93efdaa4 Add convenient delete methods in DataObject to delete dangling data
object.
2013-05-30 14:51:03 -07:00
Min Chen 614e08e19f Fix ExtractVolume bug for S3 as secondary storage. 2013-05-30 14:19:59 -07:00
Min Chen efbf9c8635 Move ExtractVolumeCmd from ManagementServerImpl to VolumeManagerImpl,
also add support for S3 for ExtractVolumeCmd.
2013-05-29 22:27:07 -07:00
Edison Su 0a5228922b clean up storage related coded 2013-05-29 18:04:37 -07:00
Min Chen bc91e7692d Fix build. 2013-05-29 15:22:04 -07:00
Min Chen 27133fba7d Simplify clean up snapshots logic in secondary storage and consolidate
to use one agent command DeleteSnapshotBackupCommand for snapshot
deletion task by removing CleanupSnapshotBackupCommand.
2013-05-25 20:48:15 -07:00
Edison Su 0ed441c690 CLOUDSTACK-2637: fix upload volume 2013-05-24 19:00:52 -07:00
Min Chen 89f351c2e5 Clean up template sync code, and update template state in template sync
phase instead of doing it in adding image store.
2013-05-24 11:56:36 -07:00
Min Chen c614c6a424 CLOUDSTACK-2674: Secondary Storage garbage collector failed with NPE in
case of S3 storage provider.
2013-05-24 11:10:45 -07:00
Wido den Hollander bef3a2edb7 CLOUDSTACK-1191: rbd: Use cloning for deploying templates instead of a copy
RBD format 2 supports cloning (aka layering) where one base image can serve
as a parent image for multiple child images.

This enables fast deployment of a large amount of virtual machines, but it also
saves spaces on the Ceph cluster and improves performance due to better caching.

Qemu-img doesn't support RBD format 2 (yet), so to enable these functions the
RADOS/RBD Java bindings are required.

This patch also enables deployment of System VMs on RBD storage pools. Since we
no longer require a patchdisk for passing the boot arguments we are able to deploy
these VMs on RBD.
2013-05-24 16:00:08 +02:00
Edison Su 3ec52807f1 CLOUDSTACK-2593: fix migrate volume between pools through secondary storage 2013-05-23 18:26:16 -07:00
Min Chen d1704a389a Exclude removed template from template sync. 2013-05-23 17:23:25 -07:00
Min Chen 1e21b0b0e8 Handle null host ip from NetUtils.getDefaultHostIp(). 2013-05-23 13:05:01 -07:00
Min Chen 8d08f9b74b CLOUDSTACK-2655: use ssvm public IP to construct extract url. 2013-05-23 12:05:58 -07:00
Min Chen 8960957605 CLOUDSTACK-2634:all delta snapshots relating to volume should be created
in the same secondary storage as the first snapshot.
2013-05-23 10:45:47 -07:00
Min Chen 039098469a CLOUDSTACK-2584: set Format value in CopyCommand. 2013-05-22 14:33:40 -07:00
Edison Su 5af888c67a add image format in volumevo 2013-05-22 13:38:11 -07:00
Min Chen f23f341922 Handle CopyCmdAnswer in VolumeObject.processEvent for image and
imageCache, and fix a bug in selecting endpoint.
2013-05-21 22:15:36 -07:00
Edison Su 383be568b3 CLOUDSTACK-2527: enable attach/detach xs tool iso 2013-05-21 19:13:29 -07:00
Edison Su 827a1a229b CLOUDSTACK-2584: fix create template from s3 2013-05-21 16:55:41 -07:00
Min Chen 235825dc33 Properly updated destroyed column in template_store_ref and
volume_store_ref in deleting template or volume.
2013-05-21 16:51:12 -07:00
Min Chen 2af36aa87b Clean up template_zone_ref entries in case of create template failure. 2013-05-21 12:00:59 -07:00
Min Chen 8f549db518 Clean up entries in template_store_ref, volume_store_ref and
snapshot_store_ref in case of operation failure.
2013-05-21 12:00:04 -07:00
Min Chen 2f6d94462e Support multiple secondary storages for backup snapshot. 2013-05-20 21:36:50 -07:00
Min Chen 5be7f7ddb1 VolumeObject.processEvent for ImageCache should only change state in
volume_store_ref, not volume table itself.
2013-05-20 21:25:59 -07:00
Min Chen b9378ae5ad Fix cache storage selection in case of zone scope id passed is null. 2013-05-20 16:44:31 -07:00
Min Chen 565bdfb27b Fix CLOUDSTACK-2525: clean up db entries in vm_template and
template_store_ref in case of template creation failure.
2013-05-20 16:21:45 -07:00
Rajesh Battala 107f492475 Fixed CLOUDSTACK-2081 Volume which is added thru upload volume is failed to attach to the instance saying Volume state must be in Allocated, Ready or in Uploaded state( Though uploaded Volume state is uploaded) 2013-05-17 14:58:25 +05:30
Edison Su f7c1b711ad merge to master 2013-05-16 23:56:20 -07:00
Prachi Damle a2eb7bab1e CLOUDSTACK-2056: DeploymentPlanner choice via ServiceOffering
- Changes merged from planner_reserve branch
- Exposing deploymentplanner as an optional parameter while creating a service offering
- changes to DeploymentPlanningManagerImpl to make sure host reserve-release happens between conflicting planner usages.
2013-05-16 15:02:17 -07:00
Edison Su 389d7c1fa0 if object is already stored in cache store, don't download again 2013-05-16 14:34:06 -07:00
Edison Su c476d7866e CLOUDSTACK-2482: fix create volume from snapshot 2013-05-15 19:41:50 -07:00
Min Chen b66c484905 Fix size issue in copying template from S3 to cache storage to make
system vm up.
2013-05-15 18:54:55 -07:00
Edison Su a0c3d280b4 merge to master 2013-05-15 00:40:53 -07:00
Edison Su ed065418b6 add license header 2013-05-14 21:58:07 -07:00
Edison Su 879a954b98 delete snapshot at the backend 2013-05-14 21:58:07 -07:00
Edison Su 68ffe1c706 fix bug in create volume from snapshot 2013-05-14 16:38:02 -07:00
Min Chen 252f384e89 Fix CLOUDSTACK-2485: ClassCastException in extracting ISO. 2013-05-14 14:57:02 -07:00
Min Chen 87af4ddab0 Only download eligible system template for S3 image store. 2013-05-14 14:57:01 -07:00
Edison Su 12583bbffc snapshot path is accountId + volumeid 2013-05-14 13:20:10 -07:00
Min Chen 2102ede2e7 Fix CLOUDSTACK-2469: default template is not downloaded. 2013-05-14 11:42:24 -07:00
Min Chen 3424ac86ad Fix size issue for template created from snapshot. 2013-05-13 17:45:22 -07:00
Edison Su 0ce01e56db set template size when creating template from snapshot 2013-05-13 16:45:03 -07:00
Min Chen 9777e56e8e Update built-in template size in vm_template table. 2013-05-13 15:46:01 -07:00
Min Chen 5341f299ff We still need the hack of Ready to Ready transition for NFS. 2013-05-13 15:46:00 -07:00
Edison Su 1694c1a502 fix download timeout 2013-05-13 14:40:49 -07:00
Min Chen e6faa47a4b Remove unnecessary state machine entry after refactoring
sendMessageAsyncWithListener to use sendMessageAsync.
2013-05-13 11:36:44 -07:00
Min Chen c13a691429 Fix some inaccurate logging message in template sync. 2013-05-13 11:27:34 -07:00
Min Chen ff735b4d81 Fix a bug in searching for eligible image store when we pass a ZoneScope
with scopeId = null.
2013-05-13 11:07:06 -07:00
Nitin Mehta f84b6abde0 merge with master 2013-05-13 17:30:42 +05:30
Rajesh Battala 756a4179e2 Fixed CLOUDSTACK-2189 Volume is stuck in resizing state
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
2013-05-13 14:41:00 +05:30
Edison Su 492127c035 fix kvm 2013-05-12 18:27:37 -07:00
Min Chen 0b9e0c0f3a Override getRootDir in LocalNfsSecondaryStorageResource to make mount
successfully on MS side. Now S3 is working.
2013-05-12 17:41:44 -07:00
Edison Su e8259e38c2 integration test is passed again 2013-05-12 14:41:06 -07:00
Min Chen 0d580ccb26 Fix NPE in adding host and also put back lost change in deployDataCenter
due to rebase with master.
2013-05-11 22:00:40 -07:00
Min Chen 1bd216fc48 Merge with latest code rebased from master 2013-05-11 21:05:54 -07:00
Min Chen f8e51f70a1 Invoke reatetmplt.sh script after copying template from S3 to cache
storage.
2013-05-11 18:19:29 -07:00
Nitin Mehta c11dbad9c9 merge master 2013-05-11 15:28:43 +05:30
Edison Su 993187622e fix compile, after rebase to master 2013-05-10 20:39:02 -07:00
Edison Su 90de46c4fe get vmware works 2013-05-10 19:26:44 -07:00
Min Chen 8a9a7a4adc Fix a bug in create cache object for S3. 2013-05-10 17:53:32 -07:00
Alex Huang 342624e0eb Merged with head 2013-05-10 16:33:57 -07:00
Alex Huang 3047929367 Merged 2013-05-10 16:21:43 -07:00
Edison Su 15fc4e137d add license header 2013-05-10 11:40:11 -07:00
Edison Su e444867e61 fix compile 2013-05-10 09:47:09 -07:00
Edison Su 42e25a22fc refactor kvm/vmware resource code 2013-05-09 23:10:05 -07:00
Min Chen 2c22488ed7 Filter out cache store in find image store by provider. 2013-05-09 21:49:12 -07:00
Min Chen 6f4fcf741c Fix listImageStores to only list store with Image role, excluding those
cache stores since they are also stored in image_store table.
2013-05-09 21:49:12 -07:00
Min Chen f965e2d892 Download system template while adding S3 image store is working. 2013-05-09 21:49:12 -07:00
Sanjay Tripathi 3aed20c9d2 CLOUDSTACK-1603: Management server .log Java Exceptions are displayed with Alerts 2013-05-10 10:18:37 +05:30
Min Chen 292e77b96b Fix marvin test to add S3 image store and cache storage. 2013-05-09 16:19:06 -07:00
Min Chen fa11575212 Add store_role to template_store_ref, and add search function to find
template_store_ref entry based on DataStoreRole and ZoneId.
2013-05-08 22:06:07 -07:00
Min Chen dc5d2f45e9 Make copyFromS3ToNFS flow work. 2013-05-08 17:39:46 -07:00
Min Chen 3b45bc9c30 Make S3 register template work. 2013-05-08 11:12:26 -07:00
Min Chen f8edb55497 Register template, delete template and create instance from registered
template are working on NFS.
2013-05-08 10:29:09 -07:00
Edison Su ac7be218d5 creat template from volume/snapshot, and then create vm from template works now 2013-05-07 20:18:19 -07:00
Edison Su 5aeca646ae make create template from volume/snapshot work 2013-05-07 20:18:19 -07:00
Min Chen d79b890904 Change VMTemplate state in destroy. 2013-05-07 17:45:06 -07:00
Min Chen 8984e430ce Add state transition for destroy scenario. 2013-05-07 17:45:06 -07:00
Min Chen 3315b23595 Relax url port validation and some MockLocalNfsSecondaryStorageResource
fix.
2013-05-07 13:34:27 -07:00
Min Chen 41272c976e Add DeleteTemplate unit test. 2013-05-06 23:02:28 -07:00
Min Chen 26424ff2a8 Make url in AddImageStoreCmd not required since S3 does not have url. 2013-05-06 22:14:49 -07:00
Min Chen e7f6333209 Merge branch 'object_store' of https://git-wip-us.apache.org/repos/asf/cloudstack into object_store 2013-05-06 21:21:40 -07:00
Edison Su 0e28772957 fix race condition, when multiple vms on the same templates are started at the same time 2013-05-06 19:04:41 -07:00
Edison Su 1b8e7aba73 uncomment out template sync 2013-05-06 18:28:47 -07:00
Edison Su 9f7bad2cef fix creating snapshot 2013-05-06 18:27:15 -07:00
Edison Su 2d544e7127 volume related operation works 2013-05-06 18:27:15 -07:00
Min Chen f5d5526d41 Fix unit test to test NFS template download flow. 2013-05-06 18:02:52 -07:00
Alex Huang 3722d66aae Fixed up unit testing to use only an in class TestConfiguration 2013-05-06 16:34:27 -07:00
Min Chen 0d78209b73 Address various UI requirements by introducing provider constants. 2013-05-06 15:12:33 -07:00
Min Chen 7ad263b67e Revert previous workaround for ListTemplateCommand, which is correct
after updating the systemvm.iso on devcloud.
2013-05-06 11:29:57 -07:00
Min Chen a221ecb484 Workaround an issue of null url when ListTemplateCommand is received on
SSVM side, also fix a bug in listTemplates with id passed.
2013-05-04 23:09:23 -07:00
Edison Su 23d6f12e9a fix db select 2013-05-03 15:44:38 -07:00
Edison Su acafccfbc6 fix volume state change 2013-05-03 15:01:09 -07:00
Min Chen 23db72d8b4 S3 template download test is working except state transition part. 2013-05-03 14:59:13 -07:00
Min Chen df7a56d63f Removed redundant getInstallPath to use getPath in VolumeInfo. 2013-05-03 10:01:23 -07:00
Edison Su fc74d76696 add volume test case 2013-05-03 09:53:06 -07:00
Edison Su 0c6e87c701 download template to primary storage works now 2013-05-03 09:45:31 -07:00
Min Chen c60c373a38 Modify DownloadCommand to pass DataTO. 2013-05-02 23:05:19 -07:00
Min Chen 87fafb7287 Add S3 register template integration test. 2013-05-02 15:20:46 -07:00
Edison Su 2384276536 make template download testable 2013-05-02 00:38:11 -07:00
Edison Su 164bedd5ae revive java integration test 2013-05-01 18:24:47 -07:00
Edison Su 4d992fd18f fix template download to primary storage 2013-05-01 17:51:24 -07:00
Min Chen 2812873ffa Remove methods of searching for secondary storages from host table from
SecondaryStorageManagerImpl.
2013-05-01 17:35:51 -07:00
Min Chen 8a5bd9e8f8 Fix a sql bug in selecting end point to send message. 2013-05-01 16:04:40 -07:00
Edison Su 52799f46a4 fix data motion 2013-05-01 14:00:53 -07:00
Min Chen 3c6b7c2a01 Populate system vm template to template_store_ref in adding an image
store.
2013-05-01 12:10:24 -07:00
Min Chen 36c1538f06 Bugfix to make deployDataCenter.py work. 2013-04-30 17:42:48 -07:00
Min Chen bbd5d13213 Fix bugs in refactored listTemplates and listIsos API. 2013-04-30 11:31:45 -07:00
Edison Su 2f689171e0 refactor snapshot 2013-04-29 18:51:14 -07:00
Edison Su 37cbe8890f refactor snapshot 2013-04-29 18:51:13 -07:00
Min Chen 91bfbdf1c4 Handle ListTemplateCommand in NfsSecondaryStorageResource for various
data store provider.
2013-04-26 12:25:06 -07:00
Nitin Mehta d1642a489c =initial changes 2013-04-25 18:29:57 +05:30
Edison Su 4029e7af44 refactor data motion service for volumes 2013-04-24 18:53:57 -07:00
Min Chen 4c174a6867 Fix DeleteSnapshotBackupCommand to pass dataStore information. 2013-04-24 17:30:38 -07:00
Min Chen 5b76e4914c Remove sendToSecStorage methods from agentManager to use EndPoint
instead.
2013-04-24 16:21:41 -07:00
Min Chen 8d29cdf804 Remove SnapshotVO.getSecHostId and SnapshotVO.setSecHostId references
from codebase to use SnapshotDataStoreVO.
2013-04-24 11:54:42 -07:00
Min Chen 792fc471a2 Remove VolumeHostDao references from codebase. 2013-04-24 11:11:58 -07:00
Min Chen 0cfef3aa4e Remove almost all VMTemplateHostDao references except S3Manager and
SwiftManager.
2013-04-23 17:51:54 -07:00
Min Chen 06b3092083 Remove VMTemplateHostDao reference from VolumeManagerImpl,
StorageManagerImpl and SecondaryStorageManagerImpl.
2013-04-23 12:10:40 -07:00
Min Chen e40a06deae Fix extractTemplateCmd. 2013-04-22 21:48:57 -07:00
Edison Su b8c5c67fbc add copycommand at resouce side 2013-04-22 19:12:50 -07:00
Min Chen 29687663e8 Rename SnapshotStrategy to SnapshotService to have consistent naming
convention for Template, Snapshot, Volume. Also rename CopyCmd to
CopyCommand to follow internal command naming convention.
2013-04-22 16:15:57 -07:00
Min Chen 5f90aa971a Implement createTemplateCmd and consolidate several copy calls in
TemplateServiceImpl.
2013-04-22 15:33:43 -07:00
Edison Su b30e39b0e3 CLOUDSTACK-1392: add create template from snapshot and volume 2013-04-22 13:24:10 -07:00
Min Chen 9c584b5500 Use EndPoint to send local/remote command, and hide agentMgr message
passing.
2013-04-22 13:21:28 -07:00
Min Chen ffdf567b58 Add implemention to pick EndPoint for secondary storage 2013-04-22 13:19:04 -07:00
Edison Su abf40435f3 refactor downloadlistener, sync system vm templates when adding a new image store 2013-04-21 22:46:02 -07:00
Min Chen 0229c75b50 Use data motion service for PrepareTemplateCmd to move template from
secondary storage to primary.
2013-04-20 20:44:31 -07:00
Min Chen 86913ab4d3 Use data motion service to implement copy template. 2013-04-19 23:01:59 -07:00
Min Chen 6f70fe28e8 Trigger system vm template download while adding image store. Just a
code skeleton, waiting for some code in EndPoint.
2013-04-19 19:37:06 -07:00
Min Chen b8229349f5 Rename HypervisorHostEndPoint to RemoteHostEndPoint to accommodate ssvm
as well.
2013-04-19 17:46:33 -07:00
Devdeep Singh b01f63123f CS-1879: NPE while migrating volume. The state transitions on the volume that has to be migrated were incorrect. A volume to be migrated is in ready state and cannot be transitioned to copying state. Similarly, the duplicated volume is in ready state too and cannot be transitioned to creating state. Fixed it by transitioning the volume to migrating state when a migrateVolume api call is made. Also, a volume has to be destroyed first before it can be expunged. Fixed that too.
Signed-off-by: Abhinandan Prateek <aprateek@apache.org>
2013-04-19 16:15:24 +05:30
Devdeep Singh 21ce3befc8 Storage motion for Xenserver changes: 1. Implemented Api findStoragePoolsForMigration. Added a new response objects to list storage pools available for migration. 2. Updated migrateVolume api for allowing migrating volumes of running vms. These changes are integrated into the latest storage refactoring changes. 3. Added the implementation for findHostsForMigration api. It lists the hosts to which an instance can be migrated, including hosts from within and across clusters to which an instance may be migrated with storage motion. The work of migrating a volume of a running vm is also done in copyAsync. 4. Updated the listHosts api for backward compatibility. 5. Added the implementation for migrateVirtualMachineWithVolume api. It migrates an instance with its volumes within a cluster and also across clusters. Also introduced a new XenServerStorageMotionStrategy for migrating volumes of a vm. When a vm is being migrated with its volumes, the vm is put in migrating state and a request is send to the volume manager to migrate the vm and its volumes. Volume manager calls into the volume service which forwards the request to data motion service after moving all the volumes to migrating state. Data motion service enumerates the strategies and the request reaches the XenServerStorageMotionStrategy. It calls in to the resource to complete the operation. 6. Resolved an issue where storage xenmotion of 2nd VM created from the same template to a host was failing with duplicate_vm exception. Made changes to remove the mac_seed key value pair from other_config when vms are created. This is was storage motion to fail. 7. Updated the db upgrade schema script. 8. Added the right permissions in commands.properties 9. Marvin tests for testing storage motion. Following scenarios are tested. 9.1. A virtual machine is migrated to another host. Its volumes are also migrated to another storage pool. 9.2. Just the volumes of a vm are migrated to another storage pool while the vm continues to run on the same host. 10. Unit tests for testing migration of a vm with its volumes.
Signed-off-by: Abhinandan Prateek <aprateek@apache.org>
2013-04-19 11:36:42 +05:30