Commit Graph

171 Commits

Author SHA1 Message Date
Mike Tutkowski 568119d437
Merge pull request #2585 from syed/upstream-snapshot-archive
Add ability to archive snapshots on primary storage
2018-08-23 19:36:04 -06:00
Rafael Weingärtner 85c88e08a7
Remove class snapshot data factory test (#2813)
* rename package from "src" to "org.apache.cloudstack.storage.vmsnapshot"

* Remove Empty test class "SnapshotDataFactoryTest"

* Remove redundant imports (importing a class from its own package)
2018-08-20 10:16:07 -03:00
Rafael Weingärtner 21c0225921
Fix the problem at #1740 when it loads all snapshots in the primary storage (#2808)
* Fix the problem at #1740 when it loads all snapshots in the primary storage

While checking a problem with @mike-tutkowski at PR #1740, we noticed that the method `org.apache.cloudstack.storage.snapshot.SnapshotDataFactoryImpl.getSnapshots(long, DataStoreRole)` was not loading the snapshots in `snapshot_store_ref` table according to their storage role. Instead, it would return a list of all snapshots (even if they are not in the storage role sent as a parameter) saying that they are in the storage that was sent as parameter.

* Add unit test
2018-08-17 08:33:25 -03:00
Syed Ahmed cd70ede3c2 Add ability to archive snapshots on primary storage 2018-05-17 04:39:34 -04:00
Rohit Yadav 41895561a7 Merge branch '4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-30 16:21:06 +05:30
Mike Tutkowski e68f5cea67 Only use the host if its Resource State is Enabled. (#2512) 2018-03-29 17:43:22 +00:00
Marc-Aurèle Brothier 893a88d225 CLOUDSTACK-10105: Use maven standard project structure in all projects (#2283)
Remove maven standard module (which only a few were using) and get ride of maven customization for the projects structure.

- moved all directories to src/main/java, src/main/resources, src/main/scripts, src/test/java, src/test/resources
- grep scan to search for src/com and src/org left over
- grep for <project>/scripts to fix pom.xml configuration
- remove custom <build> configuration in pom.xml

Signed-off-by: Marc-Aurèle Brothier <m@brothier.org>
2018-01-20 03:19:27 +05:30
Mike Tutkowski a30a31c9b7 CLOUDSTACK-9620: Enhancements for managed storage (#2298)
Allowed zone-wide primary storage based on a custom plug-in to be added via the GUI in a KVM-only environment (previously this only worked for XenServer and VMware)

Added support for root disks on managed storage with KVM

Added support for volume snapshots with managed storage on KVM

Enable creating a template directly from a volume (i.e. without having to go through a volume snapshot) on KVM with managed storage

Only allow the resizing of a volume for managed storage on KVM if the volume in question is either not attached to a VM or is attached to a VM in the Stopped state.

Included support for Reinstall VM on KVM with managed storage

Enabled offline migration on KVM from non-managed storage to managed storage and vice versa

Included support for online storage migration on KVM with managed storage (NFS and Ceph to managed storage)

Added support to download (extract) a managed-storage volume to a QCOW2 file

When uploading a file from outside of CloudStack to CloudStack, set the min and max IOPS, if applicable.

Included support for the KVM auto-convergence feature

The compression flag was actually added in version 1.0.3 (1000003) as opposed to version 1.3.0 (1003000) (changed this to reflect the correct version)

On KVM when using iSCSI-based managed storage, if the user shuts a VM down from the guest OS (as opposed to doing so from CloudStack), we need to pass to the KVM agent a list of applicable iSCSI volumes that need to be disconnected.

Added a new Global Setting: kvm.storage.live.migration.wait

For XenServer, added a check to enforce that only volumes from zone-wide managed storage can be storage motioned from a host in one cluster to a host in another cluster (cannot do so at the time being with volumes from cluster-scoped managed storage)

Don’t allow Storage XenMotion on a VM that has any managed-storage volume with one or more snapshots.

Enabled for managed storage with VMware: Template caching, create snapshot, delete snapshot, create volume from snapshot, and create template from snapshot

Added an SIOC API plug-in to support VMware SIOC

When starting a VM that uses managed storage in a cluster other than the one it last was running in, we need to remove the reference to the iSCSI volume from the original cluster.

Added the ability to revert a volume to a snapshot

Enabled cluster-scoped managed storage

Added support for VMware dynamic discovery
2018-01-15 00:05:52 +05:30
subhash yedugundla 8eca04e1f6 CLOUDSTACK-9572: Snapshot on primary storage not cleaned up after Storage migration (#1740)
Snapshot on primary storage not cleaned up after Storage migration. This happens in the following scenario:

Steps To Reproduce
Create an instance on the local storage on any host
Create a scheduled snapshot of the volume:
Wait until ACS created the snapshot. ACS is creating a snapshot on local storage and is transferring this snapshot to secondary storage. But the latest snapshot on local storage will stay there. This is as expected.
Migrate the instance to another XenServer host with ACS UI and Storage Live Migration
The Snapshot on the old host on local storage will not be cleaned up and is staying on local storage. So local storage will fill up with unneeded snapshots.
2018-01-05 11:19:56 +05:30
pavanaravapalli 2adbaeb641 CLOUDSTACK-9932: Snapshot is getting deleted while volume creation from the snapshot is in progress (#2149)
Added validation to check if any volume(s) are in creating state , before performing delete snapshot.
2018-01-04 10:54:23 +05:30
Abhinandan Prateek 391952da5b CLOUDSTACK-9867: VM snapshot on primary storage usage metrics (#2035)
VM snapshot on primary storage usage metrics.
2017-12-28 14:57:10 +05:30
Rafael Weingärtner 7f6ae15972 Remove annotation not needed at cloud-engine-storage-image (#2324)
remove bogus depends-on declaration at spring-engine-storage-snapshot-core-context
2017-11-16 23:11:15 +05:30
Harika Punna 6bb0ca2f85 This feature separates the snapshot creation on primary and its backing up on secondary.
As part of this, a new parameter, which is optional, is added to CreateSnapshotCmd, which seperates the creation and backup.

More details in the FS-
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Separate+creation+and+backup+operations+for+a+volume+snapshot
2017-10-04 14:39:03 +05:30
Rajani Karuturi 3ddac36d20 Merge pull request #1867 from anshul1886/CLOUDSTACK-9706
CLOUDSTACK-9706: Added snapshots cleanup in start and storage GC thre…
2017-06-06 15:36:23 +05:30
Rajani Karuturi 86eb5683dd Merge pull request #1840 from anshul1886/CLOUDSTACK-9685
CLOUDSTACK-9685: delete snapshot on primary associated with a volume …
2017-05-16 12:42:48 +05:30
Anshul Gangwar c68931fc64 CLOUDSTACK-9706: Added snapshots cleanup in start and storage GC thread if they are failed to cleanup during DeleteSnapshot command 2017-03-17 17:40:55 +05:30
Rajani Karuturi 25f1552e37 Merge release branch 4.9 to master
This closes #1644

* 4.9:
  CLOUDSTACK-4858 Honors the snapshot.backup.rightafter configuration variable Unhides snapshot.backup.rightafter from global configuration
2017-02-08 13:43:02 +05:30
Rajani Karuturi c101817d45 Merge pull request #1697 from myENA/feature/49_observe_snapshot_backup_rightafter
CLOUDSTACK-4858 Honors the snapshot.backup.rightafter configuration variable

Unhides snapshot.backup.rightafter from global configuration

If snapshot.backup.rightafter is set to false (defaults to true), snapshots are
not backed up to secondary storage.

This is the same as PR #1644 applied to 4.9, as per @jburwell

* pr/1697:
  CLOUDSTACK-4858 Honors the snapshot.backup.rightafter configuration variable Unhides snapshot.backup.rightafter from global configuration

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-02-08 12:34:17 +05:30
nvazquez 6ce6cf67f0 CLOUDSTACK-9738: [Vmware] Optimize vm expunge process for instances with vm snapshots 2017-02-06 23:39:01 -03:00
Rajani Karuturi 7233ac37cd Merge pull request #977 from ustcweizhou/vm-snapshot
[4.10] CLOUDSTACK-8746: VM Snapshotting implementation for KVM

* pr/977:
  Fixes for testing VM Snapshots on KVM. Related to PR 977
  CLOUDSTACK-8746: vm snapshot implementation for KVM

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-01-31 05:58:56 +05:30
Wei Zhou a2428508e2 CLOUDSTACK-8746: vm snapshot implementation for KVM
(1) add support to create/delete/revert vm snapshots on running vms with QCOW2 format
(2) add new API to create volume snapshot from vm snapshot
(3) delete metadata of vm snapshots before stopping/migrating and recover vm snapshots after starting/migrating
(4) enable deleting of VM snapshot on stopped vm or vm snapshot is not listed in qcow2 image.
(5) enable smoke tests for vmsnaphsots on KVM
2017-01-24 21:47:30 +01:00
Anshul Gangwar 336df84f17 CLOUDSTACK-9685: delete snapshot on primary associated with a volume when that volume is deleted
as that snapshot will never be going to use again and also it will fill up primary storage
2016-12-20 15:21:04 +05:30
Mike Tutkowski 06806a6523 CLOUDSTACK-9619: Updates for SAN-assisted snapshots 2016-12-06 17:32:56 -07:00
Rohit Yadav dbe57c3e50
Merge branch '4.9' 2016-11-28 16:47:44 +05:30
Syed f46651e672 Support Backup of Snapshots for Managed Storage
This PR adds an ability to Pass a new parameter, locationType,
    to the “createSnapshot” API command. Depending on the locationType,
    we decide where the snapshot should go in case of managed storage.

    There are two possible values for the locationType param

    1) `Standard`: The standard operation for managed storage is to
    keep the snapshot on the device. For non-managed storage, this will
    be to upload it to secondary storage. This option will be the
    default.

    2) `Archive`: Applicable only to managed storage. This will
    keep the snapshot on the secondary storage. For non-managed
    storage, this will result in an error.

    The reason for implementing this feature is to avoid a single
    point of failure for primary storage. Right now in case of managed
    storage, if the primary storage goes down, there is no easy way
    to recover data as all snapshots are also stored on the primary.
    This features allows us to mitigate that risk.
2016-10-30 23:19:58 -06:00
Wei Zhou 784c33585f CLOUDSTACK-9538: FIX failure in Deleting Snapshot From Primary Storage RBD Storage if vm has been removed 2016-10-17 08:25:34 +02:00
Nathan Johnson 1a55a93f04 CLOUDSTACK-4858 Honors the snapshot.backup.rightafter configuration variable
Unhides snapshot.backup.rightafter from global configuration

If snapshot.backup.rightafter is set to false (defaults to true), snapshots are
not backed up to secondary storage
2016-09-30 09:39:27 -05:00
Mike Tutkowski 9d215562eb Faster logic to see if a cluster supports resigning 2016-05-16 07:18:39 -06:00
Mike Tutkowski 2bd035d199 Support for backend snapshots with XenServer 2016-05-13 01:02:04 -06:00
Dmytro Shevchenko 10ae2aff28 CLOUDSTACK-8302 - Cleanup snapshot on KVM with RBD
Snapshot removing implemented on RBD.
1. On management side: when created new shanpshot we checking if our primary storage is RBD,
then do not remove record from cloud.snapshot_store_ref with link to Ceph
image via 'install_path' field.
2. On management side: when removing snapshot, also send command to agent 'DeleteCommand'.
3. On agent side: method implemented 'public Answer deleteSnapshot(final DeleteCommand cmd)'
2016-04-25 22:14:48 +03:00
Mike Tutkowski 158d196c05 CLOUDSTACK-9297: delete snapshot without id is failing with Unable to determine the storage pool of the snapshot 2016-03-24 12:42:48 -06:00
Wei Zhou 52412286c6 CLOUDSTACK-8845: set isRevertable of snapshot to false if the volume is removed 2015-12-04 08:21:11 +01:00
Wei Zhou c15729e71a Fix coverity scan 1323172 2015-09-10 08:01:36 +02:00
Wei Zhou c0a0aec0f9 Merge pull request #732 from ustcweizhou/revert-volume-snapshot-master
Guys, can you review it? things need to be discussed:
(1) this supports KVM/QCOW2 only. Anyone want to implement for other Hypervisor/format ?
(2) The original data volume (on primary storage) will be removed.
(3) The script uses the default timeout in libvirtComputingResource. Do we need to add one in global configuration (like copy.volume.wait or backup.snapshot.wait, create.volume.from.snapshot.wait)
(4) In scripts/storage/qcow2/managesnapshot.sh, I use "qemu-img convert -f qcow2 -O qcow2" to copy the snapshot from secondary to primary (hence there is no base image file), instead of "cp -f", this is because convert is faster than cp in my testing.

* pr/732:
  CLOUDSTACK-5863: revert volume snapshot for KVM/QCOW2

Signed-off-by: Wei Zhou <w.zhou@tech.leaseweb.com>
2015-09-01 16:18:40 +02:00
Rajani Karuturi 8bc0294014 Revert "Merge pull request #714 from rafaelweingartner/master-lrg-cs-hackday-003"
This reverts commit cd7218e241, reversing
changes made to f5a7395cc2.

Reason for Revert:

noredist build failed with the below error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project cloud-plugin-hypervisor-vmware: Compilation failure
[ERROR] /home/jenkins/acs/workspace/build-master-noredist/plugins/hypervisors/vmware/src/com/cloud/hypervisor/guru/VMwareGuru.java:[484,12] error: non-static variable logger cannot be referenced from a static context
[ERROR] -> [Help 1]

even the normal build is broken as reported by @koushik-das on dev list
http://markmail.org/message/nngimssuzkj5gpbz
2015-08-31 11:27:57 +05:30
Rafael Weingartner 3818257a68 Solved jira ticket: CLOUDSTACK-8750 2015-08-28 22:35:08 -03:00
Wei Zhou 92344c006d CLOUDSTACK-5863: revert volume snapshot for KVM/QCOW2 2015-08-24 11:01:50 +02:00
Rohit Yadav 38c9dea9c1 CLOUDSTACK-8748: VM UUID accessible in CreateVMSnapshotCommand and RevertToVMSnapshotCommand
This patch makes it possible to expose VM UUID to subsystems, this can be
useful for implementing VM Snapshots for KVM in future.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-24 14:12:49 +05:30
Remi Bergsma 044e685c92 Revert "CLOUDSTACK-8748: VM UUID accessible in CreateVMSnapshotCommand and RevertToVMSnapshotCommand"
This reverts commit 180afe52e5.

This broke the build on master:
master build broken with the below error http://jenkins.buildacloud.org/job/build-master-slowbuild/2101/consoleText

   ```
   [INFO] -------------------------------------------------------------
   [ERROR] COMPILATION ERROR :
   [INFO] -------------------------------------------------------------
   [ERROR] /home/jenkins/acs/workspace/build-master-slowbuild/plugins/hypervisors/xenserver/test/com/cloud/hypervisor/xenserver/resource/wrapper/xenbase/CitrixRequestWrapperTest.java:[1581,51] error: constructor CreateVMSnapshotCommand in class CreateVMSnapshotCommand cannot be applied to given types;
   [ERROR]   required: String,String,VMSnapshotTO,List<VolumeObjectTO>,String
     found: String,VMSnapshotTO,List<VolumeObjectTO>,String
     reason: actual and formal argument lists differ in length
   /home/jenkins/acs/workspace/build-master-slowbuild/plugins/hypervisors/xenserver/test/com/cloud/hypervisor/xenserver/resource/wrapper/xenbase/CitrixRequestWrapperTest.java:[1623,53] error: no suitable constructor found for RevertToVMSnapshotCommand(String,VMSnapshotTO,List<VolumeObjectTO>,String)
   [INFO] 2 errors
   [INFO] -------------------------------------------------------------
   ```

This was PR #717 towards 4.5
2015-08-20 07:21:27 +02:00
Rohit Yadav 180afe52e5 CLOUDSTACK-8748: VM UUID accessible in CreateVMSnapshotCommand and RevertToVMSnapshotCommand
This patch makes it possible to expose VM UUID to subsystems, this can be
useful for implementing VM Snapshots for KVM in future.

This was PR #717 towards 4.5

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

(cherry picked from commit 0062ff2672)
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-08-19 19:24:56 +02:00
Sanjay Tripathi 84d4c9bf4c Recurring snapshot issue post upgrade from 3.0.7 to 4.5
This closes #486
2015-06-19 16:03:42 +05:30
Mike Tutkowski c2330f48dd Replace synchronized with the use of database locking 2015-01-27 17:36:21 -07:00
Mike Tutkowski c847831662 Do not allow deletion of the snapshot if the snapshot is being copied 2015-01-26 14:57:28 -07:00
Sanjay Tripathi 0c4128e024 CLOUDSTACK-8097: Failed to create volume snapshot after vm live migration across clusters. 2014-12-19 14:26:48 +05:30
Sanjay Tripathi 9153b8bede CLOUDSTACK-8066: There is not way to know the size of the snapshot created. 2014-12-17 16:00:19 +05:30
Edison Su 4140811549 CLOUDSTACK-7947:
double check if parent snapshot is removed or not, when creating new
snapshot

Reviewed-by: Min
2014-11-19 16:13:05 -08:00
Edison Su 67113ff0b2 CLOUDSTACK-7946:
remove leftover state in volume and snapshot table in case of mgt server
shutdown during storage operation.
Reviewed-by: Min
2014-11-19 16:06:37 -08:00
Edison Su c05cda0d28 CLOUDSTACK-7945:
set removed field in snapshots table in case of snaphsot failure.

Reviewed-by: Frank
2014-11-19 15:58:47 -08:00
Mike Tutkowski 18091e9642 Adding support for backend snapshots to Xenserver625StorageProcessor 2014-11-10 22:41:00 -07:00
Mike Tutkowski ebb96f5e2f Handle the case where the volume a snapshot was created from is removed from the database 2014-11-06 19:44:46 -07:00