Commit Graph

292 Commits

Author SHA1 Message Date
Rafael Weingärtner c591c5ad3e CLOUDSTACK-10248: Fix errors that appeared after #2283 (#2417)
This fixes move refactoring error introduced in #2283 
For instance, the class DatadiskTO is supposed to be in com.cloud.agent.api.to package. However, the folder structure it was placed in is com.cloud.agent.api.api.to.

Skip tests for cloud-plugin-hypervisor-ovm3:
For some unknown reason, there are quite a lot of broken test cases for cloud-plugin-hypervisor-ovm3. They might have appeared after some dependency upgrade and was overlooked by the person updating them. I checked them to see if they could be fixed, but these tests are not developed in a clear and clean manner. On top of that, we do not see (at least I) people using OVM3-hypervisor with ACS. Therefore, I decided to skip them.

Identention corrected to use spaces instead of tabs in XML files
2018-01-23 12:19:36 +01:00
Marc-Aurèle Brothier 893a88d225 CLOUDSTACK-10105: Use maven standard project structure in all projects (#2283)
Remove maven standard module (which only a few were using) and get ride of maven customization for the projects structure.

- moved all directories to src/main/java, src/main/resources, src/main/scripts, src/test/java, src/test/resources
- grep scan to search for src/com and src/org left over
- grep for <project>/scripts to fix pom.xml configuration
- remove custom <build> configuration in pom.xml

Signed-off-by: Marc-Aurèle Brothier <m@brothier.org>
2018-01-20 03:19:27 +05:30
Mike Tutkowski a30a31c9b7 CLOUDSTACK-9620: Enhancements for managed storage (#2298)
Allowed zone-wide primary storage based on a custom plug-in to be added via the GUI in a KVM-only environment (previously this only worked for XenServer and VMware)

Added support for root disks on managed storage with KVM

Added support for volume snapshots with managed storage on KVM

Enable creating a template directly from a volume (i.e. without having to go through a volume snapshot) on KVM with managed storage

Only allow the resizing of a volume for managed storage on KVM if the volume in question is either not attached to a VM or is attached to a VM in the Stopped state.

Included support for Reinstall VM on KVM with managed storage

Enabled offline migration on KVM from non-managed storage to managed storage and vice versa

Included support for online storage migration on KVM with managed storage (NFS and Ceph to managed storage)

Added support to download (extract) a managed-storage volume to a QCOW2 file

When uploading a file from outside of CloudStack to CloudStack, set the min and max IOPS, if applicable.

Included support for the KVM auto-convergence feature

The compression flag was actually added in version 1.0.3 (1000003) as opposed to version 1.3.0 (1003000) (changed this to reflect the correct version)

On KVM when using iSCSI-based managed storage, if the user shuts a VM down from the guest OS (as opposed to doing so from CloudStack), we need to pass to the KVM agent a list of applicable iSCSI volumes that need to be disconnected.

Added a new Global Setting: kvm.storage.live.migration.wait

For XenServer, added a check to enforce that only volumes from zone-wide managed storage can be storage motioned from a host in one cluster to a host in another cluster (cannot do so at the time being with volumes from cluster-scoped managed storage)

Don’t allow Storage XenMotion on a VM that has any managed-storage volume with one or more snapshots.

Enabled for managed storage with VMware: Template caching, create snapshot, delete snapshot, create volume from snapshot, and create template from snapshot

Added an SIOC API plug-in to support VMware SIOC

When starting a VM that uses managed storage in a cluster other than the one it last was running in, we need to remove the reference to the iSCSI volume from the original cluster.

Added the ability to revert a volume to a snapshot

Enabled cluster-scoped managed storage

Added support for VMware dynamic discovery
2018-01-15 00:05:52 +05:30
Frank Maximus b176648f90 CLOUDSTACK-9813: Extending Config Drive support (#2097)
Extending Config Drive support

* Added support for VMware
* Build configdrive.iso on ssvm
* Added support for VPC and Isolated Networks
* Moved implementation to new Service Provider
* UI fix: add support for urlencoded userdata
* Add support for building systemvm behind a proxy

Co-Authored-By: Raf Smeets <raf.smeets@nuagenetworks.net>
Co-Authored-By: Frank Maximus <frank.maximus@nuagenetworks.net>
Co-Authored-By: Sigert Goeminne <sigert.goeminne@nuagenetworks.net>
2018-01-12 15:14:40 +05:30
Rafael Weingärtner 1c8bbddb04 [CLOUDSTACK-10128] Simplify the search for host to execute command 2018-01-11 14:36:35 -02:00
Nicolas Vazquez e86bb41e0e CLOUDSTACK-10146: Bypass Secondary Storage for KVM templates (#2379)
This feature allows using templates and ISOs avoiding secondary storage as intermediate cache on KVM. The virtual machine deployment process is enhanced to supported bypassed registered templates and ISOs, delegating the work of downloading them to primary storage to the KVM agent instead of the SSVM agent.

Template and ISO registration:
- When hypervisor is KVM, a checkbox is displayed with 'Direct Download' label.
- API methods registerTemplate and registerISO are both extended with this new parameter directdownload.
- On template or ISO registration, no download job is sent to SSVM agent, CloudStack would only persist an entry on template_store_ref indicating that template or ISO has been marked as 'Direct Download' (bypassing Secondary Storage). These entries are persisted as:
template_id = Template or ISO id on vm_template table
store_id NULL
download_state = BYPASSED
state = Ready
(Note: these entries allow users to deploy virtual machine from registered templates or ISOs)
- An URL validation command is sent to a random KVM host to check if template/ISO location can be reached. Metalink are also supported by this feature. In case of a metalink, it is fetched and URL check is performed on each of its URLs.
- Checksum should be provided as indicated on #2246: {ALGORITHM}CHKSUMHASH
- After template or ISO is registered, it would be displayed in the UI

Virtual machine deployment:
When a 'Direct Download' template is selected for deployment, CloudStack would delegate template downloading to destination storage pool via destination host by a new pluggable download manager.
Download manager would handle template downloading depending on URL protocol. In case of HTTP, request headers can be set by the user via vm_template_details. Those details should be persisted as:
Key: HTTP_HEADER
Value: HEADERNAME:HEADERVALUE

In case of HTTPS, a new API method is added uploadTemplateDirectDownloadCertificate to allow user importing a client certificate into all KVM hosts' keystore before deployment.
After template or ISO is downloaded to primary storage, usual entry would be persisted on template_spool_ref indicating the mapping between template/ISO and storage pool.
2018-01-09 12:22:18 +05:30
Khosrow Moossavi 25d7d741a7 CLOUDSTACK-10197: Rename xentools iso for XenServer 7.0+ (#2365)
The xentools iso has been renamed from xs-tools to guest-tools
starting from XenServer 7.0.
2018-01-07 16:00:39 +05:30
Rafael Weingärtner 1d0f2128f6 [CLOUDSTACK-9025][CLOUDSTACK-10128] solve problems of templates created based on snapshots (#2315)
The first PR(#1176) intended to solve #CLOUDSTACK-9025 was only tackling the problem for CloudStack deployments that use single hypervisor types (restricted to XenServer). Additionally, the lack of information regarding that solution (poor documentation, test cases and description in PRs and Jira ticket) led the code to be removed in #1124 after a long discussion and analysis in #1056. That piece of code seemed logicless (and it was!).  It would receive a hostId and then change that hostId for other hostId of the zone without doing any check; it was not even checking the hypervisor and storage in which the host was plugged into.

The problem reported in #CLOUDSTACK-9025 is caused by partial snapshots that are taken in XenServer. This means, we do not take a complete snapshot, but a partial one that contains only the modified data. This requires rebuilding the VHD hierarchy when creating a template out of the snapshot. The point is that the first hostId received is not a hostId, but a system VM ID(SSVM). That is why the code in #1176 fixed the problem for some deployment scenarios, but would cause problems for scenarios where we have multiple hypervisors in the same zone. We need to execute the creation of the VHD that represents the template in the hypervisor, so the VHD chain can be built using the parent links.

This commit changes the method com.cloud.hypervisor.XenServerGuru.getCommandHostDelegation(long, Command). From now on we replace the hostId that is intended to execute the “copy command” that will create the VHD of the template according to some conditions that were already in place. The idea is that starting with XenServer 6.2.0 hotFix ESP1004 we need to execute the command in the hypervisor host and not from the SSVM. Moreover, the method was improved making it readable and understandable; it was also created test cases assuring that from XenServer 6.2.0 hotFix ESP1004 and upward versions we change the hostId that will be used to execute the “copy command”.

Furthermore, we are not selecting a random host from a zone anymore. A new method was introduced in the HostDao called “findHostConnectedToSnapshotStoragePoolToExecuteCommand”, using this method we look for a host that is in the cluster that is using the storage pool where the volume from which the Snaphost is taken of. By doing this, we guarantee that the host that is connected to the primary storage where all of the snapshots parent VHDs are stored is used to create the template.

Consider using Disabled hosts when no Enabled hosts are found

This also closes #2317
2018-01-06 09:55:29 +01:00
Wido den Hollander 9988c269b2 CLOUDSTACK-7958: Add configuration for limit to CIDRs for Admin API calls (#2046)
* Cleanup and Improve NetUtils

This class had many unused methods, inconsistent names and redundant code.

This commit cleans up code, renames a few methods and constants.

The global/account setting 'api.allowed.source.cidr.list' is set
to 0.0.0.0/0,::/0 by default preserve the current behavior and thus
allow API calls for accounts from all IPv4 and IPv6 subnets.

Users can set it to a comma-separated list of IPv4/IPv6 subnets to
restrict API calls for Admin accounts to certain parts of their network(s).

This is to improve Security. Should an attacker steal the Access/Secret key
of an account he/she still needs to be in a subnet from where accounts are
allowed to perform API calls.

This is a good security measure for APIs which are connected to the public internet.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2018-01-04 10:56:30 +05:30
Rafael Weingärtner 3c6df7c970 CLOUDSTACK-10196: Remove ejb-api 3.0 dependency (#2348)
Fixed QuotaAlertManagerImplTest, which was injecting mock objects manually.
2017-12-18 23:59:06 +05:30
Rafael Weingärtner cb9c7adddf CLOUDSTACK-10156: Fix Coverity new problems CID(1349987, 1349986, 1347248) (#2332)
Fix Coverity new problems CID(1349987, 1349986, 1347248).
2017-11-23 11:40:53 +05:30
Wido den Hollander 632479d8f8 CLOUDSTACK-9853: Add support for Secondary IPv6 Addresses and Subnets (#2028)
This commit adds support for passing IPv6 Addresses and/or Subnets as
Secondary IPs.

This is groundwork for CLOUDSTACK-9853 where IPv6 Subnets have to be
allowed in the Security Groups of Instances to we can add DHCPv6
Prefix Delegation.

Use ; instead of : for separating addresses, otherwise it would cause
problems with IPv6 Addresses.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2017-11-22 17:30:33 +05:30
Abhinandan Prateek 4627fb2cd7 CLOUDSTACK-9972: Enhance listVolume API to include physical size and … (#2158)
* CLOUDSTACK-9972: Enhance listVolume API to include physical size and utilization.
Also fixed pool, cluster and pod info

* CLOUDSTACK-9972: Fix volume_view and duplicate API constant

* CLOUDSTACK-9972: Backport Do not allow vms to be deployed on hosts that are in disabled pod

* CLOUDSTACK-9972: Fix localization missing keys

* CLOUDSTACK-9972: Fix sql path
2017-11-05 21:44:43 +05:30
subhash yedugundla 0d81e88601 CLOUDSTACK-8939: VM Snapshot size with memory correctly calculated in cloud.usage_event (XenServer) (#914)
Fixing the error  with snapshot size calculation  in a vm with memory
2017-09-01 11:00:55 +02:00
Gabriel Beims Bräscher 01b040d19e CLOUDSTACK-9314: Remove unused code from XenServerStorageProcessor (#1443)
This commit removes unused code from
"com.cloud.hypervisor.xenserver.resource.XenServerStorageProcessor"
class and modifies some methods access level.

Remove unused methods:
- com.cloud.hypervisor.xenserver.resource.XenServerStorageProcessor.getSRByNameLabel(Connection,
String)
- com.cloud.hypervisor.xenserver.resource.XenServerStorageProcessor.getNfsSR(Connection,
StorageFilerTO)
- com.cloud.hypervisor.xenserver.resource.XenServerStorageProcessor.directDownloadHttpTemplate(CopyCommand,
DecodedDataObject, DecodedDataObject)
- com.cloud.hypervisor.xenserver.resource.XenServerStorageProcessor.execute(AttachPrimaryDataStoreCmd)


access level:
- from public to private:
com.cloud.hypervisor.xenserver.resource.XenServerStorageProcessor.swiftUpload(Connection,
SwiftTO, String, String, String, Boolean, int)
- from public to protected:
com.cloud.hypervisor.xenserver.resource.XenServerStorageProcessor.swiftBackupSnapshot(Connection,
SwiftTO, String, String, String, Boolean, int)
- from protected to private:
com.cloud.hypervisor.xenserver.resource.XenServerStorageProcessor.getSnapshotSize(Connection,
String, String, Boolean, int)
- from protected to private:
com.cloud.hypervisor.xenserver.resource.XenServerStorageProcessor.backupSnapshot(Connection,
String, String, String, String, String, String, Boolean, int)
2017-08-31 09:34:24 +02:00
Wei Zhou 960cb84083 CLOUDSTACK-7984: Collect network statistics for VMs on shared network (KVM implementation) 2017-07-23 17:15:23 +02:00
Nitin Kumar Maharana b2b422c7d7 CLOUDSTACK-8672 : NCC Integration with CloudStack. 2017-07-20 12:42:43 +05:30
Rohit Yadav ed376fcad6 Merge remote-tracking branch 'origin/4.9'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-06-07 11:21:27 +05:30
Rajani Karuturi 51974cd568 Merge pull request #899 from yvsubhash/CLOUDSTACK-8921
BUG-ID:CLOUDSTACK-8921 snapshot_store_ref table should store actual size of backup snapshot in secondary storage
2017-06-06 15:38:42 +05:30
Rohit Yadav 8323a175f1 CLOUDSTACK-9860: Power off VMs when stopVM is called with forced=true
The 'force' option provided with the stopVirtualMachine API command is
often assumed to be a hard shutdown sent to the hypervisor, when in fact
it is for CloudStacks' internal use. CloudStack should be able to send
the 'hard' power-off request to the hosts.

When forced parameter on the stopVM API is true, power off (hard shutdown)
a VM. This uses initial changes from #1635 to pass the forced parameter
to hypervisor plugin via the StopCommand, and fixes force stop (poweroff)
handling for KVM, VMware and XenServer.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-05-25 17:25:22 +05:30
Jayapal d04a3e842c CLOUDSTACK-9317: Update review comments and rule state column 2017-05-17 11:08:13 +05:30
Jayapal c20e0ef88f CLOUDSTACK-9317: Fixed disable static nat on leaving ips on interface 2017-05-17 11:03:50 +05:30
Rajani Karuturi 86eb5683dd Merge pull request #1840 from anshul1886/CLOUDSTACK-9685
CLOUDSTACK-9685: delete snapshot on primary associated with a volume …
2017-05-16 12:42:48 +05:30
Rajani Karuturi a4f3c71744 Merge pull request #1849 from sudhansu7/CLOUDSTACK-9690
CLOUDSTACK-9690: Scale CentOS7 VM fails with error
2017-05-16 11:24:50 +05:30
Rajani Karuturi 1aa6a72bc7 CLOUDSTACK-9834: prepareTemplate API call doesn't work well
with XenServer & Local SR (Db_exn.Uniqueness_constraint_violation)

removed the host uuid from SR label so that any host which has access to
the SR(all the hosts in the same pool) can reuse the same SR
2017-05-15 11:17:20 +05:30
Sudhansu e7a7db3ef3 CLOUDSTACK-9690: Scale CentOS7 VM fails with error
1. Removed XenServerGuestOsMemoryMap from CitrixHelper.java
This java file was holding a static in memory map named XenServerGuestOsMemoryMap. This was the source for xenserver dynamic memory values(max and min). These values were moved to guest_os_details table.

2. DAO layer was modified to access these values.
3. VirtualMachineTo object was modified to populate the dynamic memory values.
4. addGuestOs and UpdateGuestOS api has been modified to update memory values.
2017-05-05 10:58:12 +05:30
subhash yedugundla 4e4b67cd96 BUG-ID:CLOUDSTACK-8921
Summary: CLOUDSTACK-8921
     snapshot_store_ref table should store actual size of back snapshot in secondary storage
Calling SR scan to make sure size is updated correctly
2017-04-04 15:02:17 +05:30
Anshul And Priyank ec66256149 CLOUDSTACK-9604: Root disk resize support for VMware and XenServer. 2017-03-09 19:20:38 +05:30
Rajani Karuturi fa85151be9 Merge release branch 4.9 to master
* 4.9:
  CLOUDSTACK-9746 system-vm: logrotate config causes critical failures
  CLOUDSTACK-9788: Fix exception listNetworks with pagesize=0
  CLOUDSTACK-8663: Fixed various issues to allow VM snapshots and volume snapshots to exist together
  Fix HVM VM restart bug in XenServer
2017-02-28 05:47:06 +05:30
Rajani Karuturi d9bd01266f Merge pull request #1829 from syed/hvm-volume-attach-restart-fix
CLOUDSTACK-9363: Fix HVM VM restart bug in XenServerHere is the longer description of the problem:

By default XenServer limits HVM guests to only 4 disks. Two of those are reserved for the ROOT disk (deviceId=0) and CD ROM (device ID=3) which means that we can only attach 2 data disks. This limit however is removed when Xentools is installed on the guest. The information that a guest has Xentools installed and can handle more than 4 disks is stored in the VM metadata on XenServer. When a VM is shut down, Cloudstack removes the VM and all the metadata associated with the VM from XenServer. Now, when you start the VM again, even if it has Xentools installed, it will default to only 4 attachable disks.

Now this problem manifests itself when you have a HVM VM and you stop and start it with more than 2 data disks attached. The VM fails to start and the only way to start the VM is to detach the extra disks and then reattach them after the VM start.

In this fix, I am removing the check which is done before creating a `VBD` which enforces this limit. This will not affect current workflow and will fix the HVM issue.

@koushik-das this is related to the "autodetect" feature that you introduced a while back (https://issues.apache.org/jira/browse/CLOUDSTACK-8826). I would love your review on this fix.

* pr/1829:
  Fix HVM VM restart bug in XenServer

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-02-28 05:42:59 +05:30
Anshul Gangwar ca84fd4ffd CLOUDSTACK-8663: Fixed various issues to allow VM snapshots and volume
snapshots to exist together

Reverting VM to disk only snapshot in Xenserver corrupts VM

Stale NFS secondary storage on XS leads to volume creation failure from snapshot
2017-02-15 12:56:39 +05:30
Rajani Karuturi 4721c53ea0 Merge pull request #1749 from mike-tutkowski/archived_snapshots
CLOUDSTACK-9619: Updates for SAN-assisted snapshotsThis PR is to address a few issues in #1600 (which was recently merged to master for 4.10).

In StorageSystemDataMotionStrategy.performCopyOfVdi we call getSnapshotDetails. In one such scenario, the source snapshot in question is coming from secondary storage (when we are creating a new volume on managed storage from a snapshot of ours thats on secondary storage).

This usually worked in the regression tests due to a bit of "luck": We retrieve the ID of the snapshot (which is on secondary storage) and then try to pull out its StorageVO object (which is for primary storage). If you happen to have a primary storage that matches the ID (which is the ID of a secondary storage), then getSnapshotDetails populates its Map<String, String> with inapplicable data (that is later ignored) and you dont easily see a problem. However, if you dont have a primary storage that matches that ID (which I didnt today because I had removed that primary storage), then a NullPointerException is thrown.

I have fixed that issue by skipping getSnapshotDetails if the source is coming from secondary storage.

While fixing that, I noticed a couple more problems:

1)       We can invoke grantAccess on a snapshot thats actually on secondary storage (this doesnt amount to much because the VolumeServiceImpl ignores the call when its not for a primary-storage driver).
2)       We can invoke revokeAccess on a snapshot thats actually on secondary storage (this doesnt amount to much because the VolumeServiceImpl ignores the call when its not for a primary-storage driver).

I have corrected those issues, as well.

I then came across one more problem:
         When using a SAN snapshot and copying it to secondary storage or creating a new managed-storage volume from a snapshot of ours on secondary storage, we attach to the SR in the XenServer code, but detach from it in the StorageSystemDataMotionStrategy code (by sending a message to the XenServer code to perform an SR detach). Since we know to detach from the SR after the copy is done, we should detach from the SR in the XenServer code (without that code having to be explicitly called from outside of the XenServer logic).

I went ahead and changed that, as well.

JIRA Ticket:
https://issues.apache.org/jira/browse/CLOUDSTACK-9619

* pr/1749:
  CLOUDSTACK-9619: Updates for SAN-assisted snapshots

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2017-01-27 05:35:06 +05:30
Rohit Yadav 342162bad7 Merge branch '4.9'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-22 11:59:02 +05:30
Syed eabf862ba9 [CLOUDSTACK-9662] Add support for XenServer 7 2016-12-21 16:58:10 -05:00
Anshul Gangwar 336df84f17 CLOUDSTACK-9685: delete snapshot on primary associated with a volume when that volume is deleted
as that snapshot will never be going to use again and also it will fill up primary storage
2016-12-20 15:21:04 +05:30
Syed 966814b07d Fix HVM VM restart bug in XenServer 2016-12-15 14:12:59 -05:00
Mike Tutkowski 06806a6523 CLOUDSTACK-9619: Updates for SAN-assisted snapshots 2016-12-06 17:32:56 -07:00
Rohit Yadav f82873ebf8
Merge branch '4.9' 2016-12-05 15:32:11 +05:30
Rohit Yadav 48b28f7d6e
Merge branch '4.8' into 4.9 2016-12-05 15:32:03 +05:30
Rohit Yadav 20aea27dc0
Merge pull request #1765 from shapeblue/CLOUDSTACK-9586
Cloudstack 9586: When using local storage with Xenserver prepareTemplate does not work with multiple primary storeThe race condition will happen whenever there are multiple primary storages and the CS tries to mount the secondary store to xenserver host simultaneously.

Due to synchronised block one mount will be successful and other thread will get the already mounted SR. Without the fix the two thread will try to mount it parallely and one will fail on Xenserver.

* pr/1765:
  Cloudstack 9586: When using local storage with Xenserver prepareTemplate does not work with multiple primary store The race condition will happen whenever there are multiple primary storages and the CS tries to mount the secondary store to xenserver host simultaneously. Due to synchronised block one mount will be successful and other thread will get the already mounted SR. Without the fix the two thread will try to mount it parallely and one will fail on Xenserver.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-05 15:31:25 +05:30
Abhinandan Prateek ba32ac1a7c Cloudstack 9586: When using local storage with Xenserver prepareTemplate does not work with multiple primary store
The race condition will happen whenever there are multiple primary storages and the CS tries to mount the secondary store to xenserver host simultaneously.
Due to synchronised block one mount will be successful and other thread will get the already mounted SR. Without the fix the two thread will try to mount it parallely and one will fail on Xenserver.
2016-12-02 13:37:47 +05:30
Rohit Yadav 0642a6982f
Merge branch '4.9' 2016-11-23 14:22:15 +05:30
Rohit Yadav 55b918076f
Merge branch '4.8' into 4.9 2016-11-23 13:50:15 +05:30
Rohit Yadav ff616e700b Merge pull request #1745 from shapeblue/CLOUDSTACK-9503
CLOUDSTACK-9503: Increased the VR script timeout. Most of the changes are about converting int/long time values to joda Duration.

* pr/1745:
  CLOUDSTACK-9503: Increased the VR script timeout. Most of the changes are about converting int/long time values to joda Duration.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-11-23 13:41:52 +05:30
Koushik Das 027409d9bc Merge release branch 4.9 to master
* 4.9:
  CLOUDSTACK-9410: Data Disk shown as detached in XS
2016-11-21 11:25:38 +05:30
Koushik Das bdc806e315 Merge release branch 4.8 to 4.9
* 4.8:
  CLOUDSTACK-9410: Data Disk shown as detached in XS
2016-11-21 11:12:09 +05:30
Abhinandan Prateek 83b5a8b2b2 CLOUDSTACK-9503: Increased the VR script timeout. Most of the changes are about converting int/long time values to joda Duration. 2016-11-01 16:14:23 +05:30
Syed f46651e672 Support Backup of Snapshots for Managed Storage
This PR adds an ability to Pass a new parameter, locationType,
    to the “createSnapshot” API command. Depending on the locationType,
    we decide where the snapshot should go in case of managed storage.

    There are two possible values for the locationType param

    1) `Standard`: The standard operation for managed storage is to
    keep the snapshot on the device. For non-managed storage, this will
    be to upload it to secondary storage. This option will be the
    default.

    2) `Archive`: Applicable only to managed storage. This will
    keep the snapshot on the secondary storage. For non-managed
    storage, this will result in an error.

    The reason for implementing this feature is to avoid a single
    point of failure for primary storage. Right now in case of managed
    storage, if the primary storage goes down, there is no easy way
    to recover data as all snapshots are also stored on the primary.
    This features allows us to mitigate that risk.
2016-10-30 23:19:58 -06:00
subhash yedugundla 38c56bdf44 CLOUDSTACK-9410: Data Disk shown as detached in XS 2016-10-25 14:57:33 +05:30
Mike Tutkowski b508fb8692 Adding support for cross-cluster storage migration for managed storage when using XenServer 2016-09-12 07:39:13 -06:00
Will Stevens f7f23ec720 Merge release branch 4.8 to master
* 4.8:
  CLOUDSTACK-9353: [XenServer] Fixed VM migration with storage
  Added ASF license to unit test file
  Added unit test to verify ordering
  Fixed ordering of network ACL rules being sent to the VR. The comparator was inverted
2016-06-28 11:21:04 -04:00
Will Stevens 142f07d77f Merge release branch 4.7 to 4.8
* 4.7:
  CLOUDSTACK-9353: [XenServer] Fixed VM migration with storage
  Added ASF license to unit test file
  Added unit test to verify ordering
  Fixed ordering of network ACL rules being sent to the VR. The comparator was inverted
2016-06-28 11:20:16 -04:00
Anshul Gangwar 458264aac8 CLOUDSTACK-9353: [XenServer] Fixed VM migration with storage 2016-06-24 09:31:20 +05:30
Mike Tutkowski 2bd035d199 Support for backend snapshots with XenServer 2016-05-13 01:02:04 -06:00
Will Stevens 8c3722d953 Merge pull request #1444 from rafaelweingartner/workAroundPR780
CLOUDSTACK-8800 : Improved the listVirtualMachines API call to include memory utilization information for a VMThis PR introduces the changes proposed in PR #780 with some work to make the code null safe.

During this PR, I have also removed some unused code.

* pr/1444:
  Removed unnecessary check when creating the “userVmResponse” object.
  Fixed issues from CLOUDSTACK-8800 that were introduced in PR 780
  CLOUDSTACK-8800 : Improved the listVirtualMachines API call to include memory utilization information for a VM for xenserver,kvm and for vmware.

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-12 16:49:14 -04:00
Mike Tutkowski dad9e5d868 CLOUDSTACK-8813: Notify listeners when a host has been added to a cluster, is about to be removed from a cluster, or has been removed from a cluster 2016-05-11 08:02:46 -06:00
Will Stevens 8a3fd10615 Merge pull request #1124 from rafaelweingartner/CID-1338387
CID-1338387: Deletion of method endPointSelector.selectHypervisorHostFollowing the discussions and analysis presented on PR #1056 create by @DaanHoogland
This PR is intended to push those changes that were discussed there regarding the of endPointSelector.selectHypervisorHost method.

* pr/1124:
  Deletion of method endPointSelector.selectHypervisorHost

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-10 23:47:50 -04:00
weingartner 866ac7b6e1 Fixed issues from CLOUDSTACK-8800 that were introduced in PR 780
It was worked around some possible runtime exceptions introduced by the
changes that were added by the PR 780. Basically, the points in which a
null pointer exception could happen, we added safety checks to avoid
them. It was create a specific method do that, all together test cases
were created for this newly method that was added.
2016-05-03 14:05:18 -03:00
Maneesha.P 7ad3c5e834 CLOUDSTACK-8800 : Improved the listVirtualMachines API call to include memory utilization information for a VM for xenserver,kvm and for vmware. 2016-04-29 14:06:49 -03:00
Will Stevens f530a4c63a Merge pull request #1200 from koushik-das/CLOUDSTACK-9130
CLOUDSTACK-9130: Make RebootCommand similar to start/stop/migrate agent commands w.r.t. "execute in sequence" flag

RebootCommand now behaves in the same way as start/stop/migrate agent commands w.r.t. to sequential/parallel execution.

* pr/1200:
  CLOUDSTACK-9130: Make RebootCommand similar to start/stop/migrate agent commands w.r.t. "execute in sequence" flag RebootCommand now behaves in the same way as start/stop/migrate agent commands w.r.t. to sequential/parallel execution.

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-21 16:35:27 -04:00
Will Stevens a3371d282f Merge pull request #1262 from rafaelweingartner/lrg-cs-hackday-015
Removed unnecessary code from getGuestOsType in CitrixResourceBaseConsidering that all mapping of Guest OS Names to their respective hypervisor compatible types is made thorugh accessing a database, we've decided to remove a bit of code in the XcpOssResource class which was doing that same thing for 2 different OS's (both of which ARE in the database). That has led us to a bunch of unused parameters in the getGuestOsType method from its superclass, which we've also decided to remove. Test cases were added for four different possibilities for the platformEmulator String: one for a null String, one for a blank String, one for an empy String and one for a random case with a valid String.

* pr/1262:
  Remove test cases duplicated code.
  Removed unnecessary code from getGuestOsType in CitrixResourceBase

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-18 14:41:35 -04:00
Koushik Das d1def0a730 Merge pull request #1287 from DaanHoogland/securityrules-cleanup
SecurityGroupRulesCmd code cleanupWrote a test and cleaned some duplicate code with the objective to evaluate the jenkins pull request process at builds.a.o
worthwhile to keep, IMHO.

* pr/1287:
  SecurityGroupRulesCmd code cleanup review comments handled
  deal with PMD warnings
  code cleanup
  security rules test
  remove autogenerated pydev files

Signed-off-by: Koushik Das <koushik@apache.org>
2016-04-11 21:49:57 +05:30
weingartner b56e9c7a22 Deletion of method endPointSelector.selectHypervisorHost
As discussed on PR 1056, I am removing the method
“endPointSelector.selectHypervisorHost” that is not needed anymore
2016-04-08 11:07:18 -03:00
Will Stevens c48b6c33c0 Revert "Merge pull request #780 from maneesha-p/pull-19" due code quality issues and lack of CI results.
This reverts commit dc0ba6bd1a, reversing
changes made to 63f58dd578.
2016-03-21 17:38:04 -04:00
Maneesha.P 732a85295d CLOUDSTACK-8800 : Improved the listVirtualMachines API call to include memory utilization information for a VM for xenserver,kvm and for vmware. 2016-03-16 16:13:10 +05:30
Rohit Yadav a59cf68f50 Merge pull request #969 from rafaelweingartner/master-lrg-cs-hackday-002
Fixed return type Void to void in  DataMotionStrategy.The main changes are:
-	Changing methods Void to void.
-	Removal of the method Void copyAsync(DataObject srcData, DataObject
destData, AsyncCompletionCallback<CopyCommandResult> callback) that was
never used.
We noticed that methods form that class are using the return type Void
with capital V. This way that method has to return a null value at the
end.

Removed trim lines from XenServerStorageMotionStrategy.

The trim lines were removed from XenServerStorageMotionStrategy.

* pr/969:
  Changed return of methods from DataMotionStrategy, Void to void

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-27 16:08:07 +01:00
Remi Bergsma fac259f0fb Merge release branch 4.7 to master
* 4.7:
  Fix execution counter to support separate counts per thread
  Add test to check that each thread has it's own execution counter
  CLOUDSTACK-9231: Root volume migration from one primary to another primary storage within the same cluster is failing
2016-01-18 16:00:29 +01:00
Nitin Kumar Maharana 40e3dfc617 CLOUDSTACK-9231: Root volume migration from one primary to another primary storage within the same cluster is failing
This situation arises when there are two management server accessing the same database.
When the migration request comes the command is forwarded from one management server to another because
the host is owned by the second management server. So, serialization of map from one to another fails.
This is fixed by converting the maps to lists.
2016-01-13 22:42:53 +05:30
Daan Hoogland d39182f9d4 deal with PMD warnings 2015-12-26 09:02:30 +01:00
cirstofolini 935113fbd3 Removed unnecessary code from getGuestOsType in CitrixResourceBase 2015-12-20 15:08:44 -02:00
SudharmaJain 9ae3c0a5a9 CLOUDSTACK-9127 Missing PV-bootloader-args for "SUSE Linux Enterprise Server 10 SP2 and SP3"
Added Unit test for the new method
2015-12-14 11:26:45 +05:30
Koushik Das 5b3ffeb6cd CLOUDSTACK-9130: Make RebootCommand similar to start/stop/migrate agent commands w.r.t. "execute in sequence" flag
RebootCommand now behaves in the same way as start/stop/migrate agent commands w.r.t. to sequential/parallel execution.
2015-12-09 15:19:10 +05:30
Rohit Yadav f30fbe9a5c Merge branch '4.6' 2015-12-07 12:44:07 +05:30
Anshul Gangwar f8790c3b9e CLOUDSTACK-9025: Fixed can't create usable template from snapshot in Xenserver and Vmware
Fix also reverts below commit as below solution making assumption about hypervisor which are not applicable
in case of XenServer and VmWare

Revert "CLOUDSTACK-8964: Can't create template or volume from snapshot"

This reverts commit ccf5d75cfb.
2015-12-04 14:56:40 +05:30
Remi Bergsma 6d558c0201 Merge pull request #944 from rafaelweingartner/lrg-cs-hackday-012
Create test cases to getPatchFilePath method and class names changed In this commit we created tests cases for the respective classes in package  com.cloud.hypervisor.xenserver.resource.

We added test cases to check the implementation of  com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getPatchFiles. Therefore, we test in a more comprehensive way the tests that already exist to check the code of com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getPatchFilePath.

We added a new abstract class, called com.cloud.hypervisor.xenserver.resource.CitrixResourceBaseTest.java

This class has two tests methods:

* com.cloud.hypervisor.xenserver.resource.CitrixResourceBaseTest.testGetPathFilesExeption(CitrixResourceBase), this method tests if the getPatchFilePath() method throws the com.cloud.utils.exception.CloudRuntimeException.CloudRuntimeException when the com.cloud.utils.script.Script.findScript(String, String) return a null value;
* com.cloud.hypervisor.xenserver.resource.CitrixResourceBaseTest.testGetPathFilesListReturned(CitrixResourceBase), this method tests the correct return value from getPatchFilePath() method, basically, verify if the returned list contain the file with the same absolute path that was retrieved from the findScript method.

We also changed the name of those test classes, the change was basically remove the Path word from the name of classes.

* pr/944:
  created tests cases for method "citrixResourceBase.getPatchFiles"

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-03 10:12:07 +01:00
pedro-martins d01cdb1979 created tests cases for method "citrixResourceBase.getPatchFiles"
we also changed the name of those test classes, the change was basically
remove the “Path” word from the name of classes.

change the private method
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getPatchFiles
to protected

added new class in test cases
2015-11-28 19:06:04 -02:00
weingartner 4a9487e581 Changed return of methods from DataMotionStrategy, Void to void
The main changes are:
•	Changing methods “Void” to “void”.
•	Removal of the method “Void copyAsync(DataObject srcData, DataObject
destData, AsyncCompletionCallback callback)” that was never used. We
noticed that methods form that class are using the return type Void with
capital V. This way that method has to return a null value at the end.
2015-11-28 12:09:54 -02:00
Devdeep Singh 00522e475b CLOUDSTACK-9081 Migration of vm across clusters fails in clustered MS setup
Storage motion of vm across clusters/xenserver-pools fails in a
clustered management server setup. In xen storage motion we have to send
a migrate_receive command to the destination host, followed by
migrate_send command to the source host. The sr and network detials of
the destination host have to be passed in migrate_send command on the
source. While migrating across clusters the source and destination
resources are seperate objects. To pass this information across
resources we have to send seperate migrate with storage receive and send
commands to the resource. In a clustered ms setup these commands may
have to be forwarded to another ms as the resource may be owned by it.
The serilization of the command and answer objects fails in such case as
it doesn't understand the xapi sr and network objects. Made a change to
serialize these objects in the resource layer and pass the around as
strings in the command and answer objects.  Reviewed-By: Likitha
2015-11-26 11:59:18 +05:30
cirstofolini 1a64c247ad Removed unnecessary @Local annotations and their respective imports from the ComponentLifecycleBase class and its subclasses. 2015-11-21 18:31:11 -02:00
Boris Schrijver 5c0366c99e CLOUDSTACK-9062: Improve S3 implementation.
The S3 implementation is far from finished, this commit focusses on the bases.

 - Upgrade AWS SDK to latest version.
 - Rewrite S3 Template downloader.
 - Rewrite S3Utils utility class.
 - Improve addImageStoreS3 API command.
 - Split various classes for convenience.
 - Various minor improvements and code optimalisations.

A side effect of the new AWS SDK is that it, by default, uses the V4 signature. Therefore I added an option to specify the Signer, so it stays compatible with previous versions.
2015-11-19 15:29:10 +01:00
Funs Kessen c4afb76d8c host delegation logging for XenServerGuru.java 2015-10-23 20:09:06 +02:00
Koushik Das f5b9a96d24 CLOUDSTACK-8826: XenServer - Use device id passed as part of attach volume API properly
If device id passed as part of API and available then use it otherwise fallback on XS to automatically assign one.
For ISO device id used is 3 and it is processed before any other entry to avoid conflict.
2015-09-21 17:09:14 +05:30
Rajani Karuturi 8bc0294014 Revert "Merge pull request #714 from rafaelweingartner/master-lrg-cs-hackday-003"
This reverts commit cd7218e241, reversing
changes made to f5a7395cc2.

Reason for Revert:

noredist build failed with the below error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project cloud-plugin-hypervisor-vmware: Compilation failure
[ERROR] /home/jenkins/acs/workspace/build-master-noredist/plugins/hypervisors/vmware/src/com/cloud/hypervisor/guru/VMwareGuru.java:[484,12] error: non-static variable logger cannot be referenced from a static context
[ERROR] -> [Help 1]

even the normal build is broken as reported by @koushik-das on dev list
http://markmail.org/message/nngimssuzkj5gpbz
2015-08-31 11:27:57 +05:30
Rafael Weingartner 3818257a68 Solved jira ticket: CLOUDSTACK-8750 2015-08-28 22:35:08 -03:00
Koushik Das e1db6efc74 Removed unused agent command AttachVolumeCommand and corresponding answer AttachVolumeAnswer 2015-08-26 09:50:23 +05:30
weingartner ec8976f802 Removed duplicate code in CitrixResourceBase.getPatchFiles
@cristofolini comments:
Removed unnecessary duplicated code by having the body of the
getPatchFiles method only in the
CitrixResourceBase superclass. Given that all of its implementations
consisted of the same code
except for the path which contains the necessary file for that
implementation. An abstract method
getPatchFilePath was created in the CitrixResourceBase superclass so
that each implementation may
return the path containing the specific file needed by that
implementation.

Test cases were created for each implementation, simple as they may be.
One assert is made to verify
that the path returned by each implementation corresponds to the path
that was previously specified
on each getPatchFiles implementation.

removed trailing whitespace
2015-08-17 13:26:25 -03:00
Funs ca68bb0ef7 xenserver: Improved message for peeples to actually see what the problem is.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

This closes #576
2015-07-15 18:24:39 +05:30
Daan Hoogland e62d952375 coverity 1311706: to generic exeption handling on xapi call
Signed-off-by: Daan Hoogland <daan.hoogland@gmail.com>
2015-07-14 13:41:09 +02:00
Jayapal deb694c201 CLOUDSTACK-8324: Added migration changes for config drive 2015-07-10 09:39:27 +05:30
Daan Hoogland 6376503fe3 coverity 1308118: more resources being opened only one closed on exception nothing was closed
Signed-off-by: Daan Hoogland <daan@onecht.net>
2015-07-09 21:30:23 +02:00
weingartner 25e9918c57 Amend last commit, used 4 spaces in code formatting instead of 1 tab character.
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>

This closes #560
2015-07-09 13:37:59 +02:00
weingartner 50c4836c9f Cleaned class “com.cloud.hypervisor.xenserver.resource.XcpOssResource” that seemed to be forgotten during the evolution of the ACS. It was removed a few methods that were already coded properly in its parent class “com.cloud.hypervisor.xenserver.resource.CitrixResourceBase”. It was also removed some methods that seemed to cause weird behaviors. The methods removed/fixed are detailed as follows: • com.cloud.hypervisor.xenserver.resource.XcpOssResource.fillHostInfo(Connection, StartupRoutingCommand) – it was removed, because it always added the string “, hvm” to the host capabilities. Therefore, if one uses XCP hypervisor it could cause a lot of trouble when deploying HVM virtual machines in an environment that has PV and HVM clusters. The method is already properly coded in parent class. • com.cloud.hypervisor.xenserver.resource.XcpOssResource.launchHeartBeat(Connection) – It was removed. It was not performing anything and always returns a true value. The method of parent class is properly coded and works for XCP environments. The heartbeat plugin exists in XCP environment. • com.cloud.hypervisor.xenserver.resource.XcpOssResource.initializeLocalSR(Connection) – it was removed. The method of the parent class works properly for XCP environments. • com.cloud.hypervisor.xenserver.resource.XcpOssResource.createPatchVbd(Connection, String, VM) – It was removed. This method causes a bug in XCP environments, because of its half-implementation, it was not possible to migrate system VMs. The parent class implementation works properly for XCP. • com.cloud.hypervisor.xenserver.resource.XcpOssResource.execute(NetworkUsageCommand) – removed, hence it was already coded into parent class and its respective wrappers (“com.cloud.hypervisor.xenserver.resource.wrapper.xcp.XcpServerNetworkUsageCommandWrapper”). BTW: I noticed that the class XcpServerNetworkUsageCommandWrapper and XenServer56NetworkUsageCommandWrapper are almost the same, with the exception that XenServer56NetworkUsageCommandWrapper deals with VPC. I believe that those wrappers could be converted into one, and moved to parent. I am not doing that here because I do not have a XCP environment with advanced networking to test it. • com.cloud.hypervisor.xenserver.resource.XcpOssResource.executeRequest(Command) – removed, hence it is not needed anymor. • com.cloud.hypervisor.xenserver.resource.XcpOssResource.execute(StopCommand) – I did not understand that method. It seemed weird and its removal did not change any behavior of the environment I tested it with.
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
2015-07-09 13:37:59 +02:00
wilderrodrigues e348e76eee CLOUDSTACK-8607 - Improving unit tests
- Adding more unit tests as suggested by Daan and Rajani.
   - Mocking SshHelper and the Script classes, used by Citrix and Libvirt resources respectively.
2015-07-06 14:58:17 +02:00
wilderrodrigues 86297e70be CLOUDSTACK-8607 - Make sure the new password replaces the old one in the queue
- Added log info to show details of the operation
   - Renamed the addPwdToQueue to replaceOldPasswdInQueue
2015-07-04 11:33:52 +02:00
wilderrodrigues efa34361df CLOUDSTACK-8607 - Changed update script to return exit code based on the result
- Changed location of the update_host_passwd script
   - Updated the patch files for XenServer
   - Updated the script path on LibvirtComputing class
   - Removed the hostIP from the LibvirtUpdateHostPasswordCommandWrapper execute() method
2015-07-03 14:29:57 +02:00
wilderrodrigues 0dd02ce043 CLOUDSTACK-8607 - Adding support to update host passwd on XenServer hypervisors
- Adding update_host_passwd to VRScripts
   - Add accessor method to host password on CitrixResourceBase
   - Add implementation to CitrixUpdateHostPasswordCommandWrapper
   - Improve testUpdateHostPasswordCommand() unit test on CitrixRequestWrapperTest
   - Add line to patch files on xenserver directory

Concerning the LibVirt change:

   - I forgot to assing the return of the getDefaultHypervisorScriptsDir() method to the hypervisorScriptsDir variable
2015-07-03 10:24:44 +02:00
Sanjay Tripathi 9044a02d9a VM Snapshot Usage for DATA disk is reported as the size of the DATA disk instead of the actual snapshot size.
This closes #485
2015-06-19 15:55:21 +05:30
Sanjay Tripathi d702b63bfb CLOUDSTACK-8569: The latter snapshot export for the same volume will fail is 2 snapshot exports are queued
This closes #469
2015-06-17 15:27:03 +05:30
Sanjay Tripathi 47278556f5 CLOUDSTACK-8567 Migrating primary storage causes name_label field to blank.
This closes #468
2015-06-17 14:46:51 +05:30
Rafael da Fonseca be474f95a1 Fixed 4 findbugs warnings in CitrixResourceBase.java Unnecessary boxing/unboxing of primitive values
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

This closes #425
2015-06-15 12:09:39 +03:00