This adds support for JDK11 in CloudStack 4.14+:
- Fixes code to build against JDK11
- Bump to Debian 9 systemvmtemplate with openjdk-11
- Fix Travis to run smoketests against openjdk-11
- Use maven provided jdk11 compatible mysql-connector-java
- Remove old agent init.d scripts
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* Enable PVLAN support on L2 networks
* Fix prevent null pointer on details
* Add marvin tests
* Fixes from comments
* Fix: missing pvlan type on plugniccommand
* Fix checks on network creation for vlans overlap
* Fix remove prefix from secondary vlan id
* Improve checks on physical network for pvlans
* Fix compatibility with previous pvlan creation
* Fix shared networks backwards pvlan compatibility
* Add ui fix for pvlan type not passed to api
* Add check for isolated vlan id overlap
* Include check for dynamic vlan reserved for secondary vlan
* Fix marvin tests errors
* Fix redundant imports
* Skip marvin test for pvlan if dvswitch is not present
* spelling
Co-authored-by: Andrija Panic <45762285+andrijapanicsb@users.noreply.github.com>
The VM ingestion feature allows CloudStack to discover, on-board, import existing VMs in an infra. The feature currently works only for VMware, with a hypervisor agnostic framework which may be extended for KVM and XenServer in future.
Problem: In Vmware, appliances that have options that are required to be answered before deployments are configurable through vSphere vCenter user interface but it is not possible from the CloudStack user interface.
Root cause: CloudStack does not handle vApp configuration options during deployments if the appliance contains configurable options. These configurations are mandatory for VM deployment from the appliance on Vmware vSphere vCenter. As shown in the image below, Vmware detects there are mandatory configurations that the administrator must set before deploy the VM from the appliance (in red on the image below):
Solution:
On template registration, after it is downloaded to secondary storage, the OVF file is examined and OVF properties are extracted from the file when available.
OVF properties extracted from templates after being downloaded to secondary storage are stored on the new table 'template_ovf_properties'.
A new optional section is added to the VM deployment wizard in the UI:
If the selected template does not contain OVF properties, then the optional section is not displayed on the wizard.
If the selected template contains OVF properties, then the optional new section is displayed. Each OVF property is displayed and the user must complete every property before proceeding to the next section.
If any configuration property is empty, then a dialog is displayed indicating that there are empty properties which must be set before proceeding
image
The specific OVF properties set on deployment are stored on the 'user_vm_details' table with the prefix: 'ovfproperties-'.
The VM is configured with the vApp configuration section containing the values that the user provided on the wizard.
Problem: The VM metrics has aggregated volume bytes read/write and iops metrics but not on per volume basis.
Root Cause: The volume stats sub-system is not used to export the metrics, the support is not available for VMware.
Solution: Use the volume stats sub-system and DB table to export the metrics via the listVolumes and listVolumeMetrics API, and implement support for VMware and fix issue with network and disk metrics in the VM metrics view.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This fixes potential NPE case when memory hotpluggability is checked
based on the guest OS descriptor.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* - Offline VM and Volume migration on Vmware hypervisor hosts
- Also add VM disk consolidation call on successful VM migrations
* Fix indentation of marvin test file and reformat against PEP8
* * Fix few comment typos
* Refactor debug messages to use String.format() when debug log level is enabled.
* Send list of commands returned by hypervisor Guru instead of explicitly selecting the first one
* Fix unhandled NPE during VM migration
* Revert back to distinct event descriptions for VM to host or storage pool migration
* Reformat test_primary_storage file against PEP-8 and Remove unused imports
* Revert back the deprecation messages in the custom StringUtils class to favour the use of the ApacheUtils
[VMware] VM is not accessible after migration across clusters.
Once a VM is successfully started, don't delete the files associated with the unregistered VM, if the files are in a storage that is being used by the new VM.
Attempt to unregister a VM in another DC, only if there is a host associated with a VM.
This closes#556
* Cleaup and code-formatting POM files
* Remove obsolete mycila license-maven-plugin
* Remove obsolete console-proxy/plugin project
* Move console-proxy-rdbconsole under console-proxy parent
* Use correct parent path for rdpconsole
* Order alphabetally items in setnextversion.sh
* Unifiy License header in POMs
* Alphabetic order of modules definition
* Extract all defined versions into parent pom
* Remove obsolete files: version-info.in, configure-info.in
* Remove redundant defaultGoal
* Remove useless checkstyle plugin from checkstyle project
* Order alphabetally items in pom.xml
* Add aditional SPACEs to fix debian build
* Don't execute checkstyle on parent projects
* Use UTF-8 encoding in building checkstyle project
* Extract plugin versions into properties
* Execute PMD plugin on all the projects with -Penablefindbugs
* Upgrade maven plugins to latest version
* Make sure to always look for apache parent pom from repository
* Fix incorrect version grep in debian packaging
* Fix rebase conflicts
* Fix rebase conflicts
* Remove PMD for now to be fixed on another PR
Fixes the version in pom etc. to be consistent with versioning pattern as X.Y.Z.0-SNAPSHOT after a minor release.
Signed-off-by: Khosrow Moossavi <khos2ow@gmail.com>
* CLOUDSTACK-9184: Fixes#2631 VMware dvs portgroup autogrowth
This deprecates the vmware.ports.per.dvportgroup global setting.
The vSphere Auto Expand feature (introduced in vSphere 5.0) will take
care of dynamically increasing/decreasing the dvPorts when running out
of distributed ports . But in case of vSphere 4.1/4.0 (If used), as this
feature is not there, the new default value (=> 8) have an impact in the
existing deployments. Action item for vSphere 4.1/4.0: Admin should
modify the global configuration setting "vmware.ports.per.dvportgroup"
from 8 to any number based on their environment because the proposal
default value of 8 would be very less without auto expand feature in
general. The current default value of 256 may not need immediate
modification after deployment, but 8 would be very less which means
admin need to update immediately after upgrade.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Remove maven standard module (which only a few were using) and get ride of maven customization for the projects structure.
- moved all directories to src/main/java, src/main/resources, src/main/scripts, src/test/java, src/test/resources
- grep scan to search for src/com and src/org left over
- grep for <project>/scripts to fix pom.xml configuration
- remove custom <build> configuration in pom.xml
Signed-off-by: Marc-Aurèle Brothier <m@brothier.org>
This fixes regression failures seen in Trillian, fixes NPEs that cause Travis related failures.
This also removes the aria2 dependency from rpms that require users to enable/install epel-release.
This finally updates the checksums for 4.11 systemvmtemplates in db upgrade path.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Allowed zone-wide primary storage based on a custom plug-in to be added via the GUI in a KVM-only environment (previously this only worked for XenServer and VMware)
Added support for root disks on managed storage with KVM
Added support for volume snapshots with managed storage on KVM
Enable creating a template directly from a volume (i.e. without having to go through a volume snapshot) on KVM with managed storage
Only allow the resizing of a volume for managed storage on KVM if the volume in question is either not attached to a VM or is attached to a VM in the Stopped state.
Included support for Reinstall VM on KVM with managed storage
Enabled offline migration on KVM from non-managed storage to managed storage and vice versa
Included support for online storage migration on KVM with managed storage (NFS and Ceph to managed storage)
Added support to download (extract) a managed-storage volume to a QCOW2 file
When uploading a file from outside of CloudStack to CloudStack, set the min and max IOPS, if applicable.
Included support for the KVM auto-convergence feature
The compression flag was actually added in version 1.0.3 (1000003) as opposed to version 1.3.0 (1003000) (changed this to reflect the correct version)
On KVM when using iSCSI-based managed storage, if the user shuts a VM down from the guest OS (as opposed to doing so from CloudStack), we need to pass to the KVM agent a list of applicable iSCSI volumes that need to be disconnected.
Added a new Global Setting: kvm.storage.live.migration.wait
For XenServer, added a check to enforce that only volumes from zone-wide managed storage can be storage motioned from a host in one cluster to a host in another cluster (cannot do so at the time being with volumes from cluster-scoped managed storage)
Don’t allow Storage XenMotion on a VM that has any managed-storage volume with one or more snapshots.
Enabled for managed storage with VMware: Template caching, create snapshot, delete snapshot, create volume from snapshot, and create template from snapshot
Added an SIOC API plug-in to support VMware SIOC
When starting a VM that uses managed storage in a cluster other than the one it last was running in, we need to remove the reference to the iSCSI volume from the original cluster.
Added the ability to revert a volume to a snapshot
Enabled cluster-scoped managed storage
Added support for VMware dynamic discovery
Extending Config Drive support
* Added support for VMware
* Build configdrive.iso on ssvm
* Added support for VPC and Isolated Networks
* Moved implementation to new Service Provider
* UI fix: add support for urlencoded userdata
* Add support for building systemvm behind a proxy
Co-Authored-By: Raf Smeets <raf.smeets@nuagenetworks.net>
Co-Authored-By: Frank Maximus <frank.maximus@nuagenetworks.net>
Co-Authored-By: Sigert Goeminne <sigert.goeminne@nuagenetworks.net>
CloudStack volumes and templates are one single virtual disk in case of XenServer/XCP and KVM hypervisors since the files used for templates and volumes are virtual disks (VHD, QCOW2). However, VMware volumes and templates are in OVA format, which are archives that can contain a complete VM including multiple VMDKs and other files such as ISOs. And currently, Cloudstack only supports Template creation based on OVA files containing a single disk. If a user creates a template from a OVA file containing more than 1 disk and launches an instance using this template, only the first disk is attached to the new instance and other disks are ignored.
Similarly with uploaded volumes, attaching an uploaded volume that contains multiple disks to a VM will result in only one VMDK to being attached to the VM.
FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Support+OVA+files+containing+multiple+disks
This behavior needs to be improved in VMWare to support OVA files with multiple disks for both uploaded volumes and templates. i.e. If a user creates a template from a OVA file containing more than 1 disk and launches an instance using this template, the first disk should be attached to the new instance as the ROOT disk and volumes should be created based on other VMDK disks in the OVA file and should be attached to the instance.
Signed-off-by: Abhinandan Prateek <abhinandan.prateek@shapeblue.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* Cleanup and Improve NetUtils
This class had many unused methods, inconsistent names and redundant code.
This commit cleans up code, renames a few methods and constants.
The global/account setting 'api.allowed.source.cidr.list' is set
to 0.0.0.0/0,::/0 by default preserve the current behavior and thus
allow API calls for accounts from all IPv4 and IPv6 subnets.
Users can set it to a comma-separated list of IPv4/IPv6 subnets to
restrict API calls for Admin accounts to certain parts of their network(s).
This is to improve Security. Should an attacker steal the Access/Secret key
of an account he/she still needs to be in a subnet from where accounts are
allowed to perform API calls.
This is a good security measure for APIs which are connected to the public internet.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
* CLOUDSTACK-9972: Enhance listVolume API to include physical size and utilization.
Also fixed pool, cluster and pod info
* CLOUDSTACK-9972: Fix volume_view and duplicate API constant
* CLOUDSTACK-9972: Backport Do not allow vms to be deployed on hosts that are in disabled pod
* CLOUDSTACK-9972: Fix localization missing keys
* CLOUDSTACK-9972: Fix sql path
Allow security policies to apply on port groups:
- Accepts security policies while creating network offering
- Deployed network will have security policies from the network offering
applied on the port group (in vmware environment)
- Global settings as fallback when security policies are not defined for a network
offering
- Default promiscuous mode security policy set to REJECT as it's the default
for standard/default vswitch
Portgroup vlan-trunking options for dvswitch: This allows admins to define
a network with comma separated vlan id and vlan
range such as vlan://200-400,21,30-50 and use the provided vlan range to
configure vlan-trunking for a portgroup in dvswitch based environment.
VLAN overlap checks are performed for:
- isolated network against existing shared and isolated networks
- dedicated vlan ranges for the physical/public network for the zone
- shared network against existing isolated network
Allow shared networks to bypass vlan overlap checks: This allows admins
to create shared networks with a `bypassvlanoverlapcheck` API flag
which when set to 'true' will create a shared network without
performing vlan overlap checks against isolated network and against
the vlans allocated to the datacenter's physical network (vlan ranges).
Notes:
- No vlan-range overlap checks are performed when creating shared networks
- Multiple vlan id/ranges should include the vlan:// scheme prefix
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
If there are multiple files with the same name on vmware datastore, search operation may select any one file during volume related operations. This involves volume attach/detach, volume download, volume snapshot etc.
While using NetApp as the backup solution. This has .snapshot folder on the datastore and sometimes files from this folder gets selected during volume operations and the operation fails. Because of wrong selection of file following exception can be observed while volume deletion.
2017-02-23 19:39:05,750 ERROR [c.c.s.r.VmwareStorageProcessor] (DirectAgent-304:ctx-a1dbf5d8 ac.local) delete volume failed due to Exception: java.lang.RuntimeException
Message: Cannot delete file [4cbcd46d44c53f5c8244c0aad26a97e1] .snapshot/hourly.2017-02-23_1605/r-97-VM/ROOT-97.vmdk
To fix this behavior I have added a global configuration by name vmware.search.exclude.folders which can be comma separated list of folder paths.
I have also added a unit test to test the new method.
* 4.9:
Do not set gateway to 0.0.0.0 for windows clients
CLOUDSTACK-9904: Fix log4j to have @AGENTLOG@ replaced
ignore bogus default gateway when a shared network is secondary the default gateway gets overwritten by a bogus one dnsmasq does the right thing and replaces it with its own default which is not good for us so check for '0.0.0.0'
Activate NioTest following changes in CLOUDSTACK-9348 PR #1549
CLOUDSTACK-9828: GetDomRVersionCommand fails to get the correct version as output Fix tries to return the output as a single command, instead of appending output from two commands
CLOUDSTACK-3223 Exception observed while creating CPVM in VMware Setup with DVS
CLOUDSTACK-9787: Fix wrong return value in NetUtils.isNetworkAWithinNetworkB
- commented some occurences of cloud.com as being harmless
* examples
* identifiers (internal)
- changed the URL for vhd-util download
- changed comments from 'cloud.com' to 'Apache CloudStack'
Test scenarios:
- Enable cluster HA after VR is created. Now stop and start VR and check its restart priority, should be High.
- Enable cluster HA before VR is created. Now create some VM and verify that VR created must have High restart priority.
This removes network details from the guest VM template OVF xml before deploying
a VM which would fail in case of dvswitch-based vmware environment with no
dummy/existing vswitch.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Issue
=====
Exception observed while creating CPVM in VMware Setup with DVS.
Observed error "StartCommand failed due to Exception: com.vmware.vim25.AlreadyExists."
This is due to concurrent attempts to create same dv portgroup on same dvSwitch by
manager threads of CPVM and SSVM when both are started at same time.
Fix
===
Synchronize api calls to create/update dvportgroup.
Also maintaing local cache to avoid multiple fetch api calls to vCenter
when multiple threads try to create same object.
Signed-off-by: Sateesh Chodapuneedi <sateesh.chodapuneedi@accelerite.com>
CLOUDSTACK-9673 : Exception occured while creating the CPVM in VMware setup over standard vSwitchesJira
===
CLOUDSTACK-9673 : Exception occured while creating the CPVM in VMware setup over standard vSwitches
Issue
====
Exception occured while creating the CPVM in the VmWare Setup using standard vswitches.
```
StartCommand failed due to Exception: com.vmware.vim25.AlreadyExists
message: [] com.vmware.vim25.AlreadyExistsFaultMsg: The specified key, name, or identifier already exists
```
Fix
===
Ensure synchronization while attempting to create port group such that simultaneous attempts are not made with same port group name on same ESXi host.
Testing
======
Successfully ran manual tests (deploy user instance) on top of latest master commit `17653a86fad67447a4f13e455e336694ad5c1735`.This code change is involved in virtual network creation over VMware standard vSwitches. Existing functional tests covers this functionality.
* pr/1827:
CLOUDSTACK-9673 Exception occured while creating the CPVM in the VmWare Setup over standard vSwitches
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Issue
====
Start instance fails after reverting to a VM snapshot, when there is 1 or more child VM snapshots in the snapshot tree of the VM.
Per the code that detects the presence of a snapshot, we are checking for only current snapshot instead of checking presence of any snapshot in the snapshot tree.
The failure to detect all snapshots means ACP reconfigures the VM in wrong way assuming there are no snapshots for the VM.
This results in start failure.
Fix
===
Ensure correct detection of VM snapshots in the VM snapshot tree
This closes#1828
Signed-off-by: Sateesh Chodapuneedi <sateesh.chodapuneedi@accelerite.com>
(cherry picked from commit 673bb25b5936d1c54e9210781280e9ddc507c830)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Issue
====
Exception occured while creating the CPVM in the VmWare Setup using standard vswitches.
StartCommand failed due to Exception: com.vmware.vim25.AlreadyExists
message: [] com.vmware.vim25.AlreadyExistsFaultMsg: The specified key, name, or identifier already exists
Fix
===
Ensure synchronization while attempting to create port group such that simultaneous attempts are not made with same port group name on same ESXi host.
Signed-off-by: Sateesh Chodapuneedi <sateesh.chodapuneedi@accelerite.com>
The test assertion on a pool object may return a null object, as objects
can be randomly expired/tombstoned. This will fix a NPE sometimes seen due
to recently merge for the fix for CLOUDSTACK-9564.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9564: Fix memory leaks in VmwareContextPoolIn a recent management server crash, it was found that the largest contributor
to memory leak was in VmwareContextPool where a registry is held (arraylist)
that grows indefinitely. The list itself is not used anywhere or consumed. There
exists a hashmap (pool) that returns a list of contexts for existing poolkey
(address/username) that is used instead.
This fixes the issue by removing the arraylist registry, and limiting the
length of the context list for a given poolkey.
@blueorangutan package
* pr/1729:
CLOUDSTACK-9564: Fix memory leaks in VmwareContextPool
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
In a recent management server crash, it was found that the largest contributor
to memory leak was in VmwareContextPool where a registry is held (arraylist)
that grows indefinitely. The list itself is not used anywhere or consumed. There
exists a hashmap (pool) that returns a list of contexts for existing poolkey
(address/username) that is used instead.
This fixes the issue by removing the arraylist registry, and limiting the
length of the context list for a given poolkey.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Improve disk chain usage while attaching, migrating disks
- Gets root disk controller based diskDeviceBusName from volume's chain info
- Refactor and move VirtualMachineDiskInfo to cloud-utils
- Allows mixing of scsi controller types
- Fixes a NPE case with map passed as null, for example in case of detach volume
command
- Use a osdefault translator that allow use of recent os types added (enums of
which) are not available in the sdk
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* 4.6:
[UI] bug fix: Delete added ACL lists is not available for Domain Admin and normal users
CLOUDSTACK-4787: Allow users to select disk controller for VM/template
CLOUDSTACK-4787 Allow selection of scsi controller type in vSphere
commit #7
So far only 1 controller (scsi or ide) is supported in Cloudstack for ide or
scsi, this is existing limitation. Added support for 2nd IDE controller. Support adding IDE
virtual disk to VM. Also added check if VM is running as IDE virtual disk cannot be attached
to VM if VM is runnning.If user detaches a virtual disk on lower unit number of controller,
then subsequent attach operation should find free unit number on the controller and attach
the virtual disk there.
commit #6
Let the controllers of existing VMs continue without flip, current busInfo retrieved from
chain_info field of volume record from database would be preferred over
controller settings from all configuration settings.
commit #5
Editing global configuration param vmware.root.disk.controller osdefault value results
in loss of previous root disk controller type. Hence root disk's controller type for legacy
VMs is unknow post that modificaiton by user. If VM is stop/start then we could get this
infromation from bus info of existing volume. But if user resets VM and then try to start VM.
The existing bus info would be lost. Hence existing disk info is not available to depend on.
Using lsilogic or generic scsi controller for ROOT disk of legacy VMs if reset.
commit #4
Avoid adding additional (>1) scsi controllers to system vms. While attaching volume to legacy VM
don't use osdefault optoin which applicable only for VM created with the option enabled, use
legacy data disk controller type (lsilogic)
commit #3
If root disk's controller type is scsi and data disk controller type condenses
to any of scsi sub-types then data disk controller type would fall back to root disk controller itself. This
ensures data volumes would be accessible in all cases as controller of root volume would be reliable
and it means VM has the supported controller. It also avoids mix of scsi controller sub-types in a user instance.
Also translating disk controller type scsi to lsilogic.
commit #2
Support auto detection of recommended virtual disk controller type for specific guest OS.
commit #1
Support granual controller types. Add support for controller types in template registration as well.
Fix white spaces.
Removed stale HEAD merge lines
Removed tail of merge lines
Fixed VmwareResource, removing storage commands that moved to VmwareStorageProcessor.
removed stale code of controller that is present in processor
Fixed check style errors.
Fixed injection.
Tested with Linux and windows templates. Unable to run iso based tests due to few bugs in register iso area.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
(cherry picked from commit a4cc987a6f)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-8669: create volume failed due to null charsetAdded a new private method getCharSetFromConnection() which checks if
the connection charset is null and if it is null, returns
StringUtils.getPreferredCharset
regression caused by commit f03411ca04
* pr/695:
CLOUDSTACK-8669: create volume failed due to null charset
CLOUDSTACK-8669: making StringUtils.getDefaultCharset protected
Signed-off-by: Remi Bergsma <github@remi.nl>
Added a new private method getCharSetFromConnection() which checks if
the connection charset is null and if it is null, returns
StringUtils.getPreferredCharset
regression caused by commit f03411ca04
CLOUDSTACK-3317 - DVS does not support management\storage network
Added support for Management and Storage Network traffic over VMware DVS in CloudStack deployments. Also added support for storage VLAN over dvPortGroup.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
CLOUDSTACK-3317 - DVS does not support management\storage network
Use non-zero dvport count while updating dvportgroups of system traffic.
Updated configuration compare logic to avoid update dvportgroup operation unless required.
This would help improve speed in vm/network deployment as the update calls would reduce. Also improved logging.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Adding unit tests in class HypervisorHostHelperTest
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Added license header to new file being added to repo/branch.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
MigrateVMWithVolumes-
1. If ESXi host version is below 5.1, ensure destination datastore(s) is mounted on the source host, then migrate the storage and then finally migrate the VM.
If destination storage(s) is not mounted on the source host,
- In case of NFS storage mount the storage(s).
- In case of VMFS storage fail the request for migration.
2. If EXi host version is 5.1 or above, simultaneously migrate the VM and its storage to the destination host and storage(s) respectively for both NFS and VMFS storage.
1. If a VM by the same name exists on a different cluster in the VMware DC, unregister the existing VM and continue with the VM start.
2. If VM start succeeds, delete VM files associated with the unregistered VM.
3. If VM start fails, re-register the unregistered VM.
For AttachVolume/DetachVolume API command, improve user error message in case of RuntimeException by throwing the exception instead of 'Unexpected Exception'.
While attaching a new disk to an instance, the unit number on the controller key should be the lowest unit number
that is not in use. And in case the controller type is SCSI it shouln't be the reserved SCSI unit number.
Look for a VM in vCenter based on both the vCenter name and CS internal name (required in case 'vm.instancename.flag' is enabled).
During Attach Volume and Volume Migration, for lookup and other operations use VM's name as obtained from vCenter instead of using the name set in the agent command.
During VM start while configuring its disk devices, obtain the matching disk for a volume in storage
using both the volume's path and volume's datastore information.
Change in vCenter 5.5 API from prior versions forced code change in CloudStack. Update property value of property "VirtualE1000.deviceInfo.summary" is accommodated now.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
[VMware] Synchronize the tasks of disk preparation and reconfiguration of a VM with the disk.
This is to avoid attaching a disk with a wrong device number on the controller key.
[VMware] While attaching a new disk to an instance, the unit number on the controller key should be the lowest unit number on the key that is not in use.
Instead of a number that is 1 digit higher the highest unit number that is currently in use.
vlan id format was like "vlan://<id>" instead of just "<id>". This causes numberformatexception while converting the vlan id to integer form from string form. this was fixed for standard vswitch in bug Cloudstack-5046. now fixed for other 2 cases of dvswitch as well as pvlan.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
In vCenter 5.5, once a volume is migrated the VMDKs are renamed to match the name of the VM.
If a volume has been renamed upon migration update its volumePath to that of the new disk filename.
Conflicts:
plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java
vmware-base/src/com/cloud/hypervisor/vmware/mo/VirtualMachineMO.java
We hit these excptions whenever a management server held session that was with the old 5.1 vCenter server
is used to make resource calls to the new 5.5 vCenter.
Validate a vCenter session context before it is being used to make a resource call.
And if the context is invalid then discard the context and retrieve a new one.
During the invalidation of an old context handle the context disconnect better
by catching the appropriate exception and returning a newly created context.
Conflicts:
plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareContextFactory.java
plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java
plugins/hypervisors/vmware/src/com/cloud/storage/resource/VmwareSecondaryStorageResourceHandler.java
vmware-base/src/com/cloud/hypervisor/vmware/util/VmwareContext.java
DetachISO is succeeding even though detach opeartion is failing as cdrom is locked by VM as it was mounted inside VM.
Detect if cdrom is locked or not. If locked fail detach operation and warn user to unmount before detaching the iso/cdrom device.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Conflicts:
plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java
plugins/hypervisors/vmware/src/com/cloud/storage/resource/VmwareStorageProcessor.java
vmware-base/src/com/cloud/hypervisor/vmware/mo/VirtualMachineMO.java
During VM start, if VM already exists and CS finds a matching disk in storage for each of the existing disk then the VM is reconfigured with VM's existing disk information. This existing disk information for a VM is got from vCenter.
CS checks if a matching disk exists in storage by checking if there is a disk in storage whose name atleast partially matches with the name of the existing disk.
Post 4.1, Volume path has been changed to ROOT-<instanceId> from ROOT-<instanceId>-<volumeId>. Hence in case of an upgraded setup (pre 4.1 to 4.2) during search for an existing disk in storage even though ROOT-<instanceId>-<volumeId> has been deleted, CS does a postive match of the old existing disk because of the partial match with the new disk ROOT-<instanceId> that exists in storage. And so when VM is being powered on by vCenter it looks for ROOT-<instanceId>-<volumeId> and fails to power on.
Solution - While looking for a VM's matching disk in storage, instead of checking if a disk with a partial match exists check if a disk with the exact name exists.
During ISO detach operation, answer question from vCenter by programmatically answering for VM question in detaching process.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
If HttpNfcLease initialization fails throw appropriate error
Catch throwable in addition to exceptions during file upload task while importing the OVA into vCenter
Cause - This issue can occur in VMware setups if the guest operating system is not specified when we install the virtual machine. In which case, since we map the OS to 'Other OS', VMwareInstallerISO attachment fails.
Added the necessary guest OS mapping for VMware.
The managed context framework provides a simple way to add logic
to ACS at the various entry points of the system. As threads are
launched and ran listeners can be registered for onEntry or onLeave
of the managed context. This framework will be used specifically
to handle DB transaction checking and setting up the CallContext.
This framework is need to transition away from ACS custom AOP to
Spring AOP.
If VMware host version is ESXi 4.1 and vCenter version is 5.0 or later, number of cores per socket of a VM running on ESXi 4.1 host would be undefined as it's not supported for versions prior to 5.0.
Hence expecting undefined/null value for that and handling it.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Fix the name of the interface id in extra config, so the nic is found up using the nic uuid instead of the vm id.
getVlanInfo should return null when the nic is plugged on a Lswitch network.
On a distributed portgroup all vms should be connected to a single
portgroup and set a specific vlan on the port. On a standard switch each nic should have its own portgroup with a dedicated vlan (not related to CS vlans)
Following Hotadd memory checks are included now.
1) Check if guest operating system supports memory hotadd
2) Check if virtual machine is using hardware version 7 or later before enabling memory hotadd
Following Hotadd CPU checks are included now.
1) Check if guest operating system supports cpu hotadd
2) Check if virtual machine is using hardware version 8 or later.
3) Check if virtual machine has only 1 core per socket. If hardware version is 7, then only 1 core per socket is supported. Hot adding multi-core vcpus is not allowed if hardware version is 7.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Retrieve maximum hotadd memory limit & hotadd memory increment size from running VM's configuration to validate dynamic scale memory limit while scaling up a VM.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Now ESXi server license would be fetched to see if HOTPLUG feature is license or not.
Throw Exception if the feature is not licensed.
Also added FeatureKeyConstants enum type to maintain list of various features to be checked whether licensed or not.
instead of a dynamically configured one.
Refactor bits of the HypervisorHostHelper to better
support multiple BroadcastDomainTypes
Cleanup some parts of the code that was using tab indents and removed some dead code.
Read the global configuration setting while configuring VmwareManager.
Also enabling autoExpand feature for supported distributed virtual switch versions.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
CLOUDSTACK-659
Support for storage migration in Cloudstack deployment over VMware Added
Resource changes to perform VM live migration along with virtual disks across the clusters in a zone.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
CLOUDSTACK-659
Support for storage migration in Cloudstack deployment over VMware
Added VmwareStorageMotionStrategy to application context.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Support for volume live migration across datastores
Unit tests for vmware storage motion. These test the VmwareStorageMotionStrategy.
CLOUDSTACK-659
Fixing migrate volume.
CLOUDSTACK-659
Support for storage migration in Cloudstack deployment over VMware Added
Resource changes to perform VM live migration along with virtual disks across the clusters in a zone.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Support for volume live migration across datastores
CLOUDSTACK-659
Support for storage migration in Cloudstack deployment over VMware Added
Resource changes to perform VM live migration along with virtual disks across the clusters in a zone.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Support for volume live migration across datastores
CLOUDSTACK-659
Support for storage migration in Cloudstack deployment over VMware Added
Resource changes to perform VM live migration along with virtual disks across the clusters in a zone.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Support for volume live migration across datastores
Added vm type to prepareNetworkFromNicInfo in MigrateWithStorageCommand implementation.
CLOUDSTACK-2701 - Enable storage migration for VMware resources
Fixing attach volume and delete volume cases for volumes that are moved off original path in datastore when created.
If volume is not found in root directory or datastore, do search in sub folders.
CLOUDSTACK-2701 - Enable storage migration for VMware resources
Sending command MigrateWithStorageCommand to source host instead of target host for the case of migration of VM within cluster.
CLOUDSTACK-2701 - Enable storage migration for VMware resources
Searching for virtual disk during device tear down.
Adding dependency of 'cloud-engine-storage' to vmware hypervisor plugin.
Add hypervisor capability storage_motion_supported for VMware 5.0 and 5.1
CLOUDSTACK-659
Support for storage migration in Cloudstack deployment over VMware
Added VmwareStorageMotionStrategy to deal with storage motion tasks.
Added target host parameter to MigrateWithStorageCommand.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
CLOUDSTACK-659
Support for storage migration in Cloudstack deployment over VMware Added
Resource changes to perform VM live migration along with virtual disks across the clusters in a zone.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
CLOUDSTACK-659
Support for storage migration in Cloudstack deployment over VMware
Added VmwareStorageMotionStrategy to application context.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Support for volume live migration across datastores
Unit tests for vmware storage motion. These test the VmwareStorageMotionStrategy.
CLOUDSTACK-659
Fixing migrate volume.
CLOUDSTACK-659
Support for storage migration in Cloudstack deployment over VMware Added
Resource changes to perform VM live migration along with virtual disks across the clusters in a zone.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Support for volume live migration across datastores
Added vm type to prepareNetworkFromNicInfo in MigrateWithStorageCommand implementation.
Adding dependency of 'cloud-engine-storage' to vmware hypervisor plugin.
Add hypervisor capability storage_motion_supported for VMware 5.0 and 5.1
CLOUDSTACK-659
Support for storage migration in Cloudstack deployment over VMware
Added VmwareStorageMotionStrategy to deal with storage motion tasks.
Added target host parameter to MigrateWithStorageCommand.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
CLOUDSTACK-659
Support for storage migration in Cloudstack deployment over VMware Added
Resource changes to perform VM live migration along with virtual disks across the clusters in a zone.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
CLOUDSTACK-659
Support for storage migration in Cloudstack deployment over VMware
Added VmwareStorageMotionStrategy to application context.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Support for volume live migration across datastores
Unit tests for vmware storage motion. These test the VmwareStorageMotionStrategy.
CLOUDSTACK-659
Fixing migrate volume.
Adding dependency of 'cloud-engine-storage' to vmware hypervisor plugin.
Add hypervisor capability storage_motion_supported for VMware 5.0 and 5.1
Adding dependency of 'cloud-engine-storage' to vmware hypervisor plugin.
CLOUDSTACK-659
Support for storage migration in Cloudstack deployment over VMware
Added VmwareStorageMotionStrategy to deal with storage motion tasks.
Added target host parameter to MigrateWithStorageCommand.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
CLOUDSTACK-659
Support for storage migration in Cloudstack deployment over VMware Added
Resource changes to perform VM live migration along with virtual disks across the clusters in a zone.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
CLOUDSTACK-659
Support for storage migration in Cloudstack deployment over VMware
Added VmwareStorageMotionStrategy to application context.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Support for volume live migration across datastores
CLOUDSTACK-659
Fixing migrate volume.
Unit tests for vmware storage motion. These test the VmwareStorageMotionStrategy.
CLOUDSTACK-2701 - Enable storage migration for VMware resources
Sending command MigrateWithStorageCommand to source host instead of target host for the case of migration of VM within cluster.
CLOUDSTACK-2701 - Enable storage migration for VMware resources
Moved 2 methods that are not specific to VMware but Volume are moved to VolumeManager from VmwareManager.
Moved datastore volume path constructing code to separate method.
Added check for source and target host, if they are from different DCs/vCenter instances.
Updated error message & removed stale comment
CLOUDSTACK-2701 - Enable storage migration for VMware resources
Injecting component VolumeManager into VmwareResource.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
dahn
Rohit Yadav
Nicolas Vazquez
Abhishek Kumar
Paul Angus
Andrija Panic
Frank Maximus
GabrielBrascher
dahn
sureshanaparti
Khosrow Moossavi
Paul Angus
瓜@アイソマ:ゆるゆりはいいぞ
Rohit Yadav
lzh3636
Marc-Aurèle Brothier
Mike Tutkowski
Frank Maximus
Abhinandan Prateek
Wido den Hollander
Sigert Goeminne
Suresh Kumar Anaparti
Rohit Yadav
SudharmaJain
vedulasantosh
Rajani Karuturi
Rajani Karuturi
Rajani Karuturi
Daan Hoogland
Jayapal
Sateesh Chodapuneedi
Suresh Kumar Anaparti
Will Stevens
Remi Bergsma
Sateesh Chodapuneedi
nvazquez
Likitha Shetty
Rafael da Fonseca
Laszlo Hornyak
Hugo Trippaers
Santhosh Edukulla
Min Chen
Suresh Ramamurthy
Kelven Yang
Mike Tutkowski
Nitin Mehta
Alex Huang
Jijun
Harikrishna Patnala
Darren Shepherd
Prasanna Santhanam
Chip Childers
Vijayendra Bhamidipati