Commit Graph

2514 Commits

Author SHA1 Message Date
Harikrishna Patnala 9b923ba9dc Handle detach volume of datastore cluster if the volume name has changed at vCenter level and reconsile the chaininfo 2020-10-19 15:05:57 +05:30
Harikrishna Patnala 588b7a1c90 Cleanup volume wrapper VM(ROOT-xxx) while deploying VM in case of any failure. since the same name will be used on retries and that will eventually fails saying volume wrapper VM already exists. 2020-10-19 15:05:57 +05:30
nvazquez 897cc4bdba Fix nested virt marvin test 2020-10-19 15:05:57 +05:30
nvazquez 94bebe8792 Revert back deploy as is column on templates but keep it as default for new templates 2020-10-19 15:05:57 +05:30
nvazquez 46d412d998 Fix for system VMs 2020-10-19 15:05:57 +05:30
nvazquez 9b51a706db Set deploy-as-is to default on VMware 2020-10-19 15:05:57 +05:30
Harikrishna Patnala 2f03877723 Search in possible folders for volume while dettaching the volume 2020-10-19 15:05:57 +05:30
Harikrishna Patnala 9543fd6e6a Fix startcommand on Datastore cluster when the volume datastore in CloudStack mismatches with vCenter datastore. Volume could have migrated with in datastore cluster which caused the mismatch
Fix dettach volume when volume is not on CloudStack intended datastore
2020-10-19 15:05:57 +05:30
nvazquez 78aa3df440 Fix reboot deploy-as-is VMs 2020-10-19 15:05:57 +05:30
nvazquez 667cb53772 Fix mismatch from OVF nic adapter type and expected adapter type string 2020-10-19 15:05:57 +05:30
Harikrishna Patnala bb90ce5bd7 Fix for dettached disk migrations from vVols to other datastores 2020-10-19 15:05:57 +05:30
nvazquez 7a8dd46b32 Fix deploy as-is disks cleanup 2020-10-19 15:05:57 +05:30
Harikrishna Patnala 89aa25d023 VM deployment from ISO not working on VSAN65, fixed by adding a backup to create disk using old way 2020-10-19 15:05:57 +05:30
nvazquez c6c4bfe30c Fix deploy as is with datadisks 2020-10-19 15:05:57 +05:30
nvazquez acf872cdab Relocate deploy-as-is cloned VM if does not end on the host receiving the start command and restore deployment logs 2020-10-19 15:05:57 +05:30
nvazquez edfbed34ad Use network adapter from OVF on deploy-as-is 2020-10-19 15:05:57 +05:30
nvazquez 61e7625d49 Do not set user-configurable details for deploy-as-is VMs 2020-10-19 15:05:57 +05:30
Harikrishna Patnala 44dc0c6072 Fixed rat failure on new class DeployAsIsHelper.java
Also removed some unused imports during rebase
2020-10-19 15:05:57 +05:30
nvazquez 19c83fef64 Fix zone to zone scope volume migrations NPE 2020-10-19 15:05:57 +05:30
Harikrishna Patnala 6ebfdc6854 Dettached datadisk migration failed between NFS storages.
Fixed the issue of worker VM name being null
2020-10-19 15:05:57 +05:30
nvazquez bb4ce2118d Add new template and vm deploy as is details table and refactor 2020-10-19 15:05:57 +05:30
Harikrishna Patnala fab6b41c90 Delete VM fix which does not move detached disks to fcd folder
vVols attach disk fix
2020-10-19 15:05:56 +05:30
Harikrishna Patnala f74262b12a Snapshot on dettached volume considering wrong path, corrected it 2020-10-19 15:05:56 +05:30
Harikrishna Patnala f53f11b010 Fix seeding template issue after PR merging 2020-10-19 15:05:56 +05:30
Harikrishna Patnala f825a94dc4 Fixed volume migrations to and fro vVols datastorewq 2020-10-19 15:05:56 +05:30
Harikrishna Patnala 700ab1a5ef vVols datastore support added 2020-10-19 15:05:56 +05:30
Harikrishna Patnala 5652097ef0 Added backup on fail over scenrio in case of new create disk method 2020-10-19 15:05:56 +05:30
nvazquez d864e9dc39 [VMware] Full OVF properties support 2020-10-19 15:05:56 +05:30
Harikrishna Patnala 04d400f055 Handle import unmanaged VM into CloudStack to support any type of datastore 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 40934ba9ff Fix travis failures by removing dependency of vmware from storage.
Added a new command class to verify the vCenter details provided while adding primary storage
2020-10-19 14:57:16 +05:30
Harikrishna Patnala 570f3214b8 Handle VMFS6 sesparse format disk files 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 0a412ac6da Handle volume copy from secondary to primary in case of vSAN 2020-10-19 14:57:16 +05:30
Harikrishna Patnala af84743fd9 Fix addition of datastore clusters not to configure it as non NFS storage 2020-10-19 14:57:16 +05:30
Harikrishna Patnala b88df5ee5b Fixed a smoke test while taking snapshot 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 2dd6e184d5 Fix simulator code for SimulatorStorageProcessor.java 2020-10-19 14:57:16 +05:30
Harikrishna Patnala a5531a9e86 Changed artifact id from pbm for vmware-pbm for policy based management 2020-10-19 14:57:16 +05:30
Pearl Dsilva e211b92324 Changed workflow for storing volumes in sec storage 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 36d65905b1 Base folder creation on datstores restrict only to primary storages, for secondary storage datastore it is not required 2020-10-19 14:57:16 +05:30
Harikrishna Patnala c8a28f34ee Use clonevm task while creating template from volume
Old way is by taking snapshot of entire VM and then clone from that snapshot with the required disk
2020-10-19 14:57:16 +05:30
Pearl Dsilva 4178a7cd90 inter-cluster migration support 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 5aac6c3f0f Fix immutable list to add another entry to list 2020-10-19 14:57:16 +05:30
Harikrishna Patnala ed7efdf4d2 Generalise linked clone formats which are different for each datastore type 2020-10-19 14:57:16 +05:30
Harikrishna Patnala c9c38c0f37 Handle datastore clusters in list storage pools to list only parent not the children in the Datastore cluster 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 53fa3beecf Register Disk to first class disk whereever possible 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 41b3fc19d6 Add Datastore cluster and the child entities which are datastores in the cluster into CloudStack
Setting scope is still pending.
2020-10-19 14:57:15 +05:30
Harikrishna Patnala f858387b3c Import storagepolicies when vmware datacenter is added or updated 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 48786b2d31 DataStore Clusters addition as a storage pool 2020-10-19 14:57:15 +05:30
Harikrishna Patnala c45b83a158 Logging variable name change 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 586fa7eeaf Added listVsphereStoragePoliciesCmd 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 233e665b34 Fix response list on importVsphereStoragePoliciescmd 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 778148ba16 Fix importstorage policies to update deleted policies on vcenter 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 18340e96b1 importVsphereStoragePolices improvements 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 6df819028e UI changes and accept any type of datastore as presetup in vmware 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 4bbb747b23 VMFS to presetup change 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 3d9edeeb9d Bug fixing on default base folder 2020-10-19 14:57:15 +05:30
Harikrishna Patnala aa07959f2a Use VStorageObjectManager for disk operations.
Created disks using VStorageObjectManager
Removed redundant code around attach volume and create volumes
2020-10-19 14:57:15 +05:30
Harikrishna Patnala 487bb03c4d Added Storagepolicy while creating diskoffering. CreateDiskOffering API now takes storagepolicy as a parameter with UUID value 2020-10-19 14:57:15 +05:30
Harikrishna Patnala fb0a96e7fb Check if datastore is complaince with the storagepolicy provided in the disk offering.
Added corresponding manager objects from PBM sdk to do the job.
Made dao layer changes to read the storage policy in diskoffering
2020-10-19 14:57:15 +05:30
Harikrishna Patnala 27e5dfa633 Updated code to handle sync of storage policies when importVsphereStoragePolicies API is called multiple times 2020-10-19 14:57:15 +05:30
Harikrishna Patnala a951e5f57f Fixed unit test failure 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 8b23ea90b8 Moved Dao and VO files to different packages 2020-10-19 14:57:15 +05:30
Harikrishna Patnala a4ec86a652 Added Storage policy id to VO 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 037fcfd032 Fixed bean creation errors on VsphereStoragePolicyDaoImpl 2020-10-19 14:57:15 +05:30
Harikrishna Patnala b8b4f0f9ad Added importVsphereStoragePolicies API and scheme changes 2020-10-19 14:57:15 +05:30
Harikrishna Patnala a72782f2e5 Fix root folder issue on datastore 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 0701dc9d9c Initial commit adding few required managed object classes and added pbm sdk 2020-10-19 14:57:15 +05:30
Daan Hoogland 644731edad Merge branch '4.14' 2020-10-15 18:02:00 +00:00
Wei Zhou be5df8d9ed
kvm: fix wrong VM CPU usage (#4381) 2020-10-15 13:57:59 +02:00
Abhishek Kumar 5b419b90d0
vmware: use hotPlugMemoryIncrementSize only for valid value (#4348)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-10-14 16:05:57 +05:30
Spaceman1984 1bbb29d034
Fixed vm-templates not being removed from primary storage with storag… (#4389)
* Fixed vm-templates not being removed from primary storage with storage garbage collection

* Update vmware-base/src/main/java/com/cloud/hypervisor/vmware/mo/VirtualMachineMO.java

Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>

* Var name

Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>
2020-10-12 15:54:12 +05:30
Abhishek Kumar d3be14d4c8
hypervisor: Add Citrix Hypervisor 8x product name support (#4271)
* changes for supporting Citrix Hypervisor product name

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-10-12 15:50:43 +05:30
Spaceman1984 479ef8aafa
VMware: match hardware version for worker VM when taking a snapshot (#4321)
* Add hardware version to worker VM

* Added worker VM hardware version when creating a template from a volume and migrating a detached volume

* Add null parameter back that was removed by merge
2020-10-09 17:06:07 +02:00
Rohit Yadav 86fcb14238 Merge remote-tracking branch 'origin/4.14'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-09-24 12:20:29 +05:30
Abhishek Kumar ca1e02fab9
vmware: search unmanaged instances using hypervisor name (#4328)
VMware code keeps a cache of existing VMs on a hypervisor host using cloud.vm.internal.name property of the VM. Searching for unmanaged instances/VMs on a host might not return an expected result when this property differs from the actual name of the VM.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-09-24 11:01:33 +05:30
Rohit Yadav 4f8b88baea
vmware: while plugging in nics get existing sorted nic devices (#4336)
In large environments, with VR having multiple nics when plugging in
nic, it must get existing nics by sorted device ID otherwise it may
cause incorrect nic plugging/order.

Fixes #4246

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-09-24 10:59:39 +05:30
Pearl Dsilva 90e72b1e40
vmware: Create template from detached data-disks on VMWare (#4294)
Creation of templates from detached data disks results in a Null Pointer Exception on VMWare, as it expects the volume to be attached to a VM.
To fix this behavior and make it consistent with other hypervisors, creation of the template from the volume in case not attached to a VM is facilitated by creating a worker VM, attaching the disk to the worker VM, creating the template from it, and then destroying the VM.

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-09-18 08:25:17 +05:30
Rohit Yadav bb73bedb55 Merge remote-tracking branch 'origin/4.14' 2020-08-21 15:39:00 +05:30
Rohit Yadav 05ae3f8d81 Merge remote-tracking branch 'origin/4.13' into 4.14 2020-08-21 15:38:18 +05:30
Sid Kattoju 1da76d27f1
iscsi session cleanup now configurable, filters iscsi partitions (#4219)
Added property to agent.properties that enables or disables the iscsi session clean up feature. #4210
Added a condition to prevent disk partitions from being cleaned up. #4216
2020-08-21 14:38:36 +05:30
davidjumani 3872bf1ff9
kvm: Enable PVLAN support on L2 networks (#4040)
This is an extention of #3732 for kvm.
This is restricted to ovs > 2.9.2
Since Xen uses ovs 2.6, pvlan is unsupported.
This also fixes issues of vms on the same pvlan unable to communicate if they're on the same host
2020-08-20 15:46:34 +05:30
Spaceman1984 d57aa83517
server: Added nfs minor version support (#4180)
This PR adds minor version support when mounting nfs on the SSVM as requested in #2861

The global setting "secstorage.nfs.version" has been changed to use the String data type which allows any minor version to be specified.
2020-08-19 14:53:38 +05:30
davidjumani d949302d0f
packaging: Adding Centos8, Ubuntu 20.04, XCPNG8.1 Support (#4068)
* DB : Add support for MySQL 8

- Splits commands to create user and grant access on database, the old
statement is no longer supported by MySQL 8.x
- `NO_AUTO_CREATE_USER` is no longer supported by MySQL 8.x so remove
that from db.properties conn parameters

For mysql-server 8.x setup the following changes were added/tested to
make it work with CloudStack in /etc/mysql/mysql.conf.d/mysqld.cnf and
then restart the mysql-server process:

    server_id = 1
    sql-mode="STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION,ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE,NO_ENGINE_SUBSTITUTION"
    innodb_rollback_on_timeout=1
    innodb_lock_wait_timeout=600
    max_connections=1000
    log-bin=mysql-bin
    binlog-format = 'ROW'

    default-authentication-plugin=mysql_native_password

Notice the last line above, this is to reset the old password based
authentication used by MySQL 5.x.

Developers can set empty password as follows:

    > sudo mysql -u root
    ALTER USER 'root'@'localhost' IDENTIFIED BY '';

In libvirt repository, there are two related commits

2019-08-23 13:13 Daniel P. Berrangé            ● rpm: don't enable socket activation in upgrade if --listen present
2019-08-22 14:52 Daniel P. Berrangé            ● remote: forbid the --listen arg when systemd socket activation

In libvirt.spec.in

        /bin/systemctl mask libvirtd.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-ro.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-admin.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-tls.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-tcp.socket >/dev/null 2>&1 || :

Co-authored-by: Wei Zhou <w.zhou@global.leaseweb.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-17 16:28:30 +05:30
Spaceman1984 b586eb22f1
Human readable sizes in logs (#4207)
This PR adds outputting human readable byte sizes in the management server logs, agent logs, and usage records. A non-dynamic global variable is added (display.human.readable.sizes) to control switching this feature on and off. This setting is sent to the agent on connection and is only read from the database when the management server is started up. The setting is kept in memory by the use of a static field on the NumbersUtil class and is available throughout the codebase.

Instead of seeing things like:
2020-07-23 15:31:58,593 DEBUG [c.c.a.t.Request] (AgentManager-Handler-12:null) (logid:) Seq 8-1863645820801253428: Processing: { Ans: , MgmtId: 52238089807, via: 8, Ver: v1, Flags: 10, [{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-224-VM","bytesSent":"106496","bytesReceived":"0","result":"true","details":"","wait":"0",}}] }

The KB MB and GB values will be printed out:

2020-07-23 15:31:58,593 DEBUG [c.c.a.t.Request] (AgentManager-Handler-12:null) (logid:) Seq 8-1863645820801253428: Processing: { Ans: , MgmtId: 52238089807, via: 8, Ver: v1, Flags: 10, [{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-224-VM","bytesSent":"(104.00 KB) 106496","bytesReceived":"(0 bytes) 0","result":"true","details":"","wait":"0",}}] }

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Human+Readable+Byte+sizes
2020-08-13 15:55:16 +05:30
Pearl Dsilva c578004fe5
projects: Role based users in Projects (#4128)
Enabling Role Based users in projects
Primate PR related to the FR: apache/cloudstack-primate#382
Doc PR: https://github.com/apache/cloudstack-documentation/pull/145

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2020-08-13 15:45:39 +05:30
Rohit Yadav db665fa8df Merge remote-tracking branch 'origin/4.14'
Conflicts:
	engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-05 14:46:13 +05:30
Rohit Yadav 2fce8b733e
vmware: volume utilisation is always zero (#4225)
This fixes issues of virtual size to be twice in case the disk is a
linked-clone root disk. The virtual size of root disk (first in chain)
must be used.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-05 14:01:16 +05:30
Wido den Hollander c3554ec31d
kvm: For ceph only if a port number has been specified define in the XML (#4231)
Ceph used to use port 6789 (no need to specify it), but with the messenger v2
from Ceph it switched to port 3300 while 6789 still works.

librados/librbd/libvirt will automatically figure out the ports to use if none is
specified.

Therefor there is no need for CloudStack to explicitely define the port in the XML
passed to Libvirt or Qemu.

Leave blank if no port number has been defined by the user.
2020-08-05 13:44:40 +05:30
sureshanaparti f0a67cca7a
vmware: Support to attach more than 15 data disks in VMware VM (#4172)
Support to attach more than 15 data disks in VMware VM

Fixes #4102
2020-07-15 15:51:55 +05:30
Pearl Dsilva a73712ec4e
server: Enable sending hypervior host name via metadata - VR and Config Drive (#3976)
Enable sending hypervisor host details via metadata for VR and Config Drive providers

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-07-01 08:44:11 +05:30
Nicolas Vazquez 8c1d749360
[VMware] Enable unmanaging guest VMs (#4103)
* Enable unmanaging guest VMs

* Minor fixes

* Fix stop usage event only if VM is not stopped when unmanaging

* Rename unmanaged VMs manager

* Generate netofferingremove usage event if VM is not stopped

* Generate usage event VM snapshot primary off when unmanaging
2020-06-26 08:31:43 -03:00
Rohit Yadav 029fc51960 Merge remote-tracking branch 'origin/4.14' 2020-06-23 12:28:42 +05:30
Rohit Yadav 5f23171ff3 Merge remote-tracking branch 'origin/4.13' into 4.14 2020-06-23 12:27:48 +05:30
Spaceman1984 97f21c1835
xenserver: Fixed null pointer and deployment issue on Xenserver with L2 Guest network with configDrive (#4004)
This PR fixes an issue where an instance fails to deploy due to a null pointer when using an L2 Guest Network with DefaultL2NetworkOfferingConfigDrive on Xenserver. It also fixes migrating an instance to another host.

This has been tested by:
- Creating an L2 Guest network, using DefaultL2NetworkOfferingConfigDrive as the network offering.
- Deploying an instance using the L2 Guest network created.
- Migrating the instance away from the host and back
2020-06-23 12:21:50 +05:30
Rohit Yadav a9175d39a5 Merge remote-tracking branch 'origin/4.14' 2020-06-12 10:39:17 +05:30
Rohit Yadav 4659cff597 Merge remote-tracking branch 'origin/4.13' into 4.14 2020-06-12 10:38:30 +05:30
slavkap 8b234bd14f
kvm: fix CloudRuntimeException in volume stats for migrated volume (#3949)
When you migrate volume between data stores CS keeps the original UUID and changes the path of the volume.
When volume is not found by the given path the agent throws CloudRuntimeException but it's not catched in LibvirtGetVolumeStatsCommandWrapper.java
2020-06-12 10:32:01 +05:30
Gabriel Beims Bräscher 6e47c49fbf
server: Fix String.format arguments (#3980)
Simple log fix. I have found out a few log messages that hold unused/misused arguments on their respective String.format building.
2020-06-12 10:28:04 +05:30
mbrashearnttglobalnet 9727548dbc
kvm: Add support for DPDK trunk interfaces (#4048)
Update to support trunk interfaces for DPDK enabled hosts
2020-06-12 10:17:36 +05:30
dahn 0795cd430b
Boot into hardware setup menu on Vmware (#4021) 2020-06-11 11:47:20 +00:00
Rohit Yadav 5a557f35a1
kvm: bump jna version to latest (#4119)
This fixes issue for cloudstack-agent to make connections to libvirt
on arm64 boards.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-06-08 06:58:53 +05:30