Commit Graph

2806 Commits

Author SHA1 Message Date
Daniel Augusto Veronezi Salvador 3674e80219
Improve logs on SecondaryStorageManagerImpl and few refactors (#4955)
Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
2021-05-10 09:47:42 +02:00
Rohit Yadav cb167072a1 Merge remote-tracking branch 'origin/4.15'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-05-07 16:37:42 +05:30
Harikrishna 32e3bbdcc5
VMware Datastore Cluster primary storage pool synchronisation (#4871)
Datastore cluster as a primary storage support is already there. But if any changes at vCenter to datastore cluster like addition/removal of datastore is not synchronised with CloudStack directly. It needs removal of primary storage from CloudStack and add it again to CloudStack.

Here synchronisation of datastore cluster is fixed without need to remove or add the datastore cluster.
1. A new API is introduced syncStoragePool which takes datastore cluster storage pool UUID as the parameter. This API checks if there any changes in the datastore cluster and updates management server accordingly.
2. During synchronisation if a new child datastore is found in datastore cluster, then management server will create a new child storage pool in database under the datastore cluster. If the new child storage pool is already added as an individual storage pool then the existing storage pool entry will be converted to child storage pool (instead of creating a new storage pool entry)
3. During synchronisaton if the existing child datastore in CloudStack is found to be removed on vCenter then management server removes that child datastore from datastore cluster and makes it an individual storage pool.
The above behaviour is on par with the vCenter behaviour when adding and removing child datastore.
2021-05-07 16:30:54 +05:30
Rohit Yadav fce84e62ab Merge remote-tracking branch 'origin/4.15' 2021-05-06 00:27:43 +05:30
Pearl Dsilva bc80815cf5
server: Adding VPN options for IKE version and IKE split connections (#4953)
IKE version allows selecting ike (autoselect), ikev1, or ikev2.
Split connections gives an option of separating the first right subnet from the rest, and kicking out individual statements for each right subnet for better cross-compatibility.

Backported from PR: #4137
update per PR suggestion

Fixes #3138

Co-authored-by: Greg Goodrich <ggoodrich@ippathways.com>
Co-authored-by: Daan Hoogland <dahn@onecht.net>
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-05-05 12:54:23 +05:30
Rohit Yadav 1abd10199c Merge remote-tracking branch 'origin/4.15' 2021-05-04 19:37:45 +05:30
Gabriel Beims Bräscher ab790c11d5
server: Allow to upgrade service offerings from local <> shared storage pools (#4915)
This PR addresses the issue raised at #4545 (Fail to change Service offering from local <> shared storage).

When upgrading a VM service offering it is validated if the new offering has the same storage scope (local or shared) as the current offering. I think that the validation makes sense in a way of preventing running Root disks with an offering that does not match the current storage pool. However, the validation only compares both offerings and does not consider that it is possible to migrate Volumes between local <> shared storage pools.

The idea behind this implementation is that CloudStack should check the scope of the current storage pool which the ROOT volume is allocated; this, it is possible to migrate the volume between storage pools and list/upgrade according to the offerings that are supported for such pool.

This PR also fixes an issue where the API command that lists offerings for a VM should follow the same idea and list based on the storage pool that the volume is allocated and not the previous offering.

Fixes: #4545
2021-04-30 11:59:50 +05:30
Olivier Lemasle 72f6612971
server: Increase max length for VMInstanceVO.backupVolumes (#4967)
The default length is 255, which caused a truncation of data if
the JSON object representing the backup volumes is too big.
It caused errors when backups were made on VMs with 3 volumes
or more.

`vm_instance.backup_volumes` has the type TEXT, which has a
maximal length of 65535 characters.

Fixes #4965
2021-04-30 11:57:56 +05:30
Rohit Yadav 4742ac15f7 Merge remote-tracking branch 'origin/4.15' 2021-04-29 21:50:40 +05:30
dahn be255e4203
server: protect against stray snapshot-details without snapshot (#4924)
This PR makes sure no orphaned snapshot details are considered in the cleanup at startup job.
a real solution would be to implement some kind of cascading delete, but as the parent record is "only" marked as removed this would be a bit com

Co-authored-by: Daan Hoogland <dahn@onecht.net>
2021-04-29 20:40:29 +05:30
Abhishek Kumar 42c83b08f5 Merge remote-tracking branch 'apache/4.15' 2021-04-26 14:33:58 +05:30
Nicolas Vazquez f728287aa2
server: Fix template garbage collection cleanup (#4944) 2021-04-24 18:57:47 +05:30
Abhishek Kumar a30d518e8a
vmware: fix stopped VM volume migration (#4758)
* prevent other vm disks getting deleted

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* vmware: fix inter-cluster stopped vm migration

Fixes #4838

For inter-cluster migration without shared storage, VMware needs a host to be specified. Fix is to specify an appropriate host in the target cluster.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix detached volume inter-cluster migration

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* cleanup unused method

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* review changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* vmware: allow attached volume migration using VmwareStorageMotionStrategy

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* find vm clusterid with multiple ROOT volumes

VM can have multiple ROOT volumes and some can be on zone-wide store therefore iterate over all of them till a cluster ID is found.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix successive storage migration

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix intercluster check

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor vm cluster, host method

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove inter-pod check

Added by mistake, VMware won't have pods

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* address review comment

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-04-24 18:55:25 +05:30
slavkap b4ee4acaf3
server: Fix volume state on migrate with migrateVirtualMachineWithVolume API call (#4934)
When invoking migrateVirtualMachineWithVolume API call and a strategy isn't found the volumes are left in Migrating state

This PR puts back the volumes to Ready state.
2021-04-22 14:30:18 +05:30
Gabriel Beims Bräscher de557663ec
Migrate/Stop VMs with local storage when preparing host for maintenance (#4212) 2021-04-19 09:41:42 +02:00
Rohit Yadav f42024714c Merge remote-tracking branch 'origin/4.15'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-04-15 16:17:13 +05:30
Rohit Yadav 0302750aac
vmware: Add support for VMware 7 (#4300) 2021-04-15 16:10:14 +05:30
Harikrishna f00b5fc7ac
server: Fix for the issue of recover VM not able to attach the data disks which are there before destroy in case of VMware (#4493)
This PR fixes: #4462

Problem Statement:
In case of VMware, when a VM having multiple data disk is destroyed (without expunge) and tried to recover the VM then the previous data disks are not attached to the VM like before destroy. Only root disk is attached to the VM.

Root cause:
All data disks were removed as part of VM destroy. Only the volumes which are selected to delete (while destroying VM) are supposed to be detached and destroyed.

Solution:
During VM destroy, detach and destroy only volumes which are selected during VM destroy. Detach the other volumes during expunge of VM.
2021-04-15 12:50:53 +05:30
Daniel Augusto Veronezi Salvador 911376e495
Improve logs on KVMHostActivityChecker (#4706)
Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
2021-04-13 08:27:25 +02:00
Abhishek Kumar cce736709e Merge remote-tracking branch 'apache/4.15' 2021-04-12 11:43:57 +05:30
Pearl Dsilva a64ad9d9b7
server: Prevent vm snapshots being indefinitely stuck in Expunging state on deletion failure (#4898)
Fixes #4201

This PR addresses the issue of a vm snapshot being indefinitely stuck is Expunging state in case deletion fails. 

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-04-12 08:09:37 +05:30
Abhishek Kumar fdefee75ff
vmware: fix inter-cluster stopped vm and volume migration (#4895)
Fixes #4838

For inter-cluster migration without shared storage, VMware needs a host to be specified. Fix is to specify an appropriate host in the target cluster during a stopped VM migration. Also, find target datastore using the host in the target cluster.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-04-10 13:22:28 +05:30
Rohit Yadav 22f6c19248 Merge remote-tracking branch 'origin/4.15' 2021-04-09 13:21:07 +05:30
Abhishek Kumar 6d0d7012f1
api,engine/schema,server: vm readonly details rename (#4671)
* api,engine/schema,server: vm readonly details rename

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: update api response key

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-04-09 13:20:11 +05:30
Rohit Yadav ca8920dd36 Merge remote-tracking branch 'origin/4.14' into 4.15 2021-04-09 13:17:39 +05:30
Abhishek Kumar d8c6e00498
hypervisor: XCP-ng 8.2 support (#4672)
Adds new/missing guest os mappings for XCP-ng/Xenserver 8.1
Copy guest OS mappings from XCP-ng/Xenserver 8.1 for XCP-ng/Xenserver 8.2
Adds Ubuntu 20.04 guest os mapping for XCP-ng/Xenserver 8.2

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-04-09 13:12:06 +05:30
Abhishek Kumar cd60b8d97d
host-allocator: check capacity for suitable hosts (#4884)
Fixes #4517

Adds capacity checks for RandomAllocator (host allocator)

Factors out host cpu capability and capacity check wrt serviceoffering code into CapacityManager.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-04-09 12:35:58 +05:30
Rohit Yadav 7270ca7e25 Merge remote-tracking branch 'origin/4.14' into 4.15 2021-04-06 12:51:26 +05:30
Gabriel Beims Bräscher cb91a769d3
Fix npe when migrating vm with volume (#4698) (#4775)
Cherry-pick commit 59fba4916b and fix conflict.

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
2021-04-06 11:54:29 +05:30
Pearl Dsilva 0dbeb262e4
server: Support for persistence mode in L2 networks (#4561)
This PR aims at introducing persistence mode in L2 networks and enhancing the behavior in Isolated networks
Doc PR apache/cloudstack-documentation#183

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-04-05 14:37:11 +05:30
Rohit Yadav d4635e3442 Merge remote-tracking branch 'origin/4.15'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-04-01 14:35:01 +05:30
Wei Zhou 63c91c1458
server: Fix network statistics for vpc (#3944)
This contains 3 main changes
(1) add NETWORK_STATS_ethX for all nics with public ips in VPC VRs (current: NETWORK_STATS_eth1)
(2) DO NOT create records in user_statistics for each VPC tier (only one record per public nic per VPC VR)
(3) send NetworkUsageCommand before unplugging a NIC with public IPs from VPC VR
2021-04-01 12:43:06 +05:30
Rakesh 76ba5c62d9
server: Fix displaying public IP address of shared networks (#4675)
Public IP addresses dedicated to one domain should not be accessed
by other domains. Also, root admin should be able to display all
public ip addresses in system.

Currently following issues exist

1. Public IP address assigned to one domain can be accessed by
other sibling domains

If use.system.public.ip is false then child domains should not
see public ip of ROOT domain

Before fix
```
(test1) mgt01 > list publicipaddresses listall=true fordisplay=true allocatedonly=false forvirtualnetwork=true filter=ipaddress,
{
  "count": 59,
  "publicipaddress": [
```

After fix

```
(test) mgt01 > list publicipaddresses listall=true fordisplay=true allocatedonly=false forvirtualnetwork=true filter=ipaddress,
{
  "count": 10,
```
2021-04-01 12:39:01 +05:30
Rohit Yadav c1a02e1697 Merge remote-tracking branch 'origin/4.15'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-03-29 16:34:22 +05:30
Wei Zhou b8884efa7f
server: create DB entry for storage pool capacity when create storage pool (#4805)
* server: create DB entry for storage pool capacity when create storage pool

* Revert "server: create DB entry for storage pool capacity when create storage pool"

This reverts commit e790167bfe.

* server: create DB entry for storage pool capacity when create zone-wide storage pools
2021-03-29 16:21:24 +05:30
Abhishek Kumar d763169b1c
Restore VMware VM naming convention option (#4581)
* initial chanes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* make check explicit for instance name flag

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* allow hiding vm details (in ui)

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* condition based on name instead of displayname

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2021-03-29 16:13:14 +05:30
Daan Hoogland 8903647187 Merge branch '4.15' 2021-03-25 17:48:45 +01:00
Rohit Yadav 9b1d1e6de3
systemvmtemplate: new template for 4.15.1 (#4793)
Update new systemvmtemplate for 4.15.1.0; synced:
http://download.cloudstack.org/systemvm/4.15/

A new template is necessary due to many security fixes over the last year, the 4.15.0 systemvmtemplate was created about a year ago.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-03-24 17:08:46 +05:30
Rohit Yadav 9f730eabfa Merge remote-tracking branch 'origin/4.15' 2021-03-24 12:46:24 +05:30
Pearl Dsilva 546bf3d5a2
server: Update vm_template table to set template as removed on deletion (#4748)
* Update vm_template table removed field when template is deleted

* Update method name

* address comment

* Extracted code to separate methods

* Address test failure

* refactor test cleanup

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-03-24 12:41:03 +05:30
Rohit Yadav 775de36688 Merge remote-tracking branch 'origin/4.15' 2021-03-17 17:46:16 +05:30
Pearl Dsilva 136252d65d
server: Maintain order or project owners added to account (#4822)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-03-17 17:36:46 +05:30
Rohit Yadav c8f7c0d823 Merge remote-tracking branch 'origin/4.15'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-03-11 13:33:44 +05:30
Michael 1cfb44994f
db: add schema upgrade from 4.15.0.0 to 4.15.1.0 (#4574) 2021-03-11 13:24:29 +05:30
Daniel Augusto Veronezi Salvador 59fba4916b
Fix npe when migrating vm with volume (#4698)
Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
2021-03-08 17:56:56 +01:00
sureshanaparti 81dfcbb5f5
server: Use ACPI event to reboot VM on KVM, and Use 'forced' reboot option to stop and start the VM(s) (#4681)
* Updated libvirt's native reboot operation for VM on KVM using ACPI event, and Added 'forced' reboot option to stop and start the VM (using rebootVirtualMachine API)

* Added 'forced' reboot option for System VM and Router

- New parameter 'forced' in rebootSystemVm API, to stop and then start System VM
- New parameter 'forced' in rebootRouter API, to force stop and then start Router

* Added force reboot tests for User VM, System VM and Router
2021-03-06 14:58:56 +05:30
sureshanaparti 45e77dd6f0
server: Clean up the duplicate volume when the destination managed volume creation failed on migrate volume operation (#4730)
Duplicated volumes after failed migration in Allocated state

Fix: Clean up the duplicate volume when the destination managed volume creation failed on migrate volume operation
2021-03-03 13:30:08 +05:30
Rohit Yadav fa067e02a7 Updating pom.xml version numbers for release 4.14.2.0-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-03-02 12:32:27 +05:30
Rohit Yadav 77290df0d5 Merge remote-tracking branch 'origin/4.15'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-02-26 12:09:11 +05:30
Abhishek Kumar 88337bdea4
server: fix finding pools for volume migration (#4693)
While finding pools for volume migration list following compatible storages:
- all zone-wide storages of the same hypervisor.
- when the volume is attached to a VM, then all storages from the same cluster as that of VM.
- for detached volume, all storages that belong to clusters of the same hypervisor. 

Fixes #4692 
Fixes #4400
2021-02-25 22:13:50 +05:30