Commit Graph

2920 Commits

Author SHA1 Message Date
Daan Hoogland 8903647187 Merge branch '4.15' 2021-03-25 17:48:45 +01:00
Rohit Yadav 9b1d1e6de3
systemvmtemplate: new template for 4.15.1 (#4793)
Update new systemvmtemplate for 4.15.1.0; synced:
http://download.cloudstack.org/systemvm/4.15/

A new template is necessary due to many security fixes over the last year, the 4.15.0 systemvmtemplate was created about a year ago.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-03-24 17:08:46 +05:30
Rohit Yadav 9f730eabfa Merge remote-tracking branch 'origin/4.15' 2021-03-24 12:46:24 +05:30
Pearl Dsilva 546bf3d5a2
server: Update vm_template table to set template as removed on deletion (#4748)
* Update vm_template table removed field when template is deleted

* Update method name

* address comment

* Extracted code to separate methods

* Address test failure

* refactor test cleanup

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-03-24 12:41:03 +05:30
Rohit Yadav 775de36688 Merge remote-tracking branch 'origin/4.15' 2021-03-17 17:46:16 +05:30
Pearl Dsilva 136252d65d
server: Maintain order or project owners added to account (#4822)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-03-17 17:36:46 +05:30
Rohit Yadav c8f7c0d823 Merge remote-tracking branch 'origin/4.15'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-03-11 13:33:44 +05:30
Michael 1cfb44994f
db: add schema upgrade from 4.15.0.0 to 4.15.1.0 (#4574) 2021-03-11 13:24:29 +05:30
Daniel Augusto Veronezi Salvador 59fba4916b
Fix npe when migrating vm with volume (#4698)
Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
2021-03-08 17:56:56 +01:00
sureshanaparti 81dfcbb5f5
server: Use ACPI event to reboot VM on KVM, and Use 'forced' reboot option to stop and start the VM(s) (#4681)
* Updated libvirt's native reboot operation for VM on KVM using ACPI event, and Added 'forced' reboot option to stop and start the VM (using rebootVirtualMachine API)

* Added 'forced' reboot option for System VM and Router

- New parameter 'forced' in rebootSystemVm API, to stop and then start System VM
- New parameter 'forced' in rebootRouter API, to force stop and then start Router

* Added force reboot tests for User VM, System VM and Router
2021-03-06 14:58:56 +05:30
sureshanaparti 45e77dd6f0
server: Clean up the duplicate volume when the destination managed volume creation failed on migrate volume operation (#4730)
Duplicated volumes after failed migration in Allocated state

Fix: Clean up the duplicate volume when the destination managed volume creation failed on migrate volume operation
2021-03-03 13:30:08 +05:30
Rohit Yadav fa067e02a7 Updating pom.xml version numbers for release 4.14.2.0-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-03-02 12:32:27 +05:30
Rohit Yadav 77290df0d5 Merge remote-tracking branch 'origin/4.15'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-02-26 12:09:11 +05:30
Abhishek Kumar 88337bdea4
server: fix finding pools for volume migration (#4693)
While finding pools for volume migration list following compatible storages:
- all zone-wide storages of the same hypervisor.
- when the volume is attached to a VM, then all storages from the same cluster as that of VM.
- for detached volume, all storages that belong to clusters of the same hypervisor. 

Fixes #4692 
Fixes #4400
2021-02-25 22:13:50 +05:30
Rakesh 787491871a
server: Look for active templates for VR deployment (#4047)
If the template from which VR is created got deleted, the state
is set to inactive and removed to null.
Since the template is already deleted, the VR can't be created
using this template again.

If someone restarts network with cleanup then it will try to
deploy the vr from the old non existing template again.
So search only for active template which are not yet deleted.
2021-02-25 22:05:31 +05:30
Rohit Yadav 1d8a4978e9 Merge remote-tracking branch 'origin/4.15'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-02-24 17:39:12 +05:30
Wei Zhou 5a3ae159ca
upgrade: check systemvm template before db changes (#4582)
* Upgrade: check systemvm template before db changes

* Upgrade: move some codes to a separated method

* #4582 add txn.commit()
2021-02-24 16:26:31 +05:30
sureshanaparti eba186aa40
storage: New Dell EMC PowerFlex Plugin (formerly ScaleIO, VxFlexOS) (#4304)
Added support for PowerFlex/ScaleIO (v3.5 onwards) storage pool as a primary storage in CloudStack (for KVM hypervisor) and enabled VM/Volume operations on that pool (using pool tag).
Please find more details in the FS here:
https://cwiki.apache.org/confluence/x/cDl4CQ

Documentation PR: apache/cloudstack-documentation#169

This enables support for PowerFlex/ScaleIO (v3.5 onwards) storage pool as a primary storage in CloudStack

Other improvements addressed in addition to PowerFlex/ScaleIO support:

- Added support for config drives in host cache for KVM
	=> Changed configuration "vm.configdrive.primarypool.enabled" scope from Global to Zone level
	=> Introduced new zone level configuration "vm.configdrive.force.host.cache.use" (default: false) to force host cache for config drives
	=> Introduced new zone level configuration "vm.configdrive.use.host.cache.on.unsupported.pool" (default: true) to use host cache for config drives when storage pool doesn't support config drive
	=> Added new parameter "host.cache.location" (default: /var/cache/cloud) in KVM agent.properties for specifying the host cache path and create config drives on the "/config" directory on the host cache path
	=> Maintain the config drive location and use it when required on any config drive operation (migrate, delete)

- Detect virtual size from the template URL while registering direct download qcow2 (of KVM hypervisor) templates

- Updated full deployment destination for preparing the network(s) on VM start

- Propagate the direct download certificates uploaded to the newly added KVM hosts

- Discover the template size for direct download templates using any available host from the zones specified on template registration
	=> When zones are not specified while registering template, template size discovery is performed using any available host, which is picked up randomly from one of the available zones

- Release the VM resources when VM is sync-ed to Stopped state on PowerReportMissing (after graceful period)

- Retry VM deployment/start when the host cannot grant access to volume/template

- Mark never-used or downloaded templates as Destroyed on deletion, without sending any DeleteCommand
	=> Do not trigger any DeleteCommand for never-used or downloaded templates as these doesn't exist and cannot be deleted from the datastore

- Check the router filesystem is writable or not, before performing health checks
	=> Introduce a new test "filesystem.writable.test" to check the filesystem is writable or not
	=> The router health checks keeps the config info at "/var/cache/cloud" and updates the monitor results at "/root" for health checks, both are different partitions. So, test at both the locations.
	=> Added new script: "filesystem_writable_check.py" at /opt/cloud/bin/ to check the filesystem is writable or not

- Fixed NPE issue, template is null for DATA disks. Copy template to target storage for ROOT disk (with template id), skip DATA disk(s)

* Addressed some issues for few operations on PowerFlex storage pool.

- Updated migration volume operation to sync the status and wait for migration to complete.

- Updated VM Snapshot naming, for uniqueness in ScaleIO volume name when more than one volume exists in the VM.

- Added sync lock while spooling managed storage template before volume creation from the template (non-direct download).

- Updated resize volume error message string.

- Blocked the below operations on PowerFlex storage pool:
  -> Extract Volume
  -> Create Snapshot for VMSnapshot

* Added the PowerFlex/ScaleIO client connection pool to manage the ScaleIO gateway clients, which uses a single gateway client per Powerflex/ScaleIO storage pool and renews it when the session token expires.

- The token is valid for 8 hours from the time it was created, unless there has been no activity for 10 minutes.
  Reference: https://cpsdocs.dellemc.com/bundle/PF_REST_API_RG/page/GUID-92430F19-9F44-42B6-B898-87D5307AE59B.html

Other fixes included:

- Fail the VM deployment when the host specified in the deployVirtualMachine cmd is not in the right state (i.e. either Resource State is not Enabled or Status is not Up)

- Use the physical file size of the template to check the free space availability on the host, while downloading the direct download templates.

- Perform basic tests (for connectivity and file system) on router before updating the health check config data
	=> Validate the basic tests (connectivity and file system check) on router
	=> Cleanup the health check results when router is destroyed

* Updated PowerFlex/ScaleIO storage plugin version to 4.16.0.0

* UI Changes to support storage plugin for PowerFlex/ScaleIO storage pool.
- PowerFlex pool URL generated from the UI inputs(Gateway, Username, Password, Storage Pool) when adding "PowerFlex" Primary Storage
- Updated protocol to "custom" for PowerFlex provider
- Allow VM Snapshot for stopped VM on KVM hypervisor and PowerFlex/ScaleIO storage pool

and Minor improvements in PowerFlex/ScaleIO storage plugin code

* Added support for PowerFlex/ScaleIO volume migration across different PowerFlex storage instances.

- findStoragePoolsForMigration API returns PowerFlex pool(s) of different instance as suitable pool(s), for volume(s) on PowerFlex storage pool.
- Volume(s) with snapshots are not allowed to migrate to different PowerFlex instance.
- Volume(s) of running VM are not allowed to migrate to other PowerFlex storage pools.
- Volume migration from PowerFlex pool to Non-PowerFlex pool, and vice versa are not supported.

* Fixed change service offering smoke tests in test_service_offerings.py, test_vm_snapshots.py

* Added the PowerFlex/ScaleIO volume/snapshot name to the paths of respective CloudStack resources (Templates, Volumes, Snapshots and VM Snapshots)

* Added new response parameter “supportsStorageSnapshot” (true/false) to volume response, and Updated UI to hide the async backup option while taking snapshot for volume(s) with storage snapshot support.

* Fix to remove the duplicate zone wide pools listed while finding storage pools for migration

* Updated PowerFlex/ScaleIO volume migration checks and rollback migration on failure

* Fixed the PowerFlex/ScaleIO volume name inconsistency issue in the volume path after migration, due to rename failure
2021-02-24 14:58:33 +05:30
Pearl Dsilva aa01580381
network: Specify IP for VR in shared networks (#4503)
This PR enables admins to specify IP for a VR in a shared network.
2021-02-18 13:54:09 +05:30
Pearl Dsilva b6fe9f99eb
Network Offering: Allow enabling network and vpc offering during creation (#4564)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-02-13 10:19:06 +00:00
Sina Kashipazha 543f9827ff
add creation date as a value for domains and accounts. (#4649) 2021-02-13 10:07:53 +00:00
Abhishek Kumar d6e8b53736
vmware: vm migration improvements (#4385)
- Fixes inter-cluster migration of VMs
- Allows migration of stopped VM with disks attached to different and suitable pools
- Improves inter-cluster detached volume migration
- Allows inter-cluster migration (clusters of same Pod) for system VMs, VRs on VMware
- Allows storage migration for stopped system VMs, VRs on VMware within same Pod if StoragePool cluster scopetype

Linked Primate PR: https://github.com/apache/cloudstack-primate/pull/789 [Changes merged in this PR after new UI merge]
Documentation PR: https://github.com/apache/cloudstack-documentation/pull/170

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-02-12 12:41:41 +05:30
Rohit Yadav 66f0beda5f Updating pom.xml version numbers for release 4.14.1.0
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-02-08 16:24:09 +05:30
Rohit Yadav ba127dab3e Merge remote-tracking branch 'origin/4.15'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-02-05 16:02:26 +05:30
Rohit Yadav 6bde1384ff Merge remote-tracking branch 'origin/4.14' into 4.15
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-02-05 16:01:01 +05:30
Wei Zhou 78f73c1bc6
server: Fix update capacity for hosts take long time if there are many service offerings (#4623)
Steps to reproduce the issue:

(1)Create 10000 service offerings (by db changes below or cloudmonkey).

```
DROP PROCEDURE IF EXISTS cloud.insert_service_offering;

DELIMITER $$
CREATE PROCEDURE cloud.insert_service_offering()
BEGIN
  DECLARE count INT DEFAULT 10000;
  SET @offeringid = (select max(id)+1 from disk_offering);

  WHILE count > 0 DO
    INSERT INTO disk_offering (id,name,uuid,display_text,disk_size,type,created) values (@offeringid,'test-offering-wei',uuid(), 'test-offering-wei',0,'Service',now());
    INSERT INTO service_offering (id,cpu,speed,ram_size) values (@offeringid, 1, 500,256);
    SET @offeringid = @offeringid + 1;
    SET count = count - 1;
  END WHILE;
END $$
DELIMITER ;

CALL cloud.insert_service_offering();

mysql> CALL cloud.insert_service_offering();
Query OK, 0 rows affected (2 min 30.85 sec)
```

(2) Check the total time of periodical capacity check in cloudstack.

Without this patch, it spend 2.5 seconds (2 hosts)
```
2021-01-15 16:10:12,793 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-5d5f3b3b) (logid:f5eb68ba) Running Capacity Checker ...
2021-01-15 16:10:15,287 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-5d5f3b3b) (logid:f5eb68ba) Done running Capacity Checker ...
```

With this patch ,it spend 1.3 seconds (2 hosts)
```
2021-01-15 16:12:43,604 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-a2a7f3f1) (logid:f7e0a4c5) Running Capacity Checker ...
2021-01-15 16:12:44,927 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-a2a7f3f1) (logid:f7e0a4c5) Done running Capacity Checker ...
```

If there are 100 hosts, the total time will be reduced from 100+ seconds to around 10 seconds.
2021-02-04 14:43:57 +05:30
Daan Hoogland ff376d8187 Merge release branch 4.15 to master
* 4.15:
  server: select root disk based on user input during vm import (#4591)
  kvm: Use Q35 chipset for UEFI x86_64 (#4576)
  server: fix wrong error message when create isolated network without SourceNat (#4624)
  server: add possibility to scale vm to current customer offerings (#4622)
  server: keep networks order and ips while move a vm with multiple networks (#4602)
  server: throw exception when update vm nic on L2 network (#4625)
  doc: fix typo in install notes (#4633)
2021-02-01 09:58:52 +00:00
Daan Hoogland b6b778f003 Merge release branch 4.14 to 4.15
* 4.14:
  server: select root disk based on user input during vm import (#4591)
  kvm: Use Q35 chipset for UEFI x86_64 (#4576)
  server: fix wrong error message when create isolated network without SourceNat (#4624)
  server: add possibility to scale vm to current customer offerings (#4622)
  server: keep networks order and ips while move a vm with multiple networks (#4602)
  server: throw exception when update vm nic on L2 network (#4625)
  doc: fix typo in install notes (#4633)
2021-02-01 09:57:35 +00:00
Wei Zhou 313ae1f449
server: fix wrong error message when create isolated network without SourceNat (#4624)
This PR fixes wrong message when create isolated network without SourceNat.
2021-02-01 14:15:47 +05:30
Wei Zhou 1913c6854e
server: keep networks order and ips while move a vm with multiple networks (#4602)
This PR fixes an issue when move a vm from an account to another account.

Steps to reproduce the issue
(1) create a vm with multiple shared networks (in advanced zone, or advanced zone with security groups)
(2) create another account (in same domain who can also access the shared networks)
(3) move vm to new account, with a list of networkid

expected result: the vm has nics on the networks in same order as specified in API request, and nics have the same ips as before actual result: network order is not same as specified, ips are changed.
2021-02-01 14:14:20 +05:30
Rohit Yadav e6fef05f30 Merge remote-tracking branch 'origin/4.15' 2021-01-28 14:25:40 +05:30
Rohit Yadav 74bae56642 Merge remote-tracking branch 'origin/4.14' into 4.15
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-01-28 14:24:25 +05:30
Wei Zhou 182cea79b5
server: fix cannot create vm if another vm with same name has been added and removed on the network (#4600)
* server: fix cannot create vm if another vm with same name has been added and removed on the network

steps to reproduce the issue
(1) create vm-1 on network-1
(2) add vm-1 to network-2
(3) remove vm-1 from network-2
(4) create another vm with same name vm-1 on network-2

expected result: operation succeed
actual result: operation failed.

* #4600: add back a removed line
2021-01-27 19:28:52 +05:30
Rohit Yadav 6c346df5a3 Merge remote-tracking branch 'origin/4.15'
Fix db upgrade path conflict, add 4.15.1.0->4.16.0.0 for master, bump
systemvmtemplate version to 4.16.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-01-20 19:33:59 +05:30
Rohit Yadav 8a9ae6fcee Merge remote-tracking branch 'origin/4.14' into 4.15
Fix upgrade path conflicts, add 4.15.0.0->4.15.1.0 stub

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-01-20 18:44:48 +05:30
davidjumani f70da104a0
engine/schema: add upgrade path from 4.14.0.0 to 4.14.1.0 (#4580) 2021-01-12 18:21:30 +05:30
Rohit Yadav b482da8c91 Updating pom.xml version numbers for release 4.15.1.0-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-01-11 13:58:30 +05:30
Daan Hoogland 280c13a4bb Updating pom.xml version numbers for release 4.15.0.0
Signed-off-by: Daan Hoogland <dahn@onecht.net>
2021-01-05 15:51:02 +00:00
Rohit Yadav 015bb20caa Merge remote-tracking branch 'origin/4.15' 2021-01-05 15:25:39 +05:30
davidjumani d4580f3e50
networkorchestrator: Fix typo in exception message (#4559) 2021-01-05 12:54:26 +05:30
Daan Hoogland 81e9e6809b Updating pom.xml version numbers for release 4.15.1.0-SNAPSHOT
Signed-off-by: Daan Hoogland <dahn@onecht.net>
2021-01-04 11:34:46 +00:00
Daan Hoogland e26202f23e Updating pom.xml version numbers for release 4.16.0.0-SNAPSHOT
Signed-off-by: Daan Hoogland <dahn@onecht.net>
2021-01-04 11:32:10 +00:00
Daan Hoogland 01b3e361c7 Updating pom.xml version numbers for release 4.15.0.0
Signed-off-by: Daan Hoogland <dahn@onecht.net>
2020-12-23 16:32:25 +00:00
Harikrishna b1ddd7c2e6
vmware: Fix for mapping guest OS type read from OVF to existing guest OS in C… (#4553)
* Fix for mapping guest OS type read from OVF to existing guest OS in CloudStack database  while registering VMware template

* Added unit tests to String Utils methods and updated the code

* Updated the java doc section

* Updated os description logic to keep equals ignore match with guest os display name
2020-12-23 19:37:21 +05:30
Nicolas Vazquez 4617be4583
vmware: Fix template upload from local (#4555)
Update the guest OS from the OVF file after upload is completed
This PR fixes the template upload from local on VMware

Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2020-12-23 15:13:39 +05:30
Rohit Yadav fdf9573266 Merge remote-tracking branch 'origin/4.14' 2020-12-14 12:05:52 +05:30
Alexandru Bagu fdb2ee3165
storage: Fix hypervisor type cast to string (#4516)
This PR addresses an error that appears when you try to add a new host. I don't even understand why there was a cast to String in the first place. I will assume some classes send HypervisorType and some send a string (empty or otherwise). Shouldn't this be addressed to use the same type everywhere? With this fix adding a new xenserver host works fine.

Co-authored-by: dahn <daan.hoogland@gmail.com>
2020-12-14 11:56:44 +05:30
Wei Zhou 96cfe27504
db upgrade: fix sql exception: Access denied; you need (at least one of) the SUPER privilege(s) for this operation (#4533) 2020-12-14 11:40:33 +05:30
Wei Zhou 9de679ea1c
db: Fix description of volume.stats.interval which is in milliseconds not seconds (#4526) 2020-12-10 14:32:16 +05:30
Pearl Dsilva e4a504b084
Make global setting non-dynamic (#4505)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-12-01 14:00:35 +05:30
Wei Zhou 4fb69e695c
sql: Fix Zones are returned in a random order (#3934) (#4494) 2020-11-25 15:30:43 +05:30
Spaceman1984 dfa09fc856
server: Setting snapshot removed on timeout (#4425)
* Setting snapshot state to error on timeout

* Setting removed field so snapshot record is ignored by garbage collection

* Removed explicitly setting error status, renamed method from markFailed to markRemoved

* Renamed method, moved code a few lines down

* Moved remove logic

* Removed unused service

* Moved removed logic - last time, promise
2020-11-21 02:20:16 +05:30
Rakesh 735b6de296
Cleanup download urls when SSVM destroyed (#4078)
Co-authored-by: Rakesh Venkatesh <r.venkatesh@global.leaseweb.com>
2020-11-18 14:01:31 +01:00
Spaceman1984 acee15a530
Moved dedicated hosts to the end of the resultset when selecting an e… (#4428) 2020-11-18 12:07:14 +00:00
Pearl Dsilva 1dbb76f64b
Fix: Data migration (#4475)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-11-18 09:45:53 +01:00
Daan Hoogland 79e46f50ca Merge branch '4.14' 2020-11-13 13:25:07 +00:00
Daan Hoogland db3c12d6cf Merge branch '4.13' into 4.14 2020-11-13 13:24:39 +00:00
slavkap cc8bee7223
Fix IndexOutOfBoundsException when creating basic network (#4464)
For Basic network isolation methods are not provided, and exception is
thrown when trying to encode the Vlan id. That's why we have to check
before encoding that the list with isolation methods is not empty
2020-11-13 08:07:40 +00:00
Olivier Lemasle 5f8289ffe9
Re-enable IP address usage hiding (#4327) 2020-11-07 10:42:44 +01:00
Nicolas Vazquez 63fbbe7506
[Bug fix] VMware: Fix for SSVM recreation on deployasis systemVM templates (#4437) 2020-11-07 09:46:54 +01:00
Gabriel Beims Bräscher b3a1cb41c8
Allow to configure root disk size via Service Offering (diskoffering of type Service). (#4341) 2020-10-30 15:56:11 +00:00
Wei Zhou e447764d66
DB: fix wrong category id of guest os 'Other PV Virtio-SCSI (64-bit)' (#4426) 2020-10-29 09:08:02 +01:00
Pearl Dsilva 25dd7aad51
Modify alter view to drop/create view (#4417)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-10-28 09:53:56 +01:00
nvazquez d119a5d19f Merge branch 'master' into ovfprops-and-vsphere-adv-together 2020-10-24 13:02:31 -03:00
Rakesh 71c5dbcf49
server: Update use_bytes of storage pools (#4360)
Update the used_bytes for all default primary storage pools
Also get used_bytes of storage pool from database instead of
memory
2020-10-21 19:18:03 +02:00
Andrija Panic 5797cf5e17
guest-os-GC (#67) 2020-10-20 10:40:03 -03:00
nvazquez 50ab1b2fbe Add missing guest OS entries for VMware 2020-10-19 15:05:58 +05:30
nvazquez ee5b8763a6 Fix remove VM and its volumes for deploy-as-is if have previously failed - restore cpu flags in nested virtualization test 2020-10-19 15:05:58 +05:30
nvazquez 3600b3c7bc Reconcile root disk controller fix 2020-10-19 15:05:58 +05:30
Harikrishna Patnala 5fdabc1cb0 Added storage policy details to disk while creating disk and restricted migration of volumes to storage pools which are not storage policy compliance 2020-10-19 15:05:58 +05:30
Harikrishna Patnala 46b5322d9b Adding vSphere storage policy to disk on start command and attach volume command 2020-10-19 15:05:58 +05:30
Harikrishna Patnala 1e4e5cf4c0 Disk offering response: added vsphere storage policy to response if exists 2020-10-19 15:05:58 +05:30
Harikrishna Patnala a54436b790 service offering response: added vsphere storage policy to response if exists 2020-10-19 15:05:58 +05:30
Harikrishna Patnala 07abcf5705 During migrate volume command, when operation timed out exception or any exception is occured it is not handled properly to clean the volume_store_ref entry.
Fixed it to clean the volume_store_ref entry upon on any exception
2020-10-19 15:05:57 +05:30
Harikrishna Patnala 9b923ba9dc Handle detach volume of datastore cluster if the volume name has changed at vCenter level and reconsile the chaininfo 2020-10-19 15:05:57 +05:30
nvazquez 94bebe8792 Revert back deploy as is column on templates but keep it as default for new templates 2020-10-19 15:05:57 +05:30
nvazquez 46d412d998 Fix for system VMs 2020-10-19 15:05:57 +05:30
nvazquez 9b51a706db Set deploy-as-is to default on VMware 2020-10-19 15:05:57 +05:30
Harikrishna Patnala 9543fd6e6a Fix startcommand on Datastore cluster when the volume datastore in CloudStack mismatches with vCenter datastore. Volume could have migrated with in datastore cluster which caused the mismatch
Fix dettach volume when volume is not on CloudStack intended datastore
2020-10-19 15:05:57 +05:30
nvazquez b0d3168e0b Fail template registration when guest OS not found 2020-10-19 15:05:57 +05:30
nvazquez 32d85b0fa2 Display storage on logging when not deploy-as-is and guest OS small refactor 2020-10-19 15:05:57 +05:30
nvazquez 41354227e2 Handle guest OS read from deploy-as-is OVF descriptor 2020-10-19 15:05:57 +05:30
nvazquez edfbed34ad Use network adapter from OVF on deploy-as-is 2020-10-19 15:05:57 +05:30
nvazquez 0f5a6ee589 Remove deploy as-is details when removing templates or vms 2020-10-19 15:05:57 +05:30
nvazquez a43d0a6085 Fix VM power sync NPE on system VMs 2020-10-19 15:05:57 +05:30
Harikrishna Patnala 745dd00c9a Remove duplicate schema changes 2020-10-19 15:05:57 +05:30
Harikrishna Patnala 33ae2afc89 Removed few duplicate imports during rebase with master 2020-10-19 15:05:57 +05:30
Harikrishna Patnala 44dc0c6072 Fixed rat failure on new class DeployAsIsHelper.java
Also removed some unused imports during rebase
2020-10-19 15:05:57 +05:30
nvazquez 818981d8a7 Fix typo 2020-10-19 15:05:57 +05:30
nvazquez 44bae98664 Fix daos issue 2020-10-19 15:05:57 +05:30
nvazquez f73830acbb Refactor deploy as is constants 2020-10-19 15:05:57 +05:30
nvazquez bb4ce2118d Add new template and vm deploy as is details table and refactor 2020-10-19 15:05:57 +05:30
nvazquez d864e9dc39 [VMware] Full OVF properties support 2020-10-19 15:05:56 +05:30
Harikrishna Patnala 19745ea049 Fix enable primary datastore maintenance command seriliaztion on it 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 38b9618e4b Fix destination hostname for stopped VMs in case of reset ssh keys 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 201ebe8868 Simulator failures fixing 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 295ea50282 Fixed unexpected token Exception 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 48dd0dae07 Adding Storagepolicies to the service offering for root disks 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 61dd85876b Fix migrate vm and volume APIs in case if datastore cluster 2020-10-19 14:57:16 +05:30
Pearl Dsilva 4178a7cd90 inter-cluster migration support 2020-10-19 14:57:16 +05:30
Harikrishna Patnala fef3926469 Datastore cluster: update tag to the child storagepools also 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 873f9dd9ac Datastore Clusters operations on putting into maintenance mode, update storage pool with tags, cancelling mantenance mode and deleting storage pool 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 75fb1d91ee Fix adding Datastore clusters and listing 2020-10-19 14:57:15 +05:30
Harikrishna Patnala c9c38c0f37 Handle datastore clusters in list storage pools to list only parent not the children in the Datastore cluster 2020-10-19 14:57:15 +05:30
Harikrishna Patnala b4a23ea5f6 Allocation logic to skip datastore cluster and consider only storagepools inside the datastore cluster 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 41b3fc19d6 Add Datastore cluster and the child entities which are datastores in the cluster into CloudStack
Setting scope is still pending.
2020-10-19 14:57:15 +05:30
Harikrishna Patnala 48786b2d31 DataStore Clusters addition as a storage pool 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 586fa7eeaf Added listVsphereStoragePoliciesCmd 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 6df819028e UI changes and accept any type of datastore as presetup in vmware 2020-10-19 14:57:15 +05:30
Harikrishna Patnala fb0a96e7fb Check if datastore is complaince with the storagepolicy provided in the disk offering.
Added corresponding manager objects from PBM sdk to do the job.
Made dao layer changes to read the storage policy in diskoffering
2020-10-19 14:57:15 +05:30
Harikrishna Patnala 27e5dfa633 Updated code to handle sync of storage policies when importVsphereStoragePolicies API is called multiple times 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 8b23ea90b8 Moved Dao and VO files to different packages 2020-10-19 14:57:15 +05:30
Harikrishna Patnala b8b4f0f9ad Added importVsphereStoragePolicies API and scheme changes 2020-10-19 14:57:15 +05:30
Pearl Dsilva 0d487fc8c9
support for data migration of incremental snaps on xen (#4395)
* support for handling incremental snaps (on DB entries) on xen

* Addressed comments

* Update NfsSecondaryStorageResource.java

adjusted space in comment/ log

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-10-18 02:15:10 +05:30
Rodrigo D. Lopez c222d0bf60
enable update tags on disk offerings (#4194) 2020-10-16 09:22:42 +02:00
Wei Zhou 2acd87c41e
server: Add global configuration vm.serviceoffering.cpu.cores.max and vm.serviceoffering.ram.size.max (#4379)
vm.serviceoffering.cpu.cores.max and vm.serviceoffering.ram.size.max
2020-10-14 15:48:35 +05:30
davidjumani aab8df09aa
Fixing searchAndCount searchAndDistinctCount when sc is null (#4374) 2020-10-13 11:30:04 +02:00
Rohit Yadav b3bafffff3 Merge remote-tracking branch 'origin/4.14' 2020-09-29 14:33:58 +05:30
Abhishek Kumar 9391fa9b6b
cks: assorted fixes, test refactoring (#4269)
Fixes #4265
Fixes for:

template selection
network cleanup
capacity check
specific patch version during deployment

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-09-29 14:31:41 +05:30
Rohit Yadav 86fcb14238 Merge remote-tracking branch 'origin/4.14'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-09-24 12:20:29 +05:30
Rohit Yadav 1efe6e2df0 Merge remote-tracking branch 'origin/4.13' into 4.14
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-09-24 12:13:23 +05:30
Gabriel Beims Bräscher d6152b37ad
server: Broadcast URI not set to vxlan, but vlan (Fix #3040) (#4190)
This PR sets properly Broadcast URI to vxlan://vxlan_id when the physical network is of VXLAN.

Fixes: #3040
2020-09-24 11:09:50 +05:30
Wei Zhou 534dd475d8
agent: Compare indirect agent lb algorithm when cloudstack agent connects (#4335)
Compare not only the list of management servers but also the lb algorithm when agent connects.

Fixes: #3895
2020-09-23 12:48:16 +05:30
Pearl Dsilva cfbb4ff3dd
schema: change upgrade path to 4.14 (from 4.13) and intensify check (#4331)
* change upgrade path to 4.14 (from 4.13) and intensify check

* extracted check

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-09-22 09:40:51 +05:30
Pearl Dsilva b464fe41c6
server: Secondary Storage Usage Improvements (#4053)
This feature enables the following:
Balanced migration of data objects from source Image store to destination Image store(s)
Complete migration of data
setting an image store to read-only
viewing download progress of templates across all data stores
Related Primate PR: apache/cloudstack-primate#326
2020-09-17 10:12:10 +05:30
Pearl Dsilva 37c7a2b851
Incorrect md5sums for systemVM templates results in failure to download templates to other image stores (#4297)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-09-09 15:57:49 +05:30
Spaceman1984 cb717741fc
server: Fixed delayed power state update after vm shutdown (#4284)
After a vm is shutdown, the power state isn't updated immediately. This prevents changing the service offering.
This PR updates the power state immediately after the vm is confirmed to be shutdown.

Fixes: #3159
2020-09-01 15:53:52 +05:30
Wei Zhou 4746c8c726
server: move UpdateDefaultNic to vm work job queue (#4020)
While remove secondary nic from a Running vm, if update the default nic to the secondary nic before the nic is removed, the vm will not have default nic (and cannot be started) when both operations are completed.

It is because UpdateDefaultNic api is not handled as a vm work job (AddNicToVMCmd and RemoveNicFromVMCmd are), it is processed before nic is removed. The result is that secondary nic becomes default nic and got removed.
2020-09-01 13:54:48 +05:30
Rohit Yadav 6d6e4025f5 Merge remote-tracking branch 'origin/4.14' 2020-08-25 17:02:19 +05:30
Rohit Yadav 9c7c22eb84
engine: honour bypass VLAN id/range for L2 networks (#4274)
* engine: honour bypass VLAN id/range for L2 networks

Commit e894238d904a9c49c1140371f612a51d251efc1 (#3899) allowed private
gateways to bypass vlan check while refactoring it did not cover the
case for L2 but only shared network. This fix will re-enable honouring
the bypass vlan check option for L2 guest network (in addition to the
Shared networks).

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* Update NetworkOrchestrator.java
2020-08-25 15:19:02 +05:30
davidjumani 3872bf1ff9
kvm: Enable PVLAN support on L2 networks (#4040)
This is an extention of #3732 for kvm.
This is restricted to ovs > 2.9.2
Since Xen uses ovs 2.6, pvlan is unsupported.
This also fixes issues of vms on the same pvlan unable to communicate if they're on the same host
2020-08-20 15:46:34 +05:30
Paul Angus 400641b1cf Revert "Revert "move power sync messages to debug""
This reverts commit e06cf1a642.
2020-08-20 10:51:05 +01:00
Paul Angus e06cf1a642 Revert "move power sync messages to debug"
This reverts commit 1954ff32b7.
2020-08-20 10:50:58 +01:00
Paul Angus 1954ff32b7 move power sync messages to debug 2020-08-20 10:50:35 +01:00
Spaceman1984 d57aa83517
server: Added nfs minor version support (#4180)
This PR adds minor version support when mounting nfs on the SSVM as requested in #2861

The global setting "secstorage.nfs.version" has been changed to use the String data type which allows any minor version to be specified.
2020-08-19 14:53:38 +05:30
davidjumani d949302d0f
packaging: Adding Centos8, Ubuntu 20.04, XCPNG8.1 Support (#4068)
* DB : Add support for MySQL 8

- Splits commands to create user and grant access on database, the old
statement is no longer supported by MySQL 8.x
- `NO_AUTO_CREATE_USER` is no longer supported by MySQL 8.x so remove
that from db.properties conn parameters

For mysql-server 8.x setup the following changes were added/tested to
make it work with CloudStack in /etc/mysql/mysql.conf.d/mysqld.cnf and
then restart the mysql-server process:

    server_id = 1
    sql-mode="STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION,ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE,NO_ENGINE_SUBSTITUTION"
    innodb_rollback_on_timeout=1
    innodb_lock_wait_timeout=600
    max_connections=1000
    log-bin=mysql-bin
    binlog-format = 'ROW'

    default-authentication-plugin=mysql_native_password

Notice the last line above, this is to reset the old password based
authentication used by MySQL 5.x.

Developers can set empty password as follows:

    > sudo mysql -u root
    ALTER USER 'root'@'localhost' IDENTIFIED BY '';

In libvirt repository, there are two related commits

2019-08-23 13:13 Daniel P. Berrangé            ● rpm: don't enable socket activation in upgrade if --listen present
2019-08-22 14:52 Daniel P. Berrangé            ● remote: forbid the --listen arg when systemd socket activation

In libvirt.spec.in

        /bin/systemctl mask libvirtd.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-ro.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-admin.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-tls.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-tcp.socket >/dev/null 2>&1 || :

Co-authored-by: Wei Zhou <w.zhou@global.leaseweb.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-17 16:28:30 +05:30
Spaceman1984 b586eb22f1
Human readable sizes in logs (#4207)
This PR adds outputting human readable byte sizes in the management server logs, agent logs, and usage records. A non-dynamic global variable is added (display.human.readable.sizes) to control switching this feature on and off. This setting is sent to the agent on connection and is only read from the database when the management server is started up. The setting is kept in memory by the use of a static field on the NumbersUtil class and is available throughout the codebase.

Instead of seeing things like:
2020-07-23 15:31:58,593 DEBUG [c.c.a.t.Request] (AgentManager-Handler-12:null) (logid:) Seq 8-1863645820801253428: Processing: { Ans: , MgmtId: 52238089807, via: 8, Ver: v1, Flags: 10, [{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-224-VM","bytesSent":"106496","bytesReceived":"0","result":"true","details":"","wait":"0",}}] }

The KB MB and GB values will be printed out:

2020-07-23 15:31:58,593 DEBUG [c.c.a.t.Request] (AgentManager-Handler-12:null) (logid:) Seq 8-1863645820801253428: Processing: { Ans: , MgmtId: 52238089807, via: 8, Ver: v1, Flags: 10, [{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-224-VM","bytesSent":"(104.00 KB) 106496","bytesReceived":"(0 bytes) 0","result":"true","details":"","wait":"0",}}] }

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Human+Readable+Byte+sizes
2020-08-13 15:55:16 +05:30
Pearl Dsilva c578004fe5
projects: Role based users in Projects (#4128)
Enabling Role Based users in projects
Primate PR related to the FR: apache/cloudstack-primate#382
Doc PR: https://github.com/apache/cloudstack-documentation/pull/145

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2020-08-13 15:45:39 +05:30
Rohit Yadav 7bbb4d4bef Merge remote-tracking branch 'origin/4.14' 2020-08-12 13:46:13 +05:30
Rohit Yadav 6b6cc71be9 Merge remote-tracking branch 'origin/4.13' into 4.14 2020-08-12 13:44:41 +05:30
Spaceman1984 86939e7f9d
server: Fixed private gateway can't be deleted (#4016)
When the static route service is not available on the VPC and a static route is created, the static route is created in a revoked state.

Currently, the UI doesn't distinguish between active or revoked static routes.

This PR adds the missing state filter to the list routes command and only lists active routes in the UI.
It also ignores revoked routes when the private gateway is being removed but clears out the inactive routes before the gateway is removed.

Fixes #2908
2020-08-12 13:29:12 +05:30
Rohit Yadav 7707d3fd92 engine: fix NPE from fwd-merge 562a7db8df
This fixes NPE caused due to merge conflict fix from the forward merge
commit 562a7db8df and fixes travis test
regression:

    === TestName: test_01_reset_vm_on_reboot | Status : SUCCESS ===

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-07 15:31:43 +05:30
Rohit Yadav 562a7db8df Merge remote-tracking branch 'origin/4.14'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-05 23:59:16 +05:30
Wei Zhou cd8e28b279
server: Move restoreVM to vm work job queue (#4019) 2020-08-05 09:46:55 +00:00
Rohit Yadav db665fa8df Merge remote-tracking branch 'origin/4.14'
Conflicts:
	engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-05 14:46:13 +05:30
Wei Zhou 4527424fce
vm: Reset deviceId to fix missing nic with vm (#4000)
Co-authored-by: Wei Zhou <57355700+weizhouapache@users.noreply.github.com>
2020-08-05 13:49:03 +05:30
Rohit Yadav 9ae1170b29 Merge remote-tracking branch 'origin/4.14' 2020-08-04 11:28:43 +05:30
Abhishek Kumar a7f56d41c8
cks: assorted bug fixes (#4116)
Fixes #4056
Fixes #4107
Fixes #4113
Fixes #4133

Fixes deployment, template and network deletion.
Also allows filetering in listKubernetesSupportedVersions with keyword

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-08-04 11:27:32 +05:30
Gabriel Beims Bräscher ba6e2ac843
plugins: Redfish Client & Redfish OOBM Driver (#4175)
This PR adds support for the OOBM Redfish protocol, implementing a Java client to send HTTP requests to Redfish supported systems.

Implementation overview:
- Redfish Java client: a Java Client for Redfish that makes Redfish actions available to the HA workflow via an OOB driver.
- OOB Redfish driver: a new Out-of-band driver was created for Redfish, allowing to integrate the Redfish Client with the CloudStack Out-of-band management implementation.

Fixes: #3624
2020-07-30 10:51:16 +05:30
Rohit Yadav 36ef85012f Merge remote-tracking branch 'origin/4.14'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-07-29 14:08:02 +05:30
davidjumani eec56025c8
api: Bug fixes for primate (#4214)
Adding the following fixes so primate can work without issues :
- Adding pagination for listNetworkAclLists
- Adding pagination for listRoles
- Returning mshost uuid rather than msid in list hosts response
- Allowing listVirtualMachinesMetrics to respect hostid
- Fixing return all details in template response
2020-07-29 12:26:39 +05:30
nvazquez 7e3b61b723 Merge branch '4.14' 2020-07-18 14:17:43 -03:00
nvazquez 5c6e79b1eb Merge branch '4.13' into 4.14 2020-07-18 14:15:46 -03:00
Nicolas Vazquez f843c537f0
Fix snapshots garbage collection (#4188)
* Cleanup orphan entries from snapshot store ref for primary storage

* Add debug message
2020-07-18 14:12:53 -03:00
Spaceman1984 7b881517b7
kvm: Fixed removal of hosts from certsmap when running certificate auto-renewal (#4156)
When a host connects to a management server, the host IP address and the certificate are stored in memory on the management server. This mapping is checked periodically to determine if any certificates are due to expire.

Before a certificate is renewed, a few checks are done to determine if the host is connected to the management server by fetching the host record from the database. The problem here is if the wrong record is fetched, the host is not checked for renewal.

This PR improves the host record fetch from the database by looking only at hosts that are not removed.

Fixes: #4129
2020-07-15 15:58:07 +05:30
Andrija Panic 64c69389db
db: VMware Guest OS Mappings fix (#4196)
fix-debian10-32bit-HV-mappings-for-VMware

Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-07-08 08:07:07 +05:30
Rohit Yadav 2c82aac5aa Merge remote-tracking branch 'origin/4.14' 2020-07-07 12:53:05 +05:30
Rohit Yadav b141b8e256 Merge remote-tracking branch 'origin/4.13' into 4.14 2020-07-07 12:51:46 +05:30
sureshanaparti 5040283db9
server: Dynamic roles improvements. Add-on functionality below. (#4071)
- Create a role from any of the existing role, using new parameter roleid in createRole API
- Import a role with its rules, using a new importRole API
- New default roles for Read-Only and Support Admin & User
- No modifications allowed for Default roles
- Cleaned up old NetApp APIs from role_permissions table.
2020-07-07 10:56:43 +05:30
davidjumani b64d0b2d92
quota: Adding pagination for quotaSummary and quotaTariffList (#4186)
Adds pagination and keyword search support to quotaSummary and quotaTariffList
Fixes: #4181
2020-07-06 14:54:04 +05:30
Wei Zhou 4da374b6b4
server: Dedicated hosts should be 'Not Suitable' while find hosts for vm migration (#4001)
While migrate a vm, in the popup, the host dedicated to other accounts/domains are also 'Suitable" for migration, which is obviously wrong.

The same issue happens with api findHostsForMigration
2020-07-04 11:01:41 +05:30
Pearl Dsilva a73712ec4e
server: Enable sending hypervior host name via metadata - VR and Config Drive (#3976)
Enable sending hypervisor host details via metadata for VR and Config Drive providers

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-07-01 08:44:11 +05:30
Nicolas Vazquez 8c1d749360
[VMware] Enable unmanaging guest VMs (#4103)
* Enable unmanaging guest VMs

* Minor fixes

* Fix stop usage event only if VM is not stopped when unmanaging

* Rename unmanaged VMs manager

* Generate netofferingremove usage event if VM is not stopped

* Generate usage event VM snapshot primary off when unmanaging
2020-06-26 08:31:43 -03:00
Rohit Yadav 029fc51960 Merge remote-tracking branch 'origin/4.14' 2020-06-23 12:28:42 +05:30
Rohit Yadav 5f23171ff3 Merge remote-tracking branch 'origin/4.13' into 4.14 2020-06-23 12:27:48 +05:30
Spaceman1984 97f21c1835
xenserver: Fixed null pointer and deployment issue on Xenserver with L2 Guest network with configDrive (#4004)
This PR fixes an issue where an instance fails to deploy due to a null pointer when using an L2 Guest Network with DefaultL2NetworkOfferingConfigDrive on Xenserver. It also fixes migrating an instance to another host.

This has been tested by:
- Creating an L2 Guest network, using DefaultL2NetworkOfferingConfigDrive as the network offering.
- Deploying an instance using the L2 Guest network created.
- Migrating the instance away from the host and back
2020-06-23 12:21:50 +05:30
Rohit Yadav 972d2e201c Merge remote-tracking branch 'origin/4.14' 2020-06-20 10:49:11 +05:30
Rohit Yadav afce828dd1 Merge remote-tracking branch 'origin/4.13' into 4.14 2020-06-20 10:48:06 +05:30
Nicolas Vazquez 7e50f4a01b
server: Fix delete network with no services (#4132)
Fix delete shared network with no services on network offering
Fixes: #4096
2020-06-20 10:45:05 +05:30
harikrishna-patnala 0d4f67ad8c
server: NPE occured when dynamic scaling tried on VM and as part of this when VM tries to migrate if current host does not have capacity. (#3998)
Repro Steps:
1. Create a VM on host1
2. Make host1 capacity full by deploying multiple VMs
3. Try Dynamic scaling on VM on host1
4. NPE occurs when MS tries to find host to migrate the VM and then scale.

Root cause: VM profile is not initiated properly with serviceoffering before planning for deployment

Solution: Iniate VM profile with serviceoffering and also make sure custom compute parameters are handled
2020-06-18 09:19:19 +05:30
harikrishna-patnala a279d5c453
logging: Logging framework to use only log4j (#4003)
Currently CloudStack is using logging frameworks as log4j and Java util logging, logging wrappers as slf4j and Apache common logging.
Here changes are to made it uniform, using only log4j framework.
Removed Java util logging, slf4j and Apache common logging.
2020-06-17 07:11:23 +05:30
Rohit Yadav 567524a2a5 Merge remote-tracking branch 'origin/4.14' 2020-06-16 13:31:29 +05:30
Rohit Yadav b54d19b3b9
server: fix TransactionLegacy DB connection leaks due to DB switching by B&R thread (#4121)
BackupSync task would switch between databases to update backup usage
metrics in the cloud_usage.usage_backup table. The current framework
and the usage in ManagedContext causes database connection
(LegacyTransaction) leaks. When the thread runs faster, the issue is
easily reproducible and checking via heap dump analysis or using JMX
MBeans. This fixes by moving the task of backup data updation for
usage data to the usage server by publishing usage events instead of
switching between databases in a local thread while in a
ManagedContextRunnable.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-06-16 13:30:31 +05:30
Rohit Yadav 612100c84a Merge remote-tracking branch 'origin/4.14' 2020-06-16 12:23:23 +05:30
Rohit Yadav 77947f23fd Merge remote-tracking branch 'origin/4.13' into 4.14
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-06-16 12:21:48 +05:30
harikrishna-patnala 5054766d9f
server: Submitting multiple dynamic VM Scaling API commands for the same instance can result in two usage events in the same second causing a compound key violation in usage service (#3991)
Root cause:
Even though dynamic scaling job is handled in vmworkjob queue which ensures serilizing multiple jobs but the database updating and generating usage events are out of the job queue.

Solution:
Moved all updations into the job queue

Firstly I have tested all the scenarios to check if nothing is broken:
Scaling on a running VM with normal compute offering
Scaling on a stopped VM with normal compute offering
Scaling on a running VM with custom compute offering
Scaling on stopped VM with custom compute offering
Scaling on stopped/running VM between custom compute offering and normal compute offering and combinations among these. Checked if the custom parameters have been populated or deleted accordingly based on the offering to which the VM is scaled
Since this is a corner scenario I could not test the exact point where two usage events are recorded at the same time for two different API calls on same VM.
2020-06-16 11:41:14 +05:30
Rohit Yadav de3ccd2c29 Merge remote-tracking branch 'origin/4.14' 2020-06-15 09:56:55 +05:30
Rohit Yadav e94a54f3b4 Merge remote-tracking branch 'origin/4.13' into 4.14 2020-06-15 09:56:06 +05:30
Spaceman1984 6a683dcf77
storage: Fixed null pointer (#4130)
Fixes #4090

When trying to migrate a VM across 2 clusters, if a snapshot has been deleted and garbage collection has run to update the removed field, it is not possible to migrate the instance due to a null pointer.
2020-06-15 09:54:22 +05:30
dahn 0795cd430b
Boot into hardware setup menu on Vmware (#4021) 2020-06-11 11:47:20 +00:00
Rohit Yadav fbfab5b5eb Merge remote-tracking branch 'origin/4.14' 2020-06-09 08:30:51 +05:30
Rohit Yadav 50f0488aae Merge remote-tracking branch 'origin/4.13' into 4.14 2020-06-09 08:29:39 +05:30
Wei Zhou 851534ac17
server: Honor vm.destroy.forcestop when expunge a vm (#3965)
When expunge a Running vm, vm will be stopped with forcestop=false which does not make sense. we should honor vm.destroy.forcestop in global setting, or always set forcestop=true.
2020-06-09 08:25:51 +05:30
Rohit Yadav 1e19ea5bdd
systemvmtemplate: move to using Debian10 (#4104)
This upgrades the systemvmtemplate base to Debian 10 with openjdk-11 and a newer strongswan package.

Fixes #3654

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-06-09 08:20:51 +05:30
Abhishek Kumar d0d346524b
schema: add empty DB upgrade path from 4.14.0.0 to 4.15.0.0 (#4092)
engine/schema: add empty DB upgrade path from 4.14.0.0 to 4.15.0.0

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-05-27 10:50:51 +05:30
andrijapanicsb 5f926c3353 Updating pom.xml version numbers for release 4.15.0.0-SNAPSHOT
Signed-off-by: andrijapanicsb <andrija.panic@shapeblue.com>
2020-05-23 10:18:39 +01:00
andrijapanicsb 05e9b11694 Updating pom.xml version numbers for release 4.14.1.0-SNAPSHOT
Signed-off-by: andrijapanicsb <andrija.panic@shapeblue.com>
2020-05-23 09:59:32 +01:00
andrijapanicsb 6f96b3b2b3 Updating pom.xml version numbers for release 4.14.0.0
Signed-off-by: andrijapanicsb <andrija.panic@shapeblue.com>
2020-05-11 15:03:14 +01:00
andrijapanicsb 398e685e01 Updating pom.xml version numbers for release 4.13.2.0-SNAPSHOT
Signed-off-by: andrijapanicsb <andrija.panic@shapeblue.com>
2020-04-29 12:29:12 +01:00
Daan Hoogland 689e529d7b Merge release branch 4.13 to master
* 4.13:
  Fixed guest vlan range going missing when using zone wizzard (#4042)
  Volume migration (#4043)
2020-04-23 20:19:30 +02:00
andrijapanicsb b2ffa3efa5 Updating pom.xml version numbers for release 4.13.1.0
Signed-off-by: andrijapanicsb <andrija.panic@shapeblue.com>
2020-04-23 19:17:09 +01:00
dahn c1570b9c91
Volume migration (#4043)
* Update AncientDataMotionStrategy.java

fix When secondary storage usage is> 90%, VOLUME migration across primary storage will cause the migration to fail and lose VOLUME

* Update AncientDataMotionStrategy.java

Volume is migrated across Primary storage. If no secondary storage is available(Or used capacity> 90% ), the migration is canceled.
Before modification, if secondary storage cannot be found, copyVolumeBetweenPools return NUll

copyAsync considers answer = null to be a sign of successful task execution, so it deletes the VOLUME on the old primary storage. This is the root cause of data loss, because VOLUME did not perform the migration at all.

* code in comment removed

Co-authored-by: div8cn <35140268+div8cn@users.noreply.github.com>
Co-authored-by: Daan Hoogland <dahn@onecht.net>
2020-04-23 19:56:27 +02:00
Daan Hoogland b984184b7a Merge release branch 4.13 to master
* 4.13:
  Snapshot deletion issues (#3969)
  server: Cannot list affinity group if there are hosts dedicated… (#4025)
  server: Search zone-wide storage pool when allocation algothrim is firstfitleastconsumed (#4002)
2020-04-11 16:45:00 +02:00
dahn f18fe5e1da
Snapshot deletion issues (#3969)
* Fixes snapshot deletion

* Remove legacy '@Component', it is not necessary in this bean/class.

* Fix log message missing %d and remove snapshot on DB

* Remove "dummy" boolean return statement

* Manage snapshot deletion for KVM + NFS (primary storage)

* checkstyle trailing spaces

* rename options strings to *_OPTION

* Fix typo on deleteSnapshotOnSecondaryStorage and enhance log message

* Move the snapshotDao.remove(snapshotId); (#4006)

* Fix deletesnapshot worflow to handle both snapshots created in primary storage and snapshots backed up to secondary storage

* Fix extra space

* refactor out separate handling methods for secondary and primary (reducing returns)

* return false on unexpected error or log when expected

* != instead of ==

* secondary instead of backup storage

* init to null

* Handle snapshot deletion on primary storage. When primary store ref not found for snapshot do not fail the operation.

* Fix debug levels on log messages

Co-authored-by: GabrielBrascher <gabriel@apache.org>
Co-authored-by: Andrija Panic <45762285+andrijapanicsb@users.noreply.github.com>
Co-authored-by: Harikrishna Patnala <harikrishna.patnala@gmail.com>
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
2020-04-11 16:40:27 +02:00
Wei Zhou 6bf92fb136
server: Search zone-wide storage pool when allocation algothrim is firstfitleastconsumed (#4002) 2020-04-06 22:01:40 +02:00
Andrija Panic d52f3f4a6b
Update schema-41310to41400.sql (#3999)
* Update schema-41310to41400.sql

* update desc

* update the config key as well

* Update schema-41310to41400.sql (#4012)

* Update schema-41310to41400.sql

* update configkey desc
2020-04-04 14:07:14 +02:00
Spaceman1984 84bede2171
Updated upgrade paths (#3972)
* Updated upgrade paths

* Added license info

Co-authored-by: dahn <daan.hoogland@shapeblue.com>
2020-03-24 15:04:42 +01:00
Daan Hoogland 173174c804 Merge branch '4.13' 2020-03-24 13:41:45 +00:00
Spaceman1984 e9b652a5b7
Updated upgrade path (#3971)
* Updated upgrade path

* Added license info
2020-03-24 13:43:59 +01:00
Arthur Halet 3575f5ed52
vrouter in redundant mode acquire guest ips from first ip of th… (#3587) 2020-03-14 09:22:48 +01:00