Commit Graph

36795 Commits

Author SHA1 Message Date
Wei Zhou db1e0f49dc
kvm: set Backing Format of qcow2 images in vm start and migration (#4656)
In previous cloudstack versions, qcow2 image does not have a backing file format.
however, it is required in newer qemu versions, for example qemu 4.2 on ubuntu 20.04.

steps to reproduce the issue
(1) install cloudstack 4.14 or previous version, and ubuntu 19.04 or 18.04/16.04 LTS.
(2) create vms.
(3) upgrade to 4.15, upgrade os to ubuntu 20.04 , or install a new server with ubuntu 20.04.
(4) migrate vm from old ubuntu version to ubuntu 20.04, failed with exception below
```
2021-02-04 13:43:07,397 DEBUG [resource.wrapper.LibvirtMigrateCommandWrapper] (agentRequest-Handler-1:null) (logid:93da9385) ExecutionException : org.libvirt.LibvirtException: Requested operation is not valid: format of backing image '/mnt/03b6f487-9eaf-38bf-ad2d-d985423b832f/66990fcc-fd98-4932-9649-989bf6583d59' of image '/mnt/03b6f487-9eaf-38bf-ad2d-d985423b832f/a3dd1f0f-2557-4e07-951c-e4eb7b3f38b2' was not specified in the image metadata (See https://libvirt.org/kbase/backing_chains.html for troubleshooting)
```
(5)stop vm, and start it on ubuntu 20.04 server. failed with exception below
```
2021-02-04 13:46:29,766 WARN  [resource.wrapper.LibvirtStartCommandWrapper] (agentRequest-Handler-5:null) (logid:b54745a7) LibvirtException
org.libvirt.LibvirtException: Requested operation is not valid: format of backing image '/mnt/03b6f487-9eaf-38bf-ad2d-d985423b832f/66990fcc-fd98-4932-9649-989bf6583d59' of image '/mnt/03b6f487-9eaf-38bf-ad2d-d985423b832f/a3dd1f0f-2557-4e07-951c-e4eb7b3f38b2' was not specified in the image metadata (See https://libvirt.org/kbase/backing_chains.html for troubleshooting)
```

To make testing easier, step 1 and 2 can be replaced by
```
qemu-img create -f qcow2 -b <backing file> <qcow2 image>
```
so qcow2 image does not have a backing file format.
2021-02-19 14:06:47 +05:30
Rakesh fd49efa9c1
server: Show network name in exception message (#4396)
* Show network name in exception message

* Update server/src/main/java/com/cloud/vm/UserVmManagerImpl.java

Co-authored-by: dahn <daan.hoogland@gmail.com>
2021-02-19 14:04:39 +05:30
Rohit Yadav 9ce70afccf Merge remote-tracking branch 'origin/4.15' 2021-02-19 13:28:16 +05:30
Rakesh 13ba9d5100
Display creation date for domain and account (#4686) 2021-02-18 14:25:46 +00:00
dahn aab2447656
systemvm: loop optimisation in bash (#4451)
Co-authored-by: Daan Hoogland <dahn@onecht.net>
2021-02-18 18:18:16 +05:30
Pearl Dsilva d6509f0a21
API discovery: Prevent overwrite of API parameters in case the API names are the same (#4609)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-02-18 14:37:20 +05:30
Pearl Dsilva aa01580381
network: Specify IP for VR in shared networks (#4503)
This PR enables admins to specify IP for a VR in a shared network.
2021-02-18 13:54:09 +05:30
Pearl Dsilva b6fe9f99eb
Network Offering: Allow enabling network and vpc offering during creation (#4564)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-02-13 10:19:06 +00:00
Sina Kashipazha 543f9827ff
add creation date as a value for domains and accounts. (#4649) 2021-02-13 10:07:53 +00:00
Rohit Yadav 40c14b4552 Merge remote-tracking branch 'origin/4.15' 2021-02-12 13:06:33 +05:30
Gabriel Beims Bräscher 35055896a0
server: FIx typo on FirstFitAllocator (#4648)
From "Cusotm[UEFI]" to "Custom[UEFI]"
2021-02-12 12:53:40 +05:30
Rakesh de7b131724
server: Cleanup domain details when domain is deleted (#4445)
When domain is deleted, all the settings configured under
the domain scope still exists in domain_details table.
All the entries for the domain should be deleted as well
2021-02-12 12:52:22 +05:30
Abhishek Kumar 0d04bd621f
ui: fix tags selection for add disk offering (#4665)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-02-12 12:50:32 +05:30
Rakesh b8ec2b22d3
server: Display lb rule name instead of uuid (#4524)
Display proper error message while deleting ssl cert which is
associated with a load balancer rule
2021-02-12 12:47:01 +05:30
Abhishek Kumar d6e8b53736
vmware: vm migration improvements (#4385)
- Fixes inter-cluster migration of VMs
- Allows migration of stopped VM with disks attached to different and suitable pools
- Improves inter-cluster detached volume migration
- Allows inter-cluster migration (clusters of same Pod) for system VMs, VRs on VMware
- Allows storage migration for stopped system VMs, VRs on VMware within same Pod if StoragePool cluster scopetype

Linked Primate PR: https://github.com/apache/cloudstack-primate/pull/789 [Changes merged in this PR after new UI merge]
Documentation PR: https://github.com/apache/cloudstack-documentation/pull/170

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-02-12 12:41:41 +05:30
Rohit Yadav 66f0beda5f Updating pom.xml version numbers for release 4.14.1.0
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-02-08 16:24:09 +05:30
Rohit Yadav 0cca854613 Merge remote-tracking branch 'origin/4.15' 2021-02-05 18:29:13 +05:30
Rohit Yadav f5a44b3502 Merge remote-tracking branch 'origin/4.14' into 4.15 2021-02-05 18:28:02 +05:30
Pearl Dsilva c9c1d4a6e5
marvin: fix test failures when changing service offering of a VM (#4651)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-02-05 18:12:12 +05:30
Wei Zhou d62d5c6cd2
VR: fix expunging vm will remove dhcp entries of another vm in VR (#4627)
Steps to reproduce the issue

(1) create two vm wei-001 and wei-002, start them

(2) check /etc/cloudstack/dhcpentry.json and /etc/dhcphosts.txt in VR
They have entries for both of wei-001 and wei-002

(3) stop wei-002, and restart VR (or restart network with cleanup).
check /etc/cloudstack/dhcpentry.json and /etc/dhcphosts.txt in VR
They have entries for wei-001 only (as wei-002 is stopped)

(4) expunge wei-002. when it is done,
check /etc/cloudstack/dhcpentry.json and /etc/dhcphosts.txt in VR
They do not have entries for wei-001.
VR health check fails at dhcp_check.py and dns_check.py
2021-02-05 18:10:53 +05:30
Rohit Yadav ba127dab3e Merge remote-tracking branch 'origin/4.15'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-02-05 16:02:26 +05:30
Rohit Yadav 6bde1384ff Merge remote-tracking branch 'origin/4.14' into 4.15
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-02-05 16:01:01 +05:30
Abhishek Kumar 29e309839d
test: hardware required for changeserviceoffering (#4650)
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2021-02-05 15:58:45 +05:30
Andrija Panic b8e6c08132
setup: Update cloud-setup-databases.in (#4653) 2021-02-05 15:57:21 +05:30
Abhishek Kumar db4f9e9b53
test: fix checksums for test template (#4655)
macchinina-vmware.ova has changed at http://dl.openvm.eu/cloudstack/macchinina/x86_64/
sha1, sha256 and m5 checksum have been updated for template file in test_template.py

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-02-05 15:56:54 +05:30
Wei Zhou 4de6ac3c05
server: Get vm network/disk statistics and update database per host (#4601)
* server: Get vm network/disk statistics and update database per host

* #4601 : modify debug message
2021-02-04 14:44:47 +05:30
Wei Zhou 78f73c1bc6
server: Fix update capacity for hosts take long time if there are many service offerings (#4623)
Steps to reproduce the issue:

(1)Create 10000 service offerings (by db changes below or cloudmonkey).

```
DROP PROCEDURE IF EXISTS cloud.insert_service_offering;

DELIMITER $$
CREATE PROCEDURE cloud.insert_service_offering()
BEGIN
  DECLARE count INT DEFAULT 10000;
  SET @offeringid = (select max(id)+1 from disk_offering);

  WHILE count > 0 DO
    INSERT INTO disk_offering (id,name,uuid,display_text,disk_size,type,created) values (@offeringid,'test-offering-wei',uuid(), 'test-offering-wei',0,'Service',now());
    INSERT INTO service_offering (id,cpu,speed,ram_size) values (@offeringid, 1, 500,256);
    SET @offeringid = @offeringid + 1;
    SET count = count - 1;
  END WHILE;
END $$
DELIMITER ;

CALL cloud.insert_service_offering();

mysql> CALL cloud.insert_service_offering();
Query OK, 0 rows affected (2 min 30.85 sec)
```

(2) Check the total time of periodical capacity check in cloudstack.

Without this patch, it spend 2.5 seconds (2 hosts)
```
2021-01-15 16:10:12,793 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-5d5f3b3b) (logid:f5eb68ba) Running Capacity Checker ...
2021-01-15 16:10:15,287 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-5d5f3b3b) (logid:f5eb68ba) Done running Capacity Checker ...
```

With this patch ,it spend 1.3 seconds (2 hosts)
```
2021-01-15 16:12:43,604 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-a2a7f3f1) (logid:f7e0a4c5) Running Capacity Checker ...
2021-01-15 16:12:44,927 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-a2a7f3f1) (logid:f7e0a4c5) Done running Capacity Checker ...
```

If there are 100 hosts, the total time will be reduced from 100+ seconds to around 10 seconds.
2021-02-04 14:43:57 +05:30
Daan Hoogland 66d49c5c0d Merge release branch 4.14 to 4.15
* 4.14:
  server: prevent update vm read-only details (#4629)
2021-02-02 09:16:34 +00:00
Abhishek Kumar 05301b1e6a
server: prevent update vm read-only details (#4629) 2021-02-02 08:49:25 +00:00
Daan Hoogland ff376d8187 Merge release branch 4.15 to master
* 4.15:
  server: select root disk based on user input during vm import (#4591)
  kvm: Use Q35 chipset for UEFI x86_64 (#4576)
  server: fix wrong error message when create isolated network without SourceNat (#4624)
  server: add possibility to scale vm to current customer offerings (#4622)
  server: keep networks order and ips while move a vm with multiple networks (#4602)
  server: throw exception when update vm nic on L2 network (#4625)
  doc: fix typo in install notes (#4633)
2021-02-01 09:58:52 +00:00
Daan Hoogland b6b778f003 Merge release branch 4.14 to 4.15
* 4.14:
  server: select root disk based on user input during vm import (#4591)
  kvm: Use Q35 chipset for UEFI x86_64 (#4576)
  server: fix wrong error message when create isolated network without SourceNat (#4624)
  server: add possibility to scale vm to current customer offerings (#4622)
  server: keep networks order and ips while move a vm with multiple networks (#4602)
  server: throw exception when update vm nic on L2 network (#4625)
  doc: fix typo in install notes (#4633)
2021-02-01 09:57:35 +00:00
Abhishek Kumar 9b45ec275a
server: select root disk based on user input during vm import (#4591)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-02-01 09:55:09 +00:00
jairov4 e9dda98a87
kvm: Use Q35 chipset for UEFI x86_64 (#4576)
Fix #4245

This PR uses Q35 chipset for UEFI in x86_64.
Currently this mistakenly only enabled for secure boot
2021-02-01 14:22:29 +05:30
Wei Zhou 313ae1f449
server: fix wrong error message when create isolated network without SourceNat (#4624)
This PR fixes wrong message when create isolated network without SourceNat.
2021-02-01 14:15:47 +05:30
Wei Zhou a44fb11a02
server: add possibility to scale vm to current customer offerings (#4622)
We can use cloudmonkey to scale a vm with dynamic offering, to same offering but with different cpunumber or memory.
Enable it on UI to improve user experience.
2021-02-01 14:14:48 +05:30
Wei Zhou 1913c6854e
server: keep networks order and ips while move a vm with multiple networks (#4602)
This PR fixes an issue when move a vm from an account to another account.

Steps to reproduce the issue
(1) create a vm with multiple shared networks (in advanced zone, or advanced zone with security groups)
(2) create another account (in same domain who can also access the shared networks)
(3) move vm to new account, with a list of networkid

expected result: the vm has nics on the networks in same order as specified in API request, and nics have the same ips as before actual result: network order is not same as specified, ips are changed.
2021-02-01 14:14:20 +05:30
Wei Zhou 890e84777c
server: throw exception when update vm nic on L2 network (#4625)
without this change
```
root@mgt01:~# cmk update vmnicip nicid=afab73cb-f4f4-490f-a524-365edef432b7

{
  "accountid": "a27bffc1-48ee-11eb-8680-069fc4003392",
  "cmd": "org.apache.cloudstack.api.command.user.vm.UpdateVmNicIpCmd",
  "completed": "2020-12-28T12:55:27+0000",
  "created": "2020-12-28T12:55:27+0000",
  "jobid": "88af30e0-214b-4379-9c39-49648f998ff6",
  "jobprocstatus": 0,
  "jobresult": {
    "errorcode": 530,
    "errortext": "Failed to update ip address on vm NIC. Refer to server logs for details."
  },
  "jobresultcode": 530,
  "jobresulttype": "object",
  "jobstatus": 2,
  "userid": "a27f45f5-48ee-11eb-8680-069fc4003392"
}
🙈 Error: async API failed for job 88af30e0-214b-4379-9c39-49648f998ff6
```

with this change
```
root@mgt01:~# cmk update vmnicip nicid=22d79189-6754-4dfe-a8c4-0ba21be2ada5

{
  "accountid": "044bb5a9-32fd-11eb-b251-06b6e80033a6",
  "cmd": "org.apache.cloudstack.api.command.user.vm.UpdateVmNicIpCmd",
  "completed": "2020-12-28T12:55:15+0000",
  "created": "2020-12-28T12:55:15+0000",
  "jobid": "f4ccb545-0e47-4e9c-b562-b0e6cf52ddd7",
  "jobprocstatus": 0,
  "jobresult": {
    "errorcode": 431,
    "errortext": "UpdateVmNicIpCmd is not supported in L2 network"
  },
  "jobresultcode": 530,
  "jobresulttype": "object",
  "jobstatus": 2,
  "userid": "044cc6e3-32fd-11eb-b251-06b6e80033a6"
}
🙈 Error: async API failed for job f4ccb545-0e47-4e9c-b562-b0e6cf52ddd7
```
2021-02-01 14:13:50 +05:30
Abhishek Kumar e4972c9981
doc: fix typo in install notes (#4633)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-02-01 14:08:57 +05:30
Rohit Yadav be5e61a2f3 Merge remote-tracking branch 'origin/4.15' 2021-01-28 16:45:49 +05:30
Rohit Yadav 1bccb954c4 Fix merge issue from 74bae56642
Fixes fwd merge issue from origin/4.14

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-01-28 16:43:59 +05:30
Daniel Augusto Veronezi Salvador 3a4a82df69
kvm: Replicate `rootDiskController` behavior to `dataDiskController`. (#4569)
The bus type to `data disk` volumes is hardcoded to `virtio` or `scsi`, when using virtio-scsi (or, based on the template type). Therefore, there is no way to specify the bus type to data disk volumes (as we have for root disks).
This PR intends to replicate the `rootDiskController` behavior to `dataDiskController`, allowing the definition of the controller.

Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
2021-01-28 15:58:06 +05:30
Rohit Yadav f98940953b Merge remote-tracking branch 'origin/4.15' 2021-01-28 14:31:04 +05:30
Rohit Yadav abec6d1fff
packaging: build and bundle UI using npm in deb and rpm packages (#4605)
This changes deb and rpm packaging to build the UI using npm and bundle
it in the `cloudstack-management` package and a new `cloudstack-ui`
package. The `cloudstack-ui` package will install the UI under
`/usr/share/cloudstack-ui/`. For both packages the config.json will not
be overridden on upgrade and hosted at /etc/cloudstack/management
for the cloudstack-mangement package, and at /etc/cloudstack/ui for the
cloudstack-ui package. The cloudstack-ui package is for advanced users
who only want the UI want to setup reverse proxy (separate hosting of UI).

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-01-28 14:29:31 +05:30
Rohit Yadav e6fef05f30 Merge remote-tracking branch 'origin/4.15' 2021-01-28 14:25:40 +05:30
Rohit Yadav 74bae56642 Merge remote-tracking branch 'origin/4.14' into 4.15
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-01-28 14:24:25 +05:30
Daniel Costa 7ba0374788
ui: Fix screenshot path on README of ui (#4620) 2021-01-28 13:30:08 +05:30
Wei Zhou 182cea79b5
server: fix cannot create vm if another vm with same name has been added and removed on the network (#4600)
* server: fix cannot create vm if another vm with same name has been added and removed on the network

steps to reproduce the issue
(1) create vm-1 on network-1
(2) add vm-1 to network-2
(3) remove vm-1 from network-2
(4) create another vm with same name vm-1 on network-2

expected result: operation succeed
actual result: operation failed.

* #4600: add back a removed line
2021-01-27 19:28:52 +05:30
slavkap 4a779deab2
server: fix on changeServiceForVirtualMachine when updating read/write rate (#4491)
When changing the service offering of a VM the disk_offering_id is not
updated in volumes DB table and VM could not start

Fixes #4125
2021-01-27 19:27:34 +05:30
Sina Kashipazha b84a675de9
systemvm: Fixed typo (#4621) 2021-01-27 15:35:06 +05:30
Hoang Nguyen 2a7f84eafd
ui: Add quickview to the list of VM Snapshot (#4614)
This PR is used to fix the Quick access menu bar missing for VM Snapshots
Fixes #4607
2021-01-27 15:33:35 +05:30