Commit Graph

32593 Commits

Author SHA1 Message Date
Wei Zhou d62d5c6cd2
VR: fix expunging vm will remove dhcp entries of another vm in VR (#4627)
Steps to reproduce the issue

(1) create two vm wei-001 and wei-002, start them

(2) check /etc/cloudstack/dhcpentry.json and /etc/dhcphosts.txt in VR
They have entries for both of wei-001 and wei-002

(3) stop wei-002, and restart VR (or restart network with cleanup).
check /etc/cloudstack/dhcpentry.json and /etc/dhcphosts.txt in VR
They have entries for wei-001 only (as wei-002 is stopped)

(4) expunge wei-002. when it is done,
check /etc/cloudstack/dhcpentry.json and /etc/dhcphosts.txt in VR
They do not have entries for wei-001.
VR health check fails at dhcp_check.py and dns_check.py
2021-02-05 18:10:53 +05:30
Abhishek Kumar 29e309839d
test: hardware required for changeserviceoffering (#4650)
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2021-02-05 15:58:45 +05:30
Andrija Panic b8e6c08132
setup: Update cloud-setup-databases.in (#4653) 2021-02-05 15:57:21 +05:30
Abhishek Kumar db4f9e9b53
test: fix checksums for test template (#4655)
macchinina-vmware.ova has changed at http://dl.openvm.eu/cloudstack/macchinina/x86_64/
sha1, sha256 and m5 checksum have been updated for template file in test_template.py

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-02-05 15:56:54 +05:30
Wei Zhou 4de6ac3c05
server: Get vm network/disk statistics and update database per host (#4601)
* server: Get vm network/disk statistics and update database per host

* #4601 : modify debug message
2021-02-04 14:44:47 +05:30
Wei Zhou 78f73c1bc6
server: Fix update capacity for hosts take long time if there are many service offerings (#4623)
Steps to reproduce the issue:

(1)Create 10000 service offerings (by db changes below or cloudmonkey).

```
DROP PROCEDURE IF EXISTS cloud.insert_service_offering;

DELIMITER $$
CREATE PROCEDURE cloud.insert_service_offering()
BEGIN
  DECLARE count INT DEFAULT 10000;
  SET @offeringid = (select max(id)+1 from disk_offering);

  WHILE count > 0 DO
    INSERT INTO disk_offering (id,name,uuid,display_text,disk_size,type,created) values (@offeringid,'test-offering-wei',uuid(), 'test-offering-wei',0,'Service',now());
    INSERT INTO service_offering (id,cpu,speed,ram_size) values (@offeringid, 1, 500,256);
    SET @offeringid = @offeringid + 1;
    SET count = count - 1;
  END WHILE;
END $$
DELIMITER ;

CALL cloud.insert_service_offering();

mysql> CALL cloud.insert_service_offering();
Query OK, 0 rows affected (2 min 30.85 sec)
```

(2) Check the total time of periodical capacity check in cloudstack.

Without this patch, it spend 2.5 seconds (2 hosts)
```
2021-01-15 16:10:12,793 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-5d5f3b3b) (logid:f5eb68ba) Running Capacity Checker ...
2021-01-15 16:10:15,287 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-5d5f3b3b) (logid:f5eb68ba) Done running Capacity Checker ...
```

With this patch ,it spend 1.3 seconds (2 hosts)
```
2021-01-15 16:12:43,604 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-a2a7f3f1) (logid:f7e0a4c5) Running Capacity Checker ...
2021-01-15 16:12:44,927 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-a2a7f3f1) (logid:f7e0a4c5) Done running Capacity Checker ...
```

If there are 100 hosts, the total time will be reduced from 100+ seconds to around 10 seconds.
2021-02-04 14:43:57 +05:30
Abhishek Kumar 05301b1e6a
server: prevent update vm read-only details (#4629) 2021-02-02 08:49:25 +00:00
Abhishek Kumar 9b45ec275a
server: select root disk based on user input during vm import (#4591)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-02-01 09:55:09 +00:00
jairov4 e9dda98a87
kvm: Use Q35 chipset for UEFI x86_64 (#4576)
Fix #4245

This PR uses Q35 chipset for UEFI in x86_64.
Currently this mistakenly only enabled for secure boot
2021-02-01 14:22:29 +05:30
Wei Zhou 313ae1f449
server: fix wrong error message when create isolated network without SourceNat (#4624)
This PR fixes wrong message when create isolated network without SourceNat.
2021-02-01 14:15:47 +05:30
Wei Zhou a44fb11a02
server: add possibility to scale vm to current customer offerings (#4622)
We can use cloudmonkey to scale a vm with dynamic offering, to same offering but with different cpunumber or memory.
Enable it on UI to improve user experience.
2021-02-01 14:14:48 +05:30
Wei Zhou 1913c6854e
server: keep networks order and ips while move a vm with multiple networks (#4602)
This PR fixes an issue when move a vm from an account to another account.

Steps to reproduce the issue
(1) create a vm with multiple shared networks (in advanced zone, or advanced zone with security groups)
(2) create another account (in same domain who can also access the shared networks)
(3) move vm to new account, with a list of networkid

expected result: the vm has nics on the networks in same order as specified in API request, and nics have the same ips as before actual result: network order is not same as specified, ips are changed.
2021-02-01 14:14:20 +05:30
Wei Zhou 890e84777c
server: throw exception when update vm nic on L2 network (#4625)
without this change
```
root@mgt01:~# cmk update vmnicip nicid=afab73cb-f4f4-490f-a524-365edef432b7

{
  "accountid": "a27bffc1-48ee-11eb-8680-069fc4003392",
  "cmd": "org.apache.cloudstack.api.command.user.vm.UpdateVmNicIpCmd",
  "completed": "2020-12-28T12:55:27+0000",
  "created": "2020-12-28T12:55:27+0000",
  "jobid": "88af30e0-214b-4379-9c39-49648f998ff6",
  "jobprocstatus": 0,
  "jobresult": {
    "errorcode": 530,
    "errortext": "Failed to update ip address on vm NIC. Refer to server logs for details."
  },
  "jobresultcode": 530,
  "jobresulttype": "object",
  "jobstatus": 2,
  "userid": "a27f45f5-48ee-11eb-8680-069fc4003392"
}
🙈 Error: async API failed for job 88af30e0-214b-4379-9c39-49648f998ff6
```

with this change
```
root@mgt01:~# cmk update vmnicip nicid=22d79189-6754-4dfe-a8c4-0ba21be2ada5

{
  "accountid": "044bb5a9-32fd-11eb-b251-06b6e80033a6",
  "cmd": "org.apache.cloudstack.api.command.user.vm.UpdateVmNicIpCmd",
  "completed": "2020-12-28T12:55:15+0000",
  "created": "2020-12-28T12:55:15+0000",
  "jobid": "f4ccb545-0e47-4e9c-b562-b0e6cf52ddd7",
  "jobprocstatus": 0,
  "jobresult": {
    "errorcode": 431,
    "errortext": "UpdateVmNicIpCmd is not supported in L2 network"
  },
  "jobresultcode": 530,
  "jobresulttype": "object",
  "jobstatus": 2,
  "userid": "044cc6e3-32fd-11eb-b251-06b6e80033a6"
}
🙈 Error: async API failed for job f4ccb545-0e47-4e9c-b562-b0e6cf52ddd7
```
2021-02-01 14:13:50 +05:30
Abhishek Kumar e4972c9981
doc: fix typo in install notes (#4633)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-02-01 14:08:57 +05:30
Wei Zhou 182cea79b5
server: fix cannot create vm if another vm with same name has been added and removed on the network (#4600)
* server: fix cannot create vm if another vm with same name has been added and removed on the network

steps to reproduce the issue
(1) create vm-1 on network-1
(2) add vm-1 to network-2
(3) remove vm-1 from network-2
(4) create another vm with same name vm-1 on network-2

expected result: operation succeed
actual result: operation failed.

* #4600: add back a removed line
2021-01-27 19:28:52 +05:30
slavkap 4a779deab2
server: fix on changeServiceForVirtualMachine when updating read/write rate (#4491)
When changing the service offering of a VM the disk_offering_id is not
updated in volumes DB table and VM could not start

Fixes #4125
2021-01-27 19:27:34 +05:30
Sina Kashipazha b84a675de9
systemvm: Fixed typo (#4621) 2021-01-27 15:35:06 +05:30
davidjumani f70da104a0
engine/schema: add upgrade path from 4.14.0.0 to 4.14.1.0 (#4580) 2021-01-12 18:21:30 +05:30
Spaceman1984 ec4d83da4d
vmware: Fixed instance creation failure on dvswitch when using vlan id 4095 (#4557)
Fixed instance creation failure on dvswitch when using vlan id 4095
2021-01-12 13:23:16 +05:30
davidjumani 4d33e159f7
vr: Ensuring dnsmasq.leases file is populated (#4529) 2020-12-14 09:06:24 +00:00
Wei Zhou 96cfe27504
db upgrade: fix sql exception: Access denied; you need (at least one of) the SUPER privilege(s) for this operation (#4533) 2020-12-14 11:40:33 +05:30
Sina Kashipazha 160cdfb948
apidoc issue (#4532)
* Fix sed command failure in Mac OS.

* FIX, Removed the buggy line that I copied from Stackoverflow.
2020-12-11 13:11:44 +05:30
Wei Zhou 1a47719588
kvm: set cpu topology only if cpucore per socket is positive value (#4527)
This PR fixes a regression issue in #4497

In cloudstack 4.14 or before, the cpu topology is set only when cpucore per socket is set (to 4 or 6).
in other conditions, there is no cpu topology in vm xml definition.

with #4497, vm will have cpu topology in its xml definition, if cpucore per socket is not set.

    <topology sockets='<vm cpu cores>' cores='1' threads='1'/>
Not sure if it causes any issue. I think it would be better not to add this part in vm xml definition if cpucore per socket is not set.
2020-12-10 14:29:28 +05:30
Wei Zhou 93f3d35207
kvm: FIX cpucorespersocket is not working on KVM (#4497) 2020-12-09 15:07:51 +01:00
Pearl Dsilva 837372488f
Fix failure in validating IP address in case of multiple Management Servers (#4507)
* Fix failure in validating IP address in case of multiple Management Servers

* refactor code

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-12-08 13:42:24 +05:30
Wei Zhou 83ae6d7c01 integration test: skip vlan of public ip range in get_free_vlan 2020-12-03 15:10:43 +01:00
Wei Zhou 03e65a1266 vpc vr: plugin nics by this order: public/private/guest 2020-12-03 15:10:43 +01:00
Wei Zhou 627070c5f8 vpc vr: fix Conflicting device id on private gw nic 2020-12-03 15:10:43 +01:00
davidjumani 7f5eb6463e
Disallowing udp for lb rules for haproxy (#4501)
* Disallowing udp for lb rules

* Revert "Disallowing udp for lb rules"

This reverts commit a5f43014c9.

* Adding check for udp rules for haproxy

* Adding defensive check
2020-12-02 13:59:03 +05:30
davidjumani afebfd5bbf
Adding cpuallocated percentage and value to host and hostsformigrationresponse (#4499)
* Adding cpuallocatedwithoverprovisoning to hostresponse and hostsformigrationresponse

* Adding cpuallocatedpercentage and cpuallocatedvalue
2020-12-01 13:56:19 +05:30
Wei Zhou e82813b509
kvm: fix router.aggregation.command.each.timeout is reset to 600 when update other kvm configs (#4496)
* kvm: fix router.aggregation.command.each.timeout is reset to 600 when update other kvm configs

* remove _params
2020-11-30 16:22:16 +05:30
Wei Zhou e8ee8e9fbe
fix failures with test_multiple_nic_support.py (#4495) 2020-11-30 16:21:46 +05:30
davidjumani 2eef0e5af3
Fix hosts for migration count (#4500)
* Fixing count for findHostsForMigration

* Changing list name
2020-11-25 15:34:23 +05:30
Wei Zhou a979ab9050 integration test: update steps 2020-11-25 08:40:16 +01:00
Wei Zhou 6a91b8ace7 integration test: add private gateway in test 2020-11-25 08:40:16 +01:00
Wei Zhou b2d8fffeea integration test: verify public nics state 2020-11-25 08:40:16 +01:00
Wei Zhou 8a68617eee bugfix #9 vpc vr: Add PREROUTING rule for vm with static nat to multiple private gateways 2020-11-25 08:40:16 +01:00
Wei Zhou 69c0f71cf7 bugfix #8 vpc: add rule for traffic between vm and private gateway 2020-11-25 08:40:16 +01:00
Wei Zhou a8c9b4531b bugfix #7 vpc vr: allow servers in private gateway to reach internet via the VPC VR if it is gateway 2020-11-25 08:40:16 +01:00
Wei Zhou 8fb2efee1c bugfix #6 vpc vr: Add iptables rules for ACL of private gateway 2020-11-25 08:40:16 +01:00
Wei Zhou 7e6f484332 Revert "Fix Policy Based Routing for private gateway static routes (#3604)"
This reverts commit 82d94a87c5.
2020-11-25 08:40:16 +01:00
Wei Zhou 66d3e1f6d4 Revert "Add private gateway IP to router initialization config"
This reverts commit 65cb22216a.
2020-11-25 08:40:16 +01:00
Wei Zhou 5cc6fedb1f Revert "Handle private gateways more reliably"
This reverts commit f4f9b3ab4e.
2020-11-25 08:40:16 +01:00
Wei Zhou 51f3756030 bugfix #4 vpc vr: Do NOT send Nic plug in/out command to Stopped/Stopping VR 2020-11-25 08:40:16 +01:00
Wei Zhou 655ed10655 bugfix #2 vpc vr: fix issue if static nat is disabled but still other IP used by lb/pf 2020-11-25 08:40:16 +01:00
Wei Zhou f5ab87c153 bugfix #3 apply ip dessociation before unplugging a nic so ip is marked as add:false in ips.json 2020-11-25 08:40:16 +01:00
Wei Zhou 6491a69d63 bugfix #2 vpc: Fix remove first public ip will remove all ips on the nic 2020-11-25 08:40:16 +01:00
Wei Zhou 81ac9f90ab
vr: fix python exception when configure VRs (#4489)
before
```
root@r-27-VM:/var/cache/cloud# /opt/cloud/bin/configure.py monitor_service.json
ERROR:root:Command 'ip link show eth0 | grep 'state DOWN'' returned non-zero exit status 1
```

with this change
```
root@r-27-VM:/var/cache/cloud# /opt/cloud/bin/configure.py monitor_service.json
root@r-27-VM:/var/cache/cloud#
```
2020-11-23 14:09:40 +05:30
Wei Zhou 75fdb07387
vpc: fix ips on wrong interfaces after rebooting vpc vrs (#4467)
* vpc: fix ips on wrong interfaces after rebooting vpc vrs

* #4467: Rename to updateNicWithDeviceId

* CLSTACK-8923 vr: Force a restart of keepalived if conntrackd is not running or configuration has changed
2020-11-20 21:02:53 +05:30
davidjumani d79d24261a
Adding memoryallocatedpercentage & memoryallocatedbytes to HostsResponse & HostsForMigrationResponse (#4478) 2020-11-20 11:27:47 +00:00