Commit Graph

31889 Commits

Author SHA1 Message Date
Rohit Yadav 47f5c656ac Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-04 15:32:01 +05:30
Rohit Yadav 35b914570d
travis: use openjdk8 and xenial (ubuntu 16.04) (#3466)
This fixes Travis to use openjdk8 and Xenial (Ubuntu 16.04) as base
distro.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-04 15:30:18 +05:30
Rohit Yadav f0d7942162 Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-04 13:56:24 +05:30
Paul Angus b72aadb65d Updating pom.xml version numbers for release 4.11.4.0-SNAPSHOT
Signed-off-by: Paul Angus <paul.angus@shapeblue.com>
2019-07-03 09:28:09 +01:00
Paul Angus f855e6d3b9 Merge branch '4.11.3.0-RC20190625T2045' into 4.11 2019-07-03 09:10:24 +01:00
Rohit Yadav 0649821a21 Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-01 14:37:40 +05:30
Wei Zhou 9ac32aae38 server: VPC redundant vrs run on same hypervisor (#3421)
For VPC supports redundant VRs, when start the second VR, the pod/cluster/host of first VR should be added to avoid list. This provides higher availability.

The network VRs have the same process already.
2019-06-29 12:09:36 +05:30
Rohit Yadav a2323e1425 Merge remote-tracking branch 'origin/4.11' into 4.12 2019-06-26 16:27:29 +05:30
ustcweizhou 2408783355 ssvm: use secstorage.ssl.cert.domain as hostname if it does not start with '*' when upload a template or volume from local (#3420)
If secstorage.ssl.cert.domain does not start with '*', we should use it as host name in url when upload template/volume from local

Fixes #3305
2019-06-26 16:27:07 +05:30
ustcweizhou e76266e39b systemvm: Fix hostname is localhost in some VRs (#3422)
In some virtual routers, 'hostname -f' returns 'localhost'. The hostname is also 'localhost' in `/var/log/messages`. This change can fix the issue in new VRs.
2019-06-26 16:26:05 +05:30
Paul Angus d895983d11 Updating pom.xml version numbers for release 4.11.3.0
Signed-off-by: Paul Angus <paul.angus@shapeblue.com>
2019-06-25 20:45:12 +01:00
Rohit Yadav 018bba352b Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-25 21:50:22 +05:30
Rohit Yadav 6f1fc18332 Revert "Updating pom.xml version numbers for release 4.11.4.0-SNAPSHOT"
This reverts commit 5bfad44ef4 because
we'll need another RC on latest 4.11 branch towards 4.11.3.0.
2019-06-25 21:31:05 +05:30
Rohit Yadav 64d1316e60 engine/schema: add upgrade path for source 4.11.2.0 version
This adds a missing upgrade path for source version 4.11.2.0. Without
this fix, 4.11.2.0 users won't be able to upgrade to 4.11.3.0.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-25 21:31:05 +05:30
Paul Angus 5bfad44ef4 Updating pom.xml version numbers for release 4.11.4.0-SNAPSHOT
Signed-off-by: Paul Angus <paul.angus@shapeblue.com>
2019-06-22 10:21:02 +01:00
Paul Angus 1235f7f881 Merge branch '4.11.3.0-RC20190610T1615' into 4.11 2019-06-22 10:18:07 +01:00
Rohit Yadav 3c431bb8e4 Merge remote-tracking branch 'origin/4.11' into 4.12 2019-06-20 14:32:57 +05:30
Richard Lawley 34255061c4 .gitignore: add .vscode to ignore (#3384)
Ignore .vscode folder
2019-06-19 16:53:25 +05:30
Rohit Yadav ef1506b0b0 Merge remote-tracking branch 'origin/4.11' into 4.12 2019-06-14 15:14:25 +05:30
Rohit Yadav 425ad495b1
kvm: fix qemu hook race condition (#3405)
This fixes the qemu hooks `mkdir` race condition which can happen when
too many VMs may launch on a KVM host executing the hooks script that
tries to `mkdir` for the custom directory. On exception (multiple scripts
trying to mkdir), the VM stops.

The custom directory need not be created if it does not exist, instead
the custom hooks should only execute when there is a custom directory.

Feature documentation:
http://docs.cloudstack.apache.org/en/4.11.2.0/adminguide/hosts.html#kvm-libvirt-hook-script-include

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-14 12:55:04 +05:30
Paul Angus 51124b7b35 Updating pom.xml version numbers for release 4.11.3.0
Signed-off-by: Paul Angus <paul.angus@shapeblue.com>
2019-06-10 16:15:05 +01:00
Rohit Yadav a91a88bbc5 Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-10 18:18:42 +05:30
Rohit Yadav a3e5664b53
schema: add 4.11.2 to 4.11.3 systemvmtemplate upgrade path (#3381)
Add 4.11.2 to 4.11.3 systemvmtemplate upgrade path

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-07 23:03:05 +05:30
Rohit Yadav 73fd62a89d Merge remote-tracking branch 'origin/4.11' into 4.12 2019-06-07 09:35:56 +05:30
Nicolas Vazquez d108f322e3
KVM: Prevent regenerating keystore on provisionCertificate API (#3075)
* Prevent regenerating keystore and add background task to upload certificates

* Remove background task to upload certificates

* keystore-setup: delete old key in jks

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* also don't allow ca-cert alias

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-05 17:29:31 -03:00
Rohit Yadav ff23131701 Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-05 10:00:45 +05:30
Rohit Yadav 8fb388e931
router: support multi-homed VMs in VPC (#3373)
This does not remove VM entries in dbags when hostnames match. The
current codebase already removes entry when a VM is stopped/removed so
we don't need to handle lazy removal. This will allow a VM on
multiple-tiers in a VPC to get dns/dhcp rules as expected.

This also fixes the issue of dhcp_release based on a specific interface and
removes dhcp/dns entry when a nic is removed on a guest VM.

Fixes #3273

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-05 08:47:05 +05:30
Richard Lawley 41f569e8a8 router: Fix rule duplication with non-VPC static NAT rules (#3366)
The VR code has provision for inserting rules at the top or bottom by specifying "front" as the second parameter to self.fw.append. However, there are a number of cases where someone has been unaware of this and added a rule with the pattern self.fw.append(["mangle", "", "-I PREROUTING".... This causes the code to check for the rule already being present to fail, and duplicate rules end up being added.

This PR fixes two of these cases which apply to adding static NAT rules. I am aware of more of these cases, but I don't have the ability to easily test the outcome of fixing them. I'm happy to add these in if you're confident that the automated tests will be sufficient. Searching for "-I (case sensitive) finds these.

The code for dealing with "front" is included below to show that this shouldn't have any ill effects:

if fw[1] == "front":
    cpy = cpy.replace('-A', '-I')

Fixes #3177
2019-06-05 02:21:03 +05:30
nvazquez bea627a52e Merge branch '4.11' into 4.12 2019-06-04 09:06:09 -03:00
Vladimir Melnik c94ee1454d kvm: suspend a VM before snapshot deletion (see PR #3193) (#3194)
To make sure that a qemu2-image won't be corrupted by the snapshot deletion procedure which is being performed after copying the snapshot to a secondary store, I'd propose to put a VM in to suspended state.

Additional reference: https://bugzilla.redhat.com/show_bug.cgi?id=920020#c5

Fixes #3193
2019-06-04 16:04:45 +05:30
Rohit Yadav 42501ceecf
ssvm: apply MTU value on storage/management nic if available (#3370)
If mtu= value is defined in the parameters received by the SSVM agent
per the secstorage.vm.mtu.size setting, it applies the MTU setting on
eth1 which is the storage/management nic.

Fixes #3369

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-04 15:53:51 +05:30
Nicolas Vazquez 12c850ed2f
KVM: Improvements on upload direct download certificates (#2995)
* Improvements on upload direct download certificates

* Move upload direct download certificate logic to KVM plugin

* Extend unit test certificate expiration days

* Add marvin tests and command to revoke certificates

* Review comments

* Do not include revoke certificates API
2019-06-04 03:08:31 -03:00
Rohit Yadav 3f2399f107 server: Fix forward merge missing import causing build failure
This fixes forward merge regression that missed an import and causes
build failure in b2b99ca63e

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-03 18:55:25 +05:30
Rohit Yadav b2b99ca63e Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-03 17:15:41 +05:30
Nicolas Vazquez c9ce3e2344 router: Persistent DHCP leases file on VRs and cleanup /etc/hosts on VM deletion (#3351)
Since the CloudStack virtual router was redesigned on version 4.6 it has been observed that the DHCP leases file is not persistent across network operations. This causes conflicts on guest VMs static IPs, causing these static IPs to not be renewed by the DHCP server running on isolated and VPC networks' virtual routers (dnsmasq). On stopping or destroying a VM, its dhcp/dns records are not removed from the virtual router causing ghost effects.

Fixes #3272
Fixes #3354

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-03 17:04:16 +05:30
Richard Lawley 2484527cae srx: Fix removing static NAT rules with Juniper SRX (#3310)
Fixed the logic for deleting static NAT rules on a Juniper SRX device. Previously the private (trust) rule was not being removed.

Fixes #3309
2019-06-03 16:51:23 +05:30
Rohit Yadav fb555b11ae Merge remote-tracking branch 'origin/4.11' into 4.12 2019-05-31 12:36:45 +05:30
Richard Lawley 2f268fbb52 systemvm: fix VR issues with Multiple Public Subnets (#3361)
This PR resolves 2 issues related to Virtual Routers with multiple public interfaces, and works around a third.

- Fixes #3353 - Adds missing throw routes for eth0/eth1 to eth3+ when there are >1 public IPs
- Fixes #3168 - Incorrect marks set on some static NAT rules (some code references were changed from hex(int(interfacenum)) to hex(100 + int(interfacenum)) - this change just adds the remaining ones
- Fixes #3352 - Work around that sends Gratuitous ARP messages when a HA VR becomes master to work around the problem of the MAC address being different between HA VRs. If that issue is fixed properly (i.e. a database entry for the subsequent interfaces so they can be static) then this is unnecessary, though should not cause any problems.
2019-05-31 12:35:42 +05:30
Rohit Yadav 3737b41dbc Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-31 12:31:27 +05:30
ustcweizhou b8522c97cb server: allow dedicate ip range to a domain if ips are used by an accout in the domain (#3206)
when we dedicate public ip range to a domain but some ips are used by an account in the domain,
the operation should be allowed but actually fails for now.
It is because cloudstack check if ips are used by same account by account name,
However, accountName is null when dedicate public ip range to a domain.

Modify the code to check account id only when dedicate ip range to account.
2019-05-31 12:24:33 +05:30
ustcweizhou bd78030385 server: update dhcp configurations in vrs while update default nic of running vms (#3205)
In virtual routers, there are different dnsmasq settings for default nic and non-default nic on vm.
We need to update dhcp informations on network vrs when default nic is changed.

For example, if 172.16.1.135 is non-default nic of vm VPC1-001-001, then

root@r-22-VM:~# cat /etc/dhcphosts.txt
02:00:1d:15:00:05,set:172_16_1_135,172.16.1.135,VPC1-001-001,710h
root@r-22-VM:~# cat /etc/dhcpopts.txt
172_16_1_135,3
172_16_1_135,6
172_16_1_135,15

If it is default nic,then

root@r-22-VM:~# cat /etc/dhcpopts.txt
root@r-22-VM:~# cat /etc/dhcphosts.txt
02:00:1d:15:00:05,172.16.1.135,VPC1-001-001,757h

Fixes #3201
2019-05-31 12:23:55 +05:30
Rohit Yadav 8c387f9de6
vmware: fix potential NPE when memory hotplug capability is checked (#3362)
This fixes potential NPE case when memory hotpluggability is checked
based on the guest OS descriptor.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-31 10:27:36 +05:30
Rohit Yadav 7ef921de16 Merge remote-tracking branch 'origin/4.11' into 4.12 2019-05-30 12:30:54 +05:30
ustcweizhou f6f381fc68 ui: fix enable static nat only towards first nic and not on any other interface (#3338)
When enable static nat in a vpc on UI, it only lists the primary and secondary ips of first nic of a vm, no matter which vpc tier is selected. The same issue happens when add a vm to load balancer.

Fixes #3334
2019-05-30 11:44:52 +05:30
Rohit Yadav 31e677f0fa Merge remote-tracking branch 'origin/4.11' into 4.12 2019-05-30 11:40:19 +05:30
smlshn f1efcc1af6 ui: reset multiselect actions when refreshing listView in Instance page (#3359)
Enables the toolbar to reset to its initial state after any multiSelectAction completed.

Fixes #3337
2019-05-30 11:38:14 +05:30
ustcweizhou 8e43d258f3 server: Fail to restart VPC with cleanup if there are multiple public IPs in different subnet" (#3342)
If there are multiple IPs in different subnet assigned to a VPC, after restarting VPC with cleanup, the VRs will be FAULT state.

Step to reproduce:
(1) create vpc, source nat IP is 10.11.118.X
(2) assign two public IPs in other subnet to this VPC. 10.11.119.X and 10.11.119.Y
(3) deploy two vms in the vpc, and enable static nat 10.11.119.X and 10.11.119.Y to these two vms
(4) restart vpc with cleanup. There are more than 1 nic allocated for 10.11.119 to new VRs

Logs as below:
2019-05-10 14:12:24,652 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-36:ctx-839f6522 job-652 ctx-35fb4667) (logid:1ab7aa37) Allocating nic for vm VM[DomainRouter|r-85-VM] in network Ntwk[200|Public|1] with requested profile NicProfile[0-0-null-10.11.118.157-vlan://untagged
2019-05-10 14:12:24,676 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-36:ctx-839f6522 job-652 ctx-35fb4667) (logid:1ab7aa37) Allocating nic for vm VM[DomainRouter|r-85-VM] in network Ntwk[200|Public|1] with requested profile NicProfile[0-0-null-10.11.119.110-vlan://119
2019-05-10 14:12:24,699 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-36:ctx-839f6522 job-652 ctx-35fb4667) (logid:1ab7aa37) Allocating nic for vm VM[DomainRouter|r-85-VM] in network Ntwk[200|Public|1] with requested profile NicProfile[0-0-null-10.11.119.110-vlan://119
2019-05-10 14:12:24,723 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-36:ctx-839f6522 job-652 ctx-35fb4667) (logid:1ab7aa37) Allocating nic for vm VM[DomainRouter|r-85-VM] in network Ntwk[200|Public|1] with requested profile NicProfile[0-0-null-10.11.119.110-vlan://119

This is a regression issue caused by commit 1d382e0
2019-05-30 11:33:03 +05:30
dahn 910b08f72b server: fix duplicate tag exception as CloudRuntimeException (#3348)
See #3339: a runtime exception is thrown but it should be converted to an error return. Wrapping it in a CloudRuntimeException should do the trick.

Fixes #3339
2019-05-30 11:25:52 +05:30
Rohit Yadav d77e69a2f2 Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-23 18:16:45 +05:30
Rohit Yadav 0929866956
server: ssh-keygen in PEM format and reduce main systemvm patching script (#3333)
On first startup, the management server creates and saves a random
ssh keypair using ssh-keygen in the database. The command does
not specify keys in PEM format which is not the default as generated
by latest ssh-keygen tool.

The systemvmtemplate always needs re-building whenever there is a change
in the cloud-early-config file. This also tries to fix that by introducing a
stage 2 bootstrap.sh where the changes specific to hypervisor detection
etc are refactored/moved. The initial cloud-early-config only patches
before the other scripts are called.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-23 18:08:00 +05:30