Commit Graph

31932 Commits

Author SHA1 Message Date
nvazquez 50869fec92 Merge branch '4.12' 2019-06-04 09:49:39 -03:00
nvazquez bea627a52e Merge branch '4.11' into 4.12 2019-06-04 09:06:09 -03:00
Vladimir Melnik c94ee1454d kvm: suspend a VM before snapshot deletion (see PR #3193) (#3194)
To make sure that a qemu2-image won't be corrupted by the snapshot deletion procedure which is being performed after copying the snapshot to a secondary store, I'd propose to put a VM in to suspended state.

Additional reference: https://bugzilla.redhat.com/show_bug.cgi?id=920020#c5

Fixes #3193
2019-06-04 16:04:45 +05:30
Rohit Yadav 42501ceecf
ssvm: apply MTU value on storage/management nic if available (#3370)
If mtu= value is defined in the parameters received by the SSVM agent
per the secstorage.vm.mtu.size setting, it applies the MTU setting on
eth1 which is the storage/management nic.

Fixes #3369

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-04 15:53:51 +05:30
Nicolas Vazquez 12c850ed2f
KVM: Improvements on upload direct download certificates (#2995)
* Improvements on upload direct download certificates

* Move upload direct download certificate logic to KVM plugin

* Extend unit test certificate expiration days

* Add marvin tests and command to revoke certificates

* Review comments

* Do not include revoke certificates API
2019-06-04 03:08:31 -03:00
Rohit Yadav 3e4c55c424 Merge remote-tracking branch 'origin/4.12' 2019-06-03 18:56:54 +05:30
Rohit Yadav 3f2399f107 server: Fix forward merge missing import causing build failure
This fixes forward merge regression that missed an import and causes
build failure in b2b99ca63e

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-03 18:55:25 +05:30
Rohit Yadav cd6e84659e Revert "Fix template size for managed storage / refactor cloud-install-sys-tmplt and createtmplt.sh (#3346)"
This reverts commit 6946f41784 because it
adds regression to deployment and testing. Testing against VMware, KVM
and XenServer confirm failure to deploy an environment where systemvms
don't come online.
2019-06-03 17:29:43 +05:30
Rohit Yadav 1c5244a0c5 Merge remote-tracking branch 'origin/4.12' 2019-06-03 17:17:51 +05:30
Rohit Yadav b2b99ca63e Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-03 17:15:41 +05:30
Nicolas Vazquez c9ce3e2344 router: Persistent DHCP leases file on VRs and cleanup /etc/hosts on VM deletion (#3351)
Since the CloudStack virtual router was redesigned on version 4.6 it has been observed that the DHCP leases file is not persistent across network operations. This causes conflicts on guest VMs static IPs, causing these static IPs to not be renewed by the DHCP server running on isolated and VPC networks' virtual routers (dnsmasq). On stopping or destroying a VM, its dhcp/dns records are not removed from the virtual router causing ghost effects.

Fixes #3272
Fixes #3354

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-03 17:04:16 +05:30
Richard Lawley 2484527cae srx: Fix removing static NAT rules with Juniper SRX (#3310)
Fixed the logic for deleting static NAT rules on a Juniper SRX device. Previously the private (trust) rule was not being removed.

Fixes #3309
2019-06-03 16:51:23 +05:30
Sven Vogel 6946f41784 Fix template size for managed storage / refactor cloud-install-sys-tmplt and createtmplt.sh (#3346)
* refactor cloud-install-sys-tmplt and createtmplt.sh

* move qemu-img check to kvm / add more comments if qemu-img not available / set virtual size only for qcow2

* add comments to the failed and get options block / rename vars in failed block / typo Fix

*  add comments to the failed and get options block / rename vars in failed block / typo Fix to cloudtmplt.sh
2019-06-03 09:30:45 +02:00
Abhishek Kumar 6c6f40fd4d server: fix for public IP addresses filtering (#3368)
Added missing parameter in listPublicIpAddresses API.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-06-03 12:52:19 +05:30
Rohit Yadav 28d7eef21f Merge remote-tracking branch 'origin/4.12' 2019-05-31 12:37:31 +05:30
Rohit Yadav fb555b11ae Merge remote-tracking branch 'origin/4.11' into 4.12 2019-05-31 12:36:45 +05:30
Richard Lawley 2f268fbb52 systemvm: fix VR issues with Multiple Public Subnets (#3361)
This PR resolves 2 issues related to Virtual Routers with multiple public interfaces, and works around a third.

- Fixes #3353 - Adds missing throw routes for eth0/eth1 to eth3+ when there are >1 public IPs
- Fixes #3168 - Incorrect marks set on some static NAT rules (some code references were changed from hex(int(interfacenum)) to hex(100 + int(interfacenum)) - this change just adds the remaining ones
- Fixes #3352 - Work around that sends Gratuitous ARP messages when a HA VR becomes master to work around the problem of the MAC address being different between HA VRs. If that issue is fixed properly (i.e. a database entry for the subsequent interfaces so they can be static) then this is unnecessary, though should not cause any problems.
2019-05-31 12:35:42 +05:30
Rohit Yadav 4ba67b4c59 Merge remote-tracking branch 'origin/4.12' 2019-05-31 12:32:11 +05:30
Rohit Yadav 3737b41dbc Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-31 12:31:27 +05:30
ustcweizhou b8522c97cb server: allow dedicate ip range to a domain if ips are used by an accout in the domain (#3206)
when we dedicate public ip range to a domain but some ips are used by an account in the domain,
the operation should be allowed but actually fails for now.
It is because cloudstack check if ips are used by same account by account name,
However, accountName is null when dedicate public ip range to a domain.

Modify the code to check account id only when dedicate ip range to account.
2019-05-31 12:24:33 +05:30
ustcweizhou bd78030385 server: update dhcp configurations in vrs while update default nic of running vms (#3205)
In virtual routers, there are different dnsmasq settings for default nic and non-default nic on vm.
We need to update dhcp informations on network vrs when default nic is changed.

For example, if 172.16.1.135 is non-default nic of vm VPC1-001-001, then

root@r-22-VM:~# cat /etc/dhcphosts.txt
02:00:1d:15:00:05,set:172_16_1_135,172.16.1.135,VPC1-001-001,710h
root@r-22-VM:~# cat /etc/dhcpopts.txt
172_16_1_135,3
172_16_1_135,6
172_16_1_135,15

If it is default nic,then

root@r-22-VM:~# cat /etc/dhcpopts.txt
root@r-22-VM:~# cat /etc/dhcphosts.txt
02:00:1d:15:00:05,172.16.1.135,VPC1-001-001,757h

Fixes #3201
2019-05-31 12:23:55 +05:30
Rohit Yadav 8c387f9de6
vmware: fix potential NPE when memory hotplug capability is checked (#3362)
This fixes potential NPE case when memory hotpluggability is checked
based on the guest OS descriptor.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-31 10:27:36 +05:30
Andrija Panic fd4309cc3d server: increase POST timeout for local template upload (#3356)
Increase template upload POST timeout for local template upload to 1 hr or 60 minutes.
2019-05-31 09:32:59 +05:30
Andrija Panic 964430ab23 vmware: update vmware reservations settings description (#3358)
We do NOT always reserve VMware CPU/RAM resources - only when "vmware.reserve.cpu" or "vmware.reserve.mem" setting is set to TRUE - AND we do so, irrelevant if overprovisioning is active or not. Verified for both system VMs and user VMs.
2019-05-30 16:17:48 +05:30
Anurag Awasthi 3bf4e5c498 ui: configurable branding, keyboard list and hide-able columns through a new config.js file (#3258)
We want to support hiding table columns, specifically in metrics table, through config file so that users can make the relevant bits hidden as per their organization. Current work will support the metrics table but can be extended to any table with minimal work in future.

Config file will take the key of the metrics column from metrics.js file for the sake of minimal changes and simplicity of development.

Problem: The keyboard list in the UI is not consistent across views such as in the instance wizard and in the register template form. There is also no way to custom about url/text and doc title and help URL in the UI.
Root Cause: The list is hardcoded in the UI allowing no centralised configuration.
Solution: Introduce a new config.js file installed at the /usr/share/cloudstackmanagement/webapp/config.js location. The config.js allows configurable keyboard list, about url/text, doc title, and help URL.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-30 15:32:37 +05:30
Rohit Yadav 0f4b1f511f Merge remote-tracking branch 'origin/4.12' 2019-05-30 12:31:26 +05:30
Rohit Yadav 7ef921de16 Merge remote-tracking branch 'origin/4.11' into 4.12 2019-05-30 12:30:54 +05:30
ustcweizhou f6f381fc68 ui: fix enable static nat only towards first nic and not on any other interface (#3338)
When enable static nat in a vpc on UI, it only lists the primary and secondary ips of first nic of a vm, no matter which vpc tier is selected. The same issue happens when add a vm to load balancer.

Fixes #3334
2019-05-30 11:44:52 +05:30
Rohit Yadav c072dda88e Merge remote-tracking branch 'origin/4.12' 2019-05-30 11:41:38 +05:30
Rohit Yadav 31e677f0fa Merge remote-tracking branch 'origin/4.11' into 4.12 2019-05-30 11:40:19 +05:30
smlshn f1efcc1af6 ui: reset multiselect actions when refreshing listView in Instance page (#3359)
Enables the toolbar to reset to its initial state after any multiSelectAction completed.

Fixes #3337
2019-05-30 11:38:14 +05:30
ustcweizhou 8e43d258f3 server: Fail to restart VPC with cleanup if there are multiple public IPs in different subnet" (#3342)
If there are multiple IPs in different subnet assigned to a VPC, after restarting VPC with cleanup, the VRs will be FAULT state.

Step to reproduce:
(1) create vpc, source nat IP is 10.11.118.X
(2) assign two public IPs in other subnet to this VPC. 10.11.119.X and 10.11.119.Y
(3) deploy two vms in the vpc, and enable static nat 10.11.119.X and 10.11.119.Y to these two vms
(4) restart vpc with cleanup. There are more than 1 nic allocated for 10.11.119 to new VRs

Logs as below:
2019-05-10 14:12:24,652 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-36:ctx-839f6522 job-652 ctx-35fb4667) (logid:1ab7aa37) Allocating nic for vm VM[DomainRouter|r-85-VM] in network Ntwk[200|Public|1] with requested profile NicProfile[0-0-null-10.11.118.157-vlan://untagged
2019-05-10 14:12:24,676 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-36:ctx-839f6522 job-652 ctx-35fb4667) (logid:1ab7aa37) Allocating nic for vm VM[DomainRouter|r-85-VM] in network Ntwk[200|Public|1] with requested profile NicProfile[0-0-null-10.11.119.110-vlan://119
2019-05-10 14:12:24,699 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-36:ctx-839f6522 job-652 ctx-35fb4667) (logid:1ab7aa37) Allocating nic for vm VM[DomainRouter|r-85-VM] in network Ntwk[200|Public|1] with requested profile NicProfile[0-0-null-10.11.119.110-vlan://119
2019-05-10 14:12:24,723 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-36:ctx-839f6522 job-652 ctx-35fb4667) (logid:1ab7aa37) Allocating nic for vm VM[DomainRouter|r-85-VM] in network Ntwk[200|Public|1] with requested profile NicProfile[0-0-null-10.11.119.110-vlan://119

This is a regression issue caused by commit 1d382e0
2019-05-30 11:33:03 +05:30
dahn 910b08f72b server: fix duplicate tag exception as CloudRuntimeException (#3348)
See #3339: a runtime exception is thrown but it should be converted to an error return. Wrapping it in a CloudRuntimeException should do the trick.

Fixes #3339
2019-05-30 11:25:52 +05:30
Nicolas Vazquez 501aa7cd91
DPDK vHost User mode selection (#3153)
* DPDK vHost User mode selection

* SQL text field and DPDK classes refactor

* Fix NullPointerException after refactor

* Fix unit test

* Refactor details type
2019-05-29 08:36:33 -03:00
Rohit Yadav 4e8f14975a
ui: add memory used column in instance metrics view (#3243)
This adds memory used column in the instance metrics view. Also fixes
a bug for VMware, due to which incorrect memory usage was returned.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-28 20:35:37 +05:30
Spaceman1984 de18613206 User allowed to tag project created by him (#3323) 2019-05-28 10:32:39 +02:00
ustcweizhou 798b79fa5b kvm: disable cpu features if feature starts with '-' (#3335)
When I use SandyBridge as custom cpu in my testing, vm failed to start due to following error:
```
org.libvirt.LibvirtException: unsupported configuration: guest and host CPU are not compatible: Host CPU does not provide required features: avx, xsave, aes, tsc-deadline, x2apic, pclmuldq
```

With this patch, it works with the following setting in agent.properties:
```
  guest.cpu.mode=custom
  guest.cpu.model=SandyBridge
  guest.cpu.features=-avx -xsave -aes -tsc-deadline -x2apic -pclmuldq
```

vm cpu is defined as below:
```
  <cpu mode='custom' match='exact'>
    <model fallback='allow'>SandyBridge</model>
    <feature policy='disable' name='avx'/>
    <feature policy='disable' name='xsave'/>
    <feature policy='disable' name='aes'/>
    <feature policy='disable' name='tsc-deadline'/>
    <feature policy='disable' name='x2apic'/>
    <feature policy='disable' name='pclmuldq'/>
  </cpu>
```
2019-05-27 18:43:38 +05:30
Abhishek Kumar d1090c0577 server: fix for inactive service offering for VM (#3320)
Fixes #3315
Currently, the code was allowed to change service offering for VM to a deleted or inactive service offering. Added check for it to throw an exception.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-05-27 18:27:23 +05:30
Gabriel Beims Bräscher 25c4f7fc08 kvm: Remove code that generated /var/lib/libvirt/images/null on target host (#3280)
This commit simplifies the generateDestPath method and fixes an issue where an extra file, named as 'null', was created on the target storage pool during VM local storage volume migration. Without this fix, the VM is migrated and there is no data loss; however, 193 KB is allocated for the unused file named as 'null' and the file stays on the target storage.
2019-05-27 18:15:29 +05:30
Philipp Bankonier e8c1deb25d ui: Fix ip and ip cidr column sorting in tables (#3199)
This fixes the sorting of ip addresses and ip cidrs.
Currently they get evaluated as a numeric value via parseFloat and sorted as numeric values.
For example "10.32.123.123" got parsed to 10.32.
2019-05-27 18:09:56 +05:30
Abhishek Kumar 46a32795bc ui: instance settings visibility (#3244)
This change allows instance Settings tab to be visible but inaccessible when instance is running. A warning is shown when user tries to access Settings for a running instance and tab content is greyed out.
It also allows some admin defined instance settings/details to be made static for user. User will be able to see them in instance settings tab but cannot change their values as action buttons are disabled and greyed out. This can be achieved by providing a comma-separated list details for global settings key 'user.vm.readonly.ui.details'. A new value 'readonlyuidetails' has been added in UserVMResponse for UI manipulate editing functionality of settings/details.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-05-24 12:28:15 +02:00
Sven Vogel 9d02b3679b packaging: set correct logrotate file permissions for cloudstack-agent (#3347)
Correct the permissions on cloudstack-agent logrotate file (/etc/logrotate.d/).
2019-05-24 12:43:08 +05:30
Rohit Yadav 2aa4842a94 Merge remote-tracking branch 'origin/4.12' 2019-05-23 18:17:42 +05:30
Rohit Yadav d77e69a2f2 Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-23 18:16:45 +05:30
Rohit Yadav 0929866956
server: ssh-keygen in PEM format and reduce main systemvm patching script (#3333)
On first startup, the management server creates and saves a random
ssh keypair using ssh-keygen in the database. The command does
not specify keys in PEM format which is not the default as generated
by latest ssh-keygen tool.

The systemvmtemplate always needs re-building whenever there is a change
in the cloud-early-config file. This also tries to fix that by introducing a
stage 2 bootstrap.sh where the changes specific to hypervisor detection
etc are refactored/moved. The initial cloud-early-config only patches
before the other scripts are called.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-23 18:08:00 +05:30
Daan Hoogland 8b75d61ffc Merge release branch 4.12 to master
* 4.12:
  KVM: Fix agents dont reconnect post maintenance (#3239)
2019-05-23 14:31:21 +02:00
Daan Hoogland 29918e25e3 Merge release branch 4.11 to 4.12
* 4.11:
  KVM: Fix agents dont reconnect post maintenance (#3239)
2019-05-23 14:29:41 +02:00
Nicolas Vazquez e86f671c8e KVM: Fix agents dont reconnect post maintenance (#3239)
* Keep connection alive when on maintenance

* Refactor cancel maintenance and unit tests

* Add marvin tests

* Refactor

* Changing the way we get ssh credentials

* Add check on SSH restart and improve marvin tests
2019-05-23 14:13:17 +02:00
Philipp Bankonier 62c8a1110c Fix iops values when creating a compute offering (#3345)
@mike-tutkowski @syed If there's something more that needs to be added/changed, we'll just open another PR for this. 

For now this seems to be a very straightforward fix for the UI problem with managed storage.

* Fix iops values when creating a compute offering
* Fix iops values when creating a disk offering
2019-05-23 13:25:48 +02:00
Rohit Yadav 4f356392ab vmware: don't use redundant worker VM to extract volume (#3218)
This fixes the issue that VM with VMsnapshots fails to start after
extract volume is done on a stopped VM, on VMware.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-23 13:18:29 +02:00