Commit Graph

34084 Commits

Author SHA1 Message Date
Rohit Yadav 3950de116d Merge remote-tracking branch 'origin/4.13' 2020-02-19 13:13:04 +05:30
Wei Zhou 37d2b8537c
kvm: Enable virtio drivers based on guest os display name (#3879)
When we add new guest os, sometimes we missed the records in guest_os_hypervisor.
However, the guest disk model (virtio/ide) is determined by record in the table.
It causes the issue that some new guest os(eg Debian 8/9) uses e1000 instead of virtio nic, and ide disk instead of virtio disk.

To fix the issue permanantly, pass the guest os name in guest_os if the record for kvm is not found in guest_os_hypervisor.

Related commit:7ac9f00eeeb4cd37ec39efeba066e799b581b1a0
2020-02-19 13:12:20 +05:30
Rakesh 4ab6b42250
server: Add new command to update security group name (#3739)
By default, once we create a security group we cant change its name.
In this feature, we introduce a new API command "updateSecurityGroup"
which allows us to rename the security group name. Although we can't
change the name of the "default" security group.
2020-02-19 13:09:52 +05:30
Wei Zhou 649ed45965
kvm: fix exception in volume statts after storage migration (#3884)
On kvm, the 'path' of volume is the file name on primary storage. we should use 'path' instead of 'uuid' in volume statistics.

Fixes: #3878
2020-02-19 13:06:19 +05:30
Rohit Yadav 0c46dfa64a Merge remote-tracking branch 'origin/4.13' 2020-02-18 14:21:36 +05:30
Andrija Panic 77fc1026bb
engine/schema: remove duplicate index region (#3882)
Remove duplicate index region.
2020-02-18 14:10:51 +05:30
Rakesh bbe2bf1a6e
server: ignore site to site vpn status check on internallbvm (#3864)
When the state of the site to site vpn changes, the check
is done on all the virtual routers including the internal
load balancing vm as well. It is not needed to check the
state for internal load balancing vm
2020-02-18 14:03:30 +05:30
Rohit Yadav 5a42862d41 Merge remote-tracking branch 'origin/4.13' 2020-02-13 11:33:40 +05:30
Wei Zhou e5efb59414
simulator: fix travis failure after merging volume destroy/recover (#3880)
* test: do not associate public ip in test while deploy vm

* simulator: remove template/volume only if it exist

* simulator: fix build error
2020-02-13 09:58:31 +05:30
Spaceman1984 a443f31b78
jobs: fixed duplicate id error when creating VM work jobs (#3871)
This PR removes related VMWorkjob records when the async_job table is cleaned out.

Fixes: #3829
2020-02-12 19:06:01 +05:30
Spaceman1984 5ae65bfab0
ui: Fixed root volume resize from ui (#3873)
This PR fixes the volume resize action for a root volume from the UI.

Fixes: #3860 #3874
2020-02-12 19:03:14 +05:30
Jonathan Leitschuh d46f428757
maven: Use HTTPS instead of HTTP to resolve dependencies (#3877)
This fixes a security vulnerability in this project where the `pom.xml`
files were configuring Maven to resolve dependencies over HTTP instead of
HTTPS.

Signed-off-by: Jonathan Leitschuh <Jonathan.Leitschuh@gmail.com>
2020-02-12 18:57:26 +05:30
Rohit Yadav d90341ebf1
cloudstack: add JDK11 support (#3601)
This adds support for JDK11 in CloudStack 4.14+:

- Fixes code to build against JDK11
- Bump to Debian 9 systemvmtemplate with openjdk-11
- Fix Travis to run smoketests against openjdk-11
- Use maven provided jdk11 compatible mysql-connector-java
- Remove old agent init.d scripts

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-02-12 12:58:25 +05:30
Rohit Yadav ccda5fb776 Merge remote-tracking branch 'origin/4.13' 2020-02-10 19:02:09 +05:30
Rohit Yadav 78cc0a44c1
server: use host record related to a ssvm/cpvm (#3876)
This implements the systemvm list API response creator to find and use
the host record for a ssvm/cpvm to get the agent status and other
details like last disconnected date and agent version.

Fixes 3875

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-02-10 19:01:34 +05:30
Nicolas Vazquez ce896a477d
[Vmware] Enable PVLAN support on L2 networks (#3732)
* Enable PVLAN support on L2 networks

* Fix prevent null pointer on details

* Add marvin tests

* Fixes from comments

* Fix: missing pvlan type on plugniccommand

* Fix checks on network creation for vlans overlap

* Fix remove prefix from secondary vlan id

* Improve checks on physical network for pvlans

* Fix compatibility with previous pvlan creation

* Fix shared networks backwards pvlan compatibility

* Add ui fix for pvlan type not passed to api

* Add check for isolated vlan id overlap

* Include check for dynamic vlan reserved for secondary vlan

* Fix marvin tests errors

* Fix redundant imports

* Skip marvin test for pvlan if dvswitch is not present

* spelling

Co-authored-by: Andrija Panic <45762285+andrijapanicsb@users.noreply.github.com>
2020-02-07 15:43:01 +01:00
Rohit Yadav 70d1535df4 Merge remote-tracking branch 'origin/4.13' 2020-02-07 18:05:39 +05:30
Rohit Yadav afcbbc4b3e
systemvm: list systemvm does not return agent state and version (#3870)
This makes the listSystemVms API to return the host status (agent state),
version and last pinged information. This makes it possible for UIs
to call a single API to get this information.
2020-02-07 13:19:35 +01:00
Wei Zhou fd5bea838b
New feature: Add support to destroy/recover volumes (#3688)
* server: fix resource count of primary storage if some volumes are Expunged but not removed

Steps to reproduce the issue
(1) create a vm and stop it. check resource count of primary storage
(2) download volume. resource count of primary storage is not changed.
(3) expunge the vm, the volume will be Expunged state as there is a volume snapshot on secondary storage. The resource count of primary storage decreased.
(4) update resource count of the account (or domain), the resource count of primary storage is reset to the value in step (2).

* New feature: Add support to destroy/recover volumes

* Add integration test for volume destroy/recover

* marvin: check resource count of more types

* messages translate to JP

* Update messages for CN

* translate message for NL

* fix two issues per Daan's comments

Co-authored-by: Andrija Panic <45762285+andrijapanicsb@users.noreply.github.com>
2020-02-07 11:25:10 +01:00
Rohit Yadav d8906d3d8b Merge remote-tracking branch 'origin/4.13' 2020-02-07 11:18:58 +05:30
Wei Zhou cd55674afe packaging: install python-dnspython or python-dns to fix issue with cloudstack-setup-management (#3854)
Fixes #3817 #3841

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-02-07 11:18:00 +05:30
Spaceman1984 f769c66bc1 ui: fix default text missing from network selection on instance wizard (#3865)
This PR fixes the missing "Default" display issue on the instance wizard when selecting a network.

Fixes: #3633
2020-02-07 11:17:38 +05:30
Rohit Yadav 493415d91b
packaging: install python-dnspython or python-dns to fix issue with cloudstack-setup-management (#3854) (#3869)
Fixes #3817 #3841

Co-authored-by: Wei Zhou <ustcweizhou@gmail.com>
2020-02-07 11:13:57 +05:30
Rakesh c6e67c482c
ui: display drop down list for VPN customer gateway selection (#3805)
Display drop-down list for VPN customer gateway selection in UI
so that it can be searched according to domains
2020-02-07 11:10:56 +05:30
Bitworks LLC f78cbf4efc
ssvm: wrong SSVM behavior causes redownloading for all the templates (#3844)
As per discussion at #3838 and proposal by @weizhouapache this PR implements the fix.

Fixes #3838
2020-02-07 11:09:47 +05:30
Spaceman1984 a746b29a83
ui: fix default text missing from network selection on instance wizard (#3865)
This PR fixes the missing "Default" display issue on the instance wizard when selecting a network.

Fixes: #3633
2020-02-07 11:04:20 +05:30
Daan Hoogland 10482da136 Merge release branch 4.13 to master
* 4.13:
  vr: add missing rule for port forwarding rule in vpc (#3857)
  vpc: set traffic type of private gateway IP to Public to fix ke… (#3851)
2020-02-06 20:38:07 +01:00
Wei Zhou d88c614a35
vr: add missing rule for port forwarding rule in vpc (#3857) 2020-02-06 20:25:56 +01:00
Wei Zhou a9a1737dd9
vpc: set traffic type of private gateway IP to Public to fix ke… (#3851) 2020-02-06 20:22:08 +01:00
Rohit Yadav 33e9a50945 Merge remote-tracking branch 'origin/4.13' 2020-02-06 11:32:09 +05:30
Pearl Dsilva bfdb914693
usage: publish zone id while uploading template and volume (#3867)
After a local template is uploaded via browser, the generated usage event with type = "TEMPLATE.CREATE" is persisted with the data store ID instead of the zone ID on the zone_id column. The fix will refactor the upload monitor logic, as after the upload completes, it sets the datastore ID on the zone ID column for the created "TEMPLATE.CREATE" usage event. This refactor will query the DB for the data store and will set its associated zone ID in the usage field.
The fix produces the same behaviour as when registering a template from URL.
FIx is also for uploading VOLUME from local/via browser.
2020-02-06 11:31:24 +05:30
Wei Zhou c22e99c638
test: check more connectivity in test_privategw_acl.py (#3861) 2020-02-05 12:15:22 +01:00
Rakesh 107bba3da1
router: start all Site-to-Site VPN connections when VPC VR starts (#3863)
When we restart the VPC after destroying the master VR, the backup VR
becomes the master and the site to site connections are not in
the connected state. The passive VPN connection will be in the connected
state but the active VPN connection will be in a disconnected state.
2020-02-05 11:56:47 +05:30
Rakesh 70daee9b10 network: set restart_required to 0 after restarting network (#3803)
After restarting the network with or without cleanup option, the restart_required field in networks table should be reset to 0.
2020-02-04 14:42:46 +01:00
Abhishek Kumar 0f5b0e67f8
VM ingestion (#3606)
The VM ingestion feature allows CloudStack to discover, on-board, import existing VMs in an infra. The feature currently works only for VMware, with a hypervisor agnostic framework which may be extended for KVM and XenServer in future.
2020-02-03 15:43:52 +01:00
Rakesh 6d014fa219
network: de-associate IP address if enabling static nat fails (#3836)
Associating static NAT on IP to VM fails even though the IP is not allocated.
When we try enable static NAT on second IP address to the same VM, the operation fails but the IP address is still allocated in the db and it can't be used to enable static NAT on different VM.

Steps to reproduce the issue:

(1) create a vpc (vpc-001) and a vpc tier (vpc-001-001)

(2) create a vm (vm-001-001) in vpc-001-001

(3) acquire a public ip (ip-1) and enable static nat to vm-001-001,
operation succeeds.

(4) acquire a public ip (ip-2) and enable static nat to vm-001-001,
operation fails but the ip is still assigned to vpc tier vpc-001-001.
Note down the ip address and the id of it.

(5) create another vpc tier vpc-001-002, and vm (vm-001-002) in the tier

(6) enabled ip-2 static nat to vm-001-002, operation should succeed
2020-01-31 16:27:20 +05:30
Rakesh a2a4968f51
server: Allow creating network with duplicate name (#3807)
Add a global setting to disable creating networks with same name in an account

Add a global setting to disable creating network without
mentioning the start and end IPv4 or IPv6 address

By default we can create networks with the same name in the account.
Sometimes we should not create the networks with same name.
This change adds a global setting which prevents creating the network with same name.
The default value is true and set it to false to prevent creating network with same names.

Also its possible to create a shared network without mentioning the
start and the end IPv4 or IPv6 address.
This change adds a global setting which prevents creating a shared
network without specifying the start and the end IPv4 or IPv6 address
2020-01-31 15:52:42 +05:30
Rakesh 1a5b7c362e
engine/orchestration: display numeric value instead of variable name (#3818)
If the disk size of the vm to be created is greater
than the volume size, then the exception message should
display the numeric value instead of variable name
2020-01-31 15:42:06 +05:30
Rohit Yadav f72f39cd11 Merge remote-tracking branch 'origin/4.13' 2020-01-31 15:28:34 +05:30
Abhishek Kumar a71874682c
server: fix checking disk offering access for snapshot volume (#3791)
Fixes #3783
As reported in the issue, creating volumes from pure snapshot fails with NPE. This is due to order of calls where disk offering access is checked before checking disk offering value. This PR fixes the same.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-01-31 15:24:25 +05:30
Wei Zhou 75aa6e3659
ui: fix cannot assign vms to internal lb in VPC (#3832)
This fixes the issue that cannot assign VMs in Internal LB in VPC.
listView is overwritten due to commit 2e2b670 (#3161)

Fixes: #3827
2020-01-31 15:23:11 +05:30
Wei Zhou 76fb811e08
kvm: Fix router migration issue when router has control/public nics on other physical network than guest (#3855)
In VM migration on KVM, libvirt qemu hook script will change the bridge name to bridges for guest networks. It works for user vm. However for virtual router, it has nics on control network and public network. If control/public use different physical networks than guest network, virtual router cannot be migrated.

Fixes: #2783
2020-01-31 15:14:35 +05:30
Rohit Yadav 424f10cc77 Merge remote-tracking branch 'origin/4.13' 2020-01-31 14:18:11 +05:30
Abhishek Kumar 9d105b6546
template: copy md5 mismatch (#3383)
Fixes #3191

When a template is registered, code stores md5sum of the downloaded file in the vm_template table. However, this downloaded file could be deleted after template installation if it is not an actual (.qcow2, .ova, etc.) file. When the user copies a template using copyTemplate API, the actual template file will be copied across the image stores. Matching checksum for the copied templated file and the stored value from the vm_template table will result in a mismatch.
Changes will set an empty checksum value for the copied template while passing to download service which allows skipping wrong checksum check for the copied while install.
However, this results in a change in checksum value for concerned template entry in vm_template table post template install.

Co-authored-by: dahn <daan.hoogland@gmail.com>
2020-01-31 14:16:37 +05:30
Daan Hoogland f5551be549 Merge branch '4.13' 2020-01-31 08:40:30 +01:00
Sid Kattoju 6baa598033
Clean up inactive iscsi sessions when VMs get moved due to crashes (#3819) 2020-01-30 19:52:28 +01:00
Anurag Awasthi c0abfce8fa
Health check feature for virtual router (#3575) 2020-01-30 12:39:03 +01:00
Xia Li f1149bcb09
[CLOUDSTACK-10408] Fix String.replaceAll() to replace() for bet… (#3275)
* [CLOUDSTACK-10408] Fix String.replaceAll() to replace() for better performance

* improve with replace char but string

Co-authored-by: Rohit Yadav <rohit@apache.org>
2020-01-30 12:06:15 +01:00
Daan Hoogland 99ec8a825a Merge release branch 4.13 to master
* 4.13:
  Fix Policy Based Routing for private gateway static routes (#3604)
2020-01-30 11:39:36 +01:00
Dennis Konrad 82d94a87c5
Fix Policy Based Routing for private gateway static routes (#3604)
* Fix for routing table issue with NAT interfaces

* Mark only packets with the public ip as destination
2020-01-30 11:31:30 +01:00