Commit Graph

10762 Commits

Author SHA1 Message Date
Rohit Yadav 567524a2a5 Merge remote-tracking branch 'origin/4.14' 2020-06-16 13:31:29 +05:30
Rohit Yadav b54d19b3b9
server: fix TransactionLegacy DB connection leaks due to DB switching by B&R thread (#4121)
BackupSync task would switch between databases to update backup usage
metrics in the cloud_usage.usage_backup table. The current framework
and the usage in ManagedContext causes database connection
(LegacyTransaction) leaks. When the thread runs faster, the issue is
easily reproducible and checking via heap dump analysis or using JMX
MBeans. This fixes by moving the task of backup data updation for
usage data to the usage server by publishing usage events instead of
switching between databases in a local thread while in a
ManagedContextRunnable.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-06-16 13:30:31 +05:30
Rohit Yadav 612100c84a Merge remote-tracking branch 'origin/4.14' 2020-06-16 12:23:23 +05:30
Rohit Yadav 77947f23fd Merge remote-tracking branch 'origin/4.13' into 4.14
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-06-16 12:21:48 +05:30
harikrishna-patnala 5054766d9f
server: Submitting multiple dynamic VM Scaling API commands for the same instance can result in two usage events in the same second causing a compound key violation in usage service (#3991)
Root cause:
Even though dynamic scaling job is handled in vmworkjob queue which ensures serilizing multiple jobs but the database updating and generating usage events are out of the job queue.

Solution:
Moved all updations into the job queue

Firstly I have tested all the scenarios to check if nothing is broken:
Scaling on a running VM with normal compute offering
Scaling on a stopped VM with normal compute offering
Scaling on a running VM with custom compute offering
Scaling on stopped VM with custom compute offering
Scaling on stopped/running VM between custom compute offering and normal compute offering and combinations among these. Checked if the custom parameters have been populated or deleted accordingly based on the offering to which the VM is scaled
Since this is a corner scenario I could not test the exact point where two usage events are recorded at the same time for two different API calls on same VM.
2020-06-16 11:41:14 +05:30
sureshanaparti 185d44a265
server: Check the max data volumes limit using the actual hypervisor version (#4122)
Fix to pick the max data volumes limit using the actual hypervisor version, instead of "default" version. Use the hypervisor version in the host table when product_version parameter in host details doesn't exist or is empty

Fixes #4101
2020-06-15 08:28:57 +05:30
deargopinath b4937725b0
server: Fix error messages to replace the word 'matches' with 'match' (#3982)
Fixes #3642
2020-06-15 08:27:34 +05:30
Rakesh 20f9639842
api: Display network name for IP in shared networks (#4073)
In the list publicipaddress api call, display the network
name if ip is associated to shared network

Co-authored-by: Rakesh Venkatesh <r.venkatesh@global.leaseweb.com>
2020-06-13 06:00:46 +05:30
Rakesh 1fe96cc426
server: Search VR using redundant state (#4075)
Allow VR's to be searched using its redundant state
Under Infrastructure -> Virtual Routers -> Search box
we can search using "MASTER", "BACKUP" and this will display
the VR's matching the state.

Co-authored-by: Rakesh Venkatesh <r.venkatesh@global.leaseweb.com>
2020-06-12 12:22:31 +05:30
Gabriel Beims Bräscher 6e47c49fbf
server: Fix String.format arguments (#3980)
Simple log fix. I have found out a few log messages that hold unused/misused arguments on their respective String.format building.
2020-06-12 10:28:04 +05:30
Rakesh f4f238e18e
server: Disable searching by instance name for customers (#4077)
Only admins should be able to search VM by instance name
Customers should not see or serach VM's using the instance name (i-)

Co-authored-by: Rakesh Venkatesh <r.venkatesh@global.leaseweb.com>
2020-06-11 17:22:23 +05:30
dahn 0795cd430b
Boot into hardware setup menu on Vmware (#4021) 2020-06-11 11:47:20 +00:00
Rakesh 6bf3769c09
Display image store disk size used and total disk size in (#4046)
list imagestore api response
2020-06-08 07:05:58 +05:30
Rohit Yadav 220448aa94 Merge remote-tracking branch 'origin/4.14' 2020-06-08 07:02:34 +05:30
Rohit Yadav cd63d6812f Merge remote-tracking branch 'origin/4.13' into 4.14 2020-06-08 07:01:52 +05:30
Nicolas Vazquez 056e6768a2
server: Cannot migrate VM on PVLAN shared network (#4062)
Fix casting issue.

Fixes #4061
2020-06-08 07:01:11 +05:30
Gabriel Beims Bräscher 597f3cf1de
Enhance KVM running VM snapshot exception log (#4126) 2020-06-04 08:53:49 +00:00
Artem Smotrakov 28a19311f4
server: Enable revocation checking for uploaded certificates (#4065)
This update turns on certificate revocation checking for uploaded certificates:

- Updated `CertServiceImpl` to be able to enable revocation checking.
- Introduced a new parameter `ENABLED_REVOCATION_CHECK` for `UploadSslCertCmd`.
- Updated `CertServiceTest`.

Even if no CLRs are specified via `PKIXParameters`, the certificates
themselves may still provide info for revocation checking:

- The AIA extension may contains a URL to the OCSP responder.
- The CLRDP extension contains a URL to the CLR.

Those extensions may need to be explicitly enabled by setting the system properties `com.sun.security.enableAIAcaIssuers` and `com.sun.security.enableCRLDP` to true. See [Java PKI Programmer's Guide](https://docs.oracle.com/en/java/javase/11/security/java-pki-programmers-guide.html).

Using a revoked certificate may be dangerous. One of the most common reasons why a certificate authority (CA) revokes a certificate is that the private key has been compromised. For example, the private key might have been stolen by an adversary.

If I understand correctly, the `CertServiceImpl` bean is used for operations with certificates on a load balancer. In particular, it validates a certificate chain without revocation checking while uploading a certificate. If a compromised revoked certificate is then used by the load balancer, then it may result to compromising TLS connections. However, the attacker has to be able to implement man-in-the-middle attack to compromise the connections. So the attacker has to be quite powerful. Therefore, such an attack is definitely not easy to implement. On the other hand, the impact may be significant because of loss of confidentiality.

This has been discussed on security@cloudstack.apache.org
2020-06-04 07:47:05 +05:30
Lucas Asth 630d826ce0
cleanup of redundant check for sameOwner (#4110)
Entity ownership is already being verified at line 508.
2020-05-30 13:33:39 +02:00
andrijapanicsb 5f926c3353 Updating pom.xml version numbers for release 4.15.0.0-SNAPSHOT
Signed-off-by: andrijapanicsb <andrija.panic@shapeblue.com>
2020-05-23 10:18:39 +01:00
andrijapanicsb 638a77281c Merge branch '4.14' 2020-05-23 10:14:29 +01:00
andrijapanicsb 05e9b11694 Updating pom.xml version numbers for release 4.14.1.0-SNAPSHOT
Signed-off-by: andrijapanicsb <andrija.panic@shapeblue.com>
2020-05-23 09:59:32 +01:00
davidjumani 1756b0f64a
noVNC console integration (#3967)
* Adding noVNC repo

* Adding support for noVNC

* Adding Ctl+Esc

* Removing device name from novnc header
2020-05-19 14:14:04 +02:00
andrijapanicsb 6f96b3b2b3 Updating pom.xml version numbers for release 4.14.0.0
Signed-off-by: andrijapanicsb <andrija.panic@shapeblue.com>
2020-05-11 15:03:14 +01:00
andrijapanicsb 398e685e01 Updating pom.xml version numbers for release 4.13.2.0-SNAPSHOT
Signed-off-by: andrijapanicsb <andrija.panic@shapeblue.com>
2020-04-29 12:29:12 +01:00
andrijapanicsb b2ffa3efa5 Updating pom.xml version numbers for release 4.13.1.0
Signed-off-by: andrijapanicsb <andrija.panic@shapeblue.com>
2020-04-23 19:17:09 +01:00
Daan Hoogland 8e4be6dc60 Merge branch '4.13' 2020-04-16 15:27:52 +02:00
dahn 6a72e6e9f8 do not put in default accept rules for DNS and BOOTPS 2020-04-16 15:09:51 +02:00
Sina Kashipazha 208e185714
FIX: prevent empty sshkey name. (#4023)
* FIX: prevent empty sshkey name.

* Move sshKeyName check before database access.

Co-authored-by: Sina Kashipazha <s.kashipazha@global.leaseweb.com>
2020-04-14 16:19:24 +02:00
Daan Hoogland b984184b7a Merge release branch 4.13 to master
* 4.13:
  Snapshot deletion issues (#3969)
  server: Cannot list affinity group if there are hosts dedicated… (#4025)
  server: Search zone-wide storage pool when allocation algothrim is firstfitleastconsumed (#4002)
2020-04-11 16:45:00 +02:00
Wei Zhou e0b67a4c68
server: Cannot list affinity group if there are hosts dedicated… (#4025) 2020-04-10 09:10:51 +02:00
Wei Zhou 6bf92fb136
server: Search zone-wide storage pool when allocation algothrim is firstfitleastconsumed (#4002) 2020-04-06 22:01:40 +02:00
Andrija Panic d52f3f4a6b
Update schema-41310to41400.sql (#3999)
* Update schema-41310to41400.sql

* update desc

* update the config key as well

* Update schema-41310to41400.sql (#4012)

* Update schema-41310to41400.sql

* update configkey desc
2020-04-04 14:07:14 +02:00
Daan Hoogland 66f5ecf908 Merge branch '4.13' 2020-03-28 19:30:17 +01:00
harikrishna-patnala 78fda2d163
With basic zone and VMware hypervisor, VR fails to start since eth1 is getting empty instead of a private IP. (#3977)
Though VMware does not support security groups, but in a basic zone with VMware and no isolation VMs should be able to deploy.

Root cause:
In case of VMware and basic zone control nic is set to 0.0.0.0 assuming control network will be shared with guest network.
But to have access to VMware instances management/private needs to be assigned to it.

Solution:
Assing a private ip even in case of basic zone VMware.
2020-03-27 19:46:01 +01:00
pavanaravapalli 08699aa690
NPE fix for System VM's start Command (#3985)
Co-authored-by: Pavan Kumar Aravapalli <pavan_aravapalli@accelerite.com>
2020-03-25 10:01:13 +01:00
Rohit Yadav db2e2136dd Merge remote-tracking branch 'origin/4.13' 2020-03-25 12:54:21 +05:30
Rohit Yadav 2e3390f06e
server: export full response view for zones response when caller is root admin (#3989)
The listZonesMetrics does not return same keys are listZones as the
default response view is restricted. This fixes that by ensuring that
for root admin full response view is used.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-03-25 12:53:41 +05:30
Rohit Yadav 36efbfcfd1 Merge remote-tracking branch 'origin/4.13' 2020-03-16 15:28:39 +05:30
Arthur Halet 3575f5ed52
vrouter in redundant mode acquire guest ips from first ip of th… (#3587) 2020-03-14 09:22:48 +01:00
pavanaravapalli d4b537efa7
UEFI Implementation: Enabled UEFI Support for Guest VM's on Hypervisor KVM,VMware. enabled boot modes [Legacy,Secure] support for UEFI boot with known caveats. (#3638)
Co-authored-by: Pavan Kumar Aravapalli <pavan_aravapalli@accelerite.com>
Co-authored-by: dahn <daan.hoogland@shapeblue.com>
2020-03-13 20:56:26 +01:00
Nicolas Vazquez b9492807fd
Rename max.retries setting (#3960) 2020-03-13 19:57:45 +01:00
Gabriel Beims Bräscher cd6f0cb1e1
Prevent overflow on StatsCollector + add a few enhancements on code (#3932) 2020-03-13 19:51:12 +01:00
Gabriel Beims Bräscher 4ca69ac152
Validate disk offering IOPS normal and maximum read/write values (#3681)
* Validate API IOPS normal and maximum read/write values.

Ensures that normal read/write cannot be greater than Maximum
read/write. Additionally, it was added a global settings
'iops.maximum.rate.length'.

'iops.maximum.rate.length' sets the maximum IOPS read/write length
(seconds) accepted; thus, preventing irrealistic values for a disk
offering (e.g. hours or days of burst IOPS). The default value is 0
(zero) and allows any IOPS maximum rate length. Example:
iops.maximum.rate.length = 3600 sets the maximum IOPS length
accepted for a disk offering as 3600 seconds (60 minutes).

* Fix log String.format message from %s to %d

* Add bytes rate validation

* Refactoring to cover Read/Write Bytes and IOPS length validation

* Fix "copy-paste" issue with bytes write rate max length
2020-03-13 19:48:45 +01:00
Nicolas Vazquez efe00aa7e0
[KVM] Rolling maintenance (#3610) 2020-03-12 16:59:46 +01:00
Wei Zhou 19fb23781b
server: password is not displayed when reinstall a vm or reset… (#3948) 2020-03-12 11:14:34 +01:00
Radu Todirica d549f3bdc8
Add cache mode param properly (#3925) 2020-03-09 13:17:21 +01:00
Pearl Dsilva 51794deb62
Regression Fix: Allow full response view to Admin user (#3940)
Change Response view to Full for Admin user

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-03-07 02:40:33 +05:30
Nicolas Vazquez 73122fd0a9
[KVM] Direct download agnostic of the storage provider (#3828)
* Remove constraint for NFS storage

* Add new property on agent.properties

* Add free disk space on the host prior template download

* Add unit tests for the free space check

* Fix free space check - retrieve avaiable size in bytes

* Update default location for direct download

* Improve the method to retrieve hosts to retry on depending on the destination pool type and scope

* Verify location for temporary download exists before checking free space

* In progress - refactor and extension

* Refactor and fix

* Last fixes and marvin tests

* Remove unused test file

* Improve logging

* Change default path for direct download

* Fix upload certificate

* Fix ISO failure after retry

* Fix metalink filename mismatch error

* Fix iso direct download

* Fix for direct download ISOs on local storage and shared mount point

* Last fix iso

* Fix VM migration with ISO

* Refactor volume migration to remove secondary storage intermediate

* Fix simulator issue
2020-03-06 19:56:54 +01:00
Daan Hoogland dc225de811 Merge release branch 4.13 to master
* 4.13:
  server: fix database exception while searching network offerings (#3947)
2020-03-06 10:44:43 +01:00
Rohit Yadav 0fab5e8d60
server: fix database exception while searching network offerings (#3947)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-03-06 10:34:09 +01:00
Rohit Yadav 58cf300fb6 Merge remote-tracking branch 'origin/4.13' 2020-03-06 14:22:46 +05:30
Nicolas Vazquez bd7d41bf6d
server: fix VM with ISO attached migration issue (#3935)
As previously described by PR #3929:
If vm has attached ISO, the migration fails with error message "org.libvirt.LibvirtException: Cannot access storage file /mnt/b33e5a1d-e4ea-3465-b6ac-c98dc8ff8af0/207-2-cc5fd717-2d57-3bb3-bcf6-2c930268db6c.iso"
2020-03-06 13:32:19 +05:30
Pearl Dsilva 44d28f1994
utils: handle EOFException during VR Health Check (#3919)
VR health check throws EOFException due to malformed/Invalid JSON.

Fixes #3893

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-03-06 13:30:53 +05:30
Abhishek Kumar 8cc70c7d87
CloudStack Kubernetes Service (#3680) 2020-03-06 08:51:23 +01:00
Andrija Panic 9c6b02fd8b
Userdata to display static NAT as public ip instead of VR ip (#3862)
* Userdata to display static NAT as public ip instead of VR ip

If static nat is enabled on VM then metadata service should return
the static nat instead of gateway IP.
If static not is not enabled then it should return the gateway IP
as the public IP

Test results:

Step to reproduce:

1. Create a vm
2. Ssh to vm.
3. Run the below command inside the vm
wget http://<VR public ip>/latest/meta-data/public-ipv4

Note down the output of the above command
4. Now acquire a new public and enable static NAT on that IP to this vm
5. Now run the same command mentioned above in the VM
This should display the static NAT ip instead of VR public IP

Output:

Before enabling static nat

wget http://10.10.10.40/latest/meta-data/public-ipv4
$ cat public-ipv4
10.10.10.29

After enabling static nat

wget http://10.10.10.40/latest/meta-data/public-ipv4
$ cat public-ipv4
10.11.10.30

* server: apply vm user data when release a public ip

Co-authored-by: Wei Zhou <ustcweizhou@gmail.com>
2020-03-05 12:49:17 +01:00
Rohit Yadav d0e3c577c0 Merge remote-tracking branch 'origin/4.13' 2020-03-05 12:37:51 +05:30
Rohit Yadav b4fdf22397
kvm: fix/optimize propogating configs (#3911)
Make some changes based on @nvazquez 's comments in PR #3491
Fix a bug in #3491
2020-03-05 12:20:51 +05:30
Rohit Yadav 318924d801
CloudStack Backup & Recovery Framework (#3553) 2020-03-03 13:27:58 +01:00
Daan Hoogland 06a8ff04b1 Merge release branch 4.13 to master
* 4.13:
  VR: Fix Redundant VRouter guest network on wrong interface (#3847)
2020-02-29 19:56:07 +01:00
Wei Zhou 313e21a0da
VR: Fix Redundant VRouter guest network on wrong interface (#3847) 2020-02-29 19:52:40 +01:00
Rohit Yadav ba8fb61a33 Merge remote-tracking branch 'origin/4.13' 2020-02-28 15:06:24 +05:30
Wei Zhou 79f7f0f007
server: fix issue while list ssh keypairs by keyword (#3916)
in 4.13, list sshkeypairs with keyword will ignore the search by name if name is specifed
Fixes an issue in #3098

for example,
(local) > list sshkeypairs name=wei keyword=wei filter=name
{
  "count": 3,
  "sshkeypair": [
    {
      "name": "wei3"
    },
    {
      "name": "wei2"
    },
    {
      "name": "wei"
    }
  ]
}

with this patch ,it gives correct result.

(local) > list sshkeypairs name=wei keyword=wei filter=name
{
  "count": 1,
  "sshkeypair": [
    {
      "name": "wei"
    }
  ]
}
2020-02-28 15:05:49 +05:30
Rohit Yadav 3ca5be40d4 Merge remote-tracking branch 'origin/4.13' 2020-02-28 15:03:12 +05:30
Rakesh 7e30e3d141
router: Avoid duplicate alerts when router state changes (#3904)
When both routers of VPC is in MASTER state
then multiple alerts are sent equally to the number of tiers in the VPC.
If the VPC has 3 tiers then 6 alerts will be sent. This is not good
if VPC has more than 10 networks in it.

Instead of checking the router status for all the tiers in the VPC,
just check the status of the router for one tier in a VPC so that
multiple duplicate alerts can be avoided
2020-02-28 14:24:12 +05:30
Rakesh abb39a25af
server: send VM password to all Running VRs in network/vpc (#3903)
Currently, the cloudstack sends VM password only to the first
router in the network even if its the backup and return the result.

In some cases the first router will be back up and the second will be master.
Since password server is not running in backup, when the user resets the password,
it is sent to the first router which can be backup.
In that case, the new password is not stored in the password server and users cant log in with a new password.

This change ensures that we send the password to both the routers instead
of the first router so that a new password is stored in the master router.
2020-02-28 12:00:16 +05:30
Daan Hoogland a62a10c814 Merge branch '4.13' 2020-02-26 16:18:41 +01:00
Pearl Dsilva 4d8a2da133
api: Fix count and item issues returned by list APIs (#3894) 2020-02-26 15:14:23 +00:00
Abhishek Kumar 0ad2370baf
Enable Direct Download for System VMs (#3731)
* changes for configurable timeouts for direct download

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* server: refactor direct download config value retrieval

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactored direc download cmd, downloader classes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* server, services: allow direct download template for SSVM, CPVM

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* list bypassed system templates

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ignore direct download template during system tempalte download

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add direct download entry while adding store

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix previous change, donot add multiple entries for direct download

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* connection request timeout as hidden configuration

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix template zone ref cleanup on zone deletion

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix previous commit test error, change implementation

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactored zone template cleanup

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-02-26 13:38:31 +01:00
Daan Hoogland b96fbee9e5 Merge release branch 4.13 to master
* 4.13:
  Fix network rules issue if default egress policy is Allow (#3905)
2020-02-23 22:23:50 +01:00
Wei Zhou ce894238d9
vpc: add bypassvlanoverlapcheck parameter when create private g… (#3899) 2020-02-23 21:21:08 +00:00
Rakesh e269b14095
Fix network rules issue if default egress policy is Allow (#3905) 2020-02-23 21:12:06 +00:00
Wei Zhou 458d3b5b47
Multiple networks support for vms in advanced zone with securit… (#3639) 2020-02-19 14:02:12 +00:00
Daan Hoogland b01e011def Merge release branch 4.13 to master
* 4.13:
  KVM: Propagating changes on host parameters to the agents (#3491)
2020-02-19 14:15:52 +01:00
Wei Zhou ac7bcde45b
KVM: Propagating changes on host parameters to the agents (#3491) 2020-02-19 13:13:37 +00:00
Rohit Yadav 3950de116d Merge remote-tracking branch 'origin/4.13' 2020-02-19 13:13:04 +05:30
Wei Zhou 37d2b8537c
kvm: Enable virtio drivers based on guest os display name (#3879)
When we add new guest os, sometimes we missed the records in guest_os_hypervisor.
However, the guest disk model (virtio/ide) is determined by record in the table.
It causes the issue that some new guest os(eg Debian 8/9) uses e1000 instead of virtio nic, and ide disk instead of virtio disk.

To fix the issue permanantly, pass the guest os name in guest_os if the record for kvm is not found in guest_os_hypervisor.

Related commit:7ac9f00eeeb4cd37ec39efeba066e799b581b1a0
2020-02-19 13:12:20 +05:30
Rakesh 4ab6b42250
server: Add new command to update security group name (#3739)
By default, once we create a security group we cant change its name.
In this feature, we introduce a new API command "updateSecurityGroup"
which allows us to rename the security group name. Although we can't
change the name of the "default" security group.
2020-02-19 13:09:52 +05:30
Wei Zhou 649ed45965
kvm: fix exception in volume statts after storage migration (#3884)
On kvm, the 'path' of volume is the file name on primary storage. we should use 'path' instead of 'uuid' in volume statistics.

Fixes: #3878
2020-02-19 13:06:19 +05:30
Rohit Yadav 0c46dfa64a Merge remote-tracking branch 'origin/4.13' 2020-02-18 14:21:36 +05:30
Rakesh bbe2bf1a6e
server: ignore site to site vpn status check on internallbvm (#3864)
When the state of the site to site vpn changes, the check
is done on all the virtual routers including the internal
load balancing vm as well. It is not needed to check the
state for internal load balancing vm
2020-02-18 14:03:30 +05:30
Rohit Yadav d90341ebf1
cloudstack: add JDK11 support (#3601)
This adds support for JDK11 in CloudStack 4.14+:

- Fixes code to build against JDK11
- Bump to Debian 9 systemvmtemplate with openjdk-11
- Fix Travis to run smoketests against openjdk-11
- Use maven provided jdk11 compatible mysql-connector-java
- Remove old agent init.d scripts

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-02-12 12:58:25 +05:30
Rohit Yadav ccda5fb776 Merge remote-tracking branch 'origin/4.13' 2020-02-10 19:02:09 +05:30
Rohit Yadav 78cc0a44c1
server: use host record related to a ssvm/cpvm (#3876)
This implements the systemvm list API response creator to find and use
the host record for a ssvm/cpvm to get the agent status and other
details like last disconnected date and agent version.

Fixes 3875

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-02-10 19:01:34 +05:30
Nicolas Vazquez ce896a477d
[Vmware] Enable PVLAN support on L2 networks (#3732)
* Enable PVLAN support on L2 networks

* Fix prevent null pointer on details

* Add marvin tests

* Fixes from comments

* Fix: missing pvlan type on plugniccommand

* Fix checks on network creation for vlans overlap

* Fix remove prefix from secondary vlan id

* Improve checks on physical network for pvlans

* Fix compatibility with previous pvlan creation

* Fix shared networks backwards pvlan compatibility

* Add ui fix for pvlan type not passed to api

* Add check for isolated vlan id overlap

* Include check for dynamic vlan reserved for secondary vlan

* Fix marvin tests errors

* Fix redundant imports

* Skip marvin test for pvlan if dvswitch is not present

* spelling

Co-authored-by: Andrija Panic <45762285+andrijapanicsb@users.noreply.github.com>
2020-02-07 15:43:01 +01:00
Rohit Yadav 70d1535df4 Merge remote-tracking branch 'origin/4.13' 2020-02-07 18:05:39 +05:30
Rohit Yadav afcbbc4b3e
systemvm: list systemvm does not return agent state and version (#3870)
This makes the listSystemVms API to return the host status (agent state),
version and last pinged information. This makes it possible for UIs
to call a single API to get this information.
2020-02-07 13:19:35 +01:00
Wei Zhou fd5bea838b
New feature: Add support to destroy/recover volumes (#3688)
* server: fix resource count of primary storage if some volumes are Expunged but not removed

Steps to reproduce the issue
(1) create a vm and stop it. check resource count of primary storage
(2) download volume. resource count of primary storage is not changed.
(3) expunge the vm, the volume will be Expunged state as there is a volume snapshot on secondary storage. The resource count of primary storage decreased.
(4) update resource count of the account (or domain), the resource count of primary storage is reset to the value in step (2).

* New feature: Add support to destroy/recover volumes

* Add integration test for volume destroy/recover

* marvin: check resource count of more types

* messages translate to JP

* Update messages for CN

* translate message for NL

* fix two issues per Daan's comments

Co-authored-by: Andrija Panic <45762285+andrijapanicsb@users.noreply.github.com>
2020-02-07 11:25:10 +01:00
Bitworks LLC f78cbf4efc
ssvm: wrong SSVM behavior causes redownloading for all the templates (#3844)
As per discussion at #3838 and proposal by @weizhouapache this PR implements the fix.

Fixes #3838
2020-02-07 11:09:47 +05:30
Daan Hoogland 10482da136 Merge release branch 4.13 to master
* 4.13:
  vr: add missing rule for port forwarding rule in vpc (#3857)
  vpc: set traffic type of private gateway IP to Public to fix ke… (#3851)
2020-02-06 20:38:07 +01:00
Wei Zhou a9a1737dd9
vpc: set traffic type of private gateway IP to Public to fix ke… (#3851) 2020-02-06 20:22:08 +01:00
Rohit Yadav 33e9a50945 Merge remote-tracking branch 'origin/4.13' 2020-02-06 11:32:09 +05:30
Pearl Dsilva bfdb914693
usage: publish zone id while uploading template and volume (#3867)
After a local template is uploaded via browser, the generated usage event with type = "TEMPLATE.CREATE" is persisted with the data store ID instead of the zone ID on the zone_id column. The fix will refactor the upload monitor logic, as after the upload completes, it sets the datastore ID on the zone ID column for the created "TEMPLATE.CREATE" usage event. This refactor will query the DB for the data store and will set its associated zone ID in the usage field.
The fix produces the same behaviour as when registering a template from URL.
FIx is also for uploading VOLUME from local/via browser.
2020-02-06 11:31:24 +05:30
Rakesh 107bba3da1
router: start all Site-to-Site VPN connections when VPC VR starts (#3863)
When we restart the VPC after destroying the master VR, the backup VR
becomes the master and the site to site connections are not in
the connected state. The passive VPN connection will be in the connected
state but the active VPN connection will be in a disconnected state.
2020-02-05 11:56:47 +05:30
Abhishek Kumar 0f5b0e67f8
VM ingestion (#3606)
The VM ingestion feature allows CloudStack to discover, on-board, import existing VMs in an infra. The feature currently works only for VMware, with a hypervisor agnostic framework which may be extended for KVM and XenServer in future.
2020-02-03 15:43:52 +01:00
Rakesh 6d014fa219
network: de-associate IP address if enabling static nat fails (#3836)
Associating static NAT on IP to VM fails even though the IP is not allocated.
When we try enable static NAT on second IP address to the same VM, the operation fails but the IP address is still allocated in the db and it can't be used to enable static NAT on different VM.

Steps to reproduce the issue:

(1) create a vpc (vpc-001) and a vpc tier (vpc-001-001)

(2) create a vm (vm-001-001) in vpc-001-001

(3) acquire a public ip (ip-1) and enable static nat to vm-001-001,
operation succeeds.

(4) acquire a public ip (ip-2) and enable static nat to vm-001-001,
operation fails but the ip is still assigned to vpc tier vpc-001-001.
Note down the ip address and the id of it.

(5) create another vpc tier vpc-001-002, and vm (vm-001-002) in the tier

(6) enabled ip-2 static nat to vm-001-002, operation should succeed
2020-01-31 16:27:20 +05:30
Rakesh a2a4968f51
server: Allow creating network with duplicate name (#3807)
Add a global setting to disable creating networks with same name in an account

Add a global setting to disable creating network without
mentioning the start and end IPv4 or IPv6 address

By default we can create networks with the same name in the account.
Sometimes we should not create the networks with same name.
This change adds a global setting which prevents creating the network with same name.
The default value is true and set it to false to prevent creating network with same names.

Also its possible to create a shared network without mentioning the
start and the end IPv4 or IPv6 address.
This change adds a global setting which prevents creating a shared
network without specifying the start and the end IPv4 or IPv6 address
2020-01-31 15:52:42 +05:30
Rohit Yadav f72f39cd11 Merge remote-tracking branch 'origin/4.13' 2020-01-31 15:28:34 +05:30
Abhishek Kumar a71874682c
server: fix checking disk offering access for snapshot volume (#3791)
Fixes #3783
As reported in the issue, creating volumes from pure snapshot fails with NPE. This is due to order of calls where disk offering access is checked before checking disk offering value. This PR fixes the same.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-01-31 15:24:25 +05:30
Rohit Yadav 424f10cc77 Merge remote-tracking branch 'origin/4.13' 2020-01-31 14:18:11 +05:30
Abhishek Kumar 9d105b6546
template: copy md5 mismatch (#3383)
Fixes #3191

When a template is registered, code stores md5sum of the downloaded file in the vm_template table. However, this downloaded file could be deleted after template installation if it is not an actual (.qcow2, .ova, etc.) file. When the user copies a template using copyTemplate API, the actual template file will be copied across the image stores. Matching checksum for the copied templated file and the stored value from the vm_template table will result in a mismatch.
Changes will set an empty checksum value for the copied template while passing to download service which allows skipping wrong checksum check for the copied while install.
However, this results in a change in checksum value for concerned template entry in vm_template table post template install.

Co-authored-by: dahn <daan.hoogland@gmail.com>
2020-01-31 14:16:37 +05:30
Anurag Awasthi c0abfce8fa
Health check feature for virtual router (#3575) 2020-01-30 12:39:03 +01:00
Xia Li f1149bcb09
[CLOUDSTACK-10408] Fix String.replaceAll() to replace() for bet… (#3275)
* [CLOUDSTACK-10408] Fix String.replaceAll() to replace() for better performance

* improve with replace char but string

Co-authored-by: Rohit Yadav <rohit@apache.org>
2020-01-30 12:06:15 +01:00
Wei Zhou ac581d1546
New feature: Resource count (CPU/RAM) take only running vms into calculation (#3760)
* marvin: check resource count of more types

* New feature: add flag resource.count.running.vms.only to count resource consumption of only running vms

Stopped VMs do not use CPU/RAM actually.
A new global configuration resource.count.running.vms.only is added to determine whether resource (cpu/memory) of only running vms (including Starting/Stopping) will be taken into calculation of resource consumption.

* Add integration test for resource count of only running vms
2020-01-30 10:36:50 +01:00
Rohit Yadav 518ed5379c Merge remote-tracking branch 'origin/4.13' 2020-01-30 11:13:14 +05:30
davidjumani 7a25e40d5a
api: allow listing management server by id and name (#3840)
The List Management Server api returns a list of all the management servers but fails when trying to list by id or name. This ensures that it fetches the details as per the parameters passed.
Fixes: #3833
2020-01-30 10:38:25 +05:30
Pearl Dsilva 1c130a5dd4
api: metrics API response is not super-set of resources response keys (#3834)
The metrics API has few properties missing that are present in the corresponding resource. 

Fixes #3831

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Rohit Yadav <rohit@apache.org>
2020-01-30 08:49:45 +05:30
Rohit Yadav 0cb2db6e1d Merge remote-tracking branch 'origin/4.13' 2020-01-28 11:26:40 +05:30
Gregor Riepl 8792070f84 Rethrow takeVMSnapshot() exception instead of returning null in VMSnapshotManagerImpl (#3761)
Fixes: #3518
2020-01-28 11:05:15 +05:30
Wei Zhou a77d74ba0d server: Fix NPE while update displayvm on vm with dynamic service offering (#3758)
Steps to reproduce the issue
(1) create a custom service offering
(2) create a vm with the offering
(3) update vm with displayvm=false, returns an error

(local) > update virtualmachine id=f33fd06a-7643-40d1-833f-272845d9ba09 displayvm=false
Error 530: {"updatevirtualmachineresponse":{"uuidList":[],"errorcode":530,"cserrorcode":9999}}
2020-01-28 11:04:26 +05:30
Wei Zhou 136505b22c server: double check host capacity when start/migrate a vm (#3728)
When start a vm or migrate a vm (away from a host in host maintenance), cloudstack will check capacity of all hosts and choose one. If there are hundreds of hosts on the platform, it will take some seconds. When cloudstack choose a host and start/migrate vm to it, the resource consumption of the host might have been changed. This normally happens when we start/migrate multiple vms.
It would be better to double check the host capacity when start vm on a host.

This PR includes the fix for cpucore capacity when start/migrate a vm.
2020-01-28 10:55:11 +05:30
Wei Zhou 71e53ab01d server: Capacity check should take vms in Migrating state into calculation (#3727)
When we calculate a resource consumption of a host, we need to take the vms in following states into calculation: Running, Starting, Stopping, Migrating (to the host), and vms are Migrating from the host. Because, when stop a vm, the resource on host will be released when vm is stopped. When migrate a vm, the resource on destination host will be increased before migration starts, and resource on source host will be decreased after migraiton succeeds.

In cloudstack, there is a task named CapacityChecked which run every 5 minutes (capacity.check.period =300000 ms by default). It recalculates capacity of all hosts. However, it takes only vms in Running and Starting into consideration. We have faced some issues in host maintenance due to it.

Steps to reproduce the issue
(1) migrate N vms from host A to host B, cpu/ram resource increases before the migration.
(2) capacity check recalculate the capacity of hosts. used capacity of Host B will be reset to original value (not including the vms in Migrating).
(3) migrate some more vms from other host to host B, the migrations are allowed by cloudstack (because used capacity is incorrect). If the actual used memory exceed the physical memory on the host, there might be some critical issues (for example, libvirt dies)
2020-01-28 10:54:32 +05:30
Daan Hoogland f276485253 Merge release branch 4.13 to master
* 4.13:
  fixed inconsistency of IP on VR when VR is destroyed and recrea… (#3825)
  server: fix resource count error when upgrade a vm (#3759)
2020-01-23 17:00:12 +01:00
Pearl Dsilva b581f85d4b fixed inconsistency of IP on VR when VR is destroyed and recrea… (#3825) 2020-01-23 16:59:00 +01:00
Wei Zhou b79a539191 server: fix resource count error when upgrade a vm (#3759)
Steps to reproduce the issue
(1) create an account (test)
(2) create a vm with the account (test)
(3) login with admin, and upgrade the vm to another offering
(4) the resource count (cpu,memory) of admin increases, not the account (test).
2020-01-23 16:53:43 +01:00
dahn 5ff932eb86
Ldap fixes (#3694)
* pass domainid for list users

* passing arg in wizzard

* adding userfilter to list ldap users and usersource to response

  port of list ldap users tests to java

* assertion of differnt junit ldap methods

* broken test for directory server (and others)

* embedded context loading

* add user and query test

* UI: filter options passing filter and domain and onchange trigger

* disable tests that only work in ide

prereqs for domain-linkage fixed

move trigger to the right location in code

trigger for changing domain

* logging, comments and refactor

implement search users per domain

retrieve appropriate list of users to filter

get domain specific ldap provider

* query cloudstack users with now db filter

* recreate ldap linked account should succeed

* disable auto import users that don't exist

* ui choice and text

* import filter and potential remove from list bug fixed

* fix rights for domain admins

* list only member of linked groups not of principle group

* Do not show ldap user filter if not importing from ldap
  do not delete un-needed items from dialog permanently
  delete from temp object not from global one

* localdomain should not filterout users not imported from ldap

* several types of authentication handling errors fixed and unit tested

* conflict in output name

* add conflict source field to generic import dialog

* replace reflextion by enum member call

* conflict is now called conflict 🎉
2020-01-20 16:02:33 +01:00
Andrija Panic 9b7acfde1e
Update message when keys are NOT being injected (#3799)
* Update message when keys are NOT being injected

* Correct the message after injectkeys.ssh is done

* Update message to a more meaningful one, since sometimes nothing is injected

* Update other 2

* typo
2020-01-18 09:11:42 +01:00
Daan Hoogland 09e812f9cb Merge branch '4.13' 2020-01-15 15:46:19 +01:00
dahn 4780a27255
Add missing HA config keys (#3776) (#3814)
* Add missing HA config keys (#3776)

* merge conflict-bugs fixed

Co-authored-by: mdominka <50666672+mdominka@users.noreply.github.com>
2020-01-15 12:24:05 +01:00
Paul Angus be97470d83 Get Diagnostics: Download logs and diagnostics data from SSVM, CPVM, Router (#3350)
* * Complete API implementation
* Complete UI integration
* Complete marvin test
* Complete Secondary storage GC background task

* improve UI labels

* slight reword and add another missing description

* improve download message clarity

* Address comments

* multiple fixes and cleanups

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* fix more bugs, let it return ip rule list in another log file

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* fix missing iprule bug

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* add support for ARCHIVE type of object to be linked/setup on secstorage

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* Fix retrieving files for Xenserver

* Update get_diagnostics_files.py

* Fix bug where executable scripts weren't handled

* Fixed error on script cmd generation

* Do not filter name for log files as it would override similar prefix script names

* Addressed code review comments

* log error instead of printstacktrace

* Treat script as executable and shell script

* Check missing script name case and write to output instead of catching exception

* Use shell = true instead of shlex to support any executable

* fix xenserver bug

* don't set dir permission for vmware

* Code review comments - refactoring

* Add check for possible NPE

* Remove unused imoprt after rebase

* Add better description for configs

Co-authored-by: Nicolas Vazquez <nicovazquez90@gmail.com>
Co-authored-by: Rohit Yadav <rohit@apache.org>
Co-authored-by: Anurag Awasthi <anurag.awasthi@shapeblue.com>
2020-01-15 11:38:33 +01:00
dahn 6ebd02e7af Agent lb on svm (#3795)
* add host if needed

* allow for test with null-host

* full coverage of states

* EnumSet.contains instead of Arrays.binarySearch
2020-01-14 15:40:21 +01:00
mdominka 54cc73af08 Add missing HA config keys (#3776) 2020-01-14 09:35:34 +01:00
Gabriel Beims Bräscher d8a2f5dc20 Fix typo: the past tense of shutdown is shutdown, not shutdowned (#3659) 2020-01-13 23:39:48 +01:00
Anurag Awasthi 70fcf755f7 Allow additional configuration metadata to VMs (#3510)
* Suqash commits to a single commit and rebase against master

Update marvin tests to use white list

* * Fix marvin test failure
* Add new marvin negative tests cases
* Remove hard-coded hypervisor types in marvin tests

* Fix build error after rebase and add hugepagesless

* Fix readability of python code

* Fix failing test

* Adding cleanup of vms for negative tests

* Bug fixes - change config checks properly and block extraconfig in details

* Trim to compare the keys

* CR comments

* Don't skip extraconfig without exception

Co-authored-by: Boris Stoyanov - a.k.a Bobby <bss.stoyanov@gmail.com>
2020-01-08 17:53:45 +01:00
Rakesh dfd950cab0 Add protocol number support for security group rules (#3736)
Currently while creating ingress/egress rule for a security group,
we can specify only TCP/UDP/ICMP. Sometimes we need to add rules
for different protocol number or rules for all the above three
mentioned protocols.

In this new feature users can specify the protocol number or select
"ALL" option which will apply rules for TCP/UDP/ICMP
2020-01-08 16:13:41 +01:00
Daan Hoogland 4fa7fadd21 Merge release branch 4.13 to master
* 4.13:
  break session only on illegal origin (#3715)
2019-12-24 10:24:09 +01:00
dahn 8844f59247
break session only on illegal origin (#3715) 2019-12-24 10:22:34 +01:00
Rakesh 482e7ebf9a New feature: Acquire specific public IP for network (#3775)
Currently in cloudstack, when we click on "Acquire New Ip", it will
randomly acquire IP from the pool. With this enhancement, it is
possible to select the IP from the drop down IP list of that network.
Same thing applies for a VPC as well.
2019-12-24 10:08:53 +01:00
Daan Hoogland 06e1212e22 Merge release branch 4.13 to master
* 4.13:
  Added zone check for attach iso (#3755)
  config: add isdynamic flag in configuration response (#3729)
  filter hosts to query on zone wide storage (#3733)
  convert protocal names to be found as labels (#3747)
  Once again allow a VM to be on multiple networks from VPCs (#3754)
  create template from snapshot regression (partly reverted) (#3767)
2019-12-23 17:20:44 +01:00
Spaceman1984 100308536c Added zone check for attach iso (#3755) 2019-12-23 16:48:51 +01:00
dahn 482d0fd85e
4.13 (#3782)
* create template from snapshot regression (partly reverted) (#3767)

* Once again allow a VM to be on multiple networks from VPCs (#3754)

to once again allow a VM to be on multiple networks from VPCs

* convert protocal names to be found as labels (#3747)

* convert protocal names to be found as labels

* format

* filter hosts to query on zone wide storage (#3733)

* config: add isdynamic flag in configuration response (#3729)

Co-authored-by: Wei Zhou <ustcweizhou@gmail.com>
2019-12-20 16:43:09 +01:00
Wei Zhou 2712decf50 config: add isdynamic flag in configuration response (#3729) 2019-12-20 15:17:21 +01:00
dahn 3ac03c8858
filter hosts to query on zone wide storage (#3733) 2019-12-20 14:12:10 +01:00
dahn 0b34971b6d
Once again allow a VM to be on multiple networks from VPCs (#3754)
to once again allow a VM to be on multiple networks from VPCs
2019-12-20 14:10:02 +01:00
Anurag Awasthi 4b43c2684f Better tracking host maintanence and handling of migration jobs (#3425)
* Service layer changes for new way of tracking maintanence progress

* Fixes after offline code review

* Fix marvin tests

* Change state name and add documentation

* Fix test

* Fix and add more unit tests for different caseS

* Fix and enhance Marvin Tests

* Fixes for corner cases

* More fixes and logging

* UI fixes

* Some minor changes and reducing VMs on host for more contained tests

* Fixed ssh client auth problem causing test failure

* Code review changes + fixes + some more logging

* Fix flaky tests by adding delays between host states

* Added fetching only enabled hosts for tests

* Make port blocking KVM specific and refactor to handle failure

* Make failing migrations due to tagged host instead of port blocking

* Added additional check for migrating VMs

* Refactor to use single place for methods checking maintenance states
2019-12-19 16:36:20 +01:00
Sven Vogel cf6e616d5b
Revert "Add missing HA config keys (#3737)" (#3774)
This reverts commit 16527f1eb0.
2019-12-18 14:54:27 +01:00
mdominka 16527f1eb0 Add missing HA config keys (#3737)
* Add missing HA config keys
* Change time value to seconds
* Change Integer to Long
* Using ConfigKey defaultValue
* Do some code refactoring
* Simplify code
2019-12-17 15:24:53 +01:00
Dennis Konrad 2e8c069dd2 Load Average for KVM (#3738)
* Avgload (#2)

* Adding avgload for kvm

* Fix coding style issue

* Add getter/setter

* Fix several small errors

* Add override

* Uncomment getAverageLoad

* Override getAverageLoad()

* Checkstyle bug?

* Delete trailing spaces

* Renaming function

* Change interface to match

* Rename method in GetHostStatsAnswer

* Change method call name

* Convert double to long

* Remove trailing whitespace

* Change names around

* Make load visible to return it

* Parse string to double

* Change Long to Double

* Fix getter

* Unify naming to cpuloadaverage

* Change cpuloadaverage String to Double in listHostsMetrics
Remove some unnecessary whitespaces

* Add CPU_LOAD_AVERAGE to ApiConstants
2019-12-17 10:37:05 +01:00
Gregor Riepl 29e1bbc22d server: rethrow takeVMSnapshot() exception instead of returning null (#3546)
Fixes NPE, and throws actual exception with the error stacktrace

Fixes: #3518
2019-12-08 03:49:54 +05:30
Kai Takahashi 6f3694c431 server: save SSH.PublicKey into user_vm_details regardless of password management (#3745)
Fixes: #3744

Signed-off-by: Kai Takahashi <k-takahashi@creationline.com>
2019-12-06 18:26:38 +05:30
nvazquez 828405d297 Merge branch '4.13' 2019-12-05 08:31:14 -03:00
Nicolas Vazquez 07d64cb65a
[KVM] Agent LB Fix: Connections from disabled KVM host agents are refused (#3617)
* Remove constraint for just enabled hosts receiving the ms list

* Allow sending SetupMSList command when host is in maintenance
2019-12-04 10:43:17 -03:00
Daan Hoogland b8676aefe3 Merge release branch 4.13 to master
* 4.13:
  consoleproxy: Enable console for vms in Stopping/Migrating state (#3640)
2019-11-28 17:29:52 +01:00
Wei Zhou 94474af879 consoleproxy: Enable console for vms in Stopping/Migrating state (#3640) 2019-11-28 12:03:38 +01:00
Rohit Yadav 114df3dee0
Forward merge apache/4.13 to master
4.13 forward merge
2019-11-21 11:46:14 +05:30
Wei Zhou d981edb4bc server: acquire IPv4 address when add secondary IP to nic if IP is not specified (#3635)
After commit fbf488497f, admin need to specify an ipv4 or ipv6 addresses when add IP to nic which breaks backward compatibity. If IP is not specified, a IPv4 address should be returned.
2019-11-21 11:44:41 +05:30
Nicolas Vazquez 80abb5a74b Fix VR creation issue while creating VM on shared network using PVLAN (#3653)
* Fix VR creation issue while creating VM on shared network using PVLAN

* Fix update guest networks
2019-11-21 11:34:14 +05:30
Rohit Yadav 524b995083
IoT/ARM64 support: allow cloudstack-agent on Raspberry Pi 4 (armv8) to use kvm acceleration (#3644)
KVM is supported on arm64 Linux (https://www.linux-kvm.org/page/Processor_support#ARM:).
For a small (IoT) platform such as the new Raspberry Pi 4 that uses armv8 processor
(cortex-a72) it's possible to run Linux host with `/dev/kvm`
accleration. This adds support for IoT IaaS in CloudStack.

This PR is from a fun weekend project where:
- I set up a Raspberry Pi 4 - 4GB RAM model with 4 CPU cores @ 1.5Ghz, 128GB SD samsung evo plus card
- Installed Ubuntu 19.10 raspi3 base image: http://cdimage.ubuntu.com/releases/19.10/release/ubuntu-19.10-preinstalled-server-arm64+raspi3.img.xz
- Build a custom Linux 5.3 kernel with KVM enabled, deb here: http://dl.rohityadav.cloud/cloudstack-rpi/kernel-19.10/ and install the linux-image and linux-module
- Then install/setup CloudStack on it (fix some issues around jna, by manually installing newer libjna-java to /usr/share/cloudstack-agent/lib)
- Since the host processor is not x86_64, I had to build a new arm64 (or aarch64) systemvmtemplate: http://dl.rohityadav.cloud/cloudstack-rpi/systemvmtemplate/

I could finally get a 4.13 CloudStack + Adv zone/networking to run on it
and deployed a KVM based Ubuntu 19.10 environment and NFS storage.
Deployed a test vm with isolated network, VR works as expected. Console
proxy works as well, for this tested against arm64 openstack Debian 9/10
templates.

I raised the issue of enabling KVM in upstream Ubuntu arm64 build: https://bugs.launchpad.net/ubuntu/+source/linux-raspi2/+bug/1783961
Ubuntu kernel team has come back and future arm64 releases may have 
KVM enabled by default.

Limitation: on my aarch64 env, it did not support IDE, therefore all
default bus type for volumes are SCSI by default. With VIRTIO it fails
sometimes.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-11-11 22:01:05 +05:30
Rohit Yadav 294813e8fe Merge remote-tracking branch 'origin/4.13' 2019-10-18 12:29:24 +05:30
Wei Zhou cc27095554 server: Do NOT cleanup dhcp and dns when stop a vm (#3627)
* server: Do NOT cleanup dhcp and dns when stop a vm

According comment in PR #3608, dhcp and dns entries are cleaned up only when a VM is expunged.
Revert part of commit 8fb388e931.

* server: cleanup dns/dhcp entries in removeNic instead of finalizeExpunge
2019-10-18 01:20:55 +05:30