Commit Graph

32604 Commits

Author SHA1 Message Date
Rohit Yadav 9b6983f0e2 Merge remote-tracking branch 'origin/4.14'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-28 12:13:22 +05:30
Rohit Yadav 0501575efa
client: explicitly define SslContextFactory::Server for https (#4288)
Fixes #4199

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-28 12:09:23 +05:30
Gabriel Beims Bräscher 11e7de7d87
pom: Update Java Rados from v0.5.0 to v0.6.0 (#4287)
This PR updates the Java Rados version v0.6.0. The release artifacts are available at: https://search.maven.org/artifact/com.ceph/rados.

Fixes: #4159
2020-08-27 16:26:30 +05:30
Rohit Yadav 6d6e4025f5 Merge remote-tracking branch 'origin/4.14' 2020-08-25 17:02:19 +05:30
Greg Goodrich 9b72e5dfb8
Adding os type id to the usage record response for virtual machines (#4266) 2020-08-25 16:54:08 +05:30
Spaceman1984 39734afcbc
Changed test failure to warning (#4264)
* Added more time for capacity log

* Changed test to warning instead of fail when a timeout happens

* Update test_human_readable_logs.py

Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-25 15:29:59 +05:30
Spaceman1984 c4922c4289
Fixed rolling restart on VPC network (#4272)
This PR fixes the problem where restarting a network with cleanup within a VPC destroys the VR first before creating a new one.

Fixes: #3815
2020-08-25 15:19:51 +05:30
Rohit Yadav 9c7c22eb84
engine: honour bypass VLAN id/range for L2 networks (#4274)
* engine: honour bypass VLAN id/range for L2 networks

Commit e894238d904a9c49c1140371f612a51d251efc1 (#3899) allowed private
gateways to bypass vlan check while refactoring it did not cover the
case for L2 but only shared network. This fix will re-enable honouring
the bypass vlan check option for L2 guest network (in addition to the
Shared networks).

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* Update NetworkOrchestrator.java
2020-08-25 15:19:02 +05:30
Andrija Panic cddf69c0f8
Usage-server update message improvement (#4278)
* Update cloud.spec

* Update cloud.spec

* Update cloudstack-usage.postinst
2020-08-23 16:04:38 +05:30
Rohit Yadav bb73bedb55 Merge remote-tracking branch 'origin/4.14' 2020-08-21 15:39:00 +05:30
Rohit Yadav 05ae3f8d81 Merge remote-tracking branch 'origin/4.13' into 4.14 2020-08-21 15:38:18 +05:30
Sid Kattoju 1da76d27f1
iscsi session cleanup now configurable, filters iscsi partitions (#4219)
Added property to agent.properties that enables or disables the iscsi session clean up feature. #4210
Added a condition to prevent disk partitions from being cleaned up. #4216
2020-08-21 14:38:36 +05:30
Rohit Yadav 961f3ece41 Merge remote-tracking branch 'origin/4.14' 2020-08-21 14:31:52 +05:30
davidjumani 3872bf1ff9
kvm: Enable PVLAN support on L2 networks (#4040)
This is an extention of #3732 for kvm.
This is restricted to ovs > 2.9.2
Since Xen uses ovs 2.6, pvlan is unsupported.
This also fixes issues of vms on the same pvlan unable to communicate if they're on the same host
2020-08-20 15:46:34 +05:30
Paul Angus 400641b1cf Revert "Revert "move power sync messages to debug""
This reverts commit e06cf1a642.
2020-08-20 10:51:05 +01:00
Paul Angus e06cf1a642 Revert "move power sync messages to debug"
This reverts commit 1954ff32b7.
2020-08-20 10:50:58 +01:00
Paul Angus 1954ff32b7 move power sync messages to debug 2020-08-20 10:50:35 +01:00
Paul Angus 9ae4362908 add info only log file. 2020-08-20 10:45:11 +01:00
Rakesh 9681a28c7f
api: Display hypervisor type for VM snapshot (#4275)
The "hypervisor" field in listvmsnapshot response will
be used in primate to enable/disable creating snapshot
from vm snapshot functionality.

Creating snpashot from vm snapshot will be enabled only if
hypervisor is KVM
2020-08-20 14:13:49 +05:30
Spaceman1984 d57aa83517
server: Added nfs minor version support (#4180)
This PR adds minor version support when mounting nfs on the SSVM as requested in #2861

The global setting "secstorage.nfs.version" has been changed to use the String data type which allows any minor version to be specified.
2020-08-19 14:53:38 +05:30
davidjumani d949302d0f
packaging: Adding Centos8, Ubuntu 20.04, XCPNG8.1 Support (#4068)
* DB : Add support for MySQL 8

- Splits commands to create user and grant access on database, the old
statement is no longer supported by MySQL 8.x
- `NO_AUTO_CREATE_USER` is no longer supported by MySQL 8.x so remove
that from db.properties conn parameters

For mysql-server 8.x setup the following changes were added/tested to
make it work with CloudStack in /etc/mysql/mysql.conf.d/mysqld.cnf and
then restart the mysql-server process:

    server_id = 1
    sql-mode="STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION,ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE,NO_ENGINE_SUBSTITUTION"
    innodb_rollback_on_timeout=1
    innodb_lock_wait_timeout=600
    max_connections=1000
    log-bin=mysql-bin
    binlog-format = 'ROW'

    default-authentication-plugin=mysql_native_password

Notice the last line above, this is to reset the old password based
authentication used by MySQL 5.x.

Developers can set empty password as follows:

    > sudo mysql -u root
    ALTER USER 'root'@'localhost' IDENTIFIED BY '';

In libvirt repository, there are two related commits

2019-08-23 13:13 Daniel P. Berrangé            ● rpm: don't enable socket activation in upgrade if --listen present
2019-08-22 14:52 Daniel P. Berrangé            ● remote: forbid the --listen arg when systemd socket activation

In libvirt.spec.in

        /bin/systemctl mask libvirtd.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-ro.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-admin.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-tls.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-tcp.socket >/dev/null 2>&1 || :

Co-authored-by: Wei Zhou <w.zhou@global.leaseweb.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-17 16:28:30 +05:30
Gabriel Beims Bräscher 3fe724bd32
db: Prevent NullPointerException on GenericDaoBase (#4268)
* Prevent NullPointerException on GenericDaoBase

* Fix checkstyle: remove unused import
2020-08-17 16:13:46 +05:30
Pearl Dsilva c383269a20
testing: improve vmware ssvm test (#4262)
Add sleep before ssvm health check script is run on reboot

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-08-15 23:06:22 +05:30
Spaceman1984 b586eb22f1
Human readable sizes in logs (#4207)
This PR adds outputting human readable byte sizes in the management server logs, agent logs, and usage records. A non-dynamic global variable is added (display.human.readable.sizes) to control switching this feature on and off. This setting is sent to the agent on connection and is only read from the database when the management server is started up. The setting is kept in memory by the use of a static field on the NumbersUtil class and is available throughout the codebase.

Instead of seeing things like:
2020-07-23 15:31:58,593 DEBUG [c.c.a.t.Request] (AgentManager-Handler-12:null) (logid:) Seq 8-1863645820801253428: Processing: { Ans: , MgmtId: 52238089807, via: 8, Ver: v1, Flags: 10, [{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-224-VM","bytesSent":"106496","bytesReceived":"0","result":"true","details":"","wait":"0",}}] }

The KB MB and GB values will be printed out:

2020-07-23 15:31:58,593 DEBUG [c.c.a.t.Request] (AgentManager-Handler-12:null) (logid:) Seq 8-1863645820801253428: Processing: { Ans: , MgmtId: 52238089807, via: 8, Ver: v1, Flags: 10, [{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-224-VM","bytesSent":"(104.00 KB) 106496","bytesReceived":"(0 bytes) 0","result":"true","details":"","wait":"0",}}] }

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Human+Readable+Byte+sizes
2020-08-13 15:55:16 +05:30
Rohit Yadav 55a5470da6 Merge remote-tracking branch 'origin/4.14' 2020-08-13 15:52:47 +05:30
Pearl Dsilva a3d5b08334
vmware: Name public network appropriately to avoid conflicts (#4254)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-08-13 15:52:00 +05:30
Pearl Dsilva c578004fe5
projects: Role based users in Projects (#4128)
Enabling Role Based users in projects
Primate PR related to the FR: apache/cloudstack-primate#382
Doc PR: https://github.com/apache/cloudstack-documentation/pull/145

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2020-08-13 15:45:39 +05:30
Pearl Dsilva b68be664b7
server: Search vm snapshots using tags (#4213)
Search VM snapshots using tags
Currently, search of VM snapshots doesn't comply with tags passed as input
2020-08-13 15:39:08 +05:30
Rodrigo D. Lopez 3adee270c7
api: Prevent null pointer on listPublicIpAddress cmd (#4255)
* Prevent null pointer on listPublicIpAddress cmd

Insert an inner join between data_center table and user_ip_address where data_center.removed field is null

* Remove extra join and add a filter for VLAN removed
2020-08-13 15:36:19 +05:30
Rodrigo D. Lopez 6704ef2794
server: fix comparison using nullable objects (#4256)
This PR fix some wrongs comparison using nullable objects. Preventing null pointer exception
2020-08-13 15:35:23 +05:30
Abhishek Kumar 95c4ef167d
cks: fix for null hypervisor type (#4260)
Fixes #4241 

Failure to deploy VM when vm.instancename.flag is set to true

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-08-13 15:33:58 +05:30
Rohit Yadav 7bbb4d4bef Merge remote-tracking branch 'origin/4.14' 2020-08-12 13:46:13 +05:30
Rohit Yadav 6b6cc71be9 Merge remote-tracking branch 'origin/4.13' into 4.14 2020-08-12 13:44:41 +05:30
Spaceman1984 86939e7f9d
server: Fixed private gateway can't be deleted (#4016)
When the static route service is not available on the VPC and a static route is created, the static route is created in a revoked state.

Currently, the UI doesn't distinguish between active or revoked static routes.

This PR adds the missing state filter to the list routes command and only lists active routes in the UI.
It also ignores revoked routes when the private gateway is being removed but clears out the inactive routes before the gateway is removed.

Fixes #2908
2020-08-12 13:29:12 +05:30
Rohit Yadav 6a1ddf7d14 Merge remote-tracking branch 'origin/4.14' 2020-08-10 16:37:39 +05:30
Sina Kashipazha ce7a32c51d
build: Fix sed command failure in Mac OS. (#4253)
apidoc build failed in Mac OS because of sed in-place command. It is a minor change that fixed this issue. More information here

Fixes: #3247
Fixes: #3312
2020-08-10 13:11:45 +05:30
Rohit Yadav 7707d3fd92 engine: fix NPE from fwd-merge 562a7db8df
This fixes NPE caused due to merge conflict fix from the forward merge
commit 562a7db8df and fixes travis test
regression:

    === TestName: test_01_reset_vm_on_reboot | Status : SUCCESS ===

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-07 15:31:43 +05:30
Wido den Hollander 29fb74538b
appliance: Host SSVM Debian ISO on download.cloudstack.org (#4249)
This way we do not depend on Debian keeping the ISO online or changing
URLs which can break our build system(s).
2020-08-07 13:03:43 +05:30
Gabriel Beims Bräscher 1a2cc40905
systemvmtemplate: Update debian 10 iso to 10.5.0 (#4243)
Debian 10.4.0 is not available, therefore SystemVM templates will fail to build with a missing iso.

Fixed updating with the Debian 10.5.0 iso located at https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/
2020-08-06 15:57:17 +05:30
Rohit Yadav 562a7db8df Merge remote-tracking branch 'origin/4.14'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-05 23:59:16 +05:30
Wei Zhou cd8e28b279
server: Move restoreVM to vm work job queue (#4019) 2020-08-05 09:46:55 +00:00
Rohit Yadav db665fa8df Merge remote-tracking branch 'origin/4.14'
Conflicts:
	engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-05 14:46:13 +05:30
Gabriel Beims Bräscher 2237486079
server: Allow renaming cluster, host, and storage (#4165)
This PR adds implementation for changing host and storage name, additionally, it fixes a Bug on cluster updateCluster API command. This PRs also enhances the UI by allowing editing field name on Host and Storage pool. Due to the fact that there is no support to editing cluster via UI, it was not edited.

TODO: I will address Host, Cluster, and Storage Pool name edition on CloudStack Primate once the API implementation gets merged.

Details:
Prior to this PR the following API commands did not offer support for updating name:

updateHost (enhancement)
updateStoragePool (enhancement)
Additionally, updateCluster claims to support changing a cluster name (via clustername parameter); however, such operation did not work. (bug)
2020-08-05 14:08:29 +05:30
Rakesh a529470b16
server: Fix cpuallocated value in findHostsForMIgration api (#4220)
The findHostsForMigration api displays 0% always for
cpuallocated field which is wrong.

Fixes #4221
2020-08-05 14:02:09 +05:30
Rohit Yadav 2fce8b733e
vmware: volume utilisation is always zero (#4225)
This fixes issues of virtual size to be twice in case the disk is a
linked-clone root disk. The virtual size of root disk (first in chain)
must be used.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-05 14:01:16 +05:30
Wei Zhou 4527424fce
vm: Reset deviceId to fix missing nic with vm (#4000)
Co-authored-by: Wei Zhou <57355700+weizhouapache@users.noreply.github.com>
2020-08-05 13:49:03 +05:30
Wido den Hollander c3554ec31d
kvm: For ceph only if a port number has been specified define in the XML (#4231)
Ceph used to use port 6789 (no need to specify it), but with the messenger v2
from Ceph it switched to port 3300 while 6789 still works.

librados/librbd/libvirt will automatically figure out the ports to use if none is
specified.

Therefor there is no need for CloudStack to explicitely define the port in the XML
passed to Libvirt or Qemu.

Leave blank if no port number has been defined by the user.
2020-08-05 13:44:40 +05:30
Rohit Yadav 9ae1170b29 Merge remote-tracking branch 'origin/4.14' 2020-08-04 11:28:43 +05:30
Abhishek Kumar a7f56d41c8
cks: assorted bug fixes (#4116)
Fixes #4056
Fixes #4107
Fixes #4113
Fixes #4133

Fixes deployment, template and network deletion.
Also allows filetering in listKubernetesSupportedVersions with keyword

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-08-04 11:27:32 +05:30
Wei Zhou 407e34d4e7
vrouter: remove a POSTROUTING rule for port forwarding in VPC router (#3952)
As discussed in #3937 (comment)
a rule for port forwarding in VPC router might not be needed.

This fixes the failed result of health check for network VRs.
2020-08-04 11:25:28 +05:30