Commit Graph

32593 Commits

Author SHA1 Message Date
Andrija Panic b3eacaffe9
Update cloudstack-usage.postinst 2020-08-21 10:30:14 +02:00
Andrija Panic 62214e0b9b
Update cloud.spec 2020-08-21 10:20:59 +02:00
Andrija Panic 0e08dff8bd
Update cloud.spec 2020-08-21 10:19:04 +02:00
davidjumani 3872bf1ff9
kvm: Enable PVLAN support on L2 networks (#4040)
This is an extention of #3732 for kvm.
This is restricted to ovs > 2.9.2
Since Xen uses ovs 2.6, pvlan is unsupported.
This also fixes issues of vms on the same pvlan unable to communicate if they're on the same host
2020-08-20 15:46:34 +05:30
Paul Angus 400641b1cf Revert "Revert "move power sync messages to debug""
This reverts commit e06cf1a642.
2020-08-20 10:51:05 +01:00
Paul Angus e06cf1a642 Revert "move power sync messages to debug"
This reverts commit 1954ff32b7.
2020-08-20 10:50:58 +01:00
Paul Angus 1954ff32b7 move power sync messages to debug 2020-08-20 10:50:35 +01:00
Paul Angus 9ae4362908 add info only log file. 2020-08-20 10:45:11 +01:00
Spaceman1984 d57aa83517
server: Added nfs minor version support (#4180)
This PR adds minor version support when mounting nfs on the SSVM as requested in #2861

The global setting "secstorage.nfs.version" has been changed to use the String data type which allows any minor version to be specified.
2020-08-19 14:53:38 +05:30
davidjumani d949302d0f
packaging: Adding Centos8, Ubuntu 20.04, XCPNG8.1 Support (#4068)
* DB : Add support for MySQL 8

- Splits commands to create user and grant access on database, the old
statement is no longer supported by MySQL 8.x
- `NO_AUTO_CREATE_USER` is no longer supported by MySQL 8.x so remove
that from db.properties conn parameters

For mysql-server 8.x setup the following changes were added/tested to
make it work with CloudStack in /etc/mysql/mysql.conf.d/mysqld.cnf and
then restart the mysql-server process:

    server_id = 1
    sql-mode="STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION,ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE,NO_ENGINE_SUBSTITUTION"
    innodb_rollback_on_timeout=1
    innodb_lock_wait_timeout=600
    max_connections=1000
    log-bin=mysql-bin
    binlog-format = 'ROW'

    default-authentication-plugin=mysql_native_password

Notice the last line above, this is to reset the old password based
authentication used by MySQL 5.x.

Developers can set empty password as follows:

    > sudo mysql -u root
    ALTER USER 'root'@'localhost' IDENTIFIED BY '';

In libvirt repository, there are two related commits

2019-08-23 13:13 Daniel P. Berrangé            ● rpm: don't enable socket activation in upgrade if --listen present
2019-08-22 14:52 Daniel P. Berrangé            ● remote: forbid the --listen arg when systemd socket activation

In libvirt.spec.in

        /bin/systemctl mask libvirtd.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-ro.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-admin.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-tls.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-tcp.socket >/dev/null 2>&1 || :

Co-authored-by: Wei Zhou <w.zhou@global.leaseweb.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-17 16:28:30 +05:30
Gabriel Beims Bräscher 3fe724bd32
db: Prevent NullPointerException on GenericDaoBase (#4268)
* Prevent NullPointerException on GenericDaoBase

* Fix checkstyle: remove unused import
2020-08-17 16:13:46 +05:30
Pearl Dsilva c383269a20
testing: improve vmware ssvm test (#4262)
Add sleep before ssvm health check script is run on reboot

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-08-15 23:06:22 +05:30
Spaceman1984 b586eb22f1
Human readable sizes in logs (#4207)
This PR adds outputting human readable byte sizes in the management server logs, agent logs, and usage records. A non-dynamic global variable is added (display.human.readable.sizes) to control switching this feature on and off. This setting is sent to the agent on connection and is only read from the database when the management server is started up. The setting is kept in memory by the use of a static field on the NumbersUtil class and is available throughout the codebase.

Instead of seeing things like:
2020-07-23 15:31:58,593 DEBUG [c.c.a.t.Request] (AgentManager-Handler-12:null) (logid:) Seq 8-1863645820801253428: Processing: { Ans: , MgmtId: 52238089807, via: 8, Ver: v1, Flags: 10, [{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-224-VM","bytesSent":"106496","bytesReceived":"0","result":"true","details":"","wait":"0",}}] }

The KB MB and GB values will be printed out:

2020-07-23 15:31:58,593 DEBUG [c.c.a.t.Request] (AgentManager-Handler-12:null) (logid:) Seq 8-1863645820801253428: Processing: { Ans: , MgmtId: 52238089807, via: 8, Ver: v1, Flags: 10, [{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-224-VM","bytesSent":"(104.00 KB) 106496","bytesReceived":"(0 bytes) 0","result":"true","details":"","wait":"0",}}] }

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Human+Readable+Byte+sizes
2020-08-13 15:55:16 +05:30
Rohit Yadav 55a5470da6 Merge remote-tracking branch 'origin/4.14' 2020-08-13 15:52:47 +05:30
Pearl Dsilva a3d5b08334
vmware: Name public network appropriately to avoid conflicts (#4254)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-08-13 15:52:00 +05:30
Pearl Dsilva c578004fe5
projects: Role based users in Projects (#4128)
Enabling Role Based users in projects
Primate PR related to the FR: apache/cloudstack-primate#382
Doc PR: https://github.com/apache/cloudstack-documentation/pull/145

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2020-08-13 15:45:39 +05:30
Pearl Dsilva b68be664b7
server: Search vm snapshots using tags (#4213)
Search VM snapshots using tags
Currently, search of VM snapshots doesn't comply with tags passed as input
2020-08-13 15:39:08 +05:30
Rodrigo D. Lopez 3adee270c7
api: Prevent null pointer on listPublicIpAddress cmd (#4255)
* Prevent null pointer on listPublicIpAddress cmd

Insert an inner join between data_center table and user_ip_address where data_center.removed field is null

* Remove extra join and add a filter for VLAN removed
2020-08-13 15:36:19 +05:30
Rodrigo D. Lopez 6704ef2794
server: fix comparison using nullable objects (#4256)
This PR fix some wrongs comparison using nullable objects. Preventing null pointer exception
2020-08-13 15:35:23 +05:30
Abhishek Kumar 95c4ef167d
cks: fix for null hypervisor type (#4260)
Fixes #4241 

Failure to deploy VM when vm.instancename.flag is set to true

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-08-13 15:33:58 +05:30
Rohit Yadav 7bbb4d4bef Merge remote-tracking branch 'origin/4.14' 2020-08-12 13:46:13 +05:30
Rohit Yadav 6b6cc71be9 Merge remote-tracking branch 'origin/4.13' into 4.14 2020-08-12 13:44:41 +05:30
Spaceman1984 86939e7f9d
server: Fixed private gateway can't be deleted (#4016)
When the static route service is not available on the VPC and a static route is created, the static route is created in a revoked state.

Currently, the UI doesn't distinguish between active or revoked static routes.

This PR adds the missing state filter to the list routes command and only lists active routes in the UI.
It also ignores revoked routes when the private gateway is being removed but clears out the inactive routes before the gateway is removed.

Fixes #2908
2020-08-12 13:29:12 +05:30
Rohit Yadav 6a1ddf7d14 Merge remote-tracking branch 'origin/4.14' 2020-08-10 16:37:39 +05:30
Sina Kashipazha ce7a32c51d
build: Fix sed command failure in Mac OS. (#4253)
apidoc build failed in Mac OS because of sed in-place command. It is a minor change that fixed this issue. More information here

Fixes: #3247
Fixes: #3312
2020-08-10 13:11:45 +05:30
Rohit Yadav 7707d3fd92 engine: fix NPE from fwd-merge 562a7db8df
This fixes NPE caused due to merge conflict fix from the forward merge
commit 562a7db8df and fixes travis test
regression:

    === TestName: test_01_reset_vm_on_reboot | Status : SUCCESS ===

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-07 15:31:43 +05:30
Wido den Hollander 29fb74538b
appliance: Host SSVM Debian ISO on download.cloudstack.org (#4249)
This way we do not depend on Debian keeping the ISO online or changing
URLs which can break our build system(s).
2020-08-07 13:03:43 +05:30
Gabriel Beims Bräscher 1a2cc40905
systemvmtemplate: Update debian 10 iso to 10.5.0 (#4243)
Debian 10.4.0 is not available, therefore SystemVM templates will fail to build with a missing iso.

Fixed updating with the Debian 10.5.0 iso located at https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/
2020-08-06 15:57:17 +05:30
Rohit Yadav 562a7db8df Merge remote-tracking branch 'origin/4.14'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-05 23:59:16 +05:30
Wei Zhou cd8e28b279
server: Move restoreVM to vm work job queue (#4019) 2020-08-05 09:46:55 +00:00
Rohit Yadav db665fa8df Merge remote-tracking branch 'origin/4.14'
Conflicts:
	engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-05 14:46:13 +05:30
Gabriel Beims Bräscher 2237486079
server: Allow renaming cluster, host, and storage (#4165)
This PR adds implementation for changing host and storage name, additionally, it fixes a Bug on cluster updateCluster API command. This PRs also enhances the UI by allowing editing field name on Host and Storage pool. Due to the fact that there is no support to editing cluster via UI, it was not edited.

TODO: I will address Host, Cluster, and Storage Pool name edition on CloudStack Primate once the API implementation gets merged.

Details:
Prior to this PR the following API commands did not offer support for updating name:

updateHost (enhancement)
updateStoragePool (enhancement)
Additionally, updateCluster claims to support changing a cluster name (via clustername parameter); however, such operation did not work. (bug)
2020-08-05 14:08:29 +05:30
Rakesh a529470b16
server: Fix cpuallocated value in findHostsForMIgration api (#4220)
The findHostsForMigration api displays 0% always for
cpuallocated field which is wrong.

Fixes #4221
2020-08-05 14:02:09 +05:30
Rohit Yadav 2fce8b733e
vmware: volume utilisation is always zero (#4225)
This fixes issues of virtual size to be twice in case the disk is a
linked-clone root disk. The virtual size of root disk (first in chain)
must be used.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-05 14:01:16 +05:30
Wei Zhou 4527424fce
vm: Reset deviceId to fix missing nic with vm (#4000)
Co-authored-by: Wei Zhou <57355700+weizhouapache@users.noreply.github.com>
2020-08-05 13:49:03 +05:30
Wido den Hollander c3554ec31d
kvm: For ceph only if a port number has been specified define in the XML (#4231)
Ceph used to use port 6789 (no need to specify it), but with the messenger v2
from Ceph it switched to port 3300 while 6789 still works.

librados/librbd/libvirt will automatically figure out the ports to use if none is
specified.

Therefor there is no need for CloudStack to explicitely define the port in the XML
passed to Libvirt or Qemu.

Leave blank if no port number has been defined by the user.
2020-08-05 13:44:40 +05:30
Rohit Yadav 9ae1170b29 Merge remote-tracking branch 'origin/4.14' 2020-08-04 11:28:43 +05:30
Abhishek Kumar a7f56d41c8
cks: assorted bug fixes (#4116)
Fixes #4056
Fixes #4107
Fixes #4113
Fixes #4133

Fixes deployment, template and network deletion.
Also allows filetering in listKubernetesSupportedVersions with keyword

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-08-04 11:27:32 +05:30
Wei Zhou 407e34d4e7
vrouter: remove a POSTROUTING rule for port forwarding in VPC router (#3952)
As discussed in #3937 (comment)
a rule for port forwarding in VPC router might not be needed.

This fixes the failed result of health check for network VRs.
2020-08-04 11:25:28 +05:30
Gabriel Beims Bräscher ba6e2ac843
plugins: Redfish Client & Redfish OOBM Driver (#4175)
This PR adds support for the OOBM Redfish protocol, implementing a Java client to send HTTP requests to Redfish supported systems.

Implementation overview:
- Redfish Java client: a Java Client for Redfish that makes Redfish actions available to the HA workflow via an OOB driver.
- OOB Redfish driver: a new Out-of-band driver was created for Redfish, allowing to integrate the Redfish Client with the CloudStack Out-of-band management implementation.

Fixes: #3624
2020-07-30 10:51:16 +05:30
Rohit Yadav 36ef85012f Merge remote-tracking branch 'origin/4.14'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-07-29 14:08:02 +05:30
Gregor Riepl c856614350
Document how to pass CIDRs lists API calls (#4035) 2020-07-29 10:33:30 +02:00
davidjumani eec56025c8
api: Bug fixes for primate (#4214)
Adding the following fixes so primate can work without issues :
- Adding pagination for listNetworkAclLists
- Adding pagination for listRoles
- Returning mshost uuid rather than msid in list hosts response
- Allowing listVirtualMachinesMetrics to respect hostid
- Fixing return all details in template response
2020-07-29 12:26:39 +05:30
Spaceman1984 e225db46e4
ca: Removed check on client mode (#4226)
Fixes unit test failures due to behavior change in OpenJDK 11.0.8

JDK-8245077: Default SSLEngine Created in Server Role
=====================================================
In JDK 11 and later, `javax.net.ssl.SSLEngine` by default used client
mode when handshaking.  As a result, the set of default enabled
protocols may differ to what is expected. `SSLEngine` would usually be
used in server mode. From this JDK release onwards, `SSLEngine` will
default to server mode. The
`javax.net.ssl.SSLEngine.setUseClientMode(boolean mode)` method may be
used to configure the mode.
2020-07-29 12:25:00 +05:30
nvazquez 7e3b61b723 Merge branch '4.14' 2020-07-18 14:17:43 -03:00
nvazquez 5c6e79b1eb Merge branch '4.13' into 4.14 2020-07-18 14:15:46 -03:00
Nicolas Vazquez f843c537f0
Fix snapshots garbage collection (#4188)
* Cleanup orphan entries from snapshot store ref for primary storage

* Add debug message
2020-07-18 14:12:53 -03:00
Rohit Yadav d76caa74ce Merge remote-tracking branch 'origin/4.14' 2020-07-15 16:15:54 +05:30
Rohit Yadav 3bd1ddec88 Merge remote-tracking branch 'origin/4.13' into 4.14 2020-07-15 16:14:58 +05:30
Spaceman1984 affd010418
templates: Fixed incorrect error message on invalid template type download (#4138)
When a template is downloaded, the first 1MB of the template is validated to determine if the template is of correct file type. On failure, the download is aborted and the input stream is set to null. This leads to a second error when the try-with-resources block tries to auto-close the stream and throws an ioexception. The "stream closed" error message is then written to the db.

This PR checks if an error has been stored before setting a new error message.

Fixes: #4127
2020-07-15 15:59:18 +05:30