Commit Graph

31721 Commits

Author SHA1 Message Date
Andrija Panic df173859d7 agent: add host.reserved.mem.mb parameter documentation (#3016)
Document "host.reserved.mem.mb" parameter in agent.properties.
2018-11-12 11:40:37 +05:30
Wido den Hollander 44c080da11 server: print log on INFO if Host reached Max Guests Limit (#3013)
This should not be in DEBUG as people would want to know that the host was skipped
because it didn't have enough slots available to run the VM.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2018-11-12 11:39:17 +05:30
Rohit Yadav d4d91f355d
vmware: updateVmwareDc API for updating vmware datacenter details (#2919)
This adds a new API updateVmwareDc that allows admins to update the
VMware datacenter details of a zone. It also recursively updates
the cluster_details for any username/password updates
as well as updates the url detail in cluster_details table and guid
detail in the host_details table with any newly provided vcenter
domain/ip. The update API assumes that there is only one vCenter per
zone. And, since the username/password for each VMware host could be different
than what gets configured for vcenter at zone level, it does not update the
username/password in host_details.

Previously, one has to manually update the db with any new vcenter details for the zone.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-11-12 11:32:36 +05:30
Rafael Weingärtner 63094b2fbf
Fix delete domain button in the UI to update only after job finishes (#2905)
When the “deleteDomain” command is sent via UI, the HTML is already updated assuming that the “deleteDomain” command will actually delete it. However, there are cases (domain with allocated resources) that will not allow ACS to remove the command; as a consequence, the UI will not be in sync with ACS state, which can lead to misunderstandings.
2018-11-11 18:41:37 -02:00
Mike Tutkowski f36e0c77aa
Adding three new settings into cloud.configuration. (#3019) 2018-11-09 15:21:39 -07:00
Sven Vogel e44fc4af68 add exitcode / distribution detection and remove export CGROUP_DAEMON for RHEL7/CentOS7 (#2896)
* task/add distribution detection, remove export CGROUP_DAEMON='cpu:/virt'

* taks/add exit values

* change exit codes

* split redhat and centos / change ubuntu to /etc/lsb-release file

* turn around centos and rhel, we need first check for centos then rhel

* set variable distro to None
2018-11-07 18:22:20 -02:00
Rafael Weingärtner f18326880b Forwardmerge #2980 - Fix set initial reservation on public IP ranges 2018-11-07 10:50:09 -02:00
Nicolas Vazquez 7d8eb37924 [4.11] Fix set initial reservation on public IP ranges (#2980)
* Fix initial reservation on public IP ranges

* Do not allow dedicating a system VM IP range
2018-11-07 10:48:07 -02:00
Nicolas Vazquez 4de4eabd18
Enable DPDK support on KVM (#2839)
* Enable DPDK support on KVM

* Allow DPDK deployments on user VMs only

* Fix port name ordering
2018-11-07 09:29:01 -03:00
dahn 604d2d1bd2 make sure no time skew can interfere with comparison (#3003)
* make sure no time skew can interfere with comparison

* import
2018-11-07 10:04:40 -02:00
Rafael Weingärtner 077e692dbb Forward merge Fix DirectNetworkGuru canHandle for lowercase isolation methods
forward merge of #3010 -- Fix DirectNetworkGuru canHandle checks for lowercase isolation methods
2018-11-07 09:56:05 -02:00
Nicolas Vazquez af0c1e48cf Fix DirectNetworkGuru canHandle checks for lowercase isolation methods (#3010) 2018-11-07 09:53:01 -02:00
Rafael Weingärtner e175e93281
Fix set affinity for user overlay problem (Fixes #2986) (#2987)
This PR fixes #2986
2018-11-01 10:30:53 -03:00
Yoan Blanc 17c164d59a api: signature v3 to accept more formats (#2893)
It does it by reusing the DateUtil helpers. DateUtil uses java.time.* as that one knows how to deal
with timezones correctly.

The format expected by signatureVersion=3&expires=.... is quite limited.

It should accept the following formats that are containing a timezone and/or milliseconds.

2018-10-01T08:12:14Z
2018-10-01T08:12:14+01:00
2018-10-01T08:12:14+0100
2018-10-01T08:12:14.000Z
2018-10-01T08:12:14.000+01:00
2018-10-01T08:12:14.000+0100
afaik only 2018-10-01T08:12:14+0100 is accepted by the current codebase.

This PR echoes other pull requests I made earlier this year. #2392 and #2867

Signed-off-by: Yoan Blanc <yoan.blanc@exoscale.ch>
2018-10-31 16:57:48 +05:30
Rohit Yadav 7479e2877f Merge remote-tracking branch 'origin/4.11' 2018-10-30 15:15:35 +05:30
Rohit Yadav c6e53f6cc6
kvm: reset KVM host on heartbeat failure (#2984)
On actual testing, I could see that kvmheartbeat.sh script fails on NFS
server failure and stops the agent only. Any HA VMs could be launched
in different hosts, and recovery of NFS server could lead to a state
where a HA enabled VM runs on two hosts and can potentially cause
disk corruptions. In most cases, VM disk corruption will be worse than
VM downtime. I've kept the sleep interval between check/rounds but
reduced it to 10s. The change in behaviour was introduced in #2722.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-30 15:13:59 +05:30
Marc-Aurèle Brothier 3101612ad1 docker: add compose file for database (#2898)
A docker-compose file to setup quickly the database for CS. This helps during local development and remove the need to install a local MariaDB/MySQL.
2018-10-29 21:25:48 +05:30
Gabriel Beims Bräscher cdc6e6e50a CLOUDSTACK-9315: Removed unused Classes (#1448)
* Remove some unused Classes

These classes were deleted because they have no references in our code base. They are not in Spring execution flow nor instantiated with "new":
- com.cloud.agent.api.CheckStateAnswer
- com.cloud.agent.api.StartupVMMAgentCommand
- com.cloud.agent.api.routing.UserDataCommand
	- remove from description at
com.cloud.configuration.Config.ExecuteInSequenceNetworkElementCommands
enum
- com.cloud.agent.api.storage.UpgradeDiskCommand
- com.cloud.agent.api.storage.CreatePrivateTemplateCommand
- com.cloud.agent.api.storage.DestroyAnswer
	- Note: "FIXME: Should have an DestroyAnswer" at
com.cloud.storage.resource.StoragePoolResource
- com.cloud.agent.api.storage.UpgradeDiskAnswer
- com.cloud.agent.api.storage.ManageVolumeAvailabilityAnswer
- com.cloud.agent.api.storage.ManageVolumeAvailabilityCommand
- com.cloud.exception.UsageServerException
- com.cloud.info.SecStorageVmLoadInfo
- com.cloud.serializer.SerializerHelper

* PR#1448 update description of 'execute.in.sequence.network.element.commands' param

Update description of 'execute.in.sequence.network.element.commands'parameter to reflect an unused command that has been removed. The removed class command is 'UserDataCommand'.

* Add cloud schema to update SQL
2018-10-29 08:27:00 -03:00
Rohit Yadav 323d381767 Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-29 16:27:08 +05:30
Rene Diepstraten f7bc5807a3 vr: defer was broken in VR because of json name change
Committed at f0491d5c72 (#2979).

After upgrade from CS 4.10 to CS 4.11, multiple VRs did not start through.
It did not properly defer the finalize config in update_config.py.
Apparently, the json files are now called differently: where it used to
be vm_dhcp_entry.json it now has a uuid added, for example
vm_metadata.json.4d727b6e-2b48-49df-81c3-b8532f3d6745.
The if statement that checks if the finalize can be safely deferred
therefore no longer matches. This PR contains a fix so finalize is
defered again.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-29 16:19:33 +05:30
Nicolas Vazquez dffb430975 kvm: Fix migrating VM from ISO failures (#2928)
Prevents errors while migrating VM from ISO:

Test 1: Deploy VM from ISO -> Live migrate VM to another host -> ERROR
Test 2: Register ISO using Direct Download on KVM -> Deploy VM from ISO -> Live migrate VM to another host -> ERROR

- Prevent NullPointerException migrating VM from ISO
- Prevent mount secondary storage on ISO direct downloads on KVM
2018-10-29 16:14:20 +05:30
Rohit Yadav f0491d5c72
vr: defer was broken in VR because of json name change (#2979)
After upgrade from CS 4.10 to CS 4.11, multiple VRs did not start through.
It did not properly defer the finalize config in update_config.py.
Apparently, the json files are now called differently: where it used to
be vm_dhcp_entry.json it now has a uuid added, for example
vm_metadata.json.4d727b6e-2b48-49df-81c3-b8532f3d6745.
The if statement that checks if the finalize can be safely deferred
therefore no longer matches. This PR contains a fix so finalize is
defered again.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-29 16:11:43 +05:30
Rohit Yadav 8738ca75b1
kvm: use libvirtd as service name for all distributions (#2925)
Since we support only Ubuntu 16.04+ on master/4.12+, we can now use
the libvirt service name `libvirtd` for all distributions. This also
fixes an optional package name for libvirtd installation on Debian 9+.
Fixes #2909

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-29 11:55:28 +05:30
Rohit Yadav e2ba934c19
server: fix unwanted txn commit warning messages (#2927)
This fixes unwanted transaction commit warning messages such:

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-29 02:49:54 +05:30
Gabriel Beims Bräscher bfc326384d saml: redirect saml2 failed login message to a configurable URL (#2185)
When a user fails to authenticate with SAML2, it returns an error page
showing the content of the attached image.

To make it more user-friendly and customizable, one could configure a
desirable URL to redirect when such authentication failure happens.

This ticket proposes a global settings variable
(saml2.failed.login.redirect.url). If null, the SAML2 authentication
flow does not change from the current; however, if the user configures
an URL then ACS redirects to that URL.
2018-10-28 00:55:06 +05:30
alexanderbazhenoff a87acf93d8 kvm: improved performance on creating VM (#2923)
Improved performance on creating VM for KVM virtualization.

On a huge hosts every "ifconfig | grep" takes a lot of time (about 2.5-3 minutes on hosts with 500 machines). For example: ip link show dev $vlanDev > /dev/null is faster than ifconfig |grep -w $vlanDev > /dev/null. But using ip command is much better. Using this patch you can create 500s machine in 10 seconds. You don't need slow ifconfig prints anymore.
2018-10-25 16:28:13 +05:30
Rohit Yadav 9cf57d2568
network: on rolling restart force stop old routers (#2926)
This force stops old VRs when performing rolling restart with
cleanup=true. This will ensure that VRs are powered off quickly than
wait longer for the normal ACPI shutdown. During testing, it was found
on VMware where VM stops are slow compared to XenServer and KVM.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-25 09:20:39 +05:30
Rohit Yadav b8ed159f47 Merge remote-tracking branch 'origin/4.11' 2018-10-25 08:14:49 +05:30
Simon Weller c4b621a418 kvm: HyperV Enlightment for Improved Windows Server 2008+ Performance (#2870)
Windows has support for several paravirt features that it will use when running on Hyper-V, Microsoft's hypervisor. These features are called enlightenments. Many of the features are similar to paravirt functionality that exists with Linux on KVM (virtio, kvmclock, PV EOI, etc.)

Nowadays QEMU/KVM can also enable support for several Hyper-V enlightenments. When enabled, Windows VMs running on KVM will use many of the same paravirt optimizations they would use when running on Hyper-V.

A number of years ago, a PR was introduced that added a good portion of the code to enable this feature set, but it was never completed. This PR enables the existing features. The previous patch set detailed in #1013 also included the tests.

By selecting Windows PV, the enlightenment additions will be applied to the libvirt configuration. This is support on Windows Server 2008 and beyond, so all currently supported versions of Windows Server.

In our testing, we've seen benchmark improvements of around 20-25% running on Centos 7 hosts and it is also supported on Centos/RHEL 6.5 and later. Testing on Ubuntu would be appreciated.
2018-10-25 06:54:13 +05:30
Rohit Yadav 9b35b64b3c
packaging: install plugins at /usr/share/cloudstack-management/lib (#2915)
Install any additional plugin jars in the lib directory to be picked up
by the classpath builder, otherwise one has to manually add the jar
to /etc/default/cloudstack-management after installation. This fixes
the issue for `mysql-ha` plugin.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-24 18:21:03 +05:30
Abhishek 6f44a8d7ba template: Return requireshvm key as part of list template response (#2921)
Added requireshvm key in the response of list template API.
Fixes #2821

Signed-off-by: Abhishek Kumar <abhishek.mrt@gmail.com>
2018-10-24 16:20:27 +05:30
Rohit Yadav 8d31024a60 Merge remote-tracking branch 'origin/4.11' 2018-10-24 11:08:00 +05:30
Abhishek c8ca9e2fa0 template: create/updateTemplate should allow to set/change sshKeyEnabled (#2922)
Added sshKeyEnabled key in createTemplate and updateTemplate API.
Fixes #2822.

Signed-off-by: Abhishek Kumar <abhishek.mrt@gmail.com>
2018-10-24 11:07:03 +05:30
Rohit Yadav e092529c98
systemvm: Ensure cloud service reboots after failure (#2916)
This fixes an issue for systemvms (CPVM and SSVM) on VMware, as eth0
is not programmed (link-local) the networking.service fails to start
which is a dependency for cloud-postinit service. When cloud-postinit
service fails to start/run, it fails to start the agent (cloud) process.
This fixes the smoketest failures we saw in case of VMware 6.5 with
4.11.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-23 23:33:08 +05:30
Rafael Weingärtner 71e5a88fc1 Forward merge client: mgmt server listen default to 0.0.0.0 (#2907) 2018-10-22 16:32:06 -03:00
Abhishek d0fbd00e9b tools: Added '-B' flag for mvn current version cmd (#2910) (#2914)
maven command for finding current version might need to download packages and without batch mode it will end in variable. Added '-B' to enable batch mode.
2018-10-22 22:04:42 +05:30
Rohit Yadav 47c9c1cb58
client: mgmt server listen default to 0.0.0.0 (#2907)
This makes the management server listen on all interfaces by default.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-22 20:00:51 +05:30
Abhishek f5d52421b9 kvm: Remove libvirt-bin check for Debian (#2909) (#2912)
Debian does not have libvirt-bin package. Therefore, only for Ubuntu host distro process name libvirt-bin is used, otherwise libvirtd will be used
2018-10-22 17:48:11 +05:30
Nicolas Vazquez 5cf163d888 server: Unify templates/ISOs checksum API output (#2911)
Unify checksum API output for templates and ISOs: not list the checksum algorithm on:
KVM direct downloads

On in progress normal template downloads. The algorithm is shown on the listtemplates API, but after it is downloaded it is not shown anymore.
2018-10-21 22:33:04 +05:30
Rohit Yadav 233f46c94b Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-17 20:29:58 +05:30
Rohit Yadav 5ce14df31f
network: Allow ability to disable rolling restart feature (#2900)
This adds a global setting for admins who may not want the rolling
restart of routers or are seeing any issues around it. In future, this
setting may be removed.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-17 20:27:08 +05:30
Rohit Yadav e871638dc8 Merge remote-tracking branch 'origin/4.11' 2018-10-17 13:25:06 +05:30
Rohit Yadav 1904a70512
agent: on shutdown don't allow server reconnection (#2904)
When agent is stopped, don't allow reconnection. Previously this would
send a shutdown command to the management server which would put the
host state to Disconnected but then agent's reconnection logic may kick
in sometimes which would connect the agent to the management server
but then the agent process would terminate causing the host to be
put in Alert state (due to ping timeout or it waiting too long).

This fixes the issue by ensuring that when the agent is stopped, it
does not reconnect to the management server.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-17 06:31:13 +05:30
Gabriel Beims Bräscher 143fe5375c Allow enable the Java remote Debug for CloudStack Agent (#2897)
The idea is to create a property in /etc/defaults/cloudstack-agent that
can be enabled/disabled (uncommented/commented) to control the enabling
of Java remote debug in CloudStack's JVM.
2018-10-16 15:57:13 -03:00
Rafael Weingärtner 92f8cda64a Merge forward #2902 (Add checksum sanity validation on template registration) 2018-10-16 10:25:50 -03:00
Nicolas Vazquez 9003c7bfdc Add checksum sanity validation on template registration (#2902)
* Add checksum sanity validation on template registration

* Refactor

* Rename checksum sanity method
2018-10-16 10:21:20 -03:00
Pierre-Yves Ritschard 58b4e71b0e
params: allow signatureversion and expires without logging
This patch considers the new expires and signatureversion parameters
valid. Without this, all calls log when using the V3 signature scheme.
2018-10-16 09:01:46 +02:00
Rohit Yadav 84994c841f Merge remote-tracking branch 'origin/4.11' 2018-10-16 10:54:39 +05:30
Nicolas Vazquez 11d83fab43 agent: set log level to INFO as default for http wire (#2903)
Avoid logging bytes on direct download on KVM.
2018-10-16 10:32:03 +05:30
Rohit Yadav 933ee23104
vr: memory and swap optimizations (#2892)
This tries to provide a threshold based fix for #2873 where swappinness of VR is not used until last resort. By limiting swappiness unless actually needed, the VR system degradation can be avoided for most cases. The other change is around not starting baremetal-vr by default on all VRs, according to the spec https://cwiki.apache.org/confluence/display/CLOUDSTACK/Baremetal+Advanced+Networking+Support only vmware VRs need to run it and that too only as the last step of the setup/completion, so we don't need to run it all the time.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-16 10:29:48 +05:30