Commit Graph

31742 Commits

Author SHA1 Message Date
Wido den Hollander c499be256a ipv6: Advanced Networking Security Groups are supported (#3053)
With earlier work in Basic Networking and the security group provider IPv6 is
supported and we can allow IPv6 to be supplied in networks with SG enabled.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2018-11-24 15:10:36 +05:30
Rohit Yadav 11e34ffd5b Merge remote-tracking branch 'origin/4.11' 2018-11-21 13:10:20 +05:30
Rohit Yadav 411f368845 Merge branch 'origin/4.11.2.0-RC20181113T0924' into 4.11
Doing a ignored RC5 tree merge to get 4.11.2.0 tag on 4.11 branch. This
should have been done before merging the commit to move to
4.11.3.0-SNAPSHOT version on 4.11 branch.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-11-21 13:09:19 +05:30
Rohit Yadav 4a55ef5a7d Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-11-21 13:00:42 +05:30
Rohit Yadav 29c1a89f1f Merge branch 'origin/4.11' 2018-11-21 12:57:19 +05:30
Rohit Yadav fbb0d92687
surefire: ignore system classloader to make tests run (#3038)
Due to issue described in Surefix bug:
https://issues.apache.org/jira/browse/SUREFIRE-1588

Debian-based users/developers can no longer build CloudStack 4.11+
branches. The other workaround is to have the following jvm property:
jdk.net.URLClassPath.disableClassPathURLCheck=true

Signed-off-by: Rohit Yadav <rohit@apache.org>
2018-11-20 21:13:44 +05:30
Paul Angus fb80e51307 Updating pom.xml version numbers for release 4.11.3.0-SNAPSHOT
Signed-off-by: Paul Angus <paul.angus@shapeblue.com>
2018-11-20 13:11:52 +00:00
Boris Stoyanov - a.k.a Bobby 529152959a Adding zone disablement during deletion of the range plus one more tr… (#3028)
## Description
<!--- Describe your changes in detail -->
Disabling the execution of tests that SystemVMs acquire mocked public IPs, since it's braking other tests along the execution.
<!-- For new features, provide link to FS, dev ML discussion etc. -->
<!-- In case of bug fix, the expected and actual behaviours, steps to reproduce. -->

<!-- When "Fixes: #<id>" is specified, the issue/PR will automatically be closed when this PR gets merged -->
<!-- For addressing multiple issues/PRs, use multiple "Fixes: #<id>" -->
<!-- Fixes: # -->

## Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [X] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [X] Bug fix (non-breaking change which fixes an issue)
- [ ] Enhancement (improves an existing feature and functionality)
- [X] Cleanup (Code refactoring and cleanup, that may add test cases)

## Screenshots (if appropriate):
NA
## How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
The tests were executed and passed locally

<!-- Please read the [CONTRIBUTING](https://github.com/apache/cloudstack/blob/master/CONTRIBUTING.md) document -->
2018-11-19 09:28:15 +01:00
Philipp Bankonier a216d0df3f Display mac address in nic detail view (#3034)
## Description
Adds the MAC address to the NICs detail view of an instance.

## Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)

## Screenshots:

![detailview_with_mac_address](https://user-images.githubusercontent.com/6997263/48567118-a2785d80-e8fc-11e8-9e24-2eda8fcb815a.png)

## How Has This Been Tested?
Tested locally to verify that the correct mac address gets displayed in the detail view.
2018-11-19 09:26:03 +01:00
Rohit Yadav 97680e7a33 Merge remote-tracking branch 'origin/4.11' 2018-11-14 16:53:56 +05:30
Gabriel Beims Bräscher e45bed74a5 server: remove unused StrategyPriority.PLUGIN. (#3014)
Remove unused StrategyPriority.PLUGIN enum. The PLUGIN Strategy priority is not used, except by three JUnit test methods.
2018-11-14 15:07:37 +05:30
Nicolas Vazquez 4fabbdb1ae test: Skip network migration tests for not supported hypervisors (#3021)
Skip network migration tests for not supported hypervisors instead of failing.
2018-11-14 15:04:08 +05:30
Rohit Yadav a75cfd4d06 Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-11-13 16:13:52 +05:30
Paul Angus 5aae410dfc Updating pom.xml version numbers for release 4.11.2.0
Signed-off-by: Paul Angus <paul.angus@shapeblue.com>
2018-11-13 09:24:27 +00:00
Rohit Yadav 4d8e75cec8
systemvmtemplate: update debian 9.6 iso url and checksum (#3022)
This fixes the failing systemvmtemplate build due to 404 on old ISO url.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-11-13 06:30:09 +05:30
Bitworks LLC f6e600e4d8 CLOUDSTACK-3009: Fix resource calculation CPU, RAM for accounts. (#3012)
The view "service_offering_view" doesn't include removed SOs, as a result when SO is removed, the bug happens. The PR introduces a change for resource calculation changing "service_offering_view" to "service_offering" table which has all service offerings.

Must be fixed in:

4.12
4.11
Fixes: #3009
2018-11-13 06:29:08 +05:30
Paul Angus f95aec4a84
Merge pull request #3018 from shapeblue/fixrouterfilecreation
Prevent error on GroupAnswers on VR creation
2018-11-12 13:25:08 +00:00
Wido den Hollander 4809fe7cb4 kvm/ceph: When create a new RBD image use modern flags/features for the image (#2985)
These additional RBD features allow for faster lookups of how much space a RBD
image is using, but with the exclusive locking we prevent two VMs from writing
to the same RBD image at the same time.

These are the default features used by Ceph for any new RBD image.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2018-11-12 10:08:54 -02:00
Rohit Yadav 375264aaa4 Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-11-12 12:12:08 +05:30
Andrija Panic df173859d7 agent: add host.reserved.mem.mb parameter documentation (#3016)
Document "host.reserved.mem.mb" parameter in agent.properties.
2018-11-12 11:40:37 +05:30
Wido den Hollander 44c080da11 server: print log on INFO if Host reached Max Guests Limit (#3013)
This should not be in DEBUG as people would want to know that the host was skipped
because it didn't have enough slots available to run the VM.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2018-11-12 11:39:17 +05:30
Rohit Yadav d4d91f355d
vmware: updateVmwareDc API for updating vmware datacenter details (#2919)
This adds a new API updateVmwareDc that allows admins to update the
VMware datacenter details of a zone. It also recursively updates
the cluster_details for any username/password updates
as well as updates the url detail in cluster_details table and guid
detail in the host_details table with any newly provided vcenter
domain/ip. The update API assumes that there is only one vCenter per
zone. And, since the username/password for each VMware host could be different
than what gets configured for vcenter at zone level, it does not update the
username/password in host_details.

Previously, one has to manually update the db with any new vcenter details for the zone.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-11-12 11:32:36 +05:30
Nicolas Vazquez bb7493ad4b configdrive: Add missing ConfigDrive entries on existing zones after upgrade (#3007)
After upgrade existing environments to 4.11, ConfigDrive cannot be enabled for existing zones due to missing entry on 'physical_network_service_providers' table.
2018-11-12 11:30:00 +05:30
Rafael Weingärtner 63094b2fbf
Fix delete domain button in the UI to update only after job finishes (#2905)
When the “deleteDomain” command is sent via UI, the HTML is already updated assuming that the “deleteDomain” command will actually delete it. However, there are cases (domain with allocated resources) that will not allow ACS to remove the command; as a consequence, the UI will not be in sync with ACS state, which can lead to misunderstandings.
2018-11-11 18:41:37 -02:00
Mike Tutkowski f36e0c77aa
Adding three new settings into cloud.configuration. (#3019) 2018-11-09 15:21:39 -07:00
nvazquez dea0b3eb78 Prevent error on GroupAnswers on VR creation 2018-11-09 15:30:57 -03:00
Sven Vogel e44fc4af68 add exitcode / distribution detection and remove export CGROUP_DAEMON for RHEL7/CentOS7 (#2896)
* task/add distribution detection, remove export CGROUP_DAEMON='cpu:/virt'

* taks/add exit values

* change exit codes

* split redhat and centos / change ubuntu to /etc/lsb-release file

* turn around centos and rhel, we need first check for centos then rhel

* set variable distro to None
2018-11-07 18:22:20 -02:00
Rafael Weingärtner f18326880b Forwardmerge #2980 - Fix set initial reservation on public IP ranges 2018-11-07 10:50:09 -02:00
Nicolas Vazquez 7d8eb37924 [4.11] Fix set initial reservation on public IP ranges (#2980)
* Fix initial reservation on public IP ranges

* Do not allow dedicating a system VM IP range
2018-11-07 10:48:07 -02:00
Nicolas Vazquez 4de4eabd18
Enable DPDK support on KVM (#2839)
* Enable DPDK support on KVM

* Allow DPDK deployments on user VMs only

* Fix port name ordering
2018-11-07 09:29:01 -03:00
dahn 604d2d1bd2 make sure no time skew can interfere with comparison (#3003)
* make sure no time skew can interfere with comparison

* import
2018-11-07 10:04:40 -02:00
Rafael Weingärtner 077e692dbb Forward merge Fix DirectNetworkGuru canHandle for lowercase isolation methods
forward merge of #3010 -- Fix DirectNetworkGuru canHandle checks for lowercase isolation methods
2018-11-07 09:56:05 -02:00
Nicolas Vazquez af0c1e48cf Fix DirectNetworkGuru canHandle checks for lowercase isolation methods (#3010) 2018-11-07 09:53:01 -02:00
Rafael Weingärtner e175e93281
Fix set affinity for user overlay problem (Fixes #2986) (#2987)
This PR fixes #2986
2018-11-01 10:30:53 -03:00
Yoan Blanc 17c164d59a api: signature v3 to accept more formats (#2893)
It does it by reusing the DateUtil helpers. DateUtil uses java.time.* as that one knows how to deal
with timezones correctly.

The format expected by signatureVersion=3&expires=.... is quite limited.

It should accept the following formats that are containing a timezone and/or milliseconds.

2018-10-01T08:12:14Z
2018-10-01T08:12:14+01:00
2018-10-01T08:12:14+0100
2018-10-01T08:12:14.000Z
2018-10-01T08:12:14.000+01:00
2018-10-01T08:12:14.000+0100
afaik only 2018-10-01T08:12:14+0100 is accepted by the current codebase.

This PR echoes other pull requests I made earlier this year. #2392 and #2867

Signed-off-by: Yoan Blanc <yoan.blanc@exoscale.ch>
2018-10-31 16:57:48 +05:30
Rohit Yadav 7479e2877f Merge remote-tracking branch 'origin/4.11' 2018-10-30 15:15:35 +05:30
Rohit Yadav c6e53f6cc6
kvm: reset KVM host on heartbeat failure (#2984)
On actual testing, I could see that kvmheartbeat.sh script fails on NFS
server failure and stops the agent only. Any HA VMs could be launched
in different hosts, and recovery of NFS server could lead to a state
where a HA enabled VM runs on two hosts and can potentially cause
disk corruptions. In most cases, VM disk corruption will be worse than
VM downtime. I've kept the sleep interval between check/rounds but
reduced it to 10s. The change in behaviour was introduced in #2722.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-30 15:13:59 +05:30
Marc-Aurèle Brothier 3101612ad1 docker: add compose file for database (#2898)
A docker-compose file to setup quickly the database for CS. This helps during local development and remove the need to install a local MariaDB/MySQL.
2018-10-29 21:25:48 +05:30
Gabriel Beims Bräscher cdc6e6e50a CLOUDSTACK-9315: Removed unused Classes (#1448)
* Remove some unused Classes

These classes were deleted because they have no references in our code base. They are not in Spring execution flow nor instantiated with "new":
- com.cloud.agent.api.CheckStateAnswer
- com.cloud.agent.api.StartupVMMAgentCommand
- com.cloud.agent.api.routing.UserDataCommand
	- remove from description at
com.cloud.configuration.Config.ExecuteInSequenceNetworkElementCommands
enum
- com.cloud.agent.api.storage.UpgradeDiskCommand
- com.cloud.agent.api.storage.CreatePrivateTemplateCommand
- com.cloud.agent.api.storage.DestroyAnswer
	- Note: "FIXME: Should have an DestroyAnswer" at
com.cloud.storage.resource.StoragePoolResource
- com.cloud.agent.api.storage.UpgradeDiskAnswer
- com.cloud.agent.api.storage.ManageVolumeAvailabilityAnswer
- com.cloud.agent.api.storage.ManageVolumeAvailabilityCommand
- com.cloud.exception.UsageServerException
- com.cloud.info.SecStorageVmLoadInfo
- com.cloud.serializer.SerializerHelper

* PR#1448 update description of 'execute.in.sequence.network.element.commands' param

Update description of 'execute.in.sequence.network.element.commands'parameter to reflect an unused command that has been removed. The removed class command is 'UserDataCommand'.

* Add cloud schema to update SQL
2018-10-29 08:27:00 -03:00
Rohit Yadav 323d381767 Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-29 16:27:08 +05:30
Rene Diepstraten f7bc5807a3 vr: defer was broken in VR because of json name change
Committed at f0491d5c72 (#2979).

After upgrade from CS 4.10 to CS 4.11, multiple VRs did not start through.
It did not properly defer the finalize config in update_config.py.
Apparently, the json files are now called differently: where it used to
be vm_dhcp_entry.json it now has a uuid added, for example
vm_metadata.json.4d727b6e-2b48-49df-81c3-b8532f3d6745.
The if statement that checks if the finalize can be safely deferred
therefore no longer matches. This PR contains a fix so finalize is
defered again.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-29 16:19:33 +05:30
Nicolas Vazquez dffb430975 kvm: Fix migrating VM from ISO failures (#2928)
Prevents errors while migrating VM from ISO:

Test 1: Deploy VM from ISO -> Live migrate VM to another host -> ERROR
Test 2: Register ISO using Direct Download on KVM -> Deploy VM from ISO -> Live migrate VM to another host -> ERROR

- Prevent NullPointerException migrating VM from ISO
- Prevent mount secondary storage on ISO direct downloads on KVM
2018-10-29 16:14:20 +05:30
Rohit Yadav f0491d5c72
vr: defer was broken in VR because of json name change (#2979)
After upgrade from CS 4.10 to CS 4.11, multiple VRs did not start through.
It did not properly defer the finalize config in update_config.py.
Apparently, the json files are now called differently: where it used to
be vm_dhcp_entry.json it now has a uuid added, for example
vm_metadata.json.4d727b6e-2b48-49df-81c3-b8532f3d6745.
The if statement that checks if the finalize can be safely deferred
therefore no longer matches. This PR contains a fix so finalize is
defered again.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-29 16:11:43 +05:30
Rohit Yadav 8738ca75b1
kvm: use libvirtd as service name for all distributions (#2925)
Since we support only Ubuntu 16.04+ on master/4.12+, we can now use
the libvirt service name `libvirtd` for all distributions. This also
fixes an optional package name for libvirtd installation on Debian 9+.
Fixes #2909

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-29 11:55:28 +05:30
Rohit Yadav e2ba934c19
server: fix unwanted txn commit warning messages (#2927)
This fixes unwanted transaction commit warning messages such:

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-29 02:49:54 +05:30
Gabriel Beims Bräscher bfc326384d saml: redirect saml2 failed login message to a configurable URL (#2185)
When a user fails to authenticate with SAML2, it returns an error page
showing the content of the attached image.

To make it more user-friendly and customizable, one could configure a
desirable URL to redirect when such authentication failure happens.

This ticket proposes a global settings variable
(saml2.failed.login.redirect.url). If null, the SAML2 authentication
flow does not change from the current; however, if the user configures
an URL then ACS redirects to that URL.
2018-10-28 00:55:06 +05:30
alexanderbazhenoff a87acf93d8 kvm: improved performance on creating VM (#2923)
Improved performance on creating VM for KVM virtualization.

On a huge hosts every "ifconfig | grep" takes a lot of time (about 2.5-3 minutes on hosts with 500 machines). For example: ip link show dev $vlanDev > /dev/null is faster than ifconfig |grep -w $vlanDev > /dev/null. But using ip command is much better. Using this patch you can create 500s machine in 10 seconds. You don't need slow ifconfig prints anymore.
2018-10-25 16:28:13 +05:30
Rohit Yadav 9cf57d2568
network: on rolling restart force stop old routers (#2926)
This force stops old VRs when performing rolling restart with
cleanup=true. This will ensure that VRs are powered off quickly than
wait longer for the normal ACPI shutdown. During testing, it was found
on VMware where VM stops are slow compared to XenServer and KVM.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-25 09:20:39 +05:30
Rohit Yadav b8ed159f47 Merge remote-tracking branch 'origin/4.11' 2018-10-25 08:14:49 +05:30
Simon Weller c4b621a418 kvm: HyperV Enlightment for Improved Windows Server 2008+ Performance (#2870)
Windows has support for several paravirt features that it will use when running on Hyper-V, Microsoft's hypervisor. These features are called enlightenments. Many of the features are similar to paravirt functionality that exists with Linux on KVM (virtio, kvmclock, PV EOI, etc.)

Nowadays QEMU/KVM can also enable support for several Hyper-V enlightenments. When enabled, Windows VMs running on KVM will use many of the same paravirt optimizations they would use when running on Hyper-V.

A number of years ago, a PR was introduced that added a good portion of the code to enable this feature set, but it was never completed. This PR enables the existing features. The previous patch set detailed in #1013 also included the tests.

By selecting Windows PV, the enlightenment additions will be applied to the libvirt configuration. This is support on Windows Server 2008 and beyond, so all currently supported versions of Windows Server.

In our testing, we've seen benchmark improvements of around 20-25% running on Centos 7 hosts and it is also supported on Centos/RHEL 6.5 and later. Testing on Ubuntu would be appreciated.
2018-10-25 06:54:13 +05:30