Commit Graph

31776 Commits

Author SHA1 Message Date
Rohit Yadav 92cc4514ea Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-12-28 15:20:23 +05:30
Nicolas Vazquez 13c81a8ee4 server: Prevent corner case for infinite PrepareForMaintenance (#3095)
A corner case was found on 4.11.2 for #2493 leading to an infinite loop in state PrepareForMaintenance

To prevent such cases, in which failed migrations are detected but still running on the host, this feature adds a new cluster setting host.maintenance.retries which is the number of retries before marking the host as ErrorInMaintenance if migration errors persist.

How Has This Been Tested?
- 2 KVM hosts, pick one which has running VMs as H
- Block migrations ports on H to simulate failures on migrations:
iptables -I OUTPUT -j REJECT -m state --state NEW -m tcp -p tcp --dport 49152:49215 -m comment --comment 'test block migrations' iptables -I OUTPUT -j REJECT -m state --state NEW -m tcp -p tcp --dport 16509 -m comment --comment 'test block migrations
- Put host H in Maintenance
- Observe that host is indefinitely in PrepareForMaintenance state (after this fix it goes into ErrorInMaintenance after retrying host.maintenance.retries times)
2018-12-28 15:14:16 +05:30
Rohit Yadav 3424d9e6ff Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-12-28 15:11:15 +05:30
Anurag Awasthi 3279707030 api: Throw InvalidParameterValueException for failing ApiArgValidator (#3108)
This is important because it helps in communicating back the exact
error to the API callee.

Current behavior is that ParamProcessWorker#processParameters catches
the exception and returns an incorrect type exception without the
proper message.
2018-12-24 23:50:55 +05:30
Rohit Yadav 68b4b84101
marvin: add missing test data for test_migration smoketest (#3106)
Add missing test data for test_migration smoketest, use test_template for smoketest.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-12-24 23:43:57 +05:30
Wido den Hollander 2699586d92 network: Send userdata to Virtual Router if IPv6 is enabled (#3100)
There is no reason to not send userdata+password to the VR as all
Instances in CloudStack are Dual-Stacked. They have IPv4 and IPv6
so they can query their metadata over IPv4 at the VR.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2018-12-24 23:42:57 +05:30
Gerd Müller cabef5305d server: Enhance bypass vlan overlap check (#3026)
This PR adds the possibility to select a checkbox for the parameter bypassvlanoverlapcheck to the ajax request createNetwork. The checkbox was added for Guest Network as well as for the L2 Guest Network. For L2 Guest Network a backend check for the existence of the flag bypassvlanoverlapcheck was added.
2018-12-24 12:46:54 +05:30
Anurag Awasthi a7ccbdc790 api: allow keyword search in listSSHKeyPairs (#2920) (#3098)
Adds support for keyword search that was ignored by listsshkeypairs command.

Fixes: #2920
2018-12-23 00:34:53 +05:30
Rohit Yadav 5067d560ce
README: Ho ho ho! (#3102)
Happy holidays - Merry Christmas and new year!
2018-12-20 16:47:38 +05:30
Gabriel Beims Bräscher bf209405e7 Allow KVM VM live migration with ROOT volume on file storage type (#2997)
* Allow KVM VM live migration with ROOT volume on file

* Allow KVM VM live migration with ROOT volume on file
- Add JUnit tests

* Address reviewers and change some variable names to ease future
implementation (developers can easily guess the name and use
autocomplete)
2018-12-14 09:01:28 -02:00
Craig Squire 8d53557ba7 api: don't throttle api discovery for listApis command (#2894)
Users reported that they weren't getting all apis listed in cloudmonkey when running a sync. After some debugging, I found that the problem is that the ApiDiscoveryService is calling ApiRateLimitServiceImpl.checkAccess(), so the results of the listApis command are being truncated because Cloudstack believes the user has exceeded their API throttling rate.

I enabled throttling with a 25 request per second limit. I then created a test role with only list* permissions and assigned it to a test user. When this user calls listApis, they will typically receive anywhere from 15-18 results. Checking the logs, you see The given user has reached his/her account api limit, please retry after 218 ms..

I raised the limit to 200 requests per second, restarted the management server and tried again. This time I got 143 results and no log messages about the user being throttled.
2018-12-12 23:55:32 +05:30
Rohit Yadav ecd2b95d49 Merge remote-tracking branch 'origin/4.11' 2018-12-07 23:46:04 +05:30
Rohit Yadav 408cce48a5
travis: fail fast if --with-marvin fails with nose (#3024)
* travis: fail fast if --with-marvin fails with nose

Install missing dependency pycrypto.
This fixes issue with recent Travis runs which gave incorrect results
around smoketests with simulator where each test run failed with an
error like "nosetests: error: no such option: --with-marvin".

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-12-07 23:45:19 +05:30
Wido den Hollander d36e1a63a7 server/test: Move test files to right location (#3085)
These files were not in the right directory and thus not being executed
by Maven.

By moving the files we make sure these tests are run again.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2018-12-07 22:42:52 +05:30
Bitworks LLC 9dce8a5dea kvm: Added two more device name patterns to valid bridge slaves (lo* and dummy*) (#3000)
Added dummy and lo devices to be treated as a normal bridge slave devs.
Fixes #2998  
Added two more device names (lo* and dummy*). Implemented tests. Code was refactored.
Improved paths concatenation code from "+" to Paths.get.
2018-12-07 01:59:00 +05:30
Wido den Hollander d96bc05d10 kvm: Use 'ip route show default 0.0.0.0/0' to find the default gateway (#3080)
If a host has many routes this can be a magnitude faster then printing
all the routes and grepping for the default.

In some situations the host might have a large amount of routes due to
dynamic routing being used like OSPF or BGP.

In addition fix a couple of loglines which were throwing messages on
DEBUG while WARN and ERROR should be used there.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2018-12-06 14:04:47 +05:30
Rohit Yadav 52f68a273a Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-12-04 16:39:21 +05:30
Craig Squire 290df5f423 api: Discover tags field on superclass of API responses (#3005)
Updated ApiServiceDiscoveryImpl to check superclasses of API responses for fields.

Fixes: #3002
2018-12-04 13:59:48 +05:30
Rohit Yadav 89c567add8
security: increase keystore setup/import timeout (#3076)
This increases and uses a default 15mins timeout for VR scripts and for
KVM agent increases timeout from 60s to 5mins. The timeout can
specifically occur when keystore does not get enough entropy from CPU
and script gets killed due to timeout. This is a very specific corner
case and generally should not happen on baremetal/prod environment, but
sometimes seen in nested/test environments.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-12-04 01:28:24 +05:30
Sven Vogel 17097929b6 packaging: correct permissions in spec file and fix class path specified variable (#3030)
Install CentOS 7 e.g. Build 1804 and Java build 1.8.0_181

if you inspect systemd in debug mode you will see some errors
1.
permission of the cloudstack-managment.service are not corretly set
2.
invalid classpath specified. it seems the string which is used will be divided... we now we use ${..} like the lines above ... confused
2018-12-01 01:38:01 +05:30
Rohit Yadav 496a639ad7 Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-11-30 22:26:09 +05:30
Henko 525ddfb717 Destroyvm also removes volumes (#2793)
* Allow user to detach and delete volumes when destroyinh VMs

* Minor code refactoring
2018-11-30 10:27:31 -03:00
dahn 9a4149e5dc utils: cleanup Macaddresses utils (#2660)
Cleanup parse code, fix java docs and remove unwanted comments.
2018-11-29 22:23:50 +05:30
Boris Stoyanov - a.k.a Bobby 44bc516609 api: move ostypeid from DB id to DB uuid, backports #2528 (#3066)
This is a backport to 4.11 of #2528
2018-11-29 22:20:51 +05:30
Rohit Yadav ac9562a4a1 Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-11-29 15:06:06 +05:30
Rohit Yadav 29b8a9da48
kvm: when untagged vxlan is used, use the default guest/public bridge (#3037)
When vxlan://untagged is used for public (or guest) network, use the
default public/guest bridge device same as how vlan://untagged works.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-11-28 22:22:30 +05:30
Rene Diepstraten d425a409fc sg: add secondary ips to the correct ipset based on ip family (#2990)
Currently secondary ipv6 addresses are added to the ipv4 ipset in security_group.py.
This doesn't work, so this patch adds a function to split a set of ips in ipv4 and ipv6 addresses.
Both the default_network_rules and network_rules_vmSecondaryIp functions now utilise this function and add the ips to the appropriate ipsets.
2018-11-28 19:30:13 +05:30
Rohit Yadav 5d7b33aedb
Fix null pointer exception when using ConfigKey.Scope.ManagementServer (#3062) 2018-11-28 16:37:45 +05:30
Boris Stoyanov - a.k.a Bobby d0c9a0a23b tests: Adding a try catch for Xen licensing issue (#3061)
Adds a skip if the exception is related to licensing of the particular host, for the scaleVM API test.
2018-11-28 16:37:14 +05:30
Andrija Panic 82cdf1c163 db: Add KVM Guest OS mapping for Windows Server 2019 (#3020)
Add KVM Guest OS mapping for Windows Server 2019
2018-11-28 08:53:32 +05:30
Rohit Yadav a84f7dfde9
marvin: add missing default test data (#3055)
This add missing test data for one of the keys for a recently added
migration test.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-11-28 00:17:13 +05:30
Anurag Awasthi 6a4a4daa46 api: Allow password enabled for iso (#2745) (#3042)
Both register and update iso commands can allow for allowing password
enabled.
Fixes: #2745

Fixe: #2745
2018-11-28 00:09:20 +05:30
GabrielBrascher e6149f6996 Fix null pointer exception when using ConfigKey.Scope.ManagementServer 2018-11-27 14:28:56 -02:00
Gabriel Beims Bräscher 2334145602
Support requesting a specific IPv4 address (#2595)
This commit allows deploying VMs with a specific IPv4 address.

DirectPodBasedNetworkGuru does not support requesting a custom
IP-Address while creating a new NIC/Instance, throwing the following
error:

    Error 530: Does not support custom ip allocation at this time:
NicProfile[0-0-null-null-null
    Unknown macro: { "cserrorcode"}

Some use-cases prefer the ability to request the IPv4 address which the
Instance will get.

This implementation adds unit test cases to cover and it was manually
tested in Basic Networking. I can perform more tests if requested.
2018-11-27 09:39:10 -02:00
Wido den Hollander c499be256a ipv6: Advanced Networking Security Groups are supported (#3053)
With earlier work in Basic Networking and the security group provider IPv6 is
supported and we can allow IPv6 to be supplied in networks with SG enabled.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2018-11-24 15:10:36 +05:30
Rohit Yadav 11e34ffd5b Merge remote-tracking branch 'origin/4.11' 2018-11-21 13:10:20 +05:30
Rohit Yadav 411f368845 Merge branch 'origin/4.11.2.0-RC20181113T0924' into 4.11
Doing a ignored RC5 tree merge to get 4.11.2.0 tag on 4.11 branch. This
should have been done before merging the commit to move to
4.11.3.0-SNAPSHOT version on 4.11 branch.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-11-21 13:09:19 +05:30
Rohit Yadav 4a55ef5a7d Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-11-21 13:00:42 +05:30
Rohit Yadav 29c1a89f1f Merge branch 'origin/4.11' 2018-11-21 12:57:19 +05:30
Rohit Yadav fbb0d92687
surefire: ignore system classloader to make tests run (#3038)
Due to issue described in Surefix bug:
https://issues.apache.org/jira/browse/SUREFIRE-1588

Debian-based users/developers can no longer build CloudStack 4.11+
branches. The other workaround is to have the following jvm property:
jdk.net.URLClassPath.disableClassPathURLCheck=true

Signed-off-by: Rohit Yadav <rohit@apache.org>
2018-11-20 21:13:44 +05:30
Paul Angus fb80e51307 Updating pom.xml version numbers for release 4.11.3.0-SNAPSHOT
Signed-off-by: Paul Angus <paul.angus@shapeblue.com>
2018-11-20 13:11:52 +00:00
Boris Stoyanov - a.k.a Bobby 529152959a Adding zone disablement during deletion of the range plus one more tr… (#3028)
## Description
<!--- Describe your changes in detail -->
Disabling the execution of tests that SystemVMs acquire mocked public IPs, since it's braking other tests along the execution.
<!-- For new features, provide link to FS, dev ML discussion etc. -->
<!-- In case of bug fix, the expected and actual behaviours, steps to reproduce. -->

<!-- When "Fixes: #<id>" is specified, the issue/PR will automatically be closed when this PR gets merged -->
<!-- For addressing multiple issues/PRs, use multiple "Fixes: #<id>" -->
<!-- Fixes: # -->

## Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [X] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [X] Bug fix (non-breaking change which fixes an issue)
- [ ] Enhancement (improves an existing feature and functionality)
- [X] Cleanup (Code refactoring and cleanup, that may add test cases)

## Screenshots (if appropriate):
NA
## How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
The tests were executed and passed locally

<!-- Please read the [CONTRIBUTING](https://github.com/apache/cloudstack/blob/master/CONTRIBUTING.md) document -->
2018-11-19 09:28:15 +01:00
Philipp Bankonier a216d0df3f Display mac address in nic detail view (#3034)
## Description
Adds the MAC address to the NICs detail view of an instance.

## Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)

## Screenshots:

![detailview_with_mac_address](https://user-images.githubusercontent.com/6997263/48567118-a2785d80-e8fc-11e8-9e24-2eda8fcb815a.png)

## How Has This Been Tested?
Tested locally to verify that the correct mac address gets displayed in the detail view.
2018-11-19 09:26:03 +01:00
Rohit Yadav 97680e7a33 Merge remote-tracking branch 'origin/4.11' 2018-11-14 16:53:56 +05:30
Gabriel Beims Bräscher e45bed74a5 server: remove unused StrategyPriority.PLUGIN. (#3014)
Remove unused StrategyPriority.PLUGIN enum. The PLUGIN Strategy priority is not used, except by three JUnit test methods.
2018-11-14 15:07:37 +05:30
Nicolas Vazquez 4fabbdb1ae test: Skip network migration tests for not supported hypervisors (#3021)
Skip network migration tests for not supported hypervisors instead of failing.
2018-11-14 15:04:08 +05:30
Rohit Yadav a75cfd4d06 Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-11-13 16:13:52 +05:30
Paul Angus 5aae410dfc Updating pom.xml version numbers for release 4.11.2.0
Signed-off-by: Paul Angus <paul.angus@shapeblue.com>
2018-11-13 09:24:27 +00:00
Rohit Yadav 4d8e75cec8
systemvmtemplate: update debian 9.6 iso url and checksum (#3022)
This fixes the failing systemvmtemplate build due to 404 on old ISO url.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-11-13 06:30:09 +05:30
Bitworks LLC f6e600e4d8 CLOUDSTACK-3009: Fix resource calculation CPU, RAM for accounts. (#3012)
The view "service_offering_view" doesn't include removed SOs, as a result when SO is removed, the bug happens. The PR introduces a change for resource calculation changing "service_offering_view" to "service_offering" table which has all service offerings.

Must be fixed in:

4.12
4.11
Fixes: #3009
2018-11-13 06:29:08 +05:30