Commit Graph

121 Commits

Author SHA1 Message Date
dahn 58466c8954 api: add command to list management servers (#3150)
* api: add command to list management servers

* api: add number of mangement servers in listInfrastructure command

* ui: add block for mangement servers on infra page

* api name resolution method cleanup
2019-02-01 16:23:39 -02:00
Wido den Hollander 9c4292cc45 network: Offerings do not have to have Security Grouping enabled (#3112)
Offerings can co-exist where on does provide Security Grouping in the
network, but other guest Networks have no Security Grouping.

In V(X)LAN isolation environments the L2 separation is handled by V(X)LAN
and protection between Instances is handled by Security Grouping.

There are multiple scenarios possible where one network has Security Grouping
enabled because that is required in that network.

In the other network, but in the same zone it could be a choice to have
Security Grouping disabled and allow all traffic to flow.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2019-01-28 15:38:08 -02:00
dahn b363fd49f7 Vmware offline migration (#2848)
* - Offline VM and Volume migration on Vmware hypervisor hosts
- Also add VM disk consolidation call on successful VM migrations

* Fix indentation of marvin test file and reformat against PEP8

* * Fix few comment typos
* Refactor debug messages to use String.format() when debug log level is enabled.

* Send list of commands returned by hypervisor Guru instead of explicitly selecting the first one

* Fix unhandled NPE during VM migration

* Revert back to distinct event descriptions for VM to host or storage pool migration

* Reformat test_primary_storage file against PEP-8 and Remove unused imports

* Revert back the deprecation messages in the custom StringUtils class to favour the use of the ApacheUtils
2019-01-25 10:05:13 -02:00
Bitworks LLC d68712eb7b CLOUDSTACK-3049: Implemented role update for account. (#3058) 2019-01-25 10:02:56 -02:00
Malcolm Taylor 97ddd8dffd Fix some LGTM alerts (#3143)
* Fix some LGTM alerts

* address review comments from Gabriel Brascher and Rafael Weingartner
2019-01-24 20:22:39 -02:00
Dingane Hlaluku 323f791efc IP address acquired with associate ip address is marked as source nat (#3125)
* CLOUDSTACK-4045 added a check for network state when determining whether a new IP should be source NAT. this prevents associated IP's to be marked as source NAT when the network is in allocated state, causing disassociateIpAddress to fail later

* Remove mock object that cause other tests to fail

* Remove underscores from variable types and add documentation for the created method

* Improve exception message to include network name

* Include network UUID with the Exception message and fix failing marvin test

* Rebase against latest master and format AssociateIPAddrCmd class
2019-01-23 10:05:16 -02:00
Wido den Hollander f967944d90 ipv6: Do not allow Secondary IPv6 addresses to be EUI-64 (#3136)
* netutils: Add method to verify if IPv6 Address is EUI-64

By checking if ff:fe is present in the address we can see if an IPv6 Address
is EUI-64 or not.

Signed-off-by: Wido den Hollander <wido@widodh.nl>

* ipv6: Do not allow a Secondary IPv6 address to be EUI-64

EUI-64 addresses should not be allowed as they can be used in the future by a to be
deployed Instance which has to obtain this address because it matches it's MAC.

In a /64 subnet there are more then enough other IPs available to be allocated to
Instances, therefor we can safely disallow the allocation of EUI-64 addresses.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2019-01-21 09:20:27 -02:00
Wido den Hollander af9fefc6b4 ipv6: Calculate IPv6 address instead of fetching one from a pool (#3077)
With IPv6 we are not using DHCP to allocate addresses, but using
StateLess Address Auto Configuration (SLAAC) a Instance will calculate
it's own address based on the Router Advertisements send out by the
routers in the network.

This Advertisement contains the IPv6 Subnet in use in that subnet and
allows to calculate the stable Address the Instance will obtain based
on it's MAC Address.

The existing code is 'dead code' as it has been written, but was never
used by any production code.

SLAAC only works properly with subnets of exactly 64-bits large.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2019-01-16 16:49:02 -02:00
Anurag Awasthi 1a6eb4b856 server: use resource UUID instead of resource ID in API response (#2527) (#3099)
List resourcedetails was adding DB ID instead of UUID. This is a security risk and needs fix.

Fixes: #2527
2019-01-15 14:05:16 +05:30
Rafael Weingärtner 1725130975 server: Fix failing test_nic_secondaryip_add_remove test (#3129)
This regression was introduced with PR #2773 (Support IPv6 address in addIpToNic). The contributor did not take into consideration that the method “addIpToNic” was designed to add/allocate other IPs to a NIC. If users did not specify an IP, ACS should generate one for the network where the NIC is plugged into.

Even though I am fixing this regression here, it is still important to highlight that for IPV6, the user is not able to allocate an IP without specifying it.
2019-01-15 13:50:29 +05:30
Gabriel Beims Bräscher 1e0be522b1
Add influxdb to statscollector (#3078)
* Add Support for InfluxDB on StatsCollector

* Code refactored to fit Inner Class architecture.

Due to the inner class structure, test case for some methods will not be
implemented. On the future it will be necessary to refactor the whole
StatsCOllector architecture and extract inner classes.

Each Inner Class that is a "stats collector" and sends data to Influx
will extend AbstractStatsCollector to send metrics to the correct
measure ("table"). For instance, HostCollector sends data to host_stats,
VmStatsCollector sends data to vm_stats.

Add ping test for ensure that the target InfluxDB host is reachable

* Address PR reviews

* Enhance and tests implemented addressing reviewers.

* Set variables to private
2019-01-09 11:22:35 +01:00
Rohit Yadav 92cc4514ea Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-12-28 15:20:23 +05:30
Rohit Yadav 3424d9e6ff Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-12-28 15:11:15 +05:30
Anurag Awasthi 3279707030 api: Throw InvalidParameterValueException for failing ApiArgValidator (#3108)
This is important because it helps in communicating back the exact
error to the API callee.

Current behavior is that ParamProcessWorker#processParameters catches
the exception and returns an incorrect type exception without the
proper message.
2018-12-24 23:50:55 +05:30
Wido den Hollander 2699586d92 network: Send userdata to Virtual Router if IPv6 is enabled (#3100)
There is no reason to not send userdata+password to the VR as all
Instances in CloudStack are Dual-Stacked. They have IPv4 and IPv6
so they can query their metadata over IPv4 at the VR.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2018-12-24 23:42:57 +05:30
Gabriel Beims Bräscher bf209405e7 Allow KVM VM live migration with ROOT volume on file storage type (#2997)
* Allow KVM VM live migration with ROOT volume on file

* Allow KVM VM live migration with ROOT volume on file
- Add JUnit tests

* Address reviewers and change some variable names to ease future
implementation (developers can easily guess the name and use
autocomplete)
2018-12-14 09:01:28 -02:00
Henko 525ddfb717 Destroyvm also removes volumes (#2793)
* Allow user to detach and delete volumes when destroyinh VMs

* Minor code refactoring
2018-11-30 10:27:31 -03:00
Anurag Awasthi 6a4a4daa46 api: Allow password enabled for iso (#2745) (#3042)
Both register and update iso commands can allow for allowing password
enabled.
Fixes: #2745

Fixe: #2745
2018-11-28 00:09:20 +05:30
Gabriel Beims Bräscher 2334145602
Support requesting a specific IPv4 address (#2595)
This commit allows deploying VMs with a specific IPv4 address.

DirectPodBasedNetworkGuru does not support requesting a custom
IP-Address while creating a new NIC/Instance, throwing the following
error:

    Error 530: Does not support custom ip allocation at this time:
NicProfile[0-0-null-null-null
    Unknown macro: { "cserrorcode"}

Some use-cases prefer the ability to request the IPv4 address which the
Instance will get.

This implementation adds unit test cases to cover and it was manually
tested in Basic Networking. I can perform more tests if requested.
2018-11-27 09:39:10 -02:00
Wido den Hollander c499be256a ipv6: Advanced Networking Security Groups are supported (#3053)
With earlier work in Basic Networking and the security group provider IPv6 is
supported and we can allow IPv6 to be supplied in networks with SG enabled.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2018-11-24 15:10:36 +05:30
Rohit Yadav 375264aaa4 Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-11-12 12:12:08 +05:30
Wido den Hollander 44c080da11 server: print log on INFO if Host reached Max Guests Limit (#3013)
This should not be in DEBUG as people would want to know that the host was skipped
because it didn't have enough slots available to run the VM.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2018-11-12 11:39:17 +05:30
Rafael Weingärtner f18326880b Forwardmerge #2980 - Fix set initial reservation on public IP ranges 2018-11-07 10:50:09 -02:00
Nicolas Vazquez 4de4eabd18
Enable DPDK support on KVM (#2839)
* Enable DPDK support on KVM

* Allow DPDK deployments on user VMs only

* Fix port name ordering
2018-11-07 09:29:01 -03:00
Rafael Weingärtner 077e692dbb Forward merge Fix DirectNetworkGuru canHandle for lowercase isolation methods
forward merge of #3010 -- Fix DirectNetworkGuru canHandle checks for lowercase isolation methods
2018-11-07 09:56:05 -02:00
Yoan Blanc 17c164d59a api: signature v3 to accept more formats (#2893)
It does it by reusing the DateUtil helpers. DateUtil uses java.time.* as that one knows how to deal
with timezones correctly.

The format expected by signatureVersion=3&expires=.... is quite limited.

It should accept the following formats that are containing a timezone and/or milliseconds.

2018-10-01T08:12:14Z
2018-10-01T08:12:14+01:00
2018-10-01T08:12:14+0100
2018-10-01T08:12:14.000Z
2018-10-01T08:12:14.000+01:00
2018-10-01T08:12:14.000+0100
afaik only 2018-10-01T08:12:14+0100 is accepted by the current codebase.

This PR echoes other pull requests I made earlier this year. #2392 and #2867

Signed-off-by: Yoan Blanc <yoan.blanc@exoscale.ch>
2018-10-31 16:57:48 +05:30
Gabriel Beims Bräscher cdc6e6e50a CLOUDSTACK-9315: Removed unused Classes (#1448)
* Remove some unused Classes

These classes were deleted because they have no references in our code base. They are not in Spring execution flow nor instantiated with "new":
- com.cloud.agent.api.CheckStateAnswer
- com.cloud.agent.api.StartupVMMAgentCommand
- com.cloud.agent.api.routing.UserDataCommand
	- remove from description at
com.cloud.configuration.Config.ExecuteInSequenceNetworkElementCommands
enum
- com.cloud.agent.api.storage.UpgradeDiskCommand
- com.cloud.agent.api.storage.CreatePrivateTemplateCommand
- com.cloud.agent.api.storage.DestroyAnswer
	- Note: "FIXME: Should have an DestroyAnswer" at
com.cloud.storage.resource.StoragePoolResource
- com.cloud.agent.api.storage.UpgradeDiskAnswer
- com.cloud.agent.api.storage.ManageVolumeAvailabilityAnswer
- com.cloud.agent.api.storage.ManageVolumeAvailabilityCommand
- com.cloud.exception.UsageServerException
- com.cloud.info.SecStorageVmLoadInfo
- com.cloud.serializer.SerializerHelper

* PR#1448 update description of 'execute.in.sequence.network.element.commands' param

Update description of 'execute.in.sequence.network.element.commands'parameter to reflect an unused command that has been removed. The removed class command is 'UserDataCommand'.

* Add cloud schema to update SQL
2018-10-29 08:27:00 -03:00
Rohit Yadav 323d381767 Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-29 16:27:08 +05:30
Abhishek 6f44a8d7ba template: Return requireshvm key as part of list template response (#2921)
Added requireshvm key in the response of list template API.
Fixes #2821

Signed-off-by: Abhishek Kumar <abhishek.mrt@gmail.com>
2018-10-24 16:20:27 +05:30
Abhishek c8ca9e2fa0 template: create/updateTemplate should allow to set/change sshKeyEnabled (#2922)
Added sshKeyEnabled key in createTemplate and updateTemplate API.
Fixes #2822.

Signed-off-by: Abhishek Kumar <abhishek.mrt@gmail.com>
2018-10-24 11:07:03 +05:30
Rafael Weingärtner 71e5a88fc1 Forward merge client: mgmt server listen default to 0.0.0.0 (#2907) 2018-10-22 16:32:06 -03:00
Rohit Yadav 233f46c94b Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-17 20:29:58 +05:30
Rafael Weingärtner 92f8cda64a Merge forward #2902 (Add checksum sanity validation on template registration) 2018-10-16 10:25:50 -03:00
Pierre-Yves Ritschard 58b4e71b0e
params: allow signatureversion and expires without logging
This patch considers the new expires and signatureversion parameters
valid. Without this, all calls log when using the V3 signature scheme.
2018-10-16 09:01:46 +02:00
Rohit Yadav bd9880003f Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-10 16:10:26 +05:30
Rohit Yadav b6302d4e90 Merge remote-tracking branch 'origin/4.11'
Conflicts resolved for:
	engine/orchestration/src/org/apache/cloudstack/engine/orchestration/NetworkOrchestrator.java
	engine/schema/src/com/cloud/vm/dao/UserVmDaoImpl.java
server/src/com/cloud/network/element/VirtualRouterElement.java
server/src/com/cloud/vm/UserVmManagerImpl.java
tools/marvin/setup.py

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-05 05:18:42 +05:30
Kui LIU d53fc94485 CLOUDSTACK-10365: Change the "getXXX" boolean method names to "isXXX" (#2847)
These boolean-return methods are named as "getXXX".
Other boolean-return methods are named as "isXXX".
Considering there methods will return boolean values, it should be more clear and consistent to rename them as "isXXX".
(rebase #2602 and #2816)
2018-09-22 17:20:48 +02:00
Rohit Yadav c49807f8f4 Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-09-12 14:17:29 +05:30
Khosrow Moossavi 56f918583d Remove 'iam' projects (#2817)
* Remove 'iam' projects

* Drop corresponding tables
2018-09-11 13:06:20 -03:00
Gabriel Beims Bräscher fbf488497f Support IPv6 address in addIpToNic (#2773)
The admin will manually need to add the address to the Instance, but the
Security Grouping should allow it.
2018-09-11 12:03:19 -03:00
Mike Tutkowski d12c106a47
Restrict the number of managed clustered file systems per compute cluster (#2500)
* Restrict the number of managed clustered file systems per compute cluster
2018-09-11 08:23:19 -06:00
Mike Tutkowski 568119d437
Merge pull request #2585 from syed/upstream-snapshot-archive
Add ability to archive snapshots on primary storage
2018-08-23 19:36:04 -06:00
Bruno P. Kinoshita 34ada079d3 s/pacakge/package (#2812) 2018-08-20 10:14:25 +02:00
Dingane Hlaluku 8993c0ea99 server: refactor UserVmDetailsDao duplicate field (#2807)
Removing UserVmDetailsDao duplicate field;
Found the following repeated field in the UserVmManagerImpl class
@Inject
private UserVmDetailsDao _vmDetailsDao;
@Inject
private UserVmDetailsDao _uservmDetailsDao;

Refactored to a single field;
@Inject
private UserVmDetailsDao userVmDetailsDao;

Similar to this PR: https://github.com/apache/cloudstack/pull/2750/files
2018-08-17 16:31:14 +05:30
Rohit Yadav a4529ddcfc Merge branch '4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-08-17 15:54:19 +05:30
lzh3636 e57003f93f Fix some log messages that do not match with their function (#2796) 2018-08-16 09:49:43 +02:00
SowjanyaPatha 35e008df6c CLOUDSTACK-10223 Snapshots are not getting deleted when domain is deleted (#2399) 2018-08-12 10:17:03 -03:00
Mike Tutkowski 46c56eaaf9 Merge release branch 4.11 to master
* 4.11:
  Changed the implementation of isVolumeOnManagedStorage(VolumeInfo) to check if the data store in question is for primary storage (and added a unit test from Daan Hoogland)
  vmware: reboot VR after mac updates (#2794)
2018-08-12 00:03:37 -06:00
Rohit Yadav 5e48c0b4c9 Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-08-08 12:20:56 +05:30
ernjvr 542d4da16c asyncjobs: add endtime to async jobs (#2739)
There is currently no functional mechanism that captures or persists the end time of when an asynchronous job has finished. As a result, users are not able to do any reporting about the duration of various asynchronous jobs in Cloudstack.
Link to FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Add+End+Time+To+Asynchronous+Jobs
2018-07-25 14:48:01 +05:30