Host-HA offers investigation, fencing and recovery mechanisms for host that for
any reason are malfunctioning. It uses Activity and Health checks to determine
current host state based on which it may degrade a host or try to recover it. On
failing to recover it, it may try to fence the host.
The core feature is implemented in a hypervisor agnostic way, with two separate
implementations of the driver/provider for Simulator and KVM hypervisors. The
framework also allows for implementation of other hypervisor specific provider
implementation in future.
The Host-HA provider implementation for KVM hypervisor uses the out-of-band
management sub-system to issue IPMI calls to reset (recover) or poweroff (fence)
a host.
The Host-HA provider implementation for Simulator provides a means of testing
and validating the core framework implementation.
Signed-off-by: Abhinandan Prateek <abhinandan.prateek@shapeblue.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This introduces a new certificate authority framework that allows
pluggable CA provider implementations to handle certificate operations
around issuance, revocation and propagation. The framework injects
itself to `NioServer` to handle agent connections securely. The
framework adds assumptions in `NioClient` that a keystore if available
with known name `cloud.jks` will be used for SSL negotiations and
handshake.
This includes a default 'root' CA provider plugin which creates its own
self-signed root certificate authority on first run and uses it for
issuance and provisioning of certificate to CloudStack agents such as
the KVM, CPVM and SSVM agents and also for the management server for
peer clustering.
Additional changes and notes:
- Comma separate list of management server IPs can be set to the 'host'
global setting. Newly provisioned agents (KVM/CPVM/SSVM etc) will get
radomized comma separated list to which they will attempt connection
or reconnection in provided order. This removes need of a TCP LB on
port 8250 (default) of the management server(s).
- All fresh deployment will enforce two-way SSL authentication where
connecting agents will be required to present certificates issued
by the 'root' CA plugin.
- Existing environment on upgrade will continue to use one-way SSL
authentication and connecting agents will not be required to present
certificates.
- A script `keystore-setup` is responsible for initial keystore setup
and CSR generation on the agent/hosts.
- A script `keystore-cert-import` is responsible for import provided
certificate payload to the java keystore file.
- Agent security (keystore, certificates etc) are setup initially using
SSH, and later provisioning is handled via an existing agent connection
using command-answers. The supported clients and agents are limited to
CPVM, SSVM, and KVM agents, and clustered management server (peering).
- Certificate revocation does not revoke an existing agent-mgmt server
connection, however rejects a revoked certificate used during SSL
handshake.
- Older `cloudstackmanagement.keystore` is deprecated and will no longer
be used by mgmt server(s) for SSL negotiations and handshake. New
keystores will be named `cloud.jks`, any additional SSL certificates
should not be imported in it for use with tomcat etc. The `cloud.jks`
keystore is stricly used for agent-server communications.
- Management server keystore are validated and renewed on start up only,
the validity of them are same as the CA certificates.
New APIs:
- listCaProviders: lists all available CA provider plugins
- listCaCertificate: lists the CA certificate(s)
- issueCertificate: issues X509 client certificate with/without a CSR
- provisionCertificate: provisions certificate to a host
- revokeCertificate: revokes a client certificate using its serial
Global settings for the CA framework:
- ca.framework.provider.plugin: The configured CA provider plugin
- ca.framework.cert.keysize: The key size for certificate generation
- ca.framework.cert.signature.algorithm: The certificate signature algorithm
- ca.framework.cert.validity.period: Certificate validity in days
- ca.framework.cert.automatic.renewal: Certificate auto-renewal setting
- ca.framework.background.task.delay: CA background task delay/interval
- ca.framework.cert.expiry.alert.period: Days to check and alert expiring certificates
Global settings for the default 'root' CA provider:
- ca.plugin.root.private.key: (hidden/encrypted) CA private key
- ca.plugin.root.public.key: (hidden/encrypted) CA public key
- ca.plugin.root.ca.certificate: (hidden/encrypted) CA certificate
- ca.plugin.root.issuer.dn: The CA issue distinguished name
- ca.plugin.root.auth.strictness: Are clients required to present certificates
- ca.plugin.root.allow.expired.cert: Are clients with expired certificates allowed
UI changes:
- Button to download/save the CA certificates.
Misc changes:
- Upgrades bountycastle version and uses newer classes
- Refactors SAMLUtil to use new CertUtils
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This feature allows changing permission for existing role permissions, as those were static and could not be changed once created. It also provides the ability to change these permissions in the UI using a drop down menu for each permission rule, in which admin can select ‘Allow’ or ‘Deny’ permission.
Changes in the API:
This feature modifies behaviour of updateRolePermission API method:
New optional parameters ‘ruleid’ and ‘permission’ are introduced, they are mutual exclusive to ‘ruleorder’ parameter. This defines two use cases:
Update role permission: ‘ruleid’ and ‘permission’ parameters needed
Update rules order: ‘ruleorder’ parameter needed
Parameter ‘ruleorder’ is now optional
updateRolePermission providing ‘ruleorder’ parameter should be sent via POST
A simple if-statement would fail if either the type or code were 0
as that if-statement failed them.
By checking if they are defined and casting them to a String afterwards
this makes the if-statement properly resolve and show the rule as it should.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
The current VPN users are added in the VPN tab inside the public IP after the VPN is enabled. For each network(for which VPN is supported and enabled), the VPN users are shared. As the Cloudstack doc says “ The account owner can create and manage users for their VPN. CloudStack does not use its account database for this purpose but uses a separate table. The VPN user database is shared across all the VPNs created by the account owner. All VPN users get access to all VPNs created by the account owner.”
The current implementation of going inside each network and adding VPN users give the first feel as if the users are network based. To fix this, Shifted the VPN users to networks tab view.
This commit contains following changes
(1) add CPU CORE information in op_host_capacity
(2) add capacity name in the CapacityResponse
(3) add allocatedCapacity for CPU/MEMORY/CPU CORE for zones
(4) sort CapacityResponse by zonename and CapacityType
This allows native CloudStack users to change password in UI when LDAP
is enabled. Overall changes:
- A new usersource returned in the listUsers response
- Removed ldap check in the UI, replaced with check based on user source
- DB changes to include user.source in user_view
- Changed UI error message for non-native users trying to change password
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9669:egress destination cidr VR python script changes
CLOUDSTACK-9669:egress destination API and orchestration changes
CLOUDSTACK-9669: Added the ipset package in systemvm template
CLOUDSTACK-9669:Added licence header for new files
CLOUDSTACK-9669: replacing 0.0.0.0/0 with the network cidr
ipset member add with 0.0.0.0/0 fails. So 0.0.0.0/0 replaced with the network cidr.
In source cidr 0.0.0.0/0 is nothing but network cidr.
updated the default egress all cidr with network cidr
* 4.9:
CLOUDSTACK-9857: With this change if agent dies the systemd will catch it properly and show process as exited
CLOUDSTACK-9805: Display VR list in network details
CLOUDSTACK-9356: FIX Cannot add users in VPC VPN
- commented some occurences of cloud.com as being harmless
* examples
* identifiers (internal)
- changed the URL for vhd-util download
- changed comments from 'cloud.com' to 'Apache CloudStack'
Reviewed-By: Rashmi Dixit
Problem: All the hosts suitable for VM Migration are not shown in the UI. This could
confuse the user as the target host might never be shown in the UI.
Root Cause: The API (findHostsForMigration) always returned page 1 results which would
be always <= default.page.size global parameter. Therefore, in case of large
no. of hosts where the result can map to multiple pages, this issue would arise.
Solution: 1. Replace drop-down with listView widget.
2. Allow lazy-loading of records on listView's scroll.
3. Show additional parameters (CPU/Memory used) to assist admin in decision making.
4. Provide 'Search by host name' to limit the results.
Added change where if there are no hosts found, an empty row with message will
appear.
Displays a VR tab that lists VRs for the network in the detail views for
isolated networks, shared networks and for VPCs.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This adds support for virtio-scsi on KVM hosts, either
for guests that are associated with a new os_type of 'Other PV Virtio-SCSI (64-bit)',
or when a VM or template is regstered with a detail parameter rootDiskController=scsi.
Update cloudstack add template dialog to allow for selecting rootDiskController with KVM
Update cloudstack kvm virtio-scsi to enable discard=unmap
CLOUDSTACK-9784 : GPU detail not displayed in GPU tab of management server UI.ISSUE
==================
When GPU tab of the host is selected on the management server UI, no GPU detail is displayed.
RESOLUTION
==================
In the javascript file "system.js" while fetching the GPU details, sort functionality in dataprovider is returning value as undefined and hence it throwing an exception. So handled the output as undefined gracefully to avoid exception.
**Screenshot before applying fix :**
**Screenshot after applying fix :**
* pr/1942:
CLOUDSTACK-9784 : GPU detail not displayed in GPU tab of management server UI.
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
This improves the metrics view feature by improving the rendering performance
of metrics view tables, by reimplementing the logic at the backend and data
served via APIs. In large environments, the older implementation would
make several API calls that increases both network and database load.
List of APIs introduced for improving the performance:
listClustersMetrics
listHostsMetrics
listInfrastructure
listStoragePoolsMetrics
listVMsMetrics
listVolumesMetrics
listZonesMetrics
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9768: Time displayed for events in UI is incorrectTime displayed for events in UI is incorrect. Let's say, when we login using Japanese language the time displayed in the events is GMT instead of JST. However with English language the time is JST, as expected.
Example:
Time is displayed in the event is 10:40, if you are logged in using English language.
Whereas, time in the event shows 19:40 If you login with Japanese language.
* pr/1926:
CLOUDSTACK-9768: Time displayed for events in UI is incorrect
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
CLOUDSTACK-9228: Network update with mistmatch in services require forced option# Steps to reproduce:
1.Bring up CloudStack in advanced zone
2.Create isolated network with sourcenat, pf, lb, firewall services
3.Deploy a VM in the above network
4.Create another network offering with sourcenat, pf, firewall services
5.Try to update the network with offering created in step4
# Result:
The new offering:DefaultIsolatedNetworkOfferingForVpcNetworksNoLB will remove the following services [Lb]along with all the related configuration currently in use. will not proceed with the network update.set forced parameter to true for forcing an update."
# Workaround:
Use api with forced=true
# Fix:
Added a confirmation dialog box to check whether to make force update or not.
The dialog appears only for the Admin. Only admin can make force update.
The new dialog appears after the first CIDR unchanged confirmation dialog.
* pr/1333:
CLOUDSTACK-9228: Network update with mistmatch in services require forced option
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
CLOUDSTACK-9588: Add Load Balancer functionality in Network page is Redundant.Steps to Reproduce:
Network -> Select any network -> Observe Add Load Balancer tab
The "Add Load Balancer" functionality is redundant.
The above is used to create LB rule without any public IP.
Resolution:
There exist similar functionality in Network -> Any Network -> Details Tab -> View IP Addresses -> Any public IP -> Configuration Tab -> Observe Load Balancing.
The above is used to create LB rule with a public IP. This is a more convenient way of creating LB rule as the IP is involved.
* pr/1758:
CLOUDSTACK-9588: Add Load Balancer functionality in Network page is redundant. The "Add Load Balancer" functionality is redundant. The above is used to create LB rule without any public IP. This commit removes the tab from network page.
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
CLOUDSTACK-9679:Allow master user to manage subordinate user uploaded template
* pr/1834:
CLOUDSTACK-9679:Allow master user to manage subordinate user uploaded template
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
CLOUDSTACK-9574: Redesign storage views## Part 1: Redesign storage tags
### Actual behavior
Primary storage tags are being saved as an entry on `storage_pool_details` with:
* name = TAG_NAME
* value = "true"
When a boolean property is defined in {{storage_pool_details}} and has value = "true", it is displayed as a tag.
### Goal
Redesign `Storage Tags` for Primary Storage view, to list only tags, as it is done in Host Tags (Hosts view).
## Part 2: Remove details from listImageStores API call response and UI
### Description
In Secondary Storage view we propose removing `Details` field, as `Setting` tab list details for a given image store. We also remove details from response on `listImageStores` API method
* pr/1747:
CLOUDSTACK-9574: Redesign storage tags and remove details from listImageStores response and UI
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
Added a confirmation dialog box to check whether to make force update or not.
The dialog appears only for the Admin. Only admin can make force update.
The new dialog appears after the first CIDR unchanged confirmation dialog.
The "Add Load Balancer" functionality is redundant.
The above is used to create LB rule without any public IP.
This commit removes the tab from network page.
Updated StrongSwan VPN ImplementationThis PR is a merge of @jayapalu changes in #872 and the changes I had to make to get the functionality working.
I have done pretty extensive testing of this code so far and we are looking to be in pretty good shape. One thing to note is that a `Diffie-Hellman` group **is required** in order for this feature to work correctly. It is not highlighted in the tests below, but I have shown that the `PFS` is not required for this feature to work. In #872 I have shown a more exhaustive set of tests of this code, but I have limited this set of tests to a recommended `IKE` and `ESP` configuration in order to reduce the noise and test the other areas of functionality.
**Test Results**
I am testing this functionality by creating two VPCs with VMs in each and creating a S2S VPN connection between the two VPCs. Then I SSH into a VM in one VPC and I ping the private IP of a VM in the other VPC. Then I tear it down and try a different configuration.
_Setup_
```
VPC 1 VPC 2
===== =====
VPN Gateway VPN Gateway
VPN Customer Gateway VPN Customer Gateway
VPN Connection <---> VPN Connection
- Passive = True - Passive = False
```
_Legend_
`SKIP` => At least one of the VPN Connections did not come up, so no test was run.
`OK` => The ping test was successful over the S2S VPN connection.
`FAIL` => The ping test failed over the S2S VPN connection.
`Passive` => Specifies if either the `<vpc_1> : <vpc_2>` sides of the VPN Connection is set to passive.
`Conn State` => Specifies the connection status of the `<vpc_1> : <vpc_2>` VPN Connection in the UI.
`Requires Reset` => If the ping test does not result in an `OK`, then a VPN Connection Reset is performed on either `<vpc_1> : <vpc_2>` sides of the VPN Connection based on which side is not showing `Connected`. The results in the `Status` column is the final result after the reset is performed.
_Results_
```
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| Status | IKE & ESP | DPD | Encap | IKE Life | ESP Life | Passive | Conn State | Requires Reset |
+========+======================+=======+=======+==========+==========+===============+=============================+================+
| OK | aes128-sha1;modp1536 | True | False | 86400 | 3600 | True : False | Disconnected : Connected | False : False |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK | aes128-sha1;modp1536 | True | True | 86400 | 3600 | True : False | Disconnected : Connected | False : False |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK | aes128-sha1;modp1536 | True | False | | 3600 | True : False | Disconnected : Connected | False : False |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK | aes128-sha1;modp1536 | True | False | 86400 | | True : False | Disconnected : Connected | False : False |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK | aes128-sha1;modp1536 | True | False | | | True : False | Disconnected : Connected | False : False |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK | aes128-sha1;modp1536 | True | False | 86400 | 3600 | False : False | Connected : Connected | False : False |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK | aes128-sha1;modp1536 | True | False | 86400 | 3600 | True : True | Disconnected : Disconnected | False : False |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK | aes128-sha1;modp1536 | True | False | 86400 | 3600 | False : True | Connected : Disconnected | False : False |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK | aes128-sha1;modp1536 | False | False | 86400 | 3600 | False : False | Connected : Connected | False : False |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK | aes128-sha1;modp1536 | False | False | 86400 | 3600 | True : False | Disconnected : Connected | False : False |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK | aes128-sha1;modp1536 | False | False | 86400 | 3600 | True : True | Disconnected : Disconnected | False : False |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| OK | aes128-sha1;modp1536 | False | False | 86400 | 3600 | False : True | Connected : Disconnected | False : False |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| SKIP | aes128-sha1 | True | False | 86400 | 3600 | True : False | Disconnected : Error | True : False |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| SKIP | aes128-sha1 | False | False | 86400 | 3600 | True : False | Disconnected : Error | True : False |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| FAIL | aes128-sha1 | True | False | 86400 | 3600 | True : True | Disconnected : Disconnected | True : True |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
| SKIP | aes128-sha1 | True | False | 86400 | 3600 | False : False | Connected : Error | False : False |
+--------+----------------------+-------+-------+----------+----------+---------------+-----------------------------+----------------+
```
* pr/1741:
complete implementation of the StrongSwan VPN feature
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
deleted by deleting 1 template on any zone
added extra warning message if it's a cross-zone template ("This is a
cross zone template and will be deleted from all the zones. Are you sure
you want to proceed?").
[4.10] CLOUDSTACK-7985: assignVM in Advanced zone with Security GroupsThis commit contains the following changes:
(1) implementation of assignVM in Advanced zone with Security Groups
(2) keep the default nic on shared network when assignVM
(3) allow migrate vm from/to project;
(4) UI change for selecting account/project/network
* pr/844:
CLOUDSTACK-7985: assignVM in Advanced zone with Security Groups
CLOUDSTACK-7985: keep the default nic on shared network when assignVM
CLOUDSTACK-7985: (1) allow migrate vm from/to project; (2) UI change for selecting account/project/network
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
[4.10] CLOUDSTACK-8746: VM Snapshotting implementation for KVM
* pr/977:
Fixes for testing VM Snapshots on KVM. Related to PR 977
CLOUDSTACK-8746: vm snapshot implementation for KVM
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
This commit implements Ingress and Egress filtering for IPv6 in
Basic Networking.
It allows for opening and closing ports just as can be done with IPv4.
Rules have to be specified twice, once for IPv4 and once for IPv6, for
example:
- 22 until 22: 0.0.0.0/0
- 22 until 22: ::/0
Egress filtering works the same as with IPv4. When no rule is applied all
traffic is allowed. Otherwise only the specified traffic (with DNS being
the exception) is allowed.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
(1) add support to create/delete/revert vm snapshots on running vms with QCOW2 format
(2) add new API to create volume snapshot from vm snapshot
(3) delete metadata of vm snapshots before stopping/migrating and recover vm snapshots after starting/migrating
(4) enable deleting of VM snapshot on stopped vm or vm snapshot is not listed in qcow2 image.
(5) enable smoke tests for vmsnaphsots on KVM
CLOUDSTACK-9534: Allow VR to be destroy in UI when in running stateThis allows us to destroy a VR that is in running state from the UI. This action is possible via cloudmonkey/APIs, and would be useful for anyone who wants to simply destroy a VR from the UI without having to stop it first.
As this is a pure UI fix, manual testing would be needed but no regression integration testing is necessary.
/cc @jburwell @karuturi
@blueorangutan package
Screenshot:
* pr/1701:
CLOUDSTACK-9534: Allow VR to be destroy in UI when in running state
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9438: Fix for CLOUDSTACK-9252 - Make NFS version changeable in UIJIRA TICKET: https://issues.apache.org/jira/browse/CLOUDSTACK-9438
### Introduction
From #1361 it was possible to configure NFS version for secondary storage mount.
However, changing NFS version requires inserting an new detail on `image_store_details` table, with `name = 'nfs.version'` and `value = X` where X is desired NFS version, and then restarting management server for changes to take effect.
Our improvement aims to make NFS version changeable from UI, instead of previously described workflow.
### Proposed solution
Basically, NFS version is defined as an image store ConfigKey, this implied:
* Adding a new Config scope: **ImageStore**
* Make `ImageStoreDetailsDao` class to extend `ResourceDetailsDaoBase` and `ImageStoreDetailVO` implement `ResourceDetail`
* Insert `'display'` column on `image_store_details` table
* Extending `ListCfgsCmd` and `UpdateCfgCmd` to support **ImageStore** scope, which implied:
** Injecting `ImageStoreDetailsDao` and `ImageStoreDao` on `ConfigurationManagerImpl` class, on `cloud-server` module.
### Important
It is important to mention that `ImageStoreDaoImpl` and `ImageStoreDetailsDaoImpl` classes were moved from `cloud-engine-storage` to `cloud-engine-schema` module in order to Spring find those beans to inject on `ConfigurationManagerImpl` in `cloud-server` module.
We had this maven dependencies between modules:
* `cloud-server --> cloud-engine-schema`
* `cloud-engine-storage --> cloud-secondary-storage --> cloud-server`
As `ImageStoreDaoImpl` and `ImageStoreDetailsDao` were defined in `cloud-engine-storage`, and they needed in `cloud-server` module, to be injected on `ConfigurationManagerImpl`, if we added dependency from `cloud-server` to `cloud-engine-storage` we would introduce a dependency cycle. To avoid this cycle, we moved those classes to `cloud-engine-schema` module
* pr/1615:
CLOUDSTACK-9438: Fix for CLOUDSTACK-9252 - Make NFS version changeable in UI
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
Use available context to filter a metrics view based on zone, cluster, host
in the context object. This fixes metrics view filtering when metrics view is
viewed via Zone->Compute and Storage-> for a resource.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This allows admins/users to destroy a VR that is in running state from the UI.
This action is possible via cloudmonkey/APIs.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Add projectid to project details pageDisplay the project ID on the details view of a project.
* pr/1630:
Add to project detail page: cpu,memory,template,storage and VMs count
add projectid to project details page
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Simplifies change password transactional logic without using pessmistic locks
- Adds a re-enter password field in the UI to valid ipmi/oobm password
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The resize volume is support on all major hypervisors (Xen, VMware, KVM).
The hypervisor key is returned by the list volumes response only for admins
but not for users or domain admin users. This removes the check, as the operation
is supported on all major hypervisors that CloudStack supports.
With this changes all users would see resize volume button in the UI.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Taking fast and efficient volume snapshots with XenServer (and your storage provider)A XenServer storage repository (SR) and virtual disk image (VDI) each have UUIDs that are immutable.
This poses a problem for SAN snapshots, if you intend on mounting the underlying snapshot SR alongside the source SR (duplicate UUIDs).
VMware has a solution for this called re-signaturing (so, in other words, the snapshot UUIDs can be changed).
This PR only deals with the CloudStack side of things, but it works in concert with a new XenServer storage manager created by CloudOps (this storage manager enables re-signaturing of XenServer SR and VDI UUIDs).
I have written Marvin integration tests to go along with this, but cannot yet check those into the CloudStack repo as they rely on SolidFire hardware.
If anyone would like to see these integration tests, please let me know.
JIRA ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-9281
Here's a video I made that shows this feature in action:
https://www.youtube.com/watch?v=YQ3pBeL-WaA&list=PLqOXKM0Bt13DFnQnwUx8ZtJzoyDV0Uuye&index=13
* pr/1403:
Faster logic to see if a cluster supports resigning
Support for backend snapshots with XenServer
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Notify listeners when a host has been added to a cluster, is about to be removed from a cluster, or has been removed from a cluster
This PR addresses the following JIRA ticket:
https://issues.apache.org/jira/browse/CLOUDSTACK-8813
The problem is that there needs to be notifications sent when a host is added to, about to be removed from, and removed from a cluster.
Such notifications can be used for many purposes. For example, it can allow storage plug-ins to update ACLs on their storage systems. Also, it can allow us to clean up IQNs from ESXi hosts that are no longer needed.
* pr/816:
CLOUDSTACK-8813: Notify listeners when a host has been added to a cluster, is about to be removed from a cluster, or has been removed from a cluster
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Support access to a host’s out-of-band management interface (e.g. IPMI, iLO,
DRAC, etc.) to manage host power operations (on/off etc.) and querying current
power state in CloudStack.
Given the wide range of out-of-band management interfaces such as iLO and iDRA,
the service implementation allows for development of separate drivers as plugins.
This feature comes with a ipmitool based driver that uses the
ipmitool (http://linux.die.net/man/1/ipmitool) to communicate with any
out-of-band management interface that support IPMI 2.0.
This feature allows following common use-cases:
- Restarting stalled/failed hosts
- Powering off under-utilised hosts
- Powering on hosts for provisioning or to increase capacity
- Allowing system administrators to see the current power state of the host
For testing this feature `ipmisim` can be used:
https://pypi.python.org/pypi/ipmisim
FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Out-of-band+Management+for+CloudStack
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-8562: Dynamic Role-Based API Checker for CloudStack### CLOUDSTACK-8562: DB-Backed Dynamic Role Based API Access Checker
This feature allows root administrators to define new roles and associate API
permissions to them.
A limited form of role-based access control for the CloudStack management server
API is provided through a properties file, commands.properties, embedded in the
WAR distribution. Therefore, customizing API permissions requires unpacking the
distribution and modifying this file consistently on all servers. The old system
also does not permit the specification of additional roles.
FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+Role+Based+API+Access+Checker+for+CloudStack
DB-Backed Dynamic Role Based API Access Checker for CloudStack brings following
changes, features and use-cases:
- Moves the API access definitions from commands.properties to the mgmt server DB
- Allows defining custom roles (such as a read-only ROOT admin) beyond the
current set of four (4) roles
- All roles will resolve to one of the four known roles types (Admin, Resource
Admin, Domain Admin and User) which maintains this association by requiring
all new defined roles to specify a role type.
- Allows changes to roles and API permissions per role at runtime including additions or
removal of roles and/or modifications of permissions, without the need
of restarting management server(s)
Upgrade/installation notes:
- The feature will be enabled by default for new installations, existing
deployments will continue to use the older static role based api access checker
with an option to enable this feature
- During fresh installation or upgrade, the upgrade paths will add four default
roles based on the four default role types
- For ease of migration, at the time of upgrade commands.properties will be used
to add existing set of permissions to the default roles. cloud.account
will have a new role_id column which will be populated based on default roles
as well
Dynamic-roles migration tool: scripts/util/migrate-dynamicroles.py
- Allows admins to migrate to the dynamic role based checker at a future date
- Performs a harder one-way migrate and update
- Migrates rules from existing commands.properties file into db and deprecates it
- Enables an internal hidden switch to enable dynamic role based checker feature
* pr/1489:
maven: Fix jstl version usage
CLOUDSTACK-8562: Deprecate commands.properties
CLOUDSTACK-8562: DB-Backed Dynamic Role Based API Access Checker
CLOUDSTACK-9361: Centrally handle API validations
Signed-off-by: Will Stevens <williamstevens@gmail.com>
This feature allows root administrators to define new roles and associate API
permissions to them.
A limited form of role-based access control for the CloudStack management server
API is provided through a properties file, commands.properties, embedded in the
WAR distribution. Therefore, customizing API permissions requires unpacking the
distribution and modifying this file consistently on all servers. The old system
also does not permit the specification of additional roles.
FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+Role+Based+API+Access+Checker+for+CloudStack
DB-Backed Dynamic Role Based API Access Checker for CloudStack brings following
changes, features and use-cases:
- Moves the API access definitions from commands.properties to the mgmt server DB
- Allows defining custom roles (such as a read-only ROOT admin) beyond the
current set of four (4) roles
- All roles will resolve to one of the four known roles types (Admin, Resource
Admin, Domain Admin and User) which maintains this association by requiring
all new defined roles to specify a role type.
- Allows changes to roles and API permissions per role at runtime including additions or
removal of roles and/or modifications of permissions, without the need
of restarting management server(s)
Upgrade/installation notes:
- The feature will be enabled by default for new installations, existing
deployments will continue to use the older static role based api access checker
with an option to enable this feature
- During fresh installation or upgrade, the upgrade paths will add four default
roles based on the four default role types
- For ease of migration, at the time of upgrade commands.properties will be used
to add existing set of permissions to the default roles. cloud.account
will have a new role_id column which will be populated based on default roles
as well
Dynamic-roles migration tool: scripts/util/migrate-dynamicroles.py
- Allows admins to migrate to the dynamic role based checker at a future date
- Performs a harder one-way migrate and update
- Migrates rules from existing commands.properties file into db and deprecates it
- Enables an internal hidden switch to enable dynamic role based checker feature
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9194: Making the console popup window resizable in IE to make sure the focus is not losthttps://issues.apache.org/jira/browse/CLOUDSTACK-9194
To test:
Open any VM console in IE, and try resizing the browser window of console
It should be resizable.
* pr/1270:
CLOUDSTACK-9194: Making the console popup window resizable in IE to make sure the focus is not lost.
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CLOUDSTACK-9322: Support for Internal LB fuctionality with Nuage VSP SDN Plugin including Marvin test coverageTask: https://issues.apache.org/jira/browse/CLOUDSTACK-9322
PR contents:
1) UI changes to support LB provider InternalLbVm during VPC offering creation.
2) Bug fix for CLOUDSTACK-9320.
3) Nuage VSP SDN Plugin related enhancements for VPC network functionality.
4) Marvin test coverage for Internal LB support on master with Nuage VSP SDN Plugin.
5) Enhancements on our exiting Marvin test code (nuagevsp plugins directory).
6) PEP8 & PyFlakes compliance with our Marvin test code.
Test run:
CloudStack$ nosetests --with-marvin --marvin-config=nuage_ant.cfg test/integration/plugins/nuagevsp/ -a tags=nuagevsp
Test results:
Test user data and password reset functionality with Nuage VSP SDN plugin ... === TestName: test_nuage_UserDataPasswordReset | Status : SUCCESS ===
ok
Test Nuage VSP VPC Offering with different combinations of LB service providers ... === TestName: test_01_nuage_internallb_vpc_Offering | Status : SUCCESS ===
ok
Test Nuage VSP VPC Network Offering with and without Internal LB service ... === TestName: test_02_nuage_internallb_vpc_network_offering | Status : SUCCESS ===
ok
Test Nuage VSP VPC Networks with and without Internal LB service ... === TestName: test_03_nuage_internallb_vpc_networks | Status : SUCCESS ===
ok
Test Nuage VSP VPC Internal LB functionality with different combinations of Internal LB rules ... === TestName: test_04_nuage_internallb_rules | Status : SUCCESS ===
ok
Test Nuage VSP VPC Internal LB functionality by performing (wget) traffic tests within a VPC ... === TestName: test_05_nuage_internallb_traffic | Status : SUCCESS ===
ok
Test Nuage VSP VPC Internal LB functionality with different LB algorithms by performing (wget) traffic tests ... === TestName: test_06_nuage_internallb_algorithms_traffic | Status : SUCCESS ===
ok
Test Nuage VSP VPC Internal LB functionality with restarts of VPC network components by performing (wget) ... === TestName: test_07_nuage_internallb_vpc_network_restarts_traffic | Status : SUCCESS ===
ok
Test Nuage VSP VPC Internal LB functionality with InternalLbVm appliance operations by performing (wget) ... === TestName: test_08_nuage_internallb_appliance_operations_traffic | Status : SUCCESS ===
ok
Test Basic VPC Network Functionality with Nuage VSP SDN plugin ... === TestName: test_nuage_vpc_network | Status : SUCCESS ===
ok
Test Nuage VSP SDN plugin with basic Isolated Network functionality ... === TestName: test_nuage_vsp | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 11 tests in 12094.705s
OK
Test run logs:
[results.txt](https://github.com/apache/cloudstack/files/187587/results.txt)
[runinfo.txt](https://github.com/apache/cloudstack/files/187588/runinfo.txt)
Test config file:
[nuage_ant.txt](https://github.com/apache/cloudstack/files/222711/nuage_ant.txt)
Note: Attached the Marvin config file as .txt instead of .cfg.
PEP8 & PyFlakes Compliance:
CloudStack$ pep8 --max-line-length=150 test/integration/plugins/nuagevsp/*.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/nuageTestCase.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/test_nuage_password_reset.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/test_nuage_vpc_internal_lb.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/test_nuage_vpc_network.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/test_nuage_vsp.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/*.py
#CLOUDSTACK-9322
* pr/1452:
CLOUDSTACK-9322 : Marvin tests for Internal Lb with Nuage VSP
CLOUDSTACK-9320 : InternalLBVM is not getting destroyed when the last Internal Load Balancer rule is removed for the corresponding source IP address
CLOUDSTACK-9322 : Changes to support InternalLbVm with Nuage VSP plugin
Signed-off-by: Will Stevens <williamstevens@gmail.com>
* 4.8:
CLOUDSTACK-9172 Added cross zones check to delete template and iso
Check the existence of 'forceencap' parameter before use
systemvm: set default umask 022 in injectkeys.sh
* 4.7:
CLOUDSTACK-9172 Added cross zones check to delete template and iso
Check the existence of 'forceencap' parameter before use
systemvm: set default umask 022 in injectkeys.sh
CLOUDSTACK-9172 Added cross zones check to delete template and isoAdded a check to ignore the zoneid, in the delete template UI, if the template is cross zones.
reference : CLOUDSTACK-9172
* pr/1505:
CLOUDSTACK-9172 Added cross zones check to delete template and iso
Signed-off-by: Will Stevens <williamstevens@gmail.com>
* 4.8:
CLOUDSTACK-9272: No option in UI to add GSLB with service type "HTTP"
CLOUDSTACK-9270: UI alignment gone bad in multiple places - VM Instance, Network, Egress rules
* 4.7:
CLOUDSTACK-9272: No option in UI to add GSLB with service type "HTTP"
CLOUDSTACK-9270: UI alignment gone bad in multiple places - VM Instance, Network, Egress rules
CLOUDSTACK-9272: No option in UI to add GSLB with service type "HTTP"Steps to Repro:
============
Go to Regions -> Local -> View GSLB -> Add GSLB
Click on the service type dropdown
Observe http is missing. Please see the attached snapshot.
Expected Behaviour:
================
As it supports http also, So http should be in the list.
Actual Behaviour:
==============
http is missing from the list.
Fix:
===
It supports http also. Added http to the list.
Snapshot:
========
<img width="531" alt="gslb-http-missing-nitin" src="https://cloud.githubusercontent.com/assets/12583725/12772818/21513dc0-ca5b-11e5-822e-e2dd2426da65.png">
* pr/1399:
CLOUDSTACK-9272: No option in UI to add GSLB with service type "HTTP"
Signed-off-by: Koushik Das <koushik@apache.org>
CLOUDSTACK-9268: Display VM in Load balancing rule in UISteps of Repro:
=============
1:Create VMs
2:Make LoadBalancing rule in GUI
Name:WWW
PrivatePort:80
PublicPort:80
Add VMs:some VMs
Expected Result:
==============
The VMs which has been already assigned is should not be listed when you add the VM to an existing rule.
Actual Result:
===========
The VMs which has been already assigned is still being listed when you add the VM to an existing rule.
Fix:
===
Added jsonObj to newly created row in multiedit.js to stop listing the same VM again.
* pr/1394:
CLOUDSTACK-9268: Display VM in Load balancing rule in UI
Signed-off-by: Koushik Das <koushik@apache.org>
This fixes a typographical error in UI that did not previously send fetchLatest
flag in the listCapacity API request.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The field we use most are now on the top:
- name
- state
- hypervisor
- link local ip
- redundant state
The other fields are nice, but not needed most of the time.
* 4.8:
Display hostname the VPC router runs on
CLOUDSTACK-9266: Make deleting static routes in private gw work
CLOUDSTACK-9264: Make /32 static routes for private gw work
* 4.7:
Display hostname the VPC router runs on
CLOUDSTACK-9266: Make deleting static routes in private gw work
CLOUDSTACK-9264: Make /32 static routes for private gw work
This is displayed on Infra tab and it's annoying that on the VPC
page only the link local is mentioned and not the hostname. That is
now corrected, so you have the correct details to login to the
router straight away.
This commit includes three changes:
(1) Revert commit 13bf1ec5da to show the 'root disk size' field
(2) Set the default value from '1' to empty. This also changes the default value of customized service offerings from '1' to empty.
(3) show 'root disk size' field only when users choose a KVM template, as cloudstack supports deployvm with rootdisksize on KVM only.
trailing commas in javascripts removedAccording to SonarQube this is a bug on internet explorer. It is the only 'blocker' level issue in cloudstack. @abhinandanprateek (@agneya2001) @bhaisaab @miguelaferreira should we enforce this or, as alternative, have it disabled in SonarQube?
* pr/1288:
trailing commas in javascripts removed
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* 4.7:
CLOUDSTACK-9254: Make longer names display pretty
CLOUDSTACK-9245 - Deletes ACL items when destroying the VPC or deleting the ACL itself
CLOUDSTACK-9245 - Formatting NetworkACLServiceImpl class
CLOUDSTACK-9245 - Formatting VpcManagerImpl class
CLOUDSTACK-9245 - Formatting NetworkACLManagerImpl class
More VR performance!
* 4.7:
CLOUDSTACK-9237: Create LB Healthcheck issues - button alignment and error message goes outside the window
CLOUDSTACK-9235: Autoscale button is missing in VPC
CLOUDSTACK-9229: Autoscale policy creation failing in VPC due to zoneid missing in createAutoScaleVmProfile
CLOUDSTACK-9229: Autoscale policy creation failing in VPC due to zone id missing in createAutoScaleVmProfileAutoscale policy creation is failing on VPC while creating autoscalevmprofile since zoneid parameter is missing.
Steps to Reproduce:
================
1. Create VPC Network offering with NS as service provider for LB
2. Create VPC using offering which has NetScaler as LB provider
3. Add a tier using the network offering created in Step1
4. Launch a VM
5. Acquire a public IP
6. Create an auto scale policy with all required parameters on an LB rule
Result:
======
Fails during autoscalevmprofile creation with the following error:
GET command=createAutoScaleVmProfile&response=json&serviceofferingid=05ac866f-604c-43e2-a48b-47e83ef7c4f1&templateid=3cdd83c4-ad27-11e5-9eaf-42407779c24b&destroyvmgraceperiod=30&counterparam%5B0%5D.name=snmpcommunity&counterparam%5B0%5D.value=public&counterparam%5B1%5D.name=snmpport&counterparam%5B1%5D.value=161&_=1451458073663
DEBUG [o.a.c.a.BaseCmd] (catalina-exec-10:ctx-58bcf6cf ctx-f5123e30) (logid:ae6742c8) Ignoring paremeter fordisplay as the caller is not authorized to pass it in
INFO [c.c.a.ApiServer] (catalina-exec-10:ctx-58bcf6cf ctx-f5123e30) (logid:ae6742c8) Unable to execute API command autoscalevmprofile due to missing parameter zoneid
Expected Result:
=============
Autoscale policy creation should be successful and the "min" number of VMs specified in the policy should be deployed.
Following should be the params for autoscalevmprofile creation:
command=createAutoScaleVmProfile&response=json&zoneid=0b43fff3-c069-417b-ac0c-e4dc46b407ea&serviceofferingid=0
5ac866f-604c-43e2-a48b-47e83ef7c4f1&templateid=3cdd83c4-ad27-11e5-9eaf-42407779c24b&destroyvmgraceperiod=30&counterparam%5B0%5D
.name=snmpcommunity&counterparam%5B0%5D.value=public&counterparam%5B1%5D.name=snmpport&counterparam%5B1%5D.value=161
Fix:
===
Added a conditon to check whether the zoneid in networks is undefined,
If that is undefined, it gets the zoneid from ipaddresses argument.
Fixed a localization issue.
* pr/1334:
CLOUDSTACK-9229: Autoscale policy creation failing in VPC due to zoneid missing in createAutoScaleVmProfile
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9235: Autoscale button is missing in VPCAutoscale button is missing. This should not be the case since we are able to add NS as the external LB provider in VPC.
Steps:
=====
1. Create a VPC offering with NS as the external LB provider
2. Create a VPC and configure the public tier with the above offering
3. Acquire an IP address and try to configure Load Balancing rule
Result:
======
Autoscale option is not visible at all for the LB.
Expected Result:
=============
Autoscale option should be available and should work exactly like the way it works for a normal isolated network with NS. If we choose NS as service provider for LB, autoscale should also be visible. If VR is chosen, only then we should not display autoscale.
Fix:
===
In case of VPC, it checks the services available.
If LB is there, It checks the provider is Netscaler then it shows the button or hides it.
* pr/1337:
CLOUDSTACK-9235: Autoscale button is missing in VPC
Signed-off-by: Remi Bergsma <github@remi.nl>
Rohit Yadav
vedulasantosh
Rohit Yadav
Gabriel Beims Bräscher
Nicolas Vazquez
Wido den Hollander
Rashmi D
Nitin Kumar Maharana
Sanket Thite
Suresh Kumar Anaparti
Nitesh Sarda
Wei Zhou
Wei Zhou
Nitin Kumar Maharana
Rajani Karuturi
Jayapal
subhash yedugundla
Mowgli
Rajani Karuturi
Daan Hoogland
Nathan Johnson
Anshul And Priyank
Nitin Kumar Maharana
nvazquez
Will Stevens
Jayant Patil
Syed
pdumbre
Frank Maximus
Milamber
Nick Livens
Mike Tutkowski
Pierre-Luc Dion
Koushik Das
Patrick Dube
Remi Bergsma
Rafael Weingärtner
