- Fixed a bug, where after sorting a column of the instance list view the multi select action buttons wouldn't show up after selecting one ore more entries of the table.
- Added a behavior, so that an already starting/running/stopped/stopping instance of the instance view will (when multi selected alone or together with other instances) not create an API request if it already has the desired state.
Problem: Users can register ISOs from URL but cannot upload local ISOs.
Root cause: CloudStack provides browser-based upload support for volumes and templates, but ISOs are not supported.
Solution:
The existing browser-based upload from local functionality for templates and volumes (https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=39620237) is extended to support uploading local ISOs.
Extend the UI: A new button is created under the ISOs view: 'Upload from Local'. A new dialog form is displayed in which the user must select the ISO to upload from its local file system.
Extend the API: New 'GetUploadParamsForIso' API command is created to handle the ISO upload.
Upload template form was missing some hypervisor specific options which can be seen on Register template form.
For XenServer, XenServer tool version added
For VMware, Root disk controller, NIC adapter type, Keyboard type added and HVM is hidden
For KVM, Root disk controller added
Fixes#3363
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
We want to support hiding table columns, specifically in metrics table, through config file so that users can make the relevant bits hidden as per their organization. Current work will support the metrics table but can be extended to any table with minimal work in future.
Config file will take the key of the metrics column from metrics.js file for the sake of minimal changes and simplicity of development.
Problem: The keyboard list in the UI is not consistent across views such as in the instance wizard and in the register template form. There is also no way to custom about url/text and doc title and help URL in the UI.
Root Cause: The list is hardcoded in the UI allowing no centralised configuration.
Solution: Introduce a new config.js file installed at the /usr/share/cloudstackmanagement/webapp/config.js location. The config.js allows configurable keyboard list, about url/text, doc title, and help URL.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
When enable static nat in a vpc on UI, it only lists the primary and secondary ips of first nic of a vm, no matter which vpc tier is selected. The same issue happens when add a vm to load balancer.
Fixes#3334
This adds memory used column in the instance metrics view. Also fixes
a bug for VMware, due to which incorrect memory usage was returned.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This fixes the sorting of ip addresses and ip cidrs.
Currently they get evaluated as a numeric value via parseFloat and sorted as numeric values.
For example "10.32.123.123" got parsed to 10.32.
This change allows instance Settings tab to be visible but inaccessible when instance is running. A warning is shown when user tries to access Settings for a running instance and tab content is greyed out.
It also allows some admin defined instance settings/details to be made static for user. User will be able to see them in instance settings tab but cannot change their values as action buttons are disabled and greyed out. This can be achieved by providing a comma-separated list details for global settings key 'user.vm.readonly.ui.details'. A new value 'readonlyuidetails' has been added in UserVMResponse for UI manipulate editing functionality of settings/details.
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Added changes for creating service offerings for specified domain(s) and zone(s).
Fixed checkAccess for disk offerings.
Fixed list APIs for disk and service offerings.
UI changes for creating disk, service offerings for specified domain(s) and zone(s).
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
Allows creating storage offerings associated with particular domain(s) and zone(s). In create disk/storage offfering form UI, a mult-select control has been addded to select desired zone(s) and domain select element has been made multi-select.
createDiskOffering API has been modified to allow passing list of domain and zone IDs with keys domainids and zoneids respectively. These lists are stored in DB in cloud.disk_offering_details table with 'domainids' and 'zoneids' key as string of comma separated list of IDs. Response for create, update and list disk offering APIs will return domainids, domainnames, zoneids and zonenames in details object of offering.
listDiskOfferings API has been modified to allow passing zoneid to return only offerings which are associated with the zone.
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Problem: Custom compute offering does not allow setting min and max values for CPU and VRAM for custom VMs.
Root Cause: Custom compute offerings cannot be created with a given range of CPU number and memory instead it allows only fixed values.
Solution: createServiceOffering API has been modified to allow setting a defined range for CPU number and memory. Also, UI form for compute offering creation is provided with a new field named 'compute offering type’ with values - Fixed, Custom Constrained, Custom Constrained. It will allow the creation of compute offerings either with a fixed CPU speed and memory for fixed compute offering, or with a range of CPU number and memory for custom constrained compute offering or without predefined CPU number, CPU speed and memory for custom unconstrained compute offering.
To allow the user to set CPU number, CPU speed and memory during VM deployment, UI form for VM deployment has been modified to provide controls to change these values. These controls are depicted in screenshots below for custom constrained and custom unconstrained compute offering types.
Sample API calls using cmk to create a constrained service offering and deploying a VM using it,
create serviceoffering name=Constrained displaytext=Constrained customized=true mincpunumber=2 maxcpunumber=4 cpuspeed=400 minmemory=256 maxmemory=1024
deploy virtualmachine displayname=ConstrainedVM serviceofferingid=60f3e500-6559-40b2-9a61-2192891c2bd6 templateid=8e0f4a3e-601b-11e9-9df4-a0afbd4a2d60 zoneid=9612a0c6-ed28-4fae-9a48-6eb207af29e3 details[0].cpuNumber=3 details[0].memory=800
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
In the cloudstack UI fields get '<', '>' and '&' replaced by xml-entities
these are generic for all fields and hurt us in the case of userdata
this fix calls the existing method to reverse character replacements.
it also removes the ccs class that pretends to prevent special chars
Fixes#3202
* Add jQuery ui stylings to the project
* Adapt stylings to jquery ui stylings
- remove unnecessary (empty) style definitions
* Check if jquery autocomplete is initialized before destroying it
* Use dialog modal option instead of setting z-index styling
- set minimum heigth of dialog
* Swap removeAttr() for prop() since it's not longer recommended as of jQuery 3.0
* Fix reccuring snapshots modal view
* Further style improvements
* More improvements on the instance-wizard (and multi-wizard in general)
* Raise input hints over modal z-index
* Adapt/fix stylings
- center buttons of create form modals
- fix width of select menu in service offerings
- fix width of modal in Network -> VPC -> configure -> static NAT -> aquire new Ip -> enable static nat
* Adapt width of dynamic-input container
This allows showing complete domain, ie, domain path for accounts list view and account detail.
Added a new key, domainpath, in AccountResponse.
Fixes#2994
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Problem: When uploading template or iso in a project, the progress is not seen
Root Cause: When users upload a template/iso in project view, the template/iso is not visible in the all filter. This creates confusion that template/iso has not been uploaded.
Solution: Since the api listtemplates with a projectid acts is a superset of list templates without a project id, we should not ignore the project id when listing templates in all filter.
Using db column instead of VO variable name was causing issue with SQL select statement.
This change fixes the problem by using VO variable for adding conditional.
Fixes#3208
Using db column instead of VO variable name was causing issue with SQL select statement.
This PR fixes the problem by using VO variable for adding conditional.
Additionally in UI listAll parameter was being sent twice in the listVMSnapshot API call. It is fixed with this PR.
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Problem: When reinstalling a VM, the UI only shows featured templates and not all possible list of allowed templates.
Root Cause: The list of templates for reinstall used featured as filter in ‘listTemplates’ API which did not include all available options.
Solution: The issue is fixed by using the executable template filter of the ‘listTemplates’ API to list all
possible templates that a user is allowed to use to deploy a VM.
Hide multiselectaction buttons when selection is resetted after a action is performed
Use checked prop instead of attr to show/hide triggered selection when box was clicked by a user before.
How to reproduce:
Select one entry in a multiSelect listView (e.g. instance list) and then deselect it again. Now click on the "select all" checkbox. The entries which were clicked previously are not displayed as selected.
Select at least one entry. Use the multiSelectAction buttons to trigger an action. Wait for completion. The list refreshes and no entry is selected but the multiSelectAction buttons are displayed.
Fix:
Use "checked" property instead of the attribute to select all entries. This is necessary because when you checked an entry by clicking on it the property gets set and the browser is preferably using this. So setting the attribute by clicking on the "select all" checkbox had no impact anymore on the checkBox of the previously clicked entries.
Hide the multiSelectAction buttons when the list is refreshed after an action is performed and no element is selected anymore.
This removes CA certificate button from UI as this is not used and
confuses both admin and users how to use it. This may be re-introduced
in future when a use-case can be justified.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* Fix XenServer Security Groups 'vmops' script
- fix tokens = line.split(':') to tokens = line.split(';')
- fix expected tokens size from 5 to 4
- enhance logs
- remove unused vmops script. The XCP patch points to the vmops script
on the parent folder [1]. Thus, all XenServer versions are considering
the vmops script located at [2].
- fix UI ipv4/ipv6 cidr validator to allow a list of cidirs.
Fixing issue: #3192 Security Group rules not applied at all for
XenServer 6.5 / Advanced Zone
https://github.com/apache/cloudstack/issues/3192
* Update security group rules after VM migration
Add security group rules on target host
Cause: vmops script expected secondary IPs as "0;" but received "0:"
Remove security group network rules on source host.
Cause: destroy_network_rules_for_vm function on vmops script was not
called when migrating VM
* Add unit tests and address reviewers
* api: add command to list management servers
* api: add number of mangement servers in listInfrastructure command
* ui: add block for mangement servers on infra page
* api name resolution method cleanup
* Update jquery, jquery ui, jquery validate and flot to current versions
update jquery to 3.3.1
update jqueryui to 1.12.1
update jquery validate to 1.17.0
update jquery flot to 0.8.3
* Replace deprecated removed jquery functions
* Fix initial tab content loading in detailView
* Fix logout for new jquery version
* Fix tooltip detail displaying for new JQuery version
* Fix view all trigger in detailView for new JQuery version
* Fix breadcrumb click event handler for JQuery update
* Fix displaying of preselected zone in instanceWizard for new jQuery verion
* Remove redundant checkbox that controls the visibility of volumes to delete when destroying VMs
* Change label from Volume IDs to Delete Volumes
* Revert back deleted Volume IDs label
This PR adds the possibility to select a checkbox for the parameter bypassvlanoverlapcheck to the ajax request createNetwork. The checkbox was added for Guest Network as well as for the L2 Guest Network. For L2 Guest Network a backend check for the existence of the flag bypassvlanoverlapcheck was added.
## Description
Adds the MAC address to the NICs detail view of an instance.
## Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
## Screenshots:
## How Has This Been Tested?
Tested locally to verify that the correct mac address gets displayed in the detail view.
This adds a new API updateVmwareDc that allows admins to update the
VMware datacenter details of a zone. It also recursively updates
the cluster_details for any username/password updates
as well as updates the url detail in cluster_details table and guid
detail in the host_details table with any newly provided vcenter
domain/ip. The update API assumes that there is only one vCenter per
zone. And, since the username/password for each VMware host could be different
than what gets configured for vcenter at zone level, it does not update the
username/password in host_details.
Previously, one has to manually update the db with any new vcenter details for the zone.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
When the “deleteDomain” command is sent via UI, the HTML is already updated assuming that the “deleteDomain” command will actually delete it. However, there are cases (domain with allocated resources) that will not allow ACS to remove the command; as a consequence, the UI will not be in sync with ACS state, which can lead to misunderstandings.
Fix UI overlay issue in snapshot view on master with new jquery and other changes. This problem was created because of the jQuery-UI upgrade.
This fixes: #2830
* 4.11:
Changed the implementation of isVolumeOnManagedStorage(VolumeInfo) to check if the data store in question is for primary storage (and added a unit test from Daan Hoogland)
vmware: reboot VR after mac updates (#2794)
This re-introduces the rebooting of VR after setup of nics/macs in
case of VMware. It also adds a minor enhancement to show the console
esp. for root admins when VRs and systemvms are in starting state.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
In the UI, when a user navigates to a screen that displays a table of data and hovers the mouse over a sortable column header, the cursor does not change to a hand pointer, indicating to the user that it can be clicked on, in order to sort the table's data rows according to that column.
When the mouse hovers over the non-sortable 'Quickview' column header, it incorrectly displays a hand pointer cursor instead of a default cursor.
When the mouse hovers over a column header, it also does not display the column header's value as a tooltip.
Expected Behaviour:
The cursor type for a sortable column header must display a hand pointer.
The cursor type for the non-sortable 'Quickview' column header must display a default pointer.
When the mouse hovers over a column header it must display the column header's value as a tooltip.
Actual Behaviour:
The cursor type for a sortable column header incorrectly displays a default pointer.
The cursor type for the non-sortable 'Quickview' column header incorrectly displays a hand pointer.
When the mouse hovers over a column header it does not display the column header's value as a tooltip.
To Reproduce:
In the UI, navigate to a screen that displays a table of data and hover the mouse over a sortable column header, the cursor does not change to a hand pointer.
Hover the mouse over the non-sortable 'Quickview' column header and notice that it incorrectly displays a hand pointer cursor instead of a default cursor.
Hover the mouse over a column header and notice that it does not display the column header's value as a tooltip.
In the index.html file, the reference to the custom.css file is made on the last line of the <head> tag.
The plugins.js file appends a list of CSS files to the <head> tag dynamically, consequently placing them after the custom.css file and overriding it.
Expected Behaviour:
The plugins.js file must insert the list of CSS files before the custom.css file so that it does not get overridden.
Actual Behaviour:
The plugins.js file appends a list of CSS files to the <head> tag dynamically, consequently placing them after the custom.css file and overriding it.
To Reproduce:
Log in to the Cloudstack UI using your favourite browser.
Use your browser's development tool to view the HTML elements of the index.html loaded in memory.
Expand the <head> element and review the list of CSS <link> elements.
Notice that the list of module and plugins CSS files is appended after the custom.css file.
On the 'NICs' tab, when a user clicks the 'Add network to VM' button to add a network to a VM, the 'Add network to VM' popup screen does not have an IP Address input field to allow a user to define a specific IP Address for a NIC. A user can specify the IP address for the first NIC when creating a VM instance, but cannot do that for subsequent NICs when adding a network to a VM.
To Reproduce:
Go to the 'Instances' screen by clicking the 'Instances' tab on the lefthand side.
On the 'Instances' screen click on a specific VM instance name.
This will open the 'Details' tab for the specific VM instance.
Click on the 'NICs' tab and then click on the 'Add network to VM' button to add a network to a VM.
The 'Add network to VM' popup screen will display.
Actual Behaviour:
The 'Add network to VM' popup screen does not have an IP Address input field to allow a user to define a specific IP Address for a NIC.
Expected behaviour:
The 'Add network to VM' popup screen must have an IP Address input field to allow a user to define a specific IP Address for a NIC.
Since the addNicToVirtualMachine API's ipaddress field is not required, the IP Address input field is also not a required field.
The IP Address input field must be validated for a valid IPv4 formatted value if the user enters anything into the field.
The valid user-specified IPv4 IP Address value must be allocated to the NIC if it is within the acceptable IP range for the chosen Network.
On the 'Register Template From URL' screen, when a user selects the VMware option from the Hypervisor dropdown:
It incorrectly displays the 'HVM' checkbox.
This checkbox must be hidden in the VMware context.
This checkbox must still be visible in any other hypervisor context.
To Reproduce:
Go to the 'Register Template From URL' screen by clicking the 'Templates' tab on the lefthand side.
On the 'Templates' screen click the 'Add' button to display the 'Register Template From URL' screen.
On the 'Register Template From URL' screen, select the VMware option from the Hypervisor dropdown:
Actual Behaviour:
It incorrectly displays the 'HVM' checkbox.
Expected behaviour:
This checkbox must be hidden in the VMware context.
This checkbox must still be visible in any other hypervisor context.
In the UI, on the Instances screen, the Quickview popup window and the Details window do not display the 'Reset SSH Key Pair' button for VMs in a running state. They only display when the VM is in a stopped state. This is inconsistent with the 'Reset Password' button behaviour, where it displays in both VM states: running and stopped. This fixes the issue so that the 'Reset SSH Key Pair' button also displays in both VM states.
Expected Behaviour:
In the UI, on the Instances screen, the Quickview popup window and the Details window must display the 'Reset SSH Key Pair' button in both VM states: running and stopped. When a user clicks on the 'Reset SSH Key Pair' button and a VM is in a running state, it will display a message "Vm xxx should be stopped to do SSH Key reset".
Actual Behaviour:
In the UI, on the Instances screen, the Quickview popup window and the Details window do not display the 'Reset SSH Key Pair' button for VMs in a running state. It only displays when the VM is in a stopped state.
In the Add Zone Wizard, when a user lands on the Physical Network page, by default the first Physical Network gets initialized with the 3 Traffic Types: Guest, Management and Public. When a user drags the Management or Public Traffic Types from the first Physical Network to a second or nth Physical Network and then clicks the next button to go to the next screen, and then decides to click the previous button to go back to the Physical Network screen, the UI initializes the physicalNetwork again and thus moves these Traffic Types back to their original position in the first Physical Network as if it is the first time the user navigated to this page.
A fix was made so that when a user clicks the previous button to go back to the Physical Network screen, it does not initialize the physicalNetwork again, therefore leaving the user-defined Traffic Type configuration as it was before the next button was clicked.
On the 'Register Template From URL' screen, when a user selects the KVM option from the Hypervisor dropdown:
1) It incorrectly displays the 'Original XS Version is 6.1' checkbox. This checkbox should be hidden in the KVM context.
2) The 'Root Disk Controller' dropdown should display the default option of 'osdefault' instead of a blank default option.
There was a concurrency problem with the “moveNetworkAclItem” API method. If two users were changing the ACL rules order at the same time, this could lead to inconsistent actions.
To solve the problem we added a “consistency check ” parameter, which is used to hold the consistency hash. This hash is created using an MD5 hash function on a String that is created with all ACL rules UUIDs concatenated in their order, which is defined via the ‘number’ field.
We also lock the editing of the ACL while executing the upgrade. This allows us to handle race conditions nicely, and present a good feedback for the user.
* [CLOUDSTACK-9261] Upgrate jQuery-UI to 1.11 (JQuery UI 1.8.4 prone to XSS)
* fix problems in the UI for lbCertificatePolicy and StaticNAT
* force jenkins build
* Fix about dialog
* Fix position of network service offering
* 4.11:
comment on unencryption
ui: fix create VPC dialog box failure when zone is SG enabled (#2704)
CLOUDSTACK-10381: Fix password reset / reset ssh key with ConfigDrive
isisnot=
extra message
debug message
imports
update without decrypt doesn't work
set unsensitive attributes as not 'Secure'
remove old config artifacts from update path
This introduces a rolling restart of VRs when networks are restarted
with cleanup option for isolated and VPC networks. A make redundant option is
shown for isolated networks now in UI.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Supporting ConfigDrive user data on L2 networks.
Add UI checkbox to create L2 network offering with config drive.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* CLOUDSTACK-10147 Disabled Xenserver Cluster can still deploy VM's. Added code to skip disabled clusters when selecting a host (#2442)
(cherry picked from commit c3488a51db)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* CLOUDSTACK-10318: Bug on sorting ACL rules list in chrome (#2478)
(cherry picked from commit 4412563f19)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* CLOUDSTACK-10284:Creating a snapshot from VM Snapshot generates error if hypervisor is not KVM.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* CLOUDSTACK-10221: Allow IPv6 when creating a Basic Network (#2397)
Since CloudStack 4.10 Basic Networking supports IPv6 and thus
should be allowed to be specified when creating a network.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
(cherry picked from commit 9733a10ecd)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* CLOUDSTACK-10214: Unable to remove local primary storage (#2390)
Allow admins to remove primary storage pool.
Cherry-picked from eba2e1d8a1
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* dateutil: constistency of tzdate input and output (#2392)
Signed-off-by: Yoan Blanc <yoan.blanc@exoscale.ch>
Signed-off-by: Daan Hoogland <daan.hoogland@shapeblue.com>
(cherry picked from commit 2ad5202823)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* CLOUDSTACK-10054:Volume download times out in 3600 seconds (#2244)
(cherry picked from commit bb607d07a9)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* When creating a new account (via domain admin) it is possible to select “root admin” as the role for the new user (#2606)
* create account with domain admin showing 'root admin' role
Domain admins should not be able to assign the role of root admin to new users. Therefore, the role ‘root admin’ (or any other of the same type) should not be visible to domain admins.
* License and formatting
* Break long sentence into multiple lines
* Fix wording of method 'getCurrentAccount'
* fix typo in variable name
* [CLOUDSTACK-10259] Missing float part of secondary storage data in listAccounts
* [CLOUDSTACK-9338] ACS not accounting resources of VMs with custom service offering
ACS is accounting the resources properly when deploying VMs with custom service offerings. However, there are other methods (such as updateResourceCount) that do not execute the resource accounting properly, and these methods update the resource count for an account in the database. Therefore, if a user deploys VMs with custom service offerings, and later this user calls the “updateResourceCount” method, it (the method) will only account for VMs with normal service offerings, and update this as the number of resources used by the account. This will result in a smaller number of resources to be accounted for the given account than the real used value. The problem becomes worse because if the user starts to delete these VMs, it is possible to reach negative values of resources allocated (breaking all of the resource limiting for accounts). This is a very serious attack vector for public cloud providers!
* [CLOUDSTACK-10230] User should not be able to use removed “Guest OS type” (#2404)
* [CLOUDSTACK-10230] User is able to change to “Guest OS type” that has been removed
Users are able to change the OS type of VMs to “Guest OS type” that has been removed. This becomes a security issue when we try to force users to use HVM VMs (Meltdown/Spectre thing). A removed “guest os type” should not be usable by any users in the cloud.
* Remove trailing lines that are breaking build due to checkstyle compliance
* Remove unused imports
* fix classes that were in the wrong folder structure
* Updates to capacity management
This adds and allows Ubuntu 18.04 to be used as KVM host. In addition,
on the UI when hypervisor version key is missing, this adds and display
the host os and version detail which is useful to show the KVM host
os and version.
When cache mode 'none' is used for empty cdrom drives, systemvms
and guest VMs fail to start on newer libvirtd such as Ubuntu bionic.
The fix is ensure that cachemode is not declared when drives are empty
upon starting of the VM. Similar issue logged at redhat here:
https://bugzilla.redhat.com/show_bug.cgi?id=1342999
The workaround is to ensure that we don't configure cachemode for
cdrom devices at all. This also fixes live VM migration issue.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
After using the feature introduced by #2486 in production, we felt the need for an improvement in the UI. It is interesting to filter the displayed disk offerings according to the type of storage selected (local/shared) to migrate the volume to.
* [CLOUDSTACK-5235] Force users to enter old password when updating password
* Formatting for checkstyle
* Remove an unused import in AccountManagerImpl
* Apply Nitin's suggestions
* Change 'oldPassword' to 'currentPassword'
* Second review of Resmo
* Fix typos found by Nitin
* [CLOUDSTACK-10323] Allow changing disk offering during volume migration
This is a continuation of work developed on PR #2425 (CLOUDSTACK-10240), which provided root admins an override mechanism to move volumes between storage systems types (local/shared) even when the disk offering would not allow such operation. To complete the work, we will now provide a way for administrators to enter a new disk offering that can reflect the new placement of the volume. We will add an extra parameter to allow the root admin inform a new disk offering for the volume. Therefore, when the volume is being migrated, it will be possible to replace the disk offering to reflect the new placement of the volume.
The API method will have the following parameters:
* storageid (required)
* volumeid (required)
* livemigrate(optional)
* newdiskofferingid (optional) – this is the new parameter
The expected behavior is the following:
* If “newdiskofferingid” is not provided the current behavior is maintained. Override mechanism will also keep working as we have seen so far.
* If the “newdiskofferingid” is provided by the admin, we will execute the following checks
** new disk offering mode (local/shared) must match the target storage mode. If it does not match, an exception will be thrown and the operator will receive a message indicating the problem.
** we will check if the new disk offering tags match the target storage tags. If it does not match, an exception will be thrown and the operator will receive a message indicating the problem.
** check if the target storage has the capacity for the new volume. If it does not have enough space, then an exception is thrown and the operator will receive a message indicating the problem.
** check if the size of the volume is the same as the size of the new disk offering. If it is not the same, we will ALLOW the change of the service offering, and a warning message will be logged.
We execute the change of the Disk offering as soon as the migration of the volume finishes. Therefore, if an error happens during the migration and the volume remains in the original storage system, the disk offering will keep reflecting this situation.
* Code formatting
* Adding a test to cover migration with new disk offering (#4)
* Adding a test to cover migration with new disk offering
* Update test_volumes.py
* Update test_volumes.py
* fix test_11_migrate_volume_and_change_offering
* Fix typo in Java doc
This extends securing of KVM hosts to securing of libvirt on KVM
host as well for TLS enabled live VM migration. To simplify implementation
securing of host implies that both host and libvirtd processes are
secured with management server's CA plugin issued certificates.
Based on whether keystore and certificates files are available at
/etc/cloudstack/agent, the KVM agent determines whether to use TLS or
TCP based uris for live VM migration. It is also enforced that a secured
host will allow live VM migration to/from other secured host, and an
unsecured hosts will allow live VM migration to/from other unsecured
host only.
Post upgrade the KVM agent on startup will expose its security state
(secured detail is sent as true or false) to the managements server that
gets saved in host_details for the host. This host detail can be accesed
via the listHosts response, and in the UI unsecured KVM hosts will show
up with the host state of ‘unsecured’. Further, a button has been added
that allows admins to provision/renew certificates to KVM hosts and can
be used to secure any unsecured KVM host.
The `cloudstack-setup-agent` was modified to accept a new flag `-s`
which will reconfigure libvirtd with following settings:
listen_tcp=0
listen_tls=1
tcp_port="16509"
tls_port="16514"
auth_tcp="none"
auth_tls="none"
key_file = "/etc/pki/libvirt/private/serverkey.pem"
cert_file = "/etc/pki/libvirt/servercert.pem"
ca_file = "/etc/pki/CA/cacert.pem"
For a connected KVM host agent, when the certificate are
renewed/provisioned a background task is scheduled that waits until all
of the agent tasks finish after which libvirt process is restarted and
finally the agent is restarted via AgentShell.
There are no API or DB changes.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
While executing the find/replace, I found some blocks of duplicated code. Therefore, I extracted the duplicated part to an utils file, and then removed the duplicated blocks.
We no longer hash password in the user side. All of the password processing is executed in the server side according to the priority of hashing mechanism defined by the administrator.
* [CLOUDSTACK-10314] Add Text-Field to each ACL Rule
It is interesting to have a text field (e.g. CHAR-256) added to each ACL rule, which allows to enter a "reason" for each FW Rule created. This is valuable for customer documentation, as well as best practice for an evidence towards auditing the system
* Formatting to make check style happy and code clean ups
The ACL rules editing/addition page is not user-friendly. Users are not able to see in a single view all of the detail of the ACL rule (they need to use a scroll bar on the horizontal). The problem becomes worse when there are a considerable number of rules. Therefore, we are proposing the following changes:
1- Instead of using the table to create new ACL, we can create a button like the one presented in attached pictures, where users can click, and then a modal popup would appear and users would be able to create the new ACL there. This is similar to the workings of the ACL edit button.
2 - Remove the ability to add new ACL via table where they are presented. All ACLs should be entered via the “New ACL” button. Therefore, the section “Add ACL” would be removed as well;
3 - Move the action section of the list ACL table to the most left position;
These changes would reduce the information in the table and facilitate users to add new rules and easily edit them as well.
This fixes regression failures seen in Trillian, fixes NPEs that cause Travis related failures.
This also removes the aria2 dependency from rpms that require users to enable/install epel-release.
This finally updates the checksums for 4.11 systemvmtemplates in db upgrade path.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Allowed zone-wide primary storage based on a custom plug-in to be added via the GUI in a KVM-only environment (previously this only worked for XenServer and VMware)
Added support for root disks on managed storage with KVM
Added support for volume snapshots with managed storage on KVM
Enable creating a template directly from a volume (i.e. without having to go through a volume snapshot) on KVM with managed storage
Only allow the resizing of a volume for managed storage on KVM if the volume in question is either not attached to a VM or is attached to a VM in the Stopped state.
Included support for Reinstall VM on KVM with managed storage
Enabled offline migration on KVM from non-managed storage to managed storage and vice versa
Included support for online storage migration on KVM with managed storage (NFS and Ceph to managed storage)
Added support to download (extract) a managed-storage volume to a QCOW2 file
When uploading a file from outside of CloudStack to CloudStack, set the min and max IOPS, if applicable.
Included support for the KVM auto-convergence feature
The compression flag was actually added in version 1.0.3 (1000003) as opposed to version 1.3.0 (1003000) (changed this to reflect the correct version)
On KVM when using iSCSI-based managed storage, if the user shuts a VM down from the guest OS (as opposed to doing so from CloudStack), we need to pass to the KVM agent a list of applicable iSCSI volumes that need to be disconnected.
Added a new Global Setting: kvm.storage.live.migration.wait
For XenServer, added a check to enforce that only volumes from zone-wide managed storage can be storage motioned from a host in one cluster to a host in another cluster (cannot do so at the time being with volumes from cluster-scoped managed storage)
Don’t allow Storage XenMotion on a VM that has any managed-storage volume with one or more snapshots.
Enabled for managed storage with VMware: Template caching, create snapshot, delete snapshot, create volume from snapshot, and create template from snapshot
Added an SIOC API plug-in to support VMware SIOC
When starting a VM that uses managed storage in a cluster other than the one it last was running in, we need to remove the reference to the iSCSI volume from the original cluster.
Added the ability to revert a volume to a snapshot
Enabled cluster-scoped managed storage
Added support for VMware dynamic discovery
Extending Config Drive support
* Added support for VMware
* Build configdrive.iso on ssvm
* Added support for VPC and Isolated Networks
* Moved implementation to new Service Provider
* UI fix: add support for urlencoded userdata
* Add support for building systemvm behind a proxy
Co-Authored-By: Raf Smeets <raf.smeets@nuagenetworks.net>
Co-Authored-By: Frank Maximus <frank.maximus@nuagenetworks.net>
Co-Authored-By: Sigert Goeminne <sigert.goeminne@nuagenetworks.net>
CloudStack volumes and templates are one single virtual disk in case of XenServer/XCP and KVM hypervisors since the files used for templates and volumes are virtual disks (VHD, QCOW2). However, VMware volumes and templates are in OVA format, which are archives that can contain a complete VM including multiple VMDKs and other files such as ISOs. And currently, Cloudstack only supports Template creation based on OVA files containing a single disk. If a user creates a template from a OVA file containing more than 1 disk and launches an instance using this template, only the first disk is attached to the new instance and other disks are ignored.
Similarly with uploaded volumes, attaching an uploaded volume that contains multiple disks to a VM will result in only one VMDK to being attached to the VM.
FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Support+OVA+files+containing+multiple+disks
This behavior needs to be improved in VMWare to support OVA files with multiple disks for both uploaded volumes and templates. i.e. If a user creates a template from a OVA file containing more than 1 disk and launches an instance using this template, the first disk should be attached to the new instance as the ROOT disk and volumes should be created based on other VMDK disks in the OVA file and should be attached to the instance.
Signed-off-by: Abhinandan Prateek <abhinandan.prateek@shapeblue.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This feature allows using templates and ISOs avoiding secondary storage as intermediate cache on KVM. The virtual machine deployment process is enhanced to supported bypassed registered templates and ISOs, delegating the work of downloading them to primary storage to the KVM agent instead of the SSVM agent.
Template and ISO registration:
- When hypervisor is KVM, a checkbox is displayed with 'Direct Download' label.
- API methods registerTemplate and registerISO are both extended with this new parameter directdownload.
- On template or ISO registration, no download job is sent to SSVM agent, CloudStack would only persist an entry on template_store_ref indicating that template or ISO has been marked as 'Direct Download' (bypassing Secondary Storage). These entries are persisted as:
template_id = Template or ISO id on vm_template table
store_id NULL
download_state = BYPASSED
state = Ready
(Note: these entries allow users to deploy virtual machine from registered templates or ISOs)
- An URL validation command is sent to a random KVM host to check if template/ISO location can be reached. Metalink are also supported by this feature. In case of a metalink, it is fetched and URL check is performed on each of its URLs.
- Checksum should be provided as indicated on #2246: {ALGORITHM}CHKSUMHASH
- After template or ISO is registered, it would be displayed in the UI
Virtual machine deployment:
When a 'Direct Download' template is selected for deployment, CloudStack would delegate template downloading to destination storage pool via destination host by a new pluggable download manager.
Download manager would handle template downloading depending on URL protocol. In case of HTTP, request headers can be set by the user via vm_template_details. Those details should be persisted as:
Key: HTTP_HEADER
Value: HEADERNAME:HEADERVALUE
In case of HTTPS, a new API method is added uploadTemplateDirectDownloadCertificate to allow user importing a client certificate into all KVM hosts' keystore before deployment.
After template or ISO is downloaded to primary storage, usual entry would be persisted on template_spool_ref indicating the mapping between template/ISO and storage pool.
This feature allow admins to dedicate a range of public IP addresses to the SSVM and CPVM, such that they can be subject to specific external firewall rules. The option to dedicate a public IP range to the System VMs (SSVM & CPVM) is added to the createVlanIpRange API method and the UI.
Solution:
Global setting 'system.vm.public.ip.reservation.mode.strictness' is added to determine if the use of the system VM reservation is strict (when true) or preferred (false), false by default.
When a range has been dedicated to System VMs, CloudStack should apply IPs from that range to
the public interfaces of the CPVM and the SSVM depending on global setting's value:
If the global setting is set to false: then CloudStack will use any unused and unreserved public IP
addresses for system VMs only when the pool of reserved IPs has been exhausted
If the global setting is set to true: then CloudStack will fail to deploy the system VM when the pool
of reserved IPs has been exhausted, citing the lack of available IPs.
UI Changes
Under Infrastructure -> Zone -> Physical Network -> Public -> IP Ranges, button 'Account' label is refactored to 'Set reservation'.
When that button is clicked, dialog displayed is also refactored, including a new checkbox 'System VMs' which indicates if range should be dedicated for CPVM and SSVM, and a note indicating its usage.
When clicking on button for any created range, UI dialog displayed indicates whether IP range is dedicated for system vms or not.
This extends work presented on #2048 on which the ability to extend the management range is provided.
Aim
This PR allows separating the management network subnet on which SSVM and CPVM are from the virtual routers management subnet.
Detailed use case
PCI compliance requires that network elements are defined as ‘in scope’ or ‘out of scope’, for compliance purposes. The SSVM and CPVM are both in scope as they allow public HTTP or HTTPS connections. The virtual routers have been defined as out of scope as they have been placed entirely in a firewalled network's segment. However, all of the system VM types share management network. As SSVM and CPVM are both in scope this would bring the virtual routers into scope as well, requiring individual audits of every virtual router. As this is not practical, the ‘management network’ which the SSVM and CPVM are on, and the management network which the virtual routers are on, must be separated by a firewall.
Description
By this feature it is possible to dedicate a created range for SSVM and CPVM (system vms) and provide a VLAN ID for its range.
A new boolean global configuration is added: system.vm.management.ip.reservation.mode.strictness. If enabled, the use of System VMs management IP reservation is strict, preferred if not. Default value is false (preferred).
Strict reservation: System VMs should try to get a private IP from a range marked for system vms. If not available, deployment fails
Preferred reservation: System VMS will try to get a private IP from a range marked for system vms. If not available, IP for range not marked for system vms is taken.
Consider this scenario:
1. User launches a VM from Template and keep it running
2. Admin logins and deleted that template [CloudPlatform does not check existing / running VM etc. while the deletion is done]
3. User resets the VM
4. CloudPlatform fails to star the VM as it cannot find the corresponding template.
It throws error as
java.lang.RuntimeException: Job failed due to exception Resource [Host:11] is unreachable: Host 11: Unable to start instance due to can't find ready template: 209 for data center 1
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:113)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:495)
Client is requesting better handing of this scenario. We need to check existing / running VM's when the template is deleted and warn admin about the possible issue that may occur.
REPRO STEPS
==================
1. Launches a VM from Template and keep it running
2. Now delete that template
3. Reset the VM
4. CloudPlatform fails to star the VM as it cannot find the corresponding template.
EXPECTED BEHAVIOR
==================
Cloud platform should throw some warning message while the template is deleted if that template is being used by existing / running VM's
ACTUAL BEHAVIOR
==================
Cloud platform does not throw as waring etc.
This fixes test failures around VMware with the new systemvmtemplate.
In addition:
- Does not skip rVR related test cases for VMware
- Removes rc.local
- Processes unprocessed cmd_line.json
- Fixed NPEs around VMware tests/code
- On VMware, use udevadm to reconfigure nic/mac address than rebooting
- Fix proper acpi shutdown script for faster systemvm shutdowns
- Give at least 256MB of swap for VRs to avoid OOM on VMware
- Fixes smoke tests for environment related failures
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Several systemvmtemplate optimizations
- Uses new macchinina template for running smoke tests
- Switch to latest Debian 9.3.0 release for systemvmtemplate
- Introduce a new `get_test_template` that uses tiny test template
such as macchinina as defined test_data.py
- rVR related fixes and improvements
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Fixes timezone issue where dates show up as nvalid in UI
- Introduces new event timeline listing/filtering of events
- Several UI improvements to add columns in list views
- Bulk operations support in instance list view to shutdown and destroy
multiple-selected VMs (limitation: after operation, redundant entries
may show up in the list view, refreshing VM list view fixes that)
- Align table thead/tbody to avoid splitting of tables
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This feature allows CloudStack administrators to create layer 2 networks on CloudStack. As these networks are purely layer 2, they don't require IP addresses or Virtual Router, only VLAN is necessary (provided by administrator or assigned by CloudStack). Also, network services should be handled externally, e.g. DNS, DHCP, as they are not provided by L2 networks.
As a consequence, a new Guest Network type is created within CloudStack: L2
Description:
Network offerings and networks support new guest type: L2.
L2 Network offering creation allows administrator to select Specify VLAN or let CloudStack assign it dynamically.
L2 Network creation allows administrator to specify VLAN tag (if network offerings allows it) or simply create network.
VM deployments on L2 networks:
VMs should not IP addresses or any network service
No Virtual Router deployed on network
If Specify VLAN = true for network offering, network gets implemented using a dynamically assigned VLAN
UI changes
A new button is added on Networks tab, available for admins, to allow L2 networks creation
At present, The management IP range can only be expanded under the same subnet. According to existing range, either the last IP can be forward extended or the first IP can be backward extended. But we cannot add an entirely different range from the same subnet. So the expansion of range is subnet bound, which is fixed. But when the range gets exhausted and a user wants to deploy more system VMs, then the operation would fail. The purpose of this feature is to expand the range of management network IPs within the existing subnet. It can also delete and list the IP ranges.
Please refer the FS here: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Expansion+of+Management+IP+Range
This change adds allocatediops to the ListStoragePool API. This applies to managed storage where we have a guaranteed minimum IOPS set. This is useful for monitoring if we have reached the IOPS limit on a storage cluster.
The listNuageVspDomainTemplat cmd was called by the ui when the create vpc dialog was opened. This command failed when no nuage vsp device was present. As a consequence the ui did not show the dialog. So currently it's not possible to create a vpc through the UI with a native CloudStack deployment. This bugfix, adds robustness to the ui that in case the cmd fails the dialog will appear nonetheless . Furthermore I also changed the listNuageVspDomainTemplateCmd to always return an empty result when there is no nuage zone.
This shows the owner account and network of a VR in the VR list view,
and for VPCs shows the VPC name and redundant state of the VPC rVR.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Supports following viewing and navigation/filtering:
- Networks -> view instances (instances that have nics on this network)
- Infra-> VRs -> view instances (attached to the VR's guest network id),
view owner account (if not system, or project), view instances
- Infra -> VRs -> list of VPC redundant VRs will have redundant state in
parenthesis in the list
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* CLOUDSTACK-9972: Enhance listVolume API to include physical size and utilization.
Also fixed pool, cluster and pod info
* CLOUDSTACK-9972: Fix volume_view and duplicate API constant
* CLOUDSTACK-9972: Backport Do not allow vms to be deployed on hosts that are in disabled pod
* CLOUDSTACK-9972: Fix localization missing keys
* CLOUDSTACK-9972: Fix sql path
Allow security policies to apply on port groups:
- Accepts security policies while creating network offering
- Deployed network will have security policies from the network offering
applied on the port group (in vmware environment)
- Global settings as fallback when security policies are not defined for a network
offering
- Default promiscuous mode security policy set to REJECT as it's the default
for standard/default vswitch
Portgroup vlan-trunking options for dvswitch: This allows admins to define
a network with comma separated vlan id and vlan
range such as vlan://200-400,21,30-50 and use the provided vlan range to
configure vlan-trunking for a portgroup in dvswitch based environment.
VLAN overlap checks are performed for:
- isolated network against existing shared and isolated networks
- dedicated vlan ranges for the physical/public network for the zone
- shared network against existing isolated network
Allow shared networks to bypass vlan overlap checks: This allows admins
to create shared networks with a `bypassvlanoverlapcheck` API flag
which when set to 'true' will create a shared network without
performing vlan overlap checks against isolated network and against
the vlans allocated to the datacenter's physical network (vlan ranges).
Notes:
- No vlan-range overlap checks are performed when creating shared networks
- Multiple vlan id/ranges should include the vlan:// scheme prefix
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This commit is related to the Saml2 user authentication plugin.
The user can list its domains. His/Her domains are presented as the
following example:
username/subdomainA
username/subdomainB
However, if a user has two subdomains of the same name Dom1/subdomainA
and Dom2/subdomainA, the list is presented as follows:
username/subdomainA
username/subdomainA
With this commit it shows the full domain name to avoid such cases.
Thus, the domains will be presented as follows:
username/Dom1/subdomainA
username/Dom2/subdomainA
Root Cause:
The API returns the percentage value as floating point number but In UI it is converted to floor value of that number.
Hence for values less than 1%, the value is being converted to 0% and it doesn't show the value in the UI.
Solution:
Removed the conversion, now it represents as it is(Floating point value).
Nico Wohlfarth
Nicolas Vazquez
Abhishek Kumar
Anurag Awasthi
Rohit Yadav
ustcweizhou
smlshn
Philipp Bankonier
Abhishek Kumar
dahn
Frank Maximus
Nico Wohlfarth
Rohit Yadav
Philipp Bankonier
Dingane Hlaluku
Gabriel Beims Bräscher
Frank Maximus
Dingane Hlaluku
Gerd Müller
Henko
Rafael Weingärtner
Daan Hoogland
ernjvr
Mike Tutkowski
Mike Tutkowski
Khosrow Moossavi
Daan Hoogland
Rene Moser
Nitesh Sarda
Abhinandan Prateek
Mowgli
Sigert Goeminne
Nitin Kumar Maharana
kishankavala
Syed Mushtaq Ahmed
Rohit Yadav
Sigert Goeminne
Harika Punna
