* Use free/total instead of free metric to calculate imbalance
* Filter out hosts for condensed while checking imbalance
* Make DRS more configurable
* code refactor
* Add unit tests
* fixup
* Fix validation for drs.imbalance.condensed.skip.threshold
* Add logging and other minor changes for drs
* Add some logging for drs
* Change format for drs imbalance to string
* Show drs imbalance as percentage
* Fixup label for memorytotal in en.json
This PR fixes several issues in the testing of Veeam 11 and Veeam12
- Import Veeam.Backup.PowerShell and silently ignore the warning messages
- Fix issue when assign vm to backup offerings, which caused by separator (\r\n)
- Fix authorization failure in veeam 12a, which is because v1_4 is not supported in veeam 12a any more
- Fix exception if backup name has space
- Fix backup metrics in veeam12, which is because powershell command does not return the values needed
- Fix Incorrect datetime value, which is because powershell command returns a datetime which is not supported in Java
- Fix issue during backup restoration if VM has both ROOT and DATA disks.
This PR also has the following update
- Add integration test test/integration/smoke/test_backup_recovery_veeam.py
- Make some UI changes
- Add zone setting backup.plugin.veeam.version. If it is not set, CloudStack will get veeam version via powershell commands.
- Add zone setting backup.plugin.veeam.task.poll.interval and backup.plugin.veeam.task.poll.max.retry
Non-admin account may not have access to the readyForShutdown API therefore UI should not schedule the job.
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
The Default View has a projects toggle button that allows users to see which resources belong to projects, but there is nothing to indicate which project they belong to.
For this reason, a project column was added if the projects button is enabled, indicating the name of the project to which the resource belongs.
---------
Co-authored-by: Henrique Sato <henrique.sato@scclouds.com.br>
This PR adds the capability in CloudStack to convert VMware Instances disk(s) to KVM using virt-v2v and import them as CloudStack instances. It enables CloudStack operators to import VMware instances from vSphere into a KVM cluster managed by CloudStack. vSphere/VMware setup might be managed by CloudStack or be a standalone setup.
CloudStack will let the administrator select a VM from an existing VMware vCenter in the CloudStack environment or external vCenter requesting vCenter IP, Datacenter name and credentials.
The migrated VM will be imported as a KVM instance
The migration is done through virt-v2v: https://access.redhat.com/articles/1351473, https://www.ovirt.org/develop/release-management/features/virt/virt-v2v-integration.html
The migration process timeout can be set by the setting convert.instance.process.timeout
Before attempting the virt-v2v migration, CloudStack will create a clone of the source VM on VMware. The clone VM will be removed after the registration process finishes.
CloudStack will delegate the migration action to a KVM host and the host will attempt to migrate the VM invoking virt-v2v. In case the guest OS is not supported then CloudStack will handle the error operation as a failure
The migration process using virt-v2v may not be a fast process
CloudStack will not perform any check about the guest OS compatibility for the virt-v2v library as indicated on: https://access.redhat.com/articles/1351473.
In UI, shared network IP addresses are shown in a tab named Public IP addresses inside the network view.
Public IP addresses have their own subsection in the UI. Network → Public IP address. Shared network IP addresses are not shown in this view.
This is confusing for users and Public IP addresses tab in the network view has been renamed as IP addresses for a shared network.
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
OAuth2, the industry-standard authorization or authentication framework, simplifies the process of
granting access to resources. CloudStack supports OAuth2 authentication wherein users can login into
CloudStack without using a username and password. Support for Google and Github providers has been added.
Other OAuth2 providers can be easily integrated with CloudStack using its plugin framework.
The login page will show provider options when the OAuth2 is enabled and corresponding providers are configured.
"OAuth configuration" sub-section is present under "Configuration" where admins can register the corresponding
OAuth providers.
This pull request (PR) implements a Distributed Resource Scheduler (DRS) for a CloudStack cluster. The primary objective of this feature is to enable automatic resource optimization and workload balancing within the cluster by live migrating the VMs as per configuration.
Administrators can also execute DRS manually for a cluster, using the UI or the API.
Adds support for two algorithms - condensed & balanced. Algorithms are pluggable allowing ACS Administrators to have customized control over scheduling.
Implementation
There are three top level components:
Scheduler
A timer task which:
Generate DRS plan for clusters
Process DRS plan
Remove old DRS plan records
DRS Execution
We go through each VM in the cluster and use the specified algorithm to check if DRS is required and to calculate cost, benefit & improvement of migrating that VM to another host in the cluster. On the basis of cost, benefit & improvement, the best migration is selected for the current iteration and the VM is migrated. The maximum number of iterations (live migrations) possible on the cluster is defined by drs.iterations which is defined as a percentage (as a value between 0 and 1) of total number of workloads.
Algorithm
Every algorithms implements two methods:
needsDrs - to check if drs is required for cluster
getMetrics - to calculate cost, benefit & improvement of a migrating a VM to another host.
Algorithms
Condensed - Packs all the VMs on minimum number of hosts in the cluster.
Balanced - Distributes the VMs evenly across hosts in the cluster.
Algorithms use drs.level to decide the amount of imbalance to allow in the cluster.
APIs Added
listClusterDrsPlan
id - ID of the DRS plan to list
clusterid - to list plans for a cluster id
generateClusterDrsPlan
id - cluster id
iterations - The maximum number of iterations in a DRS job defined as a percentage (as a value between 0 and 1) of total number of workloads. Defaults to value of cluster's drs.iterations setting.
executeClusterDrsPlan
id - ID of the cluster for which DRS plan is to be executed.
migrateto - This parameter specifies the mapping between a vm and a host to migrate that VM. Format of this parameter: migrateto[vm-index].vm=<uuid>&migrateto[vm-index].host=<uuid>.
Config Keys Added
ClusterDrsPlanExpireInterval
Key drs.plan.expire.interval
Scope Global
Default Value 30 days
Description The interval in days after which old DRS records will be cleaned up.
ClusterDrsEnabled
Key drs.automatic.enable
Scope Cluster
Default Value false
Description Enable/disable automatic DRS on a cluster.
ClusterDrsInterval
Key drs.automatic.interval
Scope Cluster
Default Value 60 minutes
Description The interval in minutes after which a periodic background thread will schedule DRS for a cluster.
ClusterDrsIterations
Key drs.max.migrations
Scope Cluster
Default Value 50
Description Maximum number of live migrations in a DRS execution.
ClusterDrsAlgorithm
Key drs.algorithm
Scope Cluster
Default Value condensed
Description DRS algorithm to execute on the cluster. This PR implements two algorithms - balanced & condensed.
ClusterDrsLevel
Key drs.imbalance
Scope Cluster
Default Value 0.5
Description Percentage (as a value between 0.0 and 1.0) of imbalance allowed in the cluster. 1.0 means no imbalance
is allowed and 0.0 means imbalance is allowed.
ClusterDrsMetric
Key drs.imbalance.metric
Scope Cluster
Default Value memory
Description The cluster imbalance metric to use when checking the drs.imbalance.threshold. Possible values are memory and cpu.
This PR aims at improving the CloudStack dashboard and introduces the following:
Admin dashboard: six cards that are responsive to screen sizes and show zone specific compute, storage and network allocation, as well as instance/hosts stats, alerts and events. Now, by default, the admin dashboard shows aggegate data from all zones, with option for admin to select individual zone to see individual zone stats
Account/project dashboard: six cards that are responsive to screen sizes and show account or project specific resource lists/counts, and limits shown in three cards as (a) compute (with running stopped instances), (b) storage and (c) network allocation, an admin-defined links/docs card (via config.json) and events cards. Admin is allowed to configure project limits on project dashboards.
A global create button on the top global header/user-menu to allow for quick actions such as to deploy a VM, CKS cluster and create a volume (more actions can be added as desired via code changes) etc.
Doc PR - apache/cloudstack-documentation#349
---------
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Added a new getter 'allProjects' and mutation 'RELOAD_ALL_PROJECTS' to centralize the tracking of available projects in the state.
This eliminates direct manipulation of the Project list in separate components and improves data consistency across the application.
A watcher in ProjectMenu.vue has been implemented to handle changes to the 'allProjects' getter.
The 'RELOAD_ALL_PROJECTS' mutation was also added where necessary to ensure projects list is updated in the state whenever changes occur.
* css fixes from main
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* component: when in project, toggle the theme also
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
---------
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This PR aims to polish the UI with following tweaks and changes:
- Increase resource and os-logo icons both in list view, user-menu bar and VM deployment form
- Fix css issues in VM deployment form when resource icons are on some of the templates/isos but not all
- Replace edit icon in the resource icon editting button on the infocard, in resource view
- Fix css marging/padding issue for nav bar and left-branding/logo
- Introduce a new Limits option in the user menu, to allow users to see their own limits when they log in
- Rename resource tab to limits tab for accounts, project and domains
- Introduce a new copy-label component, that can be clicked to copy strings; use in info-card and list view for entites such as IP addresses and UUIDs
- Add router-link to /zones/ in case of user-accounts (when /zone isn't routable in the UI)
- Show better list of nics and ssh keys pairs in infocard for VM resource view
- Standardise most resources to show state/status columns right after resource name (wherever applicable)
- Remove displayname column in VM list view, add cpu number and memory by default
- Add k8s version column in k8s list view
- Add size and phy size columns in case of template and ISOs list view, only for root/domain admins
- Add phy network router-link in case of guest VLAN list view; rearrange columns list for consistency
- Add snapshot phy size column in the snapshot list view; and router-link for volume in the snapshot list view; and missing/useful details in the volume snapshot details view
- Add a create and add data disk feature in Instances tab, just like we've add nic feature in the same
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* 4.18:
UI: allow new keys for VM details (#7793)
Refactoring StorPool's smoke tests (#7392)
UI: decode userdata in EditVM dialog (#7796)
packaging: unalias cp before package upgrade (#7722)
make NoopDbUpgrade do a systemvm template check (#7564)
UI unit test: fix expected values (#7792)
* 4.18:
UI: Filter templates by zone and hypervisor type when reinstall a VM (#7739)
KVM: fix SSVM starting when overprovisioning memory (#7663)
pom.xml: add property project.systemvm.template.location (#7706)
cloudutils: fix adding rocky9 host failure due to missing /etc/sysconfig/libvirtd (#7779)
server: get id from persisted object ReservationVO (#7785)
search in (too) large result sets (#7766)
ui: fix 404 error when list volumes of system vms (#7772)
packaging: install tzdata-java on centos7/centos8 (#7768)
There are tools like cluster-api which create and manage kubernetes cluster on CloudStack. This PR adds the option to add unmanaged kubernetes cluster which are not managed by CKS plugin. This helps provide a consolidated view of unmanaged clusters on CloudStack. The changes done make sure that operations for managed clusters are not executed for unmanaged clusters.
Two new APIs have also been added:
1. addVirtualMachinesToKubernetesCluster - to add VMs to unmanaged clusters.
2. removeVirtualMachinesFromKubernetesCluster - to remove VMs to unmanaged clusters.
Two APIs have been updated:
1. createKubernetesCluster - made KUBERNETES_VERSION_ID, SERVICE_OFFERING_ID, SIZE as not required for unmanaged clusters. Add an additional parameter, managed, which is true by default.
2. listKubernetesClusters - Add a parameter managed to filter on managed field.
Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
* Guest OS mapping improvements
- Checks the OS mapping name in hypervisor (VMware, XenServer)
- Displays guest OS mappings in UI
* Added API getHypervisorGuestOsNames to list the guest OS names in the hypervisor, and code improvements
* Some static analysis fixes
* Removed commented code in listview
* Guest OS list
* UI changes for adding guest os and mappings
* Added guest os mappings in guest os form
* Added new filter to guest os mapping
* Name and description changes
* VMWare Host and cluster MO unit tests
* CheckGuestOsMapping command and answer unit tests
* GetHypervisorGuestOsNames command and answer unit tests
* VmwareResource unitests
* GuestOsMapper unittests
* icon changes
* Addressed review comments
* Renaming fixes
* Removed comments
* marvin tests for guest os operations
* Added marvin tests for OS mappings
* Document links and UI improvements
* Added deduplication for the list guest OS API
* Fixed linter failure
* Few bug fixes and UI changes
* Few improvements
* Addressed code smells
* Fixed UI issues after rebase
---------
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
Co-authored-by: Harikrishna Patnala <harikrishna.patnala@gmail.com>
Vishesh
Henrique Sato
Wei Zhou
Pearl Dsilva
Abhishek Kumar
Nicolas Vazquez
kishankavala
Rohit Yadav
dahn
Harikrishna
Gabriel Pordeus Santos
Charles Queiroz
João Jandre
John Bampton
GaOrtiga
Ilham Al Rahm