* Add persistence of VM stats
* Fix API 'since' attribute
* Add license
* Address GutoVeronezi's reviews
* Fix the order of VM stats in the API response
* Fix msid in VM stats data
* Fix disk stats and add minor improvements
* Add log message
* Build string using ReflectionToStringBuilderUtils
* Rerun checks
Co-authored-by: joseflauzino <jose@scclouds.com.br>
* VM snapshots of running KVM instance using storage providers plugins for disk snapshots
Added new virtual machine snapshot strategy which is using storage providers plugins to take/revert/delete snapshots.
You can take VM snapshot without VM memory on KVM instance, using storage providers implementations for disk snapshots.
Also revert and delete is added as functionality. Added Thaw/Freeze command for KVM instance.
The snapshots will be consistent, because we freeze the VM during the snapshotting. Backup to secondary storage is executed after
thaw of the VM and if it is enabled in global settings.
* Removed duplicated functionality
Set few methods in DefaultVMSnapshotStrategy to protected to reuse them
without duplicating the code. Remove code that is actualy not needed
* Added requirements in global setting kvm.vmstoragesnapshot.enabled
Added more information in kvm.vmstoragesnapshot.enabled global setting,
that it needs installation of:
- qemu version 1.6+
- qemu-guest-agent installed on guest virtual machine
when the option is enabled
* Added Apache license header
* Removed commented code
* If "kvm.vmstoragesnapshot.enabled" is null should be considered as false
* removed unused imports, replaced default template
Removed unused imports which causing failures and replaced template to
CentOS8
* "kvm.vmstoragesnapshot.enabled" set to dynamic
* Getting status of freeze/thaw commands not the return code
Will chacke the status if freeze/thaw of Guest VM succeded, rather than
looking for return code. Code refactoring
* removed "CreatingKVM" VMsnapshot state and events related to it
* renamed AllocatedKVM to AllocatedVM
the states should not be associated to a hypervisor type
* loggin the result of "drive-backup" command
* Check which VM snapshot strategy could handle the vm snapshots
gets the best match of VM snapshot strategy which could handle the vm
snapshots on KVM.
Other storage plugins could integrate with this functionality to support group snapshots
* Added poolId in canHandle for KVM hypervisors
Added poolId into canHandle method used to check if all volumes are on
the same PowerFlex's storage pool
* skip smoke tests if the hypervisor's OS type is CentOS
This PR works with functionality included in qemu-kvm-ev which
does not come by default on CentOS. The smoke tests will be skipped if
the hypervisor OS is CentOS
* Added missed import in smoke test
* Suggested change to use ` org.apache.commons.lang.StringUtils.isNotBlank`
* Fix getting device on Ubuntu
On Ubuntu the device isn't provided and we have to get it from
node-name parameter. For drive-backup command (for Ubuntu) is needed and job-id which
is the value of node-name (this extra param works on Ubuntu and CentOS as well).
* Removed new snapshot states and functionality for NFS
* throw CloudRuntimeException
provide a properer error message when delete VM snapshot fails
* exclude GROUP snapshots when listing snapshots
* Skip tests if there is pool with NFS/Local
* address comments
This enables jacoco, which didn't run before with the -P quality due to
missing passing of jacoco arg line to surefire plugin.
This also adds support for jacoco/quality builds using Github action and
posting of the PR coverage data using a new action step.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* Update VM priority (cpu_chares) when live scaling it
* Apply suggestions from code review
Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
* Addressing javadoc review
* Addressing review typo
Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
* [VMware] Support for removal of NIC on IP disassociation on the VR
* address comments - extract code + add appropriate operation name in logs
* Do not remove/unplug nic on removal of static nat on last IP (Public NIC)
* address comments
* address comment
* prevent role access escallation
* hierarchy issue fixed
* create api list in account manager for checking new account access
* full api list check
* strange role restriction removed for BareMetal
* add role check on upfdate account as well
* more selective use of api checkers
* error msg and var name
Co-authored-by: Daan Hoogland <dahn@onecht.net>
* maven: migrate short-term to reload4j v1.2.18
This migrate to log4j 1.x fork, reload4j 1.2.18.0 which is drop-in
replacement and addresses some immediate CVE and issues.
* log4j migration to reload4j in pom xmls
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* Exclude log4j from transitive dependencies (#73)
Co-authored-by: Marcus Sorensen <shadowsor@gmail.com>
Co-authored-by: Marcus Sorensen <mls@apple.com>
* kvm: Use lscpu to get cpu max speed
* Fix str conversion
* Reorder
* Refactor
* Apply suggestions from code review
Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
* Updated the calling method name getCpuSpeedFromCommandLscpu
* Make it more readable
Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>
* Add NFS version to mount command
* Remove extra line
* Extend NFS version to mount secondary storage
* Unused import
* Refactor NFS version to be granular
* Make use of the ConfigKey on the NFS version setting value
Currently, our compute offerings and disk offerings are tightly coupled with respect to many aspects. For example, if a compute offering is created, a corresponding disk offering entry is also created with the same ID as the reference. Also creating compute offering takes few disk-related parameters which anyway goes to the corresponding disk offering only. I think this design was initially made to address compute offering for the root volume created from a template. Also changing the offering of a volume is tightly coupled with storage tags and has to be done in different APIs either migrateVolume or resizeVolume. Changing of disk offering should be seamless and should consider new storage tags, new size and place the volume in appropriate state as defined in disk offering.
more details are mentioned here https://cwiki.apache.org/confluence/display/CLOUDSTACK/Compute+offering+and+disk+offering+refactoring
* Schema changes and disk offering column change from "type" to "compute_only"
* Few more changes
* Decoupled service offering and disk offering
* Remove diskofferingid from vminstance VO
* Decouple service offering and disk offering states
* diskoffering getsize() is only for strict disk offerings
* Fix deployVM flow
* Added new API params to compute offering creation
* Add diskofferingstrictness to serviceoffering vo under quota
* Added overrideDiskOfferingId parameter in deploy VM API which will override disk offering for the root disk both in template and ISO case
Added diskSizeStrictness parameter in create Disk offering API which will decide whether to restrict resize or disk offering change of a volume
* Fix User vm response to show proper service offering and disk offerings
* Added disk size strictness in disk offering response
* Added disk offering strictness to the service offering response
* Remove comments
* Added UI changes for Disk offering strictness in add compute offering form and Disk size strictness in add disk offering form
* Added diskoffering details to the service offering response
* Added UI changes in deployvm wizard to accept override disk offering id
* Fix delete compute offering
* Fix VM deployment from custom service offering
* Move uselocalstorage column access from service offering to disk offering
* UI: Separated compute and disk releated parameters in add compute offering wizard, also added association to disk offering
* Fixed diskoffering automatic selection on add compute offering wizard
* UI: move compute only toggle button outside the box in add compute offering wizard
* Added volumeId parameter to listDiskOfferings API and the disksizestrictness flag of the current disk offering is honored while list disk offerings
* Added configuration parameter to decide whether to check volume tags on the destination storagepool during migration
* Added disk offering change checks during resize volume operation
* Added new API changeofferingforVolume API and corresponding changes
* Add UI form for changeOfferingForVolume API
* Fix UI conflicts
* Fix service offering usage as disk offering
* Fix unit test failures
* fix user_vm_view
* Addressed review comments
* Fixed service_offering_view
* Fix service offering edit flow
* Fix service offering constructor to address custom offering
* Fix domain_router_view to get proper service offering id
* Removed unused import
* Addressed review comments and fixed update service offering flow with storage tags
* Added marvin test cases for checking disk offering strictness
* review comments addressed
* Remove system_use column from disk offering join
* update volume_view to update system_use column from service offering and not disk offering
* Fix changeOfferingForVolume API for custom disk offering
* Fix global setting implementation
* Fix list volumes, after changing system_use column from disk offering to service offering in volume_view
* Changes for override root disk offering in deployvm wizard in case of custom offering
* Fix a unit test case
* Fixed recent unit test cases with new serviceofferingvo constructor
* Fix unit test in VolumeApiServiceImpl
* Added storage id for the list disk offering API and corresponding UI changes in migrateVolume and changeOfferingForVolume flow
* Rename global configuration parameter from storage.pool.tags.disk.offering.strictness to match.storage.pool.tags.with.disk.offering
* Fix smoke test failures
* Added tool tip for migrate volume UI form
* Address review comments and fix UI form of deploy VM in case of ISO.
* Fixed resize volume UI form for data disk
* UI changes to disable override root disk size when override root disk offering is enabled
* UI fix in deploy vm wizard
* Fix listdiskoffering after rebasing with main
* Fixed UI in migrate and changeofferingfor volume to handle empty disk offering list
Removed the volume's current disk offering from listDiskOffering response list
* Added custom Iops to resize volume form and removed the current disk offering during change offering for volume UI form
* Fix false response on updateDiskOffering API
* Added search field for changeofferingforvolume UI form
* Fix resize volume and migrate volume to update volume path if DRS is applied on volume in datastore cluster
* Removed DB changes from 4.16 upgrade file
* Resolving merge conflicts with main 4.17
* Added support for auto migration and auto resize of the root volume upon changing the service offering for VM.
* UI: Added automigrate checkbox in scale VM form
* Addes since attributes to new API params
* Added shrinkOK parameter to changeofferingforvolume API
* Added shrinkOk param to UI in changeOfferingforVolume form
* Added shrinkOk flag to scaleVM and changeServiceForVirtualMachines and UI form
* Removed old foreign key constraint on IDs of service offering and disk offering
* Allow resize and automigrate of root volume if required in all cases of service offering change
* Allow only resize to higher disk size from UI
* Fixing vue syntax error
* Make UI changes to provide root disk size box when the linked disk offering is of custom
* Converted from check box to toggle in scale VM, changeoffering, resize and migrate volume forms
* Fix resize volume operation to update the VM settings
* Fix migratevolume form to pick selected storage pool id in list diskofferings API
* kvm: don't force scsi controller for aarch64 VMs
This would allow use of virtio disk controller with Ceph, etc or as
defined in the VM's root disk controller setting, rather than always
enforce SCSI.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* remove test that doesn't apply now
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* address review comment
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This PR improves the volume file search on the datastore while computing the VM snapshot chain size in VMware. The file search is now performed in the VM directory on the datastore, instead of all the directories on the datastore (which is leading to incorrect VM snapshot chain size computation when VM id has 6 digits).
With diskful set to true, linstor will fail if it cannot create local
storage for the resource. Which in turn will make it impossible to have a
setup with just compute nodes on cloudstack.
* xcp-ng: fix vm boot options
Use value boot option values from VM details directly similar to other hypervisor plugins instead of relying on boot properties explicitly set for VirtualMachineTO.
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Harikrishna Patnala <harikrishna.patnala@gmail.com>
* Fix resize volume and migrate volume to update volume path if DRS is applied on volume in datastore cluster
* Change in constructors
* Naming changes
* Remove commented code
* Refactor code for more readability
* Addressed review comments on code refactor
* KVM: Add MV Settings for virtual GPU hardware type and memory
* fix method createVideoDef argument in test package
* add available options for KVM virtual GPU hardware VM setting
* fix videoRam default value
* fix _videoRam is 0, it will use default provided by libvirt
* Fix export snapshot and export template to secondary storage in VMware to export only one required disk
* Move clone operation into virtual machine mo
* Code refactored for readability
* Added disk key check even for successful clone operation
* Delete dettached disks from cloned VM and added few logs
This adds a volume(primary) storage plugin for the Linstor SDS.
Currently it can create/delete/migrate volumes, snapshots should be possible,
but currently don't work for RAW volume types in cloudstack.
* plugin-storage-volume-linstor: notify libvirt guests about the resize
* vmware: delete snapshot disk after backup to secondary storage
WIP - This ensures that worker VM is destroyed along with any of its own
disks that are backed up to secondary storage.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* fix
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
* fix for volume backup and confuding vm var name
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
* change
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
* tag as worker vm
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
* vmware: fix copy systemvm.iso for same version
For VMware, systemvm.iso is copied from MS to secondary store. Current server checks if the desired file is present on the secondary store or not. If it is not present ISO is copied.
This change adds a check for checksum for source and destination ISO which would allow copying new ISO if there is a mismatch.
Useful in case when file is corrupted in secondary store or new systemvm.iso is generated for dev environment.
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
* changes
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
* Fix of creating volumes from snapshots without backup
When few snaphots are created onyl on primary storage, and try to create
a volume or a template from the snapshot only the first operation is
successful. Its because the snapshot is backup on secondary storage with
wrong SQL query. The problem appears on Ceph/NFS but may affects other
storage plugins.
Bypassing secondary storage is implemented only for Ceph primary storage
and it didn't cover the functionality to create volume from snapshot
which is kept only on Ceph
* Address review
Worker VM tags are missed for few cloned VMs in VMware, and so these are skipped when tracking / cleaning up of Worker VMs. Adding proper Worker VM tags to these VMs would make them trackable from CloudStack.
* Added support for removing unused port groups on VMWare
* Fixed error handling around unavailable portgroup name
* Review changes, defaulting glovbal var to false, added warning to description, changed if statement.
* Cleanup unused network port groups on all the hosts.
Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
Co-authored-by: nicolas <nicovazquez90@gmail.com>
* CLOUDSTACK-9175: [VMware DRS] Adding new host to DRS cluster does not participate in load balancing.
Summary: When a new host is added to a cluster, Cloudstack doesn't create all the port groups (created by cloudstack earlier in other hosts) present in the cluster. Since the new host doesn't have all the necessary networking port groups of cloudstack, it is not eligible to participate in DRS load balancing or HA.
Solution: When adding a host to the cluster in Cloudstack, use VMware API to find the list of unique port groups on a previously added host (older host in the cluster) if exists and then create them on the new host.
* Added few checks for cluster details
* remove hot enable cpu und memory in case of reservation
ram and cpu reservation have not relation to ram and cpu hot add
* add custom ram_reservation and it to vm details
* system vms haven't this property, for this reason add additional check
* Update plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/VmwareResource.java
Co-authored-by: dahn <daan.hoogland@gmail.com>
* replace 0.0 with NumberUtils
* remove default value and remove return MinRam(seems to be not necessary)
* Update plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/guru/VmwareVmImplementer.java
Co-authored-by: davidjumani <dj.davidjumani1994@gmail.com>
* Update plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/VmwareResource.java
Co-authored-by: davidjumani <dj.davidjumani1994@gmail.com>
Co-authored-by: DK101010 <dirk.klahre@itelligence.de>
Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: davidjumani <dj.davidjumani1994@gmail.com>
* Create utility to centralize byte convertions
* Add/change toString definitions
* Create Libvirt handler to ScaleVmCommand
* Enable dynamic scalling VM with KVM
* Move config from interface to class and rename it
As every variable declared in interfaces are already final,
this moving will be needed to mock tests in nexts commits
* Configure VM max memory and cpu cores
The values are according to service offering or global configs
* Extract dpdk configuration to a method and test it
* Extract OS desc config to a method and test it
* Extract guest resource def to a method and test it
Improve libvirt def
* Refactor LibvirtVMDef.GuestResourceDef
* Refactor ScaleVmCommand
* Improve VMInstaVO toString()
* Refactor upgradeRunningVirtualMachine method
* Turn int variables into long on utility
* Verify if VM is scalable on KVMGuru
* Rename some KVMGuruTest's methods
* Change vm's xml to work with max memory
* Verify if service offering is dynamic before scale
* Create methods to retrieve data from domain
* Create def to hotplug memory
* Adjust the way command was scaling the VM
* Fix database persistence before executing command
* Send more info to host to improve log
* Fix var name
* Fix missing "}"
* Undo unnecessary changes
* Address review
* Fix scale validation
* Add VM prepared for dynamic scaling validation
* Refactor LibvirtScaleVmCommandWrapper and improve unit tests
* Remove duplicated method
* Add RuntimeException check
* Remove copyright from header
* Remove copyright from header
* Remove copyright from header
* Remove copyright from header
* Remove copyright from header
* Update ByteScaleUtilsTest.java
Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
* Add SharedMountPoint to KVMs supported storage pool types
* Fix live migration to iSCSI and improve logs
Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
* disable hot add memory and cpu via vm settings
* add alternative implementation for hot add memory and cpu
* add log entry
* Modify and add log entry for hotadd
* Update plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/VmwareResource.java
Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>
* Update plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/VmwareResource.java
Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>
Co-authored-by: DK101010 <dirk.klahre@itelligence.de>
Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>
* server: skip zone check for PERHOST iso during attachIso
Hypervisor tools ISO - vmware-toools.iso, xs-tools.iso are marked as PERHOST in DB. They are active but not downloaded to the secondary storages and hence no template-zone entry.
Skips the template-zone check for such templates.
Fixes#5265
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
* inverted check
* use constants in TemplateManager
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
José Flauzino
slavkap
Rohit Yadav
David Jumani
Nicolas Vazquez
Wei Zhou
John Bampton
Bryan Lima
JoaoJandre
dahn
Rohit Yadav
Suresh Kumar Anaparti
Suresh Kumar Anaparti
Pearl Dsilva
dahn
Harikrishna
SadiJr
Abhishek Kumar
sureshanaparti
DH Park
Daniel Augusto Veronezi Salvador
Peinthor Rene
div8cn
Leo (Hsueh Yu-Min)
Spaceman1984
SadiJr
DK101010