cloudstack/systemvm/debian/opt/cloud/bin
sureshanaparti eba186aa40
storage: New Dell EMC PowerFlex Plugin (formerly ScaleIO, VxFlexOS) (#4304)
Added support for PowerFlex/ScaleIO (v3.5 onwards) storage pool as a primary storage in CloudStack (for KVM hypervisor) and enabled VM/Volume operations on that pool (using pool tag).
Please find more details in the FS here:
https://cwiki.apache.org/confluence/x/cDl4CQ

Documentation PR: apache/cloudstack-documentation#169

This enables support for PowerFlex/ScaleIO (v3.5 onwards) storage pool as a primary storage in CloudStack

Other improvements addressed in addition to PowerFlex/ScaleIO support:

- Added support for config drives in host cache for KVM
	=> Changed configuration "vm.configdrive.primarypool.enabled" scope from Global to Zone level
	=> Introduced new zone level configuration "vm.configdrive.force.host.cache.use" (default: false) to force host cache for config drives
	=> Introduced new zone level configuration "vm.configdrive.use.host.cache.on.unsupported.pool" (default: true) to use host cache for config drives when storage pool doesn't support config drive
	=> Added new parameter "host.cache.location" (default: /var/cache/cloud) in KVM agent.properties for specifying the host cache path and create config drives on the "/config" directory on the host cache path
	=> Maintain the config drive location and use it when required on any config drive operation (migrate, delete)

- Detect virtual size from the template URL while registering direct download qcow2 (of KVM hypervisor) templates

- Updated full deployment destination for preparing the network(s) on VM start

- Propagate the direct download certificates uploaded to the newly added KVM hosts

- Discover the template size for direct download templates using any available host from the zones specified on template registration
	=> When zones are not specified while registering template, template size discovery is performed using any available host, which is picked up randomly from one of the available zones

- Release the VM resources when VM is sync-ed to Stopped state on PowerReportMissing (after graceful period)

- Retry VM deployment/start when the host cannot grant access to volume/template

- Mark never-used or downloaded templates as Destroyed on deletion, without sending any DeleteCommand
	=> Do not trigger any DeleteCommand for never-used or downloaded templates as these doesn't exist and cannot be deleted from the datastore

- Check the router filesystem is writable or not, before performing health checks
	=> Introduce a new test "filesystem.writable.test" to check the filesystem is writable or not
	=> The router health checks keeps the config info at "/var/cache/cloud" and updates the monitor results at "/root" for health checks, both are different partitions. So, test at both the locations.
	=> Added new script: "filesystem_writable_check.py" at /opt/cloud/bin/ to check the filesystem is writable or not

- Fixed NPE issue, template is null for DATA disks. Copy template to target storage for ROOT disk (with template id), skip DATA disk(s)

* Addressed some issues for few operations on PowerFlex storage pool.

- Updated migration volume operation to sync the status and wait for migration to complete.

- Updated VM Snapshot naming, for uniqueness in ScaleIO volume name when more than one volume exists in the VM.

- Added sync lock while spooling managed storage template before volume creation from the template (non-direct download).

- Updated resize volume error message string.

- Blocked the below operations on PowerFlex storage pool:
  -> Extract Volume
  -> Create Snapshot for VMSnapshot

* Added the PowerFlex/ScaleIO client connection pool to manage the ScaleIO gateway clients, which uses a single gateway client per Powerflex/ScaleIO storage pool and renews it when the session token expires.

- The token is valid for 8 hours from the time it was created, unless there has been no activity for 10 minutes.
  Reference: https://cpsdocs.dellemc.com/bundle/PF_REST_API_RG/page/GUID-92430F19-9F44-42B6-B898-87D5307AE59B.html

Other fixes included:

- Fail the VM deployment when the host specified in the deployVirtualMachine cmd is not in the right state (i.e. either Resource State is not Enabled or Status is not Up)

- Use the physical file size of the template to check the free space availability on the host, while downloading the direct download templates.

- Perform basic tests (for connectivity and file system) on router before updating the health check config data
	=> Validate the basic tests (connectivity and file system check) on router
	=> Cleanup the health check results when router is destroyed

* Updated PowerFlex/ScaleIO storage plugin version to 4.16.0.0

* UI Changes to support storage plugin for PowerFlex/ScaleIO storage pool.
- PowerFlex pool URL generated from the UI inputs(Gateway, Username, Password, Storage Pool) when adding "PowerFlex" Primary Storage
- Updated protocol to "custom" for PowerFlex provider
- Allow VM Snapshot for stopped VM on KVM hypervisor and PowerFlex/ScaleIO storage pool

and Minor improvements in PowerFlex/ScaleIO storage plugin code

* Added support for PowerFlex/ScaleIO volume migration across different PowerFlex storage instances.

- findStoragePoolsForMigration API returns PowerFlex pool(s) of different instance as suitable pool(s), for volume(s) on PowerFlex storage pool.
- Volume(s) with snapshots are not allowed to migrate to different PowerFlex instance.
- Volume(s) of running VM are not allowed to migrate to other PowerFlex storage pools.
- Volume migration from PowerFlex pool to Non-PowerFlex pool, and vice versa are not supported.

* Fixed change service offering smoke tests in test_service_offerings.py, test_vm_snapshots.py

* Added the PowerFlex/ScaleIO volume/snapshot name to the paths of respective CloudStack resources (Templates, Volumes, Snapshots and VM Snapshots)

* Added new response parameter “supportsStorageSnapshot” (true/false) to volume response, and Updated UI to hide the async backup option while taking snapshot for volume(s) with storage snapshot support.

* Fix to remove the duplicate zone wide pools listed while finding storage pools for migration

* Updated PowerFlex/ScaleIO volume migration checks and rollback migration on failure

* Fixed the PowerFlex/ScaleIO volume name inconsistency issue in the volume path after migration, due to rename failure
2021-02-24 14:58:33 +05:30
..
cs vr: Ensuring dnsmasq.leases file is populated (#4529) 2020-12-14 09:06:24 +00:00
setup vpc: fix ips on wrong interfaces after rebooting vpc vrs (#4467) 2020-11-20 21:02:53 +05:30
baremetal-vr.py mac-check 2020-04-16 15:10:50 +02:00
baremetal_snat.sh CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
bumpup_priority.sh CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
checkbatchs2svpn.sh CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
checkrouter.sh vr: Fix vpc router in UNKNOWN state (#3465) 2019-07-11 01:48:37 +05:30
checks2svpn.sh CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
cleanup.sh Get Diagnostics: Download logs and diagnostics data from SSVM, CPVM, Router (#3350) 2020-01-15 11:38:33 +01:00
cloud-nic.sh router: Fixes #2789 fix proper mark based packet routing across interfaces (#2791) 2018-08-08 12:05:42 +05:30
configure.py bugfix #9 vpc vr: Add PREROUTING rule for vm with static nat to multiple private gateways 2020-11-25 08:40:16 +01:00
cs_cmdline.py CLOUDSTACK-10013: Fixes based on code review and test failures 2017-12-23 17:51:42 +05:30
cs_dhcp.py VR: fix expunging vm will remove dhcp entries of another vm in VR (#4627) 2021-02-05 18:10:53 +05:30
cs_firewallrules.py CLOUDSTACK-10013: Fixes based on code review and test failures 2017-12-23 17:51:42 +05:30
cs_forwardingrules.py CLOUDSTACK-10013: Fixes based on code review and test failures 2017-12-23 17:51:42 +05:30
cs_guestnetwork.py CLOUDSTACK-10013: Fixes based on code review and test failures 2017-12-23 17:51:42 +05:30
cs_ip.py router: Fixes #2719 program VR nics by device id order for VPC (#2888) 2018-10-10 15:20:36 +05:30
cs_loadbalancer.py CLOUDSTACK-10013: Fixes based on code review and test failures 2017-12-23 17:51:42 +05:30
cs_monitorservice.py Health check feature for virtual router (#3575) 2020-01-30 12:39:03 +01:00
cs_network_acl.py CLOUDSTACK-10013: Fixes based on code review and test failures 2017-12-23 17:51:42 +05:30
cs_remoteaccessvpn.py CLOUDSTACK-10013: Fixes based on code review and test failures 2017-12-23 17:51:42 +05:30
cs_site2sitevpn.py CLOUDSTACK-10013: Fixes based on code review and test failures 2017-12-23 17:51:42 +05:30
cs_staticroutes.py CLOUDSTACK-10013: Fixes based on code review and test failures 2017-12-23 17:51:42 +05:30
cs_vmdata.py CLOUDSTACK-10013: Fixes based on code review and test failures 2017-12-23 17:51:42 +05:30
cs_vmp.py CLOUDSTACK-10013: Fixes based on code review and test failures 2017-12-23 17:51:42 +05:30
cs_vpnusers.py CLOUDSTACK-10013: Fixes based on code review and test failures 2017-12-23 17:51:42 +05:30
diagnostics.py diagnostics: new diagnostics admin API for system VMs (#2721) 2018-07-13 16:58:45 +05:30
dnsmasq.sh CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
edithosts.sh CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
filesystem_writable_check.py storage: New Dell EMC PowerFlex Plugin (formerly ScaleIO, VxFlexOS) (#4304) 2021-02-24 14:58:33 +05:30
getRouterAlerts.sh cloudstack: add JDK11 support (#3601) 2020-02-12 12:58:25 +05:30
getRouterMonitorResults.sh Health check feature for virtual router (#3575) 2020-01-30 12:39:03 +01:00
get_diagnostics_files.py Get Diagnostics: Download logs and diagnostics data from SSVM, CPVM, Router (#3350) 2020-01-15 11:38:33 +01:00
get_template_version.sh CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
ilb.sh CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
ipassoc.sh router: Fixes #2789 fix proper mark based packet routing across interfaces (#2791) 2018-08-08 12:05:42 +05:30
ipsectunnel.sh CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
line_edit.py Fix Python code checkstyle execute by "systemvm\test\runtests.sh" (#2576) 2018-04-18 13:07:37 -03:00
loadbalancer.sh CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
master.py CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
merge.py Revert "Handle private gateways more reliably" 2020-11-25 08:40:16 +01:00
monitor_service.sh CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
netusage.sh CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
passwd_server_ip.py router: Set up metadata/password/dhcp server on gateway IP instead of guest IP in RVR (#3477) 2020-01-28 10:35:59 +05:30
prepare_pxe.sh CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
savepassword.sh CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
set_redundant.py CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
update_config.py VR: fix logging is not working and logs are not appended to /var/log/cloud.log (#4466) 2020-11-20 10:40:02 +00:00
vmdata.py Fix Python code checkstyle execute by "systemvm\test\runtests.sh" (#2576) 2018-04-18 13:07:37 -03:00
vpc_func.sh CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
vpc_netusage.sh CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
vpc_snat.sh CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
vpc_staticroute.sh CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
vpn_l2tp.sh CLOUDSTACK-10013: SystemVM codebase refactorings and improvements 2017-12-23 09:22:44 +05:30
vr_cfg.sh systemvm: loop optimisation in bash (#4451) 2021-02-18 18:18:16 +05:30