cloudstack/test/integration/smoke
Wei Zhou b34f093137
veeam: fix some issues with restoring volume from backup and attaching it to VM (#8570)
* veeam: detach only the restored volume during backup restore

Steps to reproduce the issue
1. create a VM (A) with ROOT and DATA disk
2. assign to a backup offering
3. create backup
4. create another VM (B)
5. restore the DATA disk of VM A, and attach to VM B
6. When operation is done, check the datastore

Without this change, the ROOT image is not removed and left over on the datastore.
```
[root@ref-trl-5933-v-Mr8-wei-zhou-esxi2:/vmfs/volumes/5f60667d-18d828eb] ls -l /vmfs/volumes/5f60667d-18d828eb/CS-RSTR-dfb6f21c-a941-49db-9963-4f0286a17dac
total 1784840
-rw-------    1 root     root     5242880000 Jan 24 09:23 ROOT-722_2-flat.vmdk
-rw-------    1 root     root           499 Jan 24 09:23 ROOT-722_2.vmdk
```

With this change, the whole temporary vm has been destroyed.
```
[root@ref-trl-5933-v-Mr8-wei-zhou-esxi2:/vmfs/volumes/5f60667d-18d828eb] ls -l /vmfs/volumes/5f60667d-18d828eb/CS-RSTR-734bee3b-640c-4ff0-a34b-bc45358565b2
ls: /vmfs/volumes/5f60667d-18d828eb/CS-RSTR-734bee3b-640c-4ff0-a34b-bc45358565b2: No such file or directory
```

* veeam: fix wrong disk size in debug message

* veeam: sync backup repository after operations are done

got exception of some operations which succeeds due to the following error
```
2024-01-19 10:59:52,846 DEBUG [o.a.c.b.v.VeeamClient] (API-Job-Executor-42:ctx-716501bb job-4373 ctx-2359b76d) (logid:b5e19a17) Veeam response for PowerShell commands [PowerShell Import-Module Veeam.Backup.PowerShell -WarningAction SilentlyContinue;$restorePoint = Get-VBRRestorePoint ^| Where-Object { $_.Id -eq '1d99106a-b5c8-4a1e-958d-066a987caa5f' };if ($restorePoint) { Remove-VBRRestorePoint -Oib $restorePoint -Confirm:$false;$repo = Get-VBRBackupRepository;Sync-VBRBackupRepository -Repository $repo;} else { ; Write-Output 'Failed to delete'; Exit 1;}] is: [^M
Restore Type       Job Name             State      Start Time             End Time               Description           ^M
------------       --------             -----      ----------             --------               -----------           ^M
ConfResynchronize  Configuration Dat... Starting   19/01/2024 10:59:52    01/01/1900 00:00:00                          ^M
^M
^M
Remove-VBRRestorePoint : Win32 internal error "Access is denied" 0x5 occurred while reading the console output buffer. ^M
Contact Microsoft Customer Support Services.^M
At line:1 char:196^M
+ ... orePoint) { Remove-VBRRestorePoint -Oib $restorePoint -Confirm:$false ...^M
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^M
    + CategoryInfo          : ReadError: (:) [Remove-VBRRestorePoint], HostException^M
    + FullyQualifiedErrorId : ReadConsoleOutput,Veeam.Backup.PowerShell.Cmdlets.RemoveVBRRestorePoint^M
 ^M
].
```

* veeam: fix unable to detach volume when restore backup and attach to vm then detach the volume

It also happened when destroy the original or backup VM

```
2024-01-24 10:10:03,401 ERROR [c.c.s.r.VmwareStorageProcessor] (DirectAgent-74:ctx-95b24ac7 10.0.35.53, job-25995/job-25996, cmd: DettachCommand) (logid:7260ffb8) Failed to detach volume!
java.lang.RuntimeException: Unable to access file [de52fdd3386b3d67b27b3960ecdb08f4] i-2-723-VM/7c2197c129464035bab062edec536a09-flat.vmdk
        at com.cloud.hypervisor.vmware.util.VmwareClient.waitForTask(VmwareClient.java:426)
        at com.cloud.hypervisor.vmware.mo.DatastoreMO.moveDatastoreFile(DatastoreMO.java:290)
        at com.cloud.storage.resource.VmwareStorageLayoutHelper.syncVolumeToRootFolder(VmwareStorageLayoutHelper.java:241)
        at com.cloud.storage.resource.VmwareStorageProcessor.attachVolume(VmwareStorageProcessor.java:2150)
        at com.cloud.storage.resource.VmwareStorageProcessor.dettachVolume(VmwareStorageProcessor.java:2408)
        at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:174)
        at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:71)
        at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:589)
        at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
2024-01-24 10:10:03,402 INFO  [c.c.h.v.u.VmwareHelper] (DirectAgent-74:ctx-95b24ac7 10.0.35.53, job-25995/job-25996, cmd: DettachCommand) (logid:7260ffb8) [ignored]failed to get message for exception: Unable to access file [de52fdd3386b3d67b27b3960ecdb08f4] i-2-723-VM/7c2197c129464035bab062edec536a09-flat.vmdk
```

* vmware: create restored volume with new UUID and attach to VM
2024-01-29 11:40:43 +01:00
..
__init__.py Add Python flake8 linting for W291 trailing whitespace with Super-Linter (#4687) 2022-03-28 11:40:26 -03:00
test_2fa.py User two factor authentication (#6924) 2023-02-13 09:14:17 +01:00
test_accounts.py Fix spelling (#7087) 2023-01-16 10:56:07 +01:00
test_affinity_groups.py Add Python flake8 linting for W291 trailing whitespace with Super-Linter (#4687) 2022-03-28 11:40:26 -03:00
test_affinity_groups_projects.py Add Python flake8 linting for W291 trailing whitespace with Super-Linter (#4687) 2022-03-28 11:40:26 -03:00
test_annotations.py Enable flake8 rule W292 No newline at end of file (#6274) 2022-06-30 12:08:27 +05:30
test_async_job.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_attach_multiple_volumes.py test,xcp-ng: fix tests for VM PV driver issue (#6549) 2022-08-09 12:44:27 +05:30
test_backup_recovery_dummy.py Add Python flake8 linting for W291 trailing whitespace with Super-Linter (#4687) 2022-03-28 11:40:26 -03:00
test_backup_recovery_veeam.py veeam: fix some issues with restoring volume from backup and attaching it to VM (#8570) 2024-01-29 11:40:43 +01:00
test_certauthority_root.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_console_endpoint.py console: Console access enhancements (#6577) 2022-09-14 12:39:59 +05:30
test_create_list_domain_account_project.py Add Python flake8 linting for W291 trailing whitespace with Super-Linter (#4687) 2022-03-28 11:40:26 -03:00
test_create_network.py Make displayText in createNetwork optional (#7112) 2023-02-08 14:41:59 +01:00
test_deploy_vgpu_enabled_vm.py Add Python flake8 linting for W291 trailing whitespace with Super-Linter (#4687) 2022-03-28 11:40:26 -03:00
test_deploy_virtio_scsi_vm.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_deploy_vm_extra_config_data.py integration test fixes for 4.18 (#7190) 2023-02-14 08:20:11 +01:00
test_deploy_vm_iso.py CLOUDSTACK-10013: Fixes based on code review and test failures 2017-12-23 17:51:42 +05:30
test_deploy_vm_iso_uefi.py Adding support for RHEL8 binary-compatible variants (#5158) 2021-08-18 10:03:03 +02:00
test_deploy_vm_root_resize.py VM Autoscaling with virtual router (#6571) 2022-12-05 15:23:03 +01:00
test_deploy_vm_with_userdata.py Fix spelling (#6898) 2022-12-13 14:58:14 +01:00
test_deploy_vms_in_parallel.py Resource reservation framework (#6694) 2022-09-16 15:44:35 +05:30
test_deploy_vms_with_varied_deploymentplanners.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_diagnostics.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_direct_download.py Direct download certificates additions and improvements (#6104) 2022-04-11 22:57:23 -03:00
test_disk_offerings.py kvm: volume encryption feature (#6522) 2022-09-27 10:20:59 +05:30
test_disk_provisioning_types.py Added disk provisioning type support for VMWare (#4640) 2021-07-16 22:37:42 -03:00
test_domain_disk_offerings.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_domain_network_offerings.py tests: Fix test failures for Local storage and Basic zones (#5106) 2021-07-01 09:45:21 +05:30
test_domain_service_offerings.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_domain_vpc_offerings.py tests: Fix test failures for Local storage and Basic zones (#5106) 2021-07-01 09:45:21 +05:30
test_dynamicroles.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_enable_account_settings_for_domain.py Enable account settings to be visible under domain settings (#4215) 2021-09-29 10:29:20 +02:00
test_enable_role_based_users_in_projects.py Enable Flake8 rule W391 (#6606) 2022-11-13 11:00:07 +01:00
test_events_resource.py VM.CREATE/VOLUME.DELETE/VOLUME.DESTROY not being emitted (#7760) 2023-08-07 10:18:17 +05:30
test_gateway_on_shared_networks.py Gateways after Nic update on Shared Network tests (#6355) 2022-05-05 19:53:31 -03:00
test_global_settings.py Improve global settings UI to be more intuitive/logical (#5797) 2023-01-31 11:23:43 +01:00
test_guest_vlan_range.py Add Python flake8 linting for W291 trailing whitespace with Super-Linter (#4687) 2022-03-28 11:40:26 -03:00
test_host_control_state.py server: add Host Control Plane State to uservm and systemvm response (#6946) 2023-01-05 09:59:28 +01:00
test_host_maintenance.py packaging: Adding Centos8, Ubuntu 20.04, XCPNG8.1 Support (#4068) 2020-08-17 16:28:30 +05:30
test_host_ping.py test: fix test_host_ping.py to restore original host state (#8380) 2023-12-20 11:52:08 +05:30
test_hostha_kvm.py Adding support for RHEL8 binary-compatible variants (#5158) 2021-08-18 10:03:03 +02:00
test_hostha_simulator.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_human_readable_logs.py Changed test failure to warning (#4264) 2020-08-25 15:29:59 +05:30
test_internal_lb.py VMware: add support for 8.0b (8.0.0.2), 8.0c (8.0.0.3) (#7380) 2023-08-17 10:42:42 +02:00
test_ipv6_infra.py test: add, refactor ipv6 network, vpc tests (#6338) 2022-07-12 12:54:53 +05:30
test_iso.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_kubernetes_clusters.py CKS: create HA cluster with 3 control VMs instead 2 (#8297) 2023-12-09 11:33:05 +05:30
test_kubernetes_supported_versions.py test: upgrade kubernetes versions to 1.25.0/1.26.0 (#7685) 2023-08-10 09:48:08 +02:00
test_list_ids_parameter.py xen: Fix volume snapshot deletion when it has child snapshots (#6296) 2022-04-22 14:36:08 -03:00
test_loadbalance.py Add Python flake8 linting for W291 trailing whitespace with Super-Linter (#4687) 2022-03-28 11:40:26 -03:00
test_login.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_metrics_api.py ui,server,api: resource metrics improvements (#6803) 2023-01-30 09:48:03 +01:00
test_migration.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_multipleips_per_nic.py CLOUDSTACK-10193: Fix smoke tests failures with new systemvmtemplate 2017-12-23 09:22:44 +05:30
test_nested_virtualization.py Add Python flake8 linting for W291 trailing whitespace with Super-Linter (#4687) 2022-03-28 11:40:26 -03:00
test_network.py Configurable MTU for VR (#6426) 2023-01-04 09:42:24 +01:00
test_network_acl.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_network_ipv6.py test: use normal user for createing networks, vpcs (#6988) 2022-12-19 11:19:56 +01:00
test_network_permissions.py Enable Flake8 rule W391 (#6606) 2022-11-13 11:00:07 +01:00
test_nic.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_nic_adapter_type.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_non_contigiousvlan.py Enable Flake8 rule W391 (#6606) 2022-11-13 11:00:07 +01:00
test_nonstrict_affinity_group.py new plugins: Add non-strict affinity groups (#6845) 2022-12-20 15:09:52 +01:00
test_outofbandmanagement.py test: Frix travis failure - test_outofbandmanagement.py (#5346) 2021-08-20 13:00:34 +02:00
test_outofbandmanagement_nestedplugin.py Use lowercase HTTP header field names so we are compatible with HTTP/2 (#7006) 2023-01-23 11:17:54 +01:00
test_over_provisioning.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_password_server.py Add Python flake8 linting for W291 trailing whitespace with Super-Linter (#4687) 2022-03-28 11:40:26 -03:00
test_persistent_network.py Enable flake8 rule W292 No newline at end of file (#6274) 2022-06-30 12:08:27 +05:30
test_portable_publicip.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_portforwardingrules.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_primary_storage.py utils,framework/db: Introduce new database encryption cipher based on AesGcmJce (#7003) 2023-02-02 16:25:49 +01:00
test_privategw_acl.py cloudstack: make code more inclusive 2021-06-08 15:47:20 +05:30
test_privategw_acl_ovs_gre.py OVS/GRE: bug fixes (#5446) 2021-10-03 14:47:52 +05:30
test_projects.py Fix spelling (#7087) 2023-01-16 10:56:07 +01:00
test_public_ip_range.py server: fix cannot get systemvm ips in dedicated ranges (#7144) 2023-08-25 11:36:39 +02:00
test_pvlan.py Fix spelling (#6860) 2022-11-13 10:56:15 +01:00
test_regions.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_register_userdata.py test: add smoke test for user role for userdata crud api (#7332) 2023-03-31 15:16:29 +05:30
test_reset_configuration_settings.py Fix spelling (#6272) 2022-07-05 09:08:53 +02:00
test_reset_vm_on_reboot.py Add Python flake8 linting for W291 trailing whitespace with Super-Linter (#4687) 2022-03-28 11:40:26 -03:00
test_resource_accounting.py CLOUDSTACK-3009: Fix resource calculation CPU, RAM for accounts. (#3012) 2018-11-13 06:29:08 +05:30
test_resource_detail.py Add Python flake8 linting for W291 trailing whitespace with Super-Linter (#4687) 2022-03-28 11:40:26 -03:00
test_router_dhcphosts.py cleanup of unused code and cleanup of cleanup procedure (#5562) 2021-12-23 10:10:38 +05:30
test_router_dns.py CLOUDSTACK-10193: Fix smoke tests failures with new systemvmtemplate 2017-12-23 09:22:44 +05:30
test_router_dnsservice.py Add Python flake8 linting for W291 trailing whitespace with Super-Linter (#4687) 2022-03-28 11:40:26 -03:00
test_routers.py server: Use ACPI event to reboot VM on KVM, and Use 'forced' reboot option to stop and start the VM(s) (#4681) 2021-03-06 14:58:56 +05:30
test_routers_iptables_default_policy.py Add Python flake8 linting for W291 trailing whitespace with Super-Linter (#4687) 2022-03-28 11:40:26 -03:00
test_routers_network_ops.py Add Python flake8 linting for W291 trailing whitespace with Super-Linter (#4687) 2022-03-28 11:40:26 -03:00
test_scale_vm.py server: fix volume migration on user vm scale (#6704) 2022-11-30 12:58:48 +05:30
test_secondary_storage.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_service_offerings.py kvm: volume encryption feature (#6522) 2022-09-27 10:20:59 +05:30
test_snapshots.py test: add test for standalone snapshot (#8104) 2023-10-19 09:48:16 +05:30
test_ssvm.py Add Python flake8 linting for W291 trailing whitespace with Super-Linter (#4687) 2022-03-28 11:40:26 -03:00
test_staticroles.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_storage_policy.py network: fix vm can be deployed on L2 network of other accounts (#5784) 2022-01-11 12:16:00 +05:30
test_templates.py marvin,test: fix directdownload template checksum test (#8096) 2023-10-19 19:40:03 +02:00
test_update_security_group.py Fix spelling (#6597) 2022-08-03 15:43:47 +05:30
test_usage.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_usage_events.py python3: Migrate Marvin and smoketests to python3 (#4727) 2021-05-04 23:19:37 +05:30
test_vm_autoscaling.py AutoScaling: support Managed User Data (#7769) 2023-08-22 11:07:16 +02:00
test_vm_deployment_planner.py smoke test: fix test_vm_deployment_planner (#7183) 2023-02-09 15:19:57 +01:00
test_vm_life_cycle.py test: additional check to ensure hosts are left in up state (#8383) 2023-12-21 20:54:43 +05:30
test_vm_lifecycle_unmanage_import.py test: fix failure on non-VMware env (#6969) 2022-12-12 10:32:29 +05:30
test_vm_snapshot_kvm.py .github/linters: Enable flake8 W293 blank line contains whitespace (#6268) 2022-04-15 20:32:52 +05:30
test_vm_snapshots.py Fix spelling (#6609) 2022-11-01 08:30:01 +01:00
test_volumes.py kvm: volume encryption feature (#6522) 2022-09-27 10:20:59 +05:30
test_vpc_ipv6.py test: use normal user for createing networks, vpcs (#6988) 2022-12-19 11:19:56 +01:00
test_vpc_redundant.py Add Python flake8 linting for W291 trailing whitespace with Super-Linter (#4687) 2022-03-28 11:40:26 -03:00
test_vpc_router_nics.py Add Python flake8 linting for W291 trailing whitespace with Super-Linter (#4687) 2022-03-28 11:40:26 -03:00
test_vpc_vpn.py Refactor test and change IP range (#6512) 2023-06-05 16:39:33 +02:00