Commit Graph

35877 Commits

Author SHA1 Message Date
slavkap f32a63be60
Storage and volumes statistics tasks for StorPool primary storage (#7404) 2023-07-19 10:48:36 +02:00
dahn 0aade286f5
proper storage construction (#6797) 2023-07-19 10:27:20 +02:00
dahn 73a269e3b3
guarantee MAC uniqueness (#7634)
Co-authored-by: Bryan Lima <42067040+BryanMLima@users.noreply.github.com>
2023-07-19 10:25:01 +02:00
dahn d575cae3e3
server: allow migration of all VMs with local storage on KVM (#7656)
This PR enables systemvm and VR migration on KVM with local storage.

Fixes: #7651
2023-07-18 12:49:17 +05:30
Ben 981d74825a
Add L2 networks to Zones with SG (#7719) 2023-07-11 09:11:23 +02:00
Abhishek Kumar f0cc76a3a8
vmware: fix unmanaged instance listing when different name & internal CS name (#7641)
Currently, ACS can continue to show an imported instance/VM as an unmanaged instance if the name and internalCSName (custom attribute, cloud.vm.internal.name) is different for the instance/VM on vCenter. This PR while filtering managed instances from the instance list received from ESXi host also checks if the internal name for the instance is not in the managed instance names list.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-07 23:00:25 +05:30
dahn 2752c49fa7
agent: get the right controll cidr (#7580)
Fixes: #7574
2023-07-07 22:57:58 +05:30
Vishesh 594c70dde0
Sync precommit config from main (#7732)
Co-authored-by: John Bampton <jbampton@users.noreply.github.com>
Co-authored-by: dahn <daan@onecht.net>
2023-07-07 11:18:16 +02:00
Abhishek Kumar 939ee9e153
server,engine-orchestration: allocate vm without transaction (#7695)
When deploying a VM is failed during the allocation process it may leave the resources that have been already allocated before the failure. They will get removed from the database as the whole code block is wrapped inside a transaction twice but the server would not inform the network or storage plugins to clean up the allocated resources.
This PR removes Transactions during VM allocation which results in the allocated VM and its resource records being persisted in DB even during failures. When failure is encountered VM is moved to Error state. This helps VM and its resources to be properly deallocated when it is expunged either by a server task such as ExpungeTask or during manual expunge.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-06 14:04:38 +05:30
Nicolas Vazquez c733a23c90
Fix direct download URL checks (#7693)
This PR fixes the URL check for direct downloads, in the case of HTTPS URLs the certificates were not loaded into the SSL context
2023-07-06 13:47:13 +05:30
dahn e6ef8a5225
use `Files.createTempDirectory()` instead of `new File()` (#7713) 2023-07-06 09:33:51 +02:00
dahn acc6f4e725
NPE guard (#7691) 2023-07-06 09:29:42 +02:00
Harikrishna 0cbe77024a
Fix foreign key constraints and the mysql procedure that is used (#7381) 2023-07-06 08:43:49 +02:00
Wei Zhou d66f83ba2f
UI: fix wrong vpcid when create network in VPC (#7707) 2023-07-05 13:27:59 +02:00
dahn 8e171785d1
cleanup removal of physical network (#7630)
Co-authored-by: Bryan Lima <42067040+BryanMLima@users.noreply.github.com>
2023-07-05 11:12:46 +02:00
Wei Zhou 3e04779f60
console proxy: use AeadBase64Encryptor instead of AES/CBC/PKCS5Padding (#7237) 2023-07-05 11:01:32 +02:00
Abhishek Kumar 5e5d194d77
server: do not check zone imagestores for directdownload template delete (#7607)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-04 09:37:06 +02:00
dahn 1275db4081
UI: Zone wizard fix (#7588)
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-03 11:56:52 +02:00
Abhishek Kumar 31dbdd0f5c
engine-orchestration: fix volume size resource count mismatch (#7666)
Fixes case of account/domain having negative storage count when there is a volume size difference while deploying volumes on certain stroages. In case of Powerflex volume size results in a multiple of 8. If user deploys a volume of 12GB it will result in 16GB in size. But currently CloudStack will not update resource count after deployment.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-03 13:00:32 +05:30
Harikrishna 70820137e6
scaleio: Avoid race condition while handling host disconnect and connect scenarios (#282) (#7689)
This PR fixes an intermittent issue where SDC id (local_path) is getting deleted and not getting populated when host connects back again.

Fix is to remove the code to delete the records from storage_pool_host_ref table. We are anyways updating the entry if the SDC ID is changed during agent restart which is anyways required inorder to get the new connections. I've quickly verified the host delete scenario to check the storage_pool_host_ref entries behavior, entries are getting deleted.
2023-07-03 12:58:42 +05:30
Abhishek Kumar de6ce503dc
api: correct error on resize volume resource allocation failure (#7687)
This PR resource throws exception with the correct error code and logs the error message when a resource allocation failure is encountered during resize volume operation.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-03 12:57:14 +05:30
Wei Zhou f3ad9e6743
marvin: fix smoke test failures if mgmt server is Ubuntu (#7583) 2023-06-27 11:57:02 +02:00
Abhishek Kumar 83dca2bf51
ui: fix vm import for L2 n/w in Setup state (#7628)
L2 networks created using offerings that allow using custom VLAN can remain in Setup state. Allow importing of VMs with such networks when VLAN matches with unmanaged instance.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-27 14:57:42 +05:30
Wei Zhou 973b0e28fd test: fix Super-Linter Check error in storpool tests
2023-06-27 08:42:08 [INFO]   File:[/github/workspace/test/integration/plugins/storpool/TestEncryptedVolumes.py]
2023-06-27 08:42:08 [ERROR]   Found errors in [flake8] linter!
2023-06-27 08:42:08 [ERROR]   Error code: 1. Command output:
------
/github/workspace/test/integration/plugins/storpool/TestEncryptedVolumes.py:681:113: W292 no newline at end of file
------
2023-06-27 08:42:08 [INFO]   ---------------------------
2023-06-27 08:42:08 [INFO]   File:[/github/workspace/test/integration/plugins/storpool/sp_util.py]
2023-06-27 08:42:08 [ERROR]   Found errors in [flake8] linter!
2023-06-27 08:42:08 [ERROR]   Error code: 1. Command output:
------
/github/workspace/test/integration/plugins/storpool/sp_util.py:798:31: W292 no newline at end of file
------
2023-06-27 10:47:25 +02:00
Wei Zhou 985f0ecb53
Tungsten: change conserve_mode of default network offering to 0 (#7511) 2023-06-26 13:36:36 +02:00
slavkap faaf72b1a4
Volume encryption support for StorPool plug-in (#7539)
Supported Virtual machine operations:
- live migration of VM to another host
- virtual machine snapshots (group snapshot without memory)
- revert VM snapshot
- delete VM snapshot
Supported Volume operations:
- attach/detach volume
- live migrate volume between two StorPool primary storages
- volume snapshot
- delete snapshot
- revert snapshot
2023-06-26 11:24:51 +02:00
Nicolas Vazquez c809201247
Fix: Volumes on lost local storage cannot be removed (#7594) 2023-06-23 12:22:15 +02:00
Vishesh 0acc66f51d
server: Add check on host's status while deleting config drive on host cache (#7584)
This PR adds a check on host's status. Without this if the agent is not in Up or Connecting state, expunging of a VM fails.

Steps to reproduce:
- Enable vm.configdrive.force.host.cache.use in Global Configuration.
- Create a L2 network with config drive
- Deploy a vm with the L2 network created in previous step
- Stop the vm and destroy vm (not expunge it)
- Stop the cloudstack-agent on the VM's host
- Expunge the vm

Fixes: #7428
2023-06-23 13:46:22 +05:30
Wei Zhou 9e1609dd5a
[4.18] Bump the version to 4.18.1.0-SNAPSHOT (#7664) 2023-06-22 06:59:08 +02:00
Harikrishna 40cc10a73d
Allow volume migrations in ScaleIO within and across ScaleIO storage clusters (#7408)
* Live storage migration of volume in scaleIO within same storage scaleio cluster

* Added migrate command

* Recent changes of migration across clusters

* Fixed uuid

* recent changes

* Pivot changes

* working blockcopy api in libvirt

* Checking block copy status

* Formatting code

* Fixed failures

* code refactoring and some changes

* Removed unused methods

* removed unused imports

* Unit tests to check if volume belongs to same or different storage scaleio cluster

* Unit tests for volume livemigration in ScaleIOPrimaryDataStoreDriver

* Fixed offline volume migration case and allowed encrypted volume migration

* Added more integration tests

* Support for migration of encrypted volumes across different scaleio clusters

* Fix UI notifications for migrate volume

* Data volume offline migration: save encryption details to destination volume entry

* Offline storage migration for scaleio encrypted volumes

* Allow multiple Volumes to be migrated with migrateVirtualMachineWithVolume API

* Removed unused unittests

* Removed duplicate keys in migrate volume vue file

* Fix Unit tests

* Add volume secrets if does not exists during volume migrations. secrets are getting cleared on package upgrades.

* Fix secret UUID for encrypted volume migration

* Added a null check for secret before removing

* Added more unit tests

* Fixed passphrase check

* Add image options to the encypted volume conversion
2023-06-21 11:57:05 +05:30
Abhishek Kumar 3748f32bc7
engine-orchestration,vmware: hypervisor migration during start vm migration (#7444)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-20 17:13:08 +02:00
Abhishek Kumar 658daef715
utils: fix check for mrtalink url (#7636)
Fixes incorrect condition while checking metalink URLs for accesibility
2023-06-19 12:23:22 +05:30
dahn 1aa4f80741
accept first word from host os string for backwards compatibility (#7620) 2023-06-15 10:09:31 +03:00
dahn fb29608f61
consider last host only for root-admin (#7597) 2023-06-14 09:35:55 +02:00
Abhishek Kumar 4ef7ebbded
ssvm: pass all accessible secondary storage to ssvm (#7410)
* ssvm: pass all accessible secondary storage to ssvm

Fixes #7162

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* license

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-12 09:02:42 +02:00
dahn 0d3ac9f8d9
interpret /etc/redhet-release better (#7570) 2023-06-09 10:29:42 +02:00
Abhishek Kumar 03910c2783 Merge remote-tracking branch 'apache/4.17' into 4.18 2023-06-08 15:34:03 +05:30
Abhishek Kumar 20865901d1 Merge remote-tracking branch 'apache/4.16' into 4.17 2023-06-08 15:33:20 +05:30
Abhishek Kumar 24da2bcf78
marvin: fix test_data for parsing failure (#7602)
PR #6512 seems to have broken the test_data.py. This PR fixes test_data.
2023-06-08 15:32:04 +05:30
Vishesh 811b6293af
.github: Use parallel builds for github actions (#7592) 2023-06-07 13:17:29 +02:00
Abhishek Kumar b030ea19a4 Merge remote-tracking branch 'apache/4.17' into 4.18 2023-06-06 13:12:12 +05:30
Abhishek Kumar f298008215 Merge remote-tracking branch 'apache/4.16' into 4.17 2023-06-06 13:11:24 +05:30
Pearl Dsilva bd5d478a3a
Refactor test and change IP range (#6512)
* Refactor test and change IP range

* countdown from 254 to allow multiple pseudo public nets counting up from 0

* cleanup

* location of asserts improved

---------

Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: Daan Hoogland <daan@onecht.net>
2023-06-05 16:39:33 +02:00
Abhishek Kumar 2d6a069812
server: improve storage GC to skip expunging possible duplicate volumes (#7313)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-05 10:03:24 +02:00
Wei Zhou b2e9993b0a
vmware: do not create vm datastore when attach an existing volume on solidfire (#7160) 2023-06-01 14:26:57 +02:00
Abhishek Kumar 0941d01e97
secondary-storage: delete backedup snapshot dir on delete (#7524)
* secondary-storage: delete backedup snapshot dir on delete

Fixes #7516

When a backed-up snapshot is deleted and the snapshot file is present in the same name directory (probably only for VMware), the whole directory can be deleted.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* resolve comment

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-01 15:48:28 +05:30
Abhishek Kumar 7319debc87
cks: k8s cluster on vpc tier (#7479)
* cks: cluster on vpc tier

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wip

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* python fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix trailing space

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-01 15:29:06 +05:30
Abhishek Kumar ef9dec11e7
guestos: add Windows Server 2022 (#7508)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-31 16:02:48 +02:00
Abhishek Kumar 17e765e6bc
test: reduce PowerMock usage for code coverage (#7452)
JaCoCo used for code coverage calculation in the project doesn't support PowerMockito classes.
This PR attemps to reduce usage of PowerMockito.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-30 11:43:01 +02:00
Abhishek Kumar f636580195
cks,ui: allow changing stopped cluster offering, improvements (#7475)
* cks,ui: allow changing stopped cluster offering, improvements

Fixes #7454

- Allows changing compute offering for a stopped cluster
- Allows compute offering change when the cluster has autoscaling enabled

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-29 15:58:06 +05:30