Commit Graph

36795 Commits

Author SHA1 Message Date
Wei Zhou 56d98ea2e7
SSVM: 'allow from' private IP in other SSVMs if the public IP is in allowed internal sites cidrs (#7288)
Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-07-20 13:38:05 +02:00
dahn 9c5e489d82
eof added to StorPoolStatsCollector (#7754) 2023-07-19 12:36:44 +02:00
Abhishek Kumar 729e6d1446
userdata: fix append scenarios (#7741)
Fixes case of appending userdata when both template and vm data are either shellscript or cloudconfig
Fixes error when appending gzip userdata
Fixes case when userdata manual text from VM is not getting decoded-encoded correctly.
Fixes case of appending multipart data when both template and vm data contain same format types.
Refactor - moved validateUserData method to UserDataManager class
Refactor userdata test to check resultant multipart userdata thoroughly

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-19 15:18:53 +05:30
Daan Hoogland 6bb95c0200 Merge release branch 4.18 to main
* 4.18:
  Storage and volumes statistics tasks for StorPool primary storage (#7404)
  proper storage construction (#6797)
  guarantee MAC uniqueness (#7634)
  server: allow migration of all VMs with local storage on KVM (#7656)
  Add L2 networks to Zones with SG (#7719)
2023-07-19 10:59:19 +02:00
slavkap f32a63be60
Storage and volumes statistics tasks for StorPool primary storage (#7404) 2023-07-19 10:48:36 +02:00
dahn 0aade286f5
proper storage construction (#6797) 2023-07-19 10:27:20 +02:00
dahn 73a269e3b3
guarantee MAC uniqueness (#7634)
Co-authored-by: Bryan Lima <42067040+BryanMLima@users.noreply.github.com>
2023-07-19 10:25:01 +02:00
Vishesh b5eebc4347
Fix codecov github action (#7747) 2023-07-18 13:27:01 +03:00
dahn d575cae3e3
server: allow migration of all VMs with local storage on KVM (#7656)
This PR enables systemvm and VR migration on KVM with local storage.

Fixes: #7651
2023-07-18 12:49:17 +05:30
Abhishek Kumar caaf25b695
test: fix ci failures on main branch (#7733)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-12 09:19:28 +02:00
Ben 981d74825a
Add L2 networks to Zones with SG (#7719) 2023-07-11 09:11:23 +02:00
Nicolas Vazquez b1fc279872
Generate cloud-init multipart user data for template append policy (#7643)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-10 09:47:03 +02:00
Rohit Yadav 5383bf64f4 Merge remote-tracking branch 'origin/4.18'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-07-07 23:04:44 +05:30
Abhishek Kumar f0cc76a3a8
vmware: fix unmanaged instance listing when different name & internal CS name (#7641)
Currently, ACS can continue to show an imported instance/VM as an unmanaged instance if the name and internalCSName (custom attribute, cloud.vm.internal.name) is different for the instance/VM on vCenter. This PR while filtering managed instances from the instance list received from ESXi host also checks if the internal name for the instance is not in the managed instance names list.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-07 23:00:25 +05:30
dahn 2752c49fa7
agent: get the right controll cidr (#7580)
Fixes: #7574
2023-07-07 22:57:58 +05:30
Vishesh c956d329f5
Fix test_vm_schedule's intermittent failures (#7726)
* Fix test_vm_schedule's intermittent failures

* Add smoke/test_vm_schedule to CI
2023-07-07 22:51:38 +05:30
Vishesh d2cd3c81cb
Fix lint errors (#7727) 2023-07-07 21:02:42 +05:30
Daan Hoogland a706bf2380 Merge branch '4.18' 2023-07-07 11:49:35 +02:00
Vishesh 594c70dde0
Sync precommit config from main (#7732)
Co-authored-by: John Bampton <jbampton@users.noreply.github.com>
Co-authored-by: dahn <daan@onecht.net>
2023-07-07 11:18:16 +02:00
Daan Hoogland 2132f46fcb Merge branch '4.18' 2023-07-06 11:24:08 +02:00
Abhishek Kumar 939ee9e153
server,engine-orchestration: allocate vm without transaction (#7695)
When deploying a VM is failed during the allocation process it may leave the resources that have been already allocated before the failure. They will get removed from the database as the whole code block is wrapped inside a transaction twice but the server would not inform the network or storage plugins to clean up the allocated resources.
This PR removes Transactions during VM allocation which results in the allocated VM and its resource records being persisted in DB even during failures. When failure is encountered VM is moved to Error state. This helps VM and its resources to be properly deallocated when it is expunged either by a server task such as ExpungeTask or during manual expunge.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-06 14:04:38 +05:30
Nicolas Vazquez c733a23c90
Fix direct download URL checks (#7693)
This PR fixes the URL check for direct downloads, in the case of HTTPS URLs the certificates were not loaded into the SSL context
2023-07-06 13:47:13 +05:30
dahn e6ef8a5225
use `Files.createTempDirectory()` instead of `new File()` (#7713) 2023-07-06 09:33:51 +02:00
dahn acc6f4e725
NPE guard (#7691) 2023-07-06 09:29:42 +02:00
Harikrishna 0cbe77024a
Fix foreign key constraints and the mysql procedure that is used (#7381) 2023-07-06 08:43:49 +02:00
Wei Zhou d66f83ba2f
UI: fix wrong vpcid when create network in VPC (#7707) 2023-07-05 13:27:59 +02:00
Daan Hoogland 02be2cdf67 Merge release branch 4.18 to main
* 4.18:
  cleanup removal of physical network (#7630)
  console proxy: use AeadBase64Encryptor instead of AES/CBC/PKCS5Padding (#7237)
2023-07-05 11:52:23 +02:00
dahn 8e171785d1
cleanup removal of physical network (#7630)
Co-authored-by: Bryan Lima <42067040+BryanMLima@users.noreply.github.com>
2023-07-05 11:12:46 +02:00
Wei Zhou 3e04779f60
console proxy: use AeadBase64Encryptor instead of AES/CBC/PKCS5Padding (#7237) 2023-07-05 11:01:32 +02:00
Daan Hoogland fbb9dec6e5 Merge release branch 4.18 to main
* 4.18:
  server: do not check zone imagestores for directdownload template delete (#7607)
  UI: Zone wizard fix (#7588)
2023-07-04 10:17:43 +02:00
Abhishek Kumar 5e5d194d77
server: do not check zone imagestores for directdownload template delete (#7607)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-04 09:37:06 +02:00
mprokopchuk 70d5470f48
If ResourceCountCheckTask throws an exception the scheduled task is not going to run again until the management servers are restarted. (#7670)
Co-authored-by: Maxim Prokopchuk <mprokopchuk@apple.com>
2023-07-04 08:45:15 +02:00
SadiJr 3c5fdeafdc
[Veeam] restored VMs without NICs (#6282) 2023-07-03 14:35:48 +02:00
dahn 1275db4081
UI: Zone wizard fix (#7588)
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-03 11:56:52 +02:00
Rohit Yadav 3b054b2665 Merge remote-tracking branch 'origin/4.18' 2023-07-03 13:17:25 +05:30
Vishesh 2fcbe6241f
cks: Add unmanaged kubernetes cluster (#7515)
There are tools like cluster-api which create and manage kubernetes cluster on CloudStack. This PR adds the option to add unmanaged kubernetes cluster which are not managed by CKS plugin. This helps provide a consolidated view of unmanaged clusters on CloudStack. The changes done make sure that operations for managed clusters are not executed for unmanaged clusters.

Two new APIs have also been added:

1. addVirtualMachinesToKubernetesCluster - to add VMs to unmanaged clusters.
2. removeVirtualMachinesFromKubernetesCluster - to remove VMs to unmanaged clusters.

Two APIs have been updated:

1. createKubernetesCluster - made KUBERNETES_VERSION_ID, SERVICE_OFFERING_ID, SIZE as not required for unmanaged clusters. Add an additional parameter, managed, which is true by default.
2. listKubernetesClusters - Add a parameter managed to filter on managed field.

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-07-03 13:07:33 +05:30
Abhishek Kumar 31dbdd0f5c
engine-orchestration: fix volume size resource count mismatch (#7666)
Fixes case of account/domain having negative storage count when there is a volume size difference while deploying volumes on certain stroages. In case of Powerflex volume size results in a multiple of 8. If user deploys a volume of 12GB it will result in 16GB in size. But currently CloudStack will not update resource count after deployment.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-03 13:00:32 +05:30
Harikrishna 70820137e6
scaleio: Avoid race condition while handling host disconnect and connect scenarios (#282) (#7689)
This PR fixes an intermittent issue where SDC id (local_path) is getting deleted and not getting populated when host connects back again.

Fix is to remove the code to delete the records from storage_pool_host_ref table. We are anyways updating the entry if the SDC ID is changed during agent restart which is anyways required inorder to get the new connections. I've quickly verified the host delete scenario to check the storage_pool_host_ref entries behavior, entries are getting deleted.
2023-07-03 12:58:42 +05:30
Abhishek Kumar de6ce503dc
api: correct error on resize volume resource allocation failure (#7687)
This PR resource throws exception with the correct error code and logs the error message when a resource allocation failure is encountered during resize volume operation.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-03 12:57:14 +05:30
Daniel Augusto Veronezi Salvador c6237c48ac
Retrieve only resource count on user dashboard (#7617)
* Allow retrieving only the count of resources on APIs listPublicIpAddresses, listNetworks, listVirtualMachines and listVolumes

* Use parameter to retrieve only the count of resources in the dashboard

* Create abstract class
2023-06-30 11:09:55 -03:00
Rohit Yadav 5975ea80c9 Merge remote-tracking branch 'origin/4.18' 2023-06-27 15:51:40 +05:30
Wei Zhou f3ad9e6743
marvin: fix smoke test failures if mgmt server is Ubuntu (#7583) 2023-06-27 11:57:02 +02:00
Wei Zhou 41403c9cf9 Merge remote-tracking branch 'apache/4.18' 2023-06-27 11:52:37 +02:00
Abhishek Kumar 83dca2bf51
ui: fix vm import for L2 n/w in Setup state (#7628)
L2 networks created using offerings that allow using custom VLAN can remain in Setup state. Allow importing of VMs with such networks when VLAN matches with unmanaged instance.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-27 14:57:42 +05:30
Vishesh c3718ab92e
Fix storpool test & remove powermock (#7652)
This PR fixes unit tests for plugins/storage/volume/storpool/ and removes powermock.
2023-06-27 14:55:19 +05:30
Vishesh 908b3b7dfa
Remove powermock from ldap & project-role-based plugin (#7658)
This PR removes powermock from tests & fixes the path for tests of project-role-based plugin.
2023-06-27 14:53:00 +05:30
Wei Zhou 973b0e28fd test: fix Super-Linter Check error in storpool tests
2023-06-27 08:42:08 [INFO]   File:[/github/workspace/test/integration/plugins/storpool/TestEncryptedVolumes.py]
2023-06-27 08:42:08 [ERROR]   Found errors in [flake8] linter!
2023-06-27 08:42:08 [ERROR]   Error code: 1. Command output:
------
/github/workspace/test/integration/plugins/storpool/TestEncryptedVolumes.py:681:113: W292 no newline at end of file
------
2023-06-27 08:42:08 [INFO]   ---------------------------
2023-06-27 08:42:08 [INFO]   File:[/github/workspace/test/integration/plugins/storpool/sp_util.py]
2023-06-27 08:42:08 [ERROR]   Found errors in [flake8] linter!
2023-06-27 08:42:08 [ERROR]   Error code: 1. Command output:
------
/github/workspace/test/integration/plugins/storpool/sp_util.py:798:31: W292 no newline at end of file
------
2023-06-27 10:47:25 +02:00
dahn 06caf32bc8
Guest os mappings improvements (#6979)
* Guest OS mapping improvements
- Checks the OS mapping name in hypervisor (VMware, XenServer)
- Displays guest OS mappings in UI

* Added API getHypervisorGuestOsNames to list the guest OS names in the hypervisor, and code improvements

* Some static analysis fixes

* Removed commented code in listview

* Guest OS list

* UI changes for adding guest os and mappings

* Added guest os mappings in guest os form

* Added new filter to guest os mapping

* Name and description changes

* VMWare Host and cluster MO unit tests

* CheckGuestOsMapping command and answer unit tests

* GetHypervisorGuestOsNames command and answer unit tests

* VmwareResource unitests

* GuestOsMapper unittests

* icon changes

* Addressed review comments

* Renaming fixes

* Removed comments

* marvin tests for guest os operations

* Added marvin tests for OS mappings

* Document links and UI improvements

* Added deduplication for the list guest OS API

* Fixed linter failure

* Few bug fixes and UI changes

* Few improvements

* Addressed code smells

* Fixed UI issues after rebase

---------

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
Co-authored-by: Harikrishna Patnala <harikrishna.patnala@gmail.com>
2023-06-27 13:49:20 +05:30
Wei Zhou 985f0ecb53
Tungsten: change conserve_mode of default network offering to 0 (#7511) 2023-06-26 13:36:36 +02:00
slavkap faaf72b1a4
Volume encryption support for StorPool plug-in (#7539)
Supported Virtual machine operations:
- live migration of VM to another host
- virtual machine snapshots (group snapshot without memory)
- revert VM snapshot
- delete VM snapshot
Supported Volume operations:
- attach/detach volume
- live migrate volume between two StorPool primary storages
- volume snapshot
- delete snapshot
- revert snapshot
2023-06-26 11:24:51 +02:00