Commit Graph

36105 Commits

Author SHA1 Message Date
Rohit Yadav 93bd5b7a69
ui: speed up compute instance listing (#7911)
* ui: speed up compute instance listing

The default listVirtualMachinesMetrics APIs assumings details=all which
isn't really used in the list view. By changing this to a subset of
details, we can see gains upto 10x in listing speed in the UI. When
moving to the resource/detail view of the UI, don't pass state or
details so `all` the details of the instance are returned.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* Update AutogenView.vue

---------

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-08-25 13:57:44 +02:00
Rohit Yadav 6f7725ab35
ui: Fix refresh and re-route behaviour (#7846)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-08-25 13:45:35 +02:00
Wei Zhou b19db10ea6
server: try with all supported hypervisors (ordered, no duplications) when deploy a virtual router (#7877) 2023-08-25 11:49:03 +02:00
Wei Zhou 8dc5fdd067
server: fix cannot get systemvm ips in dedicated ranges (#7144)
This fixes #6698
2023-08-25 11:36:39 +02:00
Wei Zhou f5a1f4130d
server: fix global setting system.vm.public.ip.reservation.mode.strictness is not really dynamic (#7909)
If the original value is `false`, and search build is configured without the condition. Now change the value to `true`, it will not get effective due to missing condition.
2023-08-25 11:35:31 +02:00
Rohit Yadav e964395bd4
vmware: improve solidfire storage plugin integration and fix cases (#3) (#7761)
This fixes the following cases in which Solidfire storage integration
caused issues when using Solidfire datadisks with VMware:

1. Take Volume Snapshot of Solidfire data disk
2. Delete an active Instance with Solidfire data disk attached
3. Attach used existing Solidfire data disk to a running/stopped VM
4. Stop and Start an instance with Solidfire data disks attached
5. Expand disk by resizing Solidfire data disk by providing size
6. Expand disk by changing disk offering for the Solidfire data disk

Additional changes:
- Use VMFS6 as managed datastore type if the host supports
- Refactor detection and splitting of managed storage ds name in storage
  processor
- Restrict storage rescanning for managed datastore when resizing

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-08-25 11:36:03 +05:30
Wei Zhou 3c38ed7a65
server: allow user to list available IPs on shared networks (#7898)
This fixes #7817
2023-08-24 21:42:01 +05:30
Abhishek Kumar 8ad1009ad2
ui: fix notification list reordering intermittently (#7609)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-08-24 11:29:25 +02:00
Marcus Sorensen a0702279aa
server Don't allow inadvertent deletion of hidden details via API (#7880)
* Don't allow inadvertent deletion of hidden details via API

* Update VM details unit test ensuring system/hidden details not removed

* Update test/integration/component/test_update_vm.py

---------

Co-authored-by: Marcus Sorensen <mls@apple.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-08-24 10:45:39 +02:00
Charles Queiroz 2e1c2821a8
UI: Add central project store and watch functionality (#7900)
Added a new getter 'allProjects' and mutation 'RELOAD_ALL_PROJECTS' to centralize the tracking of available projects in the state.
This eliminates direct manipulation of the Project list in separate components and improves data consistency across the application.
A watcher in ProjectMenu.vue has been implemented to handle changes to the 'allProjects' getter.
The 'RELOAD_ALL_PROJECTS' mutation was also added where necessary to ensure projects list is updated in the state whenever changes occur.
2023-08-24 10:13:47 +02:00
Rohit Yadav b37834fe75
ui: bump UI pkg version to match CloudStacks (#7894)
This uses the version in UI's package.json same as CloudStack's. The idea
is that static assets aren't cached per discussion on
https://github.com/apache/cloudstack/issues/7546

Fixes #7546

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-08-23 13:07:07 +02:00
Rohit Yadav d87f39ad27
ui: make it clear that deletion of a volume will not delete any snapshots (#7897)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-08-23 16:07:17 +05:30
Wei Zhou 8cb30551d5
UI: update ip ranges of shared networks (#7896)
* UI: update ip ranges of shared networks

* UI: remove ip6gateway and ip6cidr from GuestIpRanges.vue
2023-08-23 16:06:16 +05:30
Wei Zhou e4117e68d2
engine/schema: fix duplicated guest OSes in 4.18.0.0 (#7799)
Co-authored-by: Daan Hoogland <daan@onecht.net>
2023-08-23 09:44:34 +02:00
Rohit Yadav 0e9a19f24d
ui: Fix project theme on reload or refresh (#7893)
* css fixes from main

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* component: when in project, toggle the theme also

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

---------

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-08-22 14:20:45 +02:00
Wei Zhou db0e7a65af
server: check hostId when attach disk to a Stopped vm with local storage (#7886)
This fixes #7834
2023-08-22 14:52:20 +05:30
Wei Zhou 5097d2aa8b
ui: fix creating zone with vxlan if Guest physical network is not the last (#7801)
This fixes #7790
2023-08-22 14:45:30 +05:30
Wei Zhou 78bdde9e98
AutoScaling: support Managed User Data (#7769) 2023-08-22 11:07:16 +02:00
Nicolas Vazquez 405ef82aef
UI: Fix hide delete button for templates that are in Installing template state (#7882)
This PR hides the Delete Template button when a template is installing

Fixes: #7865
2023-08-22 14:32:54 +05:30
Marcus Sorensen 6cded7e010
Support GNU sed apidoc for MacOS build (#7888)
Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-08-22 14:31:20 +05:30
Fabricio Duarte 6d24217636
server: Allow admins to disable the 2FA of users in subdomains (#7870) 2023-08-21 15:48:33 +02:00
Nicolas Vazquez ddc2a362a8
UI: Add missing tooltips to Register/Upload Templates/ISOs views (#7879) 2023-08-21 11:13:25 +02:00
Sina Kashipazha d296f54c7f
Api: update command domainId/account descriptions (#7876) 2023-08-20 15:44:31 +02:00
João Jandre f7345e861f
cpvm: Fix CPVM not releasing CPU after closing console (#7826) 2023-08-18 11:12:06 +02:00
Sina Kashipazha add64bd7e6
security group: replace vm.getPrivateMacAddress() with nic.getMacAddress() (#7293) 2023-08-18 10:49:16 +02:00
sato03 5b33967310
Fix role escalation prevention (#7853)
Co-authored-by: Henrique Sato <henrique.sato@scclouds.com.br>
2023-08-18 09:33:05 +02:00
Wei Zhou 87d6c4fbd5
packaging: install tzdata-java or timezone-java (#7875)
According to https://pkgs.org/download/tzdata-java
the package name is tzdata-java on some OSes, but timezone-java on some other OSes (e.g openSUSE Leap 15.4)
2023-08-18 09:20:00 +02:00
Wei Zhou d8a5c89060
server: get only Ready ISO to mount (#7848) 2023-08-17 11:04:14 +02:00
Wei Zhou c8d6e50539
VMware: add support for 8.0b (8.0.0.2), 8.0c (8.0.0.3) (#7380)
* VMware: add support for 8.0b (8.0.0.2)

* VMware 8: add new guest os mappings in VirtualMachineGuestOsIdentifier

The full list can be found at https://developer.vmware.com/apis/1355/vsphere

* VMware: get guest os mappings of parent version

* VMware8: remove guest os mappings for 8.0.0.2

* VMware8: fix code smells

* vmware: remove annotations in VmwareVmImplementerTest which caused 0.0% code coverage

* VMware8: add a unit test case

* VMware: add support for 8.0c (8.0.0.3)

* VMware8: move to CloudStackVersion.getVMwareParentVersion

* VMware: add support for 8.0u1 (8.0.1.0)

* Copy engine/schema/src/main/java/com/cloud/upgrade/GuestOsMapper.java from PR 6979

* Copy engine/schema/src/main/java/com/cloud/storage/dao/GuestOSHypervisorDao.java from PR 6979

* VMware: ignore the last number in VMware versions

* VMware: copy guest os mapping from 8.0 to 8.0.1

* VMware: add unit tests in VmwareVmImplementerTest.java

* Copy engine/schema/src/test/java/com/cloud/upgrade/GuestOsMapperTest.java from PR 6979

* VMware8: retry vm poweron if fails due to exception "File system specific implementation of Ioctl[file] failed"

This fixes a weird issue on vmware8. When power on a vm, sometimes it fails due to error

2023-04-27 07:04:43,207 ERROR [c.c.h.v.r.VmwareResource] (DirectAgent-442:ctx-cdd42b03 10.0.32.133, job-105/job-106, cmd: StartCommand) (logid:8a24a607) StartCommand failed due to [Exception: java.lang.RuntimeException
Message: File system specific implementation of Ioctl[file] failed
].
java.lang.RuntimeException: File system specific implementation of Ioctl[file] failed
        at com.cloud.hypervisor.vmware.util.VmwareClient.waitForTask(VmwareClient.java:426)
        at com.cloud.hypervisor.vmware.mo.VirtualMachineMO.powerOn(VirtualMachineMO.java:288)

in vmware.log on ESXi host, it shows

2023-04-27T09:20:41.713Z In(05)+ vmx - Power on failure messages: File system specific implementation of Ioctl[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of Ioctl[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of Ioctl[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of Ioctl[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - Failed to lock the file
2023-04-27T09:20:41.713Z In(05)+ vmx - Cannot open the disk '/vmfs/volumes/7b29c876-ac102328/i-2-167-VM/ROOT-167.vmdk' or one of the snapshot disks it depends on.
2023-04-27T09:20:41.713Z In(05)+ vmx - Module 'Disk' power on failed.
2023-04-27T09:20:41.713Z In(05)+ vmx - Failed to start the virtual machine.

There is a KB article for it, but I still do not know why and how to fix it.
https://kb.vmware.com/s/article/1004232

* VMware: extract to method powerOnVM

* vmware: fix mistake in logs

* vmware8: use curl instead of wget to fix test failures

Traceback (most recent call last):
  File "/root/test_internal_lb.py", line 555, in test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80
    self.execute_internallb_roundrobin_tests(vpc_offering)
  File "/root/test_internal_lb.py", line 641, in execute_internallb_roundrobin_tests
    client_vm, applb.sourceipaddress, max_http_requests)
  File "/root/test_internal_lb.py", line 497, in run_ssh_test_accross_hosts
    (e, clienthost.public_ip))
AssertionError: list index out of range: SSH failed for VM with IP Address: 10.0.52.187

and

sshClient: DEBUG: {Cmd: /usr/bin/wget -T3 -qO- --user=admin --password=password http://10.1.2.253:8081/admin?stats via Host: 10.0.52.188} {returns: ["/usr/bin/wget: '/usr/lib/libpcre.so.1' is not an ELF file", "/usr/bin/wget: can't load library 'libpcre.so.1'"]}

* VMware: correct guest OS names in hypervisor mappings for VMware 8.0

el9 and variants were introduced by https://github.com/apache/cloudstack/pull/7059
they are supported with guest os identifiers since VMware 8.0

see https://vdc-repo.vmware.com/vmwb-repository/dcr-public/c476b64b-c93c-4b21-9d76-be14da0148f9/04ca12ad-59b9-4e1c-8232-fd3d4276e52c/SDK/vsphere-ws/docs/ReferenceGuide/vim.vm.GuestOsDescriptor.GuestOsIdentifier.html

* VMware: add Ubuntu 20.04 and 22.04 support for vmware 7.0+

* PR7380: only add guest os mappings for Ubuntu 20.04

* PR7380: Correct RHEL9 guest os names and others for VMware 8.0

* PR7380: correct guest os names on 8.0.0.1 as well

* PR7380: remove Windows 12 and Windows Server 2025 which are not released yet
2023-08-17 10:42:42 +02:00
Charles Queiroz e8b491177f
server: Replace Hashtable with LinkedHashMap in createIsoResponse (#7844)
* Replace Hashtable with LinkedHashMap in createIsoResponse

This change replaces the use of Hashtable with LinkedHashMap in the `createIsoResponse` method of `ViewResponseHelper`.
 The reason for this modification is to maintain the insertion order of entries, which isn't the case with Hashtable.
  This could lead to more predictable results and behaviors in calling methods.

* Replace Hashtable with LinkedHashMap in view response creation methods

Changed Hashtable to LinkedHashMap in various response creation methods within ViewResponseHelper class.
This modification ensures an ordered iteration which is beneficial for scenarios where the insertion order of responses needs to be maintained consistently.

---------

Co-authored-by: Sina Kashipazha <soreana@users.noreply.github.com>
2023-08-16 08:58:41 +02:00
dahn cf249f64a3
api: Force-stop descriptions (#7866) 2023-08-16 08:31:32 +02:00
dahn 19aacad46b
UI: Show iso urls (#7869)
* add url to details

* some cleanup
2023-08-15 13:51:53 +02:00
Wei Zhou aa02d9b3c1
test: skip live storage migration on CentOS 7 (#7862)
since #7570, The detail 'Host.OS' of centos7 host is changed from 'CentOS' to 'CentOS Linux'
2023-08-15 08:39:18 +02:00
Alexandru Bagu a47a4f4ad4
CPVM: fix cpu usage for console vm when using vnc over websockets (#6970)
Co-authored-by: Alexandru Bagu <alexandru.bagu@hotmail.com>
2023-08-14 15:45:04 +02:00
Fabricio Duarte 72e3491cef
server: Fix allocation of more public IPs than the account's limit (#7832) 2023-08-14 10:33:29 +02:00
Fabricio Duarte 2c6072273b
VMware: Fix casting exception during backup NIC synchronization (#7831) 2023-08-14 10:20:33 +02:00
Wei Zhou bf9c71430b
CKS: check access when get kubernetescluster config (#7854) 2023-08-14 10:02:20 +02:00
Rohit Yadav feb9509547
ui: assorted improvements (#7833)
This PR aims to polish the UI with following tweaks and changes:

- Increase resource and os-logo icons both in list view, user-menu bar and VM deployment form
- Fix css issues in VM deployment form when resource icons are on some of the templates/isos but not all
- Replace edit icon in the resource icon editting button on the infocard, in resource view
- Fix css marging/padding issue for nav bar and left-branding/logo
- Introduce a new Limits option in the user menu, to allow users to see their own limits when they log in
- Rename resource tab to limits tab for accounts, project and domains
- Introduce a new copy-label component, that can be clicked to copy strings; use in info-card and list view for entites such as IP addresses and UUIDs
- Add router-link to /zones/ in case of user-accounts (when /zone isn't routable in the UI)
- Show better list of nics and ssh keys pairs in infocard for VM resource view
- Standardise most resources to show state/status columns right after resource name (wherever applicable)
- Remove displayname column in VM list view, add cpu number and memory by default
- Add k8s version column in k8s list view
- Add size and phy size columns in case of template and ISOs list view, only for root/domain admins
- Add phy network router-link in case of guest VLAN list view; rearrange columns list for consistency
- Add snapshot phy size column in the snapshot list view; and router-link for volume in the snapshot list view; and missing/useful details in the volume snapshot details view
- Add a create and add data disk feature in Instances tab, just like we've add nic feature in the same

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-08-11 14:00:04 +05:30
Rohit Yadav 29c7b3167e
cloudstack: bump to use 4.18.1.0 systemvmtemplate (#7828)
* cloudstack: bump to use 4.18.1.0 systemvmtemplate

Use new 4.18.1.0 systemvmtemplate

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* fix cd ISO url

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

---------

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-08-11 07:38:47 +02:00
Wei Zhou d9319e645c
storpool: fix pom.xml which caused docker image build failures (#7845)
steps to reproduce the issue:
- git clone https://github.com/apache/cloudstack.git
- cd cloudstack
- rm -rf .git/
- run `mvn -P developer,systemvm clean install`

Without this PR, it fails with error

```
 > [ 8/10] RUN mvn -Pdeveloper -Dsimulator -DskipTests clean install:
668.1 [ERROR] Failed to execute goal pl.project13.maven:git-commit-id-plugin:4.9.10:revision (get-the-git-infos) on project cloud-plugin-storage-volume-storpool: .git directory is not found! Please specify a valid [dotGitDirectory] in your pom.xml -> [Help 1]
```
2023-08-10 16:28:49 +02:00
Wei Zhou 1605b2f0db
server: remove registered userdata when cleanup an account (#7777) 2023-08-10 14:33:59 +02:00
João Jandre fdb23dae40
server: Use max secondary storage defined on the account during upload (#7441) 2023-08-10 11:39:40 +02:00
Wei Zhou b32ba953cf
test: upgrade kubernetes versions to 1.25.0/1.26.0 (#7685)
* test: upgrade versions to 1.25.0/1.26.0

* PR7685: move settings to test_data.py
2023-08-10 09:48:08 +02:00
fermosan fa58f59619
kvm: Added VNI Devices as normal bridge slave devs (#7836)
This will allow for VXLAN configurations to utilize tags on the physical network of a zone
2023-08-10 08:59:28 +02:00
Wei Zhou a1af03f413
noVNC: fix JP keyboard on vmware7+ which uses websocket URL (#7694)
* noVNC: fix JP keyboard on vmware7+ which uses websocket URL

* noVNC: cleanup rfb.js

* noVNC: fix < and > on JP keyboard

* noVNC: fix Caps lock on JP keyboard
2023-08-09 11:43:16 +02:00
Sina Kashipazha 9df23f951b
Prometheus exporter fix cpu/memory usage labels (#7629) 2023-08-07 20:47:03 +02:00
dahn dea1373480
server: clean network offerings for domain on remove (#7775) 2023-08-07 09:56:58 +02:00
Rohit Yadav 4f85c5568f
engine/schema: fix CPU cap limitation for upgraded environment (#7808)
CPU cap limitation was enabled as part of
https://github.com/apache/cloudstack/pull/6420 that changes behaviour
for existing environments. The CPU cap limitation on KVM causes
systemvms to not start or be really slow in nested and virtualised
environments.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-08-07 10:19:33 +05:30
mprokopchuk ab0297ea9b
VM.CREATE/VOLUME.DELETE/VOLUME.DESTROY not being emitted (#7760)
VM.CREATE/VOLUME.DELETE/VOLUME.DESTROY not being emitted

* Update server/src/main/java/com/cloud/vm/UserVmManagerImpl.java

Co-authored-by: dahn <daan.hoogland@gmail.com>

* Update api/src/main/java/org/apache/cloudstack/api/command/user/vm/DeployVMCmd.java

Co-authored-by: dahn <daan.hoogland@gmail.com>

---------

Co-authored-by: Maxim Prokopchuk <mprokopchuk@apple.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-08-07 10:18:17 +05:30
Rohit Yadav dc5e4f3ec6
Allow KVM overcommit to work without reducing minimum VM memory when vm ballooning is disabled (#7810)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: Daan Hoogland <daan@onecht.net>
2023-08-06 10:39:14 +02:00