Commit Graph

86 Commits

Author SHA1 Message Date
Vishesh 72d0546d8b
Shared Network Firewall (Security groups) in Advanced zone without security groups (#9415) 2024-09-05 10:35:05 +02:00
Vishesh 90fe1d5fdc
Merge branch '4.19' 2024-06-29 03:35:24 +05:30
Abhishek Kumar 53faf0f66a
xenserver: attach regular iso with configdrive (#9216)
* xenserver: attach regular iso with configdrive

Fixes #7902

This PR allows attaching a regular ISO to a VM when it already has the
config drive ISO attached.
Config-drive ISO is now attached with the SR name-label
<VM-NAME>-CONFIGDRIVE-ISO. While regular ISOs continue to attach with SR
name-label <VM-NAME>-ISO. VM which already have the configdrive ISO
attached before this fix will return an appropriate error and will need
to be stopped-start.
2024-06-27 16:10:33 +05:30
Wei Zhou fbeee49137
fix build errors after forward-merge 2024-06-20 09:55:35 +02:00
Wei Zhou d81ffd2d86
Merge remote-tracking branch 'apache/4.19' 2024-06-20 08:57:17 +02:00
Daan Hoogland 3997e59678 Merge release branch 4.18 to 4.19
* 4.18:
  Update extraconfig for platform param in xen/xcpng (#9248)
2024-06-19 18:55:29 +02:00
Suresh Kumar Anaparti cc52b38e54
Update extraconfig for platform param in xen/xcpng (#9248)
* Update extraconfig for platform param in xen/xcpng

* Fix map param key, not to replace '-' with '_' (replace only applicable to param / map-param)

* Added unit tests

* Add license for tests file
2024-06-18 23:39:50 +05:30
Gabriel Pordeus Santos adc05f8d59
Fix XenServer nested extra configuration (#8698) 2024-03-25 11:19:16 +01:00
João Jandre 49cecaed06
Normalize loggers and upgrade log4j 1.2 to log4j 2.19 (#7131)
* Normalize logs

All classes that could have their loggers inherited from their fathers had their own loggers deleted;
Most loggers didn't have to be static, so most of them were normalized so that they wouldn't be;
All loggers are protected now;
Static logger's name are now 'LOGGER';
Non-static logger's name are now 'logger';
New class DbUpgradeAbstractImpl created so that all Upgraders extend it and inherit its logger

* Upgrade log4j

* fix errors caused by the merge

* Refactor cglibThrowableRenderer functionality to log4j2 and upgrade the last configuration files

* fix sonarcloud bug

* Fix errors caused by merge, remove some unused loggers, and rename a variable that was mistakenly renamed on the normalization commit

* Readd snmpTrapAppender, remove TestAppender

* Regenerate changes

* regenerate changes

* refactor last custom appender

* fix systemvm configuration xml

* Regenerate changes

* Regenerate changes

* regenerate changes

* Regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* Fix utils pom

* fix some tests

* regenerate changes

* Fix jar being printed on exception

* fix logging in system VMs, fix commands not having log4j2 classpath.

* regenerate changes

* Fix some unwanted renomeations

* fix end of file

* regenerate changes

* regenerate changes

* fix merge error

* regenerate changes

* fix tests

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* readd reload4j to tungsten as juniper depends on it

* Regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* re-add reload4j dependency to network-contrail, as juniper depends on it

* regenerate changes

* regenerate changes

* regenerate changes

* fix typo

* regenerate changes

* regenerate changes

* Fix end of files

* regenerate changes

* add logj42 to cloud-utils-SHADED.jar

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* Regenerate changes

* Regenerate changes

* Regenerate changes

* regenerate changes

* Regenerate changes

* regenerate changes

* Regenerate changes

* Regenerate changes

* Regenerate changes

* regenerate changes

* Regenerate changes

* Regenerate changes

* fix some tests

* Regenerate changes

* Regenerate changes

* fix test

* Regenerate changes

* Regenerate changes
2024-02-08 09:55:41 -03:00
John Bampton f090c77f41
misc: fix spelling (#7549)
Co-authored-by: Stephan Krug <stekrug@icloud.com>
2023-11-02 09:23:53 +01:00
Vishesh 5362bad442
Storage Management (#7949) 2023-11-01 10:46:22 +01:00
John Bampton 4eb110af73
Remove unneeded duplicate words (#7850) 2023-09-18 13:16:33 +02:00
Daan Hoogland 40d7da6707 Merge release branch 4.18 to main
* 4.18:
  xen/xcpng: set vmr.VCPUsMax to minimum of global setting and host cpu cores (#7887)
2023-08-28 14:53:23 +02:00
Wei Zhou 439d70fd2b
xen/xcpng: set vmr.VCPUsMax to minimum of global setting and host cpu cores (#7887)
Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-08-28 14:26:20 +02:00
John Bampton 6f4503488b
pre-commit: apply `end-of-file-fixer` to all files (#7551) 2023-08-02 13:47:21 +02:00
Vishesh d2cd3c81cb
Fix lint errors (#7727) 2023-07-07 21:02:42 +05:30
Vishesh 594c70dde0
Sync precommit config from main (#7732)
Co-authored-by: John Bampton <jbampton@users.noreply.github.com>
Co-authored-by: dahn <daan@onecht.net>
2023-07-07 11:18:16 +02:00
dahn 06caf32bc8
Guest os mappings improvements (#6979)
* Guest OS mapping improvements
- Checks the OS mapping name in hypervisor (VMware, XenServer)
- Displays guest OS mappings in UI

* Added API getHypervisorGuestOsNames to list the guest OS names in the hypervisor, and code improvements

* Some static analysis fixes

* Removed commented code in listview

* Guest OS list

* UI changes for adding guest os and mappings

* Added guest os mappings in guest os form

* Added new filter to guest os mapping

* Name and description changes

* VMWare Host and cluster MO unit tests

* CheckGuestOsMapping command and answer unit tests

* GetHypervisorGuestOsNames command and answer unit tests

* VmwareResource unitests

* GuestOsMapper unittests

* icon changes

* Addressed review comments

* Renaming fixes

* Removed comments

* marvin tests for guest os operations

* Added marvin tests for OS mappings

* Document links and UI improvements

* Added deduplication for the list guest OS API

* Fixed linter failure

* Few bug fixes and UI changes

* Few improvements

* Addressed code smells

* Fixed UI issues after rebase

---------

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
Co-authored-by: Harikrishna Patnala <harikrishna.patnala@gmail.com>
2023-06-27 13:49:20 +05:30
John Bampton c2e17310d6
Add three more `pre-commit` checks (#7083)
Co-authored-by: dahn <daan@onecht.net>
2023-03-27 13:28:55 +02:00
João Jandre 523ab58d02
Fix PR 7131 bugs and vulnerabilities (#7140) 2023-03-21 15:06:18 +01:00
John Bampton def7ce655d
Fix spelling (#6898)
Co-authored-by: davidjumani <dj.davidjumani1994@gmail.com>
2022-12-13 14:58:14 +01:00
Wei Zhou a63b2aba7a
VM Autoscaling with virtual router (#6571) 2022-12-05 15:23:03 +01:00
Rohit Yadav 3d8ea4f3b3 Merge remote-tracking branch 'origin/4.17'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2022-11-30 21:02:17 +05:30
John Bampton e65c22d883
Fix spelling (#6860) 2022-11-13 10:56:15 +01:00
Wei Zhou 296035d9a5
XenServer/XCP-ng: fix vm memory usage is always 99.9x% (#6852)
According to https://docs.citrix.com/en-us/citrix-hypervisor/monitor-performance.html
The metrics "memory_internal_free" is already in KiB, no need to convert.

"Memory used as reported by the guest agent (KiB). Enabled by default"
2022-11-02 09:36:16 +01:00
John Bampton fa39e61a4c
Fix spelling (#6609) 2022-11-01 08:30:01 +01:00
John Bampton f9347ecf2c
Fix spelling (#6597) 2022-08-03 15:43:47 +05:30
Rohit Yadav 5f04018bf0 Merge remote-tracking branch 'origin/4.17' 2022-07-27 12:41:31 +02:00
Rohit Yadav 441edf3ca7
utils: use safer parsing utility across codebase (#6562)
This addresses SonarQube/SonarCloud quality checks to use safer xml
parser to resist potential XXE attacks.

https://sonarcloud.io/organizations/apache/rules?open=java%3AS2755&rule_key=java%3AS2755

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2022-07-27 14:31:51 +05:30
dahn 731a83babf
add global setting to allow parallel execution on vmware (#6413)
* add global setting to allow parallel execution on vmware

* cleanup setting distribution for vmware.create.full.clone

* query setting in vmware guru

* don´t touch other hypervisor's commands

* guru hierarchy cleanup
2022-07-15 10:01:35 +02:00
Pearl Dsilva 3bd723ca06
Change patch path during live patching of systemVMs (#6328)
* Improve log when live patching fails

* change patching path from /tmp to /var/cache/clou

* add iptable rule for console proxy (novnc)

* temporary template paths

* revert pom xml to original paths
2022-04-29 00:01:11 -03:00
Pearl Dsilva 830f3061bc
SystemVM optimizations (#5831)
* Support for live patching systemVMs and deprecating systemVM.iso. Includes:
- fix systemVM template version
- Include agent.zip, cloud-scripts.tgz to the commons package
- Support for live-patching systemVMs - CPVM, SSVM, Routers
- Fix Unit test
- Remove systemvm.iso dependency

* The following commit:
- refactors logic added to support SystemVM deployment on KVM
- Adds support to copy specific files (required for patching) to the hosts on Xenserver
- Modifies vmops method - createFileInDomr to take cleanup param
- Adds configuratble sleep param to CitrixResourceBase::connect() used to verify if telnet to specifc port is possible (if sleep is 0, then default to _sleep = 10000ms)
- Adds Command/Answer for patch systemVMs on XenServer/Xcp

* - Support to patch SystemVMs - VMWare
- Remove attaching systemvm.iso to systemVMs
- Modify / Refactor VMware start command to copy patch related files to the systemvms
- cleanup

* Commit comprises of:
- remove docker from systemvm template - use containerd as container runtime
- update create-k8s-binaries script to use ctr for all docker operations
- Update userdata sent to the k8s nodes
- update cksnode script, run during patching of the cks/k8s nodes

* Add ssh to k8s nodes details in the Access tab on the UI

* test

* Refactor ca/cert patching logic

* Commit comprises of the following changes:
- Use restart network/VPC API to patch routers
- use livePatch API support patching of only cpvm/ssvm
- add timeout to the keystore setup/import script

* remove all references of systemvm.iso

* Fix keystore-cert-import invocation + refactor cert timeout in CP/SS VMs

* fix script timeout

* Refactor cert patching for systemVMs + update keystore-cert-import script + patch-sysvms script + remove patchSysvmCommand from networkelementcommand

* remove commented code + change core user to cloud for cks nodes

* Update ownership of ssh directory

* NEED TO DISCUSS - add on the fly template conversion as an ExecStartPre action (systemd)

* Add UI changes + move changes from patch file to runcmd

* test: validate performance for template modification during seeding

* create vms folder in cloudstack-commons directory - debian rules

* remove logic for on the fly template convert + update k8s test

* fix syntax issue - causing issue with shared network tests

* Code cleanup

* refactor patching logic - certs

* move logic of fixing rootdiskcontroller from upgrade to kubernetes service

* add livepatch option to restart network & vpc

* smooth upgrade of cks clusters

* Support for live patching systemVMs and deprecating systemVM.iso. Includes:
- fix systemVM template version
- Include agent.zip, cloud-scripts.tgz to the commons package
- Support for live-patching systemVMs - CPVM, SSVM, Routers
- Fix Unit test
- Remove systemvm.iso dependency

* The following commit:
- refactors logic added to support SystemVM deployment on KVM
- Adds support to copy specific files (required for patching) to the hosts on Xenserver
- Modifies vmops method - createFileInDomr to take cleanup param
- Adds configuratble sleep param to CitrixResourceBase::connect() used to verify if telnet to specifc port is possible (if sleep is 0, then default to _sleep = 10000ms)
- Adds Command/Answer for patch systemVMs on XenServer/Xcp

* - Support to patch SystemVMs - VMWare
- Remove attaching systemvm.iso to systemVMs
- Modify / Refactor VMware start command to copy patch related files to the systemvms
- cleanup

* Commit comprises of:
- remove docker from systemvm template - use containerd as container runtime
- update create-k8s-binaries script to use ctr for all docker operations
- Update userdata sent to the k8s nodes
- update cksnode script, run during patching of the cks/k8s nodes

* Add ssh to k8s nodes details in the Access tab on the UI

* test

* Refactor ca/cert patching logic

* Commit comprises of the following changes:
- Use restart network/VPC API to patch routers
- use livePatch API support patching of only cpvm/ssvm
- add timeout to the keystore setup/import script

* remove all references of systemvm.iso

* Fix keystore-cert-import invocation + refactor cert timeout in CP/SS VMs

* fix script timeout

* Refactor cert patching for systemVMs + update keystore-cert-import script + patch-sysvms script + remove patchSysvmCommand from networkelementcommand

* remove commented code + change core user to cloud for cks nodes

* Update ownership of ssh directory

* NEED TO DISCUSS - add on the fly template conversion as an ExecStartPre action (systemd)

* Add UI changes + move changes from patch file to runcmd

* test: validate performance for template modification during seeding

* create vms folder in cloudstack-commons directory - debian rules

* remove logic for on the fly template convert + update k8s test

* fix syntax issue - causing issue with shared network tests

* Code cleanup

* add cgroup config for containerd

* add systemd config for kubelet

* add additional info during image registry config

* address comments

* add temp links of download.cloudstack.org

* address part of the comments

* address comments

* update containerd config - as version has upgraded to 1.5 from 1.4.12 in 4.17.0

* address comments - simplify

* fix vue3 related icon changes

* allow network commands when router template version is lower but is patched

* add internal LB to the list of routers to be patched on network restart with live patch

* add unit tests for API param validations and new helper utilities - file scp & checksum validations

* perform patching only for non-user i.e., system VMs

* add test to validate params

* remove unused import

* add column to domain_router to display software version and support networkrestart with livePatch from router view

* Requires upgrade column to consider package (cloud-scripts) checksum to identify if true/false

* use router software version instead of checksum

* show N/A if no software version reported i.e., in upgraded envs

* fix deb failure

* update pom to official links of systemVM template
2022-04-21 13:40:19 -03:00
José Flauzino 16f2896940
Persistence of VM stats (#5984)
* Add persistence of VM stats

* Fix API 'since' attribute

* Add license

* Address GutoVeronezi's reviews

* Fix the order of VM stats in the API response

* Fix msid in VM stats data

* Fix disk stats and add minor improvements

* Add log message

* Build string using ReflectionToStringBuilderUtils

* Rerun checks

Co-authored-by: joseflauzino <jose@scclouds.com.br>
2022-04-11 10:42:21 -03:00
David Jumani 5ab4309298
agent: Detect existing hosts with UEFI support (#6139)
* agent: Pass uefi enabled status as part of ready command

* Cleanup

* Fix checkstyle

* Save uefi status if different
2022-04-04 23:31:54 -03:00
John Bampton 6401c850b7
Fix spelling (#6064)
* Fix spelling

- `interupted` to `interrupted`
- `paramter` to `parameter`

* Fix more typos
2022-03-08 13:02:35 -03:00
Suresh Kumar Anaparti bf70566c2c
Merge branch '4.16' into main 2022-02-02 17:30:21 +05:30
Nicolas Vazquez ddd311c695
[XenServer/XCP-ng] Sync the 'platform' setting according to the 'cpu.corespersocket' setting (#5892)
* Update platform setting if cpu.corespersocket is set

* Refactor conditions

* Allow empty map but not null
2022-02-02 12:21:47 +05:30
Nicolas Vazquez 3e92a63155
[XenServer/XCP-ng] Pass the image store NFS version on storage commands (#5886)
* Add NFS version to mount command

* Remove extra line

* Extend NFS version to mount secondary storage

* Unused import

* Refactor NFS version to be granular

* Make use of the ConfigKey on the NFS version setting value
2022-01-31 12:21:13 +05:30
Daniel Augusto Veronezi Salvador b4aabadc4d
Replace string libraries with org.apache.commons.lang3.StringUtils (#5386)
* Replace google lib for lang3 and adjust methods calls

* Replace string libs by lang3

* Prohibit others string libs

Co-authored-by: GutoVeronezi <daniel@scclouds.com.br>
2021-11-18 13:41:48 +05:30
Abhishek Kumar 2a42a01728
xcp-ng: fix vm boot options (#5543)
* xcp-ng: fix vm boot options

Use value boot option values from VM details directly similar to other hypervisor plugins instead of relying on boot properties explicitly set for VirtualMachineTO.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Co-authored-by: Harikrishna Patnala <harikrishna.patnala@gmail.com>
2021-10-07 13:22:23 +05:30
Wei Zhou 09fce75047
OVS/GRE: bug fixes (#5446)
* OVS/GRE: fix failed to fetch cloudbr* info on host

* OVS/GRE: move ovstunnel.py to python3

* OVS: add test_privategw_acl_ovs_gre.py for automatic testing

(1) VR <-> VM should work
(2) Private GW should work

* OVS: fix an issue on xenserver

ovs bridges are deleted by xenserver/ovs automatically
```
[root@ref-trl-1797-x-M7-wei-zhou-xs2 ~]# grep -r xapi7 /var/log/ |grep del-br
/var/log/xensource.log:Sep 15 07:13:44 ref-trl-1797-x-M7-wei-zhou-xs2 xcp-networkd: [ info|localhost|611 |org.xen.xapi.xenops.classic events D:4a3d931cd89f|network_utils] /usr/bin/ovs-vsctl --timeout=20 -- --if-exists del-br xapi7
/var/log/daemon.log:Sep 15 07:13:45 ref-trl-1797-x-M7-wei-zhou-xs2 ovs-vsctl: ovs|00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=20 -- --if-exists del-br xapi7
```

which results that xe network exists but bridge does not exist, and operation stuck for 20 minutes at
```
2021-09-15 16:06:56    DEBUG [root] #### VMOPS enter  create_tunnel ####
2021-09-15 16:06:56    DEBUG [root] Creating tunnel from host 2 to host 1 with GRE key 2116
2021-09-15 16:06:56    DEBUG [root] Executing:['/usr/bin/ovs-vsctl', '--timeout=0', 'wait-until', 'bridge', 'xapi7', '--', 'get', 'bridge', 'xapi7', 'name']
2021-09-15 16:26:56    DEBUG [root] bridge xapi7 for creating tunnel - VERIFIED
2021-09-15 16:26:56    DEBUG [root] Executing:['/usr/bin/ovs-vsctl', 'add-port', 'xapi7', 't2116-2-1', '--', 'set', 'interface', 't2116-2-1', 'type=gre', 'options:key=2116', 'options:remote_ip=10.0.34.230']
```

* OVS/GRE: component test test_vpc_distributed_routing_offering.py passes on xenserver

* OVS: support Shared network with GRE protocol

* update #5446: use lang3.StringUtils

* OVS/GRE: VPC with distributed routing on kvm (still not working)
2021-10-03 14:47:52 +05:30
Rohit Yadav b955b38ccb Merge remote-tracking branch 'origin/4.15' into main 2021-09-27 16:17:49 +05:30
Abhishek Kumar 25d522fb3d
server: add vm boot details for start vm api (#5469)
Add vm boot details for start vm api

Fixes #5466

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-09-27 16:16:27 +05:30
Abhishek Kumar 56f4da6dce Merge remote-tracking branch 'apache/4.15' into main 2021-09-02 16:13:33 +05:30
Abhishek Kumar 73cabcd641
xcp-ng: allow passing vm boot options (#5335)
* xenserver: honor vm boot details

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: allow boot option selection for xenserver

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix case

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* host uefi capability

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* change

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add detail only if uefi supported

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* update host detail

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix version comparison

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-08-31 16:10:12 +05:30
davidjumani cb1078aa20
Merge remote-tracking branch 'apache/4.15' into main 2021-08-09 15:10:51 +05:30
Abhishek Kumar 1ccb42017f
server: skip zone check for PERHOST iso during attachIso (#5270)
* server: skip zone check for PERHOST iso during attachIso

Hypervisor tools ISO - vmware-toools.iso, xs-tools.iso are marked as PERHOST in DB. They are active but not downloaded to the secondary storages and hence no template-zone entry.
Skips the template-zone check for such templates.

Fixes #5265

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* inverted check

* use constants in TemplateManager

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-08-09 14:02:25 +05:30
Rohit Yadav 5aefd3e204 Merge remote-tracking branch 'origin/4.15' 2021-06-02 14:03:54 +05:30
Rohit Yadav 8c7e39eb89
xenserver: attempt eject and destroying patch VBD separately (#5053)
Loop through all VMs and try to eject and remove systemvm.iso VBD upon host connect.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-06-01 20:55:57 +05:30
Rohit Yadav cb167072a1 Merge remote-tracking branch 'origin/4.15'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-05-07 16:37:42 +05:30