Commit Graph

11180 Commits

Author SHA1 Message Date
Harikrishna Patnala 31742ccf06 Remove code that reserves extra interfaces based on the global setting router.extra.public.nics in case of VMware.
Added hot plugging of vifs whenever required in case of isolated networks. VPC networks already uses hotplugging of vifs.
2022-01-31 16:09:02 +05:30
Nicolas Vazquez 84f5768e64
[VMware][Deploy-as-is] OVF properties not importing when template is uploaded from local (#5861)
* Fix ova upload missing details

* Refactor and cleanup

* Unused import
2022-01-26 11:28:52 +05:30
Wei Zhou f3289fb596
server: fix regular user can create isolated network without sourcenat (#5844)
* server: fix regular user can create isolated network without sourcenat
2022-01-20 14:10:23 +05:30
Abhishek Kumar d78a815442
server: fix update vm with unconstrained offering (#5828)
Fixes #5694

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2022-01-13 12:30:12 +05:30
Wei Zhou 216bea1978
ipv6: disable IPv6-only shared network with VR (#5810)
* ipv6: disable IPv6-only shared network with VR

* Update #5810: import org.apache.commons.lang3.StringUtils
* Update #5810: do not import org.apache.commons.lang3.StringUtils
* Update server/src/main/java/com/cloud/network/NetworkServiceImpl.java
* Update engine/orchestration/src/main/java/org/apache/cloudstack/engine/orchestration/NetworkOrchestrator.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
2022-01-12 13:33:35 +05:30
Wei Zhou 9293f5bd27
server: fix enable/disable static nat if userdata is not supported (#5839)
* server: fix enable/disable static nat if userdata is not supported

* Update #5839: rename applyUserData to applyUserDataIfNeeded

* Update server/src/main/java/com/cloud/network/rules/RulesManagerImpl.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
2022-01-11 12:22:59 +05:30
Wei Zhou fadd74aaca
network: fix vm can be deployed on L2 network of other accounts (#5784)
* Update #5769: fix domain admin can deploy vm on L2 network of other users

* test: fix test_storage_policy.py

* Update #5784: revert part of changes in #2420
2022-01-11 12:16:00 +05:30
dahn 028d338aaa
remove VmWorkJob after adding a nic to a vm (#5658)
Co-authored-by: Daan Hoogland <dahn@onecht.net>
Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
Co-authored-by: Wei Zhou <weizhou@apache.org>
2022-01-10 16:31:50 +01:00
Nicolas Vazquez 946f9156e5
vmware: Persist empty values instead of null (#5748)
This PR allows deploying VMware appliances missing default values on some properties
Fixes: #5747
2022-01-08 08:58:46 +05:30
dahn 2774bc156f
use physical size instead of virtual size for migration. (#5750)
* Use Physical size to evaluate if migration is possible

* Improve logging and consider files skipped as failure in complete migration

* skipped can't be negative

* remove useless method

* group multidisk templates for secstor migration

* use enum

* Update engine/orchestration/src/main/java/org/apache/cloudstack/engine/orchestration/DataMigrationUtility.java

Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>
Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
Co-authored-by: Daan Hoogland <dahn@onecht.net>
Co-authored-by: Pearl d'Silva <pearl.dsilva@shapeblue.com>
2022-01-06 17:18:50 +05:30
Abhishek Kumar 51f69f7134
server: do not return inaccessible entity details to normal users (#5827)
Fixes #5534

As pre 3.x APIs allow using internal DB IDs, even normal users can use internal IDs.
This fix removes additional information in error message when the caller doesn't have access to the resource.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2022-01-06 16:42:57 +05:30
Rakesh 2bd1dc1e14
Enable resetting config values to default value (#4230)
* Enable resetting config values to default value

Provide reset button to zone,cluster,domain,account,
primary and secondary storage so that config values
can be reset to default value

* fix ui issue

* Update test/integration/smoke/test_reset_configuration_settings.py

* Update test/integration/smoke/test_reset_configuration_settings.py

Co-authored-by: Rakesh Venkatesh <rakeshv@apache.org>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2022-01-03 21:55:30 +01:00
Wei Zhou 8952cd5955
kvm: add hosts using cloudstack ssh private key (ccc21 hackathon) (#5684)
This PR provides the option to add kvm hosts with empty or wrong password.
To support this, the cloudstack ssh public key needs to be added in the ~/.ssh/authorized_keys on host.

Feature work:

1. get cloudstack public key from management server (/var/lib/cloudstack/management/.ssh/id_rsa.pub)

![image](https://user-images.githubusercontent.com/57355700/141449653-85f644b5-c32e-44ca-9c6b-77570262c046.png)


2. add the key to ~/.ssh/authorized_keys on kvm hosts

![image](https://user-images.githubusercontent.com/57355700/141449722-e906eea5-74fd-4f81-a4d3-41563beeb79c.png)

3. add kvm host with empty password

<img src="https://user-images.githubusercontent.com/57355700/141449865-6ffee1f0-b0d7-4ea4-b11a-32df42e2fe91.png" height="50%" width="50%">

Tested as:
1. add host, with correct password,  works as expected.
2. put host to maitenance, stop cloudstack-agent, cancel maintenance, remove the host. all work.
3. add host, with empty password (ssh public key is added to  ~/.ssh/authorized_keys),   works as expected.
4. put host to maitenance, stop cloudstack-agent, cancel maintenance. all work as expected.
2021-12-30 16:20:04 +05:30
Nicolas Vazquez c0ddda361a
[VMware] Fix service offerings listing on appliances deployment options (#5752)
* Fix list service offerings for VMware deployment options

* UI fix to allow selecting offerings
2021-12-30 16:13:41 +05:30
Marcus Sorensen dcdcd09058
Randomize managed volume copy host (#5789)
* Randomize managed volume copy host

* Managed volume copy was always returning first host that could see storage pools

* Fix null value in logging for ScaleIOPrimaryDataStoreDriver due to if/else logic error

Signed-off-by: Marcus Sorensen <mls@apple.com>

* Use String.format for ScaleIO debug message

Signed-off-by: Marcus Sorensen <mls@apple.com>

* Update debug message for ScaleIO copy methods

Signed-off-by: Marcus Sorensen <mls@apple.com>

Co-authored-by: Marcus Sorensen <mls@apple.com>
2021-12-30 16:11:00 +05:30
Wei Zhou 7823868746
network: update ip in lb/pf/dnat tables when update vm nic ip (#5757)
* network: update ip in lb/pf/dnat tables when update vm nic ip

* Update: create 3 separated methods for DNAT/LB/PF update

* Update: invert if condition to reduce code indentation
2021-12-28 01:35:29 +05:30
Rakesh 8ba5b92a83
server: Fix NPE while deleting a domain (#5753)
* server: Fix NPE while deleting a domain

While deleting a domain, if vlan ip range cant be found
then this will throw NPE. Just return false if vlan ip
range cant be found

* return true if vlan is not found

* change output message

Co-authored-by: Rakesh Venkatesh <rakeshv@apache.org>
2021-12-28 01:21:43 +05:30
Rakesh 9607ae9639
server: Fix NPE while adding network to VPC (#5723)
* server: Fix NPE while adding network to VPC

Sometimes adding network to VPC fails because of libirt error.
In that case the new network wont have any VR's associated with it
and hence it will return null.

* use collectionutils

Co-authored-by: Rakesh Venkatesh <rakeshv@apache.org>
2021-12-25 19:57:18 +05:30
sureshanaparti 9b0a1418ec
Provision to sort ISOs from UI, and Updated Templates/ISOs API response to return in the order of sortkey (#5800) 2021-12-24 12:44:50 +05:30
sureshanaparti 39e41f6b6e
Allow force reboot VM from user account, to start VM on the same host (#5791) 2021-12-22 17:43:02 +01:00
Pearl Dsilva 936ebbb90f
api: Fix search cluster by name (#5782) 2021-12-22 10:41:06 +05:30
Gabriel Beims Bräscher 995ce06cb7
Enhance log message in FirstFitPlanner (#5762)
* Enhance log message in FirstFitPlanner

When cluster reached capacity threshold the message is:
"capacity threshold defined at each cluster/ at global value for capacity Type : 0"

Admins hardly remember the Capacity Type and it can take a while to look at which is the resource for the respective ID. This enhancement addes log message pointing to the capacity name (e.g. Memory / CPU) as well as global settings parameter name and value to be looked at.

* Change formatation in String 'warnMessageForClusterReachedCapacityThreshold'
2021-12-22 10:10:40 +05:30
dahn a47e53fcee
internal ref replaced by uuid (#5738)
* internal ref replaced by uuid

* remove all internal DB IDs from usage records

* remove superfluent .append(")")

Co-authored-by: Daan Hoogland <dahn@onecht.net>
2021-12-07 10:09:38 +05:30
Wei Zhou a277e4e08c
server: fix non-root users are able to list system networks by id (#5729) 2021-12-03 10:36:33 +01:00
Wei Zhou 4d99f84273
server: update capacity_state of host cpu core after disable/enable a host (#5701) 2021-12-03 06:45:53 +01:00
davidjumani dfd75d7bc0
api: Fix wrong logger class in *Cmd.java (#5740)
This PR changes the incorrect class name used in the logger in ListHypervisorsCmd
2021-12-02 15:16:28 +05:30
Wei Zhou dd458feeb8
IPv6: fix deploy vm issue in ipv6-only networks without VR (#5648)
* IPv6: fix deploy vm issue in ipv6-only networks

* Update #5648: add method setNicPropertiesFromNetwork
2021-11-18 13:53:55 +05:30
nicolas 93c3c3b9ac
Updating pom.xml version numbers for release 4.16.1.0-SNAPSHOT
Signed-off-by: nicolas <nicovazquez90@gmail.com>
2021-11-09 22:50:22 -03:00
nicolas 44c08b5acc
Updating pom.xml version numbers for release 4.16.0.0
Signed-off-by: nicolas <nicovazquez90@gmail.com>
2021-11-04 14:14:57 -03:00
Nicolas Vazquez 261fe326ff
Revert "parallel nic adding (#5541)" (#5665)
This reverts commit 3574d8d20b.
2021-11-04 13:29:59 -03:00
Abhishek Kumar 0ad742405c
api,server,engine/schema: admin listvm api clusterid (#5659)
* api,server,engine/schema: admin listvm api clusterid

Add clusterid parameter in listVirtualMachines API for admin

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* import order

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* set clusterid only for ListVMsCmdByAdmin

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-11-04 08:47:42 +05:30
Rohit Yadav 77fb93c513
core: use the URL scheme same as iframe for non-SSL enabled consoles (#5624)
* core: use the URL scheme same as iframe for non-SSL enabled consoles

For environments where SSL is not enabled for console, this forces the
URL scheme (http/https) in iframe to match the iframe URL scheme.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* consoleproxy: enable SSL on CPVM when both console proxy url/domain and
ssl setting are configured

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* fix unit test

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* address code review comments

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-10-29 13:42:44 -03:00
sureshanaparti 0a88e710b2
Check the pool used space from the bytes used in the storage pool stats collector, for non-default primary storage pools that cannot provide stats. (#5586)
* Check the pool used space from the bytes used in the storage pool stats collector, for  non-default primary storage pools that cannot provide stats.
Also, Update the used bytes from the pool stats answer for non-default primary storage pools if the pool can provide stats.

* Update server/src/main/java/com/cloud/storage/StorageManagerImpl.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* space fix

Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
2021-10-25 08:23:07 -03:00
Wei Zhou f29aef8b70
VPC: support LB in multiple vpc tiers if LB provider is VpcVirtualRouter (#5580)
* VPC: support LB in multiple vpc tiers if LB provider is VpcVirtualRouter

* server: fix unit test CreateNetworkOfferingTest failures

[ERROR] Tests run: 10, Failures: 0, Errors: 10, Skipped: 0, Time elapsed: 13.902 s <<< FAILURE! - in org.apache.cloudstack.networkoffering.CreateNetworkOfferingTest
[ERROR] createIsolatedNtwkOffWithVlan(org.apache.cloudstack.networkoffering.CreateNetworkOfferingTest)  Time elapsed: 0.662 s  <<< ERROR!
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'loadBalancerDaoImpl': Invocation of init method failed; nested exception is java.lang.NullPointerException
	at org.apache.cloudstack.networkoffering.CreateNetworkOfferingTest.setUp(CreateNetworkOfferingTest.java:110)
Caused by: java.lang.NullPointerException
	at org.apache.cloudstack.networkoffering.CreateNetworkOfferingTest.setUp(CreateNetworkOfferingTest.java:110)

* update #5580: use java.util.Optional

* update #5580: create method listByNetworkIdOrVpcIdAndScheme
2021-10-17 06:55:24 +05:30
sureshanaparti 07efd7c19d
Updated storage type of the volume, in the volume response, based on the underlying storage pool. (#5560) 2021-10-09 21:22:20 -03:00
Wei Zhou 798d7bebf6
VR: skip dhcp/dns health check in some cases (#5554)
this fixes #5536
2021-10-08 08:22:32 +05:30
davidjumani 6ac834a358
Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements (#4329)
Adding AutoScaling support for cks
Kubernetes PR : kubernetes/autoscaler#3629
Also replaces CoreOS with Debian
Fixes #4198

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
Co-authored-by: Wei Zhou <w.zhou@global.leaseweb.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-10-06 21:17:41 +05:30
sureshanaparti e4beb1f955
Add empty config value for scope based config setting (#5551) 2021-10-05 18:49:33 -03:00
Abhishek Kumar 6e216dd0d1
vmware, network: add maclearning option (#5471)
* vmware, network: add maclearning option

Adds option for specifying MAC Learning property for network offering (useful for VMware Distributed Virtual Portgroup). Added global config - network.mac.learning for the default value.
MAC Learning is supported for DV portgroups for VMware Distributed vSwitches v6.6.0+ and vSphere 6.7+

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix warning msg

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-10-04 20:00:45 -03:00
dahn ad4def5d1b
an inject annotation short (#5547)
Co-authored-by: Daan Hoogland <dahn@onecht.net>
2021-10-04 21:20:31 +02:00
dahn 3574d8d20b
parallel nic adding (#5541)
* trace nics additions

* work queue patch for network to add

* add secondary key to job

* logging improvements and naming of field(s)

* several naming corrections

* extra check if net already exists for vm

* placeholder job with secondary object

* constraint on entering the same job multiple times

* error handling/warning message

* review comments applied

Co-authored-by: Daan Hoogland <dahn@onecht.net>
Co-authored-by: Wei Zhou <wei.zhou@shapeblue.com>
2021-10-04 12:21:21 -03:00
Leo (Hsueh Yu-Min) 72a1c0e7f1
[KVM] Add MV Settings for virtual GPU hardware type and memory (#5513)
* KVM: Add MV Settings for virtual GPU hardware type and memory

* fix method createVideoDef argument in test package

* add available options for KVM virtual GPU hardware VM setting

* fix videoRam default value

* fix _videoRam is 0, it will use default provided by libvirt
2021-10-04 09:55:32 +05:30
Wei Zhou 669ab73efe
server: check service offering (storage) tags when reallocate a ROOT disk (#5501)
* server: check service offering (storage) tags when reallocate a ROOT disk

* server: resize volumes in Allocated state
2021-10-03 19:45:59 -03:00
Daniel Augusto Veronezi Salvador e92ce916f3
network: Remove logic that creates gap for multiple 'source NAT' in VR (#5532)
In ACS, when a VPC has more than one public IP and a user tries to use the non `source NAT` IP with some feature/option (like `static NAT`, `port forwarding`, `VPN` and others), ACS adds the public IP (used for the feature/option) as `source NAT` in the `iptables` of the VR.

Example:
- VPC has one public IP `192.168.0.50` and it is defined as the `source NAT`.
- If we assign another public IP `192.168.0.51` to the VPC and use it to execute `port forwarding`, ACS will automatically add `192.168.0.51` as `source NAT` too.

Co-authored-by: GutoVeronezi <daniel@scclouds.com.br>
2021-10-03 14:55:28 +05:30
Wei Zhou 09fce75047
OVS/GRE: bug fixes (#5446)
* OVS/GRE: fix failed to fetch cloudbr* info on host

* OVS/GRE: move ovstunnel.py to python3

* OVS: add test_privategw_acl_ovs_gre.py for automatic testing

(1) VR <-> VM should work
(2) Private GW should work

* OVS: fix an issue on xenserver

ovs bridges are deleted by xenserver/ovs automatically
```
[root@ref-trl-1797-x-M7-wei-zhou-xs2 ~]# grep -r xapi7 /var/log/ |grep del-br
/var/log/xensource.log:Sep 15 07:13:44 ref-trl-1797-x-M7-wei-zhou-xs2 xcp-networkd: [ info|localhost|611 |org.xen.xapi.xenops.classic events D:4a3d931cd89f|network_utils] /usr/bin/ovs-vsctl --timeout=20 -- --if-exists del-br xapi7
/var/log/daemon.log:Sep 15 07:13:45 ref-trl-1797-x-M7-wei-zhou-xs2 ovs-vsctl: ovs|00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=20 -- --if-exists del-br xapi7
```

which results that xe network exists but bridge does not exist, and operation stuck for 20 minutes at
```
2021-09-15 16:06:56    DEBUG [root] #### VMOPS enter  create_tunnel ####
2021-09-15 16:06:56    DEBUG [root] Creating tunnel from host 2 to host 1 with GRE key 2116
2021-09-15 16:06:56    DEBUG [root] Executing:['/usr/bin/ovs-vsctl', '--timeout=0', 'wait-until', 'bridge', 'xapi7', '--', 'get', 'bridge', 'xapi7', 'name']
2021-09-15 16:26:56    DEBUG [root] bridge xapi7 for creating tunnel - VERIFIED
2021-09-15 16:26:56    DEBUG [root] Executing:['/usr/bin/ovs-vsctl', 'add-port', 'xapi7', 't2116-2-1', '--', 'set', 'interface', 't2116-2-1', 'type=gre', 'options:key=2116', 'options:remote_ip=10.0.34.230']
```

* OVS/GRE: component test test_vpc_distributed_routing_offering.py passes on xenserver

* OVS: support Shared network with GRE protocol

* update #5446: use lang3.StringUtils

* OVS/GRE: VPC with distributed routing on kvm (still not working)
2021-10-03 14:47:52 +05:30
Abhishek Kumar 4a42e7ef9e
vmware, ui: update portgroup on network update (#5470)
Enhanced update network form in the UI.
On network offering change for an isolated network,

- VMware portgroup should be updated accordingly.
- VMs on the network should be placed on the correct VMware portgroup based on the network rate, https://docs.cloudstack.apache.org/en/latest/adminguide/service_offerings.html#network-throttling.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-10-03 14:41:42 +05:30
SadiJr 965a47fdfd
Create UpdateBackupOffering API (#5511)
* Create UpdateBackupOffering API

* Address reviews

* Address reviews

* Address reviews

Co-authored-by: SadiJr <sadi@scclouds.com.br>
2021-10-01 08:30:25 -03:00
Pearl Dsilva 32051fb5fa
Fix permission issue during Diagnostic service garbage collection (#5504) 2021-09-30 08:18:23 -03:00
Rakesh dcc02e0fbb
Enable account settings to be visible under domain settings (#4215)
* Enable account settings to be visible under domain settings

All the account settings can't be configured under domain
level settings right now.
By default, if account setting is not configured then
its value will be taken from global setting.
Add a global setting "enable.account.settings.for.domain"
so that if its enabled then all the account level settings
will be visible under domain levelsettings also.
If account level setting is configured then that value will
be considered else it will take domain scope value. If
domain scope value is not configured then it will pick
it up from global setting.

If domain level setting is not configured then by default
the value will be taken from global setting
Add another global setting "enable.domain.settings.for.child.domain"
so that when its true, if a value for domain setting is not
configured then its parent domain value is considered until
it reaches ROOT domain. If no value is configured till ROOT
domain then global setting value will be taken.

Also display all the settings configured under the domain level
in list domains api response

* rename variables

Co-authored-by: Rakesh Venkatesh <rakeshv@apache.org>
2021-09-29 10:29:20 +02:00
Pearl Dsilva 37d22646b2
simulator: Add support to scale a VM (#5515)
* simulator: Add support to scale VM

* make list a constant
2021-09-29 09:47:58 +05:30