Commit Graph

1217 Commits

Author SHA1 Message Date
Wei Zhou 09fce75047
OVS/GRE: bug fixes (#5446)
* OVS/GRE: fix failed to fetch cloudbr* info on host

* OVS/GRE: move ovstunnel.py to python3

* OVS: add test_privategw_acl_ovs_gre.py for automatic testing

(1) VR <-> VM should work
(2) Private GW should work

* OVS: fix an issue on xenserver

ovs bridges are deleted by xenserver/ovs automatically
```
[root@ref-trl-1797-x-M7-wei-zhou-xs2 ~]# grep -r xapi7 /var/log/ |grep del-br
/var/log/xensource.log:Sep 15 07:13:44 ref-trl-1797-x-M7-wei-zhou-xs2 xcp-networkd: [ info|localhost|611 |org.xen.xapi.xenops.classic events D:4a3d931cd89f|network_utils] /usr/bin/ovs-vsctl --timeout=20 -- --if-exists del-br xapi7
/var/log/daemon.log:Sep 15 07:13:45 ref-trl-1797-x-M7-wei-zhou-xs2 ovs-vsctl: ovs|00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=20 -- --if-exists del-br xapi7
```

which results that xe network exists but bridge does not exist, and operation stuck for 20 minutes at
```
2021-09-15 16:06:56    DEBUG [root] #### VMOPS enter  create_tunnel ####
2021-09-15 16:06:56    DEBUG [root] Creating tunnel from host 2 to host 1 with GRE key 2116
2021-09-15 16:06:56    DEBUG [root] Executing:['/usr/bin/ovs-vsctl', '--timeout=0', 'wait-until', 'bridge', 'xapi7', '--', 'get', 'bridge', 'xapi7', 'name']
2021-09-15 16:26:56    DEBUG [root] bridge xapi7 for creating tunnel - VERIFIED
2021-09-15 16:26:56    DEBUG [root] Executing:['/usr/bin/ovs-vsctl', 'add-port', 'xapi7', 't2116-2-1', '--', 'set', 'interface', 't2116-2-1', 'type=gre', 'options:key=2116', 'options:remote_ip=10.0.34.230']
```

* OVS/GRE: component test test_vpc_distributed_routing_offering.py passes on xenserver

* OVS: support Shared network with GRE protocol

* update #5446: use lang3.StringUtils

* OVS/GRE: VPC with distributed routing on kvm (still not working)
2021-10-03 14:47:52 +05:30
Rohit Yadav 847078b3c8 Merge remote-tracking branch 'origin/4.15' into main 2021-09-22 14:38:56 +05:30
Wei Zhou 7444bc6cd5
travis: fix test/integration/component/test_public_ip.py (#5486) 2021-09-22 09:55:13 +02:00
dahn 571a361926
tests: component test ports/fixes in python3 (#5082)
* cleanup plan

* more robust cleanup within member method

* ss_domain_limits fixed

* egress 'icmp' instead of 'all' and cleanup

* configdrive syntax, fixed to be up to par with py2

* cleanup fixed the lb secondary ip tests?

* deal with different base64 encoding in py3 for userdata

* cleanup of multiple_ips_per_nic

* cleanup and reformat of test_volumes

* cleanup and fixes for test_ps_domain_limits.py

* cleanup and fix of test_ps_limits.py

* fix occasional match of float against int

* cleanup and fix test_ps_resize_volume.py

* cleanup and fix test_snapshots

* cleanup ss_max_limits and fix for float vs int problem in API

* mere cleanup of test_volume_destroy_recover

* add missing command creation

* cleanup of test_vpc_on_host_maintenance

* cleanup test_vpc_network

* cleanup, comments and logging in test_vpc_network_lbrules

* cleanup of test_vpc_network_pfrules

* cleanup and format code for test_vpc_network_staticnatrule

* f string instead of conversion specifiers

* check http and ssh fix and cleanup (for vpc pfrules tests)

* generalise create network method

* make ip optional in creating webserver

* remove unused code and add rules to enable ssh

* more cleanup

* remove unused code and cleanup

* small cleanup, mostly precarous run environment required

* cleanup and removed unused code

* advancedsg only, cleanup, pulled in services

* reformat/cleanup

* log result of error after verify rule

* add nw_off_no_services

* tags=["TODO"] for escalations_networks

* tags=["TODO"] for organization_states

* tags=["TODO"] for browse_templates

* tags=["TODO"] for configdrive

* tags=["TODO"] for vpc_vms_deployment

* add remove network cleanup and fixes

* move tests that fail on all platforms out of the way

Co-authored-by: Daan Hoogland <dahn@onecht.net>
2021-09-02 16:03:49 +05:30
davidjumani 8bc120b288
Merge remote-tracking branch 'apache/4.15' into main 2021-08-24 13:46:28 +05:30
davidjumani f822547c3b
securitygroups: Fix security_groups for c8/suse (#5354)
* Fix security_groups for c8/suse

* security_group.py: revert a minor change in #5348

Co-authored-by: Wei Zhou <weizhou@apache.org>
2021-08-24 13:07:52 +05:30
Rohit Yadav 57a55bd777 Merge remote-tracking branch 'origin/4.15' into main 2021-08-20 17:19:40 +05:30
Wei Zhou f9fe9da734
security group: fix component test test_multiple_nic_support.py failures (#5348)
This PR fixes component test failure with test_multiple_nic_support.py, in advanced zone with security groups.

Co-authored-by: dahn <daan.hoogland@gmail.com>
2021-08-20 17:06:26 +05:30
DK101010 664a46a525
PR multi tags in compute offering [#4398] (#4399)
* [#4398] adapt code to handle multi tag string with commas

* [#4398] remove trailing spaces

* [#4398] add multi host tag support for ingest process

* [#4398] add test for multi tag support in offerings

* [#4398]  update multitag support for DeploymentPlanningManagerImpl

encapsulate multi tag check from Ingest Feature, DepolymentPlanningManager into
HostDaoImpl to prevent code duplicates

* [#4398] move logic to HostVO and add tests

* rename test method

* [#4398] Change string method to apaches StringUtils

* [#4398] modify test for multi tag support

* adapt sql for double tags

Co-authored-by: Dirk Klahre <Dirk.Klahre@Itelligence.de>
2021-08-16 12:08:40 -03:00
DK101010 c23cbdff8d
Remove condition that are prevent resizing for root volumes (vmware) (#5186)
* remove condition that are prevent resizing for root volumes

* adjust tests acordingly

* enable disk resize tests for VMware

Co-authored-by: DK101010 <dirk.klahre@itelligence.de>
Co-authored-by: mib1185 <mail@mib85.de>
2021-07-21 22:33:51 -03:00
Abhishek Kumar 5ac184edd6 Merge remote-tracking branch 'apache/4.15' into main 2021-07-15 14:02:49 +05:30
Wei Zhou cf0f1feb5e
configdrive: fix some failures in tests/component/test_configdrive.py (#5144)
* server: fix failed to apply userdata when enable static nat

* server: fix cannot expunge vm as applyUserdata fails

* configdrive: fix ISO is not recognized when plug a new nic

* configdrive: detach and attach configdrive ISO as it is changed when plug a new nic or migrate vm

* configdrive test: (1) password file does not exists in recreated ISO; (2) vm hostname should be changed after migration

* configdrive: use centos55 template with sshkey and configdrive support

* configdrive: disklabel is 'config-2' for configdrive ISO

* configdrive: use copy for configdrive ISO and move for other template/volume/iso

* configdrive: use public-keys.txt

* configdrive test: fix (1) update_template ; (2) ssh into vm by keypair
2021-07-15 12:49:37 +05:30
sureshanaparti 4f6851f4c0
Some missed inclusivity changes - use inclusive words/terms as appropriate. (#5131) 2021-06-18 17:10:11 +02:00
Suresh Kumar Anaparti 958182481e cloudstack: make code more inclusive
Inclusivity changes for CloudStack

- Change default git branch name from 'master' to 'main' (post renaming/changing default git branch to 'main' in git repo)
- Rename some offensive words/terms as appropriate for inclusiveness.

This PR updates the default git branch to 'main', as part of #4887.

Signed-off-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-06-08 15:47:20 +05:30
dahn a1f825e7c7
python3: Migrate Marvin and smoketests to python3 (#4727)
This PR prepares marvin and tests for python3. it was part of #4479, until nose2 was decided to be abandoned from that PR.

Re-PR of #4543 and #3730 to enable cooperation

Co-authored-by: Daan Hoogland <dahn@onecht.net>
Co-authored-by: Gabriel Beims Bräscher <gabriel@apache.org>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-05-04 23:19:37 +05:30
Rohit Yadav bc24268257 Merge remote-tracking branch 'origin/4.15' 2021-04-05 15:24:57 +05:30
Pearl Dsilva 467a1e7ecf
travis: fix component test failure - persistent networks (#4894)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-04-05 15:24:25 +05:30
Rohit Yadav d4635e3442 Merge remote-tracking branch 'origin/4.15'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-04-01 14:35:01 +05:30
Rakesh 76ba5c62d9
server: Fix displaying public IP address of shared networks (#4675)
Public IP addresses dedicated to one domain should not be accessed
by other domains. Also, root admin should be able to display all
public ip addresses in system.

Currently following issues exist

1. Public IP address assigned to one domain can be accessed by
other sibling domains

If use.system.public.ip is false then child domains should not
see public ip of ROOT domain

Before fix
```
(test1) mgt01 > list publicipaddresses listall=true fordisplay=true allocatedonly=false forvirtualnetwork=true filter=ipaddress,
{
  "count": 59,
  "publicipaddress": [
```

After fix

```
(test) mgt01 > list publicipaddresses listall=true fordisplay=true allocatedonly=false forvirtualnetwork=true filter=ipaddress,
{
  "count": 10,
```
2021-04-01 12:39:01 +05:30
Abhishek Kumar d6e8b53736
vmware: vm migration improvements (#4385)
- Fixes inter-cluster migration of VMs
- Allows migration of stopped VM with disks attached to different and suitable pools
- Improves inter-cluster detached volume migration
- Allows inter-cluster migration (clusters of same Pod) for system VMs, VRs on VMware
- Allows storage migration for stopped system VMs, VRs on VMware within same Pod if StoragePool cluster scopetype

Linked Primate PR: https://github.com/apache/cloudstack-primate/pull/789 [Changes merged in this PR after new UI merge]
Documentation PR: https://github.com/apache/cloudstack-documentation/pull/170

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-02-12 12:41:41 +05:30
Rohit Yadav 6bde1384ff Merge remote-tracking branch 'origin/4.14' into 4.15
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-02-05 16:01:01 +05:30
Abhishek Kumar 29e309839d
test: hardware required for changeserviceoffering (#4650)
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2021-02-05 15:58:45 +05:30
Daan Hoogland fb1e903532 Merge branch '4.14' 2020-12-03 15:11:59 +01:00
Wei Zhou 03e65a1266 vpc vr: plugin nics by this order: public/private/guest 2020-12-03 15:10:43 +01:00
Wei Zhou 627070c5f8 vpc vr: fix Conflicting device id on private gw nic 2020-12-03 15:10:43 +01:00
Rohit Yadav df07e27921 Merge remote-tracking branch 'origin/4.14' 2020-12-01 14:04:53 +05:30
Wei Zhou e8ee8e9fbe
fix failures with test_multiple_nic_support.py (#4495) 2020-11-30 16:21:46 +05:30
Daan Hoogland e9ce381c56 Merge branch '4.14' 2020-11-25 09:04:53 +01:00
Wei Zhou a979ab9050 integration test: update steps 2020-11-25 08:40:16 +01:00
Wei Zhou 6a91b8ace7 integration test: add private gateway in test 2020-11-25 08:40:16 +01:00
Wei Zhou b2d8fffeea integration test: verify public nics state 2020-11-25 08:40:16 +01:00
Wei Zhou 655ed10655 bugfix #2 vpc vr: fix issue if static nat is disabled but still other IP used by lb/pf 2020-11-25 08:40:16 +01:00
Rohit Yadav d3f18ef71c Merge remote-tracking branch 'origin/4.14' 2020-11-20 21:12:20 +05:30
Wei Zhou 75fdb07387
vpc: fix ips on wrong interfaces after rebooting vpc vrs (#4467)
* vpc: fix ips on wrong interfaces after rebooting vpc vrs

* #4467: Rename to updateNicWithDeviceId

* CLSTACK-8923 vr: Force a restart of keepalived if conntrackd is not running or configuration has changed
2020-11-20 21:02:53 +05:30
dahn a348afe91b
default teardown methods with reversed() handling (#4289)
Co-authored-by: Daan Hoogland <dahn@onecht.net>
2020-11-13 07:56:28 +00:00
Pearl Dsilva a73712ec4e
server: Enable sending hypervior host name via metadata - VR and Config Drive (#3976)
Enable sending hypervisor host details via metadata for VR and Config Drive providers

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-07-01 08:44:11 +05:30
Daan Hoogland 06a8ff04b1 Merge release branch 4.13 to master
* 4.13:
  VR: Fix Redundant VRouter guest network on wrong interface (#3847)
2020-02-29 19:56:07 +01:00
Wei Zhou 313e21a0da
VR: Fix Redundant VRouter guest network on wrong interface (#3847) 2020-02-29 19:52:40 +01:00
Wei Zhou 458d3b5b47
Multiple networks support for vms in advanced zone with securit… (#3639) 2020-02-19 14:02:12 +00:00
Wei Zhou fd5bea838b
New feature: Add support to destroy/recover volumes (#3688)
* server: fix resource count of primary storage if some volumes are Expunged but not removed

Steps to reproduce the issue
(1) create a vm and stop it. check resource count of primary storage
(2) download volume. resource count of primary storage is not changed.
(3) expunge the vm, the volume will be Expunged state as there is a volume snapshot on secondary storage. The resource count of primary storage decreased.
(4) update resource count of the account (or domain), the resource count of primary storage is reset to the value in step (2).

* New feature: Add support to destroy/recover volumes

* Add integration test for volume destroy/recover

* marvin: check resource count of more types

* messages translate to JP

* Update messages for CN

* translate message for NL

* fix two issues per Daan's comments

Co-authored-by: Andrija Panic <45762285+andrijapanicsb@users.noreply.github.com>
2020-02-07 11:25:10 +01:00
Anurag Awasthi c0abfce8fa
Health check feature for virtual router (#3575) 2020-01-30 12:39:03 +01:00
Wei Zhou ac581d1546
New feature: Resource count (CPU/RAM) take only running vms into calculation (#3760)
* marvin: check resource count of more types

* New feature: add flag resource.count.running.vms.only to count resource consumption of only running vms

Stopped VMs do not use CPU/RAM actually.
A new global configuration resource.count.running.vms.only is added to determine whether resource (cpu/memory) of only running vms (including Starting/Stopping) will be taken into calculation of resource consumption.

* Add integration test for resource count of only running vms
2020-01-30 10:36:50 +01:00
Rakesh dfd950cab0 Add protocol number support for security group rules (#3736)
Currently while creating ingress/egress rule for a security group,
we can specify only TCP/UDP/ICMP. Sometimes we need to add rules
for different protocol number or rules for all the above three
mentioned protocols.

In this new feature users can specify the protocol number or select
"ALL" option which will apply rules for TCP/UDP/ICMP
2020-01-08 16:13:41 +01:00
Rakesh 482e7ebf9a New feature: Acquire specific public IP for network (#3775)
Currently in cloudstack, when we click on "Acquire New Ip", it will
randomly acquire IP from the pool. With this enhancement, it is
possible to select the IP from the drop down IP list of that network.
Same thing applies for a VPC as well.
2019-12-24 10:08:53 +01:00
nuageci 41275ab0ee ui: Small additional NuageVsp cleanups (#3146) (#3620) 2019-11-11 21:49:52 +05:30
Rohit Yadav 97df52956c
master: travis and trillian smoketests fixes and stabilisation (#3476)
Fix failing test, add more component tests, optimise and refactor test jobs.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-12 17:27:49 +05:30
Rohit Yadav 31e677f0fa Merge remote-tracking branch 'origin/4.11' into 4.12 2019-05-30 11:40:19 +05:30
dahn 910b08f72b server: fix duplicate tag exception as CloudRuntimeException (#3348)
See #3339: a runtime exception is thrown but it should be converted to an error return. Wrapping it in a CloudRuntimeException should do the trick.

Fixes #3339
2019-05-30 11:25:52 +05:30
Dingane Hlaluku 323f791efc IP address acquired with associate ip address is marked as source nat (#3125)
* CLOUDSTACK-4045 added a check for network state when determining whether a new IP should be source NAT. this prevents associated IP's to be marked as source NAT when the network is in allocated state, causing disassociateIpAddress to fail later

* Remove mock object that cause other tests to fail

* Remove underscores from variable types and add documentation for the created method

* Improve exception message to include network name

* Include network UUID with the Exception message and fix failing marvin test

* Rebase against latest master and format AssociateIPAddrCmd class
2019-01-23 10:05:16 -02:00
Rohit Yadav a75cfd4d06 Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-11-13 16:13:52 +05:30