Commit Graph

28593 Commits

Author SHA1 Message Date
Remi Bergsma c9ad8696be Merge pull request #818 from kansal/CLOUDSTACK-8844
Fixed: Network Update from RVR offering to Standalone offering failsProblem: Moving a RVR network offering to standalone makes the status of VR's as UNKNOWN and Redundant Router marked with YES.
Fix: The network's isRedundant was not getting updated.

* pr/818:
  CLOUDSTACK-8844: Network Update from RVR offering to Standalone offering fails - Fixed

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-10-23 16:20:47 +02:00
Remi Bergsma fb0871bdca Merge pull request #936 from ustcweizhou/agent-name
Add agent name in received responseBefore change:

2015-10-15 12:55:34,268 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-d2b917a4) Seq 20-2693152577167557361: Sending  { Cmd , MgmtId: 345051313197, via: 20(KVM015), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.GetRouterAlertsCommand":{"previousAlertTimeStamp":"1970-01-01 00:00:00","accessDetails":{"router.ip":"169.254.3.13"},"wait":0}}] }
2015-10-15 12:55:34,416 DEBUG [c.c.a.t.Request] (AgentManager-Handler-7:null) Seq 20-2693152577167557361: Processing:  { Ans: , MgmtId: 345051313197, via: 20, Ver: v1, Flags: 10, [{"com.cloud.agent.api.GetRouterAlertsAnswer":{"result":true,"wait":0}}] }
2015-10-15 12:55:34,416 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-d2b917a4) Seq 20-2693152577167557361: Received:  { Ans: , MgmtId: 345051313197, via: 20, Ver: v1, Flags: 10, { GetRouterAlertsAnswer } }

After change:

2015-10-15 13:31:04,562 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-cf9a7691) Seq 22-6444651066767179831: Sending  { Cmd , MgmtId: 345051313197, via: 22(node12), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.GetRouterAlertsCommand":{"previousAlertTimeStamp":"1970-01-01 00:00:00","accessDetails":{"router.ip":"169.254.3.89"},"wait":0}}] }
2015-10-15 13:31:04,676 DEBUG [c.c.a.t.Request] (AgentManager-Handler-4:null) Seq 22-6444651066767179831: Processing:  { Ans: , MgmtId: 345051313197, via: 22, Ver: v1, Flags: 10, [{"com.cloud.agent.api.GetRouterAlertsAnswer":{"result":true,"wait":0}}] }
2015-10-15 13:31:04,677 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-cf9a7691) Seq 22-6444651066767179831: Received:  { Ans: , MgmtId: 345051313197, via: 22(node12), Ver: v1, Flags: 10, { GetRouterAlertsAnswer } }

* pr/936:
  Modify description of startvm in DeployVMCmd
  Add agent name in received response

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-10-23 15:53:42 +02:00
Remi Bergsma 2b34df0da4 Merge pull request #799 from sudhansu7/CLOUDSTACK-8815
CLOUDSTACK-8815 : Issues with cloudstack-management init script

When a management server is halted or rebooted, the cloudstack-management init script does not successfully kill the associated java process. There will always be an error about "Cannot find PID file". The script should be reworked so that there are no errors and java is killed correctly.

This appears to be due to the script using the basename of the script:

NAME="$(basename $0)"

To look for the pid file:

if [ -f /var/run/$
{NAME}

.pid ]; then

This does work correctly if the script is run directly (basename will be "cloudstack-management", so "cloudstack-management.pid" file is found). However when a server is halted or rebooted, the script is not run directly but via the symlinks in /etc/rc.d/rc0.d/ and /etc/rc.d/rc0.d/, respectively. The symlinks are named "K20cloudstack-management" so the script looks for a pid file named "K20cloudstack-management.pid" which does not exist.

screenshots:

Before fix:
![screen shot 2015-09-10 at 12 33 08 am](https://cloud.githubusercontent.com/assets/1062642/9791113/207412f0-57f7-11e5-8d21-b7ef383747bd.png)

After fix:
![screen shot 2015-09-10 at 5 46 50 pm](https://cloud.githubusercontent.com/assets/1062642/9791136/31c9ca90-57f7-11e5-98c5-9ad18f25788b.png)

* pr/799:
  CLOUDSTACK-8815 : Issues with cloudstack-management init script

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-10-23 13:57:11 +02:00
Remi Bergsma e15e1f6aca Merge pull request #963 from ustcweizhou/centos7-kvm
CLOUDSTACK-8979: add virtio support for CentOS/RHEL 7 on KVM@borisroman @NuxRo could you test it?

* pr/963:
  CLOUDSTACK-8979: add virtio support for CentOS/RHEL 7 on KVM

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-10-23 13:33:34 +02:00
Wei Zhou 6960b68443 CLOUDSTACK-8979: add virtio support for CentOS/RHEL 7 on KVM 2015-10-23 11:53:43 +02:00
Remi Bergsma cf839bdf0c Merge pull request #952 from michaelandersen/test/site2sitevpn
Improve marvin test site2site VPNAdd optional fields: iprange and fordisplay to Marvin base.py class method Vpn.create
Add optional field: passive to Marvin base.py class method Vpn.createVpnConnection

* pr/952:
  Improve marvin test site2site VPN

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-10-22 12:42:07 +02:00
Remi Bergsma 4146813309 Merge pull request #958 from miguelaferreira/cloud-apidoc-depends-on-cloud-client-ui
cloud-apidoc module depends on cloud-client-uiWhen building the cloud-apidoc module a script called `./build-apidoc.sh` is executed with an argument that points to `../../client/target/cloud-client-ui-${project.version}/WEB-INF/lib`. The script uses that argument to build a Java classpath. However, since the cloud-apidoc module does not explicitly depend on cloud-client-ui, a parallel maven build will often fail because the required jars are not present when needed.

Error message when build fails:
```
Error: Could not find or load main class com.cloud.api.doc.ApiXmlDocWriter
```

This PR makes the dependency explicit.

Resulting build:
```
mvn clean install -T4 -Pdeveloper,systemvm
....
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 09:02 min (Wall Clock)
[INFO] Finished at: 2015-10-20T13:31:30+02:00
[INFO] Final Memory: 98M/1526M
[INFO] ------------------------------------------------------------------------
```

* pr/958:
  cloud-apidoc module depends on cloud-client-ui

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-10-22 11:44:28 +02:00
Miguel Ferreira 9cbd32107d cloud-apidoc module depends on cloud-client-ui 2015-10-22 10:47:34 +02:00
Remi Bergsma e7638c0288 Merge pull request #959 from ekholabs/pr/930-CLOUDSTACK-8933
CLOUDSTACK-8933 SSVm and CPVM do not survive a reboot from APIThis closes PR #930 as well.

I Rebased @bvbharat's PR with latest Master and tested the SSVM/CPVM and the routers: rVPC; VPC; VR; and RVR.

* pr/959:
  CLOUDSTACK-8933 - Improves the the test internals of the SSVM/CPVM
  CLOUDSTACK-8933 - Replace infinite loop by a for loop
  CLOUDSTACK-8933 SSVm and CPVM do not survive a reboot from API

This closes #930

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-10-21 23:14:52 +02:00
Remi Bergsma c7ca952f2b Merge pull request #939 from sanju1010/ctxres
CLOUDSTACK-8960: Remove Citrix Resources from test_data.pyReplace URLs related to templates and ISOs with the ones accessbile to everybody in the community.

I have copied all the templates and ISOs to my webspace at http://people.apache.org/~sanjeev/ so they can be accessible from anywhere.

* pr/939:
  CLOUDSTACK-8960: Remove Citrix Resources from test_data.py Replace URLs related to templates and ISOs with the ones accessbile to everybody in the community

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-10-21 23:06:52 +02:00
Remi Bergsma 39bd17d797 Merge pull request #933 from michaelandersen/fix/loadbalancer-test
CLOUDSTACK-8697 - Refactored & Added Internal LB smoke testAdded 1VPC 2TIER internal lb roundrobin test
Added 1VPC 1TIER internal lb admin stats test
Refactored existing test
Thanks @wilderrodrigues for examples and advice!

* pr/933:
  Improve Internal Loadbalancer Smoke Test

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-10-21 14:39:09 +02:00
Wilder Rodrigues ae6aa2af88 CLOUDSTACK-8933 - Improves the the test internals of the SSVM/CPVM
- Check if the link local IP is the same a dthe one cached in the cmdline config file.
   - SSVM/CPSM can have the same IP after reboot
2015-10-21 14:18:53 +02:00
Remi Bergsma 97619f3259 Merge pull request #951 from thinktwo/patch-4
Corrected sentenceCorrected the sentence as the previous one didn't make sense

* pr/951:
  Corrected sentence

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-10-21 11:42:42 +02:00
Wilder Rodrigues 698171d210 CLOUDSTACK-8933 - Replace infinite loop by a for loop
- The loop will also sleep in between iterations, waiting for a couple of seconds and being multiplied by factor 2 per iteration
2015-10-21 11:10:37 +02:00
Remi Bergsma 260ce75aaa Merge pull request #957 from ekholabs/fix/privategw_acl-CLOUDSTACK-8971
CLOUDSTACK-8971 - Improve the current test_privategw_acls.py   - Read cofiguration from the zone: template id; template id; and domain id
   - Create Account and VPC Offering
   - Add cleanup resources

* pr/957:
  CLOUDSTACK-8971 - Improve the current test_privategw_acls.py

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-10-21 11:09:37 +02:00
Wilder Rodrigues 8708d0dc43 CLOUDSTACK-8971 - Improve the current test_privategw_acls.py
- Read cofiguration from the zone: template id; template id; and domain id
   - Create Account and VPC Offering
   - Add cleanup resources
2015-10-20 15:48:27 +02:00
Jan-Arve Nygård fdb85d13c8 Corrected sentence
Corrected the sentence as the previous one didn't make sense
2015-10-20 13:03:55 +02:00
Michael Andersen e2d13131e7 Improve marvin test site2site VPN
Add optional fields: iprange and fordisplay to Marvin base.py class method Vpn.create
    Add optional field: passive to Marvin base.py class method Vpn.createVpnConnection
2015-10-20 11:01:23 +02:00
Bharat Kumar ac68f2a116 CLOUDSTACK-8933 SSVm and CPVM do not survive a reboot from API 2015-10-20 10:50:14 +02:00
Michael Andersen 3b15890b5f Improve Internal Loadbalancer Smoke Test
Added 1VPC 2TIER internal lb roundrobin test
Added 1VPC 1TIER internal lb admin stats test
Refactored existing test

Removed double license and imports

Changed default hypervisor from xen to kvm

Fixed default network acl to default_allow

Renamed test02
2015-10-20 08:51:12 +02:00
Remi Bergsma 6fe5ae0d60 Merge pull request #940 from ekholabs/fix/rvr__keepalived_restart
CLOUDSTACK-8952 - The redundant routers are facing a race condition due to several KeepaliveD/ConntrackD restartsThis PR fixes the following issues:

* KeepAliveD being restarted for each action performed on the routers
* ConntrackD configuration being copied for each action performed on the routers, causing several restarts
* ACS Management Server relying in the JSON file to report which router is Master/Backup
* Public Interface on both routers are in UP state due to several places checking if the interface is UP/DOWN and trying to do KeepAliveD
* Removing all the sleeps from the test_vpc_redundant.py - those are no longer needed
* When KeepAliveD calls master.py during the election, update the cmdline.json to set the router in Backup mode: the election will take care of changing it afterwards.
* Add LB stats_rules to iptables INPUT chain
* The RVR public interface is set to eth2 instead of eth1 - as in the rVPC. Make sure the check works in both cases

Those fixes make all the routers very stable, with ACL, FW, PF and LB working just fine!

* pr/940:
  CLOUDSTACK-8952 - Make the checkrouter.sh compatible with RVR as well
  CLOUDSTACK-8952 - Make the tests rely on the interface state other than the json file
  CLOUDSTACK-8952 - Reduce retried from 20 to 5
  CLOUDSTACK-8952 - Do not rely in the router state on the json file to report back to ACS
  CLOUDSTACK-8952 - Make the check for master more reliable
  CLOUDSTACK-8952 - Restart dnsmasq everytime the configure.py runs
  CLOUDSTACK-8952 - Make sure the calls to CsFile use the new logic of commit/is_changed methods
  CLOUDSTACK-8952 - Make sure we restart dnsmasq if the configuration file changes
  CLOUDSTACK-8952 - The public interface was comming UP in the Backup router
  CLOUDSTACK-8952 - Do not restart conntrackd unless it's needed
  CLOUDSTACK-8952 - Do not replace the conntrackd config file unless it's needed
  CLOUDSTACK-8952 - Remove the '--vrrp' search criteria form the CsProcess constructor call

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-10-20 08:00:04 +02:00
Remi Bergsma 0827e1f850 Merge pull request #948 from miguelaferreira/marvin-depends-on-apidoc
Marvin module depends on APIdoc moduleWhen building the developer tools, if the build is sequential then the marvin module always gets build after the apidoc module. However, it the build is parallelised (-Tn option in maven) it sometimes happens that maven tries to build the marvin module before building the apidoc module. That difference in the order makes it impossible to build marvin because it depends on the artefacts of the apidoc module.

This PR makes the dependency between marvin an apidoc explicit.

* pr/948:
  Marvin module depends on APIdoc module

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-10-19 18:00:33 +02:00
Wei Zhou 948449e2ca Modify description of startvm in DeployVMCmd 2015-10-19 15:12:02 +02:00
Miguel Ferreira 3bedf31dca Marvin module depends on APIdoc module 2015-10-19 13:17:20 +02:00
Remi Bergsma 186891962e Merge pull request #861 from harikrishna-patnala/CLOUDSTACK-8888
CLOUDSTACK-8888: Xenserver 6.0.2 host stuck in disconnected state aftXenserver 6.0.2 hosts are stuck in disconnected state after CS upgrade to 4.6. This is because xenserver602resource class is not there in 4.6 branch and existing hosts running on v6.0.2 are not handled properly.

* pr/861:
  CLOUDSTACK-8888: Xenserver 6.0.2 host stuck in disconnected state after upgrade to 4.6

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-10-17 21:10:35 +02:00
sanjeev 6956e09a60 CLOUDSTACK-8960: Remove Citrix Resources from test_data.py
Replace URLs related to templates and ISOs with the ones accessbile to everybody in the community
2015-10-17 20:59:45 +05:30
Wilder Rodrigues fb33cb28ab CLOUDSTACK-8952 - Make the checkrouter.sh compatible with RVR as well 2015-10-17 15:52:28 +02:00
Wilder Rodrigues 38d03576d6 CLOUDSTACK-8952 - Make the tests rely on the interface state other than the json file 2015-10-17 14:47:05 +02:00
Wilder Rodrigues 2a747ca735 CLOUDSTACK-8952 - Reduce retried from 20 to 5
- We do not need to retry that much
2015-10-17 12:20:42 +02:00
Wilder Rodrigues 5b3c99031f CLOUDSTACK-8952 - Do not rely in the router state on the json file to report back to ACS
- If we stop/start a router, the state in the file will still say MASTER, when it is actually not
   - Checking the state based on the interface (eth1) state
   - Once master.py is called by keepalived, save the state in the json file to BACKUP just to make sure it's also written there
2015-10-17 12:20:41 +02:00
Wilder Rodrigues 41f4d8b58a CLOUDSTACK-8952 - Make the check for master more reliable
- Do not use the API call because it will read what is in the database, that might not have been updated yet
     * Check the status in the router directly instead
   - Remove all the sleeps
2015-10-17 12:20:41 +02:00
Wilder Rodrigues c7671f3cdd CLOUDSTACK-8952 - Restart dnsmasq everytime the configure.py runs 2015-10-17 12:20:41 +02:00
Wilder Rodrigues 2b286ecd73 CLOUDSTACK-8952 - Make sure the calls to CsFile use the new logic of commit/is_changed methods
- We now have to check if the file changed before commiting. Doesn't make sense to write on disk if there was nono change.
2015-10-17 12:20:40 +02:00
Wilder Rodrigues 1886c4a1b3 CLOUDSTACK-8952 - Make sure we restart dnsmasq if the configuration file changes
- It was working before because the Routers were restarting about 10 times for each operation
     e.g. adding a VM to a network ot acquiring a new IP.
   - Adding stat_rules of internal LB to iptables
     We needed one extra rule in the INPUT chain
2015-10-17 12:20:40 +02:00
Wilder Rodrigues d762dc8579 CLOUDSTACK-8952 - The public interface was comming UP in the Backup router
- There were too many places trying to put the pub interface UP. I centralised it now.
2015-10-17 12:20:40 +02:00
Wilder Rodrigues b4920aa028 CLOUDSTACK-8952 - Do not restart conntrackd unless it's needed
- With the keepalived fixed they should not be needed anymore. So first reducing them drasticaly
   - I am now making a backup of the template file, write to the template file and compare it with the existing configuration
   - The template file is recovered afer the process
   - I also check if the process is running
   - I fixed a bug in the compare method
   - I am now updating the configuration variable once the file content is flushed to disk
2015-10-17 12:20:40 +02:00
Wilder Rodrigues 5a216056b5 CLOUDSTACK-8952 - Do not replace the conntrackd config file unless it's needed
- With the new logic, the file will be replaced when the router starts, becasue the default
     conntrackd config file will be different.
2015-10-17 12:20:39 +02:00
Wilder Rodrigues 08b983fe02 CLOUDSTACK-8952 - Remove the '--vrrp' search criteria form the CsProcess constructor call
- There is no such process, which makes the CsProcess.find return false and restart keepalived all the time.
2015-10-17 12:20:39 +02:00
Rajani Karuturi 9363be3397 Merge pull request #934 from karuturi/CLOUDSTACK-8923
[4.6][BLOCKER] CLOUDSTACK-8923: Create storage network IP range failedThis is a regression from commit 0cd8c06f7d
try-with-resource used at https://github.com/apache/cloudstack/blob/master/server/src/com/cloud/network/StorageNetworkManagerImpl.java#L107 is closing the transaction.
At later point when the transaction is actually committed, it doesn't find the transaction and hence the exception.

testing:
manually tested that create storage network api succeeds.

```
cloudmonkey > create storagenetworkiprange gateway=192.168.200.67 netmask=255.255.255.0 startip=192.168.200.200 endip=192.168.200.202 podid=544e3be0-9bc6-428f-9237-bf777053ab2e
\

accountid = 120b521c-725c-11e5-9244-7278fd1d2943
cmd = org.apache.cloudstack.api.command.admin.network.CreateStorageNetworkIpRangeCmd
created = 2015-10-15T09:31:22+0000
jobid = c86c4968-bc6e-4255-aa11-39221eb2277f
jobprocstatus = 0
jobresult:
storagenetworkiprange:
id = df491e87-7a00-4b5e-9bfe-e811fe78c9cd
endip = 192.168.200.202
gateway = 192.168.200.67
netmask = 255.255.255.0
startip = 192.168.200.200
jobresultcode = 0
jobresulttype = object
jobstatus = 1
userid = 120b6414-725c-11e5-9244-7278fd1d2943
```

```
cloudmonkey > list storagenetworkiprange
count = 1
storagenetworkiprange:
id = df491e87-7a00-4b5e-9bfe-e811fe78c9cd
endip = 192.168.200.202
gateway = 192.168.200.67
netmask = 255.255.255.0
networkid = a0f7faf9-83a1-4af5-b420-d8a78f2f59d8
podid = 544e3be0-9bc6-428f-9237-bf777053ab2e
startip = 192.168.200.200
zoneid = 34b9a0a1-758c-442b-9bc6-7bce03157d4c
```

* pr/934:
  CLOUDSTACK-8923: Create storage network IP range failed

Signed-off-by: Rajani Karuturi <rajani.karuturi@citrix.com>
2015-10-16 17:24:38 +05:30
Wei Zhou 358936ffb7 Add agent name in received response 2015-10-16 11:43:25 +02:00
Rajani Karuturi 79b7391c22 CLOUDSTACK-8923: Create storage network IP range failed
transaction.getConnection() was wrapped in try-with-resource.
This caused transaction to get closed even before its committed.
2015-10-16 09:42:48 +05:30
Remi Bergsma 2ce5a0c964 Merge pull request #931 from ekholabs/fix/loadbalancer
CLOUDSTACK-8947 - Load Balancer not working with Isolated NetworksThis PR fixes the Load Balance feature by adding iptables rules for the public IP and port of the LB.

In order to cover the changes, I improved and executed the smoke/test_loadbalance.py. In addition, I also executed many other tests to make sure the main network/VM functionalities are working as expected.

Test report will follow.

* pr/931:
  CLOUDSTACK-8947 - Do not rely on the machine hostname to verify the test
  CLOUDSTACK-8947 - Fail fast!
  CLOUDSTACK-8947 - Adding some logging to better understand whay is happening with the Processes
  CLOUDSTACK-8947 - Adding some logging to better understand what's happening with the rules
  CLOUDSTACK-8947 - Configure the firewall when the load balancer is setup
  CLOUDSTACK-8947 - Avoid multiple entries in the FW_EGRESS_RULES table
  CLOUDSTACK-8947 - Open the input chain to IP when loadbalancer is configured
  CLOUDSTACK-8947 - FW_EGRESS should be added only to filter table

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-10-14 18:30:32 +02:00
Wilder Rodrigues bb3d1cde60 CLOUDSTACK-8947 - Do not rely on the machine hostname to verify the test
- The machine hostname might be different depending on the template. So do not rely on it.
   - Using the "uname" command instead.
2015-10-14 07:44:39 +02:00
Wilder Rodrigues 384b6c7cd4 CLOUDSTACK-8947 - Fail fast!
- If we canno SSH after 5 retries, it means it's not working. Do not wait for 60 attempts
2015-10-14 07:44:39 +02:00
Wilder Rodrigues f35a16c19e CLOUDSTACK-8947 - Adding some logging to better understand whay is happening with the Processes 2015-10-14 07:44:39 +02:00
Wilder Rodrigues 80b51a7972 CLOUDSTACK-8947 - Adding some logging to better understand what's happening with the rules 2015-10-14 07:44:39 +02:00
Wilder Rodrigues 59bd935f3e CLOUDSTACK-8947 - Configure the firewall when the load balancer is setup
- Only restart HAproxy if it's not running yet
2015-10-14 07:44:38 +02:00
Wilder Rodrigues 4a177031b0 CLOUDSTACK-8947 - Avoid multiple entries in the FW_EGRESS_RULES table 2015-10-14 07:44:38 +02:00
Wilder Rodrigues 052c0dc4c9 CLOUDSTACK-8947 - Open the input chain to IP when loadbalancer is configured
- Also remove the chain rule when it is removed.
2015-10-14 07:44:38 +02:00
Wilder Rodrigues cbd6464b88 CLOUDSTACK-8947 - FW_EGRESS should be added only to filter table 2015-10-14 07:44:37 +02:00