From 2660a6b7a7f226ab757d2175222db62571813120 Mon Sep 17 00:00:00 2001 From: Alena Prokharchyk Date: Thu, 9 May 2013 11:05:54 -0700 Subject: [PATCH 01/14] CLOUDSTACK-747: Internal LB between VPC tiers support Squashed commit of the following: commit def0861d5a12202260cb6672c12d77075a0de26e Author: Alena Prokharchyk Date: Thu May 9 10:53:09 2013 -0700 Inernallb: added internalLbVm to the list of network elements for nonoss build commit 56d94fc074db52ef00ace1703081a342dfb63db0 Merge: d828c15 8f9a42e Author: Alena Prokharchyk Date: Thu May 9 09:51:36 2013 -0700 Merge branch 'master' into internallb1 Conflicts: plugins/network-elements/netscaler/src/com/cloud/network/element/NetscalerElement.java server/src/com/cloud/network/vpc/VpcManagerImpl.java commit d828c154fd05fbfeb9c142b27e3d26e7dd755c77 Author: Alena Prokharchyk Date: Wed May 8 14:58:22 2013 -0700 internallb: Fixed nonoss build commit 1b8a6986a6fd1808ba7285589f9a007b52feb7e5 Merge: 9e74fa9 738d35a Author: Alena Prokharchyk Date: Wed May 8 13:20:07 2013 -0700 Merge branch 'master' into internallb1 Conflicts: api/src/com/cloud/async/AsyncJob.java api/src/com/cloud/network/NetworkModel.java api/src/com/cloud/network/rules/LoadBalancerContainer.java api/src/org/apache/cloudstack/api/BaseCmd.java api/src/org/apache/cloudstack/api/ResponseGenerator.java api/src/org/apache/cloudstack/network/lb/ApplicationLoadBalancerContainer.java client/tomcatconf/commands.properties.in engine/schema/src/com/cloud/network/dao/LoadBalancerDaoImpl.java server/src/com/cloud/api/ApiResponseHelper.java server/src/com/cloud/network/NetworkManagerImpl.java server/src/com/cloud/network/NetworkModelImpl.java server/src/com/cloud/network/NetworkServiceImpl.java server/src/com/cloud/server/ManagementServerImpl.java server/test/com/cloud/network/MockNetworkModelImpl.java server/test/com/cloud/vpc/MockNetworkManagerImpl.java server/test/com/cloud/vpc/MockNetworkModelImpl.java server/test/resources/appLoadBalancer.xml setup/db/db/schema-410to420.sql test/integration/component/test_multiple_ip_ranges.py test/integration/smoke/test_guest_vlan_range.py tools/marvin/marvin/integration/lib/base.py commit 9e74fa94067997f0ae6d469f3564df602dd274a6 Author: Prasanna Santhanam Date: Thu May 9 00:11:40 2013 +0530 marvin changes for internallbvm provider - changed the simulator context to load the internallb bean - fixed deployDataCenter to use the additional provider by default - fixed the sandbox script and the setup script for simulator checkin tests Signed-off-by: Prasanna Santhanam commit 0a2d49301b170171ebe8d6646236854f488d06ba Author: Jessica Wang Date: Tue May 7 15:50:41 2013 -0700 Internal LB - UI - zone wizard - advanced zone - enable internal LB element, enable internal LB provider. commit 43e1667f90b01e2b4095009b997b3b2665087fb4 Author: Alena Prokharchyk Date: Tue May 7 13:26:53 2013 -0700 Moved all DAOs and VOs to the cloud-engine-schema project commit 2fd94c8bbe24da86e6bd004ee8165e08038a3e1d Author: Radhika PC Date: Tue May 7 18:26:08 2013 +0530 CLOUDSTACK-893 api commit 12b64d6c00940a4b5c4aa059f2c36ae7cc1b79bd Author: Prasanna Santhanam Date: Tue May 7 17:06:15 2013 +0530 Fixing the gmaven configuration for marvin/pom.xml For the marvin checkin test custom properties had a typo when run for the *nix environment. Signed-off-by: Prasanna Santhanam commit 1e4274dd9f55a6eea7121d50e361babcafe70725 Author: Prasanna Santhanam Date: Tue May 7 15:28:43 2013 +0530 Renaming TesDedicateVlanRange -> TestDedicateVlanRange Signed-off-by: Prasanna Santhanam commit 9264ac526f854c980afb669631177c99e4bbbed7 Author: Prasanna Santhanam Date: Tue May 7 15:28:20 2013 +0530 Adding isolation method to the zone creation of marvin Signed-off-by: Prasanna Santhanam commit 3a0dc67de0285fffa7739b25286df71f92470ddf Author: Prasanna Santhanam Date: Tue May 7 15:27:26 2013 +0530 adding ACL for dedicateGuestVlanRange set of APIs Signed-off-by: Prasanna Santhanam commit 79f6e11368d4b6227e9055bb58cce8e5b9872cfe Author: Prasanna Santhanam Date: Tue May 7 15:26:48 2013 +0530 add debug logs when access checkers fail to find API When the access checkers fail for api discovery, we fail silently. instead record a debug message. Signed-off-by: Prasanna Santhanam commit 92cb7d3e0f3ef052104f698e005ab8dd243f9885 Author: Hugo Trippaers Date: Tue May 7 11:44:23 2013 +0200 commit e0d8f01ecd92a1a7b74558d02a28be3b2f20a10d enabled all tests. Fix AWSAPI build by removing broken tests and excluding failing tests. commit 4a391464c7359689915c4230dfa07d728acfb752 Author: Hugo Trippaers Date: Tue May 7 10:57:23 2013 +0200 commit e0d8f01ecd92a1a7b74558d02a28be3b2f20a10d enabled all tests, but the tests in vmware-base are horribly broken and will not compile with the current CloudStack. Removing the tests to fix the nonoss build and they are so broken they should be rewritten from scratch anyway. commit 2ca03a851360c733ddacbe6a5490b6812dfc6bec Author: Prasanna Santhanam Date: Tue May 7 13:51:34 2013 +0530 moving test data to top level dictionary Signed-off-by: Prasanna Santhanam commit ceaa4e1b0d77ccb811a7e1f56ae212034b3f4a90 Author: Prasanna Santhanam Date: Tue May 7 13:50:56 2013 +0530 Adding tracelogs to the API discovery service Signed-off-by: Prasanna Santhanam commit a3f5f01c7e78e045fdc1302ab484282b0d43e014 Author: Prasanna Santhanam Date: Tue May 7 13:50:12 2013 +0530 dedicateGuestVlanRange is admin only API Adding ACL for the dedicateGuestVlanRange API. Signed-off-by: Prasanna Santhanam commit 1c4c80fe91c7aeec0689fa9a06ae80d2bdf1b0c0 Author: Hugo Trippaers Date: Tue May 7 09:58:16 2013 +0200 Recent pom switcharoo caused the client to lose the dependecy on console-proxy, which it needs to include the systemvm commit 8103f3c386a8f57c520388ea73965a4cb7f15b91 Author: Radhika PC Date: Tue May 7 11:30:17 2013 +0530 CLOUDSTACK-893 first cut commit 67d0411d739417aae4910251e86981e6d1340da0 Author: Dave Cahill Date: Fri Apr 19 17:31:44 2013 +0900 Add docs for MidoNet networking plugin [CLOUDSTACK-996] Signed-off-by: Dave Cahill commit c745e6d28e63ef1cacdf18a98ce47d162d260978 Author: Alex Huang Date: Mon May 6 16:34:03 2013 -0700 Fixed up the simulator to run with windows paths in cygwin commit be91c037021ad17f16b22599acbdd46062a665a8 Author: Alex Huang Date: Mon May 6 10:34:22 2013 -0700 Moved over the VLAN daos commit dcc09f8472d4c94cebae5b430b7eb983037a9935 Author: Alex Huang Date: Mon May 6 06:31:47 2013 -0700 Missing file and updated .gitignore commit e9953cd1a8b1247ce569d22ffa820f51d072c841 Author: Alex Huang Date: Mon May 6 06:29:59 2013 -0700 Fixed up unit testing to use only an in class TestConfiguration commit 67275714035ea2a55c11ee7acf1e915598670845 Author: Alex Huang Date: Thu May 2 15:20:49 2013 -0700 Fixed an incorrect unit test for affinity group. Removed some useless pom.xml. commit d015fb352023a5d2ec07b4d66612836e0a928679 Author: Alex Huang Date: Thu May 2 06:50:38 2013 -0700 Moved most of the VOs and DAOs from server package into engine-schema as well commit 77547a58df5f529031516273015bb770717c5b88 Author: Alex Huang Date: Wed May 1 15:15:57 2013 -0700 Removed files that are no longer used and further separated out the files in the core project commit 345f3d34828a9b573971104cbb6132a777a58ed4 Author: Alex Huang Date: Wed May 1 05:46:15 2013 -0700 Moved agent commands to core and out of api commit e91ca00571ddf078150165a33cd3cbcee5564fe2 Author: Alex Huang Date: Tue Apr 30 19:06:20 2013 -0700 Removed framework-api which is completely useless. Changed framework-ipc to reference gson itself. Move VOs into engine-schema. commit f64564e49001ee0f52120a10665979887fd0ebd1 Author: Dennis Lawler Date: Mon Apr 29 15:10:09 2013 -0700 Removing filterwin2k option Signed-off-by: Chiradeep Vittal commit 944a7ea5d6e6728903074ec18754ce2041aaf7a3 Author: Wei Zhou Date: Mon May 6 20:57:02 2013 +0100 CLOUDSTACK-2319: fix "unable to add egress rules" in SecurityGroup Signed-off-by: Chip Childers commit ff7f8ba3623927b6cef2deb0069e7f839d933fd5 Author: Wei Zhou Date: Mon May 6 20:56:06 2013 +0100 CLOUDSTACK-2322: update network.gateway to fix deployVm error on SharedNetwork after ipv6 support Signed-off-by: Chip Childers commit a153373c7eade440c707bfaeeb9831f8179ab51a Author: Alena Prokharchyk Date: Mon May 6 11:45:19 2013 -0700 CLOUDSTACK-129: added new API - listNetworkIsolationMethods - for displaying isolation methods supported by the cloudStack commit 46f59cd49effc43ffea610cc5b7cab90f601b011 Author: Prasanna Santhanam Date: Mon May 6 20:39:36 2013 +0530 Fixed the incorrect assertion in noncontiguous_vlan test The assertion fails if the VLAN is found in which case find returns a positive number. So here the assertion should infact assert < 0 result. Signed-off-by: Prasanna Santhanam commit bd1dcc10b381b33ab5503f42075999365197168b Author: Prasanna Santhanam Date: Mon May 6 19:53:42 2013 +0530 Affinity Groups requires multiple storage pools Fixing the affinity group test which would fail to find the appropriate storage pool to satisfy the anti-affinity group of the second VM Signed-off-by: Prasanna Santhanam commit 7f853cdb8f8074408205f38c5e63c2dce187b40b Author: Prasanna Santhanam Date: Mon May 6 19:16:15 2013 +0530 fixing double calls to VM deploy This fixes regression introduced in commit 2f40a90c that made duplicate calls to deployVirtualMachine. Signed-off-by: Prasanna Santhanam commit 646e810fcf0f2c0c3fac02a37eb4301fd913b008 Author: Prasanna Santhanam Date: Mon May 6 18:51:53 2013 +0530 fixing wildcard imports Signed-off-by: Prasanna Santhanam commit b29b6e8525ed367cb6f481c7eb116abe1fb847a2 Author: Hugo Trippaers Date: Fri May 3 14:03:53 2013 +0200 BigSwitch should only respond if it is the actual provider on the network. This fixes an NPE during the release call. commit 6fcc9b009b5609756fd5195847557b56f9642ecd Author: Hugo Trippaers Date: Thu May 2 17:38:27 2013 +0200 Prevent Nicira NVP tags from exceeding the 40 character limit. commit d8e61a1c0a02cef67377f1fd1f638d77652edecf Author: Sanjay Tripathi Date: Thu May 2 13:45:39 2013 +0530 CLOUDSTACK-2188 : Parsing error with Download Monitor while checking the health of downloaded templates Signed-off-by: Sateesh Chodapuneedi commit 5b6e1140f90b4625d02b9a0bb035342139c1b6f4 Author: Sebastien Goasguen Date: Mon May 6 05:35:58 2013 -0400 CLOUDSTACK-2339: Adding libcloud example commit 7be62d2374ef5910728627b583ac33498ad7fdce Author: Sebastien Goasguen Date: Mon May 6 05:35:19 2013 -0400 CLOUDSTACK-2338: Adding example of how to sign api requests in python commit 156fd689026ca23e583fa32e5013183e13086a57 Author: Talluri Date: Fri May 3 23:11:56 2013 +0530 CLOUDSTACK-2323: fix test scripts to conform with library changes Signed-off-by: Prasanna Santhanam commit 33ff5e91079b262562b17701d5266bcfd2674ffd Author: sanjeevneelarapu Date: Tue Apr 30 20:09:46 2013 +0530 CLOUDSTACK-702: Tests for Multiple IP Ranges 1.Deleting IP Range from the existing CIDR 2.Add non-contiguous guest IP range in new CIDR 3.Add overlapped guest IP range in existing CIDR Signed-off-by: sanjeevneelarapu Signed-off-by: Prasanna Santhanam commit 33059d1f66fdc7d925f6b087a97912b8bc4598d5 Author: Pranav Saxena Date: Mon May 6 14:08:40 2013 +0530 scaleUp VM response change corresponding UI changes commit 92e18d81060a9e766e1cfce3f41bbd6ddd6d5ff5 Author: Pranav Saxena Date: Mon May 6 13:45:32 2013 +0530 CLOUDSTACK-2337:Resize button available for root/domain admin and normal users commit d5cd3f7e006d688d052f7fe797180753dae989e7 Author: Likitha Shetty Date: Wed May 1 13:47:26 2013 +0530 Dedicate guest vlan range to account commit 12c79c8377d45df2ccec217bbcb9023840a2c064 Author: Pranav Saxena Date: Sun May 5 12:02:32 2013 +0530 scale up virtual machine response change in the backend commit 94bac276228fb2ede08de063e1b789d917c8425f Author: Rohit Yadav Date: Sun May 5 09:58:04 2013 +0530 appliance: Upgrade systemvm appliance from rc1 to Debian7 GA Signed-off-by: Rohit Yadav commit c598bb0038750c18223115026774c56de1366696 Author: Isaac Chiang Date: Sun May 5 01:57:35 2013 +0530 CLOUDSTACK-2076:Listview widget infinte scrolling error commit d0615ea9a1560b7330066544f5ddf4d5c2713f70 Author: Pranav Saxena Date: Sun May 5 01:15:14 2013 +0530 CLOUDSTACK-2274:Detail view loading problem when deleting a zone commit 733b513c3ac10dadf33f187d29651494197678b4 Author: Isaac Chiang Date: Sat May 4 13:41:02 2013 +0530 CLOUDSTACK-2160:Refresh button functionality for security groups and statistics tab commit 418d75d7a4dd4f9785846e3ecbe9f0135d751fe3 Author: Marcus Sorensen Date: Fri May 3 14:09:52 2013 -0600 Summary: Release old DHCP entries Detail: Refresh dnsmasq with updated entries live, no outage BUG-ID: CLOUDSTACK-2299 Submitted-by: Dennis Lawler Signed-off-by: Marcus Sorensen 1367611792 -0600 commit b3dce6457739c0f737ed8f2f61ad108bfee13eba Author: Kelven Yang Date: Tue May 7 10:57:19 2013 -0700 fix unitest commit b17885f0f68affe060a95e3f264b9ac5e3a98a4b Author: Alena Prokharchyk Date: Tue May 7 09:55:47 2013 -0700 InternalLb: some fixes to the unittest commit 1cff609347eed3fb3a5e2e3a76b2f36404ea0b81 Merge: 053e184 a3a5c13 Author: Alena Prokharchyk Date: Fri May 3 11:23:08 2013 -0700 Merge branch 'master' into internallb1 Conflicts: server/src/com/cloud/network/NetworkModelImpl.java server/src/com/cloud/network/NetworkServiceImpl.java server/src/com/cloud/server/ManagementServerImpl.java server/test/com/cloud/network/MockNetworkManagerImpl.java server/test/com/cloud/vpc/MockNetworkManagerImpl.java commit 053e18454d105e9785768123259bacc48383bcea Author: Alena Prokharchyk Date: Fri May 3 11:07:07 2013 -0700 InternalLB: marvin integration test for internal lb feature commit 2e8e2f98f59eecfc37a2645bd353de7f592d8149 Author: Alena Prokharchyk Date: Wed May 1 13:53:20 2013 -0700 InternalLB: don't allow to upgrade the network from the network offering with internal LB to the offering with public LB, and vice versa commit c773d204c8e3b4715b3466f732bdff3100f58cfe Author: Chiradeep Vittal Date: Wed May 1 13:21:52 2013 -0700 Internal LB: if we detect that we are inside an internal lb vm, call out to the ilb script to perform LB configuration Signed-off-by: Chiradeep Vittal commit 8c8845bf77f1258e54998f509df6163df82ce24e Merge: 7e95545 471ca30 Author: Alena Prokharchyk Date: Wed May 1 10:14:06 2013 -0700 Merge branch 'master' into internallb1 commit 7e9554596f3d0e1d68ada880c2c17c01a526b0f5 Author: Alena Prokharchyk Date: Wed May 1 10:01:16 2013 -0700 InternalLb: boot args parameters cleanup for the internal lb vm commit b7cf8700749893abcac1d3a3d20ec32c4cb5d37f Author: Alena Prokharchyk Date: Tue Apr 30 10:31:28 2013 -0700 InternalLb: more unittests for internal lb element commit 63bb98ebe2fa8279138f8b32aa4264b547726224 Author: Chiradeep Vittal Date: Mon Apr 29 18:29:25 2013 -0700 allow ssh on eth1 Signed-off-by: Chiradeep Vittal commit ca1c313c2901a2ce1e7e6e3ce3a566bbf4c817b4 Author: Alena Prokharchyk Date: Mon Apr 29 17:58:45 2013 -0700 InternalLb: DB upgrade - update existing physical networks with InternalLbVm provider commit ed50caa01c12320c4fa5ff7d4e0818840a6c764d Author: Alena Prokharchyk Date: Mon Apr 29 17:48:36 2013 -0700 InternalLbVM: handle the scenario when sourceIpAddress is not passed when create internal Lb rule commit 4c22c911a9ca5d7e05e9eaf0e971cbe56345fa52 Author: Chiradeep Vittal Date: Mon Apr 29 15:56:00 2013 -0700 backend support for Internal LB Signed-off-by: Chiradeep Vittal commit 7b24a7640c53e5b5ab4493de309b43714f4646c6 Merge: 440e848 a0dbf89 Author: Alena Prokharchyk Date: Mon Apr 29 15:49:48 2013 -0700 Merge branch 'master' into internallb1 Conflicts: server/src/com/cloud/api/ApiResponseHelper.java server/src/com/cloud/api/query/QueryManagerImpl.java server/src/com/cloud/configuration/ConfigurationManagerImpl.java server/src/com/cloud/network/NetworkManagerImpl.java server/src/com/cloud/network/firewall/FirewallManagerImpl.java setup/db/db/schema-410to420.sql commit 440e8484d66e3e0366b6d61f930fa13a07d0ce5f Author: Alena Prokharchyk Date: Mon Apr 29 13:27:06 2013 -0700 InternalLB: unittests for InternalLoadBalancerVMManager commit 63babe4b7e8baa1027ac66a7d0b505caafc36d5e Author: Alena Prokharchyk Date: Fri Apr 26 13:44:01 2013 -0700 InternalLb: 1) Added unittests for InternalLoadBalancerVMService 2) Added unittests for InternalLoadBalancerElementService commit 4f9c47ce54915f9c9eb8042936f3a1b19553399d Author: Alena Prokharchyk Date: Thu Apr 25 14:46:42 2013 -0700 InternalLb: create internal lb vm specific service offering commit 408ee59d1fe64ce5d6a5e57b76e25cc505fda5dc Author: Alena Prokharchyk Date: Wed Apr 24 17:08:08 2013 -0700 Internallb: new set of Web services APIs for managing Internal LB VMs commit 7680e1cc10b4e48be78668b31ba1af44d1528a67 Author: Alena Prokharchyk Date: Wed Apr 24 17:08:08 2013 -0700 Internallb: 1) InternalLb and PublicLb can't be enabled on the same network offering 2) Can have internalLb only on VPC tier commit d73ca7ef73ce8f16cd355e48a96c0fc35b037fed Author: Alena Prokharchyk Date: Wed Apr 24 13:02:32 2013 -0700 InternalLb: 1) fixed the bug when the guest nic on internal lb vm wasnt set to be default 2) Don't send the rules to the internal lb vm if its in Stopped state commit ca2fc30655eace01857181ac50789a6632f51d29 Merge: 8057567 04a2b2d Author: Alena Prokharchyk Date: Tue Apr 23 16:56:11 2013 -0700 Merge branch 'master' into internallb1 Conflicts: server/src/com/cloud/network/vpc/VpcManagerImpl.java server/src/com/cloud/server/ManagementServerImpl.java setup/db/db/schema-410to420.sql commit 8057567aaab5000b2f6d3aec8ace65631092adc1 Author: Alena Prokharchyk Date: Tue Apr 23 13:15:36 2013 -0700 Internallb: more unittests for ApplicationLoadBalancerService commit 35c0273b857073a7018fb1a59a33f43f6f112a9c Author: Alena Prokharchyk Date: Fri Apr 19 16:17:45 2013 -0700 InternalLb: unittests for ApplicationLoadBalancerService commit 69b23f700348a56f97bc2118166799332a47b3a5 Author: Alena Prokharchyk Date: Thu Apr 18 14:01:10 2013 -0700 InternalLb: create/configure/listInternalLoadBalancerElement - fixes to the API response commit a3321ce617eab795288ab75357345d45794db93d Author: Alena Prokharchyk Date: Thu Apr 18 13:31:16 2013 -0700 Internal LB : renamed the classes responsible for managing internal lb elements. Now the names are InternalLoadBalancerVMManager, InternalLoadBalancerVMService and InternalLoadBalancerVMManagerImpl commit 2baf7c365c4847ae49cd90ee25d4e3d7d346464d Author: Alena Prokharchyk Date: Thu Apr 18 10:02:17 2013 -0700 Internallb: verify requested IP for LB rule (if specified) against guest network cidr commit 0cfe96bd00a79ccb9ddebed760c6dde257599074 Merge: 501f2ff 11162f5 Author: Alena Prokharchyk Date: Wed Apr 17 15:41:51 2013 -0700 Merge branch 'master' into internallb1 Conflicts: api/src/com/cloud/network/IpAddress.java server/src/com/cloud/network/NetworkServiceImpl.java setup/db/db/schema-410to420.sql commit 501f2ffa0b07e80c17e4d6f7a73721d034c8e46a Author: Alena Prokharchyk Date: Tue Apr 16 17:03:50 2013 -0700 InternalLb: validate source ip network as a part of LB rule creation commit 4d9a7dfd85f010918749dfe3166d75fe4db68d90 Author: Alena Prokharchyk Date: Tue Apr 16 16:03:32 2013 -0700 InternalLB: in VPC, restrict public LB to one tier only. Internal LB can be supported on multiple tiers commit 8689bf9eb38a416997d5ae4d326c51b57b57553e Author: Alena Prokharchyk Date: Tue Apr 16 15:59:56 2013 -0700 Internal LB: added internal lb vm to the list of supported providers in VPC default offering commit b7709b89ff94a7c1eeb0e6dff0eca6220a0aed25 Author: Alena Prokharchyk Date: Tue Apr 16 11:04:22 2013 -0700 Internal Lb: added 2 boolean fields - internal_lb and public_lb - to the network offering. Define if internal or public LB service is supported. In the current release it's either one or another; in the future releases we might support both on the same network commit 014689e45eeaedd9b695ec20dfb2db631f433e14 Merge: b3b16ba 90e8158 Author: Alena Prokharchyk Date: Tue Apr 16 09:55:45 2013 -0700 Merge branch 'master' into internallb1 Conflicts: api/src/com/cloud/network/Network.java plugins/network-elements/f5/src/com/cloud/network/element/F5ExternalLoadBalancerElement.java server/src/com/cloud/network/NetworkManagerImpl.java server/src/com/cloud/network/NetworkServiceImpl.java server/test/com/cloud/network/MockNetworkManagerImpl.java server/test/com/cloud/vpc/MockNetworkManagerImpl.java setup/db/db/schema-410to420.sql commit b3b16bae48976c027e3439f2e0a84cfb2cb72166 Author: Alena Prokharchyk Date: Mon Apr 15 17:00:31 2013 -0700 InternalLB: allow to specify more than one provider for the LB service when create network offering as diff providers can support internal and public LB for the same network commit a4fc1d7d65191da6a49bce9fa0a64ce6e46ac85d Author: Alena Prokharchyk Date: Mon Apr 15 13:45:25 2013 -0700 Internal LB: 1) Added network_offering_details DB table and corresponding VO/DAO objects. Change createNetworkOffering web services api to accept the map of key/value pairs as details. 2) Allow to have multiple providers for the same service for the network to support the case when LB service can have separate providers for Internal LB and Public LB commit 4530cebf2bef515f86fa29b3798d51e81089f92f Author: Alena Prokharchyk Date: Mon Apr 15 10:06:10 2013 -0700 InternalLbVm: support for start/stop Internal lb vm commit ae69f0ae5612834b8bc911bd074cfb050c0b164c Author: Alena Prokharchyk Date: Fri Apr 12 17:25:02 2013 -0700 InternalLb: fixed prepare() in InternalLbElement - have to prepare nics of User vms only commit 888a83c22111f3792850e6bb6235d30a93408d1d Author: Alena Prokharchyk Date: Fri Apr 12 15:58:27 2013 -0700 InternalLb: Start/deploy internal LB vms for the existing LB rules as a part of network implement commit 3b41d5bee1ac04d3e9cf837a2be4398b7597d1a0 Merge: bb73531 4b1a9f1 Author: Alena Prokharchyk Date: Fri Apr 12 15:32:40 2013 -0700 Merge branch 'master' into internallb1 Conflicts: server/src/com/cloud/network/NetworkModelImpl.java server/src/com/cloud/server/ManagementServerImpl.java commit bb73531fed72cc81012624fd3bae5ec22d23daa2 Author: Alena Prokharchyk Date: Fri Apr 12 15:13:55 2013 -0700 Internal Lb: list internal lb vms as a part of listRouters response. Introduced new parameter - role (can be virtual_router or internal_lb_vm) commit c113ea184b575ccae5d2be411c5bbc345473e35e Author: Alena Prokharchyk Date: Fri Apr 12 12:00:09 2013 -0700 Add Internal Lb Provider/Element as a part of adding a physical network commit 78c9db79dae3bc38699050b05baae32fd02204f6 Author: Alena Prokharchyk Date: Fri Apr 12 10:05:28 2013 -0700 InternalLbVm: destroy the internal lb vm when the last rule for the ip is being revoked commit af6201257b28a0d9177330c8c76b0ec979725870 Author: Alena Prokharchyk Date: Thu Apr 11 16:55:37 2013 -0700 Internal LB: fixed some bugs in internal lb vm startup process commit 1db240c2b668e6cbbea4e09979c34d84fe54a624 Author: Alena Prokharchyk Date: Thu Apr 11 14:23:55 2013 -0700 InternalLb: fixed searchCriteria constructor in ApplicationLoadBalancerDaoImpl commit 3795048fcc68ef23c24d82c9064946dfc082b1b6 Merge: f4c2b53 5f8a278 Author: Alena Prokharchyk Date: Thu Apr 11 14:09:31 2013 -0700 Merge branch 'master' into internallb1 Conflicts: api/src/com/cloud/async/AsyncJob.java api/src/org/apache/cloudstack/api/ApiConstants.java api/src/org/apache/cloudstack/api/BaseCmd.java api/src/org/apache/cloudstack/api/ResponseGenerator.java client/tomcatconf/commands.properties.in server/src/com/cloud/api/ApiDBUtils.java server/src/com/cloud/api/ApiResponseHelper.java server/src/com/cloud/server/ManagementServerImpl.java setup/db/db/schema-410to420.sql commit f4c2b53c218b2d11d510b9acc6baeb0d62042e5c Author: Alena Prokharchyk Date: Thu Apr 11 13:46:47 2013 -0700 InternalLB: modified InternalLbElement to start the Internal Lb vm for each ip address (if not already started) commit 76a4b1cf81610d931d6098ca94fa509c9ba2e274 Author: Alena Prokharchyk Date: Thu Apr 11 11:41:47 2013 -0700 InternalLB: added logic for acquiring guest ip address for the Internal LB rule commit 915e39fbaa9a6b6898e2aa13cbf8ef60a7b9d70e Author: Alena Prokharchyk Date: Thu Apr 11 10:19:18 2013 -0700 Removed unused methods doing ipAllocation from GuestNetworkGuru and NetworkServiceImpl. The correct method is located in NetworkModelImpl commit 3f2a62c7f62d23da23f65ff3842049243ef268e3 Merge: 20beb7a a0b5ebc Author: Alena Prokharchyk Date: Wed Apr 10 17:16:07 2013 -0700 Merge branch 'master' into internallb1 commit 20beb7a16c1a592cd7500148ea8988fe119d34fb Author: Alena Prokharchyk Date: Wed Apr 10 15:28:12 2013 -0700 Internal LB: applyLoadBalancerRules - put not null check for sourceIpAddressId (can be null when Schema is not Public) commit 87e5f5b9a6b006893ee30662fc06ea430e7b70dc Author: Alena Prokharchyk Date: Wed Apr 10 15:17:58 2013 -0700 Internal LB: intermediate checkin - added InternalLBAppliance manager and managerImpl commit 53b9c0d142bd2b99dd00c17987dcf5dc54e3ef56 Author: Alena Prokharchyk Date: Tue Apr 9 17:55:10 2013 -0700 Internal LB: added @Since to API docs for Internal LB related commands commit 867b305ea852dcbe224645dc4e5ffb950d8db8f7 Author: Alena Prokharchyk Date: Tue Apr 9 17:51:19 2013 -0700 Internal LB: Made InternalLbElement to extend the ip deployer as the LB service implements IPDeployerRequester commit 7b9af2809410132db451e48b7b829bf019512a92 Author: Alena Prokharchyk Date: Tue Apr 9 15:45:30 2013 -0700 InternalLb: new set of Web Services APIs to add InternalLB as a network element to the cloudStack (the element is packaged as an independent plugin). New APIs: 1) configureInternalLoadBalancerElement 2) createInternalLoadBalancerElement 3) listInternalLoadBalancerElements commit 039e303d4aec5618cc3914ee1a581d2f7b926e1c Author: Alena Prokharchyk Date: Tue Apr 9 14:01:11 2013 -0700 InternalLB: Modified pluggable service VirtualRouterElementService to accept only VirtualRouter and VpcVirtualRouter as a VirtualRouterProvider type when add/configure elements. Other VirtualRouterProviderTypes are are taken care by elb/internal lb plugins. commit f0018b451281bf02964dbe71d704c9a931783cae Author: Alena Prokharchyk Date: Tue Apr 9 12:56:42 2013 -0700 Internal LB: 1)Added InternalLoadBalancerManager and corresponding Impl 2)Add InternalLbVm as a default CS provider. DB upgrade is covered as well commit e344cf250af6a67a17f082f1c646e6ef5ce9a0a7 Author: Alena Prokharchyk Date: Mon Apr 8 16:57:47 2013 -0700 InternalLB - removed unused code from LoadBalancerDao commit 3588f468482e43181802730ed5a059897eb85458 Author: Alena Prokharchyk Date: Mon Apr 8 16:23:03 2013 -0700 Internal LB - added network-element plugin for internal lb service commit 76325e61681c7e191308b27514ba69e523cf12bd Author: Alena Prokharchyk Date: Mon Apr 8 13:58:08 2013 -0700 Internal Lb: 1) When network has both kinds of LB rules - Public and Internal - never send them in the same set to the provider 2) Added extra checks on the provider side to validate if the schema - Public or Internal - is supported. commit 56c2fe1d376c407a248fba789dd67ff36493a0ed Author: Alena Prokharchyk Date: Mon Apr 8 12:34:59 2013 -0700 InternalLB: 1) Added new capability for the LB service - LbSchemes. Can take 2 values - Internal and Public. 2) F5 and Netscaler LB providers - changes all LB related methods to accept LoadBalancingRule instead of ? extends FirewallRule. commit 34bcb2d026e1951f7b115d71830c69e1ad5ed89a Author: Alena Prokharchyk Date: Mon Apr 8 12:11:11 2013 -0700 InternalLB: implemented list/delete/create web services API commands that will be used for Internal LB creation commit 9ab31e11f7ebd431f9ebdf9d8582f5c126b14928 Author: Alena Prokharchyk Date: Fri Apr 5 15:59:55 2013 -0700 InternalLB: change LoadBalancingRule - reference sourceIpAddress of th load balancer by its value, not DB id commit 08f855d4e4b3888b3b4a163ce73c0e444b91e6b7 Author: Alena Prokharchyk Date: Thu Apr 4 09:26:21 2013 -0700 InternalLB: 1) Added new set of Interfaces - including the new VO - for the internal load balancer 2) DB change - added source_ip_address/source_ip_address_network_id/scheme (Internal/Public) fields to the load_balancer table --- api/src/com/cloud/async/AsyncJob.java | 2 + api/src/com/cloud/event/EventTypes.java | 24 +- api/src/com/cloud/network/IpAddress.java | 3 + api/src/com/cloud/network/Network.java | 19 +- api/src/com/cloud/network/NetworkModel.java | 7 + .../VirtualNetworkApplianceService.java | 2 + .../cloud/network/VirtualRouterProvider.java | 3 +- .../cloud/network/lb/LoadBalancingRule.java | 83 +- .../network/lb/LoadBalancingRulesService.java | 16 +- .../cloud/network/router/VirtualRouter.java | 2 +- .../com/cloud/network/rules/LoadBalancer.java | 10 +- .../network/rules/LoadBalancerContainer.java | 33 + .../com/cloud/offering/NetworkOffering.java | 11 + .../com/cloud/offering/ServiceOffering.java | 1 + api/src/com/cloud/vm/VirtualMachine.java | 3 +- .../apache/cloudstack/api/ApiConstants.java | 6 + .../org/apache/cloudstack/api/BaseCmd.java | 7 + .../cloudstack/api/ResponseGenerator.java | 44 +- ...nfigureInternalLoadBalancerElementCmd.java | 114 +++ .../CreateInternalLoadBalancerElementCmd.java | 116 +++ .../internallb/ListInternalLBVMsCmd.java | 151 +++ .../ListInternalLoadBalancerElementsCmd.java | 99 ++ .../internallb/StartInternalLBVMCmd.java | 120 +++ .../admin/internallb/StopInternalLBVMCmd.java | 123 +++ .../network/CreateNetworkOfferingCmd.java | 15 +- .../router/CreateVirtualRouterElementCmd.java | 21 +- .../command/admin/router/ListRoutersCmd.java | 7 +- .../command/admin/router/StartRouterCmd.java | 10 +- .../command/admin/router/StopRouterCmd.java | 11 +- .../CreateApplicationLoadBalancerCmd.java | 218 ++++ .../CreateLoadBalancerRuleCmd.java | 6 +- .../DeleteApplicationLoadBalancerCmd.java | 116 +++ .../ListApplicationLoadBalancersCmd.java | 131 +++ ...plicationLoadBalancerInstanceResponse.java | 63 ++ .../ApplicationLoadBalancerResponse.java | 142 +++ .../ApplicationLoadBalancerRuleResponse.java | 51 + .../api/response/DomainRouterResponse.java | 13 +- .../InternalLoadBalancerElementResponse.java | 51 + .../api/response/NetworkOfferingResponse.java | 9 + .../VirtualRouterProviderResponse.java | 1 + .../InternalLoadBalancerElementService.java | 56 ++ .../lb/ApplicationLoadBalancerContainer.java | 28 + .../lb/ApplicationLoadBalancerRule.java | 24 + .../lb/ApplicationLoadBalancerService.java | 42 + .../lb/InternalLoadBalancerVMService.java | 34 + .../apache/cloudstack/query/QueryService.java | 3 + client/pom.xml | 5 + client/tomcatconf/applicationContext.xml.in | 9 +- client/tomcatconf/commands.properties.in | 16 + client/tomcatconf/componentContext.xml.in | 2 + .../tomcatconf/nonossComponentContext.xml.in | 2 + .../simulatorComponentContext.xml.in | 1 + .../cloud/network/dao/FirewallRulesDao.java | 1 - .../com/cloud/network/dao/IPAddressVO.java | 9 +- .../cloud/network/dao/LoadBalancerDao.java | 10 +- .../network/dao/LoadBalancerDaoImpl.java | 74 +- .../com/cloud/network/dao/LoadBalancerVO.java | 18 + .../network/dao/NetworkServiceMapDao.java | 1 + .../network/dao/NetworkServiceMapDaoImpl.java | 9 + .../src/com/cloud/network/dao/NetworkVO.java | 3 - .../cloud/network/rules/FirewallRuleVO.java | 2 - .../offerings/NetworkOfferingDetailsVO.java | 90 ++ .../cloud/offerings/NetworkOfferingVO.java | 44 +- .../offerings/dao/NetworkOfferingDao.java | 4 + .../offerings/dao/NetworkOfferingDaoImpl.java | 24 + .../dao/NetworkOfferingDetailsDao.java | 31 + .../dao/NetworkOfferingDetailsDaoImpl.java | 79 ++ .../cloud/upgrade/dao/Upgrade410to420.java | 95 +- .../schema/src/com/cloud/vm/dao/NicDao.java | 2 +- .../src/com/cloud/vm/dao/NicDaoImpl.java | 2 +- .../lb/ApplicationLoadBalancerRuleVO.java | 133 +++ .../dao/ApplicationLoadBalancerRuleDao.java | 35 + .../ApplicationLoadBalancerRuleDaoImpl.java | 115 +++ .../config/etc/init.d/cloud-early-config | 26 + .../debian/config/etc/iptables/iptables-ilbvm | 33 + .../debian/config/opt/cloud/bin/ilb.sh | 211 ++++ .../config/opt/cloud/bin/patchsystemvm.sh | 23 + .../config/opt/cloud/bin/vpc_loadbalancer.sh | 23 + .../element/ElasticLoadBalancerElement.java | 22 +- .../lb/ElasticLoadBalancerManager.java | 4 +- .../lb/ElasticLoadBalancerManagerImpl.java | 32 +- .../F5ExternalLoadBalancerElement.java | 87 +- .../internal-loadbalancer/pom.xml | 50 + .../element/InternalLoadBalancerElement.java | 548 ++++++++++ .../lb/InternalLoadBalancerVMManager.java | 90 ++ .../lb/InternalLoadBalancerVMManagerImpl.java | 951 ++++++++++++++++++ .../ElementChildTestConfiguration.java | 124 +++ .../InternalLbElementServiceTest.java | 190 ++++ .../InternalLbElementTest.java | 226 +++++ .../InternalLBVMManagerTest.java | 388 +++++++ .../InternalLBVMServiceTest.java | 291 ++++++ .../LbChildTestConfiguration.java | 170 ++++ .../test/resources/lb_element.xml | 46 + .../test/resources/lb_mgr.xml | 46 + .../test/resources/lb_svc.xml | 46 + .../network/element/NetscalerElement.java | 108 +- plugins/pom.xml | 1 + server/src/com/cloud/api/ApiDBUtils.java | 10 +- .../src/com/cloud/api/ApiResponseHelper.java | 108 +- .../com/cloud/api/query/QueryManagerImpl.java | 45 +- .../query/dao/DomainRouterJoinDaoImpl.java | 9 +- .../api/query/vo/DomainRouterJoinVO.java | 17 +- .../src/com/cloud/configuration/Config.java | 5 +- .../configuration/ConfigurationManager.java | 10 +- .../ConfigurationManagerImpl.java | 123 ++- .../ExternalLoadBalancerDeviceManager.java | 6 +- ...ExternalLoadBalancerDeviceManagerImpl.java | 33 +- .../ExternalLoadBalancerUsageManagerImpl.java | 34 +- .../src/com/cloud/network/NetworkManager.java | 3 +- .../com/cloud/network/NetworkManagerImpl.java | 103 +- .../com/cloud/network/NetworkModelImpl.java | 57 +- .../com/cloud/network/NetworkServiceImpl.java | 68 +- .../network/element/VirtualRouterElement.java | 56 +- .../network/firewall/FirewallManagerImpl.java | 77 +- .../cloud/network/guru/GuestNetworkGuru.java | 43 +- .../network/lb/LBHealthCheckManager.java | 4 +- .../network/lb/LBHealthCheckManagerImpl.java | 9 +- .../network/lb/LoadBalancingRulesManager.java | 24 +- .../lb/LoadBalancingRulesManagerImpl.java | 410 +++++--- .../VirtualNetworkApplianceManager.java | 4 + .../VirtualNetworkApplianceManagerImpl.java | 59 +- ...VpcVirtualNetworkApplianceManagerImpl.java | 2 +- .../com/cloud/network/vpc/VpcManagerImpl.java | 14 +- .../cloud/server/ConfigurationServerImpl.java | 14 +- .../cloud/server/ManagementServerImpl.java | 580 ++++++++--- .../ApplicationLoadBalancerManagerImpl.java | 524 ++++++++++ .../cloud/network/MockNetworkManagerImpl.java | 27 +- .../cloud/network/MockNetworkModelImpl.java | 17 + .../vpc/MockConfigurationManagerImpl.java | 2 +- .../com/cloud/vpc/MockNetworkManagerImpl.java | 56 +- .../com/cloud/vpc/MockNetworkModelImpl.java | 17 + ...MockVpcVirtualNetworkApplianceManager.java | 14 +- .../vpc/dao/MockNetworkOfferingDaoImpl.java | 12 +- .../vpc/dao/MockNetworkServiceMapDaoImpl.java | 6 + .../lb/ApplicationLoadBalancerTest.java | 292 ++++++ .../cloudstack/lb/ChildTestConfiguration.java | 103 ++ .../CreateNetworkOfferingTest.java | 21 +- server/test/resources/appLoadBalancer.xml | 43 + setup/db/db/schema-40to410.sql | 4 +- setup/db/db/schema-410to420.sql | 45 +- setup/dev/advanced.cfg | 4 + .../component/test_multiple_ip_ranges.py | 1 + .../smoke/test_guest_vlan_range.py | 15 + test/integration/smoke/test_internal_lb.py | 250 +++++ tools/apidoc/gen_toc.py | 1 + tools/marvin/marvin/deployDataCenter.py | 12 + .../marvin/sandbox/advanced/advanced_env.py | 5 + .../marvin/sandbox/advanced/sandbox.cfg | 209 ++++ ui/scripts/zoneWizard.js | 104 ++ 149 files changed, 9497 insertions(+), 817 deletions(-) create mode 100644 api/src/com/cloud/network/rules/LoadBalancerContainer.java create mode 100644 api/src/org/apache/cloudstack/api/command/admin/internallb/ConfigureInternalLoadBalancerElementCmd.java create mode 100644 api/src/org/apache/cloudstack/api/command/admin/internallb/CreateInternalLoadBalancerElementCmd.java create mode 100644 api/src/org/apache/cloudstack/api/command/admin/internallb/ListInternalLBVMsCmd.java create mode 100644 api/src/org/apache/cloudstack/api/command/admin/internallb/ListInternalLoadBalancerElementsCmd.java create mode 100644 api/src/org/apache/cloudstack/api/command/admin/internallb/StartInternalLBVMCmd.java create mode 100644 api/src/org/apache/cloudstack/api/command/admin/internallb/StopInternalLBVMCmd.java create mode 100644 api/src/org/apache/cloudstack/api/command/user/loadbalancer/CreateApplicationLoadBalancerCmd.java create mode 100644 api/src/org/apache/cloudstack/api/command/user/loadbalancer/DeleteApplicationLoadBalancerCmd.java create mode 100644 api/src/org/apache/cloudstack/api/command/user/loadbalancer/ListApplicationLoadBalancersCmd.java create mode 100644 api/src/org/apache/cloudstack/api/response/ApplicationLoadBalancerInstanceResponse.java create mode 100644 api/src/org/apache/cloudstack/api/response/ApplicationLoadBalancerResponse.java create mode 100644 api/src/org/apache/cloudstack/api/response/ApplicationLoadBalancerRuleResponse.java create mode 100644 api/src/org/apache/cloudstack/api/response/InternalLoadBalancerElementResponse.java create mode 100644 api/src/org/apache/cloudstack/network/element/InternalLoadBalancerElementService.java create mode 100644 api/src/org/apache/cloudstack/network/lb/ApplicationLoadBalancerContainer.java create mode 100644 api/src/org/apache/cloudstack/network/lb/ApplicationLoadBalancerRule.java create mode 100644 api/src/org/apache/cloudstack/network/lb/ApplicationLoadBalancerService.java create mode 100644 api/src/org/apache/cloudstack/network/lb/InternalLoadBalancerVMService.java create mode 100644 engine/schema/src/com/cloud/offerings/NetworkOfferingDetailsVO.java create mode 100644 engine/schema/src/com/cloud/offerings/dao/NetworkOfferingDetailsDao.java create mode 100644 engine/schema/src/com/cloud/offerings/dao/NetworkOfferingDetailsDaoImpl.java create mode 100644 engine/schema/src/org/apache/cloudstack/lb/ApplicationLoadBalancerRuleVO.java create mode 100644 engine/schema/src/org/apache/cloudstack/lb/dao/ApplicationLoadBalancerRuleDao.java create mode 100644 engine/schema/src/org/apache/cloudstack/lb/dao/ApplicationLoadBalancerRuleDaoImpl.java create mode 100755 patches/systemvm/debian/config/etc/iptables/iptables-ilbvm create mode 100755 patches/systemvm/debian/config/opt/cloud/bin/ilb.sh create mode 100644 plugins/network-elements/internal-loadbalancer/pom.xml create mode 100644 plugins/network-elements/internal-loadbalancer/src/org/apache/cloudstack/network/element/InternalLoadBalancerElement.java create mode 100644 plugins/network-elements/internal-loadbalancer/src/org/apache/cloudstack/network/lb/InternalLoadBalancerVMManager.java create mode 100644 plugins/network-elements/internal-loadbalancer/src/org/apache/cloudstack/network/lb/InternalLoadBalancerVMManagerImpl.java create mode 100644 plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbelement/ElementChildTestConfiguration.java create mode 100644 plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbelement/InternalLbElementServiceTest.java create mode 100644 plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbelement/InternalLbElementTest.java create mode 100644 plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbvmmgr/InternalLBVMManagerTest.java create mode 100644 plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbvmmgr/InternalLBVMServiceTest.java create mode 100644 plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbvmmgr/LbChildTestConfiguration.java create mode 100644 plugins/network-elements/internal-loadbalancer/test/resources/lb_element.xml create mode 100644 plugins/network-elements/internal-loadbalancer/test/resources/lb_mgr.xml create mode 100644 plugins/network-elements/internal-loadbalancer/test/resources/lb_svc.xml create mode 100644 server/src/org/apache/cloudstack/network/lb/ApplicationLoadBalancerManagerImpl.java create mode 100644 server/test/org/apache/cloudstack/lb/ApplicationLoadBalancerTest.java create mode 100644 server/test/org/apache/cloudstack/lb/ChildTestConfiguration.java create mode 100644 server/test/resources/appLoadBalancer.xml create mode 100644 test/integration/smoke/test_internal_lb.py create mode 100644 tools/marvin/marvin/sandbox/advanced/sandbox.cfg diff --git a/api/src/com/cloud/async/AsyncJob.java b/api/src/com/cloud/async/AsyncJob.java index d384a7ad920..ccdc40620b7 100644 --- a/api/src/com/cloud/async/AsyncJob.java +++ b/api/src/com/cloud/async/AsyncJob.java @@ -50,7 +50,9 @@ public interface AsyncJob extends Identity, InternalIdentity { AutoScaleVmProfile, AutoScaleVmGroup, GlobalLoadBalancerRule, + LoadBalancerRule, AffinityGroup, + InternalLbVm, DedicatedGuestVlanRange } diff --git a/api/src/com/cloud/event/EventTypes.java b/api/src/com/cloud/event/EventTypes.java index 26c40abb4fb..45a904e426c 100755 --- a/api/src/com/cloud/event/EventTypes.java +++ b/api/src/com/cloud/event/EventTypes.java @@ -16,6 +16,9 @@ // under the License. package com.cloud.event; +import java.util.HashMap; +import java.util.Map; + import com.cloud.configuration.Configuration; import com.cloud.dc.DataCenter; import com.cloud.dc.Pod; @@ -23,8 +26,18 @@ import com.cloud.dc.StorageNetworkIpRange; import com.cloud.dc.Vlan; import com.cloud.domain.Domain; import com.cloud.host.Host; -import com.cloud.network.*; -import com.cloud.network.as.*; +import com.cloud.network.GuestVlan; +import com.cloud.network.Network; +import com.cloud.network.PhysicalNetwork; +import com.cloud.network.PhysicalNetworkServiceProvider; +import com.cloud.network.PhysicalNetworkTrafficType; +import com.cloud.network.PublicIpAddress; +import com.cloud.network.RemoteAccessVpn; +import com.cloud.network.as.AutoScaleCounter; +import com.cloud.network.as.AutoScalePolicy; +import com.cloud.network.as.AutoScaleVmGroup; +import com.cloud.network.as.AutoScaleVmProfile; +import com.cloud.network.as.Condition; import com.cloud.network.router.VirtualRouter; import com.cloud.network.rules.LoadBalancer; import com.cloud.network.rules.StaticNat; @@ -43,9 +56,6 @@ import com.cloud.user.Account; import com.cloud.user.User; import com.cloud.vm.VirtualMachine; -import java.util.HashMap; -import java.util.Map; - public class EventTypes { //map of Event and corresponding entity for which Event is applicable @@ -389,11 +399,15 @@ public class EventTypes { public static final String EVENT_AFFINITY_GROUP_ASSIGN = "AG.ASSIGN"; public static final String EVENT_AFFINITY_GROUP_REMOVE = "AG.REMOVE"; public static final String EVENT_VM_AFFINITY_GROUP_UPDATE = "VM.AG.UPDATE"; + + public static final String EVENT_INTERNAL_LB_VM_START = "INTERNALLBVM.START"; + public static final String EVENT_INTERNAL_LB_VM_STOP = "INTERNALLBVM.STOP"; // Dedicated guest vlan range public static final String EVENT_GUEST_VLAN_RANGE_DEDICATE = "GUESTVLANRANGE.DEDICATE"; public static final String EVENT_DEDICATED_GUEST_VLAN_RANGE_RELEASE = "GUESTVLANRANGE.RELEASE"; + static { // TODO: need a way to force author adding event types to declare the entity details as well, with out braking diff --git a/api/src/com/cloud/network/IpAddress.java b/api/src/com/cloud/network/IpAddress.java index 71c9b4e0bf3..c48e8b97ca8 100644 --- a/api/src/com/cloud/network/IpAddress.java +++ b/api/src/com/cloud/network/IpAddress.java @@ -81,4 +81,7 @@ public interface IpAddress extends ControlledEntity, Identity, InternalIdentity Long getVpcId(); String getVmIp(); + + Long getNetworkId(); + } diff --git a/api/src/com/cloud/network/Network.java b/api/src/com/cloud/network/Network.java index 4472dbacc53..fa062c6a694 100644 --- a/api/src/com/cloud/network/Network.java +++ b/api/src/com/cloud/network/Network.java @@ -16,18 +16,19 @@ // under the License. package com.cloud.network; +import java.net.URI; +import java.util.ArrayList; +import java.util.List; + +import org.apache.cloudstack.acl.ControlledEntity; +import org.apache.cloudstack.api.Identity; +import org.apache.cloudstack.api.InternalIdentity; + import com.cloud.network.Networks.BroadcastDomainType; import com.cloud.network.Networks.Mode; import com.cloud.network.Networks.TrafficType; import com.cloud.utils.fsm.StateMachine2; import com.cloud.utils.fsm.StateObject; -import org.apache.cloudstack.acl.ControlledEntity; -import org.apache.cloudstack.api.Identity; -import org.apache.cloudstack.api.InternalIdentity; - -import java.net.URI; -import java.util.ArrayList; -import java.util.List; /** * owned by an account. @@ -50,7 +51,7 @@ public interface Network extends ControlledEntity, StateObject, I Capability.MultipleIps, Capability.TrafficStatistics, Capability.SupportedTrafficDirection, Capability.SupportedEgressProtocols); public static final Service Lb = new Service("Lb", Capability.SupportedLBAlgorithms, Capability.SupportedLBIsolation, Capability.SupportedProtocols, Capability.TrafficStatistics, Capability.LoadBalancingSupportedIps, - Capability.SupportedStickinessMethods, Capability.ElasticLb); + Capability.SupportedStickinessMethods, Capability.ElasticLb, Capability.LbSchemes); public static final Service UserData = new Service("UserData"); public static final Service SourceNat = new Service("SourceNat", Capability.SupportedSourceNatTypes, Capability.RedundantRouter); public static final Service StaticNat = new Service("StaticNat", Capability.ElasticIp); @@ -124,6 +125,7 @@ public interface Network extends ControlledEntity, StateObject, I public static final Provider None = new Provider("None", false); // NiciraNvp is not an "External" provider, otherwise we get in trouble with NetworkServiceImpl.providersConfiguredForExternalNetworking public static final Provider NiciraNvp = new Provider("NiciraNvp", false); + public static final Provider InternalLbVm = new Provider("InternalLbVm", false); public static final Provider CiscoVnmc = new Provider("CiscoVnmc", true); private String name; @@ -177,6 +179,7 @@ public interface Network extends ControlledEntity, StateObject, I public static final Capability SupportedTrafficDirection = new Capability("SupportedTrafficDirection"); public static final Capability SupportedEgressProtocols = new Capability("SupportedEgressProtocols"); public static final Capability HealthCheckPolicy = new Capability("HealthCheckPolicy"); + public static final Capability LbSchemes = new Capability("LbSchemes"); private String name; diff --git a/api/src/com/cloud/network/NetworkModel.java b/api/src/com/cloud/network/NetworkModel.java index 555a09fc53e..f84a8b0c76a 100644 --- a/api/src/com/cloud/network/NetworkModel.java +++ b/api/src/com/cloud/network/NetworkModel.java @@ -33,6 +33,7 @@ import com.cloud.network.Networks.TrafficType; import com.cloud.network.element.NetworkElement; import com.cloud.network.element.UserDataServiceProvider; import com.cloud.offering.NetworkOffering; +import com.cloud.offering.NetworkOffering.Detail; import com.cloud.user.Account; import com.cloud.vm.Nic; import com.cloud.vm.NicProfile; @@ -264,5 +265,11 @@ public interface NetworkModel { Nic getPlaceholderNicForRouter(Network network, Long podId); + IpAddress getPublicIpAddress(String ipAddress, long zoneId); + + List getUsedIpsInNetwork(Network network); + + Map getNtwkOffDetails(long offId); + Networks.IsolationType[] listNetworkIsolationMethods(); } \ No newline at end of file diff --git a/api/src/com/cloud/network/VirtualNetworkApplianceService.java b/api/src/com/cloud/network/VirtualNetworkApplianceService.java index 250ecb24e91..58eead2af07 100644 --- a/api/src/com/cloud/network/VirtualNetworkApplianceService.java +++ b/api/src/com/cloud/network/VirtualNetworkApplianceService.java @@ -63,5 +63,7 @@ public interface VirtualNetworkApplianceService { VirtualRouter startRouter(long id) throws ResourceUnavailableException, InsufficientCapacityException, ConcurrentOperationException; VirtualRouter destroyRouter(long routerId, Account caller, Long callerUserId) throws ResourceUnavailableException, ConcurrentOperationException; + + VirtualRouter findRouter(long routerId); } diff --git a/api/src/com/cloud/network/VirtualRouterProvider.java b/api/src/com/cloud/network/VirtualRouterProvider.java index ed6a2741ba0..f67686e6b08 100644 --- a/api/src/com/cloud/network/VirtualRouterProvider.java +++ b/api/src/com/cloud/network/VirtualRouterProvider.java @@ -23,7 +23,8 @@ public interface VirtualRouterProvider extends InternalIdentity, Identity { public enum VirtualRouterProviderType { VirtualRouter, ElasticLoadBalancerVm, - VPCVirtualRouter + VPCVirtualRouter, + InternalLbVm } public VirtualRouterProviderType getType(); diff --git a/api/src/com/cloud/network/lb/LoadBalancingRule.java b/api/src/com/cloud/network/lb/LoadBalancingRule.java index 3e11e8c7c2c..4b37782a8c7 100644 --- a/api/src/com/cloud/network/lb/LoadBalancingRule.java +++ b/api/src/com/cloud/network/lb/LoadBalancingRule.java @@ -25,111 +25,83 @@ import com.cloud.network.as.Condition; import com.cloud.network.as.Counter; import com.cloud.network.rules.FirewallRule; import com.cloud.network.rules.LoadBalancer; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; import com.cloud.utils.Pair; +import com.cloud.utils.net.Ip; -public class LoadBalancingRule implements FirewallRule, LoadBalancer { +public class LoadBalancingRule { private LoadBalancer lb; + private Ip sourceIp; private List destinations; private List stickinessPolicies; private LbAutoScaleVmGroup autoScaleVmGroup; private List healthCheckPolicies; public LoadBalancingRule(LoadBalancer lb, List destinations, - List stickinessPolicies, List healthCheckPolicies) { + List stickinessPolicies, List healthCheckPolicies, Ip sourceIp) { this.lb = lb; this.destinations = destinations; this.stickinessPolicies = stickinessPolicies; this.healthCheckPolicies = healthCheckPolicies; + this.sourceIp = sourceIp; } - @Override public long getId() { return lb.getId(); } - @Override - public long getAccountId() { - return lb.getAccountId(); - } - - @Override - public long getDomainId() { - return lb.getDomainId(); - } - - @Override public String getName() { return lb.getName(); } - @Override public String getDescription() { return lb.getDescription(); } - @Override public int getDefaultPortStart() { return lb.getDefaultPortStart(); } - @Override public int getDefaultPortEnd() { return lb.getDefaultPortEnd(); } - @Override public String getAlgorithm() { return lb.getAlgorithm(); } - @Override public String getUuid() { return lb.getUuid(); } - @Override public String getXid() { return lb.getXid(); } - @Override - public Long getSourceIpAddressId() { - return lb.getSourceIpAddressId(); - } - - @Override public Integer getSourcePortStart() { return lb.getSourcePortStart(); } - @Override public Integer getSourcePortEnd() { return lb.getSourcePortEnd(); } - @Override public String getProtocol() { return lb.getProtocol(); } - @Override - public Purpose getPurpose() { - return Purpose.LoadBalancing; + public FirewallRule.Purpose getPurpose() { + return FirewallRule.Purpose.LoadBalancing; } - @Override - public State getState() { + public FirewallRule.State getState() { return lb.getState(); } - @Override public long getNetworkId() { return lb.getNetworkId(); } - public LoadBalancer getLb() { - return lb; - } public void setDestinations(List destinations) { this.destinations = destinations; @@ -287,36 +259,6 @@ public class LoadBalancingRule implements FirewallRule, LoadBalancer { } } - @Override - public Integer getIcmpCode() { - return null; - } - - @Override - public Integer getIcmpType() { - return null; - } - - @Override - public List getSourceCidrList() { - return null; - } - - @Override - public Long getRelated() { - return null; - } - - @Override - public TrafficType getTrafficType() { - return null; - } - - @Override - public FirewallRuleType getType() { - return FirewallRuleType.User; - } - public LbAutoScaleVmGroup getAutoScaleVmGroup() { return autoScaleVmGroup; } @@ -473,4 +415,11 @@ public class LoadBalancingRule implements FirewallRule, LoadBalancer { } } + public Ip getSourceIp() { + return sourceIp; + } + + public Scheme getScheme() { + return lb.getScheme(); + } } diff --git a/api/src/com/cloud/network/lb/LoadBalancingRulesService.java b/api/src/com/cloud/network/lb/LoadBalancingRulesService.java index ed39bedaa6f..5fc41e34c34 100644 --- a/api/src/com/cloud/network/lb/LoadBalancingRulesService.java +++ b/api/src/com/cloud/network/lb/LoadBalancingRulesService.java @@ -17,10 +17,10 @@ package com.cloud.network.lb; import java.util.List; +import java.util.Map; import org.apache.cloudstack.api.command.user.loadbalancer.CreateLBHealthCheckPolicyCmd; import org.apache.cloudstack.api.command.user.loadbalancer.CreateLBStickinessPolicyCmd; -import org.apache.cloudstack.api.command.user.loadbalancer.CreateLoadBalancerRuleCmd; import org.apache.cloudstack.api.command.user.loadbalancer.ListLBHealthCheckPoliciesCmd; import org.apache.cloudstack.api.command.user.loadbalancer.ListLBStickinessPoliciesCmd; import org.apache.cloudstack.api.command.user.loadbalancer.ListLoadBalancerRuleInstancesCmd; @@ -30,12 +30,13 @@ import org.apache.cloudstack.api.command.user.loadbalancer.UpdateLoadBalancerRul import com.cloud.exception.InsufficientAddressCapacityException; import com.cloud.exception.NetworkRuleConflictException; import com.cloud.exception.ResourceUnavailableException; -import com.cloud.network.lb.LoadBalancingRule.LbStickinessPolicy; import com.cloud.network.rules.HealthCheckPolicy; import com.cloud.network.rules.LoadBalancer; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; import com.cloud.network.rules.StickinessPolicy; import com.cloud.uservm.UserVm; import com.cloud.utils.Pair; +import com.cloud.utils.net.Ip; public interface LoadBalancingRulesService { @@ -49,7 +50,9 @@ public interface LoadBalancingRulesService { * @return the newly created LoadBalancerVO if successful, null otherwise * @throws InsufficientAddressCapacityException */ - LoadBalancer createLoadBalancerRule(CreateLoadBalancerRuleCmd lb, boolean openFirewall) throws NetworkRuleConflictException, InsufficientAddressCapacityException; + LoadBalancer createPublicLoadBalancerRule(String xId, String name, String description, + int srcPortStart, int srcPortEnd, int defPortStart, int defPortEnd, Long ipAddrId, String protocol, String algorithm, + long networkId, long lbOwnerId, boolean openFirewall) throws NetworkRuleConflictException, InsufficientAddressCapacityException; LoadBalancer updateLoadBalancerRule(UpdateLoadBalancerRuleCmd cmd); @@ -134,8 +137,9 @@ public interface LoadBalancingRulesService { List searchForLBHealthCheckPolicies(ListLBHealthCheckPoliciesCmd cmd); - List listByNetworkId(long networkId); - LoadBalancer findById(long LoadBalancer); - public void updateLBHealthChecks() throws ResourceUnavailableException; + + public void updateLBHealthChecks(Scheme scheme) throws ResourceUnavailableException; + + Map getLbInstances(long lbId); } diff --git a/api/src/com/cloud/network/router/VirtualRouter.java b/api/src/com/cloud/network/router/VirtualRouter.java index d7239dd3452..2311f489918 100755 --- a/api/src/com/cloud/network/router/VirtualRouter.java +++ b/api/src/com/cloud/network/router/VirtualRouter.java @@ -23,7 +23,7 @@ import com.cloud.vm.VirtualMachine; */ public interface VirtualRouter extends VirtualMachine { public enum Role { - VIRTUAL_ROUTER, LB + VIRTUAL_ROUTER, LB, INTERNAL_LB_VM } Role getRole(); boolean getIsRedundantRouter(); diff --git a/api/src/com/cloud/network/rules/LoadBalancer.java b/api/src/com/cloud/network/rules/LoadBalancer.java index ab6085aceb7..e6dadcaee97 100644 --- a/api/src/com/cloud/network/rules/LoadBalancer.java +++ b/api/src/com/cloud/network/rules/LoadBalancer.java @@ -19,16 +19,10 @@ package com.cloud.network.rules; /** * Definition for a LoadBalancer */ -public interface LoadBalancer extends FirewallRule { - - String getName(); - - String getDescription(); - +public interface LoadBalancer extends FirewallRule, LoadBalancerContainer { + int getDefaultPortStart(); int getDefaultPortEnd(); - String getAlgorithm(); - } diff --git a/api/src/com/cloud/network/rules/LoadBalancerContainer.java b/api/src/com/cloud/network/rules/LoadBalancerContainer.java new file mode 100644 index 00000000000..9d5ea595c9d --- /dev/null +++ b/api/src/com/cloud/network/rules/LoadBalancerContainer.java @@ -0,0 +1,33 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package com.cloud.network.rules; + +public interface LoadBalancerContainer { + + public enum Scheme { + Public, Internal; + } + + String getName(); + + String getDescription(); + + String getAlgorithm(); + + Scheme getScheme(); + +} diff --git a/api/src/com/cloud/offering/NetworkOffering.java b/api/src/com/cloud/offering/NetworkOffering.java index 6f0b9937854..72e2a2bbbab 100644 --- a/api/src/com/cloud/offering/NetworkOffering.java +++ b/api/src/com/cloud/offering/NetworkOffering.java @@ -16,6 +16,8 @@ // under the License. package com.cloud.offering; +import java.util.Map; + import org.apache.cloudstack.acl.InfrastructureEntity; import org.apache.cloudstack.api.Identity; import org.apache.cloudstack.api.InternalIdentity; @@ -38,6 +40,11 @@ public interface NetworkOffering extends InfrastructureEntity, InternalIdentity, Disabled, Enabled } + + public enum Detail { + InternalLbProvider, + PublicLbProvider + } public final static String SystemPublicNetwork = "System-Public-Network"; public final static String SystemControlNetwork = "System-Control-Network"; @@ -116,5 +123,9 @@ public interface NetworkOffering extends InfrastructureEntity, InternalIdentity, boolean isInline(); boolean getIsPersistent(); + + boolean getInternalLb(); + + boolean getPublicLb(); } diff --git a/api/src/com/cloud/offering/ServiceOffering.java b/api/src/com/cloud/offering/ServiceOffering.java index d6c215f42f0..165369c5e9b 100755 --- a/api/src/com/cloud/offering/ServiceOffering.java +++ b/api/src/com/cloud/offering/ServiceOffering.java @@ -30,6 +30,7 @@ public interface ServiceOffering extends InfrastructureEntity, InternalIdentity, public static final String ssvmDefaultOffUniqueName = "Cloud.com-SecondaryStorage"; public static final String routerDefaultOffUniqueName = "Cloud.Com-SoftwareRouter"; public static final String elbVmDefaultOffUniqueName = "Cloud.Com-ElasticLBVm"; + public static final String internalLbVmDefaultOffUniqueName = "Cloud.Com-InternalLBVm"; public enum StorageType { local, diff --git a/api/src/com/cloud/vm/VirtualMachine.java b/api/src/com/cloud/vm/VirtualMachine.java index 8f807d450c7..ce9add62469 100755 --- a/api/src/com/cloud/vm/VirtualMachine.java +++ b/api/src/com/cloud/vm/VirtualMachine.java @@ -186,6 +186,7 @@ public interface VirtualMachine extends RunningOn, ControlledEntity, Identity, I SecondaryStorageVm, ElasticIpVm, ElasticLoadBalancerVm, + InternalLoadBalancerVm, /* * UserBareMetal is only used for selecting VirtualMachineGuru, there is no @@ -196,7 +197,7 @@ public interface VirtualMachine extends RunningOn, ControlledEntity, Identity, I public static boolean isSystemVM(VirtualMachine.Type vmtype) { if (DomainRouter.equals(vmtype) || ConsoleProxy.equals(vmtype) - || SecondaryStorageVm.equals(vmtype)) { + || SecondaryStorageVm.equals(vmtype) || InternalLoadBalancerVm.equals(vmtype)) { return true; } return false; diff --git a/api/src/org/apache/cloudstack/api/ApiConstants.java b/api/src/org/apache/cloudstack/api/ApiConstants.java index d57fe058d93..c76506afc10 100755 --- a/api/src/org/apache/cloudstack/api/ApiConstants.java +++ b/api/src/org/apache/cloudstack/api/ApiConstants.java @@ -480,6 +480,12 @@ public class ApiConstants { public static final String HEALTHCHECK_HEALTHY_THRESHOLD = "healthythreshold"; public static final String HEALTHCHECK_UNHEALTHY_THRESHOLD = "unhealthythreshold"; public static final String HEALTHCHECK_PINGPATH = "pingpath"; + public static final String SOURCE_PORT = "sourceport"; + public static final String INSTANCE_PORT = "instanceport"; + public static final String SOURCE_IP = "sourceipaddress"; + public static final String SOURCE_IP_NETWORK_ID = "sourceipaddressnetworkid"; + public static final String SCHEME = "scheme"; + public static final String PROVIDER_TYPE = "providertype"; public static final String AFFINITY_GROUP_IDS = "affinitygroupids"; public static final String AFFINITY_GROUP_NAMES = "affinitygroupnames"; public static final String ASA_INSIDE_PORT_PROFILE = "insideportprofile"; diff --git a/api/src/org/apache/cloudstack/api/BaseCmd.java b/api/src/org/apache/cloudstack/api/BaseCmd.java index 48d18d029fb..8d66a8327f0 100644 --- a/api/src/org/apache/cloudstack/api/BaseCmd.java +++ b/api/src/org/apache/cloudstack/api/BaseCmd.java @@ -28,6 +28,9 @@ import java.util.regex.Pattern; import javax.inject.Inject; import org.apache.cloudstack.affinity.AffinityGroupService; +import org.apache.cloudstack.network.element.InternalLoadBalancerElementService; +import org.apache.cloudstack.network.lb.ApplicationLoadBalancerService; +import org.apache.cloudstack.network.lb.InternalLoadBalancerVMService; import org.apache.cloudstack.query.QueryService; import org.apache.cloudstack.usage.UsageService; import org.apache.log4j.Logger; @@ -139,7 +142,11 @@ public abstract class BaseCmd { @Inject public VMSnapshotService _vmSnapshotService; @Inject public DataStoreProviderApiService dataStoreProviderApiService; @Inject public VpcProvisioningService _vpcProvSvc; + @Inject public ApplicationLoadBalancerService _newLbSvc; + @Inject public ApplicationLoadBalancerService _appLbService; @Inject public AffinityGroupService _affinityGroupService; + @Inject public InternalLoadBalancerElementService _internalLbElementSvc; + @Inject public InternalLoadBalancerVMService _internalLbSvc; @Inject public NetworkModel _ntwkModel; public abstract void execute() throws ResourceUnavailableException, InsufficientCapacityException, ServerApiException, ConcurrentOperationException, ResourceAllocationException, NetworkRuleConflictException; diff --git a/api/src/org/apache/cloudstack/api/ResponseGenerator.java b/api/src/org/apache/cloudstack/api/ResponseGenerator.java index 3433003cbed..ab8f99583a8 100644 --- a/api/src/org/apache/cloudstack/api/ResponseGenerator.java +++ b/api/src/org/apache/cloudstack/api/ResponseGenerator.java @@ -19,14 +19,15 @@ package org.apache.cloudstack.api; import java.text.DecimalFormat; import java.util.EnumSet; import java.util.List; +import java.util.Map; -import com.cloud.vm.NicSecondaryIp; import org.apache.cloudstack.affinity.AffinityGroup; import org.apache.cloudstack.affinity.AffinityGroupResponse; import org.apache.cloudstack.api.ApiConstants.HostDetails; import org.apache.cloudstack.api.ApiConstants.VMDetails; import org.apache.cloudstack.api.command.user.job.QueryAsyncJobResultCmd; import org.apache.cloudstack.api.response.AccountResponse; +import org.apache.cloudstack.api.response.ApplicationLoadBalancerResponse; import org.apache.cloudstack.api.response.AsyncJobResponse; import org.apache.cloudstack.api.response.AutoScalePolicyResponse; import org.apache.cloudstack.api.response.AutoScaleVmGroupResponse; @@ -44,11 +45,15 @@ import org.apache.cloudstack.api.response.EventResponse; import org.apache.cloudstack.api.response.ExtractResponse; import org.apache.cloudstack.api.response.FirewallResponse; import org.apache.cloudstack.api.response.FirewallRuleResponse; +import org.apache.cloudstack.api.response.GlobalLoadBalancerResponse; import org.apache.cloudstack.api.response.GuestOSResponse; +import org.apache.cloudstack.api.response.GuestVlanRangeResponse; +import org.apache.cloudstack.api.response.HostForMigrationResponse; import org.apache.cloudstack.api.response.HostResponse; import org.apache.cloudstack.api.response.HypervisorCapabilitiesResponse; import org.apache.cloudstack.api.response.IPAddressResponse; import org.apache.cloudstack.api.response.InstanceGroupResponse; +import org.apache.cloudstack.api.response.InternalLoadBalancerElementResponse; import org.apache.cloudstack.api.response.IpForwardingRuleResponse; import org.apache.cloudstack.api.response.IsolationMethodResponse; import org.apache.cloudstack.api.response.LBHealthCheckResponse; @@ -84,6 +89,7 @@ import org.apache.cloudstack.api.response.SnapshotResponse; import org.apache.cloudstack.api.response.SnapshotScheduleResponse; import org.apache.cloudstack.api.response.StaticRouteResponse; import org.apache.cloudstack.api.response.StorageNetworkIpRangeResponse; +import org.apache.cloudstack.api.response.StoragePoolForMigrationResponse; import org.apache.cloudstack.api.response.StoragePoolResponse; import org.apache.cloudstack.api.response.SwiftResponse; import org.apache.cloudstack.api.response.SystemVmInstanceResponse; @@ -103,6 +109,7 @@ import org.apache.cloudstack.api.response.VpcOfferingResponse; import org.apache.cloudstack.api.response.VpcResponse; import org.apache.cloudstack.api.response.VpnUsersResponse; import org.apache.cloudstack.api.response.ZoneResponse; +import org.apache.cloudstack.network.lb.ApplicationLoadBalancerRule; import org.apache.cloudstack.region.Region; import org.apache.cloudstack.usage.Usage; @@ -119,10 +126,25 @@ import com.cloud.domain.Domain; import com.cloud.event.Event; import com.cloud.host.Host; import com.cloud.hypervisor.HypervisorCapabilities; -import com.cloud.network.*; +import com.cloud.network.GuestVlan; +import com.cloud.network.IpAddress; +import com.cloud.network.Network; import com.cloud.network.Network.Service; import com.cloud.network.Networks.IsolationType; -import com.cloud.network.as.*; +import com.cloud.network.PhysicalNetwork; +import com.cloud.network.PhysicalNetworkServiceProvider; +import com.cloud.network.PhysicalNetworkTrafficType; +import com.cloud.network.RemoteAccessVpn; +import com.cloud.network.Site2SiteCustomerGateway; +import com.cloud.network.Site2SiteVpnConnection; +import com.cloud.network.Site2SiteVpnGateway; +import com.cloud.network.VirtualRouterProvider; +import com.cloud.network.VpnUser; +import com.cloud.network.as.AutoScalePolicy; +import com.cloud.network.as.AutoScaleVmGroup; +import com.cloud.network.as.AutoScaleVmProfile; +import com.cloud.network.as.Condition; +import com.cloud.network.as.Counter; import com.cloud.network.router.VirtualRouter; import com.cloud.network.rules.FirewallRule; import com.cloud.network.rules.HealthCheckPolicy; @@ -145,7 +167,12 @@ import com.cloud.projects.ProjectAccount; import com.cloud.projects.ProjectInvitation; import com.cloud.region.ha.GlobalLoadBalancerRule; import com.cloud.server.ResourceTag; -import com.cloud.storage.*; +import com.cloud.storage.GuestOS; +import com.cloud.storage.S3; +import com.cloud.storage.Snapshot; +import com.cloud.storage.StoragePool; +import com.cloud.storage.Swift; +import com.cloud.storage.Volume; import com.cloud.storage.snapshot.SnapshotPolicy; import com.cloud.storage.snapshot.SnapshotSchedule; import com.cloud.template.VirtualMachineTemplate; @@ -153,11 +180,12 @@ import com.cloud.user.Account; import com.cloud.user.User; import com.cloud.user.UserAccount; import com.cloud.uservm.UserVm; +import com.cloud.utils.net.Ip; import com.cloud.vm.InstanceGroup; import com.cloud.vm.Nic; -import com.cloud.vm.snapshot.VMSnapshot; +import com.cloud.vm.NicSecondaryIp; import com.cloud.vm.VirtualMachine; -import org.apache.cloudstack.api.response.*; +import com.cloud.vm.snapshot.VMSnapshot; public interface ResponseGenerator { UserResponse createUserResponse(UserAccount user); @@ -397,9 +425,13 @@ public interface ResponseGenerator { NicSecondaryIpResponse createSecondaryIPToNicResponse(NicSecondaryIp result); public NicResponse createNicResponse(Nic result); + ApplicationLoadBalancerResponse createLoadBalancerContainerReponse(ApplicationLoadBalancerRule lb, Map lbInstances); + AffinityGroupResponse createAffinityGroupResponse(AffinityGroup group); Long getAffinityGroupId(String name, long entityOwnerId); + InternalLoadBalancerElementResponse createInternalLbElementResponse(VirtualRouterProvider result); + IsolationMethodResponse createIsolationMethodResponse(IsolationType method); } diff --git a/api/src/org/apache/cloudstack/api/command/admin/internallb/ConfigureInternalLoadBalancerElementCmd.java b/api/src/org/apache/cloudstack/api/command/admin/internallb/ConfigureInternalLoadBalancerElementCmd.java new file mode 100644 index 00000000000..7c3d1e95e57 --- /dev/null +++ b/api/src/org/apache/cloudstack/api/command/admin/internallb/ConfigureInternalLoadBalancerElementCmd.java @@ -0,0 +1,114 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.cloudstack.api.command.admin.internallb; + +import java.util.List; + +import javax.inject.Inject; + +import org.apache.cloudstack.api.APICommand; +import org.apache.cloudstack.api.ApiConstants; +import org.apache.cloudstack.api.ApiErrorCode; +import org.apache.cloudstack.api.BaseAsyncCmd; +import org.apache.cloudstack.api.Parameter; +import org.apache.cloudstack.api.ServerApiException; +import org.apache.cloudstack.api.response.InternalLoadBalancerElementResponse; +import org.apache.cloudstack.network.element.InternalLoadBalancerElementService; +import org.apache.log4j.Logger; + +import com.cloud.event.EventTypes; +import com.cloud.exception.ConcurrentOperationException; +import com.cloud.exception.InsufficientCapacityException; +import com.cloud.exception.ResourceUnavailableException; +import com.cloud.network.VirtualRouterProvider; +import com.cloud.user.Account; +import com.cloud.user.UserContext; + +@APICommand(name = "configureInternalLoadBalancerElement", responseObject=InternalLoadBalancerElementResponse.class, + description="Configures an Internal Load Balancer element.", since="4.2.0") +public class ConfigureInternalLoadBalancerElementCmd extends BaseAsyncCmd { + public static final Logger s_logger = Logger.getLogger(ConfigureInternalLoadBalancerElementCmd.class.getName()); + private static final String s_name = "configureinternalloadbalancerelementresponse"; + + @Inject + private List _service; + + ///////////////////////////////////////////////////// + //////////////// API parameters ///////////////////// + ///////////////////////////////////////////////////// + + @Parameter(name=ApiConstants.ID, type=CommandType.UUID, entityType = InternalLoadBalancerElementResponse.class, + required=true, description="the ID of the internal lb provider") + private Long id; + + @Parameter(name=ApiConstants.ENABLED, type=CommandType.BOOLEAN, required=true, description="Enables/Disables the Internal Load Balancer element") + private Boolean enabled; + + ///////////////////////////////////////////////////// + /////////////////// Accessors /////////////////////// + ///////////////////////////////////////////////////// + + + public Long getId() { + return id; + } + + public Boolean getEnabled() { + return enabled; + } + + ///////////////////////////////////////////////////// + /////////////// API Implementation/////////////////// + ///////////////////////////////////////////////////// + + @Override + public String getCommandName() { + return s_name; + } + + @Override + public long getEntityOwnerId() { + return Account.ACCOUNT_ID_SYSTEM; + } + + @Override + public String getEventType() { + return EventTypes.EVENT_NETWORK_ELEMENT_CONFIGURE; + } + + @Override + public String getEventDescription() { + return "configuring internal load balancer element: " + id; + } + + @Override + public void execute() throws ConcurrentOperationException, ResourceUnavailableException, InsufficientCapacityException{ + s_logger.debug("hello alena"); + UserContext.current().setEventDetails("Internal load balancer element: " + id); + s_logger.debug("hello alena"); + VirtualRouterProvider result = _service.get(0).configureInternalLoadBalancerElement(getId(), getEnabled()); + s_logger.debug("hello alena"); + if (result != null){ + InternalLoadBalancerElementResponse routerResponse = _responseGenerator.createInternalLbElementResponse(result); + routerResponse.setResponseName(getCommandName()); + this.setResponseObject(routerResponse); + } else { + throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, "Failed to configure the internal load balancer element"); + } + } +} diff --git a/api/src/org/apache/cloudstack/api/command/admin/internallb/CreateInternalLoadBalancerElementCmd.java b/api/src/org/apache/cloudstack/api/command/admin/internallb/CreateInternalLoadBalancerElementCmd.java new file mode 100644 index 00000000000..2902f7ae18a --- /dev/null +++ b/api/src/org/apache/cloudstack/api/command/admin/internallb/CreateInternalLoadBalancerElementCmd.java @@ -0,0 +1,116 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.api.command.admin.internallb; + +import java.util.List; + +import javax.inject.Inject; + +import org.apache.cloudstack.api.APICommand; +import org.apache.cloudstack.api.ApiConstants; +import org.apache.cloudstack.api.ApiErrorCode; +import org.apache.cloudstack.api.BaseAsyncCreateCmd; +import org.apache.cloudstack.api.Parameter; +import org.apache.cloudstack.api.ServerApiException; +import org.apache.cloudstack.api.response.InternalLoadBalancerElementResponse; +import org.apache.cloudstack.api.response.ProviderResponse; +import org.apache.cloudstack.network.element.InternalLoadBalancerElementService; +import org.apache.log4j.Logger; + +import com.cloud.event.EventTypes; +import com.cloud.exception.ResourceAllocationException; +import com.cloud.network.VirtualRouterProvider; +import com.cloud.user.Account; +import com.cloud.user.UserContext; + +@APICommand(name = "createInternalLoadBalancerElement", responseObject=InternalLoadBalancerElementResponse.class, description="Create an Internal Load Balancer element.",since="4.2.0") +public class CreateInternalLoadBalancerElementCmd extends BaseAsyncCreateCmd { + public static final Logger s_logger = Logger.getLogger(CreateInternalLoadBalancerElementCmd.class.getName()); + private static final String s_name = "createinternalloadbalancerelementresponse"; + + @Inject + private List _service; + + ///////////////////////////////////////////////////// + //////////////// API parameters ///////////////////// + ///////////////////////////////////////////////////// + + @Parameter(name=ApiConstants.NETWORK_SERVICE_PROVIDER_ID, type=CommandType.UUID, entityType = ProviderResponse.class, required=true, description="the network service provider ID of the internal load balancer element") + private Long nspId; + + ///////////////////////////////////////////////////// + /////////////////// Accessors /////////////////////// + ///////////////////////////////////////////////////// + + public void setNspId(Long nspId) { + this.nspId = nspId; + } + + public Long getNspId() { + return nspId; + } + + ///////////////////////////////////////////////////// + /////////////// API Implementation/////////////////// + ///////////////////////////////////////////////////// + + + + @Override + public String getCommandName() { + return s_name; + } + + @Override + public long getEntityOwnerId() { + return Account.ACCOUNT_ID_SYSTEM; + } + + @Override + public void execute(){ + UserContext.current().setEventDetails("Virtual router element Id: "+getEntityId()); + VirtualRouterProvider result = _service.get(0).getInternalLoadBalancerElement(getEntityId()); + if (result != null) { + InternalLoadBalancerElementResponse response = _responseGenerator.createInternalLbElementResponse(result); + response.setResponseName(getCommandName()); + this.setResponseObject(response); + }else { + throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, "Failed to add Virtual Router entity to physical network"); + } + } + + @Override + public void create() throws ResourceAllocationException { + VirtualRouterProvider result = _service.get(0).addInternalLoadBalancerElement(getNspId()); + if (result != null) { + setEntityId(result.getId()); + setEntityUuid(result.getUuid()); + } else { + throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, "Failed to add Internal Load Balancer entity to physical network"); + } + } + + @Override + public String getEventType() { + return EventTypes.EVENT_SERVICE_PROVIDER_CREATE; + } + + @Override + public String getEventDescription() { + return "Adding physical network element Internal Load Balancer: " + getEntityId(); + } +} diff --git a/api/src/org/apache/cloudstack/api/command/admin/internallb/ListInternalLBVMsCmd.java b/api/src/org/apache/cloudstack/api/command/admin/internallb/ListInternalLBVMsCmd.java new file mode 100644 index 00000000000..e314b3245c7 --- /dev/null +++ b/api/src/org/apache/cloudstack/api/command/admin/internallb/ListInternalLBVMsCmd.java @@ -0,0 +1,151 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.api.command.admin.internallb; + +import org.apache.cloudstack.api.APICommand; +import org.apache.cloudstack.api.ApiConstants; +import org.apache.cloudstack.api.BaseListProjectAndAccountResourcesCmd; +import org.apache.cloudstack.api.Parameter; +import org.apache.cloudstack.api.response.DomainRouterResponse; +import org.apache.cloudstack.api.response.HostResponse; +import org.apache.cloudstack.api.response.ListResponse; +import org.apache.cloudstack.api.response.NetworkResponse; +import org.apache.cloudstack.api.response.PodResponse; +import org.apache.cloudstack.api.response.UserVmResponse; +import org.apache.cloudstack.api.response.VpcResponse; +import org.apache.cloudstack.api.response.ZoneResponse; +import org.apache.log4j.Logger; + +import com.cloud.async.AsyncJob; +import com.cloud.network.router.VirtualRouter.Role; + +@APICommand(name = "listInternalLoadBalancerVMs", description="List internal LB VMs.", responseObject=DomainRouterResponse.class) +public class ListInternalLBVMsCmd extends BaseListProjectAndAccountResourcesCmd { + public static final Logger s_logger = Logger.getLogger(ListInternalLBVMsCmd.class.getName()); + + private static final String s_name = "listinternallbvmssresponse"; + + ///////////////////////////////////////////////////// + //////////////// API parameters ///////////////////// + ///////////////////////////////////////////////////// + + @Parameter(name=ApiConstants.HOST_ID, type=CommandType.UUID, entityType=HostResponse.class, + description="the host ID of the Internal LB VM") + private Long hostId; + + @Parameter(name=ApiConstants.ID, type=CommandType.UUID, entityType=UserVmResponse.class, + description="the ID of the Internal LB VM") + private Long id; + + @Parameter(name=ApiConstants.NAME, type=CommandType.STRING, description="the name of the Internal LB VM") + private String routerName; + + @Parameter(name=ApiConstants.POD_ID, type=CommandType.UUID, entityType=PodResponse.class, + description="the Pod ID of the Internal LB VM") + private Long podId; + + @Parameter(name=ApiConstants.STATE, type=CommandType.STRING, description="the state of the Internal LB VM") + private String state; + + @Parameter(name=ApiConstants.ZONE_ID, type=CommandType.UUID, entityType=ZoneResponse.class, + description="the Zone ID of the Internal LB VM") + private Long zoneId; + + @Parameter(name=ApiConstants.NETWORK_ID, type=CommandType.UUID, entityType=NetworkResponse.class, + description="list by network id") + private Long networkId; + + @Parameter(name=ApiConstants.VPC_ID, type=CommandType.UUID, entityType=VpcResponse.class, + description="List Internal LB VMs by VPC") + private Long vpcId; + + @Parameter(name=ApiConstants.FOR_VPC, type=CommandType.BOOLEAN, description="if true is passed for this parameter, list only VPC Internal LB VMs") + private Boolean forVpc; + + @Parameter(name=ApiConstants.ZONE_TYPE, type=CommandType.STRING, description="the network type of the zone that the virtual machine belongs to") + private String zoneType; + + + ///////////////////////////////////////////////////// + /////////////////// Accessors /////////////////////// + ///////////////////////////////////////////////////// + + public Long getHostId() { + return hostId; + } + + public Long getId() { + return id; + } + + public String getRouterName() { + return routerName; + } + + public Long getPodId() { + return podId; + } + + public String getState() { + return state; + } + + public Long getZoneId() { + return zoneId; + } + + public Long getNetworkId() { + return networkId; + } + + public Long getVpcId() { + return vpcId; + } + + public Boolean getForVpc() { + return forVpc; + } + + public String getRole() { + return Role.INTERNAL_LB_VM.toString(); + } + + public String getZoneType() { + return zoneType; + } + + ///////////////////////////////////////////////////// + /////////////// API Implementation/////////////////// + ///////////////////////////////////////////////////// + + @Override + public String getCommandName() { + return s_name; + } + + @Override + public AsyncJob.Type getInstanceType() { + return AsyncJob.Type.DomainRouter; + } + + @Override + public void execute(){ + ListResponse response = _queryService.searchForInternalLbVms(this); + response.setResponseName(getCommandName()); + this.setResponseObject(response); + } +} diff --git a/api/src/org/apache/cloudstack/api/command/admin/internallb/ListInternalLoadBalancerElementsCmd.java b/api/src/org/apache/cloudstack/api/command/admin/internallb/ListInternalLoadBalancerElementsCmd.java new file mode 100644 index 00000000000..18536191995 --- /dev/null +++ b/api/src/org/apache/cloudstack/api/command/admin/internallb/ListInternalLoadBalancerElementsCmd.java @@ -0,0 +1,99 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.api.command.admin.internallb; + +import java.util.ArrayList; +import java.util.List; + +import javax.inject.Inject; + +import org.apache.cloudstack.api.APICommand; +import org.apache.cloudstack.api.ApiConstants; +import org.apache.cloudstack.api.BaseListCmd; +import org.apache.cloudstack.api.Parameter; +import org.apache.cloudstack.api.ServerApiException; +import org.apache.cloudstack.api.response.InternalLoadBalancerElementResponse; +import org.apache.cloudstack.api.response.ListResponse; +import org.apache.cloudstack.api.response.ProviderResponse; +import org.apache.cloudstack.network.element.InternalLoadBalancerElementService; +import org.apache.log4j.Logger; + +import com.cloud.exception.ConcurrentOperationException; +import com.cloud.exception.InsufficientCapacityException; +import com.cloud.exception.ResourceAllocationException; +import com.cloud.exception.ResourceUnavailableException; +import com.cloud.network.VirtualRouterProvider; + +@APICommand(name = "listInternalLoadBalancerElements", description="Lists all available Internal Load Balancer elements.", + responseObject=InternalLoadBalancerElementResponse.class, since="4.2.0") +public class ListInternalLoadBalancerElementsCmd extends BaseListCmd { + public static final Logger s_logger = Logger.getLogger(ListInternalLoadBalancerElementsCmd.class.getName()); + private static final String _name = "listinternalloadbalancerelementsresponse"; + + @Inject + private InternalLoadBalancerElementService _service; + + ///////////////////////////////////////////////////// + //////////////// API parameters ///////////////////// + ///////////////////////////////////////////////////// + @Parameter(name=ApiConstants.ID, type=CommandType.UUID, entityType = InternalLoadBalancerElementResponse.class, + description="list internal load balancer elements by id") + private Long id; + + @Parameter(name=ApiConstants.NSP_ID, type=CommandType.UUID, entityType = ProviderResponse.class, + description="list internal load balancer elements by network service provider id") + private Long nspId; + + @Parameter(name=ApiConstants.ENABLED, type=CommandType.BOOLEAN, description="list internal load balancer elements by enabled state") + private Boolean enabled; + + ///////////////////////////////////////////////////// + /////////////////// Accessors /////////////////////// + ///////////////////////////////////////////////////// + + public Long getId() { + return id; + } + + public Long getNspId() { + return nspId; + } + + public Boolean getEnabled() { + return enabled; + } + + @Override + public String getCommandName() { + return _name; + } + + @Override + public void execute() throws ResourceUnavailableException, InsufficientCapacityException, ServerApiException, ConcurrentOperationException, ResourceAllocationException { + List providers = _service.searchForInternalLoadBalancerElements(getId(), getNspId(), getEnabled()); + ListResponse response = new ListResponse(); + List providerResponses = new ArrayList(); + for (VirtualRouterProvider provider : providers) { + InternalLoadBalancerElementResponse providerResponse = _responseGenerator.createInternalLbElementResponse(provider); + providerResponses.add(providerResponse); + } + response.setResponses(providerResponses); + response.setResponseName(getCommandName()); + this.setResponseObject(response); + + } +} diff --git a/api/src/org/apache/cloudstack/api/command/admin/internallb/StartInternalLBVMCmd.java b/api/src/org/apache/cloudstack/api/command/admin/internallb/StartInternalLBVMCmd.java new file mode 100644 index 00000000000..31d132b5c9c --- /dev/null +++ b/api/src/org/apache/cloudstack/api/command/admin/internallb/StartInternalLBVMCmd.java @@ -0,0 +1,120 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.api.command.admin.internallb; + +import org.apache.cloudstack.api.APICommand; +import org.apache.cloudstack.api.ApiConstants; +import org.apache.cloudstack.api.ApiErrorCode; +import org.apache.cloudstack.api.BaseAsyncCmd; +import org.apache.cloudstack.api.Parameter; +import org.apache.cloudstack.api.ServerApiException; +import org.apache.cloudstack.api.response.DomainRouterResponse; +import org.apache.log4j.Logger; + +import com.cloud.async.AsyncJob; +import com.cloud.event.EventTypes; +import com.cloud.exception.ConcurrentOperationException; +import com.cloud.exception.InsufficientCapacityException; +import com.cloud.exception.InvalidParameterValueException; +import com.cloud.exception.ResourceUnavailableException; +import com.cloud.network.router.VirtualRouter; +import com.cloud.network.router.VirtualRouter.Role; +import com.cloud.user.UserContext; + +@APICommand(name = "startInternalLoadBalancerVM", responseObject=DomainRouterResponse.class, description="Starts an existing internal lb vm.") +public class StartInternalLBVMCmd extends BaseAsyncCmd { + public static final Logger s_logger = Logger.getLogger(StartInternalLBVMCmd.class.getName()); + private static final String s_name = "startinternallbvmresponse"; + + + ///////////////////////////////////////////////////// + //////////////// API parameters ///////////////////// + ///////////////////////////////////////////////////// + + @Parameter(name=ApiConstants.ID, type=CommandType.UUID, entityType=DomainRouterResponse.class, + required=true, description="the ID of the internal lb vm") + private Long id; + + ///////////////////////////////////////////////////// + /////////////////// Accessors /////////////////////// + ///////////////////////////////////////////////////// + + public Long getId() { + return id; + } + + ///////////////////////////////////////////////////// + /////////////// API Implementation/////////////////// + ///////////////////////////////////////////////////// + + @Override + public String getCommandName() { + return s_name; + } + + public static String getResultObjectName() { + return "router"; + } + + @Override + public long getEntityOwnerId() { + VirtualRouter router = _entityMgr.findById(VirtualRouter.class, getId()); + if (router != null && router.getRole() == Role.INTERNAL_LB_VM) { + return router.getAccountId(); + } else { + throw new InvalidParameterValueException("Unable to find internal lb vm by id"); + } + } + + @Override + public String getEventType() { + return EventTypes.EVENT_INTERNAL_LB_VM_START; + } + + @Override + public String getEventDescription() { + return "starting internal lb vm: " + getId(); + } + + public AsyncJob.Type getInstanceType() { + return AsyncJob.Type.InternalLbVm; + } + + public Long getInstanceId() { + return getId(); + } + + @Override + public void execute() throws ConcurrentOperationException, ResourceUnavailableException, InsufficientCapacityException{ + UserContext.current().setEventDetails("Internal Lb Vm Id: "+getId()); + VirtualRouter result = null; + VirtualRouter router = _routerService.findRouter(getId()); + if (router == null || router.getRole() != Role.INTERNAL_LB_VM) { + throw new InvalidParameterValueException("Can't find internal lb vm by id"); + } else { + result = _internalLbSvc.startInternalLbVm(getId(), UserContext.current().getCaller(), UserContext.current().getCallerUserId()); + } + + if (result != null){ + DomainRouterResponse routerResponse = _responseGenerator.createDomainRouterResponse(result); + routerResponse.setResponseName(getCommandName()); + this.setResponseObject(routerResponse); + } else { + throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, "Failed to start internal lb vm"); + } + } +} diff --git a/api/src/org/apache/cloudstack/api/command/admin/internallb/StopInternalLBVMCmd.java b/api/src/org/apache/cloudstack/api/command/admin/internallb/StopInternalLBVMCmd.java new file mode 100644 index 00000000000..f40db49b417 --- /dev/null +++ b/api/src/org/apache/cloudstack/api/command/admin/internallb/StopInternalLBVMCmd.java @@ -0,0 +1,123 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.api.command.admin.internallb; + +import org.apache.cloudstack.api.APICommand; +import org.apache.cloudstack.api.ApiConstants; +import org.apache.cloudstack.api.ApiErrorCode; +import org.apache.cloudstack.api.BaseAsyncCmd; +import org.apache.cloudstack.api.Parameter; +import org.apache.cloudstack.api.ServerApiException; +import org.apache.cloudstack.api.response.DomainRouterResponse; +import org.apache.log4j.Logger; + +import com.cloud.async.AsyncJob; +import com.cloud.event.EventTypes; +import com.cloud.exception.ConcurrentOperationException; +import com.cloud.exception.InvalidParameterValueException; +import com.cloud.exception.ResourceUnavailableException; +import com.cloud.network.router.VirtualRouter; +import com.cloud.network.router.VirtualRouter.Role; +import com.cloud.user.UserContext; + +@APICommand(name = "stopInternalLoadBalancerVM", description = "Stops an Internal LB vm.", responseObject = DomainRouterResponse.class) +public class StopInternalLBVMCmd extends BaseAsyncCmd { + public static final Logger s_logger = Logger.getLogger(StopInternalLBVMCmd.class.getName()); + private static final String s_name = "stopinternallbvmresponse"; + + // /////////////////////////////////////////////////// + // ////////////// API parameters ///////////////////// + // /////////////////////////////////////////////////// + + @Parameter(name = ApiConstants.ID, type = CommandType.UUID, entityType = DomainRouterResponse.class, + required = true, description = "the ID of the internal lb vm") + private Long id; + + @Parameter(name = ApiConstants.FORCED, type = CommandType.BOOLEAN, required = false, description = "Force stop the VM. The caller knows the VM is stopped.") + private Boolean forced; + + // /////////////////////////////////////////////////// + // ///////////////// Accessors /////////////////////// + // /////////////////////////////////////////////////// + + public Long getId() { + return id; + } + + // /////////////////////////////////////////////////// + // ///////////// API Implementation/////////////////// + // /////////////////////////////////////////////////// + + @Override + public String getCommandName() { + return s_name; + } + + @Override + public long getEntityOwnerId() { + VirtualRouter vm = _entityMgr.findById(VirtualRouter.class, getId()); + if (vm != null && vm.getRole() == Role.INTERNAL_LB_VM) { + return vm.getAccountId(); + } else { + throw new InvalidParameterValueException("Unable to find internal lb vm by id"); + } + } + + @Override + public String getEventType() { + return EventTypes.EVENT_INTERNAL_LB_VM_STOP; + } + + @Override + public String getEventDescription() { + return "stopping internal lb vm: " + getId(); + } + + @Override + public AsyncJob.Type getInstanceType() { + return AsyncJob.Type.InternalLbVm; + } + + @Override + public Long getInstanceId() { + return getId(); + } + + public boolean isForced() { + return (forced != null) ? forced : false; + } + + @Override + public void execute() throws ConcurrentOperationException, ResourceUnavailableException { + UserContext.current().setEventDetails("Internal lb vm Id: "+getId()); + VirtualRouter result = null; + VirtualRouter vm = _routerService.findRouter(getId()); + if (vm == null || vm.getRole() != Role.INTERNAL_LB_VM) { + throw new InvalidParameterValueException("Can't find internal lb vm by id"); + } else { + result = _internalLbSvc.stopInternalLbVm(getId(), isForced(), UserContext.current().getCaller(), UserContext.current().getCallerUserId()); + } + + if (result != null) { + DomainRouterResponse response = _responseGenerator.createDomainRouterResponse(result); + response.setResponseName(getCommandName()); + this.setResponseObject(response); + } else { + throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, "Failed to stop internal lb vm"); + } + } +} diff --git a/api/src/org/apache/cloudstack/api/command/admin/network/CreateNetworkOfferingCmd.java b/api/src/org/apache/cloudstack/api/command/admin/network/CreateNetworkOfferingCmd.java index b48bf9e763e..6410715727c 100644 --- a/api/src/org/apache/cloudstack/api/command/admin/network/CreateNetworkOfferingCmd.java +++ b/api/src/org/apache/cloudstack/api/command/admin/network/CreateNetworkOfferingCmd.java @@ -31,7 +31,6 @@ import org.apache.cloudstack.api.Parameter; import org.apache.cloudstack.api.ServerApiException; import org.apache.cloudstack.api.response.NetworkOfferingResponse; import org.apache.cloudstack.api.response.ServiceOfferingResponse; - import org.apache.log4j.Logger; import com.cloud.exception.InvalidParameterValueException; @@ -95,6 +94,10 @@ public class CreateNetworkOfferingCmd extends BaseCmd { @Parameter(name=ApiConstants.IS_PERSISTENT, type=CommandType.BOOLEAN, description="true if network offering supports persistent networks; defaulted to false if not specified") private Boolean isPersistent; + + @Parameter(name=ApiConstants.DETAILS, type=CommandType.MAP, since="4.2.0", description="Template details in key/value pairs." + + " Supported keys are internallbprovider/publiclbprovider with service provider as a value") + protected Map details; ///////////////////////////////////////////////////// /////////////////// Accessors /////////////////////// @@ -215,6 +218,16 @@ public class CreateNetworkOfferingCmd extends BaseCmd { return capabilityMap; } + + public Map getDetails() { + if (details == null || details.isEmpty()) { + return null; + } + + Collection paramsCollection = details.values(); + Map params = (Map) (paramsCollection.toArray())[0]; + return params; + } ///////////////////////////////////////////////////// /////////////// API Implementation/////////////////// diff --git a/api/src/org/apache/cloudstack/api/command/admin/router/CreateVirtualRouterElementCmd.java b/api/src/org/apache/cloudstack/api/command/admin/router/CreateVirtualRouterElementCmd.java index 39fac136233..b3fca5addf1 100644 --- a/api/src/org/apache/cloudstack/api/command/admin/router/CreateVirtualRouterElementCmd.java +++ b/api/src/org/apache/cloudstack/api/command/admin/router/CreateVirtualRouterElementCmd.java @@ -31,6 +31,7 @@ import org.apache.cloudstack.api.response.VirtualRouterProviderResponse; import org.apache.log4j.Logger; import com.cloud.event.EventTypes; +import com.cloud.exception.InvalidParameterValueException; import com.cloud.exception.ResourceAllocationException; import com.cloud.network.VirtualRouterProvider; import com.cloud.network.VirtualRouterProvider.VirtualRouterProviderType; @@ -52,6 +53,9 @@ public class CreateVirtualRouterElementCmd extends BaseAsyncCreateCmd { @Parameter(name=ApiConstants.NETWORK_SERVICE_PROVIDER_ID, type=CommandType.UUID, entityType = ProviderResponse.class, required=true, description="the network service provider ID of the virtual router element") private Long nspId; + + @Parameter(name=ApiConstants.PROVIDER_TYPE, type=CommandType.UUID, entityType = ProviderResponse.class, description="The provider type. Supported types are VirtualRouter (default) and VPCVirtualRouter") + private String providerType; ///////////////////////////////////////////////////// /////////////////// Accessors /////////////////////// @@ -61,16 +65,27 @@ public class CreateVirtualRouterElementCmd extends BaseAsyncCreateCmd { this.nspId = nspId; } - - public Long getNspId() { return nspId; } + + public VirtualRouterProviderType getProviderType() { + if (providerType != null) { + if (providerType.equalsIgnoreCase(VirtualRouterProviderType.VirtualRouter.toString())) { + return VirtualRouterProviderType.VirtualRouter; + } else if (providerType.equalsIgnoreCase(VirtualRouterProviderType.VPCVirtualRouter.toString())) { + return VirtualRouterProviderType.VPCVirtualRouter; + } else throw new InvalidParameterValueException("Invalid providerType specified"); + } + return VirtualRouterProviderType.VirtualRouter; + } ///////////////////////////////////////////////////// /////////////// API Implementation/////////////////// ///////////////////////////////////////////////////// + + @Override public String getCommandName() { return s_name; @@ -96,7 +111,7 @@ public class CreateVirtualRouterElementCmd extends BaseAsyncCreateCmd { @Override public void create() throws ResourceAllocationException { - VirtualRouterProvider result = _service.get(0).addElement(getNspId(), VirtualRouterProviderType.VirtualRouter); + VirtualRouterProvider result = _service.get(0).addElement(getNspId(), getProviderType()); if (result != null) { setEntityId(result.getId()); setEntityUuid(result.getUuid()); diff --git a/api/src/org/apache/cloudstack/api/command/admin/router/ListRoutersCmd.java b/api/src/org/apache/cloudstack/api/command/admin/router/ListRoutersCmd.java index 9fbc9401532..78c3554ae73 100644 --- a/api/src/org/apache/cloudstack/api/command/admin/router/ListRoutersCmd.java +++ b/api/src/org/apache/cloudstack/api/command/admin/router/ListRoutersCmd.java @@ -31,6 +31,7 @@ import org.apache.cloudstack.api.response.ZoneResponse; import org.apache.log4j.Logger; import com.cloud.async.AsyncJob; +import com.cloud.network.router.VirtualRouter.Role; @APICommand(name = "listRouters", description="List routers.", responseObject=DomainRouterResponse.class) public class ListRoutersCmd extends BaseListProjectAndAccountResourcesCmd { @@ -77,7 +78,7 @@ public class ListRoutersCmd extends BaseListProjectAndAccountResourcesCmd { @Parameter(name=ApiConstants.FOR_VPC, type=CommandType.BOOLEAN, description="if true is passed for this parameter, list only VPC routers") private Boolean forVpc; - + ///////////////////////////////////////////////////// /////////////////// Accessors /////////////////////// ///////////////////////////////////////////////////// @@ -121,6 +122,10 @@ public class ListRoutersCmd extends BaseListProjectAndAccountResourcesCmd { public Boolean getForVpc() { return forVpc; } + + public String getRole() { + return Role.VIRTUAL_ROUTER.toString(); + } ///////////////////////////////////////////////////// /////////////// API Implementation/////////////////// diff --git a/api/src/org/apache/cloudstack/api/command/admin/router/StartRouterCmd.java b/api/src/org/apache/cloudstack/api/command/admin/router/StartRouterCmd.java index 1d3930b6b63..ad0461e0eb7 100644 --- a/api/src/org/apache/cloudstack/api/command/admin/router/StartRouterCmd.java +++ b/api/src/org/apache/cloudstack/api/command/admin/router/StartRouterCmd.java @@ -29,8 +29,10 @@ import com.cloud.async.AsyncJob; import com.cloud.event.EventTypes; import com.cloud.exception.ConcurrentOperationException; import com.cloud.exception.InsufficientCapacityException; +import com.cloud.exception.InvalidParameterValueException; import com.cloud.exception.ResourceUnavailableException; import com.cloud.network.router.VirtualRouter; +import com.cloud.network.router.VirtualRouter.Role; import com.cloud.user.Account; import com.cloud.user.UserContext; @@ -100,7 +102,13 @@ public class StartRouterCmd extends BaseAsyncCmd { @Override public void execute() throws ConcurrentOperationException, ResourceUnavailableException, InsufficientCapacityException{ UserContext.current().setEventDetails("Router Id: "+getId()); - VirtualRouter result = _routerService.startRouter(id); + VirtualRouter result = null; + VirtualRouter router = _routerService.findRouter(getId()); + if (router == null || router.getRole() != Role.VIRTUAL_ROUTER) { + throw new InvalidParameterValueException("Can't find router by id"); + } else { + result = _routerService.startRouter(getId()); + } if (result != null){ DomainRouterResponse routerResponse = _responseGenerator.createDomainRouterResponse(result); routerResponse.setResponseName(getCommandName()); diff --git a/api/src/org/apache/cloudstack/api/command/admin/router/StopRouterCmd.java b/api/src/org/apache/cloudstack/api/command/admin/router/StopRouterCmd.java index 60dd9386c75..94473cf9ffc 100644 --- a/api/src/org/apache/cloudstack/api/command/admin/router/StopRouterCmd.java +++ b/api/src/org/apache/cloudstack/api/command/admin/router/StopRouterCmd.java @@ -28,8 +28,10 @@ import org.apache.log4j.Logger; import com.cloud.async.AsyncJob; import com.cloud.event.EventTypes; import com.cloud.exception.ConcurrentOperationException; +import com.cloud.exception.InvalidParameterValueException; import com.cloud.exception.ResourceUnavailableException; import com.cloud.network.router.VirtualRouter; +import com.cloud.network.router.VirtualRouter.Role; import com.cloud.user.Account; import com.cloud.user.UserContext; @@ -103,7 +105,14 @@ public class StopRouterCmd extends BaseAsyncCmd { @Override public void execute() throws ConcurrentOperationException, ResourceUnavailableException { UserContext.current().setEventDetails("Router Id: "+getId()); - VirtualRouter result = _routerService.stopRouter(getId(), isForced()); + VirtualRouter result = null; + VirtualRouter router = _routerService.findRouter(getId()); + if (router == null || router.getRole() != Role.VIRTUAL_ROUTER) { + throw new InvalidParameterValueException("Can't find router by id"); + } else { + result = _routerService.stopRouter(getId(), isForced()); + } + if (result != null) { DomainRouterResponse response = _responseGenerator.createDomainRouterResponse(result); response.setResponseName(getCommandName()); diff --git a/api/src/org/apache/cloudstack/api/command/user/loadbalancer/CreateApplicationLoadBalancerCmd.java b/api/src/org/apache/cloudstack/api/command/user/loadbalancer/CreateApplicationLoadBalancerCmd.java new file mode 100644 index 00000000000..17ae959aa6e --- /dev/null +++ b/api/src/org/apache/cloudstack/api/command/user/loadbalancer/CreateApplicationLoadBalancerCmd.java @@ -0,0 +1,218 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.api.command.user.loadbalancer; + +import org.apache.cloudstack.api.APICommand; +import org.apache.cloudstack.api.ApiConstants; +import org.apache.cloudstack.api.ApiErrorCode; +import org.apache.cloudstack.api.BaseAsyncCreateCmd; +import org.apache.cloudstack.api.Parameter; +import org.apache.cloudstack.api.ServerApiException; +import org.apache.cloudstack.api.response.ApplicationLoadBalancerResponse; +import org.apache.cloudstack.api.response.NetworkResponse; +import org.apache.cloudstack.network.lb.ApplicationLoadBalancerRule; +import org.apache.log4j.Logger; + +import com.cloud.async.AsyncJob; +import com.cloud.event.EventTypes; +import com.cloud.exception.InsufficientAddressCapacityException; +import com.cloud.exception.InsufficientVirtualNetworkCapcityException; +import com.cloud.exception.InvalidParameterValueException; +import com.cloud.exception.NetworkRuleConflictException; +import com.cloud.exception.ResourceAllocationException; +import com.cloud.exception.ResourceUnavailableException; +import com.cloud.network.Network; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; +import com.cloud.user.UserContext; +import com.cloud.utils.net.NetUtils; + +@APICommand(name = "createLoadBalancer", description="Creates a Load Balancer", responseObject=ApplicationLoadBalancerResponse.class, since="4.2.0") +public class CreateApplicationLoadBalancerCmd extends BaseAsyncCreateCmd { + public static final Logger s_logger = Logger.getLogger(CreateApplicationLoadBalancerCmd.class.getName()); + + private static final String s_name = "createloadbalancerresponse"; + + ///////////////////////////////////////////////////// + //////////////// API parameters ///////////////////// + ///////////////////////////////////////////////////// + @Parameter(name=ApiConstants.NAME, type=CommandType.STRING, required=true, description="name of the Load Balancer") + private String loadBalancerName; + + @Parameter(name=ApiConstants.DESCRIPTION, type=CommandType.STRING, description="the description of the Load Balancer", length=4096) + private String description; + + @Parameter(name=ApiConstants.NETWORK_ID, type=CommandType.UUID, required=true, entityType = NetworkResponse.class, + description="The guest network the Load Balancer will be created for") + private Long networkId; + + @Parameter(name=ApiConstants.SOURCE_PORT, type=CommandType.INTEGER, required=true, description="the source port the network traffic will be load balanced from") + private Integer sourcePort; + + @Parameter(name=ApiConstants.ALGORITHM, type=CommandType.STRING, required=true, description="load balancer algorithm (source, roundrobin, leastconn)") + private String algorithm; + + @Parameter(name=ApiConstants.INSTANCE_PORT, type=CommandType.INTEGER, required=true, description="the TCP port of the virtual machine where the network traffic will be load balanced to") + private Integer instancePort; + + @Parameter(name=ApiConstants.SOURCE_IP, type=CommandType.STRING, description="the source ip address the network traffic will be load balanced from") + private String sourceIp; + + @Parameter(name=ApiConstants.SOURCE_IP_NETWORK_ID, type=CommandType.UUID, entityType = NetworkResponse.class, required=true, + description="the network id of the source ip address") + private Long sourceIpNetworkId; + + @Parameter(name=ApiConstants.SCHEME, type=CommandType.STRING, required=true, description="the load balancer scheme. Supported value in this release is Internal") + private String scheme; + + + + ///////////////////////////////////////////////////// + /////////////////// Accessors /////////////////////// + ///////////////////////////////////////////////////// + + public String getAlgorithm() { + return algorithm; + } + + public String getDescription() { + return description; + } + + public String getLoadBalancerName() { + return loadBalancerName; + } + + public Integer getPrivatePort() { + return instancePort; + } + + public long getNetworkId() { + return networkId; + } + + public String getName() { + return loadBalancerName; + } + + public Integer getSourcePort() { + return sourcePort.intValue(); + } + + public String getProtocol() { + return NetUtils.TCP_PROTO; + } + + public long getAccountId() { + //get account info from the network object + Network ntwk = _networkService.getNetwork(networkId); + if (ntwk == null) { + throw new InvalidParameterValueException("Invalid network id specified"); + } + + return ntwk.getAccountId(); + + } + + public int getInstancePort() { + return instancePort.intValue(); + } + + @Override + public String getEventType() { + return EventTypes.EVENT_LOAD_BALANCER_CREATE; + } + + @Override + public String getEventDescription() { + return "creating load balancer: " + getName() + " account: " + getAccountId(); + + } + + @Override + public AsyncJob.Type getInstanceType() { + return AsyncJob.Type.LoadBalancerRule; + } + + public String getSourceIp() { + return sourceIp; + } + + public long getSourceIpNetworkId() { + return sourceIpNetworkId; + } + + public Scheme getScheme() { + if (scheme.equalsIgnoreCase(Scheme.Internal.toString())) { + return Scheme.Internal; + } else { + throw new InvalidParameterValueException("Invalid value for scheme. Supported value is Internal"); + } + } + + @Override + public long getEntityOwnerId() { + return getAccountId(); + } + + ///////////////////////////////////////////////////// + /////////////// API Implementation/////////////////// + ///////////////////////////////////////////////////// + @Override + public String getCommandName() { + return s_name; + } + + @Override + public void execute() throws ResourceAllocationException, ResourceUnavailableException { + ApplicationLoadBalancerRule rule = null; + try { + UserContext.current().setEventDetails("Load Balancer Id: " + getEntityId()); + // State might be different after the rule is applied, so get new object here + rule = _entityMgr.findById(ApplicationLoadBalancerRule.class, getEntityId()); + ApplicationLoadBalancerResponse lbResponse = _responseGenerator.createLoadBalancerContainerReponse(rule, _lbService.getLbInstances(getEntityId())); + setResponseObject(lbResponse); + lbResponse.setResponseName(getCommandName()); + } catch (Exception ex) { + s_logger.warn("Failed to create Load Balancer due to exception ", ex); + } finally { + if (rule == null) { + throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, "Failed to create Load Balancer"); + } + } + } + + @Override + public void create() { + try { + + ApplicationLoadBalancerRule result = _appLbService.createApplicationLoadBalancer(getName(), getDescription(), getScheme(), + getSourceIpNetworkId(), getSourceIp(), getSourcePort(), getInstancePort(), getAlgorithm(), getNetworkId(), getEntityOwnerId()); + this.setEntityId(result.getId()); + this.setEntityUuid(result.getUuid()); + }catch (NetworkRuleConflictException e) { + s_logger.warn("Exception: ", e); + throw new ServerApiException(ApiErrorCode.NETWORK_RULE_CONFLICT_ERROR, e.getMessage()); + } catch (InsufficientAddressCapacityException e) { + s_logger.warn("Exception: ", e); + throw new ServerApiException(ApiErrorCode.INSUFFICIENT_CAPACITY_ERROR, e.getMessage()); + } catch (InsufficientVirtualNetworkCapcityException e) { + s_logger.warn("Exception: ", e); + throw new ServerApiException(ApiErrorCode.INSUFFICIENT_CAPACITY_ERROR, e.getMessage()); + } + } +} + diff --git a/api/src/org/apache/cloudstack/api/command/user/loadbalancer/CreateLoadBalancerRuleCmd.java b/api/src/org/apache/cloudstack/api/command/user/loadbalancer/CreateLoadBalancerRuleCmd.java index 5f1d97b2803..f6cc1f130bd 100644 --- a/api/src/org/apache/cloudstack/api/command/user/loadbalancer/CreateLoadBalancerRuleCmd.java +++ b/api/src/org/apache/cloudstack/api/command/user/loadbalancer/CreateLoadBalancerRuleCmd.java @@ -148,7 +148,7 @@ public class CreateLoadBalancerRuleCmd extends BaseAsyncCreateCmd /*implements } - public Long getNetworkId() { + public long getNetworkId() { if (networkId != null) { return networkId; } @@ -278,7 +278,9 @@ public class CreateLoadBalancerRuleCmd extends BaseAsyncCreateCmd /*implements throw new InvalidParameterValueException("Parameter cidrList is deprecated; if you need to open firewall rule for the specific cidr, please refer to createFirewallRule command"); } try { - LoadBalancer result = _lbService.createLoadBalancerRule(this, getOpenFirewall()); + LoadBalancer result = _lbService.createPublicLoadBalancerRule(getXid(), getName(), getDescription(), + getSourcePortStart(), getSourcePortEnd(), getDefaultPortStart(), getDefaultPortEnd(), getSourceIpAddressId(), getProtocol(), getAlgorithm(), + getNetworkId(), getEntityOwnerId(), getOpenFirewall()); this.setEntityId(result.getId()); this.setEntityUuid(result.getUuid()); } catch (NetworkRuleConflictException e) { diff --git a/api/src/org/apache/cloudstack/api/command/user/loadbalancer/DeleteApplicationLoadBalancerCmd.java b/api/src/org/apache/cloudstack/api/command/user/loadbalancer/DeleteApplicationLoadBalancerCmd.java new file mode 100644 index 00000000000..bc6cd09526c --- /dev/null +++ b/api/src/org/apache/cloudstack/api/command/user/loadbalancer/DeleteApplicationLoadBalancerCmd.java @@ -0,0 +1,116 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.api.command.user.loadbalancer; + +import org.apache.cloudstack.api.APICommand; +import org.apache.cloudstack.api.ApiConstants; +import org.apache.cloudstack.api.ApiErrorCode; +import org.apache.cloudstack.api.BaseAsyncCmd; +import org.apache.cloudstack.api.Parameter; +import org.apache.cloudstack.api.ServerApiException; +import org.apache.cloudstack.api.response.FirewallRuleResponse; +import org.apache.cloudstack.api.response.SuccessResponse; +import org.apache.cloudstack.network.lb.ApplicationLoadBalancerRule; +import org.apache.log4j.Logger; + +import com.cloud.async.AsyncJob; +import com.cloud.event.EventTypes; +import com.cloud.exception.InvalidParameterValueException; +import com.cloud.user.UserContext; + +@APICommand(name = "deleteLoadBalancer", description="Deletes a load balancer", responseObject=SuccessResponse.class, since="4.2.0") +public class DeleteApplicationLoadBalancerCmd extends BaseAsyncCmd { + public static final Logger s_logger = Logger.getLogger(DeleteApplicationLoadBalancerCmd.class.getName()); + private static final String s_name = "deleteloadbalancerresponse"; + ///////////////////////////////////////////////////// + //////////////// API parameters ///////////////////// + ///////////////////////////////////////////////////// + + @Parameter(name=ApiConstants.ID, type=CommandType.UUID, entityType = FirewallRuleResponse.class, + required=true, description="the ID of the Load Balancer") + private Long id; + + + ///////////////////////////////////////////////////// + /////////////////// Accessors /////////////////////// + ///////////////////////////////////////////////////// + + public Long getId() { + return id; + } + + ///////////////////////////////////////////////////// + /////////////// API Implementation/////////////////// + ///////////////////////////////////////////////////// + + @Override + public String getCommandName() { + return s_name; + } + + @Override + public long getEntityOwnerId() { + ApplicationLoadBalancerRule lb = _entityMgr.findById(ApplicationLoadBalancerRule.class, getId()); + if (lb != null) { + return lb.getAccountId(); + } else { + throw new InvalidParameterValueException("Can't find load balancer by id specified"); + } + } + + @Override + public String getEventType() { + return EventTypes.EVENT_LOAD_BALANCER_DELETE; + } + + @Override + public String getEventDescription() { + return "deleting load balancer: " + getId(); + } + + @Override + public void execute(){ + UserContext.current().setEventDetails("Load balancer Id: " + getId()); + boolean result = _appLbService.deleteApplicationLoadBalancer(getId()); + + if (result) { + SuccessResponse response = new SuccessResponse(getCommandName()); + this.setResponseObject(response); + } else { + throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, "Failed to delete load balancer"); + } + } + + @Override + public String getSyncObjType() { + return BaseAsyncCmd.networkSyncObject; + } + + @Override + public Long getSyncObjId() { + ApplicationLoadBalancerRule lb = _appLbService.getApplicationLoadBalancer(id); + if(lb == null){ + throw new InvalidParameterValueException("Unable to find load balancer by id "); + } + return lb.getNetworkId(); + } + + @Override + public AsyncJob.Type getInstanceType() { + return AsyncJob.Type.FirewallRule; + } +} diff --git a/api/src/org/apache/cloudstack/api/command/user/loadbalancer/ListApplicationLoadBalancersCmd.java b/api/src/org/apache/cloudstack/api/command/user/loadbalancer/ListApplicationLoadBalancersCmd.java new file mode 100644 index 00000000000..8e5df31ed29 --- /dev/null +++ b/api/src/org/apache/cloudstack/api/command/user/loadbalancer/ListApplicationLoadBalancersCmd.java @@ -0,0 +1,131 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.api.command.user.loadbalancer; + +import java.util.ArrayList; +import java.util.List; + +import org.apache.cloudstack.api.APICommand; +import org.apache.cloudstack.api.ApiConstants; +import org.apache.cloudstack.api.BaseListTaggedResourcesCmd; +import org.apache.cloudstack.api.Parameter; +import org.apache.cloudstack.api.response.ApplicationLoadBalancerResponse; +import org.apache.cloudstack.api.response.FirewallRuleResponse; +import org.apache.cloudstack.api.response.ListResponse; +import org.apache.cloudstack.api.response.NetworkResponse; +import org.apache.cloudstack.network.lb.ApplicationLoadBalancerRule; +import org.apache.log4j.Logger; + +import com.cloud.exception.InvalidParameterValueException; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; +import com.cloud.utils.Pair; + +@APICommand(name = "listLoadBalancers", description = "Lists Load Balancers", responseObject = ApplicationLoadBalancerResponse.class, since="4.2.0") +public class ListApplicationLoadBalancersCmd extends BaseListTaggedResourcesCmd { + public static final Logger s_logger = Logger.getLogger(ListApplicationLoadBalancersCmd.class.getName()); + + private static final String s_name = "listloadbalancerssresponse"; + + // /////////////////////////////////////////////////// + // ////////////// API parameters ///////////////////// + // /////////////////////////////////////////////////// + + @Parameter(name = ApiConstants.ID, type = CommandType.UUID, entityType = FirewallRuleResponse.class, + description = "the ID of the Load Balancer") + private Long id; + + @Parameter(name = ApiConstants.NAME, type = CommandType.STRING, description = "the name of the Load Balancer") + private String loadBalancerName; + + @Parameter(name = ApiConstants.SOURCE_IP, type = CommandType.STRING, description = "the source ip address of the Load Balancer") + private String sourceIp; + + @Parameter(name=ApiConstants.SOURCE_IP_NETWORK_ID, type=CommandType.UUID, entityType = NetworkResponse.class, + description="the network id of the source ip address") + private Long sourceIpNetworkId; + + @Parameter(name = ApiConstants.SCHEME, type = CommandType.STRING, description = "the scheme of the Load Balancer. Supported value is Internal in the current release") + private String scheme; + + @Parameter(name=ApiConstants.NETWORK_ID, type=CommandType.UUID, entityType = NetworkResponse.class, + description="the network id of the Load Balancer") + private Long networkId; + + + // /////////////////////////////////////////////////// + // ///////////////// Accessors /////////////////////// + // /////////////////////////////////////////////////// + + public Long getId() { + return id; + } + + public String getLoadBalancerRuleName() { + return loadBalancerName; + } + + public String getLoadBalancerName() { + return loadBalancerName; + } + + public String getSourceIp() { + return sourceIp; + } + + public Long getSourceIpNetworkId() { + return sourceIpNetworkId; + } + + @Override + public String getCommandName() { + return s_name; + } + + public Scheme getScheme() { + if (scheme != null) { + if (scheme.equalsIgnoreCase(Scheme.Internal.toString())) { + return Scheme.Internal; + } else { + throw new InvalidParameterValueException("Invalid value for scheme. Supported value is Internal"); + } + } + return null; + } + + public Long getNetworkId() { + return networkId; + } + // /////////////////////////////////////////////////// + // ///////////// API Implementation/////////////////// + // /////////////////////////////////////////////////// + + @Override + public void execute() { + Pair, Integer> loadBalancers = _appLbService.listApplicationLoadBalancers(this); + ListResponse response = new ListResponse(); + List lbResponses = new ArrayList(); + for (ApplicationLoadBalancerRule loadBalancer : loadBalancers.first()) { + ApplicationLoadBalancerResponse lbResponse = _responseGenerator.createLoadBalancerContainerReponse(loadBalancer, _lbService.getLbInstances(loadBalancer.getId())); + lbResponse.setObjectName("loadbalancer"); + lbResponses.add(lbResponse); + } + response.setResponses(lbResponses, loadBalancers.second()); + response.setResponseName(getCommandName()); + this.setResponseObject(response); + } + +} diff --git a/api/src/org/apache/cloudstack/api/response/ApplicationLoadBalancerInstanceResponse.java b/api/src/org/apache/cloudstack/api/response/ApplicationLoadBalancerInstanceResponse.java new file mode 100644 index 00000000000..2d6614d217b --- /dev/null +++ b/api/src/org/apache/cloudstack/api/response/ApplicationLoadBalancerInstanceResponse.java @@ -0,0 +1,63 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.api.response; + +import org.apache.cloudstack.api.ApiConstants; +import org.apache.cloudstack.api.BaseResponse; + +import com.cloud.serializer.Param; +import com.google.gson.annotations.SerializedName; + +/** + * + * Load Balancer instance is the User Vm instance participating in the Load Balancer + * + */ + +@SuppressWarnings("unused") +public class ApplicationLoadBalancerInstanceResponse extends BaseResponse{ + + @SerializedName(ApiConstants.ID) @Param(description = "the instance ID") + private String id; + + @SerializedName(ApiConstants.NAME) @Param(description = "the name of the instance") + private String name; + + @SerializedName(ApiConstants.STATE) @Param(description="the state of the instance") + private String state; + + @SerializedName(ApiConstants.IP_ADDRESS) + @Param(description="the ip address of the instance") + private String ipAddress; + + + public void setId(String id) { + this.id = id; + } + + public void setName(String name) { + this.name = name; + } + + public void setState(String state) { + this.state = state; + } + + public void setIpAddress(String ipAddress) { + this.ipAddress = ipAddress; + } +} diff --git a/api/src/org/apache/cloudstack/api/response/ApplicationLoadBalancerResponse.java b/api/src/org/apache/cloudstack/api/response/ApplicationLoadBalancerResponse.java new file mode 100644 index 00000000000..de9bce6c658 --- /dev/null +++ b/api/src/org/apache/cloudstack/api/response/ApplicationLoadBalancerResponse.java @@ -0,0 +1,142 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.cloudstack.api.response; + +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Set; + +import org.apache.cloudstack.api.ApiConstants; +import org.apache.cloudstack.api.BaseResponse; + +import com.cloud.serializer.Param; +import com.google.gson.annotations.SerializedName; + +@SuppressWarnings("unused") +public class ApplicationLoadBalancerResponse extends BaseResponse implements ControlledEntityResponse{ + @SerializedName(ApiConstants.ID) @Param(description = "the Load Balancer ID") + private String id; + + @SerializedName(ApiConstants.NAME) @Param(description = "the name of the Load Balancer") + private String name; + + @SerializedName(ApiConstants.DESCRIPTION) @Param(description = "the description of the Load Balancer") + private String description; + + @SerializedName(ApiConstants.ALGORITHM) @Param(description = "the load balancer algorithm (source, roundrobin, leastconn)") + private String algorithm; + + @SerializedName(ApiConstants.NETWORK_ID) @Param(description="Load Balancer network id") + private String networkId; + + @SerializedName(ApiConstants.SOURCE_IP) @Param(description="Load Balancer source ip") + private String sourceIp; + + @SerializedName(ApiConstants.SOURCE_IP_NETWORK_ID) @Param(description="Load Balancer source ip network id") + private String sourceIpNetworkId; + + @SerializedName(ApiConstants.ACCOUNT) + @Param(description = "the account of the Load Balancer") + private String accountName; + + @SerializedName(ApiConstants.PROJECT_ID) @Param(description="the project id of the Load Balancer") + private String projectId; + + @SerializedName(ApiConstants.PROJECT) @Param(description="the project name of the Load Balancer") + private String projectName; + + @SerializedName(ApiConstants.DOMAIN_ID) + @Param(description = "the domain ID of the Load Balancer") + private String domainId; + + @SerializedName(ApiConstants.DOMAIN) + @Param(description = "the domain of the Load Balancer") + private String domainName; + + @SerializedName("loadbalancerrule") @Param(description="the list of rules associated with the Load Balancer", responseObject = ApplicationLoadBalancerRuleResponse.class) + private List lbRules; + + @SerializedName("loadbalancerinstance") @Param(description="the list of instances associated with the Load Balancer", responseObject = ApplicationLoadBalancerInstanceResponse.class) + private List lbInstances; + + @SerializedName(ApiConstants.TAGS) @Param(description="the list of resource tags associated with the Load Balancer", responseObject = ResourceTagResponse.class) + private List tags; + + public void setAccountName(String accountName) { + this.accountName = accountName; + } + + @Override + public void setDomainId(String domainId) { + this.domainId = domainId; + } + + public void setDomainName(String domainName) { + this.domainName = domainName; + } + + @Override + public void setProjectId(String projectId) { + this.projectId = projectId; + } + + @Override + public void setProjectName(String projectName) { + this.projectName = projectName; + } + + public void setTags(List tags) { + this.tags = tags; + } + + public void setId(String id) { + this.id = id; + } + + public void setName(String name) { + this.name = name; + } + + public void setDescription(String description) { + this.description = description; + } + + public void setAlgorithm(String algorithm) { + this.algorithm = algorithm; + } + + public void setNetworkId(String networkId) { + this.networkId = networkId; + } + + public void setSourceIp(String sourceIp) { + this.sourceIp = sourceIp; + } + + public void setSourceIpNetworkId(String sourceIpNetworkId) { + this.sourceIpNetworkId = sourceIpNetworkId; + } + + public void setLbRules(List lbRules) { + this.lbRules = lbRules; + } + + public void setLbInstances(List lbInstances) { + this.lbInstances = lbInstances; + } +} diff --git a/api/src/org/apache/cloudstack/api/response/ApplicationLoadBalancerRuleResponse.java b/api/src/org/apache/cloudstack/api/response/ApplicationLoadBalancerRuleResponse.java new file mode 100644 index 00000000000..ffc64d5ca46 --- /dev/null +++ b/api/src/org/apache/cloudstack/api/response/ApplicationLoadBalancerRuleResponse.java @@ -0,0 +1,51 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.cloudstack.api.response; + +import org.apache.cloudstack.api.ApiConstants; +import org.apache.cloudstack.api.BaseResponse; + +import com.cloud.serializer.Param; +import com.google.gson.annotations.SerializedName; + +/** + * Subobject of the load balancer container response + */ +@SuppressWarnings("unused") +public class ApplicationLoadBalancerRuleResponse extends BaseResponse{ + @SerializedName(ApiConstants.SOURCE_PORT) @Param(description = "source port of the load balancer rule") + private Integer sourcePort; + + @SerializedName(ApiConstants.INSTANCE_PORT) @Param(description = "instance port of the load balancer rule") + private Integer instancePort; + + @SerializedName(ApiConstants.STATE) @Param(description = "the state of the load balancer rule") + private String state; + + public void setSourcePort(Integer sourcePort) { + this.sourcePort = sourcePort; + } + + public void setInstancePort(Integer instancePort) { + this.instancePort = instancePort; + } + + public void setState(String state) { + this.state = state; + } +} diff --git a/api/src/org/apache/cloudstack/api/response/DomainRouterResponse.java b/api/src/org/apache/cloudstack/api/response/DomainRouterResponse.java index 79c8596a8d1..852d98815a3 100644 --- a/api/src/org/apache/cloudstack/api/response/DomainRouterResponse.java +++ b/api/src/org/apache/cloudstack/api/response/DomainRouterResponse.java @@ -153,8 +153,11 @@ public class DomainRouterResponse extends BaseResponse implements ControlledView @SerializedName("scriptsversion") @Param(description="the version of scripts") private String scriptsVersion; - @SerializedName(ApiConstants.VPC_ID) @Param(description="VPC the network belongs to") + @SerializedName(ApiConstants.VPC_ID) @Param(description="VPC the router belongs to") private String vpcId; + + @SerializedName(ApiConstants.ROLE) @Param(description="role of the domain router") + private String role; @SerializedName("nic") @Param(description="the list of nics associated with the router", responseObject = NicResponse.class, since="4.0") @@ -164,15 +167,11 @@ public class DomainRouterResponse extends BaseResponse implements ControlledView nics = new LinkedHashSet(); } - - @Override public String getObjectId() { return this.getId(); } - - public String getId() { return id; } @@ -372,4 +371,8 @@ public class DomainRouterResponse extends BaseResponse implements ControlledView public void setIp6Dns2(String ip6Dns2) { this.ip6Dns2 = ip6Dns2; } + + public void setRole(String role) { + this.role = role; + } } diff --git a/api/src/org/apache/cloudstack/api/response/InternalLoadBalancerElementResponse.java b/api/src/org/apache/cloudstack/api/response/InternalLoadBalancerElementResponse.java new file mode 100644 index 00000000000..b7e8634ee8f --- /dev/null +++ b/api/src/org/apache/cloudstack/api/response/InternalLoadBalancerElementResponse.java @@ -0,0 +1,51 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.api.response; + +import org.apache.cloudstack.api.ApiConstants; +import org.apache.cloudstack.api.BaseResponse; +import org.apache.cloudstack.api.EntityReference; + +import com.cloud.network.VirtualRouterProvider; +import com.cloud.serializer.Param; +import com.google.gson.annotations.SerializedName; + +@EntityReference(value=VirtualRouterProvider.class) +@SuppressWarnings("unused") +public class InternalLoadBalancerElementResponse extends BaseResponse { + @SerializedName(ApiConstants.ID) @Param(description="the id of the internal load balancer element") + private String id; + + @SerializedName(ApiConstants.NSP_ID) @Param(description="the physical network service provider id of the element") + private String nspId; + + @SerializedName(ApiConstants.ENABLED) @Param(description="Enabled/Disabled the element") + private Boolean enabled; + + + public void setId(String id) { + this.id = id; + } + + public void setNspId(String nspId) { + this.nspId = nspId; + } + + public void setEnabled(Boolean enabled) { + this.enabled = enabled; + } +} diff --git a/api/src/org/apache/cloudstack/api/response/NetworkOfferingResponse.java b/api/src/org/apache/cloudstack/api/response/NetworkOfferingResponse.java index b1dcd423117..7a7e371e180 100644 --- a/api/src/org/apache/cloudstack/api/response/NetworkOfferingResponse.java +++ b/api/src/org/apache/cloudstack/api/response/NetworkOfferingResponse.java @@ -18,6 +18,7 @@ package org.apache.cloudstack.api.response; import java.util.Date; import java.util.List; +import java.util.Map; import org.apache.cloudstack.api.ApiConstants; import org.apache.cloudstack.api.BaseResponse; @@ -83,6 +84,10 @@ public class NetworkOfferingResponse extends BaseResponse { @SerializedName(ApiConstants.IS_PERSISTENT) @Param(description="true if network offering supports persistent networks, false otherwise") private Boolean isPersistent; + + @SerializedName(ApiConstants.DETAILS) @Param(description="additional key/value details tied with network offering", since="4.2.0") + private Map details; + public void setId(String id) { this.id = id; @@ -156,5 +161,9 @@ public class NetworkOfferingResponse extends BaseResponse { public void setIsPersistent(Boolean isPersistent) { this.isPersistent = isPersistent; } + + public void setDetails(Map details) { + this.details = details; + } } diff --git a/api/src/org/apache/cloudstack/api/response/VirtualRouterProviderResponse.java b/api/src/org/apache/cloudstack/api/response/VirtualRouterProviderResponse.java index 92d9a1d0cc1..de355bd0c25 100644 --- a/api/src/org/apache/cloudstack/api/response/VirtualRouterProviderResponse.java +++ b/api/src/org/apache/cloudstack/api/response/VirtualRouterProviderResponse.java @@ -25,6 +25,7 @@ import com.cloud.serializer.Param; import com.google.gson.annotations.SerializedName; @EntityReference(value=VirtualRouterProvider.class) +@SuppressWarnings("unused") public class VirtualRouterProviderResponse extends BaseResponse implements ControlledEntityResponse { @SerializedName(ApiConstants.ID) @Param(description="the id of the router") private String id; diff --git a/api/src/org/apache/cloudstack/network/element/InternalLoadBalancerElementService.java b/api/src/org/apache/cloudstack/network/element/InternalLoadBalancerElementService.java new file mode 100644 index 00000000000..33a0c64058e --- /dev/null +++ b/api/src/org/apache/cloudstack/network/element/InternalLoadBalancerElementService.java @@ -0,0 +1,56 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.network.element; + +import java.util.List; + + +import com.cloud.network.VirtualRouterProvider; +import com.cloud.utils.component.PluggableService; + +public interface InternalLoadBalancerElementService extends PluggableService{ + /** + * Configures existing Internal Load Balancer Element (enables or disables it) + * @param id + * @param enable + * @return + */ + VirtualRouterProvider configureInternalLoadBalancerElement(long id, boolean enable); + + /** + * Adds Internal Load Balancer element to the Network Service Provider + * @param ntwkSvcProviderId + * @return + */ + VirtualRouterProvider addInternalLoadBalancerElement(long ntwkSvcProviderId); + + /** + * Retrieves existing Internal Load Balancer element + * @param id + * @return + */ + VirtualRouterProvider getInternalLoadBalancerElement(long id); + + /** + * Searches for existing Internal Load Balancer elements based on parameters passed to the call + * @param id + * @param ntwkSvsProviderId + * @param enabled + * @return + */ + List searchForInternalLoadBalancerElements(Long id, Long ntwkSvsProviderId, Boolean enabled); +} diff --git a/api/src/org/apache/cloudstack/network/lb/ApplicationLoadBalancerContainer.java b/api/src/org/apache/cloudstack/network/lb/ApplicationLoadBalancerContainer.java new file mode 100644 index 00000000000..df94d3d4338 --- /dev/null +++ b/api/src/org/apache/cloudstack/network/lb/ApplicationLoadBalancerContainer.java @@ -0,0 +1,28 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.network.lb; + +import com.cloud.network.rules.LoadBalancerContainer; +import com.cloud.utils.net.Ip; + +public interface ApplicationLoadBalancerContainer extends LoadBalancerContainer{ + + public Long getSourceIpNetworkId(); + + public Ip getSourceIp(); + +} diff --git a/api/src/org/apache/cloudstack/network/lb/ApplicationLoadBalancerRule.java b/api/src/org/apache/cloudstack/network/lb/ApplicationLoadBalancerRule.java new file mode 100644 index 00000000000..f4acb734c8b --- /dev/null +++ b/api/src/org/apache/cloudstack/network/lb/ApplicationLoadBalancerRule.java @@ -0,0 +1,24 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.cloudstack.network.lb; + +import com.cloud.network.rules.LoadBalancer; + +public interface ApplicationLoadBalancerRule extends ApplicationLoadBalancerContainer, LoadBalancer{ + int getInstancePort(); +} diff --git a/api/src/org/apache/cloudstack/network/lb/ApplicationLoadBalancerService.java b/api/src/org/apache/cloudstack/network/lb/ApplicationLoadBalancerService.java new file mode 100644 index 00000000000..b2ac358555b --- /dev/null +++ b/api/src/org/apache/cloudstack/network/lb/ApplicationLoadBalancerService.java @@ -0,0 +1,42 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.cloudstack.network.lb; + +import java.util.List; + +import org.apache.cloudstack.api.command.user.loadbalancer.ListApplicationLoadBalancersCmd; + +import com.cloud.exception.InsufficientAddressCapacityException; +import com.cloud.exception.InsufficientVirtualNetworkCapcityException; +import com.cloud.exception.NetworkRuleConflictException; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; +import com.cloud.utils.Pair; + +public interface ApplicationLoadBalancerService { + + ApplicationLoadBalancerRule createApplicationLoadBalancer(String name, String description, Scheme scheme, long sourceIpNetworkId, String sourceIp, + int sourcePort, int instancePort, String algorithm, long networkId, long lbOwnerId) throws InsufficientAddressCapacityException, + NetworkRuleConflictException, InsufficientVirtualNetworkCapcityException; + + boolean deleteApplicationLoadBalancer(long id); + + Pair, Integer> listApplicationLoadBalancers(ListApplicationLoadBalancersCmd cmd); + + ApplicationLoadBalancerRule getApplicationLoadBalancer(long ruleId); + +} diff --git a/api/src/org/apache/cloudstack/network/lb/InternalLoadBalancerVMService.java b/api/src/org/apache/cloudstack/network/lb/InternalLoadBalancerVMService.java new file mode 100644 index 00000000000..91cd88d91c1 --- /dev/null +++ b/api/src/org/apache/cloudstack/network/lb/InternalLoadBalancerVMService.java @@ -0,0 +1,34 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.network.lb; + +import com.cloud.exception.ConcurrentOperationException; +import com.cloud.exception.InsufficientCapacityException; +import com.cloud.exception.ResourceUnavailableException; +import com.cloud.exception.StorageUnavailableException; +import com.cloud.network.router.VirtualRouter; +import com.cloud.user.Account; + +public interface InternalLoadBalancerVMService { + + VirtualRouter startInternalLbVm(long internalLbVmId, Account caller, long callerUserId) + throws StorageUnavailableException, InsufficientCapacityException, ConcurrentOperationException, ResourceUnavailableException; + + VirtualRouter stopInternalLbVm(long vmId, boolean forced, Account caller, long callerUserId) + throws ConcurrentOperationException, ResourceUnavailableException; + +} diff --git a/api/src/org/apache/cloudstack/query/QueryService.java b/api/src/org/apache/cloudstack/query/QueryService.java index 443c5df65b5..2f50d63828c 100644 --- a/api/src/org/apache/cloudstack/query/QueryService.java +++ b/api/src/org/apache/cloudstack/query/QueryService.java @@ -18,6 +18,7 @@ package org.apache.cloudstack.query; import org.apache.cloudstack.affinity.AffinityGroupResponse; import org.apache.cloudstack.api.command.admin.host.ListHostsCmd; +import org.apache.cloudstack.api.command.admin.internallb.ListInternalLBVMsCmd; import org.apache.cloudstack.api.command.admin.router.ListRoutersCmd; import org.apache.cloudstack.api.command.admin.storage.ListStoragePoolsCmd; import org.apache.cloudstack.api.command.admin.user.ListUsersCmd; @@ -101,4 +102,6 @@ public interface QueryService { public ListResponse listAffinityGroups(Long affinityGroupId, String affinityGroupName, String affinityGroupType, Long vmId, Long startIndex, Long pageSize); + + ListResponse searchForInternalLbVms(ListInternalLBVMsCmd cmd); } diff --git a/client/pom.xml b/client/pom.xml index 147959bd7f0..197ba27975c 100644 --- a/client/pom.xml +++ b/client/pom.xml @@ -85,6 +85,11 @@ cloud-plugin-network-midonet ${project.version} + + org.apache.cloudstack + cloud-plugin-network-internallb + ${project.version} + org.apache.cloudstack cloud-plugin-hypervisor-xen diff --git a/client/tomcatconf/applicationContext.xml.in b/client/tomcatconf/applicationContext.xml.in index 36f232c737c..67c8ccf9355 100644 --- a/client/tomcatconf/applicationContext.xml.in +++ b/client/tomcatconf/applicationContext.xml.in @@ -363,6 +363,9 @@ + + + diff --git a/client/tomcatconf/commands.properties.in b/client/tomcatconf/commands.properties.in index 687c3c16c75..cdc19929c19 100644 --- a/client/tomcatconf/commands.properties.in +++ b/client/tomcatconf/commands.properties.in @@ -577,6 +577,17 @@ revertToVMSnapshot=15 #### Baremetal commands addBaremetalHost=1 +#### New Load Balancer commands +createLoadBalancer=15 +listLoadBalancers=15 +deleteLoadBalancer=15 + +#Internal Load Balancer Element commands +configureInternalLoadBalancerElement=1 +createInternalLoadBalancerElement=1 +listInternalLoadBalancerElements=1 + + #### Affinity group commands createAffinityGroup=15 deleteAffinityGroup=15 @@ -594,5 +605,10 @@ addCiscoAsa1000vResource=1 deleteCiscoAsa1000vResource=1 listCiscoAsa1000vResources=1 +#### Internal LB VM commands +stopInternalLoadBalancerVM=1 +startInternalLoadBalancerVM=1 +listInternalLoadBalancerVMs=1 + ### Network Isolation methods listing listNetworkIsolationMethods=1 diff --git a/client/tomcatconf/componentContext.xml.in b/client/tomcatconf/componentContext.xml.in index 7a469816f82..8a45e5fea85 100644 --- a/client/tomcatconf/componentContext.xml.in +++ b/client/tomcatconf/componentContext.xml.in @@ -198,6 +198,7 @@ + @@ -241,6 +242,7 @@ + + + 4.0.0 + cloud-plugin-network-internallb + Apache CloudStack Plugin - Network Internal Load Balancer + + org.apache.cloudstack + cloudstack-plugins + 4.2.0-SNAPSHOT + ../../pom.xml + + + install + src + test + + + resources + + **/*.xml + + + + + + test/resources + + %regex[.*[0-9]*To[0-9]*.*Test.*] + + + + + diff --git a/plugins/network-elements/internal-loadbalancer/src/org/apache/cloudstack/network/element/InternalLoadBalancerElement.java b/plugins/network-elements/internal-loadbalancer/src/org/apache/cloudstack/network/element/InternalLoadBalancerElement.java new file mode 100644 index 00000000000..4b9308b6606 --- /dev/null +++ b/plugins/network-elements/internal-loadbalancer/src/org/apache/cloudstack/network/element/InternalLoadBalancerElement.java @@ -0,0 +1,548 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.cloudstack.network.element; + +import java.util.ArrayList; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; + +import javax.ejb.Local; +import javax.inject.Inject; + +import org.apache.cloudstack.api.command.admin.internallb.ConfigureInternalLoadBalancerElementCmd; +import org.apache.cloudstack.api.command.admin.internallb.CreateInternalLoadBalancerElementCmd; +import org.apache.cloudstack.api.command.admin.internallb.ListInternalLoadBalancerElementsCmd; +import org.apache.cloudstack.lb.dao.ApplicationLoadBalancerRuleDao; +import org.apache.cloudstack.network.lb.InternalLoadBalancerVMManager; +import org.apache.log4j.Logger; + +import com.cloud.agent.api.to.LoadBalancerTO; +import com.cloud.configuration.ConfigurationManager; +import com.cloud.dc.DataCenter; +import com.cloud.dc.DataCenter.NetworkType; +import com.cloud.deploy.DeployDestination; +import com.cloud.exception.ConcurrentOperationException; +import com.cloud.exception.InsufficientCapacityException; +import com.cloud.exception.InvalidParameterValueException; +import com.cloud.exception.ResourceUnavailableException; +import com.cloud.network.Network; +import com.cloud.network.Network.Capability; +import com.cloud.network.Network.Provider; +import com.cloud.network.Network.Service; +import com.cloud.network.NetworkModel; +import com.cloud.network.Networks.TrafficType; +import com.cloud.network.PhysicalNetworkServiceProvider; +import com.cloud.network.PublicIpAddress; +import com.cloud.network.VirtualRouterProvider; +import com.cloud.network.VirtualRouterProvider.VirtualRouterProviderType; +import com.cloud.network.dao.NetworkServiceMapDao; +import com.cloud.network.dao.PhysicalNetworkServiceProviderDao; +import com.cloud.network.dao.VirtualRouterProviderDao; +import com.cloud.network.element.IpDeployer; +import com.cloud.network.element.LoadBalancingServiceProvider; +import com.cloud.network.element.NetworkElement; +import com.cloud.network.element.VirtualRouterElement; +import com.cloud.network.element.VirtualRouterProviderVO; +import com.cloud.network.lb.LoadBalancingRule; +import com.cloud.network.router.VirtualRouter; +import com.cloud.network.router.VirtualRouter.Role; +import com.cloud.network.rules.FirewallRule; +import com.cloud.network.rules.LoadBalancerContainer; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; +import com.cloud.offering.NetworkOffering; +import com.cloud.user.Account; +import com.cloud.user.AccountManager; +import com.cloud.user.User; +import com.cloud.utils.component.AdapterBase; +import com.cloud.utils.db.SearchCriteria.Op; +import com.cloud.utils.db.SearchCriteria2; +import com.cloud.utils.db.SearchCriteriaService; +import com.cloud.utils.exception.CloudRuntimeException; +import com.cloud.utils.net.Ip; +import com.cloud.vm.DomainRouterVO; +import com.cloud.vm.NicProfile; +import com.cloud.vm.ReservationContext; +import com.cloud.vm.VirtualMachine; +import com.cloud.vm.VirtualMachineProfile; +import com.cloud.vm.dao.DomainRouterDao; + +@Local(value = {NetworkElement.class}) +public class InternalLoadBalancerElement extends AdapterBase implements LoadBalancingServiceProvider, InternalLoadBalancerElementService, IpDeployer{ + private static final Logger s_logger = Logger.getLogger(InternalLoadBalancerElement.class); + protected static final Map> capabilities = setCapabilities(); + private static InternalLoadBalancerElement internalLbElement = null; + + @Inject NetworkModel _ntwkModel; + @Inject NetworkServiceMapDao _ntwkSrvcDao; + @Inject DomainRouterDao _routerDao; + @Inject VirtualRouterProviderDao _vrProviderDao; + @Inject PhysicalNetworkServiceProviderDao _pNtwkSvcProviderDao; + @Inject InternalLoadBalancerVMManager _internalLbMgr; + @Inject ConfigurationManager _configMgr; + @Inject AccountManager _accountMgr; + @Inject ApplicationLoadBalancerRuleDao _appLbDao; + + protected InternalLoadBalancerElement() { + } + + + public static InternalLoadBalancerElement getInstance() { + if ( internalLbElement == null) { + internalLbElement = new InternalLoadBalancerElement(); + } + return internalLbElement; + } + + + private boolean canHandle(Network config, Scheme lbScheme) { + //works in Advance zone only + DataCenter dc = _configMgr.getZone(config.getDataCenterId()); + if (dc.getNetworkType() != NetworkType.Advanced) { + s_logger.trace("Not hanling zone of network type " + dc.getNetworkType()); + return false; + } + if (config.getGuestType() != Network.GuestType.Isolated || config.getTrafficType() != TrafficType.Guest) { + s_logger.trace("Not handling network with Type " + config.getGuestType() + " and traffic type " + config.getTrafficType()); + return false; + } + + Map lbCaps = this.getCapabilities().get(Service.Lb); + if (!lbCaps.isEmpty()) { + String schemeCaps = lbCaps.get(Capability.LbSchemes); + if (schemeCaps != null && lbScheme != null) { + if (!schemeCaps.contains(lbScheme.toString())) { + s_logger.debug("Scheme " + lbScheme.toString() + " is not supported by the provider " + this.getName()); + return false; + } + } + } + + if (!_ntwkModel.isProviderSupportServiceInNetwork(config.getId(), Service.Lb, getProvider())) { + s_logger.trace("Element " + getProvider().getName() + " doesn't support service " + Service.Lb + + " in the network " + config); + return false; + } + return true; + } + + + @Override + public Map> getCapabilities() { + return capabilities; + } + + + @Override + public Provider getProvider() { + return Provider.InternalLbVm; + } + + + @Override + public boolean implement(Network network, NetworkOffering offering, DeployDestination dest, ReservationContext context) + throws ConcurrentOperationException, ResourceUnavailableException, + InsufficientCapacityException { + + if (!canHandle(network, null)) { + s_logger.trace("No need to implement " + this.getName()); + return true; + } + + //1) Get all the Ips from the network having LB rules assigned + List ips = _appLbDao.listLbIpsBySourceIpNetworkIdAndScheme(network.getId(), Scheme.Internal); + + //2) Start those vms + for (String ip : ips) { + Ip sourceIp = new Ip(ip); + List internalLbVms; + try { + internalLbVms = _internalLbMgr.deployInternalLbVm(network, sourceIp, dest, _accountMgr.getAccount(network.getAccountId()), null); + } catch (InsufficientCapacityException e) { + s_logger.warn("Failed to deploy element " + this.getName() + " for ip " + sourceIp + " due to:", e); + return false; + } catch (ConcurrentOperationException e) { + s_logger.warn("Failed to deploy element " + this.getName() + " for ip " + sourceIp + " due to:", e); + return false; + } + + if (internalLbVms == null || internalLbVms.isEmpty()) { + throw new ResourceUnavailableException("Can't deploy " + this.getName() + " to handle LB rules", + DataCenter.class, network.getDataCenterId()); + } + } + + return true; + } + + + @Override + public boolean prepare(Network network, NicProfile nic, VirtualMachineProfile vm, DeployDestination dest, ReservationContext context) throws ConcurrentOperationException, + ResourceUnavailableException, InsufficientCapacityException { + + if (!canHandle(network, null)) { + s_logger.trace("No need to prepare " + this.getName()); + return true; + } + + if (vm.getType() == VirtualMachine.Type.User) { + //1) Get all the Ips from the network having LB rules assigned + List ips = _appLbDao.listLbIpsBySourceIpNetworkIdAndScheme(network.getId(), Scheme.Internal); + + //2) Start those vms + for (String ip : ips) { + Ip sourceIp = new Ip(ip); + List internalLbVms; + try { + internalLbVms = _internalLbMgr.deployInternalLbVm(network, sourceIp, dest, _accountMgr.getAccount(network.getAccountId()), null); + } catch (InsufficientCapacityException e) { + s_logger.warn("Failed to deploy element " + this.getName() + " for ip " + sourceIp + " due to:", e); + return false; + } catch (ConcurrentOperationException e) { + s_logger.warn("Failed to deploy element " + this.getName() + " for ip " + sourceIp + " due to:", e); + return false; + } + + if (internalLbVms == null || internalLbVms.isEmpty()) { + throw new ResourceUnavailableException("Can't deploy " + this.getName() + " to handle LB rules", + DataCenter.class, network.getDataCenterId()); + } + } + } + + return true; + } + + @Override + public boolean release(Network network, NicProfile nic, VirtualMachineProfile vm, ReservationContext context) throws ConcurrentOperationException, ResourceUnavailableException { + return true; + } + + + @Override + public boolean shutdown(Network network, ReservationContext context, boolean cleanup) throws ConcurrentOperationException, ResourceUnavailableException { + List internalLbVms = _routerDao.listByNetworkAndRole(network.getId(), Role.INTERNAL_LB_VM); + if (internalLbVms == null || internalLbVms.isEmpty()) { + return true; + } + boolean result = true; + for (VirtualRouter internalLbVm : internalLbVms) { + result = result && _internalLbMgr.destroyInternalLbVm(internalLbVm.getId(), + context.getAccount(), context.getCaller().getId()); + if (cleanup) { + if (!result) { + s_logger.warn("Failed to stop internal lb element " + internalLbVm + ", but would try to process clean up anyway."); + } + result = (_internalLbMgr.destroyInternalLbVm(internalLbVm.getId(), + context.getAccount(), context.getCaller().getId())); + if (!result) { + s_logger.warn("Failed to clean up internal lb element " + internalLbVm); + } + } + } + return result; + } + + + @Override + public boolean destroy(Network network, ReservationContext context) throws ConcurrentOperationException, ResourceUnavailableException { + List internalLbVms = _routerDao.listByNetworkAndRole(network.getId(), Role.INTERNAL_LB_VM); + if (internalLbVms == null || internalLbVms.isEmpty()) { + return true; + } + boolean result = true; + for (VirtualRouter internalLbVm : internalLbVms) { + result = result && (_internalLbMgr.destroyInternalLbVm(internalLbVm.getId(), + context.getAccount(), context.getCaller().getId())); + } + return result; + } + + + @Override + public boolean isReady(PhysicalNetworkServiceProvider provider) { + VirtualRouterProviderVO element = _vrProviderDao.findByNspIdAndType(provider.getId(), + VirtualRouterProviderType.InternalLbVm); + if (element == null) { + return false; + } + return element.isEnabled(); + } + + + @Override + public boolean shutdownProviderInstances(PhysicalNetworkServiceProvider provider, ReservationContext context) + throws ConcurrentOperationException, ResourceUnavailableException { + VirtualRouterProviderVO element = _vrProviderDao.findByNspIdAndType(provider.getId(), + VirtualRouterProviderType.InternalLbVm); + if (element == null) { + return true; + } + long elementId = element.getId(); + List internalLbVms = _routerDao.listByElementId(elementId); + boolean result = true; + for (DomainRouterVO internalLbVm : internalLbVms) { + result = result && (_internalLbMgr.destroyInternalLbVm(internalLbVm.getId(), + context.getAccount(), context.getCaller().getId())); + } + _vrProviderDao.remove(elementId); + + return result; + } + + + @Override + public boolean canEnableIndividualServices() { + return true; + } + + + @Override + public boolean verifyServicesCombination(Set services) { + return true; + } + + + @Override + public IpDeployer getIpDeployer(Network network) { + return this; + } + + + @Override + public boolean applyLBRules(Network network, List rules) throws ResourceUnavailableException { + //1) Get Internal LB VMs to destroy + Set vmsToDestroy = getVmsToDestroy(rules); + + //2) Get rules to apply + Map> rulesToApply = getLbRulesToApply(rules); + s_logger.debug("Applying " + rulesToApply.size() + " on element " + this.getName()); + + + for (Ip sourceIp : rulesToApply.keySet()) { + if (vmsToDestroy.contains(sourceIp)) { + //2.1 Destroy internal lb vm + List vms = _internalLbMgr.findInternalLbVms(network.getId(), sourceIp); + if (vms.size() > 0) { + //only one internal lb per IP exists + try { + s_logger.debug("Destroying internal lb vm for ip " + sourceIp.addr() + " as all the rules for this vm are in Revoke state"); + return _internalLbMgr.destroyInternalLbVm(vms.get(0).getId(), _accountMgr.getAccount(Account.ACCOUNT_ID_SYSTEM), + _accountMgr.getUserIncludingRemoved(User.UID_SYSTEM).getId()); + } catch (ConcurrentOperationException e) { + s_logger.warn("Failed to apply lb rule(s) for ip " + sourceIp.addr() + " on the element " + this.getName() + " due to:", e); + return false; + } + } + } else { + //2.2 Start Internal LB vm per IP address + List internalLbVms; + try { + DeployDestination dest = new DeployDestination(_configMgr.getZone(network.getDataCenterId()), null, null, null); + internalLbVms = _internalLbMgr.deployInternalLbVm(network, sourceIp, dest, _accountMgr.getAccount(network.getAccountId()), null); + } catch (InsufficientCapacityException e) { + s_logger.warn("Failed to apply lb rule(s) for ip " + sourceIp.addr() + "on the element " + this.getName() + " due to:", e); + return false; + } catch (ConcurrentOperationException e) { + s_logger.warn("Failed to apply lb rule(s) for ip " + sourceIp.addr() + "on the element " + this.getName() + " due to:", e); + return false; + } + + if (internalLbVms == null || internalLbVms.isEmpty()) { + throw new ResourceUnavailableException("Can't find/deploy internal lb vm to handle LB rules", + DataCenter.class, network.getDataCenterId()); + } + + //2.3 Apply Internal LB rules on the VM + if (!_internalLbMgr.applyLoadBalancingRules(network, rulesToApply.get(sourceIp), internalLbVms)) { + throw new CloudRuntimeException("Failed to apply load balancing rules for ip " + sourceIp.addr() + + " in network " + network.getId() + " on element " + this.getName()); + } + } + } + + return true; + } + + + protected Map> getLbRulesToApply(List rules) { + //Group rules by the source ip address as NetworkManager always passes the entire network lb config to the element + Map> rulesToApply = groupBySourceIp(rules); + + return rulesToApply; + } + + + protected Set getVmsToDestroy(List rules) { + //1) Group rules by the source ip address as NetworkManager always passes the entire network lb config to the element + Map> groupedRules = groupBySourceIp(rules); + + //2) Count rules in revoke state + Set vmsToDestroy = new HashSet(); + + for (Ip sourceIp : groupedRules.keySet()) { + List rulesToCheck = groupedRules.get(sourceIp); + int revoke = 0; + for (LoadBalancingRule ruleToCheck : rulesToCheck) { + if (ruleToCheck.getState() == FirewallRule.State.Revoke){ + revoke++; + } + } + + if (revoke == rulesToCheck.size()) { + s_logger.debug("Have to destroy internal lb vm for source ip " + sourceIp); + vmsToDestroy.add(sourceIp); + } + } + return vmsToDestroy; + } + + + protected Map> groupBySourceIp(List rules) { + Map> groupedRules = new HashMap>(); + for (LoadBalancingRule rule : rules) { + Ip sourceIp = rule.getSourceIp(); + if (!groupedRules.containsKey(sourceIp)) { + groupedRules.put(sourceIp, null); + } + + List rulesToApply = groupedRules.get(sourceIp); + if (rulesToApply == null) { + rulesToApply = new ArrayList(); + } + rulesToApply.add(rule); + groupedRules.put(sourceIp, rulesToApply); + } + return groupedRules; + } + + @Override + public boolean validateLBRule(Network network, LoadBalancingRule rule) { + List rules = new ArrayList(); + rules.add(rule); + if (canHandle(network, rule.getScheme())) { + List routers = _routerDao.listByNetworkAndRole(network.getId(), Role.INTERNAL_LB_VM); + if (routers == null || routers.isEmpty()) { + return true; + } + return VirtualRouterElement.validateHAProxyLBRule(rule); + } + return true; + } + + @Override + public List updateHealthChecks(Network network, List lbrules) { + return null; + } + + private static Map> setCapabilities() { + Map> capabilities = new HashMap>(); + + // Set capabilities for LB service + Map lbCapabilities = new HashMap(); + lbCapabilities.put(Capability.SupportedLBAlgorithms, "roundrobin,leastconn,source"); + lbCapabilities.put(Capability.SupportedLBIsolation, "dedicated"); + lbCapabilities.put(Capability.SupportedProtocols, "tcp, udp"); + lbCapabilities.put(Capability.SupportedStickinessMethods, VirtualRouterElement.getHAProxyStickinessCapability()); + lbCapabilities.put(Capability.LbSchemes, LoadBalancerContainer.Scheme.Internal.toString()); + + capabilities.put(Service.Lb, lbCapabilities); + return capabilities; + } + + @Override + public List> getCommands() { + List> cmdList = new ArrayList>(); + cmdList.add(CreateInternalLoadBalancerElementCmd.class); + cmdList.add(ConfigureInternalLoadBalancerElementCmd.class); + cmdList.add(ListInternalLoadBalancerElementsCmd.class); + return cmdList; + } + + @Override + public VirtualRouterProvider configureInternalLoadBalancerElement(long id, boolean enable) { + VirtualRouterProviderVO element = _vrProviderDao.findById(id); + if (element == null || element.getType() != VirtualRouterProviderType.InternalLbVm) { + throw new InvalidParameterValueException("Can't find " + this.getName() + " element with network service provider id " + id + + " to be used as a provider for " + this.getName()); + } + + element.setEnabled(enable); + element = _vrProviderDao.persist(element); + + return element; + } + + @Override + public VirtualRouterProvider addInternalLoadBalancerElement(long ntwkSvcProviderId) { + VirtualRouterProviderVO element = _vrProviderDao.findByNspIdAndType(ntwkSvcProviderId, VirtualRouterProviderType.InternalLbVm); + if (element != null) { + s_logger.debug("There is already an " + this.getName() + " with service provider id " + ntwkSvcProviderId); + return null; + } + + PhysicalNetworkServiceProvider provider = _pNtwkSvcProviderDao.findById(ntwkSvcProviderId); + if (provider == null || !provider.getProviderName().equalsIgnoreCase(this.getName())) { + throw new InvalidParameterValueException("Invalid network service provider is specified"); + } + + element = new VirtualRouterProviderVO(ntwkSvcProviderId, VirtualRouterProviderType.InternalLbVm); + element = _vrProviderDao.persist(element); + return element; + } + + + @Override + public VirtualRouterProvider getInternalLoadBalancerElement(long id) { + VirtualRouterProvider provider = _vrProviderDao.findById(id); + if (provider == null || provider.getType() != VirtualRouterProviderType.InternalLbVm) { + throw new InvalidParameterValueException("Unable to find " + this.getName() + " by id"); + } + return provider; + } + + @Override + public List searchForInternalLoadBalancerElements(Long id, Long ntwkSvsProviderId, Boolean enabled) { + + SearchCriteriaService sc = SearchCriteria2.create(VirtualRouterProviderVO.class); + if (id != null) { + sc.addAnd(sc.getEntity().getId(), Op.EQ, id); + } + if (ntwkSvsProviderId != null) { + sc.addAnd(sc.getEntity().getNspId(), Op.EQ, ntwkSvsProviderId); + } + if (enabled != null) { + sc.addAnd(sc.getEntity().isEnabled(), Op.EQ, enabled); + } + + //return only Internal LB elements + sc.addAnd(sc.getEntity().getType(), Op.EQ, VirtualRouterProvider.VirtualRouterProviderType.InternalLbVm); + + return sc.list(); + } + + @Override + public boolean applyIps(Network network, List ipAddress, Set services) throws ResourceUnavailableException { + //do nothing here; this element just has to extend the ip deployer + //as the LB service implements IPDeployerRequester + return true; + } + +} diff --git a/plugins/network-elements/internal-loadbalancer/src/org/apache/cloudstack/network/lb/InternalLoadBalancerVMManager.java b/plugins/network-elements/internal-loadbalancer/src/org/apache/cloudstack/network/lb/InternalLoadBalancerVMManager.java new file mode 100644 index 00000000000..9faca562bfb --- /dev/null +++ b/plugins/network-elements/internal-loadbalancer/src/org/apache/cloudstack/network/lb/InternalLoadBalancerVMManager.java @@ -0,0 +1,90 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.network.lb; + +import java.util.List; +import java.util.Map; + +import com.cloud.deploy.DeployDestination; +import com.cloud.exception.ConcurrentOperationException; +import com.cloud.exception.InsufficientCapacityException; +import com.cloud.exception.ResourceUnavailableException; +import com.cloud.network.Network; +import com.cloud.network.lb.LoadBalancingRule; +import com.cloud.network.router.VirtualRouter; +import com.cloud.user.Account; +import com.cloud.utils.component.Manager; +import com.cloud.utils.net.Ip; +import com.cloud.vm.VirtualMachineProfile.Param; + +public interface InternalLoadBalancerVMManager extends Manager, InternalLoadBalancerVMService{ + //RAM/CPU for the system offering used by Internal LB VMs + public static final int DEFAULT_INTERNALLB_VM_RAMSIZE = 128; // 128 MB + public static final int DEFAULT_INTERNALLB_VM_CPU_MHZ = 256; // 256 MHz + + /** + * Destroys Internal LB vm instance + * @param vmId + * @param caller + * @param callerUserId + * @return + * @throws ResourceUnavailableException + * @throws ConcurrentOperationException + */ + boolean destroyInternalLbVm(long vmId, Account caller, Long callerUserId) + throws ResourceUnavailableException, ConcurrentOperationException; + + + /** + * Deploys internal lb vm + * @param guestNetwork + * @param requestedGuestIp + * @param dest + * @param owner + * @param params + * @return + * @throws InsufficientCapacityException + * @throws ConcurrentOperationException + * @throws ResourceUnavailableException + */ + List deployInternalLbVm(Network guestNetwork, Ip requestedGuestIp, DeployDestination dest, Account owner, + Map params) throws InsufficientCapacityException, + ConcurrentOperationException, ResourceUnavailableException; + + + + /** + * + * @param network + * @param rules + * @param internalLbVms + * @return + * @throws ResourceUnavailableException + */ + boolean applyLoadBalancingRules(Network network, List rules, List internalLbVms) + throws ResourceUnavailableException; + + + /** + * Returns existing Internal Load Balancer elements based on guestNetworkId (required) and requestedIp (optional) + * @param guestNetworkId + * @param requestedGuestIp + * @return + */ + List findInternalLbVms(long guestNetworkId, Ip requestedGuestIp); + +} diff --git a/plugins/network-elements/internal-loadbalancer/src/org/apache/cloudstack/network/lb/InternalLoadBalancerVMManagerImpl.java b/plugins/network-elements/internal-loadbalancer/src/org/apache/cloudstack/network/lb/InternalLoadBalancerVMManagerImpl.java new file mode 100644 index 00000000000..fe32a7ba26f --- /dev/null +++ b/plugins/network-elements/internal-loadbalancer/src/org/apache/cloudstack/network/lb/InternalLoadBalancerVMManagerImpl.java @@ -0,0 +1,951 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.network.lb; + +import java.util.ArrayList; +import java.util.Iterator; +import java.util.List; +import java.util.Map; + +import javax.ejb.Local; +import javax.inject.Inject; +import javax.naming.ConfigurationException; + +import org.apache.cloudstack.lb.ApplicationLoadBalancerRuleVO; +import org.apache.cloudstack.lb.dao.ApplicationLoadBalancerRuleDao; +import org.apache.log4j.Logger; +import org.springframework.stereotype.Component; + +import com.cloud.agent.AgentManager; +import com.cloud.agent.AgentManager.OnError; +import com.cloud.agent.api.Answer; +import com.cloud.agent.api.GetDomRVersionAnswer; +import com.cloud.agent.api.GetDomRVersionCmd; +import com.cloud.agent.api.StopAnswer; +import com.cloud.agent.api.check.CheckSshAnswer; +import com.cloud.agent.api.check.CheckSshCommand; +import com.cloud.agent.api.routing.LoadBalancerConfigCommand; +import com.cloud.agent.api.routing.NetworkElementCommand; +import com.cloud.agent.api.to.LoadBalancerTO; +import com.cloud.agent.api.to.NicTO; +import com.cloud.agent.api.to.VirtualMachineTO; +import com.cloud.agent.manager.Commands; +import com.cloud.configuration.Config; +import com.cloud.configuration.dao.ConfigurationDao; +import com.cloud.dc.DataCenter; +import com.cloud.dc.DataCenterVO; +import com.cloud.dc.dao.DataCenterDao; +import com.cloud.deploy.DataCenterDeployment; +import com.cloud.deploy.DeployDestination; +import com.cloud.deploy.DeploymentPlan; +import com.cloud.exception.AgentUnavailableException; +import com.cloud.exception.ConcurrentOperationException; +import com.cloud.exception.InsufficientAddressCapacityException; +import com.cloud.exception.InsufficientCapacityException; +import com.cloud.exception.InsufficientServerCapacityException; +import com.cloud.exception.InvalidParameterValueException; +import com.cloud.exception.OperationTimedoutException; +import com.cloud.exception.ResourceUnavailableException; +import com.cloud.exception.StorageUnavailableException; +import com.cloud.hypervisor.Hypervisor.HypervisorType; +import com.cloud.network.Network; +import com.cloud.network.Network.Provider; +import com.cloud.network.Network.Service; +import com.cloud.network.NetworkManager; +import com.cloud.network.NetworkModel; +import com.cloud.network.Networks.TrafficType; +import com.cloud.network.PhysicalNetworkServiceProvider; +import com.cloud.network.VirtualRouterProvider; +import com.cloud.network.VirtualRouterProvider.VirtualRouterProviderType; +import com.cloud.network.dao.NetworkDao; +import com.cloud.network.dao.NetworkVO; +import com.cloud.network.dao.PhysicalNetworkServiceProviderDao; +import com.cloud.network.dao.VirtualRouterProviderDao; +import com.cloud.network.lb.LoadBalancingRule; +import com.cloud.network.lb.LoadBalancingRule.LbDestination; +import com.cloud.network.lb.LoadBalancingRule.LbHealthCheckPolicy; +import com.cloud.network.lb.LoadBalancingRule.LbStickinessPolicy; +import com.cloud.network.lb.LoadBalancingRulesManager; +import com.cloud.network.router.VirtualRouter; +import com.cloud.network.router.VirtualRouter.RedundantState; +import com.cloud.network.router.VirtualRouter.Role; +import com.cloud.network.rules.FirewallRule; +import com.cloud.offering.NetworkOffering; +import com.cloud.offering.ServiceOffering; +import com.cloud.offerings.dao.NetworkOfferingDao; +import com.cloud.resource.ResourceManager; +import com.cloud.server.ConfigurationServer; +import com.cloud.service.ServiceOfferingVO; +import com.cloud.service.dao.ServiceOfferingDao; +import com.cloud.storage.VMTemplateVO; +import com.cloud.storage.dao.VMTemplateDao; +import com.cloud.user.Account; +import com.cloud.user.AccountManager; +import com.cloud.user.User; +import com.cloud.utils.Pair; +import com.cloud.utils.component.ManagerBase; +import com.cloud.utils.db.DB; +import com.cloud.utils.exception.CloudRuntimeException; +import com.cloud.utils.net.Ip; +import com.cloud.utils.net.NetUtils; +import com.cloud.vm.DomainRouterVO; +import com.cloud.vm.Nic; +import com.cloud.vm.NicProfile; +import com.cloud.vm.NicVO; +import com.cloud.vm.ReservationContext; +import com.cloud.vm.VirtualMachine; +import com.cloud.vm.VirtualMachine.State; +import com.cloud.vm.VirtualMachineGuru; +import com.cloud.vm.VirtualMachineManager; +import com.cloud.vm.VirtualMachineName; +import com.cloud.vm.VirtualMachineProfile; +import com.cloud.vm.VirtualMachineProfile.Param; +import com.cloud.vm.dao.DomainRouterDao; +import com.cloud.vm.dao.NicDao; + +@Component +@Local(value = { InternalLoadBalancerVMManager.class, InternalLoadBalancerVMService.class}) +public class InternalLoadBalancerVMManagerImpl extends ManagerBase implements + InternalLoadBalancerVMManager, VirtualMachineGuru { + private static final Logger s_logger = Logger + .getLogger(InternalLoadBalancerVMManagerImpl.class); + static final private String _internalLbVmNamePrefix = "b"; + + private String _instance; + private String _mgmtHost; + private String _mgmtCidr; + private long _internalLbVmOfferingId; + + @Inject VirtualMachineManager _itMgr; + @Inject DomainRouterDao _internalLbVmDao; + @Inject ConfigurationDao _configDao; + @Inject AgentManager _agentMgr; + @Inject DataCenterDao _dcDao; + @Inject VirtualRouterProviderDao _vrProviderDao; + @Inject ApplicationLoadBalancerRuleDao _lbDao; + @Inject NetworkModel _ntwkModel; + @Inject LoadBalancingRulesManager _lbMgr; + @Inject NicDao _nicDao; + @Inject AccountManager _accountMgr; + @Inject NetworkDao _networkDao; + @Inject NetworkManager _ntwkMgr; + @Inject ServiceOfferingDao _serviceOfferingDao; + @Inject PhysicalNetworkServiceProviderDao _physicalProviderDao; + @Inject NetworkOfferingDao _networkOfferingDao; + @Inject VMTemplateDao _templateDao; + @Inject ResourceManager _resourceMgr; + @Inject ConfigurationServer _configServer; + + @Override + public DomainRouterVO findByName(String name) { + if (!VirtualMachineName.isValidSystemVmName(name, _instance, _internalLbVmNamePrefix)) { + return null; + } + + return _internalLbVmDao.findById(VirtualMachineName.getRouterId(name)); + } + + @Override + public DomainRouterVO findById(long id) { + return _internalLbVmDao.findById(id); + } + + @Override + public DomainRouterVO persist(DomainRouterVO vm) { + DomainRouterVO virtualRouter = _internalLbVmDao.persist(vm); + return virtualRouter; + } + + @Override + public boolean finalizeVirtualMachineProfile(VirtualMachineProfile profile, + DeployDestination dest, ReservationContext context) { + + //Internal LB vm starts up with 2 Nics + //Nic #1 - Guest Nic with IP address that would act as the LB entry point + //Nic #2 - Control/Management Nic + + StringBuilder buf = profile.getBootArgsBuilder(); + buf.append(" template=domP"); + buf.append(" name=").append(profile.getHostName()); + + if (Boolean.valueOf(_configDao.getValue("system.vm.random.password"))) { + buf.append(" vmpassword=").append(_configDao.getValue("system.vm.password")); + } + + NicProfile controlNic = null; + Network guestNetwork = null; + + for (NicProfile nic : profile.getNics()) { + int deviceId = nic.getDeviceId(); + buf.append(" eth").append(deviceId).append("ip=").append(nic.getIp4Address()); + buf.append(" eth").append(deviceId).append("mask=").append(nic.getNetmask()); + + if (nic.isDefaultNic()) { + buf.append(" gateway=").append(nic.getGateway()); + buf.append(" dns1=").append(nic.getGateway()); + } + + if (nic.getTrafficType() == TrafficType.Guest) { + guestNetwork = _ntwkModel.getNetwork(nic.getNetworkId()); + } else if (nic.getTrafficType() == TrafficType.Management) { + buf.append(" localgw=").append(dest.getPod().getGateway()); + } else if (nic.getTrafficType() == TrafficType.Control) { + controlNic = nic; + // Internal LB control command is sent over management server in VMware + if (dest.getHost().getHypervisorType() == HypervisorType.VMware) { + if (s_logger.isInfoEnabled()) { + s_logger.info("Check if we need to add management server explicit route to Internal LB. pod cidr: " + + dest.getPod().getCidrAddress() + "/" + dest.getPod().getCidrSize() + + ", pod gateway: " + dest.getPod().getGateway() + ", management host: " + _mgmtHost); + } + + if (s_logger.isInfoEnabled()) { + s_logger.info("Add management server explicit route to Internal LB."); + } + + + buf.append(" mgmtcidr=").append(_mgmtCidr); + buf.append(" localgw=").append(dest.getPod().getGateway()); + } + } + } + + if (controlNic == null) { + throw new CloudRuntimeException("Didn't start a control port"); + } + + if (guestNetwork != null) { + String domain = guestNetwork.getNetworkDomain(); + if (domain != null) { + buf.append(" domain=" + domain); + } + } + + String type = "ilbvm"; + buf.append(" type=" + type); + + if (s_logger.isDebugEnabled()) { + s_logger.debug("Boot Args for " + profile + ": " + buf.toString()); + } + + return true; + } + + @Override + public boolean finalizeDeployment(Commands cmds, VirtualMachineProfile profile, DeployDestination dest, ReservationContext context) throws ResourceUnavailableException { + DomainRouterVO internalLbVm = profile.getVirtualMachine(); + + List nics = profile.getNics(); + for (NicProfile nic : nics) { + if (nic.getTrafficType() == TrafficType.Control) { + internalLbVm.setPrivateIpAddress(nic.getIp4Address()); + internalLbVm.setPrivateMacAddress(nic.getMacAddress()); + } + } + _internalLbVmDao.update(internalLbVm.getId(), internalLbVm); + + finalizeCommandsOnStart(cmds, profile); + return true; + } + + @Override + public boolean finalizeStart(VirtualMachineProfile profile, long hostId, Commands cmds, ReservationContext context) { + DomainRouterVO internalLbVm = profile.getVirtualMachine(); + + boolean result = true; + + Answer answer = cmds.getAnswer("checkSsh"); + if (answer != null && answer instanceof CheckSshAnswer) { + CheckSshAnswer sshAnswer = (CheckSshAnswer) answer; + if (sshAnswer == null || !sshAnswer.getResult()) { + s_logger.warn("Unable to ssh to the internal LB VM: " + sshAnswer.getDetails()); + result = false; + } + } else { + result = false; + } + if (result == false) { + return result; + } + + //Get guest network info + List guestNetworks = new ArrayList(); + List internalLbVmNics = _nicDao.listByVmId(profile.getId()); + for (Nic internalLbVmNic : internalLbVmNics) { + Network network = _ntwkModel.getNetwork(internalLbVmNic.getNetworkId()); + if (network.getTrafficType() == TrafficType.Guest) { + guestNetworks.add(network); + } + } + + answer = cmds.getAnswer("getDomRVersion"); + if (answer != null && answer instanceof GetDomRVersionAnswer) { + GetDomRVersionAnswer versionAnswer = (GetDomRVersionAnswer)answer; + if (answer == null || !answer.getResult()) { + s_logger.warn("Unable to get the template/scripts version of internal LB VM " + internalLbVm.getInstanceName() + + " due to: " + versionAnswer.getDetails()); + result = false; + } else { + internalLbVm.setTemplateVersion(versionAnswer.getTemplateVersion()); + internalLbVm.setScriptsVersion(versionAnswer.getScriptsVersion()); + internalLbVm = _internalLbVmDao.persist(internalLbVm, guestNetworks); + } + } else { + result = false; + } + + return result; + } + + @Override + public boolean finalizeCommandsOnStart(Commands cmds, VirtualMachineProfile profile) { + DomainRouterVO internalLbVm = profile.getVirtualMachine(); + NicProfile controlNic = getNicProfileByTrafficType(profile, TrafficType.Control); + + if (controlNic == null) { + s_logger.error("Control network doesn't exist for the internal LB vm " + internalLbVm); + return false; + } + + finalizeSshAndVersionOnStart(cmds, profile, internalLbVm, controlNic); + + // restart network if restartNetwork = false is not specified in profile parameters + boolean reprogramGuestNtwk = true; + if (profile.getParameter(Param.ReProgramGuestNetworks) != null + && (Boolean) profile.getParameter(Param.ReProgramGuestNetworks) == false) { + reprogramGuestNtwk = false; + } + + VirtualRouterProvider lbProvider = _vrProviderDao.findById(internalLbVm.getElementId()); + if (lbProvider == null) { + throw new CloudRuntimeException("Cannot find related element " + VirtualRouterProviderType.InternalLbVm + " of vm: " + internalLbVm.getHostName()); + } + + Provider provider = Network.Provider.getProvider(lbProvider.getType().toString()); + if (provider == null) { + throw new CloudRuntimeException("Cannot find related provider of provider: " + lbProvider.getType().toString()); + } + + if (reprogramGuestNtwk) { + NicProfile guestNic = getNicProfileByTrafficType(profile, TrafficType.Guest); + finalizeLbRulesForIp(cmds, internalLbVm, provider, new Ip(guestNic.getIp4Address()), guestNic.getNetworkId()); + } + + return true; + } + + @Override + public void finalizeStop(VirtualMachineProfile profile, StopAnswer answer) { + } + + @Override + public void finalizeExpunge(DomainRouterVO vm) { + } + + @Override + public Long convertToId(String vmName) { + if (!VirtualMachineName.isValidSystemVmName(vmName, _instance, _internalLbVmNamePrefix)) { + return null; + } + + return VirtualMachineName.getRouterId(vmName); + } + + @Override + public boolean plugNic(Network network, NicTO nic, VirtualMachineTO vm, ReservationContext context, DeployDestination dest) throws ConcurrentOperationException, ResourceUnavailableException, + InsufficientCapacityException { + //not supported + throw new UnsupportedOperationException("Plug nic is not supported for vm of type " + vm.getType()); + } + + @Override + public boolean unplugNic(Network network, NicTO nic, VirtualMachineTO vm, ReservationContext context, DeployDestination dest) throws ConcurrentOperationException, ResourceUnavailableException { + //not supported + throw new UnsupportedOperationException("Unplug nic is not supported for vm of type " + vm.getType()); + } + + @Override + public void prepareStop(VirtualMachineProfile profile) { + } + + @Override + public boolean configure(String name, Map params) throws ConfigurationException { + final Map configs = _configDao.getConfiguration("AgentManager", params); + _instance = configs.get("instance.name"); + if (_instance == null) { + _instance = "DEFAULT"; + } + + _mgmtHost = configs.get("host"); + _mgmtCidr = _configDao.getValue(Config.ManagementNetwork.key()); + + String offIdStr = configs.get(Config.InternalLbVmServiceOfferingId.key()); + if (offIdStr != null && !offIdStr.isEmpty()) { + _internalLbVmOfferingId = Long.parseLong(offIdStr); + } else { + boolean useLocalStorage = Boolean.parseBoolean(configs.get(Config.SystemVMUseLocalStorage.key())); + ServiceOfferingVO newOff = new ServiceOfferingVO("System Offering For Internal LB VM", 1, InternalLoadBalancerVMManager.DEFAULT_INTERNALLB_VM_RAMSIZE, InternalLoadBalancerVMManager.DEFAULT_INTERNALLB_VM_CPU_MHZ, null, + null, true, null, useLocalStorage, true, null, true, VirtualMachine.Type.InternalLoadBalancerVm, true); + newOff.setUniqueName(ServiceOffering.internalLbVmDefaultOffUniqueName); + newOff = _serviceOfferingDao.persistSystemServiceOffering(newOff); + _internalLbVmOfferingId = newOff.getId(); + } + + _itMgr.registerGuru(VirtualMachine.Type.InternalLoadBalancerVm, this); + + if (s_logger.isInfoEnabled()) { + s_logger.info(getName() + " has been configured"); + } + + return true; + } + + @Override + public String getName() { + return _name; + } + + protected NicProfile getNicProfileByTrafficType(VirtualMachineProfile profile, TrafficType trafficType) { + for (NicProfile nic : profile.getNics()) { + if (nic.getTrafficType() == trafficType && nic.getIp4Address() != null) { + return nic; + } + } + return null; + } + + protected void finalizeSshAndVersionOnStart(Commands cmds, VirtualMachineProfile profile, DomainRouterVO router, NicProfile controlNic) { + cmds.addCommand("checkSsh", new CheckSshCommand(profile.getInstanceName(), controlNic.getIp4Address(), 3922)); + + // Update internal lb vm template/scripts version + final GetDomRVersionCmd command = new GetDomRVersionCmd(); + command.setAccessDetail(NetworkElementCommand.ROUTER_IP, controlNic.getIp4Address()); + command.setAccessDetail(NetworkElementCommand.ROUTER_NAME, router.getInstanceName()); + cmds.addCommand("getDomRVersion", command); + } + + + protected void finalizeLbRulesForIp(Commands cmds, DomainRouterVO internalLbVm, Provider provider, Ip sourceIp, long guestNtwkId) { + s_logger.debug("Resending load balancing rules as a part of start for " + internalLbVm); + List lbs = _lbDao.listBySrcIpSrcNtwkId(sourceIp, guestNtwkId); + List lbRules = new ArrayList(); + if (_ntwkModel.isProviderSupportServiceInNetwork(guestNtwkId, Service.Lb, provider)) { + // Re-apply load balancing rules + for (ApplicationLoadBalancerRuleVO lb : lbs) { + List dstList = _lbMgr.getExistingDestinations(lb.getId()); + List policyList = _lbMgr.getStickinessPolicies(lb.getId()); + List hcPolicyList = _lbMgr.getHealthCheckPolicies(lb.getId()); + LoadBalancingRule loadBalancing = new LoadBalancingRule(lb, dstList, policyList, hcPolicyList, sourceIp); + lbRules.add(loadBalancing); + } + } + + s_logger.debug("Found " + lbRules.size() + " load balancing rule(s) to apply as a part of Intenrnal LB vm" + internalLbVm + " start."); + if (!lbRules.isEmpty()) { + createApplyLoadBalancingRulesCommands(lbRules, internalLbVm, cmds, guestNtwkId); + } + } + + private void createApplyLoadBalancingRulesCommands(List rules, VirtualRouter internalLbVm, Commands cmds, long guestNetworkId) { + + LoadBalancerTO[] lbs = new LoadBalancerTO[rules.size()]; + int i = 0; + boolean inline = false; + for (LoadBalancingRule rule : rules) { + boolean revoked = (rule.getState().equals(FirewallRule.State.Revoke)); + String protocol = rule.getProtocol(); + String algorithm = rule.getAlgorithm(); + String uuid = rule.getUuid(); + + String srcIp = rule.getSourceIp().addr(); + int srcPort = rule.getSourcePortStart(); + List destinations = rule.getDestinations(); + List stickinessPolicies = rule.getStickinessPolicies(); + LoadBalancerTO lb = new LoadBalancerTO(uuid, srcIp, srcPort, protocol, algorithm, revoked, false, inline, destinations, stickinessPolicies); + lbs[i++] = lb; + } + + Network guestNetwork = _ntwkModel.getNetwork(guestNetworkId); + Nic guestNic = _nicDao.findByNtwkIdAndInstanceId(guestNetwork.getId(), internalLbVm.getId()); + NicProfile guestNicProfile = new NicProfile(guestNic, guestNetwork, guestNic.getBroadcastUri(), guestNic.getIsolationUri(), + _ntwkModel.getNetworkRate(guestNetwork.getId(), internalLbVm.getId()), + _ntwkModel.isSecurityGroupSupportedInNetwork(guestNetwork), + _ntwkModel.getNetworkTag(internalLbVm.getHypervisorType(), guestNetwork)); + + LoadBalancerConfigCommand cmd = new LoadBalancerConfigCommand(lbs, guestNic.getIp4Address(), + guestNic.getIp4Address(), internalLbVm.getPrivateIpAddress(), + _itMgr.toNicTO(guestNicProfile, internalLbVm.getHypervisorType()), internalLbVm.getVpcId()); + + cmd.lbStatsVisibility = _configDao.getValue(Config.NetworkLBHaproxyStatsVisbility.key()); + cmd.lbStatsUri = _configDao.getValue(Config.NetworkLBHaproxyStatsUri.key()); + cmd.lbStatsAuth = _configDao.getValue(Config.NetworkLBHaproxyStatsAuth.key()); + cmd.lbStatsPort = _configDao.getValue(Config.NetworkLBHaproxyStatsPort.key()); + + cmd.setAccessDetail(NetworkElementCommand.ROUTER_IP, getInternalLbControlIp(internalLbVm.getId())); + cmd.setAccessDetail(NetworkElementCommand.ROUTER_GUEST_IP, guestNic.getIp4Address()); + cmd.setAccessDetail(NetworkElementCommand.ROUTER_NAME, internalLbVm.getInstanceName()); + DataCenterVO dcVo = _dcDao.findById(internalLbVm.getDataCenterId()); + cmd.setAccessDetail(NetworkElementCommand.ZONE_NETWORK_TYPE, dcVo.getNetworkType().toString()); + cmds.addCommand(cmd); + } + + + protected String getInternalLbControlIp(long internalLbVmId) { + String controlIpAddress = null; + List nics = _nicDao.listByVmId(internalLbVmId); + for (NicVO nic : nics) { + Network ntwk = _ntwkModel.getNetwork(nic.getNetworkId()); + if (ntwk.getTrafficType() == TrafficType.Control) { + controlIpAddress = nic.getIp4Address(); + } + } + + if(controlIpAddress == null) { + s_logger.warn("Unable to find Internal LB control ip in its attached NICs!. Internal LB vm: " + internalLbVmId); + DomainRouterVO internalLbVm = _internalLbVmDao.findById(internalLbVmId); + return internalLbVm.getPrivateIpAddress(); + } + + return controlIpAddress; + } + + @Override + public boolean destroyInternalLbVm(long vmId, Account caller, Long callerUserId) + throws ResourceUnavailableException, ConcurrentOperationException { + if (s_logger.isDebugEnabled()) { + s_logger.debug("Attempting to destroy Internal LB vm " + vmId); + } + + DomainRouterVO internalLbVm = _internalLbVmDao.findById(vmId); + if (internalLbVm == null) { + return true; + } + + _accountMgr.checkAccess(caller, null, true, internalLbVm); + + return _itMgr.expunge(internalLbVm, _accountMgr.getActiveUser(callerUserId), caller); + } + + + @Override + public VirtualRouter stopInternalLbVm(long vmId, boolean forced, Account caller, long callerUserId) throws ConcurrentOperationException, + ResourceUnavailableException { + DomainRouterVO internalLbVm = _internalLbVmDao.findById(vmId); + if (internalLbVm == null || internalLbVm.getRole() != Role.INTERNAL_LB_VM) { + throw new InvalidParameterValueException("Can't find internal lb vm by id specified"); + } + + return stopInternalLbVm(internalLbVm, forced, caller, callerUserId); + } + + protected VirtualRouter stopInternalLbVm(DomainRouterVO internalLbVm, boolean forced, Account caller, long callerUserId) throws ResourceUnavailableException, ConcurrentOperationException { + s_logger.debug("Stopping internal lb vm " + internalLbVm); + try { + if (_itMgr.advanceStop((DomainRouterVO) internalLbVm, forced, _accountMgr.getActiveUser(callerUserId), caller)) { + return _internalLbVmDao.findById(internalLbVm.getId()); + } else { + return null; + } + } catch (OperationTimedoutException e) { + throw new CloudRuntimeException("Unable to stop " + internalLbVm, e); + } + } + + + @Override + public List deployInternalLbVm(Network guestNetwork, Ip requestedGuestIp, DeployDestination dest, + Account owner, Map params) throws InsufficientCapacityException, + ConcurrentOperationException, ResourceUnavailableException { + + List internalLbVms = findOrDeployInternalLbVm(guestNetwork, requestedGuestIp, dest, owner, params); + + return startInternalLbVms(params, internalLbVms); + } + + protected List startInternalLbVms(Map params, List internalLbVms) + throws StorageUnavailableException, InsufficientCapacityException, ConcurrentOperationException, ResourceUnavailableException { + List runningInternalLbVms = null; + + if (internalLbVms != null) { + runningInternalLbVms = new ArrayList(); + } else { + s_logger.debug("Have no internal lb vms to start"); + return null; + } + + for (DomainRouterVO internalLbVm : internalLbVms) { + if (internalLbVm.getState() != VirtualMachine.State.Running) { + internalLbVm = startInternalLbVm(internalLbVm, _accountMgr.getSystemAccount(), User.UID_SYSTEM, params); + } + + if (internalLbVm != null) { + runningInternalLbVms.add(internalLbVm); + } + } + return runningInternalLbVms; + } + + + + @DB + protected List findOrDeployInternalLbVm(Network guestNetwork, Ip requestedGuestIp, DeployDestination dest, + Account owner, Map params) throws ConcurrentOperationException, + InsufficientCapacityException, ResourceUnavailableException { + + List internalLbVms = new ArrayList(); + Network lock = _networkDao.acquireInLockTable(guestNetwork.getId(), _ntwkMgr.getNetworkLockTimeout()); + if (lock == null) { + throw new ConcurrentOperationException("Unable to lock network " + guestNetwork.getId()); + } + + if (s_logger.isDebugEnabled()) { + s_logger.debug("Lock is acquired for network id " + lock.getId() + " as a part of internal lb startup in " + dest); + } + + long internalLbProviderId = getInternalLbProviderId(guestNetwork); + + try { + assert guestNetwork.getState() == Network.State.Implemented || guestNetwork.getState() == Network.State.Setup || + guestNetwork.getState() == Network.State.Implementing : "Network is not yet fully implemented: " + + guestNetwork; + assert guestNetwork.getTrafficType() == TrafficType.Guest; + + //deploy internal lb vm + Pair> planAndInternalLbVms = getDeploymentPlanAndInternalLbVms(dest, guestNetwork.getId(), requestedGuestIp); + internalLbVms = planAndInternalLbVms.second(); + DeploymentPlan plan = planAndInternalLbVms.first(); + + if (internalLbVms.size() > 0) { + s_logger.debug("Found " + internalLbVms.size() + " internal lb vms for the requested IP " + requestedGuestIp.addr()); + return internalLbVms; + } + + List> networks = createInternalLbVmNetworks(guestNetwork, plan, requestedGuestIp); + //Pass startVm=false as we are holding the network lock that needs to be released at the end of vm allocation + DomainRouterVO internalLbVm = deployInternalLbVm(owner, dest, plan, params, internalLbProviderId, _internalLbVmOfferingId, guestNetwork.getVpcId(), + networks, false); + if (internalLbVm != null) { + _internalLbVmDao.addRouterToGuestNetwork(internalLbVm, guestNetwork); + internalLbVms.add(internalLbVm); + } + } finally { + if (lock != null) { + _networkDao.releaseFromLockTable(lock.getId()); + if (s_logger.isDebugEnabled()) { + s_logger.debug("Lock is released for network id " + lock.getId() + " as a part of internal lb vm startup in " + dest); + } + } + } + return internalLbVms; + } + + protected long getInternalLbProviderId(Network guestNetwork) { + VirtualRouterProviderType type = VirtualRouterProviderType.InternalLbVm; + long physicalNetworkId = _ntwkModel.getPhysicalNetworkId(guestNetwork); + + PhysicalNetworkServiceProvider provider = _physicalProviderDao.findByServiceProvider(physicalNetworkId, type.toString()); + if (provider == null) { + throw new CloudRuntimeException("Cannot find service provider " + type.toString() + " in physical network " + physicalNetworkId); + } + + VirtualRouterProvider internalLbProvider = _vrProviderDao.findByNspIdAndType(provider.getId(), type); + if (internalLbProvider == null) { + throw new CloudRuntimeException("Cannot find provider " + type.toString() + " as service provider " + provider.getId()); + } + + return internalLbProvider.getId(); + } + + protected List> createInternalLbVmNetworks(Network guestNetwork, DeploymentPlan plan, Ip guestIp) throws ConcurrentOperationException, + InsufficientAddressCapacityException { + + //Form networks + List> networks = new ArrayList>(3); + + //1) Guest network - default + if (guestNetwork != null) { + s_logger.debug("Adding nic for Internal LB in Guest network " + guestNetwork); + NicProfile guestNic = new NicProfile(); + if (guestIp != null) { + guestNic.setIp4Address(guestIp.addr()); + } else { + guestNic.setIp4Address(_ntwkMgr.acquireGuestIpAddress(guestNetwork, null)); + } + guestNic.setGateway(guestNetwork.getGateway()); + guestNic.setBroadcastUri(guestNetwork.getBroadcastUri()); + guestNic.setBroadcastType(guestNetwork.getBroadcastDomainType()); + guestNic.setIsolationUri(guestNetwork.getBroadcastUri()); + guestNic.setMode(guestNetwork.getMode()); + String gatewayCidr = guestNetwork.getCidr(); + guestNic.setNetmask(NetUtils.getCidrNetmask(gatewayCidr)); + guestNic.setDefaultNic(true); + networks.add(new Pair((NetworkVO) guestNetwork, guestNic)); + } + + //2) Control network + s_logger.debug("Adding nic for Internal LB vm in Control network "); + List offerings = _ntwkModel.getSystemAccountNetworkOfferings(NetworkOffering.SystemControlNetwork); + NetworkOffering controlOffering = offerings.get(0); + NetworkVO controlConfig = _ntwkMgr.setupNetwork(_accountMgr.getSystemAccount(), controlOffering, plan, null, null, false).get(0); + networks.add(new Pair(controlConfig, null)); + + return networks; + } + + + protected Pair> getDeploymentPlanAndInternalLbVms(DeployDestination dest, long guestNetworkId, Ip requestedGuestIp) { + long dcId = dest.getDataCenter().getId(); + DeploymentPlan plan = new DataCenterDeployment(dcId); + List internalLbVms = findInternalLbVms(guestNetworkId, requestedGuestIp); + + return new Pair>(plan, internalLbVms); + + } + + @Override + public List findInternalLbVms(long guestNetworkId, Ip requestedGuestIp) { + List internalLbVms = _internalLbVmDao.listByNetworkAndRole(guestNetworkId, Role.INTERNAL_LB_VM); + if (requestedGuestIp != null && !internalLbVms.isEmpty()) { + Iterator it = internalLbVms.iterator(); + while (it.hasNext()) { + DomainRouterVO vm = it.next(); + Nic nic = _nicDao.findByNtwkIdAndInstanceId(guestNetworkId, vm.getId()); + if (!nic.getIp4Address().equalsIgnoreCase(requestedGuestIp.addr())) { + it.remove(); + } + } + } + return internalLbVms; + } + + + protected DomainRouterVO deployInternalLbVm(Account owner, DeployDestination dest, DeploymentPlan plan, Map params, + long internalLbProviderId, long svcOffId, Long vpcId, + List> networks, boolean startVm) throws ConcurrentOperationException, + InsufficientAddressCapacityException, InsufficientServerCapacityException, InsufficientCapacityException, + StorageUnavailableException, ResourceUnavailableException { + + long id = _internalLbVmDao.getNextInSequence(Long.class, "id"); + if (s_logger.isDebugEnabled()) { + s_logger.debug("Creating the internal lb vm " + id + " in datacenter " + dest.getDataCenter()); + } + + ServiceOfferingVO routerOffering = _serviceOfferingDao.findById(svcOffId); + + // Internal lb is the network element, we don't know the hypervisor type yet. + // Try to allocate the internal lb twice using diff hypervisors, and when failed both times, throw the exception up + List hypervisors = getHypervisors(dest, plan, null); + + int allocateRetry = 0; + int startRetry = 0; + DomainRouterVO internalLbVm = null; + for (Iterator iter = hypervisors.iterator(); iter.hasNext();) { + HypervisorType hType = iter.next(); + try { + s_logger.debug("Allocating the Internal lb with the hypervisor type " + hType); + String templateName = null; + switch (hType) { + case XenServer: + templateName = _configServer.getConfigValue(Config.RouterTemplateXen.key(), Config.ConfigurationParameterScope.zone.toString(), dest.getDataCenter().getId()); + break; + case KVM: + templateName = _configServer.getConfigValue(Config.RouterTemplateKVM.key(), Config.ConfigurationParameterScope.zone.toString(), dest.getDataCenter().getId()); + break; + case VMware: + templateName = _configServer.getConfigValue(Config.RouterTemplateVmware.key(), Config.ConfigurationParameterScope.zone.toString(), dest.getDataCenter().getId()); + break; + case Hyperv: + templateName = _configServer.getConfigValue(Config.RouterTemplateHyperv.key(), Config.ConfigurationParameterScope.zone.toString(), dest.getDataCenter().getId()); + break; + case LXC: + templateName = _configServer.getConfigValue(Config.RouterTemplateLXC.key(), Config.ConfigurationParameterScope.zone.toString(), dest.getDataCenter().getId()); + break; + default: break; + } + VMTemplateVO template = _templateDao.findRoutingTemplate(hType, templateName); + + if (template == null) { + s_logger.debug(hType + " won't support system vm, skip it"); + continue; + } + + internalLbVm = new DomainRouterVO(id, routerOffering.getId(), internalLbProviderId, + VirtualMachineName.getSystemVmName(id, _instance, _internalLbVmNamePrefix), template.getId(), template.getHypervisorType(), + template.getGuestOSId(), owner.getDomainId(), owner.getId(), false, 0, false, + RedundantState.UNKNOWN, false, false, VirtualMachine.Type.InternalLoadBalancerVm, vpcId); + internalLbVm.setRole(Role.INTERNAL_LB_VM); + internalLbVm = _itMgr.allocate(internalLbVm, template, routerOffering, networks, plan, null, owner); + } catch (InsufficientCapacityException ex) { + if (allocateRetry < 2 && iter.hasNext()) { + s_logger.debug("Failed to allocate the Internal lb vm with hypervisor type " + hType + ", retrying one more time"); + continue; + } else { + throw ex; + } + } finally { + allocateRetry++; + } + + if (startVm) { + try { + internalLbVm = startInternalLbVm(internalLbVm, _accountMgr.getSystemAccount(), User.UID_SYSTEM, params); + break; + } catch (InsufficientCapacityException ex) { + if (startRetry < 2 && iter.hasNext()) { + s_logger.debug("Failed to start the Internal lb vm " + internalLbVm + " with hypervisor type " + hType + ", " + + "destroying it and recreating one more time"); + // destroy the internal lb vm + destroyInternalLbVm(internalLbVm.getId(), _accountMgr.getSystemAccount(), User.UID_SYSTEM); + continue; + } else { + throw ex; + } + } finally { + startRetry++; + } + } else { + //return stopped internal lb vm + return internalLbVm; + } + } + return internalLbVm; + } + + + + protected DomainRouterVO startInternalLbVm(DomainRouterVO internalLbVm, Account caller, long callerUserId, Map params) + throws StorageUnavailableException, InsufficientCapacityException, + ConcurrentOperationException, ResourceUnavailableException { + s_logger.debug("Starting Internal LB VM " + internalLbVm); + if (_itMgr.start(internalLbVm, params, _accountMgr.getUserIncludingRemoved(callerUserId), caller, null) != null) { + if (internalLbVm.isStopPending()) { + s_logger.info("Clear the stop pending flag of Internal LB VM " + internalLbVm.getHostName() + " after start router successfully!"); + internalLbVm.setStopPending(false); + internalLbVm = _internalLbVmDao.persist(internalLbVm); + } + return _internalLbVmDao.findById(internalLbVm.getId()); + } else { + return null; + } + } + + + protected List getHypervisors(DeployDestination dest, DeploymentPlan plan, + List supportedHypervisors) throws InsufficientServerCapacityException { + List hypervisors = new ArrayList(); + + HypervisorType defaults = _resourceMgr.getDefaultHypervisor(dest.getDataCenter().getId()); + if (defaults != HypervisorType.None) { + hypervisors.add(defaults); + } else { + //if there is no default hypervisor, get it from the cluster + hypervisors = _resourceMgr.getSupportedHypervisorTypes(dest.getDataCenter().getId(), true, + plan.getPodId()); + } + + //keep only elements defined in supported hypervisors + StringBuilder hTypesStr = new StringBuilder(); + if (supportedHypervisors != null && !supportedHypervisors.isEmpty()) { + hypervisors.retainAll(supportedHypervisors); + for (HypervisorType hType : supportedHypervisors) { + hTypesStr.append(hType).append(" "); + } + } + + if (hypervisors.isEmpty()) { + throw new InsufficientServerCapacityException("Unable to create internal lb vm, " + + "there are no clusters in the zone ", DataCenter.class, dest.getDataCenter().getId()); + } + return hypervisors; + } + + @Override + public boolean applyLoadBalancingRules(Network network, final List rules, List internalLbVms) + throws ResourceUnavailableException { + if (rules == null || rules.isEmpty()) { + s_logger.debug("No lb rules to be applied for network " + network); + return true; + } + + //only one internal lb vm is supported per ip address at this time + if (internalLbVms == null || internalLbVms.isEmpty()) { + throw new CloudRuntimeException("Can't apply the lb rules on network " + network + " as the list of internal lb vms is empty"); + } + + VirtualRouter lbVm = internalLbVms.get(0); + if (lbVm.getState() == State.Running) { + return sendLBRules(lbVm, rules, network.getId()); + } else if (lbVm.getState() == State.Stopped || lbVm.getState() == State.Stopping) { + s_logger.debug("Internal LB VM " + lbVm.getInstanceName() + " is in " + lbVm.getState() + + ", so not sending apply lb rules commands to the backend"); + return true; + } else { + s_logger.warn("Unable to apply lb rules, Internal LB VM is not in the right state " + lbVm.getState()); + throw new ResourceUnavailableException("Unable to apply lb rules; Internal LB VM is not in the right state", DataCenter.class, lbVm.getDataCenterId()); + } + } + + protected boolean sendLBRules(VirtualRouter internalLbVm, List rules, long guestNetworkId) throws ResourceUnavailableException { + Commands cmds = new Commands(OnError.Continue); + createApplyLoadBalancingRulesCommands(rules, internalLbVm, cmds, guestNetworkId); + return sendCommandsToInternalLbVm(internalLbVm, cmds); + } + + + protected boolean sendCommandsToInternalLbVm(final VirtualRouter internalLbVm, Commands cmds) throws AgentUnavailableException { + Answer[] answers = null; + try { + answers = _agentMgr.send(internalLbVm.getHostId(), cmds); + } catch (OperationTimedoutException e) { + s_logger.warn("Timed Out", e); + throw new AgentUnavailableException("Unable to send commands to virtual router ", internalLbVm.getHostId(), e); + } + + if (answers == null) { + return false; + } + + if (answers.length != cmds.size()) { + return false; + } + + boolean result = true; + if (answers.length > 0) { + for (Answer answer : answers) { + if (!answer.getResult()) { + result = false; + break; + } + } + } + return result; + } + + + @Override + public VirtualRouter startInternalLbVm(long internalLbVmId, Account caller, long callerUserId) + throws StorageUnavailableException, InsufficientCapacityException, + ConcurrentOperationException, ResourceUnavailableException { + + DomainRouterVO internalLbVm = _internalLbVmDao.findById(internalLbVmId); + if (internalLbVm == null || internalLbVm.getRole() != Role.INTERNAL_LB_VM) { + throw new InvalidParameterValueException("Can't find internal lb vm by id specified"); + } + + return startInternalLbVm(internalLbVm, caller, callerUserId, null); + } +} diff --git a/plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbelement/ElementChildTestConfiguration.java b/plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbelement/ElementChildTestConfiguration.java new file mode 100644 index 00000000000..6959b951fc3 --- /dev/null +++ b/plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbelement/ElementChildTestConfiguration.java @@ -0,0 +1,124 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.internallbelement; + +import java.io.IOException; + +import org.apache.cloudstack.lb.dao.ApplicationLoadBalancerRuleDao; +import org.apache.cloudstack.network.lb.InternalLoadBalancerVMManager; +import org.apache.cloudstack.test.utils.SpringUtils; +import org.mockito.Mockito; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.ComponentScan; +import org.springframework.context.annotation.ComponentScan.Filter; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.FilterType; +import org.springframework.core.type.classreading.MetadataReader; +import org.springframework.core.type.classreading.MetadataReaderFactory; +import org.springframework.core.type.filter.TypeFilter; + +import com.cloud.configuration.ConfigurationManager; +import com.cloud.dc.dao.AccountVlanMapDaoImpl; +import com.cloud.dc.dao.DataCenterDao; +import com.cloud.network.NetworkManager; +import com.cloud.network.NetworkModel; +import com.cloud.network.dao.NetworkServiceMapDao; +import com.cloud.network.dao.PhysicalNetworkServiceProviderDao; +import com.cloud.network.dao.VirtualRouterProviderDao; +import com.cloud.user.AccountManager; +import com.cloud.vm.dao.DomainRouterDao; + +@Configuration +@ComponentScan( + basePackageClasses={ + AccountVlanMapDaoImpl.class + }, + includeFilters={@Filter(value=ElementChildTestConfiguration.Library.class, type=FilterType.CUSTOM)}, + useDefaultFilters=false + ) + +public class ElementChildTestConfiguration { + public static class Library implements TypeFilter { + @Bean + public AccountManager accountManager() { + return Mockito.mock(AccountManager.class); + } + + + @Bean + public DomainRouterDao domainRouterDao() { + return Mockito.mock(DomainRouterDao.class); + } + + @Bean + public VirtualRouterProviderDao virtualRouterProviderDao() { + return Mockito.mock(VirtualRouterProviderDao.class); + } + + @Bean + public NetworkModel networkModel() { + return Mockito.mock(NetworkModel.class); + } + + + @Bean + public NetworkManager networkManager() { + return Mockito.mock(NetworkManager.class); + } + + + @Bean + public PhysicalNetworkServiceProviderDao physicalNetworkServiceProviderDao() { + return Mockito.mock(PhysicalNetworkServiceProviderDao.class); + } + + @Bean + public NetworkServiceMapDao networkServiceMapDao() { + return Mockito.mock(NetworkServiceMapDao.class); + } + + @Bean + public InternalLoadBalancerVMManager internalLoadBalancerVMManager() { + return Mockito.mock(InternalLoadBalancerVMManager.class); + } + + @Bean + public ConfigurationManager confugurationManager() { + return Mockito.mock(ConfigurationManager.class); + } + + + @Bean + public ApplicationLoadBalancerRuleDao applicationLoadBalancerRuleDao() { + return Mockito.mock(ApplicationLoadBalancerRuleDao.class); + } + + @Bean + public DataCenterDao dataCenterDao() { + return Mockito.mock(DataCenterDao.class); + } + + + + @Override + public boolean match(MetadataReader mdr, MetadataReaderFactory arg1) throws IOException { + mdr.getClassMetadata().getClassName(); + ComponentScan cs = ElementChildTestConfiguration.class.getAnnotation(ComponentScan.class); + return SpringUtils.includedInBasePackageClasses(mdr.getClassMetadata().getClassName(), cs); + } + } +} diff --git a/plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbelement/InternalLbElementServiceTest.java b/plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbelement/InternalLbElementServiceTest.java new file mode 100644 index 00000000000..f0e951cdc7a --- /dev/null +++ b/plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbelement/InternalLbElementServiceTest.java @@ -0,0 +1,190 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.internallbelement; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; + +import javax.inject.Inject; + +import org.apache.cloudstack.network.element.InternalLoadBalancerElementService; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.mockito.Mockito; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +import com.cloud.exception.InvalidParameterValueException; +import com.cloud.network.VirtualRouterProvider; +import com.cloud.network.VirtualRouterProvider.VirtualRouterProviderType; +import com.cloud.network.dao.PhysicalNetworkServiceProviderDao; +import com.cloud.network.dao.PhysicalNetworkServiceProviderVO; +import com.cloud.network.dao.VirtualRouterProviderDao; +import com.cloud.network.element.VirtualRouterProviderVO; +import com.cloud.user.AccountManager; +import com.cloud.utils.component.ComponentContext; + +@RunWith(SpringJUnit4ClassRunner.class) +@ContextConfiguration(locations="classpath:/lb_element.xml") +public class InternalLbElementServiceTest { + //The interface to test + @Inject InternalLoadBalancerElementService _lbElSvc; + + //Mocked interfaces + @Inject AccountManager _accountMgr; + @Inject VirtualRouterProviderDao _vrProviderDao; + @Inject PhysicalNetworkServiceProviderDao _pNtwkProviderDao; + + long validElId = 1L; + long nonExistingElId = 2L; + long invalidElId = 3L; //not of VirtualRouterProviderType + + long validProviderId = 1L; + long nonExistingProviderId = 2L; + long invalidProviderId = 3L; + + + @Before + public void setUp() { + + ComponentContext.initComponentsLifeCycle(); + VirtualRouterProviderVO validElement = new VirtualRouterProviderVO(1, VirtualRouterProviderType.InternalLbVm); + VirtualRouterProviderVO invalidElement = new VirtualRouterProviderVO(1, VirtualRouterProviderType.VirtualRouter); + + Mockito.when(_vrProviderDao.findById(validElId)).thenReturn(validElement); + Mockito.when(_vrProviderDao.findById(invalidElId)).thenReturn(invalidElement); + + Mockito.when(_vrProviderDao.persist(validElement)).thenReturn(validElement); + + Mockito.when(_vrProviderDao.findByNspIdAndType(validProviderId, VirtualRouterProviderType.InternalLbVm)).thenReturn(validElement); + + PhysicalNetworkServiceProviderVO validProvider = new PhysicalNetworkServiceProviderVO(1, "InternalLoadBalancerElement"); + PhysicalNetworkServiceProviderVO invalidProvider = new PhysicalNetworkServiceProviderVO(1, "Invalid name!"); + + Mockito.when(_pNtwkProviderDao.findById(validProviderId)).thenReturn(validProvider); + Mockito.when(_pNtwkProviderDao.findById(invalidProviderId)).thenReturn(invalidProvider); + + Mockito.when(_vrProviderDao.persist(Mockito.any(VirtualRouterProviderVO.class))).thenReturn(validElement); + } + + //TESTS FOR getInternalLoadBalancerElement METHOD + + + @Test (expected = InvalidParameterValueException.class) + public void findNonExistingVm() { + String expectedExcText = null; + try { + _lbElSvc.getInternalLoadBalancerElement(nonExistingElId); + } catch (InvalidParameterValueException e) { + expectedExcText = e.getMessage(); + throw e; + } finally { + assertEquals("Test failed. The non-existing intenral lb provider was found" + + expectedExcText, expectedExcText, "Unable to find InternalLoadBalancerElementService by id"); + } + } + + + @Test (expected = InvalidParameterValueException.class) + public void findInvalidVm() { + String expectedExcText = null; + try { + _lbElSvc.getInternalLoadBalancerElement(invalidElId); + } catch (InvalidParameterValueException e) { + expectedExcText = e.getMessage(); + throw e; + } finally { + assertEquals("Test failed. The non-existing intenral lb provider was found" + + expectedExcText, expectedExcText, "Unable to find InternalLoadBalancerElementService by id"); + } + } + + + @Test + public void findValidVm() { + VirtualRouterProvider provider = null; + try { + provider = _lbElSvc.getInternalLoadBalancerElement(validElId); + } finally { + assertNotNull("Test failed. Couldn't find the VR provider by the valid id",provider); + } + } + + + //TESTS FOR configureInternalLoadBalancerElement METHOD + + @Test (expected = InvalidParameterValueException.class) + public void configureNonExistingVm() { + + _lbElSvc.configureInternalLoadBalancerElement(nonExistingElId, true); + + } + + + @Test (expected = InvalidParameterValueException.class) + public void ConfigureInvalidVm() { + _lbElSvc.configureInternalLoadBalancerElement(invalidElId, true); + } + + + @Test + public void enableProvider() { + VirtualRouterProvider provider = null; + try { + provider = _lbElSvc.configureInternalLoadBalancerElement(validElId, true); + } finally { + assertNotNull("Test failed. Couldn't find the VR provider by the valid id ",provider); + assertTrue("Test failed. The provider wasn't eanbled ", provider.isEnabled()); + } + } + + @Test + public void disableProvider() { + VirtualRouterProvider provider = null; + try { + provider = _lbElSvc.configureInternalLoadBalancerElement(validElId, false); + } finally { + assertNotNull("Test failed. Couldn't find the VR provider by the valid id ",provider); + assertFalse("Test failed. The provider wasn't disabled ", provider.isEnabled()); + } + } + + //TESTS FOR addInternalLoadBalancerElement METHOD + + @Test (expected = InvalidParameterValueException.class) + public void addToNonExistingProvider() { + + _lbElSvc.addInternalLoadBalancerElement(nonExistingProviderId); + + } + + @Test (expected = InvalidParameterValueException.class) + public void addToInvalidProvider() { + _lbElSvc.addInternalLoadBalancerElement(invalidProviderId); + } + + @Test + public void addToExistingProvider() { + _lbElSvc.addInternalLoadBalancerElement(validProviderId); + } + +} + + diff --git a/plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbelement/InternalLbElementTest.java b/plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbelement/InternalLbElementTest.java new file mode 100644 index 00000000000..f19612f6b0f --- /dev/null +++ b/plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbelement/InternalLbElementTest.java @@ -0,0 +1,226 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.internallbelement; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; + +import java.lang.reflect.Field; +import java.util.ArrayList; +import java.util.HashSet; +import java.util.List; + +import javax.inject.Inject; + +import org.apache.cloudstack.lb.ApplicationLoadBalancerRuleVO; +import org.apache.cloudstack.network.element.InternalLoadBalancerElement; +import org.apache.cloudstack.network.lb.InternalLoadBalancerVMManager; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.mockito.Mockito; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +import com.cloud.agent.api.to.LoadBalancerTO; +import com.cloud.configuration.ConfigurationManager; +import com.cloud.dc.DataCenter.NetworkType; +import com.cloud.dc.DataCenterVO; +import com.cloud.exception.ResourceUnavailableException; +import com.cloud.network.Network.Provider; +import com.cloud.network.Network.Service; +import com.cloud.network.VirtualRouterProvider.VirtualRouterProviderType; +import com.cloud.network.addr.PublicIp; +import com.cloud.network.dao.NetworkVO; +import com.cloud.network.dao.PhysicalNetworkServiceProviderDao; +import com.cloud.network.dao.PhysicalNetworkServiceProviderVO; +import com.cloud.network.dao.VirtualRouterProviderDao; +import com.cloud.network.element.VirtualRouterProviderVO; +import com.cloud.network.lb.LoadBalancingRule; +import com.cloud.network.rules.FirewallRule; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; +import com.cloud.user.AccountManager; +import com.cloud.utils.component.ComponentContext; +import com.cloud.utils.net.Ip; + +@RunWith(SpringJUnit4ClassRunner.class) +@ContextConfiguration(locations="classpath:/lb_element.xml") +public class InternalLbElementTest { + //The class to test + @Inject InternalLoadBalancerElement _lbEl; + + //Mocked interfaces + @Inject AccountManager _accountMgr; + @Inject VirtualRouterProviderDao _vrProviderDao; + @Inject PhysicalNetworkServiceProviderDao _pNtwkProviderDao; + @Inject InternalLoadBalancerVMManager _internalLbMgr; + @Inject ConfigurationManager _configMgr; + + long validElId = 1L; + long nonExistingElId = 2L; + long invalidElId = 3L; //not of VirtualRouterProviderType + long notEnabledElId = 4L; + + long validProviderId = 1L; + long nonExistingProviderId = 2L; + long invalidProviderId = 3L; + + + @Before + public void setUp() { + + ComponentContext.initComponentsLifeCycle(); + VirtualRouterProviderVO validElement = new VirtualRouterProviderVO(1, VirtualRouterProviderType.InternalLbVm); + validElement.setEnabled(true); + VirtualRouterProviderVO invalidElement = new VirtualRouterProviderVO(1, VirtualRouterProviderType.VirtualRouter); + VirtualRouterProviderVO notEnabledElement = new VirtualRouterProviderVO(1, VirtualRouterProviderType.InternalLbVm); + + Mockito.when(_vrProviderDao.findByNspIdAndType(validElId, VirtualRouterProviderType.InternalLbVm)).thenReturn(validElement); + Mockito.when(_vrProviderDao.findByNspIdAndType(invalidElId, VirtualRouterProviderType.InternalLbVm)).thenReturn(invalidElement); + Mockito.when(_vrProviderDao.findByNspIdAndType(notEnabledElId, VirtualRouterProviderType.InternalLbVm)).thenReturn(notEnabledElement); + + Mockito.when(_vrProviderDao.persist(validElement)).thenReturn(validElement); + + Mockito.when(_vrProviderDao.findByNspIdAndType(validProviderId, VirtualRouterProviderType.InternalLbVm)).thenReturn(validElement); + + PhysicalNetworkServiceProviderVO validProvider = new PhysicalNetworkServiceProviderVO(1, "InternalLoadBalancerElement"); + PhysicalNetworkServiceProviderVO invalidProvider = new PhysicalNetworkServiceProviderVO(1, "Invalid name!"); + + Mockito.when(_pNtwkProviderDao.findById(validProviderId)).thenReturn(validProvider); + Mockito.when(_pNtwkProviderDao.findById(invalidProviderId)).thenReturn(invalidProvider); + + Mockito.when(_vrProviderDao.persist(Mockito.any(VirtualRouterProviderVO.class))).thenReturn(validElement); + + DataCenterVO dc = new DataCenterVO + (1L, null, null, null, null, null, null, null, null, null, NetworkType.Advanced, null, null); + Mockito.when(_configMgr.getZone(Mockito.anyLong())).thenReturn(dc); + } + + //TEST FOR getProvider() method + + @Test + public void verifyProviderName() { + Provider pr = _lbEl.getProvider(); + assertEquals("Wrong provider is returned", pr.getName(), Provider.InternalLbVm.getName()); + } + + //TEST FOR isReady() METHOD + + @Test + public void verifyValidProviderState() { + PhysicalNetworkServiceProviderVO provider = new PhysicalNetworkServiceProviderVO(); + provider = setId(provider, validElId); + boolean isReady = _lbEl.isReady(provider); + assertTrue("Valid provider is returned as not ready", isReady); + } + + + @Test + public void verifyNonExistingProviderState() { + PhysicalNetworkServiceProviderVO provider = new PhysicalNetworkServiceProviderVO(); + provider = setId(provider, nonExistingElId); + boolean isReady = _lbEl.isReady(provider); + assertFalse("Non existing provider is returned as ready", isReady); + } + + + @Test + public void verifyInvalidProviderState() { + PhysicalNetworkServiceProviderVO provider = new PhysicalNetworkServiceProviderVO(); + provider = setId(provider, invalidElId); + boolean isReady = _lbEl.isReady(provider); + assertFalse("Not valid provider is returned as ready", isReady); + } + + @Test + public void verifyNotEnabledProviderState() { + PhysicalNetworkServiceProviderVO provider = new PhysicalNetworkServiceProviderVO(); + provider = setId(provider, notEnabledElId); + boolean isReady = _lbEl.isReady(provider); + assertFalse("Not enabled provider is returned as ready", isReady); + } + + //TEST FOR canEnableIndividualServices METHOD + @Test + public void verifyCanEnableIndividualSvc() { + boolean result = _lbEl.canEnableIndividualServices(); + assertTrue("Wrong value is returned by canEnableIndividualSvc", result); + } + + //TEST FOR verifyServicesCombination METHOD + @Test + public void verifyServicesCombination() { + boolean result = _lbEl.verifyServicesCombination(new HashSet()); + assertTrue("Wrong value is returned by verifyServicesCombination", result); + } + + + //TEST FOR applyIps METHOD + @Test + public void verifyApplyIps() throws ResourceUnavailableException { + List ips = new ArrayList(); + boolean result = _lbEl.applyIps(new NetworkVO(), ips, new HashSet()); + assertTrue("Wrong value is returned by applyIps method", result); + } + + + //TEST FOR updateHealthChecks METHOD + @Test + public void verifyUpdateHealthChecks() throws ResourceUnavailableException { + List check = _lbEl.updateHealthChecks(new NetworkVO(), new ArrayList()); + assertNull("Wrong value is returned by updateHealthChecks method", check); + } + + //TEST FOR validateLBRule METHOD + @Test + public void verifyValidateLBRule() throws ResourceUnavailableException { + ApplicationLoadBalancerRuleVO lb = new ApplicationLoadBalancerRuleVO(null, null, 22, 22, "roundrobin", + 1L, 1L, 1L, new Ip("10.10.10.1"), 1L, Scheme.Internal); + lb.setState(FirewallRule.State.Add); + + LoadBalancingRule rule = new LoadBalancingRule(lb, null, + null, null, new Ip("10.10.10.1")); + + + boolean result = _lbEl.validateLBRule(new NetworkVO(), rule); + assertTrue("Wrong value is returned by validateLBRule method", result); + } + + + private static PhysicalNetworkServiceProviderVO setId(PhysicalNetworkServiceProviderVO vo, long id) { + PhysicalNetworkServiceProviderVO voToReturn = vo; + Class c = voToReturn.getClass(); + try { + Field f = c.getDeclaredField("id"); + f.setAccessible(true); + f.setLong(voToReturn, id); + } catch (NoSuchFieldException ex) { + return null; + } catch (IllegalAccessException ex) { + return null; + } + + return voToReturn; + } + + + +} + + diff --git a/plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbvmmgr/InternalLBVMManagerTest.java b/plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbvmmgr/InternalLBVMManagerTest.java new file mode 100644 index 00000000000..a19a82e30c1 --- /dev/null +++ b/plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbvmmgr/InternalLBVMManagerTest.java @@ -0,0 +1,388 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.internallbvmmgr; + +import java.lang.reflect.Field; +import java.net.URI; +import java.net.URISyntaxException; +import java.util.ArrayList; +import java.util.List; + +import javax.inject.Inject; + +import junit.framework.TestCase; + +import org.apache.cloudstack.lb.ApplicationLoadBalancerRuleVO; +import org.apache.cloudstack.network.lb.InternalLoadBalancerVMManager; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.mockito.Mockito; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +import com.cloud.agent.AgentManager; +import com.cloud.agent.api.Answer; +import com.cloud.agent.manager.Commands; +import com.cloud.dc.DataCenter.NetworkType; +import com.cloud.dc.DataCenterVO; +import com.cloud.dc.dao.DataCenterDao; +import com.cloud.exception.AgentUnavailableException; +import com.cloud.exception.ConcurrentOperationException; +import com.cloud.exception.OperationTimedoutException; +import com.cloud.exception.ResourceUnavailableException; +import com.cloud.hypervisor.Hypervisor.HypervisorType; +import com.cloud.network.NetworkModel; +import com.cloud.network.dao.NetworkVO; +import com.cloud.network.lb.LoadBalancingRule; +import com.cloud.network.router.VirtualRouter; +import com.cloud.network.router.VirtualRouter.Role; +import com.cloud.network.rules.FirewallRule; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; +import com.cloud.service.ServiceOfferingVO; +import com.cloud.service.dao.ServiceOfferingDao; +import com.cloud.user.Account; +import com.cloud.user.AccountManager; +import com.cloud.user.AccountVO; +import com.cloud.user.User; +import com.cloud.utils.component.ComponentContext; +import com.cloud.utils.exception.CloudRuntimeException; +import com.cloud.utils.net.Ip; +import com.cloud.vm.DomainRouterVO; +import com.cloud.vm.NicProfile; +import com.cloud.vm.NicVO; +import com.cloud.vm.VirtualMachine; +import com.cloud.vm.VirtualMachine.State; +import com.cloud.vm.VirtualMachineManager; +import com.cloud.vm.dao.DomainRouterDao; +import com.cloud.vm.dao.NicDao; + +/** + * Set of unittests for InternalLoadBalancerVMManager + * + */ + +@RunWith(SpringJUnit4ClassRunner.class) +@ContextConfiguration(locations="classpath:/lb_mgr.xml") +public class InternalLBVMManagerTest extends TestCase { + //The interface to test + @Inject InternalLoadBalancerVMManager _lbVmMgr; + + //Mocked interfaces + @Inject AccountManager _accountMgr; + @Inject ServiceOfferingDao _svcOffDao; + @Inject DomainRouterDao _domainRouterDao; + @Inject NicDao _nicDao; + @Inject AgentManager _agentMgr; + @Inject NetworkModel _ntwkModel; + @Inject VirtualMachineManager _itMgr; + @Inject DataCenterDao _dcDao; + + long validNtwkId = 1L; + long invalidNtwkId = 2L; + String requestedIp = "10.1.1.1"; + DomainRouterVO vm = null; + NetworkVO ntwk = createNetwork(); + long validVmId = 1L; + long invalidVmId = 2L; + + @Before + public void setUp() { + //mock system offering creation as it's used by configure() method called by initComponentsLifeCycle + Mockito.when(_accountMgr.getAccount(1L)).thenReturn(new AccountVO()); + ServiceOfferingVO off = new ServiceOfferingVO("alena", 1, 1, + 1, 1, 1, false, "alena", false, false, null, false, VirtualMachine.Type.InternalLoadBalancerVm, false); + off = setId(off, 1); + Mockito.when(_svcOffDao.persistSystemServiceOffering(Mockito.any(ServiceOfferingVO.class))).thenReturn(off); + + ComponentContext.initComponentsLifeCycle(); + + vm = new DomainRouterVO(1L,off.getId(),1,"alena",1,HypervisorType.XenServer,1,1,1, + false, 0,false,null,false,false, + VirtualMachine.Type.InternalLoadBalancerVm, null); + vm.setRole(Role.INTERNAL_LB_VM); + vm = setId(vm, 1); + vm.setPrivateIpAddress("10.2.2.2"); + NicVO nic = new NicVO("somereserver", 1L, 1L, VirtualMachine.Type.InternalLoadBalancerVm); + nic.setIp4Address(requestedIp); + + List emptyList = new ArrayList(); + List nonEmptyList = new ArrayList(); + nonEmptyList.add(vm); + + Mockito.when(_domainRouterDao.listByNetworkAndRole(invalidNtwkId, Role.INTERNAL_LB_VM)).thenReturn(emptyList); + Mockito.when(_domainRouterDao.listByNetworkAndRole(validNtwkId, Role.INTERNAL_LB_VM)).thenReturn(nonEmptyList); + + Mockito.when(_nicDao.findByNtwkIdAndInstanceId(validNtwkId, 1)).thenReturn(nic); + Mockito.when(_nicDao.findByNtwkIdAndInstanceId(invalidNtwkId, 1)).thenReturn(nic); + + Answer answer= new Answer(null, true, null); + Answer[] answers = new Answer[1]; + answers[0] = answer; + + try { + Mockito.when(_agentMgr.send(Mockito.anyLong(), Mockito.any(Commands.class))).thenReturn(answers); + } catch (AgentUnavailableException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (OperationTimedoutException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } + + createNetwork(); + Mockito.when(_ntwkModel.getNetwork(Mockito.anyLong())).thenReturn(ntwk); + + + Mockito.when(_itMgr.toNicTO(Mockito.any(NicProfile.class), Mockito.any(HypervisorType.class))).thenReturn(null); + Mockito.when(_domainRouterDao.findById(Mockito.anyLong())).thenReturn(vm); + DataCenterVO dc = new DataCenterVO + (1L, null, null, null, null, null, null, null, null, null, NetworkType.Advanced, null, null); + Mockito.when(_dcDao.findById(Mockito.anyLong())).thenReturn(dc); + + + try { + Mockito.when(_itMgr.expunge(Mockito.any(DomainRouterVO.class), Mockito.any(User.class), Mockito.any(Account.class))).thenReturn(true); + } catch (ResourceUnavailableException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } + + Mockito.when(_domainRouterDao.findById(validVmId)).thenReturn(vm); + Mockito.when(_domainRouterDao.findById(invalidVmId)).thenReturn(null); + + } + + protected NetworkVO createNetwork() { + ntwk = new NetworkVO(); + try { + ntwk.setBroadcastUri(new URI("somevlan")); + } catch (URISyntaxException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } + ntwk = setId(ntwk, 1L); + return ntwk; + } + + //TESTS FOR findInternalLbVms METHOD + + @Test + public void findInternalLbVmsForInvalidNetwork() { + List vms = _lbVmMgr.findInternalLbVms(invalidNtwkId, new Ip(requestedIp)); + assertTrue("Non empty vm list was returned for invalid network id", vms.isEmpty()); + } + + @Test + public void findInternalLbVmsForValidNetwork() { + List vms = _lbVmMgr.findInternalLbVms(validNtwkId, new Ip(requestedIp)); + assertTrue("Empty vm list was returned for valid network id", !vms.isEmpty()); + } + + + //TESTS FOR applyLoadBalancingRules METHOD + @Test + public void applyEmptyRulesSet() { + boolean result = false; + List vms = new ArrayList(); + try { + result = _lbVmMgr.applyLoadBalancingRules(new NetworkVO(), new ArrayList(), vms); + } catch (ResourceUnavailableException e) { + + } finally { + assertTrue("Got failure when tried to apply empty list of rules", result); + } + } + + @Test (expected = CloudRuntimeException.class) + public void applyWithEmptyVmsSet() { + boolean result = false; + List vms = new ArrayList(); + List rules = new ArrayList(); + LoadBalancingRule rule = new LoadBalancingRule(null, null, + null, null, null); + + rules.add(rule); + try { + result = _lbVmMgr.applyLoadBalancingRules(new NetworkVO(), rules, vms); + } catch (ResourceUnavailableException e) { + } finally { + assertFalse("Got success when tried to apply with the empty internal lb vm list", result); + } + } + + @Test (expected = ResourceUnavailableException.class) + public void applyToVmInStartingState() throws ResourceUnavailableException { + boolean result = false; + List vms = new ArrayList(); + vm.setState(State.Starting); + vms.add(vm); + + List rules = new ArrayList(); + LoadBalancingRule rule = new LoadBalancingRule(null, null, + null, null, null); + + rules.add(rule); + try { + result = _lbVmMgr.applyLoadBalancingRules(new NetworkVO(), rules, vms); + } finally { + assertFalse("Rules were applied to vm in Starting state", result); + } + } + + + @Test + public void applyToVmInStoppedState() throws ResourceUnavailableException { + boolean result = false; + List vms = new ArrayList(); + vm.setState(State.Stopped); + vms.add(vm); + + List rules = new ArrayList(); + LoadBalancingRule rule = new LoadBalancingRule(null, null, + null, null, null); + + rules.add(rule); + try { + result = _lbVmMgr.applyLoadBalancingRules(new NetworkVO(), rules, vms); + } finally { + assertTrue("Rules failed to apply to vm in Stopped state", result); + } + } + + + @Test + public void applyToVmInStoppingState() throws ResourceUnavailableException { + boolean result = false; + List vms = new ArrayList(); + vm.setState(State.Stopping); + vms.add(vm); + + List rules = new ArrayList(); + LoadBalancingRule rule = new LoadBalancingRule(null, null, + null, null, null); + + rules.add(rule); + try { + result = _lbVmMgr.applyLoadBalancingRules(new NetworkVO(), rules, vms); + } finally { + assertTrue("Rules failed to apply to vm in Stopping state", result); + } + } + + + @Test + public void applyToVmInRunningState() throws ResourceUnavailableException { + boolean result = false; + List vms = new ArrayList(); + vm.setState(State.Running); + vms.add(vm); + + List rules = new ArrayList(); + ApplicationLoadBalancerRuleVO lb = new ApplicationLoadBalancerRuleVO(null, null, 22, 22, "roundrobin", + 1L, 1L, 1L, new Ip(requestedIp), 1L, Scheme.Internal); + lb.setState(FirewallRule.State.Add); + + LoadBalancingRule rule = new LoadBalancingRule(lb, null, + null, null, new Ip(requestedIp)); + + rules.add(rule); + + ntwk.getId(); + + try { + result = _lbVmMgr.applyLoadBalancingRules(ntwk, rules, vms); + } finally { + assertTrue("Rules failed to apply to vm in Running state", result); + } + } + + + //TESTS FOR destroyInternalLbVm METHOD + @Test + public void destroyNonExistingVM() throws ResourceUnavailableException, ConcurrentOperationException { + boolean result = false; + + try { + result = _lbVmMgr.destroyInternalLbVm(invalidVmId, new AccountVO(), 1L); + } finally { + assertTrue("Failed to destroy non-existing vm", result); + } + } + + @Test + public void destroyExistingVM() throws ResourceUnavailableException, ConcurrentOperationException { + boolean result = false; + + try { + result = _lbVmMgr.destroyInternalLbVm(validVmId, new AccountVO(), 1L); + } finally { + assertTrue("Failed to destroy valid vm", result); + } + } + + + private static ServiceOfferingVO setId(ServiceOfferingVO vo, long id) { + ServiceOfferingVO voToReturn = vo; + Class c = voToReturn.getClass(); + try { + Field f = c.getSuperclass().getDeclaredField("id"); + f.setAccessible(true); + f.setLong(voToReturn, id); + } catch (NoSuchFieldException ex) { + return null; + } catch (IllegalAccessException ex) { + return null; + } + + return voToReturn; + } + + + private static NetworkVO setId(NetworkVO vo, long id) { + NetworkVO voToReturn = vo; + Class c = voToReturn.getClass(); + try { + Field f = c.getDeclaredField("id"); + f.setAccessible(true); + f.setLong(voToReturn, id); + } catch (NoSuchFieldException ex) { + return null; + } catch (IllegalAccessException ex) { + return null; + } + + return voToReturn; + } + + private static DomainRouterVO setId(DomainRouterVO vo, long id) { + DomainRouterVO voToReturn = vo; + Class c = voToReturn.getClass(); + try { + Field f = c.getSuperclass().getDeclaredField("id"); + f.setAccessible(true); + f.setLong(voToReturn, id); + } catch (NoSuchFieldException ex) { + return null; + } catch (IllegalAccessException ex) { + return null; + } + + return voToReturn; + } + +} diff --git a/plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbvmmgr/InternalLBVMServiceTest.java b/plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbvmmgr/InternalLBVMServiceTest.java new file mode 100644 index 00000000000..75f54faf8f0 --- /dev/null +++ b/plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbvmmgr/InternalLBVMServiceTest.java @@ -0,0 +1,291 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.internallbvmmgr; + +import java.lang.reflect.Field; +import java.util.Map; + +import javax.inject.Inject; + +import junit.framework.TestCase; + +import org.apache.cloudstack.network.lb.InternalLoadBalancerVMService; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.mockito.Mockito; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +import com.cloud.deploy.DeploymentPlan; +import com.cloud.exception.ConcurrentOperationException; +import com.cloud.exception.InsufficientCapacityException; +import com.cloud.exception.InvalidParameterValueException; +import com.cloud.exception.OperationTimedoutException; +import com.cloud.exception.ResourceUnavailableException; +import com.cloud.exception.StorageUnavailableException; +import com.cloud.hypervisor.Hypervisor.HypervisorType; +import com.cloud.network.router.VirtualRouter; +import com.cloud.network.router.VirtualRouter.Role; +import com.cloud.service.ServiceOfferingVO; +import com.cloud.service.dao.ServiceOfferingDao; +import com.cloud.user.Account; +import com.cloud.user.AccountManager; +import com.cloud.user.AccountVO; +import com.cloud.user.User; +import com.cloud.utils.component.ComponentContext; +import com.cloud.vm.DomainRouterVO; +import com.cloud.vm.VirtualMachine; +import com.cloud.vm.VirtualMachineManager; +import com.cloud.vm.dao.DomainRouterDao; + +/** + * Set of unittests for InternalLoadBalancerVMService + * + */ + +@RunWith(SpringJUnit4ClassRunner.class) +@ContextConfiguration(locations="classpath:/lb_svc.xml") +@SuppressWarnings("unchecked") +public class InternalLBVMServiceTest extends TestCase { + //The interface to test + @Inject InternalLoadBalancerVMService _lbVmSvc; + + //Mocked interfaces + @Inject AccountManager _accountMgr; + @Inject ServiceOfferingDao _svcOffDao; + @Inject DomainRouterDao _domainRouterDao; + @Inject VirtualMachineManager _itMgr; + + long validVmId = 1L; + long nonExistingVmId = 2L; + long nonInternalLbVmId = 3L; + + @Before + public void setUp() { + //mock system offering creation as it's used by configure() method called by initComponentsLifeCycle + Mockito.when(_accountMgr.getAccount(1L)).thenReturn(new AccountVO()); + ServiceOfferingVO off = new ServiceOfferingVO("alena", 1, 1, + 1, 1, 1, false, "alena", false, false, null, false, VirtualMachine.Type.InternalLoadBalancerVm, false); + off = setId(off, 1); + Mockito.when(_svcOffDao.persistSystemServiceOffering(Mockito.any(ServiceOfferingVO.class))).thenReturn(off); + + ComponentContext.initComponentsLifeCycle(); + + DomainRouterVO validVm = new DomainRouterVO(validVmId,off.getId(),1,"alena",1,HypervisorType.XenServer,1,1,1, + false, 0,false,null,false,false, + VirtualMachine.Type.InternalLoadBalancerVm, null); + validVm.setRole(Role.INTERNAL_LB_VM); + DomainRouterVO nonInternalLbVm = new DomainRouterVO(validVmId,off.getId(),1,"alena",1,HypervisorType.XenServer,1,1,1, + false, 0,false,null,false,false, + VirtualMachine.Type.DomainRouter, null); + nonInternalLbVm.setRole(Role.VIRTUAL_ROUTER); + + Mockito.when(_domainRouterDao.findById(validVmId)).thenReturn(validVm); + Mockito.when(_domainRouterDao.findById(nonExistingVmId)).thenReturn(null); + Mockito.when(_domainRouterDao.findById(nonInternalLbVmId)).thenReturn(nonInternalLbVm); + + try { + Mockito.when(_itMgr.start(Mockito.any(DomainRouterVO.class), + Mockito.any(Map.class), Mockito.any(User.class), Mockito.any(Account.class), Mockito.any(DeploymentPlan.class))).thenReturn(validVm); + } catch (InsufficientCapacityException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (ResourceUnavailableException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } + + try { + Mockito.when(_itMgr.advanceStop(Mockito.any(DomainRouterVO.class), Mockito.any(Boolean.class), Mockito.any(User.class), Mockito.any(Account.class))).thenReturn(true); + } catch (ResourceUnavailableException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (OperationTimedoutException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (ConcurrentOperationException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } + + } + + //TESTS FOR START COMMAND + + + @Test (expected = InvalidParameterValueException.class) + public void startNonExistingVm() { + String expectedExcText = null; + try { + _lbVmSvc.startInternalLbVm(nonExistingVmId, _accountMgr.getAccount(1L), 1L); + } catch (StorageUnavailableException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (InsufficientCapacityException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (ConcurrentOperationException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (ResourceUnavailableException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (InvalidParameterValueException e) { + expectedExcText = e.getMessage(); + throw e; + } finally { + assertEquals("Test failed. The non-existing internal lb vm was attempted to start" + + expectedExcText, expectedExcText, "Can't find internal lb vm by id specified"); + } + } + + @Test (expected = InvalidParameterValueException.class) + public void startNonInternalLbVmVm() { + String expectedExcText = null; + try { + _lbVmSvc.startInternalLbVm(nonInternalLbVmId, _accountMgr.getAccount(1L), 1L); + } catch (StorageUnavailableException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (InsufficientCapacityException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (ConcurrentOperationException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (ResourceUnavailableException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + }catch (InvalidParameterValueException e) { + expectedExcText = e.getMessage(); + throw e; + } finally { + assertEquals("Test failed. The existing vm of not Internal lb vm type was attempted to start" + + expectedExcText, expectedExcText, "Can't find internal lb vm by id specified"); + } + } + + @Test + public void startValidLbVmVm() { + VirtualRouter vr = null; + try { + vr = _lbVmSvc.startInternalLbVm(validVmId, _accountMgr.getAccount(1L), 1L); + } catch (StorageUnavailableException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (InsufficientCapacityException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (ConcurrentOperationException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (ResourceUnavailableException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } finally { + assertNotNull("Internal LB vm is null which means it failed to start " + vr, vr); + } + } + + + //TEST FOR STOP COMMAND + @Test (expected = InvalidParameterValueException.class) + public void stopNonExistingVm() { + String expectedExcText = null; + try { + _lbVmSvc.stopInternalLbVm(nonExistingVmId, false,_accountMgr.getAccount(1L), 1L); + } catch (StorageUnavailableException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (ConcurrentOperationException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (ResourceUnavailableException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (InvalidParameterValueException e) { + expectedExcText = e.getMessage(); + throw e; + } finally { + assertEquals("Test failed. The non-existing internal lb vm was attempted to stop" + + expectedExcText, expectedExcText, "Can't find internal lb vm by id specified"); + } + } + + + @Test (expected = InvalidParameterValueException.class) + public void stopNonInternalLbVmVm() { + String expectedExcText = null; + try { + _lbVmSvc.stopInternalLbVm(nonInternalLbVmId, false, _accountMgr.getAccount(1L), 1L); + } catch (StorageUnavailableException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (ConcurrentOperationException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (ResourceUnavailableException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + }catch (InvalidParameterValueException e) { + expectedExcText = e.getMessage(); + throw e; + } finally { + assertEquals("Test failed. The existing vm of not Internal lb vm type was attempted to stop" + + expectedExcText, expectedExcText, "Can't find internal lb vm by id specified"); + } + } + + + @Test + public void stopValidLbVmVm() { + VirtualRouter vr = null; + try { + vr = _lbVmSvc.stopInternalLbVm(validVmId, false, _accountMgr.getAccount(1L), 1L); + } catch (StorageUnavailableException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (ConcurrentOperationException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } catch (ResourceUnavailableException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } finally { + assertNotNull("Internal LB vm is null which means it failed to stop " + vr, vr); + } + } + + + + private static ServiceOfferingVO setId(ServiceOfferingVO vo, long id) { + ServiceOfferingVO voToReturn = vo; + Class c = voToReturn.getClass(); + try { + Field f = c.getSuperclass().getDeclaredField("id"); + f.setAccessible(true); + f.setLong(voToReturn, id); + } catch (NoSuchFieldException ex) { + return null; + } catch (IllegalAccessException ex) { + return null; + } + + return voToReturn; + } +} diff --git a/plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbvmmgr/LbChildTestConfiguration.java b/plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbvmmgr/LbChildTestConfiguration.java new file mode 100644 index 00000000000..0f24f963ae6 --- /dev/null +++ b/plugins/network-elements/internal-loadbalancer/test/org/apache/cloudstack/internallbvmmgr/LbChildTestConfiguration.java @@ -0,0 +1,170 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.internallbvmmgr; + +import java.io.IOException; + +import org.apache.cloudstack.lb.dao.ApplicationLoadBalancerRuleDao; +import org.apache.cloudstack.test.utils.SpringUtils; +import org.mockito.Mockito; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.ComponentScan; +import org.springframework.context.annotation.ComponentScan.Filter; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.FilterType; +import org.springframework.core.type.classreading.MetadataReader; +import org.springframework.core.type.classreading.MetadataReaderFactory; +import org.springframework.core.type.filter.TypeFilter; + +import com.cloud.agent.AgentManager; +import com.cloud.configuration.dao.ConfigurationDao; +import com.cloud.dc.dao.AccountVlanMapDaoImpl; +import com.cloud.dc.dao.DataCenterDao; +import com.cloud.network.NetworkManager; +import com.cloud.network.NetworkModel; +import com.cloud.network.dao.NetworkDao; +import com.cloud.network.dao.PhysicalNetworkServiceProviderDao; +import com.cloud.network.dao.VirtualRouterProviderDao; +import com.cloud.network.lb.LoadBalancingRulesManager; +import com.cloud.offerings.dao.NetworkOfferingDao; +import com.cloud.resource.ResourceManager; +import com.cloud.server.ConfigurationServer; +import com.cloud.service.dao.ServiceOfferingDao; +import com.cloud.storage.dao.VMTemplateDao; +import com.cloud.user.AccountManager; +import com.cloud.vm.VirtualMachineManager; +import com.cloud.vm.dao.DomainRouterDao; +import com.cloud.vm.dao.NicDao; + +@Configuration +@ComponentScan( + basePackageClasses={ + AccountVlanMapDaoImpl.class + }, + includeFilters={@Filter(value=LbChildTestConfiguration.Library.class, type=FilterType.CUSTOM)}, + useDefaultFilters=false + ) + + public class LbChildTestConfiguration { + + public static class Library implements TypeFilter { + + + @Bean + public AccountManager accountManager() { + return Mockito.mock(AccountManager.class); + } + + @Bean + public VirtualMachineManager virtualMachineManager() { + return Mockito.mock(VirtualMachineManager.class); + } + + @Bean + public DomainRouterDao domainRouterDao() { + return Mockito.mock(DomainRouterDao.class); + } + + @Bean + public ConfigurationDao configurationDao() { + return Mockito.mock(ConfigurationDao.class); + } + + @Bean + public VirtualRouterProviderDao virtualRouterProviderDao() { + return Mockito.mock(VirtualRouterProviderDao.class); + } + + @Bean + public ApplicationLoadBalancerRuleDao applicationLoadBalancerRuleDao() { + return Mockito.mock(ApplicationLoadBalancerRuleDao.class); + } + + @Bean + public NetworkModel networkModel() { + return Mockito.mock(NetworkModel.class); + } + + @Bean + public LoadBalancingRulesManager loadBalancingRulesManager() { + return Mockito.mock(LoadBalancingRulesManager.class); + } + + @Bean + public NicDao nicDao() { + return Mockito.mock(NicDao.class); + } + + @Bean + public NetworkDao networkDao() { + return Mockito.mock(NetworkDao.class); + } + + @Bean + public NetworkManager networkManager() { + return Mockito.mock(NetworkManager.class); + } + + @Bean + public ServiceOfferingDao serviceOfferingDao() { + return Mockito.mock(ServiceOfferingDao.class); + } + + @Bean + public PhysicalNetworkServiceProviderDao physicalNetworkServiceProviderDao() { + return Mockito.mock(PhysicalNetworkServiceProviderDao.class); + } + + @Bean + public NetworkOfferingDao networkOfferingDao() { + return Mockito.mock(NetworkOfferingDao.class); + } + + @Bean + public VMTemplateDao vmTemplateDao() { + return Mockito.mock(VMTemplateDao.class); + } + + @Bean + public ResourceManager resourceManager() { + return Mockito.mock(ResourceManager.class); + } + + @Bean + public AgentManager agentManager() { + return Mockito.mock(AgentManager.class); + } + + @Bean + public DataCenterDao dataCenterDao() { + return Mockito.mock(DataCenterDao.class); + } + + @Bean + public ConfigurationServer configurationServer() { + return Mockito.mock(ConfigurationServer.class); + } + + @Override + public boolean match(MetadataReader mdr, MetadataReaderFactory arg1) throws IOException { + mdr.getClassMetadata().getClassName(); + ComponentScan cs = LbChildTestConfiguration.class.getAnnotation(ComponentScan.class); + return SpringUtils.includedInBasePackageClasses(mdr.getClassMetadata().getClassName(), cs); + } + + } +} diff --git a/plugins/network-elements/internal-loadbalancer/test/resources/lb_element.xml b/plugins/network-elements/internal-loadbalancer/test/resources/lb_element.xml new file mode 100644 index 00000000000..5dec9c314f6 --- /dev/null +++ b/plugins/network-elements/internal-loadbalancer/test/resources/lb_element.xml @@ -0,0 +1,46 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/plugins/network-elements/internal-loadbalancer/test/resources/lb_mgr.xml b/plugins/network-elements/internal-loadbalancer/test/resources/lb_mgr.xml new file mode 100644 index 00000000000..1ad6403861c --- /dev/null +++ b/plugins/network-elements/internal-loadbalancer/test/resources/lb_mgr.xml @@ -0,0 +1,46 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/plugins/network-elements/internal-loadbalancer/test/resources/lb_svc.xml b/plugins/network-elements/internal-loadbalancer/test/resources/lb_svc.xml new file mode 100644 index 00000000000..fa822f35302 --- /dev/null +++ b/plugins/network-elements/internal-loadbalancer/test/resources/lb_svc.xml @@ -0,0 +1,46 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/plugins/network-elements/netscaler/src/com/cloud/network/element/NetscalerElement.java b/plugins/network-elements/netscaler/src/com/cloud/network/element/NetscalerElement.java index b8144301810..4f2a0a1da42 100644 --- a/plugins/network-elements/netscaler/src/com/cloud/network/element/NetscalerElement.java +++ b/plugins/network-elements/netscaler/src/com/cloud/network/element/NetscalerElement.java @@ -16,6 +16,22 @@ // under the License. package com.cloud.network.element; +import java.net.URI; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; + +import javax.ejb.Local; +import javax.inject.Inject; + +import org.apache.cloudstack.api.ApiConstants; +import org.apache.cloudstack.network.ExternalNetworkDeviceManager.NetworkDevice; +import org.apache.cloudstack.region.gslb.GslbServiceProvider; +import org.apache.log4j.Logger; + import com.cloud.agent.AgentManager; import com.cloud.agent.api.Answer; import com.cloud.agent.api.routing.GlobalLoadBalancerConfigCommand; @@ -27,7 +43,11 @@ import com.cloud.agent.api.routing.SetStaticNatRulesCommand; import com.cloud.agent.api.to.LoadBalancerTO; import com.cloud.agent.api.to.StaticNatRuleTO; import com.cloud.api.ApiDBUtils; -import com.cloud.api.commands.*; +import com.cloud.api.commands.AddNetscalerLoadBalancerCmd; +import com.cloud.api.commands.ConfigureNetscalerLoadBalancerCmd; +import com.cloud.api.commands.DeleteNetscalerLoadBalancerCmd; +import com.cloud.api.commands.ListNetscalerLoadBalancerNetworksCmd; +import com.cloud.api.commands.ListNetscalerLoadBalancersCmd; import com.cloud.api.response.NetscalerLoadBalancerResponse; import com.cloud.configuration.Config; import com.cloud.configuration.ConfigurationManager; @@ -39,27 +59,48 @@ import com.cloud.dc.HostPodVO; import com.cloud.dc.dao.DataCenterDao; import com.cloud.dc.dao.DataCenterIpAddressDao; import com.cloud.deploy.DeployDestination; -import com.cloud.exception.*; +import com.cloud.exception.ConcurrentOperationException; +import com.cloud.exception.InsufficientCapacityException; +import com.cloud.exception.InsufficientNetworkCapacityException; +import com.cloud.exception.InvalidParameterValueException; +import com.cloud.exception.ResourceUnavailableException; import com.cloud.host.Host; import com.cloud.host.HostVO; import com.cloud.host.dao.HostDao; import com.cloud.host.dao.HostDetailsDao; -import com.cloud.network.*; +import com.cloud.network.ExternalLoadBalancerDeviceManager; +import com.cloud.network.ExternalLoadBalancerDeviceManagerImpl; +import com.cloud.network.IpAddress; +import com.cloud.network.NetScalerPodVO; +import com.cloud.network.Network; import com.cloud.network.Network.Capability; import com.cloud.network.Network.Provider; import com.cloud.network.Network.Service; +import com.cloud.network.NetworkModel; import com.cloud.network.Networks.TrafficType; +import com.cloud.network.PhysicalNetwork; +import com.cloud.network.PhysicalNetworkServiceProvider; +import com.cloud.network.PublicIpAddress; import com.cloud.network.as.AutoScaleCounter; import com.cloud.network.as.AutoScaleCounter.AutoScaleCounterType; -import com.cloud.network.dao.*; +import com.cloud.network.dao.ExternalLoadBalancerDeviceDao; +import com.cloud.network.dao.ExternalLoadBalancerDeviceVO; import com.cloud.network.dao.ExternalLoadBalancerDeviceVO.LBDeviceState; +import com.cloud.network.dao.NetScalerPodDao; +import com.cloud.network.dao.NetworkDao; +import com.cloud.network.dao.NetworkExternalLoadBalancerDao; +import com.cloud.network.dao.NetworkExternalLoadBalancerVO; +import com.cloud.network.dao.NetworkServiceMapDao; +import com.cloud.network.dao.NetworkVO; +import com.cloud.network.dao.PhysicalNetworkDao; +import com.cloud.network.dao.PhysicalNetworkVO; import com.cloud.network.lb.LoadBalancingRule; import com.cloud.network.lb.LoadBalancingRule.LbDestination; import com.cloud.network.resource.NetscalerResource; import com.cloud.network.rules.FirewallRule; -import com.cloud.network.rules.FirewallRule.Purpose; import com.cloud.network.rules.LbStickinessMethod; import com.cloud.network.rules.LbStickinessMethod.StickinessMethodType; +import com.cloud.network.rules.LoadBalancerContainer; import com.cloud.network.rules.StaticNat; import com.cloud.network.vpc.PrivateGateway; import com.cloud.network.vpc.StaticRouteProfile; @@ -75,15 +116,6 @@ import com.cloud.vm.ReservationContext; import com.cloud.vm.VirtualMachine; import com.cloud.vm.VirtualMachineProfile; import com.google.gson.Gson; -import org.apache.cloudstack.api.ApiConstants; -import org.apache.cloudstack.network.ExternalNetworkDeviceManager.NetworkDevice; -import org.apache.cloudstack.region.gslb.GslbServiceProvider; -import org.apache.log4j.Logger; - -import javax.ejb.Local; -import javax.inject.Inject; -import java.net.URI; -import java.util.*; @Local(value = {NetworkElement.class, StaticNatServiceProvider.class, LoadBalancingServiceProvider.class, GslbServiceProvider.class}) public class NetscalerElement extends ExternalLoadBalancerDeviceManagerImpl implements LoadBalancingServiceProvider, @@ -207,6 +239,10 @@ public class NetscalerElement extends ExternalLoadBalancerDeviceManagerImpl impl if (!canHandle(config, Service.Lb)) { return false; } + + if (canHandleLbRules(rules)) { + return false; + } if (isBasicZoneNetwok(config)) { return applyElasticLoadBalancerRules(config, rules); @@ -237,6 +273,9 @@ public class NetscalerElement extends ExternalLoadBalancerDeviceManagerImpl impl // Specifies that load balancing rules can only be made with public IPs that aren't source NAT IPs lbCapabilities.put(Capability.LoadBalancingSupportedIps, "additional"); + // Supports only Public load balancing + lbCapabilities.put(Capability.LbSchemes, LoadBalancerContainer.Scheme.Public.toString()); + // Specifies that load balancing rules can support autoscaling and the list of counters it supports AutoScaleCounter counter; List counterList = new ArrayList(); @@ -644,14 +683,7 @@ public class NetscalerElement extends ExternalLoadBalancerDeviceManagerImpl impl return this; } - public boolean applyElasticLoadBalancerRules(Network network, List rules) throws ResourceUnavailableException { - - List loadBalancingRules = new ArrayList(); - for (FirewallRule rule : rules) { - if (rule.getPurpose().equals(Purpose.LoadBalancing)) { - loadBalancingRules.add((LoadBalancingRule) rule); - } - } + public boolean applyElasticLoadBalancerRules(Network network, List loadBalancingRules) throws ResourceUnavailableException { if (loadBalancingRules == null || loadBalancingRules.isEmpty()) { return true; @@ -682,7 +714,7 @@ public class NetscalerElement extends ExternalLoadBalancerDeviceManagerImpl impl String protocol = rule.getProtocol(); String algorithm = rule.getAlgorithm(); String lbUuid = rule.getUuid(); - String srcIp = _networkMgr.getIp(rule.getSourceIpAddressId()).getAddress().addr(); + String srcIp = rule.getSourceIp().addr(); int srcPort = rule.getSourcePortStart(); List destinations = rule.getDestinations(); @@ -813,16 +845,10 @@ public class NetscalerElement extends ExternalLoadBalancerDeviceManagerImpl impl return null; } - public List getElasticLBRulesHealthCheck(Network network, List rules) + public List getElasticLBRulesHealthCheck(Network network, List loadBalancingRules) throws ResourceUnavailableException { HealthCheckLBConfigAnswer answer = null; - List loadBalancingRules = new ArrayList(); - for (FirewallRule rule : rules) { - if (rule.getPurpose().equals(Purpose.LoadBalancing)) { - loadBalancingRules.add((LoadBalancingRule) rule); - } - } if (loadBalancingRules == null || loadBalancingRules.isEmpty()) { return null; @@ -849,7 +875,7 @@ public class NetscalerElement extends ExternalLoadBalancerDeviceManagerImpl impl String protocol = rule.getProtocol(); String algorithm = rule.getAlgorithm(); String lbUuid = rule.getUuid(); - String srcIp = _networkMgr.getIp(rule.getSourceIpAddressId()).getAddress().addr(); + String srcIp = rule.getSourceIp().addr(); int srcPort = rule.getSourcePortStart(); List destinations = rule.getDestinations(); @@ -874,7 +900,7 @@ public class NetscalerElement extends ExternalLoadBalancerDeviceManagerImpl impl public List updateHealthChecks(Network network, List lbrules) { - if (canHandle(network, Service.Lb)) { + if (canHandle(network, Service.Lb) && canHandleLbRules(lbrules)) { try { if (isBasicZoneNetwok(network)) { @@ -891,7 +917,7 @@ public class NetscalerElement extends ExternalLoadBalancerDeviceManagerImpl impl return null; } - public List getLBHealthChecks(Network network, List rules) + public List getLBHealthChecks(Network network, List rules) throws ResourceUnavailableException { return super.getLBHealthChecks(network, rules); } @@ -960,6 +986,22 @@ public class NetscalerElement extends ExternalLoadBalancerDeviceManagerImpl impl } return null; } + + private boolean canHandleLbRules(List rules) { + Map lbCaps = this.getCapabilities().get(Service.Lb); + if (!lbCaps.isEmpty()) { + String schemeCaps = lbCaps.get(Capability.LbSchemes); + if (schemeCaps != null) { + for (LoadBalancingRule rule : rules) { + if (!schemeCaps.contains(rule.getScheme().toString())) { + s_logger.debug("Scheme " + rules.get(0).getScheme() + " is not supported by the provider " + this.getName()); + return false; + } + } + } + } + return true; + } @Override public boolean implementVpc(Vpc vpc, DeployDestination dest, ReservationContext context) diff --git a/plugins/pom.xml b/plugins/pom.xml index b9402718b89..e49fac9533a 100755 --- a/plugins/pom.xml +++ b/plugins/pom.xml @@ -62,6 +62,7 @@ storage/volume/default alert-handlers/snmp-alerts alert-handlers/syslog-alerts + network-elements/internal-loadbalancer diff --git a/server/src/com/cloud/api/ApiDBUtils.java b/server/src/com/cloud/api/ApiDBUtils.java index b8eea12b4cf..fce1f719086 100755 --- a/server/src/com/cloud/api/ApiDBUtils.java +++ b/server/src/com/cloud/api/ApiDBUtils.java @@ -25,8 +25,6 @@ import java.util.Set; import javax.annotation.PostConstruct; import javax.inject.Inject; -import com.cloud.network.rules.LoadBalancer; -import com.cloud.region.ha.GlobalLoadBalancingRulesService; import org.apache.cloudstack.affinity.AffinityGroup; import org.apache.cloudstack.affinity.AffinityGroupResponse; import org.apache.cloudstack.affinity.dao.AffinityGroupDao; @@ -37,8 +35,8 @@ import org.apache.cloudstack.api.response.AsyncJobResponse; import org.apache.cloudstack.api.response.DiskOfferingResponse; import org.apache.cloudstack.api.response.DomainRouterResponse; import org.apache.cloudstack.api.response.EventResponse; -import org.apache.cloudstack.api.response.HostResponse; import org.apache.cloudstack.api.response.HostForMigrationResponse; +import org.apache.cloudstack.api.response.HostResponse; import org.apache.cloudstack.api.response.InstanceGroupResponse; import org.apache.cloudstack.api.response.ProjectAccountResponse; import org.apache.cloudstack.api.response.ProjectInvitationResponse; @@ -52,6 +50,7 @@ import org.apache.cloudstack.api.response.UserResponse; import org.apache.cloudstack.api.response.UserVmResponse; import org.apache.cloudstack.api.response.VolumeResponse; import org.apache.cloudstack.api.response.ZoneResponse; +import org.apache.cloudstack.lb.dao.ApplicationLoadBalancerRuleDao; import org.apache.cloudstack.storage.datastore.db.PrimaryDataStoreDao; import org.apache.cloudstack.storage.datastore.db.StoragePoolVO; import org.springframework.stereotype.Component; @@ -157,6 +156,8 @@ import com.cloud.network.as.dao.AutoScaleVmGroupPolicyMapDao; import com.cloud.network.as.dao.AutoScaleVmProfileDao; import com.cloud.network.as.dao.ConditionDao; import com.cloud.network.as.dao.CounterDao; +import com.cloud.network.dao.AccountGuestVlanMapDao; +import com.cloud.network.dao.AccountGuestVlanMapVO; import com.cloud.network.dao.FirewallRulesCidrsDao; import com.cloud.network.dao.FirewallRulesDao; import com.cloud.network.dao.IPAddressDao; @@ -181,6 +182,7 @@ import com.cloud.network.dao.Site2SiteVpnGatewayDao; import com.cloud.network.dao.Site2SiteVpnGatewayVO; import com.cloud.network.router.VirtualRouter; import com.cloud.network.rules.FirewallRuleVO; +import com.cloud.network.rules.LoadBalancer; import com.cloud.network.security.SecurityGroup; import com.cloud.network.security.SecurityGroupManager; import com.cloud.network.security.SecurityGroupVO; @@ -204,6 +206,7 @@ import com.cloud.projects.Project; import com.cloud.projects.ProjectAccount; import com.cloud.projects.ProjectInvitation; import com.cloud.projects.ProjectService; +import com.cloud.region.ha.GlobalLoadBalancingRulesService; import com.cloud.resource.ResourceManager; import com.cloud.server.Criteria; import com.cloud.server.ManagementServer; @@ -499,6 +502,7 @@ public class ApiDBUtils { @Inject private VMSnapshotDao vmSnapshotDao; @Inject private NicSecondaryIpDao nicSecondaryIpDao; @Inject private VpcProvisioningService vpcProvSvc; + @Inject private ApplicationLoadBalancerRuleDao _appLbDao; @Inject private AffinityGroupDao affinityGroupDao; @Inject private AffinityGroupJoinDao affinityGroupJoinDao; @Inject private GlobalLoadBalancingRulesService gslbService; diff --git a/server/src/com/cloud/api/ApiResponseHelper.java b/server/src/com/cloud/api/ApiResponseHelper.java index d7eaa2604b1..d5960abba8b 100755 --- a/server/src/com/cloud/api/ApiResponseHelper.java +++ b/server/src/com/cloud/api/ApiResponseHelper.java @@ -44,6 +44,9 @@ import org.apache.cloudstack.api.BaseCmd; import org.apache.cloudstack.api.ResponseGenerator; import org.apache.cloudstack.api.command.user.job.QueryAsyncJobResultCmd; import org.apache.cloudstack.api.response.AccountResponse; +import org.apache.cloudstack.api.response.ApplicationLoadBalancerInstanceResponse; +import org.apache.cloudstack.api.response.ApplicationLoadBalancerResponse; +import org.apache.cloudstack.api.response.ApplicationLoadBalancerRuleResponse; import org.apache.cloudstack.api.response.AsyncJobResponse; import org.apache.cloudstack.api.response.AutoScalePolicyResponse; import org.apache.cloudstack.api.response.AutoScaleVmGroupResponse; @@ -72,6 +75,7 @@ import org.apache.cloudstack.api.response.HostResponse; import org.apache.cloudstack.api.response.HypervisorCapabilitiesResponse; import org.apache.cloudstack.api.response.IPAddressResponse; import org.apache.cloudstack.api.response.InstanceGroupResponse; +import org.apache.cloudstack.api.response.InternalLoadBalancerElementResponse; import org.apache.cloudstack.api.response.IpForwardingRuleResponse; import org.apache.cloudstack.api.response.IsolationMethodResponse; import org.apache.cloudstack.api.response.LBHealthCheckPolicyResponse; @@ -130,6 +134,7 @@ import org.apache.cloudstack.api.response.VpcOfferingResponse; import org.apache.cloudstack.api.response.VpcResponse; import org.apache.cloudstack.api.response.VpnUsersResponse; import org.apache.cloudstack.api.response.ZoneResponse; +import org.apache.cloudstack.network.lb.ApplicationLoadBalancerRule; import org.apache.cloudstack.region.Region; import org.apache.cloudstack.storage.datastore.db.StoragePoolVO; import org.apache.cloudstack.usage.Usage; @@ -189,6 +194,7 @@ import com.cloud.network.Network; import com.cloud.network.Network.Capability; import com.cloud.network.Network.Provider; import com.cloud.network.Network.Service; +import com.cloud.network.NetworkModel; import com.cloud.network.NetworkProfile; import com.cloud.network.Networks.IsolationType; import com.cloud.network.Networks.TrafficType; @@ -216,6 +222,7 @@ import com.cloud.network.rules.FirewallRule; import com.cloud.network.rules.FirewallRuleVO; import com.cloud.network.rules.HealthCheckPolicy; import com.cloud.network.rules.LoadBalancer; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; import com.cloud.network.rules.PortForwardingRule; import com.cloud.network.rules.StaticNatRule; import com.cloud.network.rules.StickinessPolicy; @@ -229,6 +236,7 @@ import com.cloud.network.vpc.Vpc; import com.cloud.network.vpc.VpcOffering; import com.cloud.offering.DiskOffering; import com.cloud.offering.NetworkOffering; +import com.cloud.offering.NetworkOffering.Detail; import com.cloud.offering.ServiceOffering; import com.cloud.offerings.NetworkOfferingVO; import com.cloud.org.Cluster; @@ -269,6 +277,7 @@ import com.cloud.user.UserContext; import com.cloud.uservm.UserVm; import com.cloud.utils.Pair; import com.cloud.utils.StringUtils; +import com.cloud.utils.net.Ip; import com.cloud.utils.net.NetUtils; import com.cloud.vm.ConsoleProxyVO; import com.cloud.vm.InstanceGroup; @@ -289,6 +298,7 @@ public class ApiResponseHelper implements ResponseGenerator { private static final DecimalFormat s_percentFormat = new DecimalFormat("##.##"); @Inject private EntityManager _entityMgr = null; @Inject private UsageService _usageSvc = null; + @Inject NetworkModel _ntwkModel; @Override public UserResponse createUserResponse(User user) { @@ -750,7 +760,7 @@ public class ApiResponseHelper implements ResponseGenerator { } //set tag information - List tags = ApiDBUtils.listByResourceTypeAndId(TaggedResourceType.UserVm, loadBalancer.getId()); + List tags = ApiDBUtils.listByResourceTypeAndId(TaggedResourceType.LoadBalancer, loadBalancer.getId()); List tagResponses = new ArrayList(); for (ResourceTag tag : tags) { ResourceTagResponse tagResponse = createResourceTagResponse(tag, true); @@ -2263,6 +2273,13 @@ public class ApiResponseHelper implements ResponseGenerator { response.setForVpc(ApiDBUtils.isOfferingForVpc(offering)); response.setServices(serviceResponses); + + //set network offering details + Map details = _ntwkModel.getNtwkOffDetails(offering.getId()); + if (details != null && !details.isEmpty()) { + response.setDetails(details); + } + response.setObjectName("networkoffering"); return response; } @@ -2826,6 +2843,11 @@ public class ApiResponseHelper implements ResponseGenerator { @Override public VirtualRouterProviderResponse createVirtualRouterProviderResponse(VirtualRouterProvider result) { + //generate only response of the VR/VPCVR provider type + if (!(result.getType() == VirtualRouterProvider.VirtualRouterProviderType.VirtualRouter + || result.getType() == VirtualRouterProvider.VirtualRouterProviderType.VPCVirtualRouter)) { + return null; + } VirtualRouterProviderResponse response = new VirtualRouterProviderResponse(); response.setId(result.getUuid()); PhysicalNetworkServiceProvider nsp = ApiDBUtils.findPhysicalNetworkServiceProviderById(result.getNspId()); @@ -3689,6 +3711,73 @@ public class ApiResponseHelper implements ResponseGenerator { return response; } + + @Override + public ApplicationLoadBalancerResponse createLoadBalancerContainerReponse(ApplicationLoadBalancerRule lb, Map lbInstances) { + + ApplicationLoadBalancerResponse lbResponse = new ApplicationLoadBalancerResponse(); + lbResponse.setId(lb.getUuid()); + lbResponse.setName(lb.getName()); + lbResponse.setDescription(lb.getDescription()); + lbResponse.setAlgorithm(lb.getAlgorithm()); + Network nw = ApiDBUtils.findNetworkById(lb.getNetworkId()); + lbResponse.setNetworkId(nw.getUuid()); + populateOwner(lbResponse, lb); + + if (lb.getScheme() == Scheme.Internal) { + lbResponse.setSourceIp(lb.getSourceIp().addr()); + //TODO - create the view for the load balancer rule to reflect the network uuid + Network network = ApiDBUtils.findNetworkById(lb.getNetworkId()); + lbResponse.setSourceIpNetworkId(network.getUuid()); + } else { + //for public, populate the ip information from the ip address + IpAddress publicIp = ApiDBUtils.findIpAddressById(lb.getSourceIpAddressId()); + lbResponse.setSourceIp(publicIp.getAddress().addr()); + Network ntwk = ApiDBUtils.findNetworkById(publicIp.getNetworkId()); + lbResponse.setSourceIpNetworkId(ntwk.getUuid()); + } + + //set load balancer rules information (only one rule per load balancer in this release) + List ruleResponses = new ArrayList(); + ApplicationLoadBalancerRuleResponse ruleResponse = new ApplicationLoadBalancerRuleResponse(); + ruleResponse.setInstancePort(lb.getDefaultPortStart()); + ruleResponse.setSourcePort(lb.getSourcePortStart()); + String stateToSet = lb.getState().toString(); + if (stateToSet.equals(FirewallRule.State.Revoke)) { + stateToSet = "Deleting"; + } + ruleResponse.setState(stateToSet); + ruleResponse.setObjectName("loadbalancerrule"); + ruleResponses.add(ruleResponse); + lbResponse.setLbRules(ruleResponses); + + //set Lb instances information + List instanceResponses = new ArrayList(); + for (Ip ip : lbInstances.keySet()) { + ApplicationLoadBalancerInstanceResponse instanceResponse = new ApplicationLoadBalancerInstanceResponse(); + instanceResponse.setIpAddress(ip.addr()); + UserVm vm = lbInstances.get(ip); + instanceResponse.setId(vm.getUuid()); + instanceResponse.setName(vm.getInstanceName()); + instanceResponse.setObjectName("loadbalancerinstance"); + instanceResponses.add(instanceResponse); + } + + lbResponse.setLbInstances(instanceResponses); + + //set tag information + List tags = ApiDBUtils.listByResourceTypeAndId(TaggedResourceType.LoadBalancer, lb.getId()); + List tagResponses = new ArrayList(); + for (ResourceTag tag : tags) { + ResourceTagResponse tagResponse = createResourceTagResponse(tag, true); + tagResponses.add(tagResponse); + } + lbResponse.setTags(tagResponses); + + lbResponse.setObjectName("loadbalancer"); + return lbResponse; + } + @Override public AffinityGroupResponse createAffinityGroupResponse(AffinityGroup group) { @@ -3719,8 +3808,25 @@ public class ApiResponseHelper implements ResponseGenerator { } } + + @Override + public InternalLoadBalancerElementResponse createInternalLbElementResponse(VirtualRouterProvider result) { + if (result.getType() != VirtualRouterProvider.VirtualRouterProviderType.InternalLbVm) { + return null; + } + InternalLoadBalancerElementResponse response = new InternalLoadBalancerElementResponse(); + response.setId(result.getUuid()); + PhysicalNetworkServiceProvider nsp = ApiDBUtils.findPhysicalNetworkServiceProviderById(result.getNspId()); + if (nsp != null) { + response.setNspId(nsp.getUuid()); + } + response.setEnabled(result.isEnabled()); + response.setObjectName("internalloadbalancerelement"); + return response; + } + @Override public IsolationMethodResponse createIsolationMethodResponse(IsolationType method) { IsolationMethodResponse response = new IsolationMethodResponse(); diff --git a/server/src/com/cloud/api/query/QueryManagerImpl.java b/server/src/com/cloud/api/query/QueryManagerImpl.java index 50018e53efb..808b1efceb1 100644 --- a/server/src/com/cloud/api/query/QueryManagerImpl.java +++ b/server/src/com/cloud/api/query/QueryManagerImpl.java @@ -29,7 +29,9 @@ import javax.inject.Inject; import org.apache.cloudstack.affinity.AffinityGroupResponse; import org.apache.cloudstack.affinity.AffinityGroupVMMapVO; import org.apache.cloudstack.affinity.dao.AffinityGroupVMMapDao; +import org.apache.cloudstack.api.BaseListProjectAndAccountResourcesCmd; import org.apache.cloudstack.api.command.admin.host.ListHostsCmd; +import org.apache.cloudstack.api.command.admin.internallb.ListInternalLBVMsCmd; import org.apache.cloudstack.api.command.admin.router.ListRoutersCmd; import org.apache.cloudstack.api.command.admin.storage.ListStoragePoolsCmd; import org.apache.cloudstack.api.command.admin.user.ListUsersCmd; @@ -981,7 +983,21 @@ public class QueryManagerImpl extends ManagerBase implements QueryService { @Override public ListResponse searchForRouters(ListRoutersCmd cmd) { - Pair, Integer> result = searchForRoutersInternal(cmd); + Pair, Integer> result = searchForRoutersInternal(cmd, cmd.getId(), cmd.getRouterName(), + cmd.getState(), cmd.getZoneId(), cmd.getPodId(), cmd.getHostId(), cmd.getKeyword(), cmd.getNetworkId(), + cmd.getVpcId(), cmd.getForVpc(), cmd.getRole(), cmd.getZoneType()); + ListResponse response = new ListResponse(); + + List routerResponses = ViewResponseHelper.createDomainRouterResponse(result.first().toArray(new DomainRouterJoinVO[result.first().size()])); + response.setResponses(routerResponses, result.second()); + return response; + } + + @Override + public ListResponse searchForInternalLbVms(ListInternalLBVMsCmd cmd) { + Pair, Integer> result = searchForRoutersInternal(cmd, cmd.getId(), cmd.getRouterName(), + cmd.getState(), cmd.getZoneId(), cmd.getPodId(), cmd.getHostId(), cmd.getKeyword(), cmd.getNetworkId(), + cmd.getVpcId(), cmd.getForVpc(), cmd.getRole(), cmd.getZoneType()); ListResponse response = new ListResponse(); List routerResponses = ViewResponseHelper.createDomainRouterResponse(result.first().toArray(new DomainRouterJoinVO[result.first().size()])); @@ -990,18 +1006,9 @@ public class QueryManagerImpl extends ManagerBase implements QueryService { } - private Pair, Integer> searchForRoutersInternal(ListRoutersCmd cmd) { - Long id = cmd.getId(); - String name = cmd.getRouterName(); - String state = cmd.getState(); - Long zoneId = cmd.getZoneId(); - String zoneType = cmd.getZoneType(); - Long pod = cmd.getPodId(); - Long hostId = cmd.getHostId(); - String keyword = cmd.getKeyword(); - Long networkId = cmd.getNetworkId(); - Long vpcId = cmd.getVpcId(); - Boolean forVpc = cmd.getForVpc(); + private Pair, Integer> searchForRoutersInternal(BaseListProjectAndAccountResourcesCmd cmd, Long id, + String name, String state, Long zoneId, Long podId, Long hostId, String keyword, Long networkId, Long vpcId, Boolean forVpc, String role, String zoneType) { + Account caller = UserContext.current().getCaller(); List permittedAccounts = new ArrayList(); @@ -1032,6 +1039,7 @@ public class QueryManagerImpl extends ManagerBase implements QueryService { sb.and("podId", sb.entity().getPodId(), SearchCriteria.Op.EQ); sb.and("hostId", sb.entity().getHostId(), SearchCriteria.Op.EQ); sb.and("vpcId", sb.entity().getVpcId(), SearchCriteria.Op.EQ); + sb.and("role", sb.entity().getRole(), SearchCriteria.Op.EQ); if (forVpc != null) { if (forVpc) { @@ -1073,13 +1081,14 @@ public class QueryManagerImpl extends ManagerBase implements QueryService { sc.setParameters("dataCenterId", zoneId); } + if (podId != null) { + sc.setParameters("podId", podId); + } + if (zoneType != null) { sc.setParameters("dataCenterType", zoneType); } - if (pod != null) { - sc.setParameters("podId", pod); - } if (hostId != null) { sc.setParameters("hostId", hostId); @@ -1092,6 +1101,10 @@ public class QueryManagerImpl extends ManagerBase implements QueryService { if (vpcId != null) { sc.setParameters("vpcId", vpcId); } + + if (role != null) { + sc.setParameters("role", role); + } // search VR details by ids Pair, Integer> uniqueVrPair = _routerJoinDao.searchAndCount(sc, searchFilter); diff --git a/server/src/com/cloud/api/query/dao/DomainRouterJoinDaoImpl.java b/server/src/com/cloud/api/query/dao/DomainRouterJoinDaoImpl.java index 125db17c760..a7a83de14a1 100644 --- a/server/src/com/cloud/api/query/dao/DomainRouterJoinDaoImpl.java +++ b/server/src/com/cloud/api/query/dao/DomainRouterJoinDaoImpl.java @@ -32,6 +32,7 @@ import com.cloud.api.query.vo.DomainRouterJoinVO; import com.cloud.configuration.dao.ConfigurationDao; import com.cloud.network.Networks.TrafficType; import com.cloud.network.router.VirtualRouter; +import com.cloud.network.router.VirtualRouter.Role; import com.cloud.user.Account; import com.cloud.utils.db.GenericDaoBase; import com.cloud.utils.db.SearchBuilder; @@ -156,6 +157,8 @@ public class DomainRouterJoinDaoImpl extends GenericDaoBase> serviceProviderMap, boolean isDefault, Network.GuestType type, boolean systemOnly, Long serviceOfferingId, boolean conserveMode, Map> serviceCapabilityMap, - boolean specifyIpRanges, boolean isPersistent); + boolean specifyIpRanges, boolean isPersistent, Map details); Vlan createVlanAndPublicIpRange(long zoneId, long networkId, long physicalNetworkId, boolean forVirtualNetwork, Long podId, String startIP, String endIP, String vlanGateway, String vlanNetmask, String vlanId, Account vlanOwner, String startIPv6, String endIPv6, String vlanIp6Gateway, String vlanIp6Cidr) throws InsufficientCapacityException, ConcurrentOperationException, InvalidParameterValueException; diff --git a/server/src/com/cloud/configuration/ConfigurationManagerImpl.java b/server/src/com/cloud/configuration/ConfigurationManagerImpl.java index 37ca793c556..fdc0ffbabe1 100755 --- a/server/src/com/cloud/configuration/ConfigurationManagerImpl.java +++ b/server/src/com/cloud/configuration/ConfigurationManagerImpl.java @@ -39,6 +39,7 @@ import javax.naming.NamingException; import javax.naming.directory.DirContext; import javax.naming.directory.InitialDirContext; + import com.cloud.dc.*; import com.cloud.dc.dao.*; import com.cloud.user.*; @@ -81,6 +82,17 @@ import com.cloud.configuration.Resource.ResourceType; import com.cloud.configuration.dao.ConfigurationDao; import com.cloud.dc.DataCenter.NetworkType; import com.cloud.dc.Vlan.VlanType; +import com.cloud.dc.VlanVO; +import com.cloud.dc.dao.AccountVlanMapDao; +import com.cloud.dc.dao.ClusterDao; +import com.cloud.dc.dao.DataCenterDao; +import com.cloud.dc.dao.DataCenterIpAddressDao; +import com.cloud.dc.dao.DataCenterLinkLocalIpAddressDao; +import com.cloud.dc.dao.DcDetailsDao; +import com.cloud.dc.dao.HostPodDao; +import com.cloud.dc.dao.PodVlanMapDao; +import com.cloud.dc.dao.VlanDao; + import com.cloud.deploy.DataCenterDeployment; import com.cloud.domain.Domain; import com.cloud.domain.DomainVO; @@ -115,10 +127,12 @@ import com.cloud.network.dao.PhysicalNetworkDao; import com.cloud.network.dao.PhysicalNetworkTrafficTypeDao; import com.cloud.network.dao.PhysicalNetworkTrafficTypeVO; import com.cloud.network.dao.PhysicalNetworkVO; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; import com.cloud.network.vpc.VpcManager; import com.cloud.offering.DiskOffering; import com.cloud.offering.NetworkOffering; import com.cloud.offering.NetworkOffering.Availability; +import com.cloud.offering.NetworkOffering.Detail; import com.cloud.offering.ServiceOffering; import com.cloud.offerings.NetworkOfferingServiceMapVO; import com.cloud.offerings.NetworkOfferingVO; @@ -1919,6 +1933,8 @@ public class ConfigurationManagerImpl extends ManagerBase implements Configurati vmType = VirtualMachine.Type.ConsoleProxy; } else if (VirtualMachine.Type.SecondaryStorageVm.toString().toLowerCase().equals(vmTypeString)) { vmType = VirtualMachine.Type.SecondaryStorageVm; + } else if (VirtualMachine.Type.InternalLoadBalancerVm.toString().toLowerCase().equals(vmTypeString)) { + vmType = VirtualMachine.Type.InternalLoadBalancerVm; } else { throw new InvalidParameterValueException("Invalid systemVmType. Supported types are: " + VirtualMachine.Type.DomainRouter + ", " + VirtualMachine.Type.ConsoleProxy + ", " + VirtualMachine.Type.SecondaryStorageVm); @@ -3340,6 +3356,7 @@ public class ConfigurationManagerImpl extends ManagerBase implements Configurati Network.GuestType guestType = null; boolean specifyIpRanges = cmd.getSpecifyIpRanges(); boolean isPersistent = cmd.getIsPersistent(); + Map detailsStr = cmd.getDetails(); // Verify traffic type for (TrafficType tType : TrafficType.values()) { @@ -3432,10 +3449,10 @@ public class ConfigurationManagerImpl extends ManagerBase implements Configurati Network.Service service = Network.Service.getService(serviceStr); if (serviceProviderMap.containsKey(service)) { Set providers = new HashSet(); - // in Acton, don't allow to specify more than 1 provider per service - if (svcPrv.get(serviceStr) != null && svcPrv.get(serviceStr).size() > 1) { + // Allow to specify more than 1 provider per service only if the service is LB + if (!serviceStr.equalsIgnoreCase(Service.Lb.getName()) && svcPrv.get(serviceStr) != null && svcPrv.get(serviceStr).size() > 1) { throw new InvalidParameterValueException("In the current release only one provider can be " + - "specified for the service"); + "specified for the service if the service is not LB"); } for (String prvNameStr : svcPrv.get(serviceStr)) { // check if provider is supported @@ -3508,9 +3525,26 @@ public class ConfigurationManagerImpl extends ManagerBase implements Configurati firewallProviderSet.add(firewallProvider); serviceProviderMap.put(Service.Firewall, firewallProviderSet); } + + Map details = new HashMap(); + if (detailsStr != null) { + for (String detailStr : detailsStr.keySet()) { + NetworkOffering.Detail offDetail = null; + for (NetworkOffering.Detail supportedDetail: NetworkOffering.Detail.values()) { + if (detailStr.equalsIgnoreCase(supportedDetail.toString())) { + offDetail = supportedDetail; + break; + } + } + if (offDetail == null) { + throw new InvalidParameterValueException("Unsupported detail " + detailStr); + } + details.put(offDetail, detailsStr.get(detailStr)); + } + } return createNetworkOffering(name, displayText, trafficType, tags, specifyVlan, availability, networkRate, serviceProviderMap, false, guestType, false, - serviceOfferingId, conserveMode, serviceCapabilityMap, specifyIpRanges, isPersistent); + serviceOfferingId, conserveMode, serviceCapabilityMap, specifyIpRanges, isPersistent, details); } void validateLoadBalancerServiceCapabilities(Map lbServiceCapabilityMap) { @@ -3539,8 +3573,16 @@ public class ConfigurationManagerImpl extends ManagerBase implements Configurati if (!enabled && !disabled) { throw new InvalidParameterValueException("Unknown specified value for " + Capability.InlineMode.getName()); } + } else if (cap == Capability.LbSchemes) { + boolean internalLb = value.contains("internal"); + boolean publicLb = value.contains("public"); + if (!internalLb && !publicLb) { + throw new InvalidParameterValueException("Unknown specified value for " + Capability.LbSchemes.getName()); + } } else { - throw new InvalidParameterValueException("Only " + Capability.SupportedLBIsolation.getName() + ", " + Capability.ElasticLb.getName() + ", " + Capability.InlineMode.getName() + " capabilities can be sepcified for LB service"); + throw new InvalidParameterValueException("Only " + Capability.SupportedLBIsolation.getName() + + ", " + Capability.ElasticLb.getName() + ", " + Capability.InlineMode.getName() + + ", " + Capability.LbSchemes.getName() + " capabilities can be sepcified for LB service"); } } } @@ -3612,7 +3654,7 @@ public class ConfigurationManagerImpl extends ManagerBase implements Configurati @DB public NetworkOfferingVO createNetworkOffering(String name, String displayText, TrafficType trafficType, String tags, boolean specifyVlan, Availability availability, Integer networkRate, Map> serviceProviderMap, boolean isDefault, Network.GuestType type, boolean systemOnly, Long serviceOfferingId, - boolean conserveMode, Map> serviceCapabilityMap, boolean specifyIpRanges, boolean isPersistent) { + boolean conserveMode, Map> serviceCapabilityMap, boolean specifyIpRanges, boolean isPersistent, Map details) { String multicastRateStr = _configDao.getValue("multicast.throttling.rate"); int multicastRate = ((multicastRateStr == null) ? 10 : Integer.parseInt(multicastRateStr)); @@ -3666,6 +3708,8 @@ public class ConfigurationManagerImpl extends ManagerBase implements Configurati boolean elasticIp = false; boolean associatePublicIp = false; boolean inline = false; + boolean publicLb = false; + boolean internalLb = false; if (serviceCapabilityMap != null && !serviceCapabilityMap.isEmpty()) { Map lbServiceCapabilityMap = serviceCapabilityMap.get(Service.Lb); @@ -3690,6 +3734,23 @@ public class ConfigurationManagerImpl extends ManagerBase implements Configurati } else { inline = false; } + + String publicLbStr = lbServiceCapabilityMap.get(Capability.LbSchemes); + if (serviceProviderMap.containsKey(Service.Lb)) { + if (publicLbStr != null) { + _networkModel.checkCapabilityForProvider(serviceProviderMap.get(Service.Lb), Service.Lb, Capability.LbSchemes, publicLbStr); + internalLb = publicLbStr.contains("internal"); + publicLb = publicLbStr.contains("public"); + } else { + //if not specified, default public lb to true + publicLb = true; + } + } + } + + //in the current version of the code, publicLb and specificLb can't both be set to true for the same network offering + if (publicLb && internalLb) { + throw new InvalidParameterValueException("Public lb and internal lb can't be enabled at the same time on the offering"); } Map sourceNatServiceCapabilityMap = serviceCapabilityMap.get(Service.SourceNat); @@ -3724,18 +3785,23 @@ public class ConfigurationManagerImpl extends ManagerBase implements Configurati NetworkOfferingVO offering = new NetworkOfferingVO(name, displayText, trafficType, systemOnly, specifyVlan, networkRate, multicastRate, isDefault, availability, tags, type, conserveMode, dedicatedLb, - sharedSourceNat, redundantRouter, elasticIp, elasticLb, specifyIpRanges, inline, isPersistent, associatePublicIp); + sharedSourceNat, redundantRouter, elasticIp, elasticLb, specifyIpRanges, inline, isPersistent, associatePublicIp, publicLb, internalLb); if (serviceOfferingId != null) { offering.setServiceOfferingId(serviceOfferingId); } + + //validate the details + if (details != null) { + validateNtwkOffDetails(details, serviceProviderMap); + } Transaction txn = Transaction.currentTxn(); txn.start(); - // create network offering object + //1) create network offering object s_logger.debug("Adding network offering " + offering); - offering = _networkOfferingDao.persist(offering); - // populate services and providers + offering = _networkOfferingDao.persist(offering, details); + //2) populate services and providers if (serviceProviderMap != null) { for (Network.Service service : serviceProviderMap.keySet()) { Set providers = serviceProviderMap.get(service); @@ -3769,6 +3835,42 @@ public class ConfigurationManagerImpl extends ManagerBase implements Configurati return offering; } + protected void validateNtwkOffDetails(Map details, Map> serviceProviderMap) { + for (Detail detail : details.keySet()) { + + Provider lbProvider = null; + if (detail == NetworkOffering.Detail.InternalLbProvider || detail == NetworkOffering.Detail.PublicLbProvider) { + //1) Vaidate the detail values - have to match the lb provider name + String providerStr = details.get(detail); + if (Network.Provider.getProvider(providerStr) == null) { + throw new InvalidParameterValueException("Invalid value " + providerStr + " for the detail " + detail); + } + if (serviceProviderMap.get(Service.Lb) != null) { + for (Provider provider : serviceProviderMap.get(Service.Lb)) { + if (provider.getName().equalsIgnoreCase(providerStr)) { + lbProvider = provider; + break; + } + } + } + + if (lbProvider == null) { + throw new InvalidParameterValueException("Invalid value " + details.get(detail) + + " for the detail " + detail + ". The provider is not supported by the network offering"); + } + + //2) validate if the provider supports the scheme + Set lbProviders = new HashSet(); + lbProviders.add(lbProvider); + if (detail == NetworkOffering.Detail.InternalLbProvider) { + _networkModel.checkCapabilityForProvider(lbProviders, Service.Lb, Capability.LbSchemes, Scheme.Internal.toString()); + } else if (detail == NetworkOffering.Detail.PublicLbProvider){ + _networkModel.checkCapabilityForProvider(lbProviders, Service.Lb, Capability.LbSchemes, Scheme.Public.toString()); + } + } + } + } + @Override public List searchForNetworkOfferings(ListNetworkOfferingsCmd cmd) { @@ -3994,6 +4096,7 @@ public class ConfigurationManagerImpl extends ManagerBase implements Configurati public boolean isOfferingForVpc(NetworkOffering offering) { boolean vpcProvider = _ntwkOffServiceMapDao.isProviderForNetworkOffering(offering.getId(), Provider.VPCVirtualRouter); + boolean internalLb = offering.getInternalLb(); return vpcProvider; } diff --git a/server/src/com/cloud/network/ExternalLoadBalancerDeviceManager.java b/server/src/com/cloud/network/ExternalLoadBalancerDeviceManager.java index 9f11b850180..cb00614b086 100644 --- a/server/src/com/cloud/network/ExternalLoadBalancerDeviceManager.java +++ b/server/src/com/cloud/network/ExternalLoadBalancerDeviceManager.java @@ -23,7 +23,7 @@ import com.cloud.exception.InsufficientCapacityException; import com.cloud.exception.ResourceUnavailableException; import com.cloud.host.Host; import com.cloud.network.dao.ExternalLoadBalancerDeviceVO; -import com.cloud.network.rules.FirewallRule; +import com.cloud.network.lb.LoadBalancingRule; import com.cloud.resource.ServerResource; import com.cloud.utils.component.Manager; @@ -89,7 +89,7 @@ public interface ExternalLoadBalancerDeviceManager extends Manager{ * @return true if successfully applied rules * @throws ResourceUnavailableException */ - public boolean applyLoadBalancerRules(Network network, List rules) throws ResourceUnavailableException; + public boolean applyLoadBalancerRules(Network network, List rules) throws ResourceUnavailableException; /** * implements or shutdowns guest network on the load balancer device assigned to the guest network @@ -102,6 +102,6 @@ public interface ExternalLoadBalancerDeviceManager extends Manager{ public boolean manageGuestNetworkWithExternalLoadBalancer(boolean add, Network guestConfig) throws ResourceUnavailableException, InsufficientCapacityException; - public List getLBHealthChecks(Network network, List rules) + public List getLBHealthChecks(Network network, List rules) throws ResourceUnavailableException; } diff --git a/server/src/com/cloud/network/ExternalLoadBalancerDeviceManagerImpl.java b/server/src/com/cloud/network/ExternalLoadBalancerDeviceManagerImpl.java index 686f5bc2a05..f93bf7ae9b5 100644 --- a/server/src/com/cloud/network/ExternalLoadBalancerDeviceManagerImpl.java +++ b/server/src/com/cloud/network/ExternalLoadBalancerDeviceManagerImpl.java @@ -829,19 +829,11 @@ public abstract class ExternalLoadBalancerDeviceManagerImpl extends AdapterBase } @Override - public boolean applyLoadBalancerRules(Network network, List rules) throws ResourceUnavailableException { + public boolean applyLoadBalancerRules(Network network, List loadBalancingRules) throws ResourceUnavailableException { // Find the external load balancer in this zone long zoneId = network.getDataCenterId(); DataCenterVO zone = _dcDao.findById(zoneId); - List loadBalancingRules = new ArrayList(); - - for (FirewallRule rule : rules) { - if (rule.getPurpose().equals(Purpose.LoadBalancing)) { - loadBalancingRules.add((LoadBalancingRule) rule); - } - } - if (loadBalancingRules == null || loadBalancingRules.isEmpty()) { return true; } @@ -870,12 +862,13 @@ public abstract class ExternalLoadBalancerDeviceManagerImpl extends AdapterBase String protocol = rule.getProtocol(); String algorithm = rule.getAlgorithm(); String uuid = rule.getUuid(); - String srcIp = _networkModel.getIp(rule.getSourceIpAddressId()).getAddress().addr(); + String srcIp = rule.getSourceIp().addr(); int srcPort = rule.getSourcePortStart(); List destinations = rule.getDestinations(); if (externalLoadBalancerIsInline) { - MappingNic nic = getLoadBalancingIpNic(zone, network, rule.getSourceIpAddressId(), revoked, null); + long ipId = _networkModel.getPublicIpAddress(rule.getSourceIp().addr(), network.getDataCenterId()).getId(); + MappingNic nic = getLoadBalancingIpNic(zone, network, ipId, revoked, null); mappingStates.add(nic.getState()); NicVO loadBalancingIpNic = nic.getNic(); if (loadBalancingIpNic == null) { @@ -927,7 +920,8 @@ public abstract class ExternalLoadBalancerDeviceManagerImpl extends AdapterBase } else { continue; } - getLoadBalancingIpNic(zone, network, rule.getSourceIpAddressId(), revoke, existedGuestIp); + long sourceIpId = _networkModel.getPublicIpAddress(rule.getSourceIp().addr(), network.getDataCenterId()).getId(); + getLoadBalancingIpNic(zone, network, sourceIpId, revoke, existedGuestIp); } } throw new ResourceUnavailableException(ex.getMessage(), DataCenter.class, network.getDataCenterId()); @@ -1113,7 +1107,7 @@ public abstract class ExternalLoadBalancerDeviceManagerImpl extends AdapterBase } @Override - public List getLBHealthChecks(Network network, List rules) + public List getLBHealthChecks(Network network, List loadBalancingRules) throws ResourceUnavailableException { // Find the external load balancer in this zone @@ -1121,14 +1115,6 @@ public abstract class ExternalLoadBalancerDeviceManagerImpl extends AdapterBase DataCenterVO zone = _dcDao.findById(zoneId); HealthCheckLBConfigAnswer answer = null; - List loadBalancingRules = new ArrayList(); - - for (FirewallRule rule : rules) { - if (rule.getPurpose().equals(Purpose.LoadBalancing)) { - loadBalancingRules.add((LoadBalancingRule) rule); - } - } - if (loadBalancingRules == null || loadBalancingRules.isEmpty()) { return null; } @@ -1158,12 +1144,13 @@ public abstract class ExternalLoadBalancerDeviceManagerImpl extends AdapterBase String protocol = rule.getProtocol(); String algorithm = rule.getAlgorithm(); String uuid = rule.getUuid(); - String srcIp = _networkModel.getIp(rule.getSourceIpAddressId()).getAddress().addr(); + String srcIp = rule.getSourceIp().addr(); int srcPort = rule.getSourcePortStart(); List destinations = rule.getDestinations(); if (externalLoadBalancerIsInline) { - MappingNic nic = getLoadBalancingIpNic(zone, network, rule.getSourceIpAddressId(), revoked, null); + long sourceIpId = _networkModel.getPublicIpAddress(rule.getSourceIp().addr(), network.getDataCenterId()).getId(); + MappingNic nic = getLoadBalancingIpNic(zone, network, sourceIpId, revoked, null); mappingStates.add(nic.getState()); NicVO loadBalancingIpNic = nic.getNic(); if (loadBalancingIpNic == null) { diff --git a/server/src/com/cloud/network/ExternalLoadBalancerUsageManagerImpl.java b/server/src/com/cloud/network/ExternalLoadBalancerUsageManagerImpl.java index d405382f89c..2c8031c64f0 100644 --- a/server/src/com/cloud/network/ExternalLoadBalancerUsageManagerImpl.java +++ b/server/src/com/cloud/network/ExternalLoadBalancerUsageManagerImpl.java @@ -16,6 +16,22 @@ // under the License. package com.cloud.network; +import java.net.URI; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.concurrent.Executors; +import java.util.concurrent.ScheduledExecutorService; +import java.util.concurrent.TimeUnit; + +import javax.ejb.Local; +import javax.inject.Inject; +import javax.naming.ConfigurationException; + +import org.apache.log4j.Logger; +import org.springframework.stereotype.Component; + import com.cloud.agent.AgentManager; import com.cloud.agent.api.ExternalNetworkResourceUsageAnswer; import com.cloud.agent.api.ExternalNetworkResourceUsageCommand; @@ -48,6 +64,7 @@ import com.cloud.network.dao.NetworkServiceMapDao; import com.cloud.network.dao.NetworkVO; import com.cloud.network.dao.PhysicalNetworkDao; import com.cloud.network.dao.PhysicalNetworkServiceProviderDao; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; import com.cloud.network.rules.PortForwardingRuleVO; import com.cloud.network.rules.dao.PortForwardingRulesDao; import com.cloud.offerings.dao.NetworkOfferingDao; @@ -68,20 +85,6 @@ import com.cloud.vm.DomainRouterVO; import com.cloud.vm.NicVO; import com.cloud.vm.dao.DomainRouterDao; import com.cloud.vm.dao.NicDao; -import org.apache.log4j.Logger; -import org.springframework.stereotype.Component; - -import javax.ejb.Local; -import javax.inject.Inject; -import javax.naming.ConfigurationException; -import java.net.URI; -import java.util.ArrayList; -import java.util.HashMap; -import java.util.List; -import java.util.Map; -import java.util.concurrent.Executors; -import java.util.concurrent.ScheduledExecutorService; -import java.util.concurrent.TimeUnit; @Component @Local(value = { ExternalLoadBalancerUsageManager.class }) @@ -647,9 +650,10 @@ public class ExternalLoadBalancerUsageManagerImpl extends ManagerBase implements // If an external load balancer is added, manage one entry for each load balancing rule in this network if (externalLoadBalancer != null && lbAnswer != null) { boolean inline = _networkMgr.isNetworkInlineMode(network); - List loadBalancers = _loadBalancerDao.listByNetworkId(network.getId()); + List loadBalancers = _loadBalancerDao.listByNetworkIdAndScheme(network.getId(), Scheme.Public); for (LoadBalancerVO loadBalancer : loadBalancers) { String publicIp = _networkMgr.getIp(loadBalancer.getSourceIpAddressId()).getAddress().addr(); + if (!createOrUpdateStatsEntry(create, accountId, zoneId, network.getId(), publicIp, externalLoadBalancer.getId(), lbAnswer, inline)) { throw new ExecutionException(networkErrorMsg + ", load balancing rule public IP = " + publicIp); } diff --git a/server/src/com/cloud/network/NetworkManager.java b/server/src/com/cloud/network/NetworkManager.java index 4af716ca12a..34a092a465a 100755 --- a/server/src/com/cloud/network/NetworkManager.java +++ b/server/src/com/cloud/network/NetworkManager.java @@ -43,6 +43,7 @@ import com.cloud.network.element.StaticNatServiceProvider; import com.cloud.network.element.UserDataServiceProvider; import com.cloud.network.guru.NetworkGuru; import com.cloud.network.rules.FirewallRule; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; import com.cloud.network.rules.StaticNat; import com.cloud.offering.NetworkOffering; import com.cloud.offerings.NetworkOfferingVO; @@ -333,7 +334,7 @@ public interface NetworkManager { int getRuleCountForIp(Long addressId, FirewallRule.Purpose purpose, FirewallRule.State state); - LoadBalancingServiceProvider getLoadBalancingProviderForNetwork(Network network); + LoadBalancingServiceProvider getLoadBalancingProviderForNetwork(Network network, Scheme lbScheme); boolean isSecondaryIpSetForNic(long nicId); diff --git a/server/src/com/cloud/network/NetworkManagerImpl.java b/server/src/com/cloud/network/NetworkManagerImpl.java index 4fffbc11072..c91243095da 100755 --- a/server/src/com/cloud/network/NetworkManagerImpl.java +++ b/server/src/com/cloud/network/NetworkManagerImpl.java @@ -62,6 +62,13 @@ import com.cloud.network.guru.NetworkGuru; import com.cloud.network.lb.LoadBalancingRulesManager; import com.cloud.network.rules.*; import com.cloud.network.rules.FirewallRule.Purpose; +import com.cloud.network.rules.FirewallRuleVO; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; +import com.cloud.network.rules.PortForwardingRuleVO; +import com.cloud.network.rules.RulesManager; +import com.cloud.network.rules.StaticNat; +import com.cloud.network.rules.StaticNatRule; +import com.cloud.network.rules.StaticNatRuleImpl; import com.cloud.network.rules.dao.PortForwardingRulesDao; import com.cloud.network.vpc.NetworkACLManager; import com.cloud.network.vpc.VpcManager; @@ -72,6 +79,7 @@ import com.cloud.offering.NetworkOffering.Availability; import com.cloud.offerings.NetworkOfferingServiceMapVO; import com.cloud.offerings.NetworkOfferingVO; import com.cloud.offerings.dao.NetworkOfferingDao; +import com.cloud.offerings.dao.NetworkOfferingDetailsDao; import com.cloud.offerings.dao.NetworkOfferingServiceMapDao; import com.cloud.org.Grouping; import com.cloud.user.*; @@ -156,6 +164,8 @@ public class NetworkManagerImpl extends ManagerBase implements NetworkManager, L @Inject PodVlanMapDao _podVlanMapDao; @Inject + NetworkOfferingDetailsDao _ntwkOffDetailsDao; + @Inject ConfigurationServer _configServer; @Inject AccountGuestVlanMapDao _accountGuestVlanMapDao; @@ -948,7 +958,7 @@ public class NetworkManagerImpl extends ManagerBase implements NetworkManager, L _configMgr.createNetworkOffering(NetworkOffering.QuickCloudNoServices, "Offering for QuickCloud with no services", TrafficType.Guest, null, true, Availability.Optional, null, new HashMap>(), true, - Network.GuestType.Shared, false, null, true, null, true, false); + Network.GuestType.Shared, false, null, true, null, true, false, null); offering.setState(NetworkOffering.State.Enabled); _networkOfferingDao.update(offering.getId(), offering); } @@ -957,14 +967,14 @@ public class NetworkManagerImpl extends ManagerBase implements NetworkManager, L _configMgr.createNetworkOffering(NetworkOffering.DefaultSharedNetworkOfferingWithSGService, "Offering for Shared Security group enabled networks", TrafficType.Guest, null, true, Availability.Optional, null, defaultSharedNetworkOfferingProviders, true, - Network.GuestType.Shared, false, null, true, null, true, false); + Network.GuestType.Shared, false, null, true, null, true, false, null); offering.setState(NetworkOffering.State.Enabled); _networkOfferingDao.update(offering.getId(), offering); } if (_networkOfferingDao.findByUniqueName(NetworkOffering.DefaultSharedNetworkOffering) == null) { offering = _configMgr.createNetworkOffering(NetworkOffering.DefaultSharedNetworkOffering, "Offering for Shared networks", TrafficType.Guest, null, true, Availability.Optional, null, - defaultSharedNetworkOfferingProviders, true, Network.GuestType.Shared, false, null, true, null, true, false); + defaultSharedNetworkOfferingProviders, true, Network.GuestType.Shared, false, null, true, null, true, false, null); offering.setState(NetworkOffering.State.Enabled); _networkOfferingDao.update(offering.getId(), offering); } @@ -987,7 +997,7 @@ public class NetworkManagerImpl extends ManagerBase implements NetworkManager, L offering = _configMgr.createNetworkOffering(NetworkOffering.DefaultIsolatedNetworkOfferingWithSourceNatService, "Offering for Isolated networks with Source Nat service enabled", TrafficType.Guest, null, false, Availability.Required, null, defaultINetworkOfferingProvidersForVpcNetwork, - true, Network.GuestType.Isolated, false, null, true, null, false, false); + true, Network.GuestType.Isolated, false, null, true, null, false, false, null); offering.setState(NetworkOffering.State.Enabled); _networkOfferingDao.update(offering.getId(), offering); } @@ -996,7 +1006,7 @@ public class NetworkManagerImpl extends ManagerBase implements NetworkManager, L offering = _configMgr.createNetworkOffering(NetworkOffering.DefaultIsolatedNetworkOfferingForVpcNetworks, "Offering for Isolated VPC networks with Source Nat service enabled", TrafficType.Guest, null, false, Availability.Optional, null, defaultVPCOffProviders, - true, Network.GuestType.Isolated, false, null, false, null, false, false); + true, Network.GuestType.Isolated, false, null, false, null, false, false, null); offering.setState(NetworkOffering.State.Enabled); _networkOfferingDao.update(offering.getId(), offering); } @@ -1007,7 +1017,7 @@ public class NetworkManagerImpl extends ManagerBase implements NetworkManager, L offering = _configMgr.createNetworkOffering(NetworkOffering.DefaultIsolatedNetworkOfferingForVpcNetworksNoLB, "Offering for Isolated VPC networks with Source Nat service enabled and LB service disabled", TrafficType.Guest, null, false, Availability.Optional, null, defaultVPCOffProviders, - true, Network.GuestType.Isolated, false, null, false, null, false, false); + true, Network.GuestType.Isolated, false, null, false, null, false, false, null); offering.setState(NetworkOffering.State.Enabled); _networkOfferingDao.update(offering.getId(), offering); } @@ -1016,7 +1026,7 @@ public class NetworkManagerImpl extends ManagerBase implements NetworkManager, L offering = _configMgr.createNetworkOffering(NetworkOffering.DefaultIsolatedNetworkOffering, "Offering for Isolated networks with no Source Nat service", TrafficType.Guest, null, true, Availability.Optional, null, defaultIsolatedNetworkOfferingProviders, true, Network.GuestType.Isolated, - false, null, true, null, true, false); + false, null, true, null, true, false, null); offering.setState(NetworkOffering.State.Enabled); _networkOfferingDao.update(offering.getId(), offering); } @@ -1045,7 +1055,7 @@ public class NetworkManagerImpl extends ManagerBase implements NetworkManager, L if (_networkOfferingDao.findByUniqueName(NetworkOffering.DefaultSharedEIPandELBNetworkOffering) == null) { offering = _configMgr.createNetworkOffering(NetworkOffering.DefaultSharedEIPandELBNetworkOffering, "Offering for Shared networks with Elastic IP and Elastic LB capabilities", TrafficType.Guest, null, true, - Availability.Optional, null, netscalerServiceProviders, true, Network.GuestType.Shared, false, null, true, serviceCapabilityMap, true, false); + Availability.Optional, null, netscalerServiceProviders, true, Network.GuestType.Shared, false, null, true, serviceCapabilityMap, true, false, null); offering.setState(NetworkOffering.State.Enabled); offering.setDedicatedLB(false); _networkOfferingDao.update(offering.getId(), offering); @@ -2651,9 +2661,15 @@ public class NetworkManagerImpl extends ManagerBase implements NetworkManager, L success = false; } - // apply load balancer rules - if (!_lbMgr.applyLoadBalancersForNetwork(networkId)) { - s_logger.warn("Failed to reapply load balancer rules as a part of network id=" + networkId + " restart"); + // apply public load balancer rules + if (!_lbMgr.applyLoadBalancersForNetwork(networkId, Scheme.Public)) { + s_logger.warn("Failed to reapply Public load balancer rules as a part of network id=" + networkId + " restart"); + success = false; + } + + // apply internal load balancer rules + if (!_lbMgr.applyLoadBalancersForNetwork(networkId, Scheme.Internal)) { + s_logger.warn("Failed to reapply internal load balancer rules as a part of network id=" + networkId + " restart"); success = false; } @@ -3234,12 +3250,22 @@ public class NetworkManagerImpl extends ManagerBase implements NetworkManager, L } try { - if (!_lbMgr.revokeLoadBalancersForNetwork(networkId)) { - s_logger.warn("Failed to cleanup lb rules as a part of shutdownNetworkRules"); + if (!_lbMgr.revokeLoadBalancersForNetwork(networkId, Scheme.Public)) { + s_logger.warn("Failed to cleanup public lb rules as a part of shutdownNetworkRules"); success = false; } } catch (ResourceUnavailableException ex) { - s_logger.warn("Failed to cleanup lb rules as a part of shutdownNetworkRules due to ", ex); + s_logger.warn("Failed to cleanup public lb rules as a part of shutdownNetworkRules due to ", ex); + success = false; + } + + try { + if (!_lbMgr.revokeLoadBalancersForNetwork(networkId, Scheme.Internal)) { + s_logger.warn("Failed to cleanup internal lb rules as a part of shutdownNetworkRules"); + success = false; + } + } catch (ResourceUnavailableException ex) { + s_logger.warn("Failed to cleanup public lb rules as a part of shutdownNetworkRules due to ", ex); success = false; } @@ -3645,7 +3671,7 @@ public class NetworkManagerImpl extends ManagerBase implements NetworkManager, L } } } else { - NicVO nicVO = _nicDao.findByInstanceIdAndNetworkId(network.getId(), vm.getId()); + NicVO nicVO = _nicDao.findByNtwkIdAndInstanceId(network.getId(), vm.getId()); if (nicVO != null) { nic = _networkModel.getNicProfile(vm, network.getId(), null); } @@ -3747,35 +3773,62 @@ public class NetworkManagerImpl extends ManagerBase implements NetworkManager, L return null; } - protected NetworkElement getElementForServiceInNetwork(Network network, Service service) { + protected List getElementForServiceInNetwork(Network network, Service service) { + List elements = new ArrayList(); List providers = getProvidersForServiceInNetwork(network, service); //Only support one provider now if (providers == null) { s_logger.error("Cannot find " + service.getName() + " provider for network " + network.getId()); return null; } - if (providers.size() != 1) { + if (providers.size() != 1 && service != Service.Lb) { + //support more than one LB providers only s_logger.error("Found " + providers.size() + " " + service.getName() + " providers for network!" + network.getId()); return null; + } + + for (Provider provider : providers) { + NetworkElement element = _networkModel.getElementImplementingProvider(provider.getName()); + s_logger.info("Let " + element.getName() + " handle " + service.getName() + " in network " + network.getId()); + elements.add(element); } - NetworkElement element = _networkModel.getElementImplementingProvider(providers.get(0).getName()); - s_logger.info("Let " + element.getName() + " handle " + service.getName() + " in network " + network.getId()); - return element; + return elements; } @Override public StaticNatServiceProvider getStaticNatProviderForNetwork(Network network) { - NetworkElement element = getElementForServiceInNetwork(network, Service.StaticNat); + //only one provider per Static nat service is supoprted + NetworkElement element = getElementForServiceInNetwork(network, Service.StaticNat).get(0); assert element instanceof StaticNatServiceProvider; return (StaticNatServiceProvider)element; } @Override - public LoadBalancingServiceProvider getLoadBalancingProviderForNetwork(Network network) { - NetworkElement element = getElementForServiceInNetwork(network, Service.Lb); - assert element instanceof LoadBalancingServiceProvider; - return (LoadBalancingServiceProvider)element; + public LoadBalancingServiceProvider getLoadBalancingProviderForNetwork(Network network, Scheme lbScheme) { + List lbElements = getElementForServiceInNetwork(network, Service.Lb); + NetworkElement lbElement = null; + if (lbElements.size() > 1) { + String providerName = null; + //get network offering details + NetworkOffering off = _configMgr.getNetworkOffering(network.getNetworkOfferingId()); + if (lbScheme == Scheme.Public) { + providerName = _ntwkOffDetailsDao.getDetail(off.getId(), NetworkOffering.Detail.PublicLbProvider); + } else { + providerName = _ntwkOffDetailsDao.getDetail(off.getId(), NetworkOffering.Detail.InternalLbProvider); + } + if (providerName == null) { + throw new InvalidParameterValueException("Can't find Lb provider supporting scheme " + lbScheme.toString() + " in network " + network); + } + lbElement = _networkModel.getElementImplementingProvider(providerName); + } else if (lbElements.size() == 1){ + lbElement = lbElements.get(0); + } + + assert lbElement != null; + assert lbElement instanceof LoadBalancingServiceProvider; + return (LoadBalancingServiceProvider)lbElement; } + @Override public boolean isNetworkInlineMode(Network network) { NetworkOfferingVO offering = _networkOfferingDao.findById(network.getNetworkOfferingId()); diff --git a/server/src/com/cloud/network/NetworkModelImpl.java b/server/src/com/cloud/network/NetworkModelImpl.java index eaec6a6b42f..135fd290535 100755 --- a/server/src/com/cloud/network/NetworkModelImpl.java +++ b/server/src/com/cloud/network/NetworkModelImpl.java @@ -32,6 +32,7 @@ import javax.ejb.Local; import javax.inject.Inject; import javax.naming.ConfigurationException; +import org.apache.cloudstack.lb.dao.ApplicationLoadBalancerRuleDao; import org.apache.log4j.Logger; import org.springframework.stereotype.Component; @@ -52,13 +53,11 @@ import com.cloud.exception.InvalidParameterValueException; import com.cloud.exception.PermissionDeniedException; import com.cloud.exception.UnsupportedServiceException; import com.cloud.hypervisor.Hypervisor.HypervisorType; -import com.cloud.server.ConfigurationServer; import com.cloud.network.IpAddress.State; import com.cloud.network.Network.Capability; import com.cloud.network.Network.GuestType; import com.cloud.network.Network.Provider; import com.cloud.network.Network.Service; -import com.cloud.network.Networks.IsolationType; import com.cloud.network.Networks.TrafficType; import com.cloud.network.addr.PublicIp; import com.cloud.network.dao.FirewallRulesDao; @@ -86,11 +85,14 @@ import com.cloud.network.rules.FirewallRuleVO; import com.cloud.network.rules.dao.PortForwardingRulesDao; import com.cloud.network.vpc.dao.PrivateIpDao; import com.cloud.offering.NetworkOffering; +import com.cloud.offering.NetworkOffering.Detail; import com.cloud.offerings.NetworkOfferingServiceMapVO; import com.cloud.offerings.NetworkOfferingVO; import com.cloud.offerings.dao.NetworkOfferingDao; +import com.cloud.offerings.dao.NetworkOfferingDetailsDao; import com.cloud.offerings.dao.NetworkOfferingServiceMapDao; import com.cloud.projects.dao.ProjectAccountDao; +import com.cloud.server.ConfigurationServer; import com.cloud.user.Account; import com.cloud.user.AccountVO; import com.cloud.user.DomainManager; @@ -183,9 +185,13 @@ public class NetworkModelImpl extends ManagerBase implements NetworkModel { @Inject UserIpv6AddressDao _ipv6Dao; @Inject - NicSecondaryIpDao _nicSecondaryIpDao;; + NicSecondaryIpDao _nicSecondaryIpDao; + @Inject + ApplicationLoadBalancerRuleDao _appLbRuleDao; @Inject private ProjectAccountDao _projectAccountDao; + @Inject + NetworkOfferingDetailsDao _ntwkOffDetailsDao; private final HashMap _systemNetworks = new HashMap(5); static Long _privateOfferingId = null; @@ -604,7 +610,6 @@ public class NetworkModelImpl extends ManagerBase implements NetworkModel { NetworkElement element = getElementImplementingProvider(instance.getProvider()); if (element != null) { Map> elementCapabilities = element.getCapabilities(); - ; if (elementCapabilities != null) { networkCapabilities.put(service, elementCapabilities.get(service)); } @@ -917,7 +922,7 @@ public class NetworkModelImpl extends ManagerBase implements NetworkModel { boolean isUserVmsDefaultNetwork = false; boolean isDomRGuestOrPublicNetwork = false; if (vm != null) { - Nic nic = _nicDao.findByInstanceIdAndNetworkId(networkId, vmId); + Nic nic = _nicDao.findByNtwkIdAndInstanceId(networkId, vmId); if (vm.getType() == Type.User && nic != null && nic.isDefaultNic()) { isUserVmsDefaultNetwork = true; } else if (vm.getType() == Type.DomainRouter && ntwkOff != null && (ntwkOff.getTrafficType() == TrafficType.Public || ntwkOff.getTrafficType() == TrafficType.Guest)) { @@ -1465,10 +1470,8 @@ public class NetworkModelImpl extends ManagerBase implements NetworkModel { throw new UnsupportedServiceException("Service " + service.getName() + " doesn't have capability " + cap.getName() + " for element=" + element.getName() + " implementing Provider=" + provider.getName()); } - - capValue = capValue.toLowerCase(); - - if (!value.contains(capValue)) { + + if (!value.toLowerCase().contains(capValue.toLowerCase())) { throw new UnsupportedServiceException("Service " + service.getName() + " doesn't support value " + capValue + " for capability " + cap.getName() + " for element=" + element.getName() + " implementing Provider=" + provider.getName()); } @@ -1664,9 +1667,7 @@ public class NetworkModelImpl extends ManagerBase implements NetworkModel { @Override public Set getAvailableIps(Network network, String requestedIp) { String[] cidr = network.getCidr().split("/"); - List ips = _nicDao.listIpAddressInNetwork(network.getId()); - List secondaryIps = _nicSecondaryIpDao.listSecondaryIpAddressInNetwork(network.getId()); - ips.addAll(secondaryIps); + List ips = getUsedIpsInNetwork(network); Set usedIps = new TreeSet(); for (String ip : ips) { @@ -1677,6 +1678,7 @@ public class NetworkModelImpl extends ManagerBase implements NetworkModel { usedIps.add(NetUtils.ip2Long(ip)); } + Set allPossibleIps = NetUtils.getAllIpsFromCidr(cidr[0], Integer.parseInt(cidr[1]), usedIps); String gateway = network.getGateway(); @@ -1685,6 +1687,19 @@ public class NetworkModelImpl extends ManagerBase implements NetworkModel { return allPossibleIps; } + + @Override + public List getUsedIpsInNetwork(Network network) { + //Get all ips used by vms nics + List ips = _nicDao.listIpAddressInNetwork(network.getId()); + //Get all secondary ips for nics + List secondaryIps = _nicSecondaryIpDao.listSecondaryIpAddressInNetwork(network.getId()); + ips.addAll(secondaryIps); + //Get ips used by load balancers + List lbIps = _appLbRuleDao.listLbIpsBySourceIpNetworkId(network.getId()); + ips.addAll(lbIps); + return ips; + } @Override public String getDomainNetworkDomain(long domainId, long zoneId) { @@ -1792,7 +1807,7 @@ public class NetworkModelImpl extends ManagerBase implements NetworkModel { if (broadcastUri != null) { nic = _nicDao.findByNetworkIdInstanceIdAndBroadcastUri(networkId, vm.getId(), broadcastUri); } else { - nic = _nicDao.findByInstanceIdAndNetworkId(networkId, vm.getId()); + nic = _nicDao.findByNtwkIdAndInstanceId(networkId, vm.getId()); } if (nic == null) { return null; @@ -2051,6 +2066,22 @@ public class NetworkModelImpl extends ManagerBase implements NetworkModel { return null; } + + @Override + public IpAddress getPublicIpAddress(String ipAddress, long zoneId) { + List networks = _networksDao.listByZoneAndTrafficType(zoneId, TrafficType.Public); + if (networks.isEmpty() || networks.size() > 1) { + throw new CloudRuntimeException("Can't find public network in the zone specified"); + } + + return _ipAddressDao.findByIpAndSourceNetworkId(networks.get(0).getId(), ipAddress); + } + + @Override + public Map getNtwkOffDetails(long offId) { + return _ntwkOffDetailsDao.getNtwkOffDetails(offId); + } + @Override public Networks.IsolationType[] listNetworkIsolationMethods() { diff --git a/server/src/com/cloud/network/NetworkServiceImpl.java b/server/src/com/cloud/network/NetworkServiceImpl.java index f44688c7594..88155582569 100755 --- a/server/src/com/cloud/network/NetworkServiceImpl.java +++ b/server/src/com/cloud/network/NetworkServiceImpl.java @@ -47,6 +47,7 @@ import org.apache.cloudstack.api.command.user.network.CreateNetworkCmd; import org.apache.cloudstack.api.command.user.network.ListNetworksCmd; import org.apache.cloudstack.api.command.user.network.RestartNetworkCmd; import org.apache.cloudstack.api.command.user.vm.ListNicsCmd; +import org.apache.cloudstack.network.element.InternalLoadBalancerElementService; import org.apache.log4j.Logger; import org.springframework.stereotype.Component; @@ -170,6 +171,33 @@ import com.cloud.vm.dao.NicSecondaryIpDao; import com.cloud.vm.dao.NicSecondaryIpVO; import com.cloud.vm.dao.UserVmDao; import com.cloud.vm.dao.VMInstanceDao; +import com.cloud.vm.*; +import com.cloud.vm.dao.*; +import org.apache.cloudstack.acl.ControlledEntity.ACLType; +import org.apache.cloudstack.acl.SecurityChecker; +import org.apache.cloudstack.acl.SecurityChecker.AccessType; +import org.apache.cloudstack.api.command.admin.network.DedicateGuestVlanRangeCmd; +import org.apache.cloudstack.api.command.admin.network.ListDedicatedGuestVlanRangesCmd; +import org.apache.cloudstack.api.command.admin.usage.ListTrafficTypeImplementorsCmd; +import org.apache.cloudstack.api.command.user.network.CreateNetworkCmd; +import org.apache.cloudstack.api.command.user.network.ListNetworksCmd; +import org.apache.cloudstack.api.command.user.network.RestartNetworkCmd; +import org.apache.cloudstack.api.command.user.vm.ListNicsCmd; +import org.apache.log4j.Logger; +import org.springframework.stereotype.Component; + +import javax.ejb.Local; +import javax.inject.Inject; +import javax.naming.ConfigurationException; +import java.net.Inet6Address; +import java.net.InetAddress; +import java.net.UnknownHostException; +import java.security.InvalidParameterException; +import java.sql.PreparedStatement; +import java.sql.ResultSet; +import java.sql.SQLException; +import java.util.*; + /** * NetworkServiceImpl implements NetworkService. @@ -267,6 +295,8 @@ public class NetworkServiceImpl extends ManagerBase implements NetworkService { HostDao _hostDao; @Inject HostPodDao _hostPodDao; + @Inject + InternalLoadBalancerElementService _internalLbElementSvc; @Inject DataCenterVnetDao _datacneter_vnet; @Inject @@ -1187,6 +1217,10 @@ public class NetworkServiceImpl extends ManagerBase implements NetworkService { if (_configMgr.isOfferingForVpc(ntwkOff)){ throw new InvalidParameterValueException("Network offering can be used for VPC networks only"); } + if (ntwkOff.getInternalLb()) { + throw new InvalidParameterValueException("Internal Lb can be enabled on vpc networks only"); + } + network = _networkMgr.createGuestNetwork(networkOfferingId, name, displayText, gateway, cidr, vlanId, networkDomain, owner, sharedDomainId, pNtwk, zoneId, aclType, subdomainAccess, vpcId, ip6Gateway, ip6Cidr); } @@ -2134,8 +2168,6 @@ public class NetworkServiceImpl extends ManagerBase implements NetworkService { } - - protected Set getAvailableIps(Network network, String requestedIp) { String[] cidr = network.getCidr().split("/"); List ips = _nicDao.listIpAddressInNetwork(network.getId()); @@ -2159,7 +2191,6 @@ public class NetworkServiceImpl extends ManagerBase implements NetworkService { } - protected boolean canUpgrade(Network network, long oldNetworkOfferingId, long newNetworkOfferingId) { NetworkOffering oldNetworkOffering = _networkOfferingDao.findByIdIncludingRemoved(oldNetworkOfferingId); NetworkOffering newNetworkOffering = _networkOfferingDao.findById(newNetworkOfferingId); @@ -2225,6 +2256,14 @@ public class NetworkServiceImpl extends ManagerBase implements NetworkService { return false; } } + + //can't update from internal LB to public LB + if (areServicesSupportedByNetworkOffering(oldNetworkOfferingId, Service.Lb) && areServicesSupportedByNetworkOffering(newNetworkOfferingId, Service.Lb)) { + if (oldNetworkOffering.getPublicLb() != newNetworkOffering.getPublicLb() || oldNetworkOffering.getInternalLb() != newNetworkOffering.getInternalLb()) { + throw new InvalidParameterValueException("Original and new offerings support different types of LB - Internal vs Public," + + " can't upgrade"); + } + } return canIpsUseOffering(publicIps, newNetworkOfferingId); } @@ -2345,7 +2384,10 @@ public class NetworkServiceImpl extends ManagerBase implements NetworkService { // add baremetal as the defualt network service provider /* addDefaultBaremetalProvidersToPhysicalNetwork(pNetwork.getId()); */ - + + //Add Internal Load Balancer element as a default network service provider + addDefaultInternalLbProviderToPhysicalNetwork(pNetwork.getId()); + txn.commit(); return pNetwork; } catch (Exception ex) { @@ -3564,6 +3606,22 @@ public class NetworkServiceImpl extends ManagerBase implements NetworkService { return nsp; } + + + protected PhysicalNetworkServiceProvider addDefaultInternalLbProviderToPhysicalNetwork(long physicalNetworkId) { + + PhysicalNetworkServiceProvider nsp = addProviderToPhysicalNetwork(physicalNetworkId, + Network.Provider.InternalLbVm.getName(), null, null); + + NetworkElement networkElement = _networkModel.getElementImplementingProvider(Network.Provider.InternalLbVm.getName()); + if (networkElement == null) { + throw new CloudRuntimeException("Unable to find the Network Element implementing the " + Network.Provider.InternalLbVm.getName() + " Provider"); + } + + _internalLbElementSvc.addInternalLoadBalancerElement(nsp.getId()); + + return nsp; + } protected PhysicalNetworkServiceProvider addDefaultSecurityGroupProviderToPhysicalNetwork(long physicalNetworkId) { @@ -3572,6 +3630,8 @@ public class NetworkServiceImpl extends ManagerBase implements NetworkService { return nsp; } + + private PhysicalNetworkServiceProvider addDefaultBaremetalProvidersToPhysicalNetwork(long physicalNetworkId) { PhysicalNetworkVO pvo = _physicalNetworkDao.findById(physicalNetworkId); diff --git a/server/src/com/cloud/network/element/VirtualRouterElement.java b/server/src/com/cloud/network/element/VirtualRouterElement.java index f601f4fa2e4..28473cc7bc2 100755 --- a/server/src/com/cloud/network/element/VirtualRouterElement.java +++ b/server/src/com/cloud/network/element/VirtualRouterElement.java @@ -25,7 +25,6 @@ import java.util.Set; import javax.ejb.Local; import javax.inject.Inject; -import com.cloud.utils.PropertiesUtil; import org.apache.cloudstack.api.command.admin.router.ConfigureVirtualRouterElementCmd; import org.apache.cloudstack.api.command.admin.router.CreateVirtualRouterElementCmd; import org.apache.cloudstack.api.command.admin.router.ListVirtualRouterElementsCmd; @@ -66,6 +65,7 @@ import com.cloud.network.router.VpcVirtualNetworkApplianceManager; import com.cloud.network.rules.FirewallRule; import com.cloud.network.rules.LbStickinessMethod; import com.cloud.network.rules.LbStickinessMethod.StickinessMethodType; +import com.cloud.network.rules.LoadBalancerContainer; import com.cloud.network.rules.PortForwardingRule; import com.cloud.network.rules.RulesManager; import com.cloud.network.rules.StaticNat; @@ -242,7 +242,7 @@ public class VirtualRouterElement extends AdapterBase implements VirtualRouterEl * number like 12 2) time or tablesize like 12h, 34m, 45k, 54m , here * last character is non-digit but from known characters . */ - private boolean containsOnlyNumbers(String str, String endChar) { + private static boolean containsOnlyNumbers(String str, String endChar) { if (str == null) return false; @@ -271,7 +271,7 @@ public class VirtualRouterElement extends AdapterBase implements VirtualRouterEl return true; } - private boolean validateHAProxyLBRule(LoadBalancingRule rule) { + public static boolean validateHAProxyLBRule(LoadBalancingRule rule) { String timeEndChar = "dhms"; for (LbStickinessPolicy stickinessPolicy : rule.getStickinessPolicies()) { @@ -338,7 +338,9 @@ public class VirtualRouterElement extends AdapterBase implements VirtualRouterEl @Override public boolean validateLBRule(Network network, LoadBalancingRule rule) { - if (canHandle(network, Service.Lb)) { + List rules = new ArrayList(); + rules.add(rule); + if (canHandle(network, Service.Lb) && canHandleLbRules(rules)) { List routers = _routerDao.listByNetworkAndRole(network.getId(), Role.VIRTUAL_ROUTER); if (routers == null || routers.isEmpty()) { return true; @@ -351,6 +353,10 @@ public class VirtualRouterElement extends AdapterBase implements VirtualRouterEl @Override public boolean applyLBRules(Network network, List rules) throws ResourceUnavailableException { if (canHandle(network, Service.Lb)) { + if (!canHandleLbRules(rules)) { + return false; + } + List routers = _routerDao.listByNetworkAndRole(network.getId(), Role.VIRTUAL_ROUTER); if (routers == null || routers.isEmpty()) { s_logger.debug("Virtual router elemnt doesn't need to apply firewall rules on the backend; virtual " + @@ -358,8 +364,8 @@ public class VirtualRouterElement extends AdapterBase implements VirtualRouterEl return true; } - if (!_routerMgr.applyFirewallRules(network, rules, routers)) { - throw new CloudRuntimeException("Failed to apply firewall rules in network " + network.getId()); + if (!_routerMgr.applyLoadBalancingRules(network, rules, routers)) { + throw new CloudRuntimeException("Failed to apply load balancing rules in network " + network.getId()); } else { return true; } @@ -452,7 +458,7 @@ public class VirtualRouterElement extends AdapterBase implements VirtualRouterEl return capabilities; } - private static String getHAProxyStickinessCapability() { + public static String getHAProxyStickinessCapability() { LbStickinessMethod method; List methodList = new ArrayList(1); @@ -557,8 +563,8 @@ public class VirtualRouterElement extends AdapterBase implements VirtualRouterEl lbCapabilities.put(Capability.SupportedLBAlgorithms, "roundrobin,leastconn,source"); lbCapabilities.put(Capability.SupportedLBIsolation, "dedicated"); lbCapabilities.put(Capability.SupportedProtocols, "tcp, udp"); - lbCapabilities.put(Capability.SupportedStickinessMethods, getHAProxyStickinessCapability()); + lbCapabilities.put(Capability.LbSchemes, LoadBalancerContainer.Scheme.Public.toString()); capabilities.put(Service.Lb, lbCapabilities); @@ -715,8 +721,8 @@ public class VirtualRouterElement extends AdapterBase implements VirtualRouterEl @Override public VirtualRouterProvider configure(ConfigureVirtualRouterElementCmd cmd) { VirtualRouterProviderVO element = _vrProviderDao.findById(cmd.getId()); - if (element == null) { - s_logger.debug("Can't find element with network service provider id " + cmd.getId()); + if (element == null || !(element.getType() == VirtualRouterProviderType.VirtualRouter || element.getType() == VirtualRouterProviderType.VPCVirtualRouter)) { + s_logger.debug("Can't find Virtual Router element with network service provider id " + cmd.getId()); return null; } @@ -728,6 +734,10 @@ public class VirtualRouterElement extends AdapterBase implements VirtualRouterEl @Override public VirtualRouterProvider addElement(Long nspId, VirtualRouterProviderType providerType) { + if (!(providerType == VirtualRouterProviderType.VirtualRouter || providerType == VirtualRouterProviderType.VPCVirtualRouter)) { + throw new InvalidParameterValueException("Element " + this.getName() + " supports only providerTypes: " + + VirtualRouterProviderType.VirtualRouter.toString() + " and " + VirtualRouterProviderType.VPCVirtualRouter); + } VirtualRouterProviderVO element = _vrProviderDao.findByNspIdAndType(nspId, providerType); if (element != null) { s_logger.debug("There is already a virtual router element with service provider id " + nspId); @@ -801,7 +811,11 @@ public class VirtualRouterElement extends AdapterBase implements VirtualRouterEl @Override public VirtualRouterProvider getCreatedElement(long id) { - return _vrProviderDao.findById(id); + VirtualRouterProvider provider = _vrProviderDao.findById(id); + if (!(provider.getType() == VirtualRouterProviderType.VirtualRouter || provider.getType() == VirtualRouterProviderType.VPCVirtualRouter)) { + throw new InvalidParameterValueException("Unable to find provider by id"); + } + return provider; } @Override @@ -911,6 +925,10 @@ public class VirtualRouterElement extends AdapterBase implements VirtualRouterEl if (enabled != null) { sc.addAnd(sc.getEntity().isEnabled(), Op.EQ, enabled); } + + //return only VR and VPC VR + sc.addAnd(sc.getEntity().getType(), Op.IN, VirtualRouterProvider.VirtualRouterProviderType.VPCVirtualRouter, VirtualRouterProvider.VirtualRouterProviderType.VirtualRouter); + return sc.list(); } @@ -946,4 +964,20 @@ public class VirtualRouterElement extends AdapterBase implements VirtualRouterEl // TODO Auto-generated method stub return null; } + + private boolean canHandleLbRules(List rules) { + Map lbCaps = this.getCapabilities().get(Service.Lb); + if (!lbCaps.isEmpty()) { + String schemeCaps = lbCaps.get(Capability.LbSchemes); + if (schemeCaps != null) { + for (LoadBalancingRule rule : rules) { + if (!schemeCaps.contains(rule.getScheme().toString())) { + s_logger.debug("Scheme " + rules.get(0).getScheme() + " is not supported by the provider " + this.getName()); + return false; + } + } + } + } + return true; + } } diff --git a/server/src/com/cloud/network/firewall/FirewallManagerImpl.java b/server/src/com/cloud/network/firewall/FirewallManagerImpl.java index 4ad8868b86a..def4c1ed06f 100644 --- a/server/src/com/cloud/network/firewall/FirewallManagerImpl.java +++ b/server/src/com/cloud/network/firewall/FirewallManagerImpl.java @@ -27,17 +27,12 @@ import javax.ejb.Local; import javax.inject.Inject; import javax.naming.ConfigurationException; +import org.apache.cloudstack.api.command.user.firewall.ListEgressFirewallRulesCmd; import com.cloud.network.dao.*; import org.apache.cloudstack.api.command.user.firewall.ListFirewallRulesCmd; import org.apache.log4j.Logger; import org.springframework.stereotype.Component; -import com.mysql.jdbc.ConnectionPropertiesImpl; -import org.apache.log4j.Logger; - -import org.apache.cloudstack.api.BaseListCmd; -import org.apache.cloudstack.api.command.user.firewall.ListEgressFirewallRulesCmd; -import org.apache.cloudstack.api.command.user.firewall.ListFirewallRulesCmd; import com.cloud.configuration.Config; import com.cloud.configuration.dao.ConfigurationDao; import com.cloud.domain.dao.DomainDao; @@ -53,7 +48,6 @@ import com.cloud.network.IpAddress; import com.cloud.network.Network; import com.cloud.network.Network.Capability; import com.cloud.network.Network.Service; -import com.cloud.network.Networks.TrafficType; import com.cloud.network.NetworkManager; import com.cloud.network.NetworkModel; import com.cloud.network.NetworkRuleApplier; @@ -61,10 +55,15 @@ import com.cloud.network.element.FirewallServiceProvider; import com.cloud.network.element.NetworkACLServiceProvider; import com.cloud.network.element.PortForwardingServiceProvider; import com.cloud.network.element.StaticNatServiceProvider; -import com.cloud.network.rules.*; +import com.cloud.network.rules.FirewallManager; +import com.cloud.network.rules.FirewallRule; import com.cloud.network.rules.FirewallRule.FirewallRuleType; import com.cloud.network.rules.FirewallRule.Purpose; import com.cloud.network.rules.FirewallRule.State; +import com.cloud.network.rules.FirewallRuleVO; +import com.cloud.network.rules.PortForwardingRule; +import com.cloud.network.rules.PortForwardingRuleVO; +import com.cloud.network.rules.StaticNat; import com.cloud.network.rules.dao.PortForwardingRulesDao; import com.cloud.network.vpc.VpcManager; import com.cloud.projects.Project.ListProjectResourcesCriteria; @@ -83,8 +82,8 @@ import com.cloud.utils.db.Filter; import com.cloud.utils.db.JoinBuilder; import com.cloud.utils.db.SearchBuilder; import com.cloud.utils.db.SearchCriteria; -import com.cloud.utils.db.*; import com.cloud.utils.db.SearchCriteria.Op; +import com.cloud.utils.db.Transaction; import com.cloud.utils.exception.CloudRuntimeException; import com.cloud.utils.net.NetUtils; import com.cloud.vm.UserVmVO; @@ -438,22 +437,28 @@ public class FirewallManagerImpl extends ManagerBase implements FirewallService, return; } - if (ipAddress!=null){ - if (ipAddress.getAssociatedWithNetworkId() == null) { - throw new InvalidParameterValueException("Unable to create firewall rule ; ip with specified id is not associated with any network"); - } else { - networkId = ipAddress.getAssociatedWithNetworkId(); - } - + if (ipAddress != null){ + if (ipAddress.getAssociatedWithNetworkId() == null) { + throw new InvalidParameterValueException("Unable to create firewall rule ; ip with specified id is not associated with any network"); + } else { + networkId = ipAddress.getAssociatedWithNetworkId(); + } + // Validate ip address _accountMgr.checkAccess(caller, null, true, ipAddress); - + } + + //network id either has to be passed explicitly, or implicitly as a part of ipAddress object + if (networkId == null) { + throw new InvalidParameterValueException("Unable to retrieve network id to validate the rule"); + } + Network network = _networkModel.getNetwork(networkId); - assert network != null : "Can't create port forwarding rule as network associated with public ip address is null?"; + assert network != null : "Can't create rule as network associated with public ip address is null?"; - if (trafficType == FirewallRule.TrafficType.Egress) { - _accountMgr.checkAccess(caller, null, true, network); - } + if (trafficType == FirewallRule.TrafficType.Egress) { + _accountMgr.checkAccess(caller, null, true, network); + } // Verify that the network guru supports the protocol specified Map caps = null; @@ -464,32 +469,32 @@ public class FirewallManagerImpl extends ManagerBase implements FirewallService, } } else if (purpose == Purpose.PortForwarding) { caps = _networkModel.getNetworkServiceCapabilities(network.getId(), Service.PortForwarding); - }else if (purpose == Purpose.Firewall){ - caps = _networkModel.getNetworkServiceCapabilities(network.getId(),Service.Firewall); + } else if (purpose == Purpose.Firewall){ + caps = _networkModel.getNetworkServiceCapabilities(network.getId(),Service.Firewall); } if (caps != null) { - String supportedProtocols; - String supportedTrafficTypes = null; - if (purpose == FirewallRule.Purpose.Firewall) { - supportedTrafficTypes = caps.get(Capability.SupportedTrafficDirection).toLowerCase(); - } + String supportedProtocols; + String supportedTrafficTypes = null; + if (purpose == FirewallRule.Purpose.Firewall) { + supportedTrafficTypes = caps.get(Capability.SupportedTrafficDirection).toLowerCase(); + } - if (purpose == FirewallRule.Purpose.Firewall && trafficType == FirewallRule.TrafficType.Egress) { - supportedProtocols = caps.get(Capability.SupportedEgressProtocols).toLowerCase(); - } else { - supportedProtocols = caps.get(Capability.SupportedProtocols).toLowerCase(); - } + if (purpose == FirewallRule.Purpose.Firewall && trafficType == FirewallRule.TrafficType.Egress) { + supportedProtocols = caps.get(Capability.SupportedEgressProtocols).toLowerCase(); + } else { + supportedProtocols = caps.get(Capability.SupportedProtocols).toLowerCase(); + } if (!supportedProtocols.contains(proto.toLowerCase())) { throw new InvalidParameterValueException("Protocol " + proto + " is not supported in zone " + network.getDataCenterId()); } else if (proto.equalsIgnoreCase(NetUtils.ICMP_PROTO) && purpose != Purpose.Firewall) { throw new InvalidParameterValueException("Protocol " + proto + " is currently supported only for rules with purpose " + Purpose.Firewall); - } else if (purpose == Purpose.Firewall && !supportedTrafficTypes.contains(trafficType.toString().toLowerCase())) { - throw new InvalidParameterValueException("Traffic Type " + trafficType + " is currently supported by Firewall in network " + networkId); - } + } else if (purpose == Purpose.Firewall && !supportedTrafficTypes.contains(trafficType.toString().toLowerCase())) { + throw new InvalidParameterValueException("Traffic Type " + trafficType + " is currently supported by Firewall in network " + networkId); } } + } @Override diff --git a/server/src/com/cloud/network/guru/GuestNetworkGuru.java b/server/src/com/cloud/network/guru/GuestNetworkGuru.java index 291e3ccbc77..32ce744979b 100755 --- a/server/src/com/cloud/network/guru/GuestNetworkGuru.java +++ b/server/src/com/cloud/network/guru/GuestNetworkGuru.java @@ -223,48 +223,7 @@ public abstract class GuestNetworkGuru extends AdapterBase implements NetworkGur nic.deallocate(); } } - - public Ip4Address acquireIp4Address(Network network, Ip4Address requestedIp, String reservationId) { - List ips = _nicDao.listIpAddressInNetwork(network.getId()); - String[] cidr = network.getCidr().split("/"); - SortedSet usedIps = new TreeSet(); - - if (requestedIp != null && requestedIp.equals(network.getGateway())) { - s_logger.warn("Requested ip address " + requestedIp + " is used as a gateway address in network " + network); - return null; - } - - for (String ip : ips) { - usedIps.add(NetUtils.ip2Long(ip)); - } - - if (network.getGateway() != null) { - usedIps.add(NetUtils.ip2Long(network.getGateway())); - } - - if (requestedIp != null) { - if (usedIps.contains(requestedIp.toLong())) { - s_logger.warn("Requested ip address " + requestedIp + " is already in used in " + network); - return null; - } - //check that requested ip has the same cidr - boolean isSameCidr = NetUtils.sameSubnetCIDR(requestedIp.ip4(), cidr[0], Integer.parseInt(cidr[1])); - if (!isSameCidr) { - s_logger.warn("Requested ip address " + requestedIp + " doesn't belong to the network " + network + " cidr"); - return null; - } - - return requestedIp; - } - - long ip = NetUtils.getRandomIpFromCidr(cidr[0], Integer.parseInt(cidr[1]), usedIps); - if (ip == -1) { - s_logger.warn("Unable to allocate any more ip address in " + network); - return null; - } - - return new Ip4Address(ip); - } + public int getVlanOffset(long physicalNetworkId, int vlanTag) { PhysicalNetworkVO pNetwork = _physicalNetworkDao.findById(physicalNetworkId); diff --git a/server/src/com/cloud/network/lb/LBHealthCheckManager.java b/server/src/com/cloud/network/lb/LBHealthCheckManager.java index 2e24965aa35..a9969eb7ce1 100644 --- a/server/src/com/cloud/network/lb/LBHealthCheckManager.java +++ b/server/src/com/cloud/network/lb/LBHealthCheckManager.java @@ -16,9 +16,11 @@ // under the License. package com.cloud.network.lb; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; + public interface LBHealthCheckManager { - void updateLBHealthCheck(); + void updateLBHealthCheck(Scheme scheme); } diff --git a/server/src/com/cloud/network/lb/LBHealthCheckManagerImpl.java b/server/src/com/cloud/network/lb/LBHealthCheckManagerImpl.java index 90547328714..62b738bb498 100644 --- a/server/src/com/cloud/network/lb/LBHealthCheckManagerImpl.java +++ b/server/src/com/cloud/network/lb/LBHealthCheckManagerImpl.java @@ -19,7 +19,6 @@ package com.cloud.network.lb; import static java.lang.String.format; import java.util.Map; - import java.util.concurrent.Executors; import java.util.concurrent.ScheduledExecutorService; import java.util.concurrent.TimeUnit; @@ -34,6 +33,7 @@ import org.springframework.stereotype.Component; import com.cloud.configuration.Config; import com.cloud.configuration.dao.ConfigurationDao; import com.cloud.exception.ResourceUnavailableException; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; import com.cloud.utils.NumbersUtil; import com.cloud.utils.component.Manager; import com.cloud.utils.component.ManagerBase; @@ -90,7 +90,8 @@ public class LBHealthCheckManagerImpl extends ManagerBase implements LBHealthChe @Override public void run() { try { - updateLBHealthCheck(); + updateLBHealthCheck(Scheme.Public); + updateLBHealthCheck(Scheme.Internal); } catch (Exception e) { s_logger.error("Exception in LB HealthCheck Update Checker", e); } @@ -98,9 +99,9 @@ public class LBHealthCheckManagerImpl extends ManagerBase implements LBHealthChe } @Override - public void updateLBHealthCheck() { + public void updateLBHealthCheck(Scheme scheme) { try { - _lbService.updateLBHealthChecks(); + _lbService.updateLBHealthChecks(scheme); } catch (ResourceUnavailableException e) { s_logger.debug("Error while updating the LB HealtCheck ", e); } diff --git a/server/src/com/cloud/network/lb/LoadBalancingRulesManager.java b/server/src/com/cloud/network/lb/LoadBalancingRulesManager.java index d98872a0906..a23d96f8aea 100644 --- a/server/src/com/cloud/network/lb/LoadBalancingRulesManager.java +++ b/server/src/com/cloud/network/lb/LoadBalancingRulesManager.java @@ -16,23 +16,24 @@ // under the License. package com.cloud.network.lb; +import java.util.List; + import com.cloud.exception.NetworkRuleConflictException; import com.cloud.exception.ResourceUnavailableException; -import com.cloud.network.Network; import com.cloud.network.lb.LoadBalancingRule.LbDestination; import com.cloud.network.lb.LoadBalancingRule.LbHealthCheckPolicy; import com.cloud.network.lb.LoadBalancingRule.LbStickinessPolicy; -import com.cloud.network.rules.FirewallRule; import com.cloud.network.rules.LbStickinessMethod; import com.cloud.network.rules.LoadBalancer; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; import com.cloud.user.Account; -import org.apache.cloudstack.api.command.user.loadbalancer.CreateLoadBalancerRuleCmd; - -import java.util.List; +import com.cloud.user.UserContext; public interface LoadBalancingRulesManager extends LoadBalancingRulesService { - LoadBalancer createLoadBalancer(CreateLoadBalancerRuleCmd lb, boolean openFirewall) throws NetworkRuleConflictException; + LoadBalancer createPublicLoadBalancer(String xId, String name, String description, + int srcPort, int destPort, long sourceIpId, String protocol, String algorithm, boolean openFirewall, UserContext caller) + throws NetworkRuleConflictException; boolean removeAllLoadBalanacersForIp(long ipId, Account caller, long callerUserId); boolean removeAllLoadBalanacersForNetwork(long networkId, Account caller, long callerUserId); @@ -47,9 +48,14 @@ public interface LoadBalancingRulesManager extends LoadBalancingRulesService { * @return true if removal is successful */ boolean removeVmFromLoadBalancers(long vmId); - boolean applyRules(Network network, FirewallRule.Purpose purpose, List rules) throws ResourceUnavailableException ; - boolean applyLoadBalancersForNetwork(long networkId) throws ResourceUnavailableException; + boolean applyLoadBalancersForNetwork(long networkId, Scheme scheme) throws ResourceUnavailableException; String getLBCapability(long networkid, String capabilityName); boolean configureLbAutoScaleVmGroup(long vmGroupid, String currentState) throws ResourceUnavailableException; - boolean revokeLoadBalancersForNetwork(long networkId) throws ResourceUnavailableException; + boolean revokeLoadBalancersForNetwork(long networkId, Scheme scheme) throws ResourceUnavailableException; + + boolean validateLbRule(LoadBalancingRule lbRule); + + void removeLBRule(LoadBalancer rule); + + void isLbServiceSupportedInNetwork(long networkId, Scheme scheme); } diff --git a/server/src/com/cloud/network/lb/LoadBalancingRulesManagerImpl.java b/server/src/com/cloud/network/lb/LoadBalancingRulesManagerImpl.java index 7ad1070e1c7..520dd763667 100755 --- a/server/src/com/cloud/network/lb/LoadBalancingRulesManagerImpl.java +++ b/server/src/com/cloud/network/lb/LoadBalancingRulesManagerImpl.java @@ -16,6 +16,34 @@ // under the License. package com.cloud.network.lb; +import java.security.InvalidParameterException; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collection; +import java.util.HashMap; +import java.util.HashSet; +import java.util.Iterator; +import java.util.List; +import java.util.Map; +import java.util.Set; + +import javax.ejb.Local; +import javax.inject.Inject; + +import org.apache.cloudstack.api.ApiConstants; +import org.apache.cloudstack.api.command.user.loadbalancer.CreateLBHealthCheckPolicyCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.CreateLBStickinessPolicyCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.ListLBHealthCheckPoliciesCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.ListLBStickinessPoliciesCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.ListLoadBalancerRuleInstancesCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.ListLoadBalancerRulesCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.UpdateLoadBalancerRuleCmd; +import org.apache.cloudstack.api.response.ServiceResponse; +import org.apache.cloudstack.lb.ApplicationLoadBalancerRuleVO; +import org.apache.cloudstack.lb.dao.ApplicationLoadBalancerRuleDao; +import org.apache.log4j.Logger; +import org.springframework.stereotype.Component; + import com.cloud.agent.api.to.LoadBalancerTO; import com.cloud.configuration.Config; import com.cloud.configuration.ConfigurationManager; @@ -30,21 +58,70 @@ import com.cloud.event.EventTypes; import com.cloud.event.UsageEventUtils; import com.cloud.event.dao.EventDao; import com.cloud.event.dao.UsageEventDao; -import com.cloud.exception.*; -import com.cloud.network.*; +import com.cloud.exception.InsufficientAddressCapacityException; +import com.cloud.exception.InvalidParameterValueException; +import com.cloud.exception.NetworkRuleConflictException; +import com.cloud.exception.PermissionDeniedException; +import com.cloud.exception.ResourceUnavailableException; +import com.cloud.network.ExternalLoadBalancerUsageManager; +import com.cloud.network.IpAddress; +import com.cloud.network.LBHealthCheckPolicyVO; +import com.cloud.network.Network; import com.cloud.network.Network.Capability; import com.cloud.network.Network.Provider; import com.cloud.network.Network.Service; -import com.cloud.network.as.*; +import com.cloud.network.NetworkManager; +import com.cloud.network.NetworkModel; +import com.cloud.network.addr.PublicIp; +import com.cloud.network.as.AutoScalePolicy; +import com.cloud.network.as.AutoScalePolicyConditionMapVO; +import com.cloud.network.as.AutoScaleVmGroup; +import com.cloud.network.as.AutoScaleVmGroupPolicyMapVO; +import com.cloud.network.as.AutoScaleVmGroupVO; +import com.cloud.network.as.AutoScaleVmProfile; import com.cloud.network.as.Condition; -import com.cloud.network.as.dao.*; -import com.cloud.network.dao.*; +import com.cloud.network.as.Counter; +import com.cloud.network.as.dao.AutoScalePolicyConditionMapDao; +import com.cloud.network.as.dao.AutoScalePolicyDao; +import com.cloud.network.as.dao.AutoScaleVmGroupDao; +import com.cloud.network.as.dao.AutoScaleVmGroupPolicyMapDao; +import com.cloud.network.as.dao.AutoScaleVmProfileDao; +import com.cloud.network.as.dao.ConditionDao; +import com.cloud.network.as.dao.CounterDao; +import com.cloud.network.dao.FirewallRulesCidrsDao; +import com.cloud.network.dao.FirewallRulesDao; +import com.cloud.network.dao.IPAddressDao; +import com.cloud.network.dao.IPAddressVO; +import com.cloud.network.dao.LBHealthCheckPolicyDao; +import com.cloud.network.dao.LBStickinessPolicyDao; +import com.cloud.network.dao.LBStickinessPolicyVO; +import com.cloud.network.dao.LoadBalancerDao; +import com.cloud.network.dao.LoadBalancerVMMapDao; +import com.cloud.network.dao.LoadBalancerVMMapVO; +import com.cloud.network.dao.LoadBalancerVO; +import com.cloud.network.dao.NetworkDao; +import com.cloud.network.dao.NetworkServiceMapDao; +import com.cloud.network.dao.NetworkVO; import com.cloud.network.element.LoadBalancingServiceProvider; -import com.cloud.network.lb.LoadBalancingRule.*; -import com.cloud.network.rules.*; +import com.cloud.network.lb.LoadBalancingRule.LbAutoScalePolicy; +import com.cloud.network.lb.LoadBalancingRule.LbAutoScaleVmGroup; +import com.cloud.network.lb.LoadBalancingRule.LbAutoScaleVmProfile; +import com.cloud.network.lb.LoadBalancingRule.LbCondition; +import com.cloud.network.lb.LoadBalancingRule.LbDestination; +import com.cloud.network.lb.LoadBalancingRule.LbHealthCheckPolicy; +import com.cloud.network.lb.LoadBalancingRule.LbStickinessPolicy; +import com.cloud.network.rules.FirewallManager; +import com.cloud.network.rules.FirewallRule; import com.cloud.network.rules.FirewallRule.FirewallRuleType; import com.cloud.network.rules.FirewallRule.Purpose; +import com.cloud.network.rules.FirewallRuleVO; +import com.cloud.network.rules.HealthCheckPolicy; +import com.cloud.network.rules.LbStickinessMethod; import com.cloud.network.rules.LbStickinessMethod.LbStickinessMethodParam; +import com.cloud.network.rules.LoadBalancer; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; +import com.cloud.network.rules.RulesManager; +import com.cloud.network.rules.StickinessPolicy; import com.cloud.network.vpc.VpcManager; import com.cloud.offering.NetworkOffering; import com.cloud.projects.Project.ListProjectResourcesCriteria; @@ -53,15 +130,25 @@ import com.cloud.service.dao.ServiceOfferingDao; import com.cloud.storage.dao.VMTemplateDao; import com.cloud.tags.ResourceTagVO; import com.cloud.tags.dao.ResourceTagDao; -import com.cloud.user.*; +import com.cloud.user.Account; +import com.cloud.user.AccountManager; +import com.cloud.user.DomainService; +import com.cloud.user.User; +import com.cloud.user.UserContext; import com.cloud.user.dao.AccountDao; import com.cloud.user.dao.UserDao; import com.cloud.uservm.UserVm; import com.cloud.utils.Pair; import com.cloud.utils.Ternary; import com.cloud.utils.component.ManagerBase; -import com.cloud.utils.db.*; +import com.cloud.utils.db.DB; +import com.cloud.utils.db.Filter; +import com.cloud.utils.db.JoinBuilder; +import com.cloud.utils.db.SearchBuilder; +import com.cloud.utils.db.SearchCriteria; +import com.cloud.utils.db.Transaction; import com.cloud.utils.exception.CloudRuntimeException; +import com.cloud.utils.net.Ip; import com.cloud.utils.net.NetUtils; import com.cloud.vm.Nic; import com.cloud.vm.UserVmVO; @@ -70,21 +157,11 @@ import com.cloud.vm.dao.NicDao; import com.cloud.vm.dao.UserVmDao; import com.google.gson.Gson; import com.google.gson.reflect.TypeToken; -import org.apache.cloudstack.api.ApiConstants; -import org.apache.cloudstack.api.command.user.loadbalancer.*; -import org.apache.cloudstack.api.response.ServiceResponse; -import org.apache.log4j.Logger; -import org.springframework.stereotype.Component; - -import javax.ejb.Local; -import javax.inject.Inject; -import java.security.InvalidParameterException; -import java.util.*; @Component @Local(value = { LoadBalancingRulesManager.class, LoadBalancingRulesService.class }) public class LoadBalancingRulesManagerImpl extends ManagerBase implements LoadBalancingRulesManager, - LoadBalancingRulesService, NetworkRuleApplier { + LoadBalancingRulesService { private static final Logger s_logger = Logger.getLogger(LoadBalancingRulesManagerImpl.class); @Inject @@ -166,6 +243,7 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements UserDao _userDao; @Inject List _lbProviders; + @Inject ApplicationLoadBalancerRuleDao _appLbRuleDao; // Will return a string. For LB Stickiness this will be a json, for // autoscale this will be "," separated values @@ -261,8 +339,9 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements * Regular config like destinations need not be packed for applying * autoscale config as of today. */ - List policyList = getStickinessPolicies(lb.getId()); - LoadBalancingRule rule = new LoadBalancingRule(lb, null, policyList, null); + List policyList = getStickinessPolicies(lb.getId()); + Ip sourceIp = getSourceIp(lb); + LoadBalancingRule rule = new LoadBalancingRule(lb, null, policyList, null, sourceIp); rule.setAutoScaleVmGroup(lbAutoScaleVmGroup); if (!isRollBackAllowedForProvider(lb)) { @@ -273,7 +352,7 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements List rules = Arrays.asList(rule); - if (!_networkMgr.applyRules(rules, FirewallRule.Purpose.LoadBalancing, this, false)) { + if (!applyLbRules(rules, false)) { s_logger.debug("LB rules' autoscale config are not completely applied"); return false; } @@ -281,6 +360,17 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements return true; } + private Ip getSourceIp(LoadBalancer lb) { + Ip sourceIp = null; + if (lb.getScheme() == Scheme.Public) { + sourceIp = _networkModel.getPublicIpAddress(lb.getSourceIpAddressId()).getAddress(); + } else if (lb.getScheme() == Scheme.Internal) { + ApplicationLoadBalancerRuleVO appLbRule = _appLbRuleDao.findById(lb.getId()); + sourceIp = appLbRule.getSourceIp(); + } + return sourceIp; + } + @Override @DB public boolean configureLbAutoScaleVmGroup(long vmGroupid, String currentState) throws ResourceUnavailableException { @@ -454,9 +544,10 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements cmd.getStickinessMethodName(), cmd.getparamList(), cmd.getDescription()); List policyList = new ArrayList(); policyList.add(new LbStickinessPolicy(cmd.getStickinessMethodName(), lbpolicy.getParams())); + Ip sourceIp = getSourceIp(loadBalancer); LoadBalancingRule lbRule = new LoadBalancingRule(loadBalancer, getExistingDestinations(lbpolicy.getId()), - policyList, null); - if (!validateRule(lbRule)) { + policyList, null, sourceIp); + if (!validateLbRule(lbRule)) { throw new InvalidParameterValueException("Failed to create Stickiness policy: Validation Failed " + cmd.getLbRuleId()); } @@ -539,7 +630,8 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements return policy; } - private boolean validateRule(LoadBalancingRule lbRule) { + @Override + public boolean validateLbRule(LoadBalancingRule lbRule) { Network network = _networkDao.findById(lbRule.getNetworkId()); Purpose purpose = lbRule.getPurpose(); if (purpose != Purpose.LoadBalancing) { @@ -748,7 +840,7 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements // by CloudStack and update them in lbvmmap table @DB @Override - public void updateLBHealthChecks() throws ResourceUnavailableException { + public void updateLBHealthChecks(Scheme scheme) throws ResourceUnavailableException { List rules = _lbDao.listAll(); List networks = _networkDao.listAll(); List stateRules = null; @@ -763,7 +855,7 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements * "HealthCheck Manager :: LB Provider in the Network has the Healthcheck policy capability :: " * + provider.get(0).getName()); */ - rules = _lbDao.listByNetworkId(network.getId()); + rules = _lbDao.listByNetworkIdAndScheme(network.getId(), scheme); if (rules != null && rules.size() > 0) { List lbrules = new ArrayList(); for (LoadBalancerVO lb : rules) { @@ -772,7 +864,8 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements // adding to lbrules list only if the LB rule // hashealtChecks if (hcPolicyList != null && hcPolicyList.size() > 0) { - LoadBalancingRule loadBalancing = new LoadBalancingRule(lb, dstList, null, hcPolicyList); + Ip sourceIp = getSourceIp(lb); + LoadBalancingRule loadBalancing = new LoadBalancingRule(lb, dstList, null, hcPolicyList, sourceIp); lbrules.add(loadBalancing); } } @@ -1168,31 +1261,21 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements @Override @ActionEvent(eventType = EventTypes.EVENT_LOAD_BALANCER_CREATE, eventDescription = "creating load balancer") - public LoadBalancer createLoadBalancerRule(CreateLoadBalancerRuleCmd lb, boolean openFirewall) + public LoadBalancer createPublicLoadBalancerRule(String xId, String name, String description, + int srcPortStart, int srcPortEnd, int defPortStart, int defPortEnd, Long ipAddrId, String protocol, String algorithm, long networkId, long lbOwnerId, boolean openFirewall) throws NetworkRuleConflictException, InsufficientAddressCapacityException { - Account lbOwner = _accountMgr.getAccount(lb.getEntityOwnerId()); - - int defPortStart = lb.getDefaultPortStart(); - int defPortEnd = lb.getDefaultPortEnd(); - - if (!NetUtils.isValidPort(defPortEnd)) { - throw new InvalidParameterValueException("privatePort is an invalid value: " + defPortEnd); - } - if (defPortStart > defPortEnd) { - throw new InvalidParameterValueException("private port range is invalid: " + defPortStart + "-" - + defPortEnd); - } - if ((lb.getAlgorithm() == null) || !NetUtils.isValidAlgorithm(lb.getAlgorithm())) { - throw new InvalidParameterValueException("Invalid algorithm: " + lb.getAlgorithm()); + Account lbOwner = _accountMgr.getAccount(lbOwnerId); + + if (srcPortStart != srcPortEnd) { + throw new InvalidParameterValueException("Port ranges are not supported by the load balancer"); } - Long ipAddrId = lb.getSourceIpAddressId(); IPAddressVO ipVO = null; if (ipAddrId != null) { ipVO = _ipAddressDao.findById(ipAddrId); } - Network network = _networkModel.getNetwork(lb.getNetworkId()); + Network network = _networkModel.getNetwork(networkId); // FIXME: breaking the dependency on ELB manager. This breaks // functionality of ELB using virtual router @@ -1204,8 +1287,7 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements IpAddress systemIp = null; NetworkOffering off = _configMgr.getNetworkOffering(network.getNetworkOfferingId()); if (off.getElasticLb() && ipVO == null && network.getVpcId() == null) { - systemIp = _networkMgr.assignSystemIp(lb.getNetworkId(), lbOwner, true, false); - lb.setSourceIpAddressId(systemIp.getId()); + systemIp = _networkMgr.assignSystemIp(networkId, lbOwner, true, false); ipVO = _ipAddressDao.findById(systemIp.getId()); } @@ -1224,11 +1306,11 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements && ipVO.getVpcId().longValue() == network.getVpcId(); if (assignToVpcNtwk) { // set networkId just for verification purposes - _networkModel.checkIpForService(ipVO, Service.Lb, lb.getNetworkId()); + _networkModel.checkIpForService(ipVO, Service.Lb, networkId); - s_logger.debug("The ip is not associated with the VPC network id=" + lb.getNetworkId() + s_logger.debug("The ip is not associated with the VPC network id=" + networkId + " so assigning"); - ipVO = _networkMgr.associateIPToGuestNetwork(ipAddrId, lb.getNetworkId(), false); + ipVO = _networkMgr.associateIPToGuestNetwork(ipAddrId, networkId, false); performedIpAssoc = true; } } else { @@ -1240,10 +1322,7 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements + network); } - if (lb.getSourceIpAddressId() == null) { - throw new CloudRuntimeException("No ip address is defined to assign the LB to"); - } - result = createLoadBalancer(lb, openFirewall); + result = createPublicLoadBalancer(xId, name, description, srcPortStart, defPortStart, ipVO.getId(), protocol, algorithm, openFirewall, UserContext.current()); } catch (Exception ex) { s_logger.warn("Failed to create load balancer due to ", ex); if (ex instanceof NetworkRuleConflictException) { @@ -1258,27 +1337,31 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements // release ip address if ipassoc was perfored if (performedIpAssoc) { ipVO = _ipAddressDao.findById(ipVO.getId()); - _vpcMgr.unassignIPFromVpcNetwork(ipVO.getId(), lb.getNetworkId()); + _vpcMgr.unassignIPFromVpcNetwork(ipVO.getId(), networkId); } } } if (result == null) { - throw new CloudRuntimeException("Failed to create load balancer rule: " + lb.getName()); + throw new CloudRuntimeException("Failed to create load balancer rule: " + name); } return result; } - @Override @DB - public LoadBalancer createLoadBalancer(CreateLoadBalancerRuleCmd lb, boolean openFirewall) + @Override + public LoadBalancer createPublicLoadBalancer(String xId, String name, String description, + int srcPort, int destPort, long sourceIpId, String protocol, String algorithm, boolean openFirewall, UserContext caller) throws NetworkRuleConflictException { - UserContext caller = UserContext.current(); - int srcPortStart = lb.getSourcePortStart(); - int defPortStart = lb.getDefaultPortStart(); - int srcPortEnd = lb.getSourcePortEnd(); - long sourceIpId = lb.getSourceIpAddressId(); + + if (!NetUtils.isValidPort(destPort)) { + throw new InvalidParameterValueException("privatePort is an invalid value: " + destPort); + } + + if ((algorithm == null) || !NetUtils.isValidAlgorithm(algorithm)) { + throw new InvalidParameterValueException("Invalid algorithm: " + algorithm); + } IPAddressVO ipAddr = _ipAddressDao.findById(sourceIpId); // make sure ip address exists @@ -1293,6 +1376,9 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements ex.addProxyObject(ipAddr, sourceIpId, "sourceIpId"); throw ex; } + + _accountMgr.checkAccess(caller.getCaller(), null, true, ipAddr); + Long networkId = ipAddr.getAssociatedWithNetworkId(); if (networkId == null) { @@ -1301,39 +1387,34 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements ex.addProxyObject(ipAddr, sourceIpId, "sourceIpId"); throw ex; } - - _firewallMgr.validateFirewallRule(caller.getCaller(), ipAddr, srcPortStart, srcPortEnd, lb.getProtocol(), - Purpose.LoadBalancing, FirewallRuleType.User, networkId, null); - NetworkVO network = _networkDao.findById(networkId); - _accountMgr.checkAccess(caller.getCaller(), null, true, ipAddr); - + // verify that lb service is supported by the network - if (!_networkModel.areServicesSupportedInNetwork(network.getId(), Service.Lb)) { - InvalidParameterValueException ex = new InvalidParameterValueException( - "LB service is not supported in specified network id"); - ex.addProxyObject(network, networkId, "networkId"); - throw ex; + isLbServiceSupportedInNetwork(networkId, Scheme.Public); + + _firewallMgr.validateFirewallRule(caller.getCaller(), ipAddr, srcPort, srcPort, protocol, + Purpose.LoadBalancing, FirewallRuleType.User, networkId, null); + + LoadBalancerVO newRule = new LoadBalancerVO(xId, name, description, + sourceIpId, srcPort, srcPort, algorithm, + networkId, ipAddr.getAllocatedToAccountId(), ipAddr.getAllocatedInDomainId()); + + // verify rule is supported by Lb provider of the network + Ip sourceIp = getSourceIp(newRule); + LoadBalancingRule loadBalancing = new LoadBalancingRule(newRule, new ArrayList(), + new ArrayList(), new ArrayList(), sourceIp); + if (!validateLbRule(loadBalancing)) { + throw new InvalidParameterValueException("LB service provider cannot support this rule"); } Transaction txn = Transaction.currentTxn(); txn.start(); - - LoadBalancerVO newRule = new LoadBalancerVO(lb.getXid(), lb.getName(), lb.getDescription(), - lb.getSourceIpAddressId(), lb.getSourcePortEnd(), lb.getDefaultPortStart(), lb.getAlgorithm(), - network.getId(), ipAddr.getAllocatedToAccountId(), ipAddr.getAllocatedInDomainId()); - - // verify rule is supported by Lb provider of the network - LoadBalancingRule loadBalancing = new LoadBalancingRule(newRule, new ArrayList(), - new ArrayList(), new ArrayList()); - if (!validateRule(loadBalancing)) { - throw new InvalidParameterValueException("LB service provider cannot support this rule"); - } - + newRule = _lbDao.persist(newRule); + //create rule for all CIDRs if (openFirewall) { - _firewallMgr.createRuleForAllCidrs(sourceIpId, caller.getCaller(), lb.getSourcePortStart(), - lb.getSourcePortEnd(), lb.getProtocol(), null, null, newRule.getId(), networkId); + _firewallMgr.createRuleForAllCidrs(sourceIpId, caller.getCaller(), srcPort, + srcPort, protocol, null, null, newRule.getId(), networkId); } boolean success = true; @@ -1344,7 +1425,7 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements throw new CloudRuntimeException("Unable to update the state to add for " + newRule); } s_logger.debug("Load balancer " + newRule.getId() + " for Ip address id=" + sourceIpId + ", public port " - + srcPortStart + ", private port " + defPortStart + " is added successfully."); + + srcPort + ", private port " + destPort + " is added successfully."); UserContext.current().setEventDetails("Load balancer Id: " + newRule.getId()); UsageEventUtils.publishUsageEvent(EventTypes.EVENT_LOAD_BALANCER_CREATE, ipAddr.getAllocatedToAccountId(), ipAddr.getDataCenterId(), newRule.getId(), null, LoadBalancingRule.class.getName(), @@ -1380,14 +1461,17 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements lbs = Arrays.asList(lb); } else { // get all rules in transition state - lbs = _lbDao.listInTransitionStateByNetworkId(lb.getNetworkId()); + lbs = _lbDao.listInTransitionStateByNetworkIdAndScheme(lb.getNetworkId(), lb.getScheme()); } return applyLoadBalancerRules(lbs, true); } @Override - public boolean revokeLoadBalancersForNetwork(long networkId) throws ResourceUnavailableException { - List lbs = _lbDao.listByNetworkId(networkId); + public boolean revokeLoadBalancersForNetwork(long networkId, Scheme scheme) throws ResourceUnavailableException { + List lbs = _lbDao.listByNetworkIdAndScheme(networkId, scheme); + if (s_logger.isDebugEnabled()) { + s_logger.debug("Revoking " + lbs.size() + " " + scheme + " load balancing rules for network id=" + networkId); + } if (lbs != null) { for(LoadBalancerVO lb : lbs) { // called during restart, not persisting state in db lb.setState(FirewallRule.State.Revoke); @@ -1400,20 +1484,20 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements } @Override - public boolean applyLoadBalancersForNetwork(long networkId) throws ResourceUnavailableException { - List lbs = _lbDao.listByNetworkId(networkId); + public boolean applyLoadBalancersForNetwork(long networkId, Scheme scheme) throws ResourceUnavailableException { + List lbs = _lbDao.listByNetworkIdAndScheme(networkId, scheme); if (lbs != null) { + s_logger.debug("Applying load balancer rules of scheme " + scheme + " in network id=" + networkId); return applyLoadBalancerRules(lbs, true); } else { - s_logger.info("Network id=" + networkId + " doesn't have load balancer rules, nothing to apply"); + s_logger.info("Network id=" + networkId + " doesn't have load balancer rules of scheme " + scheme + ", nothing to apply"); return true; } } - @Override - public boolean applyRules(Network network, Purpose purpose, List rules) + + protected boolean applyLbRules(Network network, List rules) throws ResourceUnavailableException { - assert (purpose == Purpose.LoadBalancing) : "LB Manager asked to handle non-LB rules"; boolean handled = false; for (LoadBalancingServiceProvider lbElement : _lbProviders) { Provider provider = lbElement.getProvider(); @@ -1422,7 +1506,7 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements if (!isLbProvider) { continue; } - handled = lbElement.applyLBRules(network, (List) rules); + handled = lbElement.applyLBRules(network, rules); if (handled) break; } @@ -1432,7 +1516,8 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements private LoadBalancingRule getLoadBalancerRuleToApply(LoadBalancerVO lb) { List policyList = getStickinessPolicies(lb.getId()); - LoadBalancingRule loadBalancing = new LoadBalancingRule(lb, null, policyList, null); + Ip sourceIp = getSourceIp(lb); + LoadBalancingRule loadBalancing = new LoadBalancingRule(lb, null, policyList, null, sourceIp); if (_autoScaleVmGroupDao.isAutoScaleLoadBalancer(lb.getId())) { // Get the associated VmGroup @@ -1442,7 +1527,7 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements } else { List dstList = getExistingDestinations(lb.getId()); loadBalancing.setDestinations(dstList); - List hcPolicyList = getHealthCheckPolicies(lb.getId()); + List hcPolicyList = getHealthCheckPolicies(lb.getId()); loadBalancing.setHealthCheckPolicies(hcPolicyList); } @@ -1458,7 +1543,7 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements rules.add(getLoadBalancerRuleToApply(lb)); } - if (!_networkMgr.applyRules(rules, FirewallRule.Purpose.LoadBalancing, this, false)) { + if (!applyLbRules(rules, false)) { s_logger.debug("LB rules are not completely applied"); return false; } @@ -1515,7 +1600,7 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements } txn.commit(); - if (checkForReleaseElasticIp) { + if (checkForReleaseElasticIp && lb.getSourceIpAddressId() != null) { boolean success = true; long count = _firewallDao.countRulesByIpId(lb.getSourceIpAddressId()); if (count == 0) { @@ -1534,8 +1619,10 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements } // if the rule is the last one for the ip address assigned to // VPC, unassign it from the network - IpAddress ip = _ipAddressDao.findById(lb.getSourceIpAddressId()); - _vpcMgr.unassignIPFromVpcNetwork(ip.getId(), lb.getNetworkId()); + if (lb.getSourceIpAddressId() != null) { + IpAddress ip = _ipAddressDao.findById(lb.getSourceIpAddressId()); + _vpcMgr.unassignIPFromVpcNetwork(ip.getId(), lb.getNetworkId()); + } } } @@ -1902,32 +1989,115 @@ public class LoadBalancingRulesManagerImpl extends ManagerBase implements count++; } } + + //list only Public load balancers using this command + sc.setParameters("scheme", Scheme.Public); Pair, Integer> result = _lbDao.searchAndCount(sc, searchFilter); return new Pair, Integer>(result.first(), result.second()); } - @Override - public List listByNetworkId(long networkId) { - List lbs = _lbDao.listByNetworkId(networkId); - List lbRules = new ArrayList(); - for (LoadBalancerVO lb : lbs) { - List dstList = getExistingDestinations(lb.getId()); - List policyList = this.getStickinessPolicies(lb.getId()); - List hcPolicyList = this.getHealthCheckPolicies(lb.getId()); - LoadBalancingRule loadBalancing = new LoadBalancingRule(lb, dstList, policyList, hcPolicyList); - lbRules.add(loadBalancing); - } - return lbRules; - } @Override public LoadBalancerVO findById(long lbId) { return _lbDao.findById(lbId); } - protected void removeLBRule(LoadBalancerVO rule) { + @Override + public void removeLBRule(LoadBalancer rule) { // remove the rule _lbDao.remove(rule.getId()); } + + + public boolean applyLbRules(List rules, boolean continueOnError) throws ResourceUnavailableException { + if (rules == null || rules.size() == 0) { + s_logger.debug("There are no Load Balancing Rules to forward to the network elements"); + return true; + } + + boolean success = true; + Network network = _networkModel.getNetwork(rules.get(0).getNetworkId()); + List publicIps = new ArrayList(); + + + // get the list of public ip's owned by the network + List userIps = _ipAddressDao.listByAssociatedNetwork(network.getId(), null); + if (userIps != null && !userIps.isEmpty()) { + for (IPAddressVO userIp : userIps) { + PublicIp publicIp = PublicIp.createFromAddrAndVlan(userIp, _vlanDao.findById(userIp.getVlanId())); + publicIps.add(publicIp); + } + } + + // rules can not programmed unless IP is associated with network + // service provider, so run IP assoication for + // the network so as to ensure IP is associated before applying + // rules (in add state) + _networkMgr.applyIpAssociations(network, false, continueOnError, publicIps); + + + try { + applyLbRules(network, rules); + } catch (ResourceUnavailableException e) { + if (!continueOnError) { + throw e; + } + s_logger.warn("Problems with applying load balancing rules but pushing on", e); + success = false; + } + + // if all the rules configured on public IP are revoked then + // dis-associate IP with network service provider + _networkMgr.applyIpAssociations(network, true, continueOnError, publicIps); + + return success; + } + + @Override + public Map getLbInstances(long lbId) { + Map dstList = new HashMap(); + List lbVmMaps = _lb2VmMapDao.listByLoadBalancerId(lbId); + LoadBalancerVO lb = _lbDao.findById(lbId); + + for (LoadBalancerVMMapVO lbVmMap : lbVmMaps) { + UserVm vm = _vmDao.findById(lbVmMap.getInstanceId()); + Nic nic = _nicDao.findByInstanceIdAndNetworkIdIncludingRemoved(lb.getNetworkId(), vm.getId()); + Ip ip = new Ip(nic.getIp4Address()); + dstList.put(ip, vm); + } + return dstList; + } + + @Override + public void isLbServiceSupportedInNetwork(long networkId, Scheme scheme) { + Network network = _networkDao.findById(networkId); + + //1) Check if the LB service is supported + if (!_networkModel.areServicesSupportedInNetwork(network.getId(), Service.Lb)) { + InvalidParameterValueException ex = new InvalidParameterValueException( + "LB service is not supported in specified network id"); + ex.addProxyObject(network, network.getId(), "networkId"); + throw ex; + } + + //2) Check if the Scheme is supported\ + NetworkOffering off = _configMgr.getNetworkOffering(network.getNetworkOfferingId()); + if (scheme == Scheme.Public) { + if (!off.getPublicLb()) { + throw new InvalidParameterValueException("Scheme " + scheme + " is not supported by the network offering " + off); + } + } else { + if (!off.getInternalLb()) { + throw new InvalidParameterValueException("Scheme " + scheme + " is not supported by the network offering " + off); + } + } + + //3) Check if the provider supports the scheme + LoadBalancingServiceProvider lbProvider = _networkMgr.getLoadBalancingProviderForNetwork(network, scheme); + if (lbProvider == null) { + throw new InvalidParameterValueException("Lb rule with scheme " + scheme.toString() + " is not supported by lb providers in network " + network); + } + } + } diff --git a/server/src/com/cloud/network/router/VirtualNetworkApplianceManager.java b/server/src/com/cloud/network/router/VirtualNetworkApplianceManager.java index f49ab79b500..fcf650f900c 100644 --- a/server/src/com/cloud/network/router/VirtualNetworkApplianceManager.java +++ b/server/src/com/cloud/network/router/VirtualNetworkApplianceManager.java @@ -28,6 +28,7 @@ import com.cloud.network.PublicIpAddress; import com.cloud.network.RemoteAccessVpn; import com.cloud.network.VirtualNetworkApplianceService; import com.cloud.network.VpnUser; +import com.cloud.network.lb.LoadBalancingRule; import com.cloud.network.rules.FirewallRule; import com.cloud.network.rules.StaticNat; import com.cloud.user.Account; @@ -103,4 +104,7 @@ public interface VirtualNetworkApplianceManager extends Manager, VirtualNetworkA boolean applyUserData(Network config, NicProfile nic, VirtualMachineProfile vm, DeployDestination dest, List routers) throws ResourceUnavailableException; + + boolean applyLoadBalancingRules(Network network, List rules, List routers) throws ResourceUnavailableException; + } diff --git a/server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java b/server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java index 6620e0a6379..e3dd06ba47c 100755 --- a/server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java +++ b/server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java @@ -173,6 +173,7 @@ import com.cloud.network.router.VirtualRouter.RedundantState; import com.cloud.network.router.VirtualRouter.Role; import com.cloud.network.rules.FirewallRule; import com.cloud.network.rules.FirewallRule.Purpose; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; import com.cloud.network.rules.PortForwardingRule; import com.cloud.network.rules.RulesManager; import com.cloud.network.rules.StaticNat; @@ -218,6 +219,7 @@ import com.cloud.utils.db.SearchBuilder; import com.cloud.utils.db.SearchCriteria; import com.cloud.utils.db.Transaction; import com.cloud.utils.exception.CloudRuntimeException; +import com.cloud.utils.net.Ip; import com.cloud.utils.net.MacAddress; import com.cloud.utils.net.NetUtils; import com.cloud.vm.DomainRouterVO; @@ -1526,7 +1528,7 @@ public class VirtualNetworkApplianceManagerImpl extends ManagerBase implements V for (int i = 0; i < count; i++) { List> networks = createRouterNetworks(owner, isRedundant, plan, guestNetwork, new Pair(publicNetwork, sourceNatIp)); - //don't start the router as we are holding the network lock that needs to be released at the end of router allocation + //don't start the router as we are holding the network lock that needs to be released at the end of router allocation DomainRouterVO router = deployRouter(owner, destination, plan, params, isRedundant, vrProvider, offeringId, null, networks, false, null); @@ -2410,7 +2412,7 @@ public class VirtualNetworkApplianceManagerImpl extends ManagerBase implements V } } - List lbs = _loadBalancerDao.listByNetworkId(guestNetworkId); + List lbs = _loadBalancerDao.listByNetworkIdAndScheme(guestNetworkId, Scheme.Public); List lbRules = new ArrayList(); if (_networkModel.isProviderSupportServiceInNetwork(guestNetworkId, Service.Lb, provider)) { // Re-apply load balancing rules @@ -2418,7 +2420,8 @@ public class VirtualNetworkApplianceManagerImpl extends ManagerBase implements V List dstList = _lbMgr.getExistingDestinations(lb.getId()); List policyList = _lbMgr.getStickinessPolicies(lb.getId()); List hcPolicyList = _lbMgr.getHealthCheckPolicies(lb.getId()); - LoadBalancingRule loadBalancing = new LoadBalancingRule(lb, dstList, policyList, hcPolicyList); + Ip sourceIp = _networkModel.getPublicIpAddress(lb.getSourceIpAddressId()).getAddress(); + LoadBalancingRule loadBalancing = new LoadBalancingRule(lb, dstList, policyList, hcPolicyList, sourceIp); lbRules.add(loadBalancing); } } @@ -2509,7 +2512,7 @@ public class VirtualNetworkApplianceManagerImpl extends ManagerBase implements V Network network = _networkModel.getNetwork(routerNic.getNetworkId()); if (network.getTrafficType() == TrafficType.Guest) { guestNetworks.add(network); - } + } } answer = cmds.getAnswer("getDomRVersion"); @@ -3036,7 +3039,7 @@ public class VirtualNetworkApplianceManagerImpl extends ManagerBase implements V String algorithm = rule.getAlgorithm(); String uuid = rule.getUuid(); - String srcIp = _networkModel.getIp(rule.getSourceIpAddressId()).getAddress().addr(); + String srcIp = rule.getSourceIp().addr(); int srcPort = rule.getSourcePortStart(); List destinations = rule.getDestinations(); List stickinessPolicies = rule.getStickinessPolicies(); @@ -3051,7 +3054,7 @@ public class VirtualNetworkApplianceManagerImpl extends ManagerBase implements V } Network guestNetwork = _networkModel.getNetwork(guestNetworkId); - Nic nic = _nicDao.findByInstanceIdAndNetworkId(guestNetwork.getId(), router.getId()); + Nic nic = _nicDao.findByNtwkIdAndInstanceId(guestNetwork.getId(), router.getId()); NicProfile nicProfile = new NicProfile(nic, guestNetwork, nic.getBroadcastUri(), nic.getIsolationUri(), _networkModel.getNetworkRate(guestNetwork.getId(), router.getId()), _networkModel.isSecurityGroupSupportedInNetwork(guestNetwork), @@ -3144,7 +3147,7 @@ public class VirtualNetworkApplianceManagerImpl extends ManagerBase implements V } if (createVmData) { - NicVO nic = _nicDao.findByInstanceIdAndNetworkId(guestNetworkId, vm.getId()); + NicVO nic = _nicDao.findByNtwkIdAndInstanceId(guestNetworkId, vm.getId()); if (nic != null) { s_logger.debug("Creating user data entry for vm " + vm + " on domR " + router); createVmDataCommand(router, vm, nic, null, cmds); @@ -3197,7 +3200,7 @@ public class VirtualNetworkApplianceManagerImpl extends ManagerBase implements V createDhcp = false; } if (createDhcp) { - NicVO nic = _nicDao.findByInstanceIdAndNetworkId(guestNetworkId, vm.getId()); + NicVO nic = _nicDao.findByNtwkIdAndInstanceId(guestNetworkId, vm.getId()); if (nic != null) { s_logger.debug("Creating dhcp entry for vm " + vm + " on domR " + router + "."); createDhcpEntryCommand(router, vm, nic, cmds); @@ -3315,13 +3318,14 @@ public class VirtualNetworkApplianceManagerImpl extends ManagerBase implements V public boolean execute(Network network, VirtualRouter router) throws ResourceUnavailableException { if (rules.get(0).getPurpose() == Purpose.LoadBalancing) { // for load balancer we have to resend all lb rules for the network - List lbs = _loadBalancerDao.listByNetworkId(network.getId()); + List lbs = _loadBalancerDao.listByNetworkIdAndScheme(network.getId(), Scheme.Public); List lbRules = new ArrayList(); for (LoadBalancerVO lb : lbs) { List dstList = _lbMgr.getExistingDestinations(lb.getId()); List policyList = _lbMgr.getStickinessPolicies(lb.getId()); - List hcPolicyList = _lbMgr.getHealthCheckPolicies(lb.getId() ); - LoadBalancingRule loadBalancing = new LoadBalancingRule(lb, dstList, policyList, hcPolicyList); + List hcPolicyList = _lbMgr.getHealthCheckPolicies(lb.getId()); + Ip sourceIp = _networkModel.getPublicIpAddress(lb.getSourceIpAddressId()).getAddress(); + LoadBalancingRule loadBalancing = new LoadBalancingRule(lb, dstList, policyList, hcPolicyList, sourceIp); lbRules.add(loadBalancing); } return sendLBRules(router, lbRules, network.getId()); @@ -3338,6 +3342,32 @@ public class VirtualNetworkApplianceManagerImpl extends ManagerBase implements V } }); } + + + @Override + public boolean applyLoadBalancingRules(Network network, final List rules, List routers) throws ResourceUnavailableException { + if (rules == null || rules.isEmpty()) { + s_logger.debug("No lb rules to be applied for network " + network.getId()); + return true; + } + return applyRules(network, routers, "loadbalancing rules", false, null, false, new RuleApplier() { + @Override + public boolean execute(Network network, VirtualRouter router) throws ResourceUnavailableException { + // for load balancer we have to resend all lb rules for the network + List lbs = _loadBalancerDao.listByNetworkIdAndScheme(network.getId(), Scheme.Public); + List lbRules = new ArrayList(); + for (LoadBalancerVO lb : lbs) { + List dstList = _lbMgr.getExistingDestinations(lb.getId()); + List policyList = _lbMgr.getStickinessPolicies(lb.getId()); + List hcPolicyList = _lbMgr.getHealthCheckPolicies(lb.getId()); + Ip sourceIp = _networkModel.getPublicIpAddress(lb.getSourceIpAddressId()).getAddress(); + LoadBalancingRule loadBalancing = new LoadBalancingRule(lb, dstList, policyList, hcPolicyList, sourceIp); + lbRules.add(loadBalancing); + } + return sendLBRules(router, lbRules, network.getId()); + } + }); + } protected boolean sendLBRules(VirtualRouter router, List rules, long guestNetworkId) throws ResourceUnavailableException { Commands cmds = new Commands(OnError.Continue); @@ -3734,4 +3764,11 @@ public class VirtualNetworkApplianceManagerImpl extends ManagerBase implements V } } } + + + + @Override + public VirtualRouter findRouter(long routerId) { + return _routerDao.findById(routerId); + } } diff --git a/server/src/com/cloud/network/router/VpcVirtualNetworkApplianceManagerImpl.java b/server/src/com/cloud/network/router/VpcVirtualNetworkApplianceManagerImpl.java index ebf2d4257e3..611100955e7 100644 --- a/server/src/com/cloud/network/router/VpcVirtualNetworkApplianceManagerImpl.java +++ b/server/src/com/cloud/network/router/VpcVirtualNetworkApplianceManagerImpl.java @@ -440,7 +440,7 @@ public class VpcVirtualNetworkApplianceManagerImpl extends VirtualNetworkApplian defaultDns2 = guestNic.getDns2(); } - Nic nic = _nicDao.findByInstanceIdAndNetworkId(network.getId(), router.getId()); + Nic nic = _nicDao.findByNtwkIdAndInstanceId(network.getId(), router.getId()); String networkDomain = network.getNetworkDomain(); String dhcpRange = getGuestDhcpRange(guestNic, network, _configMgr.getZone(network.getDataCenterId())); diff --git a/server/src/com/cloud/network/vpc/VpcManagerImpl.java b/server/src/com/cloud/network/vpc/VpcManagerImpl.java index 9be22084e11..55656d8b4fb 100644 --- a/server/src/com/cloud/network/vpc/VpcManagerImpl.java +++ b/server/src/com/cloud/network/vpc/VpcManagerImpl.java @@ -184,7 +184,9 @@ public class VpcManagerImpl extends ManagerBase implements VpcManager, VpcProvis private final ScheduledExecutorService _executor = Executors.newScheduledThreadPool(1, new NamedThreadFactory("VpcChecker")); private List vpcElements = null; private final List nonSupportedServices = Arrays.asList(Service.SecurityGroup, Service.Firewall); - private final List supportedProviders = Arrays.asList(Provider.VPCVirtualRouter, Provider.NiciraNvp, Provider.Netscaler); + private final List supportedProviders = Arrays.asList(Provider.VPCVirtualRouter, Provider.NiciraNvp, Provider.InternalLbVm, Provider.Netscaler); + + int _cleanupInterval; int _maxNetworks; SearchBuilder IpAddressSearch; @@ -206,6 +208,7 @@ public class VpcManagerImpl extends ManagerBase implements VpcManager, VpcProvis if (svc == Service.Lb) { Set lbProviders = new HashSet(); lbProviders.add(Provider.VPCVirtualRouter); + lbProviders.add(Provider.InternalLbVm); svcProviderMap.put(svc, lbProviders); } else { svcProviderMap.put(svc, defaultProviders); @@ -1057,16 +1060,17 @@ public class VpcManagerImpl extends ManagerBase implements VpcManager, VpcProvis } } - //4) Only one network in the VPC can support LB - if (_ntwkModel.areServicesSupportedByNetworkOffering(guestNtwkOff.getId(), Service.Lb)) { + //4) Only one network in the VPC can support public LB inside the VPC. Internal LB can be supported on multiple VPC tiers + if (_ntwkModel.areServicesSupportedByNetworkOffering(guestNtwkOff.getId(), Service.Lb) && guestNtwkOff.getPublicLb()) { List networks = getVpcNetworks(vpc.getId()); for (Network network : networks) { if (networkId != null && network.getId() == networkId.longValue()) { //skip my own network continue; } else { - if (_ntwkModel.areServicesSupportedInNetwork(network.getId(), Service.Lb)) { - throw new InvalidParameterValueException("LB service is already supported " + + NetworkOffering otherOff = _configMgr.getNetworkOffering(network.getNetworkOfferingId()); + if (_ntwkModel.areServicesSupportedInNetwork(network.getId(), Service.Lb) && otherOff.getPublicLb()) { + throw new InvalidParameterValueException("Public LB service is already supported " + "by network " + network + " in VPC " + vpc); } } diff --git a/server/src/com/cloud/server/ConfigurationServerImpl.java b/server/src/com/cloud/server/ConfigurationServerImpl.java index 3d97447fe40..bc52e9a881c 100755 --- a/server/src/com/cloud/server/ConfigurationServerImpl.java +++ b/server/src/com/cloud/server/ConfigurationServerImpl.java @@ -1017,7 +1017,7 @@ public class ConfigurationServerImpl extends ManagerBase implements Configuratio "Offering for Shared Security group enabled networks", TrafficType.Guest, false, true, null, null, true, Availability.Optional, - null, Network.GuestType.Shared, true, true, false); + null, Network.GuestType.Shared, true, true, false, false, false); defaultSharedSGNetworkOffering.setState(NetworkOffering.State.Enabled); defaultSharedSGNetworkOffering = _networkOfferingDao.persistDefaultNetworkOffering(defaultSharedSGNetworkOffering); @@ -1034,7 +1034,7 @@ public class ConfigurationServerImpl extends ManagerBase implements Configuratio "Offering for Shared networks", TrafficType.Guest, false, true, null, null, true, Availability.Optional, - null, Network.GuestType.Shared, true, true, false); + null, Network.GuestType.Shared, true, true, false, false, false); defaultSharedNetworkOffering.setState(NetworkOffering.State.Enabled); defaultSharedNetworkOffering = _networkOfferingDao.persistDefaultNetworkOffering(defaultSharedNetworkOffering); @@ -1051,7 +1051,7 @@ public class ConfigurationServerImpl extends ManagerBase implements Configuratio "Offering for Isolated networks with Source Nat service enabled", TrafficType.Guest, false, false, null, null, true, Availability.Required, - null, Network.GuestType.Isolated, true, false, false); + null, Network.GuestType.Isolated, true, false, false, false, true); defaultIsolatedSourceNatEnabledNetworkOffering.setState(NetworkOffering.State.Enabled); defaultIsolatedSourceNatEnabledNetworkOffering = _networkOfferingDao.persistDefaultNetworkOffering(defaultIsolatedSourceNatEnabledNetworkOffering); @@ -1069,7 +1069,7 @@ public class ConfigurationServerImpl extends ManagerBase implements Configuratio "Offering for Isolated networks with no Source Nat service", TrafficType.Guest, false, true, null, null, true, Availability.Optional, - null, Network.GuestType.Isolated, true, true, false); + null, Network.GuestType.Isolated, true, true, false, false, false); defaultIsolatedEnabledNetworkOffering.setState(NetworkOffering.State.Enabled); defaultIsolatedEnabledNetworkOffering = _networkOfferingDao.persistDefaultNetworkOffering(defaultIsolatedEnabledNetworkOffering); @@ -1086,7 +1086,7 @@ public class ConfigurationServerImpl extends ManagerBase implements Configuratio "Offering for Shared networks with Elastic IP and Elastic LB capabilities", TrafficType.Guest, false, true, null, null, true, Availability.Optional, - null, Network.GuestType.Shared, true, false, false, false, true, true, true, false, false, true); + null, Network.GuestType.Shared, true, false, false, false, true, true, true, false, false, true, true, false); defaultNetscalerNetworkOffering.setState(NetworkOffering.State.Enabled); defaultNetscalerNetworkOffering = _networkOfferingDao.persistDefaultNetworkOffering(defaultNetscalerNetworkOffering); @@ -1103,7 +1103,7 @@ public class ConfigurationServerImpl extends ManagerBase implements Configuratio "Offering for Isolated Vpc networks with Source Nat service enabled", TrafficType.Guest, false, false, null, null, true, Availability.Optional, - null, Network.GuestType.Isolated, false, false, false); + null, Network.GuestType.Isolated, false, false, false, false, true); defaultNetworkOfferingForVpcNetworks.setState(NetworkOffering.State.Enabled); defaultNetworkOfferingForVpcNetworks = _networkOfferingDao.persistDefaultNetworkOffering(defaultNetworkOfferingForVpcNetworks); @@ -1133,7 +1133,7 @@ public class ConfigurationServerImpl extends ManagerBase implements Configuratio "Offering for Isolated Vpc networks with Source Nat service enabled and LB service Disabled", TrafficType.Guest, false, false, null, null, true, Availability.Optional, - null, Network.GuestType.Isolated, false, false, false); + null, Network.GuestType.Isolated, false, false, false, false, false); defaultNetworkOfferingForVpcNetworksNoLB.setState(NetworkOffering.State.Enabled); defaultNetworkOfferingForVpcNetworksNoLB = _networkOfferingDao.persistDefaultNetworkOffering(defaultNetworkOfferingForVpcNetworksNoLB); diff --git a/server/src/com/cloud/server/ManagementServerImpl.java b/server/src/com/cloud/server/ManagementServerImpl.java index da8f30ed759..86c1a640da9 100755 --- a/server/src/com/cloud/server/ManagementServerImpl.java +++ b/server/src/com/cloud/server/ManagementServerImpl.java @@ -34,7 +34,6 @@ import java.util.List; import java.util.Map; import java.util.Set; import java.util.TimeZone; -import java.util.UUID; import java.util.concurrent.Executors; import java.util.concurrent.ScheduledExecutorService; import java.util.concurrent.TimeUnit; @@ -44,52 +43,371 @@ import javax.crypto.spec.SecretKeySpec; import javax.inject.Inject; import javax.naming.ConfigurationException; -import com.cloud.configuration.*; -import com.cloud.storage.dao.*; +import org.apache.cloudstack.acl.ControlledEntity; import org.apache.cloudstack.acl.SecurityChecker.AccessType; -import org.apache.cloudstack.api.ApiConstants; - -import com.cloud.event.ActionEventUtils; -import org.apache.cloudstack.api.BaseUpdateTemplateOrIsoCmd; -import org.apache.cloudstack.api.command.admin.account.*; -import org.apache.cloudstack.api.command.admin.domain.*; -import org.apache.cloudstack.api.command.admin.host.*; -import org.apache.cloudstack.api.command.admin.network.*; -import org.apache.cloudstack.api.command.admin.offering.*; -import org.apache.cloudstack.api.command.admin.resource.*; -import org.apache.cloudstack.api.command.admin.router.*; -import org.apache.cloudstack.api.command.admin.storage.*; -import org.apache.cloudstack.api.command.admin.systemvm.*; -import org.apache.cloudstack.api.command.admin.usage.*; -import org.apache.cloudstack.api.command.admin.user.*; -import org.apache.cloudstack.api.command.admin.vlan.*; -import org.apache.cloudstack.api.command.admin.vpc.*; -import org.apache.cloudstack.api.command.user.autoscale.*; -import org.apache.cloudstack.api.command.user.firewall.*; -import org.apache.cloudstack.api.command.user.iso.*; -import org.apache.cloudstack.api.command.user.loadbalancer.*; -import org.apache.cloudstack.api.command.user.nat.*; -import org.apache.cloudstack.api.command.user.network.*; -import org.apache.cloudstack.api.command.user.project.*; -import org.apache.cloudstack.api.command.user.resource.*; -import org.apache.cloudstack.api.command.user.securitygroup.*; -import org.apache.cloudstack.api.command.user.snapshot.*; -import org.apache.cloudstack.api.command.user.template.*; -import org.apache.cloudstack.api.command.user.vm.*; -import org.apache.cloudstack.api.command.user.volume.*; -import org.apache.cloudstack.api.command.user.vpc.*; -import org.apache.cloudstack.api.command.user.vpn.*; -import org.apache.cloudstack.api.response.ExtractResponse; -import org.apache.commons.codec.binary.Base64; -import org.apache.log4j.Logger; import org.apache.cloudstack.affinity.AffinityGroupProcessor; import org.apache.cloudstack.affinity.dao.AffinityGroupVMMapDao; - +import org.apache.cloudstack.api.ApiConstants; +import org.apache.cloudstack.api.BaseUpdateTemplateOrIsoCmd; +import org.apache.cloudstack.api.command.admin.account.CreateAccountCmd; +import org.apache.cloudstack.api.command.admin.account.DeleteAccountCmd; +import org.apache.cloudstack.api.command.admin.account.DisableAccountCmd; +import org.apache.cloudstack.api.command.admin.account.EnableAccountCmd; +import org.apache.cloudstack.api.command.admin.account.LockAccountCmd; +import org.apache.cloudstack.api.command.admin.account.UpdateAccountCmd; +import org.apache.cloudstack.api.command.admin.autoscale.CreateCounterCmd; +import org.apache.cloudstack.api.command.admin.autoscale.DeleteCounterCmd; +import org.apache.cloudstack.api.command.admin.cluster.AddClusterCmd; +import org.apache.cloudstack.api.command.admin.cluster.DeleteClusterCmd; +import org.apache.cloudstack.api.command.admin.cluster.ListClustersCmd; +import org.apache.cloudstack.api.command.admin.cluster.UpdateClusterCmd; +import org.apache.cloudstack.api.command.admin.config.ListCfgsByCmd; +import org.apache.cloudstack.api.command.admin.config.ListHypervisorCapabilitiesCmd; +import org.apache.cloudstack.api.command.admin.config.UpdateCfgCmd; +import org.apache.cloudstack.api.command.admin.config.UpdateHypervisorCapabilitiesCmd; +import org.apache.cloudstack.api.command.admin.domain.CreateDomainCmd; +import org.apache.cloudstack.api.command.admin.domain.DeleteDomainCmd; +import org.apache.cloudstack.api.command.admin.domain.ListDomainChildrenCmd; +import org.apache.cloudstack.api.command.admin.domain.ListDomainsCmd; +import org.apache.cloudstack.api.command.admin.domain.UpdateDomainCmd; +import org.apache.cloudstack.api.command.admin.host.AddHostCmd; +import org.apache.cloudstack.api.command.admin.host.AddSecondaryStorageCmd; +import org.apache.cloudstack.api.command.admin.host.CancelMaintenanceCmd; +import org.apache.cloudstack.api.command.admin.host.DeleteHostCmd; +import org.apache.cloudstack.api.command.admin.host.FindHostsForMigrationCmd; +import org.apache.cloudstack.api.command.admin.host.ListHostsCmd; +import org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd; +import org.apache.cloudstack.api.command.admin.host.ReconnectHostCmd; +import org.apache.cloudstack.api.command.admin.host.UpdateHostCmd; +import org.apache.cloudstack.api.command.admin.host.UpdateHostPasswordCmd; +import org.apache.cloudstack.api.command.admin.internallb.ConfigureInternalLoadBalancerElementCmd; +import org.apache.cloudstack.api.command.admin.internallb.CreateInternalLoadBalancerElementCmd; +import org.apache.cloudstack.api.command.admin.internallb.ListInternalLBVMsCmd; +import org.apache.cloudstack.api.command.admin.internallb.ListInternalLoadBalancerElementsCmd; +import org.apache.cloudstack.api.command.admin.internallb.StartInternalLBVMCmd; +import org.apache.cloudstack.api.command.admin.internallb.StopInternalLBVMCmd; +import org.apache.cloudstack.api.command.admin.ldap.LDAPConfigCmd; +import org.apache.cloudstack.api.command.admin.ldap.LDAPRemoveCmd; +import org.apache.cloudstack.api.command.admin.network.AddNetworkDeviceCmd; +import org.apache.cloudstack.api.command.admin.network.AddNetworkServiceProviderCmd; +import org.apache.cloudstack.api.command.admin.network.CreateNetworkOfferingCmd; +import org.apache.cloudstack.api.command.admin.network.CreatePhysicalNetworkCmd; +import org.apache.cloudstack.api.command.admin.network.CreateStorageNetworkIpRangeCmd; +import org.apache.cloudstack.api.command.admin.network.DedicateGuestVlanRangeCmd; +import org.apache.cloudstack.api.command.admin.network.DeleteNetworkDeviceCmd; +import org.apache.cloudstack.api.command.admin.network.DeleteNetworkOfferingCmd; +import org.apache.cloudstack.api.command.admin.network.DeleteNetworkServiceProviderCmd; +import org.apache.cloudstack.api.command.admin.network.DeletePhysicalNetworkCmd; +import org.apache.cloudstack.api.command.admin.network.DeleteStorageNetworkIpRangeCmd; +import org.apache.cloudstack.api.command.admin.network.ListDedicatedGuestVlanRangesCmd; +import org.apache.cloudstack.api.command.admin.network.ListNetworkDeviceCmd; +import org.apache.cloudstack.api.command.admin.network.ListNetworkIsolationMethodsCmd; +import org.apache.cloudstack.api.command.admin.network.ListNetworkServiceProvidersCmd; +import org.apache.cloudstack.api.command.admin.network.ListPhysicalNetworksCmd; +import org.apache.cloudstack.api.command.admin.network.ListStorageNetworkIpRangeCmd; +import org.apache.cloudstack.api.command.admin.network.ListSupportedNetworkServicesCmd; +import org.apache.cloudstack.api.command.admin.network.ReleaseDedicatedGuestVlanRangeCmd; +import org.apache.cloudstack.api.command.admin.network.UpdateNetworkOfferingCmd; +import org.apache.cloudstack.api.command.admin.network.UpdateNetworkServiceProviderCmd; +import org.apache.cloudstack.api.command.admin.network.UpdatePhysicalNetworkCmd; +import org.apache.cloudstack.api.command.admin.network.UpdateStorageNetworkIpRangeCmd; +import org.apache.cloudstack.api.command.admin.offering.CreateDiskOfferingCmd; +import org.apache.cloudstack.api.command.admin.offering.CreateServiceOfferingCmd; +import org.apache.cloudstack.api.command.admin.offering.DeleteDiskOfferingCmd; +import org.apache.cloudstack.api.command.admin.offering.DeleteServiceOfferingCmd; +import org.apache.cloudstack.api.command.admin.offering.UpdateDiskOfferingCmd; +import org.apache.cloudstack.api.command.admin.offering.UpdateServiceOfferingCmd; +import org.apache.cloudstack.api.command.admin.pod.CreatePodCmd; +import org.apache.cloudstack.api.command.admin.pod.DeletePodCmd; +import org.apache.cloudstack.api.command.admin.pod.ListPodsByCmd; +import org.apache.cloudstack.api.command.admin.pod.UpdatePodCmd; +import org.apache.cloudstack.api.command.admin.region.AddRegionCmd; +import org.apache.cloudstack.api.command.admin.region.RemoveRegionCmd; +import org.apache.cloudstack.api.command.admin.region.UpdateRegionCmd; +import org.apache.cloudstack.api.command.admin.resource.ArchiveAlertsCmd; +import org.apache.cloudstack.api.command.admin.resource.DeleteAlertsCmd; +import org.apache.cloudstack.api.command.admin.resource.ListAlertsCmd; +import org.apache.cloudstack.api.command.admin.resource.ListCapacityCmd; +import org.apache.cloudstack.api.command.admin.resource.UploadCustomCertificateCmd; +import org.apache.cloudstack.api.command.admin.router.ConfigureVirtualRouterElementCmd; +import org.apache.cloudstack.api.command.admin.router.CreateVirtualRouterElementCmd; +import org.apache.cloudstack.api.command.admin.router.DestroyRouterCmd; +import org.apache.cloudstack.api.command.admin.router.ListRoutersCmd; +import org.apache.cloudstack.api.command.admin.router.ListVirtualRouterElementsCmd; +import org.apache.cloudstack.api.command.admin.router.RebootRouterCmd; +import org.apache.cloudstack.api.command.admin.router.StartRouterCmd; +import org.apache.cloudstack.api.command.admin.router.StopRouterCmd; +import org.apache.cloudstack.api.command.admin.router.UpgradeRouterCmd; +import org.apache.cloudstack.api.command.admin.storage.AddS3Cmd; +import org.apache.cloudstack.api.command.admin.storage.CancelPrimaryStorageMaintenanceCmd; +import org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd; +import org.apache.cloudstack.api.command.admin.storage.DeletePoolCmd; +import org.apache.cloudstack.api.command.admin.storage.FindStoragePoolsForMigrationCmd; +import org.apache.cloudstack.api.command.admin.storage.ListS3sCmd; +import org.apache.cloudstack.api.command.admin.storage.ListStoragePoolsCmd; +import org.apache.cloudstack.api.command.admin.storage.ListStorageProvidersCmd; +import org.apache.cloudstack.api.command.admin.storage.PreparePrimaryStorageForMaintenanceCmd; +import org.apache.cloudstack.api.command.admin.storage.UpdateStoragePoolCmd; +import org.apache.cloudstack.api.command.admin.swift.AddSwiftCmd; +import org.apache.cloudstack.api.command.admin.swift.ListSwiftsCmd; +import org.apache.cloudstack.api.command.admin.systemvm.DestroySystemVmCmd; +import org.apache.cloudstack.api.command.admin.systemvm.ListSystemVMsCmd; +import org.apache.cloudstack.api.command.admin.systemvm.MigrateSystemVMCmd; +import org.apache.cloudstack.api.command.admin.systemvm.RebootSystemVmCmd; +import org.apache.cloudstack.api.command.admin.systemvm.StartSystemVMCmd; +import org.apache.cloudstack.api.command.admin.systemvm.StopSystemVmCmd; +import org.apache.cloudstack.api.command.admin.systemvm.UpgradeSystemVMCmd; +import org.apache.cloudstack.api.command.admin.template.PrepareTemplateCmd; +import org.apache.cloudstack.api.command.admin.usage.AddTrafficMonitorCmd; +import org.apache.cloudstack.api.command.admin.usage.AddTrafficTypeCmd; +import org.apache.cloudstack.api.command.admin.usage.DeleteTrafficMonitorCmd; +import org.apache.cloudstack.api.command.admin.usage.DeleteTrafficTypeCmd; +import org.apache.cloudstack.api.command.admin.usage.GenerateUsageRecordsCmd; +import org.apache.cloudstack.api.command.admin.usage.GetUsageRecordsCmd; +import org.apache.cloudstack.api.command.admin.usage.ListTrafficMonitorsCmd; +import org.apache.cloudstack.api.command.admin.usage.ListTrafficTypeImplementorsCmd; +import org.apache.cloudstack.api.command.admin.usage.ListTrafficTypesCmd; +import org.apache.cloudstack.api.command.admin.usage.ListUsageTypesCmd; +import org.apache.cloudstack.api.command.admin.usage.UpdateTrafficTypeCmd; +import org.apache.cloudstack.api.command.admin.user.CreateUserCmd; +import org.apache.cloudstack.api.command.admin.user.DeleteUserCmd; +import org.apache.cloudstack.api.command.admin.user.DisableUserCmd; +import org.apache.cloudstack.api.command.admin.user.EnableUserCmd; +import org.apache.cloudstack.api.command.admin.user.GetUserCmd; +import org.apache.cloudstack.api.command.admin.user.ListUsersCmd; +import org.apache.cloudstack.api.command.admin.user.LockUserCmd; +import org.apache.cloudstack.api.command.admin.user.RegisterCmd; +import org.apache.cloudstack.api.command.admin.user.UpdateUserCmd; +import org.apache.cloudstack.api.command.admin.vlan.CreateVlanIpRangeCmd; +import org.apache.cloudstack.api.command.admin.vlan.DedicatePublicIpRangeCmd; +import org.apache.cloudstack.api.command.admin.vlan.DeleteVlanIpRangeCmd; +import org.apache.cloudstack.api.command.admin.vlan.ListVlanIpRangesCmd; +import org.apache.cloudstack.api.command.admin.vlan.ReleasePublicIpRangeCmd; +import org.apache.cloudstack.api.command.admin.vm.AssignVMCmd; +import org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd; +import org.apache.cloudstack.api.command.admin.vm.MigrateVirtualMachineWithVolumeCmd; +import org.apache.cloudstack.api.command.admin.vm.RecoverVMCmd; +import org.apache.cloudstack.api.command.admin.vpc.CreatePrivateGatewayCmd; +import org.apache.cloudstack.api.command.admin.vpc.CreateVPCOfferingCmd; +import org.apache.cloudstack.api.command.admin.vpc.DeletePrivateGatewayCmd; +import org.apache.cloudstack.api.command.admin.vpc.DeleteVPCOfferingCmd; +import org.apache.cloudstack.api.command.admin.vpc.UpdateVPCOfferingCmd; +import org.apache.cloudstack.api.command.admin.zone.CreateZoneCmd; +import org.apache.cloudstack.api.command.admin.zone.DeleteZoneCmd; +import org.apache.cloudstack.api.command.admin.zone.MarkDefaultZoneForAccountCmd; +import org.apache.cloudstack.api.command.admin.zone.UpdateZoneCmd; +import org.apache.cloudstack.api.command.user.account.AddAccountToProjectCmd; +import org.apache.cloudstack.api.command.user.account.DeleteAccountFromProjectCmd; +import org.apache.cloudstack.api.command.user.account.ListAccountsCmd; +import org.apache.cloudstack.api.command.user.account.ListProjectAccountsCmd; +import org.apache.cloudstack.api.command.user.address.AssociateIPAddrCmd; +import org.apache.cloudstack.api.command.user.address.DisassociateIPAddrCmd; +import org.apache.cloudstack.api.command.user.address.ListPublicIpAddressesCmd; import org.apache.cloudstack.api.command.user.affinitygroup.CreateAffinityGroupCmd; import org.apache.cloudstack.api.command.user.affinitygroup.DeleteAffinityGroupCmd; import org.apache.cloudstack.api.command.user.affinitygroup.ListAffinityGroupTypesCmd; import org.apache.cloudstack.api.command.user.affinitygroup.ListAffinityGroupsCmd; import org.apache.cloudstack.api.command.user.affinitygroup.UpdateVMAffinityGroupCmd; +import org.apache.cloudstack.api.command.user.autoscale.CreateAutoScalePolicyCmd; +import org.apache.cloudstack.api.command.user.autoscale.CreateAutoScaleVmGroupCmd; +import org.apache.cloudstack.api.command.user.autoscale.CreateAutoScaleVmProfileCmd; +import org.apache.cloudstack.api.command.user.autoscale.CreateConditionCmd; +import org.apache.cloudstack.api.command.user.autoscale.DeleteAutoScalePolicyCmd; +import org.apache.cloudstack.api.command.user.autoscale.DeleteAutoScaleVmGroupCmd; +import org.apache.cloudstack.api.command.user.autoscale.DeleteAutoScaleVmProfileCmd; +import org.apache.cloudstack.api.command.user.autoscale.DeleteConditionCmd; +import org.apache.cloudstack.api.command.user.autoscale.DisableAutoScaleVmGroupCmd; +import org.apache.cloudstack.api.command.user.autoscale.EnableAutoScaleVmGroupCmd; +import org.apache.cloudstack.api.command.user.autoscale.ListAutoScalePoliciesCmd; +import org.apache.cloudstack.api.command.user.autoscale.ListAutoScaleVmGroupsCmd; +import org.apache.cloudstack.api.command.user.autoscale.ListAutoScaleVmProfilesCmd; +import org.apache.cloudstack.api.command.user.autoscale.ListConditionsCmd; +import org.apache.cloudstack.api.command.user.autoscale.ListCountersCmd; +import org.apache.cloudstack.api.command.user.autoscale.UpdateAutoScalePolicyCmd; +import org.apache.cloudstack.api.command.user.autoscale.UpdateAutoScaleVmGroupCmd; +import org.apache.cloudstack.api.command.user.autoscale.UpdateAutoScaleVmProfileCmd; +import org.apache.cloudstack.api.command.user.config.ListCapabilitiesCmd; +import org.apache.cloudstack.api.command.user.event.ArchiveEventsCmd; +import org.apache.cloudstack.api.command.user.event.DeleteEventsCmd; +import org.apache.cloudstack.api.command.user.event.ListEventTypesCmd; +import org.apache.cloudstack.api.command.user.event.ListEventsCmd; +import org.apache.cloudstack.api.command.user.firewall.CreateEgressFirewallRuleCmd; +import org.apache.cloudstack.api.command.user.firewall.CreateFirewallRuleCmd; +import org.apache.cloudstack.api.command.user.firewall.CreatePortForwardingRuleCmd; +import org.apache.cloudstack.api.command.user.firewall.DeleteEgressFirewallRuleCmd; +import org.apache.cloudstack.api.command.user.firewall.DeleteFirewallRuleCmd; +import org.apache.cloudstack.api.command.user.firewall.DeletePortForwardingRuleCmd; +import org.apache.cloudstack.api.command.user.firewall.ListEgressFirewallRulesCmd; +import org.apache.cloudstack.api.command.user.firewall.ListFirewallRulesCmd; +import org.apache.cloudstack.api.command.user.firewall.ListPortForwardingRulesCmd; +import org.apache.cloudstack.api.command.user.firewall.UpdatePortForwardingRuleCmd; +import org.apache.cloudstack.api.command.user.guest.ListGuestOsCategoriesCmd; +import org.apache.cloudstack.api.command.user.guest.ListGuestOsCmd; +import org.apache.cloudstack.api.command.user.iso.AttachIsoCmd; +import org.apache.cloudstack.api.command.user.iso.CopyIsoCmd; +import org.apache.cloudstack.api.command.user.iso.DeleteIsoCmd; +import org.apache.cloudstack.api.command.user.iso.DetachIsoCmd; +import org.apache.cloudstack.api.command.user.iso.ExtractIsoCmd; +import org.apache.cloudstack.api.command.user.iso.ListIsoPermissionsCmd; +import org.apache.cloudstack.api.command.user.iso.ListIsosCmd; +import org.apache.cloudstack.api.command.user.iso.RegisterIsoCmd; +import org.apache.cloudstack.api.command.user.iso.UpdateIsoCmd; +import org.apache.cloudstack.api.command.user.iso.UpdateIsoPermissionsCmd; +import org.apache.cloudstack.api.command.user.job.ListAsyncJobsCmd; +import org.apache.cloudstack.api.command.user.job.QueryAsyncJobResultCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.AssignToLoadBalancerRuleCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.CreateApplicationLoadBalancerCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.CreateLBHealthCheckPolicyCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.CreateLBStickinessPolicyCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.CreateLoadBalancerRuleCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.DeleteApplicationLoadBalancerCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.DeleteLBHealthCheckPolicyCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.DeleteLBStickinessPolicyCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.DeleteLoadBalancerRuleCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.ListApplicationLoadBalancersCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.ListLBHealthCheckPoliciesCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.ListLBStickinessPoliciesCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.ListLoadBalancerRuleInstancesCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.ListLoadBalancerRulesCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.RemoveFromLoadBalancerRuleCmd; +import org.apache.cloudstack.api.command.user.loadbalancer.UpdateLoadBalancerRuleCmd; +import org.apache.cloudstack.api.command.user.nat.CreateIpForwardingRuleCmd; +import org.apache.cloudstack.api.command.user.nat.DeleteIpForwardingRuleCmd; +import org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd; +import org.apache.cloudstack.api.command.user.nat.EnableStaticNatCmd; +import org.apache.cloudstack.api.command.user.nat.ListIpForwardingRulesCmd; +import org.apache.cloudstack.api.command.user.network.CreateNetworkACLCmd; +import org.apache.cloudstack.api.command.user.network.CreateNetworkCmd; +import org.apache.cloudstack.api.command.user.network.DeleteNetworkACLCmd; +import org.apache.cloudstack.api.command.user.network.DeleteNetworkCmd; +import org.apache.cloudstack.api.command.user.network.ListNetworkACLsCmd; +import org.apache.cloudstack.api.command.user.network.ListNetworkOfferingsCmd; +import org.apache.cloudstack.api.command.user.network.ListNetworksCmd; +import org.apache.cloudstack.api.command.user.network.RestartNetworkCmd; +import org.apache.cloudstack.api.command.user.network.UpdateNetworkCmd; +import org.apache.cloudstack.api.command.user.offering.ListDiskOfferingsCmd; +import org.apache.cloudstack.api.command.user.offering.ListServiceOfferingsCmd; +import org.apache.cloudstack.api.command.user.project.ActivateProjectCmd; +import org.apache.cloudstack.api.command.user.project.CreateProjectCmd; +import org.apache.cloudstack.api.command.user.project.DeleteProjectCmd; +import org.apache.cloudstack.api.command.user.project.DeleteProjectInvitationCmd; +import org.apache.cloudstack.api.command.user.project.ListProjectInvitationsCmd; +import org.apache.cloudstack.api.command.user.project.ListProjectsCmd; +import org.apache.cloudstack.api.command.user.project.SuspendProjectCmd; +import org.apache.cloudstack.api.command.user.project.UpdateProjectCmd; +import org.apache.cloudstack.api.command.user.project.UpdateProjectInvitationCmd; +import org.apache.cloudstack.api.command.user.region.ListRegionsCmd; +import org.apache.cloudstack.api.command.user.region.ha.gslb.AssignToGlobalLoadBalancerRuleCmd; +import org.apache.cloudstack.api.command.user.region.ha.gslb.CreateGlobalLoadBalancerRuleCmd; +import org.apache.cloudstack.api.command.user.region.ha.gslb.DeleteGlobalLoadBalancerRuleCmd; +import org.apache.cloudstack.api.command.user.region.ha.gslb.ListGlobalLoadBalancerRuleCmd; +import org.apache.cloudstack.api.command.user.region.ha.gslb.RemoveFromGlobalLoadBalancerRuleCmd; +import org.apache.cloudstack.api.command.user.resource.GetCloudIdentifierCmd; +import org.apache.cloudstack.api.command.user.resource.ListHypervisorsCmd; +import org.apache.cloudstack.api.command.user.resource.ListResourceLimitsCmd; +import org.apache.cloudstack.api.command.user.resource.UpdateResourceCountCmd; +import org.apache.cloudstack.api.command.user.resource.UpdateResourceLimitCmd; +import org.apache.cloudstack.api.command.user.securitygroup.AuthorizeSecurityGroupEgressCmd; +import org.apache.cloudstack.api.command.user.securitygroup.AuthorizeSecurityGroupIngressCmd; +import org.apache.cloudstack.api.command.user.securitygroup.CreateSecurityGroupCmd; +import org.apache.cloudstack.api.command.user.securitygroup.DeleteSecurityGroupCmd; +import org.apache.cloudstack.api.command.user.securitygroup.ListSecurityGroupsCmd; +import org.apache.cloudstack.api.command.user.securitygroup.RevokeSecurityGroupEgressCmd; +import org.apache.cloudstack.api.command.user.securitygroup.RevokeSecurityGroupIngressCmd; +import org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd; +import org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotPolicyCmd; +import org.apache.cloudstack.api.command.user.snapshot.DeleteSnapshotCmd; +import org.apache.cloudstack.api.command.user.snapshot.DeleteSnapshotPoliciesCmd; +import org.apache.cloudstack.api.command.user.snapshot.ListSnapshotPoliciesCmd; +import org.apache.cloudstack.api.command.user.snapshot.ListSnapshotsCmd; +import org.apache.cloudstack.api.command.user.ssh.CreateSSHKeyPairCmd; +import org.apache.cloudstack.api.command.user.ssh.DeleteSSHKeyPairCmd; +import org.apache.cloudstack.api.command.user.ssh.ListSSHKeyPairsCmd; +import org.apache.cloudstack.api.command.user.ssh.RegisterSSHKeyPairCmd; +import org.apache.cloudstack.api.command.user.tag.CreateTagsCmd; +import org.apache.cloudstack.api.command.user.tag.DeleteTagsCmd; +import org.apache.cloudstack.api.command.user.tag.ListTagsCmd; +import org.apache.cloudstack.api.command.user.template.CopyTemplateCmd; +import org.apache.cloudstack.api.command.user.template.CreateTemplateCmd; +import org.apache.cloudstack.api.command.user.template.DeleteTemplateCmd; +import org.apache.cloudstack.api.command.user.template.ExtractTemplateCmd; +import org.apache.cloudstack.api.command.user.template.ListTemplatePermissionsCmd; +import org.apache.cloudstack.api.command.user.template.ListTemplatesCmd; +import org.apache.cloudstack.api.command.user.template.RegisterTemplateCmd; +import org.apache.cloudstack.api.command.user.template.UpdateTemplateCmd; +import org.apache.cloudstack.api.command.user.template.UpdateTemplatePermissionsCmd; +import org.apache.cloudstack.api.command.user.vm.AddIpToVmNicCmd; +import org.apache.cloudstack.api.command.user.vm.AddNicToVMCmd; +import org.apache.cloudstack.api.command.user.vm.DeployVMCmd; +import org.apache.cloudstack.api.command.user.vm.DestroyVMCmd; +import org.apache.cloudstack.api.command.user.vm.GetVMPasswordCmd; +import org.apache.cloudstack.api.command.user.vm.ListNicsCmd; +import org.apache.cloudstack.api.command.user.vm.ListVMsCmd; +import org.apache.cloudstack.api.command.user.vm.RebootVMCmd; +import org.apache.cloudstack.api.command.user.vm.RemoveIpFromVmNicCmd; +import org.apache.cloudstack.api.command.user.vm.RemoveNicFromVMCmd; +import org.apache.cloudstack.api.command.user.vm.ResetVMPasswordCmd; +import org.apache.cloudstack.api.command.user.vm.ResetVMSSHKeyCmd; +import org.apache.cloudstack.api.command.user.vm.RestoreVMCmd; +import org.apache.cloudstack.api.command.user.vm.ScaleVMCmd; +import org.apache.cloudstack.api.command.user.vm.StartVMCmd; +import org.apache.cloudstack.api.command.user.vm.StopVMCmd; +import org.apache.cloudstack.api.command.user.vm.UpdateDefaultNicForVMCmd; +import org.apache.cloudstack.api.command.user.vm.UpdateVMCmd; +import org.apache.cloudstack.api.command.user.vm.UpgradeVMCmd; +import org.apache.cloudstack.api.command.user.vmgroup.CreateVMGroupCmd; +import org.apache.cloudstack.api.command.user.vmgroup.DeleteVMGroupCmd; +import org.apache.cloudstack.api.command.user.vmgroup.ListVMGroupsCmd; +import org.apache.cloudstack.api.command.user.vmgroup.UpdateVMGroupCmd; +import org.apache.cloudstack.api.command.user.vmsnapshot.CreateVMSnapshotCmd; +import org.apache.cloudstack.api.command.user.vmsnapshot.DeleteVMSnapshotCmd; +import org.apache.cloudstack.api.command.user.vmsnapshot.ListVMSnapshotCmd; +import org.apache.cloudstack.api.command.user.vmsnapshot.RevertToVMSnapshotCmd; +import org.apache.cloudstack.api.command.user.volume.AttachVolumeCmd; +import org.apache.cloudstack.api.command.user.volume.CreateVolumeCmd; +import org.apache.cloudstack.api.command.user.volume.DeleteVolumeCmd; +import org.apache.cloudstack.api.command.user.volume.DetachVolumeCmd; +import org.apache.cloudstack.api.command.user.volume.ExtractVolumeCmd; +import org.apache.cloudstack.api.command.user.volume.ListVolumesCmd; +import org.apache.cloudstack.api.command.user.volume.MigrateVolumeCmd; +import org.apache.cloudstack.api.command.user.volume.ResizeVolumeCmd; +import org.apache.cloudstack.api.command.user.volume.UploadVolumeCmd; +import org.apache.cloudstack.api.command.user.vpc.CreateStaticRouteCmd; +import org.apache.cloudstack.api.command.user.vpc.CreateVPCCmd; +import org.apache.cloudstack.api.command.user.vpc.DeleteStaticRouteCmd; +import org.apache.cloudstack.api.command.user.vpc.DeleteVPCCmd; +import org.apache.cloudstack.api.command.user.vpc.ListPrivateGatewaysCmd; +import org.apache.cloudstack.api.command.user.vpc.ListStaticRoutesCmd; +import org.apache.cloudstack.api.command.user.vpc.ListVPCOfferingsCmd; +import org.apache.cloudstack.api.command.user.vpc.ListVPCsCmd; +import org.apache.cloudstack.api.command.user.vpc.RestartVPCCmd; +import org.apache.cloudstack.api.command.user.vpc.UpdateVPCCmd; +import org.apache.cloudstack.api.command.user.vpn.AddVpnUserCmd; +import org.apache.cloudstack.api.command.user.vpn.CreateRemoteAccessVpnCmd; +import org.apache.cloudstack.api.command.user.vpn.CreateVpnConnectionCmd; +import org.apache.cloudstack.api.command.user.vpn.CreateVpnCustomerGatewayCmd; +import org.apache.cloudstack.api.command.user.vpn.CreateVpnGatewayCmd; +import org.apache.cloudstack.api.command.user.vpn.DeleteRemoteAccessVpnCmd; +import org.apache.cloudstack.api.command.user.vpn.DeleteVpnConnectionCmd; +import org.apache.cloudstack.api.command.user.vpn.DeleteVpnCustomerGatewayCmd; +import org.apache.cloudstack.api.command.user.vpn.DeleteVpnGatewayCmd; +import org.apache.cloudstack.api.command.user.vpn.ListRemoteAccessVpnsCmd; +import org.apache.cloudstack.api.command.user.vpn.ListVpnConnectionsCmd; +import org.apache.cloudstack.api.command.user.vpn.ListVpnCustomerGatewaysCmd; +import org.apache.cloudstack.api.command.user.vpn.ListVpnGatewaysCmd; +import org.apache.cloudstack.api.command.user.vpn.ListVpnUsersCmd; +import org.apache.cloudstack.api.command.user.vpn.RemoveVpnUserCmd; +import org.apache.cloudstack.api.command.user.vpn.ResetVpnConnectionCmd; +import org.apache.cloudstack.api.command.user.vpn.UpdateVpnCustomerGatewayCmd; +import org.apache.cloudstack.api.command.user.zone.ListZonesByCmd; +import org.apache.cloudstack.api.response.ExtractResponse; +import org.apache.cloudstack.engine.subsystem.api.storage.DataStoreManager; +import org.apache.cloudstack.engine.subsystem.api.storage.StoragePoolAllocator; +import org.apache.cloudstack.storage.datastore.db.PrimaryDataStoreDao; +import org.apache.cloudstack.storage.datastore.db.StoragePoolVO; +import org.apache.commons.codec.binary.Base64; +import org.apache.log4j.Logger; + import com.cloud.agent.AgentManager; import com.cloud.agent.api.GetVncPortAnswer; import com.cloud.agent.api.GetVncPortCommand; @@ -103,28 +421,53 @@ import com.cloud.alert.AlertManager; import com.cloud.alert.AlertVO; import com.cloud.alert.dao.AlertDao; import com.cloud.api.ApiDBUtils; -import com.cloud.async.*; +import com.cloud.async.AsyncJobExecutor; +import com.cloud.async.AsyncJobManager; +import com.cloud.async.AsyncJobResult; +import com.cloud.async.AsyncJobVO; +import com.cloud.async.BaseAsyncJobExecutor; import com.cloud.capacity.Capacity; import com.cloud.capacity.CapacityVO; import com.cloud.capacity.dao.CapacityDao; import com.cloud.capacity.dao.CapacityDaoImpl.SummedCapacity; import com.cloud.cluster.ClusterManager; +import com.cloud.configuration.Config; +import com.cloud.configuration.Configuration; +import com.cloud.configuration.ConfigurationManager; +import com.cloud.configuration.ConfigurationVO; import com.cloud.configuration.dao.ConfigurationDao; import com.cloud.consoleproxy.ConsoleProxyManagementState; import com.cloud.consoleproxy.ConsoleProxyManager; -import com.cloud.dc.*; +import com.cloud.dc.AccountVlanMapVO; +import com.cloud.dc.ClusterVO; +import com.cloud.dc.DataCenterVO; +import com.cloud.dc.HostPodVO; +import com.cloud.dc.Pod; +import com.cloud.dc.PodVlanMapVO; +import com.cloud.dc.Vlan; import com.cloud.dc.Vlan.VlanType; -import com.cloud.dc.dao.*; +import com.cloud.dc.VlanVO; +import com.cloud.dc.dao.AccountVlanMapDao; +import com.cloud.dc.dao.ClusterDao; +import com.cloud.dc.dao.DataCenterDao; +import com.cloud.dc.dao.HostPodDao; +import com.cloud.dc.dao.PodVlanMapDao; +import com.cloud.dc.dao.VlanDao; import com.cloud.deploy.DataCenterDeployment; -import com.cloud.deploy.DeploymentPlanner; import com.cloud.deploy.DeploymentPlanner.ExcludeList; import com.cloud.domain.DomainVO; import com.cloud.domain.dao.DomainDao; import com.cloud.event.ActionEvent; +import com.cloud.event.ActionEventUtils; import com.cloud.event.EventTypes; import com.cloud.event.EventVO; import com.cloud.event.dao.EventDao; -import com.cloud.exception.*; +import com.cloud.exception.ConcurrentOperationException; +import com.cloud.exception.InvalidParameterValueException; +import com.cloud.exception.OperationTimedoutException; +import com.cloud.exception.PermissionDeniedException; +import com.cloud.exception.ResourceUnavailableException; +import com.cloud.exception.StorageUnavailableException; import com.cloud.ha.HighAvailabilityManager; import com.cloud.host.DetailVO; import com.cloud.host.Host; @@ -141,7 +484,12 @@ import com.cloud.hypervisor.dao.HypervisorCapabilitiesDao; import com.cloud.info.ConsoleProxyInfo; import com.cloud.keystore.KeystoreManager; import com.cloud.network.IpAddress; -import com.cloud.network.dao.*; +import com.cloud.network.dao.IPAddressDao; +import com.cloud.network.dao.IPAddressVO; +import com.cloud.network.dao.LoadBalancerDao; +import com.cloud.network.dao.LoadBalancerVO; +import com.cloud.network.dao.NetworkDao; +import com.cloud.network.dao.NetworkVO; import com.cloud.org.Cluster; import com.cloud.org.Grouping.AllocationState; import com.cloud.projects.Project; @@ -150,11 +498,29 @@ import com.cloud.projects.ProjectManager; import com.cloud.resource.ResourceManager; import com.cloud.server.ResourceTag.TaggedResourceType; import com.cloud.server.auth.UserAuthenticator; -import com.cloud.service.ServiceOfferingVO; import com.cloud.service.dao.ServiceOfferingDao; -import com.cloud.storage.*; +import com.cloud.storage.DiskOfferingVO; +import com.cloud.storage.GuestOS; +import com.cloud.storage.GuestOSCategoryVO; +import com.cloud.storage.GuestOSVO; +import com.cloud.storage.GuestOsCategory; +import com.cloud.storage.Storage; import com.cloud.storage.Storage.ImageFormat; +import com.cloud.storage.StorageManager; +import com.cloud.storage.StoragePool; +import com.cloud.storage.Upload; import com.cloud.storage.Upload.Mode; +import com.cloud.storage.UploadVO; +import com.cloud.storage.VMTemplateVO; +import com.cloud.storage.Volume; +import com.cloud.storage.VolumeManager; +import com.cloud.storage.VolumeVO; +import com.cloud.storage.dao.DiskOfferingDao; +import com.cloud.storage.dao.GuestOSCategoryDao; +import com.cloud.storage.dao.GuestOSDao; +import com.cloud.storage.dao.UploadDao; +import com.cloud.storage.dao.VMTemplateDao; +import com.cloud.storage.dao.VolumeDao; import com.cloud.storage.s3.S3Manager; import com.cloud.storage.secondary.SecondaryStorageVmManager; import com.cloud.storage.snapshot.SnapshotManager; @@ -164,7 +530,13 @@ import com.cloud.tags.ResourceTagVO; import com.cloud.tags.dao.ResourceTagDao; import com.cloud.template.TemplateManager; import com.cloud.template.VirtualMachineTemplate.TemplateFilter; -import com.cloud.user.*; +import com.cloud.user.Account; +import com.cloud.user.AccountManager; +import com.cloud.user.SSHKeyPair; +import com.cloud.user.SSHKeyPairVO; +import com.cloud.user.User; +import com.cloud.user.UserContext; +import com.cloud.user.UserVO; import com.cloud.user.dao.AccountDao; import com.cloud.user.dao.SSHKeyPairDao; import com.cloud.user.dao.UserDao; @@ -177,91 +549,39 @@ import com.cloud.utils.component.ComponentLifecycle; import com.cloud.utils.component.ManagerBase; import com.cloud.utils.concurrency.NamedThreadFactory; import com.cloud.utils.crypt.DBEncryptionUtil; -import com.cloud.utils.db.*; +import com.cloud.utils.db.DB; +import com.cloud.utils.db.Filter; +import com.cloud.utils.db.GlobalLock; +import com.cloud.utils.db.JoinBuilder; import com.cloud.utils.db.JoinBuilder.JoinType; +import com.cloud.utils.db.SearchBuilder; +import com.cloud.utils.db.SearchCriteria; +import com.cloud.utils.db.Transaction; import com.cloud.utils.exception.CloudRuntimeException; import com.cloud.utils.net.MacAddress; import com.cloud.utils.net.NetUtils; import com.cloud.utils.ssh.SSHKeysHelper; -import com.cloud.vm.*; +import com.cloud.vm.ConsoleProxyVO; +import com.cloud.vm.DiskProfile; +import com.cloud.vm.InstanceGroupVO; +import com.cloud.vm.SecondaryStorageVmVO; +import com.cloud.vm.UserVmVO; +import com.cloud.vm.VMInstanceVO; +import com.cloud.vm.VirtualMachine; import com.cloud.vm.VirtualMachine.State; -import com.cloud.vm.dao.*; +import com.cloud.vm.VirtualMachineManager; +import com.cloud.vm.VirtualMachineProfile; +import com.cloud.vm.VirtualMachineProfileImpl; +import com.cloud.vm.dao.ConsoleProxyDao; +import com.cloud.vm.dao.DomainRouterDao; +import com.cloud.vm.dao.InstanceGroupDao; +import com.cloud.vm.dao.SecondaryStorageVmDao; +import com.cloud.vm.dao.UserVmDao; +import com.cloud.vm.dao.VMInstanceDao; + import edu.emory.mathcs.backport.java.util.Arrays; import edu.emory.mathcs.backport.java.util.Collections; -import org.apache.cloudstack.acl.ControlledEntity; -import org.apache.cloudstack.api.command.admin.autoscale.CreateCounterCmd; -import org.apache.cloudstack.api.command.admin.autoscale.DeleteCounterCmd; -import org.apache.cloudstack.api.command.admin.cluster.AddClusterCmd; -import org.apache.cloudstack.api.command.admin.cluster.DeleteClusterCmd; -import org.apache.cloudstack.api.command.admin.cluster.ListClustersCmd; -import org.apache.cloudstack.api.command.admin.cluster.UpdateClusterCmd; -import org.apache.cloudstack.api.command.admin.config.ListCfgsByCmd; -import org.apache.cloudstack.api.command.admin.config.ListHypervisorCapabilitiesCmd; -import org.apache.cloudstack.api.command.admin.config.UpdateCfgCmd; -import org.apache.cloudstack.api.command.admin.config.UpdateHypervisorCapabilitiesCmd; -import org.apache.cloudstack.api.command.admin.ldap.LDAPConfigCmd; -import org.apache.cloudstack.api.command.admin.ldap.LDAPRemoveCmd; -import org.apache.cloudstack.api.command.admin.pod.CreatePodCmd; -import org.apache.cloudstack.api.command.admin.pod.DeletePodCmd; -import org.apache.cloudstack.api.command.admin.pod.ListPodsByCmd; -import org.apache.cloudstack.api.command.admin.pod.UpdatePodCmd; -import org.apache.cloudstack.api.command.admin.region.AddRegionCmd; -import org.apache.cloudstack.api.command.admin.region.RemoveRegionCmd; -import org.apache.cloudstack.api.command.admin.region.UpdateRegionCmd; -import org.apache.cloudstack.api.command.admin.swift.AddSwiftCmd; -import org.apache.cloudstack.api.command.admin.swift.ListSwiftsCmd; -import org.apache.cloudstack.api.command.admin.template.PrepareTemplateCmd; -import org.apache.cloudstack.api.command.admin.vlan.CreateVlanIpRangeCmd; -import org.apache.cloudstack.api.command.admin.vlan.DeleteVlanIpRangeCmd; -import org.apache.cloudstack.api.command.admin.vlan.ListVlanIpRangesCmd; -import org.apache.cloudstack.api.command.admin.vm.AssignVMCmd; -import org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd; -import org.apache.cloudstack.api.command.admin.vm.MigrateVirtualMachineWithVolumeCmd; -import org.apache.cloudstack.api.command.admin.vm.RecoverVMCmd; -import org.apache.cloudstack.api.command.admin.zone.CreateZoneCmd; -import org.apache.cloudstack.api.command.admin.zone.DeleteZoneCmd; -import org.apache.cloudstack.api.command.admin.zone.MarkDefaultZoneForAccountCmd; -import org.apache.cloudstack.api.command.admin.zone.UpdateZoneCmd; -import org.apache.cloudstack.api.command.user.account.AddAccountToProjectCmd; -import org.apache.cloudstack.api.command.user.account.DeleteAccountFromProjectCmd; -import org.apache.cloudstack.api.command.user.account.ListAccountsCmd; -import org.apache.cloudstack.api.command.user.account.ListProjectAccountsCmd; -import org.apache.cloudstack.api.command.user.address.AssociateIPAddrCmd; -import org.apache.cloudstack.api.command.user.address.DisassociateIPAddrCmd; -import org.apache.cloudstack.api.command.user.address.ListPublicIpAddressesCmd; -import org.apache.cloudstack.api.command.user.config.ListCapabilitiesCmd; -import org.apache.cloudstack.api.command.user.event.ArchiveEventsCmd; -import org.apache.cloudstack.api.command.user.event.DeleteEventsCmd; -import org.apache.cloudstack.api.command.user.event.ListEventTypesCmd; -import org.apache.cloudstack.api.command.user.event.ListEventsCmd; -import org.apache.cloudstack.api.command.user.guest.ListGuestOsCategoriesCmd; -import org.apache.cloudstack.api.command.user.guest.ListGuestOsCmd; -import org.apache.cloudstack.api.command.user.job.ListAsyncJobsCmd; -import org.apache.cloudstack.api.command.user.job.QueryAsyncJobResultCmd; -import org.apache.cloudstack.api.command.user.offering.ListDiskOfferingsCmd; -import org.apache.cloudstack.api.command.user.offering.ListServiceOfferingsCmd; -import org.apache.cloudstack.api.command.user.region.ListRegionsCmd; -import org.apache.cloudstack.api.command.user.region.ha.gslb.*; -import org.apache.cloudstack.api.command.user.ssh.CreateSSHKeyPairCmd; -import org.apache.cloudstack.api.command.user.ssh.DeleteSSHKeyPairCmd; -import org.apache.cloudstack.api.command.user.ssh.ListSSHKeyPairsCmd; -import org.apache.cloudstack.api.command.user.ssh.RegisterSSHKeyPairCmd; -import org.apache.cloudstack.api.command.user.tag.CreateTagsCmd; -import org.apache.cloudstack.api.command.user.tag.DeleteTagsCmd; -import org.apache.cloudstack.api.command.user.tag.ListTagsCmd; -import org.apache.cloudstack.api.command.user.vmgroup.CreateVMGroupCmd; -import org.apache.cloudstack.api.command.user.vmgroup.DeleteVMGroupCmd; -import org.apache.cloudstack.api.command.user.vmgroup.ListVMGroupsCmd; -import org.apache.cloudstack.api.command.user.vmgroup.UpdateVMGroupCmd; -import org.apache.cloudstack.api.command.user.vmsnapshot.CreateVMSnapshotCmd; -import org.apache.cloudstack.api.command.user.vmsnapshot.DeleteVMSnapshotCmd; -import org.apache.cloudstack.api.command.user.vmsnapshot.ListVMSnapshotCmd; -import org.apache.cloudstack.api.command.user.vmsnapshot.RevertToVMSnapshotCmd; -import org.apache.cloudstack.api.command.user.zone.ListZonesByCmd; -import org.apache.cloudstack.engine.subsystem.api.storage.DataStoreManager; -import org.apache.cloudstack.engine.subsystem.api.storage.StoragePoolAllocator; -import org.apache.cloudstack.storage.datastore.db.PrimaryDataStoreDao; -import org.apache.cloudstack.storage.datastore.db.StoragePoolVO; + public class ManagementServerImpl extends ManagerBase implements ManagementServer { public static final Logger s_logger = Logger.getLogger(ManagementServerImpl.class.getName()); @@ -2542,11 +2862,21 @@ public class ManagementServerImpl extends ManagerBase implements ManagementServe cmdList.add(AssignToGlobalLoadBalancerRuleCmd.class); cmdList.add(RemoveFromGlobalLoadBalancerRuleCmd.class); cmdList.add(ListStorageProvidersCmd.class); + cmdList.add(CreateApplicationLoadBalancerCmd.class); + cmdList.add(ListApplicationLoadBalancersCmd.class); + cmdList.add(DeleteApplicationLoadBalancerCmd.class); + cmdList.add(ConfigureInternalLoadBalancerElementCmd.class); + cmdList.add(CreateInternalLoadBalancerElementCmd.class); + cmdList.add(ListInternalLoadBalancerElementsCmd.class); cmdList.add(CreateAffinityGroupCmd.class); cmdList.add(DeleteAffinityGroupCmd.class); cmdList.add(ListAffinityGroupsCmd.class); cmdList.add(UpdateVMAffinityGroupCmd.class); cmdList.add(ListAffinityGroupTypesCmd.class); + cmdList.add(StopInternalLBVMCmd.class); + cmdList.add(StartInternalLBVMCmd.class); + cmdList.add(ListInternalLBVMsCmd.class); + cmdList.add(ListNetworkIsolationMethodsCmd.class); cmdList.add(ListNetworkIsolationMethodsCmd.class); return cmdList; diff --git a/server/src/org/apache/cloudstack/network/lb/ApplicationLoadBalancerManagerImpl.java b/server/src/org/apache/cloudstack/network/lb/ApplicationLoadBalancerManagerImpl.java new file mode 100644 index 00000000000..ec0be8c9d96 --- /dev/null +++ b/server/src/org/apache/cloudstack/network/lb/ApplicationLoadBalancerManagerImpl.java @@ -0,0 +1,524 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.cloudstack.network.lb; + +import java.util.ArrayList; +import java.util.List; +import java.util.Map; + +import javax.ejb.Local; +import javax.inject.Inject; + +import org.apache.cloudstack.acl.SecurityChecker.AccessType; +import org.apache.cloudstack.api.command.user.loadbalancer.ListApplicationLoadBalancersCmd; +import org.apache.cloudstack.lb.ApplicationLoadBalancerRuleVO; +import org.apache.cloudstack.lb.dao.ApplicationLoadBalancerRuleDao; +import org.apache.log4j.Logger; +import org.springframework.stereotype.Component; + +import com.cloud.event.ActionEvent; +import com.cloud.event.EventTypes; +import com.cloud.event.UsageEventUtils; +import com.cloud.exception.InsufficientAddressCapacityException; +import com.cloud.exception.InsufficientVirtualNetworkCapcityException; +import com.cloud.exception.InvalidParameterValueException; +import com.cloud.exception.NetworkRuleConflictException; +import com.cloud.exception.UnsupportedServiceException; +import com.cloud.network.Network; +import com.cloud.network.Network.Capability; +import com.cloud.network.Network.Service; +import com.cloud.network.NetworkManager; +import com.cloud.network.NetworkModel; +import com.cloud.network.Networks.TrafficType; +import com.cloud.network.dao.FirewallRulesDao; +import com.cloud.network.lb.LoadBalancingRule; +import com.cloud.network.lb.LoadBalancingRule.LbDestination; +import com.cloud.network.lb.LoadBalancingRule.LbHealthCheckPolicy; +import com.cloud.network.lb.LoadBalancingRule.LbStickinessPolicy; +import com.cloud.network.lb.LoadBalancingRulesManager; +import com.cloud.network.rules.FirewallRule.State; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; +import com.cloud.projects.Project.ListProjectResourcesCriteria; +import com.cloud.server.ResourceTag.TaggedResourceType; +import com.cloud.tags.ResourceTagVO; +import com.cloud.tags.dao.ResourceTagDao; +import com.cloud.user.Account; +import com.cloud.user.AccountManager; +import com.cloud.user.UserContext; +import com.cloud.utils.Pair; +import com.cloud.utils.Ternary; +import com.cloud.utils.component.ManagerBase; +import com.cloud.utils.db.DB; +import com.cloud.utils.db.Filter; +import com.cloud.utils.db.JoinBuilder; +import com.cloud.utils.db.SearchBuilder; +import com.cloud.utils.db.SearchCriteria; +import com.cloud.utils.db.Transaction; +import com.cloud.utils.exception.CloudRuntimeException; +import com.cloud.utils.net.Ip; +import com.cloud.utils.net.NetUtils; + +@Component +@Local(value = { ApplicationLoadBalancerService.class }) +public class ApplicationLoadBalancerManagerImpl extends ManagerBase implements ApplicationLoadBalancerService { + private static final Logger s_logger = Logger.getLogger(ApplicationLoadBalancerManagerImpl.class); + + @Inject NetworkModel _networkModel; + @Inject ApplicationLoadBalancerRuleDao _lbDao; + @Inject AccountManager _accountMgr; + @Inject LoadBalancingRulesManager _lbMgr; + @Inject FirewallRulesDao _firewallDao; + @Inject ResourceTagDao _resourceTagDao; + @Inject NetworkManager _ntwkMgr; + + + @Override + @ActionEvent(eventType = EventTypes.EVENT_LOAD_BALANCER_CREATE, eventDescription = "creating load balancer") + public ApplicationLoadBalancerRule createApplicationLoadBalancer(String name, String description, Scheme scheme, long sourceIpNetworkId, String sourceIp, + int sourcePort, int instancePort, String algorithm, long networkId, long lbOwnerId) throws InsufficientAddressCapacityException, + NetworkRuleConflictException, InsufficientVirtualNetworkCapcityException { + + //Validate LB rule guest network + Network guestNtwk = _networkModel.getNetwork(networkId); + if (guestNtwk == null || guestNtwk.getTrafficType() != TrafficType.Guest) { + throw new InvalidParameterValueException("Can't find guest network by id"); + } + + Account caller = UserContext.current().getCaller(); + _accountMgr.checkAccess(caller, AccessType.UseNetwork, false, guestNtwk); + + Network sourceIpNtwk = _networkModel.getNetwork(sourceIpNetworkId); + if (sourceIpNtwk == null) { + throw new InvalidParameterValueException("Can't find source ip network by id"); + } + + Account lbOwner = _accountMgr.getAccount(lbOwnerId); + if (lbOwner == null) { + throw new InvalidParameterValueException("Can't find the lb owner account"); + } + + return createApplicationLoadBalancer(name, description, scheme, sourceIpNtwk, sourceIp, sourcePort, instancePort, algorithm, lbOwner, guestNtwk); + } + + + protected ApplicationLoadBalancerRule createApplicationLoadBalancer(String name, String description, Scheme scheme, Network sourceIpNtwk, String sourceIp, int sourcePort, int instancePort, String algorithm, + Account lbOwner, Network guestNtwk) throws NetworkRuleConflictException, InsufficientVirtualNetworkCapcityException { + + //Only Internal scheme is supported in this release + if (scheme != Scheme.Internal) { + throw new UnsupportedServiceException("Only scheme of type " + Scheme.Internal + " is supported"); + } + + //1) Validate LB rule's parameters + validateLbRule(sourcePort, instancePort, algorithm, guestNtwk, scheme); + + //2) Validate source network + validateSourceIpNtwkForLbRule(sourceIpNtwk, scheme); + + //3) Get source ip address + Ip sourceIpAddr = getSourceIp(scheme, sourceIpNtwk, sourceIp); + + ApplicationLoadBalancerRuleVO newRule = new ApplicationLoadBalancerRuleVO(name, description, sourcePort, instancePort, algorithm, guestNtwk.getId(), + lbOwner.getId(), lbOwner.getDomainId(), sourceIpAddr, sourceIpNtwk.getId(), scheme); + + //4) Validate Load Balancing rule on the providers + LoadBalancingRule loadBalancing = new LoadBalancingRule(newRule, new ArrayList(), + new ArrayList(), new ArrayList(), sourceIpAddr); + if (!_lbMgr.validateLbRule(loadBalancing)) { + throw new InvalidParameterValueException("LB service provider cannot support this rule"); + } + + //5) Persist Load Balancer rule + return persistLbRule(newRule); + } + + + @DB + protected ApplicationLoadBalancerRule persistLbRule(ApplicationLoadBalancerRuleVO newRule) throws NetworkRuleConflictException { + + Transaction txn = Transaction.currentTxn(); + txn.start(); + + //1) Persist the rule + newRule = _lbDao.persist(newRule); + boolean success = true; + + try { + //2) Detect conflicts + detectLbRulesConflicts(newRule); + if (!_firewallDao.setStateToAdd(newRule)) { + throw new CloudRuntimeException("Unable to update the state to add for " + newRule); + } + s_logger.debug("Load balancer " + newRule.getId() + " for Ip address " + newRule.getSourceIp().addr() + ", source port " + + newRule.getSourcePortStart() + ", instance port " + newRule.getDefaultPortStart() + " is added successfully."); + UserContext.current().setEventDetails("Load balancer Id: " + newRule.getId()); + Network ntwk = _networkModel.getNetwork(newRule.getNetworkId()); + UsageEventUtils.publishUsageEvent(EventTypes.EVENT_LOAD_BALANCER_CREATE, newRule.getAccountId(), + ntwk.getDataCenterId(), newRule.getId(), null, LoadBalancingRule.class.getName(), + newRule.getUuid()); + txn.commit(); + + return newRule; + } catch (Exception e) { + success = false; + if (e instanceof NetworkRuleConflictException) { + throw (NetworkRuleConflictException) e; + } + throw new CloudRuntimeException("Unable to add lb rule for ip address " + newRule.getSourceIpAddressId(), e); + } finally { + if (!success && newRule != null) { + _lbMgr.removeLBRule(newRule); + } + } + } + + /** + * Validates Lb rule parameters + * @param sourcePort + * @param instancePort + * @param algorithm + * @param network + * @param scheme TODO + * @param networkId + */ + protected void validateLbRule(int sourcePort, int instancePort, String algorithm, Network network, Scheme scheme) { + //1) verify that lb service is supported by the network + if (!_networkModel.areServicesSupportedInNetwork(network.getId(), Service.Lb)) { + InvalidParameterValueException ex = new InvalidParameterValueException( + "LB service is not supported in specified network id"); + ex.addProxyObject(network, network.getId(), "networkId"); + throw ex; + } + + //2) verify that lb service is supported by the network + _lbMgr.isLbServiceSupportedInNetwork(network.getId(), scheme); + + Map caps = _networkModel.getNetworkServiceCapabilities(network.getId(), Service.Lb); + String supportedProtocols = caps.get(Capability.SupportedProtocols).toLowerCase(); + if (!supportedProtocols.contains(NetUtils.TCP_PROTO.toLowerCase())) { + throw new InvalidParameterValueException("Protocol " + NetUtils.TCP_PROTO.toLowerCase() + " is not supported in zone " + network.getDataCenterId()); + } + + //3) Validate rule parameters + if (!NetUtils.isValidPort(instancePort)) { + throw new InvalidParameterValueException("Invalid value for instance port: " + instancePort); + } + + if (!NetUtils.isValidPort(sourcePort)) { + throw new InvalidParameterValueException("Invalid value for source port: " + sourcePort); + } + + if ((algorithm == null) || !NetUtils.isValidAlgorithm(algorithm)) { + throw new InvalidParameterValueException("Invalid algorithm: " + algorithm); + } + } + + + /** + * Gets source ip address based on the LB rule scheme/source IP network/requested IP address + * @param scheme + * @param sourceIpNtwk + * @param requestedIp + * @return + * @throws InsufficientVirtualNetworkCapcityException + */ + protected Ip getSourceIp(Scheme scheme, Network sourceIpNtwk, String requestedIp) throws InsufficientVirtualNetworkCapcityException { + + if (requestedIp != null) { + if (_lbDao.countBySourceIp(new Ip(requestedIp), sourceIpNtwk.getId()) > 0) { + s_logger.debug("IP address " + requestedIp + " is already used by existing LB rule, returning it"); + return new Ip(requestedIp); + } + + validateRequestedSourceIpForLbRule(sourceIpNtwk, new Ip(requestedIp), scheme); + } + + requestedIp = allocateSourceIpForLbRule(scheme, sourceIpNtwk, requestedIp); + + if (requestedIp == null) { + throw new InsufficientVirtualNetworkCapcityException("Unable to acquire IP address for network " + sourceIpNtwk, Network.class, sourceIpNtwk.getId()); + } + return new Ip(requestedIp); + } + + + /** + * Allocates new Source IP address for the Load Balancer rule based on LB rule scheme/sourceNetwork + * @param scheme + * @param sourceIpNtwk + * @param requestedIp TODO + * @param sourceIp + * @return + */ + protected String allocateSourceIpForLbRule(Scheme scheme, Network sourceIpNtwk, String requestedIp) { + String sourceIp = null; + if (scheme != Scheme.Internal) { + throw new InvalidParameterValueException("Only scheme " + Scheme.Internal + " is supported"); + } else { + sourceIp = allocateSourceIpForInternalLbRule(sourceIpNtwk, requestedIp); + } + return sourceIp; + } + + + /** + * Allocates sourceIp for the Internal LB rule + * @param sourceIpNtwk + * @param requestedIp TODO + * @return + */ + protected String allocateSourceIpForInternalLbRule(Network sourceIpNtwk, String requestedIp) { + return _ntwkMgr.acquireGuestIpAddress(sourceIpNtwk, requestedIp); + } + + + /** + * Validates requested source ip address of the LB rule based on Lb rule scheme/sourceNetwork + * @param sourceIpNtwk + * @param requestedSourceIp + * @param scheme + */ + void validateRequestedSourceIpForLbRule(Network sourceIpNtwk, Ip requestedSourceIp, Scheme scheme) { + //only Internal scheme is supported in this release + if (scheme != Scheme.Internal) { + throw new UnsupportedServiceException("Only scheme of type " + Scheme.Internal + " is supported"); + } else { + //validate guest source ip + validateRequestedSourceIpForInternalLbRule(sourceIpNtwk, requestedSourceIp); + } + } + + + /** + * Validates requested source IP address of Internal Lb rule against sourceNetworkId + * @param sourceIpNtwk + * @param requestedSourceIp + */ + protected void validateRequestedSourceIpForInternalLbRule(Network sourceIpNtwk, Ip requestedSourceIp) { + //Check if the IP is within the network cidr + Pair cidr = NetUtils.getCidr(sourceIpNtwk.getCidr()); + if (!NetUtils.getCidrSubNet(requestedSourceIp.addr(), cidr.second()).equalsIgnoreCase(NetUtils.getCidrSubNet(cidr.first(), cidr.second()))) { + throw new InvalidParameterValueException("The requested IP is not in the network's CIDR subnet."); + } + } + + + /** + * Validates source IP network for the LB rule + * @param sourceNtwk + * @param scheme + * @return + */ + protected Network validateSourceIpNtwkForLbRule(Network sourceNtwk, Scheme scheme) { + //only Internal scheme is supported in this release + if (scheme != Scheme.Internal) { + throw new UnsupportedServiceException("Only scheme of type " + Scheme.Internal + " is supported"); + } else { + //validate source ip network + return validateSourceIpNtwkForInternalLbRule(sourceNtwk); + } + + } + + /** + * Validates source IP network for the Internal LB rule + * @param sourceIpNtwk + * @return + */ + protected Network validateSourceIpNtwkForInternalLbRule(Network sourceIpNtwk) { + if (sourceIpNtwk.getTrafficType() != TrafficType.Guest) { + throw new InvalidParameterValueException("Only traffic type " + TrafficType.Guest + " is supported"); + } + + //Can't create the LB rule if the network's cidr is NULL + String ntwkCidr = sourceIpNtwk.getCidr(); + if (ntwkCidr == null) { + throw new InvalidParameterValueException("Can't create the application load balancer rule for the network having NULL cidr"); + } + + //check if the requested ip address is within the cidr + return sourceIpNtwk; + } + + + @Override + public boolean deleteApplicationLoadBalancer(long id) { + return _lbMgr.deleteLoadBalancerRule(id, true); + } + + @Override + public Pair, Integer> listApplicationLoadBalancers(ListApplicationLoadBalancersCmd cmd) { + Long id = cmd.getId(); + String name = cmd.getLoadBalancerRuleName(); + String ip = cmd.getSourceIp(); + Long ipNtwkId = cmd.getSourceIpNetworkId(); + String keyword = cmd.getKeyword(); + Scheme scheme = cmd.getScheme(); + Long networkId = cmd.getNetworkId(); + + Map tags = cmd.getTags(); + + Account caller = UserContext.current().getCaller(); + List permittedAccounts = new ArrayList(); + + Ternary domainIdRecursiveListProject = new Ternary( + cmd.getDomainId(), cmd.isRecursive(), null); + _accountMgr.buildACLSearchParameters(caller, id, cmd.getAccountName(), cmd.getProjectId(), permittedAccounts, + domainIdRecursiveListProject, cmd.listAll(), false); + Long domainId = domainIdRecursiveListProject.first(); + Boolean isRecursive = domainIdRecursiveListProject.second(); + ListProjectResourcesCriteria listProjectResourcesCriteria = domainIdRecursiveListProject.third(); + + Filter searchFilter = new Filter(ApplicationLoadBalancerRuleVO.class, "id", true, cmd.getStartIndex(), cmd.getPageSizeVal()); + SearchBuilder sb = _lbDao.createSearchBuilder(); + _accountMgr.buildACLSearchBuilder(sb, domainId, isRecursive, permittedAccounts, listProjectResourcesCriteria); + + sb.and("id", sb.entity().getId(), SearchCriteria.Op.EQ); + sb.and("name", sb.entity().getName(), SearchCriteria.Op.EQ); + sb.and("sourceIpAddress", sb.entity().getSourceIp(), SearchCriteria.Op.EQ); + sb.and("sourceIpAddressNetworkId", sb.entity().getSourceIpNetworkId(), SearchCriteria.Op.EQ); + sb.and("scheme", sb.entity().getScheme(), SearchCriteria.Op.EQ); + sb.and("networkId", sb.entity().getNetworkId(), SearchCriteria.Op.EQ); + + //list only load balancers having not null sourceIp/sourceIpNtwkId + sb.and("sourceIpAddress", sb.entity().getSourceIp(), SearchCriteria.Op.NNULL); + sb.and("sourceIpAddressNetworkId", sb.entity().getSourceIpNetworkId(), SearchCriteria.Op.NNULL); + + if (tags != null && !tags.isEmpty()) { + SearchBuilder tagSearch = _resourceTagDao.createSearchBuilder(); + for (int count = 0; count < tags.size(); count++) { + tagSearch.or().op("key" + String.valueOf(count), tagSearch.entity().getKey(), SearchCriteria.Op.EQ); + tagSearch.and("value" + String.valueOf(count), tagSearch.entity().getValue(), SearchCriteria.Op.EQ); + tagSearch.cp(); + } + tagSearch.and("resourceType", tagSearch.entity().getResourceType(), SearchCriteria.Op.EQ); + sb.groupBy(sb.entity().getId()); + sb.join("tagSearch", tagSearch, sb.entity().getId(), tagSearch.entity().getResourceId(), + JoinBuilder.JoinType.INNER); + } + + SearchCriteria sc = sb.create(); + _accountMgr.buildACLSearchCriteria(sc, domainId, isRecursive, permittedAccounts, listProjectResourcesCriteria); + + if (keyword != null) { + SearchCriteria ssc = _lbDao.createSearchCriteria(); + ssc.addOr("name", SearchCriteria.Op.LIKE, "%" + keyword + "%"); + ssc.addOr("description", SearchCriteria.Op.LIKE, "%" + keyword + "%"); + sc.addAnd("name", SearchCriteria.Op.SC, ssc); + } + + if (name != null) { + sc.setParameters("name", name); + } + + if (id != null) { + sc.setParameters("id", id); + } + + if (ip != null) { + sc.setParameters("sourceIpAddress", ip); + } + + if (ipNtwkId != null) { + sc.setParameters("sourceIpAddressNetworkId", ipNtwkId); + } + + if (scheme != null) { + sc.setParameters("scheme", scheme); + } + + if (networkId != null) { + sc.setParameters("networkId", networkId); + } + + if (tags != null && !tags.isEmpty()) { + int count = 0; + sc.setJoinParameters("tagSearch", "resourceType", TaggedResourceType.LoadBalancer.toString()); + for (String key : tags.keySet()) { + sc.setJoinParameters("tagSearch", "key" + String.valueOf(count), key); + sc.setJoinParameters("tagSearch", "value" + String.valueOf(count), tags.get(key)); + count++; + } + } + + Pair, Integer> result = _lbDao.searchAndCount(sc, searchFilter); + return new Pair, Integer>(result.first(), result.second()); + } + + @Override + public ApplicationLoadBalancerRule getApplicationLoadBalancer(long ruleId) { + ApplicationLoadBalancerRule lbRule = _lbDao.findById(ruleId); + if (lbRule == null) { + throw new InvalidParameterValueException("Can't find the load balancer by id"); + } + return lbRule; + } + + + /** + * Detects lb rule conflicts against other rules + * @param newLbRule + * @throws NetworkRuleConflictException + */ + protected void detectLbRulesConflicts(ApplicationLoadBalancerRule newLbRule) throws NetworkRuleConflictException { + if (newLbRule.getScheme() != Scheme.Internal) { + throw new UnsupportedServiceException("Only scheme of type " + Scheme.Internal + " is supported"); + } else { + detectInternalLbRulesConflict(newLbRule); + } + } + + + /** + * Detects Internal Lb Rules conflicts + * @param newLbRule + * @throws NetworkRuleConflictException + */ + protected void detectInternalLbRulesConflict(ApplicationLoadBalancerRule newLbRule) throws NetworkRuleConflictException { + List lbRules = _lbDao.listBySourceIpAndNotRevoked(newLbRule.getSourceIp(), newLbRule.getSourceIpNetworkId()); + + for (ApplicationLoadBalancerRuleVO lbRule : lbRules) { + if (lbRule.getId() == newLbRule.getId()) { + continue; // Skips my own rule. + } + + if (lbRule.getNetworkId() != newLbRule.getNetworkId() && lbRule.getState() != State.Revoke) { + throw new NetworkRuleConflictException("New rule is for a different network than what's specified in rule " + + lbRule.getXid()); + } + + if ((lbRule.getSourcePortStart().intValue() <= newLbRule.getSourcePortStart().intValue() + && lbRule.getSourcePortEnd().intValue() >= newLbRule.getSourcePortStart().intValue()) + || (lbRule.getSourcePortStart().intValue() <= newLbRule.getSourcePortEnd().intValue() + && lbRule.getSourcePortEnd().intValue() >= newLbRule.getSourcePortEnd().intValue()) + || (newLbRule.getSourcePortStart().intValue() <= lbRule.getSourcePortStart().intValue() + && newLbRule.getSourcePortEnd().intValue() >= lbRule.getSourcePortStart().intValue()) + || (newLbRule.getSourcePortStart().intValue() <= lbRule.getSourcePortEnd().intValue() + && newLbRule.getSourcePortEnd().intValue() >= lbRule.getSourcePortEnd().intValue())) { + + + throw new NetworkRuleConflictException("The range specified, " + newLbRule.getSourcePortStart() + "-" + newLbRule.getSourcePortEnd() + ", conflicts with rule " + lbRule.getId() + + " which has " + lbRule.getSourcePortStart() + "-" + lbRule.getSourcePortEnd()); + } + } + + if (s_logger.isDebugEnabled()) { + s_logger.debug("No network rule conflicts detected for " + newLbRule + " against " + (lbRules.size() - 1) + " existing rules"); + } + } +} diff --git a/server/test/com/cloud/network/MockNetworkManagerImpl.java b/server/test/com/cloud/network/MockNetworkManagerImpl.java index 2f717c8c156..eb5fc253784 100755 --- a/server/test/com/cloud/network/MockNetworkManagerImpl.java +++ b/server/test/com/cloud/network/MockNetworkManagerImpl.java @@ -16,13 +16,33 @@ // under the License. package com.cloud.network; +import java.util.List; +import java.util.Map; +import java.util.Set; + +import javax.ejb.Local; +import javax.naming.ConfigurationException; + +import org.apache.cloudstack.acl.ControlledEntity.ACLType; +import org.apache.cloudstack.api.command.admin.usage.ListTrafficTypeImplementorsCmd; +import org.apache.cloudstack.api.command.user.network.CreateNetworkCmd; +import org.apache.cloudstack.api.command.user.network.ListNetworksCmd; +import org.apache.cloudstack.api.command.user.network.RestartNetworkCmd; +import org.apache.cloudstack.api.command.user.vm.ListNicsCmd; +import org.springframework.stereotype.Component; + import com.cloud.dc.DataCenter; import com.cloud.dc.Pod; import com.cloud.dc.Vlan.VlanType; import com.cloud.deploy.DataCenterDeployment; import com.cloud.deploy.DeployDestination; import com.cloud.deploy.DeploymentPlan; -import com.cloud.exception.*; +import com.cloud.exception.ConcurrentOperationException; +import com.cloud.exception.InsufficientAddressCapacityException; +import com.cloud.exception.InsufficientCapacityException; +import com.cloud.exception.InsufficientVirtualNetworkCapcityException; +import com.cloud.exception.ResourceAllocationException; +import com.cloud.exception.ResourceUnavailableException; import com.cloud.network.Network.Provider; import com.cloud.network.Network.Service; import com.cloud.network.Networks.TrafficType; @@ -37,6 +57,7 @@ import com.cloud.network.guru.NetworkGuru; import com.cloud.network.rules.FirewallRule; import com.cloud.network.rules.FirewallRule.Purpose; import com.cloud.network.rules.FirewallRule.State; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; import com.cloud.network.rules.StaticNat; import com.cloud.offering.NetworkOffering; import com.cloud.offerings.NetworkOfferingVO; @@ -46,6 +67,7 @@ import com.cloud.utils.Pair; import com.cloud.utils.component.ManagerBase; import com.cloud.vm.*; import com.cloud.vm.VirtualMachine.Type; +import com.cloud.vm.VirtualMachineProfile; import org.apache.cloudstack.acl.ControlledEntity.ACLType; import org.apache.cloudstack.api.command.admin.network.DedicateGuestVlanRangeCmd; import org.apache.cloudstack.api.command.admin.network.ListDedicatedGuestVlanRangesCmd; @@ -62,6 +84,7 @@ import java.util.List; import java.util.Map; import java.util.Set; + @Component @Local(value = { NetworkManager.class, NetworkService.class }) public class MockNetworkManagerImpl extends ManagerBase implements NetworkManager, NetworkService { @@ -820,7 +843,7 @@ public class MockNetworkManagerImpl extends ManagerBase implements NetworkManage } @Override - public LoadBalancingServiceProvider getLoadBalancingProviderForNetwork(Network network) { + public LoadBalancingServiceProvider getLoadBalancingProviderForNetwork(Network network, Scheme lbScheme) { // TODO Auto-generated method stub return null; } diff --git a/server/test/com/cloud/network/MockNetworkModelImpl.java b/server/test/com/cloud/network/MockNetworkModelImpl.java index 7df45a01715..c3a0d6c5ae9 100644 --- a/server/test/com/cloud/network/MockNetworkModelImpl.java +++ b/server/test/com/cloud/network/MockNetworkModelImpl.java @@ -40,6 +40,7 @@ import com.cloud.network.dao.NetworkVO; import com.cloud.network.element.NetworkElement; import com.cloud.network.element.UserDataServiceProvider; import com.cloud.offering.NetworkOffering; +import com.cloud.offering.NetworkOffering.Detail; import com.cloud.offerings.NetworkOfferingVO; import com.cloud.user.Account; import com.cloud.utils.component.ManagerBase; @@ -853,6 +854,22 @@ public class MockNetworkModelImpl extends ManagerBase implements NetworkModel { } @Override + public IpAddress getPublicIpAddress(String ipAddress, long zoneId) { + // TODO Auto-generated method stub + return null; + } + + @Override + public List getUsedIpsInNetwork(Network network) { + // TODO Auto-generated method stub + return null; + } + + @Override + public Map getNtwkOffDetails(long offId) { + return null; + } + public IsolationType[] listNetworkIsolationMethods() { // TODO Auto-generated method stub return null; diff --git a/server/test/com/cloud/vpc/MockConfigurationManagerImpl.java b/server/test/com/cloud/vpc/MockConfigurationManagerImpl.java index 83b19247093..90587985f74 100755 --- a/server/test/com/cloud/vpc/MockConfigurationManagerImpl.java +++ b/server/test/com/cloud/vpc/MockConfigurationManagerImpl.java @@ -501,7 +501,7 @@ public class MockConfigurationManagerImpl extends ManagerBase implements Configu @Override public NetworkOfferingVO createNetworkOffering(String name, String displayText, TrafficType trafficType, String tags, boolean specifyVlan, Availability availability, Integer networkRate, Map> serviceProviderMap, boolean isDefault, GuestType type, boolean systemOnly, Long serviceOfferingId, boolean conserveMode, - Map> serviceCapabilityMap, boolean specifyIpRanges, boolean isPersistent) { + Map> serviceCapabilityMap, boolean specifyIpRanges, boolean isPersistent, Map details) { // TODO Auto-generated method stub return null; } diff --git a/server/test/com/cloud/vpc/MockNetworkManagerImpl.java b/server/test/com/cloud/vpc/MockNetworkManagerImpl.java index f884ba1d767..84ae818f489 100644 --- a/server/test/com/cloud/vpc/MockNetworkManagerImpl.java +++ b/server/test/com/cloud/vpc/MockNetworkManagerImpl.java @@ -16,17 +16,51 @@ // under the License. package com.cloud.vpc; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + +import javax.ejb.Local; +import javax.inject.Inject; +import javax.naming.ConfigurationException; + +import org.apache.cloudstack.acl.ControlledEntity.ACLType; +import org.apache.cloudstack.api.command.admin.network.DedicateGuestVlanRangeCmd; +import org.apache.cloudstack.api.command.admin.network.ListDedicatedGuestVlanRangesCmd; +import org.apache.cloudstack.api.command.admin.usage.ListTrafficTypeImplementorsCmd; +import org.apache.cloudstack.api.command.user.network.CreateNetworkCmd; +import org.apache.cloudstack.api.command.user.network.ListNetworksCmd; +import org.apache.cloudstack.api.command.user.network.RestartNetworkCmd; +import org.apache.cloudstack.api.command.user.vm.ListNicsCmd; +import org.apache.log4j.Logger; +import org.springframework.stereotype.Component; + import com.cloud.dc.DataCenter; import com.cloud.dc.Pod; import com.cloud.dc.Vlan.VlanType; import com.cloud.deploy.DataCenterDeployment; import com.cloud.deploy.DeployDestination; import com.cloud.deploy.DeploymentPlan; -import com.cloud.exception.*; -import com.cloud.network.*; +import com.cloud.exception.ConcurrentOperationException; +import com.cloud.exception.InsufficientAddressCapacityException; +import com.cloud.exception.InsufficientCapacityException; +import com.cloud.exception.InsufficientVirtualNetworkCapcityException; +import com.cloud.exception.ResourceAllocationException; +import com.cloud.exception.ResourceUnavailableException; +import com.cloud.network.GuestVlan; +import com.cloud.network.IpAddress; +import com.cloud.network.Network; import com.cloud.network.Network.Provider; import com.cloud.network.Network.Service; +import com.cloud.network.NetworkManager; +import com.cloud.network.NetworkProfile; +import com.cloud.network.NetworkRuleApplier; +import com.cloud.network.NetworkService; import com.cloud.network.Networks.TrafficType; +import com.cloud.network.PhysicalNetwork; +import com.cloud.network.PhysicalNetworkServiceProvider; +import com.cloud.network.PhysicalNetworkTrafficType; +import com.cloud.network.PublicIpAddress; import com.cloud.network.addr.PublicIp; import com.cloud.network.dao.AccountGuestVlanMapVO; import com.cloud.network.dao.IPAddressVO; @@ -40,6 +74,7 @@ import com.cloud.network.guru.NetworkGuru; import com.cloud.network.rules.FirewallRule; import com.cloud.network.rules.FirewallRule.Purpose; import com.cloud.network.rules.FirewallRule.State; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; import com.cloud.network.rules.StaticNat; import com.cloud.offering.NetworkOffering; import com.cloud.offerings.NetworkOfferingVO; @@ -48,8 +83,15 @@ import com.cloud.user.Account; import com.cloud.user.User; import com.cloud.utils.Pair; import com.cloud.utils.component.ManagerBase; -import com.cloud.vm.*; +import com.cloud.vm.Nic; +import com.cloud.vm.NicProfile; +import com.cloud.vm.NicSecondaryIp; +import com.cloud.vm.NicVO; +import com.cloud.vm.ReservationContext; +import com.cloud.vm.VMInstanceVO; +import com.cloud.vm.VirtualMachine; import com.cloud.vm.VirtualMachine.Type; +import com.cloud.vm.VirtualMachineProfile; import org.apache.cloudstack.acl.ControlledEntity.ACLType; import org.apache.cloudstack.api.command.admin.network.DedicateGuestVlanRangeCmd; import org.apache.cloudstack.api.command.admin.network.ListDedicatedGuestVlanRangesCmd; @@ -61,12 +103,6 @@ import org.apache.cloudstack.api.command.user.vm.ListNicsCmd; import org.apache.log4j.Logger; import org.springframework.stereotype.Component; -import javax.ejb.Local; -import javax.inject.Inject; -import javax.naming.ConfigurationException; -import java.util.HashMap; -import java.util.List; -import java.util.Map; @Component @Local(value = { NetworkManager.class, NetworkService.class }) @@ -1301,7 +1337,7 @@ public class MockNetworkManagerImpl extends ManagerBase implements NetworkManage } @Override - public LoadBalancingServiceProvider getLoadBalancingProviderForNetwork(Network network) { + public LoadBalancingServiceProvider getLoadBalancingProviderForNetwork(Network network, Scheme lbScheme) { // TODO Auto-generated method stub return null; } diff --git a/server/test/com/cloud/vpc/MockNetworkModelImpl.java b/server/test/com/cloud/vpc/MockNetworkModelImpl.java index c9d043ad0d1..d9e33b75616 100644 --- a/server/test/com/cloud/vpc/MockNetworkModelImpl.java +++ b/server/test/com/cloud/vpc/MockNetworkModelImpl.java @@ -47,6 +47,7 @@ import com.cloud.network.dao.NetworkVO; import com.cloud.network.element.NetworkElement; import com.cloud.network.element.UserDataServiceProvider; import com.cloud.offering.NetworkOffering; +import com.cloud.offering.NetworkOffering.Detail; import com.cloud.offerings.NetworkOfferingVO; import com.cloud.offerings.dao.NetworkOfferingServiceMapDao; import com.cloud.user.Account; @@ -865,6 +866,22 @@ public class MockNetworkModelImpl extends ManagerBase implements NetworkModel { } @Override + public IpAddress getPublicIpAddress(String ipAddress, long zoneId) { + // TODO Auto-generated method stub + return null; + } + + @Override + public List getUsedIpsInNetwork(Network network) { + // TODO Auto-generated method stub + return null; + } + + @Override + public Map getNtwkOffDetails(long offId) { + return null; + } + public IsolationType[] listNetworkIsolationMethods() { // TODO Auto-generated method stub return null; diff --git a/server/test/com/cloud/vpc/MockVpcVirtualNetworkApplianceManager.java b/server/test/com/cloud/vpc/MockVpcVirtualNetworkApplianceManager.java index ef5478bb1f8..9010f1f5acb 100644 --- a/server/test/com/cloud/vpc/MockVpcVirtualNetworkApplianceManager.java +++ b/server/test/com/cloud/vpc/MockVpcVirtualNetworkApplianceManager.java @@ -36,6 +36,7 @@ import com.cloud.network.RemoteAccessVpn; import com.cloud.network.Site2SiteVpnConnection; import com.cloud.network.VpcVirtualNetworkApplianceService; import com.cloud.network.VpnUser; +import com.cloud.network.lb.LoadBalancingRule; import com.cloud.network.router.VirtualRouter; import com.cloud.network.router.VpcVirtualNetworkApplianceManager; import com.cloud.network.rules.FirewallRule; @@ -46,7 +47,6 @@ import com.cloud.network.vpc.Vpc; import com.cloud.user.Account; import com.cloud.user.User; import com.cloud.uservm.UserVm; -import com.cloud.utils.component.Manager; import com.cloud.utils.component.ManagerBase; import com.cloud.vm.DomainRouterVO; import com.cloud.vm.NicProfile; @@ -402,4 +402,16 @@ VpcVirtualNetworkApplianceService { return null; } + @Override + public boolean applyLoadBalancingRules(Network network, List rules, List routers) throws ResourceUnavailableException { + // TODO Auto-generated method stub + return false; + } + + @Override + public VirtualRouter findRouter(long routerId) { + // TODO Auto-generated method stub + return null; + } + } diff --git a/server/test/com/cloud/vpc/dao/MockNetworkOfferingDaoImpl.java b/server/test/com/cloud/vpc/dao/MockNetworkOfferingDaoImpl.java index dbf14113de4..a8208dd7d9c 100644 --- a/server/test/com/cloud/vpc/dao/MockNetworkOfferingDaoImpl.java +++ b/server/test/com/cloud/vpc/dao/MockNetworkOfferingDaoImpl.java @@ -101,28 +101,28 @@ public class MockNetworkOfferingDaoImpl extends NetworkOfferingDaoImpl implement if (id.longValue() == 1) { //network offering valid for vpc vo = new NetworkOfferingVO("vpc", "vpc", TrafficType.Guest, false, true, null, null, false, - Availability.Optional, null, Network.GuestType.Isolated, false, false, false); + Availability.Optional, null, Network.GuestType.Isolated, false, false, false, false, false); } else if (id.longValue() == 2) { //invalid offering - source nat is not included vo = new NetworkOfferingVO("vpc", "vpc", TrafficType.Guest, false, true, null, null, false, - Availability.Optional, null, Network.GuestType.Isolated, false, false, false); + Availability.Optional, null, Network.GuestType.Isolated, false, false, false, false, false); } else if (id.longValue() == 3) { //network offering invalid for vpc (conserve mode off) vo = new NetworkOfferingVO("non vpc", "non vpc", TrafficType.Guest, false, true, null, null, false, - Availability.Optional, null, Network.GuestType.Isolated, true, false, false); + Availability.Optional, null, Network.GuestType.Isolated, true, false, false, false, false); } else if (id.longValue() == 4) { //network offering invalid for vpc (Shared) vo = new NetworkOfferingVO("non vpc", "non vpc", TrafficType.Guest, false, true, null, null, false, - Availability.Optional, null, Network.GuestType.Shared, false, false, false); + Availability.Optional, null, Network.GuestType.Shared, false, false, false, false, false); } else if (id.longValue() == 5) { //network offering invalid for vpc (has redundant router) vo = new NetworkOfferingVO("vpc", "vpc", TrafficType.Guest, false, true, null, null, false, - Availability.Optional, null, Network.GuestType.Isolated, false, false, false); + Availability.Optional, null, Network.GuestType.Isolated, false, false, false, false, false); vo.setRedundantRouter(true); } else if (id.longValue() == 6) { //network offering invalid for vpc (has lb service) vo = new NetworkOfferingVO("vpc", "vpc", TrafficType.Guest, false, true, null, null, false, - Availability.Optional, null, Network.GuestType.Isolated, false, false, false); + Availability.Optional, null, Network.GuestType.Isolated, false, false, false, false, false); } if (vo != null) { diff --git a/server/test/com/cloud/vpc/dao/MockNetworkServiceMapDaoImpl.java b/server/test/com/cloud/vpc/dao/MockNetworkServiceMapDaoImpl.java index 002b61dcbc4..103f04ea8b9 100644 --- a/server/test/com/cloud/vpc/dao/MockNetworkServiceMapDaoImpl.java +++ b/server/test/com/cloud/vpc/dao/MockNetworkServiceMapDaoImpl.java @@ -95,4 +95,10 @@ public class MockNetworkServiceMapDaoImpl extends GenericDaoBase getProvidersForServiceInNetwork(long networkId, Service service) { + // TODO Auto-generated method stub + return null; + } } diff --git a/server/test/org/apache/cloudstack/lb/ApplicationLoadBalancerTest.java b/server/test/org/apache/cloudstack/lb/ApplicationLoadBalancerTest.java new file mode 100644 index 00000000000..461cbbdf012 --- /dev/null +++ b/server/test/org/apache/cloudstack/lb/ApplicationLoadBalancerTest.java @@ -0,0 +1,292 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.lb; + +import java.lang.reflect.Field; +import java.util.HashMap; +import java.util.Map; + +import javax.inject.Inject; + +import junit.framework.TestCase; + +import org.apache.cloudstack.lb.dao.ApplicationLoadBalancerRuleDao; +import org.apache.cloudstack.network.lb.ApplicationLoadBalancerManagerImpl; +import org.apache.cloudstack.network.lb.ApplicationLoadBalancerRule; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.mockito.Mockito; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; + +import com.cloud.event.dao.UsageEventDao; +import com.cloud.exception.InsufficientAddressCapacityException; +import com.cloud.exception.InsufficientVirtualNetworkCapcityException; +import com.cloud.exception.InvalidParameterValueException; +import com.cloud.exception.NetworkRuleConflictException; +import com.cloud.exception.UnsupportedServiceException; +import com.cloud.network.Network; +import com.cloud.network.Network.Capability; +import com.cloud.network.Network.Service; +import com.cloud.network.NetworkModel; +import com.cloud.network.Networks.TrafficType; +import com.cloud.network.dao.FirewallRulesDao; +import com.cloud.network.dao.NetworkVO; +import com.cloud.network.lb.LoadBalancingRule; +import com.cloud.network.lb.LoadBalancingRulesManager; +import com.cloud.network.rules.FirewallRuleVO; +import com.cloud.network.rules.LoadBalancerContainer.Scheme; +import com.cloud.user.AccountManager; +import com.cloud.user.AccountVO; +import com.cloud.user.UserContext; +import com.cloud.user.UserVO; +import com.cloud.utils.component.ComponentContext; +import com.cloud.utils.exception.CloudRuntimeException; +import com.cloud.utils.net.Ip; +import com.cloud.utils.net.NetUtils; + +/** + * This class is responsible for unittesting the methods defined in ApplicationLoadBalancerService + * + */ + +@RunWith(SpringJUnit4ClassRunner.class) +@ContextConfiguration(locations="classpath:/appLoadBalancer.xml") +public class ApplicationLoadBalancerTest extends TestCase{ + //The interface to test + @Inject ApplicationLoadBalancerManagerImpl _appLbSvc; + + //The interfaces below are mocked + @Inject ApplicationLoadBalancerRuleDao _lbDao; + @Inject LoadBalancingRulesManager _lbMgr; + @Inject NetworkModel _ntwkModel; + @Inject AccountManager _accountMgr; + @Inject FirewallRulesDao _firewallDao; + @Inject UsageEventDao _usageEventDao; + + + public static long existingLbId = 1L; + public static long nonExistingLbId = 2L; + + public static long validGuestNetworkId = 1L; + public static long invalidGuestNetworkId = 2L; + public static long validPublicNetworkId = 3L; + + public static long validAccountId = 1L; + public static long invalidAccountId = 2L; + + public String validRequestedIp = "10.1.1.1"; + + + + @Before + public void setUp() { + ComponentContext.initComponentsLifeCycle(); + //mockito for .getApplicationLoadBalancer tests + Mockito.when(_lbDao.findById(1L)).thenReturn(new ApplicationLoadBalancerRuleVO()); + Mockito.when(_lbDao.findById(2L)).thenReturn(null); + + //mockito for .deleteApplicationLoadBalancer tests + Mockito.when(_lbMgr.deleteLoadBalancerRule(existingLbId, true)).thenReturn(true); + Mockito.when(_lbMgr.deleteLoadBalancerRule(nonExistingLbId, true)).thenReturn(false); + + //mockito for .createApplicationLoadBalancer tests + NetworkVO guestNetwork = new NetworkVO(TrafficType.Guest, null, null, 1, + null, 1, 1L); + setId(guestNetwork, validGuestNetworkId); + guestNetwork.setCidr("10.1.1.1/24"); + + NetworkVO publicNetwork = new NetworkVO(TrafficType.Public, null, null, 1, + null, 1, 1L); + + Mockito.when(_ntwkModel.getNetwork(validGuestNetworkId)).thenReturn(guestNetwork); + Mockito.when(_ntwkModel.getNetwork(invalidGuestNetworkId)).thenReturn(null); + Mockito.when(_ntwkModel.getNetwork(validPublicNetworkId)).thenReturn(publicNetwork); + + Mockito.when(_accountMgr.getAccount(validAccountId)).thenReturn(new AccountVO()); + Mockito.when(_accountMgr.getAccount(invalidAccountId)).thenReturn(null); + Mockito.when(_ntwkModel.areServicesSupportedInNetwork(validGuestNetworkId, Service.Lb)).thenReturn(true); + Mockito.when(_ntwkModel.areServicesSupportedInNetwork(invalidGuestNetworkId, Service.Lb)).thenReturn(false); + + ApplicationLoadBalancerRuleVO lbRule = new ApplicationLoadBalancerRuleVO("new", "new", 22, 22, "roundrobin", + validGuestNetworkId, validAccountId, 1L, new Ip(validRequestedIp), validGuestNetworkId, Scheme.Internal); + Mockito.when(_lbDao.persist(Mockito.any(ApplicationLoadBalancerRuleVO.class))).thenReturn(lbRule); + + Mockito.when(_lbMgr.validateLbRule(Mockito.any(LoadBalancingRule.class))).thenReturn(true); + + Mockito.when(_firewallDao.setStateToAdd(Mockito.any(FirewallRuleVO.class))).thenReturn(true); + + Mockito.when(_accountMgr.getSystemUser()).thenReturn(new UserVO(1)); + Mockito.when(_accountMgr.getSystemAccount()).thenReturn(new AccountVO(2)); + UserContext.registerContext(_accountMgr.getSystemUser().getId(), _accountMgr.getSystemAccount(), null, false); + + Mockito.when(_ntwkModel.areServicesSupportedInNetwork(Mockito.anyLong(), Mockito.any(Network.Service.class))).thenReturn(true); + + Map caps = new HashMap(); + caps.put(Capability.SupportedProtocols, NetUtils.TCP_PROTO); + Mockito.when(_ntwkModel.getNetworkServiceCapabilities(Mockito.anyLong(), Mockito.any(Network.Service.class))).thenReturn(caps); + + + Mockito.when(_lbDao.countBySourceIp(new Ip(validRequestedIp), validGuestNetworkId)).thenReturn(1L); + + } + + /** + * TESTS FOR .getApplicationLoadBalancer + */ + + @Test + //Positive test - retrieve existing lb + public void searchForExistingLoadBalancer() { + ApplicationLoadBalancerRule rule = _appLbSvc.getApplicationLoadBalancer(existingLbId); + assertNotNull("Couldn't find existing application load balancer", rule); + } + + @Test + //Negative test - try to retrieve non-existing lb + public void searchForNonExistingLoadBalancer() { + boolean notFound = false; + ApplicationLoadBalancerRule rule = null; + try { + rule = _appLbSvc.getApplicationLoadBalancer(nonExistingLbId); + if (rule != null) { + notFound = false; + } + } catch (InvalidParameterValueException ex) { + notFound = true; + } + + assertTrue("Found non-existing load balancer; no invalid parameter value exception was thrown", notFound); + } + + /** + * TESTS FOR .deleteApplicationLoadBalancer + */ + + + @Test + //Positive test - delete existing lb + public void deleteExistingLoadBalancer() { + boolean result = false; + try { + result = _appLbSvc.deleteApplicationLoadBalancer(existingLbId); + } finally { + assertTrue("Couldn't delete existing application load balancer", result); + } + } + + + @Test + //Negative test - try to delete non-existing lb + public void deleteNonExistingLoadBalancer() { + boolean result = true; + try { + result = _appLbSvc.deleteApplicationLoadBalancer(nonExistingLbId); + } finally { + assertFalse("Didn't fail when try to delete non-existing load balancer", result); + } + } + + /** + * TESTS FOR .createApplicationLoadBalancer + * @throws NetworkRuleConflictException + * @throws InsufficientVirtualNetworkCapcityException + * @throws InsufficientAddressCapacityException + */ + + @Test (expected = CloudRuntimeException.class) + //Positive test + public void createValidLoadBalancer() throws InsufficientAddressCapacityException, + InsufficientVirtualNetworkCapcityException, NetworkRuleConflictException { + _appLbSvc.createApplicationLoadBalancer("alena", "alena", Scheme.Internal, validGuestNetworkId, validRequestedIp, + 22, 22, "roundrobin", validGuestNetworkId, validAccountId); + } + + + @Test(expected = UnsupportedServiceException.class) + //Negative test - only internal scheme value is supported in the current release + public void createPublicLoadBalancer() throws InsufficientAddressCapacityException, + InsufficientVirtualNetworkCapcityException, NetworkRuleConflictException { + _appLbSvc.createApplicationLoadBalancer("alena", "alena", Scheme.Public, validGuestNetworkId, validRequestedIp, + 22, 22, "roundrobin", validGuestNetworkId, validAccountId); + } + + + @Test(expected = InvalidParameterValueException.class) + //Negative test - invalid SourcePort + public void createWithInvalidSourcePort() throws InsufficientAddressCapacityException, + InsufficientVirtualNetworkCapcityException, NetworkRuleConflictException { + _appLbSvc.createApplicationLoadBalancer("alena", "alena", Scheme.Internal, validGuestNetworkId, validRequestedIp, + 65536, 22, "roundrobin", validGuestNetworkId, validAccountId); + } + + @Test(expected = InvalidParameterValueException.class) + //Negative test - invalid instancePort + public void createWithInvalidInstandePort() throws InsufficientAddressCapacityException, + InsufficientVirtualNetworkCapcityException, NetworkRuleConflictException { + _appLbSvc.createApplicationLoadBalancer("alena", "alena", Scheme.Internal, validGuestNetworkId, validRequestedIp, + 22, 65536, "roundrobin", validGuestNetworkId, validAccountId); + + } + + + @Test(expected = InvalidParameterValueException.class) + //Negative test - invalid algorithm + public void createWithInvalidAlgorithm() throws InsufficientAddressCapacityException, InsufficientVirtualNetworkCapcityException, NetworkRuleConflictException { + String expectedExcText = null; + _appLbSvc.createApplicationLoadBalancer("alena", "alena", Scheme.Internal, validGuestNetworkId, validRequestedIp, + 22, 22, "invalidalgorithm", validGuestNetworkId, validAccountId); + + } + + @Test(expected = InvalidParameterValueException.class) + //Negative test - invalid sourceNetworkId (of Public type, which is not supported) + public void createWithInvalidSourceIpNtwk() throws InsufficientAddressCapacityException, + InsufficientVirtualNetworkCapcityException, NetworkRuleConflictException { + _appLbSvc.createApplicationLoadBalancer("alena", "alena", Scheme.Internal, validPublicNetworkId, validRequestedIp, + 22, 22, "roundrobin", validGuestNetworkId, validAccountId); + + } + + + @Test(expected = InvalidParameterValueException.class) + //Negative test - invalid requested IP (outside of guest network cidr range) + public void createWithInvalidRequestedIp() throws InsufficientAddressCapacityException, + InsufficientVirtualNetworkCapcityException, NetworkRuleConflictException { + + _appLbSvc.createApplicationLoadBalancer("alena", "alena", Scheme.Internal, validGuestNetworkId, "10.2.1.1", + 22, 22, "roundrobin", validGuestNetworkId, validAccountId); + } + + + private static NetworkVO setId(NetworkVO vo, long id) { + NetworkVO voToReturn = vo; + Class c = voToReturn.getClass(); + try { + Field f = c.getDeclaredField("id"); + f.setAccessible(true); + f.setLong(voToReturn, id); + } catch (NoSuchFieldException ex) { + return null; + } catch (IllegalAccessException ex) { + return null; + } + + return voToReturn; + } +} diff --git a/server/test/org/apache/cloudstack/lb/ChildTestConfiguration.java b/server/test/org/apache/cloudstack/lb/ChildTestConfiguration.java new file mode 100644 index 00000000000..918de81c0c5 --- /dev/null +++ b/server/test/org/apache/cloudstack/lb/ChildTestConfiguration.java @@ -0,0 +1,103 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.cloudstack.lb; + +import java.io.IOException; + +import org.apache.cloudstack.lb.dao.ApplicationLoadBalancerRuleDao; +import org.apache.cloudstack.test.utils.SpringUtils; +import org.mockito.Mockito; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.ComponentScan; +import org.springframework.context.annotation.ComponentScan.Filter; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.FilterType; +import org.springframework.core.type.classreading.MetadataReader; +import org.springframework.core.type.classreading.MetadataReaderFactory; +import org.springframework.core.type.filter.TypeFilter; + +import com.cloud.dc.dao.AccountVlanMapDaoImpl; +import com.cloud.event.dao.UsageEventDao; +import com.cloud.network.NetworkManager; +import com.cloud.network.NetworkModel; +import com.cloud.network.dao.FirewallRulesDao; +import com.cloud.network.lb.LoadBalancingRulesManager; +import com.cloud.tags.dao.ResourceTagDao; +import com.cloud.user.AccountManager; + +@Configuration +@ComponentScan( + basePackageClasses={ + AccountVlanMapDaoImpl.class + }, + includeFilters={@Filter(value=ChildTestConfiguration.Library.class, type=FilterType.CUSTOM)}, + useDefaultFilters=false + ) + + public class ChildTestConfiguration { + + public static class Library implements TypeFilter { + + @Bean + public ApplicationLoadBalancerRuleDao applicationLoadBalancerDao() { + return Mockito.mock(ApplicationLoadBalancerRuleDao.class); + } + + @Bean + public NetworkModel networkModel() { + return Mockito.mock(NetworkModel.class); + } + + @Bean + public AccountManager accountManager() { + return Mockito.mock(AccountManager.class); + } + + @Bean + public LoadBalancingRulesManager loadBalancingRulesManager() { + return Mockito.mock(LoadBalancingRulesManager.class); + } + + @Bean + public FirewallRulesDao firewallRulesDao() { + return Mockito.mock(FirewallRulesDao.class); + } + + @Bean + public ResourceTagDao resourceTagDao() { + return Mockito.mock(ResourceTagDao.class); + } + + @Bean + public NetworkManager networkManager() { + return Mockito.mock(NetworkManager.class); + } + + @Bean + public UsageEventDao UsageEventDao() { + return Mockito.mock(UsageEventDao.class); + } + + @Override + public boolean match(MetadataReader mdr, MetadataReaderFactory arg1) throws IOException { + mdr.getClassMetadata().getClassName(); + ComponentScan cs = ChildTestConfiguration.class.getAnnotation(ComponentScan.class); + return SpringUtils.includedInBasePackageClasses(mdr.getClassMetadata().getClassName(), cs); + } + + } +} diff --git a/server/test/org/apache/cloudstack/networkoffering/CreateNetworkOfferingTest.java b/server/test/org/apache/cloudstack/networkoffering/CreateNetworkOfferingTest.java index 7946c1aa740..92aa2a2c8ff 100644 --- a/server/test/org/apache/cloudstack/networkoffering/CreateNetworkOfferingTest.java +++ b/server/test/org/apache/cloudstack/networkoffering/CreateNetworkOfferingTest.java @@ -84,6 +84,7 @@ public class CreateNetworkOfferingTest extends TestCase{ Mockito.when(configDao.findByName(Mockito.anyString())).thenReturn(configVO); Mockito.when(offDao.persist(Mockito.any(NetworkOfferingVO.class))).thenReturn(new NetworkOfferingVO()); + Mockito.when(offDao.persist(Mockito.any(NetworkOfferingVO.class), Mockito.anyMap())).thenReturn(new NetworkOfferingVO()); Mockito.when(mapDao.persist(Mockito.any(NetworkOfferingServiceMapVO.class))).thenReturn(new NetworkOfferingServiceMapVO()); Mockito.when(accountMgr.getSystemUser()).thenReturn(new UserVO(1)); Mockito.when(accountMgr.getSystemAccount()).thenReturn(new AccountVO(2)); @@ -96,7 +97,7 @@ public class CreateNetworkOfferingTest extends TestCase{ public void createSharedNtwkOffWithVlan() { NetworkOfferingVO off = configMgr.createNetworkOffering("shared", "shared", TrafficType.Guest, null, true, Availability.Optional, 200, null, false, Network.GuestType.Shared, false, - null, false, null, true, false); + null, false, null, true, false, null); assertNotNull("Shared network offering with specifyVlan=true failed to create ", off); } @@ -105,7 +106,7 @@ public class CreateNetworkOfferingTest extends TestCase{ try { NetworkOfferingVO off = configMgr.createNetworkOffering("shared", "shared", TrafficType.Guest, null, false, Availability.Optional, 200, null, false, Network.GuestType.Shared, false, - null, false, null, true, false); + null, false, null, true, false, null); assertNull("Shared network offering with specifyVlan=false was created", off); } catch (InvalidParameterValueException ex) { } @@ -115,7 +116,7 @@ public class CreateNetworkOfferingTest extends TestCase{ public void createSharedNtwkOffWithSpecifyIpRanges() { NetworkOfferingVO off = configMgr.createNetworkOffering("shared", "shared", TrafficType.Guest, null, true, Availability.Optional, 200, null, false, Network.GuestType.Shared, false, - null, false, null, true, false); + null, false, null, true, false, null); assertNotNull("Shared network offering with specifyIpRanges=true failed to create ", off); } @@ -125,7 +126,7 @@ public class CreateNetworkOfferingTest extends TestCase{ try { NetworkOfferingVO off = configMgr.createNetworkOffering("shared", "shared", TrafficType.Guest, null, true, Availability.Optional, 200, null, false, Network.GuestType.Shared, false, - null, false, null, false, false); + null, false, null, false, false, null); assertNull("Shared network offering with specifyIpRanges=false was created", off); } catch (InvalidParameterValueException ex) { } @@ -140,7 +141,7 @@ public class CreateNetworkOfferingTest extends TestCase{ serviceProviderMap.put(Network.Service.SourceNat, vrProvider); NetworkOfferingVO off = configMgr.createNetworkOffering("isolated", "isolated", TrafficType.Guest, null, false, Availability.Optional, 200, serviceProviderMap, false, Network.GuestType.Isolated, false, - null, false, null, false, false); + null, false, null, false, false, null); assertNotNull("Isolated network offering with specifyIpRanges=false failed to create ", off); } @@ -153,7 +154,7 @@ public class CreateNetworkOfferingTest extends TestCase{ serviceProviderMap.put(Network.Service.SourceNat, vrProvider); NetworkOfferingVO off = configMgr.createNetworkOffering("isolated", "isolated", TrafficType.Guest, null, true, Availability.Optional, 200, serviceProviderMap, false, Network.GuestType.Isolated, false, - null, false, null, false, false); + null, false, null, false, false, null); assertNotNull("Isolated network offering with specifyVlan=true wasn't created", off); } @@ -167,7 +168,7 @@ public class CreateNetworkOfferingTest extends TestCase{ serviceProviderMap.put(Network.Service.SourceNat, vrProvider); NetworkOfferingVO off = configMgr.createNetworkOffering("isolated", "isolated", TrafficType.Guest, null, false, Availability.Optional, 200, serviceProviderMap, false, Network.GuestType.Isolated, false, - null, false, null, true, false); + null, false, null, true, false, null); assertNull("Isolated network offering with specifyIpRanges=true and source nat service enabled, was created", off); } catch (InvalidParameterValueException ex) { } @@ -180,7 +181,7 @@ public class CreateNetworkOfferingTest extends TestCase{ Set vrProvider = new HashSet(); NetworkOfferingVO off = configMgr.createNetworkOffering("isolated", "isolated", TrafficType.Guest, null, false, Availability.Optional, 200, serviceProviderMap, false, Network.GuestType.Isolated, false, - null, false, null, true, false); + null, false, null, true, false, null); assertNotNull("Isolated network offering with specifyIpRanges=true and with no sourceNatService, failed to create", off); } @@ -198,7 +199,7 @@ public class CreateNetworkOfferingTest extends TestCase{ serviceProviderMap.put(Network.Service.Lb , vrProvider); NetworkOfferingVO off = configMgr.createNetworkOffering("isolated", "isolated", TrafficType.Guest, null, true, Availability.Optional, 200, serviceProviderMap, false, Network.GuestType.Isolated, false, - null, false, null, false, false); + null, false, null, false, false, null); // System.out.println("Creating Vpc Network Offering"); assertNotNull("Vpc Isolated network offering with Vpc provider ", off); } @@ -218,7 +219,7 @@ public class CreateNetworkOfferingTest extends TestCase{ serviceProviderMap.put(Network.Service.Lb, lbProvider); NetworkOfferingVO off = configMgr.createNetworkOffering("isolated", "isolated", TrafficType.Guest, null, true, Availability.Optional, 200, serviceProviderMap, false, Network.GuestType.Isolated, false, null, false, - null, false, false); + null, false, false, null); // System.out.println("Creating Vpc Network Offering"); assertNotNull("Vpc Isolated network offering with Vpc and Netscaler provider ", off); } diff --git a/server/test/resources/appLoadBalancer.xml b/server/test/resources/appLoadBalancer.xml new file mode 100644 index 00000000000..d7c1502a715 --- /dev/null +++ b/server/test/resources/appLoadBalancer.xml @@ -0,0 +1,43 @@ + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/setup/db/db/schema-40to410.sql b/setup/db/db/schema-40to410.sql index a3426e2caaa..b7b1c7a91dd 100644 --- a/setup/db/db/schema-40to410.sql +++ b/setup/db/db/schema-40to410.sql @@ -644,6 +644,7 @@ CREATE VIEW `cloud`.`domain_router_view` AS data_center.id data_center_id, data_center.uuid data_center_uuid, data_center.name data_center_name, + data_center.networktype data_center_type, data_center.dns1 dns1, data_center.dns2 dns2, data_center.ip6_dns1 ip6_dns1, @@ -684,7 +685,8 @@ CREATE VIEW `cloud`.`domain_router_view` AS domain_router.scripts_version scripts_version, domain_router.is_redundant_router is_redundant_router, domain_router.redundant_state redundant_state, - domain_router.stop_pending stop_pending + domain_router.stop_pending stop_pending, + domain_router.role role from `cloud`.`domain_router` inner join diff --git a/setup/db/db/schema-410to420.sql b/setup/db/db/schema-410to420.sql index c8ac1ecfc2e..065b35ca501 100644 --- a/setup/db/db/schema-410to420.sql +++ b/setup/db/db/schema-410to420.sql @@ -339,7 +339,7 @@ CREATE TABLE `cloud`.`vm_snapshots` ( ALTER TABLE `cloud`.`hypervisor_capabilities` ADD COLUMN `vm_snapshot_enabled` tinyint(1) DEFAULT 0 NOT NULL COMMENT 'Whether VM snapshot is supported by hypervisor'; UPDATE `cloud`.`hypervisor_capabilities` SET `vm_snapshot_enabled`=1 WHERE `hypervisor_type` in ('VMware', 'XenServer'); - + DROP VIEW IF EXISTS `cloud`.`user_vm_view`; CREATE VIEW `cloud`.`user_vm_view` AS select @@ -450,7 +450,7 @@ CREATE VIEW `cloud`.`user_vm_view` AS async_job.uuid job_uuid, async_job.job_status job_status, async_job.account_id job_account_id, - affinity_group.id affinity_group_id, + affinity_group.id affinity_group_id, affinity_group.uuid affinity_group_uuid, affinity_group.name affinity_group_name, affinity_group.description affinity_group_description @@ -515,7 +515,7 @@ CREATE VIEW `cloud`.`user_vm_view` AS and async_job.job_status = 0 left join `cloud`.`affinity_group_vm_map` ON vm_instance.id = affinity_group_vm_map.instance_id - left join + left join `cloud`.`affinity_group` ON affinity_group_vm_map.affinity_group_id = affinity_group.id; DROP VIEW IF EXISTS `cloud`.`affinity_group_view`; @@ -844,7 +844,8 @@ CREATE VIEW `cloud`.`domain_router_view` AS domain_router.scripts_version scripts_version, domain_router.is_redundant_router is_redundant_router, domain_router.redundant_state redundant_state, - domain_router.stop_pending stop_pending + domain_router.stop_pending stop_pending, + domain_router.role role from `cloud`.`domain_router` inner join @@ -919,7 +920,7 @@ CREATE TABLE `cloud`.`network_asa1000v_map` ( ALTER TABLE `cloud`.`network_offerings` ADD COLUMN `eip_associate_public_ip` int(1) unsigned NOT NULL DEFAULT 0 COMMENT 'true if public IP is associated with user VM creation by default when EIP service is enabled.' AFTER `elastic_ip_service`; -- Re-enable foreign key checking, at the end of the upgrade path -SET foreign_key_checks = 1; +SET foreign_key_checks = 1; -- Add "default" field to account/user tables @@ -1116,6 +1117,40 @@ CREATE VIEW `cloud`.`account_view` AS and async_job.job_status = 0; + +ALTER TABLE `cloud`.`load_balancing_rules` ADD COLUMN `source_ip_address` varchar(40) COMMENT 'source ip address for the load balancer rule'; +ALTER TABLE `cloud`.`load_balancing_rules` ADD COLUMN `source_ip_address_network_id` bigint unsigned COMMENT 'the id of the network where source ip belongs to'; +ALTER TABLE `cloud`.`load_balancing_rules` ADD COLUMN `scheme` varchar(40) NOT NULL COMMENT 'load balancer scheme; can be Internal or Public'; +UPDATE `cloud`.`load_balancing_rules` SET `scheme`='Public'; + + + +-- Add details talbe for the network offering +CREATE TABLE `cloud`.`network_offering_details` ( + `id` bigint unsigned NOT NULL auto_increment, + `network_offering_id` bigint unsigned NOT NULL COMMENT 'network offering id', + `name` varchar(255) NOT NULL, + `value` varchar(1024) NOT NULL, + PRIMARY KEY (`id`), + CONSTRAINT `fk_network_offering_details__network_offering_id` FOREIGN KEY `fk_network_offering_details__network_offering_id`(`network_offering_id`) REFERENCES `network_offerings`(`id`) ON DELETE CASCADE +) ENGINE=InnoDB DEFAULT CHARSET=utf8; + +-- Change the constraint for the network service map table. Now we support multiple provider for the same service +ALTER TABLE `cloud`.`ntwk_service_map` DROP FOREIGN KEY `fk_ntwk_service_map__network_id`; +ALTER TABLE `cloud`.`ntwk_service_map` DROP INDEX `network_id`; + +ALTER TABLE `cloud`.`ntwk_service_map` ADD UNIQUE `network_id` (`network_id`,`service`,`provider`); +ALTER TABLE `cloud`.`ntwk_service_map` ADD CONSTRAINT `fk_ntwk_service_map__network_id` FOREIGN KEY (`network_id`) REFERENCES `networks` (`id`) ON DELETE CASCADE; + + +ALTER TABLE `cloud`.`network_offerings` ADD COLUMN `internal_lb` int(1) unsigned NOT NULL DEFAULT '0' COMMENT 'true if the network offering supports Internal lb service'; +ALTER TABLE `cloud`.`network_offerings` ADD COLUMN `public_lb` int(1) unsigned NOT NULL DEFAULT '0' COMMENT 'true if the network offering supports Public lb service'; +UPDATE `cloud`.`network_offerings` SET public_lb=1 where id IN (SELECT DISTINCT network_offering_id FROM `cloud`.`ntwk_offering_service_map` WHERE service='Lb'); + + +INSERT IGNORE INTO `cloud`.`configuration` VALUES ('Advanced', 'DEFAULT', 'NetworkManager', 'internallbvm.service.offering', null, 'Uuid of the service offering used by internal lb vm; if NULL - default system internal lb offering will be used'); + + alter table `cloud_usage`.`usage_network_offering` add column nic_id bigint(20) unsigned NOT NULL; ALTER TABLE `cloud`.`data_center_details` MODIFY value varchar(1024); ALTER TABLE `cloud`.`cluster_details` MODIFY value varchar(255); diff --git a/setup/dev/advanced.cfg b/setup/dev/advanced.cfg index 63e435bb18f..83357866ca7 100644 --- a/setup/dev/advanced.cfg +++ b/setup/dev/advanced.cfg @@ -45,6 +45,10 @@ { "broadcastdomainrange": "ZONE", "name": "VpcVirtualRouter" + }, + { + "broadcastdomainrange": "ZONE", + "name": "InternalLbVm" } ], "isolationmethods": [ diff --git a/test/integration/component/test_multiple_ip_ranges.py b/test/integration/component/test_multiple_ip_ranges.py index 7e9eeb1028d..7e9e712aef0 100644 --- a/test/integration/component/test_multiple_ip_ranges.py +++ b/test/integration/component/test_multiple_ip_ranges.py @@ -369,6 +369,7 @@ class TestMultipleIpRanges(cloudstackTestCase): self.fail("CS should not accept overlapped ip ranges in guest traffic, but it allowed") return + @attr(tags=["advanced_sg", "sg"]) def test_06_add_ip_range_overlapped_with_two_ranges(self): """Test adding overlapped ip range in existing cidr diff --git a/test/integration/smoke/test_guest_vlan_range.py b/test/integration/smoke/test_guest_vlan_range.py index bd533430f51..704fe59bfff 100644 --- a/test/integration/smoke/test_guest_vlan_range.py +++ b/test/integration/smoke/test_guest_vlan_range.py @@ -44,6 +44,7 @@ class Services: "password": "password", }, "name": "testphysicalnetwork", + "vlan": "2118-2120", } @@ -149,6 +150,19 @@ class TestDedicateGuestVlanRange(cloudstackTestCase): ) self.debug("Releasing guest vlan range"); +<<<<<<< HEAD + dedicated_guest_vlan_response.release(self.apiclient) + list_dedicated_guest_vlan_range_response = PhysicalNetwork.listDedicated( + self.apiclient, + id=dedicate_guest_vlan_range_response.id + ) + dedicated_guest_vlan_response = list_dedicated_guest_vlan_range_response[0] + self.assertEqual( + dedicated_guest_vlan_response.account, + "system", + "Check account name is system account in listDedicatedGuestVlanRanges" + ) +======= dedicate_guest_vlan_range_response.release(self.apiclient) list_dedicated_guest_vlan_range_response = PhysicalNetwork.listDedicated(self.apiclient) self.assertEqual( @@ -157,4 +171,5 @@ class TestDedicateGuestVlanRange(cloudstackTestCase): "Check vlan range is not available in listDedicatedGuestVlanRanges" ) +>>>>>>> master diff --git a/test/integration/smoke/test_internal_lb.py b/test/integration/smoke/test_internal_lb.py new file mode 100644 index 00000000000..ae64297bf1c --- /dev/null +++ b/test/integration/smoke/test_internal_lb.py @@ -0,0 +1,250 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +""" Tests for configuring Internal Load Balancing Rules. +""" +#Import Local Modules +from marvin.cloudstackTestCase import * +from marvin.cloudstackAPI import * +from marvin.integration.lib.utils import * +from marvin.integration.lib.base import * +from marvin.integration.lib.common import * + + +class TestInternalLb(cloudstackTestCase): + networkOfferingId = None + networkId = None + vmId = None + lbId = None + + zoneId = 1 + serviceOfferingId = 1 + templateId = 5 + + + serviceProviderList = [ + { + "provider": "VpcVirtualRouter", + "service": "Vpn" + }, + { + "provider": "VpcVirtualRouter", + "service": "UserData" + }, + { + "provider": "VpcVirtualRouter", + "service": "Dhcp" + }, + { + "provider": "VpcVirtualRouter", + "service": "Dns" + }, + { + "provider": "InternalLbVM", + "service": "Lb" + }, + { + "provider": "VpcVirtualRouter", + "service": "SourceNat" + }, + { + "provider": "VpcVirtualRouter", + "service": "StaticNat" + }, + { + "provider": "VpcVirtualRouter", + "service": "PortForwarding" + }, + { + "provider": "VpcVirtualRouter", + "service": "NetworkACL" + } + ] + + serviceCapsList = [ + { + "service": "SourceNat", + "capabilitytype": "SupportedSourceNatTypes", + "capabilityvalue": "peraccount" + }, + { + "service": "Lb", + "capabilitytype": "SupportedLbIsolation", + "capabilityvalue": "dedicated" + }, + { + "service": "Lb", + "capabilitytype": "lbSchemes", + "capabilityvalue": "internal" + } + ] + + def setUp(self): + self.apiClient = self.testClient.getApiClient() + + + + def test_internallb(self): + + #1) Create and enable network offering with Internal Lb vm service + self.createNetworkOffering() + + #2) Create VPC and network in it + self.createNetwork() + + #3) Deploy a vm + self.deployVm() + + #4) Create an Internal Load Balancer + self.createInternalLoadBalancer() + + #5) Assign the VM to the Internal Load Balancer + self.assignToLoadBalancerRule() + + #6) Remove the vm from the Interanl Load Balancer + self.removeFromLoadBalancerRule() + + #7) Delete the Load Balancer + self.deleteLoadBalancer() + + + def deployVm(self): + deployVirtualMachineCmd = deployVirtualMachine.deployVirtualMachineCmd() + deployVirtualMachineCmd.networkids = TestInternalLb.networkId + deployVirtualMachineCmd.serviceofferingid = TestInternalLb.serviceOfferingId + deployVirtualMachineCmd.zoneid = TestInternalLb.zoneId + deployVirtualMachineCmd.templateid = TestInternalLb.templateId + deployVirtualMachineCmd.hypervisor = "XenServer" + deployVMResponse = self.apiClient.deployVirtualMachine(deployVirtualMachineCmd) + TestInternalLb.vmId = deployVMResponse.id + + + def createInternalLoadBalancer(self): + createLoadBalancerCmd = createLoadBalancer.createLoadBalancerCmd() + createLoadBalancerCmd.name = "lb rule" + createLoadBalancerCmd.sourceport = 22 + createLoadBalancerCmd.instanceport = 22 + createLoadBalancerCmd.algorithm = "roundrobin" + createLoadBalancerCmd.scheme = "internal" + createLoadBalancerCmd.sourceipaddressnetworkid = TestInternalLb.networkId + createLoadBalancerCmd.networkid = TestInternalLb.networkId + createLoadBalancerResponse = self.apiClient.createLoadBalancer(createLoadBalancerCmd) + TestInternalLb.lbId = createLoadBalancerResponse.id + self.assertIsNotNone(createLoadBalancerResponse.id, "Failed to create a load balancer") + + + def assignToLoadBalancerRule(self): + assignToLoadBalancerRuleCmd = assignToLoadBalancerRule.assignToLoadBalancerRuleCmd() + assignToLoadBalancerRuleCmd.id = TestInternalLb.lbId + assignToLoadBalancerRuleCmd.virtualMachineIds = TestInternalLb.vmId + assignToLoadBalancerRuleResponse = self.apiClient.assignToLoadBalancerRule(assignToLoadBalancerRuleCmd) + self.assertTrue(assignToLoadBalancerRuleResponse.success, "Failed to assign the vm to the load balancer") + + + + def removeFromLoadBalancerRule(self): + removeFromLoadBalancerRuleCmd = removeFromLoadBalancerRule.removeFromLoadBalancerRuleCmd() + removeFromLoadBalancerRuleCmd.id = TestInternalLb.lbId + removeFromLoadBalancerRuleCmd.virtualMachineIds = TestInternalLb.vmId + removeFromLoadBalancerRuleResponse = self.apiClient.removeFromLoadBalancerRule(removeFromLoadBalancerRuleCmd) + self.assertTrue(removeFromLoadBalancerRuleResponse.success, "Failed to remove the vm from the load balancer") + + + + #def removeInternalLoadBalancer(self): + def deleteLoadBalancer(self): + deleteLoadBalancerCmd = deleteLoadBalancer.deleteLoadBalancerCmd() + deleteLoadBalancerCmd.id = TestInternalLb.lbId + deleteLoadBalancerResponse = self.apiClient.deleteLoadBalancer(deleteLoadBalancerCmd) + self.assertTrue(deleteLoadBalancerResponse.success, "Failed to remove the load balancer") + + + + def createNetwork(self): + createVPCCmd = createVPC.createVPCCmd() + createVPCCmd.name = "new vpc" + createVPCCmd.cidr = "10.1.1.0/24" + createVPCCmd.displaytext = "new vpc" + createVPCCmd.vpcofferingid = 1 + createVPCCmd.zoneid = self.zoneId + createVPCResponse = self.apiClient.createVPC(createVPCCmd) + + + createNetworkCmd = createNetwork.createNetworkCmd() + createNetworkCmd.name = "vpc network" + createNetworkCmd.displaytext = "vpc network" + createNetworkCmd.netmask = "255.255.255.0" + createNetworkCmd.gateway = "10.1.1.1" + createNetworkCmd.zoneid = self.zoneId + createNetworkCmd.vpcid = createVPCResponse.id + createNetworkCmd.networkofferingid = TestInternalLb.networkOfferingId + createNetworkResponse = self.apiClient.createNetwork(createNetworkCmd) + TestInternalLb.networkId = createNetworkResponse.id + + self.assertIsNotNone(createNetworkResponse.id, "Network failed to create") + + + def createNetworkOffering(self): + createNetworkOfferingCmd = createNetworkOffering.createNetworkOfferingCmd() + createNetworkOfferingCmd.name = "Network offering for internal lb service - " + str(random.randrange(1,100+1)) + createNetworkOfferingCmd.displaytext = "Network offering for internal lb service" + createNetworkOfferingCmd.guestiptype = "isolated" + createNetworkOfferingCmd.traffictype = "Guest" + createNetworkOfferingCmd.conservemode = "false" + createNetworkOfferingCmd.supportedservices = "Vpn,Dhcp,Dns,Lb,UserData,SourceNat,StaticNat,PortForwarding,NetworkACL" + + + createNetworkOfferingCmd.serviceproviderlist = [] + for item in self.serviceProviderList: + createNetworkOfferingCmd.serviceproviderlist.append({ + 'service': item['service'], + 'provider': item['provider'] + }) + + createNetworkOfferingCmd.servicecapabilitylist = [] + for item in self.serviceCapsList: + createNetworkOfferingCmd.servicecapabilitylist.append({ + 'service': item['service'], + 'capabilitytype': item['capabilitytype'], + 'capabilityvalue': item['capabilityvalue'] + }) + + + createNetworkOfferingResponse = self.apiClient.createNetworkOffering(createNetworkOfferingCmd) + TestInternalLb.networkOfferingId = createNetworkOfferingResponse.id + + #enable network offering + updateNetworkOfferingCmd = updateNetworkOffering.updateNetworkOfferingCmd() + updateNetworkOfferingCmd.id = TestInternalLb.networkOfferingId + updateNetworkOfferingCmd.state = "Enabled" + updateNetworkOfferingResponse = self.apiClient.updateNetworkOffering(updateNetworkOfferingCmd) + + + #list network offering to see if its enabled + listNetworkOfferingsCmd = listNetworkOfferings.listNetworkOfferingsCmd() + listNetworkOfferingsCmd.id = TestInternalLb.networkOfferingId + listOffResponse = self.apiClient.listNetworkOfferings(listNetworkOfferingsCmd) + + self.assertNotEqual(len(listOffResponse), 0, "Check if the list network offerings API \ + returns a non-empty response") + + + def tearDown(self): + #destroy the vm + if TestInternalLb.vmId is not None: + destroyVirtualMachineCmd = destroyVirtualMachine.destroyVirtualMachineCmd() + destroyVirtualMachineCmd.id = TestInternalLb.vmId + destroyVirtualMachineResponse = self.apiClient.destroyVirtualMachine(destroyVirtualMachineCmd) diff --git a/tools/apidoc/gen_toc.py b/tools/apidoc/gen_toc.py index f8bdae281b2..bd8c0f1668f 100644 --- a/tools/apidoc/gen_toc.py +++ b/tools/apidoc/gen_toc.py @@ -140,6 +140,7 @@ known_categories = { 'removeIpFromNic': 'Nic', 'listNics':'Nic', 'AffinityGroup': 'Affinity Group', + 'InternalLoadBalancer': 'Internal LB', } diff --git a/tools/marvin/marvin/deployDataCenter.py b/tools/marvin/marvin/deployDataCenter.py index 21685923b37..7059059beb1 100644 --- a/tools/marvin/marvin/deployDataCenter.py +++ b/tools/marvin/marvin/deployDataCenter.py @@ -216,6 +216,18 @@ class deployDataCenters(): vrconfig.id = vrprovid self.apiClient.configureVirtualRouterElement(vrconfig) self.enableProvider(pnetprovres[0].id) + elif provider.name == 'InternalLbVm': + internallbprov = listInternalLoadBalancerElements.listInternalLoadBalancerElementsCmd() + internallbprov.nspid = pnetprovres[0].id + internallbresponse = self.apiClient.listInternalLoadBalancerElements(internallbprov) + internallbid = internallbresponse[0].id + + internallbconfig = \ + configureInternalLoadBalancerElement.configureInternalLoadBalancerElementCmd() + internallbconfig.enabled = "true" + internallbconfig.id = internallbid + self.apiClient.configureInternalLoadBalancerElement(internallbconfig) + self.enableProvider(pnetprovres[0].id) elif provider.name == 'SecurityGroupProvider': self.enableProvider(pnetprovres[0].id) elif provider.name in ['Netscaler', 'JuniperSRX', 'F5BigIp']: diff --git a/tools/marvin/marvin/sandbox/advanced/advanced_env.py b/tools/marvin/marvin/sandbox/advanced/advanced_env.py index bf880f38055..6343293aa62 100644 --- a/tools/marvin/marvin/sandbox/advanced/advanced_env.py +++ b/tools/marvin/marvin/sandbox/advanced/advanced_env.py @@ -50,6 +50,9 @@ def describeResources(config): vpcprovider = provider() vpcprovider.name = 'VpcVirtualRouter' + + lbprovider = provider() + lbprovider.name = 'InternalLbVm' pn = physical_network() pn.name = "Sandbox-pnet" @@ -60,6 +63,7 @@ def describeResources(config): traffictype("Public", {"simulator":"cloud-simulator-public"})] pn.isolationmethods = ["VLAN"] pn.providers.append(vpcprovider) + pn.providers.append(lbprovider) pn2 = physical_network() pn2.name = "Sandbox-pnet2" @@ -68,6 +72,7 @@ def describeResources(config): pn2.traffictypes = [traffictype('Guest', {'simulator': 'cloud-simulator-guest'})] pn2.isolationmethods = ["VLAN"] pn2.providers.append(vpcprovider) + pn2.providers.append(lbprovider) z.physical_networks.append(pn) z.physical_networks.append(pn2) diff --git a/tools/marvin/marvin/sandbox/advanced/sandbox.cfg b/tools/marvin/marvin/sandbox/advanced/sandbox.cfg new file mode 100644 index 00000000000..01a84730dad --- /dev/null +++ b/tools/marvin/marvin/sandbox/advanced/sandbox.cfg @@ -0,0 +1,209 @@ +{ + "zones": [ + { + "name": "Sandbox-Simulator", + "guestcidraddress": "10.1.1.0/24", + "dns1": "10.147.28.6", + "physical_networks": [ + { + "providers": [ + { + "broadcastdomainrange": "ZONE", + "name": "VirtualRouter" + }, + { + "broadcastdomainrange": "ZONE", + "name": "VpcVirtualRouter" + }, + { + "broadcastdomainrange": "ZONE", + "name": "InternalLb" + } + ], + "name": "Sandbox-pnet", + "tags": [ + "cloud-simulator-public" + ], + "broadcastdomainrange": "Zone", + "vlan": "675-679", + "traffictypes": [ + { + "typ": "Guest" + }, + { + "typ": "Management", + "simulator": "cloud-simulator-mgmt" + }, + { + "typ": "Public", + "simulator": "cloud-simulator-public" + } + ], + "isolationmethods": [ + "VLAN" + ] + }, + { + "providers": [ + { + "broadcastdomainrange": "ZONE", + "name": "VirtualRouter" + }, + { + "broadcastdomainrange": "ZONE", + "name": "VpcVirtualRouter" + }, + { + "broadcastdomainrange": "ZONE", + "name": "InternalLb" + } + ], + "name": "Sandbox-pnet2", + "tags": [ + "cloud-simulator-guest" + ], + "broadcastdomainrange": "Zone", + "vlan": "800-1000", + "traffictypes": [ + { + "typ": "Guest", + "simulator": "cloud-simulator-guest" + } + ], + "isolationmethods": [ + "VLAN" + ] + } + ], + "securitygroupenabled": "false", + "ipranges": [ + { + "startip": "10.147.31.150", + "endip": "10.147.31.159", + "netmask": "255.255.255.0", + "vlan": "31", + "gateway": "10.147.31.1" + } + ], + "networktype": "Advanced", + "pods": [ + { + "endip": "10.147.29.159", + "name": "POD0", + "startip": "10.147.29.150", + "netmask": "255.255.255.0", + "clusters": [ + { + "clustername": "C0", + "hypervisor": "Simulator", + "hosts": [ + { + "username": "root", + "url": "http://simulator0", + "password": "password" + } + ], + "clustertype": "CloudManaged", + "primaryStorages": [ + { + "url": "nfs://10.147.28.6:/export/home/sandbox/primary", + "name": "PS0" + } + ] + } + ], + "gateway": "10.147.29.1" + } + ], + "internaldns1": "10.147.28.6", + "secondaryStorages": [ + { + "url": "nfs://10.147.28.6:/export/home/sandbox/sstor" + } + ] + } + ], + "dbSvr": { + "dbSvr": "localhost", + "passwd": "cloud", + "db": "cloud", + "port": 3306, + "user": "cloud" + }, + "logger": [ + { + "name": "TestClient", + "file": "testclient.log" + }, + { + "name": "TestCase", + "file": "testcase.log" + } + ], + "globalConfig": [ + { + "name": "storage.cleanup.interval", + "value": "300" + }, + { + "name": "direct.agent.load.size", + "value": "1000" + }, + { + "name": "default.page.size", + "value": "10000" + }, + { + "name": "instance.name", + "value": "QA" + }, + { + "name": "workers", + "value": "10" + }, + { + "name": "vm.op.wait.interval", + "value": "5" + }, + { + "name": "account.cleanup.interval", + "value": "600" + }, + { + "name": "guest.domain.suffix", + "value": "sandbox.simulator" + }, + { + "name": "expunge.delay", + "value": "60" + }, + { + "name": "vm.allocation.algorithm", + "value": "random" + }, + { + "name": "expunge.interval", + "value": "60" + }, + { + "name": "expunge.workers", + "value": "3" + }, + { + "name": "secstorage.allowed.internal.sites", + "value": "10.147.28.0/24" + }, + { + "name": "check.pod.cidrs", + "value": "true" + } + ], + "mgtSvr": [ + { + "mgtSvrIp": "localhost", + "passwd": "password", + "user": "root", + "port": 8096 + } + ] +} \ No newline at end of file diff --git a/ui/scripts/zoneWizard.js b/ui/scripts/zoneWizard.js index b28258544b7..9b28c327b78 100755 --- a/ui/scripts/zoneWizard.js +++ b/ui/scripts/zoneWizard.js @@ -2376,6 +2376,110 @@ }); // ***** Virtual Router ***** (end) ***** + // ***** Internal LB ***** (begin) ***** + var internalLbProviderId; + $.ajax({ + url: createURL("listNetworkServiceProviders&name=Internallbvm&physicalNetworkId=" + thisPhysicalNetwork.id), + dataType: "json", + async: false, + success: function(json) { + var items = json.listnetworkserviceprovidersresponse.networkserviceprovider; + if(items != null && items.length > 0) { + internalLbProviderId = items[0].id; + } + } + }); + if(internalLbProviderId == null) { + alert("error: listNetworkServiceProviders API doesn't return internalLb provider ID"); + return; + } + + var internalLbElementId; + $.ajax({ + url: createURL("listInternalLoadBalancerElements&nspid=" + internalLbProviderId), + dataType: "json", + async: false, + success: function(json) { + var items = json.listinternalloadbalancerelementsresponse.internalloadbalancerelement; + if(items != null && items.length > 0) { + internalLbElementId = items[0].id; + } + } + }); + if(internalLbElementId == null) { + alert("error: listInternalLoadBalancerElements API doesn't return Internal LB Element Id"); + return; + } + + $.ajax({ + url: createURL("configureInternalLoadBalancerElement&enabled=true&id=" + internalLbElementId), + dataType: "json", + async: false, + success: function(json) { + var jobId = json.configureinternalloadbalancerelementresponse.jobid; + var enableInternalLbElementIntervalID = setInterval(function() { + $.ajax({ + url: createURL("queryAsyncJobResult&jobId="+jobId), + dataType: "json", + success: function(json) { + var result = json.queryasyncjobresultresponse; + if (result.jobstatus == 0) { + return; //Job has not completed + } + else { + clearInterval(enableInternalLbElementIntervalID); + + if (result.jobstatus == 1) { //configureVirtualRouterElement succeeded + $.ajax({ + url: createURL("updateNetworkServiceProvider&state=Enabled&id=" + internalLbProviderId), + dataType: "json", + async: false, + success: function(json) { + var jobId = json.updatenetworkserviceproviderresponse.jobid; + var enableInternalLbProviderIntervalID = setInterval(function() { + $.ajax({ + url: createURL("queryAsyncJobResult&jobId="+jobId), + dataType: "json", + success: function(json) { + var result = json.queryasyncjobresultresponse; + if (result.jobstatus == 0) { + return; //Job has not completed + } + else { + clearInterval(enableInternalLbProviderIntervalID); + + if (result.jobstatus == 1) { //Internal LB has been enabled successfully + //don't need to do anything here + } + else if (result.jobstatus == 2) { + alert("failed to enable Internal LB Provider. Error: " + _s(result.jobresult.errortext)); + } + } + }, + error: function(XMLHttpResponse) { + var errorMsg = parseXMLHttpResponse(XMLHttpResponse); + alert("failed to enable Internal LB Provider. Error: " + errorMsg); + } + }); + }, g_queryAsyncJobResultInterval); + } + }); + } + else if (result.jobstatus == 2) { + alert("configureVirtualRouterElement failed. Error: " + _s(result.jobresult.errortext)); + } + } + }, + error: function(XMLHttpResponse) { + var errorMsg = parseXMLHttpResponse(XMLHttpResponse); + alert("configureVirtualRouterElement failed. Error: " + errorMsg); + } + }); + }, g_queryAsyncJobResultInterval); + } + }); + // ***** Internal LB ***** (end) ***** + if(args.data.zone.sgEnabled != true) { //Advanced SG-disabled zone // ***** VPC Virtual Router ***** (begin) ***** var vpcVirtualRouterProviderId; From 19f014585e1d2a85b66ecd5ad57ee0c6338beb4c Mon Sep 17 00:00:00 2001 From: Alena Prokharchyk Date: Thu, 9 May 2013 14:47:49 -0700 Subject: [PATCH 02/14] Marvin test: exclude sandbox.cfg from the rat check --- pom.xml | 1 + 1 file changed, 1 insertion(+) diff --git a/pom.xml b/pom.xml index 9dbd1bf8d7a..e8fdb2f83ea 100644 --- a/pom.xml +++ b/pom.xml @@ -437,6 +437,7 @@ patches/systemvm/debian/config/var/www/html/latest/.htaccess patches/systemvm/debian/vpn/etc/ipsec.d/l2tp.conf tools/transifex/.tx/config + tools/marvin/marvin/sandbox/advanced/sandbox.cfg From 75a2457af639e6bcf751c2e97d1360df1a0265e4 Mon Sep 17 00:00:00 2001 From: Hari Patanala Date: Fri, 10 May 2013 09:29:41 +0530 Subject: [PATCH 03/14] CLOUDSTACK-658 - Adding scale vm support for vmware with unit tests Signed off by : Nitin Mehta --- plugins/hypervisors/vmware/pom.xml | 10 +++ .../vmware/resource/VmwareResource.java | 31 ++++++- .../vmware/resource/VmwareResourceTest.java | 82 +++++++++++++++++++ .../src/com/cloud/vm/UserVmManagerImpl.java | 2 +- .../hypervisor/vmware/util/VmwareHelper.java | 26 +++++- 5 files changed, 148 insertions(+), 3 deletions(-) create mode 100644 plugins/hypervisors/vmware/test/com/cloud/hypervisor/vmware/resource/VmwareResourceTest.java diff --git a/plugins/hypervisors/vmware/pom.xml b/plugins/hypervisors/vmware/pom.xml index d65ef640655..79779decf62 100644 --- a/plugins/hypervisors/vmware/pom.xml +++ b/plugins/hypervisors/vmware/pom.xml @@ -58,5 +58,15 @@ wsdl4j 1.4 + + junit + junit + 4.10 + + + org.mockito + mockito-all + 1.9.5 + diff --git a/plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java b/plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java index d87da64040b..6d7e0e7289c 100755 --- a/plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java +++ b/plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java @@ -111,6 +111,8 @@ import com.cloud.agent.api.RebootCommand; import com.cloud.agent.api.RebootRouterCommand; import com.cloud.agent.api.RevertToVMSnapshotAnswer; import com.cloud.agent.api.RevertToVMSnapshotCommand; +import com.cloud.agent.api.ScaleVmCommand; +import com.cloud.agent.api.ScaleVmAnswer; import com.cloud.agent.api.SetupAnswer; import com.cloud.agent.api.SetupCommand; import com.cloud.agent.api.SetupGuestNetworkAnswer; @@ -485,6 +487,8 @@ public class VmwareResource implements StoragePoolResource, ServerResource, Vmwa return execute((ResizeVolumeCommand) cmd); } else if (clz == UnregisterVMCommand.class) { return execute((UnregisterVMCommand) cmd); + } else if (clz == ScaleVmCommand.class) { + return execute((ScaleVmCommand) cmd); } else { answer = Answer.createUnsupportedCommandAnswer(cmd); } @@ -2088,6 +2092,28 @@ public class VmwareResource implements StoragePoolResource, ServerResource, Vmwa return validatedDisks.toArray(new VolumeTO[0]); } + protected ScaleVmAnswer execute(ScaleVmCommand cmd) { + + VmwareContext context = getServiceContext(); + VirtualMachineTO vmSpec = cmd.getVirtualMachine(); + try{ + VmwareHypervisorHost hyperHost = getHyperHost(context); + VirtualMachineMO vmMo = hyperHost.findVmOnHyperHost(cmd.getVmName()); + VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec(); + int ramMb = (int) (vmSpec.getMinRam()); + + VmwareHelper.setVmScaleUpConfig(vmConfigSpec, vmSpec.getCpus(), vmSpec.getSpeed(), vmSpec.getSpeed(),(int) (vmSpec.getMaxRam()), ramMb, vmSpec.getLimitCpuUse()); + + if(!vmMo.configureVm(vmConfigSpec)) { + throw new Exception("Unable to execute ScaleVmCommand"); + } + }catch(Exception e) { + s_logger.error("Unexpected exception: ", e); + return new ScaleVmAnswer(cmd, false, "Unable to execute ScaleVmCommand due to " + e.toString()); + } + return new ScaleVmAnswer(cmd, true, null); + } + protected StartAnswer execute(StartCommand cmd) { if (s_logger.isInfoEnabled()) { @@ -2191,7 +2217,10 @@ public class VmwareResource implements StoragePoolResource, ServerResource, Vmwa VmwareHelper.setBasicVmConfig(vmConfigSpec, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmSpec.getMinSpeed(),(int) (vmSpec.getMaxRam()/(1024*1024)), ramMb, translateGuestOsIdentifier(vmSpec.getArch(), vmSpec.getOs()).value(), vmSpec.getLimitCpuUse()); - + + vmConfigSpec.setMemoryHotAddEnabled(true); + vmConfigSpec.setCpuHotAddEnabled(true); + if ("true".equals(vmSpec.getDetails().get(VmDetailConstants.NESTED_VIRTUALIZATION_FLAG))) { s_logger.debug("Nested Virtualization enabled in configuration, checking hypervisor capability"); ManagedObjectReference hostMor = vmMo.getRunningHost().getMor(); diff --git a/plugins/hypervisors/vmware/test/com/cloud/hypervisor/vmware/resource/VmwareResourceTest.java b/plugins/hypervisors/vmware/test/com/cloud/hypervisor/vmware/resource/VmwareResourceTest.java new file mode 100644 index 00000000000..3ca0b600e36 --- /dev/null +++ b/plugins/hypervisors/vmware/test/com/cloud/hypervisor/vmware/resource/VmwareResourceTest.java @@ -0,0 +1,82 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package com.cloud.hypervisor.vmware.resource; + +import com.cloud.agent.api.Command; +import com.cloud.agent.api.ScaleVmAnswer; +import com.cloud.agent.api.ScaleVmCommand; +import com.cloud.agent.api.to.VirtualMachineTO; +import com.cloud.hypervisor.vmware.mo.VirtualMachineMO; +import com.cloud.hypervisor.vmware.mo.VmwareHypervisorHost; +import com.cloud.hypervisor.vmware.util.VmwareContext; +import com.cloud.hypervisor.vmware.util.VmwareHelper; +import com.vmware.vim25.VirtualMachineConfigSpec; +import org.junit.Test; +import org.junit.Before; + +import org.mockito.Mock; +import org.mockito.Spy; +import org.mockito.MockitoAnnotations; +import static org.mockito.Mockito.*; + + +public class VmwareResourceTest { + + @Spy VmwareResource _resource = new VmwareResource() { + + @Override + public ScaleVmAnswer execute(ScaleVmCommand cmd) { + return super.execute(cmd); + } + @Override + public VmwareHypervisorHost getHyperHost(VmwareContext context, Command cmd) { + return hyperHost; + } + }; + + @Mock VmwareContext context; + @Mock ScaleVmCommand cmd; + @Mock VirtualMachineTO vmSpec; + @Mock + VmwareHypervisorHost hyperHost; + @Mock VirtualMachineMO vmMo; + @Mock VirtualMachineConfigSpec vmConfigSpec; + + @Before + public void setup(){ + MockitoAnnotations.initMocks(this); + doReturn(context).when(_resource).getServiceContext(null); + when(cmd.getVirtualMachine()).thenReturn(vmSpec); + } + //Test successful scaling up the vm + @Test + public void testScaleVMF1() throws Exception { + when(_resource.getHyperHost(context, null)).thenReturn(hyperHost); + doReturn("i-2-3-VM").when(cmd).getVmName(); + when(hyperHost.findVmOnHyperHost("i-2-3-VM")).thenReturn(vmMo); + doReturn(1024L).when(vmSpec).getMinRam(); + doReturn(1).when(vmSpec).getCpus(); + doReturn(1000).when(vmSpec).getSpeed(); + doReturn(1024L).when(vmSpec).getMaxRam(); + doReturn(false).when(vmSpec).getLimitCpuUse(); + when(vmMo.configureVm(vmConfigSpec)).thenReturn(true); + + ScaleVmAnswer answer = _resource.execute(cmd); + verify(_resource).execute(cmd); + } + +} diff --git a/server/src/com/cloud/vm/UserVmManagerImpl.java b/server/src/com/cloud/vm/UserVmManagerImpl.java index eee8503b3f5..9fbc5099374 100755 --- a/server/src/com/cloud/vm/UserVmManagerImpl.java +++ b/server/src/com/cloud/vm/UserVmManagerImpl.java @@ -1064,7 +1064,7 @@ public class UserVmManagerImpl extends ManagerBase implements UserVmManager, Use // Verify input parameters VMInstanceVO vmInstance = _vmInstanceDao.findById(vmId); - if(vmInstance.getHypervisorType() != HypervisorType.XenServer){ + if(vmInstance.getHypervisorType() != HypervisorType.XenServer && vmInstance.getHypervisorType() != HypervisorType.VMware){ throw new InvalidParameterValueException("This operation not permitted for this hypervisor of the vm"); } diff --git a/vmware-base/src/com/cloud/hypervisor/vmware/util/VmwareHelper.java b/vmware-base/src/com/cloud/hypervisor/vmware/util/VmwareHelper.java index 9c467dc8b6b..4a6a135a5b8 100644 --- a/vmware-base/src/com/cloud/hypervisor/vmware/util/VmwareHelper.java +++ b/vmware-base/src/com/cloud/hypervisor/vmware/util/VmwareHelper.java @@ -524,7 +524,31 @@ public class VmwareHelper { return options; } - public static void setBasicVmConfig(VirtualMachineConfigSpec vmConfig, int cpuCount, int cpuSpeedMHz, int cpuReservedMhz, + public static void setVmScaleUpConfig(VirtualMachineConfigSpec vmConfig, int cpuCount, int cpuSpeedMHz, int cpuReservedMhz, + int memoryMB, int memoryReserveMB, boolean limitCpuUse) { + + // VM config for scaling up + vmConfig.setMemoryMB((long)memoryMB); + vmConfig.setNumCPUs(cpuCount); + + ResourceAllocationInfo cpuInfo = new ResourceAllocationInfo(); + if (limitCpuUse) { + cpuInfo.setLimit((long)(cpuSpeedMHz * cpuCount)); + } else { + cpuInfo.setLimit(-1L); + } + + cpuInfo.setReservation((long)cpuReservedMhz); + vmConfig.setCpuAllocation(cpuInfo); + + ResourceAllocationInfo memInfo = new ResourceAllocationInfo(); + memInfo.setLimit((long)memoryMB); + memInfo.setReservation((long)memoryReserveMB); + vmConfig.setMemoryAllocation(memInfo); + + } + + public static void setBasicVmConfig(VirtualMachineConfigSpec vmConfig, int cpuCount, int cpuSpeedMHz, int cpuReservedMhz, int memoryMB, int memoryReserveMB, String guestOsIdentifier, boolean limitCpuUse) { // VM config basics From 7fea21e31f25e0a42a4e2d311faa3cba429e0101 Mon Sep 17 00:00:00 2001 From: Radhika PC Date: Fri, 10 May 2013 10:03:11 +0530 Subject: [PATCH 04/14] CLOUDSTACK-893 conceptual info around gslb --- docs/en-US/added-API-commands-4.2.xml | 33 ++++--- docs/en-US/gslb.xml | 120 ++++++++++++++++++++------ 2 files changed, 118 insertions(+), 35 deletions(-) diff --git a/docs/en-US/added-API-commands-4.2.xml b/docs/en-US/added-API-commands-4.2.xml index 7417bd15f35..3abb780663e 100644 --- a/docs/en-US/added-API-commands-4.2.xml +++ b/docs/en-US/added-API-commands-4.2.xml @@ -91,20 +91,31 @@ listGlobalLoadBalancerRule - Lists load balancer rules. account (lists resources by account. Use with the domainId - parameter); domainid (lists only resources belonging to the domain specified) id (the unique - ID of the global load balancer rule) isrecursive (defaults to false, but if true, lists all - resources from the parent specified by the domainId till leaves); keyword (List by keyword); - listall (if set to false, list only resources belonging to the command's caller; if set to - true - list resources that the caller is authorized to see. Default value is false); page; - pagesize; projectid (lists objects by project); regionid (region ID); tags (lists resources - by tags: key/value pairs). + Lists load balancer rules. + The request parameters are: account (lists resources by account. Use with the domainid + parameter); domainid (lists only resources belonging to the domain specified); id (the + unique ID of the global load balancer rule); isrecursive (defaults to false; but if true, + lists all the resources from the parent specified by the domainid); keyword (lists by + keyword); listall (if set to false, lists only resources belonging to the command's caller; + if set to true, lists resources that the caller is authorized to see. Default value is + false); page; pagesize; projectid (lists objects by project); regionid ; tags (lists + resources by tags: key/value pairs). updateGlobalLoadBalancerRule - Archives the specified events. The request parameters are: ids (allowed to pass one or - more IDs separated by comma); type (string); olderthan (yyyy-mm-dd format). - The response parameters are: true, false + Updates global load balancer rules. + The request parameters are: id (the unique ID of the global load balancer rule); account + (lists resources by account. Use with the domainid parameter); description (the description + of the load balancer rule); domainid (lists only resources belonging to the domain + specified); gslblbmethod (the load balancer algorithm that is used to distributed traffic + across the zones participating in global server load balancing, if not specified defaults to + round robin); gslbstickysessionmethodname (the session sticky method; if not specified + defaults to sourceip); isrecursive (defaults to false, but if true, lists all resources from + the parent specified by the domainid till leaves); keyword (lists by keyword); listall (if + set to false, list only those resources belonging to the command's caller; if set to true, + lists resources that the caller is authorized to see. Default value is false); page; + pagesize; projectid (lists objects by project); regionid; tags (lists resources by tags: + key/value pairs) diff --git a/docs/en-US/gslb.xml b/docs/en-US/gslb.xml index 385642d394d..ac17d61d69c 100644 --- a/docs/en-US/gslb.xml +++ b/docs/en-US/gslb.xml @@ -41,6 +41,60 @@ alternate location for accessing a resource in the event of a failure, or to provide a means of shifting traffic easily to simplify maintenance, or both. +
+ Components of GSLB + A typical GSLB environment is comprised of the following components: + + + GSLB Site: In &PRODUCT;terminology, GSLB sites are + represented by zones that are mapped to data centers, each of which has various network + appliances. Each GSLB site is managed by a NetScaler appliance that is local to that site. + Each of these appliances treats its own site as the local site and all other sites, + managed by other appliances, as remote sites. + + + GSLB Services: A GSLB service is typically + represented by a load balancing or content switching virtual server. In a GSLB + environment, you can have a local as well as remote GSLB services. A local GSLB service + represents a local load balancing or content switching virtual server. A remote GSLB + service is the one configured at one of the other sites in the GSLB setup. At each site in + the GSLB setup, you can create one local GSLB service and any number of remote GSLB + services. + + + GSLB Virtual Servers: A GSLB virtual server refers to + one or more GSLB services and balances traffic between traffic across the VMs in multiple + zones by using the &PRODUCT; functionality. It evaluates the configured GSLB methods or + algorithms to select a GSLB service to which to send the client requests. One or more + virtual servers from different zones are bound to the GSLB virtual server. GSLB virtual + server does not have a public IP associated with it, instead it will have a FQDN DNS + name. + + + Load Balancing or Content Switching Virtual Servers: + According to Citrix NetScaler terminology, a load balancing or content switching virtual + server represents one or many servers on the local network. Clients send their requests to + the load balancing or content switching virtual server’s virtual IP (VIP) address, and the + virtual server balances the load across the local servers. After a GSLB virtual server + selects a GSLB service representing either a local or a remote load balancing or content + switching virtual server, the client sends the request to that virtual server’s VIP + address. + + + DNS VIPs: DNS virtual IP represents a load balancing + DNS virtual server on the GSLB service provider. The DNS requests for domains for which + the GSLB service provider is authoritative can be sent to a DNS VIP. + + + ADNS: ADNS (Authoritative Domain Name Server) is a + service that provides actual answer to DNS queries, such as web site IP address. In a GSLB + environment, an ADNS service responds only to DNS requests for domains for which the GSLB + service provider is authoritative. When an ADNS service is configured, the service + provider owns that IP address and advertises it. When you create an ADNS service, the + NetScaler responds to DNS queries on the configured ADNS service IP and port. + + +
Prerequisites and Guidelines @@ -63,23 +117,23 @@ When users have VMs deployed in multiple availability zones which are GSLB enabled, - user is allowed to use the GSLB functionality to load balance traffic across the VMs in - multiple zones. + they can use the GSLB functionality to load balance traffic across the VMs in multiple + zones. - The users are allowed to use GSLB to load balance across the VMs across zones in a - region only if the admin has enabled GSLB in that region. + The users can use GSLB to load balance across the VMs across zones in a region only if + the admin has enabled GSLB in that region. - The users are allowed to load balance traffic across the availability zones in the - same region or different regions. + The users can load balance traffic across the availability zones in the same region or + different regions. - The admin is allowed to configure DNS name for the entire cloud. + The admin can configure DNS name for the entire cloud. - The users can specify an unique name, across the cloud, for a globally load balanced - service. The provided name will be used as the domain under the DNS name associated with + The users can specify an unique name across the cloud for a globally load balanced + service. The provided name is used as the domain name under the DNS name associated with the cloud. The user-provided name along with the admin-provided DNS name is used to produce a globally resolvable FQDN for the globally load balanced service of the user. For example, @@ -88,13 +142,12 @@ foo.xyztelco.com. - While setting up GSLB, users can select a load balancing method, such as round robin - or least RTT, that would be the load balance traffic used across the zones that are part - of GSLB. + While setting up GSLB, users can select a load balancing method, such as round robin, + for using across the zones that are part of GSLB. The user shall be able to set weight to zone-level virtual server. Weight shall be - considered by the load balancing method is distributing the traffic. + considered by the load balancing method for distributing the traffic. The GSLB functionality shall support session persistence, where series of client @@ -103,6 +156,24 @@
+
+ Configuring GSLB + To configure GSLB in your cloud environment, as a cloud administrator you must perform the + following. + To configure such a GSLB setup, you must first configure a standard load balancing setup + for each server farm or data center. This enables you to balance load across the different + servers in each server farm. Then, configure both NetScaler appliances as authoritative DNS + (ADNS) servers. Next, create a GSLB site for each server farm, configure GSLB virtual servers + for each site, create GLSB services, and bind the GSLB services to the GSLB virtual servers. + Finally, bind the domain to the GSLB virtual servers. The GSLB configurations on the two + appliances at the two different sites are identical, although each sites load-balancing + configuration is specific to that site. + + + + + +
Adding a GSLB Rule @@ -174,17 +245,18 @@
How Does GSLB Works in &PRODUCT;? - The following is an illustrated conceptual model of how GLSB functionality is provided in - &PRODUCT;: An organization, xyztelco, has set up a public cloud that spans two zones, Zone-1 - and Zone-2, across geographically separated data centers that are managed by &PRODUCT;. - Tenant-A of the cloud launches a highly available solution by using xyztelco cloud. For that - purpose, they launch two instances each in both the zones: VM1 and VM2 in Zone-1 and VM5 and - VM6 in Zone-2. Tenant-A acquires a public IP, IP-1 in Zone-1, and configures a load balancer - rule to load balance the traffic between VM1 and VM2 instances. &PRODUCT; orchestrates setting - up a virtual server on the LB service provider in Zone-1. Virtual server 1 that is set up on - the LB service provider in Zone-1 represents a publicly accessible virtual server that client - reaches at IP-1. The client traffic to virtual server 1 at IP-1 will be load balanced across - VM1 and VM2 instances. + Global server load balancing is used to manage traffic flow to a web site hosted on two + separate zones that ideally are in different geographic locations. The following is an + illustration of how GLSB functionality is provided in &PRODUCT;: An organization, xyztelco, + has set up a public cloud that spans two zones, Zone-1 and Zone-2, across geographically + separated data centers that are managed by &PRODUCT;. Tenant-A of the cloud launches a highly + available solution by using xyztelco cloud. For that purpose, they launch two instances each + in both the zones: VM1 and VM2 in Zone-1 and VM5 and VM6 in Zone-2. Tenant-A acquires a public + IP, IP-1 in Zone-1, and configures a load balancer rule to load balance the traffic between + VM1 and VM2 instances. &PRODUCT; orchestrates setting up a virtual server on the LB service + provider in Zone-1. Virtual server 1 that is set up on the LB service provider in Zone-1 + represents a publicly accessible virtual server that client reaches at IP-1. The client + traffic to virtual server 1 at IP-1 will be load balanced across VM1 and VM2 instances. Tenant-A acquires another public IP, IP-2 in Zone-2 and sets up a load balancer rule to load balance the traffic between VM5 and VM6 instances. Similarly in Zone-2, &PRODUCT; orchestrates setting up a virtual server on the LB service provider. Virtual server 2 that is From fa9987753c530c8e9560297986ac8848b7526854 Mon Sep 17 00:00:00 2001 From: Sanjay Tripathi Date: Thu, 9 May 2013 18:01:40 +0530 Subject: [PATCH 05/14] CLOUDSTACK-2419 : Localization - Some Japanese are not working --- services/console-proxy/server/js/ajaxkeys.js | 70 +++++++++++++------- 1 file changed, 45 insertions(+), 25 deletions(-) diff --git a/services/console-proxy/server/js/ajaxkeys.js b/services/console-proxy/server/js/ajaxkeys.js index 1ed41c562d4..5f497bbb785 100644 --- a/services/console-proxy/server/js/ajaxkeys.js +++ b/services/console-proxy/server/js/ajaxkeys.js @@ -155,7 +155,7 @@ KEYBOARD_TYPE_JP = "jp"; KEYBOARD_TYPE_UK = "uk"; //JP keyboard type -// + var keyboardTables = [ {tindex: 0, keyboardType: KEYBOARD_TYPE_COOKED, mappingTable: {X11: [ {keycode: 222, entry: X11_KEY_CIRCUMFLEX_ACCENT}, @@ -220,15 +220,31 @@ var keyboardTables = [ {keycode: JS_KEY_SHIFT, entry : X11_KEY_SHIFT}, {keycode: JS_KEY_CTRL, entry : X11_KEY_CTRL}, {keycode: JS_KEY_ALT, entry : X11_KEY_ALT}, - {keycode: JS_KEY_GRAVE_ACCENT, entry : X11_KEY_GRAVE_ACCENT}, - {keycode: JS_KEY_SUBSTRACT, entry : X11_KEY_SUBSTRACT}, - {keycode: JS_KEY_ADD, entry : X11_KEY_ADD}, - {keycode: JS_KEY_OPEN_BRACKET, entry : X11_KEY_OPEN_BRACKET}, - {keycode: JS_KEY_CLOSE_BRACKET, entry : X11_KEY_CLOSE_BRACKET}, + //{keycode: JS_KEY_GRAVE_ACCENT, entry : X11_KEY_GRAVE_ACCENT}, + //[192 / 64 = "' @"] + {keycode: 192, entry : 0x5b, browser: "IE"}, + {keycode: 64, entry : 0x5b, browser: "Firefox"}, + //{keycode: JS_KEY_ADD, entry : X11_KEY_ADD}, + //[187 / 59 = "; +"] + {keycode: 187, entry : 0x3a, browser: "IE"}, + {keycode: 59, entry : 0x3b, browser: "Firefox"}, + //{keycode: JS_KEY_OPEN_BRACKET, entry : X11_KEY_OPEN_BRACKET}, + //[219 = "[{"] + {keycode: 219, entry : 0x5d, browser: "IE"}, + {keycode: 219, entry : 0x5d, browser: "Firefox"}, + //{keycode: JS_KEY_CLOSE_BRACKET, entry : X11_KEY_CLOSE_BRACKET}, + //[221 = "]}"] + {keycode: 221, entry : 0x5c, browser: "IE"}, + {keycode: 221, entry : 0x5c, browser: "Firefox"}, {keycode: JS_KEY_BACK_SLASH, entry : X11_KEY_BACK_SLASH}, - {keycode: JS_KEY_SINGLE_QUOTE, entry : X11_KEY_SINGLE_QUOTE}, + //{keycode: JS_KEY_SINGLE_QUOTE, entry : X11_KEY_SINGLE_QUOTE}, + //[222 / 160 = "~^"] + {keycode: 222, entry : 0x3d, browser: "IE"}, + {keycode: 160, entry : 0x3d, browser: "Firefox"}, + //[173 = "-=" ] specific to Firefox browser + {keycode: 173, entry : 0x2d, browser: "Firefox"}, {keycode: JS_KEY_COMMA, entry : X11_KEY_COMMA}, - {keycode: JS_KEY_PERIOD, entry : X11_KEY_PERIOD}, + {keycode: JS_KEY_PERIOD, entry : X11_KEY_PERIOD}, {keycode: JS_KEY_FORWARD_SLASH, entry : X11_KEY_FORWARD_SLASH}, {keycode: JS_KEY_DASH, entry : X11_KEY_DASH}, {keycode: JS_KEY_SEMI_COLON, entry : X11_KEY_SEMI_COLON}, @@ -243,8 +259,16 @@ var keyboardTables = [ {keycode: JS_KEY_NUMPAD8, entry : X11_KEY_NUMPAD8}, {keycode: JS_KEY_NUMPAD9, entry : X11_KEY_NUMPAD9}, {keycode: JS_KEY_DECIMAL_POINT, entry : X11_KEY_DECIMAL_POINT}, - {keycode: JS_KEY_DIVIDE, entry : X11_KEY_DIVIDE}, - + {keycode: JS_KEY_DIVIDE, entry : 0xffaf}, + {keycode: JS_KEY_MULTIPLY, entry : 0xffaa}, + {keycode: JS_KEY_ADD, entry : 0xffab}, + {keycode: JS_KEY_SUBSTRACT, entry : 0xffad}, + //Kanji Key = 243 / 244 + {keycode: 243, entry : 0x7e, browser: "IE"}, + {keycode: 244, entry : 0x7e, browser: "IE"}, + //Caps Lock = 240 + {keycode: 240, entry : 0xffe5}, + /* {keycode: JS_KEY_MULTIPLY, entry : [ {type: KEY_DOWN, code: X11_KEY_SHIFT, modifiers: 0 }, {type: KEY_DOWN, code: X11_KEY_ASTERISK, modifiers: 0 }, @@ -252,21 +276,17 @@ var keyboardTables = [ {type: KEY_UP, code: X11_KEY_SHIFT, modifiers: 0 } ]}, {keycode: JS_KEY_ADD, entry : false} - ], - keyPress: [ - {keycode: 61, entry: [ - {type: KEY_DOWN, code: X11_KEY_ADD, modifiers: 0, shift: false }, - {type: KEY_UP, code: X11_KEY_ADD, modifiers: 0, shift: false } - ]}, - {keycode: 43, entry: [ - {type: KEY_DOWN, code: X11_KEY_SHIFT, modifiers: 0, shift: false }, - {type: KEY_DOWN, code: X11_KEY_ADD, modifiers: 0, shift: false }, - {type: KEY_UP, code: X11_KEY_ADD, modifiers: 0, shift: false }, - {type: KEY_UP, code: X11_KEY_SHIFT, modifiers: 0, shift: false }, - {type: KEY_DOWN, code: X11_KEY_ADD, modifiers: 0, shift: true }, - {type: KEY_UP, code: X11_KEY_ADD, modifiers: 0, shift: true } - ]} - ] + */ + //[186 / 58 = "~^"] + {keycode: 186, entry : 0x22, browser: "IE"}, + {keycode: 58, entry : 0x22, browser: "Firefox"}, + ], + keyPress: [ + {keycode: 61, entry: [ + {type: KEY_DOWN, code: X11_KEY_ADD, modifiers: 0, shift: false }, + {type: KEY_UP, code: X11_KEY_ADD, modifiers: 0, shift: false } + ]}, + ] } }, {tindex: 2, keyboardType: KEYBOARD_TYPE_UK, mappingTable: {X11: [], From 3aed20c9d27cbe97afc32fd47909a2f08c4550c8 Mon Sep 17 00:00:00 2001 From: Sanjay Tripathi Date: Wed, 8 May 2013 14:17:20 +0530 Subject: [PATCH 06/14] CLOUDSTACK-1603: Management server .log Java Exceptions are displayed with Alerts --- .../storage/datastore/provider/DefaultHostListener.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/engine/storage/volume/src/org/apache/cloudstack/storage/datastore/provider/DefaultHostListener.java b/engine/storage/volume/src/org/apache/cloudstack/storage/datastore/provider/DefaultHostListener.java index f2cb1c45c82..2f0b43ad9f6 100644 --- a/engine/storage/volume/src/org/apache/cloudstack/storage/datastore/provider/DefaultHostListener.java +++ b/engine/storage/volume/src/org/apache/cloudstack/storage/datastore/provider/DefaultHostListener.java @@ -56,7 +56,7 @@ public class DefaultHostListener implements HypervisorHostListener { } if (!answer.getResult()) { - String msg = "Add host failed due to ModifyStoragePoolCommand failed" + answer.getDetails(); + String msg = "Unable to attach storage pool" + poolId + " to the host" + hostId; alertMgr.sendAlert(AlertManager.ALERT_TYPE_HOST, pool.getDataCenterId(), pool.getPodId(), msg, msg); throw new CloudRuntimeException("Unable establish connection from storage head to storage pool " + pool.getId() + " due to " + answer.getDetails() + pool.getId()); } From e40aba3bc099d8ad49b87829803c58b4a5975717 Mon Sep 17 00:00:00 2001 From: Pranav Saxena Date: Fri, 10 May 2013 11:05:47 +0530 Subject: [PATCH 07/14] CLOUDSTACK-2411:UI:fail to enable conserve mode while creating NO with guest type as shared --- ui/scripts/configuration.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ui/scripts/configuration.js b/ui/scripts/configuration.js index 4a64eeac1a5..9a08c4c56b1 100644 --- a/ui/scripts/configuration.js +++ b/ui/scripts/configuration.js @@ -1210,7 +1210,7 @@ } } }); - if(havingVpcVirtualRouterForAtLeastOneService == true || $guestTypeField.val() == 'Shared') { + if(havingVpcVirtualRouterForAtLeastOneService == true ) { $conservemode.find("input[type=checkbox]").attr("disabled", "disabled"); $conservemode.find("input[type=checkbox]").attr('checked', false); From dfad178a9e334776e4ba6c056523b5bd627ccace Mon Sep 17 00:00:00 2001 From: Likitha Shetty Date: Mon, 6 May 2013 09:58:18 +0530 Subject: [PATCH 08/14] Bug CLOUDSTACK-1390: Allow Root/Domain admin to move a User VM to another user under a different domain Add unit tests --- client/tomcatconf/commands.properties.in | 2 +- .../src/com/cloud/vm/UserVmManagerImpl.java | 14 ++-- .../test/com/cloud/vm/UserVmManagerTest.java | 83 ++++++++++++++++++- 3 files changed, 88 insertions(+), 11 deletions(-) diff --git a/client/tomcatconf/commands.properties.in b/client/tomcatconf/commands.properties.in index cdc19929c19..0a6ec708166 100644 --- a/client/tomcatconf/commands.properties.in +++ b/client/tomcatconf/commands.properties.in @@ -67,7 +67,7 @@ getVMPassword=15 restoreVirtualMachine=15 changeServiceForVirtualMachine=15 scaleVirtualMachine=15 -assignVirtualMachine=1 +assignVirtualMachine=7 migrateVirtualMachine=1 migrateVirtualMachineWithVolume=1 recoverVirtualMachine=7 diff --git a/server/src/com/cloud/vm/UserVmManagerImpl.java b/server/src/com/cloud/vm/UserVmManagerImpl.java index 9fbc5099374..f7f5fc7ad74 100755 --- a/server/src/com/cloud/vm/UserVmManagerImpl.java +++ b/server/src/com/cloud/vm/UserVmManagerImpl.java @@ -3725,19 +3725,14 @@ public class UserVmManagerImpl extends ManagerBase implements UserVmManager, Use + cmd.getAccountName() + " is disabled."); } - // make sure the accounts are under same domain - if (oldAccount.getDomainId() != newAccount.getDomainId()) { - throw new InvalidParameterValueException( - "The account should be under same domain for moving VM between two accounts. Old owner domain =" - + oldAccount.getDomainId() - + " New owner domain=" - + newAccount.getDomainId()); - } + //check caller has access to both the old and new account + _accountMgr.checkAccess(caller, null, true, oldAccount); + _accountMgr.checkAccess(caller, null, true, newAccount); // make sure the accounts are not same if (oldAccount.getAccountId() == newAccount.getAccountId()) { throw new InvalidParameterValueException( - "The account should be same domain for moving VM between two accounts. Account id =" + "The new account is the same as the old account. Account id =" + oldAccount.getAccountId()); } @@ -3829,6 +3824,7 @@ public class UserVmManagerImpl extends ManagerBase implements UserVmManager, Use _resourceLimitMgr.decrementResourceCount(oldAccount.getAccountId(), ResourceType.primary_storage, new Long(volume.getSize())); volume.setAccountId(newAccount.getAccountId()); + volume.setDomainId(newAccount.getDomainId()); _volsDao.persist(volume); _resourceLimitMgr.incrementResourceCount(newAccount.getAccountId(), ResourceType.volume); _resourceLimitMgr.incrementResourceCount(newAccount.getAccountId(), ResourceType.primary_storage, diff --git a/server/test/com/cloud/vm/UserVmManagerTest.java b/server/test/com/cloud/vm/UserVmManagerTest.java index 08f2a9c2abc..dfd7465aba0 100755 --- a/server/test/com/cloud/vm/UserVmManagerTest.java +++ b/server/test/com/cloud/vm/UserVmManagerTest.java @@ -17,19 +17,26 @@ package com.cloud.vm; +import static org.mockito.Matchers.any; import static org.mockito.Matchers.anyBoolean; import static org.mockito.Matchers.anyFloat; import static org.mockito.Matchers.anyInt; import static org.mockito.Matchers.anyLong; +import static org.mockito.Matchers.anyString; import static org.mockito.Matchers.eq; import static org.mockito.Mockito.doNothing; import static org.mockito.Mockito.doReturn; +import static org.mockito.Mockito.doThrow; import static org.mockito.Mockito.when; import java.lang.reflect.Field; import java.util.List; +import java.util.UUID; +import org.apache.cloudstack.acl.ControlledEntity; +import org.apache.cloudstack.acl.SecurityChecker.AccessType; import org.apache.cloudstack.api.ServerApiException; +import org.apache.cloudstack.api.command.admin.vm.AssignVMCmd; import org.apache.cloudstack.api.command.user.vm.RestoreVMCmd; import org.apache.cloudstack.api.command.user.vm.ScaleVMCmd; import org.junit.Before; @@ -44,9 +51,11 @@ import com.cloud.configuration.dao.ConfigurationDao; import com.cloud.exception.ConcurrentOperationException; import com.cloud.exception.InsufficientCapacityException; import com.cloud.exception.InvalidParameterValueException; +import com.cloud.exception.PermissionDeniedException; import com.cloud.exception.ResourceAllocationException; import com.cloud.exception.ResourceUnavailableException; import com.cloud.hypervisor.Hypervisor; +import com.cloud.hypervisor.Hypervisor.HypervisorType; import com.cloud.offering.ServiceOffering; import com.cloud.service.ServiceOfferingVO; import com.cloud.storage.VMTemplateVO; @@ -57,6 +66,7 @@ import com.cloud.storage.dao.VMTemplateDao; import com.cloud.storage.dao.VolumeDao; import com.cloud.user.Account; import com.cloud.user.AccountManager; +import com.cloud.user.AccountService; import com.cloud.user.AccountVO; import com.cloud.user.UserContext; import com.cloud.user.UserVO; @@ -73,6 +83,7 @@ public class UserVmManagerTest { @Mock VolumeManager _storageMgr; @Mock Account _account; @Mock AccountManager _accountMgr; + @Mock AccountService _accountService; @Mock ConfigurationManager _configMgr; @Mock CapacityManager _capacityMgr; @Mock AccountDao _accountDao; @@ -91,6 +102,7 @@ public class UserVmManagerTest { @Mock VMTemplateVO _templateMock; @Mock VolumeVO _volumeMock; @Mock List _rootVols; + @Mock Account _accountMock2; @Before public void setup(){ MockitoAnnotations.initMocks(this); @@ -102,6 +114,7 @@ public class UserVmManagerTest { _userVmMgr._itMgr = _itMgr; _userVmMgr.volumeMgr = _storageMgr; _userVmMgr._accountDao = _accountDao; + _userVmMgr._accountService = _accountService; _userVmMgr._userDao = _userDao; _userVmMgr._accountMgr = _accountMgr; _userVmMgr._configMgr = _configMgr; @@ -370,6 +383,74 @@ public class UserVmManagerTest { return serviceOffering; } - + // Test Move VM b/w accounts where caller is not ROOT/Domain admin + @Test(expected=InvalidParameterValueException.class) + public void testMoveVmToUser1() throws Exception { + AssignVMCmd cmd = new AssignVMCmd(); + Class _class = cmd.getClass(); + + Field virtualmachineIdField = _class.getDeclaredField("virtualMachineId"); + virtualmachineIdField.setAccessible(true); + virtualmachineIdField.set(cmd, 1L); + + Field accountNameField = _class.getDeclaredField("accountName"); + accountNameField.setAccessible(true); + accountNameField.set(cmd, "account"); + + Field domainIdField = _class.getDeclaredField("domainId"); + domainIdField.setAccessible(true); + domainIdField.set(cmd, 1L); + + // caller is of type 0 + Account caller = (Account) new AccountVO("testaccount", 1, "networkdomain", (short) 0, + UUID.randomUUID().toString()); + UserContext.registerContext(1, caller, null, true); + + _userVmMgr.moveVMToUser(cmd); + } + + + // Test Move VM b/w accounts where caller doesn't have access to the old or new account + @Test(expected=PermissionDeniedException.class) + public void testMoveVmToUser2() throws Exception { + AssignVMCmd cmd = new AssignVMCmd(); + Class _class = cmd.getClass(); + + Field virtualmachineIdField = _class.getDeclaredField("virtualMachineId"); + virtualmachineIdField.setAccessible(true); + virtualmachineIdField.set(cmd, 1L); + + Field accountNameField = _class.getDeclaredField("accountName"); + accountNameField.setAccessible(true); + accountNameField.set(cmd, "account"); + + Field domainIdField = _class.getDeclaredField("domainId"); + domainIdField.setAccessible(true); + domainIdField.set(cmd, 1L); + + // caller is of type 0 + Account caller = (Account) new AccountVO("testaccount", 1, "networkdomain", (short) 1, + UUID.randomUUID().toString()); + UserContext.registerContext(1, caller, null, true); + + Account oldAccount = (Account) new AccountVO("testaccount", 1, "networkdomain", (short) 0, + UUID.randomUUID().toString()); + Account newAccount = (Account) new AccountVO("testaccount", 1, "networkdomain", (short) 1, + UUID.randomUUID().toString()); + + UserVmVO vm = new UserVmVO(10L, "test", "test", 1L, HypervisorType.Any, 1L, false, false, 1L, 1L, + 5L, "test", "test", 1L); + vm.setState(VirtualMachine.State.Stopped); + when(_vmDao.findById(anyLong())).thenReturn(vm); + + when(_accountService.getActiveAccountById(anyLong())).thenReturn(oldAccount); + + when(_accountService.getActiveAccountByName(anyString(), anyLong())).thenReturn(newAccount); + + doThrow(new PermissionDeniedException("Access check failed")).when(_accountMgr).checkAccess(any(Account.class), any(AccessType.class), + any(Boolean.class), any(ControlledEntity.class)); + + _userVmMgr.moveVMToUser(cmd); + } } \ No newline at end of file From c704c90b1163d0f3386370e75fc5903dd76498c5 Mon Sep 17 00:00:00 2001 From: Koushik Das Date: Fri, 10 May 2013 16:35:15 +0530 Subject: [PATCH 09/14] CLOUDSTACK-2035: Fix source NAT configuration with Cisco VNMC/ASA Due to VNMC limitation source nat ip cannot be assigned to ASA 1000v outside interface. Working around this issue by acquiring additional public ip during network implement and assigning that to outside interface of ASA. Also made changes to ensure that source nat policy comes after pf and static nat policies in terms of evaluation by assigning a high 'order' value for it. --- .../cisco/CiscoVnmcConnectionImpl.java | 35 ++++++++++++------- .../network/element/CiscoVnmcElement.java | 33 ++++++++++++++--- 2 files changed, 50 insertions(+), 18 deletions(-) diff --git a/plugins/network-elements/cisco-vnmc/src/com/cloud/network/cisco/CiscoVnmcConnectionImpl.java b/plugins/network-elements/cisco-vnmc/src/com/cloud/network/cisco/CiscoVnmcConnectionImpl.java index 527fb04698e..714eb82a559 100644 --- a/plugins/network-elements/cisco-vnmc/src/com/cloud/network/cisco/CiscoVnmcConnectionImpl.java +++ b/plugins/network-elements/cisco-vnmc/src/com/cloud/network/cisco/CiscoVnmcConnectionImpl.java @@ -279,7 +279,7 @@ public class CiscoVnmcConnectionImpl implements CiscoVnmcConnection { String xml = VnmcXml.CREATE_VDC.getXml(); String service = VnmcXml.CREATE_VDC.getService(); xml = replaceXmlValue(xml, "cookie", _cookie); - xml = replaceXmlValue(xml, "descr", "VDC for Tenant" + tenantName); + xml = replaceXmlValue(xml, "descr", "VDC for Tenant " + tenantName); xml = replaceXmlValue(xml, "name", getNameForTenantVDC(tenantName)); xml = replaceXmlValue(xml, "dn", getDnForTenantVDC(tenantName)); @@ -304,7 +304,7 @@ public class CiscoVnmcConnectionImpl implements CiscoVnmcConnection { String xml = VnmcXml.CREATE_EDGE_DEVICE_PROFILE.getXml(); String service = VnmcXml.CREATE_EDGE_DEVICE_PROFILE.getService(); xml = replaceXmlValue(xml, "cookie", _cookie); - xml = replaceXmlValue(xml, "descr", "Edge Device Profile for Tenant VDC" + tenantName); + xml = replaceXmlValue(xml, "descr", "Edge Device Profile for Tenant VDC " + tenantName); xml = replaceXmlValue(xml, "name", getNameForEdgeDeviceServiceProfile(tenantName)); xml = replaceXmlValue(xml, "dn", getDnForTenantVDCEdgeDeviceProfile(tenantName)); @@ -407,7 +407,7 @@ public class CiscoVnmcConnectionImpl implements CiscoVnmcConnection { String xml = VnmcXml.CREATE_EDGE_SECURITY_PROFILE.getXml(); String service = VnmcXml.CREATE_EDGE_SECURITY_PROFILE.getService(); xml = replaceXmlValue(xml, "cookie", _cookie); - xml = replaceXmlValue(xml, "descr", "Edge Security Profile for Tenant VDC" + tenantName); + xml = replaceXmlValue(xml, "descr", "Edge Security Profile for Tenant VDC " + tenantName); xml = replaceXmlValue(xml, "name", getNameForEdgeDeviceSecurityProfile(tenantName)); xml = replaceXmlValue(xml, "espdn", getDnForTenantVDCEdgeSecurityProfile(tenantName)); xml = replaceXmlValue(xml, "egressref", "default-egress"); @@ -505,7 +505,8 @@ public class CiscoVnmcConnectionImpl implements CiscoVnmcConnection { return createTenantVDCNatPolicyRef( getDnForSourceNatPolicyRef(tenantName), getNameForSourceNatPolicy(tenantName), - tenantName); + tenantName, + true); } @Override @@ -545,7 +546,7 @@ public class CiscoVnmcConnectionImpl implements CiscoVnmcConnection { String xml = VnmcXml.RESOLVE_NAT_POLICY_SET.getXml(); String service = VnmcXml.RESOLVE_NAT_POLICY_SET.getService(); xml = replaceXmlValue(xml, "cookie", _cookie); - xml = replaceXmlValue(xml, "descr", "Edge Security Profile for Tenant VDC" + tenantName); + xml = replaceXmlValue(xml, "descr", "Edge Security Profile for Tenant VDC " + tenantName); xml = replaceXmlValue(xml, "name", getNameForEdgeDeviceSecurityProfile(tenantName)); xml = replaceXmlValue(xml, "espdn", getDnForTenantVDCEdgeSecurityProfile(tenantName)); xml = replaceXmlValue(xml, "natpolicysetname", getNameForNatPolicySet(tenantName)); @@ -656,7 +657,7 @@ public class CiscoVnmcConnectionImpl implements CiscoVnmcConnection { String xml = VnmcXml.RESOLVE_ACL_POLICY_SET.getXml(); String service = VnmcXml.RESOLVE_ACL_POLICY_SET.getService(); xml = replaceXmlValue(xml, "cookie", _cookie); - xml = replaceXmlValue(xml, "descr", "Edge Security Profile for Tenant VDC" + tenantName); + xml = replaceXmlValue(xml, "descr", "Edge Security Profile for Tenant VDC " + tenantName); xml = replaceXmlValue(xml, "name", getNameForEdgeDeviceSecurityProfile(tenantName)); xml = replaceXmlValue(xml, "espdn", getDnForTenantVDCEdgeSecurityProfile(tenantName)); //xml = replaceXmlValue(xml, "egresspolicysetname", getNameForAclPolicySet(tenantName, false)); @@ -838,17 +839,23 @@ public class CiscoVnmcConnectionImpl implements CiscoVnmcConnection { return verifySuccess(response); } - private boolean createTenantVDCNatPolicyRef(String policyRefDn, String name, String tenantName) throws ExecutionException { + private boolean createTenantVDCNatPolicyRef(String policyRefDn, String name, String tenantName, boolean isSourceNat) throws ExecutionException { String xml = VnmcXml.CREATE_NAT_POLICY_REF.getXml(); String service = VnmcXml.CREATE_NAT_POLICY_REF.getService(); xml = replaceXmlValue(xml, "cookie", _cookie); xml = replaceXmlValue(xml, "natpolicyrefdn", policyRefDn); xml = replaceXmlValue(xml, "natpolicyname", name); - List policies = listNatPolicies(tenantName); - int order = 100; - if (policies != null) { - order += policies.size(); + // PF and static NAT policies need to come before source NAT, so leaving buffer + // and creating source NAT with a high order value. + // Initially tried setting MAX_INT as the order but VNMC complains about it + int order = 10000; // TODO: For now value should be sufficient, if required may need to increase + if (!isSourceNat) { + List policies = listNatPolicies(tenantName); + order = 100; // order starts at 100 + if (policies != null) { + order += policies.size(); + } } xml = replaceXmlValue(xml, "order", Integer.toString(order)); @@ -1062,7 +1069,8 @@ public class CiscoVnmcConnectionImpl implements CiscoVnmcConnection { return createTenantVDCNatPolicyRef( getDnForPFPolicyRef(tenantName, identifier), getNameForPFPolicy(tenantName, identifier), - tenantName); + tenantName, + false); } @Override @@ -1180,7 +1188,8 @@ public class CiscoVnmcConnectionImpl implements CiscoVnmcConnection { return createTenantVDCNatPolicyRef( getDnForDNatPolicyRef(tenantName, identifier), getNameForDNatPolicy(tenantName, identifier), - tenantName); + tenantName, + false); } @Override diff --git a/plugins/network-elements/cisco-vnmc/src/com/cloud/network/element/CiscoVnmcElement.java b/plugins/network-elements/cisco-vnmc/src/com/cloud/network/element/CiscoVnmcElement.java index 470c4e88217..33cc40a67da 100644 --- a/plugins/network-elements/cisco-vnmc/src/com/cloud/network/element/CiscoVnmcElement.java +++ b/plugins/network-elements/cisco-vnmc/src/com/cloud/network/element/CiscoVnmcElement.java @@ -70,6 +70,7 @@ import com.cloud.deploy.DeployDestination; import com.cloud.exception.ConcurrentOperationException; import com.cloud.exception.InsufficientCapacityException; import com.cloud.exception.InvalidParameterValueException; +import com.cloud.exception.ResourceAllocationException; import com.cloud.exception.ResourceUnavailableException; import com.cloud.host.DetailVO; import com.cloud.host.Host; @@ -113,6 +114,7 @@ import com.cloud.resource.ResourceStateAdapter; import com.cloud.resource.ServerResource; import com.cloud.resource.UnableDeleteHostException; import com.cloud.user.Account; +import com.cloud.user.UserContext; import com.cloud.utils.component.AdapterBase; import com.cloud.utils.db.Transaction; import com.cloud.utils.exception.CloudRuntimeException; @@ -338,10 +340,31 @@ public class CiscoVnmcElement extends AdapterBase implements SourceNatServicePro publicGateways.add(vlanVO.getVlanGateway()); } + // due to VNMC limitation of not allowing source NAT ip as the outside ip of firewall, + // an additional public ip needs to acquired for assigning as firewall outside ip + IpAddress outsideIp = null; + try { + Account caller = UserContext.current().getCaller(); + long callerUserId = UserContext.current().getCallerUserId(); + outsideIp = _networkMgr.allocateIp(owner, false, caller, callerUserId, zone); + } catch (ResourceAllocationException e) { + s_logger.error("Unable to allocate additional public Ip address. Exception details " + e); + return false; + } + + try { + outsideIp = _networkMgr.associateIPToGuestNetwork(outsideIp.getId(), network.getId(), true); + } catch (ResourceAllocationException e) { + s_logger.error("Unable to assign allocated additional public Ip " + outsideIp.getAddress().addr() + " to network with vlan " + vlanId + ". Exception details " + e); + return false; + } + // create logical edge firewall in VNMC String gatewayNetmask = NetUtils.getCidrNetmask(network.getCidr()); + // due to ASA limitation of allowing single subnet to be assigned to firewall interfaces, + // all public ip addresses must be from same subnet, this essentially means single public subnet in zone if (!createLogicalEdgeFirewall(vlanId, network.getGateway(), gatewayNetmask, - sourceNatIp.getAddress().addr(), sourceNatIp.getNetmask(), publicGateways, ciscoVnmcHost.getId())) { + outsideIp.getAddress().addr(), sourceNatIp.getNetmask(), publicGateways, ciscoVnmcHost.getId())) { s_logger.error("Failed to create logical edge firewall in Cisco VNMC device for network " + network.getName()); return false; } @@ -356,10 +379,10 @@ public class CiscoVnmcElement extends AdapterBase implements SourceNatServicePro } // configure source NAT - //if (!configureSourceNat(vlanId, network.getCidr(), sourceNatIp, ciscoVnmcHost.getId())) { - // s_logger.error("Failed to configure source NAT in Cisco VNMC device for network " + network.getName()); - // return false; - //} + if (!configureSourceNat(vlanId, network.getCidr(), sourceNatIp, ciscoVnmcHost.getId())) { + s_logger.error("Failed to configure source NAT in Cisco VNMC device for network " + network.getName()); + return false; + } // associate Asa 1000v instance with logical edge firewall if (!associateAsaWithLogicalEdgeFirewall(vlanId, assignedAsa.getManagementIp(), ciscoVnmcHost.getId())) { From edd2fbb266d316fc130520876dfb2f7152223d37 Mon Sep 17 00:00:00 2001 From: Pranav Saxena Date: Fri, 10 May 2013 16:47:09 +0530 Subject: [PATCH 10/14] CLOUDSTACK-760:Allow ACL on all layer 4 protocols --- ui/scripts/vpc.js | 33 +++++++++++++++++++++++++++++++-- 1 file changed, 31 insertions(+), 2 deletions(-) diff --git a/ui/scripts/vpc.js b/ui/scripts/vpc.js index db964e6ffcd..dc26265c232 100644 --- a/ui/scripts/vpc.js +++ b/ui/scripts/vpc.js @@ -48,6 +48,22 @@ return name != 'icmptype' && name != 'icmpcode' && name != 'cidrlist'; }); + var $protocolinput = args.$form.find('th,td'); + var $protocolFields = $protocolinput.filter(function(){ + var name = $(this).attr('rel'); + + return $.inArray(name,['protocolnumber']) > -1; + }); + + if($(this).val() == 'protocolnumber' ){ + + $protocolFields.show(); + } + else{ + $protocolFields.hide(); + } + + if ($(this).val() == 'icmp') { $icmpFields.show(); $icmpFields.attr('disabled', false); @@ -68,11 +84,16 @@ data: [ { name: 'tcp', description: 'TCP' }, { name: 'udp', description: 'UDP' }, - { name: 'icmp', description: 'ICMP' } + { name: 'icmp', description: 'ICMP' }, + { name: 'all', description: 'ALL'}, + { name: 'protocolnumber', description: 'Protocol Number'} + ] }); } }, + + 'protocolnumber': {label:'Protocol Number',isDisabled:true,isHidden:true,edit:true}, 'startport': { edit: true, label: 'label.start.port' }, 'endport': { edit: true, label: 'label.end.port' }, 'networkid': { @@ -136,7 +157,15 @@ label: 'label.add', action: function(args) { var $multi = args.$multi; - + //Support for Protocol Number between 0 to 255 + if(args.data.protocol == 'protocolnumber'){ + $.extend(args.data,{protocol:args.data.protocolnumber}); + delete args.data.protocolnumber; + } + else + delete args.data.protocolnumber; + + $.ajax({ url: createURL('createNetworkACL'), data: $.extend(args.data, { From b4aff6190f741cffa0bd945cfa9d7d7b11ada47b Mon Sep 17 00:00:00 2001 From: Kishan Kavala Date: Fri, 10 May 2013 17:27:10 +0530 Subject: [PATCH 11/14] CLOUDSTACK-2309: Add fix to handle Vmware 4.0 template upgrade --- .../com/cloud/upgrade/dao/Upgrade302to40.java | 51 ++++++++++++++++++- 1 file changed, 50 insertions(+), 1 deletion(-) diff --git a/engine/schema/src/com/cloud/upgrade/dao/Upgrade302to40.java b/engine/schema/src/com/cloud/upgrade/dao/Upgrade302to40.java index 753f64ec682..ecda872dfa4 100644 --- a/engine/schema/src/com/cloud/upgrade/dao/Upgrade302to40.java +++ b/engine/schema/src/com/cloud/upgrade/dao/Upgrade302to40.java @@ -63,6 +63,7 @@ public class Upgrade302to40 extends Upgrade30xBase implements DbUpgrade { @Override public void performDataMigration(Connection conn) { + updateVmWareSystemVms(conn); correctVRProviders(conn); correctMultiplePhysicaNetworkSetups(conn); addHostDetailsUniqueKey(conn); @@ -82,7 +83,55 @@ public class Upgrade302to40 extends Upgrade30xBase implements DbUpgrade { return new File[] { new File(script) }; } - + + private void updateVmWareSystemVms(Connection conn){ + PreparedStatement pstmt = null; + ResultSet rs = null; + boolean VMware = false; + try { + pstmt = conn.prepareStatement("select distinct(hypervisor_type) from `cloud`.`cluster` where removed is null"); + rs = pstmt.executeQuery(); + while(rs.next()){ + if("VMware".equals(rs.getString(1))){ + VMware = true; + } + } + } catch (SQLException e) { + throw new CloudRuntimeException("Error while iterating through list of hypervisors in use", e); + } + // Just update the VMware system template. Other hypervisor templates are unchanged from previous 3.0.x versions. + s_logger.debug("Updating VMware System Vms"); + try { + //Get 4.0 VMware system Vm template Id + pstmt = conn.prepareStatement("select id from `cloud`.`vm_template` where name = 'systemvm-vmware-4.0' and removed is null"); + rs = pstmt.executeQuery(); + if(rs.next()){ + long templateId = rs.getLong(1); + rs.close(); + pstmt.close(); + // change template type to SYSTEM + pstmt = conn.prepareStatement("update `cloud`.`vm_template` set type='SYSTEM' where id = ?"); + pstmt.setLong(1, templateId); + pstmt.executeUpdate(); + pstmt.close(); + // update templete ID of system Vms + pstmt = conn.prepareStatement("update `cloud`.`vm_instance` set vm_template_id = ? where type <> 'User' and hypervisor_type = 'VMware'"); + pstmt.setLong(1, templateId); + pstmt.executeUpdate(); + pstmt.close(); + } else { + if (VMware){ + throw new CloudRuntimeException("4.0 VMware SystemVm template not found. Cannot upgrade system Vms"); + } else { + s_logger.warn("4.0 VMware SystemVm template not found. VMware hypervisor is not used, so not failing upgrade"); + } + } + } catch (SQLException e) { + throw new CloudRuntimeException("Error while updating VMware systemVm template", e); + } + s_logger.debug("Updating System Vm Template IDs Complete"); + } + private void correctVRProviders(Connection conn) { PreparedStatement pstmtVR = null; ResultSet rsVR = null; From 530b0beb3c8b172fbfaf89584cd24785b7513998 Mon Sep 17 00:00:00 2001 From: Koushik Das Date: Fri, 10 May 2013 17:54:59 +0530 Subject: [PATCH 12/14] CLOUDSTACK-2417: NPE while creating Egress rules with Networking using Cisco ASA firewall provider An input parameter was incorrectly interpreted during egress rule creation and so resulted in NPE. Created a new vnmc xml for handling creation of egress rule with protocol as 'All' --- .../network/cisco/create-egress-acl-rule.xml | 53 ++--------- ...te-generic-egress-acl-no-protocol-rule.xml | 94 +++++++++++++++++++ .../cisco/create-generic-egress-acl-rule.xml | 1 - .../network/cisco/create-ingress-acl-rule.xml | 43 +-------- .../network/cisco/CiscoVnmcConnection.java | 10 +- .../cisco/CiscoVnmcConnectionImpl.java | 30 +++--- .../network/element/CiscoVnmcElement.java | 9 +- .../network/resource/CiscoVnmcResource.java | 12 +-- .../resource/CiscoVnmcResourceTest.java | 4 +- 9 files changed, 144 insertions(+), 112 deletions(-) create mode 100755 plugins/network-elements/cisco-vnmc/scripts/network/cisco/create-generic-egress-acl-no-protocol-rule.xml diff --git a/plugins/network-elements/cisco-vnmc/scripts/network/cisco/create-egress-acl-rule.xml b/plugins/network-elements/cisco-vnmc/scripts/network/cisco/create-egress-acl-rule.xml index 930272ed8ee..05c066d6d53 100755 --- a/plugins/network-elements/cisco-vnmc/scripts/network/cisco/create-egress-acl-rule.xml +++ b/plugins/network-elements/cisco-vnmc/scripts/network/cisco/create-egress-acl-rule.xml @@ -118,70 +118,38 @@ under the License. - - - - - - - - - - - - - - + + value="%deststartport%"/> - + + value="%destendport%"/> @@ -195,7 +163,6 @@ under the License. protocolvalue = "TCP" or "UDP" deststartip="destination start ip" destendip="destination end ip" - sourcestartport="start port at source" - sourceendport="end port at source" - sourceip="source ip" + deststartport="start port at destination" + destendport="end port at destination" --!> diff --git a/plugins/network-elements/cisco-vnmc/scripts/network/cisco/create-generic-egress-acl-no-protocol-rule.xml b/plugins/network-elements/cisco-vnmc/scripts/network/cisco/create-generic-egress-acl-no-protocol-rule.xml new file mode 100755 index 00000000000..17cfa54a34e --- /dev/null +++ b/plugins/network-elements/cisco-vnmc/scripts/network/cisco/create-generic-egress-acl-no-protocol-rule.xml @@ -0,0 +1,94 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/plugins/network-elements/cisco-vnmc/scripts/network/cisco/create-generic-egress-acl-rule.xml b/plugins/network-elements/cisco-vnmc/scripts/network/cisco/create-generic-egress-acl-rule.xml index 92c25043dad..436e3eae790 100755 --- a/plugins/network-elements/cisco-vnmc/scripts/network/cisco/create-generic-egress-acl-rule.xml +++ b/plugins/network-elements/cisco-vnmc/scripts/network/cisco/create-generic-egress-acl-rule.xml @@ -118,5 +118,4 @@ under the License. protocolvalue = "TCP" or "UDP" or "ICMP" deststartip="destination start ip" destendip="destination end ip" - sourceip="source ip" --!> diff --git a/plugins/network-elements/cisco-vnmc/scripts/network/cisco/create-ingress-acl-rule.xml b/plugins/network-elements/cisco-vnmc/scripts/network/cisco/create-ingress-acl-rule.xml index 1af30b44416..f283ffeb333 100755 --- a/plugins/network-elements/cisco-vnmc/scripts/network/cisco/create-ingress-acl-rule.xml +++ b/plugins/network-elements/cisco-vnmc/scripts/network/cisco/create-ingress-acl-rule.xml @@ -118,7 +118,7 @@ under the License. @@ -127,56 +127,24 @@ under the License. dn="%aclruledn%/rule-cond-4/nw-expr2/nw-attr-qual" status="created"/> - - - - - - - - - - - - - - + - + diff --git a/plugins/network-elements/cisco-vnmc/src/com/cloud/network/cisco/CiscoVnmcConnection.java b/plugins/network-elements/cisco-vnmc/src/com/cloud/network/cisco/CiscoVnmcConnection.java index f137148ab48..fed6724418d 100644 --- a/plugins/network-elements/cisco-vnmc/src/com/cloud/network/cisco/CiscoVnmcConnection.java +++ b/plugins/network-elements/cisco-vnmc/src/com/cloud/network/cisco/CiscoVnmcConnection.java @@ -140,23 +140,23 @@ public interface CiscoVnmcConnection { public boolean createTenantVDCIngressAclRule(String tenantName, String identifier, String policyIdentifier, String protocol, String sourceStartIp, String sourceEndIp, - String destStartPort, String destEndPort, String destIp) + String destStartPort, String destEndPort) throws ExecutionException; public boolean createTenantVDCIngressAclRule(String tenantName, String identifier, String policyIdentifier, - String protocol, String sourceStartIp, String sourceEndIp, String destIp) + String protocol, String sourceStartIp, String sourceEndIp) throws ExecutionException; public boolean createTenantVDCEgressAclRule(String tenantName, String identifier, String policyIdentifier, - String protocol, String sourceStartPort, String sourceEndPort, String sourceIp, - String destStartIp, String destEndIp) + String protocol, String destStartIp, String destEndIp, + String destStartPort, String destEndPort) throws ExecutionException; public boolean createTenantVDCEgressAclRule(String tenantName, String identifier, String policyIdentifier, - String protocol, String sourceIp, String destStartIp, String destEndIp) + String protocol, String destStartIp, String destEndIp) throws ExecutionException; public boolean deleteTenantVDCAclRule(String tenantName, diff --git a/plugins/network-elements/cisco-vnmc/src/com/cloud/network/cisco/CiscoVnmcConnectionImpl.java b/plugins/network-elements/cisco-vnmc/src/com/cloud/network/cisco/CiscoVnmcConnectionImpl.java index 714eb82a559..c7380ab11d8 100644 --- a/plugins/network-elements/cisco-vnmc/src/com/cloud/network/cisco/CiscoVnmcConnectionImpl.java +++ b/plugins/network-elements/cisco-vnmc/src/com/cloud/network/cisco/CiscoVnmcConnectionImpl.java @@ -95,6 +95,7 @@ public class CiscoVnmcConnectionImpl implements CiscoVnmcConnection { CREATE_EGRESS_ACL_RULE("create-egress-acl-rule.xml", "policy-mgr"), CREATE_GENERIC_INGRESS_ACL_RULE("create-generic-ingress-acl-rule.xml", "policy-mgr"), CREATE_GENERIC_EGRESS_ACL_RULE("create-generic-egress-acl-rule.xml", "policy-mgr"), + CREATE_GENERIC_EGRESS_ACL_NO_PROTOCOL_RULE("create-generic-egress-acl-no-protocol-rule.xml", "policy-mgr"), DELETE_RULE("delete-rule.xml", "policy-mgr"), @@ -660,8 +661,7 @@ public class CiscoVnmcConnectionImpl implements CiscoVnmcConnection { xml = replaceXmlValue(xml, "descr", "Edge Security Profile for Tenant VDC " + tenantName); xml = replaceXmlValue(xml, "name", getNameForEdgeDeviceSecurityProfile(tenantName)); xml = replaceXmlValue(xml, "espdn", getDnForTenantVDCEdgeSecurityProfile(tenantName)); - //xml = replaceXmlValue(xml, "egresspolicysetname", getNameForAclPolicySet(tenantName, false)); - xml = replaceXmlValue(xml, "egresspolicysetname", "default-egress"); + xml = replaceXmlValue(xml, "egresspolicysetname", getNameForAclPolicySet(tenantName, false)); xml = replaceXmlValue(xml, "ingresspolicysetname", getNameForAclPolicySet(tenantName, true)); xml = replaceXmlValue(xml, "natpolicysetname", getNameForNatPolicySet(tenantName)); @@ -673,7 +673,7 @@ public class CiscoVnmcConnectionImpl implements CiscoVnmcConnection { public boolean createTenantVDCIngressAclRule(String tenantName, String identifier, String policyIdentifier, String protocol, String sourceStartIp, String sourceEndIp, - String destStartPort, String destEndPort, String destIp) throws ExecutionException { + String destStartPort, String destEndPort) throws ExecutionException { String xml = VnmcXml.CREATE_INGRESS_ACL_RULE.getXml(); String service = VnmcXml.CREATE_INGRESS_ACL_RULE.getService(); @@ -687,7 +687,6 @@ public class CiscoVnmcConnectionImpl implements CiscoVnmcConnection { xml = replaceXmlValue(xml, "sourceendip", sourceEndIp); xml = replaceXmlValue(xml, "deststartport", destStartPort); xml = replaceXmlValue(xml, "destendport", destEndPort); - xml = replaceXmlValue(xml, "destip", destIp); List rules = listChildren(getDnForAclPolicy(tenantName, policyIdentifier)); int order = 100; @@ -703,8 +702,7 @@ public class CiscoVnmcConnectionImpl implements CiscoVnmcConnection { @Override public boolean createTenantVDCIngressAclRule(String tenantName, String identifier, String policyIdentifier, - String protocol, String sourceStartIp, String sourceEndIp, - String destIp) throws ExecutionException { + String protocol, String sourceStartIp, String sourceEndIp) throws ExecutionException { String xml = VnmcXml.CREATE_GENERIC_INGRESS_ACL_RULE.getXml(); String service = VnmcXml.CREATE_GENERIC_INGRESS_ACL_RULE.getService(); @@ -731,8 +729,8 @@ public class CiscoVnmcConnectionImpl implements CiscoVnmcConnection { @Override public boolean createTenantVDCEgressAclRule(String tenantName, String identifier, String policyIdentifier, - String protocol, String sourceStartPort, String sourceEndPort, String sourceIp, - String destStartIp, String destEndIp) throws ExecutionException { + String protocol, String destStartIp, String destEndIp, + String destStartPort, String destEndPort) throws ExecutionException { String xml = VnmcXml.CREATE_EGRESS_ACL_RULE.getXml(); String service = VnmcXml.CREATE_EGRESS_ACL_RULE.getService(); @@ -744,9 +742,8 @@ public class CiscoVnmcConnectionImpl implements CiscoVnmcConnection { xml = replaceXmlValue(xml, "protocolvalue", protocol); xml = replaceXmlValue(xml, "deststartip", destStartIp); xml = replaceXmlValue(xml, "destendip", destEndIp); - xml = replaceXmlValue(xml, "sourcestartport", sourceStartPort); - xml = replaceXmlValue(xml, "sourceendport", sourceEndPort); - xml = replaceXmlValue(xml, "sourceip", sourceIp); + xml = replaceXmlValue(xml, "deststartport", destStartPort); + xml = replaceXmlValue(xml, "destendport", destEndPort); List rules = listChildren(getDnForAclPolicy(tenantName, policyIdentifier)); int order = 100; @@ -762,17 +759,20 @@ public class CiscoVnmcConnectionImpl implements CiscoVnmcConnection { @Override public boolean createTenantVDCEgressAclRule(String tenantName, String identifier, String policyIdentifier, - String protocol, String sourceIp, - String destStartIp, String destEndIp) throws ExecutionException { + String protocol, String destStartIp, String destEndIp) throws ExecutionException { String xml = VnmcXml.CREATE_GENERIC_EGRESS_ACL_RULE.getXml(); String service = VnmcXml.CREATE_GENERIC_EGRESS_ACL_RULE.getService(); - + if (protocol.equalsIgnoreCase("all")) { // any protocol + xml = VnmcXml.CREATE_GENERIC_EGRESS_ACL_NO_PROTOCOL_RULE.getXml(); + service = VnmcXml.CREATE_GENERIC_EGRESS_ACL_NO_PROTOCOL_RULE.getService(); + } else { // specific protocol + xml = replaceXmlValue(xml, "protocolvalue", protocol); + } xml = replaceXmlValue(xml, "cookie", _cookie); xml = replaceXmlValue(xml, "aclruledn", getDnForAclRule(tenantName, identifier, policyIdentifier)); xml = replaceXmlValue(xml, "aclrulename", getNameForAclRule(tenantName, identifier)); xml = replaceXmlValue(xml, "descr", "Egress ACL rule for Tenant VDC " + tenantName); xml = replaceXmlValue(xml, "actiontype", "permit"); - xml = replaceXmlValue(xml, "protocolvalue", protocol); xml = replaceXmlValue(xml, "deststartip", destStartIp); xml = replaceXmlValue(xml, "destendip", destEndIp); diff --git a/plugins/network-elements/cisco-vnmc/src/com/cloud/network/element/CiscoVnmcElement.java b/plugins/network-elements/cisco-vnmc/src/com/cloud/network/element/CiscoVnmcElement.java index 33cc40a67da..b335edb9f63 100644 --- a/plugins/network-elements/cisco-vnmc/src/com/cloud/network/element/CiscoVnmcElement.java +++ b/plugins/network-elements/cisco-vnmc/src/com/cloud/network/element/CiscoVnmcElement.java @@ -105,6 +105,7 @@ import com.cloud.network.dao.PhysicalNetworkServiceProviderDao; import com.cloud.network.dao.PhysicalNetworkServiceProviderVO; import com.cloud.network.resource.CiscoVnmcResource; import com.cloud.network.rules.FirewallRule; +import com.cloud.network.rules.FirewallRule.TrafficType; import com.cloud.network.rules.PortForwardingRule; import com.cloud.network.rules.StaticNat; import com.cloud.offering.NetworkOffering; @@ -677,8 +678,12 @@ public class CiscoVnmcElement extends AdapterBase implements SourceNatServicePro List rulesTO = new ArrayList(); for (FirewallRule rule : rules) { - IpAddress sourceIp = _networkModel.getIp(rule.getSourceIpAddressId()); - FirewallRuleTO ruleTO = new FirewallRuleTO(rule, null, sourceIp.getAddress().addr(), rule.getPurpose(), rule.getTrafficType()); + String address = "0.0.0.0"; + if (rule.getTrafficType() == TrafficType.Ingress) { + IpAddress sourceIp = _networkModel.getIp(rule.getSourceIpAddressId()); + address = sourceIp.getAddress().addr(); + } + FirewallRuleTO ruleTO = new FirewallRuleTO(rule, null, address, rule.getPurpose(), rule.getTrafficType()); rulesTO.add(ruleTO); } diff --git a/plugins/network-elements/cisco-vnmc/src/com/cloud/network/resource/CiscoVnmcResource.java b/plugins/network-elements/cisco-vnmc/src/com/cloud/network/resource/CiscoVnmcResource.java index 91559782304..906e0ae6e85 100644 --- a/plugins/network-elements/cisco-vnmc/src/com/cloud/network/resource/CiscoVnmcResource.java +++ b/plugins/network-elements/cisco-vnmc/src/com/cloud/network/resource/CiscoVnmcResource.java @@ -368,29 +368,29 @@ public class CiscoVnmcResource implements ServerResource { if (!_connection.createTenantVDCIngressAclRule(tenant, Long.toString(rule.getId()), policyIdentifier, rule.getProtocol().toUpperCase(), externalIpRange[0], externalIpRange[1], - Integer.toString(rule.getSrcPortRange()[0]), Integer.toString(rule.getSrcPortRange()[1]), publicIp)) { + Integer.toString(rule.getSrcPortRange()[0]), Integer.toString(rule.getSrcPortRange()[1]))) { throw new Exception("Failed to create ACL ingress rule in VNMC for guest network with vlan " + vlanId); } } else { if (!_connection.createTenantVDCIngressAclRule(tenant, Long.toString(rule.getId()), policyIdentifier, - rule.getProtocol().toUpperCase(), externalIpRange[0], externalIpRange[1], publicIp)) { + rule.getProtocol().toUpperCase(), externalIpRange[0], externalIpRange[1])) { throw new Exception("Failed to create ACL ingress rule in VNMC for guest network with vlan " + vlanId); } } } else { - if (!rule.getProtocol().equalsIgnoreCase("icmp")) { + if (rule.getProtocol().equalsIgnoreCase("tcp") || rule.getProtocol().equalsIgnoreCase("udp")) { if (!_connection.createTenantVDCEgressAclRule(tenant, Long.toString(rule.getId()), policyIdentifier, rule.getProtocol().toUpperCase(), - Integer.toString(rule.getSrcPortRange()[0]), Integer.toString(rule.getSrcPortRange()[1]), publicIp, - externalIpRange[0], externalIpRange[1])) { + externalIpRange[0], externalIpRange[1], + Integer.toString(rule.getSrcPortRange()[0]), Integer.toString(rule.getSrcPortRange()[1]))) { throw new Exception("Failed to create ACL egress rule in VNMC for guest network with vlan " + vlanId); } } else { if (!_connection.createTenantVDCEgressAclRule(tenant, Long.toString(rule.getId()), policyIdentifier, - rule.getProtocol().toUpperCase(), publicIp, externalIpRange[0], externalIpRange[1])) { + rule.getProtocol().toUpperCase(), externalIpRange[0], externalIpRange[1])) { throw new Exception("Failed to create ACL egress rule in VNMC for guest network with vlan " + vlanId); } } diff --git a/plugins/network-elements/cisco-vnmc/test/com/cloud/network/resource/CiscoVnmcResourceTest.java b/plugins/network-elements/cisco-vnmc/test/com/cloud/network/resource/CiscoVnmcResourceTest.java index e814fdcd4d5..acfc5ebaaa7 100755 --- a/plugins/network-elements/cisco-vnmc/test/com/cloud/network/resource/CiscoVnmcResourceTest.java +++ b/plugins/network-elements/cisco-vnmc/test/com/cloud/network/resource/CiscoVnmcResourceTest.java @@ -171,11 +171,11 @@ public class CiscoVnmcResourceTest { when(_connection.createTenantVDCIngressAclRule( anyString(), anyString(), anyString(), anyString(), anyString(), anyString(), - anyString(), anyString(), anyString())).thenReturn(true); + anyString(), anyString())).thenReturn(true); when(_connection.createTenantVDCEgressAclRule( anyString(), anyString(), anyString(), anyString(), anyString(), anyString(), - anyString(), anyString(), anyString())).thenReturn(true); + anyString(), anyString())).thenReturn(true); when(_connection.associateAclPolicySet(anyString())).thenReturn(true); Answer answer = _resource.executeRequest(cmd); From da74554ba1cce53a939d115cab04ce965b68617b Mon Sep 17 00:00:00 2001 From: Radhika PC Date: Fri, 10 May 2013 18:26:10 +0530 Subject: [PATCH 13/14] CLOUDSTACK-893 conceptual info around gslb --- .../external-firewalls-and-load-balancers.xml | 1 + docs/en-US/gslb.xml | 678 +++++++++++------- docs/en-US/images/gslb.png | Bin 184080 -> 60354 bytes docs/en-US/networks.xml | 3 +- 4 files changed, 438 insertions(+), 244 deletions(-) diff --git a/docs/en-US/external-firewalls-and-load-balancers.xml b/docs/en-US/external-firewalls-and-load-balancers.xml index b947daf7361..42ecacf9f75 100644 --- a/docs/en-US/external-firewalls-and-load-balancers.xml +++ b/docs/en-US/external-firewalls-and-load-balancers.xml @@ -29,5 +29,6 @@ xmlns:xi="http://www.w3.org/2001/XInclude"/> +
diff --git a/docs/en-US/gslb.xml b/docs/en-US/gslb.xml index ac17d61d69c..23033317381 100644 --- a/docs/en-US/gslb.xml +++ b/docs/en-US/gslb.xml @@ -26,7 +26,7 @@ achieve this by extending its functionality of integrating with NetScaler Application Delivery Controller (ADC), which also provides various GSLB capabilities, such as disaster recovery and load balancing. The DNS redirection technique is used to achieve GSLB in &PRODUCT;. - In order to support his functionality, region level services and service provider are + In order to support this functionality, region level services and service provider are introduced. A new service 'GSLB' is introduced as a region level service. The GSLB service provider is introduced that will provider the GSLB service. Currently, NetScaler is the supported GSLB provider in &PRODUCT;. GSLB functionality works in an Active-Active data center @@ -40,264 +40,456 @@ multiple data centers situated at geographically separated locations. GSLB can also provide an alternate location for accessing a resource in the event of a failure, or to provide a means of shifting traffic easily to simplify maintenance, or both. - -
- Components of GSLB - A typical GSLB environment is comprised of the following components: - - - GSLB Site: In &PRODUCT;terminology, GSLB sites are - represented by zones that are mapped to data centers, each of which has various network - appliances. Each GSLB site is managed by a NetScaler appliance that is local to that site. - Each of these appliances treats its own site as the local site and all other sites, - managed by other appliances, as remote sites. - - - GSLB Services: A GSLB service is typically - represented by a load balancing or content switching virtual server. In a GSLB - environment, you can have a local as well as remote GSLB services. A local GSLB service - represents a local load balancing or content switching virtual server. A remote GSLB - service is the one configured at one of the other sites in the GSLB setup. At each site in - the GSLB setup, you can create one local GSLB service and any number of remote GSLB - services. - - - GSLB Virtual Servers: A GSLB virtual server refers to - one or more GSLB services and balances traffic between traffic across the VMs in multiple - zones by using the &PRODUCT; functionality. It evaluates the configured GSLB methods or - algorithms to select a GSLB service to which to send the client requests. One or more - virtual servers from different zones are bound to the GSLB virtual server. GSLB virtual - server does not have a public IP associated with it, instead it will have a FQDN DNS - name. - - - Load Balancing or Content Switching Virtual Servers: - According to Citrix NetScaler terminology, a load balancing or content switching virtual - server represents one or many servers on the local network. Clients send their requests to - the load balancing or content switching virtual server’s virtual IP (VIP) address, and the - virtual server balances the load across the local servers. After a GSLB virtual server - selects a GSLB service representing either a local or a remote load balancing or content - switching virtual server, the client sends the request to that virtual server’s VIP - address. - - - DNS VIPs: DNS virtual IP represents a load balancing - DNS virtual server on the GSLB service provider. The DNS requests for domains for which - the GSLB service provider is authoritative can be sent to a DNS VIP. - - - ADNS: ADNS (Authoritative Domain Name Server) is a - service that provides actual answer to DNS queries, such as web site IP address. In a GSLB - environment, an ADNS service responds only to DNS requests for domains for which the GSLB - service provider is authoritative. When an ADNS service is configured, the service - provider owns that IP address and advertises it. When you create an ADNS service, the - NetScaler responds to DNS queries on the configured ADNS service IP and port. - - -
-
- Prerequisites and Guidelines - - - The GSLB functionality is supported both Basic and Advanced zones. - - - GSLB is added as a new network service. - - - GSLB service provider can be added to a physical network in a zone. - - - The admin is allowed to enable or disable GSLB functionality at region level. - - - The admin is allowed to configure a zone as GSLB capable or enabled. - A zone shall be considered as GSLB capable only if a GSLB service provider is - provisioned in the zone. - - - When users have VMs deployed in multiple availability zones which are GSLB enabled, - they can use the GSLB functionality to load balance traffic across the VMs in multiple - zones. - - - The users can use GSLB to load balance across the VMs across zones in a region only if - the admin has enabled GSLB in that region. - - - The users can load balance traffic across the availability zones in the same region or - different regions. - - - The admin can configure DNS name for the entire cloud. - - - The users can specify an unique name across the cloud for a globally load balanced - service. The provided name is used as the domain name under the DNS name associated with - the cloud. - The user-provided name along with the admin-provided DNS name is used to produce a - globally resolvable FQDN for the globally load balanced service of the user. For example, - if the admin has configured xyztelco.com as the DNS name for the cloud, and user specifies - 'foo' for the GSLB virtual service, then the FQDN name of the GSLB virtual service is - foo.xyztelco.com. - - - While setting up GSLB, users can select a load balancing method, such as round robin, - for using across the zones that are part of GSLB. - - - The user shall be able to set weight to zone-level virtual server. Weight shall be - considered by the load balancing method for distributing the traffic. - - - The GSLB functionality shall support session persistence, where series of client - requests for particular domain name is sent to a virtual server on the same zone. - Statistics is collected from each GSLB virtual server. - - +
+ Components of GSLB + A typical GSLB environment is comprised of the following components: + + + GSLB Site: In &PRODUCT;terminology, GSLB sites are + represented by zones that are mapped to data centers, each of which has various network + appliances. Each GSLB site is managed by a NetScaler appliance that is local to that + site. Each of these appliances treats its own site as the local site and all other + sites, managed by other appliances, as remote sites. It is the central entity in a GSLB + deployment, and is represented by a name and an IP address. + + + GSLB Services: A GSLB service is typically + represented by a load balancing or content switching virtual server. In a GSLB + environment, you can have a local as well as remote GSLB services. A local GSLB service + represents a local load balancing or content switching virtual server. A remote GSLB + service is the one configured at one of the other sites in the GSLB setup. At each site + in the GSLB setup, you can create one local GSLB service and any number of remote GSLB + services. + + + GSLB Virtual Servers: A GSLB virtual server refers + to one or more GSLB services and balances traffic between traffic across the VMs in + multiple zones by using the &PRODUCT; functionality. It evaluates the configured GSLB + methods or algorithms to select a GSLB service to which to send the client requests. One + or more virtual servers from different zones are bound to the GSLB virtual server. GSLB + virtual server does not have a public IP associated with it, instead it will have a FQDN + DNS name. + + + Load Balancing or Content Switching Virtual + Servers: According to Citrix NetScaler terminology, a load balancing or + content switching virtual server represents one or many servers on the local network. + Clients send their requests to the load balancing or content switching virtual server’s + virtual IP (VIP) address, and the virtual server balances the load across the local + servers. After a GSLB virtual server selects a GSLB service representing either a local + or a remote load balancing or content switching virtual server, the client sends the + request to that virtual server’s VIP address. + + + DNS VIPs: DNS virtual IP represents a load + balancing DNS virtual server on the GSLB service provider. The DNS requests for domains + for which the GSLB service provider is authoritative can be sent to a DNS VIP. + + + Authoritative DNS: ADNS (Authoritative Domain Name + Server) is a service that provides actual answer to DNS queries, such as web site IP + address. In a GSLB environment, an ADNS service responds only to DNS requests for + domains for which the GSLB service provider is authoritative. When an ADNS service is + configured, the service provider owns that IP address and advertises it. When you create + an ADNS service, the NetScaler responds to DNS queries on the configured ADNS service IP + and port. + + +
+
+ How Does GSLB Works in &PRODUCT;? + Global server load balancing is used to manage the traffic flow to a web site hosted on + two separate zones that ideally are in different geographic locations. The following is an + illustration of how GLSB functionality is provided in &PRODUCT;: An organization, xyztelco, + has set up a public cloud that spans two zones, Zone-1 and Zone-2, across geographically + separated data centers that are managed by &PRODUCT;. Tenant-A of the cloud launches a + highly available solution by using xyztelco cloud. For that purpose, they launch two + instances each in both the zones: VM1 and VM2 in Zone-1 and VM5 and VM6 in Zone-2. Tenant-A + acquires a public IP, IP-1 in Zone-1, and configures a load balancer rule to load balance + the traffic between VM1 and VM2 instances. &PRODUCT; orchestrates setting up a virtual + server on the LB service provider in Zone-1. Virtual server 1 that is set up on the LB + service provider in Zone-1 represents a publicly accessible virtual server that client + reaches at IP-1. The client traffic to virtual server 1 at IP-1 will be load balanced across + VM1 and VM2 instances. + Tenant-A acquires another public IP, IP-2 in Zone-2 and sets up a load balancer rule to + load balance the traffic between VM5 and VM6 instances. Similarly in Zone-2, &PRODUCT; + orchestrates setting up a virtual server on the LB service provider. Virtual server 2 that + is setup on the LB service provider in Zone-2 represents a publicly accessible virtual + server that client reaches at IP-2. The client traffic that reaches virtual server 2 at IP-2 + is load balanced across VM5 and VM6 instances. At this point Tenant-A has the service + enabled in both the zones, but has no means to set up a disaster recovery plan if one of the + zone fails. Additionally, there is no way for Tenant-A to load balance the traffic + intelligently to one of the zones based on load, proximity and so on. The cloud + administrator of xyztelco provisions a GSLB service provider to both the zones. A GSLB + provider is typically an ADC that has the ability to act as an ADNS (Authoritative Domain + Name Server) and has the mechanism to monitor health of virtual servers both at local and + remote sites. The cloud admin enables GSLB as a service to the tenants that use zones 1 and + 2. + + + + + + gslb.png: GSLB architecture + + + Tenant-A wishes to leverage the GSLB service provided by the xyztelco cloud. Tenant-A + configures a GSLB rule to load balance traffic across virtual server 1 at Zone-1 and virtual + server 2 at Zone-2. The domain name is provided as A.xyztelco.com. &PRODUCT; orchestrates + setting up GSLB virtual server 1 on the GSLB service provider at Zone-1. &PRODUCT; binds + virtual server 1 of Zone-1 and virtual server 2 of Zone-2 to GLSB virtual server 1. GSLB + virtual server 1 is configured to start monitoring the health of virtual server 1 and 2 in + Zone-1. &PRODUCT; will also orchestrate setting up GSLB virtual server 2 on GSLB service + provider at Zone-2. &PRODUCT; will bind virtual server 1 of Zone-1 and virtual server 2 of + Zone-2 to GLSB virtual server 2. GSLB virtual server 2 is configured to start monitoring the + health of virtual server 1 and 2. &PRODUCT; will bind the domain A.xyztelco.com to both the + GSLB virtual server 1 and 2. At this point, Tenant-A service will be globally reachable at + A.xyztelco.com. The private DNS server for the domain xyztelcom.com is configured by the + admin out-of-band to resolve the domain A.xyztelco.com to the GSLB providers at both the + zones, which are configured as ADNS for the domain A.xyztelco.com. A client when sends a DNS + request to resolve A.xyztelcom.com, will eventually get DNS delegation to the address of + GSLB providers at zone 1 and 2. A client DNS request will be received by the GSLB provider. + The GSLB provider, depending on the domain for which it needs to resolve, will pick up the + GSLB virtual server associated with the domain. Depending on the health of the virtual + servers being load balanced, DNS request for the domain will be resolved to the public IP + associated with the selected virtual server. +
Configuring GSLB + A GSLB deployment is the logical collection of GSLB virtual server, GSLB service, LB + virtual server, service, domain, and ADNS service. To create a GSLB site, you must configure + load balancing in the zone. You must create GSLB vservers and GSLB services for each site. You + must bind GSLB services to GSLB vservers. You must then create an ADNS service that provides + the IP address of the best performing site to the client's request. A GSLB vserver is an + entity that performs load balancing for the domains bound to it by returning the IP address of + the best GSLB service. A GSLB service is a representation of the load balancing/content + switching vserver. An LB vserver load balances incoming traffic by identifying the best + server, then directs traffic to the corresponding service. It can also load-balance external + DNS name servers. Services are entities that represent the servers. The domain is the domain + name for which the system is the authoritative DNS server. By creating an ADNS service, the + system can be configured as an authoritative DNS server. To configure GSLB in your cloud environment, as a cloud administrator you must perform the following. To configure such a GSLB setup, you must first configure a standard load balancing setup - for each server farm or data center. This enables you to balance load across the different - servers in each server farm. Then, configure both NetScaler appliances as authoritative DNS - (ADNS) servers. Next, create a GSLB site for each server farm, configure GSLB virtual servers - for each site, create GLSB services, and bind the GSLB services to the GSLB virtual servers. - Finally, bind the domain to the GSLB virtual servers. The GSLB configurations on the two - appliances at the two different sites are identical, although each sites load-balancing - configuration is specific to that site. + for each zone. This enables you to balance load across the different servers in each zone in + the region. Then, configure both NetScaler appliances that you plan to add to each zone as + authoritative DNS (ADNS) servers. Next, create a GSLB site for each zone, configure GSLB + virtual servers for each site, create GLSB services, and bind the GSLB services to the GSLB + virtual servers. Finally, bind the domain to the GSLB virtual servers. The GSLB configurations + on the two appliances at the two different sites are identical, although each sites + load-balancing configuration is specific to that site. + Perform the following as a cloud administrator. As per the above example, the + administrator of xyztelco is the one who sets up GSLB: - + In the cloud.dns.name global parameter, specify the DNS name of your tenant's cloud + that make use of the GSLB service. + + + On the NetScaler side, configure GSLB as given in Configuring Global Server Load Balancing (GSLB): + + + Configuring a standard load balancing setup. + + + Configure Authoritative DNS, as explained in Configuring an Authoritative DNS Service. + + + Configure a GSLB site with site name formed from the domain name details. + For more information, see Configuring a Basic GSLB Site. + + + Configure a GSLB virtual server. + For more information, see Configuring a GSLB Virtual Server. + + + Configure a GSLB service for each virtual server. + For more information, see Configuring a GSLB Service. + + + Bind the GSLB services to the GSLB virtual server. + For more information, see Binding GSLB Services to a GSLB Virtual Server. + + + Bind domain name to GSLB virtual server. Domain name is obtained from the domain + details. + For more information, see Binding a Domain to a GSLB Virtual Server. + + + + + In each zone that are participating in GSLB, add GSLB-enabled NetScaler device. + For more information, see . -
-
- Adding a GSLB Rule + As a domain administrator/ user perform the following: - Log in to the &PRODUCT; UI as administrator. + Add a GSLB rule on both the sites. + See . - In the left navigation pane, click Region. - - - Select the region for which you want to create a GSLB rule. - - - In the Details tab, click View GSLB. - - - Click Add GSLB. - The Add GSLB page is displayed as follows: - - - - - - gslb-add.png: adding a gslb rule - - - - - Specify the following: - - - Name: Name for the GSLB rule. - - - Description: (Optional) A short description of - the GSLB rule that can be displayed to users. - - - GSLB Domain Name: A preferred domain name for the - service. - - - Algorithm: (Optional) The algorithm to use to - load balance the traffic across the zones. The options are Round Robin, Least - Connection, and Proximity. - - - Service Type: The transport protocol to use for - GSLB. The options are TCP and UDP. - - - Domain: (Optional) The domain for which you want - to create the GSLB rule. - - - Account: (Optional) The account on which you want - to apply the GSLB rule. - - - - - Click OK to confirm. + Assign load balancer rules. + See . -
-
- Assigning Load Balancing Rules to GSLB - -
-
- How Does GSLB Works in &PRODUCT;? - Global server load balancing is used to manage traffic flow to a web site hosted on two - separate zones that ideally are in different geographic locations. The following is an - illustration of how GLSB functionality is provided in &PRODUCT;: An organization, xyztelco, - has set up a public cloud that spans two zones, Zone-1 and Zone-2, across geographically - separated data centers that are managed by &PRODUCT;. Tenant-A of the cloud launches a highly - available solution by using xyztelco cloud. For that purpose, they launch two instances each - in both the zones: VM1 and VM2 in Zone-1 and VM5 and VM6 in Zone-2. Tenant-A acquires a public - IP, IP-1 in Zone-1, and configures a load balancer rule to load balance the traffic between - VM1 and VM2 instances. &PRODUCT; orchestrates setting up a virtual server on the LB service - provider in Zone-1. Virtual server 1 that is set up on the LB service provider in Zone-1 - represents a publicly accessible virtual server that client reaches at IP-1. The client - traffic to virtual server 1 at IP-1 will be load balanced across VM1 and VM2 instances. - Tenant-A acquires another public IP, IP-2 in Zone-2 and sets up a load balancer rule to - load balance the traffic between VM5 and VM6 instances. Similarly in Zone-2, &PRODUCT; - orchestrates setting up a virtual server on the LB service provider. Virtual server 2 that is - setup on the LB service provider in Zone-2 represents a publicly accessible virtual server - that client reaches at IP-2. The client traffic that reaches virtual server 2 at IP-2 is load - balanced across VM5 and VM6 instances. At this point Tenant-A has the service enabled in both - the zones, but has no means to set up a disaster recovery plan if one of the zone fails. - Additionally, there is no way for Tenant-A to load balance the traffic intelligently to one of - the zones based on load, proximity and so on. The cloud administrator of xyztelco provisions a - GSLB service provider to both the zones. A GSLB provider is typically an ADC that has the - ability to act as an ADNS (Authoritative Domain Name Server) and has the mechanism to monitor - health of virtual servers both at local and remote sites. The cloud admin enables GSLB as a - service to the tenants that use zones 1 and 2. - - - - - - gslb.png: GSLB architecture - - - Tenant-A wishes to leverage the GSLB service provided by the xyztelco cloud. Tenant-A - configures a GSLB rule to load balance traffic across virtual server 1 at Zone-1 and virtual - server 2 at Zone-2. The domain name is provided as A.xyztelco.com. &PRODUCT; orchestrates - setting up GSLB virtual server 1 on the GSLB service provider at Zone-1. &PRODUCT; binds - virtual server 1 of Zone-1 and virtual server 2 of Zone-2 to GLSB virtual server 1. GSLB - virtual server 1 is configured to start monitoring the health of virtual server 1 and 2 in - Zone-1. &PRODUCT; will also orchestrate setting up GSLB virtual server 2 on GSLB service - provider at Zone-2. &PRODUCT; will bind virtual server 1 of Zone-1 and virtual server 2 of - Zone-2 to GLSB virtual server 2. GSLB virtual server 2 is configured to start monitoring the - health of virtual server 1 and 2. &PRODUCT; will bind the domain A.xyztelco.com to both the - GSLB virtual server 1 and 2. At this point, Tenant-A service will be globally reachable at - A.xyztelco.com. The private DNS server for the domain xyztelcom.com is configured by the admin - out-of-band to resolve the domain A.xyztelco.com to the GSLB providers at both the zones, - which are configured as ADNS for the domain A.xyztelco.com. A client when sends a DNS request - to resolve A.xyztelcom.com, will eventually get DNS delegation to the address of GSLB - providers at Zone 1 and 2. A client DNS request will be received by the GSLB provider. The - GSLB provider, depending on the domain for which it needs to resolve, will pick up the GSLB - virtual server associated with the domain. Depending on the health of the virtual servers - being load balanced, DNS request for the domain will be resolved to the public IP associated - with the selected virtual server. +
+ Prerequisites and Guidelines + + + The GSLB functionality is supported both Basic and Advanced zones. + + + GSLB is added as a new network service. + + + GSLB service provider can be added to a physical network in a zone. + + + The admin is allowed to enable or disable GSLB functionality at region level. + + + The admin is allowed to configure a zone as GSLB capable or enabled. + A zone shall be considered as GSLB capable only if a GSLB service provider is + provisioned in the zone. + + + When users have VMs deployed in multiple availability zones which are GSLB enabled, + they can use the GSLB functionality to load balance traffic across the VMs in multiple + zones. + + + The users can use GSLB to load balance across the VMs across zones in a region only + if the admin has enabled GSLB in that region. + + + The users can load balance traffic across the availability zones in the same region + or different regions. + + + The admin can configure DNS name for the entire cloud. + + + The users can specify an unique name across the cloud for a globally load balanced + service. The provided name is used as the domain name under the DNS name associated with + the cloud. + The user-provided name along with the admin-provided DNS name is used to produce a + globally resolvable FQDN for the globally load balanced service of the user. For + example, if the admin has configured xyztelco.com as the DNS name for the cloud, and + user specifies 'foo' for the GSLB virtual service, then the FQDN name of the GSLB + virtual service is foo.xyztelco.com. + + + While setting up GSLB, users can select a load balancing method, such as round + robin, for using across the zones that are part of GSLB. + + + The user shall be able to set weight to zone-level virtual server. Weight shall be + considered by the load balancing method for distributing the traffic. + + + The GSLB functionality shall support session persistence, where series of client + requests for particular domain name is sent to a virtual server on the same zone. + Statistics is collected from each GSLB virtual server. + + +
+
+ Enabling GSLB in NetScaler + In each zone, add GSLB-enabled NetScaler device for load balancing. + + + Log in as administrator to the &PRODUCT; UI. + + + In the left navigation bar, click Infrastructure. + + + In Zones, click View More. + + + Choose the zone you want to work with. + + + Click the Physical Network tab, then click the name of the physical network. + + + In the Network Service Providers node of the diagram, click Configure. + You might have to scroll down to see this. + + + Click NetScaler. + + + Click Add NetScaler device and provide the following: + For NetScaler: + + + IP Address: The IP address of the SRX. + + + Username/Password: The authentication + credentials to access the device. &PRODUCT; uses these credentials to access the + device. + + + Type: The type of device that is being added. + It could be F5 Big Ip Load Balancer, NetScaler VPX, NetScaler MPX, or NetScaler SDX. + For a comparison of the NetScaler types, see the &PRODUCT; Administration + Guide. + + + Public interface: Interface of device that is + configured to be part of the public network. + + + Private interface: Interface of device that is + configured to be part of the private network. + + + GSLB service: Select this option. + + + GSLB service Public IP: The public IP address + of the NAT translator for a GSLB service that is on a private network. + + + GSLB service Private IP: The private IP of the + GSLB service. + + + Number of Retries. Number of times to attempt a + command on the device before considering the operation failed. Default is 2. + + + Capacity: The number of networks the device can + handle. + + + Dedicated: When marked as dedicated, this + device will be dedicated to a single account. When Dedicated is checked, the value + in the Capacity field has no significance implicitly, its value is 1. + + + + + Click OK. + + +
+
+ Adding a GSLB Rule + + + Log in to the &PRODUCT; UI as a domain administrator or user. + + + In the left navigation pane, click Region. + + + Select the region for which you want to create a GSLB rule. + + + In the Details tab, click View GSLB. + + + Click Add GSLB. + The Add GSLB page is displayed as follows: + + + + + + gslb-add.png: adding a gslb rule + + + + + Specify the following: + + + Name: Name for the GSLB rule. + + + Description: (Optional) A short description of + the GSLB rule that can be displayed to users. + + + GSLB Domain Name: A preferred domain name for + the service. + + + Algorithm: (Optional) The algorithm to use to + load balance the traffic across the zones. The options are Round Robin, Least + Connection, and Proximity. + + + Service Type: The transport protocol to use for + GSLB. The options are TCP and UDP. + + + Domain: (Optional) The domain for which you + want to create the GSLB rule. + + + Account: (Optional) The account on which you + want to apply the GSLB rule. + + + + + Click OK to confirm. + + +
+
+ Assigning Load Balancing Rules to GSLB + + + + Log in to the &PRODUCT; UI as a domain administrator or user. + + + In the left navigation pane, click Region. + + + Select the region for which you want to create a GSLB rule. + + + In the Details tab, click View GSLB. + + + Select the desired GSLB. + + + Click view assigned load balancing. + + + Click assign more load balancing. + + + Select the load balancing rule you have created for the zone. + + + Click OK to confirm. + + +
Known Limitation diff --git a/docs/en-US/images/gslb.png b/docs/en-US/images/gslb.png index 8d1a389936c31e3ed46c24f7a6787fd21a873c65..9f13580c56095985891f09e9c33bc363567eb57e 100644 GIT binary patch literal 60354 zcmdqJWm8;T5H5-{xVt+9hhV{7f=eL5J-7w;;O_2j!F6zVcXto&&KcfgIltgm-4E2% zELN?(+Pk0bJt2znl1T9Q@L*tINYYYb%3xrSmSAAuX8;J$iscdR3+MyfL0M7+tYVz- z2=oQYR9H?J46Hf|;YA-B^c~Jt>bnCN7;?|YANYVxu@M;9Uv+6QVO1BM6Hizl)w#L% zm34#_1cVk+T5U9NVOwD;^FJ|qv;N6f1k&{5VCLvhn}9#{U-^N5h0HEG!%yBV6|o*oT_FbpPoFHu;fuQpTnM{^KXLIM~f4 zeIxVBytn6h1lI;zN$?gWwz?$u9B0xT71YC-Ws=^uP*;GEr>Cf~v9YM(P71pzsjGpA zfsT%~iHV86zVG#Kp)uRz*`wjoQip?aG9zJOVW)53q@}BSH1_sP%`Gf;wztOxz3b>7 z6t{_ph<6~=eY-c>&u9E~`*ApIZ8#~^2p3l9xqxBHlJx0=}M_>zpuX!!>4ey_N z0@9t%*RL-Wyyd|++zb;CSbsAIa~a>DS-6_@(f=S3?s+~YYD9u6i-NK(z?NIAf(#C% z54OXV=*$A3h8Q_C2(liudW3<=dWB(c9QX^9KNn?s!Q2VcqMeF$)s>n?tLySBOyFN8 z+qYr>U@d;bR@EN0$qseFCfu0Y zqHJ!xXe~VKi{~pu)|omeL^kn=9|AV2GtqGo+dr-Moc7r+cO6j<=1if{I`3#^oV$1d zi_vqNwapd3bnIT>0hR7vovXPlWVV|EAsMayii?X&1^Z^-d4m6MuD~>Vbwux4a9wa$ z)3&9an6r!tB#uLH;#az-nCY|J;q-+akKm;=sZryC5)ndC41q)-W zgE3UAgbzoIj_4p;(d5%k=p5r&2kzddiGGD* zS|>zwo9J=vjc4R(|CL;&J}Sx2r$IG7YBAFrjW!)&c#q#LaSH6Ya0fBGw06hNsI6l! z3K=QBsrUc_4PTO>=-=-puQAi9^r3zugL9g~Y9~~8-U0O9W=0*}U;WNe>QDl6eVcu& z$1)vL1hC`~xF@mk@{h*)anz3ykz+7VxH?|j=(o`N5e-^sKf$uoK(L#Rl6GxYfjhQK z`|k!?#fIv!;`$1M903ol7-8ouKyQD>15Y2rzkf>GuFcu$7nfwvmZ72Hn#=%d^%jAQ zfa}B2IZbI2?9!3L?CNrz24*J@S`Pc&38gbtF7-i|Uyzm^$*(r8S60a<)FEjmKll?F zJUUrVzhn3il$%2D)i-qW0>8=tO{+q0y+0*Vcp(RZ?@Nbr<`sHpG9?jKixr=7+jf0s z-PumzF`;g=*M`&Zx0=FzNKo~%qy&9NzdxXYk9opNm>1*nk4QV9IjoVO;klk*9NBU; z?pp`MYp)iaq%1w2R^GRG{ct;J~hebMjA z_!Ghcns9b#l!K2fXDqoHeooLFy<)>36 zDG*~I!HY}AK38YpV8D|RhUS}={djnP{)>Qc{_@Q`B8F>k8hzoSC=T;9Q>ADw36coH z_Ha@^l+ALH1?TTnpPmr0Y+J`QdFg1{>73rDA2D2(zclg80Q*h=m-(tSbC&EKZXXBI z1+>pBma`7kI(2&MLR(duKzgtsQ>d3Hu(aCOD4v6#ln|-Us?v7V14rg>!O1z^*Q@7x z5ZByv6Xd^LUJd62>CVJ)v=0r)GeeLfrJw*R9G?{wHw;M7IS;nkm83`iK*pdeIvF@- z69J-LB_4z{BR`YdA65dpb6hxt9 zfhZhd_h^!NlTYVf(j9mF>o%cxY$FE~MP^A>i!lM;v{K3kBHP-|IDp~-(u0416K<}XwJXw*H6{i<_kY+ zxpD-^FwNOjM%oj-U(@AUM|6aNXaG_OS#n5#faxhCgxfuxjjW(zzS#_q%kC=*AAl(N zyqGdR(hZi|pdw$tn{eMvm=BfUG4>96hUTvH?=?Dp35iEmDO93$kfUfk(KVBGlCaAp zga>KaMw6W(-SE8C3Q_-Doxr%WfRfe)yMbZyl&}Mb9L?2l{@3+BXFtKd-rk>huHTv3 zI>p6Bk0##CKiPV=kt@!~m`~BummOYbyV)(LWn~sNUu^q%^}GoYL5@MXcLy6lV9aJa zrIv`c9SI#Rw!xMzqY2uyzcS+(y7~aa3huJ?TUPY4X<(9n{#g84#HE>Mw{G{W4)9%b zzk5Q4_Z*gQP6&grWsx9N1h9l4VqT?JPN4b7Lk~ppz8symMgj3YG09OQ{>_eEtl3HA z?Cv8cGftkaBiR+1g%X>``du*5r(L}TB{5a`*xmXz{W~LC1nmkjKSl&B$bFIZ(&AJ4 z+l9FdFHXd5lT&c-jLEOIy5!rgtefL6=OZAA_i`X*5dpZ(*Kh5Y)?*HrSiSV3nlTms!nS0h6Waidx;NPX%Ilnk$kGm9Yu?X!5e8)`sa^bWN zZKFAz#1Z-Y5RCqrw7e|uiuX{&)$HV*f@Q3+o(lbFT%n;0L*}iqSb&E`_@i;bY2j=Iz7NmZ|ctVLAbohg{VGSi74v9_;(2eoKXo^h8p={yc3w2&m z=%Um4CPjpy4J4^@68wQnLd0+mRqNo$S@kWzuCGnXxxp`6@OFU(2`7}7;01e(V zQh+7av7}}AiABvKRHAJ~>k;o(#3{m{nY^{6`=>mz-`OlHp#u#!;&N4e6B8${r%u(H zp#tI`ByM%7XS*`x7q|e;-*$QuohTsXzJVM}52!?)JV&rwfafcgO>Wj}4wzS@&{@wq zJ>i>SUh~LHOlldHVl>X{?cdw+6PXqCX~+??0nA%2ndqrAC4Qw3-iIv#3ZeStvp3&b zJ1j?%_gF0vtNNQNWz)=$6O;V(i-)Ca2?k!rN0=}EV9JiVAnq!KgEu*ihDDJ%_=YK( zZA&kSJPLTg1l0=y0ZAD5JP@gUGvl6P#o^T0&C3?GGa245rz@KO$axIuL$W?qGG*$ z8KM>H1zDozIAqT}kRRIs{KTigIp9OtovGnAtW|f54h~a_jf>MlZ97bfxRie<$S{!veGEIK+Jm-?EonA^8ge);S(#O&S}=>F&F~+zXf%R_j2P>Z1R4XF0!Wl5((gEDYMc3-d+b}h^DT( zf-cs><|=lJZ2fn|mCbF|;Mcx+&3A1jHrK>kDix-};Cup*pz{*4sv=O5Ga<3RX z%Ck2kPT%ZGtrhd~5*R9U2J6dO9@+&0d6=+g>bh)qNQPjru7vRINc7L(sb2{iT99Dc z$@BgEacWHQo!ipE{kyT6xDhEfRX#3-sUyF7063hG?oSU|=~*4KS4_>#V9v-$*t6cY z4j5Q1d^9l&t?*U8khL6JblV+NYBX}-uX57h`d&Ab7_qcGgv8_&k|)oB^V@f~j=r6F zSnabazawE#C=<{wRw1KS8$Rc!BOxVV37i| zIb|--DQIPWX|zVqwScb_b1U|z@@?)}LZSp$Zi3Oz)t zRs9Ed7rxjoiuVgUw2#AFcKir?HHRVnS-ze9&ls))Jbj4;d4`3Gt@+tFrUp(ck~UGE z9GsWA8q-^~{V;p8IV4V5)7PI1J&JU7o+2b34>oQ))~p@(;hu}Zz=xr^;uY44z5dAfGT-3q$DKPX1jfBduTa{BEt#fL zFPhA8n;$lGJi#!#>t9^O0BKHG05`<@CtH7F3=Z4@#h6SmXYnseUS3iKXQVz32iNhdim&q4F>GZ%YxuQ+WSJom- zzzC}c_2LJnBsnkk1Zy?kM*{HC*sQ@zUp~Dk#PC4qLq8-vjLC!-%G#yGPVc!W7Zq)I z``zmsS_FKd@QJ%fZoFU|#{(C42oNUci0YHjqP6z*6OF$>2U`bTls%z?GM~w;5V+&S zB;hiQke~WYDddQG$Rsp$kg$`zZ`nD-b|XB~=jWb>)<76WXw2eNxRo5uA{{QCNu|5- zDbWav0hk!RVhF1(R@y5+7+Vbq-j)|3ZoTpXj#yu(1lK@doU;4sNB%nqExT5Mpg1EE zvTb$GxkR=@Xyu%9@#hT2s*cbH?Scm4Psk0;a(dD`s*iA0L4n!`iqUui5sFjwO$+S{ zP4hF4D?`Ho+FH_78E)zr(RBqqC9lSCfp0F{i2-<<$P8Buf2YT!vU@Z&*7z+<-b8ot z+gbHudkLO=0@?)}M1F`7`VodLkJ7(5jtCa0Fbpyuvj_eQ6J78?fT|*gU>?Z-XDJbSRCzT~SiuZqEDliIEw~uVD?aLlI>pxJ{ro)FDBQzNQ2d2XHodlsv zMu&5BNx}ajNm(DE{Q%Vg=6_>fA4r!!G05YHR>}j#{#|__{0L`3$?X3J`vTEIdIK_c zJU7Q-DuaJOuFivxK+Myd4*oB$mJie;NYHjf20i{RdA$Q=l>dG#C=^A|ubvtiLKm1tb-|l{$s* zE5*bpDJflEUdF`4j8eZs^Bwd6{Fy;K?rA&&?uPqp0raHd&labrrwyxdGr4IF$sus( zF)+}#x3`len}cy-4M8%W04cx`o;6_@NUcuLNMV*1XM3qF**{0_=2Py+m=psNebE5= zAL6v#v!B8R?;t0|H8Q05OZp1mJTp2yGi_jCFiL^)wHqWovOtiApq42zAyc5@LS=he zg7C$kmi`JldX$@5&*YOcGjXwqjz;JN^*}Qal|E4)&!*7h`7S})iV9}zi(gY;o*|tt z{fW3vfRp+QKYxZrFC|B}0Sm-Gx`zq|r8iPhUS5Aerw`ugwaHMPcye-bc=%HVuCE(( zs?Z<^!elcljYa4NK%s+250;jGN>|2}BI7=92WT)Rrl%7+h3z_Zf$l3H0@B?m9}c^k zw5+zROGmfI{TG+H?VL*HW{W;nBJip6QZY(hZ{yrAkRhXEV~z=8sYXVqh9c)0s#o=UBQ|!^zWgWQd~C_ZDK)fR;b1XR z#P@zcf>^*#L?<;FDZ|z%kEfSFPis!k_3jtbPw01#5W#G9bj&gs_SF@ z)q3Sq5FssvwmtH+T_blpe24Lln=Tbm_mRQptpPqj)57#9JKIg9_=st_Zi5WhXp=g5 zmj8ZC7fqFYu5biMjM)_ey6}P?U9tM~NmdOb1|$6#&I}rSZ4UtpZXI@n(?H9}>FPdO z@6&by>r`g1GgocF5TUV>b{Z#ywt$Fzkrtur2>ktiQ?6q;dh+4u?G0HSomC1&MZ=V{w#(5pPy5k5OUuwLi4gg5;$Vxpl3lo%Mhvz0%F3N^BX<44C-$k0I zgW_qPt2I+{u=mG~WWHAdX0S(SM~vXZisUtI2QDLit0dYiEAkndZ!s!_7-hBObRUl`dA6by zF?lJv)-4pb8SfzsEC?_)?ceM2wcla2rA{*C`-Q8@rAduRe+jjJif6O4>5e9|t`Ex8)hlg+UI6z-GrQLJE&PkyyrUYp$Ff2w9!+?TWXs-~R{nUNxpPeGWC zj0rE8v*nOFK1pu;ik(2#z2S{_;}~m-MAY!D6eWO^U;zsyf$v%jhzK+`OwmrhQlYsK ziLd<8tUvF^zQ9bit?6>~CypS=>P!0tcGs5Px3d9KI?1Lxi;CPffj3VoOpoRcy?Eet zbWf;ANn6$1geu^n)$Gy0$d>7I6rw<-Dk#R<5`f~OwJ7#)v>S;suQ%`RR{BUV2Ho3G zxi}g!pD$IFY|7OEzal4vjx4`RocRQBF{jiEb5sfz^3}OCr>3XZNihSis~2A(19Hic zF1f6gWu&P!lL?ZR?)piWOq14Xf4=_dyDl1vH)iU7({Zw&|HYzoF@iO)2_-%6q1*z2 z3YF$`e@cT>^6f3n(mMW?Dt`Y~;5Qv)d>>POxlztkE`&tRC&3?X6Q1`BiGufhZOzj$ zq=4F+Jb{hLw6}92Y{Ln74o^>Pa$Uxj;kmC?S&qKCs~9Y*;by8{M0ojw&4hE-0*W@< zF&&tm)Q4=1BuJZ3afP07wE|c|>VEYqxJydv?Uv{~&u+vA+>Z?j z-?{4o?_XO-crJFhVSP{`0GU; zpZL{jUFbRr7>G9=y#swfyIut_sLvLOAlFuJqQO*4)+zatTrQzOEJ#Lr z=dxa}KmUrX@cSkXF(*l`^OyB@DWT^65)k@93%FlWG%wX=ICor>2)C9GaJbu0;I}k(QObWi zLdnwk%tg=A>8}HMEgIrtbOAye#DmE?_fI~;*7znM_;k_6%?0^!vn0qOxYL#@rc6j{ ztnKOq?-F<)aT=GKzEm>m{!&tKuj78#mjtblZ*zFxo|LudxP|WV{19s~@#fgq{$_kM z)@?pEK2lcvC%0**GQIA4gS<6W$?2%I6(zIjTvG5QN0jYYFuJ=Q&rQvm7_aqr6h#o# zN93+q22O_iOaUIENNz`@d9J(S?%y8-Uifrf$B2?d(H6^i>O7U>r7`;9Za15x#lYuS zlu5pSS7NbCpCvJw0JvrP6pSfe$#(8u=FSwf6RdZ#jAdpxc)tA_mPo48uTI8Sti-n=Z7 z6Io3SWH+^lDr0et!yYh(!u5@o@QHeWJ38L$W`%&H01LI<9Ub4|F*L~CvDNS!P@F;& zbi73(F6u->lGKxOONEtu$(H`y$mvma`-t?c8x<1z+Lv+O+uQ3h!->d@Lo*1Xidxtw z@7eY~@=?@HdHi;!!+XoMASQ-*nJQQ;v#jji0-T$@2w;RL z?Hnd%z*eQf@DrWn)T5-Wmg)H+!FP!BhLhl61oaXEitFCFW3DOC z;PwjaVV}WcApyUB{YvL`uE@>Z7JBp+D0?g}FN3QY#dVmOOY)%LVI3={i<2^Pl(X`Y z=~?k zK^P5NOG``A%cD%72`SXJVO&M)p@joe>zpG6bRtq}0HbHG4HLCy>u)DX9C_^qyrmd;z{<1q&p7s9MWyeG;JVKU19{9^>{sxrj z;t&WGlSlHfadL99s~rB)pGR@{EK;8c)FS?<1`Q9n4XVg})+mvXkoNZWZf|cN2>q4xsRS z6bf1i_P!t!{~W`?!SN6=;RlU!dEQ0pobjS()7RntXH5yMT0BHW(o{Dwfoe2+vYnEW zvZ>e&91R?r$7?%2ET6qspF{YPAu)5SC)D2F9)>FB5W2X1e*POnS$6E7vNHP074PD$ zyoR&DrsUqf1wj-wt>kMdJpk$=%{0_$syGV{$dit#kuXG%pps@uC0|^upe&_uB7B)J zp7Cu|KYyh6CrtI&3a(4}E_&5Pl=I8?IN*kfsRq^u?#=Gz+7Jzmm>AvYNA_=3mahsE z!DxL;OG}FRNI#&=5f{rhR#y=}g`9qq8$;TYYuUlr+1I{OTfmz3Gr`OUcX4)#{ze#f zY5`)y6TZIuMmXT&%yJf<=+fcjrc3U5LpqFUSe{P)>sLrmSvk1?h-qz8ZhLfYrxOV_{u2+ozLq z*N`KMaOnfxd>EZw5ZW(P>h>9>S3C~j;Bcqsa(y2z^gs^EJ3anpejNu0c`h=M*akE; zaWl8Ick*S2^LakqEY+Hp%Rr%rM-m%-4dN;D!JNY-)c(AeR)kqRIX)g)F+RS!nx30W zewdJ>beD~}%)ddK3QG@v4b~m|;e-`FN_!O-9yk0sw z?`A1;Vj>y~2Sj%G;sL_b;_R&d#si@ih~sZo1MiN-cRN!lNxE7dMB&d}-94l<`OqP~ zg|wCdzB)X?xK$--90&uTqagpAet`ku8@M50r`OO}e)#C~bDPs*BK9s$UQMSq?&BWZ zIeP1@VI9a(oq9P2xMO6WP8PCbJ#ZsptWsw_Kon3Ih4~1hHUOxjH|OZV{{H?(t~a}b zl6xkG=KUwUjV`Y|J+c^8VAtXBQKQKa9w`2TaQ$fN*a=5=Y&H1^G=MXd6g_sng1Q3l zF*D~6!aCw#3RSopJ#9g@k9fYQprX|Xphk@o0P@GJLPF_LGi#wx+qoDi(XQZ>k{@x! zRv5IUY4&qc9B4~AX?A0Gt#)!6{C}`b1&G9HtauPy4g!hD%(D9xU#;A-ySEu+u zexX9sKNtX#)c>5C(*G#xS;}mgnEqXr_&5fD$%FJS$kmMqqF(>omX_APtKmYRDjSW~ z9{*o7${+6|+xeI{0><5IP11XaplKFE1`G?x@*@-&26}+`lPGR} zBwapeU?6?~KhQxTMG7!YnFtS|Vvz3(L?VEpDma%BqnyV4!EZajsM#O{>)Hj4Bbz1Q zd2={5O6sY8na&21%pqF#3i7Fz8)3WA>VF5aFha1*MdGp=9M6{E=(vy_?#!2KDrE8< zS)g=+90+cOl02;mjY6i!y6Zt#h0XG@e%A+SH>hCW{}l7wb!P3C|G1~8r6}=n`R;nw zE9%{5Jr>#V>;^#Qm--J`MAd-OTL{FrAC9MEITaCAS5w>Lb%l^1S88Z;zt0r##9x-= z^Kt^*5@r8)K`1l`(#S<1?0nPXCNB*=O6)FfhR({xFL%=sV4#fU3Hw%i52;A9_=f`f zkb)UgV#ZBQc6GLSK06^Ax1&<9P@*SVune}gwuWHP)Z9(s+zy+4r~y4dXbKl6h3%6m z41{;x!FzKH@5PpF-+t&*9vmxOT%Zn;$Ph^5DS19LJ{cAK=V5;&5yDqyDVz*@!xIcJ zbhsT3ct7&$oxEztp^^_{YyoW61r-+;SGziF=f#z!DWqHQMqmn5K4Nyz>}q2WVSP1~ zXij4Qjbnfq*2xFx)J9jmn#>UmOlx?RBd>-g$jrQWGJ?)`O~nDhV*ZQ}o565!a6Bmi z6oCL^N`C(NvvB{d101c?*jSL-^-!`|rorN*qXF=sHfiCGp=UU`lYIlO=tH3Vda-XV zcLtt+tpI%wZIwZGfw~Ov5i_vt8*0M{8j}kr772(@&86P+MuUyeFCXz!`^g2yL_~oq z0MKjl6F_JBG7v?=RXY1d5mRO02eI=&EWo&0JiPPk?*R_*py%lkLshN7(8LiKFWg~-^vb7#B9jjUyWeMpuLjbfQ-ltQ zC@AcD$Y^&K_1Ob^QNki*)A_WD$?X0@MyaoYvc(UD>Wtww!n(24?N&gY$)VsY7`O`v(WGI3Y$6B}t5ldF&AS0buek zUD&3@cbgMC-@9I&Pswn1K`gJXxtCRp{FuWO_5~}}0O4z(mbIiLEcbWC!9Q>zkQDmc zWO3l|bX?FVeQLz1@au zUWl6w(UpNj^229eQ-BfaSXo(BVc=XyB|D02{<52e)~z#q)4|X|Z@*vjf;!`Wq!#)9 zmdU2#^>F6imh7S4o&wZTq8Ju*_iiuqfm8nTCK%ioWdZ0sR#sHp919T?po~!z-8Q$p z&{`Po&iN1`EgG0Jq6E!gcJDGrP8aD`9R5lAdVP(3F{$uo>NA#I`n$y5U(UeB7aZlA zbK0;9CE~Zphlc^(;x8h_i0`x=9xh7Jn9RX_P$B*+IRji4&(UHcNFiZJT@HS@+SnPO zFa;wDHXsB8SG%i!7_!k}y>t$6L&+kYxGSenwWcb<#}lSXg>U zk>>>7jB3tAOgkuk$m25vDgAoWEYy#e$+0G%7sqmvvHU1(s;{ z`MI!g*C(8VWtgSGs6V#r#u*mp8mn}^k`xQ?%@&H38_fBXqo$c2`o8XW%^ z3l3!6hE6eH2=HIYHtYTX<(!`9r%0!lh>JL?ps==$+w55{;igaoF~{=J^|sg3D$;XUE+W*as4ATp~)f z`8p9CD}+b)XO2C-<485Iltq9d)X!4DX&AMxCOaYp78}p0N&-OBDryyV;E4pfB^u z?DuEx2PP=N5VV{0&l&okAIr$cXmE6!Ebyj!Uxt9Zg2xfc^^I5E&uE*~EB%jz<{Ks~ zr)Q1p8n^QVU-vak1t|so{NR%_(Xbyl-_5!)ba%ZnmW*(?9>(e`7#?wh*O+GB-Na+e zs`pA6u$zLQ%Xw3xLA<7ui&bR4NbhXcAYX}k~`Q;)Op zf>P`0N@zq&lO7xfyoMTe@Dli1ov__;Hd4rzcG-`c^Dr(VY{-Nj6Ss$%3=0Krc*dZm4^Sy52UZ?e&mL@t42BBw?`8A19z+kP*xaUWW9s6RlLjAb~n3!x8bbxm5K zAVs-qXN&&jO#VWMG}3?^%7aasJJk>dSCfOb+ch@1LE~z0GFil9qltc9lX`J02&kJ- zIuO?Jwy5&erl1VDUm^Z@$kY83bp(Ony>&^|=o(Nx#eJalcUb$=?DlkNlmH(FnTYku zsVb{Ol*VPVszvRu`= z1*n+PthY53(Z-;BL^5p$nwb1?BeGO7^CYxaDG2nUrW5n#vgnCREp)i9jr#OeN<1`k zI%;;9vW~?@UVYdNqhP$=GDTMZa@GvtjJsXQcKAw;Ake1@aJj0toBE~g(nT*0QY@eq4lDp zFf&k{C2?1%_RsgBxkTch7fmwtD#<6}V+6=zxNUzDj>Rp_*&tt{cS z#&a%fp6R>Of3Ye}MANj|)kzuH4NR+UWK1{@Q2XI-HZpNnFWN0Gi z@sJVgl@L52W8o9`LaDmtC;kwKo8O)ZF`BDG1pE)5J6L>^mXUrnqO%a5bz zP#7~S3dvzWavT+5FB>cfLwvo47!dJxwp006Is+Mi9kTD`_j<& znJ%#|W_3b8XrGgZe5(K#0O#r1iUQO0UDbVR?J4DdJsqWXKUjo52_x^qsPP5cBO#AT z6~Z^Y)3x>;N~9LO)IyB&kWKT6mT%&Y88@o=dcg;HA>o}3GR@*Eoor_t9VEP-_Vyx< zLL8{2LR0}(9cd;%P3Rb!&VSg+y7^7{Io@3zX=0Z%XjOcoL4A|n>)I2qWKcQtHg7js z_7&Jn$lX}0l7gV1Q#;I%!)REF8EH znu&HNJcx2YZvb#Lz0VFY8=9;zSm@*C_Y!C+d;_eX!RtA2dvGT)wO zSny2B7r)v|;EDU(Exl>g^>SI7VQHz=f#C7*RwP(IPSyu10V=@a0T2FS+x?8IxrW16 zHH!bC9^Aru@AUMt4$frin4pxA1!7;KlSUv0G&E39P_WO${q+N->L=YK$QnH67)fl? z^lej#JL@&uQiS?8-UhST|)4(Vb z3@~E5Z@NoxysWXX#J-v~=2~tzN~HWc$QZunE>@NsPHvu5##^OPwk+axdaiyfE9>pFC9}?EY+U-7adBdWsiES>HGG>Eo7w%>=CW$(&{B@F1l~R+-s`JzP`l-k zH#?-V_3D$-&}E}1$9o&c)tjiD%R{~TW<(Rt^YROyNB>k5j^F7T=eMt?C9Vl=uD1#1 zLasMmi43)IkE2Vgl}w(NLmUYc=b5C#_HuifLd60^s{+gJy%doYO$u#|&F}RK&0oRS z-5_Z=7e&69A+`-n9$WWf2OSFn=kusB^|rV=?z(MPh!@X~$w5>(0oAxVAHw`dPhT%wI zO{LAG?+oGmr0w{J?eN<3y;r^t3#0`%;%e$+8u{;#Q1`|?-(CE$`ZyUpA!5&}1Lt%{ zS_i$d-lfw#Wj3whTqEvNKhTk_dzsR`!}1X93AOz_Egyb2 z{&Hqa6>`z+;uL&-jYD|KJjGuO-CM;2_x|XlYy`vd31T32@p(hFVkPVN*foxqX%((h zxda_sgQ}DU1m}<>g|pNLwbjplEma#B-d-->ii!0@zGdPxi$19+-F1*J7TGGS@*Fh) zcY4(p6wpb==rW4_yeXz|>lBmSd*7@tv7e8$o_RaTsX0|#ZN*idEGYkVKrqlP>0cSj z+o!E7O1)TEZ)}PhEd*OX0+A-7g;KswLi={?eR@0FXH&Xhzpb3uZv81z=5SZyq`iws z$6uRG*4r}gyFU*8OvH?w7n%=UDggr8G)NYRN5_te+_i{Zui`t86sya**H?GIuePh@ z_VluMt9{BG#b*XF3s~F7-F%vP;?gUFmy}OdNTxH2oy06~1X-b>B{}oC#dr0}xIU ze;~^9)UvEly#471QQ6jE{QUe~WP(mc%-`><*VSL;)?R`mXDlpX+qar!+Ya%bwc}y? z)Y6}d?w9|XIls%d^gILnbEyIFE$=F6cF>Kf5% zQ-qr^@HK7SBHO((VqU1ag@VA9zl5QI2w(ueFG!4viH_;~og*HrggYJtZ75D=4MkU8+_*C!okJ{y>!bKf_R;XO5SIR zepDHH<0DqXv!a{O^9UNw3k8czkBEo}z^`d<7Z3}>mF+XPvg!-aXJutAPST4_Gd6|- z`0lY6z~;Yv7x2NMd}R9qa1u+Q+vqSAt?gLYajp;EFdl6CnD@NLwTBNE(x3pVfq+R# zOoVBfYv%J3R*)0DYYui(MX}N@ic}^<*s-fY;Cg9kX-Ns3BN_?G zg!4pdb*u*U0l-Iiy|$WbkYCriDg{Pg&u?DvK3uuIY)YaC&qw%9;x*6nix%%0Md!Zd zJ?joGC`wM^fNRug{`j$p4TnCnxVX5q1c#2zKV{1%CVBk8wu>;PqMxbnq9%eOuJ6mybF$(E*u&Jzr?GBA|UB@k-={*f0n z0(^peB}HK2U_xL?ejC(sCZ7QlueA5hzj2blK*h`_{|UCMX1(O#hh1hwa>2kt>7upxXZ}lbRwYAucZb3_wpwO-)Ts2Edq^n6+8i5(-kHDSDE0Rz8{XwgcL0#HfASQwUBO1B$_IATyMD}eT=XGzH4 zEXiv$l7A^FO;d$j8rVcH+N`_XSU$HOa|$zEVwx(nf{YvoF~j=DCpbCEV2X5=oTy$? zlW5^dQw=d*sNm^avS2nSP%oYXi#p~_B=uS!l)p#k7yEu~Q! zNk7%09VHb~nF>e1(zD=dxV!W6AykVNl(gMxywaGa-a9%WNKV}3r{fvgq`JLvNW+y- z8<~mO%F!m95kMlCFhQx-9L}bfr#95Hr1GTwisO{qbzWC{19NxM^{e;0DL73VTWW9u zIcF{Y-vuA{xF!XAT3lULY_QmC=BICA)JC=ZdF(%x;vj^i4?)ak1dyU+=qA<*YpGhfflN`7DdGts0&*5NlnB-WR`_$=oGw4>=KEB(LSdck-a<*p+P0v(E zDdgua2F2`sTchQWU&29*p#Bm*LOG3qKGaexnzLbt_mRoLL1ae}Wmu6|dGA$sY_yv! zz_Sd31{`XGyzgUkp~YP(!gl?}XtQHUV_afmV{J8@L9e;2qY!ZZA+0=oX&tG+t^WBm z$$O zZA^_FF`k~5#)j@TaC7quRZ>yn0nNZLG=nU8TZA5tSz%hnobznrFK6bTjVZ%ZiQkGw ziSn(t+f(W(GySJXZlD(@^q{)oKPKYU93AxWz$NsApawY{4q|HV1zsKG40*Z>`c1R4 zcp6C^83;*)N$irz4=J2-Z;GOx#OR2YS;RQ4Ed>Y88zT2rB6y?btf&c`JuIj5W2fn< zH)2v!CpfLGlR$Qk8vYTp`hOUvfF9AQ2bC?%_si|MCGFj%bU$0qWoih=NSl@6m3> zR+8_*2XZ=NX5dW~{Qz5-H3TyJb01~Hrs`)R?zyHN%Na6w!NS97Ji|&7O%J+rJJu#; z9-a^0u!6d7o`@_o9|J-A5}=lnjU(Z*4Uiq#BYg*^E>aGQXK( z_slWbf${8pJZ7<-0VX4rC(z&v@+&oFSgtOai82%17nm7V^p(W5YA>(K1r~e^SYv&3 zgfTxrbXhbo&p&3|*ZqMTgxt4qq@7by2gW*qJMPMHK|gO;$;iN$B_|FIz5DV_cBntW z8FJm(WfgC@xC8me@pXoMfu5B!Q&F}agbA!^kzV;@ z3)xYN9;YgI5{&{PtkRs;iqePFL;bI=Jv2psoh>-erbBdrE;k95JDjEt1|_(@ERv+H zQj}}e63&KDed2BU>tBEm;Xb~A>>X#P`ghZFK`xKGiO7m?`RQrDyrhrA!}CSb*F;-P zZMX9T&%r32rNhM+2asT-B_VuO8k`%_t(Wa;%q-lU*>}Ukq|@!!=Ts)%$t_n;n9TIz zHuG_PH^sq3!Iz}-N3c($N4(G82o3Cr{5xET*F$@f+GloZSD*VBkg8hItrNd zks7V!#l1Ieh~qK>rbU4G*`YIA)_?X&bH_QznH-Z{5O;BA+{nblL|Cph`?W2>@X=jyVxl10Ccwp@A>}hF;I9hK#++}S; zbev>ak~!sq+}Vk7X*_~?*;+U{BM>YIkX;48(d$7aK_WpM;fjM{4WEhCz%9pbtK!fk zk-!(@FLaPJ95*yH)YjHwK~LO~Xgm26&=V}Jhvj4h7y(xh5NG!^o!WZxjhdEY2p?d=j{$X1w;>IO~ecc427jBFhP=q&N&nJ{Z>OEk_$8j<#5;Dax?)Nn`Ra z&rT`k^{@vxa}-7(xDX(F1@TCYO@hW?Jq*vBjEoFYQi56Has)ptFGG+-0$K1yjJ&`R zAI_XPgLOPW2g(F1X@E%y91s_%;A*lgEh7*_2w+sLZ>d_l_r+t?J0Ny+WL#d{#QgY4 z$%$#OT+@0T%k&^xJ=)p0Y3B__w)*z+>aGKoZTqQqVB?-#a`pIOm-CnwM9q~&Wdz(s zfVd-V$b(rGt4i|OBa#D#Q&3QV+gXt;*d&d0?q=au_)sH}dIkyO^~sYb0WLuaSn*pU zk(xfz4^-W)D&fG403+Z50_giV`f<&!6}a0VbKW70`(2Wl|)96fpz>>Jug7_cyr=#PJ6;%KV= zz``&B9wQ(u51nmgJ6C?ZcO}kNL^}B?V=l;?J0v;Z?&Rrgv7y?8kH4Pd@18C5&w#4|5pP!$dosIHB z8A(fMUz9yWNF>>Mz#wUC{p_>P@Ig~TXgigB1fnl0hSOvO7y%~;P?!JDp4V6Jc(J*? z9wD=mNGTOYU=lWNXeCcI>}>05f>g=z=`+XP3@VwP zlFehF|Eq~}WCZ+x069v+5{q%5zP=t*5{t}{BS(^dBup_r+D8~7B$D&L$ly47y(AW4gq3LyN-VR_NL!q4-LLePsj&-%u78VUMtw2NV{jq zcO@J+lM@^F>4rU@*KcfW7yG!QBV#6xD4jiFeo@Y7wsCW}Vgy1D0U8_O>jEBm?%X+0 zN!YSSjT%K~(SS$#!Z|V&NF-?=;RgVuz#|VHJczjj4gr`>m_deKKj9LEEds=r4xQfq z=EmQgtrKVWB*hN7AZzZ>w2A2QbZ;R(gb#h}Pp|E+y&)TSb<43+4cqlL?pYIVnLO%B zwsD7TSK@+)DgqjhMBlHesR56KXCxGAAeWk7xUppq1rkZr8}t!l;)xR{Fld570yW?1 zZ;h=3PK^;@1O`O_-TY+b!8bp7?s(-MVV#Ld88KwW$c#(L?-?sbe88d+EAMd7K01L* zvvJeulG=UTjcqm9#C^PeYiGA`9EXkjvTrk|HFGdW3|`-uKdXZK);^o6k|NTDOGEgd6AM*@+%WP3+z ze;fA|7v7$i$!C=ZX;rh>jDS4?WS&KThcXfmJR`Bd#%O?{((q+=(342ol!PG?HZ1(E zsHn)z&BdYZemG;3TP`EO2>28M^zORGst>pQZS%o*kP(i49G*U9bmrwLNy70F^bu>t z;63`$r_gDMle;|zNl;1sKJLn~tb=1Vv7g3py6w~Y=ENC+^4IQr zQdzwGmO_RC2%sb5=*N$Ctk8~rEE#gisEpZkayZdPx+|7Rk1GvEZ{vo3f^6IvHeusF zUccp3<4&@1rzhoHHvUV~F1R)+A(d_1t^mtn7y)x4peZ9^(Z&L@xVRW366RS9n{+Rt zQI7{C(zjRWxLYvNMB+ChLVpXBKb_CcBcQuTa`?UoY*}{W{2T6hy?2kk@yV(+L4Ve4_gg7GV;Y6kR>y8Y7>!2ZF#LR zYxb=7TN@37jEYN?6)Oi|j*g4J`PoNL-0{0NR=xGcs@qS#=2p}&z;loy($aa|uGb!Y z`L++X{}qRaXD5%CJ^Z`VhusdY1Hw~OR0N|8G?8Ju|Jut&qRlJVgan@-oJ9;Oi6=KF z|NOkmN{fFmedrx|DWgH$;Xkl+)i-|s_I*2#l$l}H@v`8KqVp=z2cUA2WaG8R^a z3qR1SEd>nI7T-OM;Mn}E8vaI7SjyRpQc&O6!`3$od@Gcl%a)0JLsFSR^l``GwZB;L zl{Y^5H4YEQ(T_8R-8pmEox}4k(9=ggw9Ak#hM^Lk08mlDwG?Vyu=C-=hl5j&$h&BI z(O0h+u?WrsvC#>K&un{P^&|i9rCZnR_&dA{LQX5mt3*Ru6DML)1ZdF+QiN4E(IosP ztz=*ybCF1zerR+9MzlkdZP|fsBB7J}^t&G_UH7uyFkSxolj~=auJIw|$z_`h=RdQ2 zVR7-o=T^P-%*}<uXD@qo@#*OMA8~3BH+y)zWRo!VbOhQKCz^g=~kmPjC zi2#uzjGV-gOk!0q9f{O%0a#&jM};}tYe-5pZNVMN3%!Qv@=Y6#9TzkaO2J1zR*s*x zcYK_-80>KQto4gjE#u}*YfENc+IP21ucGnsb^SpbAdBLK3#Nt`!sxk2%Xk0!jl2Ky z!GkzF97jJ+%>Ck(BkwscZwj?^wHOq-rvRzB1c*nHl+JbOj3qn1)>Y>u(>o+;tTLVSo3&S2FP2_WTlsE)r@GOE3$36mMEW^HyKfn9n zb8p{`!^3g(G}~;z=_4UxUJ}WNeFVPEhN1*= znpv?{7Oiml%#yNA1MQfBigB6xUHPdKO7YM_WoS{!u??T-$>ix%$4ZKZ%G5TmTp=#+ z>-#v*BG>`OD0Z%)a>YlFVc!N04+kk3oiz)V)A5B@=I04>D$z%rLqYz{!Cu~=lV7uO zV*?XzZP0AoMR}vJL4MwO_l?iJnrz(gUWJYOwROv@>&t`K#?7n5plhQnusaBtkQbSE zST`Y9?$kAhH7f#Qnz6}Nc=M8lCm-#zEvE3&Sqg2eQEpkFJVu*l7QHZh$ztrR5#i;x z+BO=S-=r{zE3@bHFSLU!%!-UVa)CZJwbX-iz$Ci;$iER4mli_%II-ZG!u(;dpMtNz zGY@wv_?bSc-UW`2aKe_RJ_j~wM>wflqi&i{%Z6STh2Bk9?x16`u$oK0-9GO8+@WLh zOHpg^TO}L!hW)QU_R8&0DhIEPn^%b@Yke$Eh$9ez$pAx)v3gOr&a2b_7Dvjpx8p2L zns{n!Yq8Ui@2Uy$2FS$=HU!X5J38A|Z(jkz0onoaho(+AKWi?IwzfMtJRt5U;D6s= zuaq1+t}I+`zi$aU>mNH=QgU3GrG5nxDNb?1CaS)*AQPEsdyftHPp&O-5G>fLPL-GT zKjC~3jJG(e95*Q9ta6-S+|XM2dBZ2C>v#89yAd=yZ7A;Jo;K#%*qC_J?`}jLtt|R2 z?R8T`wU*6(=uL5=f~d+DkBhg~;)ODR`lc8aTDTH58zEtV&&guprrbZgNbO?^Gkea7 zq6Mt`*Qwf@QV?vY4& zt`m0U+c=6D>&AVVs5WsL2B@TVA2+VOX>D&hQNQ&>!?u>rI;0vOn{@Gc^DdwC<;>K4 zc1Um^W{$xK7>0m3Nu+tr*@gkn2{8hUfOiPc*;}7}_5qH5!~rp2keP`^IQsF@VPC`1 zkFZ>e+fwAo;U+oyaVWf9kM%uUzI@C2W5u(kdvp;cd||x<(1fwcT@p4QSbpHQO4r_q zP4u}##}!-w8~4Qpx8Qtu&Bpz=4<7mKGp%| zQ*OrzjtH8&q_&S66$&WXKRxo*Z5{|F+pm z*-G0r=vy|HDRX8xI-ApWP^j(JqJB`6zrN_Pb&AIXG)Nj3*d*2fP$a`MLEFbIbj-Qu zU0nDjocD=in{f7Lb;Id@tbgi}7jJoM1#CCXv2EmeXYff)|1Q zM?c~$7??OOD3}Qyl{mb*punv3(T4i8>2nnIbkFkFR$^z&0`=y~zSV*a7Ie%fe&p5) z_0h=X4_080PM@!Ww_XR6oNU~?BbWUUi`Ny`#Kp*8Nr# z%J}@f1%~6B7y+n}PE{SkDUbC{)tX9)j(&V;^`qL+kLPEV;^;@51%sm>i9Ui}1jEsf zQx_~OJ|-?G+OqP6<0Z4Ex@XGPk;@RfopR77soA(u%XC(`X5$`}H|?_F-=01EyPA#r zH*fvV(v{z28~327asWmkBoPQZ5_w?XUi*ryTJsw(*;h?jY&HvH-ceb_XQ zm(K2cMNRR-=g5?~eBp^Fb;Pl_MCrS?S5U}j#hHTA?s@yk-V3+~5Y?9QbG^593l6$* z`a_rdqHu1f3D)@0wpTFyC&y(Jrj2W9ZF*0OVIFWB3#NaMj+fsWOY>^)}-nsbu3CX467Dzy%|~QHH1TR zC;zZ%@F-HRKCf>gkLXgNZ}6f|Vgon8?3|oFbdgk!eI8Xesj49dlwW7YCwat+-^k>k ziVXvGWcerSm01gJnN_lGql1eMaY6QyIl4n(X3d(t_Aylf*}r9p} zGV{_x!7B78zvx7oI-^uMe%f|1_Vl|SDqZ)oekQQpaOfJ(IIiA~`1m@vz43eu6DD=s zR`Smc%b{_Mi%QyF{^o{5uR@K4LjiI0=SV@@9+p_XX8dwAscsb-Z$8u zCX%Fw)q0$+;r5!cYt{`Yzb%lo;UiO_d4&mH&7NfOgDpOg{jGe{28Fg(VAF2z$uIUI z6%Xz8oRgRE_Ay&a&ar&f`b9U~c%SX;n1KqV8OoOwzp#jIWH!kI<`om02u?uirL?p(IKyGDZlLa-F$r8OhC>9L?BgDve-+#%VdG9u z$j9R3qn$6o#*M97zTL-dqE#a3>~Z}_!cB+~2w?=mi$szhkQf{>sJ*Y+Y|S=;sc9*t zPEtA=sv1B|W2v?Wn-SdRX9-25gOGg;(yGdG?thc_r2401pM5oh4%C7~PleT*k`(k- zXUcXWzcOZ=&|Ol=R#nfwOQp^F>_fPXi=v7BxyN2yBt4dJTdsQYOW#7{D8b^Qp}Kd4 zH{Wijj1*rARn|kL4pVRlVqXPkpmBy8vbC-Golk$K7bGkGyq4~olP#O$QxwogY{)g` z(T{<~j`Qz_Kz|#z*nu@{7&h|5#(jD5_ht?Mc2U|y#M^%4-(=&)y+^*VahqqASpM3w z7ltoc?5e*Dw<<;;xDf~|5@}<5lC))w9WRx9s#RDebo$y9dPhmORPLz}JDaw#uQ6Ms zY9A8Zkaofl;Z!w(wtXx*_6d2^%{gRe@3%##TJFlW+dQM`CP7KntaSy@fd!uOH2k3O%4kjp@C z9cVf)bQM~Q-*#SHtx|`v&hDcdyin_{$R`B`rEj%!f9x(+5P!C_uVMXvPz_@ z+kdA&a5UuVnhV4T=n)7j64`fGjcO_GYiL$YyOPMM`Z~J?1D{87FSHr6`%pm&Y$K*H zQIn4)aY*}+u4n>PMP}7LY04)0X|r+f({z(s+79FwPMSj9sw0w8NC$FUJ6pR;n7m6es2+U8!D z0f7wd?d{do)wnDHCyCV7)?!5quPp2dbvX^E0oC0l;8~>S1G|zGk6!fMdqN4RL2Uv1 zqz*kFu}q3ce>x0lOQugqEu!|VW7+j7YLe?@wG*RnA~Hh1f~isTDQl@1gEXgNP6W`! zIyyS)>+5T3YFb)a^z$^nf(aWFKOTHwKS*=dU~owVc8CYNxZ2v<(B2vw8=IP%#30{t z>{P?{Qw_VRI7M0K&zf+{Mdw|C#<}b8hJSqWXmn)Ex4!VisGDhiy&cfDXmcc%_JIM;EmS1JFf%p;SU_SZn8m%!mLw zJSKHa)p($B)ZbEr_n5FjAMvm`+l-YLv@%hvLwf^#Y;A1?kHjN5Wn0au#_h-Jw{~{7 zp>XL*ITw{&RXpU9O$XnCb46-G);%}>Aw4yd*E2y|^DMZ*2r!8p1W-Q!WJHy^$o-7S ziql{Oj7EU$hI-3`{xh8f$wK`DA&dyFy$>!!}C?@f{qqFUF!=BIUH#WA5 zb1bI_W6jmv2klcV60ioUYwY8Qtk-lnY4lLb$G=RR!HyqNn z%0fnn?x84fjj}{Uz=AVU4g@0F?M482djxik**RM3l9kiO}=1c!mJS3SR{DHg1qe zQcOZgS=n^xM8o!qrap(}&b#QFS53JCS1*PyUR-iUK%3c2Ng@q$Hw>$47Q3E(f5Bmx zKnnUI8I06_MMXwJ>mUv$v?F=+#`ZXdhEftgl2rBm>6?Hf0w}|)4`rYwsLxMJ*0uTp z;nV+sCF6?fF56ExtbF39r?W@?boX?qKi!>B*~lN*oyAXAS3CXSCD=B2Hsr;Bkx`0Z z-8LAt(MHQRLWz!0qO_k#1rc;*vybRnu{abX;3)#ocGG>_IEYl+$Bp}MN^&n8n>}w_ z=C#Q&bo{P(*6e!m@Tr~dZL;xHFpkFvFajPR5JodwKMYU!~M z(Xm>@e;`WoU(hD-9pPLhw{p^-^iGozp9J}Yj!yh1zSPL5P-T)AaBo~So`^*sWiBEn zkxY?}(2Pp0q>N6X)W9~KWMADR1%`+s5$O#k!lPVB6~9#hD1Diod?@V*5}S&p|3zh? z!eS1Ni;IKjBF^HG|0X6R#K*u&Y=VwivjAIEQhL0GqQ1 zX(4_9h})>y5DK<5yE_$7ixM!@+`1p`TtCm_vXLL=I{CzR3`^iK5_6na1LL`JC}RB|Vn zq6k9bsRT`MJ~bPWDTJ3KlS%*e4v{okr=3Wkv0!24m#M0!p~#ZRXd>50loL-B zIAknzJh9-Dv9ZZX$%#pc(Q)zkoS2vh&KVVrk^vC5KbWl_usl9PKrA@r2%vX!k&7se zjFK~r?U8;h;;}ztNDHM4u72!YkJ(Z|*AzWcbT{W4%x#NOG05~jY$5Pjg9ze;{lJlU zcXklBk8l-T4Dnsv9l!qdv!_p=jE#%2c3L7;4M%@Sh*OCfz(&q`Nu;1*eaz8<4}w>b z?hxHiBYF5BwTw2kpI~AdTO*^kjUd`=+>-2yKw2z~=gDL>HmJ%>sy;#QWL`w}Bx{5i zkiIuNl8r>L`8M_{DfMj94h|%u#_i>-5Dz~>Su2<Gc%!HU25KX8u;{u|OMDc4H&V0D-Z@10)kzwa~g==>jsN4Ygz==(j^(SYJyD=u( zCZ|4eSgCP+*F7CKhJY1<8{{ zrFyExK*UwG??eIRYEum(o41T<>uz`>Lz5YS61RP){i~_@DPdvakYkCu=|`-}WXHsE z4yr@V3ZY-@*=!xXLx#U)NMsfuz9?P)rD7yi_LwVum!4kbxT0 zlWqPGew5-XX&t5+Kz!E1XCZS@Kre`sCAY!C&$CSC+vJ^xR zEKt=QibQOTO+=8>m^x_)VjE$bJB3;l;vzqb)MrQ~sc>Bs3I;jH$N>1(LbQqO*EUnT zkVu_#ohsiGyAL3%6?R3_Vq*sstZI9^V`=9<5cj6)BFBZOw ztnGMnA0-hn2dy#mO`YFsghuTwHhhFut*7UcegAr6)AE#rA(v0Q^|J9d#m2~#7?h0S1~Sq#9zw8b6` z5~_C=t@o((Xy2O2L49Le!s&~TL4jx|4H=RiTWN)=kw|rYsTmjKT2@U{zmBxEPod3} z>p2=Eq=0ZX5xp)tTCC=1VV9&@421arpFuP6tuQRa#KI`zya(~OVTFJnI1+0j!>UDv zPGJR!eD>LMHP!KPu`O+_7RlRs*@I+1R9ZU1Q156@eI%+X-dO~f)xjyk+O$MOPX)su`ilpj z<<-VQF7g2&#Yt>zd}=E8LZqdnrY0xh5x=0vghB<=UPN@PTg}mqK01v1!x7W=*&mRMNGIQ!NI+`OU9kF^Gwc)+`2nqSIo? z_KOy66u?FYWtWuQ($=s=7#MzKOxc!LP}iYmkulj6ok4Y+0Ip|g^c)-ekrQ+ zwDh47MU@tC>6#LQni0xN`ACZae0^fz{*1rG8`jyo)^0;86qCq7s?!eug;awIp~8Sf z{^r?dE6<+E%ou|6KCnx<4~e9iL~159sz;;^FTrmJ37Ty5V=E2X@{QzKHr)nAB*mE) zMdO58IyR~&7%hEExlq>hhR*1g^H2oNz|$0;c!T%gku<~E}(DB?3f4cS$9six9iJx?&@vDR&f2Z0m_nh)amajE4y>IVA;-n6Qke zq-Lh1q!Lz`_CP!FTWtD?>%%gU=7v3Qt=Y8#Cr%I4kRQ4IEqwLJ%Qx5s;P(WGyEsfQ zzID^_S2Zw4jCGBUh{o2{+4kITeqCErjULt6*@Xr}4O`vEW@A?x*%B55 z+oR>uyVw&tLh?3Nt;i}-Q+0Nt@!M8JYDUEBqykb~779v7iBJa@oz!M_ppL2jrZtze z7zl?C*>TXc0g_EXH4j+DnpPH3{a0r(5W|j|j&ROa zEd~fF><>0>D$^Hnsgq(6M@wSayCb`M#Ek%WFL|o~0c$o`!a*X%NfxoOL((&ZJ4$Q} zYzVmN4HOd^PV`JNlQ@2;PTa8=444#u;a>#&z>!!J>E9~l+&o2qXqiUNy1F`k_v>Gu zJaH^LD-$DNWpy>}!Pb&TU|ey5Q{}`AAs+-FJtB?H7>mglP@{2Tz>SU;oJfYbfh?7a5_6e5&A~Q= z#e0#6ghG}}Y`jH|G?htp36ksq@x&E&xa|h7kzi|kTYFb0&MfNc2E**?>g+@e3#h48 z8tbKX9#j+mB_$@|`}Fj5*v4^_a&vp@&fUB3xc&C*>}(ja51(7tVQE6X^NN229Sf0E z1kTOP#mmadO5D?^+28p+wRyL2pk{xxddv1RuOoV9+{g(9^Ug265D2HFq_}V{(RBei@Y5$ds0O=#0o zN&`1+5sFv>X`R>BXoUt*Bw=gDB$e7><$KXFX*Frfk-91xs=dX4;!;PpEg+>EjJlqr zwVy1!T)XOw2?PcuKp*IT!bkq4sxqnU5+V8^2M6+KCdX3M9=3{c{Ho?2nF$%*f2Dt*fIMgmh-4YXheNi3G!w6iVCSIl^O-1W5xW6^BEr1(AV; ztqs8hmm{qq9_TQTf=ERCPjG?B;SYRAf7)g>X-t#)RD4DDwD$H+q>2P-+Cp7`D_Vqd zQ~}RJrk!1#*o=chgWSs^L_j+N9@*X11H`1@-FTx-bj{5z=U;IC_;KT}yZ*Y!*qCpA z`&&;u@x-)g)2geh!4Xf?Z9mnz<%?q(R zfS`T0jetUI%3B?Rq%na!WH1`&>>eULt&EflECw=Zj9>x~q>R!D1BF=TAwMAohDXWi zF^-}0;tdw+Nl9r*i78;S5C$)?CV?gsoeu2*3>5g()z#quTggY=SZ27`kT{rU2%yH9 zL^^}c4*+>qa~#Vk1Zb4UApYs6pB_GZ804U>wWYnS6>j8*4jrsIdsZ-p*4FXkCyX68 zwz{fXbw!ekZ#f%bEEXdyMobV(ZO$OFD6GFU6A<4LgR7u)f&jriQBqQjx-=ybf5Ru4 zEb%@X*?YO2m{CX_2X-ZinJ}Ami^IW%6$^$YY4Ek}LlxQtnzv})6JJTMdO1mms2Xe3 z0%%N*z9SWjRQ{=&!d?eP6gi?RIw}?uLR)(iA|@pyTRJ;h+gqT|6tjs`Z9*iPO_5%5 zN^(?0WPRN^dnN@INk>LNBi))HauXm7_H z3GD@pX3^2Hs50#GmOH<#smUo-l~tYXotS+wq2m_2xY+oNjI@Tv`nvi?q%3TmpqO#- z!-fy zq~X@0BPmpLeSIDEFOA+wHMwdd))qyAO-VgEF+yx6Q4PCN`wp)_)H`?&ev^>YSu|@i zkw`6Ne5S=7S~XlOi0YASv@Fq=kz}LwB=vn-$U%=si$*CeIwRJZl57fD=~yrtVFdRQ zb`PNyRXer_rD^n$HByEG7+GjWcG*j%%Syvu$({ZYW|kiT^}`@}9|#uFf7TD|^Iykup@o-KsX1h(47NJ}MZK zaRPj1X4b4(vk>RcfBnBlfANbeuDHV07Q*3tiGUwC5}UJqS%I9k*9g$qefI3x1q&8{ zF3g!T2Xv;Rz3tt9{p(y^9TbhQT{gFO-Fx>UOhuo3@(CsqG3G)8CNB2_4->{4!QJo) zJC3E+P!6AB*cAj*dL`1(DIB)MFfFc#5u>dzOGOG@n$$89E5(o=4L?7jI}>w3Z~4@} z)xFptPnO6Ya7_666PT+KvC?y)K6@Vrr$E0VbL)ViSm)h0dMp{ph z2B;9KtIt+eR&?P;;;5b|n4T>$X*n5r+4-1ZaEc@>l9(?AGQz$VdF=R+=9c;>TpA>} zY!~cdBhNd(y{jW3DY2!sLsB{@I^nVzb#VWI_O>>ds=>@VySj!I4a>{T-?e*3OGjsZ zb`J9W{MfM}>FFa!js!2qg~>n$tn~2VBghWv-0_v$zV%<<&Mhp+$jreVJnik>xJ(&s z18PXvyR|uz5YuxMf{EX|ivy8JfDA^CDxj03b zBA`o<#ekN_R7Pt1a3nzXz#laTKw4YdrNcDVxAMp&aTo-~1OX^)+3+{%=kjU9K<#qIc(IZDN($PRKxf!gq(sn; zHcS;Tj3!Dmt_3Ud6bnc@e*xAJ#AdVdtmmUi19_-PisuVQS=KCdPL@9v? z5ETU-o}4ElG4aE(2^w42>Jt)@4j(#NQ*#<&;$ssmR-szR%*h;9Qryzo4T{;*2@RsS z?+3FM6i!Ev9j$CQ6_=c#v?nH5TN4s`Vq=Hqc%9&EiIkxU2Uh%oWAgqiH8pyN>HqGuAIDg z_o+xr^49I2V#>x`tpyfr_`yQa zSYK09)6m#_^G!FQiV=WP6aFW1N1P)3lZ7o+E|9SJ61%?0x+<1`v_iAF>Po?+j$e{c z@>!Jb9Fo+?muh%bdT z31QPC%Kob^=xlzIYL6s7D*sm{AT=ZvS+K_5tEUuI;V87W*5;_#Xe@~l5w_{{^b8C|vGED*?Hyv{JlJ@*B{4qX^odiQZS7)|63n(I zEp13ncIK(mC+nMqnZ83C3{{S+kf^J6E#_Tf=^-50-j-sn-cEco&T zW5%QJ4|ZyLxOJ%jl-Ca&i8YaGy^<%HWEkQJKy7oDju3za4P)-flPACRt#AF{2S2#< z(&-?Ctu3v~mpzS9a_hEj+qZ3RYU{Z1i`T(b4zz=ao&1e49`pj^wIFF25hqR<18Rr_ z!Gu7Vva+%c>_1p~`JB4?I?NZ{JzXH473Zolv$DW1F-%vUJ$vfZsr>9b7;_sN8{{Q{ zv2fI@Z#XCPjq(xpEMZyek`7%$If0y^w8J2bSq}5XWj#^z57Z}HV^*D?{BbAXBswy)uDaQB5~^FXun$B;#&kjJX6x)?^K~Rl2J*^NsW`-<3d9TSOf@FU zMIO10P$c!hNTGof3$sFtGPN5==SOXdZ9f?yE{>N4SBtAVUC3rf(Ao2K_^2P{lG7HQ z)Qb{DeYxl}ntfYWH(VtX;*%=R)K;E74sqar0tXn35gFMjd4)M3go*KK-R&)8a|c0$ zkH?wvlc&y}?6gG1$Hpk_(8?=$g~JOA3ftS;l9N-Lo0~9Lfo=kq9wq9tqkCFAni8Uu z!4|u_o05~$M~$A)+R>Vj1QlZ!#t^F_l$D^_2lnrei3N19nOLG?y3#b7N&S?c z{s{PO{9zI~Nb@rUAan*yS4mtVc%+Al&20&+td4$v(@5fr$PW8MHEgO;un4tvlvs8%+yr6#(gqlFxp(ghjvF^_!o(N;`hOpN^bsZxb22_Q^2)0(ZiQt%x}dYCImt#MF~i{m7B~!S z-?ptw>8z=5Zf$Iiu_~iS4$sWXPfZ&F7lOn@$~qnx#zrMIHMQ?Kw4=VUrn6l&6i&Tx z(y*dYjV+A{$??!c#>OSU*Ab94G}e`$K7HckY1#sw7#B6~s%v2=Ut9JeoFieji;j*& zL9l59b}P7;aGlF~bR87#X0<#ttn| zD1;a0*S6Nih6Z`Riy&}vGc~lGa7Ph5QaVuLL@Hqc7N+pn*wPN=dq=ucd!qkP67f!A zPbUOsGbC;EoADVi+Q5qZ=NnW0j|9iQI*eJrXCR zwzRYgdolWHdmHFrYg1cvtQB|K@7}Y|8r7DVlGN4Moe&=}Br~O{r8_sb7}|Oq_Sn!k;e9Iva2g$+9~H5Qs;EcP(I^(dXqtxdH}b+fOUx%yupB}ORU z{_c(I%l5W4W39d!1TifwGczNrwY3Ecp3$R6W2KG3Pq>|QbYKt$k{MZ<8L8>8cR%yo zZ{}X{1;hKPaedBQDfx8#F@z&I$1AKgk-!)IfzBTROc)@fQ;zuU%HrExEit5}fluVu zzy9^As%oL>!*C5#PTe`E%@!|SylT~|_gAmE{+es0-%=Ni@{-#1U}s@_8!Xoe2M-=S z*B%j-lqEd~_l|KIw;!i>Bd1 zX{p&YbwqUPkPN($>J#i$jBRgkq{egWS8koZ;48BIKsVC!mlt37(t`OZaq-)?emZ8% zxRg{exgI@q0HkX6+^gzqDxvQ@fBd9(-u~x?4I3Z(<#=l?2@ZS;X#0Oldg62*gg3NDB^qlOz%cLEA{AP~KQ!u>N$z_O_;` zvyC;FexbC)WioTFx-32+37SsWrn}lYq(vB$Q#W)B+dkcjN&Q55MM7dU>`?^;MPnw8 z>4d(bvkQ|GWi2$3(eX_UO;wd=v$Hbl8XFRm6R=SeXYpW%W^-$MQgRCHn6a4ou<^*M zoU5sYy?pxAX@ZPL^x*uE>grl(dDpF5D>jKnL_>FiK8Ga%QUE8wLI8h)5Nz4<>DAX< z{lza{xBOR6AN~B;nspyfxn#O5gnGZZ$E}_W4y4@Gx<8#3g=d z+SkrM|NJRaE=H3&S6lton{UFKdBcVc+qUmGc;s_f4A4T*Vr@!9Vp>A;hXmkUi+(n8 z#Bkal1GP66HefEtj($FN%orG6$+rQWx3QxIN7uuB1tbiXgmbmEDe=iTloZ?%3Vn1F z1ncbS#1=2I7)bFzBGI9QzY5t9(3NuXaw6m7YOAV?+dHpnjYx`)+nAepCJkPj()T4Y z5^N0o7~7E1*(=VRId%FJdfd2iW8vzHy<%g>jz!?I3(1!VzHN+_UX1yi_RYs9Ut?_M|%phQ*OO=&W;`BN+itO=%)!l zhAalLk+BWU9s5u1E*@2|cgvBU*3PL@My8~tM_7lz%Q68LgBDoQldKAM=XAqjaH`@o z_5q$fR)vZwE*>(fWI{q*X4$$AvFQz^0x0l!LDigh-g(%2LoNwe5`6N>=AZoJ;cx!e zcW$45%Z81cK3cctvRQKs>8I}YsbC53xABKbB315Kl1-7%1{06 z|Nhd^3F~kwaDQ1akI4}m28{AWKKUR=ER{s>J9g&O$;p>aef1wJ z4jeu9;E#UNQdRr&A3s=K*Z8BK{^a6IF0QOPz4Y2nz3Co5!+>2T7H_+4fHv{?hew9w_(O|5EDwNTlwxUb!&0Jc*sx*u-h1!5_3MBA+dp8j2X?1f3`CE^ zc`}h`QP32|HP$uNH@3np8GT9~l`l{9w)8~BMYeaKuh|ZX#QHB9n?=M8F<1wP9oAw4 zToha<(Z0nVI@rK_tkFH^CXUWaNKBFkND8OM*x2Zf&L)iZ03SN<+it)8qKl`*B_tj{ zeB_OH-n{OMU%)P{Z<(L2Cm|NO^4f=Vv^)zet}J^AF5 z|Ml%}kH7HZ{d;y6myARoT>8Z0J9g}#(xE5n7JDF(16mB~+uEaI;E$O6$@ytf8%KEr|(nU2W44OJsCpZf^Dk7mTg0tS(!(M(}$pG~~VGeGwr$(! zbexXe>DcM8lUsS-^PcnEPxlYFpY|_nSE_2)syQasnq#y!P~m*>9125{#85$_rh_{D zY482^)VKAgB_vHTuhnNPmi<()*W?7ABB3@~xtGANdsnJdAXM!2S?37jmk)bABT;dJF~)igk&5K>@| ze=hHAMJLV0(d~_^q)hkt1SD!WxNz7gPBLqL6-p1PCJplK@VX&Ko;?9J!XZ7`SozMt8C3AVauBa3G>Q}?Ga3a;X|HY2H+_T5n2UeDic ze=b+A7w5}}SjSC$g$Lf>_Ar&q_#XEOU7FqKd`_s9`pF-*<1+3G^4^liXKQtJ(inA8 z)AX;B^4_~^J5MN)UM)1|!U^9V2il7t)60*Lw!Kb~`rb)x?+UA2Zd+gID$S%w|toa2j;B*ZeA z+i$0)?S3fKDG*nYm98Et>h zxXPMFD05w#*J#hUs|KIQFGjwmh22G6o!!=!>t!?22VB4XB&z1T<$YBX<8y+)PY$p~ z0RAF|-2l~JDRhwv8ezlhaRXroo9e03P+3Pj10y~;0Wp$em1A;*KIz%dJ|5`9Y zAa%gmdmRJFALLJK@;GeCVyXo_Jw1_1=C;{wj-|Bx?tKiYQf?8_aX*9oi3pkHy^B3R zZx$0Fx6Um#_S5MCt#o?{E-CY$hZaCa# zwp`X~wY}|z5m=U2dOQX|8||YJx$cCbE9IR z|5N-!2Xtt3e0B}d)j~czX1&>wm#@Cck;%)5?rfW@*WfWc@Y_9tEq;zP>~)T<=2-=0 zQ=pjL^FdMqsZ63uqC)_`%`10D$L|xXQ|psXo~{V7!EuJzw&++Z8;juxIQ{NJK>4e@ zI9bfb?+H))ukqO08m}R!g{5t5h%KmZx#ro@GT45I{tM2Zy?$+kn)6b{*1e79Mi)YH z$+00F+~1G0wvj+rX?@Ii%`KC8+vP?FD+hzwW4)oey85&{Jz@3R!vXbz_ne|Uuc!xg z`sFC$x5V*$ji0cxl(F6^Uu>c+z7zGFfe6GXluCT3#6mWt9#S?jri{+x@xeKy;SQcy zCmkm<A0se={{9Fz~A zcTIa#q5TN@{qJ_vWzR!}>6F1$pbxqZ-7&D!&n_yT zFJjYijvOw!{Gg7f7z?@h(A)^m%U^LrL2(v(lLreSeXB;IrMwtL11 zFK6-R^~1xZb{luce+8eP63O4aw!sCxA1RU4)HQ^nW|y}7R=@I%+^l#$c#F6SvpE# z(J=fA8vySxjKACZ@4!RYnnJ^zzuT%dHW*s?p#;&I8L-}ecP||RILi|qHi8b>@19F3 z8e6U}_?%m>C9i=6Wr`Zzf1XbfV&siTV>V9ZPhHbJjg9sly^*uA9H}XcbV!UlRU@sE zB(h@|tHUR1!>4HDChEcozhI~77LaL-6l>@tRdu4Bkf>Ja=$z8)x};uo{zzS0?tEsz zV<4QvLD^nhT3A_A*qh53%WpMKSGgaIvMz{&kL9AgbK#FrDJ2LQ3oyLs6%`7V=#aFnCzn`S?F zF?3CUCQd21x+cd~Qo+afXnfA+<-vZtp{y0kx}*I^EI~SEC+`kkFvLiBKtgkG!!JcY zpWQjiTwkN(#_jiur&Nk!QW$rDSVnoy!QCrJps;C{E9eE%K?CHOD+j8@>(Z_r!(bJJ z%6rjGgk8%oR?w`U=_Emt;OS|`FLoPB=G*Jz_1>Nl66$bx+@Y*_IxMZ8N+Q?nN`)~R zb$mr$lEhe$H+SSVGIHy$*jT@}+dQr-$j?5nw=)mtn_Xwe`Z;_ak%wMP?I{Jw%%+XQnclijQL3DN1XHxiEIoiRNV%eBrXl^#$TpSJfKj0f=6r&;E2VYH7#8_zt za6VJcBt#ExNj7xNtySJN@3Pqr2=nrgoC*@At_1<|rwpB2kT$JF@_0HhdG(nmlYkc0 zSTwfISC+!gMyH31O(ls|HqIKY9-L~6$NIt~N`-!L_<%5Y>_(TV(f4Y7%7>}pFWGzv zD(gyJF6Q6NPp`kV)V=#GWiDAi;Pc!)?GBmGWRIFaA?HW-KSLLB2>8nxp-Xy9(1M>6 zHp6RzLO8Gpk24Ls?R!S2PfnghpNYLB-AXR2apqK2g}Gsw37^5nFY`3C z{r#R1IaS*}o@h`}WAj});$F@AY|>LSFTw z&GzxP-zSNQ1%hCYQ@|G_bk!q>{#FGmFQChHG`vav|SX2LWnz$z>X|560`cPw*AF z^J%q2V+Wb&5icmg=Q(fM0loL;#2RNkOK`Q}sqgKSH;JLs;{<-rhw z2sL7Gj+?jHQKr*8{5*?IF`*`V*#@}>df5bWq|VOL`psma>{M{8+sqc*IcCn0Yo!W$ z6I-C@h}TF-)mU8g(!I3sWE1IU?VH7UpWvgz)Tn$1H)MY4tnjD(=m94Ncx@2<1*4P0k7Ll@_Y70y%nfSD9IUOqg`{VANsrT(V<=)xz8#al7fjAHxTa zdU&=nKCJjaz=z3BM^VHzALwP`KvO&q6ke8tFBGyXf|7)f{0S`R42he>{7c1#wRuUr z`h^V3LfSCo%(q@S3ygV(iDNf&4({WE>y^2QOC&!uRmuVS7qD5>`y~} zwlh-^eviMmx7v5Fy6o)*HS}{b6R9}D*1Smq2x=iapJL~$6FSxf-@L!-O=fc&Z~r|S zPgO*6X>Kd*%iqt3QSvVx9yLz1k1p?Nc2*JlW+YZcUdB|=So(A%>>lt%a?Hi8*==l? z!Z^v`T9}L-qa|c1}rD#5G$DdJEPhIHg8@JZOGEnT3}Z z?L|m3&u^<>1~CT9V=Io}nR-#g$%*AVrOLL2bO+qsk&UfBz(D5mTHyOh1A@ty>c_4t z2_l0($KMCP$6t(o(ytXgU8k8HC~5&^->@%s&^M`;<4zS)5_Bh@@XTyz5NSPz zJ|dxP8PG177@t177&w99TgD;8{RtZTIfQ=xdi&|Z?YEvwi}p)u?eF)?ce~CfGaH)s z-JS;#+UvVU(`k8m!RNWSPjH`RPySqPgPxv8PaR(HyYKY40jf52+ge_S2dOizfOKxB z?)&O<9N=vxacg_rHy#xQpO9Qji;j(iqd4Rkru-p&DGpo%1BXy!imxbx9-`f%cy*^%M7%|sYp;f7zoNMF)PL6a%#q% z`}}JXJ1H!W^ZF{GsC@=@JJE|xn-wbcu=Pi|$!R%UXwR)@MoG=-pAe0-XuQ9dZf#3y zKT8k>JmBQ#xCAhMo4U6B7@Jk8EQx2mcD?XBRdCtmp)MBdAQt0N_!%xKnW&5 zWEGY_=f6qxQqtW%Kx}if{r+|JnA!UTr z=IPZE69|LJzyqZMrNrng0r7;rE#ADlk{|nLinpkDJ7dl==FM1Sa}T4@g zS!>&Uwty~m3PyU;FtBH~PEHJ>Sx9a0als zm4w-?O{fxL5s`idCMQRxMlMifr-&?{VRPoF{8A#~i|_e8S74Xr7B=wEtfTrzAy);% z*V(IDtK#CKP2ZdxToAsv|BfTbNy&*ZmQkuv!V&f=$)%)H+`JW!8o`V6$XOKk50tSw zL#y16Yh4-Av*B_ zkhF7{K6nKi6P5%!$EadT);28QXCTuaGO;6y5Yi9p_%>KkP>%`nv}iuXY`5?slmcH| zRx|kEVH6EOlj?!FlRlOl3^SP474GgzetM!qON!^gW>bABVJ;_iNoyIuVRJUvwZ)b6 z_`uzfv2|KjrgK%(aZ?tLo~|J|unYoi zKDSGAon=D2M1uRPci-RNkCz{|R=?*Ui(nT(@=_+rpv9W;n=OO$%% zl|od+^&FpJt^_t%;_N|i1wO7du-qMKeXWA(NTc&}cSzDO7@ymSteY(Fb%_;IK0W-N z^AL)rabr1Ri9P$eAQ0J0-~cx6#Anj!FR4iX`-;z@-8d02ggNEay2b(E6J7aQr(cW=I%JQnaI3^dsOQ3}&zJVH zNgOYUqQADw1{uxOs%Y_zFVp(x3`S$Pr`ya9UI%=UK|qKxq8N$^xv55L7{h06IPbT; z0YSM_c&>`5S~_qf7zZzvU~*EKIwm{|FT0(uP`ZhC5w1V}PWqWgXSD+LSF;SVl?$1` zxGkG<0~a$31A%$BIXre?m3vfVq>PM=Ll+JrS7^pMruPrS%lKz%d>z_5>#Fk1vJTgE6kp$+B!wp!xk3ygMq)bB?+>4u-LH#;{+04g2^T6s%0egkX8XBF>M_ORUt)neUBED9^dj z$F!5F<_b{rv~-7Pvx#;R$1`Y)NjOL+NPdM)A-KnJ^vgdYGUAL|p(2F}8Rw8yl#@K| zj$QG>S>UN=?jx92N?lnFZ6ZkKYNwYFS;D4HB@8FG*G0_%iQ+Go7T{?1AGSYn+pAZf zLmvpi%jVNMwXrjT+@Vb-w)yq%m&;`2DV&Jxn;pLA$nvqO= z4ahXoXv!NCdRyCixy`RmYG-4|(hcXRaKWI3Wf+Uf$$d4J#Qz~aE;Aj~2$XL*#Ky?N z;pA)Z4OaFnHjcRFZb?atZ%`GO+8;j&Y#fzqhEeWlt{<3GKQOtA?RrCHGXxK%sO(f{ zXM$%`c8$a+UyO&}Irn!w zQXX3C*4PZf!fEi?7uY(9Mj@x!>+K=(l`i=c9SAvU5 zs+{=X6uEZtiJ{*I%0teOHCNurcnaiBI0XsYlT_9Mq*zz)Z{dh*#*F|SPFasdeLw|%NYzyhMKW0BMQ0Izw?WA zu@yYW;!}evgWIIm4hfRHs?g{j~rC z(jIz4v&rMuY*Cx;$K&O3t-VF?HnPN;R+D+p-Q;8xr7p}R;!UqOx<@-G6^?>zKO$*QWEn6vJNX}%;xX2#66C=nhm0*JG)WFUynQhs3 z5tJ`FKb4W!tt7WE3cw)8ng(V3<=I1YbDEMLvB(e6tQQ*6F95!Q#oAh9mF>^fnjEY(T>78p8S8 zh8&S5N!e$0_w5;f%@Xk8ks`A@OnFEV@9sTW;?(!seFr>r>(IE`OAi7E7T>LU~sK;|e;w(fy zo`b$yAP1 zh=Q|s#ZV3=CTK)UOe`_VOxQuC$Y_w1*&Kr#W7aLJ)#`b$N^JO$7Gnr*a50*Z_!NCH zD0hhJ+X)d5U4@}rn1<0K5a(qlGqx7YpLgoM>`M!++)s+zQq=~1pLC`+SX#fZ1+ZKSo# zQ6qy8hO9wJFb^bqiD<|>1c706bn=4JgvimBE*l1tOhArMd2yfKfV z+o=E0D0k9|9^$KdS=t~=w|hV>f-Xy9*|t|gtPq^0Q?+DDTvB@fN&984Q9db5&q_BP zOv$aZKnN~wX7+1g*3NgSc3-2Cuo>rJEI|TNgGWG6RPsjXrhY+8q%5SVdUg*KBS{KL zJ36n(qKMd=WZV088k^OeY;0^G3E6xur>1)H$B$9!;rfk8$N9<~9A+oGU=)mHx`vxn zw1{VgN3n=&WyHF@G%jsX1(D_^2qbs(J(;dFyv7kbT$VYRm-d~UK@URGjuSEJG5gFI zHWtb4`=2Kj=t(P8JhHo3eC7Bf4HJ7ZoQ_3|cCRDJ=%yo~M6cLrwh;kAn2YXYeD1iT zH=3IHQhDZ*k5X-vdVyQSt*=4O0fAe}sP3fZu-8RDOe+J?-i0`}x$t=GxU~O1cjzJ{ z<0I^8nj5*HOBFXHV}>A{UqibRWOIQNL`<^-Oi|2`TiLv-G2Jv_T}w%8rFCe zOoi)s4a3XnA%xBIlLo3fBg{~}JK1Lu%G9us1yYfynZZE%C37H_qnH36F~ux-BTt%L zTt^S*ir>_Z+Ok-44ld@r3m4oKr2mpIchw;lDbiQbABLqpK5pl|9fa@DI0ffY+&YR@ z>C#Swyw397nIH_Gn9mIa_!p)8cmG%+=j>zglRp#3%n zz1M}FAYcuPdR&~=ny^5|&(1c&hCGSUomr)aZ1eUOudm}O1`I6DO!`YXnfld*r?UdZOmiDJ%23?DYY3?+3)g z%M@}#lVH4$VN5(gD~{~;#2n{tgA4l22$1nYkKm{u4AU>#R)VfItVW~+DMKN5her!h|X3-x<^a&hXW?XmJE5=5g>IYM$LuWW+VIb0h`6po}JcR!1QWwSKYL2sq; zHzZsi#a|`H2K;V08&s*GOtU*0H_B?2IFnLT^o|{ci07g^WHK5b5WeF2REM?t3nY8*Y^Z#1_4j#xA9Agz#mgDQz{yhseMNq(xM*ZA6t_%D1xyM1`2zp z4u4@04f%9bb-HX%zhtY1JQ1)*J(sk!_%4>Jx3sQ(iD^DTuC1M6R!MqRs9w8{`{5~T z?W+PYzrz5V%hPE5SlS(K92xMn>;#7HOLiAD43+q|H~4blC6L2o$p_RyZUdk}~EX zC-+;&zZP>Q5td~KBFP!rUC ziB71V=zmZhgbIl*p;Vw?2pon>=88n<1tUnqHx*D2SBE%T40FxJEzq`Nzyv4=0Sb1Y zv6y%dF9^yoeFur z&!z(mK18sg$=%(2l%x^nF=OS|l;J0|;bQmI3g85AF(-4FecB67B>sse6o_8#yO9B< zx~SjbokZX@#1M!Qft=yOi41A-;yp*(2t2k3{o z5EVqa7*0sB;QX%R1|iiqg8#G59WcN+bwMJ6{@><#C+;sz)&>LU+ZX_X1OM~UhGKwv z%wd8T@b5v1j|2aEganZ$^7zyI_HRXv5d4RrzBG&&hQA}DVZIJ6^1j!#O`T}cz zWKjI{SlEO^XZ>9Hp%Jk+VA}bN^!WAhH<&>oO2-+ZXhzVQl>5aH;DsormVOOLBFZd` z#Bpdobx<)xMH)ICk*la}Qk|F^2C;NiD*iOpl}o-pVSYOOlc$m#c@KU2JGy9I?tK~Z z^QraK?xMZl`#E-;uh(ea7EpRF3jz&VKhGx_;_?<)h(?Y z)JasvX09T(>RE-&IaLbd=$vj*ZsM!9zqJp>&gP0z*1px2*||WFy0Er6HI z{5x@%Ju}jALHk!{ms8|LW=ZvC894cGSvQ*Z=Nht1P>-*l*?IL>uf9>qhs}J7|HV&q z6f9CIN3vNl(ptF&5kE5geRx?LSGu}JJrq%14okaw@k>`ys{K$WPizu8EO&;CtdLfn z{oIxQNo{QUOKwu;xa~LRT)DR-mBWL=@*-JAfd&R8xt}xf={&rc(0Q%uCVPlBUcK!v4>2u%r)_q+X6*~R8 zidMOhGQos|<``&xrTi7OrKW~F$U6WPE#-?7#NL=G#Fk*b_kG}}M$@X9-nM1a`+}6K zxa_2owJu$Sl}DgVKm$)InBwK{YXue%6GoJ*YJz%fb}q8X#nzQ#vw8yloSvLtmXdF} zz(|+^IyMZ#6WzrYYy3*UL*2z;aG`1!NQge5i^bfcn*{0#;v}q=W&eiDt6=x|1n$YI zrKBcd0ed{TQ{=%-Nq|&bR6tv}Odyy;G`(7syij&eYGtvMStkQ-J$m^ry>5z_!cU2? zd{l!}Mfhb-Ba5Iud?G5YZS6z^_gn*rhNIAJ5FYc-t|g zi3X(Hv#v#?mtPt%q|=qNztJkLePP5xgb zC5ng9tciS79**NCyp^?3*pQ?7oFrvTGiueMeGr5+T0CA6j01K^$TQ7oSl$9FI%)hi zF!qXyqk=*qHEh+OtIm z7|o)K-~FGTot|s_Ai8pt63&}|aEn)+pW|%pDfa3w$Bm}UB}P_0TsHMVLoqmP=ixgu zRF-0dEZdKW#ftnonfIl=+d&>NAOIdTl&~9zB-%6OfQu4Kz`QGrn`7oAV zcA3N7gT`#ttD)&Inpc3%+tE>pXl??s5Y68|a&&aw*!F%d6 zPf5wkyBLArX)0WjFOs>5t06J5F-7j~$93UN1U1%E%!AJcmY#Zwntp_4rYn0_I~N$9 zhu9mywPVN=w%TaN_y1O($@qAXRz9FYq1ZcAm90QRi39SefpK5V%_%`1h^WQFZb!ET zw1V69`<%I*BOTzs(c&XZiUstc`3`9wGu3iBc`_DIB+?dks=E$>B6!7R=T7OyJYZl+yfuF8gUhkIWA{j8M;CEd+RG;0UB{&XMQAT}uP1`W zL5~@R=oSQO?zs_DJ}stXB5->l^6D|u!gz3a*eOebg0hi7K*{1xM~M^kfNB-kK~iZh zg@ZcvKW_*%K;;8!K@vHpJD8`NHs*+A{WIJ`cuu6-a)Ow<9akV6SC*uwf{{Ps&i9L} zDa)UBxL}h779%BQ0ffNk>nSE6;O}=U?_tVb114{ zi8kPcX^mxtMeaL(LhgUsL)tddil{ji4TTT*+0=z(ln|UN{r!04m&5-s!VKXzXCcAJ>a@u_f=!NM>hl;*lq;h9emZQ101lT#qX6_Nr}nOTaf(%?P_AO9N$ z``ZJEevJE-Tm8Q$HDH2pK!B}nX1^+cN06JnD*jGUz^NgR_@$@d-3MZ42xXZ-fB6`h zxUE)yQN`gwhLG{{2S@$pth&O~))UTT<6{(K=ATCDCTS_oJFdPmN`KvW9Vw83vg#fi z;!u6wQt08V>#(dZ^_=uY2a>W!7Z-XrziAPqXXlM=ZBOJB;zX^c@9`t38xoX%)b?|b zskWgqmj#o1pj~$QV%v$cxU#T#zdELocBr$Xulz%kDkn|mrppS?AaK%2aF_k2+#V7v z1jh^{F2Q0>3!ef?O~;8up9fP@Q}cSAbAVo|q{91S_R``GaV_J%hmo@V{e8#xGUCs| z5kRftqv-Dny!*KFAiz1J_knYUMuTmLh~$R=)e2k@k`Budp#>I`cwrL2>mVQ?0R2q> z#yp;xHcAyVG+b-`A}Cf(a^M8=r_#@wX2z8RVPDYM z+1VdsX7plpr3wH}m1^|k;*t__GBT{G$ZqsFwz|?pq z+9(e=vYXlCbAOAw*$JWqP{J{vpDw!gPzoZUI&iQP1Dn}oiCJF^GGY`xg1Ul`o;aMH zop;*09ACo97I%u`>Wqwxm_~6<4=@u^!AJ}seaK$KSR$N*#6op`mUvXv)*{xJjK>p8 zOG^jfj}eDncGf4VHgb_kG)_RDWFcs)9!p9efei1BCy)0Wb$w(u^h7T3yV{J z@}YWm1lZ?~e$-}ar}Cg{lBZYg{4u`hrXAJ*l(8VertEns~gSHC&dJ1`=$#wT>7zC0v`pZk4e04hl}S7a(`?#t|_MY`W|9P?~w2} z!XL<(mC5Gwh#d3)13*UpZ}~BK4P3rB0e6i*KD`LgimL8592_-Pzzy4W*opo>!|F%= z$R)u@DF^>$Ktk?7oWs(6k|p~gCHlW+MGP1=8-#>H@V^qqfU6JHQTHXzwA6o%f{K2` zR4~!mF#opp-4Fl+t}5*r=>I($N(97I$XMJc|25VRAQ*%LQpa8O?XTMZ84bu6&VRiZ z=ls#POY!t=-=}I40rjiwpHXv8+~i^(maIJ8!>VE*GG@wCSO6xVg{CESH4a)$GXjhQ zM6Wv@Jm)btqW6n127|h??vVord={JCuV6=8RoriHI5m3i_gBC9cAwO6=4I+1DPzwT zrrI_{X>~h1>DX64Z;Jd%rX5~X{zMQlm>_}x+!y#_`AlbRqeqeokIi2S4W8QM7_T~R z-kce27^I8CYKM2}UrnVWL2>VZ4iEP#2A0S~+taG}!v-aLvPhQZ8;(!v+N6MfN}YD+ z-pZKbmcz{Hv)1oNxG3bem(wVQrEtq_h>=Col>8}rq$4-DfW z+zzd}vEq>vowgj$o0hE(sl0rKIw$&&CswiNUu98;R9PIfIvly_TFsUvH#;;LEjNvF zGBom+xlqJ}dGE!Ik2NEz2x9(8EVv)JbtTGv=S6DFyz04)x6t%@;@i2KgDf_D06yUh2av9WWy zwGA)bYS<vl=(SaZ{y*y@4?#mtXsIrS$0?Zw!XKdc~@Qh`Hv5hK{RA zu4blPA~Ma`1CIaOIcuSM5nPxS=wB-ohq(Q%Xfo=Iv7-O(%)Kxic1-K^*s`& zSA%{kA%_9eqw))T$%WINucDT8TT|tA&>H+)cYo_tr6!#^UYQ=kbZ=l&?lZ2tRR##WYY01e)Hu1Rm0D}!? z0+W}Qr^5f=LL|)9<36?>UK9!p+J7-;=j@*%W2O|6!y(mwapR+&9Lx*607E^H4p0B~ z^j9E&H4*raD*qoI`J=$NeYk%wxSbo;|An4^#7*T$_(5Wd{tu4;u)6>d3H|}2|Cjmy zrE?#0yXpTF=0C*1{|hQa8y~UC1-<{&hp_X1Z_OY2BUS-ni~WD0U;9T;v7rN!miKQ+ zL;A2ZPz$_}e+>pWNCN2B-Uh-T_-{xf_<%ISj)_vP^{_5U;BnA)YV{RPCwSB!w44i`eWnj%5ftjE$zrD`5r7- zf0Cf_K*Rm*{iSwQi$wBS2OAKzxE+g!xRJN_9~c=Vddy;1??iW2KI#&%)%I=lg(xKi2X$X_MZy$!*EK$=0~Qa1q8!CF2u1}?f6L4)YVm+Gp&r=y&mwD7794I zR}+mq|C&tq!mSB<+I{Ax-5;wkSvqZQff!cV%B{pn<0%^k=C4=Qe<63tB!J^3gzXP0g@1)WbTTqoQue` z2T>2AtH+RssLb>_ee9`Ad{!xIai)tV8$>p|bdOxr2a(5)XZ7uFbgRP{)RWHVq6!xk z4^%+Taz`}%rkUECsim$hIrod#^kO5M-TKdo;gu-(E}eIf|86EoPx?S@#F;fHY=on< z20vnQ3^0YIX^xfP#6UzalsR5+YqQGDtt5W2$@mq#s;aVo=TJnvytjO{5A#gk0*;+S zT1ISLNX=5xYdwvT zO7^-h)5@RIy!1y<#kTIE9+Y@K!*5G4$V$4zrqrx;xoIK(GN6X_HF z1`8etP>%lv@mgNF4ix_s^e4b4aMd*Kyd=e`ISEh2I7x)e(&mKS18x6%CVw8)ZocAl zXUZ&{6}rl_n_UH)P1Ox@)n`RSp8}lr%ImcXrXNmaV6lX>!+5GR6Sdy`ztnoOQt0)* zM0`CX2s6Bcr7=DvMfu%$&lsi4z#vYDZh+Vi1HlVcz{?;#YJt0K!hYS3epZw6i<1@r zn7_)bY}!GnG7pkWqg%UKP;Gb?_ZH{f59&*K%F%&0*sS{P93n1My$(AEqB;n_j8z#> zL)1oR8k7~XNK0epqzQhXVN0bC%r;I*5#lYAc{V5BD^~kqK_eFBPL`Ve=Id0htq1}g z*=7X;l_X*tBZm)f{-$hRvg=5_6B7dothJ?!J(rlNpRf0_thkXIHzRCUIY;smM_9GC zG>9}-#%9>Fg4241aJO9+txFrPzw1&^AJp`4Xie&SechPcSHk-3zG}LdRfM-Pk*Tom z^DM~r$P2%qZ+{$M^NiOMeUTU_e4HDZlR+ z9RICGlww(bk$RR3CBHw`M}A@Qk-vog0t(k=dYTorJ_pUn7G?kuh!%j1n`UttT|_$V z{`J!js7aM9!T>6ZP+y>RXumC>kwDP5yS;rS2%@X2OGih?*x0zJun`kxCZ)7&M zSUUo1;EM;TvG@Yr#pgNOulG4DC_b+>5BjWMA*LfX`b;mk=uJ&uNXPU^jMsC)3G2|G z)Xq=kC4w_pOuY}@TC7d>bYfd9_HCwpi?h^N)F}#Ih8mbo^l*^ZDc_$F6NO9Z$B4u; z;?;aJxL#4wji$`58$DlZ7PSB1izYB)vr?@7aV6xi)lhFyN_`HL%Mq+qkEwu5Vt-B4 zo2VM|Gk!}S4luzjr*n0X&)9?ceYT92S1884zZCdbW9lFwAe!IQgl1_hH89r%)S+f> zkZP~Z^I1aY+tr#b2;WhS6Yb6xLL~{(_^f~7O~FBUGrzTuB=E&=ek6hkagWy^=Kr`S zVSoQ!Nt>f7yjRIr2w{ZA@Mxhck`F+Ue0{8B2J#ys%jo0YNq<1&4R9R>AQ0}ayQP4f z1(;!!5Y?mD;Mb7n)8k{M9X`JZzv~*@=mOG=TUX1X4p2vHNtidp#!E@6;GAR8P7Sc0ERh49OfZ3=<2j z;~=%#pDC5ZMvW1K!UPV4M+PAgE{8%{x7`UzV}$mz`|Z&}7Nk3;*JFG~!2dvmV~sT& zH&#`_@u9j%WEDID-a3Fu62uFOna)72{}@5j9Ceo!4~@VNJ}eo{1VRJ^^gUXT0+gm* zn#>uV`G(v5iU@?9#uC*;3m$N&Ffd@vVJKRDZ%@A;Z60?8v zbSt^LMpuuv57$o{9p8YalmYC0cL#)kxD67IS6G9XZ_CmC9>_4`kH(36(AS3hLP}k1 zU^HQok&ys%*mrEzQn`q@I8NAJIw&A|c*Fsvf@DTlVsNlnZL{C5P%UF&Vfpmw)6LC| zimGbE;$4U$S)5>Rt(HO_3X>q1FRtHXz2iPE`1y~}5i1~r&B(}jL=b#ls?xaJ>Up_4 zUU0rc68uOL*I*26LVgu=cH+3r`Q6P)8pK;#T54%&VIiZU0WWI|Bl6Dm7-qRsX0(t4 zF>*OD(NvIa6p6_R&hMJW#-wZsEiElTslXwjxF;jicaY4X2@5#w4jjbMxWrk(KqBf7 zdQBBGb8}$fc3*KjL5jf7TRA10f)v>q=F$9>xr~D4NSZ8W-?LUBXx%krfRc4XFVa*c-MV;4Ff~P9(n#;eMhV2kg{Lf=FqjL6D-`oBbQF z;VKrWJ`rFoVbaC=(#4FKFl7_}4qV?+m?$IQFL142^`o*${~c#Wg8+f*#Pt7`)BaiU z?Y9MdQzrtpd}GwqW7)6r^%EE;#`GSucHkRFQnS|YieO@&!_I#Tscp_2@Y-*8!LRA#?ZQFI{=?zKkxOV4=q1CYV&|t5aDvsb5M%l$KO!_ zDF#wB`4B8PG!__t{vc5#wQPctu8OX;-L~cMT#l-s7WuT2qTZT8hROQs22ZGr#B{BA z_Q1iNp4>Q8=%sI}rMbilD}zl$zB^LY5%4pmBPD-0!H&M?`~Sw7X?5iocm(*v9IK5u zVdqw~b$&Xf3Y==?2{zUrd>h(m#)fxzK;nU$e3Du_MTjXr)X z&kZUaHa9CiP_9pWjc8zY>^L4$MPpS(W!*u|r_21CmO-;@-a?O71OO3^j8w z`4I#HJwAY#SZwYl0bx!eauP<(@GuQEkzd6ece0M0iHMPN=N$Jp&kqRY20KqJjkQ~v zE^6qSEWC3TZ+6vM=cT8o$9EZOqWs?L@K%hH)>JwFEu6db66FdU*wTwO;q}E^|2A7x zipU>vV{`Lwn47Qr)%Vob;08!XG`PaClwo}yK1V0zz2fE-?`K0OY}A(N5EyERJyO+X zg2=5%0bY__mAmc0qwA;Z{n#%TkEhT`fE(;+QKu~i*&Ad6%O&Z@wdNC#-4DSA!-y)W zs5ma8t&O23w2H8omx@w0qCb}b|2j}eQaHa42NQQtPPb+Ob_w+Evmub3o%C-7G7UvX zO~E{V1}(?R;;Cw;Tc?}GPo;QnS}pW6S#HZaZupZwLm_5DwV7axkdsgop&Rl`upQ5 zARtXZKq-QBkfsy`rAU@iIxES{nmJ|8nK{?K&NX{SxH-|#6^zqb z>m95)Y}AQs6fOk_waW9vWLc3g2IR3znyIO%H~wLK7)wS=%`SU$@OhinU8mCE36`13 zLbQ|+^1xfg6w$s%D5b&?u#*~OXJ@CZqVg2nWm=eU>Gb#0Dlt|upUaaA6pDv>U|{fx z&fULRK5`!pcGgIv{ZtTNTBE&bt7uY5>$;ZfQ!*wp(kXboW~N4Wm!L11IwL&Po9Nl( z38Q<i;(jem-i462IPG;-|&JJJE++D2y#)A2%>UXkGBfp zSU^^#@}yCb>uUWfUi_V^@oS$}=((Qa(~a3Y<0D@FWVI;vQ<9ljmQM?CH(-+ehP5wr zmF)@%oRrug+g=BzGMlR>XDO8$(eS2oIGH#%dV|>mAF8$J`1oK)cUWy%^9E1id|t2T zAKb#Zg2y2N{JhL@h#Ls2dzF?&WbpyNb0U6T+_}7(z|0a=0w1BCN z$iA&+;yb&IpXhMm<%Tm}C9QDH{9fzA%1L1I? z?vnakBvW&0QN%$<+MXHN#=}x8L;1x9m>};E{$+tABDj)_pVR8EzP2WIPY z`{~zrXWxfEVg%p*S>vq|%Va$ae)KE9D6S4K^mjKylx8|Z|K>GSb|E#1PYh|#aO^e} zj@>p~F|}3(qi}WGj;;1PGt~aABwgg!yGIpt_gPRk6OlN;Lb|ecBof*p`%Rc7FUs*(Bw?l8FSa>e01SflacPP>L%^#dH(L(*(VTWl`XWRA>2Ka~ zSs!-Kox{scX6LD3IUkKY;g_@(2n%#C9(KrB1QgS;NV3fIO);yyKN-y>z(35Vrch-! znzPe}@mfgf*;?2hY(6(vIdg%Jx)}y8I8!%zYVMlXJ*(EKYPB=!#!!7}x6aDH>%X7! zD(AwrMQ#sph)ERmXMU{YY((d&ZU=a~?bpD^a)$FG42?Rk(a^K@rrKXtT*G^;iJkCo z6qGc8K?+zp;5-}&$oD&266kiH$c~%^ItKle*dT(7pB07l)qCy!-@~Hfy_u5tF9JV*l)u~fE}}`KvrXXCWh50-<$AT}EwRyIe14mPltBgO zBzm(HQ*$6=#DbD>V+>Tx^I9Io30Lg0S63b6ZTn_~KyHvl;<)911h!*?PRom5aWd=0 z@{_kPk){Xsj(>1E>LBRQNpn0_QWYj}@}*ie4riO&saP7hRpqaWD`c5jrp8~6hD9ad zDYx^9MSTD`V2GMp-1x}r`GXICxojX7rE1}5#6uibpf)1(sHm6dRgwNF{c6l(>+%5z zgjv^sC^a}WTmJX+@R$;RFA3%Hd)KT+()RLB{+nM2CBA+QR5S?{-5sL#Jvktx%Jykw zhr}3IL#@C20j^uHG(xPPVyTG!7+5X9WO&;huRXJKCcdFP!i5!+-VhvFd28oKp~}1;x(h9V)QSZ);3=lC(?`_tHyI)W6elmOMaH+#}UVJ zOAv55`2ck{r7BruA7B!wv5AaKwTm37Ky@Z3CjmwG`&$o4Pv7@QOG)8T2in`)6M#(( zlT%W<+GaHdCH!p_h?TPtXI-I`F?GGWMC}TKyZBTA4y!Wb%M z|1@eiTBAGKcevwdn(#KFwucu{JEeK1=Wse3yZR zg$1zjNl8g7>lTTeO$deBE!TqsZ*p^AX%{}jUT48nSd1j2m}`fhOA`lZ>*~@JKgb@Y ze5GM~F7CQ#-T51fHw(KbVMLbyBA;L>SQDRWemu7ey4IL3;Yt&3Z;#xF|8t1nT4B`tX)rQOh{KuXlt80^=WM z|C-_GC3<=S5m?%)Gw|-(co`xx^+zWGrYgDyQUI|TCtCR75V`7BH$jzRS4f+|TsAN5 z$OJ^v6yzR4XB2L=@NtVtn#6+k=w%l}NECAw2=5F%m5*c}W4$ROjM@|!;F%5m=)Jo* zJUwkR(BZ)#*@ z6tNXsb`3_$N#4Z=_8!g%CkKK37k&4$jKDMsnw(`<@~Ez977l|m5iHlQ(&&Ix)RTP* z5A|#^{K)IO4uUtw);MeG&)90TWm`5a?hMWu$f{M3P591zYRnD4vk-c^+0f8$l=A5b)k|`rde+g#jA}67c1s0)qUlzT=%nZA|YV?~K zl@OAHu#R$dasow$F|cl32PX>M#)FCpAD6^F6m;Xt8&$}+opqPJfcB^^&~CpTN6hLf z2jE31w(3VCBb|SLibLq3^r8<}(lRX(zMEqk7U&hzx*#7-{zl7*v=3Z1hv52==x^Bx z6ei|_;nEJMt9et)l9Bmq7etvo3AY^2H&9%*FYJT;-lE3V3mkqqusAk2`W1XFpmJ29 zs866bKih=x;FumTh?!LZ0`SEz?uF{I-oAe87S&KFP6VBBlT3%U&2g7x7;Ha$H}l%}dAwaP^43oqI-ZF`+Js?Eve z2IDW^#AOv1AAqZVHWOk{ykRaQ`CNiLtW3muu?doQ>DKqwhc`E1Yt;&`q0yNKGlIsN zkEi7oS`rBUIOh4;>G((j2AeaX5b9m>RT)~drRBcl)vRfQA zL(0WUPaXByf?Yxf5vu~h3b+jTGV?A_?mrXApfqs;mO;XZK+IgK-9$vBafJ898YUL; zr5}3Qu0kjkv>~cipGQXUXi1-Jo~-0*|J0RR?d@jl0M-<3Xgm@XoHE|9&Ywl}q)z=V z^Bt&?iC9&j-y`@yD*{q@ig#~ndYlv;CKze+!*Zu@aPY~;+#1S6EH}W{_2a#vI38s@xGg0f{e-Syjk{bN|8hwIg!nCjd>HgZlfJP}-fG0YZ7&i7u zrC;HIq(417n|u?>enhZ{1&hu2R4$WL5cNK}SXqoURu)$Z1 zEh`R@KUrf-l^XcU!`-6EiKt_sU5tD^VJj$t8m}lu`K?Trhg|~OD)0< zUrWEDkB(rj+le|w!8(fg8L6pu{C8koqR>bfhiXpZbPyp&q`1cw39ldI?5!Fr4WXK# zdz)9Oy&8y8qb>p6-+6k4Bw-uM2i2nIU6E}qD=!bBq|GL7LBOA)C>+blQXybI=B%2x z;cl)wEF||7snt}nk3-p2B?8&}Y3$rK)IlkKZcT_WRDQYtcgnx~VWtm0PrK3KT3u5^ zL!KmNJI)@cqZFbx_D4Ikc+D8E@dK-_3V;~kdAlfTFCA4&HB3rgA*`(mJyyIebWq77 z!FUN0|2A0=0)H%+z(Y&bylF*1F-_Tg7`$_wqM$gW15NCL21A>mF;n0~I3sm(0p1AV z+x?Ed*B9Ouf|G2-|DSYln8cwGh)q$GCzU6@Tu6M6V3MRnZbU_Dyq7A=Q858Q8(;Wm zbB7l1Sr@@mlvLaE%ALY+!pH~F4Y|^I_WvaT%;CzLV^rsi7{=ZR5qbBJVk=_7BP6VU zKz^U+wkqrs1r0iXU1u)@@vYOo75_M%W<-E;g*um4dQoCjg-ud#Of_UsCarC-Ci+OU#~$w{#3 z?9hbF>cwG6N2J(Zh6xlYM&+wflOB+LQ$iLt=sC*2cwHV=jzH*UX>$=GDe_Vjxr(2J zlqtCoH}_Ohk&B2qV_ZXBcD4@X_qO7CiR~Vv(XzckcWTk-`6b`L?UtMixou$>95hM? z|E<1}K+(td2o{oPos`o=AHZOhY8gaRrq)NQU}%3}?B4#76(T z`N{NlDj}+)fzS@E`^vS(WKfu$%$X1Ja`MAu+XJ-Ov$N;IhY2BZHSVNJ{hTwr!86i^5mf4EEjn5 zFRtOn58lfw6@edIQc$-txKL}=>nYQfvHO7;Ps}^fE)p>Wys7KsGd)r2VCql0_;_=? zM4Qw5lQq3GSXJhq>T00C*BxIst}t#bk}LNd@i=~(wHC>iwI5>RBdUPBm=gvs9-Zx4 zI3BAWRT2h)vB%<6v_eT8?`=(U!qiWV_S10#&-CV`Inj=M%RFVqgEvOFJ*54oDzH{9O) zY>9HNp;jW5;TW23+Ef41DV*=>(0OBEf-pQhJTo(MvdWCL_*wcU*5XFP|H0 z35y~j^hvrPEa6 z-O=RtWsN)U%LCHX#(D?TZ+=PTZzet?zmvXjgH|X4Ppba-@nBj3iRCM5FB+(~Bo8FX z3j%c|;w@wfg$4#;iAh(X2_}I;hHs3HPPW@#m{0hY?L0IiP@}WfvZ&GjF~-QJMs