This patch adds support for trust chains in the netscaler.
I initially planned on using the 10.1 API's "bundle" feature but during
my testing I found that was not working. So I am doing the chain linking
myself. Also NS can have only one entity of a certificate ie lets say
two different users try to add the same certificate on the netscaler
only one of them will go through. The other one says resouce already
exists even though they have different files.
This can be a problem in trust chains where the chain can be shared
between multiple accounts/certificates. So, I am using the figerprint as
an identifier of a certificate and making sure that we delete it only
when no one references it.
Resource limit shouldnt be counted for resources with display flag = 0. Adding functions to resourcelimitmanager and doing it for the volumes at the moment.
Added a field to the command for serviceOfferingId and changed the internal interface to accept the service offering as parameter
Applied automated code cleanup
This patch adds a network plugin to support Palo Alto Networks firewall (their appliance and their VM series firewall).
More information in the FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Palo+Alto+Firewall+Integration
Features supported are:
- List/Add/Delete Palo Alto service provider
- List/Add/Delete Palo Alto network service offering
- List/Add/Delete Palo Alto network with above service offering
- Add instance to the new network (creates the public IP and private gateway/cidr on the PA as well as the source nat rule)
- List/Add/Delete Ingress Firewall rule
- List/Add/Delete Egress Firewall rule
- List/Add/Delete Port Forwarding rule
- List/Add/Delete Static Nat rule
- Supports Palo Alto Networks 'Log Forwarding' profile globally per device (additional docs to come)
- Supports Palo Alto Networks 'Security Profile Groups' functionality globally per device (additional docs to come)
Knowns limitations:
- Only supports one public IP range in CloudStack.
- Currently not verifying SSL certificates when creating a connection between CloudStack and the Palo Alto Networks firewall.
- Currently not tracking usage on Public IPs.
Signed-off-by: Sheng Yang <sheng.yang@citrix.com>
Changes:
- Consider if VM requires the local storage or shared storage or both for its disks.
- Accordingly all pools in the cluster should consider local or shared or both pools
Conflicts:
server/src/com/cloud/agent/manager/allocator/HostAllocator.java
Now VPN connection can be created as "passive", which would enable the ability
of remote peer initiate the connection. So it's possible for VPC VR to
establish the connection to another VPC VR of CloudStack.
Test case also included.
The test case would create 2 vpcs and using VPN to connect them.
TODO:
1. error handling of no domains present, nested hierarchy
2. handling the case when the api call fails for a specific user/users
3. test cases for LdapUserManager
Signed-off-by: Ian Duffy <ian@ianduffy.ie>
1) added createDetail to ResourceDetailDao interface to provide generic way of creating resourceDetail DB objects
2) added resource details support for firewall rules
* changed name for TaggedResourceType enum to ResourceObjectType as this enum is used both by ResourceMetaData and ResourceTags code
* enhanced the enum with extra fields resourceTagsSupport (boolean) and metadataSupport identifying if the resource supports tags and/or metadata.
* cleanup unused @Inject objects from the ResourceMetaDataManager
Introduction of a new Transaction API that is more consistent with the style
of Spring's transaction managment. The existing Transaction class was renamed
to TransactionLegacy. All of the non-DAO code in the management server has been
updated to use the new Transaction API.
fix listDiskOffering to not display offerings to the normal users with forDisplay=false. But display them to ROOT Admin irrespective of the flag.
Added display flag in the updateVolume/offeirng commands
Signed off by : nitin mehta<nitin.mehta@citrix.com>
These changes are a joint effort between Edison and I to refactor some
of the code around snapshotting VM volumes and creating
templates/volumes from VM volume snapshots. In general, we were working
towards allowing PrimaryDataStoreDrivers to create snapshots on primary
storage and not requiring the snapshots to be transferred to secondary
storage.
High level changes:
-Added uuid to NfsTO, SwiftTO & S3TO to cut down on the requirement of
PrimaryDataStoreTO and ImageStoreTO which don't really serve much of a
purpose
-Initial work towards enable reverting VM volume from snapshots
-Added hypervisor commands for introducing and forgetting new hypervisor
objects (snapshots, templates & volumes)
Signed-off-by: Edison Su <sudison@gmail.com>
Also fixed existing bugs for the API:
* corrected action event to be VOLUME.UPDATE (was VOLUME.ATTACH)
* all parameters to update, should be optional - fixed that. If nothing is specified, the db object will remain with its original fields
Initial patch for VXLAN support.
Fully functional, hopefully, for GuestNetwork - AdvancedZone.
Patch Note:
in cloudstack-server
- Add isolation method VXLAN
- Add VxlanGuestNetworkGuru as plugin for VXLAN isolation
- Modify NetworkServiceImpl to handle extended vNet range for VXLAN isolation
- Add VXLAN isolation option in zoneWizard UI
in cloudstack-agent (kvm)
- Add modifyvxlan.sh script that handle bridge/vxlan interface manipulation script
-- Usage is exactly same to modifyvlan.sh
- BridgeVifDriver will call modifyvxlan.sh instead of modifyvlan.sh when VXLAN is used for isolation
Database changes:
- No change in database structure.
- VXLAN isolation uses same tables that VLAN uses to store vNet allocation status.
Known Issue and/or TODO:
- Some resource still says 'VLAN' in log even if VXLAN is used
- in UI, "Network - GuestNetworks" dosen't display VNI
-- VLAN ID field displays "N/A"
- Documentation!
Signed-off-by : Toshiaki Hatano <haeena@haeena.net>
- change the AccountService::isRootAdmin(short) to isRootAdmin(long accountId);
- Change all callers
- Change all places that check the account.getType() directly to call the AccountManagerImpl.
Changes:
- Implict creation of the 'ExplicitDedication' Affinity group during resource dedication
- Only one group per account or per domain will be present
- ListDedicatedResources by affinityGroup
- Deployment should consider dedicated resources associated to the group only
- Deleting affinity group should release the dedicated resouces
- Releasing the dedicated resources should remove the group associated if there are no more resources.
Conflicts:
plugins/dedicated-resources/src/org/apache/cloudstack/dedicated/DedicatedResourceManagerImpl.java
plugins/dedicated-resources/test/org/apache/cloudstack/dedicated/manager/DedicatedApiUnitTest.java
server/src/com/cloud/configuration/ConfigurationManagerImpl.java
Initial patch for VXLAN support.
Fully functional, hopefully, for GuestNetwork - AdvancedZone.
Patch Note:
in cloudstack-server
- Add isolation method VXLAN
- Add VxlanGuestNetworkGuru as plugin for VXLAN isolation
- Modify NetworkServiceImpl to handle extended vNet range for VXLAN isolation
- Add VXLAN isolation option in zoneWizard UI
in cloudstack-agent (kvm)
- Add modifyvxlan.sh script that handle bridge/vxlan interface manipulation script
-- Usage is exactly same to modifyvlan.sh
- BridgeVifDriver will call modifyvxlan.sh instead of modifyvlan.sh when VXLAN is used for isolation
Database changes:
- No change in database structure.
- VXLAN isolation uses same tables that VLAN uses to store vNet allocation status.
Known Issue:
- Some resource still says 'VLAN' in log even if VXLAN is used
- in UI, "Network - GuestNetworks" dosen't display VNI
-- VLAN ID field displays "N/A"
The time increased due to the newly added dedicated resources feature. During regular VM deployment, all dedicated resources are put in avoid list so that they are not considered for deployment.
Now the way to compute the list of dedicated resources is not optimal and performance deteriorates in an environment having lot of pods, clusters and hosts as the logic is to query db. for each suc resource.
The fix is to optimize the logic not to loop through all resources but get the list of each resource type in a single query.
Conflicts:
server/src/com/cloud/deploy/DeploymentPlanningManagerImpl.java
Removing memoryovercommitratio and cpuovercommitratio parameters from addCluster and updateCluster APIs,
since these can be configurable using updateConfiguration API at cluster level.
By default while creating cluster these values are taken from global configuration parameters.
Changes:
- Locking the group and save reservation mechanism done by DPM
- Added admin operation to cleanup VM reservations
- DPM will also cleanup VM reservations on startup
- Move vnetBridge clean up function from LibvirtComputingResource to BridgeVifDriver
-- since only BridgeVifDriver have to handle this event
- LibvirtComputingResource now properly call VifDriver.unplug() when it receives UnPlugCommand
- Remove not working and no longer used method getVnet(String) from VirtualMachineName
- Remove not working and no longer used method getVnet() from StopCommand
- Remove unused constructer StopCommand(VirtualMachine, String, boolean) from StopCommand
- Remove unused member vnet from StopCommand
- Remove unused member _modifyVlanPath from OvsVifDriver
Tested with 2 KVM hosts and confirmed it correctly manipulate vnetBridge with start, stop, migrate, plug, and unplug event
Signed-off-by: Hugo Trippaers <htrippaers@schubergphilis.com>
include the UUID of the user/domain that was disabled.
- added enity type and enity UUID details to UserContext
- publish the entity type and UUID details for the action events
generated for accout/user/domain
in case of networks with external devices after GC
add an exception for networks that use external networking devices and has
secondary guest IP's allocated. On network GC, when network goes through
implement phase a new vlan is allocated, based on the acquired VLAN id cidr
of the network is decided in case of external networking case. While NIC
uses reservation strategy 'Start' which ensures that new primary ip is
allocated for the NiC from the new CIDR. Secondary IP's have hardcoded
IP's in network rules. So prevent network GC.
Update ImageFormat enum to include VHDX format introduced with Hyper-V
Server 2012.
Remove existing Hyper-V plugin, because it does not work and is dead
code.
Remove references to existing Hyper-V plugin from config files.
Remove Hypervisor.HypervisorType.Hyperv special cases from manager code
that are unused or unsupported.
Specifically, there is no CIFS secondary storage class
"CifsSecondaryStorageResource". Also, the Hyper-V plugin's
ServerResource is contacted by the management server and not the other
way around.
Add Hyperv-V support to ListHypervisorsCmd API call
Signed-off-by: Edison Su <sudison@gmail.com>
Stratosphere SSP is an SDN solution which creates virtual L2
networks backed by vxlan and vlan. SSP will ask hypervisor to set a
specific vlan, then SSP will interact with openflow switches and
put vxlan/vlan translation flow rules.
This plugin provides SSP as "connctivity" service provider.
Signed-off-by: Hiroaki KAWAI <kawai@stratosphere.co.jp>
CLOUDSTACK-3042 - handle Scaling up of vm memory/CPU based on the presence of XS tools in the template
This also takes care of updation of VM after XS tools are installed in the vm and set memory values accordingly to support dynamic scaling after stop start of VM
Signed-off-by: Abhinandan Prateek <aprateek@apache.org>
for number of commands participating in Vm deployment process, as parallel deployment is supported on the hypervisor side.
The behavior is controlled by global config varirables:
"execute.in.sequence.hypervisor.commands" (false by default) sets/resets the synchronization for commands:
=========================
StartCommand
StopCommand
CreateCommand
CopyVolumeCommand
"execute.in.sequence.network.element.commands" (false by default) sets/resets the synchronization for commands:
==========================
DhcpEntryCommand
SavePasswordCommand
UserDataCommand
VmDataCommand
As a part of the fix, increased the global lock timeout to 30 mins in several VR scripts:
===========================
edithosts.sh
savepassword.sh
userdata.sh
to support situations when multiple concurrent calls to the script are being made.
Issues:
In Implicit planner resource usage is fixed to "Dedicated". It should be Dedicated/Shared depending upon the Implict Planner strict/preferred modes and hosts availability.
Fixed:
Issue is fixed by determining the resource usage to be "Dedicated/Shared" depending upon the Implicit strict/preferred mode and the hosts availability for the planner.
This feature allows a user to deploy VMs only in the resources dedicated to his account or domain.
1. Resources(Zones, Pods, Clusters or hosts) can be dedicated to an account or domain.
Implemented 12 new APIs to dedicate/list/release resources:
- dedicateZone, listDedicatedZones, releaseDedicatedZone for a Zone.
- dedicatePod, listDedicatedPods, releaseDedicatedPod for a Pod.
- dedicateCluster, listDedicatedClusters, releaseDedicatedCluster for a Cluster
- dedicateHost, listDedicatedHosts, releaseDedicatedHost for a Host.
2. Once a resource(eg. pod) is dedicated to an account, other resources(eg. clusters/hosts) inside that cannot be further dedicated.
3. Once a resource is dedicated to a domain, other resources inside that can be further dedicated to its sub-domain or account.
4. If any resource (eg.cluster) is dedicated to a account/domain, then resources(eg. Pod) above that cannot be dedicated to different accounts/domain (not belonging to the same domain)
5. To use Explicit dedication, user needs to create an Affinity Group of type 'ExplicitDedication'
6. A VM can be deployed with the above affinity group parameter as an input.
7. A new ExplicitDedicationProcessor has been added which will process the affinity group of type 'Explicit Dedication' for a deployment of a VM that demands dedicated resources.
This processor implements the AffinityGroupProcessor adapter. This processor will update the avoid list.
8. A VM requesting dedication will be deployed on dedicatd resources if available with the user account.
9. A VM requesting dedication can also be deployed on the dedicated resources available with the parent domains iff no dedicated resources are available with the current user's account or
domain.
10. A VM (without dedication) can be deployed on shared host but not on dedicated hosts.
11. To modify the dedication, the resource has to be released first.
12. Existing Private zone functionality has been redirected to Explicit dedication of zones.
13. Updated the db upgrade schema script. A new table "dedicated_resources" has been added.
14. Added the right permissions in commands.properties
15. Unit tests: For the new APIs and Service, added unit tests under : plugins/dedicated-resources/test/org/apache/cloudstack/dedicated/DedicatedApiUnitTest.java
16. Marvin Test: To dedicate host, create affinity group, deploy-vm, check if vm is deployed on the dedicated host.
associated with account
Unlike public ip which gets dis-associated (released) with the account
on network/VPC delete, portable IP should continue to be associated with
the account even when the network/VPC with which it is currently
associated in deleted. This fix ensures portable IP are associated to
account even after network/vpc is deleted.
The location of the virtual machine is provided by DeployDestination, which will
be passed in NetworkGuru#reserve and NetworkElement#prepare.
During the virtual machine migration, it actually changes DeployDestination and
it looks like that it will tell that event to network components as it has
NetworkManager#prepareNicForMigration. The problem is that althogh the interface
has that method, NetworkManagerImpl does not tell the DeployDestination changes
to network components.
So IMHO, we need to add calls of NetworkGuru#reserve and NetworkElement#prepare
in NetworkManagerImpl#prepareNicForMigration . And then, we also need to add
calls NetworkGuru#release and NetworkElement#release after the migration,
otherwise the network resources that plugin reserved will be kept even when the
vm leaves off.
(Sheng Yang: rebase code, add license header)
Signed-off-by: Sheng Yang <sheng.yang@citrix.com>
Squashed commit of the following:
commit f244f9ce7982db16984dd87c31545f1c0240c704
Merge: 993cbb0 f5c8e38
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Mon May 20 18:54:05 2013 +0530
Merge branch 'master' into portablepublicip
Conflicts:
server/src/com/cloud/server/ManagementServerImpl.java
server/test/org/apache/cloudstack/networkoffering/ChildTestConfiguration.java
commit 993cbb0df9fa6e64b96b18ed775b73cdf4a8f5d7
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Mon May 20 18:49:54 2013 +0530
introduce 'transferPortableIP' interface method in network manger. This
method will transfer association of portable ip from one network to
another network.
commit 0c1c2652c1b39e9a81ca35464360e11ed9ef23f1
Merge: a718d35a29e393
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Fri May 17 02:48:54 2013 +0530
Merge branch 'master' into portablepublicip
Conflicts:
utils/src/com/cloud/utils/net/NetUtils.java
commit a718d353f7
Merge: ecca117c211818
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Mon May 13 21:22:19 2013 +0530
Merge branch 'master' into portablepublicip
Conflicts:
api/src/org/apache/cloudstack/api/ResponseGenerator.java
server/src/com/cloud/api/ApiResponseHelper.java
server/src/com/cloud/network/NetworkServiceImpl.java
server/src/com/cloud/network/addr/PublicIp.java
server/src/com/cloud/server/ManagementServerImpl.java
server/test/com/cloud/network/MockNetworkManagerImpl.java
server/test/com/cloud/vpc/MockConfigurationManagerImpl.java
server/test/com/cloud/vpc/MockNetworkManagerImpl.java
setup/db/db/schema-410to420.sql
commit ecca117e34
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Mon May 13 20:05:29 2013 +0530
added integration tests for testing portable ip ranges
commit 895a27c277
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Mon May 13 15:12:19 2013 +0530
- establish model for transferring portable IP association from a network
with which it is associated to another network.
- enabling static nat api, extended to transfer potrtable IP across the
networks if the VM/network is different from the current associate
network of the portable ip
commit 51509751b2
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Mon May 13 12:05:33 2013 +0530
seperate out associate/disassociate with guest network operations from
alloc and release of portable ip
commit bd058f58c2
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Sun May 12 21:14:48 2013 +0530
enhance disasociateIPAddr API to release protable IP associated with a
guest network or VPC
commit 27504d9098
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Sun May 12 16:53:45 2013 +0530
enhance asociateIPAddr API to acquire a protable IP and associate with a
guest network or VPC
commit f82c6a8431
Merge: 3dbfb440749013
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Sat May 11 23:32:13 2013 +0530
Merge branch 'master' into portablepublicip
Conflicts:
api/src/com/cloud/network/IpAddress.java
api/src/org/apache/cloudstack/api/ResponseGenerator.java
client/tomcatconf/commands.properties.in
server/src/com/cloud/api/ApiResponseHelper.java
server/src/com/cloud/configuration/ConfigurationManagerImpl.java
server/src/com/cloud/server/ManagementServerImpl.java
server/test/org/apache/cloudstack/affinity/AffinityApiTestConfiguration.java
server/test/org/apache/cloudstack/networkoffering/ChildTestConfiguration.java
setup/db/db/schema-410to420.sql
commit 3dbfb44eb5
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Sat May 11 20:33:19 2013 +0530
- add 'portable' boolean as property of IpAddress, persist the property in
IPAddressVO, return the property in IpAddressResponse
- add ability to request portable IP in associateIpAddress api
commit bf3cb274cf
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Sat May 11 16:08:40 2013 +0530
add the status of each portable IP (its state, details of associated data
center/VPC/guest network etc) in the PortableIpRangeResponse returned by
listPortableIpRanges API
commit e7b2fb2255
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Sat May 11 14:36:01 2013 +0530
Introdcues notion of 'portable IP' pool at region level.
Introduces root admin only API's to provision portable ip to a region
- createPortableIpRange
- deletePortableIpRange
- listPortableIpRanges
Changes:
- There is no good mechanism currently to figure out if the deployment failed due to affinity groups only
- We can just hint the user that the deployment might have failed due to the affinity groups and ask to review the input
1. A new implicit planner which extends the functionality provided by FirstFitPlanner.
2. Implicit planner can be used in either strict or preferred mode. In strict mode it tries to deploy a vm of a given account on a host on which vms of the account are already running. If no such host is found it'll search for an empty host to service the request. Otherwise the deploy vm request fails.
3. In preferred mode, if a host which is running vms of the account or an empty host isn't found, the planner then tries to deploy on any other host provided it isn't running implicitly dedicated strict vms of any other account.
4. Updated the createServiceOffering api to configure the details for the planner that the service offering is using.
5. Made db changes to store the service offering details for the planner.
6. Unit tests for testing the implicit planner functionality.
7. Marvin test for validating the functionality.
- Changes merged from planner_reserve branch
- Exposing deploymentplanner as an optional parameter while creating a service offering
- changes to DeploymentPlanningManagerImpl to make sure host reserve-release happens between conflicting planner usages.
This feature enables adding of guest ip ranges (public ips) form different subnets.
In order to provide the dhcp service to a different subnet we create an ipalias on the router. This allows the router to listen to the dhcp request from the guest vms and respond accordingly. Every time a vm is deployed in the new subnet we configure an ip alias on the router. Cloudstack uses dnsmasq to provide dhcp service. We need to configure the dnsmasq to issue ips on the new subnets. Added a new class dnsmasqconfigurator which generates the dnsmasq confg file, this file replaces the old config in the router.
The details of the alias ips are stored in db in the nic_ip_alias table. Every time a new subnet is added one of the ip from the subnet is used to configure the ip alias.
I have pushed the code to https://github.com/bvbharatk/cloud-stack/tree/Cloudstack-702 , also rebased the code with master.
I need to test the code for advanced sg enabled network using kvm.
I have added the unit test
Marvin tests are at https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=53e4965
Also accomodated some of the changes suggested by koushik.
corrected the import statements. renamed the IpAlias command to createIpAlias command.
This feature supports only ipv4
tuna
Kishan Kavala
Min Chen
Syed Ahmed
Alena Prokharchyk
Bharat Kumar
Nitin Mehta
Alex Huang
Kelven Yang
Prachi Damle
Wido den Hollander
Harikrishna Patnala
Laszlo Hornyak
Rajesh Battala
Edison Su
Mike Tutkowski
Hugo Trippaers
Will Stevens
Jayapal
Pedro Marques
Sheng Yang
Rajani Karuturi
Wei Zhou
Darren Shepherd
dhoogland
Chris Suich
ynojima
Daan Hoogland
Ian Duffy
Toshiaki Hatano
Koushik Das
Mice Xia
frank
Edison Su
Murali Reddy
Ryan Dietrich
Alex Huang
Donal Lafferty
Hiroaki KAWAI
Jayapal
Saksham Srivastava
Devdeep Singh