Changes:
- We need to reset the avoid set to its original state while calling the storage pool allocators for each volume.
- This will prevent affecting allocation of the disks due to the avoid set output of the prior disk allocations.
Conflicts:
server/src/com/cloud/deploy/DeploymentPlanningManagerImpl.java
Changes:
- Added 'virtualmachineid' parameter to the createVolume API to specify a VM for the volume. The Vm should be in 'Running' or 'Stopped' state.
- This parameter is used only when createVolume API is called using snapshotid parameter
- When this parameter is set, the volume is created from the snapshot in the pod/cluster of the VM. Also the volume is then attached to the VM in the same request
- If attach Volume fails but create has succeeded, the API errors out but the Volume created remains available. User may attach the same volume later
- When Vm is provided, but if no storage pool is available in the VM's pod/cluster then the volume is not created and API fails.
Changes:
- Do not do the stoarge capacity check for a VM having READY Volumes that are already present on the storagepool.
Conflicts:
server/src/com/cloud/deploy/DeploymentPlanningManagerImpl.java
This patch adds support for trust chains in the netscaler.
I initially planned on using the 10.1 API's "bundle" feature but during
my testing I found that was not working. So I am doing the chain linking
myself. Also NS can have only one entity of a certificate ie lets say
two different users try to add the same certificate on the netscaler
only one of them will go through. The other one says resouce already
exists even though they have different files.
This can be a problem in trust chains where the chain can be shared
between multiple accounts/certificates. So, I am using the figerprint as
an identifier of a certificate and making sure that we delete it only
when no one references it.
Resource limit shouldnt be counted for resources with display flag = 0. Adding functions to resourcelimitmanager and doing it for the volumes at the moment.
"Public IP Address" for system VMs
In system VM response object return EIP as public IP in case of basic
zone with EIP service.
Conflicts:
server/src/com/cloud/api/ApiResponseHelper.java
This patch adds an assumption that OpenJDK is used in some of the tests.
OpenJDK detection utility also added. By specifying -Dcloudstack.jce.enabled=true in the test parameters the OpenJDK detection can be overridden.
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
Tests:
- all tests are @Test rather than having one test to call them, so they can be run one by one
- tests that expect exception from a method fail if there is none
- no longer extends TestCase so that the original method names could be kept as test
Implementation:
- include root cause in exceptions when possible - helps at troubleshuting
- close readers
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
ResourceUnavailableException won't be catched by VirtualNetworkApplianceManager
after Alex's refactor, because VirtualMachineManager.start() would cover the
exceptions as well. So changed the calling to advanceStart() instead.
Added a field to the command for serviceOfferingId and changed the internal interface to accept the service offering as parameter
Applied automated code cleanup
This patch adds a network plugin to support Palo Alto Networks firewall (their appliance and their VM series firewall).
More information in the FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Palo+Alto+Firewall+Integration
Features supported are:
- List/Add/Delete Palo Alto service provider
- List/Add/Delete Palo Alto network service offering
- List/Add/Delete Palo Alto network with above service offering
- Add instance to the new network (creates the public IP and private gateway/cidr on the PA as well as the source nat rule)
- List/Add/Delete Ingress Firewall rule
- List/Add/Delete Egress Firewall rule
- List/Add/Delete Port Forwarding rule
- List/Add/Delete Static Nat rule
- Supports Palo Alto Networks 'Log Forwarding' profile globally per device (additional docs to come)
- Supports Palo Alto Networks 'Security Profile Groups' functionality globally per device (additional docs to come)
Knowns limitations:
- Only supports one public IP range in CloudStack.
- Currently not verifying SSL certificates when creating a connection between CloudStack and the Palo Alto Networks firewall.
- Currently not tracking usage on Public IPs.
Signed-off-by: Sheng Yang <sheng.yang@citrix.com>
Fix is use the storage overprovisioning factor (global configuration parameter "storage.overprovisioning.factor") to calculate total provisioning capacity for storage space allocation over VMFS based storage pools as well.
There are two level of thin provisioning provided in VMware, storage level and file-level (VMDK) thin provisioning. in CloudStack, all volumes are provisioned with thin VMDK format, so at hypervisor level, we ALWAYS do thin provisioning. If storage vendor has the ability to provide storage level thin provisioning in addition to VMDK thin provisioning, it is also allowed since it is transparent to Cloudstack.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Changes:
- Set total capacity of a host if it has changed in the CapacityChecker thread
- Fix bug while setting the reserved/used cpu/mem capacity - only one of them used to get set
Changes:
- Consider if VM requires the local storage or shared storage or both for its disks.
- Accordingly all pools in the cluster should consider local or shared or both pools
Conflicts:
server/src/com/cloud/agent/manager/allocator/HostAllocator.java
architecture allows additional functionality to be easily added. Incorporating the plugin in CloudStack will allow
the community to participate in improving the features available with Hyper-V. The plugin uses a Director Connect
Agent architecture described here: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Progress
Add ability to pass kvp data via the key cloudstack-vm-userdata
Rearrange code to make it clearer what .NET objects are being used.
Test failures are easier to deal with if test key is not deleted.
Acquire management/pod ip for control ip when VR deploys in HyperV
Fixed deletion on VM's on hyperv host when mgmt server gets restarted due to HA
Implementation for attach iso command. Attaches an iso to a given vm.
Now VPN connection can be created as "passive", which would enable the ability
of remote peer initiate the connection. So it's possible for VPC VR to
establish the connection to another VPC VR of CloudStack.
Test case also included.
The test case would create 2 vpcs and using VPN to connect them.
1) vxlan will use bridge scheme 'brvx-<vni>'. Multiple physical networks can host guest
traffic type with vxlan isolation, so long as they don't use the same VNI range.
2) Guest traffic labels can be physical interface if bridge by given name is not found.
Normally we take traffic label name, find the matching bridge, then resolve that to a
physical interface. Then we create guest bridges on that interface. Now we can just
specify the interface.
TODO:
1. error handling of no domains present, nested hierarchy
2. handling the case when the api call fails for a specific user/users
3. test cases for LdapUserManager
Signed-off-by: Ian Duffy <ian@ianduffy.ie>
In case of VMware, once the state of the old root volume has been updated to destroyed force expunge it from primary storage to avoid the garbage collector from trying to delete the new root volume
1) added createDetail to ResourceDetailDao interface to provide generic way of creating resourceDetail DB objects
2) added resource details support for firewall rules
The following types were not setup to discover extensions
com.cloud.network.element.FirewallServiceProvider
com.cloud.network.element.PortForwardingServiceProvider
com.cloud.network.element.StaticNatServiceProvider
com.cloud.network.element.NetworkACLServiceProvider
com.cloud.network.element.LoadBalancingServiceProvider
com.cloud.network.element.Site2SiteVpnServiceProvider
1) Added support for Zone resource details
2) Renamed DcDetailsDao to DataCenterDetailsDao to follow the CS name convention for DataCenter related classes
This is not perfect fix but should address 99% of the issues. The only
time an issue could happen is if something in the bootstrap or system
context tried to lock a row. Which shouldn't happen at the moment. If
we find that to be an issue, the LockMasterListener could move to system
context and be inject here.
* changed name for TaggedResourceType enum to ResourceObjectType as this enum is used both by ResourceMetaData and ResourceTags code
* enhanced the enum with extra fields resourceTagsSupport (boolean) and metadataSupport identifying if the resource supports tags and/or metadata.
* cleanup unused @Inject objects from the ResourceMetaDataManager
Detail: Previously the cloud user has full password-less sudo access.
This commit changes that to only allow access to a specific list of
commands. Been tested in production on ACS 4.0 and 4.2 mangement servers.
BUG-ID: CLOUDSTACK-967
Bugfix-for:
Reviewed-by:
Reported-by:
Signed-off-by: John Kinsella <jlk@stratosec.co> 1382560936 -0700
commit c9ee0d12e191e803fb341f3f96e95ca434a36f6c
Author: Wei Zhou <w.zhou@leaseweb.com>
Date: Wed Oct 23 16:55:10 2013 +0200
CLOUDSTACK-4931, CLOUDSTACK-4937: setDetails to user VMs only
(cherry picked from commit a94acc5a43)
commit fe1586c71377bc6d219db2dcf088c40b65dd1fc4
Author: Anthony Xu <anthony.xu@citrix.com>
Date: Tue Oct 22 11:20:27 2013 -0700
CLOUDSTACK-4649:
vm sync tracks the pv driver version for xenserver
Anthony
commit 56a218f66eda540b4b4b04030ee71fc6863f8532
Author: Anthony Xu <anthony.xu@citrix.com>
Date: Mon Oct 21 16:10:07 2013 -0700
CLOUDSTACK-4649:
xs 6.1/6.2 introduce the new virtual platform, so there are two virtual platforms, windows PV driver version must match virtual platforms,
this patch tracks PV driver versions in vm details and template details.
Anthony
commit 4e85d28c678a6f96b5b70d8d33fc60f9d1ea3df6
Author: Laszlo Hornyak <laszlo.hornyak@gmail.com>
Date: Mon Oct 21 21:17:33 2013 +0200
removed unused static field
- s_httpClientManager was not used
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
commit d4121fa26023db236f7396cea455ef090672ae9a
Author: Chris Suich <chris.suich@netapp.com>
Date: Tue Oct 22 10:45:22 2013 -0400
Updated DataMotionServiceImpl and ApiResponseHelper based on review feedback.
commit aaf026e1e4204d405bcda2ae4f1a01b1d0f7e7cb
Author: Chris Suich <chris.suich@netapp.com>
Date: Thu Oct 17 14:27:12 2013 -0400
Added context to strategy sorting error responses
Added TODOs for DRYing out pickStrategy() overloading
commit a221f4aa3fb2ddc255bc35cf753f98f88f5bf44e
Author: Chris Suich <chris.suich@netapp.com>
Date: Wed Oct 16 09:57:28 2013 -0400
Updated inefficient strategy sorting/selection
Removed unnecessary canRevertSnapshot from PrimaryDataStoreDriver
Other general cleaup and fixes from reviews
commit 7d58949c6a1b7e853e891b59387a9620e8cd7a91
Author: Chris Suich <chris.suich@netapp.com>
Date: Mon Oct 14 14:01:22 2013 -0400
Added volume snapshot revert capability to SnapshotResponse
Updated UI to hide/show snapshot revert action per snapshot
Signed-off-by: Edison Su <sudison@gmail.com>
- new utility method introduced in PropertiesUtil to load properties objects from files
- RegionManagerImpl modified to use the utility method
- Tests added for both
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
xs 6.1/6.2 introduce the new virtual platform, so there are two virtual platforms, windows PV driver version must match virtual platforms,
this patch tracks PV driver versions in vm details and template details.
Anthony
public range to see whether the same VLAN exists in portable IP range.
added check to enusre a VLAN id used for a public IP range is not used
for portable ip range
Conflicts:
server/src/com/cloud/configuration/ConfigurationManagerImpl.java
The cluster and zone wide storage pool allocators returned shared pools even for volumes meant to be on local storage pool.
If the VM uses local disk then cluster and zone storage allocators should not handle it and return null or empty list.
Also fixed the deployment planner to avoid a cluster if
a. avoid set returned by storage pool allocators is empty OR
b. all local or shared pools in a cluster are in avoid state
Conflicts:
engine/storage/src/org/apache/cloudstack/storage/allocator/ClusterScopeStoragePoolAllocator.java
engine/storage/src/org/apache/cloudstack/storage/allocator/ZoneWideStoragePoolAllocator.java
defined in different VLANs across public and portable ranges
added checks to restric same ip range to be configure as both public ip
range and portable ip range
This patch would reset the priority in such condition:
1. All redundant routers are stopped, e.g. due to network GC
2. User start one VM in the network
3. The routers would be brought up with reseted priority(100 & 99).
This would resolve the issue of network GC result in lower limit of redundant router priority reached.
Even though the volume may get migrated from shared to local storage, it is not possible to update the disk offering.
The fix is to disallow migration from shared to local store.
VM deployment is fine, issue is in attach volume where all possible scenarios are not handled.
The following needs to be logic of attached volume:
1. if data volume scope is zone then allow attach (this is already there)
2. if data volume scope is cluster
a. if root volume scope is host, allow if both are in same cluster (already there)
b. if root volume scope is zone, allow if vm and data volume in same cluster (fixed as part of this commit)
3. if data volume scope is host allow if vm and data volume in same host (fixed as part of this commit)
list Clusters/pods/zones API not accounting for reserved in the used capacity percentage.
Fix listCapacity cmd for reserved as well
Signed off by : nitin mehta<nitin.mehta@citrix.com>
Introduction of a new Transaction API that is more consistent with the style
of Spring's transaction managment. The existing Transaction class was renamed
to TransactionLegacy. All of the non-DAO code in the management server has been
updated to use the new Transaction API.
1) Fixed the way we build the response for resource details
2) Fixed the response name - should be generic "resourcedetail". The type is being carried in resourceType response parameter
3) Throw unsupported excpetion if unsupported resourceType is passed in. Before, user vm was returned
fix listDiskOffering to not display offerings to the normal users with forDisplay=false. But display them to ROOT Admin irrespective of the flag.
Added display flag in the updateVolume/offeirng commands
Signed off by : nitin mehta<nitin.mehta@citrix.com>
This code was recently switched from using HashMap to ConcurrentHashMap.
ConcurrentHashMap does not accept null keys and will through a NPE. Adding
a null check just in case somebody passes null.
fix listVolumes to not display volumes to the normal users with forDisplay=false. But display volumes to ROOT Admin irrespective of the flag.
Signed off by : nitin mehta<nitin.mehta@citrix.com>
These changes are a joint effort between Edison and I to refactor some
of the code around snapshotting VM volumes and creating
templates/volumes from VM volume snapshots. In general, we were working
towards allowing PrimaryDataStoreDrivers to create snapshots on primary
storage and not requiring the snapshots to be transferred to secondary
storage.
High level changes:
-Added uuid to NfsTO, SwiftTO & S3TO to cut down on the requirement of
PrimaryDataStoreTO and ImageStoreTO which don't really serve much of a
purpose
-Initial work towards enable reverting VM volume from snapshots
-Added hypervisor commands for introducing and forgetting new hypervisor
objects (snapshots, templates & volumes)
Signed-off-by: Edison Su <sudison@gmail.com>
fix listVirtualMachines to not display vms to the normal users with forDisplay=false. But displayvms to ROOT Admin irrespective of the flag.
Signed off by : nitin mehta<nitin.mehta@citrix.com>
All of the code changes for the Spring Modularization will work in
a modularized context or a non-modularized context. This commit
is the final commit to turn modularization on. Revert this commit
to disable the modularization and go back to monolithic Spring
configuration.
ACS is now comprised of a hierarchy of spring application contexts.
Each plugin can contribute configuration files to add to an existing
module or create it's own module.
Additionally, for the mgmt server, ACS custom AOP is no longer used
and instead we use Spring AOP to manage interceptors.
The managed context framework provides a simple way to add logic
to ACS at the various entry points of the system. As threads are
launched and ran listeners can be registered for onEntry or onLeave
of the managed context. This framework will be used specifically
to handle DB transaction checking and setting up the CallContext.
This framework is need to transition away from ACS custom AOP to
Spring AOP.
Various classes are using member injection to inject extensible objects.
Really those object should come from an AdapterList that is injected in.
This patch switches the code to use setter injection that will later allow
spring to inject an AdapterList or something similar to allow
extensibility.
DefaultUserAuthenticator maskes the _name varible in ComponentLifecycleBase
making the setName() method not work as expected. This patch cleans up the
code such that getName() will be getClass().getSimpleName() unless
overridden in the Spring configuration.
Also fixed existing bugs for the API:
* corrected action event to be VOLUME.UPDATE (was VOLUME.ATTACH)
* all parameters to update, should be optional - fixed that. If nothing is specified, the db object will remain with its original fields
Initial patch for VXLAN support.
Fully functional, hopefully, for GuestNetwork - AdvancedZone.
Patch Note:
in cloudstack-server
- Add isolation method VXLAN
- Add VxlanGuestNetworkGuru as plugin for VXLAN isolation
- Modify NetworkServiceImpl to handle extended vNet range for VXLAN isolation
- Add VXLAN isolation option in zoneWizard UI
in cloudstack-agent (kvm)
- Add modifyvxlan.sh script that handle bridge/vxlan interface manipulation script
-- Usage is exactly same to modifyvlan.sh
- BridgeVifDriver will call modifyvxlan.sh instead of modifyvlan.sh when VXLAN is used for isolation
Database changes:
- No change in database structure.
- VXLAN isolation uses same tables that VLAN uses to store vNet allocation status.
Known Issue and/or TODO:
- Some resource still says 'VLAN' in log even if VXLAN is used
- in UI, "Network - GuestNetworks" dosen't display VNI
-- VLAN ID field displays "N/A"
- Documentation!
Signed-off-by : Toshiaki Hatano <haeena@haeena.net>
- change the AccountService::isRootAdmin(short) to isRootAdmin(long accountId);
- Change all callers
- Change all places that check the account.getType() directly to call the AccountManagerImpl.
replaced Long object instantiation with static call to Long.toString in case it was only to generate a string
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
- variables inlined
- cpu utilization is not cast to float from double
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
Signed-off-by: Min Chen <min.chen@citrix.com>
double values do not need a Double object to be casted to long
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
Signed-off-by: Min Chen <min.chen@citrix.com>
It was implemented by extending the NFS provider. Its validation was updated so that you can pass it a URL containing the
details of a CIFS share. The code that mounts NFS shares was extended to allow it do the same for CIFS shares. Otherwise,
the secondary storage code is left unchanged.
This reverts commit 5a8a2a259e.
We would fix it in another way, since mgmt server may get state updated in
time.
Conflicts:
server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java
Changes:
- When listing a zone, add clause in the search to check the account_id for a dedicated zone
- When listsing a zone with a domainid, add a similar clause.
- DomainCheck:: checkAccess() for a zone should consider that zone can now be dediacted to a specific account and check access accordingly.
Conflicts:
server/src/com/cloud/api/query/vo/DataCenterJoinVO.java
setup/db/db/schema-410to420.sql
Changes:
createZone API:
- This API takes in domainid, set it to the zone record in the data_center table
updateZone API:
- This API uses 'isPublic' flag to set a private zone to public - if this flag is set and the zone is dedicated, release the dedication and remove the domainid from the data_center table
listZone API:
- This API already has 'domainid' parameter. We should allow list zones by domain for Root admin.
DedicateZone API:
- set domainid in the data_center table
ReleaseDedicatedZone API:
- remove zoneid from the data_center table
Changes:
- Adding mocks in unit tests for new injected components
Conflicts:
server/test/org/apache/cloudstack/networkoffering/ChildTestConfiguration.java
Changes:
- Implict creation of the 'ExplicitDedication' Affinity group during resource dedication
- Only one group per account or per domain will be present
- ListDedicatedResources by affinityGroup
- Deployment should consider dedicated resources associated to the group only
- Deleting affinity group should release the dedicated resouces
- Releasing the dedicated resources should remove the group associated if there are no more resources.
Conflicts:
plugins/dedicated-resources/src/org/apache/cloudstack/dedicated/DedicatedResourceManagerImpl.java
plugins/dedicated-resources/test/org/apache/cloudstack/dedicated/manager/DedicatedApiUnitTest.java
server/src/com/cloud/configuration/ConfigurationManagerImpl.java
Changes:
- 'ExcplicitDedication' type of group can be created/deleted by Root admin only
- Users can no longer create this type of affinity group
- RootAdmin can create this type of affinitygroup at domain level. Such a domain level group is available for all accounts in that domain for listing and for use during deployVM.
- The domain level affinitygroup should be visible to the users in that domain, domain admins and Root admin.
Conflicts:
server/src/com/cloud/api/query/QueryManagerImpl.java
server/src/org/apache/cloudstack/affinity/AffinityGroupServiceImpl.java
server/test/org/apache/cloudstack/affinity/AffinityApiUnitTest.java
There are three issues in resource_count table
(1) expunge a vm, the public_ip decreases and becomes -1 in basic zone.
(2) recover a vm, the volume increase.
(3) restore a vm, the volume decrease.
deployment fails with error "Message: Invalid configuration for device
'2'."
ensuring that direct network guru assigns a mac address for the nic that
it designs
Improving the error message saying ssh key pair should be assinged to VM to get the encrypted password.
Signed-off-by: Koushik Das <koushik@apache.org>
During host connect multiple listeners gets invoked, one of them is the download listener.
As part of processConnect() method, it checks if templates needs to be downloaded to secondary
store for a particular HV type. As part of that check it computes list of HVs present in the
zone. The earlier logic was to query all hosts (excluding current one) and iterate over them to
make the list. This is not optimal and is bound to have some latency as the number of hosts
increases.
Optimized the logic by querying the list of HVs from the db. directly instead of iterating over
all hosts in the zone.
Conflicts:
server/src/com/cloud/resource/ResourceManagerImpl.java
Initial patch for VXLAN support.
Fully functional, hopefully, for GuestNetwork - AdvancedZone.
Patch Note:
in cloudstack-server
- Add isolation method VXLAN
- Add VxlanGuestNetworkGuru as plugin for VXLAN isolation
- Modify NetworkServiceImpl to handle extended vNet range for VXLAN isolation
- Add VXLAN isolation option in zoneWizard UI
in cloudstack-agent (kvm)
- Add modifyvxlan.sh script that handle bridge/vxlan interface manipulation script
-- Usage is exactly same to modifyvlan.sh
- BridgeVifDriver will call modifyvxlan.sh instead of modifyvlan.sh when VXLAN is used for isolation
Database changes:
- No change in database structure.
- VXLAN isolation uses same tables that VLAN uses to store vNet allocation status.
Known Issue:
- Some resource still says 'VLAN' in log even if VXLAN is used
- in UI, "Network - GuestNetworks" dosen't display VNI
-- VLAN ID field displays "N/A"
The locking code in implement/shutdown network code was not efficient. Even in order to check the current state of the network lock was getting acquired which is not required. This resulted in delays in deploy VM as can be seen from attached logs where the code waited on the lock just to check if network is implemented.
As part of the fix moved out code that is checking if the network is already implemented or shutdowned outside the lock.
For some scenarios like prepare nic, all network service providers are checked which is not efficient and also introduces unnecessary dependencies.
The check to use only the required providers is already there for implement, shutdown operation on network. Put the same check for all missing cases.
If you shut down the agent while VMs are running, the management
server assumes that the VMs are continuing to run. You can then
delete the host while it is in 'disconnected' state, and those VMs
will be unusable, forever in running state. They can't change state
because the host no longer exists. This patch checks for any VMs
that may have been tied to the removed host and resets their state
so that cloudstack can continue to manage them.
Prachi Damle
Min Chen
tuna
Kishan Kavala
yichi.lu
Alex Ough
Kelven Yang
Alena Prokharchyk
Fang Wang
Anthony Xu
Nitin Mehta
Syed Ahmed
Likitha Shetty
Bharat Kumar
Devdeep Singh
Murali Reddy
lafferty
Jayapal
Wei Zhou
Hugo Trippaers
Abhinandan Prateek
wilderrodrigues
Anshul Gangwar
Alex Huang
Murali Reddy
Wido den Hollander
fjkdf
Harikrishna Patnala
Prasanna Santhanam
Edison Su
Darren Shepherd
Kelven Yang
Laszlo Hornyak
Daan Hoogland
Sheng Yang
Rajesh Battala
Mike Tutkowski
Will Stevens
Sateesh Chodapuneedi
Donal Lafferty
Rajani Karuturi
ynojima
Marcus Sorensen
John Kinsella
Chris Suich
dhoogland
alex@alexhitchins.com
Koushik Das
Damodar Reddy
Darren Shepherd
Daan Hoogland
Chip Childers
Ian Duffy
Chiradeep Vittal
frank
Toshiaki Hatano