Snapshot object is being accessed even when it is null. In case snapshot is not present in backup store the code should return after cleaning db entry.
Also noticed discrepancy in the upgraded db setup but couldn't fully verify it, so added logging in the upgrade logic for template/snapshots etc.
Track the Datacenter of previous cluster correctly while going through each cluster in the zone to see if 2 clusters are from different DC/vCenter.
Cherry picked from 4.2 commit a3450afff5
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
System template upgrade is not required during 4.0 upgrade since we handle the same during 4.2 upgrade. So removing the system template update during 4.0 upgrade.
KVM.snapshot.enabled is lowercased by f025db95 to keep the configs
uniformly lower-case. But it missed the upgrade script and the
references in SnapshotManagerImpl. This commit will fix the issue in all
locations
Signed-off-by: Prasanna Santhanam <tsp@apache.org>
(cherry picked from commit 0e216fa7e41bdfe0cc744006bb896c8b00138ca2)
Private templates would now get copied to only one of image storage chosen randamly as was the case earlier. Dont throw an exception for uploading volumes when there are multiple image stores, instead choose one of them randomly
Signed off by : nitin mehta<nitin.mehta@citrix.com>
Persist the download url in the db for volume download.
Signed off by : nitin mehta<nitin.mehta@citrix.com>
Conflicts:
server/src/com/cloud/storage/VolumeManagerImpl.java
Marked the system template new system template as dynamicallyScalable
- handled upgrade case
- moved "dynamicallyScalable" flag to vm_instance table from user_vm_details to support dynamic scaling of system vm
Signed off by : Nitin Mehta<nitin.mehta@citrix.com>
Changes:
- During host deletion, host entry in databse gets removed prior to the disconnect task getting processed.
- This causes the disconnect task to get NPE while trying to do the host state transition
Some existing scenarios for root and data volume combination was not working. These are
a. Local root + Shared data
b. Shared root + Local data
Enabled these scenarios as part of this fix
The request to migrate was coming till ancient data motion strategy but wasn't worked on and forwarded
to the resource. The code probably got removed by a bad merge. Bringing it back.
The issue is that while calculating the used primary storage size, the updateResourceCount
API is also calculating the disk size of virtual router VM, created for that account and
because of this, the API is returning the incorrect result.
Changes:
- Passing the avoid set generated by the first pass of deployment to the second try.
- The second try is done, when the first pass that uses a reserved plan fails to deploy on the reserved host, to search over the entire zone again
Changes:
- Locking the group and save reservation mechanism done by DPM
- Added admin operation to cleanup VM reservations
- DPM will also cleanup VM reservations on startup
Available bytes was getting stored in the used bytes property of local storage pools. As a result of this, for newly added local pools Cloudstack thinks that there is no space available and generated alerts.
User should be able to delete/archive alerts and events by selecting a time period or by
choosing the alerts and events older than a date. Added the ability to choose a time period
too.
Summary of changes in the fix
- Optimized host scan logic, now instead of iterating over each cluster host scan is done for a batch of clusters
- Made host scan task interval configurable
only on first rule is created on the IP and last rule is revoked on the
IP
Current suboptima logic of IP Assoc
- On associate IP to GuestNetwork there is an IPAssoc command sent to
corresponding network service providers of the network
- On every rule apply on IP associated with the network send IP assoc
to the network service providers
- On every rule deletion on IP associated with a network sernd IP assoc
command to the network service providers
With this fix logic of IP assoc is changed as below which eliminates
executio of unnessary and expensive IpAssocCommand resource command
- On associate IP to GuestNetwork, associate IP only to the network,
Untill any service is associated with the IP dont send IP Assoc
- On creation of first rule on the IP send IPAssoc to corresponding
network service provider. Since IP is used for a service, IPAssoc
need to be sent to correpondign service provider
- On deletion of last rule on the IP send IPAssoc to corresponding
network service provider. When last rule is deleted, IP has no
service associated with it, so send IP assoc to service provider to
remove the IP association
Filter the detail map sent over the wire into Map<String, String> before
processing underneath by storage life cycle
Signed-off-by: Prasanna Santhanam <tsp@apache.org>
Description:
When retrieving primary datastore, handle case for non-existing datastores/hosts.
Throw exception and handle the exception in datastore mgmt layer and pass onward
to create storage pool API.
involved in the GSLB
add weights to each site participating in the GSLB. Traffic will be load
balanced across the sites based on the weigths associated with each
site. If not specified weight of site is defaulted to 1.
Breaking down storage components among oss, nonoss and simulator
contexts. The default components are loaded by
OSS - applicationContext + componentContext
NonOSS - applicationContext + nonossComponentContext
Simulator - applicationContext + simulatorComponentContext
provider beans are are selectively overridden for simpler configuration.
Where possible beans are loaded by local reference.
<list merge=true> does not unfortunately work perfectly for bean merging
the providers causing a bit of bloat. Explore for later.
Signed-off-by: Prasanna Santhanam <tsp@apache.org>
Update ImageFormat enum to include VHDX format introduced with Hyper-V
Server 2012.
Remove existing Hyper-V plugin, because it does not work and is dead
code.
Remove references to existing Hyper-V plugin from config files.
Remove Hypervisor.HypervisorType.Hyperv special cases from manager code
that are unused or unsupported.
Specifically, there is no CIFS secondary storage class
"CifsSecondaryStorageResource". Also, the Hyper-V plugin's
ServerResource is contacted by the management server and not the other
way around.
Add Hyperv-V support to ListHypervisorsCmd API call
Signed-off-by: Edison Su <sudison@gmail.com>
NAT does not work
making an exception for portabe IP, so that if the current datacenter with
portable IP is associated is different from destiantion data center
also on transfer on to new zone, transfer the portable ip association to
new data center, physical network id's
CLOUDSTACK-3042 - handle Scaling up of vm memory/CPU based on the presence of XS tools in the template
This also takes care of updation of VM after XS tools are installed in the vm and set memory values accordingly to support dynamic scaling after stop start of VM
Signed-off-by: Abhinandan Prateek <aprateek@apache.org>
for number of commands participating in Vm deployment process, as parallel deployment is supported on the hypervisor side.
The behavior is controlled by global config varirables:
"execute.in.sequence.hypervisor.commands" (false by default) sets/resets the synchronization for commands:
=========================
StartCommand
StopCommand
CreateCommand
CopyVolumeCommand
"execute.in.sequence.network.element.commands" (false by default) sets/resets the synchronization for commands:
==========================
DhcpEntryCommand
SavePasswordCommand
UserDataCommand
VmDataCommand
As a part of the fix, increased the global lock timeout to 30 mins in several VR scripts:
===========================
edithosts.sh
savepassword.sh
userdata.sh
to support situations when multiple concurrent calls to the script are being made.
has dedicated resources and the dedicated resources have all been consumed - use.system.public.ips and use.system.guest.vlans
Both configs are configurable at the account level too.
Recurring snapshot schedule not showing up in UI
For some of the volumes Recurring snapshot schedule was not showing up in UI because the active column was set to false. Since we dont use this column anymore I am removing the active=true check in the listSnapshotPolicies call.
combination prior to 3.0 release
Fix does following:
- add F5 network service provider into a physical network if there if F5
deployed in the zone
- add instance of F5 network service provider
- add SRX network service provider into a physical network if there if
SRX deployed in the zone
- add instance of SRX network service provider
- upgrade all the guest networks to network offering '"Isolated with
external providers"
Added hypervisor type to CreateStoragePoolCmd & Storage pool responses.
DatastoreLifeCycle would consider hypervisor type while attaching datastore to zone.
ZoneWideStoragePoolAllocator would filter zone wide primary storage pools by hypervisor type along with tags in disk profile.
hypervisor type is mandatory parameter if scope is specified as ZONE while creating primary storage pool.
As of now KVM, VMware are allowed to use ZoneWideStoragePoolAllocator.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
DB changes to support hypervisor specific zone wide storage pool.
Added method findZoneWideStoragePoolsByHypervisor to PrimaryStorageDaoImpl to find suitable zone wide storage pool of specific hypervisor type.
Added column 'hypervisor' to table storage_pool. This column can be NULL. Used/populated only for zone wide primary storage pools.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Upgrade handling.
Detect legacy zones during db upgrade and perform data migration.
If legacy zone is detected the table 'cloud'.'legacy_zones' is populated.
If an existing zone which has resources that belong to single VMware datacenter then such zone would not be marked as legacy zone. Such zone would be automatically associated with the specific Vmware datacenter of the clusters inside the zone.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
This feature allows a user to deploy VMs only in the resources dedicated to his account or domain.
1. Resources(Zones, Pods, Clusters or hosts) can be dedicated to an account or domain.
Implemented 12 new APIs to dedicate/list/release resources:
- dedicateZone, listDedicatedZones, releaseDedicatedZone for a Zone.
- dedicatePod, listDedicatedPods, releaseDedicatedPod for a Pod.
- dedicateCluster, listDedicatedClusters, releaseDedicatedCluster for a Cluster
- dedicateHost, listDedicatedHosts, releaseDedicatedHost for a Host.
2. Once a resource(eg. pod) is dedicated to an account, other resources(eg. clusters/hosts) inside that cannot be further dedicated.
3. Once a resource is dedicated to a domain, other resources inside that can be further dedicated to its sub-domain or account.
4. If any resource (eg.cluster) is dedicated to a account/domain, then resources(eg. Pod) above that cannot be dedicated to different accounts/domain (not belonging to the same domain)
5. To use Explicit dedication, user needs to create an Affinity Group of type 'ExplicitDedication'
6. A VM can be deployed with the above affinity group parameter as an input.
7. A new ExplicitDedicationProcessor has been added which will process the affinity group of type 'Explicit Dedication' for a deployment of a VM that demands dedicated resources.
This processor implements the AffinityGroupProcessor adapter. This processor will update the avoid list.
8. A VM requesting dedication will be deployed on dedicatd resources if available with the user account.
9. A VM requesting dedication can also be deployed on the dedicated resources available with the parent domains iff no dedicated resources are available with the current user's account or
domain.
10. A VM (without dedication) can be deployed on shared host but not on dedicated hosts.
11. To modify the dedication, the resource has to be released first.
12. Existing Private zone functionality has been redirected to Explicit dedication of zones.
13. Updated the db upgrade schema script. A new table "dedicated_resources" has been added.
14. Added the right permissions in commands.properties
15. Unit tests: For the new APIs and Service, added unit tests under : plugins/dedicated-resources/test/org/apache/cloudstack/dedicated/DedicatedApiUnitTest.java
16. Marvin Test: To dedicate host, create affinity group, deploy-vm, check if vm is deployed on the dedicated host.
RBD format 2 supports cloning (aka layering) where one base image can serve
as a parent image for multiple child images.
This enables fast deployment of a large amount of virtual machines, but it also
saves spaces on the Ceph cluster and improves performance due to better caching.
Qemu-img doesn't support RBD format 2 (yet), so to enable these functions the
RADOS/RBD Java bindings are required.
This patch also enables deployment of System VMs on RBD storage pools. Since we
no longer require a patchdisk for passing the boot arguments we are able to deploy
these VMs on RBD.
Changes:
- In VolumeReservationVO, the getter method of a column had a typo, causing us to create a wrong searchbuilder. It was searching over the 'id' column instead of 'vm_reservation_id' causing
- This bug was causing the vm deployment to choose a wrong pool during deployment since the search was choosing incorrectly
- This bug in the GenericSearchBuilder is also fixed - if the getter method does not use the standard 'get' or 'is' prefix, one should annotate that method using
@Column(name = "<column_name>") and indicate which column this method refers to. This will cause the GenericSearchBuilder to identify the field correctly.
Squashed commit of the following:
commit f244f9ce7982db16984dd87c31545f1c0240c704
Merge: 993cbb0 f5c8e38
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Mon May 20 18:54:05 2013 +0530
Merge branch 'master' into portablepublicip
Conflicts:
server/src/com/cloud/server/ManagementServerImpl.java
server/test/org/apache/cloudstack/networkoffering/ChildTestConfiguration.java
commit 993cbb0df9fa6e64b96b18ed775b73cdf4a8f5d7
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Mon May 20 18:49:54 2013 +0530
introduce 'transferPortableIP' interface method in network manger. This
method will transfer association of portable ip from one network to
another network.
commit 0c1c2652c1b39e9a81ca35464360e11ed9ef23f1
Merge: a718d35a29e393
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Fri May 17 02:48:54 2013 +0530
Merge branch 'master' into portablepublicip
Conflicts:
utils/src/com/cloud/utils/net/NetUtils.java
commit a718d353f7
Merge: ecca117c211818
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Mon May 13 21:22:19 2013 +0530
Merge branch 'master' into portablepublicip
Conflicts:
api/src/org/apache/cloudstack/api/ResponseGenerator.java
server/src/com/cloud/api/ApiResponseHelper.java
server/src/com/cloud/network/NetworkServiceImpl.java
server/src/com/cloud/network/addr/PublicIp.java
server/src/com/cloud/server/ManagementServerImpl.java
server/test/com/cloud/network/MockNetworkManagerImpl.java
server/test/com/cloud/vpc/MockConfigurationManagerImpl.java
server/test/com/cloud/vpc/MockNetworkManagerImpl.java
setup/db/db/schema-410to420.sql
commit ecca117e34
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Mon May 13 20:05:29 2013 +0530
added integration tests for testing portable ip ranges
commit 895a27c277
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Mon May 13 15:12:19 2013 +0530
- establish model for transferring portable IP association from a network
with which it is associated to another network.
- enabling static nat api, extended to transfer potrtable IP across the
networks if the VM/network is different from the current associate
network of the portable ip
commit 51509751b2
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Mon May 13 12:05:33 2013 +0530
seperate out associate/disassociate with guest network operations from
alloc and release of portable ip
commit bd058f58c2
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Sun May 12 21:14:48 2013 +0530
enhance disasociateIPAddr API to release protable IP associated with a
guest network or VPC
commit 27504d9098
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Sun May 12 16:53:45 2013 +0530
enhance asociateIPAddr API to acquire a protable IP and associate with a
guest network or VPC
commit f82c6a8431
Merge: 3dbfb440749013
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Sat May 11 23:32:13 2013 +0530
Merge branch 'master' into portablepublicip
Conflicts:
api/src/com/cloud/network/IpAddress.java
api/src/org/apache/cloudstack/api/ResponseGenerator.java
client/tomcatconf/commands.properties.in
server/src/com/cloud/api/ApiResponseHelper.java
server/src/com/cloud/configuration/ConfigurationManagerImpl.java
server/src/com/cloud/server/ManagementServerImpl.java
server/test/org/apache/cloudstack/affinity/AffinityApiTestConfiguration.java
server/test/org/apache/cloudstack/networkoffering/ChildTestConfiguration.java
setup/db/db/schema-410to420.sql
commit 3dbfb44eb5
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Sat May 11 20:33:19 2013 +0530
- add 'portable' boolean as property of IpAddress, persist the property in
IPAddressVO, return the property in IpAddressResponse
- add ability to request portable IP in associateIpAddress api
commit bf3cb274cf
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Sat May 11 16:08:40 2013 +0530
add the status of each portable IP (its state, details of associated data
center/VPC/guest network etc) in the PortableIpRangeResponse returned by
listPortableIpRanges API
commit e7b2fb2255
Author: Murali Reddy <muralimmreddy@gmail.com>
Date: Sat May 11 14:36:01 2013 +0530
Introdcues notion of 'portable IP' pool at region level.
Introduces root admin only API's to provision portable ip to a region
- createPortableIpRange
- deletePortableIpRange
- listPortableIpRanges
Changes:
- There is no good mechanism currently to figure out if the deployment failed due to affinity groups only
- We can just hint the user that the deployment might have failed due to the affinity groups and ask to review the input
1. A new implicit planner which extends the functionality provided by FirstFitPlanner.
2. Implicit planner can be used in either strict or preferred mode. In strict mode it tries to deploy a vm of a given account on a host on which vms of the account are already running. If no such host is found it'll search for an empty host to service the request. Otherwise the deploy vm request fails.
3. In preferred mode, if a host which is running vms of the account or an empty host isn't found, the planner then tries to deploy on any other host provided it isn't running implicitly dedicated strict vms of any other account.
4. Updated the createServiceOffering api to configure the details for the planner that the service offering is using.
5. Made db changes to store the service offering details for the planner.
6. Unit tests for testing the implicit planner functionality.
7. Marvin test for validating the functionality.
- Changes merged from planner_reserve branch
- Exposing deploymentplanner as an optional parameter while creating a service offering
- changes to DeploymentPlanningManagerImpl to make sure host reserve-release happens between conflicting planner usages.
Public IP addresses resource count of an account - number of ip addresses dedicated to an account
plus the number of ip addresses belonging to the system that have been allocated to the account
1) Only PrivateNetworkGuru handles network creation for the private gateway. Exluded Guest Network Guru from this list (was mistakenly included as a part of merge for Nicira integration)
2) Pass vpc_id to createNetwork call when the network is created as a part of private gateway creation
3) Fixed VPC restart when there are multiple private gateways present (have to grab all the private gateways when creating nic profiles for the VPC router that is being re-created)
4) 41-42 db upgarde: set vpc_id for the private networks of the existing VPCs
Public IP addresses resource count of an account - number of ip addresses dedicated to an account
plus the number of ip addresses belonging to the system that have been allocated to the account
This feature enables adding of guest ip ranges (public ips) form different subnets.
In order to provide the dhcp service to a different subnet we create an ipalias on the router. This allows the router to listen to the dhcp request from the guest vms and respond accordingly. Every time a vm is deployed in the new subnet we configure an ip alias on the router. Cloudstack uses dnsmasq to provide dhcp service. We need to configure the dnsmasq to issue ips on the new subnets. Added a new class dnsmasqconfigurator which generates the dnsmasq confg file, this file replaces the old config in the router.
The details of the alias ips are stored in db in the nic_ip_alias table. Every time a new subnet is added one of the ip from the subnet is used to configure the ip alias.
I have pushed the code to https://github.com/bvbharatk/cloud-stack/tree/Cloudstack-702 , also rebased the code with master.
I need to test the code for advanced sg enabled network using kvm.
I have added the unit test
Marvin tests are at https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=53e4965
Also accomodated some of the changes suggested by koushik.
corrected the import statements. renamed the IpAlias command to createIpAlias command.
This feature supports only ipv4
Alex Huang
Alena Prokharchyk
Koushik Das
Prasanna Santhanam
Sateesh Chodapuneedi
Edison Su
Min Chen
Bharat Kumar
Marcus Sorensen
Rajesh Battala
Harikrishna Patnala
Kishan Kavala
Nitin Mehta
Chiradeep Vittal
Wei Zhou
Chip Childers
Mice Xia
Prachi Damle
Edison Su
Likitha Shetty
Saksham Srivastava
Devdeep Singh
Anthony Xu
Koushik Das
Sanjay Tripathi
Mike Tutkowski
Vijayendra Bhamidipati
Sheng Yang
Murali Reddy
Dave Brosius
Donal Lafferty
Jayapal
Jessica Wang
Jayapal
Laszlo Hornyak
Wido den Hollander