This would fix regression from recent mvn version changes. Without this
patch users get redirected to error.jsp as jstl-1.2 jar is not installed
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Removes commands.properties file
- Fixes apidocs and marvin to be independent of commands.properties usage
- Removes bundling of commands.properties in deb/rpm packaging
- Removes file references across codebase
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This feature allows root administrators to define new roles and associate API
permissions to them.
A limited form of role-based access control for the CloudStack management server
API is provided through a properties file, commands.properties, embedded in the
WAR distribution. Therefore, customizing API permissions requires unpacking the
distribution and modifying this file consistently on all servers. The old system
also does not permit the specification of additional roles.
FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+Role+Based+API+Access+Checker+for+CloudStack
DB-Backed Dynamic Role Based API Access Checker for CloudStack brings following
changes, features and use-cases:
- Moves the API access definitions from commands.properties to the mgmt server DB
- Allows defining custom roles (such as a read-only ROOT admin) beyond the
current set of four (4) roles
- All roles will resolve to one of the four known roles types (Admin, Resource
Admin, Domain Admin and User) which maintains this association by requiring
all new defined roles to specify a role type.
- Allows changes to roles and API permissions per role at runtime including additions or
removal of roles and/or modifications of permissions, without the need
of restarting management server(s)
Upgrade/installation notes:
- The feature will be enabled by default for new installations, existing
deployments will continue to use the older static role based api access checker
with an option to enable this feature
- During fresh installation or upgrade, the upgrade paths will add four default
roles based on the four default role types
- For ease of migration, at the time of upgrade commands.properties will be used
to add existing set of permissions to the default roles. cloud.account
will have a new role_id column which will be populated based on default roles
as well
Dynamic-roles migration tool: scripts/util/migrate-dynamicroles.py
- Allows admins to migrate to the dynamic role based checker at a future date
- Performs a harder one-way migrate and update
- Migrates rules from existing commands.properties file into db and deprecates it
- Enables an internal hidden switch to enable dynamic role based checker feature
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Validate API arguments based on annotations. Introduces:
- NotNullOrEmpty: for doing null and empty string checks
- PositiveNumber: number > 0 (natural number)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-8901: PrepareTemplate job thread hard-coded to max 8 threads The thread pool was hardcoded to use 8 threads,
com.cloud.template.TemplateManagerImpl.configure(String, Map<String, Object>):
_preloadExecutor = Executors.newFixedThreadPool(8, new NamedThreadFactory("Template-Preloader"));
Added the change to pick threadpool size from configuration.
* pr/880:
CLOUDSTACK-8901: PrepareTemplate job thread hard-coded to max 8 threads
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CLOUDSTACK-9351: Add ids parameter to resource listing API calls## General behaviour
A new parameter is added in each method, its type a list of IDs of the entity, it will be mutually exclusive with id. (Similar to <code>id</code> and <code>ids</code> parameters in <code>listVirtualMachines</code> method)
### API Methods affected
* <code>listTemplates</code>: new parameter **<code>ids</code>**, mutually exclusive with <code>id</code>
* <code>listVolumes</code>: new parameter **<code>ids</code>**, mutually exclusive with <code>id</code>
* <code>listSnapshots</code>: new parameter **<code>ids</code>**, mutually exclusive with <code>id</code>
* <code>listVMSnapshots</code>: new parameter **<code>vmsnapshotids</code>**, mutually exclusive with <code>vmsnapshotid</code>
* pr/1497:
CLOUDSTACK-9351: Add marvin test and add it to travis file
CLOUDSTACK-9351: Add ids parameter to resource listing API calls
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CLOUDSTACK-9350: KVM-HA- Fix CheckOnHost for Local storage- KVM-HA- Fix CheckOnHost for Local storage
- Also skip HA on VMs that are using local storage
* pr/1496:
CLOUDSTACK-9350: KVM-HA- Fix CheckOnHost for Local storage - Also skip HA on VMs that are using local storage
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CID-1338387: Deletion of method endPointSelector.selectHypervisorHostFollowing the discussions and analysis presented on PR #1056 create by @DaanHoogland
This PR is intended to push those changes that were discussed there regarding the of endPointSelector.selectHypervisorHost method.
* pr/1124:
Deletion of method endPointSelector.selectHypervisorHost
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Fixes JSON deserialization of cmdInfo (current process fails with
StringIndexOutOfBoundsException when cmdEventType is the last parameter
in the JSON string.
CLOUDSTACK-8847: ListServiceOfferings is returning incompatible tagged offerings when called with VM idWhen calling listServiceOfferings with VM id as parameter. It is returning incompatible tagged offerings. It should only list all compatible tagged offerings. Compatible means the new service offering should contain all the tags of the existing service offering(Existing offering SUBSET of new offering). If that is the case It should list in the result and can be upgraded to that offering.
* pr/1321:
CLOUDSTACK-8847: ListServiceOfferings is returning incompatible tagged offerings when called with VM id
Signed-off-by: Will Stevens <williamstevens@gmail.com>
[4.7] vmware: improve support for disks- Improve disk chain usage while attaching, migrating disks
- Gets root disk controller based diskDeviceBusName from volume's chain info
* pr/1365:
vmware: improve support for disks
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CLOUDSTACK-9323: Fix cancel host maintenance canFix cancel host maintenance so that if maintenance is cancelled the host come back to normal state gracefully.
Added marvin tests for host maintennace.
* pr/1454:
CLOUDSTACK-9323: Fix Cancel maintenance so that if maintenance is cancelled the host come back to normal state gracefully. Added marvin tests for host maintennace.
Signed-off-by: Will Stevens <williamstevens@gmail.com>
* 4.8:
Removed sleeps and used validateList as requested.
Added required_hardware="false" attr above test_02_root_volume_attach_detach
Modified test_volumes.py to include a hypervisor test for root attach/detach testing
Let hypervisor type KVM and Simulator detach root volumes. Updated test_volumes.py to include a test for detaching and reattaching a root volume from a vm. I also had to update base.py to allow attach_volume to have the parameter deviceid to be passed as needed.
* 4.7:
Removed sleeps and used validateList as requested.
Added required_hardware="false" attr above test_02_root_volume_attach_detach
Modified test_volumes.py to include a hypervisor test for root attach/detach testing
Let hypervisor type KVM and Simulator detach root volumes. Updated test_volumes.py to include a test for detaching and reattaching a root volume from a vm. I also had to update base.py to allow attach_volume to have the parameter deviceid to be passed as needed.
CLOUDSTACK-9349: Enable root disk detach for KVM with new Marvin testsThis PR addresses the KVM detach/attach ROOT disks from VMs (CLOUDSTACK-9349). In short, this allows the KVM Hypervisor, and I added the Simulator as a valid hypervisor for ease of development and testing of marvin, to detach a root volume and the reattach a root volume using the deviceid=0 flag to the attachVolume API. I have also written a marvin integration test that verifies this feature works for both KVM and the Simulator.
Below is the marvin results files of the full marvin test_volumes.py. All tests pass, including the new root detach/attach, on our KVM lab running with the patches in this PR.
[test_volumes_KIR4G3.zip](https://github.com/apache/cloudstack/files/223799/test_volumes_KIR4G3.zip)
* pr/1500:
Removed sleeps and used validateList as requested.
Added required_hardware="false" attr above test_02_root_volume_attach_detach
Modified test_volumes.py to include a hypervisor test for root attach/detach testing
Let hypervisor type KVM and Simulator detach root volumes. Updated test_volumes.py to include a test for detaching and reattaching a root volume from a vm. I also had to update base.py to allow attach_volume to have the parameter deviceid to be passed as needed.
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Log asynchronous responses in the api logCurrently API responses for synchronous calls are logged, but asynchronous call responses are not. This pull request makes a minor modification that logs the response including the JobId of all asynchronous requests.
As an example, here is what a stopVirtualMachine request looked like in the logs:
2016-04-27 10:43:11,084 INFO [a.c.c.a.ApiServer] (catalina-exec-3:ctx-37d9f693 ctx-d2368de3) (logid:3a0fad97) (userId=2 accountId=2 sessionId=AF8B1F726ACB5C3A637B8B300AA218A7) 10.103.0.207 -- GET command=stopVirtualMachine&id=f63b6fcc-e0b0-480f-8f7a-cba329634ba1&forced=false&response=json&_=1461771791036 200
After this modification, here is what the logs look like:
2016-04-27 13:37:11,338 INFO [a.c.c.a.ApiServer] (catalina-exec-6:ctx-915b5c84 ctx-a03152fa) (logid:66249df0) (userId=2 accountId=2 sessionId=9EF127EED5CA6E74797DFE487D980FAF) 10.103.0.207 -- GET command=stopVirtualMachine&id=f63b6fcc-e0b0-480f-8f7a-cba329634ba1&forced=false&response=json&_=1461782231194 200 {"stopvirtualmachineresponse":{"jobid":"5b9f4a9b-eabe-4fa4-849d-3d004bb65634"}}
* pr/1522:
Log responses from asynchronous api commands
Signed-off-by: Will Stevens <williamstevens@gmail.com>
4.9 mvn version safeupgradeonlyUpgrades maven dependencies versions that can be safely upgraded without breaking console-proxy/crypto usage.
Bisected changes from: https://github.com/apache/cloudstack/pull/1397
cc @swill @DaanHoogland
* pr/1510:
maven: fix dependency version support by JDK7
further maven dependency updates from Daan
framework/quota: fix checkstyle issue
maven: Upgrade dependency versions
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Fixed: Error given when creating VPN user in one network if VR for another network is stopped.
Problem: Adding a VPN user to an IP on an isolated network fails with an exception if the end-user has two isolated networks and one of them is not in use (i.e. VR is stopped).
Fix: Presently in the code, all the VR which are not in Running state are forced to throw error. Added a check for the Stopped and Stopping state routers. Configurations will be applied to them when they get restarted.
* pr/826:
Fixed Coverity Issue: Unintentional Integer FLow
Fixed: Error given when creating VPN user in one network if VR for another network is stopped
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CLOUDSTACK-9322: Support for Internal LB fuctionality with Nuage VSP SDN Plugin including Marvin test coverageTask: https://issues.apache.org/jira/browse/CLOUDSTACK-9322
PR contents:
1) UI changes to support LB provider InternalLbVm during VPC offering creation.
2) Bug fix for CLOUDSTACK-9320.
3) Nuage VSP SDN Plugin related enhancements for VPC network functionality.
4) Marvin test coverage for Internal LB support on master with Nuage VSP SDN Plugin.
5) Enhancements on our exiting Marvin test code (nuagevsp plugins directory).
6) PEP8 & PyFlakes compliance with our Marvin test code.
Test run:
CloudStack$ nosetests --with-marvin --marvin-config=nuage_ant.cfg test/integration/plugins/nuagevsp/ -a tags=nuagevsp
Test results:
Test user data and password reset functionality with Nuage VSP SDN plugin ... === TestName: test_nuage_UserDataPasswordReset | Status : SUCCESS ===
ok
Test Nuage VSP VPC Offering with different combinations of LB service providers ... === TestName: test_01_nuage_internallb_vpc_Offering | Status : SUCCESS ===
ok
Test Nuage VSP VPC Network Offering with and without Internal LB service ... === TestName: test_02_nuage_internallb_vpc_network_offering | Status : SUCCESS ===
ok
Test Nuage VSP VPC Networks with and without Internal LB service ... === TestName: test_03_nuage_internallb_vpc_networks | Status : SUCCESS ===
ok
Test Nuage VSP VPC Internal LB functionality with different combinations of Internal LB rules ... === TestName: test_04_nuage_internallb_rules | Status : SUCCESS ===
ok
Test Nuage VSP VPC Internal LB functionality by performing (wget) traffic tests within a VPC ... === TestName: test_05_nuage_internallb_traffic | Status : SUCCESS ===
ok
Test Nuage VSP VPC Internal LB functionality with different LB algorithms by performing (wget) traffic tests ... === TestName: test_06_nuage_internallb_algorithms_traffic | Status : SUCCESS ===
ok
Test Nuage VSP VPC Internal LB functionality with restarts of VPC network components by performing (wget) ... === TestName: test_07_nuage_internallb_vpc_network_restarts_traffic | Status : SUCCESS ===
ok
Test Nuage VSP VPC Internal LB functionality with InternalLbVm appliance operations by performing (wget) ... === TestName: test_08_nuage_internallb_appliance_operations_traffic | Status : SUCCESS ===
ok
Test Basic VPC Network Functionality with Nuage VSP SDN plugin ... === TestName: test_nuage_vpc_network | Status : SUCCESS ===
ok
Test Nuage VSP SDN plugin with basic Isolated Network functionality ... === TestName: test_nuage_vsp | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 11 tests in 12094.705s
OK
Test run logs:
[results.txt](https://github.com/apache/cloudstack/files/187587/results.txt)
[runinfo.txt](https://github.com/apache/cloudstack/files/187588/runinfo.txt)
Test config file:
[nuage_ant.txt](https://github.com/apache/cloudstack/files/222711/nuage_ant.txt)
Note: Attached the Marvin config file as .txt instead of .cfg.
PEP8 & PyFlakes Compliance:
CloudStack$ pep8 --max-line-length=150 test/integration/plugins/nuagevsp/*.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/nuageTestCase.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/test_nuage_password_reset.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/test_nuage_vpc_internal_lb.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/test_nuage_vpc_network.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/test_nuage_vsp.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/*.py
#CLOUDSTACK-9322
* pr/1452:
CLOUDSTACK-9322 : Marvin tests for Internal Lb with Nuage VSP
CLOUDSTACK-9320 : InternalLBVM is not getting destroyed when the last Internal Load Balancer rule is removed for the corresponding source IP address
CLOUDSTACK-9322 : Changes to support InternalLbVm with Nuage VSP plugin
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CLOUDSTACK-9130: Make RebootCommand similar to start/stop/migrate agent commands w.r.t. "execute in sequence" flag
RebootCommand now behaves in the same way as start/stop/migrate agent commands w.r.t. to sequential/parallel execution.
* pr/1200:
CLOUDSTACK-9130: Make RebootCommand similar to start/stop/migrate agent commands w.r.t. "execute in sequence" flag RebootCommand now behaves in the same way as start/stop/migrate agent commands w.r.t. to sequential/parallel execution.
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Updated test_volumes.py to include a test for detaching and reattaching a root volume from a vm. I also had to update base.py to allow attach_volume to have the parameter deviceid to be passed as needed.
- Improve disk chain usage while attaching, migrating disks
- Gets root disk controller based diskDeviceBusName from volume's chain info
- Refactor and move VirtualMachineDiskInfo to cloud-utils
- Allows mixing of scsi controller types
- Fixes a NPE case with map passed as null, for example in case of detach volume
command
- Use a osdefault translator that allow use of recent os types added (enums of
which) are not available in the sdk
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Removed unused code from com.cloud.api.ApiServer**Removed \_ from variables names**: private variables with \_ at the beginning is common in C++ but not in Java.
**Removed unused code from ApiServer:**
- com.cloud.api.ApiServer.getPluggableServices(): unused method;
- com.cloud.api.ApiServer.getApiAccessCheckers(): unused method;
**Methods and variables access level reviewed:**
- com.cloud.api.ApiServer.handleAsyncJobPublishEvent(String, String ,Object): this method was private but the annotation @MessageHandler requests public methods, as can be seen in org.apache.cloudstack.framework.messagebus.MessageDispatcher.buildHandlerMethodCache(Class\<?\>), which searches methods with the @MessageHandler annotation and changes
it to be accessible (setAccessible(true)). Thus, there is no reason for handleAsyncJobPublishEvent be a private method and lead some other dev to wrong conclusions about the use of the method;
- Global variables and methods called just by this class (ApiServer) were changed to private.
**Changed variables and methods from static to non-static (if possible):** as some variables/methods are used just by one object of this class, instantiated by Spring, they were changed to non-static.
With that, calls from com.cloud.api.ApiServlet.ApiServlet() that used static methods from ApiServer, were changed from ApiServer.\<staticMethodName\> to \_apiServer.\<methodName\> that refers to the org.apache.cloudstack.api.ApiServerService interface. Thus, methods com.cloud.api.ApiServer.getJSONContentType() and com.cloud.api.ApiServer.isSecureSessionCookieEnabled() had to be added in the interface (org.apache.cloudstack.api.ApiServerService, interface implemented by class ApiServer).
* pr/1263:
The goal of this PR is to review com.cloud.api.ApiServer class, with the following actions:
Signed-off-by: Will Stevens <williamstevens@gmail.com>
following actions:
Removed “_” in beginning of global variables names:
Variables was changed from “_<variablename>” to “<variablename>”, as
this convension (private veriables with “_”) is common in C++ but not in
Java.
Removed unused code from ApiServer:
- com.cloud.api.ApiServer.getPluggableServices():
Unused method.
- com.cloud.api.ApiServer.getApiAccessCheckers():
Unused method.
Methods and variables access level reviewed:
- com.cloud.api.ApiServer.handleAsyncJobPublishEvent(String, String,
Object):
This method was private but the annotation @MessageHandler requests
public methods, as can be seen in
org.apache.cloudstack.framework.messagebus.MessageDispatcher.buildHandlerMethodCache(Class<?>),
which searches methods with the @MessageHandler annotation and changes
it to accessible (“setAccessible(true)”). Thus, there is no reason for
handleAsyncJobPublishEvent be a private method.
- Global variables and methods called just by this class (ApiServer)
were changed to private.
Changed variables and methods from static to non static (if possible):
As some variables/methods are used just by one object of this class
(instantiated by springer), they were changed to non static.
With that, calls from com.cloud.api.ApiServlet.ApiServlet() that used
static methods from ApiServer, was changed from
ApiServer.<staticMethodName> to _apiServer.<methodName> that refers to
the org.apache.cloudstack.api.ApiServerService interface. Thus, methods
com.cloud.api.ApiServer.getJSONContentType() and
com.cloud.api.ApiServer.isSecureSessionCookieEnabled() had to be
included in the interface (org.apache.cloudstack.api.ApiServerService,
interface implemented by class ApiServer).
However, com.cloud.api.ApiServer.isEncodeApiResponse() was keept static,
as its call hierarchy would have to be changed (more than planed for
this PR).
SecurityGroupRulesCmd code cleanupWrote a test and cleaned some duplicate code with the objective to evaluate the jenkins pull request process at builds.a.o
worthwhile to keep, IMHO.
* pr/1287:
SecurityGroupRulesCmd code cleanup review comments handled
deal with PMD warnings
code cleanup
security rules test
remove autogenerated pydev files
Signed-off-by: Koushik Das <koushik@apache.org>
- In case of redundant VPCs, the ACL items are revoked in the first iteration. Since the econd iteration
is needed in order to remove the private network, we have to check if the nic profile is gone before trying
to revoke the ACL items again, which would throw a NPE.
- Some variable extraction in order to ease debugging.
CLOUDSTACK-9333: Exclude clusters from OVF operationsJIRA TICKET: https://issues.apache.org/jira/browse/CLOUDSTACK-9333
## Introduction
In some environments there is a need to exclude certain VMware clusters from performing OVF operations. This operations are part of:
* create template
* create volume snaphsot
* copy template, volume, images from primary storage to secondary storage
* migrate volume
* participate when a template gets cached over to primary storage.
In ESX/ESXi, OVF operations are low priority and bound to a single CPU and most likely get throttled to certain IOPS and network limits.
If the hypervisor chosen for OVF operations is weak or overloaded this results in significantly longer execution of such OVF command and therefore degraded performance of underlying CloudStack API call.
### Proposed solution
It is proposed to add a way to exclude hosts from selected clusters for OVF operations.
To exclude a cluster, would be necessary to insert a record in <code>cluster_details</code> specifying property **vmware.exclude_from_ovf** in this way: (supposing we want to exclude cluster X)
| cluster_id | name| value |
|:-------------:|:-------------:|:-------------:|
|X|vmware.exclude_from_ovf|true|
* pr/1457:
CLOUDSTACK-9333: Exclude clusters for OVF operations
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Add ability to download templates in SwiftThis PR adds the ability to download templates when using Swift as a secondary storage. Uses the "temp_url" feature of Swift so that tempates can be downloaded without authenticaiton.
* pr/1332:
Add ability to download templates in Swift
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CLOUDSTACK-9298: Improve performance of resource retrieval that have tags associated and target volumes, VMs and templatesJIRA TICKET: https://issues.apache.org/jira/browse/CLOUDSTACK-9298
## Description of the problem
When retrieving a large number of resources which have tags associated with, retrieval methods took too long. Our goal is to improve performance of this methods avoiding query the database for each tag, managing that information in memory.
API methods to improve: <code>listTemplates</code>, <code>listVolumes</code>, <code>listVirtualMachines</code>
To achive it, it's necessary to include new columns in <code>template_view</code>, <code>volume_view</code> and <code>user_vm_view</code>:
* tag_account_name
* tag_domain_name
* tag_domain_uuid
* pr/1425:
CLOUDSTACK-9298: Remove user definer from view creations
CLOUDSTACK-9298: Add @MappedSuperClass support for persistence inheritance
CLOUDSTACK-9298: Improve ListTemplatesCmd, ListVolumesCmd and ListVMsCmd performance
Signed-off-by: Will Stevens <williamstevens@gmail.com>
It is now broken into separate methods for more readability and
flexibility. Each zone type (basic, advanced) has its own method for
getting the default network when creating the VM.
CLOUDSTACK-8800 : Improved the listVirtualMachines API call to include memory utilization information for a VMfor xenserver,kvm and for vmware.
* pr/780:
CLOUDSTACK-8800 : Improved the listVirtualMachines API call to include memory utilization information for a VM for xenserver,kvm and for vmware.
Signed-off-by: Kishan Kavala <kishan@apache.org>
Removed unused variables from "NetworkStateListener" classWe removed the following variables from "com.cloud.network.NetworkStateListener"
. UsageEventDao _usageEventDao
. NetworkDao _networkDao
We changed the EventBus s_eventBus variable to private, the constructor not to use those variables and applied this change in classes com.cloud.network.IpAddressManagerImpl and org.apache.cloudstack.engine.orchestration.NetworkOrchestrator
* pr/1261:
Removed unused variables from class NetworkStateListener
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Removal of class AgentBasedStandaloneConsoleProxyManagerThe AgentBasedStandaloneConsoleProxyManager class is neither manually
instatiated anywhere in the code nor via Spring. Further checking in the
interface which the class implements, shows that the use for classes
implementing the EventBus interface relies on instances generated by the
Spring framework, which discards the possibility of the class being
created by EJB. Being the class useless, it was discarded.
* pr/855:
Add Javadoc to AgentBasedStandaloneConsoleProxyManager
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-8858: listVolumes API fails for a particular domain with NPE.CLOUDSTACK-8858: listVolumes API fails for a particular domain with NPE.
Summary: listVolumes API fails when volume associated vm instance has NULL or invalid state. Fix the code to guard this situation since this should not block volume listing.
* pr/830:
CLOUDSTACK-8858: listVolumes API fails for a particular domain with NPE.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* 4.7:
CLOUDSTACK-9254: Make longer names display pretty
CLOUDSTACK-9245 - Deletes ACL items when destroying the VPC or deleting the ACL itself
CLOUDSTACK-9245 - Formatting NetworkACLServiceImpl class
CLOUDSTACK-9245 - Formatting VpcManagerImpl class
CLOUDSTACK-9245 - Formatting NetworkACLManagerImpl class
More VR performance!
* 4.7:
CLOUDSTACK-9237: Create LB Healthcheck issues - button alignment and error message goes outside the window
CLOUDSTACK-9235: Autoscale button is missing in VPC
CLOUDSTACK-9229: Autoscale policy creation failing in VPC due to zoneid missing in createAutoScaleVmProfile
CLOUDSTACK-8860: improve error messages in VM deployment code path.improved the error messages in vm deployment code path. added some more data to the error messages and also fixed some errors using internal ids to use uuids.
* pr/864:
CLOUDSTACK-8860: improve error messages in VM deployment code path.
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9132: API createVolume takes empty string for name parameterSteps to Reproduce:
================
Create a volume using createVolume API where parameter name is empty.
It creates a volume with empty name.
But the name parameter is mandatory.(Issue)
Expected Behaviour:
================
It shouldn't create a volume with an empty name. Error should be returned.
Solution:
=======
Added a condition to check in case of empty string. If the name is an empty string, it generates a random name for the volume. Made the name field optional in UI as well as in API.
* pr/1319:
CLOUDSTACK-9132: API createVolume takes empty string for name parameter
Signed-off-by: Remi Bergsma <github@remi.nl>
* 4.7:
CLOUDSTACK-9222 Prevent cloud.log.1 filling up the disk
Add integration test for restartVPC with cleanup, and Private Gateway enabled.
Nullpointer Exception in NicProfileHelperImpl
NicProfileHelperImpl NullpointerException when ipVO is nullWhen a VPC has a private gateway, and one would like to restart the VPC with **cleanup** it would fail.
This PR adds a NullPointer check and verifies it with an integration test.
```
test_01_vpc_privategw_acl (integration.smoke.test_privategw_acl.TestPrivateGwACL) ... === TestName: test_01_vpc_privategw_acl | Status : SUCCESS ===
ok
test_02_vpc_privategw_static_routes (integration.smoke.test_privategw_acl.TestPrivateGwACL) ... === TestName: test_02_vpc_privategw_static_routes | Status : SUCCESS ===
ok
test_03_vpc_privategw_restart_vpc_cleanup (integration.smoke.test_privategw_acl.TestPrivateGwACL) ... === TestName: test_03_vpc_privategw_restart_vpc_cleanup | Status : SUCCESS ===
ok
test_04_rvpc_privategw_static_routes (integration.smoke.test_privategw_acl.TestPrivateGwACL) ... === TestName: test_04_rvpc_privategw_static_routes | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 4 tests in 2945.055s
OK
```
* pr/1328:
Add integration test for restartVPC with cleanup, and Private Gateway enabled.
Nullpointer Exception in NicProfileHelperImpl
Signed-off-by: Remi Bergsma <github@remi.nl>
* 4.7:
Fix unable to setup more than one Site2Site VPN Connection
FIX S2S VPN rVPC: Check only redundant routers in state MASTER
PEP8 of integration/smoke/test_vpc_vpn
Add S2S VPN test for Redundant VPC
Make integration/smoke/test_vpc_vpn Hypervisor independant
FIX VPN: non-working ipsec commands
[UI] MADNESS
[DB] Add force_encap field to s2s_customer_gateway table
[ROUTER] Add forceencaps field to python router ipsec config method
[TEST] unittest needs rework
[MARVIN] Add forceencap field to VpnCustomerGateway class in marvin base
[CORE] Add Force UDP Encapsulation option to Site2Site VPN
CLOUDSTACK-9186: Root admin cannot see VPC created by Domain admin user
CLOUDSTACK-9192: UpdateVpnCustomerGateway is failing
CLOUDSTACK-6485 prevent ip asignment of private gw iface
CLOUDSTACK-9204 Do not error when staticroute is already gone
make both check lines consistent
CLOUDSTACK-9181 Prevent syntax error in checkrouter.sh
CLOUDSTACK-9202 Bump ssh timeout
[4.7] FIX Site2SiteVPN on redundant VPCThis PR:
- fixes the inability to setup more than one Site2Site VPN connection from a VPC
- fixes starting of Site2Site VPN on redundant VPC
- fixes Site2Site VPN state checking on redundant VPC
- improves the vpc_vpn test to allow multple hypervisors
- adds an integration test for Site2Site VPN on redundant VPC
Tested it on 4.7 single Xen server zone:
command:
```
nosetests --with-marvin --marvin-config=/data/shared/marvin/mct-zone1-xen1.cfg -a tags=advanced,required_hardware=true /tmp/test_vpc_vpn.py
```
results:
```
Test Site 2 Site VPN Across redundant VPCs ... === TestName: test_01_redundant_vpc_site2site_vpn | Status : SUCCESS ===
ok
Test Remote Access VPN in VPC ... === TestName: test_01_vpc_remote_access_vpn | Status : SUCCESS ===
ok
Test Site 2 Site VPN Across VPCs ... === TestName: test_01_vpc_site2site_vpn | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 3 tests in 1490.076s
OK
```
also performed numerous manual inspections of state of VPN connections and connectivity between VPC's
* pr/1276:
Fix unable to setup more than one Site2Site VPN Connection
FIX S2S VPN rVPC: Check only redundant routers in state MASTER
PEP8 of integration/smoke/test_vpc_vpn
Add S2S VPN test for Redundant VPC
Make integration/smoke/test_vpc_vpn Hypervisor independant
FIX VPN: non-working ipsec commands
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-6485 prevent ip asignment of private gw ifacePrevent ipaddress asignment of gateway to gateway-interface on vpc router by setting vpcid to null in network. This was fixed in 4.4 by 1f209ff226, reimplemented for 4.7
* pr/1299:
CLOUDSTACK-6485 prevent ip asignment of private gw iface
Signed-off-by: Remi Bergsma <github@remi.nl>
Added conditions to check if the name is empty or blank.
If it is empty or blank, then it generates a random name.
Made the name field as optional in UI as well as in API.
Added required unit tests.
Prevent ipaddress asignment of gateway to gateway-interface on vpc router by setting vpcid to null in network
Was fixed in 4.4 by 1f209ff226
Reimplemented for 4.7
* 4.6:
[UI] bug fix: Delete added ACL lists is not available for Domain Admin and normal users
CLOUDSTACK-4787: Allow users to select disk controller for VM/template
CLOUDSTACK-4787 Allow selection of scsi controller type in vSphere
[4.6] CLOUDSTACK-4787 - vmware diskcontrollersSame as #1131 (see this for screenshots etc)
* pr/1132:
CLOUDSTACK-4787: Allow users to select disk controller for VM/template
CLOUDSTACK-4787 Allow selection of scsi controller type in vSphere
Signed-off-by: Daan Hoogland <daan@onecht.net>
CLOUDSTACK-4374 - As a Developer I want to have HA enabled for routers that are part or a redundant network or VPCThis PR fixes the issue we were facing with crashed redundant routers not being picked up by the HA monitor due to an IF statement in the code.
* Redundant Routers are not the same as Haigh Available. Having a router HA is actually saying that the given router will be controlled by the High Availability monitor. Hence fix any problem we might face.
* pr/1222:
CLOUDSTACK-4374 - Adds HA capabilities to redundant routers
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9086: ACS allows to create isolated networks with invalide gateway IP address - Fixed and Test cases added
Problem: There was no check for the network and broadcast IP addresses in the case where we provide the gateway and netmask while creating the isolated network. As a result the provided IP gets assigned to the eth0 interface of the VR.
Note: This is in continuation of PR #1125 which I closed.
* pr/1188:
CLOUDSTACK-9086: ACS allows to create isolated networks with invalide gateway IP address - Fixed and Test cases added
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9138 - Adds multiple providers back to VPC implementation - It is need and already allowed/used in the current implementation. For example, the Default [redundant] VPC offerings use two LB providers. If we cannot create offerings with 2 LB providers, the whole internal loadbalancer implementation won't work.
* pr/1215:
CLOUDSTACK-9138 - Adds multiple providers back to VPC implementation
Signed-off-by: Remi Bergsma <github@remi.nl>
[4.7] CLOUDSTACK-9129: list vpc routers by keyword in Infrastructure -> Virtual Routers
and two more changes:
(1) add network name/vpc name in the listRouters response
(2) add network name/vpc id, vpc name in the router details page
* pr/1197:
CLOUDSTACK-9129: list vpc routers by keyword in Infrastructure -> Virtual Routers
Signed-off-by: Remi Bergsma <github@remi.nl>
- Redundant Routers are not the same as Haigh Available.
Having a router HA is actually saying that the given router will be controlled by the High Availability monitor.
Hence fix any problem we might face.
- It is need and already allowed/used in the current implementation. For example, the Default [redundant] VPC offerings use
two LB providers. If we cannot create offerings with 2 LB providers, the whole internal loadbalancer implementation won't work
* 4.6:
CLOUDSTACK-9106 - Makes Enum name compliant with Java code conventions.
CLOUDSTACK-9106 - Adds a test to cover the changes in the applyVpnUsers() method
CLOUDSTACK-9106 - Makes the router commands call more consistent.
CLOUDSTACK-9106 - Enables private gateway tests on Redundant VPCs
CLOUDSTACK-9106 - Refactor the createPrivateNicProfileForGateway() method
CLOUDSTACK-9106 - Reduces the amount of iterations through the routers of a VPC
Add support for not (re)starting server after cloud-setup-management.
Closed PRs that will not be considered for merge:
This closes#1158
This closes#1097
Quota service while allowing for scalability will make sure that the cloud is
not exploited by attacks, careless use and program errors. To address this
problem, we propose to employ a quota-enforcement service that allows resource
usage within certain bounds as defined by policies and available quotas for
various entities. Quota service extends the functionality of usage server to
provide a measurement for the resources used by the accounts and domains using a
common unit referred to as cloud currency in this document. It can be configured
to ensure that your usage won’t exceed the budget allocated to accounts/domain
in cloud currency. It will let user know how much of the cloud resources he is
using. It will help the cloud admins, if they want, to ensure that a user does
not go beyond his allocated quota. Per usage cycle if a account is found to be
exceeding its quota then it is locked. Locking an account means that it will not
be able to initiat e a new resource allocation request, whether it is more
storage or an additional ip. Needless to say quota service as well as any action
on the account is configurable.
Changes from Github code review:
- Added marvin test for quota plugin API
- removed unused commented code
- debug messages in debug enabled check
- checks for nulls, fixed access to member variables and feature
- changes based on PR comments
- unit tests for UsageTypes
- unit tests for all Cmd classes
- unit tests for all service and manager impls
- try-catch-finally or try-with-resource in dao impls for failsafe db switching
- remove dead code
- add missing quota calculation case (regression fixed)
- replace tabs with spaces in pom.xmls
- quota: though default value for quota_calculated is 0, the usage server
makes it null while entering usage entries. Flipping the condition so
as to acocunt for that.
- quotatypes: fix NPE in quota type
- quota framework test fixes
- made statement period configurable
- changed default email templates to reflect the fact that exhausted quota may not result in a locked account
- added quotaUpdateCmd that refreshes quota balances and sends alerts and statements
- report quotaSummary command returns quota balance, quota usage and state for all account
- made UI framework changes to allow for text area input in edit views
- process usage entries that have greater than 0 usage
- orocess quota entries only if tariff is non zero
- if there are credit entries but no balance entry create a dummy balance entry
- remove any credit entries that are before the last balance entry
when displaying balance statement
- on a rerun the last balance is now getting added
FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Quota+Service+-+FS
PR: https://github.com/apache/cloudstack/pull/768
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Changed the NetworkTopologyContext class just to make the private member accessible from the test
- Added a test class to cover the positive scenario of the VpcVirtualRouterElementTest.applyVpnUsers() method.
- Covering when there is either no VPC or no routers.
- Checks the result of a call against the previous result. Either both are true or the method returns false.
- Do not thrown exceptions because some calls are not handling/rethrowing them. It would cause runtime problems.
- When doing a list.addAll(Arrays.asList(String[]{}) will cause problems when trying to cast the list.toArray() into an aray of String
It would only work if instead of calling addAll() I would pass it straight into the constructor:
e.g. List<String> l = new ArrayList(Arrays.asList(new String[]{});
Stirng [] s = (String[]) l.toArray();
But I did not like that implementation because it would require 2 arrays of string and combine them at the end.
- Use the router to retrieve the instance ID
- Check if the VPC is redundant in order to reuse the private gateway address.
- Brings the private gateways interfaces up.
- It was causing problems because Nics were expected to be plugged before they actually exist. Only in rVPC cases.
- Applies ACL items to routers only after the Pvt GW is setup.
CLOUDSTACK-9094: Multiple threads are being used to collect the statsCLOUDSTACK-9094: Multiple threads are being used to collect the stats from the same VR
Same thread is being intialised by two managers, VirtualNetworkApplianceManager and VpcVirtualNetworkApplianceManager
* pr/1140:
CLOUDSTACK-9094: Multiple threads are being used to collect the stats from the same VR
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9025: Fixed can't create usable template from snapshot in Xenserver and Vmwarehttps://issues.apache.org/jira/browse/CLOUDSTACK-9025
Fix also reverts below commit as below solution making assumption about hypervisor which are not applicable in case of XenServer and VmWare
Revert "CLOUDSTACK-8964: Can't create template or volume from snapshot"
This reverts commit ccf5d75cfb.
Testing:
Able to deploy VM successfully from template created from linked clone snapshot on XenServer.
* pr/1176:
CLOUDSTACK-9025: Fixed can't create usable template from snapshot in Xenserver and Vmware
Signed-off-by: Daan Hoogland <daan@onecht.net>
CLOUDSTACK-9047 rename enumsmake enums adhere to best practice naming conventions
* pr/1049:
CLOUDSTACK-9046 rename enums to adhere to naming conventions
CLOUDSTACK-9046 renamed enums in kvm plugin
CLOUDSTACK-9047 use 'State's only with context there are more types called 'State' (or to be called so but now 'state') So remove imports and prepend their enclosing class/context to them.
Signed-off-by: Daan Hoogland <daan@onecht.net>
CLOUDSTACK-9101: fix some issues in resize volume(1) fix issue: volume size is not updated even if the operation succeed
(2) Add ui support for root volume resize
(3) resize on qcow2 type ROOT volume of stopped vm does not really work
see https://issues.apache.org/jira/browse/CLOUDSTACK-9101
* pr/1161:
CLOUDSTACK-9101: resize root volume of stopped vm on KVM
CLOUDSTACK-9101: add UI support for root volume resize
CLOUDSTACK-9101: update volume size after resizevolume
Signed-off-by: Daan Hoogland <daan@onecht.net>
CLOUDSTACK-9051: update nic IP address of stopped vmThis provides the feature to change ip address of NIC on a stopped vm by API and UI.
* pr/1086:
CLOUDSTACK-9051: reprogram network as a part of vm nic ip update
CLOUDSTACK-9051: add unit tests for UpdateVmNicIp
CLOUDSTACK-9051: update nic IP address of stopped vm
Signed-off-by: Daan Hoogland <daan@onecht.net>
Removed the PlannerBase class because it is does not bring contribution to ACS' code.Removed the PlannerBase class because it is does not bring contribution to ACS' code.
We changed com.cloud.deploy.FirstFitPlanner, now it doesnt extends the PlannerBase and implements the com.cloud.deploy.DeploymentPlanner.
We also removed the method com.cloud.deploy.DeploymentPlanner.check(VirtualMachineProfile, DeploymentPlan, DeployDestination, ExcludeList) that was not used anywhere.
Additionally, we removed the _ from FirstFitPlanner's attributes name, in order to have them in a more standard way.
* pr/1108:
Removed PlannerBase empty class
Signed-off-by: Remi Bergsma <github@remi.nl>
Fix also reverts below commit as below solution making assumption about hypervisor which are not applicable
in case of XenServer and VmWare
Revert "CLOUDSTACK-8964: Can't create template or volume from snapshot"
This reverts commit ccf5d75cfb.
* 4.6:
CLOUDSTACK-9075 - Uses the same vlan since it should have been already released
CLOUDSTACK-9075 - Adds VPC static routes test
CLOUDSTACK-9075 - Covers Private GW ACL with Redundant VPCs
CLOUDSTACK-9075 - Add method to get list of Physical Networks per zone
CLOUDSTACK-6276 Removing unused parameter in integration test for projects
CLOUDSTACK-6276 Removing unused parameter in integration test
CLOUDSTACK-6276 Fixing affinity groups for projects
CLOUDSTACK-6276 Fixing affinity groups for projectsWith some contributions from @resmo and @ustcweizhou.
This closes https://github.com/apache/cloudstack/pull/508
To test manually (need at least 2 hosts):
Create a project
Create an affinity group in that project
Deploy a vm with that affinity group
Deploy a second vm with that affinity group
They should be on different hosts
Ran old and new tests for affinity groups on the simulator
Test create affinity group as admin in project ... === TestName: test_01_admin_create_aff_grp_for_project | Status : SUCCESS ===
ok
Test create affinity group as domain admin for projects ... === TestName: test_02_doadmin_create_aff_grp_for_project | Status : SUCCESS ===
ok
Test create affinity group as user for projects ... === TestName: test_03_user_create_aff_grp_for_project | Status : SUCCESS ===
ok
Test create affinity group that exists (same name) for projects ... === TestName: test_4_user_create_aff_grp_existing_name_for_project | Status : SUCCESS ===
ok
#Delete Affinity Group by id. ... === TestName: test_01_delete_aff_grp_by_id | Status : SUCCESS ===
ok
#Delete Affinity Group by id should fail for user not in project ... === TestName: test_02_delete_aff_grp_by_id_another_user | Status : SUCCESS ===
ok
test DeployVM in anti-affinity groups ... === TestName: test_01_deploy_vm_anti_affinity_group | Status : SUCCESS ===
ok
test DeployVM in anti-affinity groups with more vms than hosts. ... === TestName: test_02_deploy_vm_anti_affinity_group_fail_on_not_enough_hosts | Status : SUCCESS ===
ok
List affinity group for a vm for projects ... === TestName: test_01_list_aff_grps_for_vm | Status : SUCCESS ===
ok
List multiple affinity groups associated with a vm for projects ... === TestName: test_02_list_multiple_aff_grps_for_vm | Status : SUCCESS ===
ok
List affinity groups by id for projects ... === TestName: test_03_list_aff_grps_by_id | Status : SUCCESS ===
ok
List Affinity Groups by name for projects ... === TestName: test_04_list_aff_grps_by_name | Status : SUCCESS ===
ok
List Affinity Groups by non-existing id for projects ... === TestName: test_05_list_aff_grps_by_non_existing_id | Status : SUCCESS ===
ok
List Affinity Groups by non-existing name for projects ... === TestName: test_06_list_aff_grps_by_non_existing_name | Status : SUCCESS ===
ok
List affinity group should list all for a vms associated with that group for projects ... === TestName: test_07_list_all_vms_in_aff_grp | Status : SUCCESS ===
ok
Update the list of affinityGroups by using affinity groupids ... === TestName: test_01_update_aff_grp_by_ids | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 16 tests in 581.706s
OK
Deploy vm as Admin in Affinity Group belonging to regular user (should fail) ... === TestName: test_01_deploy_vm_another_user | Status : SUCCESS ===
ok
Create Affinity Group as admin for regular user ... === TestName: test_02_create_aff_grp_user | Status : SUCCESS ===
ok
List Affinity Groups as admin for all the users ... === TestName: test_03_list_aff_grp_all_users | Status : SUCCESS ===
ok
List Affinity Groups belonging to admin user ... === TestName: test_04_list_all_admin_aff_grp | Status : SUCCESS ===
ok
List Affinity Groups belonging to regular user passing account id and domain id ... === TestName: test_05_list_all_users_aff_grp | Status : SUCCESS ===
ok
List Affinity Groups belonging to regular user passing group id ... === TestName: test_06_list_all_users_aff_grp_by_id | Status : SUCCESS ===
ok
Delete Affinity Group belonging to regular user ... === TestName: test_07_delete_aff_grp_of_other_user | Status : SUCCESS ===
ok
Test create affinity group as admin ... === TestName: test_01_admin_create_aff_grp | Status : SUCCESS ===
ok
Test create affinity group as domain admin ... === TestName: test_02_doadmin_create_aff_grp | Status : SUCCESS ===
ok
Test create affinity group as user ... === TestName: test_03_user_create_aff_grp | Status : SUCCESS ===
ok
Test create affinity group that exists (same name) ... === TestName: test_04_user_create_aff_grp_existing_name | Status : SUCCESS ===
ok
Test create affinity group with existing name but within different account ... === TestName: test_05_create_aff_grp_same_name_diff_acc | Status : SUCCESS ===
ok
Test create affinity group of non-existing type ... === TestName: test_06_create_aff_grp_nonexisting_type | Status : SUCCESS ===
ok
Delete Affinity Group by name ... === TestName: test_01_delete_aff_grp_by_name | Status : SUCCESS ===
ok
Delete Affinity Group as admin for an account ... === TestName: test_02_delete_aff_grp_for_acc | Status : SUCCESS ===
ok
Delete Affinity Group which has vms in it ... === TestName: test_03_delete_aff_grp_with_vms | Status : SUCCESS ===
ok
Delete Affinity Group with id which does not belong to this user ... === TestName: test_05_delete_aff_grp_id | Status : SUCCESS ===
ok
Delete Affinity Group by name which does not belong to this user ... === TestName: test_06_delete_aff_grp_name | Status : SUCCESS ===
ok
Delete Affinity Group by id. ... === TestName: test_08_delete_aff_grp_by_id | Status : SUCCESS ===
ok
Root admin should be able to delete affinity group of other users ... === TestName: test_09_delete_aff_grp_root_admin | Status : SUCCESS ===
ok
Deploy VM without affinity group ... === TestName: test_01_deploy_vm_without_aff_grp | Status : SUCCESS ===
ok
Deploy VM by aff grp name ... === TestName: test_02_deploy_vm_by_aff_grp_name | Status : SUCCESS ===
ok
Deploy VM by aff grp id ... === TestName: test_03_deploy_vm_by_aff_grp_id | Status : SUCCESS ===
ok
test DeployVM in anti-affinity groups ... === TestName: test_04_deploy_vm_anti_affinity_group | Status : SUCCESS ===
ok
Deploy vms by affinity group id ... === TestName: test_05_deploy_vm_by_id | Status : SUCCESS ===
ok
Deploy vm in affinity group of another user by name ... === TestName: test_06_deploy_vm_aff_grp_of_other_user_by_name | Status : SUCCESS ===
ok
Deploy vm in affinity group of another user by id ... === TestName: test_07_deploy_vm_aff_grp_of_other_user_by_id | Status : SUCCESS ===
ok
Deploy vm in multiple affinity groups ... === TestName: test_08_deploy_vm_multiple_aff_grps | Status : SUCCESS ===
ok
Deploy multiple vms in multiple affinity groups ... === TestName: test_09_deploy_vm_multiple_aff_grps | Status : SUCCESS ===
ok
Deploy VM by aff grp name and id ... === TestName: test_10_deploy_vm_by_aff_grp_name_and_id | Status : SUCCESS ===
ok
List affinity group for a vm ... === TestName: test_01_list_aff_grps_for_vm | Status : SUCCESS ===
ok
List multiple affinity groups associated with a vm ... === TestName: test_02_list_multiple_aff_grps_for_vm | Status : SUCCESS ===
ok
List affinity groups by id ... === TestName: test_03_list_aff_grps_by_id | Status : SUCCESS ===
ok
List Affinity Groups by name ... === TestName: test_04_list_aff_grps_by_name | Status : SUCCESS ===
ok
List Affinity Groups by non-existing id ... === TestName: test_05_list_aff_grps_by_non_existing_id | Status : SUCCESS ===
ok
List Affinity Groups by non-existing name ... === TestName: test_06_list_aff_grps_by_non_existing_name | Status : SUCCESS ===
ok
List affinity group should list all for a vms associated with that group ... === TestName: test_07_list_all_vms_in_aff_grp | Status : SUCCESS ===
ok
Update the list of affinityGroups by using affinity groupids ... === TestName: test_01_update_aff_grp_by_ids | Status : SUCCESS ===
ok
Update the list of affinityGroups by using affinity groupnames ... === TestName: test_02_update_aff_grp_by_names | Status : SUCCESS ===
ok
Update the list of affinityGroups for vm which is not associated ... === TestName: test_03_update_aff_grp_for_vm_with_no_aff_grp | Status : SUCCESS ===
ok
Update the list of Affinity Groups to empty list ... SKIP: Skip - Failing - work in progress
Update the list of Affinity Groups on running vm ... === TestName: test_05_update_aff_grp_on_running_vm | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 42 tests in 976.432s
OK (SKIP=1)
* pr/1134:
CLOUDSTACK-6276 Removing unused parameter in integration test for projects
CLOUDSTACK-6276 Removing unused parameter in integration test
CLOUDSTACK-6276 Fixing affinity groups for projects
Signed-off-by: Remi Bergsma <github@remi.nl>
Merge 4.6there is one (simple) conflict so for formailty's sake I make this a pull request.
@utscweizhou en @remibergsma please have a look/test-run
* pr/1155:
CLOUDSTACK-9022: move storage.cleanup related global configurations to StorageManager
CLOUDSTACK-9022: keep Destroyed volumes for sometime
Signed-off-by: Remi Bergsma <github@remi.nl>
* 4.6:
CLOUDSTACK-9022: move storage.cleanup related global configurations to StorageManager
CLOUDSTACK-9022: keep Destroyed volumes for sometime
Conflicts:
server/src/com/cloud/storage/StorageManagerImpl.java
CLOUDSTACK-8988: Removed unused adapters from async-job-component.xml.I was looking the file /cloud-server/test/async-job-component.xml, and I found an adapter configuration that seems to have no use. The reason for that is explained as follows.
The adapter configuration is the following:
<*adapters key="com.cloud.agent.manager.allocator.StorageAllocator"*>
<*adapter name="Storage"
class="com.cloud.agent.manager.allocator.impl.FirstFitStorageAllocator"*>
<*param name="storage.overprovisioning.factor"*>2<*/param*>
<*/adapter*>
<*adapter name="
class="com.cloud.agent.manager.allocator.impl.RandomStoragePoolAllocator"*>
<*param name="storage.overprovisioning.factor"*>2<*/param*>
<*/adapter*>
<*/adapters*>
class="com.cloud.agent.manager.allocator.impl.FirstFitStorageAllocator"
The class "com.cloud.agent.manager.allocator.impl.FirstFitStorageAllocator" does not exist. The only reference for it is found in the following file:
- /cloud-server/test/async-job-component.xml
Therefore, we can conclude that there is no need for this line at that file.
class="com.cloud.agent.manager.allocator.impl.RandomStoragePoolAllocator"
Additionally, the class RandomStoragePoolAllocator.java is never used. The only reference is found in the following file:
- /cloud-server/test/async-job-component.xml
I found a project called cloud-plugin-storage-allocator-random. This project has only one package that contains only one class, which is the RandomStoragePoolAllocator.java. Despite the names that are the same, the class in cloud-plugin-storage-allocator-random project and the class referenced in - /cloud-server/test/async-job-component.xml have different packages. Therefore, I removed that configuration from async-job-component.xml and the project that contains only the RandomStoragePoolAllocator class that is never used.
Consequently, I had to remove the following lines from the */cloud-client-ui/pom.xml*:
<*dependency*>
<*groupId*>org.apache.cloudstack<*/groupId*>
<*artifactId*>cloud-plugin-storage-allocator-random<*/artifactId*>
<*version*>${project.version}<*/version*>
<*/dependency*>
Furthermore, after I removed that configuration I noticed that there is no such class StorageAllocator.java. However, it appears that exists test for it, like the following classes:
- StorageAllocatorTestConfiguration.java
- StorageAllocatorTest.java.
I am not sure if these classes are tests for the class StorageAllocator.java and for the possible configuration I have just removed. If they are, we can remove both classes. Can someone help me on checking that?
* pr/943:
fixed references in async-job-component.xml for storage allocators
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8868: use PasswordGenerator.generateRandomPassword() to generate systemvm passwordsgenerateRandomPassword() is supposed to create root user passwords. Right now it is only used on the guest VMs. The format of the passwords it creates are of the form "random 3-character string with a lowercase character, uppercase character, and a digit" + random n-character string with only lowercase characters".
For whatever reason it was that we use generateRandomPassword() for guest VM root user passwords(maybe more secure?) we should use the same function for system VM root user passwords.
Tests:
manually tested that password is generated in proper format and am able to login to cpvm with the new password. ex: zD2ztm, tR8snbwhq
```
$ mvn -pl server test -Dtest=ConfigurationServerImplTest#testUpdateSystemvmPassword
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.cloud.server.ConfigurationServerImplTest
log4j:WARN No appenders could be found for logger (com.cloud.utils.crypt.EncryptionSecretKeyChecker).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.487 sec - in com.cloud.server.ConfigurationServerImplTest
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7.781 s
[INFO] Finished at: 2015-09-16T14:17:07+05:30
[INFO] Final Memory: 60M/466M
[INFO] ------------------------------------------------------------------------
```
* pr/841:
CLOUDSTACK-8868: change the default vm.password.length to 10
CLOUDSTACK-8868: use same method to generate passwords for system/guest vms
removed commented code
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9080: Resource limits for Primary arent respected during attachprimary store resource limit check is not performed while attaching a
volume to a vm. Added them same.
Also added a marvin test case to verify the same.
Testing:
BEFORE
No error is shown in UI when trying to attach a volume even after reaching the resource limits.
```
mysql> select * from resource_limit where type="primary_storage";
+----+-----------+------------+-----------------+-------------+
| id | domain_id | account_id | type | max |
+----+-----------+------------+-----------------+-------------+
| 10 | NULL | 4 | primary_storage | 21474836480 |
+----+-----------+------------+-----------------+-------------+
1 row in set (0.00 sec)
mysql> select * from resource_count where account_id=4 and type='primary_storage';
+----+------------+-----------+-----------------+-------------+
| id | account_id | domain_id | type | count |
+----+------------+-----------+-----------------+-------------+
| 63 | 4 | NULL | primary_storage | 48318382080 |
+----+------------+-----------+-----------------+-------------+
1 row in set (0.00 sec)
```
AFTER
Following error message is shown in UI and the volume is not attached
The resource limits stays the same
```
mysql> select * from resource_limit where type="primary_storage";
+----+-----------+------------+-----------------+-------------+
| id | domain_id | account_id | type | max |
+----+-----------+------------+-----------------+-------------+
| 10 | NULL | 4 | primary_storage | 21474836480 |
+----+-----------+------------+-----------------+-------------+
1 row in set (0.01 sec)
mysql> select * from resource_count where account_id=4 and type='primary_storage';
+----+------------+-----------+-----------------+-------------+
| id | account_id | domain_id | type | count |
+----+------------+-----------+-----------------+-------------+
| 63 | 4 | NULL | primary_storage | 48318382080 |
+----+------------+-----------+-----------------+-------------+
1 row in set (0.00 sec)
```
Marvin test: nosetests --with-marvin --marvin-config=setup/dev/advanced.cfg --zone=xen-zone0 --hypervisor=xenserver test/integration/component/test_ps_resource_limits_volume.py
before the change
```
# do ... === TestName: test_attach_volume_exceeding_primary_limits | Status : FAILED ===
AssertionError: Resource count 23 should match with the expected resource count 22\n
```
After the change
```
# do ... === TestName: test_attach_volume_exceeding_primary_limits | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 1 test in 1178.354s
OK
```
* pr/1107:
CLOUDSTACK-9080: Resource limits for Primary arent respected during attach.
Signed-off-by: Remi Bergsma <github@remi.nl>
The PlannerBase is an empty class, and does not bring contribution to
ACS code.
We changed com.cloud.deploy.FirstFitPlanner to extend the AdapterBase
and implements the com.cloud.deploy.DeploymentPlanner.
We also removed the method
com.cloud.deploy.DeploymentPlanner.check(VirtualMachineProfile,
DeploymentPlan, DeployDestination, ExcludeList) that was not used
anywhere.
Additionally, we removed the “_” from FirstFitPlanner's attributes name,
in order to have them in a more standard way.
CLOUDSTACK-8866 : restart.retry.interval is being used instead of migrate.retry.interval during host maintenanceThe frequency at which Cloudstack tries to migrate the VMs is currently controlled by the global parameter "restart.retry.interval" which has a default value of 600 seconds or 10 minutes.This has to be changed to use "migrate.retry.interval" which by default is 120 seconds or 2 minutes . Cloudstack uses restart.retry.interval for all operations-migrate,restart,stop,destroy.
* pr/834:
CLOUDSTACK-8866 : restart.retry.interval is being used instead of migrate.retry.interval during host maintenance
Signed-off-by: Daan Hoogland <daan@onecht.net>
* 4.6:
CLOUDSTACK-9020: UI enhancements from metrics view
CLOUDSTACK-9064: The users should be able to create multiple Guest Shared Networks in same Vlan ID, same Physical Network and same network, just with a different IP ranges.
CLOUDSTACK-9078: Gave scripts executable permissions.
CLOUDSTACK-9076: Changed ownership of directory /var/lib/cloudstack to cloud.
Cannot list vlanipranges by keyword
[4.6] Cannot list vlanipranges by keywordBefore change:
cloudmonkey> list vlanipranges keyword=118
: Caught: com.mysql.jdbc.JDBC4PreparedStatement@18f36b6e: SELECT vlan.id, vlan.vlan_id, vlan.vlan_gateway, vlan.vlan_netmask, vlan.ip6_gateway, vlan.ip6_cidr, vlan.data_center_id, vlan.description, vlan.ip6_range, vlan.network_id, vlan.physical_network_id, vlan.vlan_type, vlan.uuid, vlan.removed, vlan.created FROM vlan WHERE ( OR vlan.description LIKE ** NOT SPECIFIED ** ) AND vlan.removed IS NULL ORDER BY vlan.id ASC LIMIT 0, 500
After change:
cloudmonkey> list vlanipranges keyword='118'
count = 1
vlaniprange:
id = 0d80fd9c-cd6b-4f99-96c6-261420e75f58
account = system
domain = ROOT
domainid = 2044762d-c4a5-11e3-8379-005056ac4490
......
* pr/1085:
Cannot list vlanipranges by keyword
Signed-off-by: Remi Bergsma <github@remi.nl>
[4.7] CLOUDSTACK-8958: add dedicated ips to domain (account for now)For now, we dedicate ip pool to account, however, other accounts in the same domain cannot fetch the ip from this ip pool.
By dedicating ip pool to domain, accounts in the domain can fetch the public ip from same ip pool.
* pr/1007:
CLOUDSTACK-8958: throw an exception if project account cannot be found
CLOUDSTACK-8958: add dedicated ips to domain (account for now)
Signed-off-by: Remi Bergsma <github@remi.nl>
Removal of DefaultUserAuthenticator empty class.The DefaultUserAuthenticator is an empty class, extending from the
AdapterBase and implementing the UserAuthenticator interface. The class
is not being used as a marker and it's sole use is to be extended by
other UserAuthenticators. Noticing that the class had no purpose, I
removed it and made it's children extend from it's superclass and
implement it's interface instead. Also, I removed the @Local tags from
those classes, since EJB is not used anymore.
* pr/1100:
Removal of DefaultUserAuthenticator empty class
Signed-off-by: Remi Bergsma <github@remi.nl>
* 4.6:
Fix event UUIDS missing on event bus
Add select template dropdown when reinstall VM
CLOUDSTACK-9068: Listing Port Forwarding Rules take too much time to load
The DefaultUserAuthenticator is an empty class, extending the
AdapterBase and implementing the UserAuthenticator interface. The class
is only being used as a marker and it's sole use is to be extended by
other UserAuthenticators. Noticing that the class had no purpose, I
removed it and made it's children extend from it's superclass and
implement it's interface instead. Also, I removed the @Local tags from
those classes, since EJB is not used anymore.
- Adds new controller types in the UI, for selecting root disk controller while
registering templates
- Fixes bug to not override disk controller type if provided in the details (either
vm details or from template details)
(cherry picked from commit c7d67628b3)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
commit #7
So far only 1 controller (scsi or ide) is supported in Cloudstack for ide or
scsi, this is existing limitation. Added support for 2nd IDE controller. Support adding IDE
virtual disk to VM. Also added check if VM is running as IDE virtual disk cannot be attached
to VM if VM is runnning.If user detaches a virtual disk on lower unit number of controller,
then subsequent attach operation should find free unit number on the controller and attach
the virtual disk there.
commit #6
Let the controllers of existing VMs continue without flip, current busInfo retrieved from
chain_info field of volume record from database would be preferred over
controller settings from all configuration settings.
commit #5
Editing global configuration param vmware.root.disk.controller osdefault value results
in loss of previous root disk controller type. Hence root disk's controller type for legacy
VMs is unknow post that modificaiton by user. If VM is stop/start then we could get this
infromation from bus info of existing volume. But if user resets VM and then try to start VM.
The existing bus info would be lost. Hence existing disk info is not available to depend on.
Using lsilogic or generic scsi controller for ROOT disk of legacy VMs if reset.
commit #4
Avoid adding additional (>1) scsi controllers to system vms. While attaching volume to legacy VM
don't use osdefault optoin which applicable only for VM created with the option enabled, use
legacy data disk controller type (lsilogic)
commit #3
If root disk's controller type is scsi and data disk controller type condenses
to any of scsi sub-types then data disk controller type would fall back to root disk controller itself. This
ensures data volumes would be accessible in all cases as controller of root volume would be reliable
and it means VM has the supported controller. It also avoids mix of scsi controller sub-types in a user instance.
Also translating disk controller type scsi to lsilogic.
commit #2
Support auto detection of recommended virtual disk controller type for specific guest OS.
commit #1
Support granual controller types. Add support for controller types in template registration as well.
Fix white spaces.
Removed stale HEAD merge lines
Removed tail of merge lines
Fixed VmwareResource, removing storage commands that moved to VmwareStorageProcessor.
removed stale code of controller that is present in processor
Fixed check style errors.
Fixed injection.
Tested with Linux and windows templates. Unable to run iso based tests due to few bugs in register iso area.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
(cherry picked from commit a4cc987a6f)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The fixing of CLOUDSTACK-8816 introduced a regression that removed the
first class entities in the event bus description property. This is
because everything was changed to use the Class as a key... Everything
but the populateFirstClassEntities method in ActionEventUtils.
primary store resource limit check is not performed while attaching a
volume to a vm. Added them same.
Also added a marvin test case to verify the same.
CLOUDSTACK-8485: listAPIs are taking too long to return results- Removed regex. based search/replace of sensitive data on API response introduced as part of commit b0c6d47347
- Added new response serializer to skip sensitive data from getting logged based on annotation present in resposne object fields
- Added annotation (@LogLevel(Log4jLevel.Off)) to sensitive response object fields
Ran the following tests on simulator:
test_vm_life_cycle.py
Test advanced zone virtual router ... === TestName: test_advZoneVirtualRouter | Status : SUCCESS ===
ok
Test Deploy Virtual Machine ... === TestName: test_deploy_vm | Status : SUCCESS ===
ok
Test Multiple Deploy Virtual Machine ... === TestName: test_deploy_vm_multiple | Status : SUCCESS ===
ok
Test Stop Virtual Machine ... === TestName: test_01_stop_vm | Status : SUCCESS ===
ok
Test Start Virtual Machine ... === TestName: test_02_start_vm | Status : SUCCESS ===
ok
Test Reboot Virtual Machine ... === TestName: test_03_reboot_vm | Status : SUCCESS ===
ok
Test destroy Virtual Machine ... === TestName: test_06_destroy_vm | Status : SUCCESS ===
ok
Test recover Virtual Machine ... === TestName: test_07_restore_vm | Status : SUCCESS ===
ok
Test migrate VM ... === TestName: test_08_migrate_vm | Status : SUCCESS ===
ok
Test destroy(expunge) Virtual Machine ... === TestName: test_09_expunge_vm | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 10 tests in 306.429s
OK
test_volumes.py
Download a Volume attached to a VM ... === TestName: test_03_download_attached_volume | Status : SUCCESS ===
ok
Delete a Volume attached to a VM ... === TestName: test_04_delete_attached_volume | Status : SUCCESS ===
ok
Detach a Volume attached to a VM ... === TestName: test_05_detach_volume | Status : SUCCESS ===
ok
Delete a Volume unattached to an VM ... === TestName: test_09_delete_detached_volume | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 4 tests in 184.132s
OK
test_network.py
Test for delete account ... === TestName: test_delete_account | Status : SUCCESS ===
ok
Test for Associate/Disassociate public IP address for admin account ... === TestName: test_public_ip_admin_account | Status : SUCCESS ===
ok
Test for Associate/Disassociate public IP address for user account ... === TestName: test_public_ip_user_account | Status : SUCCESS ===
ok
Test for release public IP address ... === TestName: test_releaseIP | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 4 tests in 783.726s
OK
test_routers.py
Test router internal advanced zone ... SKIP: Marvin configuration has no host credentials to check router services
Test restart network ... === TestName: test_03_restart_network_cleanup | Status : SUCCESS ===
ok
Test router basic setup ... === TestName: test_05_router_basic | Status : SUCCESS ===
ok
Test router advanced setup ... === TestName: test_06_router_advanced | Status : SUCCESS ===
ok
Test stop router ... === TestName: test_07_stop_router | Status : SUCCESS ===
ok
Test start router ... === TestName: test_08_start_router | Status : SUCCESS ===
ok
Test reboot router ... === TestName: test_09_reboot_router | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 7 tests in 42.958s
OK (SKIP=1)
test_global_settings.py
test update configuration setting at zone level scope ... === TestName: test_UpdateConfigParamWithScope | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 1 test in 0.127s
OK
test_resource_detail.py
Test volume detail ... === TestName: test_01_updatevolumedetail | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 1 test in 11.492s
OK
* pr/1021:
CLOUDSTACK-8485: listAPIs are taking too long to return results - Removed regex. based search/replace of sensitive data on API response introduced as part of commit b0c6d47347 - Added new response serializer to skip sensitive data from getting logged based on annotation present in resposne object fields - Added new parameter 'isSensitive' to @Param for marking a field as sensitive in response objects
Signed-off-by: Remi Bergsma <github@remi.nl>
Removed unnecessary @Local annotations and their respective importsFollowing @rafaelweingartner 's findings in PR #714 that many spring beans contained an @Local annotation, we've decided to remove said annotations and their imports from the ComponentLifecycleBase class and its subclasses seeking a reduction of a few hundred lines of useless code.
I had already opened a pull request for this (#853) but at some point my commit disappeared from the PR entirely, showing no new changes in code, which caused it to be merged automatically (with no changes).
* pr/1102:
Removed unnecessary @Local annotations and their respective imports from the ComponentLifecycleBase class and its subclasses.
Signed-off-by: Remi Bergsma <github@remi.nl>
[4.7] CLOUDSTACK-8951: validation for config param "remote.access.vpn.psk.length"throwing error for value < 8 and value > 256
right now, 8, 256 are hardcoded in the code. They should be moved to a
constant and has to be reused everywhere.
will update with test cases/testing later.
* pr/929:
CLOUDSTACK-8951: validation for "remote.access.vpn.psk.length"
Signed-off-by: Remi Bergsma <github@remi.nl>
update for comment in cancelMaintin on start VRchange comment
//if the instance is of type ssvm, call the ssvm manager
to
//if the instance is of type domain router vm, call the network manager
* pr/1065:
update for comment in cancelMaintin on start VR
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9062: Improve S3 implementation.The S3 implementation is far from finished, this commit focuses on the bases.
- Upgrade AWS SDK to latest version.
- Rewrite S3 Template downloader.
- Rewrite S3Utils utility class.
- Improve addImageStoreS3 API command.
- Split various classes for convenience.
- Various minor improvements and code optimizations.
A side effect of the new AWS SDK is that it, by default, uses the V4 signature. Therefore I added an option to specify the Signer, so it stays compatible with previous versions.
Please review thoroughly, both code inspection and (automated) integration tests. Currently no integration tests are available specifically for S3. Therefore the implementation is needed to be tested manually, for now...
What I tested:
- Greenfield install -> will download latest systemvm template automatically to S3.
- Upload a template/iso
- Download a template/iso
- Restart of management server -> list available templates -> doesn't download them again if available.
* pr/1083:
CLOUDSTACK-9062: Improve S3 implementation.
Signed-off-by: Remi Bergsma <github@remi.nl>
* 4.6:
CLOUDSTACK-9053 security upgrade as per COLLECTIONS-580
CLOUDSTACK-9055: fix NPE in updating Redundant State of VPC networks
CLOUDSTACK-9057 remove old system vm upgrade code
- Removed regex. based search/replace of sensitive data on API response introduced as part of commit b0c6d47347
- Added new response serializer to skip sensitive data from getting logged based on annotation present in resposne object fields
- Added new parameter 'isSensitive' to @Param for marking a field as sensitive in response objects
The S3 implementation is far from finished, this commit focusses on the bases.
- Upgrade AWS SDK to latest version.
- Rewrite S3 Template downloader.
- Rewrite S3Utils utility class.
- Improve addImageStoreS3 API command.
- Split various classes for convenience.
- Various minor improvements and code optimalisations.
A side effect of the new AWS SDK is that it, by default, uses the V4 signature. Therefore I added an option to specify the Signer, so it stays compatible with previous versions.
Before change:
cloudmonkey> list vlanipranges keyword=118
: Caught: com.mysql.jdbc.JDBC4PreparedStatement@18f36b6e: SELECT vlan.id, vlan.vlan_id, vlan.vlan_gateway, vlan.vlan_netmask, vlan.ip6_gateway, vlan.ip6_cidr, vlan.data_center_id, vlan.description, vlan.ip6_range, vlan.network_id, vlan.physical_network_id, vlan.vlan_type, vlan.uuid, vlan.removed, vlan.created FROM vlan WHERE ( OR vlan.description LIKE ** NOT SPECIFIED ** ) AND vlan.removed IS NULL ORDER BY vlan.id ASC LIMIT 0, 500
After change:
cloudmonkey> list vlanipranges keyword='118'
count = 1
vlaniprange:
id = 0d80fd9c-cd6b-4f99-96c6-261420e75f58
account = system
domain = ROOT
domainid = 2044762d-c4a5-11e3-8379-005056ac4490
......
generateRandomPassword() is supposed to create root user passwords.
Right now it is only used on the guest VMs. The format of the passwords
it creates are of the form "random 3-character string with a lowercase
character, uppercase character, and a digit" + random n-character string
with only lowercase characters".
For whatever reason it was that we use generateRandomPassword() for
guest VM root user passwords(maybe more secure?) we should use the same
function for system VM root user passwords.
throwing error for value < 8 and value > 256
right now, 8, 256 are hardcoded in the code. They should be moved to a
constant and has to be reused everywhere.
CLOUDSTACK-9006 - ListTemplates API returns result in inconsistent order when called concurrentlyThe order of templates returned in the response is based on a field called sortkey and by default value for the field is set to 0.
With more than 1000 templates, we tried listing the templates with different page sizes concurrently, and we noticed the results being inconsistent.
Thus we added a secondary order by clause to list templates call on tempZonePair column to make sure the results are consistent.
The addOrderby method of Filter class was also not appending , if we added more orderby clauses.
* pr/1009:
CLOUDSTACK-9006 - ListTemplates API returns result in inconsistent order when called concurrently
CLOUDSTACK-9006 - ListTemplates API returns result in inconsistent order when called concurrently
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8902 Restart Network fails in EIP/ELB zoneThe restart network was failing when using external loadbalencer. The failure was because of a number format exception. When BroadcastDomainType.getValue(guestConfig.getBroadcastUri() is executed this returns a string untagged. We were trying to parse this as long so there was a number pointer exception.
This happens only when the vlan uri is vlan://untagged. in other cases were there is a number instead of untagged (vlan tag) this used to succeed. Although we were trying to convert the number to long we were not really using it. we were converting the number to long and then back to string when creating the IpAddressTo. so I removed this unnecessary conversion in this case for fixing the issue at hand.
I did a manual restart of the network and checked for this number format exception in a EIP/ELB setup.
* pr/898:
CLOUDSTACK-89027 Restart Network fails in EIP/ELB zone
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9016 Fail to create VM instance within VPCbug link https://issues.apache.org/jira/browse/CLOUDSTACK-9016.
CS doe not allocate the ip of the form x.x.x.1 to a guest VM. We seem to incorrectly assume that the first ip in the subnet belongs to the gateway.
* pr/1020:
CLOUDSTACK-9016: Deploy vm with gateway ip address in VPC
CLOUDSTACK-9016 Fail to create VM instance within VPC
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACk-9002: VM deployment is successful even when dhcp entry command fails - Fixed
Reason: The return value of the call to accept() function in the applyRules() function of BasicNetworkTopology.java was not checked for success or failure. As a result even if it fails, exception was not thrown and VM deployment went ahead without any errors.
Fix: Added the necessary checks.
* pr/995:
CLOUDSTACk-9002: VM deployment is successful even when dhcp entry command fails - Fixed
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8940: Wrong value is inserted into nics table netmask field when creating a VM - Fixed
Problem: When creating a VM in shared network with no service, the value of netmask is added in the table in the CIDR format unlike other cases where it is added as normal string in the format xxx.xxx.xxx.xxx. The netmask column in the nics table has a length of 15 chars which gets violated if the CIDR exceeds it(Max CIDR length can be 18).
Fix: Before storing the netmask convert from CIDR to native format.
* pr/916:
CLOUDSTACK-8940: Wrong value is inserted into nics table netmask field when creating a VM - Fixed
Signed-off-by: Remi Bergsma <github@remi.nl>
Fixed: Network Update from RVR offering to Standalone offering failsProblem: Moving a RVR network offering to standalone makes the status of VR's as UNKNOWN and Redundant Router marked with YES.
Fix: The network's isRedundant was not getting updated.
* pr/818:
CLOUDSTACK-8844: Network Update from RVR offering to Standalone offering fails - Fixed
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8793 Enable s2s VPN connection for projects
* pr/879:
CLOUDSTACK-8793 Added project id to create vpn customer gateway, and to the impl of list vpn connections and list vpn customer gateways
Signed-off-by: Remi Bergsma <github@remi.nl>
Pass LbProtocol down to the HAProxyConfiguratorThis will let us specify a new load balancer protocol (tcp-proxy) which enables HAProxy's `send-proxy` functionality.
`send-proxy` / [the PROXY protocol][1] will send the real connection origin IP through to the servers behind HAProxy, without requiring any protocol specific changes (such as HTTP header rewriting).
[1]: http://www.haproxy.org/download/1.5/doc/proxy-protocol.txt
This is also inline with what [Amazon ELB now supports][2].
[2]: http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/enable-proxy-protocol.html
* pr/848:
Pass LbProtocol down to the HAProxyConfigurator
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8962: Dedicated cluster is used for virtual routers that belong to non-dedicated account
Earlier the deployment planner was not handling the case of virtual routers.(In Explicit Dedication)
It was only handling for all instance VMs/user VMs.
Added code for checking the case of Virtual Routers.
* pr/945:
CLOUDSTACK-8962: Dedicated cluster is used for virtual routers that belong to non-dedicated account
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8911: VM start job got stuck in loop looking for suitable host
VM instance creation job get stuck in the loop, when VMs require local storage there are host that reached max guest limit and remain hosts does have storage available. This happens because the hosts that reach the max guest limit were not getting added to the avoid list and hence the cluster.
Verified the fix on my local setup.
Repro Steps:
1. Take an environment with single cluster and 2 hosts.
2. change the max guest limit for the hypervisor such that on one host max guest limit should reach.
3. change thresholds so that other host should not have enough storage. If required create a VM for sufficient bigger disk.
4. Now deploy a VM with local storage.
5. cluster will never be put in the avoid set and job will keep looking for suitable host.
6. once we increase the max guest limit, VM will deploy or will fail if there is a lack of storage.
* pr/895:
CLOUDSTACK-8911: VM start job got stuck in loop looking for suitable host
Signed-off-by: Remi Bergsma <github@remi.nl>
Cloudstack-8816 some of the events do not have resource uuidsThe key objects in the context map are sometimes String and sometimes object. This causes missing uuids when an entity put in the context map with key entity.toString is queried with key entity
Testing:
manually tested by deploying a vm and checked that the created events in rabbitmq now has uuids.
events before and after the change are update at https://issues.apache.org/jira/browse/CLOUDSTACK-8816?focusedCommentId=14805239
unittests
```
$ mvn -pl :cloud-api test -Dtest=CallContextTest
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.cloudstack.context.CallContextTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.152 sec - in org.apache.cloudstack.context.CallContextTest
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.445 s
[INFO] Finished at: 2015-09-18T14:58:53+05:30
[INFO] Final Memory: 55M/448M
[INFO] ------------------------------------------------------------------------
```
* pr/849:
CLOUDSTACK-8816 added missing events
CLOUDSTACK-8816: fixed missing resource uuid in delete network cmd
CLOUDSTACK-8816: fixed missing resource uuid in destroy vm event
Cloudstack-8816: Fixed missing resource uuid in delete snapshot events
CLOUDSTACK-8816: some of the events do not have resource uuids
CLOUDSTACK-8816: some of the events do not have resource uuids
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8917 : Instance tab takes long time to load with 12K Vmsmodified sql that is used for retrieving vm count .
In load test environment listVirtualmachine takes 8-11 sec to load. This environment has around 12k active VMs. Total number of rows is 190K.
Performance bottleneck in listVirtualmachine command is fetching the count and distinct vms.
{noformat}
// search vm details by ids
Pair<List<UserVmJoinVO>, Integer> uniqueVmPair = _userVmJoinDao.searchAndCount(sc, searchFilter);
Integer count = uniqueVmPair.second();
{noformat}
This takes 95% of the total time.
To fetch the count and distinct vms we are using below sqls.
Query 1:
{noformat}
SELECT DISTINCT(user_vm_view.id) FROM user_vm_view WHERE user_vm_view.account_type != 5 AND user_vm_view.display_vm = 1 AND user_vm_view.removed IS NULL ORDER BY user_vm_view.id ASC LIMIT 0, 20
{noformat}
Query 2:
select count(distinct id) from user_vm_view WHERE user_vm_view.account_type != 5 AND user_vm_view.display_vm = 1 AND user_vm_view.removed IS NULL
Query 2 is a problematic query.
If we rewrite the query as mentioned below then it will be ~2x faster.
select count(*) from (select distinct id from user_vm_view WHERE user_vm_view.account_type != 5 AND user_vm_view.display_vm = 1 AND user_vm_view.removed IS NULL) as temp;
Mysql Test result:
With 134 active Vms (total rows 349)
mysql> select count(*) from vm_instance;
+----------+
| count(*) |
+----------+
| 349 |
+----------+
1 row in set (0.00 sec)
mysql> select count(*) from user_vm_view;
+----------+
| count(*) |
+----------+
| 135 |
+----------+
1 row in set (0.02 sec)
mysql> select count(distinct id) from user_vm_view WHERE user_vm_view.account_type != 5 AND user_vm_view.display_vm = 1 AND user_vm_view.removed IS NULL;
+--------------------+
| count(distinct id) |
+--------------------+
| 134 |
+--------------------+
1 row in set (0.02 sec)
mysql> select count(*) from (select distinct id from user_vm_view WHERE user_vm_view.account_type != 5 AND user_vm_view.display_vm = 1 AND user_vm_view.removed IS NULL) as temp;
+----------+
| count(*) |
+----------+
| 134 |
+----------+
1 row in set (0.01 sec)
With 14326 active Vms (total rows 195660)
mysql> select count(*) from vm_instance;
+----------+
| count(*) |
+----------+
| 195660 |
+----------+
1 row in set (0.04 sec)
mysql> select count(*) from user_vm_view;
+----------+
| count(*) |
+----------+
| 41313 |
+----------+
1 row in set (4.55 sec)
mysql> select count(distinct id) from user_vm_view WHERE user_vm_view.account_type != 5 AND user_vm_view.display_vm = 1 AND user_vm_view.removed IS NULL;
+--------------------+
| count(distinct id) |
+--------------------+
| 14326 |
+--------------------+
1 row in set (7.39 sec)
mysql> select count(*) from (select distinct id from user_vm_view WHERE user_vm_view.account_type != 5 AND user_vm_view.display_vm = 1 AND user_vm_view.removed IS NULL) as temp;
+----------+
| count(*) |
+----------+
| 14326 |
+----------+
1 row in set (2.08 sec)
UI test Results:
Before:
After
* pr/894:
CLOUDSTACK-8917 : Instance tab takes long time to load with 12K active VM (total vms: 190K)
Signed-off-by: Rajani Karuturi <rajani.karuturi@citrix.com>
Primary Storage count for an account does not decrease when a Data Disk
is deleted belonging to the account unless the VM to which volume
belonged is destroyed
The resource counts are updated even before the disk is actually deleted
resulting in the same value.
Moved the resource counts updation to after the expunge operation as
thats when the disk is actually deleted.
all the tests in test/integration/component/test_ps_limits.py now pass
CLOUDSTACK-8964: Can't create template or volume from snapshot on KVM
* pr/954:
CLOUDSTACK-8964: Can't create template or volume from snapshot
Signed-off-by: Remi Bergsma <github@remi.nl>
uuid is missing in the first event of VM create as the entity is just
created and never put in the Context.
Added the entity uuid to context on successful creation.
This issue happens on KVM.
Normally the SSVM will process the CopyCommand from snapshot to template.
However, Ovm3HypervisorGuru chooses a KVM hypervisor to process the CopyCommand.
This is obviously wrong.
The labeling was broken. Only labels assigned at zone creation
were used, changing labels was not working. Tested with changing
a label and checking it.
As a bonus fixed the consistency of KVM in Dutch compared to other
traffic labels in dutch and copied in the OVM3 translated label
in other languages.
Earlier the deployment planner was not handling the case of virtual routers.(In Explicit Dedication)
It was only handling for all instance VMs/user VMs.
Added code for checking the case of Virtual Routers.
- The default is Accept and will be changed based on the configuration of the offering.
CLOUDSTACK-8934 - The default egress is set as Deny in the router.
- We had to change it on the Java side in order to make the apply it once the default is defined as allowed on the net offering
CLOUDSTACK-8864: Not able to add TCP port forwarding rule in VPN for specific ports
Setting port forwarding rules for port 500,1701 and 4500 after enabling VPN, gives the error message "The range specified, xxxx, conflicts with rule xxxx which has xxxx." This happens because the rules added for vpn doesn't have a matching condition to allow port forwarding rules.
Added a unit test to verify the detectRulesConflict function of FirewallManagerImpl.
* pr/851:
CLOUDSTACK-8864: Not able to add TCP port forwarding rule in VPN for specific ports
Signed-off-by: Remi Bergsma <github@remi.nl>
Summary: listVolumes API fails when volume associated vm instance has NULL or invalid state. Fix the code to guard this situation since this should not block volume listing.
CLOUDSTACK-8834: Fixed unable to download Template , when in multi zonesWe were listing image stores by zone id which was resulting in listing of only one image store
If in that image store its download state is not DOWNLOADED then download template is failing
* pr/804:
CLOUDSTACK-8834: Fixed unable to download Template , when in multi zones We were listing image stores by zone id which was resulting in listing of only one image store If in that image store its download state is not DOWNLOADED then download template is failing
Signed-off-by: Wido den Hollander <wido@widodh.nl>
Cloudstack 8816 entityuuid missing in some of the eventsIn some of the events generated, entity uuid was missing making it difficult to find the entity. Fixed the same.
Tested it on rabbitmq instance.
There are the events before after the fix:
Before
--------------------------------------------------------------------------------
routing_key: management-server.ActionEvent.ACCOUNT-DELETE.Account.*
exchange: cloudstack-events
message_count: 2
payload:
{"eventDateTime":"2015-09-04 17:59:24 +0530","status":"Scheduled","description":"deleting User test4 (id: 28) and accountId \u003d 28","event":"ACCOUNT.DELETE","Account":"c09e2e81-8edc-4c27-b072-25005b522b63","account":"bd73dc2e-35c0-11e5-b094-d4ae52cb9af0","user":"bd7ea748-35c0-11e5-b094-d4ae52cb9af0"}
payload_bytes: 304
payload_encoding: string
redelivered: False
--------------------------------------------------------------------------------
routing_key: management-server.AsyncJobEvent.complete.Account.*
exchange: cloudstack-events
message_count: 0
payload: {"cmdInfo":"{\"id\":\"9dd3abc2-3f8b-4852-aa60-a74b234acb13\",\"response\":\"json\",\"sessionkey\":\"5ig1ItP2_5v-mgY4cVJbJN5hw_w\",\"ctxDetails\":\"
{\\\"interface com.cloud.user.Account\\\":\\\"9dd3abc2-3f8b-4852-aa60-a74b234acb13\\\"}
\",\"cmdEventType\":\"ACCOUNT.DELETE\",\"expires\":\"2015-09-07T11:11:56+0000\",\"ctxUserId\":\"2\",\"signatureversion\":\"3\",\"httpmethod\":\"GET\",\"uuid\":\"9dd3abc2-3f8b-4852-aa60-a74b234acb13\",\"ctxAccountId\":\"2\",\"ctxStartEventId\":\"447\"}","instanceType":"Account","jobId":"5004989d-0cde-4922-8afa-66bf38b75ea7","status":"SUCCEEDED","processStatus":"0","commandEventType":"ACCOUNT.DELETE","resultCode":"0","command":"org.apache.cloudstack.api.command.admin.account.DeleteAccountCmd","jobResult":"org.apache.cloudstack.api.response.SuccessResponse/null/
{\"success\":true}
","account":"bd73dc2e-35c0-11e5-b094-d4ae52cb9af0","user":"bd7ea748-35c0-11e5-b094-d4ae52cb9af0"}
payload_bytes: 914
payload_encoding: string
redelivered: False
--------------------------------------------------------------------------------
After
--------------------------------------------------------------------------------
routing_key: management-server.ActionEvent.ACCOUNT-DELETE.Account.e5e2db91-414d-484c-99d5-c4e265c14ad8
exchange: cloudstack-events
message_count: 13
payload: {"eventDateTime":"2015-09-07 17:32:26 +0530","status":"Completed","description":"Successfully completed deleting account. Account Id: 45","event":"ACCOUNT.DELETE","entityuuid":"e5e2db91-414d-484c-99d5-c4e265c14ad8","entity":"com.cloud.user.Account","account":"bd73dc2e-35c0-11e5-b094-d4ae52cb9af0","user":"bd7ea748-35c0-11e5-b094-d4ae52cb9af0"}
payload_bytes: 344
payload_encoding: string
redelivered: True
--------------------------------------------------------------------------------
routing_key: management-server.AsyncJobEvent.complete.Account.e5e2db91-414d-484c-99d5-c4e265c14ad8
exchange: cloudstack-events
message_count: 12
payload: {"cmdInfo":"{\"id\":\"e5e2db91-414d-484c-99d5-c4e265c14ad8\",\"response\":\"json\",\"sessionkey\":\"8AJVbn8HIpg5LZ_VaVfSPs_QN2k\",\"ctxDetails\":\"{\\\"interface com.cloud.user.Account\\\":\\\"e5e2db91-414d-484c-99d5-c4e265c14ad8\\\"}\",\"cmdEventType\":\"ACCOUNT.DELETE\",\"expires\":\"2015-09-07T12:17:42+0000\",\"ctxUserId\":\"2\",\"signatureversion\":\"3\",\"httpmethod\":\"GET\",\"uuid\":\"e5e2db91-414d-484c-99d5-c4e265c14ad8\",\"ctxAccountId\":\"2\",\"ctxStartEventId\":\"465\"}","instanceType":"Account","instanceUuid":"e5e2db91-414d-484c-99d5-c4e265c14ad8","jobId":"0bb08486-6d9f-4e9f-bfef-b7463c42e71b","status":"SUCCEEDED","processStatus":"0","commandEventType":"ACCOUNT.DELETE","resultCode":"0","command":"org.apache.cloudstack.api.command.admin.account.DeleteAccountCmd","jobResult":"org.apache.cloudstack.api.response.SuccessResponse/null/{\"success\":true}","account":"bd73dc2e-35c0-11e5-b094-d4ae52cb9af0","user":"bd7ea748-35c0-11e5-b094-d4ae52cb9af0"}
payload_bytes: 968
payload_encoding: string
redelivered: True
--------------------------------------------------------------------------------
* pr/782:
CLOUDSTACK-8816 Systemvm reboot event doesnt have uuids. Fixed the same
CLOUDSTACK-8816: Project UUID is not showing for some of operations in RabbitMQ.
CLOUDSTACK-8816: entity uuid missing in create network event
CLOUDSTACK-8816: instance uuid is missing in events for delete account
CLOUDSTACK-8816 Fixed entityUuid missing in some cases is events
Signed-off-by: Rajani Karuturi <rajani.karuturi@citrix.com>
unittests to verify empty password is not allowed during account/user create
* pr/807:
unittests to verify empty password is not allowed during account create
Signed-off-by: Wido den Hollander <wido@widodh.nl>
We were listing image stores by zone id which was resulting in listing of only one image store
If in that image store its download state is not DOWNLOADED then download template is failing
CLOUDSTACK-8814 - Refactoring the configuration of Routers and VPC routers nicsHi there,
I refactored the configureDefaultNics() method in order to split the implementations for Routers and VPC Routers.
The following tests were executed:
* test_vm_life_cycle
* test_routers
* test_vpc_router_nics
* test_vpc_routers
* test_vpc_offerings
@remibergsma @bhaisaab @koushik-das @miguelaferreira @DaanHoogland @karuturi , could you please have a look/test this PR?
Thanks in advance.
Cheers,
Wilder
* pr/788:
CLOUDSTACK-8814 - Refactoring the configuration of Routers and VPC routers nics
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
context parameters is Map<object,Object>. This has been used
interchangeably with String and class object resulting the param value
not being available in some cases if its put with object and get with
class.forName()
made the put and get consistent by using Object as key everywhere.
CLOUDSTACK-8727: API call listVirtualMachines returns same keypairCurrently the user can register same key with different names. Upon listing the VM's the name which got registered first is being returned and not the actual one. Anyhow this behavior is rare and not good. I have added a UNIQUE constraint on the ssh_keypairs table and also made sure that the previous registered keys(with duplicates) get deleted.
* pr/685:
CLOUDSTACK-8727: API call listVirtualMachines returns same keypair: Corrected and test cases added
Signed-off-by: Rajani Karuturi <rajani.karuturi@citrix.com>
config: doc: add default value 10 in description for vm.network.nic.max.secondary.ipaddresses
value hardcoded by commit 4925b9f6a1
* pr/777:
config: doc: add default value 10 in description for vm.network.nic.max.secondary.ipaddresses
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-8759 - Destroying VPC router results in a new unusable VPC routerHow to reproduce the problem:
1. Stop/Destroy the VPC router
2. Add a virtual machine to one of the VPC tier - it will trigger a VPC router creation
3. Router is created, but the NICs are not configured
How to recover without this fix:
1. Stop/destroy the VPC router and restart the VPC
Side effects: private gateways could be misconfigured.
Root cause:
In the VpcNetworkHelperImpl.configureDefaultNics() method, the guest network nic was added in the map prior to the control and public NICs. The order in the map should not matter, however in the LibvirtComputingResource.createVifs() method, there is a logic that relies on the device index - the array index - in order to create the control nic. I advise a refactor on the data model in order to be able to identify the NIC type instead of relying in the array index.
An integration test was added to cover the fix:
* test_vpc_router_nics.py
Environment:
Management Server running on CentOS 7.1
KVM host running on CentOS 7.1
CloudStack Agent/Common 4.6.0-SNAPSHOT
Executing the test:
```
nosetests --with-marvin --marvin-config=/data/shared/marvin/mct-zone2-kvm2-ISOLATED.cfg -s -a tags=advanced,required_hardware=true component/test_vpc_router_nics.py
```
Remark: during the SSH there might be stack traces on the console due to the connection retry. It takes some time to get the PF rules in place and reach the VMs. So, just let the test run until the end.
```
Test results:
Create a vpc with two networks with two vms in each network ... === TestName: test_01_VPC_nics_after_destroy | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 1 test in 774.020s
OK
/tmp//MarvinLogs/test_vpc_router_nics_VH6E9S/results.txt (END)
```
* pr/773:
CLOUDSTACK-8759 - Fix guets nic allocation
CLOUDSTACK-8759 - Adding a marvin test in order to cover the fix
CLOUDSTACK-8759 - The guest nic has to be added after the control nic
Signed-off-by: Remi Bergsma <github@remi.nl>
Cloudstack:8647 LDAP Trust AD and AutoimportToday, CloudStack can automatically import LDAP users based on the configuration to a domain or an account. However, any new users in LDAP aren't automatically reflected. The admin has to manually import them again.
This feature enables admin to map LDAP group/OU to a CloudStack domain and any changes are reflected in ACS as well.
FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/WIP%3A+LDAP%3A+Trust+AD+and+Auto+Import
testcases output:
```
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running groovy.org.apache.cloudstack.ldap.NoLdapUserMatchingQueryExceptionSpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.216 sec - in groovy.org.apache.cloudstack.ldap.NoLdapUserMatchingQueryExceptionSpec
Running groovy.org.apache.cloudstack.ldap.LdapManagerImplSpec
log4j:WARN No appenders could be found for logger (org.apache.cloudstack.ldap.LdapManagerImpl).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
using type:
using type: null
using type: TEST
using type: TEST TEST
using name:
using name: null
using accountType: -1
using accountType: 1
using accountType: 3
using accountType: 4
using accountType: 5
using accountType: 6
using accountType: 20000
using accountType: -500000
Tests run: 29, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.387 sec - in groovy.org.apache.cloudstack.ldap.LdapManagerImplSpec
Running groovy.org.apache.cloudstack.ldap.LdapListUsersCmdSpec
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.041 sec - in groovy.org.apache.cloudstack.ldap.LdapListUsersCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapAddConfigurationCmdSpec
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.019 sec - in groovy.org.apache.cloudstack.ldap.LdapAddConfigurationCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapUserSpec
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.021 sec - in groovy.org.apache.cloudstack.ldap.LdapUserSpec
Running groovy.org.apache.cloudstack.ldap.LdapAuthenticatorSpec
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.082 sec - in groovy.org.apache.cloudstack.ldap.LdapAuthenticatorSpec
Running groovy.org.apache.cloudstack.ldap.LdapConfigurationVOSpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 sec - in groovy.org.apache.cloudstack.ldap.LdapConfigurationVOSpec
Running groovy.org.apache.cloudstack.ldap.OpenLdapUserManagerSpec
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.094 sec - in groovy.org.apache.cloudstack.ldap.OpenLdapUserManagerSpec
Running groovy.org.apache.cloudstack.ldap.LdapDeleteConfigurationCmdSpec
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 sec - in groovy.org.apache.cloudstack.ldap.LdapDeleteConfigurationCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapUserResponseSpec
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec - in groovy.org.apache.cloudstack.ldap.LdapUserResponseSpec
Running groovy.org.apache.cloudstack.ldap.LdapUserManagerFactorySpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.027 sec - in groovy.org.apache.cloudstack.ldap.LdapUserManagerFactorySpec
Running groovy.org.apache.cloudstack.ldap.ADLdapUserManagerImplSpec
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.012 sec - in groovy.org.apache.cloudstack.ldap.ADLdapUserManagerImplSpec
Running groovy.org.apache.cloudstack.ldap.LdapCreateAccountCmdSpec
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.168 sec - in groovy.org.apache.cloudstack.ldap.LdapCreateAccountCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapImportUsersCmdSpec
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.063 sec - in groovy.org.apache.cloudstack.ldap.LdapImportUsersCmdSpec
Running groovy.org.apache.cloudstack.ldap.LinkDomainToLdapCmdSpec
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.019 sec - in groovy.org.apache.cloudstack.ldap.LinkDomainToLdapCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapSearchUserCmdSpec
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.011 sec - in groovy.org.apache.cloudstack.ldap.LdapSearchUserCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapListConfigurationCmdSpec
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec - in groovy.org.apache.cloudstack.ldap.LdapListConfigurationCmdSpec
Running groovy.org.apache.cloudstack.ldap.NoSuchLdapUserExceptionSpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 sec - in groovy.org.apache.cloudstack.ldap.NoSuchLdapUserExceptionSpec
Running groovy.org.apache.cloudstack.ldap.LdapConfigurationResponseSpec
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.006 sec - in groovy.org.apache.cloudstack.ldap.LdapConfigurationResponseSpec
Running groovy.org.apache.cloudstack.ldap.LdapConfigurationSpec
asserting for provider configuration: openldap
asserting for provider configuration: microsoftad
asserting for provider configuration:
asserting for provider configuration:
asserting for provider configuration: xyz
asserting for provider configuration: MicrosoftAd
asserting for provider configuration: OpenLdap
asserting for provider configuration: MicrosoftAD
Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.053 sec - in groovy.org.apache.cloudstack.ldap.LdapConfigurationSpec
Running groovy.org.apache.cloudstack.ldap.LdapContextFactorySpec
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.099 sec - in groovy.org.apache.cloudstack.ldap.LdapContextFactorySpec
Running groovy.org.apache.cloudstack.ldap.LdapConfigurationDaoImplSpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.027 sec - in groovy.org.apache.cloudstack.ldap.LdapConfigurationDaoImplSpec
Running groovy.org.apache.cloudstack.ldap.LdapUtilsSpec
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec - in groovy.org.apache.cloudstack.ldap.LdapUtilsSpec
Results :
Tests run: 156, Failures: 0, Errors: 0, Skipped: 0
```
* pr/755:
CLOUDSTACK-8647: linkdomaintoldap shouldnt fail when createuseraccount fails
CLOUDSTACK-8647 removed duplicate key in create sql of ldap_trust_map
CLOUDSTACK-8647: string formatting
CLOUDSTACK-8647: updated with review comments
CLOUDSTACK-8647: unittests for LdapAuthenticatorSpec
CLOUDSTACK-8647: formatted LdapAuthenticatorSpec
CLOUDSTACK-8647: UI for trust AD feature
CLOUDSTACK-8647 added unittests for new methods in ldapmanager
CLOUDSTACK-8647 unittests for LinkDomainToLdap api command
CLOUDSTACK-8647: fixed unittests
CLOUDSTACK-8647 support for assigning and admin to linked ldap domain
CLOUDSTACK-8647 added nested group enabled config in ldap
CLOUDSTACK-8647 added account_type to the linkDomainToLdap API
CLOUDSTACK-8647 changed the authentication flow
CLOUDSTACK-8647 added new api linkLdapToDomain
CLOUDSTACK-8647: added cmd and response class for the new api
Signed-off-by: Rajani Karuturi <rajani.karuturi@citrix.com>
- When stopping/detroying a VPC router and creating a new one via the addition os a VM, the networks are no plugged correctly
- The ETH1 ends up with the IP of the guest nic, which remains down. This causes routes problems and the VMs are not reachable via the pub IPs
- The test will create a VPC, add 2 tier, 2 VMs, ACL, PF and SSH into the VM
- Then it will stop the router, destroy the router, add another VM to 1 tier and check that we can reach all the VMs
Guys, can you review it? things need to be discussed:
(1) this supports KVM/QCOW2 only. Anyone want to implement for other Hypervisor/format ?
(2) The original data volume (on primary storage) will be removed.
(3) The script uses the default timeout in libvirtComputingResource. Do we need to add one in global configuration (like copy.volume.wait or backup.snapshot.wait, create.volume.from.snapshot.wait)
(4) In scripts/storage/qcow2/managesnapshot.sh, I use "qemu-img convert -f qcow2 -O qcow2" to copy the snapshot from secondary to primary (hence there is no base image file), instead of "cp -f", this is because convert is faster than cp in my testing.
* pr/732:
CLOUDSTACK-5863: revert volume snapshot for KVM/QCOW2
Signed-off-by: Wei Zhou <w.zhou@tech.leaseweb.com>
CLOUDSTACK-8785: Proper enforcement of retry count (max.retries) for all work type handled by HighAvailability manager
Retry count is properly enforced for all work types in HA manager. Also reorganized some of the code for easy testing.
Signed-off-by: Koushik Das <koushik@apache.org>
Made following fixes in simulator
- Support for ScaleVmCommand/NetworkRulesVmSecondaryIpCommand in resource layer
- Added support for scaling up a running VM in simulator
- Fixed some method names not following convention
In order to test PR #725 using simulator some of these changes are needed.
Based on the way HV check is present in the scale VM API, had to explicitly put simulator related check to allow support. The ideal way would be to remove all these HV specific check from code and made them some configuration (by putting them in hypervisor_capabilities table in DB). But that will be a bigger effort outside the scope of this PR.
Signed-off-by: Koushik Das <koushik@apache.org>
This reverts commit cd7218e241, reversing
changes made to f5a7395cc2.
Reason for Revert:
noredist build failed with the below error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project cloud-plugin-hypervisor-vmware: Compilation failure
[ERROR] /home/jenkins/acs/workspace/build-master-noredist/plugins/hypervisors/vmware/src/com/cloud/hypervisor/guru/VMwareGuru.java:[484,12] error: non-static variable logger cannot be referenced from a static context
[ERROR] -> [Help 1]
even the normal build is broken as reported by @koushik-das on dev list
http://markmail.org/message/nngimssuzkj5gpbz
CLOUDSTACK-8721: Fixed Setting details of VM through API results in removal of all other details except the one passed in API
* pr/673:
CLOUDSTACK-8721: Fixed Setting details of VM through API results in removal of all other details except the one passed in API
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Support for ScaleVmCommand/NetworkRulesVmSecondaryIpCommand in resource layer
- Added support for scaling up a running VM in simulator
- Fixed some method names not following convention
Fix for the NicVO.java regression.Renamed set*() methods to correct naming.
* pr/726:
Fix for the NicVO.java regression.
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8687: Prepare template only on a given storage poolUpdate prepare template api to seed/prepare a template only on a given primary storage. Currently, the prepare template api will seed/prepare a given template on all the primary storage pools in a zone. If however, a user wishes to prepare a template only a particular storage pool, it isn't possible. Updated the api to take storage pool id as an optional parameter. If the pool id is provided then the template is prepared only on the given primary storage pool. Also added unit tests to validate the functionality
* pr/715:
CLOUDSTACK-8687: Unit tests for validating the prepare template functionality. These tests validate that the templates get scheduled for seeding. Additionally, if a template is already seeded, we do not try and seed it again. Tests also validate that templates are seeded to storage pools which are available.
CLOUDSTACK-8687: Update prepare template api to seed/prepare a template only on a given primary storage. Currently, the prepare template api will seed/prepare a given template on all the primary storage pools in a zone. If however, a user wishes to prepare a template only a particular storage pool, it isn't possible. Updated the api to take storage pool id as an optional parameter. If the pool id is provided then the template is prepared only on the given primary storage pool
Signed-off-by: Rajani Karuturi <rajanikaruturi@gmail.com>
Refactored Nic.java for readability.Only changed names and reorganized them! No functional implementation change.
See issue CLOUDSTACK-8736 for ongoing effort to clean up network code.
* pr/707:
Renamed internal variables of NicProfile.java to adhere to the naming convention.
Refactored Nic.java for readability.
Signed-off-by: Rajani Karuturi <rajanikaruturi@gmail.com>
These tests validate that the templates get scheduled for seeding. Additionally,
if a template is already seeded, we do not try and seed it again. Tests also
validate that templates are seeded to storage pools which are available.
only on a given primary storage. Currently, the prepare template api
will seed/prepare a given template on all the primary storage pools in
a zone. If however, a user wishes to prepare a template only a
particular storage pool, it isn't possible. Updated the api to take
storage pool id as an optional parameter. If the pool id is provided
then the template is prepared only on the given primary storage pool
Changed methodnames according to Nic.java refactor.
Fixed NicVO.java due to regression from Nic.java refactor.
Fixed VmWareGuru.java after Nic.java refactor.
See issue CLOUDSTACK-8736 for ongoing effort to clean up network code.
Refactored NicProfile.java for readability.Only changed names and reorganized them! No functional implementation change.
See issue CLOUDSTACK-8736 for ongoing effort to clean up network code.
* pr/705:
Refactored NicProfile.java for readability.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
Cloudstack 8656: do away with more silently ignoring exceptions.a lot of messages added.
some restructuring for test exception assertions and try-with-resource blocks
* pr/654: (29 commits)
CLOUDSTACK-8656: more logging instead of sysout
CLOUDSTACK-8656: use catch block for validation
CLOUDSTACK-8656: class in json specified not found
CLOUDSTACK-8656: removed unused classes
CLOUDSTACK-8656: restructure of tests
CLOUDSTACK-8656: reorganise sychronized block
CLOUDSTACK-8656: restructure tests to ensure exception throwing
CLOUDSTACK-8656: validate the throwing of ServerApiException
CLOUDSTACK-8656: logging ignored exceptions
CLOUDSTACK-8656: try-w-r removes need for empty catch block
CLOUDSTACK-8656: try-w-r instead of clunckey close-except
CLOUDSTACK-8656: deal with empty SQLException catch block by try-w-r
CLOUDSTACK-8656: unnecessary close construct removed
CLOUDSTACK-8656: message about timed buffer logging
CLOUDSTACK-8656: message about invalid number from store
CLOUDSTACK-8656: move cli test tool to separate file
CLOUDSTACK-8656: exception is the rule for some tests
CLOUDSTACK-8656: network related exception logging
CLOUDSTACK-8656: reporting ignored exceptions in server
CLOUDSTACK-8656: log in case we are on a platform not supporting UTF8
...
Signed-off-by: Remi Bergsma <github@remi.nl>
Interface changes related to CLOUDSTACK-8580See issue CLOUDSTACK-8580 and individual commits.
* pr/680:
Made interface changes related to CLOUDSTACK-8580
Signed-off-by: Remi Bergsma <github@remi.nl>
Added responses to ListCapabilities to reflect CLOUDSTACK-8580 changes.
This to add these options to the gui. See issue CLOUDSTACK-8580.
3be14e978a
Removed comments as proposed by Daan Hoogland.
Tests will confirm the behaviour of the newly added response fields of listSecurityGroups.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
This closes#679
See issue CLOUDSTACK-8133 for more information.
Added null check by comment of Koushik Das.
Added brackets by comment of Wido den Hollander.
Removed a call to findById() by comment of Koushik Das.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
Implemented condition that only admin or owner of the template can change its permissions ..... using updateTemplatePermissions API
Consider this scenario :
In a domain, there are three User Accounts UA1, UA2,UA3
A private template is registered by UA1
Through the updateTemplatePermissions API, UA1 gives permission to both UA2 and UA3
Now, UA2, having been shared the template, can remove the permission of UA3(or add permissions to another account).
EXPECTED BEHAVIOR :
UA2 should not be able to to add/remove permissions of other accounts.
* pr/658:
Implemented condition that only admin or owner of the template can change its permissions using updateTemplatePermissions API
Signed-off-by: Remi Bergsma <github@remi.nl>
* pr/547:
CLOUDSTACK-8601. VMFS storage added as local storage can be re-added as shared storage. Fail addition of a VMFS shared storage pool in case it has already been added as local storage in CS.
Signed-off-by: Mike Tutkowski <mike.tutkowski@solidfire.com>
CLOUDSTACK-8704: Schedule restart of router VMs ahead of user VMs as part of HA
VRs are scheduled for HA ahead of user VMs.
Refer to the bug for more details.
* pr/656:
CLOUDSTACK-8704: Schedule restart of router VMs ahead of user VMs as part of HA VRs are scheduled for HA ahead of user VMs
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8711: public_ip type resource count for an account is not decremented upon IP range deletionProblem:
--------------------------
When you add an IP range and associate it to an account then resource count of public_ip will be updated to the range length.
After some time try to delete this range and the resource count of public_up for this account is not descremented and is causing account not to add any more public IPs to it once it reaches the resource limit.
RCA:
----------------
We were not decrement the count while deleting the IP range that was associated to an account.
Fix:
-------------
Up on deletion we are decrementing the resource count for public_up now.
* pr/662:
Bug-Id: CS-27335: public_ip type resource count for an account is not decremented upon IP range deletion
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8709 No out of band migrate alert for non-routers
* pr/666:
CLOUDSTACK-8709 No out of band migrate alert for non-routers
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8714 Restore VM (Re-install VM) with enable.storage.migration set to false fails
* pr/659:
Bug-ID:CS-27160: Restore VM (Re-install VM) with enable.storage.migration set to false fails, later fails to start up VM too
Signed-off-by: Remi Bergsma <github@remi.nl>
Rohit Yadav
Will Stevens
Syed
weingartner
nvazquez
Olivier Lemasle
Maneesha.P
Abhinandan Prateek
Nathan Johnson
Abhinandan Prateek
Daan Hoogland
Kshitij Kansal
SudharmaJain
David Mabry
gabrascher
Koushik Das
Wilder Rodrigues
Miguel Ferreira
jeff
Nick Livens
Kishan Kavala
Remi Bergsma
Nitin Kumar Maharana
Michael Andersen
Boris Schrijver
Nitin Kumar Maharana
pedro-martins
Wei Zhou
Anshul Gangwar
Harikrishna Patnala
Patrick Dube
Sateesh Chodapuneedi
Rajani Karuturi
cirstofolini
Boris Schrijver
Rajani Karuturi
Daan Hoogland
wenwenxiong
Bharat Kumar
ramamurtis
Aaron Brady
Funs Kessen
Sudhansu
Suresh Kumar Anaparti
Wido den Hollander
Ashwin Swaroop
Sanjay Tripathi
Kshitij Kansal
Rene Moser
radu-stefanache
Devdeep Singh
Milamber
Mike Tutkowski
