Commit Graph

2994 Commits

Author SHA1 Message Date
Remi Bergsma cdfcea3da2 Merge release branch 4.6 to master
* 4.6:
  [UI] bug fix: Delete added ACL lists is not available for Domain Admin and normal users
  CLOUDSTACK-4787: Allow users to select disk controller for VM/template
  CLOUDSTACK-4787 Allow selection of scsi controller type in vSphere
2015-12-12 22:29:19 +01:00
Daan Hoogland 8c78f89c91 Merge pull request #1132 from shapeblue/4.6-vmware-diskcontrollers
[4.6] CLOUDSTACK-4787 - vmware diskcontrollersSame as #1131 (see this for screenshots etc)

* pr/1132:
  CLOUDSTACK-4787: Allow users to select disk controller for VM/template
  CLOUDSTACK-4787 Allow selection of scsi controller type in vSphere

Signed-off-by: Daan Hoogland <daan@onecht.net>
2015-12-12 14:38:28 +01:00
Remi Bergsma bdaa60fd6c Merge release branch 4.6 to master
* 4.6:
  Show actual diff in commits after merge
  CLOUDSTACK-9113: skip vm with inconsistent state when getVmStats/getVmDiskStats
2015-12-12 14:03:43 +01:00
Remi Bergsma 65bf2edfa1 Merge pull request #1182 from ustcweizhou/CLOUDSTACK-9113-getVmStats
[4.6] CLOUDSTACK-9113: skip vm with inconsistent state when getVmStats/getVmDiskStatson KVM, if there is a vm has inconsistent state between hypervisor and db, the getVmStat will terminate and return null, all vm stats will not be updated.
we should skip the vm which has inconsistent state, and continue on others.

* pr/1182:
  CLOUDSTACK-9113: skip vm with inconsistent state when getVmStats/getVmDiskStats

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-12 13:58:49 +01:00
Abhinandan Prateek 1e17c3e38c QUOTA: Ensuring that the dates displayed are as per user expectations
When querying db we use start of next day to query quota usage for
    today, but while displaying it to user we still need to show it as
    todays date
2015-12-11 14:53:51 +05:30
Abhinandan Prateek 3e22fbe457 CLOUDSTACK-9131: Create a new API to check if the plugin is enabled.
fixing type
2015-12-10 09:37:54 +05:30
Remi Bergsma e08294a95f Merge release branch 4.6 to master
* 4.6:
  CLOUDSTACK-9106 - Makes Enum name compliant with Java code conventions.
  CLOUDSTACK-9106 - Adds a test to cover the changes in the applyVpnUsers() method
  CLOUDSTACK-9106 - Makes the router commands call more consistent.
  CLOUDSTACK-9106 - Enables private gateway tests on Redundant VPCs
  CLOUDSTACK-9106 - Refactor the createPrivateNicProfileForGateway() method
  CLOUDSTACK-9106 - Reduces the amount of iterations through the routers of a VPC
  Add support for not (re)starting server after cloud-setup-management.

Closed PRs that will not be considered for merge:
This closes #1158
This closes #1097
2015-12-07 21:36:36 +01:00
Remi Bergsma e443a0cb12 Merge pull request #1142 from nlivens/master_nuage
CLOUDSTACK-9095 : Hypervisor changes to support UserData for Nuage VSP

* pr/1142:
  CLOUDSTACK-9095 : Hypervisor changes to support UserData for Nuage VSP

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-07 21:21:48 +01:00
Remi Bergsma 8757daf2d1 Merge pull request #1178 from miguelaferreira/nsx-livelock
Prevent live-lock in NSX API clientThe NSX api client relies on a sequence of responses to identify the need to authenticate and to follow redirects. In order to avoid live-locks, the NSX API client has a counter that will abort the execution after 5 consecutive requests that do not produce a Success (200) response.
When a NSX controller enters a faulty state it can allow authentication requests but deny any other API call. In such cases the NSX API client will consider the denied request a reason to follow a redirect and will enter into a live-lock (because the actual redirection is not happening).
This PR changes the NSX API client to no reset it's counter on a Success response from an authentication request. That is, the counter is only reset if another type of API call yields a Success response.

In addition, this PR also:
* changes the configuration of the license-maven-plugin to ignore files generated by pmd
* moves the NSX marvin test to a plugins folder
* refactors the NSX marvin test to reduce duplication
* adds an extra test case to the NSX marvin test that checks that NSX tunnels are properly created

* pr/1178:
  Test NSX tunnel in guest network
  Refactor test cases to reduce duplication
  Use logger to print debug messages during test
  Move NSX integrationt test to new plugins folder
  Ignore pmd generated files during license check
  Fix NSX rest client to not reset execution counter after a login
  Add test for NSX plugin that simulates a live lock

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-07 19:34:20 +01:00
Abhinandan Prateek 987fcbd441 CLOUDSTACK-8592: Implement Quota service
Quota service while allowing for scalability will make sure that the cloud is
not exploited by attacks, careless use and program errors. To address this
problem, we propose to employ a quota-enforcement service that allows resource
usage within certain bounds as defined by policies and available quotas for
various entities.  Quota service extends the functionality of usage server to
provide a measurement for the resources used by the accounts and domains using a
common unit referred to as cloud currency in this document. It can be configured
to ensure that your usage won’t exceed the budget allocated to accounts/domain
in cloud currency.  It will let user know how much of the cloud resources he is
using. It will help the cloud admins, if they want, to ensure that a user does
not go beyond his allocated quota. Per usage cycle if a account is found to be
exceeding its quota then it is locked. Locking an account means that it will not
be able to initiat e a new resource allocation request, whether it is more
storage or an additional ip. Needless to say quota service as well as any action
on the account is configurable.

Changes from Github code review:

- Added marvin test for quota plugin API
- removed unused commented code
- debug messages in debug enabled check
- checks for nulls, fixed access to member variables and feature
- changes based on PR comments
- unit tests for UsageTypes
- unit tests for all Cmd classes
- unit tests for all service and manager impls
- try-catch-finally or try-with-resource in dao impls for failsafe db switching
- remove dead code
- add missing quota calculation case (regression fixed)
- replace tabs with spaces in pom.xmls
- quota: though default value for quota_calculated is 0, the usage server
  makes it null while entering usage entries. Flipping the condition so
  as to acocunt for that.
- quotatypes: fix NPE in quota type
- quota framework test fixes
- made statement period configurable
- changed default email templates to reflect the fact that exhausted quota may not result in a locked account
- added quotaUpdateCmd that refreshes quota balances and sends alerts and statements
- report quotaSummary command returns quota balance, quota usage and state for all account
- made UI framework changes to allow for text area input in edit views
- process usage entries that have greater than 0 usage
- orocess quota entries only if tariff is non zero
- if there are credit entries but no balance entry create a dummy balance entry
- remove any credit entries that are before the last balance entry
  when displaying balance statement
- on a rerun the last balance is now getting added

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Quota+Service+-+FS
PR: https://github.com/apache/cloudstack/pull/768

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-12-07 23:02:48 +05:30
Wilder Rodrigues 13eb789388 CLOUDSTACK-9106 - Makes the router commands call more consistent.
- Checks the result of a call against the previous result. Either both are true or the method returns false.
   - Do not thrown exceptions because some calls are not handling/rethrowing them. It would cause runtime problems.
   - When doing a list.addAll(Arrays.asList(String[]{}) will cause problems when trying to cast the list.toArray() into an aray of String
     It would only work if instead of calling addAll() I would pass it straight into the constructor:
     e.g. List<String> l = new ArrayList(Arrays.asList(new String[]{});
          Stirng [] s = (String[]) l.toArray();
     But I did not like that implementation because it would require 2 arrays of string and combine them at the end.
2015-12-07 13:40:12 +01:00
Wei Zhou 24edf1c1e4 CLOUDSTACK-9113: skip vm with inconsistent state when getVmStats/getVmDiskStats 2015-12-07 12:15:24 +01:00
Wilder Rodrigues a168a69c3c CLOUDSTACK-9106 - Reduces the amount of iterations through the routers of a VPC
- It was causing problems because Nics were expected to be plugged before they actually exist. Only in rVPC cases.
   - Applies ACL items to routers only after the Pvt GW is setup.
2015-12-07 10:30:17 +01:00
Nick Livens 0957268e92 CLOUDSTACK-9095 : Hypervisor changes to support UserData for Nuage VSP 2015-12-07 10:26:47 +01:00
Rohit Yadav f30fbe9a5c Merge branch '4.6' 2015-12-07 12:44:07 +05:30
Daan Hoogland 0d61941101 Merge pull request #1175 from shapeblue/4.6-rate-limit-testfix
[4.6/master] rate-limit: increase JVM memory and enable fork mode for unit testsEnables forkmode for surefire plugin and increases memory opts for JVM
for rate-limit tests. This tries to fix intermittent Jenkins failures
which look like:

multipleClientsCanAccessWithoutBlocking(org.apache.cloudstack.ratelimit.ApiRateLimitTest):
unable to create new native thread

Previous PR (again master by mistake) -- #1171

* pr/1175:
  rate-limit: increase JVM memory and enable fork mode for unit tests

Signed-off-by: Daan Hoogland <daan@onecht.net>
2015-12-06 20:36:44 +01:00
Daan Hoogland 2c4ea503f9 Merge pull request #1176 from anshul1886/CLOUDSTACK-9025-4.6
CLOUDSTACK-9025: Fixed can't create usable template from snapshot in Xenserver and Vmwarehttps://issues.apache.org/jira/browse/CLOUDSTACK-9025

Fix also reverts below commit as below solution making assumption about hypervisor which are not applicable in case of XenServer and VmWare

Revert "CLOUDSTACK-8964: Can't create template or volume from snapshot"

This reverts commit ccf5d75cfb.

Testing:

Able to deploy VM successfully from template created from linked clone snapshot on XenServer.

* pr/1176:
  CLOUDSTACK-9025: Fixed can't create usable template from snapshot in Xenserver and Vmware

Signed-off-by: Daan Hoogland <daan@onecht.net>
2015-12-06 20:34:34 +01:00
Daan Hoogland 638f1cf091 Merge pull request #1049 from DaanHoogland/CLOUDSTACK-9047
CLOUDSTACK-9047 rename enumsmake enums adhere to best practice naming conventions

* pr/1049:
  CLOUDSTACK-9046 rename enums to adhere to naming conventions
  CLOUDSTACK-9046 renamed enums in kvm plugin
  CLOUDSTACK-9047 use 'State's only with context   there are more types called 'State'   (or to be called so but now 'state')   So remove imports and prepend their enclosing class/context to them.

Signed-off-by: Daan Hoogland <daan@onecht.net>
2015-12-06 20:17:04 +01:00
Remi Bergsma e37842cbd3 Merge pull request #1108 from rafaelweingartner/master-lrg-cs-hackday-004
Removed the PlannerBase class because it is does not bring contribution to ACS' code.Removed the PlannerBase class because it is does not bring contribution to ACS' code.

We changed com.cloud.deploy.FirstFitPlanner, now it doesnt extends the PlannerBase and implements the com.cloud.deploy.DeploymentPlanner.

We also removed the method com.cloud.deploy.DeploymentPlanner.check(VirtualMachineProfile, DeploymentPlan, DeployDestination, ExcludeList) that was not used anywhere.

Additionally, we removed the _ from FirstFitPlanner's attributes name, in order to have them in a more standard way.

* pr/1108:
  Removed PlannerBase empty class

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-05 19:11:21 +01:00
Remi Bergsma 74e33dc5ae Merge pull request #840 from bvbharatk/CLOUDSTACK-8852
CLOUDSTACK-8852 Database shows that management server is UP when it iDatabase shows that management server is UP when it is actually stopped.

 This was happening as the stop method in clusterMnanagerImpl was not getting callled. Added shutdown hooks to all spring sub contexts, this enables spring to call the stop mehtods of the beans when management server is shutting down.

Conflicts:
	framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/web/CloudStackContextLoaderListener.java

* pr/840:
  CLOUDSTACK-8852 Database shows that management server is UP when it is actually stopped from the CCP GUI

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-05 18:47:53 +01:00
Miguel Ferreira cfe14462a3 Fix NSX rest client to not reset execution counter after a login 2015-12-05 15:52:22 +01:00
Miguel Ferreira d248e61a31 Add test for NSX plugin that simulates a live lock 2015-12-05 15:52:22 +01:00
Wei Zhou a90b953dbc CLOUDSTACK-8964: Ovm3HypervisorGuru handle only srcData with HypervisorType is Ovm3 2015-12-04 19:19:04 +01:00
Wei Zhou 69e031d9be Revert "simple change to prevent failure and keep OVM3 snapshots working"
This reverts commit 66fed462b6.
2015-12-04 18:18:28 +01:00
Wei Zhou e0501be87e Revert "CLOUDSTACK-8964 side effect isolation"
This reverts commit fc18d1e8b1.
2015-12-04 18:18:15 +01:00
Anshul Gangwar f8790c3b9e CLOUDSTACK-9025: Fixed can't create usable template from snapshot in Xenserver and Vmware
Fix also reverts below commit as below solution making assumption about hypervisor which are not applicable
in case of XenServer and VmWare

Revert "CLOUDSTACK-8964: Can't create template or volume from snapshot"

This reverts commit ccf5d75cfb.
2015-12-04 14:56:40 +05:30
Rohit Yadav dcc66651c0 rate-limit: increase JVM memory and enable fork mode for unit tests
Enables forkmode for surefire plugin and increases memory opts for JVM
for rate-limit tests. This tries to fix intermittent Jenkins failures
which look like:

multipleClientsCanAccessWithoutBlocking(org.apache.cloudstack.ratelimit.ApiRateLimitTest):
unable to create new native thread

(cherry picked from commit 421bc00cd6)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-12-04 13:51:36 +05:30
Josh Harshman b6c900fdde CLOUDSTACK-9004: Add features to HyperVEnlightenmentFeatureDef
Add function to set vapic, spinlock and retries
Add function to get retry value
Modify toString to output appropriate XML for spinlock value if set

CLOUDSTACK-9004: Add features to HyperVEnlightenmentFeatureDef

Refactored set methods to get rid of code duplication.
Modified unit tests accordingly
2015-12-03 16:49:00 -08:00
Remi Bergsma 7e902cd505 Merge release branch 4.6 to master
* 4.6:
  CLOUDSTACK-9075 - Uses the same vlan since it should have been already released
  CLOUDSTACK-9075 - Adds VPC static routes test
  CLOUDSTACK-9075 - Covers Private GW ACL with Redundant VPCs
  CLOUDSTACK-9075 - Add method to get list of Physical Networks per zone
  CLOUDSTACK-6276 Removing unused parameter in integration test for projects
  CLOUDSTACK-6276 Removing unused parameter in integration test
  CLOUDSTACK-6276 Fixing affinity groups for projects
2015-12-03 20:42:41 +01:00
Remi Bergsma 9a21873c4a Merge pull request #1134 from pdube/CLOUDSTACK-6276
CLOUDSTACK-6276 Fixing affinity groups for projectsWith some contributions from @resmo and @ustcweizhou.
This closes https://github.com/apache/cloudstack/pull/508

To test manually (need at least 2 hosts):
Create a project
Create an affinity group in that project
Deploy a vm with that affinity group
Deploy a second vm with that affinity group
They should be on different hosts

Ran old and new tests for affinity groups on the simulator

Test create affinity group as admin in project ... === TestName: test_01_admin_create_aff_grp_for_project | Status : SUCCESS ===
ok
Test create affinity group as domain admin for projects ... === TestName: test_02_doadmin_create_aff_grp_for_project | Status : SUCCESS ===
ok
Test create affinity group as user for projects ... === TestName: test_03_user_create_aff_grp_for_project | Status : SUCCESS ===
ok
Test create affinity group that exists (same name) for projects ... === TestName: test_4_user_create_aff_grp_existing_name_for_project | Status : SUCCESS ===
ok
#Delete Affinity Group by id. ... === TestName: test_01_delete_aff_grp_by_id | Status : SUCCESS ===
ok
#Delete Affinity Group by id should fail for user not in project ... === TestName: test_02_delete_aff_grp_by_id_another_user | Status : SUCCESS ===
ok
test DeployVM in anti-affinity groups ... === TestName: test_01_deploy_vm_anti_affinity_group | Status : SUCCESS ===
ok
test DeployVM in anti-affinity groups with more vms than hosts. ... === TestName: test_02_deploy_vm_anti_affinity_group_fail_on_not_enough_hosts | Status : SUCCESS ===
ok
List affinity group for a vm for projects ... === TestName: test_01_list_aff_grps_for_vm | Status : SUCCESS ===
ok
List multiple affinity groups associated with a vm for projects ... === TestName: test_02_list_multiple_aff_grps_for_vm | Status : SUCCESS ===
ok
List affinity groups by id for projects ... === TestName: test_03_list_aff_grps_by_id | Status : SUCCESS ===
ok
List Affinity Groups by name for projects ... === TestName: test_04_list_aff_grps_by_name | Status : SUCCESS ===
ok
List Affinity Groups by non-existing id for projects ... === TestName: test_05_list_aff_grps_by_non_existing_id | Status : SUCCESS ===
ok
List Affinity Groups by non-existing name for projects ... === TestName: test_06_list_aff_grps_by_non_existing_name | Status : SUCCESS ===
ok
List affinity group should list all for a vms associated with that group for projects ... === TestName: test_07_list_all_vms_in_aff_grp | Status : SUCCESS ===
ok
Update the list of affinityGroups by using affinity groupids ... === TestName: test_01_update_aff_grp_by_ids | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 16 tests in 581.706s

OK

Deploy vm as Admin in Affinity Group belonging to regular user (should fail) ... === TestName: test_01_deploy_vm_another_user | Status : SUCCESS ===
ok
Create Affinity Group as admin for regular user ... === TestName: test_02_create_aff_grp_user | Status : SUCCESS ===
ok
List Affinity Groups as admin for all the users ... === TestName: test_03_list_aff_grp_all_users | Status : SUCCESS ===
ok
List Affinity Groups belonging to admin user ... === TestName: test_04_list_all_admin_aff_grp | Status : SUCCESS ===
ok
List Affinity Groups belonging to regular user passing account id and domain id ... === TestName: test_05_list_all_users_aff_grp | Status : SUCCESS ===
ok
List Affinity Groups belonging to regular user passing group id ... === TestName: test_06_list_all_users_aff_grp_by_id | Status : SUCCESS ===
ok
Delete Affinity Group belonging to regular user ... === TestName: test_07_delete_aff_grp_of_other_user | Status : SUCCESS ===
ok
Test create affinity group as admin ... === TestName: test_01_admin_create_aff_grp | Status : SUCCESS ===
ok
Test create affinity group as domain admin ... === TestName: test_02_doadmin_create_aff_grp | Status : SUCCESS ===
ok
Test create affinity group as user ... === TestName: test_03_user_create_aff_grp | Status : SUCCESS ===
ok
Test create affinity group that exists (same name) ... === TestName: test_04_user_create_aff_grp_existing_name | Status : SUCCESS ===
ok
Test create affinity group with existing name but within different account ... === TestName: test_05_create_aff_grp_same_name_diff_acc | Status : SUCCESS ===
ok
Test create affinity group of non-existing type ... === TestName: test_06_create_aff_grp_nonexisting_type | Status : SUCCESS ===
ok
Delete Affinity Group by name ... === TestName: test_01_delete_aff_grp_by_name | Status : SUCCESS ===
ok
Delete Affinity Group as admin for an account ... === TestName: test_02_delete_aff_grp_for_acc | Status : SUCCESS ===
ok
Delete Affinity Group which has vms in it ... === TestName: test_03_delete_aff_grp_with_vms | Status : SUCCESS ===
ok
Delete Affinity Group with id which does not belong to this user ... === TestName: test_05_delete_aff_grp_id | Status : SUCCESS ===
ok
Delete Affinity Group by name which does not belong to this user ... === TestName: test_06_delete_aff_grp_name | Status : SUCCESS ===
ok
Delete Affinity Group by id. ... === TestName: test_08_delete_aff_grp_by_id | Status : SUCCESS ===
ok
Root admin should be able to delete affinity group of other users ... === TestName: test_09_delete_aff_grp_root_admin | Status : SUCCESS ===
ok
Deploy VM without affinity group ... === TestName: test_01_deploy_vm_without_aff_grp | Status : SUCCESS ===
ok
Deploy VM by aff grp name ... === TestName: test_02_deploy_vm_by_aff_grp_name | Status : SUCCESS ===
ok
Deploy VM by aff grp id ... === TestName: test_03_deploy_vm_by_aff_grp_id | Status : SUCCESS ===
ok
test DeployVM in anti-affinity groups ... === TestName: test_04_deploy_vm_anti_affinity_group | Status : SUCCESS ===
ok
Deploy vms by affinity group id ... === TestName: test_05_deploy_vm_by_id | Status : SUCCESS ===
ok
Deploy vm in affinity group of another user by name ... === TestName: test_06_deploy_vm_aff_grp_of_other_user_by_name | Status : SUCCESS ===
ok
Deploy vm in affinity group of another user by id ... === TestName: test_07_deploy_vm_aff_grp_of_other_user_by_id | Status : SUCCESS ===
ok
Deploy vm in multiple affinity groups ... === TestName: test_08_deploy_vm_multiple_aff_grps | Status : SUCCESS ===
ok
Deploy multiple vms in multiple affinity groups ... === TestName: test_09_deploy_vm_multiple_aff_grps | Status : SUCCESS ===
ok
Deploy VM by aff grp name and id ... === TestName: test_10_deploy_vm_by_aff_grp_name_and_id | Status : SUCCESS ===
ok
List affinity group for a vm ... === TestName: test_01_list_aff_grps_for_vm | Status : SUCCESS ===
ok
List multiple affinity groups associated with a vm ... === TestName: test_02_list_multiple_aff_grps_for_vm | Status : SUCCESS ===
ok
List affinity groups by id ... === TestName: test_03_list_aff_grps_by_id | Status : SUCCESS ===
ok
List Affinity Groups by name ... === TestName: test_04_list_aff_grps_by_name | Status : SUCCESS ===
ok
List Affinity Groups by non-existing id ... === TestName: test_05_list_aff_grps_by_non_existing_id | Status : SUCCESS ===
ok
List Affinity Groups by non-existing name ... === TestName: test_06_list_aff_grps_by_non_existing_name | Status : SUCCESS ===
ok
List affinity group should list all for a vms associated with that group ... === TestName: test_07_list_all_vms_in_aff_grp | Status : SUCCESS ===
ok
Update the list of affinityGroups by using affinity groupids ... === TestName: test_01_update_aff_grp_by_ids | Status : SUCCESS ===
ok
Update the list of affinityGroups by using affinity groupnames ... === TestName: test_02_update_aff_grp_by_names | Status : SUCCESS ===
ok
Update the list of affinityGroups for vm which is not associated ... === TestName: test_03_update_aff_grp_for_vm_with_no_aff_grp | Status : SUCCESS ===
ok
Update the list of Affinity Groups to empty list ... SKIP: Skip - Failing - work in progress
Update the list of Affinity Groups on running vm ... === TestName: test_05_update_aff_grp_on_running_vm | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 42 tests in 976.432s

OK (SKIP=1)

* pr/1134:
  CLOUDSTACK-6276 Removing unused parameter in integration test for projects
  CLOUDSTACK-6276 Removing unused parameter in integration test
  CLOUDSTACK-6276 Fixing affinity groups for projects

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-03 20:10:16 +01:00
Remi Bergsma 6d558c0201 Merge pull request #944 from rafaelweingartner/lrg-cs-hackday-012
Create test cases to getPatchFilePath method and class names changed In this commit we created tests cases for the respective classes in package  com.cloud.hypervisor.xenserver.resource.

We added test cases to check the implementation of  com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getPatchFiles. Therefore, we test in a more comprehensive way the tests that already exist to check the code of com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getPatchFilePath.

We added a new abstract class, called com.cloud.hypervisor.xenserver.resource.CitrixResourceBaseTest.java

This class has two tests methods:

* com.cloud.hypervisor.xenserver.resource.CitrixResourceBaseTest.testGetPathFilesExeption(CitrixResourceBase), this method tests if the getPatchFilePath() method throws the com.cloud.utils.exception.CloudRuntimeException.CloudRuntimeException when the com.cloud.utils.script.Script.findScript(String, String) return a null value;
* com.cloud.hypervisor.xenserver.resource.CitrixResourceBaseTest.testGetPathFilesListReturned(CitrixResourceBase), this method tests the correct return value from getPatchFilePath() method, basically, verify if the returned list contain the file with the same absolute path that was retrieved from the findScript method.

We also changed the name of those test classes, the change was basically remove the Path word from the name of classes.

* pr/944:
  created tests cases for method "citrixResourceBase.getPatchFiles"

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-03 10:12:07 +01:00
weingartner 7453727c7b Removed PlannerBase empty class
The PlannerBase is an empty class, and does not bring contribution to
ACS code.
We changed com.cloud.deploy.FirstFitPlanner to extend the AdapterBase
and implements the com.cloud.deploy.DeploymentPlanner.

We also removed the method
com.cloud.deploy.DeploymentPlanner.check(VirtualMachineProfile,
DeploymentPlan, DeployDestination, ExcludeList) that was not used
anywhere.

Additionally, we removed the “_” from FirstFitPlanner's attributes name,
in order to have them in a more standard way.
2015-12-01 11:45:23 -02:00
Remi Bergsma 4ecfc29267 Merge release branch 4.6 to master
* 4.6:
  Use version for RC branch name instead of branch
  make sure all files are updates with new version
  Update L10N resource files with 4.6 strings from Transifex (20151129)
  Fix secondary storage not working with swift
  CLOUDSTACK-9083: Add disk serial to kvm virt xml
2015-11-30 22:30:52 +01:00
Remi Bergsma 9710046fb9 Merge pull request #1100 from rafaelweingartner/master-lrg-cs-hackday-005
Removal of DefaultUserAuthenticator empty class.The DefaultUserAuthenticator is an empty class, extending from the
AdapterBase and implementing the UserAuthenticator interface. The class
is not being used as a marker and it's sole use is to be extended by
other UserAuthenticators. Noticing that the class had no purpose, I
removed it and made it's children extend from it's superclass and
implement it's interface instead. Also, I removed the @Local tags from
those classes, since EJB is not used anymore.

* pr/1100:
  Removal of DefaultUserAuthenticator empty class

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-29 09:15:50 +01:00
Remi Bergsma f823d06dd9 Merge pull request #1118 from karuturi/CLOUDSTACK-9081
CLOUDSTACK-9081 Migration of vm across clusters fails in clustered MS setupStorage motion of vm across clusters/xenserver-pools fails in a
clustered management server setup. In xen storage motion we have to send
a migrate_receive command to the destination host, followed by
migrate_send command to the source host. The sr and network detials of
the destination host have to be passed in migrate_send command on the
source. While migrating across clusters the source and destination
resources are seperate objects. To pass this information across
resources we have to send seperate migrate with storage receive and send
commands to the resource. In a clustered ms setup these commands may
have to be forwarded to another ms as the resource may be owned by it.
The serilization of the command and answer objects fails in such case as
it doesn't understand the xapi sr and network objects. Made a change to
serialize these objects in the resource layer and pass the around as
strings in the command and answer objects.
Reviewed-By: @likitha
Author: @devdeep
Tested-By: @sailajamada

* pr/1118:
  CLOUDSTACK-9081 Migration of vm across clusters fails in clustered MS setup

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-29 09:09:03 +01:00
pedro-martins d01cdb1979 created tests cases for method "citrixResourceBase.getPatchFiles"
we also changed the name of those test classes, the change was basically
remove the “Path” word from the name of classes.

change the private method
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getPatchFiles
to protected

added new class in test cases
2015-11-28 19:06:04 -02:00
weingartner 98594b709b Removal of DefaultUserAuthenticator empty class
The DefaultUserAuthenticator is an empty class, extending the
AdapterBase and implementing the UserAuthenticator interface. The class
is only being used as a marker and it's sole use is to be extended by
other UserAuthenticators. Noticing that the class had no purpose, I
removed it and made it's children extend from it's superclass and
implement it's interface instead. Also, I removed the @Local tags from
those classes, since EJB is not used anymore.
2015-11-27 21:29:50 -02:00
Patrick Dube c76d317150 CLOUDSTACK-6276 Fixing affinity groups for projects 2015-11-27 14:43:02 -05:00
Remi Bergsma 3358d6509b Merge pull request #1035 from DaanHoogland/CLOUDSTACK-9037
CLOUDSTACK-9037 patterns can be more elaborate then prefixes.little fix to make sure for instance "eth" is not regarded as interface when it is part of "methamfetamine"

* pr/1035:
  CLOUDSTACK-9037 patterns can be more elaborate then prefixes.

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-27 19:44:35 +01:00
Sateesh Chodapuneedi 6a9956e0f5 CLOUDSTACK-4787 Allow selection of scsi controller type in vSphere
commit #7
    So far only 1 controller (scsi or ide) is supported in Cloudstack for ide or
    scsi, this is existing limitation. Added support for 2nd IDE controller. Support adding IDE
    virtual disk to VM. Also added check if VM is running as IDE virtual disk cannot be attached
    to VM if VM is runnning.If user detaches a virtual disk on lower unit number of controller,
    then subsequent attach operation should find free unit number on the controller and attach
    the virtual disk there.

    commit #6
    Let the controllers of existing VMs continue without flip, current busInfo retrieved from
    chain_info field of volume record from database would be preferred over
    controller settings from all configuration settings.

    commit #5
    Editing global configuration param vmware.root.disk.controller osdefault value results
    in loss of previous root disk controller type. Hence root disk's controller type for legacy
    VMs is unknow post that modificaiton by user. If VM is stop/start then we could get this
    infromation from bus info of existing volume. But if user resets VM and then try to start VM.
    The existing bus info would be lost. Hence existing disk info is not available to depend on.
    Using lsilogic or generic scsi controller for ROOT disk of legacy VMs if reset.

    commit #4
    Avoid adding additional (>1) scsi controllers to system vms. While attaching volume to legacy VM
    don't use osdefault optoin which applicable only for VM created with the option enabled, use
    legacy data disk controller type (lsilogic)

    commit #3
    If root disk's controller type is scsi and data disk controller type condenses
    to any of scsi sub-types then data disk controller type would fall back to root disk controller itself. This
    ensures data volumes would be accessible in all cases as controller of root volume would be reliable
    and it means VM has the supported controller. It also avoids mix of scsi controller sub-types in a user instance.
    Also translating disk controller type scsi to lsilogic.

    commit #2
    Support auto detection of recommended virtual disk controller type for specific guest OS.

    commit #1
    Support granual controller types. Add support for controller types in template registration as well.

    Fix white spaces.
    Removed stale HEAD merge lines
    Removed tail of merge lines
    Fixed VmwareResource, removing storage commands that moved to VmwareStorageProcessor.
    removed stale code of controller that is present in processor
    Fixed check style errors.
    Fixed injection.
    Tested with Linux and windows templates. Unable to run iso based tests due to few bugs in register iso area.

    Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>

(cherry picked from commit a4cc987a6f)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-11-27 18:39:41 +05:30
Nick Livens 453333cc67 Fix NuageVsp errors for build-master-slowbuild 2015-11-26 09:08:40 +01:00
Devdeep Singh 00522e475b CLOUDSTACK-9081 Migration of vm across clusters fails in clustered MS setup
Storage motion of vm across clusters/xenserver-pools fails in a
clustered management server setup. In xen storage motion we have to send
a migrate_receive command to the destination host, followed by
migrate_send command to the source host. The sr and network detials of
the destination host have to be passed in migrate_send command on the
source. While migrating across clusters the source and destination
resources are seperate objects. To pass this information across
resources we have to send seperate migrate with storage receive and send
commands to the resource. In a clustered ms setup these commands may
have to be forwarded to another ms as the resource may be owned by it.
The serilization of the command and answer objects fails in such case as
it doesn't understand the xapi sr and network objects. Made a change to
serialize these objects in the resource layer and pass the around as
strings in the command and answer objects.  Reviewed-By: Likitha
2015-11-26 11:59:18 +05:30
Rohit Yadav 12c395b560 CLOUDSTACK-9083: Add disk serial to kvm virt xml
Adds disk serial ids based on volume uuids to the virt xml. This may be useful
for appliances/software that needs some serial ids on the VM disks. This does not
impact existing/running VMs, the vm virt xmls will be updates for running VMs
the next time they are stopped/started.

For testing, disk serial (of debian based systemvm) in the virt xml matched that
in /sys/devices/pci0000:00:0000:00:07.0/virtio4/block/vda/serial.

We currently don't support scsi-blcok devices for which serial is not supported,
for this we've added a DeviceType (LUN) which may be used in future and a check
to not add the serial to the xml if disk type is LUN.
Refer: https://libvirt.org/formatdomain.html#elementsDisks

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-11-25 14:58:42 +05:30
Daan Hoogland 4bc6bcfa9a PR1102 regression fix; half the annotation was removed at several locations 2015-11-23 19:18:35 +01:00
Remi Bergsma a2a72887d2 Merge pull request #1021 from koushik-das/CLOUDSTACK-8485
CLOUDSTACK-8485: listAPIs are taking too long to return results- Removed regex. based search/replace of sensitive data on API response introduced as part of commit b0c6d47347
- Added new response serializer to skip sensitive data from getting logged based on annotation present in resposne object fields
- Added annotation (@LogLevel(Log4jLevel.Off)) to sensitive response object fields

Ran the following tests on simulator:

test_vm_life_cycle.py

Test advanced zone virtual router ... === TestName: test_advZoneVirtualRouter | Status : SUCCESS ===
ok
Test Deploy Virtual Machine ... === TestName: test_deploy_vm | Status : SUCCESS ===
ok
Test Multiple Deploy Virtual Machine ... === TestName: test_deploy_vm_multiple | Status : SUCCESS ===
ok
Test Stop Virtual Machine ... === TestName: test_01_stop_vm | Status : SUCCESS ===
ok
Test Start Virtual Machine ... === TestName: test_02_start_vm | Status : SUCCESS ===
ok
Test Reboot Virtual Machine ... === TestName: test_03_reboot_vm | Status : SUCCESS ===
ok
Test destroy Virtual Machine ... === TestName: test_06_destroy_vm | Status : SUCCESS ===
ok
Test recover Virtual Machine ... === TestName: test_07_restore_vm | Status : SUCCESS ===
ok
Test migrate VM ... === TestName: test_08_migrate_vm | Status : SUCCESS ===
ok
Test destroy(expunge) Virtual Machine ... === TestName: test_09_expunge_vm | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 10 tests in 306.429s

OK

test_volumes.py

Download a Volume attached to a VM ... === TestName: test_03_download_attached_volume | Status : SUCCESS ===
ok
Delete a Volume attached to a VM ... === TestName: test_04_delete_attached_volume | Status : SUCCESS ===
ok
Detach a Volume attached to a VM ... === TestName: test_05_detach_volume | Status : SUCCESS ===
ok
Delete a Volume unattached to an VM ... === TestName: test_09_delete_detached_volume | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 4 tests in 184.132s

OK

test_network.py

Test for delete account ... === TestName: test_delete_account | Status : SUCCESS ===
ok
Test for Associate/Disassociate public IP address for admin account ... === TestName: test_public_ip_admin_account | Status : SUCCESS ===
ok
Test for Associate/Disassociate public IP address for user account ... === TestName: test_public_ip_user_account | Status : SUCCESS ===
ok
Test for release public IP address ... === TestName: test_releaseIP | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 4 tests in 783.726s

OK

test_routers.py

Test router internal advanced zone ... SKIP: Marvin configuration has no host credentials                            to check router services
Test restart network ... === TestName: test_03_restart_network_cleanup | Status : SUCCESS ===
ok
Test router basic setup ... === TestName: test_05_router_basic | Status : SUCCESS ===
ok
Test router advanced setup ... === TestName: test_06_router_advanced | Status : SUCCESS ===
ok
Test stop router ... === TestName: test_07_stop_router | Status : SUCCESS ===
ok
Test start router ... === TestName: test_08_start_router | Status : SUCCESS ===
ok
Test reboot router ... === TestName: test_09_reboot_router | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 7 tests in 42.958s

OK (SKIP=1)

test_global_settings.py

test update configuration setting at zone level scope ... === TestName: test_UpdateConfigParamWithScope | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 1 test in 0.127s

OK

test_resource_detail.py

Test volume detail ... === TestName: test_01_updatevolumedetail | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 1 test in 11.492s

OK

* pr/1021:
  CLOUDSTACK-8485: listAPIs are taking too long to return results - Removed regex. based search/replace of sensitive data on API response introduced as part of commit b0c6d47347 - Added new response serializer to skip sensitive data from getting logged based on annotation present in resposne object fields - Added new parameter 'isSensitive' to @Param for marking a field as sensitive in response objects

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-23 12:56:25 +01:00
Daan Hoogland 08eb96776a CLOUDSTACK-9037 patterns can be more elaborate then prefixes. 2015-11-23 11:48:44 +01:00
Remi Bergsma a6b6f4104d Merge pull request #1102 from rafaelweingartner/master-lrg-cs-hackday-006
Removed unnecessary @Local annotations and their respective importsFollowing @rafaelweingartner 's findings in PR #714 that many spring beans contained an @Local annotation, we've decided to remove said annotations and their imports from the ComponentLifecycleBase class and its subclasses seeking a reduction of a few hundred lines of useless code.

I had already opened a pull request for this (#853) but at some point my commit disappeared from the PR entirely, showing no new changes in code, which caused it to be merged automatically (with no changes).

* pr/1102:
  Removed unnecessary @Local annotations and their respective imports from the ComponentLifecycleBase class and its subclasses.

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-23 11:25:43 +01:00
Remi Bergsma a3234349b9 Merge pull request #1019 from DaanHoogland/PMD
findbugs: ! equals instead of !=simple pmd warning fixed

* pr/1019:
  use equals instead of comparison operators on strings
  findbugs: instead of !=, use string utils

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-22 13:20:53 +01:00
cirstofolini 1a64c247ad Removed unnecessary @Local annotations and their respective imports from the ComponentLifecycleBase class and its subclasses. 2015-11-21 18:31:11 -02:00
Remi Bergsma 7665bdc815 Merge pull request #1083 from borisroman/CLOUDSTACK-9062
CLOUDSTACK-9062: Improve S3 implementation.The S3 implementation is far from finished, this commit focuses on the bases.

 - Upgrade AWS SDK to latest version.
 - Rewrite S3 Template downloader.
 - Rewrite S3Utils utility class.
 - Improve addImageStoreS3 API command.
 - Split various classes for convenience.
 - Various minor improvements and code optimizations.

A side effect of the new AWS SDK is that it, by default, uses the V4 signature. Therefore I added an option to specify the Signer, so it stays compatible with previous versions.

Please review thoroughly, both code inspection and (automated) integration tests. Currently no integration tests are available specifically for S3. Therefore the implementation is needed to be tested manually, for now...

What I tested:
 - Greenfield install -> will download latest systemvm template automatically to S3.
 - Upload a template/iso
 - Download a template/iso
 - Restart of management server -> list available templates -> doesn't download them again if available.

* pr/1083:
  CLOUDSTACK-9062: Improve S3 implementation.

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-20 22:27:17 +01:00
Koushik Das e13df96348 CLOUDSTACK-8485: listAPIs are taking too long to return results
- Removed regex. based search/replace of sensitive data on API response introduced as part of commit b0c6d47347
- Added new response serializer to skip sensitive data from getting logged based on annotation present in resposne object fields
- Added new parameter 'isSensitive' to @Param for marking a field as sensitive in response objects
2015-11-20 16:40:53 +05:30
Remi Bergsma 791f9dfe5f Merge pull request #1084 from ekholabs/improvement/remove_scripts-CLOUDSTACK-9067
CLOUDSTACK-9067 - As I developer I want to remove all the unused router-shell scripts from ACSThis PR removes the unused shell scripts that were present in the ACS project. Those script were replaced by the.

Some of the scripts are used by the HyperV Resource, which were hardcoded. I took the opportunity to use the Java constants over there as well, so the next one touching the code will know they exist and won't hardcode anything.

The following task were applied:

* Remove the shell files and the Java constants that were mapping them;
* Apply the use of the Java constants to the HyperV Resource class;
* Wrap the String.format() method in the StringUtils so we can test the changes in the HyperV Resource class.

The last point was added because I do not have a HyperV test environment. Hence, I wanted to make sure the tiny code I changed is covered at least by unit tests.

* pr/1084:
  CLOUDSTACK-9067 - Replaces hardcoded paths with the VRScripts constants.
  CLOUDSTACK-9067 - Fomatting the code of HypervDirectConnectResource class
  CLOUDSTACK-9067 - Remove old script file from the project

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-19 17:58:42 +01:00
Boris Schrijver 5c0366c99e CLOUDSTACK-9062: Improve S3 implementation.
The S3 implementation is far from finished, this commit focusses on the bases.

 - Upgrade AWS SDK to latest version.
 - Rewrite S3 Template downloader.
 - Rewrite S3Utils utility class.
 - Improve addImageStoreS3 API command.
 - Split various classes for convenience.
 - Various minor improvements and code optimalisations.

A side effect of the new AWS SDK is that it, by default, uses the V4 signature. Therefore I added an option to specify the Signer, so it stays compatible with previous versions.
2015-11-19 15:29:10 +01:00
Wilder Rodrigues 942aa17d0c CLOUDSTACK-9067 - Replaces hardcoded paths with the VRScripts constants. 2015-11-18 13:12:45 +01:00
Remi Bergsma 219da64027 Merge pull request #935 from nvazquez/from4.5.1
CLOUDSTACK-8956: NSX/Nicira Plugin does not support NSX v4.2.1JIRA Ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-8956

### Description of the problem:
Prior to version 4.2. Nicira/VmWare NSX used a variation of Open vSwitch as means of integrating SDN into hypervisor layer. Cloudstack NiciraNVP plugin was written to support OVS as a bridge to NSX.
In version 4.2 VMware introduced NSX vSwitch as a replacement for OVS in ESX hypervisors. It is a fork of distributed vSwitch leveraging one of the recent features of ESX called opaque networks. Because of that change the current version of NiciraNVP plugin doesnt support versions of NSX-MH above 4.2 specifically in Vsphere environment. Proposed fix will analyze a version of NVP/NSX API and use proper support for ESX hypervisors.

vSphere hypervisor mode operations when NV is deployed onto NSX managed network changes:
* Current mode. A portgroup = UUID of CS VM NIC is created on a local standard switch of the Hypervisor where VM is starting. VM nic is attached to that port group.
* New mode. No additional port group is created on a HW. No port group cleanup is needed after VM/NIC is destroyed. VM is attached to 1st port group having the following attributes:
** opaqueNetworkId string "br-int
** opaqueNetworkType string "nsx.network"

If portgroup with such attributes is not found a deployment should fail with exception.

### VMware vSphere API version from 5.1 to 5.5:
Since vSphere API version 5.5, [OpaqueNetworks](https://www.vmware.com/support/developer/converter-sdk/conv55_apireference/vim.OpaqueNetwork.html) are introduced.
Its description says:
> This interface defines an opaque network, in the sense that the detail and configuration of the network is unknown to vShpere and is managed by a management plane outside of vSphere. However, the identifier and name of these networks is made available to vSphere so that host and virtual machine virtual ethernet device can connect to them.

In order to connect a vm's virtual ethernet device to the proper opaque network when deploying a vm into a NSX managed network, we first need to look for a particular opaque network on hosts. This opaque network's id has to be **"br-int"** and its type **"nsx.network"**.

Since vSphere API version 5.5 [HostNetworkInfo](https://www.vmware.com/support/developer/converter-sdk/conv55_apireference/vim.host.NetworkInfo.html#opaqueNetwork) introduces a list of available opaque networks for each host.
If NSX API version >= 4.2 we look for a [OpaqueNetworkInfo](https://www.vmware.com/support/developer/converter-sdk/conv55_apireference/vim.host.OpaqueNetworkInfo.html) which satisfies:
* opaqueNetworkId = "br-int"
* opaqueNetworkType = "nsx.netork"

If that opaque network is found, then we need to attach vm's NIC to a virtual ethernet device which support this, so we use [VirtualEthernetCardOpaqueNetworkBackingInfo](https://www.vmware.com/support/developer/converter-sdk/conv55_apireference/vim.vm.device.VirtualEthernetCard.OpaqueNetworkBackingInfo.html) setting:
* opaqueNetworkId = "br-int"
* opaqueNetworkType = "nsx.netork"

* pr/935:
  CLOUDSTACK-8956: Remove assert(false) on opaque network and ping method on NiciraNvpApiVersion
  CLOUDSTACK-8956: Deploy VM on NSX managed network changes if NSX Api Version >= 4.2: has to connect to "br-int" of "nsx.network" type
  CLOUDSTACK-8956: Log NSX Api Version
  CLOUDSTACK-8956: Add VMware Api v5.5 and change pom.xml to use VMware Api v5.5

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-18 13:11:25 +01:00
Remi Bergsma 95ae7963d5 Merge pull request #801 from nlivens/updated-nuage-vsp-plugin
CLOUDSTACK-8832 : Update Nuage VSP plugin to work with Nuage VSP release 3.2

* pr/801:
  CLOUDSTACK-8832 : Update Nuage VSP plugin to work with Nuage VSP release 3.2

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-18 12:09:09 +01:00
Wilder Rodrigues 1cf57f74d7 CLOUDSTACK-9067 - Fomatting the code of HypervDirectConnectResource class 2015-11-17 15:58:22 +01:00
Rajani Karuturi a723988aec CLOUDSTACK-8885: added blocked connection listener for rabbitmqeventbus
When rabbitmq connections are blocked(for example when rabbitmq is
server is out of space), all the cloudstack threads which does any
action and publishes to rabbitmq(for example login, launch vm etc.) are
all blocked.

Added a blocked connection listener to handle this and unblock the
parent thread.
2015-11-17 16:04:16 +05:30
Rajani Karuturi 693a001c63 updating rabbitmq amqp client to 3.5.4 from 3.4.2 2015-11-17 16:04:16 +05:30
nvazquez b20c5be1ce CLOUDSTACK-8956: Remove assert(false) on opaque network and ping method on NiciraNvpApiVersion 2015-11-16 11:29:31 -08:00
nvazquez e1d1a40325 CLOUDSTACK-8956: Deploy VM on NSX managed network changes if NSX Api Version >= 4.2: has to connect to "br-int" of "nsx.network" type 2015-11-16 11:29:30 -08:00
nvazquez 2386ae5e5a CLOUDSTACK-8956: Log NSX Api Version 2015-11-16 11:29:30 -08:00
Daan Hoogland 3d5f09357b use equals instead of comparison operators on strings 2015-11-16 18:30:36 +01:00
Daan Hoogland 9a798c537e findbugs: instead of !=, use string utils 2015-11-16 18:30:36 +01:00
Daan Hoogland 962d4b4afe CLOUDSTACK-9046 renamed enums in kvm plugin 2015-11-16 13:41:14 +00:00
Remi Bergsma 1f53f2a93e Updating pom.xml version numbers for release 4.7.0-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-15 18:54:13 +01:00
Remi Bergsma b38c3bed0c Updating pom.xml version numbers for release 4.6.1-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-13 21:27:57 +01:00
Remi Bergsma e31ade03c6 Updating pom.xml version numbers for release 4.6.0
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-10 15:45:34 +01:00
Remi Bergsma 0b4cc5de87 Merge pull request #986 from wido/kvm-libvirt-unittests
Add Unit Tests for Libvirt/KVM storage codeThese classes were not covered by Unit Tests and this commit
adds some tests for their basic functionality.

* pr/986:
  Add Unit Tests for Libvirt/KVM storage code

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-02 17:53:31 +01:00
Remi Bergsma 9c90eff87f Merge pull request #1015 from DaanHoogland/pr/975
CLOUDSTACK-8964: Ovm3HypervisorGuru answer fix and snapshot to volume implementation.some clean up of code after the fix from #975

* pr/1015:
  CLOUDSTACK-8964 side effect isolation  extract side effect away to emphasize the main commandDelegation objective
  host delegation logging for XenServerGuru.java
  Getting volume from snapshot working again... odd
  simple change to prevent failure and keep OVM3 snapshots working

This closes #975

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-02 13:37:19 +01:00
Bharat Kumar 4849368c94 CLOUDSTACK-8852 Database shows that management server is UP when it is actually stopped from the CCP GUI
This was happening as the stop method in clusterMnanagerImpl was not getting callled. Added shutdown hooks to all sub contexts,
    this enables spring to call the stop mehtods of the beans when management server is shutting down.

Conflicts:
	framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/web/CloudStackContextLoaderListener.java
2015-11-02 15:26:39 +05:30
Wido den Hollander 602231132d kvm: Add UnitTests for LibvirtUtilitiesHelper
These were lacking, but this helper is used in various places
inside the KVM code.

Some simple tests to verify the helper is doing what we expect it
to do.
2015-10-30 12:34:00 +01:00
Daan Hoogland fc18d1e8b1 CLOUDSTACK-8964 side effect isolation
extract side effect away to emphasize the main commandDelegation
objective
2015-10-30 09:00:47 +01:00
Mike Tutkowski 7d1dc97423 CLOUDSTACK-8985: Deleted volume's removed column not updated 2015-10-27 15:04:15 -06:00
Nick Livens 645f8758df CLOUDSTACK-8832 : Update Nuage VSP plugin to work with Nuage VSP release 3.2 2015-10-27 14:56:57 +01:00
Wido den Hollander 7568f2123c Add Unit Tests for Libvirt/KVM storage code
These classes were not covered by Unit Tests and this commit
adds some tests for their basic functionality.
2015-10-27 13:21:28 +01:00
Daan Hoogland 5a134a6fe6 unit test for interface patterns in libvirt compute resource 2015-10-24 00:57:01 +02:00
Funs Kessen c4afb76d8c host delegation logging for XenServerGuru.java 2015-10-23 20:09:06 +02:00
Daan Hoogland f1ea27ed37 Merge branch 'pr/966' into interfacePatternCheck 2015-10-23 17:10:03 +02:00
Funs Kessen 6db4b7b0be Getting volume from snapshot working again... odd 2015-10-23 16:50:26 +02:00
Funs Kessen 66fed462b6 simple change to prevent failure and keep OVM3 snapshots working 2015-10-23 15:58:40 +02:00
David Mabry e1a401c023 Added support for KVM teamd devices to LibvirtComputingResource.java. This will allow users to utilze teamd nic teaming devices named team*. 2015-10-22 10:19:50 -05:00
Wido den Hollander d413b2c375 CLOUDSTACK-8879: Depend in rados-java 0.2.0
This should fix the CloudStack Agent from crashing when it has to
handle more then 16 RBD snapshots on one Volume.
2015-09-25 11:01:03 +02:00
Boris Schrijver 80cb3adad0 Added isEmpty() check to prevent nullPointerException. 2015-09-23 15:07:16 +02:00
Koushik Das 86325cbc95 Merge pull request #792 from koushik-das/CLOUDSTACK-8826
CLOUDSTACK-8826: XenServer - Use device id passed as part of attach volume API properly

If device id passed as part of API and available then use it otherwise fallback on XS to automatically assign one.
For ISO device id used is 3 and it is processed before any other entry to avoid conflict.

Signed-off-by: Koushik Das <koushik@apache.org>
2015-09-22 15:43:42 +05:30
Koushik Das f5b9a96d24 CLOUDSTACK-8826: XenServer - Use device id passed as part of attach volume API properly
If device id passed as part of API and available then use it otherwise fallback on XS to automatically assign one.
For ISO device id used is 3 and it is processed before any other entry to avoid conflict.
2015-09-21 17:09:14 +05:30
wilderrodrigues 6e426fa46d Revert "Merge pull request #825 from borisroman/charset"
This reverts commit a44e2bff7e, reversing
changes made to 06cefaf493.

We are reverting the PR #825 because it breaks Master.
2015-09-15 14:58:18 +02:00
Daan Hoogland a44e2bff7e Merge pull request #825 from borisroman/charset
Replaced all occurences of Charset.forName(UTF-8) with StringUtils.getPreferredCharset().

* pr/825:
  Replaced all occurences of Charset.forName(UTF-8) with StringUtils.getPreferredCharset().

Signed-off-by: Daan Hoogland <daan@onecht.net>
2015-09-15 13:38:18 +02:00
Boris Schrijver 71929fdedd Replaced all occurences of Charset.forName(UTF-8) with StringUtils.getPreferredCharset(). 2015-09-15 12:00:33 +02:00
Wido den Hollander 4f409a7c17 CLOUDSTACK-8645: Improve logging of RBD functionality in KVM
A simple commit which changes a couple of log lines.
2015-09-15 10:10:04 +02:00
Satoru Nakaya e8c5ed4e3d CLOUDSTACK-8838: Allow ensX enoX enpX enxX format for nics in CentOS 7 2015-09-13 22:13:30 +09:00
Wido den Hollander 28d18dce00 Merge pull request #776 from exoscale/fix/firewall-sysctl
sysctl: don't modify /etc/sysctl.confTo configure firewall rules, CloudStack modifies `/etc/sysctl.conf` and
execute those modifications. This may be harmful for several reasons:

 1. `/etc/sysctl.conf` may be managed by some configuration management
    system. Such a system will constantly restore the previous version.

 2. `/etc/sysctl.conf` may contain additional properties that have been
    changed later by some system administrator (for example, once a
    firewall has been configured, forwarding may have been activated
    while it is disabled in `/etc/sysctl.conf`). Executing the file
    again at a later time may disrupt the system.

 3. Entries are added again and again. `/etc/sysctl.conf` will contain
    the same directives repeated several times.

Using a configuration file is not needed as `sysctl` is able to directly
modify sysctl values with `-w` flag.

Signed-off-by: Vincent Bernat <Vincent.Bernat@exoscale.ch>

* pr/776:
  sysctl: don't modify /etc/sysctl.conf

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2015-09-09 10:29:20 +02:00
Rajani Karuturi b69a69def7 Fixing findbugs error due to PR #755 merge
issue: Boxing/unboxing to parse a primitive
2015-09-07 10:40:55 +05:30
Remi Bergsma 1077f86144 Merge pull request #766 from kmccormick/CLOUDSTACK-8678
CLOUDSTACK-8678: Reserve RAM for KVM host OSUse host.reserved.ram.mb agent property to modify total system RAM
before reporting to management server.

* pr/766:
  CLOUDSTACK-8678: Reserve RAM for KVM host OS

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-09-05 09:10:58 +02:00
Rajani Karuturi 5881035e7b Merge pull request #755 from karuturi/CLOUDSTACK-8647-2
Cloudstack:8647 LDAP Trust AD and AutoimportToday, CloudStack can automatically import LDAP users based on the configuration to a domain or an account. However, any new users in LDAP aren't automatically reflected. The admin has to manually import them again.
This feature enables admin to map LDAP group/OU to a CloudStack domain and any changes are reflected in ACS as well.
FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/WIP%3A+LDAP%3A+Trust+AD+and+Auto+Import

testcases output:
```
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running groovy.org.apache.cloudstack.ldap.NoLdapUserMatchingQueryExceptionSpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.216 sec - in groovy.org.apache.cloudstack.ldap.NoLdapUserMatchingQueryExceptionSpec
Running groovy.org.apache.cloudstack.ldap.LdapManagerImplSpec
log4j:WARN No appenders could be found for logger (org.apache.cloudstack.ldap.LdapManagerImpl).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
using type:
using type: null
using type: TEST
using type: TEST TEST
using name:
using name: null
using accountType: -1
using accountType: 1
using accountType: 3
using accountType: 4
using accountType: 5
using accountType: 6
using accountType: 20000
using accountType: -500000
Tests run: 29, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.387 sec - in groovy.org.apache.cloudstack.ldap.LdapManagerImplSpec
Running groovy.org.apache.cloudstack.ldap.LdapListUsersCmdSpec
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.041 sec - in groovy.org.apache.cloudstack.ldap.LdapListUsersCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapAddConfigurationCmdSpec
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.019 sec - in groovy.org.apache.cloudstack.ldap.LdapAddConfigurationCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapUserSpec
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.021 sec - in groovy.org.apache.cloudstack.ldap.LdapUserSpec
Running groovy.org.apache.cloudstack.ldap.LdapAuthenticatorSpec
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.082 sec - in groovy.org.apache.cloudstack.ldap.LdapAuthenticatorSpec
Running groovy.org.apache.cloudstack.ldap.LdapConfigurationVOSpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 sec - in groovy.org.apache.cloudstack.ldap.LdapConfigurationVOSpec
Running groovy.org.apache.cloudstack.ldap.OpenLdapUserManagerSpec
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.094 sec - in groovy.org.apache.cloudstack.ldap.OpenLdapUserManagerSpec
Running groovy.org.apache.cloudstack.ldap.LdapDeleteConfigurationCmdSpec
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 sec - in groovy.org.apache.cloudstack.ldap.LdapDeleteConfigurationCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapUserResponseSpec
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec - in groovy.org.apache.cloudstack.ldap.LdapUserResponseSpec
Running groovy.org.apache.cloudstack.ldap.LdapUserManagerFactorySpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.027 sec - in groovy.org.apache.cloudstack.ldap.LdapUserManagerFactorySpec
Running groovy.org.apache.cloudstack.ldap.ADLdapUserManagerImplSpec
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.012 sec - in groovy.org.apache.cloudstack.ldap.ADLdapUserManagerImplSpec
Running groovy.org.apache.cloudstack.ldap.LdapCreateAccountCmdSpec
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.168 sec - in groovy.org.apache.cloudstack.ldap.LdapCreateAccountCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapImportUsersCmdSpec
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.063 sec - in groovy.org.apache.cloudstack.ldap.LdapImportUsersCmdSpec
Running groovy.org.apache.cloudstack.ldap.LinkDomainToLdapCmdSpec
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.019 sec - in groovy.org.apache.cloudstack.ldap.LinkDomainToLdapCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapSearchUserCmdSpec
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.011 sec - in groovy.org.apache.cloudstack.ldap.LdapSearchUserCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapListConfigurationCmdSpec
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec - in groovy.org.apache.cloudstack.ldap.LdapListConfigurationCmdSpec
Running groovy.org.apache.cloudstack.ldap.NoSuchLdapUserExceptionSpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 sec - in groovy.org.apache.cloudstack.ldap.NoSuchLdapUserExceptionSpec
Running groovy.org.apache.cloudstack.ldap.LdapConfigurationResponseSpec
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.006 sec - in groovy.org.apache.cloudstack.ldap.LdapConfigurationResponseSpec
Running groovy.org.apache.cloudstack.ldap.LdapConfigurationSpec
asserting for provider configuration: openldap
asserting for provider configuration: microsoftad
asserting for provider configuration:
asserting for provider configuration:
asserting for provider configuration: xyz
asserting for provider configuration: MicrosoftAd
asserting for provider configuration: OpenLdap
asserting for provider configuration: MicrosoftAD
Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.053 sec - in groovy.org.apache.cloudstack.ldap.LdapConfigurationSpec
Running groovy.org.apache.cloudstack.ldap.LdapContextFactorySpec
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.099 sec - in groovy.org.apache.cloudstack.ldap.LdapContextFactorySpec
Running groovy.org.apache.cloudstack.ldap.LdapConfigurationDaoImplSpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.027 sec - in groovy.org.apache.cloudstack.ldap.LdapConfigurationDaoImplSpec
Running groovy.org.apache.cloudstack.ldap.LdapUtilsSpec
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec - in groovy.org.apache.cloudstack.ldap.LdapUtilsSpec

Results :

Tests run: 156, Failures: 0, Errors: 0, Skipped: 0
```

* pr/755:
  CLOUDSTACK-8647: linkdomaintoldap shouldnt fail when createuseraccount fails
  CLOUDSTACK-8647 removed duplicate key in create sql of ldap_trust_map
  CLOUDSTACK-8647: string formatting
  CLOUDSTACK-8647: updated with review comments
  CLOUDSTACK-8647: unittests for LdapAuthenticatorSpec
  CLOUDSTACK-8647: formatted LdapAuthenticatorSpec
  CLOUDSTACK-8647: UI for trust AD feature
  CLOUDSTACK-8647 added unittests for new methods  in ldapmanager
  CLOUDSTACK-8647 unittests for LinkDomainToLdap api command
  CLOUDSTACK-8647: fixed unittests
  CLOUDSTACK-8647 support for assigning and admin to linked ldap domain
  CLOUDSTACK-8647 added nested group enabled config in ldap
  CLOUDSTACK-8647 added account_type to the linkDomainToLdap API
  CLOUDSTACK-8647 changed the authentication flow
  CLOUDSTACK-8647 added new api linkLdapToDomain
  CLOUDSTACK-8647: added cmd and response class for the new api

Signed-off-by: Rajani Karuturi <rajani.karuturi@citrix.com>
2015-09-05 09:30:17 +05:30
Vincent Bernat f2b8f2eade sysctl: don't modify /etc/sysctl.conf
To configure firewall rules, CloudStack modifies `/etc/sysctl.conf` and
execute those modifications. This may be harmful for several reasons:

 1. `/etc/sysctl.conf` may be managed by some configuration management
    system. Such a system will constantly restore the previous version.

 2. `/etc/sysctl.conf` may contain additional properties that have been
    changed later by some system administrator (for example, once a
    firewall has been configured, forwarding may have been activated
    while it is disabled in `/etc/sysctl.conf`). Executing the file
    again at a later time may disrupt the system.

 3. Entries are added again and again. `/etc/sysctl.conf` will contain
    the same directives repeated several times.

Using a configuration file is not needed as `sysctl` is able to directly
modify sysctl values with `-w` flag.

Signed-off-by: Vincent Bernat <Vincent.Bernat@exoscale.ch>
2015-09-04 14:31:09 +02:00
Kevin McCormick 99d20234d4 CLOUDSTACK-8678: Reserve RAM for KVM host OS
Use host.reserved.ram.mb agent property to modify total system RAM
before reporting to management server.

Remove dom0ram variable and its min/max calculation. Just reserve a
default of 1GB, unless overridden by host.reserved.mem.mb property.
2015-09-03 10:07:22 -07:00
Rajani Karuturi 53a441faf6 CLOUDSTACK-8647: linkdomaintoldap shouldnt fail when createuseraccount fails
Incase create useraccount fails with any runtime exception,
linkdomaintoldap api shouldnt fail. It just will not return the admin id
as it didnt create the account.
added test cases to verify this as well.
2015-09-03 18:05:50 +05:30
Rohit Yadav dd9ba48efa Merge pull request #753 from shapeblue/master-CLOUDSTACK-8762
CLOUDSTACK-8762: Check to confirm disk activity before starting a VMImplements a VM volume/disk file activity checker that checks if QCOW2 file
has been changed before starting the VM. This is useful as a pessimistic
approach to save VMs that were running on faulty hosts that CloudStack could
try to launch on other hosts while the host was not cleanly fenced. This is
optional and available only if you enable the settings in agent.properties
file, on per-host basis.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* pr/753:
  CLOUDSTACK-8762: Check to confirm disk activity before starting a VM

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-02 14:55:54 +05:30
Wei Zhou c0a0aec0f9 Merge pull request #732 from ustcweizhou/revert-volume-snapshot-master
Guys, can you review it? things need to be discussed:
(1) this supports KVM/QCOW2 only. Anyone want to implement for other Hypervisor/format ?
(2) The original data volume (on primary storage) will be removed.
(3) The script uses the default timeout in libvirtComputingResource. Do we need to add one in global configuration (like copy.volume.wait or backup.snapshot.wait, create.volume.from.snapshot.wait)
(4) In scripts/storage/qcow2/managesnapshot.sh, I use "qemu-img convert -f qcow2 -O qcow2" to copy the snapshot from secondary to primary (hence there is no base image file), instead of "cp -f", this is because convert is faster than cp in my testing.

* pr/732:
  CLOUDSTACK-5863: revert volume snapshot for KVM/QCOW2

Signed-off-by: Wei Zhou <w.zhou@tech.leaseweb.com>
2015-09-01 16:18:40 +02:00