config: doc: add default value 10 in description for vm.network.nic.max.secondary.ipaddresses
value hardcoded by commit 4925b9f6a1
* pr/777:
config: doc: add default value 10 in description for vm.network.nic.max.secondary.ipaddresses
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-8759 - Destroying VPC router results in a new unusable VPC routerHow to reproduce the problem:
1. Stop/Destroy the VPC router
2. Add a virtual machine to one of the VPC tier - it will trigger a VPC router creation
3. Router is created, but the NICs are not configured
How to recover without this fix:
1. Stop/destroy the VPC router and restart the VPC
Side effects: private gateways could be misconfigured.
Root cause:
In the VpcNetworkHelperImpl.configureDefaultNics() method, the guest network nic was added in the map prior to the control and public NICs. The order in the map should not matter, however in the LibvirtComputingResource.createVifs() method, there is a logic that relies on the device index - the array index - in order to create the control nic. I advise a refactor on the data model in order to be able to identify the NIC type instead of relying in the array index.
An integration test was added to cover the fix:
* test_vpc_router_nics.py
Environment:
Management Server running on CentOS 7.1
KVM host running on CentOS 7.1
CloudStack Agent/Common 4.6.0-SNAPSHOT
Executing the test:
```
nosetests --with-marvin --marvin-config=/data/shared/marvin/mct-zone2-kvm2-ISOLATED.cfg -s -a tags=advanced,required_hardware=true component/test_vpc_router_nics.py
```
Remark: during the SSH there might be stack traces on the console due to the connection retry. It takes some time to get the PF rules in place and reach the VMs. So, just let the test run until the end.
```
Test results:
Create a vpc with two networks with two vms in each network ... === TestName: test_01_VPC_nics_after_destroy | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 1 test in 774.020s
OK
/tmp//MarvinLogs/test_vpc_router_nics_VH6E9S/results.txt (END)
```
* pr/773:
CLOUDSTACK-8759 - Fix guets nic allocation
CLOUDSTACK-8759 - Adding a marvin test in order to cover the fix
CLOUDSTACK-8759 - The guest nic has to be added after the control nic
Signed-off-by: Remi Bergsma <github@remi.nl>
Cloudstack:8647 LDAP Trust AD and AutoimportToday, CloudStack can automatically import LDAP users based on the configuration to a domain or an account. However, any new users in LDAP aren't automatically reflected. The admin has to manually import them again.
This feature enables admin to map LDAP group/OU to a CloudStack domain and any changes are reflected in ACS as well.
FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/WIP%3A+LDAP%3A+Trust+AD+and+Auto+Import
testcases output:
```
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running groovy.org.apache.cloudstack.ldap.NoLdapUserMatchingQueryExceptionSpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.216 sec - in groovy.org.apache.cloudstack.ldap.NoLdapUserMatchingQueryExceptionSpec
Running groovy.org.apache.cloudstack.ldap.LdapManagerImplSpec
log4j:WARN No appenders could be found for logger (org.apache.cloudstack.ldap.LdapManagerImpl).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
using type:
using type: null
using type: TEST
using type: TEST TEST
using name:
using name: null
using accountType: -1
using accountType: 1
using accountType: 3
using accountType: 4
using accountType: 5
using accountType: 6
using accountType: 20000
using accountType: -500000
Tests run: 29, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.387 sec - in groovy.org.apache.cloudstack.ldap.LdapManagerImplSpec
Running groovy.org.apache.cloudstack.ldap.LdapListUsersCmdSpec
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.041 sec - in groovy.org.apache.cloudstack.ldap.LdapListUsersCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapAddConfigurationCmdSpec
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.019 sec - in groovy.org.apache.cloudstack.ldap.LdapAddConfigurationCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapUserSpec
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.021 sec - in groovy.org.apache.cloudstack.ldap.LdapUserSpec
Running groovy.org.apache.cloudstack.ldap.LdapAuthenticatorSpec
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.082 sec - in groovy.org.apache.cloudstack.ldap.LdapAuthenticatorSpec
Running groovy.org.apache.cloudstack.ldap.LdapConfigurationVOSpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 sec - in groovy.org.apache.cloudstack.ldap.LdapConfigurationVOSpec
Running groovy.org.apache.cloudstack.ldap.OpenLdapUserManagerSpec
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.094 sec - in groovy.org.apache.cloudstack.ldap.OpenLdapUserManagerSpec
Running groovy.org.apache.cloudstack.ldap.LdapDeleteConfigurationCmdSpec
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 sec - in groovy.org.apache.cloudstack.ldap.LdapDeleteConfigurationCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapUserResponseSpec
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec - in groovy.org.apache.cloudstack.ldap.LdapUserResponseSpec
Running groovy.org.apache.cloudstack.ldap.LdapUserManagerFactorySpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.027 sec - in groovy.org.apache.cloudstack.ldap.LdapUserManagerFactorySpec
Running groovy.org.apache.cloudstack.ldap.ADLdapUserManagerImplSpec
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.012 sec - in groovy.org.apache.cloudstack.ldap.ADLdapUserManagerImplSpec
Running groovy.org.apache.cloudstack.ldap.LdapCreateAccountCmdSpec
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.168 sec - in groovy.org.apache.cloudstack.ldap.LdapCreateAccountCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapImportUsersCmdSpec
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.063 sec - in groovy.org.apache.cloudstack.ldap.LdapImportUsersCmdSpec
Running groovy.org.apache.cloudstack.ldap.LinkDomainToLdapCmdSpec
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.019 sec - in groovy.org.apache.cloudstack.ldap.LinkDomainToLdapCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapSearchUserCmdSpec
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.011 sec - in groovy.org.apache.cloudstack.ldap.LdapSearchUserCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapListConfigurationCmdSpec
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec - in groovy.org.apache.cloudstack.ldap.LdapListConfigurationCmdSpec
Running groovy.org.apache.cloudstack.ldap.NoSuchLdapUserExceptionSpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 sec - in groovy.org.apache.cloudstack.ldap.NoSuchLdapUserExceptionSpec
Running groovy.org.apache.cloudstack.ldap.LdapConfigurationResponseSpec
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.006 sec - in groovy.org.apache.cloudstack.ldap.LdapConfigurationResponseSpec
Running groovy.org.apache.cloudstack.ldap.LdapConfigurationSpec
asserting for provider configuration: openldap
asserting for provider configuration: microsoftad
asserting for provider configuration:
asserting for provider configuration:
asserting for provider configuration: xyz
asserting for provider configuration: MicrosoftAd
asserting for provider configuration: OpenLdap
asserting for provider configuration: MicrosoftAD
Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.053 sec - in groovy.org.apache.cloudstack.ldap.LdapConfigurationSpec
Running groovy.org.apache.cloudstack.ldap.LdapContextFactorySpec
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.099 sec - in groovy.org.apache.cloudstack.ldap.LdapContextFactorySpec
Running groovy.org.apache.cloudstack.ldap.LdapConfigurationDaoImplSpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.027 sec - in groovy.org.apache.cloudstack.ldap.LdapConfigurationDaoImplSpec
Running groovy.org.apache.cloudstack.ldap.LdapUtilsSpec
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec - in groovy.org.apache.cloudstack.ldap.LdapUtilsSpec
Results :
Tests run: 156, Failures: 0, Errors: 0, Skipped: 0
```
* pr/755:
CLOUDSTACK-8647: linkdomaintoldap shouldnt fail when createuseraccount fails
CLOUDSTACK-8647 removed duplicate key in create sql of ldap_trust_map
CLOUDSTACK-8647: string formatting
CLOUDSTACK-8647: updated with review comments
CLOUDSTACK-8647: unittests for LdapAuthenticatorSpec
CLOUDSTACK-8647: formatted LdapAuthenticatorSpec
CLOUDSTACK-8647: UI for trust AD feature
CLOUDSTACK-8647 added unittests for new methods in ldapmanager
CLOUDSTACK-8647 unittests for LinkDomainToLdap api command
CLOUDSTACK-8647: fixed unittests
CLOUDSTACK-8647 support for assigning and admin to linked ldap domain
CLOUDSTACK-8647 added nested group enabled config in ldap
CLOUDSTACK-8647 added account_type to the linkDomainToLdap API
CLOUDSTACK-8647 changed the authentication flow
CLOUDSTACK-8647 added new api linkLdapToDomain
CLOUDSTACK-8647: added cmd and response class for the new api
Signed-off-by: Rajani Karuturi <rajani.karuturi@citrix.com>
- When stopping/detroying a VPC router and creating a new one via the addition os a VM, the networks are no plugged correctly
- The ETH1 ends up with the IP of the guest nic, which remains down. This causes routes problems and the VMs are not reachable via the pub IPs
- The test will create a VPC, add 2 tier, 2 VMs, ACL, PF and SSH into the VM
- Then it will stop the router, destroy the router, add another VM to 1 tier and check that we can reach all the VMs
Guys, can you review it? things need to be discussed:
(1) this supports KVM/QCOW2 only. Anyone want to implement for other Hypervisor/format ?
(2) The original data volume (on primary storage) will be removed.
(3) The script uses the default timeout in libvirtComputingResource. Do we need to add one in global configuration (like copy.volume.wait or backup.snapshot.wait, create.volume.from.snapshot.wait)
(4) In scripts/storage/qcow2/managesnapshot.sh, I use "qemu-img convert -f qcow2 -O qcow2" to copy the snapshot from secondary to primary (hence there is no base image file), instead of "cp -f", this is because convert is faster than cp in my testing.
* pr/732:
CLOUDSTACK-5863: revert volume snapshot for KVM/QCOW2
Signed-off-by: Wei Zhou <w.zhou@tech.leaseweb.com>
CLOUDSTACK-8785: Proper enforcement of retry count (max.retries) for all work type handled by HighAvailability manager
Retry count is properly enforced for all work types in HA manager. Also reorganized some of the code for easy testing.
Signed-off-by: Koushik Das <koushik@apache.org>
Made following fixes in simulator
- Support for ScaleVmCommand/NetworkRulesVmSecondaryIpCommand in resource layer
- Added support for scaling up a running VM in simulator
- Fixed some method names not following convention
In order to test PR #725 using simulator some of these changes are needed.
Based on the way HV check is present in the scale VM API, had to explicitly put simulator related check to allow support. The ideal way would be to remove all these HV specific check from code and made them some configuration (by putting them in hypervisor_capabilities table in DB). But that will be a bigger effort outside the scope of this PR.
Signed-off-by: Koushik Das <koushik@apache.org>
This reverts commit cd7218e241, reversing
changes made to f5a7395cc2.
Reason for Revert:
noredist build failed with the below error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project cloud-plugin-hypervisor-vmware: Compilation failure
[ERROR] /home/jenkins/acs/workspace/build-master-noredist/plugins/hypervisors/vmware/src/com/cloud/hypervisor/guru/VMwareGuru.java:[484,12] error: non-static variable logger cannot be referenced from a static context
[ERROR] -> [Help 1]
even the normal build is broken as reported by @koushik-das on dev list
http://markmail.org/message/nngimssuzkj5gpbz
CLOUDSTACK-8721: Fixed Setting details of VM through API results in removal of all other details except the one passed in API
* pr/673:
CLOUDSTACK-8721: Fixed Setting details of VM through API results in removal of all other details except the one passed in API
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Support for ScaleVmCommand/NetworkRulesVmSecondaryIpCommand in resource layer
- Added support for scaling up a running VM in simulator
- Fixed some method names not following convention
Fix for the NicVO.java regression.Renamed set*() methods to correct naming.
* pr/726:
Fix for the NicVO.java regression.
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8687: Prepare template only on a given storage poolUpdate prepare template api to seed/prepare a template only on a given primary storage. Currently, the prepare template api will seed/prepare a given template on all the primary storage pools in a zone. If however, a user wishes to prepare a template only a particular storage pool, it isn't possible. Updated the api to take storage pool id as an optional parameter. If the pool id is provided then the template is prepared only on the given primary storage pool. Also added unit tests to validate the functionality
* pr/715:
CLOUDSTACK-8687: Unit tests for validating the prepare template functionality. These tests validate that the templates get scheduled for seeding. Additionally, if a template is already seeded, we do not try and seed it again. Tests also validate that templates are seeded to storage pools which are available.
CLOUDSTACK-8687: Update prepare template api to seed/prepare a template only on a given primary storage. Currently, the prepare template api will seed/prepare a given template on all the primary storage pools in a zone. If however, a user wishes to prepare a template only a particular storage pool, it isn't possible. Updated the api to take storage pool id as an optional parameter. If the pool id is provided then the template is prepared only on the given primary storage pool
Signed-off-by: Rajani Karuturi <rajanikaruturi@gmail.com>
Refactored Nic.java for readability.Only changed names and reorganized them! No functional implementation change.
See issue CLOUDSTACK-8736 for ongoing effort to clean up network code.
* pr/707:
Renamed internal variables of NicProfile.java to adhere to the naming convention.
Refactored Nic.java for readability.
Signed-off-by: Rajani Karuturi <rajanikaruturi@gmail.com>
These tests validate that the templates get scheduled for seeding. Additionally,
if a template is already seeded, we do not try and seed it again. Tests also
validate that templates are seeded to storage pools which are available.
only on a given primary storage. Currently, the prepare template api
will seed/prepare a given template on all the primary storage pools in
a zone. If however, a user wishes to prepare a template only a
particular storage pool, it isn't possible. Updated the api to take
storage pool id as an optional parameter. If the pool id is provided
then the template is prepared only on the given primary storage pool
Changed methodnames according to Nic.java refactor.
Fixed NicVO.java due to regression from Nic.java refactor.
Fixed VmWareGuru.java after Nic.java refactor.
See issue CLOUDSTACK-8736 for ongoing effort to clean up network code.
Refactored NicProfile.java for readability.Only changed names and reorganized them! No functional implementation change.
See issue CLOUDSTACK-8736 for ongoing effort to clean up network code.
* pr/705:
Refactored NicProfile.java for readability.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
Cloudstack 8656: do away with more silently ignoring exceptions.a lot of messages added.
some restructuring for test exception assertions and try-with-resource blocks
* pr/654: (29 commits)
CLOUDSTACK-8656: more logging instead of sysout
CLOUDSTACK-8656: use catch block for validation
CLOUDSTACK-8656: class in json specified not found
CLOUDSTACK-8656: removed unused classes
CLOUDSTACK-8656: restructure of tests
CLOUDSTACK-8656: reorganise sychronized block
CLOUDSTACK-8656: restructure tests to ensure exception throwing
CLOUDSTACK-8656: validate the throwing of ServerApiException
CLOUDSTACK-8656: logging ignored exceptions
CLOUDSTACK-8656: try-w-r removes need for empty catch block
CLOUDSTACK-8656: try-w-r instead of clunckey close-except
CLOUDSTACK-8656: deal with empty SQLException catch block by try-w-r
CLOUDSTACK-8656: unnecessary close construct removed
CLOUDSTACK-8656: message about timed buffer logging
CLOUDSTACK-8656: message about invalid number from store
CLOUDSTACK-8656: move cli test tool to separate file
CLOUDSTACK-8656: exception is the rule for some tests
CLOUDSTACK-8656: network related exception logging
CLOUDSTACK-8656: reporting ignored exceptions in server
CLOUDSTACK-8656: log in case we are on a platform not supporting UTF8
...
Signed-off-by: Remi Bergsma <github@remi.nl>
Interface changes related to CLOUDSTACK-8580See issue CLOUDSTACK-8580 and individual commits.
* pr/680:
Made interface changes related to CLOUDSTACK-8580
Signed-off-by: Remi Bergsma <github@remi.nl>
Added responses to ListCapabilities to reflect CLOUDSTACK-8580 changes.
This to add these options to the gui. See issue CLOUDSTACK-8580.
3be14e978a
Removed comments as proposed by Daan Hoogland.
See issue CLOUDSTACK-8133 for more information.
Added null check by comment of Koushik Das.
Added brackets by comment of Wido den Hollander.
Removed a call to findById() by comment of Koushik Das.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
Implemented condition that only admin or owner of the template can change its permissions ..... using updateTemplatePermissions API
Consider this scenario :
In a domain, there are three User Accounts UA1, UA2,UA3
A private template is registered by UA1
Through the updateTemplatePermissions API, UA1 gives permission to both UA2 and UA3
Now, UA2, having been shared the template, can remove the permission of UA3(or add permissions to another account).
EXPECTED BEHAVIOR :
UA2 should not be able to to add/remove permissions of other accounts.
* pr/658:
Implemented condition that only admin or owner of the template can change its permissions using updateTemplatePermissions API
Signed-off-by: Remi Bergsma <github@remi.nl>
* pr/547:
CLOUDSTACK-8601. VMFS storage added as local storage can be re-added as shared storage. Fail addition of a VMFS shared storage pool in case it has already been added as local storage in CS.
Signed-off-by: Mike Tutkowski <mike.tutkowski@solidfire.com>
CLOUDSTACK-8704: Schedule restart of router VMs ahead of user VMs as part of HA
VRs are scheduled for HA ahead of user VMs.
Refer to the bug for more details.
* pr/656:
CLOUDSTACK-8704: Schedule restart of router VMs ahead of user VMs as part of HA VRs are scheduled for HA ahead of user VMs
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8711: public_ip type resource count for an account is not decremented upon IP range deletionProblem:
--------------------------
When you add an IP range and associate it to an account then resource count of public_ip will be updated to the range length.
After some time try to delete this range and the resource count of public_up for this account is not descremented and is causing account not to add any more public IPs to it once it reaches the resource limit.
RCA:
----------------
We were not decrement the count while deleting the IP range that was associated to an account.
Fix:
-------------
Up on deletion we are decrementing the resource count for public_up now.
* pr/662:
Bug-Id: CS-27335: public_ip type resource count for an account is not decremented upon IP range deletion
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8709 No out of band migrate alert for non-routers
* pr/666:
CLOUDSTACK-8709 No out of band migrate alert for non-routers
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8714 Restore VM (Re-install VM) with enable.storage.migration set to false fails
* pr/659:
Bug-ID:CS-27160: Restore VM (Re-install VM) with enable.storage.migration set to false fails, later fails to start up VM too
Signed-off-by: Remi Bergsma <github@remi.nl>
Reviewed-By: Anthony
Changes:
- Try to reuse the storage pools for READY disks if the pool fits the deployment plan
- Try to use the last_host if it has free capacity but no reserved capacity
Signed-off-by: Maneesha.P <maneesha.papireddygari@citrix.com>
Instead of putting the host to Alert state immediately, the investigators should be allowed to run for some time based on alert.wait global config.
At the end of this interval if the host state still cannot be determined then put the host in Alert. Also updated some of the log messages.
This closes#621
Also change the logging a bit so that you get useful logs when
not running on DEBUG level
Signed-off-by: Wido den Hollander <wido@widodh.nl>
This closes#580
If SSVM cannot be started then the existing entry is removed and a new SSVM gets created. Made similar changes for CPVM as well.
Also cleaned up some log messages in SSVM and CPVM manager code.
- Login is based on sessionkey HttpOnly Cookie
- ApiServlet does login verification using sessionKey from both the request cookies
and the API parameters. In both cases, if either or both are passed they should
match the sessionKey stored in the current session of the HttpRequest
- UI: it no longer needs to read or set sessionkey cookie
- UI: it no longer needs to return g_sessionKey value in the API requests, though
to support a sso mechanism g_sessionKey is still passed in the API is not null
- Secure jsessionid cookie is set to be HttpOnly and Secure
- SAML login should also set HttpOnly cookie before redirecting to UI
- SAML: listIdps & getSPMetadata APIs are readonly now, won't log out a logged in user
Performed tests (login, saml login if applicable, page refreshes, opening
multiple tabs, logout) with following combinations:
- SAML disabled, normal auth as admin, domain-admin and user
- SAML enabled, normal auth as admin, domain-admin and user; and saml sso as
admin, domain-admin and user
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#574
This closes#308
- Make sure the new password replaces the old one in the queue
- Updated the patch files for XenServer
- Updated the script path on LibvirtComputing class
- Adding update_host_passwd to VRScripts
- Add implementation to CitrixUpdateHostPasswordCommandWrapper
- Improve testUpdateHostPasswordCommand() unit test on CitrixRequestWrapperTest
- Adding update_host_passwd.sh script
- Adding the host IP address as an instance variable on UpdateHostPasswordCommand
- Improving the Unit Test (LibvirtComputingResourceTest) to get it covering the new code
- Make sure doUpdateHostPassword() doesn't get called if flag is set to false
- Do not update XenServer hosts if the cluster ID is not informed
This closes#559
- Changed location of the update_host_passwd script
- Updated the patch files for XenServer
- Updated the script path on LibvirtComputing class
- Removed the hostIP from the LibvirtUpdateHostPasswordCommandWrapper execute() method
listing the hosts for migration for instance with tag
While preparing the suitable hosts we are accidentally removing the
incompatible (host does not have host tag) hosts from otherhost list(
incorrect use of List.retainAll).
- Modifying the LibvirtUpdateHostPasswordCommandWrapper in order to execute the script on the host
- Adding the script path to LibvirtComputingResource
- Adding the host IP address as an instance variable on UpdateHostPasswordCommand
- Improving the Unit Test (LibvirtComputingResourceTest) to get it covering the new code
* Move config options to SAML plugin
This moves all configuration options from Config.java to SAML auth manager. This
allows us to use the config framework.
* Make SAML2UserAuthenticator validate SAML token in httprequest
* Make logout API use ConfigKeys defined in saml auth manager
* Before doing SAML auth, cleanup local states and cookies
* Fix configurations in 4.5.1 to 4.5.2 upgrade path
* Fail if idp has no sso URL defined
* Add a default set of SAML SP cert for testing purposes
Now to enable and use saml, one needs to do a deploydb-saml after doing a deploydb
* UI remembers login selections, IDP server
- CLOUDSTACK-8458:
* On UI show dropdown list of discovered IdPs
* Support SAML Federation, where there may be more than one IdP
- New datastructure to hold metadata of SP or IdP
- Recursive processing of IdP metadata
- Fix login/logout APIs to get new interface and metadata data structure
- Add org/contact information to metadata
- Add new API: listIdps that returns list of all discovered IdPs
- Refactor and cleanup code and tests
- CLOUDSTACK-8459:
* Add HTTP-POST binding to SP metadata
* Authn requests must use either HTTP POST/Artifact binding
- CLOUDSTACK-8461:
* Use unspecified x509 cert as a fallback encryption/signing key
In case a IDP's metadata does not clearly say if their certificates need to be
used as signing or encryption and we don't find that, fallback to use the
unspecified key itself.
- CLOUDSTACK-8462:
* SAML Auth plugin should not do authorization
This removes logic to create user if they don't exist. This strictly now
assumes that users have been already created/imported/authorized by admins.
As per SAML v2.0 spec section 4.1.2, the SP provider should create authn requests using
either HTTP POST or HTTP Artifact binding to transfer the message through a
user agent (browser in our case). The use of HTTP Redirect was one of the reasons
why this plugin failed to work for some IdP servers that enforce this.
* Add new User Source
By reusing the source field, we can find if a user has been SAML enabled or not.
The limitation is that, once say a user is imported by LDAP and then SAML
enabled - they won't be able to use LDAP for authentication
* UI should allow users to pass in domain they want to log into, though it is
optional and needed only when a user has accounts across domains with same
username and authorized IDP server
* SAML users need to be authorized before they can authenticate
- New column entity to track saml entity id for a user
- Reusing source column to check if user is saml enabled or not
- Add new source types, saml2 and saml2disabled
- New table saml_token to solve the issue of multiple users across domains and
to enforce security by tracking authn token and checking the samlresponse for
the tokens
- Implement API: authorizeSamlSso to enable/disable saml authentication for a
user
- Stubs to implement saml token flushing/expiry
- CLOUDSTACK-8463:
* Use username attribute specified in global setting
Use username attribute defined by admin from a global setting
In case of encrypted assertion/attributes:
- Decrypt them
- Check signature if provided to check authenticity of message using IdP's
public key and SP's private key
- Loop through attributes to find the username
- CLOUDSTACK-8538:
* Add new global config for SAML request sig algorithm
- CLOUDSTACK-8539:
* Add metadata refresh timer task and token expiring
- Fix domain path and save it to saml_tokens
- Expire hour old saml tokens
- Refresh metadata based on timer task
- Fix unit tests
This closes#489
(cherry picked from commit 20ce346f3a)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
client/WEB-INF/classes/resources/messages_hu.properties
plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resource/wrapper/xenbase/CitrixCheckHealthCommandWrapper.java
plugins/user-authenticators/saml2/src/org/apache/cloudstack/api/command/SAML2LoginAPIAuthenticatorCmd.java
ui/scripts/ui-custom/login.js
- Make sure the doUpdateHostPassword() method returns the result from the answer, not always true
- Added the LibvirtUpdateHostPasswordCommandWrapper class, which will handle the changes in the KVM agent
That's will be further implemented in the future. The command will avoid the agent to complain about Unsupported command
- Added a test to make sure the current implementation of the comment works
- If changes in the future, it will also require changes. In that way, we make sure nobody will break it
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
This closes#527
- 2 years ago a change in that method caused an issue when trying to update the hosts password:
- they would be updated in the databse but not in the host it self, causing the hosts to disconnect after a management server resstart
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
- By doing so the check is simple and will require less effort when adding other hypervisors.
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
- The code was hard to maintain because updating a host or all the hosts in a cluster was handled in the same method
- Created updateHost and updateCluster password in both ResourceManager and ManagementServer interfaces/classes
- The chck for whihc method to use is done in the API level
- Started adding the support for KVM host passwd update
No API changes are needed and it will be backwards compatible.
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
- New implementation uses nanoseconds. Due to that, the places where the Profiler is used as a Monitor and/or
a stopwatch will suffer with the difference in the return
- Also added a getDuration(), which returns the time in nanoseconds in case someone wants to use it instead
- Added an extra test to check if the getDuration() works fine with nanoseconds
- Fixed the test that checks the time in milliseconds: I added an error margin to cover the test better
Signed-off-by: Daan Hoogland <daan@onecht.net>
Removing the previous logic and adding a method to check if the CIDR is 31 bit prefixed
Adding tests for the new method
Signed-off-by: Daan Hoogland <daan.hoogland@gmail.com>
This closes#303
This allows non-root users to add KVM hosts, the user should be an admin or
added to sudoers to execute sudo cloudstack-setup-agent.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Signed-off-by: Remi Bergsma <apache@remi.nl>
This closes#288
(cherry picked from commit d2b0c1a32b)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This allows non-root users to add KVM hosts, the user should be an admin or
added to sudoers to execute sudo cloudstack-setup-agent.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Signed-off-by: Remi Bergsma <apache@remi.nl>
This closes#288
- Changing the implementation of the NetUtils.ipRangesOverlap() a little bit in order to be compliant with RFC 3021
- 2 unit tests added
- ranges from 0 to 255 covered by the tests, which also test the negative cases.
For KVM agent, guid is configurable in agent.properties, this fix allows the
configuration to work by removing string tail (the -LibvirtComputingResource suffix).
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#286
(cherry picked from commit 803b946c2f)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
For KVM agent, guid is configurable in agent.properties, this fix allows the
configuration to work by removing string tail (the -LibvirtComputingResource suffix).
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#286
We add a new contract to pass Http request to authentication plugin system. In
the default login API, we disallow non-POST requests.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This commit does not implement new functionality:
* Fixes duplicate parameter in API docs.
* Fixes a bunch of typos.
* Add interfaces to make it easier for the FirewallService interface.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#249
... of new volumes. Following changes are implemented 1. Disable or enable a pool with the
updateStoragePool api. A new 'enabled' parameter added for the same. 2. When a
pool is disabled the state of the pool is updated to 'Disabled' in the db. On
enabling it is updated back to 'Up'. Alert is raised when a pool is disabled or
enabled. 3. Updated other storage providers to also honour the disabled state.
4. A disabled pool is skipped by allocators for provisioing of new volumes. 5.
Since the allocators skip a disabled pool for provisioning of volumes, the
volumes are also not listed as a destination for volume migration.
FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Disabling+Storage+Pool+for+Provisioning
This closes#257
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
VMs in transition states - Starting, Stopping, Migrating - are also taken into account for enforcing "max. guest limit"
(cherry picked from commit 3100fc1554)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The PR #211 introduced changes where the abstract investigator testIpAddress()
would return other Status, which previously only returned null, Up or Down. In
this patch we introduce a new Status "Unknown" that replaces null's semantics.
The important changes #211 introduced was the debugging statements as semantically
the changes would work same as the consumers of testIpAddress() method only used
if returned values were Up or Down and in other cases (null, Alert etc) it would
simply continue to loop through the resources being investigated.
Keeping the debug logs, this commit only replaces the previously returned null
values with Status.Unknown and fixed the debug statements to reflect the same.
In case of trapped exceptions too, we return Unknown status but log the exception
we trapped.
server: add null assertions and remove dead code with testIpAddress usage
This closes#222
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 7a1cb28c9f)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The PR #211 introduced changes where the abstract investigator testIpAddress()
would return other Status, which previously only returned null, Up or Down. In
this patch we introduce a new Status "Unknown" that replaces null's semantics.
The important changes #211 introduced was the debugging statements as semantically
the changes would work same as the consumers of testIpAddress() method only used
if returned values were Up or Down and in other cases (null, Alert etc) it would
simply continue to loop through the resources being investigated.
Keeping the debug logs, this commit only replaces the previously returned null
values with Status.Unknown and fixed the debug statements to reflect the same.
In case of trapped exceptions too, we return Unknown status but log the exception
we trapped.
server: add null assertions and remove dead code with testIpAddress usage
This closes#222
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
From b3f18e7d74, the zone level systemvm local
storage setting never worked as it needed to be moved to config depot.
(cherry picked from commit 279efb0432)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
plugins/network-elements/internal-loadbalancer/src/org/apache/cloudstack/network/lb/InternalLoadBalancerVMManagerImpl.java
server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java
From b3f18e7d74, the zone level systemvm local
storage setting never worked as it needed to be moved to config depot.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
When a full cluster is down or unreachable,
CloudStack currently reports everything the
same as the last known state, which is usually
Up. When it cannot reach a host and cannot
reach another host in the same cluster either,
it returns null and says "I don't know". This
prevents it from reporting the problem. Now,
we return an Alert or Disconnected state so
proper action can be taken.
Also logging was added, so we know what part
of the code put it to Alert or Disconnected.
When a full cluster is down or unreachable,
CloudStack currently reports everything the
same as the last known state, which is usually
Up. When it cannot reach a host and cannot
reach another host in the same cluster either,
it returns null and says "I don't know". This
prevents it from reporting the problem. Now,
we return an Alert or Disconnected state so
proper action can be taken.
Also logging was added, so we know what part
of the code put it to Alert or Disconnected.
ToDayStart/End because ParamProcessWorker.java is taking care of building proper
date format with day start/end
This closes#185
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This allows changing the hvm flag for a template after the template has been
registered.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 3e8b12bbb1)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
inconsistent state (NotUploaded or UploadInProgress) and doesn't transition to any terminal
state (Uploaded, Uploaded). As a result these volumes/templates cannot be removed. Fix is to handle such
volume/template entries correctly.
- pool allocation checks for both root and data disks
- NPE checks to not add null object in collection or try to migrate null VM
- HA work tries need to increment and be given up when max retries are crossed
- VM creation should check IP address format for IPv4 and IPv6
- If userdata is not supported by a network, then fail early if userdata, ssh key,
or password enabled template is passed/used
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 24435dd6bc)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- pool allocation checks for both root and data disks
- NPE checks to not add null object in collection or try to migrate null VM
- HA work tries need to increment and be given up when max retries are crossed
- VM creation should check IP address format for IPv4 and IPv6
- If userdata is not supported by a network, then fail early if userdata, ssh key,
or password enabled template is passed/used
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Rohit Yadav
Remi Bergsma
Rajani Karuturi
Rene Moser
wilderrodrigues
Wei Zhou
Koushik Das
Rafael Weingartner
Kshitij Kansal
Rajani Karuturi
Boris Schrijver
radu-stefanache
Devdeep Singh
Wido den Hollander
Milamber
Maneesha.P
Mike Tutkowski
Anshul Gangwar
Damodar
Prachi Damle
Daan Hoogland
Kshitij Kansal
Daan Hoogland
Jayapal
Sudhansu
Likitha Shetty
Rene Moser
Sanjay Tripathi
Abhinandan Prateek
Rafael da Fonseca
Kishan Kavala
Daan Hoogland
Santhosh Edukulla
Ilia Shakitko