Reviewed-By: Anthony
Changes:
- Try to reuse the storage pools for READY disks if the pool fits the deployment plan
- Try to use the last_host if it has free capacity but no reserved capacity
Signed-off-by: Maneesha.P <maneesha.papireddygari@citrix.com>
Instead of putting the host to Alert state immediately, the investigators should be allowed to run for some time based on alert.wait global config.
At the end of this interval if the host state still cannot be determined then put the host in Alert. Also updated some of the log messages.
This closes#621
Also change the logging a bit so that you get useful logs when
not running on DEBUG level
Signed-off-by: Wido den Hollander <wido@widodh.nl>
This closes#580
If SSVM cannot be started then the existing entry is removed and a new SSVM gets created. Made similar changes for CPVM as well.
Also cleaned up some log messages in SSVM and CPVM manager code.
- Login is based on sessionkey HttpOnly Cookie
- ApiServlet does login verification using sessionKey from both the request cookies
and the API parameters. In both cases, if either or both are passed they should
match the sessionKey stored in the current session of the HttpRequest
- UI: it no longer needs to read or set sessionkey cookie
- UI: it no longer needs to return g_sessionKey value in the API requests, though
to support a sso mechanism g_sessionKey is still passed in the API is not null
- Secure jsessionid cookie is set to be HttpOnly and Secure
- SAML login should also set HttpOnly cookie before redirecting to UI
- SAML: listIdps & getSPMetadata APIs are readonly now, won't log out a logged in user
Performed tests (login, saml login if applicable, page refreshes, opening
multiple tabs, logout) with following combinations:
- SAML disabled, normal auth as admin, domain-admin and user
- SAML enabled, normal auth as admin, domain-admin and user; and saml sso as
admin, domain-admin and user
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#574
This closes#308
- Make sure the new password replaces the old one in the queue
- Updated the patch files for XenServer
- Updated the script path on LibvirtComputing class
- Adding update_host_passwd to VRScripts
- Add implementation to CitrixUpdateHostPasswordCommandWrapper
- Improve testUpdateHostPasswordCommand() unit test on CitrixRequestWrapperTest
- Adding update_host_passwd.sh script
- Adding the host IP address as an instance variable on UpdateHostPasswordCommand
- Improving the Unit Test (LibvirtComputingResourceTest) to get it covering the new code
- Make sure doUpdateHostPassword() doesn't get called if flag is set to false
- Do not update XenServer hosts if the cluster ID is not informed
This closes#559
- Changed location of the update_host_passwd script
- Updated the patch files for XenServer
- Updated the script path on LibvirtComputing class
- Removed the hostIP from the LibvirtUpdateHostPasswordCommandWrapper execute() method
listing the hosts for migration for instance with tag
While preparing the suitable hosts we are accidentally removing the
incompatible (host does not have host tag) hosts from otherhost list(
incorrect use of List.retainAll).
- Modifying the LibvirtUpdateHostPasswordCommandWrapper in order to execute the script on the host
- Adding the script path to LibvirtComputingResource
- Adding the host IP address as an instance variable on UpdateHostPasswordCommand
- Improving the Unit Test (LibvirtComputingResourceTest) to get it covering the new code
* Move config options to SAML plugin
This moves all configuration options from Config.java to SAML auth manager. This
allows us to use the config framework.
* Make SAML2UserAuthenticator validate SAML token in httprequest
* Make logout API use ConfigKeys defined in saml auth manager
* Before doing SAML auth, cleanup local states and cookies
* Fix configurations in 4.5.1 to 4.5.2 upgrade path
* Fail if idp has no sso URL defined
* Add a default set of SAML SP cert for testing purposes
Now to enable and use saml, one needs to do a deploydb-saml after doing a deploydb
* UI remembers login selections, IDP server
- CLOUDSTACK-8458:
* On UI show dropdown list of discovered IdPs
* Support SAML Federation, where there may be more than one IdP
- New datastructure to hold metadata of SP or IdP
- Recursive processing of IdP metadata
- Fix login/logout APIs to get new interface and metadata data structure
- Add org/contact information to metadata
- Add new API: listIdps that returns list of all discovered IdPs
- Refactor and cleanup code and tests
- CLOUDSTACK-8459:
* Add HTTP-POST binding to SP metadata
* Authn requests must use either HTTP POST/Artifact binding
- CLOUDSTACK-8461:
* Use unspecified x509 cert as a fallback encryption/signing key
In case a IDP's metadata does not clearly say if their certificates need to be
used as signing or encryption and we don't find that, fallback to use the
unspecified key itself.
- CLOUDSTACK-8462:
* SAML Auth plugin should not do authorization
This removes logic to create user if they don't exist. This strictly now
assumes that users have been already created/imported/authorized by admins.
As per SAML v2.0 spec section 4.1.2, the SP provider should create authn requests using
either HTTP POST or HTTP Artifact binding to transfer the message through a
user agent (browser in our case). The use of HTTP Redirect was one of the reasons
why this plugin failed to work for some IdP servers that enforce this.
* Add new User Source
By reusing the source field, we can find if a user has been SAML enabled or not.
The limitation is that, once say a user is imported by LDAP and then SAML
enabled - they won't be able to use LDAP for authentication
* UI should allow users to pass in domain they want to log into, though it is
optional and needed only when a user has accounts across domains with same
username and authorized IDP server
* SAML users need to be authorized before they can authenticate
- New column entity to track saml entity id for a user
- Reusing source column to check if user is saml enabled or not
- Add new source types, saml2 and saml2disabled
- New table saml_token to solve the issue of multiple users across domains and
to enforce security by tracking authn token and checking the samlresponse for
the tokens
- Implement API: authorizeSamlSso to enable/disable saml authentication for a
user
- Stubs to implement saml token flushing/expiry
- CLOUDSTACK-8463:
* Use username attribute specified in global setting
Use username attribute defined by admin from a global setting
In case of encrypted assertion/attributes:
- Decrypt them
- Check signature if provided to check authenticity of message using IdP's
public key and SP's private key
- Loop through attributes to find the username
- CLOUDSTACK-8538:
* Add new global config for SAML request sig algorithm
- CLOUDSTACK-8539:
* Add metadata refresh timer task and token expiring
- Fix domain path and save it to saml_tokens
- Expire hour old saml tokens
- Refresh metadata based on timer task
- Fix unit tests
This closes#489
(cherry picked from commit 20ce346f3a)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
client/WEB-INF/classes/resources/messages_hu.properties
plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resource/wrapper/xenbase/CitrixCheckHealthCommandWrapper.java
plugins/user-authenticators/saml2/src/org/apache/cloudstack/api/command/SAML2LoginAPIAuthenticatorCmd.java
ui/scripts/ui-custom/login.js
- Make sure the doUpdateHostPassword() method returns the result from the answer, not always true
- Added the LibvirtUpdateHostPasswordCommandWrapper class, which will handle the changes in the KVM agent
That's will be further implemented in the future. The command will avoid the agent to complain about Unsupported command
- Added a test to make sure the current implementation of the comment works
- If changes in the future, it will also require changes. In that way, we make sure nobody will break it
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
This closes#527
- 2 years ago a change in that method caused an issue when trying to update the hosts password:
- they would be updated in the databse but not in the host it self, causing the hosts to disconnect after a management server resstart
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
- By doing so the check is simple and will require less effort when adding other hypervisors.
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
- The code was hard to maintain because updating a host or all the hosts in a cluster was handled in the same method
- Created updateHost and updateCluster password in both ResourceManager and ManagementServer interfaces/classes
- The chck for whihc method to use is done in the API level
- Started adding the support for KVM host passwd update
No API changes are needed and it will be backwards compatible.
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
- New implementation uses nanoseconds. Due to that, the places where the Profiler is used as a Monitor and/or
a stopwatch will suffer with the difference in the return
- Also added a getDuration(), which returns the time in nanoseconds in case someone wants to use it instead
- Added an extra test to check if the getDuration() works fine with nanoseconds
- Fixed the test that checks the time in milliseconds: I added an error margin to cover the test better
Signed-off-by: Daan Hoogland <daan@onecht.net>
Removing the previous logic and adding a method to check if the CIDR is 31 bit prefixed
Adding tests for the new method
Signed-off-by: Daan Hoogland <daan.hoogland@gmail.com>
This closes#303
This allows non-root users to add KVM hosts, the user should be an admin or
added to sudoers to execute sudo cloudstack-setup-agent.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Signed-off-by: Remi Bergsma <apache@remi.nl>
This closes#288
(cherry picked from commit d2b0c1a32b)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This allows non-root users to add KVM hosts, the user should be an admin or
added to sudoers to execute sudo cloudstack-setup-agent.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Signed-off-by: Remi Bergsma <apache@remi.nl>
This closes#288
- Changing the implementation of the NetUtils.ipRangesOverlap() a little bit in order to be compliant with RFC 3021
- 2 unit tests added
- ranges from 0 to 255 covered by the tests, which also test the negative cases.
For KVM agent, guid is configurable in agent.properties, this fix allows the
configuration to work by removing string tail (the -LibvirtComputingResource suffix).
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#286
(cherry picked from commit 803b946c2f)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
For KVM agent, guid is configurable in agent.properties, this fix allows the
configuration to work by removing string tail (the -LibvirtComputingResource suffix).
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#286
We add a new contract to pass Http request to authentication plugin system. In
the default login API, we disallow non-POST requests.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This commit does not implement new functionality:
* Fixes duplicate parameter in API docs.
* Fixes a bunch of typos.
* Add interfaces to make it easier for the FirewallService interface.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#249
... of new volumes. Following changes are implemented 1. Disable or enable a pool with the
updateStoragePool api. A new 'enabled' parameter added for the same. 2. When a
pool is disabled the state of the pool is updated to 'Disabled' in the db. On
enabling it is updated back to 'Up'. Alert is raised when a pool is disabled or
enabled. 3. Updated other storage providers to also honour the disabled state.
4. A disabled pool is skipped by allocators for provisioing of new volumes. 5.
Since the allocators skip a disabled pool for provisioning of volumes, the
volumes are also not listed as a destination for volume migration.
FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Disabling+Storage+Pool+for+Provisioning
This closes#257
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Removes awsapi db properties usage across codebase
- Removes references from spring xmls, test cases and TransactionLegacy
- Adds sql command to drop database cloudbridge in schema-451to460-cleanup.sql
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
VMs in transition states - Starting, Stopping, Migrating - are also taken into account for enforcing "max. guest limit"
(cherry picked from commit 3100fc1554)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The PR #211 introduced changes where the abstract investigator testIpAddress()
would return other Status, which previously only returned null, Up or Down. In
this patch we introduce a new Status "Unknown" that replaces null's semantics.
The important changes #211 introduced was the debugging statements as semantically
the changes would work same as the consumers of testIpAddress() method only used
if returned values were Up or Down and in other cases (null, Alert etc) it would
simply continue to loop through the resources being investigated.
Keeping the debug logs, this commit only replaces the previously returned null
values with Status.Unknown and fixed the debug statements to reflect the same.
In case of trapped exceptions too, we return Unknown status but log the exception
we trapped.
server: add null assertions and remove dead code with testIpAddress usage
This closes#222
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 7a1cb28c9f)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The PR #211 introduced changes where the abstract investigator testIpAddress()
would return other Status, which previously only returned null, Up or Down. In
this patch we introduce a new Status "Unknown" that replaces null's semantics.
The important changes #211 introduced was the debugging statements as semantically
the changes would work same as the consumers of testIpAddress() method only used
if returned values were Up or Down and in other cases (null, Alert etc) it would
simply continue to loop through the resources being investigated.
Keeping the debug logs, this commit only replaces the previously returned null
values with Status.Unknown and fixed the debug statements to reflect the same.
In case of trapped exceptions too, we return Unknown status but log the exception
we trapped.
server: add null assertions and remove dead code with testIpAddress usage
This closes#222
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
From b3f18e7d74, the zone level systemvm local
storage setting never worked as it needed to be moved to config depot.
(cherry picked from commit 279efb0432)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
plugins/network-elements/internal-loadbalancer/src/org/apache/cloudstack/network/lb/InternalLoadBalancerVMManagerImpl.java
server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java
From b3f18e7d74, the zone level systemvm local
storage setting never worked as it needed to be moved to config depot.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
When a full cluster is down or unreachable,
CloudStack currently reports everything the
same as the last known state, which is usually
Up. When it cannot reach a host and cannot
reach another host in the same cluster either,
it returns null and says "I don't know". This
prevents it from reporting the problem. Now,
we return an Alert or Disconnected state so
proper action can be taken.
Also logging was added, so we know what part
of the code put it to Alert or Disconnected.
When a full cluster is down or unreachable,
CloudStack currently reports everything the
same as the last known state, which is usually
Up. When it cannot reach a host and cannot
reach another host in the same cluster either,
it returns null and says "I don't know". This
prevents it from reporting the problem. Now,
we return an Alert or Disconnected state so
proper action can be taken.
Also logging was added, so we know what part
of the code put it to Alert or Disconnected.
ToDayStart/End because ParamProcessWorker.java is taking care of building proper
date format with day start/end
This closes#185
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This allows changing the hvm flag for a template after the template has been
registered.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 3e8b12bbb1)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
inconsistent state (NotUploaded or UploadInProgress) and doesn't transition to any terminal
state (Uploaded, Uploaded). As a result these volumes/templates cannot be removed. Fix is to handle such
volume/template entries correctly.
- pool allocation checks for both root and data disks
- NPE checks to not add null object in collection or try to migrate null VM
- HA work tries need to increment and be given up when max retries are crossed
- VM creation should check IP address format for IPv4 and IPv6
- If userdata is not supported by a network, then fail early if userdata, ssh key,
or password enabled template is passed/used
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 24435dd6bc)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- pool allocation checks for both root and data disks
- NPE checks to not add null object in collection or try to migrate null VM
- HA work tries need to increment and be given up when max retries are crossed
- VM creation should check IP address format for IPv4 and IPv6
- If userdata is not supported by a network, then fail early if userdata, ssh key,
or password enabled template is passed/used
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
1. provide compatibility with the Big Cloud Fabric (BCF) controller
L2 Connectivity Service in both VPC and non-VPC modes
2. virtual network terminology updates: VNS --> BCF_SEGMENT
3. uses HTTPS with trust-always certificate handling
4. topology sync support with BCF controller
5. support multiple (two) BCF controllers with HA
6. support VM migration
7. support Firewall, Static NAT, and Source NAT with NAT enabled option
8. add VifDriver for Indigo Virtual Switch (IVS)
This closes#151
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- With the changes added by the rVPC work, the bump priority became deprecated.
This commit includes a refactor to get it removed from the following resources:
* Java classes
* domain_router table - removing the is_priority_bumpup column
* Fixing unit tests
All changes were tested with:
XenServer 6.2 running under our VMWare zone
CloudStack Management Server running on MacBook Pro
MySql running on MackBook Pro
Storage Type: Local
also fixed the issue wherein the successful uploads where also moving to
error state as the channelinactive is called after the end of successful
upload as well.
added a fileuploaded boolean to check when the channel is inactive.
Customise the messages showed during VPC restart based on redundancy
Do not show "Make Redundant" option if VPC is already redundant
Show on VPC details if a VPC is redundant
Fix the router state. do not show UNKNOW, but MASTER or BACKUP depending on the type of router
Implement the virtual_router_id to be passed as a boot parameter to the router
- it is needed for the keepalived configuration
Fix: RTNETLINK errors
- Management Server health check trying to create already existing interface
- Changes on update_config.py, cs_guestnetwork.py, merger.py
Fix: replace RRouTER_LOG in the CsRedundant.py per log file location
Fix: Guest Net address association during Router restart
- Changes on NicProfileHelper, NicProfileHelperImpl
Fix: aggregationExecution() method on VirtualNetworkApplianceManagerImpl
- Do not send an AggregationControlCommand to a non-configured router
Some classes have been formatted.
Fixing assign IPv4 on GuestNetworkGuru.java
Fixing getRouters() on VpcVirtualRouterElement.java
Fixing Fixing paths on keepalived.conf.templ
Refactor on the other files
* removed the "is redundant" flag form the addVpcRouterToGuestNetwork() method
* removed the "is redundant" flag from the removeVpcRouterFromGuestNetwork() method
* changed the path of the master.py file in the keepalived.conf.temp file
* the call to routerDao.addRouterToGuestNetwork() in the VpcRouterDeploymentDefinition is not needed. That step will be performed once a VM is created
- In addition, when restarting a VPC the routers will have the guest net configured, if any exists.
* Pushing the POM.xml as well, to use the old Jetty for now. Could not fix the logging problem. Will replace the POM with master version after VPC is done.
Fixing Finalize methods in the VpcVirtualNetworkApplianceManager
Making use ot eh RouterControlHelper instead of duplicating methods in the VirtualNetworkApplianceManager
[witp] - redundant vpc virtual routers have the same mac address, have to fix that.
There were more places were the amount of routers is checked.
Not throwing exception when routers > 1
[wip] - we need to refactor VpcViortualRouterElement in order to cope with methods returning true only for the sake of it.
Added a source column to the user table.
Source now has only two values UNKNOWN,LDAP with UNKNOWN being the
default and is an enum is com.cloud.User.
When the source is UNKNOWN, the old method of authenticating against all
the available authenticators is used. If a source is available, only
that particular authenticator will be used.
added overloaded methods in AccountService to createUserAccount and
createUser with source specified.
(cherry picked from commit 5da733072e)
on restart of management server, template sync runs. It checks for
templates in ssvm using the uniquename. If it doesnt find any, cleans
the directory. In case of uploaded templates, these are getting saved
using name instead on uniquename and hence template sync cant find them
and does cleanup. Using uniquename in template.properties now.
We do not throw the exceptions any more, so added info in the logs to say
what happend to this command.
Signed-off-by: Rajani Karuturi <rajanikaruturi@gmail.com>
(cherry picked from commit 659edb4d9b)
Added a source column to the user table.
Source now has only two values UNKNOWN,LDAP with UNKNOWN being the
default and is an enum is com.cloud.User.
When the source is UNKNOWN, the old method of authenticating against all
the available authenticators is used. If a source is available, only
that particular authenticator will be used.
added overloaded methods in AccountService to createUserAccount and
createUser with source specified.
We do not throw the exceptions any more, so added info in the logs to say
what happend to this command.
Signed-off-by: Rajani Karuturi <rajanikaruturi@gmail.com>
This is a plugin that puts in ovm3 support ranging from 3.3.1 to 3.3.2. Basic
functionality is in here, advanced networking etc..
Snapshots only work when a VM is stopped now due to the semantics of OVM's raw
image implementation (so snapshots should work on a storage level underneath the
hypervisor shrug)
This closes#113
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
During upload, POST url is saved in template_store_ref DB table. Now during download, same url is incorrectly returned back.
Fixed the code to cleanup POST url from DB on successful template upload.
- UI: use post when updating user
- S3: don't send s3 key in the response
- VPN: don't send preshared key in remoteaccessvpn api response
- Snapshot response should set zone id not volume's device id
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 02cadc3fb3)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- UI: use post when updating user
- S3: don't send s3 key in the response
- VPN: don't send preshared key in remoteaccessvpn api response
- Snapshot response should set zone id not volume's device id
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Adds X-XSS-Protection header
- Adds X-Content-Type-Options header
- Fixes to use json content type defined from global settings
- Uses secure cookie if enabled in global settings
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit b6b3494782)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Adds X-XSS-Protection header
- Adds X-Content-Type-Options header
- Fixes to use json content type defined from global settings
- Uses secure cookie if enabled in global settings
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- insecure authenticators excluded in configuration
- snapshot response should have zone
- remove vmsnapshots when removing accounts
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Backend should update if state was diabled and now has changed
- UI's fetch latest does not actually fetch latest
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 985a61652e)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Backend should update if state was diabled and now has changed
- UI's fetch latest does not actually fetch latest
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Design Document:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Allow+VM+listing+by+User+ID
- Adds column to VMInstance DAO
- Adds column in vm_instance table
- Adds column in the UserVMJoinVO
- Adds default admin user which has UID = 2
- Adds migration path that sets user_id to first user of the accountId that
owns the vm in vm_instance table
- Add arg on list VMs API to query by userId, add support in query layer
- Refactor VMInstanceVO and child classes to accept userId
- Add code to let service layer pass userId if loggedIn user belongs to same
account as the owner executing an API call or use first user from owner account
- In case of CPVM and SSVM use system user ID
- Fix unit tests and spring injections
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
1. getRouters() doesn't handle RestartNetwork with cleanup=true for basic zone,
because pod wouldn't be specific at the time.
2. The regression caused by the following fix. The variable "routers" was
overrided with some local values, result in only one of the routers in multiple
pods would return, thus only one router would be started.
commit 6dd5c3fd42
Author: Rohit Yadav <bhaisaab@apache.org>
Date: Thu Oct 11 18:30:00 2012 +0530
CLOUDSTACK-70: Improve restart network behaviour for basic network
(cherry picked from commit aaeadc5c44)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
pool, the source and destination pools cannot be local and cluster/zone and vice versa.
Cloudstack detects it and throws a exception. However, the end user only sees an
unexpected exception and not the reason for failure. Fixed it by making sure the
reason for the failure is correctly captured and shown to the end user.
(cherry picked from commit cffae8eef0)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
server/src/com/cloud/storage/VolumeApiServiceImpl.java
During VM creation, if vm.instancename.flag is set to true and hypervisor type is VMware, check if VM with the same hostname already exists in the zone.
(cherry picked from commit 5f9e4fddf3)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
For AttachVolume/DetachVolume API command, improve user error message in case of RuntimeException by throwing the exception instead of 'Unexpected Exception'.
(cherry picked from commit 4d7ede535d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
server/src/com/cloud/storage/VolumeApiServiceImpl.java
When migration fails instead of returning NULL, throw the exception.
(cherry picked from commit a5a65c7b55)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
If VM has been cold migrated across different VMware DCs, then unregister the VM from source host.
(cherry picked from commit 15b348632d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
If user hasn't supplied a display name for a VM, default it to VM name in listVolume response.
This behaviour is identical to listVirtualMachine response.
(cherry picked from commit ff7997a2b1)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The volume/template size and install path was not getting updated properly.
The upload status response should return these properties on completion of upload.
Fixed some issues with the polling logic to handle scenarios when SSVM is destroyed/down.
Also changed the status of the volume_store_ref entries so that subsequent operations work fine.
Call removeRawUsageRecords with interval (> 0) and it will clean up cloud_usage
table by removing records older than interval no. of days from today (current date)
and in case it runs when the job exec time is near, it will fail alerting
user to try again after a 15 min window.
There is an issue with async job scheduler, if this API were async it tries
to search and remove job from cloud_usage.async_job table and fails which is
why this API is sync and extends BaseCmd.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Separate global config to enable/disable Storage Migration during normal deployment
Introduced a configuration parameter named enable.storage.migration
(cherry picked from commit c55bc0b2d1)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-4201: listServiceOfferings API needs to be able to take virtualmachineid of SystemVM and return service offerings available for the vm to change service offering
(cherry picked from commit ee0f0a1cff)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
removing nio server as it is currently handling only https connections
and the parsing logic is also specific to agent communication.
current limitation of httpcomponents server is that the entire file is
read in memory. need figure out how to read it in chunks and send it
through a inputstreamreader to save on secondary storage.
This limits the likeliness of timing attacks against the API.
See http://codahale.com/a-lesson-in-timing-attacks/ for the
full rationale.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
server/src/com/cloud/api/ApiServer.java
server/src/com/cloud/user/AccountManagerImpl.java
(cherry picked from commit 9b4e39e837)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This limits the likeliness of timing attacks against the API.
See http://codahale.com/a-lesson-in-timing-attacks/ for the
full rationale.
Conflicts:
server/src/com/cloud/api/ApiServer.java
server/src/com/cloud/user/AccountManagerImpl.java
This removes extra whitespaces from the JSON serialized response.
After the fix, tested to work with:
- Present UI
- CloudMonkey
- Old buggy json parsers
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 921ad057de)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This removes extra whitespaces from the JSON serialized response.
After the fix, tested to work with:
- Present UI
- CloudMonkey
- Old buggy json parsers
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
If user hasn't supplied a display name for a VM, default it to VM name in listVolume response.
This behaviour is identical to listVirtualMachine response.
For AttachVolume/DetachVolume API command, improve user error message in case of RuntimeException by throwing the exception instead of 'Unexpected Exception'.
During VM creation, if vm.instancename.flag is set to true and hypervisor type is VMware, check if VM with the same hostname already exists in the zone.
During vmsync if StopCommand (issued as part of PowerOff/PowerMissing report) fails to stop VM (since VM is running on HV),
don't transition VM state to "Stopped" in CS db. Also added a check to throw ConcurrentOperationException if vm state is not
"Running" after start operation.
During vmsync if StopCommand (issued as part of PowerOff/PowerMissing report) fails to stop VM (since VM is running on HV),
don't transition VM state to "Stopped" in CS db. Also added a check to throw ConcurrentOperationException if vm state is not
"Running" after start operation.
on calling GetUploadParamsForVolume, persisting the metadata to db
validating the account limits and incrementing the appropriate limits
encoded the metadata on management server using preshared key
Changes:
- The event of deleteing an affinity group is published on the MessageBus so that IAM Service can listen and process the event, However the publish operation should not be handled within a DB transaction, since it may take longer and hold the DB transaction for long unnecessarily
- Publish any events to MessageBus outside of the transaction
Conflicts:
server/src/org/apache/cloudstack/affinity/AffinityGroupServiceImpl.java
Changes:
- When there is HA we try to redeploy the affected vm using regular planners and if that fails we retry using the special planner for HA (which skips checking disable threshold)
Now because of job framework the InsufficientCapacittyException gets masked and the special planners are not called. Job framework needs to be fixed to rethrow the correct exception.
- Also the VM Work Job framework is not setting the DeploymentPlanner to the VmWorkJob. So the HA Planner being passed by HAMgr was not getting used.
- Now the job framework sets the planner passed in by any caller of the VM Start operation, to the job
Changes:
- When there is HA we try to redeploy the affected vm using regular planners and if that fails we retry using the special planner for HA (which skips checking disable threshold)
Now because of job framework the InsufficientCapacittyException gets masked and the special planners are not called. Job framework needs to be fixed to rethrow the correct exception.
- Also the VM Work Job framework is not setting the DeploymentPlanner to the VmWorkJob. So the HA Planner being passed by HAMgr was not getting used.
- Now the job framework sets the planner passed in by any caller of the VM Start operation, to the job
Changes:
- The event of deleteing an affinity group is published on the MessageBus so that IAM Service can listen and process the event, However the publish operation should not be handled within a DB transaction, since it may take longer and hold the DB transaction for long unnecessarily
- Publish any events to MessageBus outside of the transaction
on calling GetUploadParamsForTemplate, persisting the metadata to db
validating the account limits and incrementing the appropriate limits
encoded the metadata on management server using preshared key
Steps to reproduce if you have this issue:
- Create a VM's volume snapshot
- Remove VM's template and mark the template as removed with timestamp in DB
- Restart mgmt server and create a volume out of snapshot you should get NPE
Fix: In `storagePoolHasEnoughSpace`, we're only searching for a VM's volume's
snapshot's template by Id and not including removed templates. This is a corner
case and NPE hits when template has been marked removed for a VM's volume's
template so we should search including removed templates.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit f189c105d8)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Steps to reproduce if you have this issue:
- Create a VM's volume snapshot
- Remove VM's template and mark the template as removed with timestamp in DB
- Restart mgmt server and create a volume out of snapshot you should get NPE
Fix: In `storagePoolHasEnoughSpace`, we're only searching for a VM's volume's
snapshot's template by Id and not including removed templates. This is a corner
case and NPE hits when template has been marked removed for a VM's volume's
template so we should search including removed templates.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit f189c105d8)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
On default iptables rules are updated to add ACCEPT egress traffic.
If the network egress default policy is false, CS remove ACCEPT and adds the DROP rule which
is egress default rule when there are no other egress rules.
If the CS network egress default policy is true, CS won't configure any default rule for egress because
router already came up to accept egress traffic. If there are already egress rules for network then the
egress rules get applied on VR.
For isolated network with out firewall service, VR default allows egress traffic (guestnetwork --> public network)
The last commit 513adab51b didn't fully fix it. Scenario #1 was not handled, only #2 was handled.
Fixed#1 as part of this commit.
1. If VM is in stopped state in CS due to 'PowerMissing' report from old host (hostId is null) and then there is a 'PowerOn' report from new host
2. If VM is in running state in CS and there is a 'PowerOn' report from new host
On default iptables rules are updated to add ACCEPT egress traffic.
If the network egress default policy is false, CS remove ACCEPT and adds the DROP rule which
is egress default rule when there are no other egress rules.
If the CS network egress default policy is true, CS won't configure any default rule for egress because
router already came up to accept egress traffic. If there are already egress rules for network then the
egress rules get applied on VR.
For isolated network with out firewall service, VR default allows egress traffic (guestnetwork --> public network)
Ensure that VR is re-booted when it is moved to another host out-of-band. This is necessary to re-program all network rules
Conflicts:
server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java
The previous fix tried to access StatsCollector from UsageManagerImpl
which is not possible due to dependency cycle.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 1e0880cbab)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java
The previous fix tried to access StatsCollector from UsageManagerImpl
which is not possible due to dependency cycle.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
If an auth API call (such as login, logout) is called on unauthenticated port
such as the 8096 integration server port, we need to ignore such API calls
as calling auth APIs on 8096 is un-necessary and is undefined.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 21a6bef53b)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
If an auth API call (such as login, logout) is called on unauthenticated port
such as the 8096 integration server port, we need to ignore such API calls
as calling auth APIs on 8096 is un-necessary and is undefined.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
If we create VM with shared service offering and attach disk with local disk offering,
and one of storage pool is full(cannot be allocated) and other is not full then
we are not putting the cluster in avoid list which is causing this infinite loop.
Fixed by putting the cluster in avoid list even if one of the storage pool is full(cannot be allocated)
pool, the source and destination pools cannot be local and cluster/zone and vice versa.
Cloudstack detects it and throws a exception. However, the end user only sees an
unexpected exception and not the reason for failure. Fixed it by making sure the
reason for the failure is correctly captured and shown to the end user.
This allows external processing of VmStats information without using
the usage server of CloudStack
Statistics are being send to Graphite using UDP and not TCP.
UDP is used to prevent the management server waiting for TCP timeouts
when the Graphite server is unavailable
Allows updating details (key/value) pair which updatse entries
invm_template_details and user_vm_details tables using updateVM and updateTemplate
APIs. This allows sys admins to update nics, controllers etc without DB hacking.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 97fa4a023e)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Allows updating details (key/value) pair which updatse entries
invm_template_details and user_vm_details tables using updateVM and updateTemplate
APIs. This allows sys admins to update nics, controllers etc without DB hacking.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Unnecessary exception in MS logs while removing default NIC from VM. Following changes are made:
1. Changed the exception from CloudRuntimeException to InvalidParameterValueExecption.
2. Moved out validation logic to UserVMManagerImpl from VirtualMachineManagerImpl.
3. Handling InvalidParameterValueException from async API calls so that they are not logged as ERROR in MS logs.
In ConfigurationVo, changed the setter to do the encryption if required
like the getter. Called the setter in constructor as well.
Removed references of encryption check in different places.
Reviewed-by: Santhosh Edukulla
This closes#35
Unnecessary exception in MS logs while removing default NIC from VM. Following changes are made:
1. Changed the exception from CloudRuntimeException to InvalidParameterValueExecption.
2. Moved out validation logic to UserVMManagerImpl from VirtualMachineManagerImpl.
3. Handling InvalidParameterValueException from async API calls so that they are not logged as ERROR in MS logs.
failed to migrate to the other host in the cluster.
Migration for vGPU VMs is not supported in XS, so instead of migrating them to
new server, stopping them.
failed to migrate to the other host in the cluster.
Migration for vGPU VMs is not supported in XS, so instead of migrating them to
new server, stopping them.
If we create VM with shared service offering and attach disk with local disk offering,
and one of storage pool is full(cannot be allocated) and other is not full then
we are not putting the cluster in avoid list which is causing this infinite loop.
Fixed by putting the cluster in avoid list even if one of the storage pool is full(cannot be allocated)
Remi Bergsma
Damodar
Mike Tutkowski
Prachi Damle
Daan Hoogland
Kshitij Kansal
Koushik Das
Daan Hoogland
Boris Schrijver
Wido den Hollander
Rohit Yadav
Jayapal
Rajani Karuturi
wilderrodrigues
Sudhansu
Likitha Shetty
Rene Moser
Rafael da Fonseca
Sanjay Tripathi
Abhinandan Prateek
Kishan Kavala
Milamber
Devdeep Singh
Anshul Gangwar
Daan Hoogland
Santhosh Edukulla
Ilia Shakitko
KC Wang
Amogh Vasekar
Ian Southam
Hugo Trippaers
Antonio Fornie
Sander Botman
Bharat Kumar
Nitin Mehta
Min Chen
Funs
Laszlo Hornyak
Sheng Yang
Saksham Srivastava
Rajesh Battala
Anthony Xu
Harikrishna Patnala
cloudtrends
Pierre-Yves Ritschard
Wei Zhou
Edison Su
Sebastien Goasguen
Ilia Shakitko
Daniel Vega
Syed
Will Stevens
Pierre-Luc Dion