This setting works on CentOS 6 / RHEL 6 but does nothing, as
"cpu" cgroup is not mounted. On CentOS 7 / RHEL 7 systemd does
mount cgroups and "cpu" is co-mounted with "cpuacc". Hence, if
we specify "cpu" then this results in an error because it can
only use them both, or none.
By removing the setting, we rely on the default of qemu, which
is:
cgroup_controllers = ["cpu", "devices", "memory", "blkio", "cpuacct", "net_cls"]
Only if they are really mounted, they will be used. So, this will
work on both version 6 and 7.
The 'fix script' didn't work well, as after a reboot you'd still have qemu
throwing errors. Now we can handle the co-mountedcgroups.
Fix for the NicVO.java regression.Renamed set*() methods to correct naming.
* pr/726:
Fix for the NicVO.java regression.
Signed-off-by: Remi Bergsma <github@remi.nl>
Removed duplicate code in CitrixResourceBase and its subclassesRemoved unnecessary duplicated code by having the body of the getPatchFiles method only in the CitrixResourceBase superclass. Given that all of its implementations consisted of the same code except for the path which contains the necessary file for that implementation. An abstract method getPatchFilePath was created in the CitrixResourceBase superclass so that each implementation may return the path containing the specific file needed by that implementation.
Test cases were created for each implementation, simple as they may be. One assert is made to verify that the path returned by each implementation corresponds to the path that was previously specified on each getPatchFiles implementation.
* pr/700:
Removed duplicate code in CitrixResourceBase.getPatchFiles
Signed-off-by: Remi Bergsma <github@remi.nl>
This tries to avoid cleaning by a device name.
This closes#718
(cherry picked from commit 74f697a2dd)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Changed methodnames according to Nic.java refactor.
Fixed NicVO.java due to regression from Nic.java refactor.
Fixed VmWareGuru.java after Nic.java refactor.
See issue CLOUDSTACK-8736 for ongoing effort to clean up network code.
@cristofolini comments:
Removed unnecessary duplicated code by having the body of the
getPatchFiles method only in the
CitrixResourceBase superclass. Given that all of its implementations
consisted of the same code
except for the path which contains the necessary file for that
implementation. An abstract method
getPatchFilePath was created in the CitrixResourceBase superclass so
that each implementation may
return the path containing the specific file needed by that
implementation.
Test cases were created for each implementation, simple as they may be.
One assert is made to verify
that the path returned by each implementation corresponds to the path
that was previously specified
on each getPatchFiles implementation.
removed trailing whitespace
Cloudstack 8656: do away with more silently ignoring exceptions.a lot of messages added.
some restructuring for test exception assertions and try-with-resource blocks
* pr/654: (29 commits)
CLOUDSTACK-8656: more logging instead of sysout
CLOUDSTACK-8656: use catch block for validation
CLOUDSTACK-8656: class in json specified not found
CLOUDSTACK-8656: removed unused classes
CLOUDSTACK-8656: restructure of tests
CLOUDSTACK-8656: reorganise sychronized block
CLOUDSTACK-8656: restructure tests to ensure exception throwing
CLOUDSTACK-8656: validate the throwing of ServerApiException
CLOUDSTACK-8656: logging ignored exceptions
CLOUDSTACK-8656: try-w-r removes need for empty catch block
CLOUDSTACK-8656: try-w-r instead of clunckey close-except
CLOUDSTACK-8656: deal with empty SQLException catch block by try-w-r
CLOUDSTACK-8656: unnecessary close construct removed
CLOUDSTACK-8656: message about timed buffer logging
CLOUDSTACK-8656: message about invalid number from store
CLOUDSTACK-8656: move cli test tool to separate file
CLOUDSTACK-8656: exception is the rule for some tests
CLOUDSTACK-8656: network related exception logging
CLOUDSTACK-8656: reporting ignored exceptions in server
CLOUDSTACK-8656: log in case we are on a platform not supporting UTF8
...
Signed-off-by: Remi Bergsma <github@remi.nl>
Before this commit the call to `getResizeScriptType` would throw
an exception (earlier versions returned `null`, which was fine) -
this caused the RBD case to fail. By changing the default to
notify only we fix the case for any non-CLVM and non-QCOW cases.
This is RBD for now, but this should extend to new storage
types suppored by Libvirt natively in future.
- Adds unit test for ListAndSwitchSAMLAccountCmd
- Checks and logs in user only if they are enabled
- If saml user switches to a locked account, send appropriate error message
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit b30977911d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
SAML authorized accounts might be across various domains, this allows for
switching of accounts only in case of SAML authenticated user accounts across
other accounts with the same SAML uid/username.
Moves the previous switch account logic to its own ui-custom module
(cherry picked from commit 1065661cd5)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
plugins/user-authenticators/saml2/src/org/apache/cloudstack/api/command/SAML2LoginAPIAuthenticatorCmd.java
ui/index.jsp
When dumping XML use appropriate flags:
1, VIR_DOMAIN_XML_SECURE (dump security sensitive information too)
8, VIR_DOMAIN_XML_MIGRATABLE (dump XML suitable for migration)
Source:
https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainXMLFlags
This fixes CVE 2015-3252: VNC password lost during VM migration across KVM
hosts. The issue is also seen when a VM is rebooted.
(cherry picked from commit cb2aca7516)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* pr/674:
getUsedBytes should query the SolidFire cluster to acquire the size of the given volume if there is no volume_details info for that volume (and then create a volume_details row for this volume so we don't have to make that cluster call for this purpose again)
Signed-off-by: Mike Tutkowski
Signed-off-by: Mike Tutkowski <mike.tutkowski@solidfire.com>
* pr/547:
CLOUDSTACK-8601. VMFS storage added as local storage can be re-added as shared storage. Fail addition of a VMFS shared storage pool in case it has already been added as local storage in CS.
Signed-off-by: Mike Tutkowski <mike.tutkowski@solidfire.com>
In LdapUserManagerFactory moved the beans to a map
used a Enum for LdapProvider and made the corresponding changes in
LdapConfiguration and the callers.
the changes in commit ec32ea30f7 changed
gmaven version in ldap plugin which broke running the spock tests from
command line. Hence reverting the change until we fix it with
the new version.
added a new configuration to select the appropriate ldap implementation
incase of microsoft AD enabled nested querying of group members
moved LdapUserManager to an interface and added separate implementations
for openLdap and microsoft AD
Added unit tests
On NFS we write a heartbeat, but without those we can not safely
fence off a host.
If we fence without knowing about a heartbeat we can cause a split-brain situation.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
Add option to authorize SAML SSO for user when adding user. Appends a domain to
user/account name if global setting is enabled, useful in case of multiple IDP
server. By default the setting is set to false (keep as it is).
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#575
(cherry picked from commit 7d11c7bc70)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Login is based on sessionkey HttpOnly Cookie
- ApiServlet does login verification using sessionKey from both the request cookies
and the API parameters. In both cases, if either or both are passed they should
match the sessionKey stored in the current session of the HttpRequest
- UI: it no longer needs to read or set sessionkey cookie
- UI: it no longer needs to return g_sessionKey value in the API requests, though
to support a sso mechanism g_sessionKey is still passed in the API is not null
- Secure jsessionid cookie is set to be HttpOnly and Secure
- SAML login should also set HttpOnly cookie before redirecting to UI
- SAML: listIdps & getSPMetadata APIs are readonly now, won't log out a logged in user
Performed tests (login, saml login if applicable, page refreshes, opening
multiple tabs, logout) with following combinations:
- SAML disabled, normal auth as admin, domain-admin and user
- SAML enabled, normal auth as admin, domain-admin and user; and saml sso as
admin, domain-admin and user
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#574
This closes#308
- Make sure the new password replaces the old one in the queue
- Updated the patch files for XenServer
- Updated the script path on LibvirtComputing class
- Adding update_host_passwd to VRScripts
- Add implementation to CitrixUpdateHostPasswordCommandWrapper
- Improve testUpdateHostPasswordCommand() unit test on CitrixRequestWrapperTest
- Adding update_host_passwd.sh script
- Adding the host IP address as an instance variable on UpdateHostPasswordCommand
- Improving the Unit Test (LibvirtComputingResourceTest) to get it covering the new code
- Make sure doUpdateHostPassword() doesn't get called if flag is set to false
- Do not update XenServer hosts if the cluster ID is not informed
This closes#559
- Adding more unit tests as suggested by Daan and Rajani.
- Mocking SshHelper and the Script classes, used by Citrix and Libvirt resources respectively.
- Changed location of the update_host_passwd script
- Updated the patch files for XenServer
- Updated the script path on LibvirtComputing class
- Removed the hostIP from the LibvirtUpdateHostPasswordCommandWrapper execute() method
- Adding update_host_passwd to VRScripts
- Add accessor method to host password on CitrixResourceBase
- Add implementation to CitrixUpdateHostPasswordCommandWrapper
- Improve testUpdateHostPasswordCommand() unit test on CitrixRequestWrapperTest
- Add line to patch files on xenserver directory
Concerning the LibVirt change:
- I forgot to assing the return of the getDefaultHypervisorScriptsDir() method to the hypervisorScriptsDir variable
listing the hosts for migration for instance with tag
While preparing the suitable hosts we are accidentally removing the
incompatible (host does not have host tag) hosts from otherhost list(
incorrect use of List.retainAll).
- Modifying the LibvirtUpdateHostPasswordCommandWrapper in order to execute the script on the host
- Adding the script path to LibvirtComputingResource
- Adding the host IP address as an instance variable on UpdateHostPasswordCommand
- Improving the Unit Test (LibvirtComputingResourceTest) to get it covering the new code
- Change the retry() method to return Answer.createUnsupportedCommandAnswer(command) instead of throwing an exception
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
This closes#546
- Adding command wrappers NiciraNvpCreateLogicalSwitchCommandWrapper and NiciraNvpDeleteLogicalSwitchCommandWrapper
- Refactoring the retry mechanism
- Applying the new retry mechanism to current wrappers and old methods in NiciraNvpResource
- Adding 2 tests
- Fixing the testRetries() in NiciraNvpResourceTest class
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
- Adding the NiciraNvpWrapper
- This class will keep track of all Wrappers of the Nicira NVP Plugin
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
This commit has a small refactoring of cloud-plugin-acl-static-role-based
to allow it to read files on the classpath that might have a different name
than "commands.properties". It also allows more than one file to be read from.
Rationale: Third-party plugins may want to keep their API command access level
configuration separate from the main file so as to reduce configuration
maintenance work during packaging and deployments.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#354
* Move config options to SAML plugin
This moves all configuration options from Config.java to SAML auth manager. This
allows us to use the config framework.
* Make SAML2UserAuthenticator validate SAML token in httprequest
* Make logout API use ConfigKeys defined in saml auth manager
* Before doing SAML auth, cleanup local states and cookies
* Fix configurations in 4.5.1 to 4.5.2 upgrade path
* Fail if idp has no sso URL defined
* Add a default set of SAML SP cert for testing purposes
Now to enable and use saml, one needs to do a deploydb-saml after doing a deploydb
* UI remembers login selections, IDP server
- CLOUDSTACK-8458:
* On UI show dropdown list of discovered IdPs
* Support SAML Federation, where there may be more than one IdP
- New datastructure to hold metadata of SP or IdP
- Recursive processing of IdP metadata
- Fix login/logout APIs to get new interface and metadata data structure
- Add org/contact information to metadata
- Add new API: listIdps that returns list of all discovered IdPs
- Refactor and cleanup code and tests
- CLOUDSTACK-8459:
* Add HTTP-POST binding to SP metadata
* Authn requests must use either HTTP POST/Artifact binding
- CLOUDSTACK-8461:
* Use unspecified x509 cert as a fallback encryption/signing key
In case a IDP's metadata does not clearly say if their certificates need to be
used as signing or encryption and we don't find that, fallback to use the
unspecified key itself.
- CLOUDSTACK-8462:
* SAML Auth plugin should not do authorization
This removes logic to create user if they don't exist. This strictly now
assumes that users have been already created/imported/authorized by admins.
As per SAML v2.0 spec section 4.1.2, the SP provider should create authn requests using
either HTTP POST or HTTP Artifact binding to transfer the message through a
user agent (browser in our case). The use of HTTP Redirect was one of the reasons
why this plugin failed to work for some IdP servers that enforce this.
* Add new User Source
By reusing the source field, we can find if a user has been SAML enabled or not.
The limitation is that, once say a user is imported by LDAP and then SAML
enabled - they won't be able to use LDAP for authentication
* UI should allow users to pass in domain they want to log into, though it is
optional and needed only when a user has accounts across domains with same
username and authorized IDP server
* SAML users need to be authorized before they can authenticate
- New column entity to track saml entity id for a user
- Reusing source column to check if user is saml enabled or not
- Add new source types, saml2 and saml2disabled
- New table saml_token to solve the issue of multiple users across domains and
to enforce security by tracking authn token and checking the samlresponse for
the tokens
- Implement API: authorizeSamlSso to enable/disable saml authentication for a
user
- Stubs to implement saml token flushing/expiry
- CLOUDSTACK-8463:
* Use username attribute specified in global setting
Use username attribute defined by admin from a global setting
In case of encrypted assertion/attributes:
- Decrypt them
- Check signature if provided to check authenticity of message using IdP's
public key and SP's private key
- Loop through attributes to find the username
- CLOUDSTACK-8538:
* Add new global config for SAML request sig algorithm
- CLOUDSTACK-8539:
* Add metadata refresh timer task and token expiring
- Fix domain path and save it to saml_tokens
- Expire hour old saml tokens
- Refresh metadata based on timer task
- Fix unit tests
This closes#489
(cherry picked from commit 20ce346f3a)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
client/WEB-INF/classes/resources/messages_hu.properties
plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resource/wrapper/xenbase/CitrixCheckHealthCommandWrapper.java
plugins/user-authenticators/saml2/src/org/apache/cloudstack/api/command/SAML2LoginAPIAuthenticatorCmd.java
ui/scripts/ui-custom/login.js
- Make sure the doUpdateHostPassword() method returns the result from the answer, not always true
- Added the LibvirtUpdateHostPasswordCommandWrapper class, which will handle the changes in the KVM agent
That's will be further implemented in the future. The command will avoid the agent to complain about Unsupported command
- Added a test to make sure the current implementation of the comment works
- If changes in the future, it will also require changes. In that way, we make sure nobody will break it
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
This closes#527
CLOUDSTACK-3317 - DVS does not support management\storage network
Added support for Management and Storage Network traffic over VMware DVS in CloudStack deployments. Also added support for storage VLAN over dvPortGroup.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
CLOUDSTACK-3317 - DVS does not support management\storage network
Use non-zero dvport count while updating dvportgroups of system traffic.
Updated configuration compare logic to avoid update dvportgroup operation unless required.
This would help improve speed in vm/network deployment as the update calls would reduce. Also improved logging.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Adding unit tests in class HypervisorHostHelperTest
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Added license header to new file being added to repo/branch.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Assertion is not used in runtime, correct way is throw and handle exception without killing app
Signed-off-by: Daan Hoogland <daan.hoogland@gmail.com>
This closes#362
VmwareResource.java:315, MS_SHOULD_BE_FINAL, Priority: High
com.cloud.hypervisor.vmware.resource.VmwareResource.s_serviceContext isn't final but should be
VmwareResource.java:331, MS_SHOULD_BE_FINAL, Priority: High
com.cloud.hypervisor.vmware.resource.VmwareResource.s_powerStatesTable isn't final but should be
Signed-off-by: Daan Hoogland <daan.hoogland@gmail.com>
This closes#363
KVM hosts which are actuall up, but if their agents are shutdown should be put
in disconnected state. This would avoid getting the VMs HA'd and other commands
such as deploying a VM will exclude that host and save us from errors.
The improvement is that, we first try to contact the KVM host itself. If it fails
we assume that it's disconnected, and then ask its KVM neighbours if they can
check its status. If all of the KVM neighbours tell us that it's Down and we're
unable to reach the KVM host, then the host is possibly down. In case any of the
KVM neighbours tell us that it's Up but we're unable to reach the KVM host then
we can be sure that the agent is offline but the host is running.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
This closes#340
When executing the tests in an environment where Libvirt is also installed, it
caused errors.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#342
Maven was bulding the project and executing the tests just fine because it uses "-sourcepath". However,
with IDEs and javac it would fail.
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
This closes#317
Pull average Cpu util report between polling intervals instead of since boot
instead of using values since uptime
(cherry picked from commit 04176eaf17)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
This closes#297
Pull average Cpu util report between polling intervals instead of since boot
instead of using values since uptime
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#289
Passing the file argument to the xml break for EL 7.1, the fix removes
the argument as just passing rombar='off' with its file arg to be empty string.
This closes#290
(cherry picked from commit aafa0c80b3)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Passing the file argument to the xml break for EL 7.1, the fix removes
the argument as just passing rombar='off' with its file arg to be empty string.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#290
EL7 has a different output to 'free', use /proc/meminfo instead of a tool to be
more consistent across distros
(cherry picked from commit 212a05a345)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
We add a new contract to pass Http request to authentication plugin system. In
the default login API, we disallow non-POST requests.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
EL7 has a different output to 'free', use /proc/meminfo instead of a tool to be
more consistent across distros
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Removing real IPs from the tests because they cause a long running time for LibvirtComputingResourceTest
- In a local machine it takes 1.977s, but in a KVM test environment it's taking 257.879 sec
... of new volumes. Following changes are implemented 1. Disable or enable a pool with the
updateStoragePool api. A new 'enabled' parameter added for the same. 2. When a
pool is disabled the state of the pool is updated to 'Disabled' in the db. On
enabling it is updated back to 'Up'. Alert is raised when a pool is disabled or
enabled. 3. Updated other storage providers to also honour the disabled state.
4. A disabled pool is skipped by allocators for provisioing of new volumes. 5.
Since the allocators skip a disabled pool for provisioning of volumes, the
volumes are also not listed as a destination for volume migration.
FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Disabling+Storage+Pool+for+Provisioning
This closes#257
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Fixing typo on LibvirtRequestWrapper
- Replace linbvirtCommands by libvirtCommands on LibvirtRequestWrapper
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#255
- The test was okay, but when running in an environment where a /root/.ssh/id_rsa existed, it would return true then fail
- We now mock the calls to methods that return the key paths, instead of relying in the static variables
- Adding LibvirtNetworkElementCommandWrapper and LibvirtStorageSubSystemCommandWrapper
- 2 unit tests added
- KVM hypervisor plugin with 22.2% coverage
I also refactored the StorageSubSystemCommand interface into an abstract class
- Remove the pseudo-multiple-inheritance implementation
- The StorageSubSystemCommand was an interface, not related to the Command class
and its implementation were extending the Command class anyway. The whole structure is better now.
- Addin LibvirtPvlanSetupCommandWrapper
- 6 unit tests added
- KVM hypervisor plugin with 21% coverage
From the 6 tests added, 2 were extra tests to increase the coverage of the LibvirtStopCommandWrapper
- Increased from 35% to 78.7%
- Adding LibvirtCopyVolumeCommandWrapper
Refactoring the LibvirtUtilitiesHelper
- Changing method name
Did not add any test to this commit due to the refactor mentioned abot.
Will proceed and add the tests
i# Please enter the commit message for your changes. Lines starting
- Gave it a better, more suggestive, name since I now added other methods to the class.
- It makes easier to mock objects and get a better coverage of the classes
- Adding LibvirtBackupSnapshotCommandWrapper, LibvirtCreatePrivateTemplateFromVolumeCommandWrapper and LibvirtManageSnapshotCommandWrapper
- 3 unit tests added
- KVM hypervisor plugin with 18.3% coverage
Less tests added to those classes because the code is quite complex and way too long.
The tests added are just covering the new flow, to make sure it works fine. I will come back to those classes later.
- Adding LibvirtOvsDestroyBridgeCommandWrapper, LibvirtOvsSetupBridgeCommandWrapper
- 4 unit tests added
- KVM hypervisor plugin with 13.9% coverage
More tests added to cover LibvirtPrepareForMigrationCommandWrapper
- Coverage of this wrapper broght from 37% to 90.6%
- 4 new tests added
- Adding LibvirtCheckConsoleProxyLoadCommandWrapper, LibvirtConsoleProxyLoadCommandWrapper, LibvirtWatchConsoleProxyLoadCommandWrapperand CitrixConsoleProxyLoadCommandWrapper
- 2 unit tests added
- KVM hypervisor plugin with 12% coverage
Refactored the CommandWrapper interface in order to remove the esecuteProxyLoadScan, which is now
implemented bu subclasses.
- Adding LibvirtGetHosStatsCommandWrapper
- 1 unit test added
- KVM hypervisor with 10.5% coverage
Tests are a bit limited on this one becuause of the current implementation. Would clean it up later in a separate branch
- Adding LibvirtGetVmStatsCommandWrapper
- 3 unit tests
Refactored the LibvirtConnectiobn by surrounding it with an wrapper.
- Make it easier to cover the static/native calls
- Added better coverage to StopCommand tests
- Adding LibvirtStopCommandWrapper
- LibvirtRequestWrapper
- 1 unit tests
Refactored the RequestWrapper to make it better.
- Changes also applied to the CitrixRequestWrapper
- Removes awsapi db properties usage across codebase
- Removes references from spring xmls, test cases and TransactionLegacy
- Adds sql command to drop database cloudbridge in schema-451to460-cleanup.sql
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
From b3f18e7d74, the zone level systemvm local
storage setting never worked as it needed to be moved to config depot.
(cherry picked from commit 279efb0432)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
plugins/network-elements/internal-loadbalancer/src/org/apache/cloudstack/network/lb/InternalLoadBalancerVMManagerImpl.java
server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java
From b3f18e7d74, the zone level systemvm local
storage setting never worked as it needed to be moved to config depot.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Linux kernel supports vmxnet3, allowing it in KVM plugin would allow us to
run ESX hosts on KVM hosts using CloudStack with vmxnet3 nic which can be
passed as VM's nicAdapter detail
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit e02d787f30)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Linux kernel supports vmxnet3, allowing it in KVM plugin would allow us to
run ESX hosts on KVM hosts using CloudStack with vmxnet3 nic which can be
passed as VM's nicAdapter detail
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This improvements checks for "guest.cpu.features" property which is a space
separated list of cpu features that is specific for a host. When added, it
will add <feature policy='require' name='{{feature-you-listed}}'/> in the
<cpu> section of the generated vm spec xml.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit ea7fd37783)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This improvements checks for "guest.cpu.features" property which is a space
separated list of cpu features that is specific for a host. When added, it
will add <feature policy='require' name='{{feature-you-listed}}'/> in the
<cpu> section of the generated vm spec xml.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Dont evict template when a delete command has been sent to VMware resource for deletion of volume.
(cherry picked from commit f45e6b94ed)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The only artifact resolved from libvirt.org was org.libvirt:libvirt:0.5.1
this artifact is now available from maven's default central repository
This closes#180
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 9cf31b0714)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The only artifact resolved from libvirt.org was org.libvirt:libvirt:0.5.1
this artifact is now available from maven's default central repository
This closes#180
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
MigrateVMWithVolumes-
1. If ESXi host version is below 5.1, ensure destination datastore(s) is mounted on the source host, then migrate the storage and then finally migrate the VM.
If destination storage(s) is not mounted on the source host,
- In case of NFS storage mount the storage(s).
- In case of VMFS storage fail the request for migration.
2. If EXi host version is 5.1 or above, simultaneously migrate the VM and its storage to the destination host and storage(s) respectively for both NFS and VMFS storage.
(cherry picked from commit adc836cc5e)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Correctly register node info for a newly created VMware context.
(cherry picked from commit 13bdc1cef4)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
As suggested by Wido on the dev ML changing the repo to eu.ceph.com to avoid
build failures. Will revert if ceph.com is up again.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit c9fd57fff3)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
As suggested by Wido on the dev ML changing the repo to eu.ceph.com to avoid
build failures. Will revert if ceph.com is up again.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CentOS 7 does not ship with ifconfig anymore. We should use ip commands instead.
This also works on older versions, like CentOS 6 and Ubuntu 12.x/14.x, that we
support.
(cherry picked from commit ac06ec02eb)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/IvsVifDriver.java
CentOS 7 does not ship with ifconfig anymore. We should use ip commands instead.
This also works on older versions, like CentOS 6 and Ubuntu 12.x/14.x, that we
support.
This closes#165
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- use sticky chmod 1777 on the mountpoint
- remove dead code
(cherry picked from commit eea716b791)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- use sticky chmod 1777 on the mountpoint
- remove dead code
- port improved code for moving disk into corresponding folder from master
- for dummy worker case, check case for powered off vm state
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
It causes extreme long vm start time when managment server has slow connection to secondary storage
(cherry picked from commit 49c01e2ae4)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
While looking for an ongoing VM snapshot task, check the task status to identify if the task is still running.
(cherry picked from commit 25a4f0dc53)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Handling spaces in datastore name while extracting vmdk base name
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
(cherry picked from commit aa84b05491)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Update the path and size of data volumes after snapshot creation/deletion by correctly trimming only the snapshot postfix of a disk.
(cherry picked from commit ac491c9607)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Fix the OVA path that is returned once an OVA is packaged using a META file.
(cherry picked from commit 507d9d337d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
1. provide compatibility with the Big Cloud Fabric (BCF) controller
L2 Connectivity Service in both VPC and non-VPC modes
2. virtual network terminology updates: VNS --> BCF_SEGMENT
3. uses HTTPS with trust-always certificate handling
4. topology sync support with BCF controller
5. support multiple (two) BCF controllers with HA
6. support VM migration
7. support Firewall, Static NAT, and Source NAT with NAT enabled option
8. add VifDriver for Indigo Virtual Switch (IVS)
This closes#151
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- With the changes added by the rVPC work, the bump priority became deprecated.
This commit includes a refactor to get it removed from the following resources:
* Java classes
* domain_router table - removing the is_priority_bumpup column
* Fixing unit tests
All changes were tested with:
XenServer 6.2 running under our VMWare zone
CloudStack Management Server running on MacBook Pro
MySql running on MackBook Pro
Storage Type: Local
Started the refactor of the XenServer56Resource class
- Unit test added
Changing the CitrixRequestWrapper in order to cope with multiple resources classes and commands
Still have to remove few methods form CitrixResourceBase
- Unit tests added
In the executeRequest I needed to keep the following:
// We need this one because the StorageSubSystemCommand is from another hierarchy.
if (cmd instanceof StorageSubSystemCommand) {
return storageHandler.handleStorageCommands((StorageSubSystemCommand) cmd);
}
- Added basic tests
- Changed the way RebootCommand gets called from RebootRouterCommand
- Made a couple of methods public in the CitrixResourceBase and its subclasses
Sonarcube is great, but has no context and can be wrong, in this
case File.separator is nice if you're running platform independent,
the agent on OVM3 is however NOT platform independent and will break
if we feed it windows separators.
The copy command reply can have null size returned, so check and set values
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 53ca0b1861)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Fix: RTNETLINK errors
- Management Server health check trying to create already existing interface
- Changes on update_config.py, cs_guestnetwork.py, merger.py
Fix: replace RRouTER_LOG in the CsRedundant.py per log file location
Fix: Guest Net address association during Router restart
- Changes on NicProfileHelper, NicProfileHelperImpl
Fix: aggregationExecution() method on VirtualNetworkApplianceManagerImpl
- Do not send an AggregationControlCommand to a non-configured router
Some classes have been formatted.
Added a source column to the user table.
Source now has only two values UNKNOWN,LDAP with UNKNOWN being the
default and is an enum is com.cloud.User.
When the source is UNKNOWN, the old method of authenticating against all
the available authenticators is used. If a source is available, only
that particular authenticator will be used.
added overloaded methods in AccountService to createUserAccount and
createUser with source specified.
(cherry picked from commit 5da733072e)
Added a source column to the user table.
Source now has only two values UNKNOWN,LDAP with UNKNOWN being the
default and is an enum is com.cloud.User.
When the source is UNKNOWN, the old method of authenticating against all
the available authenticators is used. If a source is available, only
that particular authenticator will be used.
added overloaded methods in AccountService to createUserAccount and
createUser with source specified.
The 19e3c0168e commit breaks SAML login and any
login where redirection is used.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit b79f13ccb5)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
MigrateVMWithVolumes-
1. If ESXi host version is below 5.1, ensure destination datastore(s) is mounted on the source host, then migrate the storage and then finally migrate the VM.
If destination storage(s) is not mounted on the source host,
- In case of NFS storage mount the storage(s).
- In case of VMFS storage fail the request for migration.
2. If EXi host version is 5.1 or above, simultaneously migrate the VM and its storage to the destination host and storage(s) respectively for both NFS and VMFS storage.
This is a plugin that puts in ovm3 support ranging from 3.3.1 to 3.3.2. Basic
functionality is in here, advanced networking etc..
Snapshots only work when a VM is stopped now due to the semantics of OVM's raw
image implementation (so snapshots should work on a storage level underneath the
hypervisor shrug)
This closes#113
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This commit produces event bus messages to a "cloudstack" topic
in Apache Kafka. Configuration is expected to be found in
/etc/cloudstack/management/kafka.producer.properties and will
generally be of the form:
bootstrap.servers=kafka-host1:9092,kafka-host2:9092
key.serializer=org.apache.kafka.common.serialization.StringSerializer
value.serializer=org.apache.kafka.common.serialization.StringSerializer
There is no way to parameterize the topic yet, and the consuming
code is just place-holder. I think adding a consumer within cloudstack
is very debatable and likely not needed.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
When there is large size VR configuration (aggregate commands) copying data to VR using vmops plugin was failed
because of the ARG_MAX size limitation. The configuration data size is around 300KB.
Updated this to create file in host by scp with file contents. This will create file in host.
Then copy the file from the host to VR using hte vmops createFileInDomr method.
In host file get created in /tmp/ with name VR-<UUID>.cfg, once it copied to VR this file will be removed.
(cherry picked from commit 619f014255)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
When there is large size VR configuration (aggregate commands) copying data to VR using vmops plugin was failed
because of the ARG_MAX size limitation. The configuration data size is around 300KB.
Updated this to create file in host by scp with file contents. This will create file in host.
Then copy the file from the host to VR using hte vmops createFileInDomr method.
In host file get created in /tmp/ with name VR-<UUID>.cfg, once it copied to VR this file will be removed.
The test testFindSecurityProfileByUuid built an assumptio on the order of the arguments after transforming through a HashMap.
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
Refactored to use the XPatch expressions to check the generated domain xml rathern than string comparison.
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
Design Document:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Allow+VM+listing+by+User+ID
- Adds column to VMInstance DAO
- Adds column in vm_instance table
- Adds column in the UserVMJoinVO
- Adds default admin user which has UID = 2
- Adds migration path that sets user_id to first user of the accountId that
owns the vm in vm_instance table
- Add arg on list VMs API to query by userId, add support in query layer
- Refactor VMInstanceVO and child classes to accept userId
- Add code to let service layer pass userId if loggedIn user belongs to same
account as the owner executing an API call or use first user from owner account
- In case of CPVM and SSVM use system user ID
- Fix unit tests and spring injections
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Earlier host addition of multiple hosts with local storage failed due to
same local storage UUID being used where the storage path is same.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit bf17f640c6)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Earlier host addition of multiple hosts with local storage failed due to
same local storage UUID being used where the storage path is same.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Fix cleanup of NFS mounts on management server during server starup by correcting how mount points are listed for a management server.
(cherry picked from commit 647532376f)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
For AttachVolume/DetachVolume API command, improve user error message in case of RuntimeException by throwing the exception instead of 'Unexpected Exception'.
(cherry picked from commit 4d7ede535d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
server/src/com/cloud/storage/VolumeApiServiceImpl.java
If VM has been cold migrated across different VMware DCs, then unregister the VM from source host.
(cherry picked from commit 15b348632d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
1. If a VM by the same name exists on a different cluster in the VMware DC, unregister the existing VM and continue with the VM start.
2. If VM start succeeds, delete VM files associated with the unregistered VM.
3. If VM start fails, re-register the unregistered VM.
(cherry picked from commit 974b0180dd)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
If an exact match is being done while locating disk chain by name, don't trim snapshot postfix appended to the disk name.
(cherry picked from commit ddcae8a930)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Before registering a VM check if a different CS VM with same name exists in vCenter.
(cherry picked from commit 33179cce56)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Look for a VM in vCenter based on both the vCenter name and CS internal name (required in case 'vm.instancename.flag' is enabled).
During Attach Volume and Volume Migration, for lookup and other operations use VM's name as obtained from vCenter instead of using the name set in the agent command.
(cherry picked from commit b8fdda0a34)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Fixed some issues with the polling logic to handle scenarios when SSVM is destroyed/down.
Also changed the status of the volume_store_ref entries so that subsequent operations work fine.
KVMStoragePoolManager is a singleton in practice, any plugin
or extension of LibvirtComputingResource will need to act on
the specific instance of KVMStoragePoolManager that LibvirtComputingResource
has initialized. Therefore, expose this variable for those who
wish to call storage commands from plugins or extensions.
Conflicts:
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
KVMStoragePoolManager is a singleton in practice, any plugin
or extension of LibvirtComputingResource will need to act on
the specific instance of KVMStoragePoolManager that LibvirtComputingResource
has initialized. Therefore, expose this variable for those who
wish to call storage commands from plugins or extensions.
Conflicts:
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
The fix generates X509Certificate if missing from DB and uses that for eternity.
SAML SP metadata remains same since it's using the same X509 certificate and
it remains same after restarts. The certificate is serialized, base64 encoded
and stored in the keystore table under a specific name. For reading, it's
retrieved, base64 decoded and deserialized.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 4358714381)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The fix generates X509Certificate if missing from DB and uses that for eternity.
SAML SP metadata remains same since it's using the same X509 certificate and
it remains same after restarts. The certificate is serialized, base64 encoded
and stored in the keystore table under a specific name. For reading, it's
retrieved, base64 decoded and deserialized.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
As per Version 1 cookies, certain characters are now allowed such as space,
colons etc but they should be url encoded using UTF8 encoding. The frontend
has a cookie value unboxing method that removes any double quotes that are added.
As per the doc http://download.oracle.com/javase/6/docs/api/java/net/URLEncoder.html
values are application/x-www-form-urlencoded and as per
http://www.w3.org/TR/html4/interact/forms.html#h-17.13.4 whitespaces are encoded
as +, therefore '+' are replaced by %20 (whitespace).
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 734bd70173)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
As per Version 1 cookies, certain characters are now allowed such as space,
colons etc but they should be url encoded using UTF8 encoding. The frontend
has a cookie value unboxing method that removes any double quotes that are added.
As per the doc http://download.oracle.com/javase/6/docs/api/java/net/URLEncoder.html
values are application/x-www-form-urlencoded and as per
http://www.w3.org/TR/html4/interact/forms.html#h-17.13.4 whitespaces are encoded
as +, therefore '+' are replaced by %20 (whitespace).
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The User table's UUID column is restricted to 40 chars only, since we don't
know how long the nameID/userID of a SAML authenticated user will be - the fix
hashes that user ID and takes a substring of length 40 chars. For hashing,
SHA256 is used which returns a 64 char length string.
- Fix tests, add test cases
- Improve checkSAMLUser method
- Use SHA256 one way hashing to create unique UUID for SAML users
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit b2b496288d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The User table's UUID column is restricted to 40 chars only, since we don't
know how long the nameID/userID of a SAML authenticated user will be - the fix
hashes that user ID and takes a substring of length 40 chars. For hashing,
SHA256 is used which returns a 64 char length string.
- Fix tests, add test cases
- Improve checkSAMLUser method
- Use SHA256 one way hashing to create unique UUID for SAML users
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Remi Bergsma
Wei Zhou
Miguel Ferreira
Boris Schrijver
radu-stefanache
Rohit Yadav
weingartner
manuiiit
Daan Hoogland
Aaron Brady
Mike Tutkowski
Daan Hoogland
Sten Spans
Kshitij Kansal
Rajani Karuturi
Wido den Hollander
Likitha Shetty
Funs
Wido den Hollander
Jayapal
wilderrodrigues
Kishan Kavala
Sudhansu
jeff
Rafael da Fonseca
Sanjay Tripathi
Sateesh Chodapuneedi
Koushik Das
Devdeep Singh
Laszlo Hornyak
Anshul Gangwar
Daan Hoogland
Santhosh Edukulla
KC Wang
Frank Zhang
Antonio Fornie
Hugo Trippaers
Pierre-Yves Ritschard
punith-cloudbyte
Marcus Sorensen
Damodar
Rajesh Battala
Anthony Xu
Saksham Srivastava