CLOUDSTACK-8762: Check to confirm disk activity before starting a VMImplements a VM volume/disk file activity checker that checks if QCOW2 file
has been changed before starting the VM. This is useful as a pessimistic
approach to save VMs that were running on faulty hosts that CloudStack could
try to launch on other hosts while the host was not cleanly fenced. This is
optional and available only if you enable the settings in agent.properties
file, on per-host basis.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* pr/753:
CLOUDSTACK-8762: Check to confirm disk activity before starting a VM
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Guys, can you review it? things need to be discussed:
(1) this supports KVM/QCOW2 only. Anyone want to implement for other Hypervisor/format ?
(2) The original data volume (on primary storage) will be removed.
(3) The script uses the default timeout in libvirtComputingResource. Do we need to add one in global configuration (like copy.volume.wait or backup.snapshot.wait, create.volume.from.snapshot.wait)
(4) In scripts/storage/qcow2/managesnapshot.sh, I use "qemu-img convert -f qcow2 -O qcow2" to copy the snapshot from secondary to primary (hence there is no base image file), instead of "cp -f", this is because convert is faster than cp in my testing.
* pr/732:
CLOUDSTACK-5863: revert volume snapshot for KVM/QCOW2
Signed-off-by: Wei Zhou <w.zhou@tech.leaseweb.com>
made domainId compulsory in api LinkDomainToLdapCmd
used accountServive from BaseCmd in LinkDomainToLdapCmd
changed the allowed account type values to 0 and 2
Made following fixes in simulator
- Support for ScaleVmCommand/NetworkRulesVmSecondaryIpCommand in resource layer
- Added support for scaling up a running VM in simulator
- Fixed some method names not following convention
In order to test PR #725 using simulator some of these changes are needed.
Based on the way HV check is present in the scale VM API, had to explicitly put simulator related check to allow support. The ideal way would be to remove all these HV specific check from code and made them some configuration (by putting them in hypervisor_capabilities table in DB). But that will be a bigger effort outside the scope of this PR.
Signed-off-by: Koushik Das <koushik@apache.org>
This reverts commit cd7218e241, reversing
changes made to f5a7395cc2.
Reason for Revert:
noredist build failed with the below error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project cloud-plugin-hypervisor-vmware: Compilation failure
[ERROR] /home/jenkins/acs/workspace/build-master-noredist/plugins/hypervisors/vmware/src/com/cloud/hypervisor/guru/VMwareGuru.java:[484,12] error: non-static variable logger cannot be referenced from a static context
[ERROR] -> [Help 1]
even the normal build is broken as reported by @koushik-das on dev list
http://markmail.org/message/nngimssuzkj5gpbz
Implements a VM volume/disk file activity checker that checks if QCOW2 file
has been changed before starting the VM. This is useful as a pessimistic
approach to save VMs that were running on faulty hosts that CloudStack could
try to launch on other hosts while the host was not cleanly fenced. This is
optional and available only if you enable the settings in agent.properties
file, on per-host basis.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
if an admin username is given to the linkDomainToLdap, added support to
import this user
User will be imported only if the user is available in the group/ou in
ldap and an account with the name doesnt exist in cloudstack.
on successful import, accountid will be returned in response.
messages: Removed unused agent command AttachVolumeCommand and corresponding answer AttachVolumeAnswer
These types are not used anywhere in the orchestration layer.
Signed-off-by: Koushik Das <koushik@apache.org>
Default to notify only script to handle non-CLVM/QCOW cases.This relates to [CLOUDSTACK-8732](https://issues.apache.org/jira/browse/CLOUDSTACK-8732)
Before this commit the call to `getResizeScriptType` would throw an exception (earlier versions returned `null`, which was fine) - this caused the RBD case to fail. By changing the default to notify only we fix the case for any non-CLVM and non-QCOW cases, too.
This is RBD for now, but this should extend to new storage types supported by Libvirt natively in future.
This is my first attempted contribution: I can see a case for adding RBD logic to the actual getResizeScriptType call, too, but I felt that putting it `LibvirtResizeVolumeCommandWrapper.java` kept the special-casing of RBD (and comments about that) in one place.
### Caveat:
With Libvirt 1.2.2 this actually doesn't do the right thing - but it does do what the documentation *says* should be the right thing, so I'm going to test if this is a Libvirt bug which is fixed in a later version.
(To make it work I need to execute something like:
virsh blockresize --path vda --size 100G i-7-44-VM
where vda is the path as far as the *guest* is concerned, and not an `rbd/` path - which *should* work, but doesn't.)
* pr/696:
Default to notify only script to handle non-CLVM/QCOW cases.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Add -noverify JVM arg to surefire plugin, to allow Powermockito to
de-encapsulate private methods
- Add dependency on cloud-utils test-jar to use custom HttpRequest
matchers
- Support for ScaleVmCommand/NetworkRulesVmSecondaryIpCommand in resource layer
- Added support for scaling up a running VM in simulator
- Fixed some method names not following convention
CLOUDSTACK-8443: Support CentOS 7 as KVM hypervisorThis adds support for CentOS 7 as a KVM hypervisor. As discussed in the Jira issue, the root cause of the issue was the `cloudstack-setup-agent` script that added this line to `/etc/libvirt/qemu.conf`:
`cgroup_controllers=["cpu"]`
On CentOS 6 this 'works', but since by default no cgroups are mounted, this setting is ignored by `libvirtd`. On CentOS 7 is does not work, because the `cpu` and `cpuacct` are 'co-mounted'. This simpy means you cannot use one without the order. The config line above instructs `libvirtd` to only use the `cpu` cgroup, which it can't.
There was code added to make sure the `cpu` cgroup was not co-mounted. It did this with a shell script, but after a reboot it was gone. It wasn't reliable and not even a solution at all. I went ahead and removed the code, because the co-mounted cgroups work fine. The real issue was the config line we put in `/etc/libvirt/qemu.conf` so much better to fix that instead.
The default for `libvirtd` is to use only cgroups that are mounted. When you remove the 'cgroup_controllers' line, the default kicks in. According to the `qemu.conf` file, the default is:
`cgroup_controllers = ["cpu", "devices", "memory", "blkio", "cpuacct", "net_cls"]`
Then again, if it's not mounted, it's not used. So, that's a nice way to control it.
This works just fine for both RHEL/CentOS 6 and RHEL/CentOS 7.
Along the debugging, I came across some small issues here and there which I also fixed in this PR.
If you test this, be sure to build RPMs from this branch and install those on the KVM hypervisor.
At Schuberg Philis we run KVM hypervisors on CentOS 7 for months with these fixes. Now I took the time to fix the issues in CloudStack, rather than work around them.
@bhaisaab @karuturi could you please review this? Thanks!
Please note:
The next step is to fix CLOUDSTACK-8625 (Systemd profile for KVM Agent) as currently a sysvinit script is still used. Although it works, we do not have any systemd benefits. As it is a separate issue, I'll try to address it in a separate PR.
* pr/731:
CLOUDSTACK-8443: detect CentOS 7.x as RHEL 7 alike
CLOUDSTACK-8443: mention the correct logfile
CLOUDSTACK-8443: display the right hypervisor type
CLOUDSTACK-8443: don't try to fix co-mounted cgroups
Signed-off-by: Remi Bergsma <github@remi.nl>
This setting works on CentOS 6 / RHEL 6 but does nothing, as
"cpu" cgroup is not mounted. On CentOS 7 / RHEL 7 systemd does
mount cgroups and "cpu" is co-mounted with "cpuacc". Hence, if
we specify "cpu" then this results in an error because it can
only use them both, or none.
By removing the setting, we rely on the default of qemu, which
is:
cgroup_controllers = ["cpu", "devices", "memory", "blkio", "cpuacct", "net_cls"]
Only if they are really mounted, they will be used. So, this will
work on both version 6 and 7.
The 'fix script' didn't work well, as after a reboot you'd still have qemu
throwing errors. Now we can handle the co-mountedcgroups.
Fix for the NicVO.java regression.Renamed set*() methods to correct naming.
* pr/726:
Fix for the NicVO.java regression.
Signed-off-by: Remi Bergsma <github@remi.nl>
Removed duplicate code in CitrixResourceBase and its subclassesRemoved unnecessary duplicated code by having the body of the getPatchFiles method only in the CitrixResourceBase superclass. Given that all of its implementations consisted of the same code except for the path which contains the necessary file for that implementation. An abstract method getPatchFilePath was created in the CitrixResourceBase superclass so that each implementation may return the path containing the specific file needed by that implementation.
Test cases were created for each implementation, simple as they may be. One assert is made to verify that the path returned by each implementation corresponds to the path that was previously specified on each getPatchFiles implementation.
* pr/700:
Removed duplicate code in CitrixResourceBase.getPatchFiles
Signed-off-by: Remi Bergsma <github@remi.nl>
This tries to avoid cleaning by a device name.
This closes#718
(cherry picked from commit 74f697a2dd)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Changed methodnames according to Nic.java refactor.
Fixed NicVO.java due to regression from Nic.java refactor.
Fixed VmWareGuru.java after Nic.java refactor.
See issue CLOUDSTACK-8736 for ongoing effort to clean up network code.
@cristofolini comments:
Removed unnecessary duplicated code by having the body of the
getPatchFiles method only in the
CitrixResourceBase superclass. Given that all of its implementations
consisted of the same code
except for the path which contains the necessary file for that
implementation. An abstract method
getPatchFilePath was created in the CitrixResourceBase superclass so
that each implementation may
return the path containing the specific file needed by that
implementation.
Test cases were created for each implementation, simple as they may be.
One assert is made to verify
that the path returned by each implementation corresponds to the path
that was previously specified
on each getPatchFiles implementation.
removed trailing whitespace
Cloudstack 8656: do away with more silently ignoring exceptions.a lot of messages added.
some restructuring for test exception assertions and try-with-resource blocks
* pr/654: (29 commits)
CLOUDSTACK-8656: more logging instead of sysout
CLOUDSTACK-8656: use catch block for validation
CLOUDSTACK-8656: class in json specified not found
CLOUDSTACK-8656: removed unused classes
CLOUDSTACK-8656: restructure of tests
CLOUDSTACK-8656: reorganise sychronized block
CLOUDSTACK-8656: restructure tests to ensure exception throwing
CLOUDSTACK-8656: validate the throwing of ServerApiException
CLOUDSTACK-8656: logging ignored exceptions
CLOUDSTACK-8656: try-w-r removes need for empty catch block
CLOUDSTACK-8656: try-w-r instead of clunckey close-except
CLOUDSTACK-8656: deal with empty SQLException catch block by try-w-r
CLOUDSTACK-8656: unnecessary close construct removed
CLOUDSTACK-8656: message about timed buffer logging
CLOUDSTACK-8656: message about invalid number from store
CLOUDSTACK-8656: move cli test tool to separate file
CLOUDSTACK-8656: exception is the rule for some tests
CLOUDSTACK-8656: network related exception logging
CLOUDSTACK-8656: reporting ignored exceptions in server
CLOUDSTACK-8656: log in case we are on a platform not supporting UTF8
...
Signed-off-by: Remi Bergsma <github@remi.nl>
Before this commit the call to `getResizeScriptType` would throw
an exception (earlier versions returned `null`, which was fine) -
this caused the RBD case to fail. By changing the default to
notify only we fix the case for any non-CLVM and non-QCOW cases.
This is RBD for now, but this should extend to new storage
types suppored by Libvirt natively in future.
- Adds unit test for ListAndSwitchSAMLAccountCmd
- Checks and logs in user only if they are enabled
- If saml user switches to a locked account, send appropriate error message
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit b30977911d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
SAML authorized accounts might be across various domains, this allows for
switching of accounts only in case of SAML authenticated user accounts across
other accounts with the same SAML uid/username.
Moves the previous switch account logic to its own ui-custom module
(cherry picked from commit 1065661cd5)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
plugins/user-authenticators/saml2/src/org/apache/cloudstack/api/command/SAML2LoginAPIAuthenticatorCmd.java
ui/index.jsp
When dumping XML use appropriate flags:
1, VIR_DOMAIN_XML_SECURE (dump security sensitive information too)
8, VIR_DOMAIN_XML_MIGRATABLE (dump XML suitable for migration)
Source:
https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainXMLFlags
This fixes CVE 2015-3252: VNC password lost during VM migration across KVM
hosts. The issue is also seen when a VM is rebooted.
(cherry picked from commit cb2aca7516)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* pr/674:
getUsedBytes should query the SolidFire cluster to acquire the size of the given volume if there is no volume_details info for that volume (and then create a volume_details row for this volume so we don't have to make that cluster call for this purpose again)
Signed-off-by: Mike Tutkowski
Signed-off-by: Mike Tutkowski <mike.tutkowski@solidfire.com>
* pr/547:
CLOUDSTACK-8601. VMFS storage added as local storage can be re-added as shared storage. Fail addition of a VMFS shared storage pool in case it has already been added as local storage in CS.
Signed-off-by: Mike Tutkowski <mike.tutkowski@solidfire.com>
In LdapUserManagerFactory moved the beans to a map
used a Enum for LdapProvider and made the corresponding changes in
LdapConfiguration and the callers.
the changes in commit ec32ea30f7 changed
gmaven version in ldap plugin which broke running the spock tests from
command line. Hence reverting the change until we fix it with
the new version.
added a new configuration to select the appropriate ldap implementation
incase of microsoft AD enabled nested querying of group members
moved LdapUserManager to an interface and added separate implementations
for openLdap and microsoft AD
Added unit tests
On NFS we write a heartbeat, but without those we can not safely
fence off a host.
If we fence without knowing about a heartbeat we can cause a split-brain situation.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
Add option to authorize SAML SSO for user when adding user. Appends a domain to
user/account name if global setting is enabled, useful in case of multiple IDP
server. By default the setting is set to false (keep as it is).
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#575
(cherry picked from commit 7d11c7bc70)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Login is based on sessionkey HttpOnly Cookie
- ApiServlet does login verification using sessionKey from both the request cookies
and the API parameters. In both cases, if either or both are passed they should
match the sessionKey stored in the current session of the HttpRequest
- UI: it no longer needs to read or set sessionkey cookie
- UI: it no longer needs to return g_sessionKey value in the API requests, though
to support a sso mechanism g_sessionKey is still passed in the API is not null
- Secure jsessionid cookie is set to be HttpOnly and Secure
- SAML login should also set HttpOnly cookie before redirecting to UI
- SAML: listIdps & getSPMetadata APIs are readonly now, won't log out a logged in user
Performed tests (login, saml login if applicable, page refreshes, opening
multiple tabs, logout) with following combinations:
- SAML disabled, normal auth as admin, domain-admin and user
- SAML enabled, normal auth as admin, domain-admin and user; and saml sso as
admin, domain-admin and user
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#574
This closes#308
- Make sure the new password replaces the old one in the queue
- Updated the patch files for XenServer
- Updated the script path on LibvirtComputing class
- Adding update_host_passwd to VRScripts
- Add implementation to CitrixUpdateHostPasswordCommandWrapper
- Improve testUpdateHostPasswordCommand() unit test on CitrixRequestWrapperTest
- Adding update_host_passwd.sh script
- Adding the host IP address as an instance variable on UpdateHostPasswordCommand
- Improving the Unit Test (LibvirtComputingResourceTest) to get it covering the new code
- Make sure doUpdateHostPassword() doesn't get called if flag is set to false
- Do not update XenServer hosts if the cluster ID is not informed
This closes#559
- Adding more unit tests as suggested by Daan and Rajani.
- Mocking SshHelper and the Script classes, used by Citrix and Libvirt resources respectively.
- Changed location of the update_host_passwd script
- Updated the patch files for XenServer
- Updated the script path on LibvirtComputing class
- Removed the hostIP from the LibvirtUpdateHostPasswordCommandWrapper execute() method
- Adding update_host_passwd to VRScripts
- Add accessor method to host password on CitrixResourceBase
- Add implementation to CitrixUpdateHostPasswordCommandWrapper
- Improve testUpdateHostPasswordCommand() unit test on CitrixRequestWrapperTest
- Add line to patch files on xenserver directory
Concerning the LibVirt change:
- I forgot to assing the return of the getDefaultHypervisorScriptsDir() method to the hypervisorScriptsDir variable
listing the hosts for migration for instance with tag
While preparing the suitable hosts we are accidentally removing the
incompatible (host does not have host tag) hosts from otherhost list(
incorrect use of List.retainAll).
- Modifying the LibvirtUpdateHostPasswordCommandWrapper in order to execute the script on the host
- Adding the script path to LibvirtComputingResource
- Adding the host IP address as an instance variable on UpdateHostPasswordCommand
- Improving the Unit Test (LibvirtComputingResourceTest) to get it covering the new code
- Change the retry() method to return Answer.createUnsupportedCommandAnswer(command) instead of throwing an exception
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
This closes#546
- Adding command wrappers NiciraNvpCreateLogicalSwitchCommandWrapper and NiciraNvpDeleteLogicalSwitchCommandWrapper
- Refactoring the retry mechanism
- Applying the new retry mechanism to current wrappers and old methods in NiciraNvpResource
- Adding 2 tests
- Fixing the testRetries() in NiciraNvpResourceTest class
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
- Adding the NiciraNvpWrapper
- This class will keep track of all Wrappers of the Nicira NVP Plugin
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
This commit has a small refactoring of cloud-plugin-acl-static-role-based
to allow it to read files on the classpath that might have a different name
than "commands.properties". It also allows more than one file to be read from.
Rationale: Third-party plugins may want to keep their API command access level
configuration separate from the main file so as to reduce configuration
maintenance work during packaging and deployments.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#354
* Move config options to SAML plugin
This moves all configuration options from Config.java to SAML auth manager. This
allows us to use the config framework.
* Make SAML2UserAuthenticator validate SAML token in httprequest
* Make logout API use ConfigKeys defined in saml auth manager
* Before doing SAML auth, cleanup local states and cookies
* Fix configurations in 4.5.1 to 4.5.2 upgrade path
* Fail if idp has no sso URL defined
* Add a default set of SAML SP cert for testing purposes
Now to enable and use saml, one needs to do a deploydb-saml after doing a deploydb
* UI remembers login selections, IDP server
- CLOUDSTACK-8458:
* On UI show dropdown list of discovered IdPs
* Support SAML Federation, where there may be more than one IdP
- New datastructure to hold metadata of SP or IdP
- Recursive processing of IdP metadata
- Fix login/logout APIs to get new interface and metadata data structure
- Add org/contact information to metadata
- Add new API: listIdps that returns list of all discovered IdPs
- Refactor and cleanup code and tests
- CLOUDSTACK-8459:
* Add HTTP-POST binding to SP metadata
* Authn requests must use either HTTP POST/Artifact binding
- CLOUDSTACK-8461:
* Use unspecified x509 cert as a fallback encryption/signing key
In case a IDP's metadata does not clearly say if their certificates need to be
used as signing or encryption and we don't find that, fallback to use the
unspecified key itself.
- CLOUDSTACK-8462:
* SAML Auth plugin should not do authorization
This removes logic to create user if they don't exist. This strictly now
assumes that users have been already created/imported/authorized by admins.
As per SAML v2.0 spec section 4.1.2, the SP provider should create authn requests using
either HTTP POST or HTTP Artifact binding to transfer the message through a
user agent (browser in our case). The use of HTTP Redirect was one of the reasons
why this plugin failed to work for some IdP servers that enforce this.
* Add new User Source
By reusing the source field, we can find if a user has been SAML enabled or not.
The limitation is that, once say a user is imported by LDAP and then SAML
enabled - they won't be able to use LDAP for authentication
* UI should allow users to pass in domain they want to log into, though it is
optional and needed only when a user has accounts across domains with same
username and authorized IDP server
* SAML users need to be authorized before they can authenticate
- New column entity to track saml entity id for a user
- Reusing source column to check if user is saml enabled or not
- Add new source types, saml2 and saml2disabled
- New table saml_token to solve the issue of multiple users across domains and
to enforce security by tracking authn token and checking the samlresponse for
the tokens
- Implement API: authorizeSamlSso to enable/disable saml authentication for a
user
- Stubs to implement saml token flushing/expiry
- CLOUDSTACK-8463:
* Use username attribute specified in global setting
Use username attribute defined by admin from a global setting
In case of encrypted assertion/attributes:
- Decrypt them
- Check signature if provided to check authenticity of message using IdP's
public key and SP's private key
- Loop through attributes to find the username
- CLOUDSTACK-8538:
* Add new global config for SAML request sig algorithm
- CLOUDSTACK-8539:
* Add metadata refresh timer task and token expiring
- Fix domain path and save it to saml_tokens
- Expire hour old saml tokens
- Refresh metadata based on timer task
- Fix unit tests
This closes#489
(cherry picked from commit 20ce346f3a)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
client/WEB-INF/classes/resources/messages_hu.properties
plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resource/wrapper/xenbase/CitrixCheckHealthCommandWrapper.java
plugins/user-authenticators/saml2/src/org/apache/cloudstack/api/command/SAML2LoginAPIAuthenticatorCmd.java
ui/scripts/ui-custom/login.js
- Make sure the doUpdateHostPassword() method returns the result from the answer, not always true
- Added the LibvirtUpdateHostPasswordCommandWrapper class, which will handle the changes in the KVM agent
That's will be further implemented in the future. The command will avoid the agent to complain about Unsupported command
- Added a test to make sure the current implementation of the comment works
- If changes in the future, it will also require changes. In that way, we make sure nobody will break it
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
This closes#527
CLOUDSTACK-3317 - DVS does not support management\storage network
Added support for Management and Storage Network traffic over VMware DVS in CloudStack deployments. Also added support for storage VLAN over dvPortGroup.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
CLOUDSTACK-3317 - DVS does not support management\storage network
Use non-zero dvport count while updating dvportgroups of system traffic.
Updated configuration compare logic to avoid update dvportgroup operation unless required.
This would help improve speed in vm/network deployment as the update calls would reduce. Also improved logging.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Adding unit tests in class HypervisorHostHelperTest
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Added license header to new file being added to repo/branch.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Assertion is not used in runtime, correct way is throw and handle exception without killing app
Signed-off-by: Daan Hoogland <daan.hoogland@gmail.com>
This closes#362
VmwareResource.java:315, MS_SHOULD_BE_FINAL, Priority: High
com.cloud.hypervisor.vmware.resource.VmwareResource.s_serviceContext isn't final but should be
VmwareResource.java:331, MS_SHOULD_BE_FINAL, Priority: High
com.cloud.hypervisor.vmware.resource.VmwareResource.s_powerStatesTable isn't final but should be
Signed-off-by: Daan Hoogland <daan.hoogland@gmail.com>
This closes#363
KVM hosts which are actuall up, but if their agents are shutdown should be put
in disconnected state. This would avoid getting the VMs HA'd and other commands
such as deploying a VM will exclude that host and save us from errors.
The improvement is that, we first try to contact the KVM host itself. If it fails
we assume that it's disconnected, and then ask its KVM neighbours if they can
check its status. If all of the KVM neighbours tell us that it's Down and we're
unable to reach the KVM host, then the host is possibly down. In case any of the
KVM neighbours tell us that it's Up but we're unable to reach the KVM host then
we can be sure that the agent is offline but the host is running.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
This closes#340
When executing the tests in an environment where Libvirt is also installed, it
caused errors.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#342
Maven was bulding the project and executing the tests just fine because it uses "-sourcepath". However,
with IDEs and javac it would fail.
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
This closes#317
Pull average Cpu util report between polling intervals instead of since boot
instead of using values since uptime
(cherry picked from commit 04176eaf17)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
This closes#297
Pull average Cpu util report between polling intervals instead of since boot
instead of using values since uptime
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#289
Passing the file argument to the xml break for EL 7.1, the fix removes
the argument as just passing rombar='off' with its file arg to be empty string.
This closes#290
(cherry picked from commit aafa0c80b3)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Passing the file argument to the xml break for EL 7.1, the fix removes
the argument as just passing rombar='off' with its file arg to be empty string.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#290
EL7 has a different output to 'free', use /proc/meminfo instead of a tool to be
more consistent across distros
(cherry picked from commit 212a05a345)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
We add a new contract to pass Http request to authentication plugin system. In
the default login API, we disallow non-POST requests.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
EL7 has a different output to 'free', use /proc/meminfo instead of a tool to be
more consistent across distros
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Removing real IPs from the tests because they cause a long running time for LibvirtComputingResourceTest
- In a local machine it takes 1.977s, but in a KVM test environment it's taking 257.879 sec
... of new volumes. Following changes are implemented 1. Disable or enable a pool with the
updateStoragePool api. A new 'enabled' parameter added for the same. 2. When a
pool is disabled the state of the pool is updated to 'Disabled' in the db. On
enabling it is updated back to 'Up'. Alert is raised when a pool is disabled or
enabled. 3. Updated other storage providers to also honour the disabled state.
4. A disabled pool is skipped by allocators for provisioing of new volumes. 5.
Since the allocators skip a disabled pool for provisioning of volumes, the
volumes are also not listed as a destination for volume migration.
FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Disabling+Storage+Pool+for+Provisioning
This closes#257
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Fixing typo on LibvirtRequestWrapper
- Replace linbvirtCommands by libvirtCommands on LibvirtRequestWrapper
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#255
- The test was okay, but when running in an environment where a /root/.ssh/id_rsa existed, it would return true then fail
- We now mock the calls to methods that return the key paths, instead of relying in the static variables
- Adding LibvirtNetworkElementCommandWrapper and LibvirtStorageSubSystemCommandWrapper
- 2 unit tests added
- KVM hypervisor plugin with 22.2% coverage
I also refactored the StorageSubSystemCommand interface into an abstract class
- Remove the pseudo-multiple-inheritance implementation
- The StorageSubSystemCommand was an interface, not related to the Command class
and its implementation were extending the Command class anyway. The whole structure is better now.
- Addin LibvirtPvlanSetupCommandWrapper
- 6 unit tests added
- KVM hypervisor plugin with 21% coverage
From the 6 tests added, 2 were extra tests to increase the coverage of the LibvirtStopCommandWrapper
- Increased from 35% to 78.7%
- Adding LibvirtCopyVolumeCommandWrapper
Refactoring the LibvirtUtilitiesHelper
- Changing method name
Did not add any test to this commit due to the refactor mentioned abot.
Will proceed and add the tests
i# Please enter the commit message for your changes. Lines starting
- Gave it a better, more suggestive, name since I now added other methods to the class.
- It makes easier to mock objects and get a better coverage of the classes
- Adding LibvirtBackupSnapshotCommandWrapper, LibvirtCreatePrivateTemplateFromVolumeCommandWrapper and LibvirtManageSnapshotCommandWrapper
- 3 unit tests added
- KVM hypervisor plugin with 18.3% coverage
Less tests added to those classes because the code is quite complex and way too long.
The tests added are just covering the new flow, to make sure it works fine. I will come back to those classes later.
- Adding LibvirtOvsDestroyBridgeCommandWrapper, LibvirtOvsSetupBridgeCommandWrapper
- 4 unit tests added
- KVM hypervisor plugin with 13.9% coverage
More tests added to cover LibvirtPrepareForMigrationCommandWrapper
- Coverage of this wrapper broght from 37% to 90.6%
- 4 new tests added
- Adding LibvirtCheckConsoleProxyLoadCommandWrapper, LibvirtConsoleProxyLoadCommandWrapper, LibvirtWatchConsoleProxyLoadCommandWrapperand CitrixConsoleProxyLoadCommandWrapper
- 2 unit tests added
- KVM hypervisor plugin with 12% coverage
Refactored the CommandWrapper interface in order to remove the esecuteProxyLoadScan, which is now
implemented bu subclasses.
- Adding LibvirtGetHosStatsCommandWrapper
- 1 unit test added
- KVM hypervisor with 10.5% coverage
Tests are a bit limited on this one becuause of the current implementation. Would clean it up later in a separate branch
- Adding LibvirtGetVmStatsCommandWrapper
- 3 unit tests
Refactored the LibvirtConnectiobn by surrounding it with an wrapper.
- Make it easier to cover the static/native calls
- Added better coverage to StopCommand tests
- Adding LibvirtStopCommandWrapper
- LibvirtRequestWrapper
- 1 unit tests
Refactored the RequestWrapper to make it better.
- Changes also applied to the CitrixRequestWrapper
- Removes awsapi db properties usage across codebase
- Removes references from spring xmls, test cases and TransactionLegacy
- Adds sql command to drop database cloudbridge in schema-451to460-cleanup.sql
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
From b3f18e7d74, the zone level systemvm local
storage setting never worked as it needed to be moved to config depot.
(cherry picked from commit 279efb0432)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
plugins/network-elements/internal-loadbalancer/src/org/apache/cloudstack/network/lb/InternalLoadBalancerVMManagerImpl.java
server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java
From b3f18e7d74, the zone level systemvm local
storage setting never worked as it needed to be moved to config depot.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Linux kernel supports vmxnet3, allowing it in KVM plugin would allow us to
run ESX hosts on KVM hosts using CloudStack with vmxnet3 nic which can be
passed as VM's nicAdapter detail
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit e02d787f30)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Linux kernel supports vmxnet3, allowing it in KVM plugin would allow us to
run ESX hosts on KVM hosts using CloudStack with vmxnet3 nic which can be
passed as VM's nicAdapter detail
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This improvements checks for "guest.cpu.features" property which is a space
separated list of cpu features that is specific for a host. When added, it
will add <feature policy='require' name='{{feature-you-listed}}'/> in the
<cpu> section of the generated vm spec xml.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit ea7fd37783)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This improvements checks for "guest.cpu.features" property which is a space
separated list of cpu features that is specific for a host. When added, it
will add <feature policy='require' name='{{feature-you-listed}}'/> in the
<cpu> section of the generated vm spec xml.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Dont evict template when a delete command has been sent to VMware resource for deletion of volume.
(cherry picked from commit f45e6b94ed)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The only artifact resolved from libvirt.org was org.libvirt:libvirt:0.5.1
this artifact is now available from maven's default central repository
This closes#180
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 9cf31b0714)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The only artifact resolved from libvirt.org was org.libvirt:libvirt:0.5.1
this artifact is now available from maven's default central repository
This closes#180
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
MigrateVMWithVolumes-
1. If ESXi host version is below 5.1, ensure destination datastore(s) is mounted on the source host, then migrate the storage and then finally migrate the VM.
If destination storage(s) is not mounted on the source host,
- In case of NFS storage mount the storage(s).
- In case of VMFS storage fail the request for migration.
2. If EXi host version is 5.1 or above, simultaneously migrate the VM and its storage to the destination host and storage(s) respectively for both NFS and VMFS storage.
(cherry picked from commit adc836cc5e)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Correctly register node info for a newly created VMware context.
(cherry picked from commit 13bdc1cef4)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
As suggested by Wido on the dev ML changing the repo to eu.ceph.com to avoid
build failures. Will revert if ceph.com is up again.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit c9fd57fff3)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
As suggested by Wido on the dev ML changing the repo to eu.ceph.com to avoid
build failures. Will revert if ceph.com is up again.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CentOS 7 does not ship with ifconfig anymore. We should use ip commands instead.
This also works on older versions, like CentOS 6 and Ubuntu 12.x/14.x, that we
support.
(cherry picked from commit ac06ec02eb)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/IvsVifDriver.java
CentOS 7 does not ship with ifconfig anymore. We should use ip commands instead.
This also works on older versions, like CentOS 6 and Ubuntu 12.x/14.x, that we
support.
This closes#165
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- use sticky chmod 1777 on the mountpoint
- remove dead code
(cherry picked from commit eea716b791)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- use sticky chmod 1777 on the mountpoint
- remove dead code
- port improved code for moving disk into corresponding folder from master
- for dummy worker case, check case for powered off vm state
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
It causes extreme long vm start time when managment server has slow connection to secondary storage
(cherry picked from commit 49c01e2ae4)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
While looking for an ongoing VM snapshot task, check the task status to identify if the task is still running.
(cherry picked from commit 25a4f0dc53)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Handling spaces in datastore name while extracting vmdk base name
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
(cherry picked from commit aa84b05491)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Update the path and size of data volumes after snapshot creation/deletion by correctly trimming only the snapshot postfix of a disk.
(cherry picked from commit ac491c9607)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Fix the OVA path that is returned once an OVA is packaged using a META file.
(cherry picked from commit 507d9d337d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
1. provide compatibility with the Big Cloud Fabric (BCF) controller
L2 Connectivity Service in both VPC and non-VPC modes
2. virtual network terminology updates: VNS --> BCF_SEGMENT
3. uses HTTPS with trust-always certificate handling
4. topology sync support with BCF controller
5. support multiple (two) BCF controllers with HA
6. support VM migration
7. support Firewall, Static NAT, and Source NAT with NAT enabled option
8. add VifDriver for Indigo Virtual Switch (IVS)
This closes#151
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- With the changes added by the rVPC work, the bump priority became deprecated.
This commit includes a refactor to get it removed from the following resources:
* Java classes
* domain_router table - removing the is_priority_bumpup column
* Fixing unit tests
All changes were tested with:
XenServer 6.2 running under our VMWare zone
CloudStack Management Server running on MacBook Pro
MySql running on MackBook Pro
Storage Type: Local
Started the refactor of the XenServer56Resource class
- Unit test added
Changing the CitrixRequestWrapper in order to cope with multiple resources classes and commands
Still have to remove few methods form CitrixResourceBase
- Unit tests added
In the executeRequest I needed to keep the following:
// We need this one because the StorageSubSystemCommand is from another hierarchy.
if (cmd instanceof StorageSubSystemCommand) {
return storageHandler.handleStorageCommands((StorageSubSystemCommand) cmd);
}
- Added basic tests
- Changed the way RebootCommand gets called from RebootRouterCommand
- Made a couple of methods public in the CitrixResourceBase and its subclasses
Sonarcube is great, but has no context and can be wrong, in this
case File.separator is nice if you're running platform independent,
the agent on OVM3 is however NOT platform independent and will break
if we feed it windows separators.
The copy command reply can have null size returned, so check and set values
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 53ca0b1861)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Fix: RTNETLINK errors
- Management Server health check trying to create already existing interface
- Changes on update_config.py, cs_guestnetwork.py, merger.py
Fix: replace RRouTER_LOG in the CsRedundant.py per log file location
Fix: Guest Net address association during Router restart
- Changes on NicProfileHelper, NicProfileHelperImpl
Fix: aggregationExecution() method on VirtualNetworkApplianceManagerImpl
- Do not send an AggregationControlCommand to a non-configured router
Some classes have been formatted.
Added a source column to the user table.
Source now has only two values UNKNOWN,LDAP with UNKNOWN being the
default and is an enum is com.cloud.User.
When the source is UNKNOWN, the old method of authenticating against all
the available authenticators is used. If a source is available, only
that particular authenticator will be used.
added overloaded methods in AccountService to createUserAccount and
createUser with source specified.
(cherry picked from commit 5da733072e)
Added a source column to the user table.
Source now has only two values UNKNOWN,LDAP with UNKNOWN being the
default and is an enum is com.cloud.User.
When the source is UNKNOWN, the old method of authenticating against all
the available authenticators is used. If a source is available, only
that particular authenticator will be used.
added overloaded methods in AccountService to createUserAccount and
createUser with source specified.
The 19e3c0168e commit breaks SAML login and any
login where redirection is used.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit b79f13ccb5)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
MigrateVMWithVolumes-
1. If ESXi host version is below 5.1, ensure destination datastore(s) is mounted on the source host, then migrate the storage and then finally migrate the VM.
If destination storage(s) is not mounted on the source host,
- In case of NFS storage mount the storage(s).
- In case of VMFS storage fail the request for migration.
2. If EXi host version is 5.1 or above, simultaneously migrate the VM and its storage to the destination host and storage(s) respectively for both NFS and VMFS storage.
This is a plugin that puts in ovm3 support ranging from 3.3.1 to 3.3.2. Basic
functionality is in here, advanced networking etc..
Snapshots only work when a VM is stopped now due to the semantics of OVM's raw
image implementation (so snapshots should work on a storage level underneath the
hypervisor shrug)
This closes#113
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This commit produces event bus messages to a "cloudstack" topic
in Apache Kafka. Configuration is expected to be found in
/etc/cloudstack/management/kafka.producer.properties and will
generally be of the form:
bootstrap.servers=kafka-host1:9092,kafka-host2:9092
key.serializer=org.apache.kafka.common.serialization.StringSerializer
value.serializer=org.apache.kafka.common.serialization.StringSerializer
There is no way to parameterize the topic yet, and the consuming
code is just place-holder. I think adding a consumer within cloudstack
is very debatable and likely not needed.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
When there is large size VR configuration (aggregate commands) copying data to VR using vmops plugin was failed
because of the ARG_MAX size limitation. The configuration data size is around 300KB.
Updated this to create file in host by scp with file contents. This will create file in host.
Then copy the file from the host to VR using hte vmops createFileInDomr method.
In host file get created in /tmp/ with name VR-<UUID>.cfg, once it copied to VR this file will be removed.
(cherry picked from commit 619f014255)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
When there is large size VR configuration (aggregate commands) copying data to VR using vmops plugin was failed
because of the ARG_MAX size limitation. The configuration data size is around 300KB.
Updated this to create file in host by scp with file contents. This will create file in host.
Then copy the file from the host to VR using hte vmops createFileInDomr method.
In host file get created in /tmp/ with name VR-<UUID>.cfg, once it copied to VR this file will be removed.
The test testFindSecurityProfileByUuid built an assumptio on the order of the arguments after transforming through a HashMap.
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
Refactored to use the XPatch expressions to check the generated domain xml rathern than string comparison.
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
Design Document:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Allow+VM+listing+by+User+ID
- Adds column to VMInstance DAO
- Adds column in vm_instance table
- Adds column in the UserVMJoinVO
- Adds default admin user which has UID = 2
- Adds migration path that sets user_id to first user of the accountId that
owns the vm in vm_instance table
- Add arg on list VMs API to query by userId, add support in query layer
- Refactor VMInstanceVO and child classes to accept userId
- Add code to let service layer pass userId if loggedIn user belongs to same
account as the owner executing an API call or use first user from owner account
- In case of CPVM and SSVM use system user ID
- Fix unit tests and spring injections
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Earlier host addition of multiple hosts with local storage failed due to
same local storage UUID being used where the storage path is same.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit bf17f640c6)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Earlier host addition of multiple hosts with local storage failed due to
same local storage UUID being used where the storage path is same.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Fix cleanup of NFS mounts on management server during server starup by correcting how mount points are listed for a management server.
(cherry picked from commit 647532376f)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
For AttachVolume/DetachVolume API command, improve user error message in case of RuntimeException by throwing the exception instead of 'Unexpected Exception'.
(cherry picked from commit 4d7ede535d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
server/src/com/cloud/storage/VolumeApiServiceImpl.java
If VM has been cold migrated across different VMware DCs, then unregister the VM from source host.
(cherry picked from commit 15b348632d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
1. If a VM by the same name exists on a different cluster in the VMware DC, unregister the existing VM and continue with the VM start.
2. If VM start succeeds, delete VM files associated with the unregistered VM.
3. If VM start fails, re-register the unregistered VM.
(cherry picked from commit 974b0180dd)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
If an exact match is being done while locating disk chain by name, don't trim snapshot postfix appended to the disk name.
(cherry picked from commit ddcae8a930)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Before registering a VM check if a different CS VM with same name exists in vCenter.
(cherry picked from commit 33179cce56)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Look for a VM in vCenter based on both the vCenter name and CS internal name (required in case 'vm.instancename.flag' is enabled).
During Attach Volume and Volume Migration, for lookup and other operations use VM's name as obtained from vCenter instead of using the name set in the agent command.
(cherry picked from commit b8fdda0a34)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Fixed some issues with the polling logic to handle scenarios when SSVM is destroyed/down.
Also changed the status of the volume_store_ref entries so that subsequent operations work fine.
KVMStoragePoolManager is a singleton in practice, any plugin
or extension of LibvirtComputingResource will need to act on
the specific instance of KVMStoragePoolManager that LibvirtComputingResource
has initialized. Therefore, expose this variable for those who
wish to call storage commands from plugins or extensions.
Conflicts:
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
KVMStoragePoolManager is a singleton in practice, any plugin
or extension of LibvirtComputingResource will need to act on
the specific instance of KVMStoragePoolManager that LibvirtComputingResource
has initialized. Therefore, expose this variable for those who
wish to call storage commands from plugins or extensions.
Conflicts:
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
The fix generates X509Certificate if missing from DB and uses that for eternity.
SAML SP metadata remains same since it's using the same X509 certificate and
it remains same after restarts. The certificate is serialized, base64 encoded
and stored in the keystore table under a specific name. For reading, it's
retrieved, base64 decoded and deserialized.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 4358714381)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The fix generates X509Certificate if missing from DB and uses that for eternity.
SAML SP metadata remains same since it's using the same X509 certificate and
it remains same after restarts. The certificate is serialized, base64 encoded
and stored in the keystore table under a specific name. For reading, it's
retrieved, base64 decoded and deserialized.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
As per Version 1 cookies, certain characters are now allowed such as space,
colons etc but they should be url encoded using UTF8 encoding. The frontend
has a cookie value unboxing method that removes any double quotes that are added.
As per the doc http://download.oracle.com/javase/6/docs/api/java/net/URLEncoder.html
values are application/x-www-form-urlencoded and as per
http://www.w3.org/TR/html4/interact/forms.html#h-17.13.4 whitespaces are encoded
as +, therefore '+' are replaced by %20 (whitespace).
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 734bd70173)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
As per Version 1 cookies, certain characters are now allowed such as space,
colons etc but they should be url encoded using UTF8 encoding. The frontend
has a cookie value unboxing method that removes any double quotes that are added.
As per the doc http://download.oracle.com/javase/6/docs/api/java/net/URLEncoder.html
values are application/x-www-form-urlencoded and as per
http://www.w3.org/TR/html4/interact/forms.html#h-17.13.4 whitespaces are encoded
as +, therefore '+' are replaced by %20 (whitespace).
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The User table's UUID column is restricted to 40 chars only, since we don't
know how long the nameID/userID of a SAML authenticated user will be - the fix
hashes that user ID and takes a substring of length 40 chars. For hashing,
SHA256 is used which returns a 64 char length string.
- Fix tests, add test cases
- Improve checkSAMLUser method
- Use SHA256 one way hashing to create unique UUID for SAML users
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit b2b496288d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The User table's UUID column is restricted to 40 chars only, since we don't
know how long the nameID/userID of a SAML authenticated user will be - the fix
hashes that user ID and takes a substring of length 40 chars. For hashing,
SHA256 is used which returns a 64 char length string.
- Fix tests, add test cases
- Improve checkSAMLUser method
- Use SHA256 one way hashing to create unique UUID for SAML users
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
1. If a VM by the same name exists on a different cluster in the VMware DC, unregister the existing VM and continue with the VM start.
2. If VM start succeeds, delete VM files associated with the unregistered VM.
3. If VM start fails, re-register the unregistered VM.
For AttachVolume/DetachVolume API command, improve user error message in case of RuntimeException by throwing the exception instead of 'Unexpected Exception'.
on calling GetUploadParamsForTemplate, persisting the metadata to db
validating the account limits and incrementing the appropriate limits
encoded the metadata on management server using preshared key
On default iptables rules are updated to add ACCEPT egress traffic.
If the network egress default policy is false, CS remove ACCEPT and adds the DROP rule which
is egress default rule when there are no other egress rules.
If the CS network egress default policy is true, CS won't configure any default rule for egress because
router already came up to accept egress traffic. If there are already egress rules for network then the
egress rules get applied on VR.
For isolated network with out firewall service, VR default allows egress traffic (guestnetwork --> public network)
added 3 new method to strip partition information from VirtualServer, LBPool, VLAN api response.
With BigIP V11.x VirtualServer, LBPool, VLAN api response has been modified.
Now BigIP returns resource name with user partition information
ex: if vlanname is vlan-100 then the get_list() will return /Common/vlan-100 (/Common -> Suer portition)
This method will strip the partition information and only returns a list with vlan name (vlan-100)
Signed-off-by: Rajani Karuturi <rajanikaruturi@gmail.com>
On default iptables rules are updated to add ACCEPT egress traffic.
If the network egress default policy is false, CS remove ACCEPT and adds the DROP rule which
is egress default rule when there are no other egress rules.
If the CS network egress default policy is true, CS won't configure any default rule for egress because
router already came up to accept egress traffic. If there are already egress rules for network then the
egress rules get applied on VR.
For isolated network with out firewall service, VR default allows egress traffic (guestnetwork --> public network)
Changes;
- Upgrades maven-war plugin to 4.5 (faster war packaging)
- Upgrade spring framework to latest minor release
- Upgrade ehcache, jasypt, httpclient, httpcore and other core dependencies
- Upgrade to latest ipv6 library, fix unit test NetUtilsTest
- httpcore and httpclient are sharing same version variable
- commons-httpclient is different that httpclient, the fix gives it a separate var
- Apidocs failed to generate and get stuck with new reflections version, for now
we will continue using 0.9.8
Newer dependencies can be listed using:
mvn versions:display-dependency-updates -Dnoredist -Dsimulator -P developer,systemvm
Testing;
- Tested using Maven 3.2.1
- Local noredist build with unit tests succeeds
- CloudStack mgmt server started, basic business layer tests work
- Observed 10-15% build time improvement using new maven-war plugin
Branch: bugfix/4.5-8011 (commits are squashed in favour of a linear history)
Pull request:
https://github.com/apache/cloudstack/pull/50
This closes#50
TravisCI build summary:
https://travis-ci.org/shapeblue/cloudstack/builds/42902172
- Build passes with unit tests
- Apidocs generates successfully
- Most integration tests pass, some fail due to timeout errors, second re-run
passes some of them
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit fac7bfc5d5)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
pom.xml
Changes;
- Upgrades maven-war plugin to 4.5 (faster war packaging)
- Upgrade spring framework to latest minor release
- Upgrade ehcache, jasypt, httpclient, httpcore and other core dependencies
- Upgrade to latest ipv6 library, fix unit test NetUtilsTest
- httpcore and httpclient are sharing same version variable
- commons-httpclient is different that httpclient, the fix gives it a separate var
- Apidocs failed to generate and get stuck with new reflections version, for now
we will continue using 0.9.8
Newer dependencies can be listed using:
mvn versions:display-dependency-updates -Dnoredist -Dsimulator -P developer,systemvm
Testing;
- Tested using Maven 3.2.1
- Local noredist build with unit tests succeeds
- CloudStack mgmt server started, basic business layer tests work
- Observed 10-15% build time improvement using new maven-war plugin
Branch: bugfix/4.5-8011 (commits are squashed in favour of a linear history)
Pull request:
https://github.com/apache/cloudstack/pull/50
This closes#50
TravisCI build summary:
https://travis-ci.org/shapeblue/cloudstack/builds/42902172
- Build passes with unit tests
- Apidocs generates successfully
- Most integration tests pass, some fail due to timeout errors, second re-run
passes some of them
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This should fix that we can't gather CPU statistics on hypervisors
> Ubuntu 12.04
(cherry picked from commit 69ee01af9d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Findbugs will report error on this as it is expecting true/false for Boolean value.
But we have diffrent meaning for null so it is false positive case from findbug
This closes#39
Findbugs will report error on this as it is expecting true/false for Boolean value.
But we have diffrent meaning for null so it is false positive case from findbug
This closes#39
Look for a VM in vCenter based on both the vCenter name and CS internal name (required in case 'vm.instancename.flag' is enabled).
During Attach Volume and Volume Migration, for lookup and other operations use VM's name as obtained from vCenter instead of using the name set in the agent command.
Web UI shows all DHCP/PXE providers in cloud when admin click DHCP/PXE IP for A zone
Conflicts:
plugins/hypervisors/baremetal/src/com/cloud/baremetal/networkservice/BaremetalKickStartServiceImpl.java
GPU enabled hosts from non-GPU VM deployment.
Cluster reordering is based on the number of unique host tags in a cluster,
cluster with most number of unique host tags will put at the end of list.
Hosts with GPU capability will get tagged with implicit tags defined by
global config param 'implicit.host.tags' at the time os host discovery.
Also added FirstFitPlannerTest unit test file.
GPU enabled hosts from non-GPU VM deployment.
Cluster reordering is based on the number of unique host tags in a cluster,
cluster with most number of unique host tags will put at the end of list.
Hosts with GPU capability will get tagged with implicit tags defined by
global config param 'implicit.host.tags' at the time os host discovery.
Also added FirstFitPlannerTest unit test file.
Also when the ssvm is destroyed all the download urls are expired to be cleaned up in the next run by the new ssvm.
(cherry picked from commit ce90837357)
Clearly show if a volume is found and if not, that the pool is being refreshed
and the fetch is tried again.
Due to my commit b53a9dcc9f the chance of a volume
not being found is slightly bigger, but the performance gain is enormous on larger
deployments.
This is why we clearly have to log that we are refreshing the pool information
when a volume is not found.
It could be that a volume is created on host A and a few seconds later host B tries
to access the volume. In that case host B's libvirt doesn't know about the volume
yet and has to refresh the pool before it does.
(cherry picked from commit 4ee82f1f40)
2.8.7 to 3.3.5
3.3.5 is the latest stable version of AMQP client, which is also
backward comaptible. Successfully tested with updated client library.
(cherry picked from commit ff797dfa59)
modules can use it.
(cherry picked from commit b6401b04f22b0a5b686c7c477da4c6e0fd18df84)
Conflicts:
plugins/hypervisors/baremetal/src/com/cloud/baremetal/networkservice/BaremetalKickStartServiceImpl.java
GPU enabled hosts from non-GPU VM deployment.
Cluster reordering is based on the number of unique host tags in a cluster,
cluster with most number of unique host tags will put at the end of list.
Hosts with GPU capability will get tagged with implicit tags defined by
global config param 'implicit.host.tags' at the time os host discovery.
Also added FirstFitPlannerTest unit test file.
(cherry picked from commit 39fe766c2b)
Support offline volume resize on ESX by creating a worker VM to attach the unattached volume to and then resize it.
(cherry picked from commit 65ed25b7a6)
While expunging a volume, CS chooses the endpoint to perform delete operation by selecting any host that has the storage containing the volume mounted on it.
Instead, if the volume to be deleted is attached to a VM, the endpoint chosen by CCP should be the host that contains the VM.
(cherry picked from commit f1e3e83bbf)
If CCP thread local context which is used to handle connections to a vCenter is being re-used, validate that the context corresponds to the right vCenter API session.
(cherry picked from commit 6b06970366)
On larger (especially RBD) storage pools this can take a lot of
time slowing operations like creating volumes down.
The getStorageStats command will still ask a pool to be refreshed so
that the management server has accurate information about the storage pools.
On larger deployments, with thousands of volumes in one pool, this should
significantly improve storage related operations
(cherry picked from commit b53a9dcc9f)
cleaning up vmsync changes, checkvirtualmachine command was updated to return the
power state of the vm. The change was missed for Hyper-V. This caused migration to
fail on cloudstack even though it used to succeed on the hypervisor. Updated the
hyper-v agent code to return the cloudstack equivalent power state for check virtual
machine answer.
(cherry picked from commit 5350e61187)
Clearly show if a volume is found and if not, that the pool is being refreshed
and the fetch is tried again.
Due to my commit b53a9dcc9f the chance of a volume
not being found is slightly bigger, but the performance gain is enormous on larger
deployments.
This is why we clearly have to log that we are refreshing the pool information
when a volume is not found.
It could be that a volume is created on host A and a few seconds later host B tries
to access the volume. In that case host B's libvirt doesn't know about the volume
yet and has to refresh the pool before it does.
GPU enabled hosts from non-GPU VM deployment.
Cluster reordering is based on the number of unique host tags in a cluster,
cluster with most number of unique host tags will put at the end of list.
Hosts with GPU capability will get tagged with implicit tags defined by
global config param 'implicit.host.tags' at the time os host discovery.
Also added FirstFitPlannerTest unit test file.
GPU enabled hosts from non-GPU VM deployment.
Cluster reordering is based on the number of unique host tags in a cluster,
cluster with most number of unique host tags will put at the end of list.
Hosts with GPU capability will get tagged with implicit tags defined by
global config param 'implicit.host.tags' at the time os host discovery.
Also added FirstFitPlannerTest unit test file.
While expunging a volume, CS chooses the endpoint to perform delete operation by selecting any host that has the storage containing the volume mounted on it.
Instead, if the volume to be deleted is attached to a VM, the endpoint chosen by CCP should be the host that contains the VM.
If CCP thread local context which is used to handle connections to a vCenter is being re-used, validate that the context corresponds to the right vCenter API session.
On larger (especially RBD) storage pools this can take a lot of
time slowing operations like creating volumes down.
The getStorageStats command will still ask a pool to be refreshed so
that the management server has accurate information about the storage pools.
On larger deployments, with thousands of volumes in one pool, this should
significantly improve storage related operations
cleaning up vmsync changes, checkvirtualmachine command was updated to return the
power state of the vm. The change was missed for Hyper-V. This caused migration to
fail on cloudstack even though it used to succeed on the hypervisor. Updated the
hyper-v agent code to return the cloudstack equivalent power state for check virtual
machine answer.
Fix the bug by removing the command CreateEntityDownloadURLCommand from the host delegation. This results in same ssvm for creating the symlink on ssvm and same public ip being used for generating the url on MS.
NPE can happen if Spring fails to inject api authenticator, so better check
and set list of commands if the authenticator is not null or returning null cmds
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Return super.true() even if plugin is not enabled
- Return empty list when getCommands is called
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Latest httpclient library suggests when creating Protocol object for use
with the apache common httpclient class, they should avoid using the
deprecated Protocol signature which takes in SecureProtocolSocketFactory
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
If session is null, probably logout (local) happened removing the name id and
session index which is needed for global logout. The limitation by design is that
local logout will void possibility of global logout. To globally logout, one
use the SLO api which would logout locally as well.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Fixes signatures on plugin manager for ease of testing
- Fixes authenticator
- Adds unit testing for getType and authenticate methods for all cmd classes
- Adds SAMLAuthenticator test
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This adds GetServiceProviderMetaDataCmd which returns SP metadata XML, since
this information should be public for IdPs to discover, we implement this as a
login/cmd api so this does not require any kind of authentication to GET this
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Use opensaml version from root pom
- Add utils and api as explicit dependency
- Add org.apache.cloudstack.saml.SAML2AuthServiceImpl bean
- Fix imports in all source files and resource xmls
- Use methods available from SAMLUtils to encode/decode SAML request/response
- SAML logout api is not the global logout api
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Move interfaces and classes from server to api module
- This can be then used for pluggable api authenticators
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This is a feature to handle DNS entries by means of an external DNS Provider,
such as Bind. These entries include DNS domains and reverse domains, VM records
and reverse records.
For a complete description, please refer to the design document available at
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Bind+and+PowerDNS+integration+by+Globo+DNSAPI
For the discussion about this feature on the dev mailing list, please refer to
http://markmail.org/thread/fvwf36hpxotiibka
Summary:
- new Network Service Provider called GloboDNS
- new Network Element to manage network domains and VM records (entries) on an external API
- new Network Resource to communicate with GloboDNS (open source)
- new API command to add DNS server
- new global option to determine if this provider should override VM entries on external DNS server
- changes in UI to include GloboDNS in Providers list
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
1. While destroying a ROOT volume do the lookup of the associated VM under the DC and not just cluster.
2. In case of VMware, during VM start if a volume is being recreated no need to detach the old volume because
we now expunge it immediately and don't wait for the storage cleanup task to run.
During VM start while configuring its disk devices, obtain the matching disk for a volume in storage
using both the volume's path and volume's datastore information.
While adding host to existing cluster which is using Nexus 1000v as a network backend, skip validation of Nexus VSM as it was already done while adding that cluster.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
determined. To exclude hosted system we filter the result on Caption='Virtual Machine'
but this string is locale dependent so it may not not work properly for locales other
than english. To overcome this now we started using ProcessId >= 0 filter
For ResizeVolume API command -
1. If hypervisor resource throws an exception, handle the NPE thrown by the job framework.
2. Improve user error message in case of RuntimeException by throwing the exception instead of 'Unexpected Exception'.
We don't need an external script to investigate the format of the RBD volume,
we only have to ask Libvirt to resize the volume and that will ask librbd to
do so.
1. Fixed JSON response deserialization. While creating a mock a JSON can be passed which will be deserialized into a response object and returned from agent layer.
For e.g. for a mock corresponding to StopCommand, a response like "{"com.cloud.agent.api.StopAnswer":{"result":false,"wait":0}}" can be passed.
2. Ability to mock PingCommand (returned as part of getCurrentStatus() agent method). As a part of this a mocked VM state report can be returned.
For e.g. {"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"newStates":{},"_hostVmStateReport":{"v-2-VM":{"state":"PowerOn","host":"SimulatedAgent.e6df7732-69b2-429b-9b6a-3e24dddfa2e0"},"i-2-5-VM":{"state":"PowerOff","host":"SimulatedAgent.e6df7732-69b2-429b-9b6a-3e24dddfa2e0"}},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":3,"contextMap":{},"wait":0}}
in the XenServer pool for GRE tunnel networks
Fix uses XenServer recommended way
Network.other_config:assume_network_is_shared=true
which ensures bridge is created automatically on hosts in the pool for
GRE tunnel networks. Fix also gets rid of error prone custom logic that ensures
bridge is created by plugging a VIF into the dom0 and connected to
GRE tunnel network.
Conflicts:
plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resource/CitrixResourceBase.java
xen cluster
XenServer does not create a bridge automatically when VIF from domU is connected
to internal network. So there is logic to force bridge creation by
creating VIF in dom0 connected to GRE tunnel network. But there is no
logic to delete the VIF after bridge gets created. So this fix ensure
VIF is delted when atleast there is one domU VIF connected to the
network.
1. Adding the missing Template/Volume URLs expiration functionality
2. Improvement - While deleting the volume during expiration use rm -rf as vmware now contains directoy
3. Improvement - Use standard Answer so that the error gets logged in case deletion of expiration link didnt work fine.
4. Improvement - In case of domain change, expire the old urls
In situations where libvirt lost the storage pool the KVM Agent will re-create the
storage pool in libvirt.
This could be then libvirt is restarted for example.
The object returned internally was missing essential information like the sourceDir
aka the Ceph pool, the monitor IPs, cephx information and such.
In this case the first operation on this newly created pool would fail. All operations
afterwards would succeed.
The proxy-arp add/del is done on firewall rule add/del.
The proxy-arp rule is deleted only when there is no static nat or dest nat rule is not using the ip.
When there is static nat or PF and firewall rule
a. Delete firewall rule. It skips delete proxy-arp because the rule is used by static nat rule.
b. After deleting fw rule if we disable static nat there is no way to delete proxy-arp rule.
On VM expunge we are deleting firewall rules first then static nat rules. This caused the stale proxy-arp
rules.
With this fix adding/deleting proxy arp rule on static nat/PF rule add/del.
to get nic info of 0.0.0.0. To get it, it iterates through all nics and return the last NIC in
the list if it doesn't match with any IP address. In case last NIC doesn't have unicastAddress,
Hyper-V agent will fail to start. We don't need IP address during initialization. It get
initialized with startupcommand later.
last VM from the VPC is deleted on a host
OVS distributed routing: ensure bridge is deleted when last VM from the
VPC is deleted on a host. This fix ensures that bridge is
destroyed.
the folder column. For an smb share the smb credentials are in the query string of the path.
Before adding the path, smb shares query string should be cleaned up.
that case cloudstack was not doing anything and not updating the state of the vms to stopped.
Now the agent returns empty list of hostvmstatereport. Management server will then update the
vm state to stopped (instead of not acting upon the return state).
not created already when OvsVpcPhysicalTopologyConfigCommand update is
recived
Currently if the tunnel creation fails, there is no retry logic. Fix
ensures OvsVpcPhysicalTopologyConfigCommand updates as an opputiunity to ensure
proper tunnels are established between the hosts.
Check if switch name detected from traffic label for management, storage, control traffic is null before falling back to default value.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
We used to create the snapshot after the copy from Secondary Storage,
but it could be that we never use the snapshot.
Now we check if the snapshot exists prior to performing the cloning operation
Since we use qemu-img to copy from RBD to Secondary Storage we no
longer have to force to RAW images, but can stick with QCOW2
When the snapshot backups are QCOW2 format they can easily be deployed
again when restoring from a backup
The KVMStorageProcessor no longer has a hardcoded if-statement which sets
RBD volumes to RAW, this is now handled in the LibvirtStorageAdapter
The Management Server still sends QCOW2 as format. That's a fix for later.
templatefilter="shared" is used , we see public templates also being
included in the list. This commit reverts listTemplates behavior to 4.3
old logic without using consistent interpretation of list parameters
adopted in new IAM model.
invalid password is provided.
- AccountManager now works using accountId instead of accountType in
following methods too:
- isResourceDomainAdmin()
- isAdmin()
OvsVpcPhysicalTopologyConfigCommand and OvsVpcRoutingPolicyConfigCommand
fix ensures only latest updates are applied (new openflow rules) to the
bidge enabled for distributed routing.
fix mismatch of ovs-host-setup, ovs_host_setup used Libvirt resource and
scripts
plug the nic to OVS bridges created for the tunnel network.
Conflicts:
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/OvsVifDriver.java
In case some environments has different performance or we found some commands
would took too long to execute, one global configuration item is introduced to
specify "time out in seconds per one command in aggregation commands".
By default it's 3 seconds. If admin feel it's too long, it can be adjust to as
low as 1 seconds, which runs still well in my machine.
Conflicts:
setup/db/db/schema-430to440.sql
Added a new flag 'checkBeforeCleanup' to StopCommand based on which check is done to see if VM is running in HV host.
If VM is running then in this case it is not stopped and the operation bails out.
Also modified the MS code to call the StopCommand with appropriate value for the flag based on the context.
Currently it is only set to 'true' when called from the new vmsync logic based on powerstate of VM. For rest it
is set to 'false' meaning no change in behaviour.
This reduces the amount of time and storage it takes dramatically. We no longer
do a full copy, but a sparse copy. The destination image is still in RAW
format, but we only copy over used blocks.
Qemu is also better in doing this then us doing it in Java code.
Otherwise a RBDException will be thrown with the message that the snapshot
isn't protected.
modified: plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java
clusers
across the poll members an internal network created is visible to al the
members but bridge is not necessariliy created. This fix enables
plugging a temp VIF connected to internal network to dom0 and then
unplug-it. this action creates a bridge on the host of the network.
This saves the step of writing to a temporary image in /tmp first before
writing to RBD.
This is possible due to a new version in librbd. With the rbd_default_format
setting we can now force qemu-img to create format 2 RBD images.
This is available since Ceph version 0.67.5 (Dumpling).
introduce 'RegionLevelVpc' as capability of 'Connectivity' service. Add
support for CreateVPCOffering to take the 'regionlevelvpc' as capability
of service 'connectivity'.
introduces new capability 'StretchedL2Subnet' for 'Connectivity'
service. Also add support to createNetworkOffering api to allow
StretchedL2Subnet capablity for the connectivity service.
adds check to ensure 'Connectivity' service provider supports
'StretchedL2Subnet' and 'RegionLevelVpc' capabilities when specified in
createNetworkOffering and createVpcOffering respectivley
enable ovs plug-in to support both StretchedL2Subnet and RegionLevelVpc
capabilities
make zone id optional parameter in createVpc, zone id can be null only
if vpc offfering supports region level VPC
in region level vpc, let the network/tier to be created in any zone of
the region
keep zoneid as required param for createVpc
skip external guest network guru if 'Connectivy' service is present in
network offering
fix build break in contrail manager
permit VM's to be created in different zone that in which network is
created if the network support streched L2 subnet
add integration tests for region level VPC
rebase to master
Conflicts:
setup/db/db/schema-430to440.sql
Conflicts:
api/src/org/apache/cloudstack/api/ApiConstants.java
engine/schema/src/com/cloud/network/vpc/VpcVO.java
setup/db/db/schema-430to440.sql
Add executeInVR() with timeout interface to VirtualRouterDeployer
AggregationControlCommand with Action.Finish may take longer than normal command
since it would execute all the commands in one execution, and it may result in
SSH timeout for SshHelper or other mechanism communicate with VR.
Introduce an new executeInVR() interface with added timeout period for waiting
FinishAggregationCommand to complete execution.
to flow rules and applies them on the bridge
add event subscriber in OvsTunnelManager, that listens to
replaceNetworkAcl events. On event sends the updated policy info to all
the hosts in the VPC
- get the hosts on which VPC spans given vpc id
- get the VM's in the VPC
- get the hosts on which a network spans
- get the VPC's to which a hosts is part of
- get VM's of a VPC on a hosts
introduces capability to build a physical toplogy representation of a
VPC. This json file is encapsulated in
OvsVpcPhysicalTopologyConfigCommand, and is used to send full topology
to hypervisor hosts. On hypervisor this json config can be used to setup
tunnels, configure bridge, add flow rules etc
Ovs GURU, to use different broasdcast scheme VS://vpcid.gerkey for the
networks in VPC that use distributed routing
each VIF and tunnel interface to carry the network UUID in other/options
config
storage pool (SMB) and attached to a running vm can be live migrated to another shared storage
pool. Also a vm and its volumes can be live migrated to another host and storage pool respectively.
2) Corrected some logging in MidoNetPublicNetworkGuru - removed .toString method call on the objects in the log body as toString is called on the object by default when use log4j
CLOUDSTACK-4762 : Enabling VGPU support for XenServer.
This feature is to enable the GPU-passthrough and vGPU functionality,
with the help of this feature, admins/users will be able to leverage
the GPU graphics unit power by deploying a virtul machine with GPU or
vGPU support or by changing the service offering of an existing VM
at any later point of time. There GPU/vGPU enabled VMs are able to run
graphical applications.
For now, this feature is only supported with XenServer hypervisor but
can be extended to add the support of other hypervisors.
Moving default transport for console proxy, SSVM to http.
See
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Realhost+IP+changes
for more info.
jlk ported Amogh's patch for 4.3 to master - code base is different
enough that patch has multiple issues.
Author: Amogh Vasekar <Amogh Vasekar <amogh.vasekar@citrix.com>
Signed-off-by: John Kinsella <jlk@stratosec.co> 1394398017 -0700
In vCenter 5.5, once a volume is migrated the VMDKs are renamed to match the name of the VM.
Update volume path for every volume belonging to the VM to the corresponding new disk filename.
Conflicts:
plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java
plugins/hypervisors/vmware/src/org/apache/cloudstack/storage/motion/VmwareStorageMotionStrategy.java
In vCenter 5.5, once a volume is migrated the VMDKs are renamed to match the name of the VM.
If a volume has been renamed upon migration update its volumePath to that of the new disk filename.
Conflicts:
plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java
vmware-base/src/com/cloud/hypervisor/vmware/mo/VirtualMachineMO.java
With VirtIO enabled on KVM. FreeBSD 10 supports VirtIO for both the
network and the disks. This frees us from IDE and E1000 which should
also improve performance.
By default all network disks are in RAW format. Gluster works fine with
QCOW2 which has some advantages.
Disks are by default in QCOW2 format. It is possible to run into
a mismatch, where the disk is in QCOW2 format, but QEMU gets started
with format=raw. This causes the virtual machines to lockup on boot.
Failures to start a virtual machine can be verified by checking the log
of the virtual machine, and compare the output of 'qemu-img info'.
In /var/log/libvirt/qemu/<VM>.log find the URL for the drive:
-drive file=gluster+tcp://...,format=raw,..
Compare this with the 'qemu-img info' output of the same file, mounted
under /mnt/<pool-uuid>/<img-uuid>:
# qemu-img info /mnt/<pool-uuid>/<img-uuid>
...
file format: qcow2
...
This change makes passes the format when creating a disk located on RBD
(RAW only) and Gluster (QCOW2).
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The support for Gluster as Primary Storage is mostly based on the
implementation for NFS. Like NFS, libvirt can address a Gluster environment
through the 'netfs' pool-type.
return null. Changed to retrieve the first entry in the map.
Removed the ExecutionException try catch, this would prevent the
unittest from giving accurate information on exceptions. Avoid catching
checked exceptions in a unittest, use expected keyword on @Test instead.
PrepareForMigrationCommand, so that destination hypervisor can
mount pool. This further exposed an issue for KVM where iso
was not getting cleaned up upon successful migration, fixed as well.
When project/domains are created, our plugin code won't get invoked. Our
contrail plugin uses Event infrastructure provided by cloudstack to
receive these events and handle them accordingly. It is must to create
domains/projects before creating a virtual network/vm object in contrail
implementation. Hence our plugin must need a way to get notified about
project/domain events.
By default only the Integers between -128..127 are cached (unless overridden by java.lang.Integer.IntegerCache.high system property)
If the inbound or outbound values are higher, the reference comparison won't work.
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
- minor resource leak cleaned up
- cpu-speed reading method extracted
- test added
- logging added in case of exception
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
Replaced HttpClient#execute(HttpUriRequest) with
HttpClient#execute(HttpUriRequest,ResponseHandler<T>).
The former requires extra EntityUtils#consume(HttpEntity).
Rohit Yadav
Wei Zhou
Rajani Karuturi
Mike Tutkowski
Koushik Das
Rafael Weingartner
Rajani Karuturi
Likitha Shetty
Miguel Ferreira
Remi Bergsma
Boris Schrijver
radu-stefanache
manuiiit
Daan Hoogland
Aaron Brady
Daan Hoogland
Sten Spans
Kshitij Kansal
Wido den Hollander
Funs
Wido den Hollander
Jayapal
wilderrodrigues
Kishan Kavala
Sudhansu
jeff
Rafael da Fonseca
Sanjay Tripathi
Sateesh Chodapuneedi
Devdeep Singh
Laszlo Hornyak
Anshul Gangwar
Daan Hoogland
Santhosh Edukulla
KC Wang
Frank Zhang
Antonio Fornie
Hugo Trippaers
Pierre-Yves Ritschard
punith-cloudbyte
Marcus Sorensen
Damodar
Rajesh Battala
Anthony Xu
Saksham Srivastava
Logan Barfield
Wei Zhou
Harikrishna Patnala
Bharat Kumar
Edison Su
amoghvk
punith-cloudbyte
Nitin Mehta
Murali Reddy
David Nalley
imduffy15
Fred Clift
Vania Xu
Kishan Kavala
Daniel Vega
Min Chen
Ritu Sabharwal
Sheng Yang
Will Stevens
Suresh Ramamurthy
Marcus Sorensen
Joris van Lieshout
Kelven Yang
ynojima
Vania Xu
Sujaya Maiyya
Girish chaudhari
Tim Mackey
Sachchidanand Vaidya
Prachi Damle
Chris Suich
Ding Yuan
Kishan Kavala
sbalineni
Abhinandan Prateek
Alex Huang
Santhosh Edukulla
Alena Prokharchyk
John Kinsella
Mandar Barve
Niels de Vos
Miguel Ferreira
Alex Hitchins
Sander Botman
Darren Shepherd
Syed Ahmed
Hiroaki KAWAI