test: Fix Libvirt test so that it works on WindowsThis test failed on Windows, using the File.separator it should run fine on Windows.
* pr/1242:
test: Fix Libvirt test so that it works on Windows
Signed-off-by: Remi Bergsma <github@remi.nl>
* 4.7:
Fix execution counter to support separate counts per thread
Add test to check that each thread has it's own execution counter
CLOUDSTACK-9231: Root volume migration from one primary to another primary storage within the same cluster is failing
This situation arises when there are two management server accessing the same database.
When the migration request comes the command is forwarded from one management server to another because
the host is owned by the second management server. So, serialization of map from one to another fails.
This is fixed by converting the maps to lists.
CLOUDSTACK-9127 Missing PV-bootloader-args for "SUSE Linux Enterprise Server 10 SP2 and SP3"
ISSUE
--------
STOP-START of SUSE Linux VMs fail, as PV-bootloader-args are missing during the start command.
DESCRIPTION
----------------------
Repro steps
1. Upload Suse ISO
2. Create a VM with this ISO, and install it.
3. Detach ISO from the VM.
4. Reboot the VM, :>>>> This will work fine, as the pv-bootloader-args are not missing during reboot.
5.Stop the VM from CCP(VM will get destroyed in Xencenter)
6. Start the same VM from CCP , it will try to start but will fail.
Before Applying the fix
--------------------------------
Before applying the starting the VM failed with following exception
com.cloud.utils.exception.CloudRuntimeException: Unable to start VM(i-2-6-VM) on host(7cfd6388-b763-4c09-b3a3-9679db2904a3) due to Task failed! Task record: uuid: 21a6799f-9523-7c0e-bb86-1de750a38d74
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Wed Dec 09 07:00:29 UTC 2015
finished: Wed Dec 09 07:00:31 UTC 2015
status: failure
residentOn: com.xensource.xenapi.Host@513d238c
progress: 1.0
type: <none/>
result:
errorInfo: [BOOTLOADER_FAILED, OpaqueRef:0b10b6ac-837d-29af-da9d-6ef1e11a064a, Unable to find partition containing kernel
]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
After Applying the fix
--------------------------
After applying the fix I am able to start the vm.
* pr/1196:
CLOUDSTACK-9127 Missing PV-bootloader-args for "SUSE Linux Enterprise Server 10 SP2 and SP3"
Signed-off-by: Daan Hoogland <daan@onecht.net>
* 4.6:
[UI] bug fix: Delete added ACL lists is not available for Domain Admin and normal users
CLOUDSTACK-4787: Allow users to select disk controller for VM/template
CLOUDSTACK-4787 Allow selection of scsi controller type in vSphere
[4.6] CLOUDSTACK-4787 - vmware diskcontrollersSame as #1131 (see this for screenshots etc)
* pr/1132:
CLOUDSTACK-4787: Allow users to select disk controller for VM/template
CLOUDSTACK-4787 Allow selection of scsi controller type in vSphere
Signed-off-by: Daan Hoogland <daan@onecht.net>
CLOUDSTACK-9095 : Hypervisor changes to support UserData for Nuage VSP
* pr/1142:
CLOUDSTACK-9095 : Hypervisor changes to support UserData for Nuage VSP
Signed-off-by: Remi Bergsma <github@remi.nl>
Quota service while allowing for scalability will make sure that the cloud is
not exploited by attacks, careless use and program errors. To address this
problem, we propose to employ a quota-enforcement service that allows resource
usage within certain bounds as defined by policies and available quotas for
various entities. Quota service extends the functionality of usage server to
provide a measurement for the resources used by the accounts and domains using a
common unit referred to as cloud currency in this document. It can be configured
to ensure that your usage won’t exceed the budget allocated to accounts/domain
in cloud currency. It will let user know how much of the cloud resources he is
using. It will help the cloud admins, if they want, to ensure that a user does
not go beyond his allocated quota. Per usage cycle if a account is found to be
exceeding its quota then it is locked. Locking an account means that it will not
be able to initiat e a new resource allocation request, whether it is more
storage or an additional ip. Needless to say quota service as well as any action
on the account is configurable.
Changes from Github code review:
- Added marvin test for quota plugin API
- removed unused commented code
- debug messages in debug enabled check
- checks for nulls, fixed access to member variables and feature
- changes based on PR comments
- unit tests for UsageTypes
- unit tests for all Cmd classes
- unit tests for all service and manager impls
- try-catch-finally or try-with-resource in dao impls for failsafe db switching
- remove dead code
- add missing quota calculation case (regression fixed)
- replace tabs with spaces in pom.xmls
- quota: though default value for quota_calculated is 0, the usage server
makes it null while entering usage entries. Flipping the condition so
as to acocunt for that.
- quotatypes: fix NPE in quota type
- quota framework test fixes
- made statement period configurable
- changed default email templates to reflect the fact that exhausted quota may not result in a locked account
- added quotaUpdateCmd that refreshes quota balances and sends alerts and statements
- report quotaSummary command returns quota balance, quota usage and state for all account
- made UI framework changes to allow for text area input in edit views
- process usage entries that have greater than 0 usage
- orocess quota entries only if tariff is non zero
- if there are credit entries but no balance entry create a dummy balance entry
- remove any credit entries that are before the last balance entry
when displaying balance statement
- on a rerun the last balance is now getting added
FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Quota+Service+-+FS
PR: https://github.com/apache/cloudstack/pull/768
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9025: Fixed can't create usable template from snapshot in Xenserver and Vmwarehttps://issues.apache.org/jira/browse/CLOUDSTACK-9025
Fix also reverts below commit as below solution making assumption about hypervisor which are not applicable in case of XenServer and VmWare
Revert "CLOUDSTACK-8964: Can't create template or volume from snapshot"
This reverts commit ccf5d75cfb.
Testing:
Able to deploy VM successfully from template created from linked clone snapshot on XenServer.
* pr/1176:
CLOUDSTACK-9025: Fixed can't create usable template from snapshot in Xenserver and Vmware
Signed-off-by: Daan Hoogland <daan@onecht.net>
CLOUDSTACK-9047 rename enumsmake enums adhere to best practice naming conventions
* pr/1049:
CLOUDSTACK-9046 rename enums to adhere to naming conventions
CLOUDSTACK-9046 renamed enums in kvm plugin
CLOUDSTACK-9047 use 'State's only with context there are more types called 'State' (or to be called so but now 'state') So remove imports and prepend their enclosing class/context to them.
Signed-off-by: Daan Hoogland <daan@onecht.net>
Removed the PlannerBase class because it is does not bring contribution to ACS' code.Removed the PlannerBase class because it is does not bring contribution to ACS' code.
We changed com.cloud.deploy.FirstFitPlanner, now it doesnt extends the PlannerBase and implements the com.cloud.deploy.DeploymentPlanner.
We also removed the method com.cloud.deploy.DeploymentPlanner.check(VirtualMachineProfile, DeploymentPlan, DeployDestination, ExcludeList) that was not used anywhere.
Additionally, we removed the _ from FirstFitPlanner's attributes name, in order to have them in a more standard way.
* pr/1108:
Removed PlannerBase empty class
Signed-off-by: Remi Bergsma <github@remi.nl>
Fix also reverts below commit as below solution making assumption about hypervisor which are not applicable
in case of XenServer and VmWare
Revert "CLOUDSTACK-8964: Can't create template or volume from snapshot"
This reverts commit ccf5d75cfb.
Add function to set vapic, spinlock and retries
Add function to get retry value
Modify toString to output appropriate XML for spinlock value if set
CLOUDSTACK-9004: Add features to HyperVEnlightenmentFeatureDef
Refactored set methods to get rid of code duplication.
Modified unit tests accordingly
Create test cases to getPatchFilePath method and class names changed In this commit we created tests cases for the respective classes in package com.cloud.hypervisor.xenserver.resource.
We added test cases to check the implementation of com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getPatchFiles. Therefore, we test in a more comprehensive way the tests that already exist to check the code of com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getPatchFilePath.
We added a new abstract class, called com.cloud.hypervisor.xenserver.resource.CitrixResourceBaseTest.java
This class has two tests methods:
* com.cloud.hypervisor.xenserver.resource.CitrixResourceBaseTest.testGetPathFilesExeption(CitrixResourceBase), this method tests if the getPatchFilePath() method throws the com.cloud.utils.exception.CloudRuntimeException.CloudRuntimeException when the com.cloud.utils.script.Script.findScript(String, String) return a null value;
* com.cloud.hypervisor.xenserver.resource.CitrixResourceBaseTest.testGetPathFilesListReturned(CitrixResourceBase), this method tests the correct return value from getPatchFilePath() method, basically, verify if the returned list contain the file with the same absolute path that was retrieved from the findScript method.
We also changed the name of those test classes, the change was basically remove the Path word from the name of classes.
* pr/944:
created tests cases for method "citrixResourceBase.getPatchFiles"
Signed-off-by: Remi Bergsma <github@remi.nl>
The PlannerBase is an empty class, and does not bring contribution to
ACS code.
We changed com.cloud.deploy.FirstFitPlanner to extend the AdapterBase
and implements the com.cloud.deploy.DeploymentPlanner.
We also removed the method
com.cloud.deploy.DeploymentPlanner.check(VirtualMachineProfile,
DeploymentPlan, DeployDestination, ExcludeList) that was not used
anywhere.
Additionally, we removed the “_” from FirstFitPlanner's attributes name,
in order to have them in a more standard way.
* 4.6:
Use version for RC branch name instead of branch
make sure all files are updates with new version
Update L10N resource files with 4.6 strings from Transifex (20151129)
Fix secondary storage not working with swift
CLOUDSTACK-9083: Add disk serial to kvm virt xml
CLOUDSTACK-9081 Migration of vm across clusters fails in clustered MS setupStorage motion of vm across clusters/xenserver-pools fails in a
clustered management server setup. In xen storage motion we have to send
a migrate_receive command to the destination host, followed by
migrate_send command to the source host. The sr and network detials of
the destination host have to be passed in migrate_send command on the
source. While migrating across clusters the source and destination
resources are seperate objects. To pass this information across
resources we have to send seperate migrate with storage receive and send
commands to the resource. In a clustered ms setup these commands may
have to be forwarded to another ms as the resource may be owned by it.
The serilization of the command and answer objects fails in such case as
it doesn't understand the xapi sr and network objects. Made a change to
serialize these objects in the resource layer and pass the around as
strings in the command and answer objects.
Reviewed-By: @likitha
Author: @devdeep
Tested-By: @sailajamada
* pr/1118:
CLOUDSTACK-9081 Migration of vm across clusters fails in clustered MS setup
Signed-off-by: Remi Bergsma <github@remi.nl>
we also changed the name of those test classes, the change was basically
remove the “Path” word from the name of classes.
change the private method
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getPatchFiles
to protected
added new class in test cases
The main changes are:
• Changing methods “Void” to “void”.
• Removal of the method “Void copyAsync(DataObject srcData, DataObject
destData, AsyncCompletionCallback callback)” that was never used. We
noticed that methods form that class are using the return type Void with
capital V. This way that method has to return a null value at the end.
commit #7
So far only 1 controller (scsi or ide) is supported in Cloudstack for ide or
scsi, this is existing limitation. Added support for 2nd IDE controller. Support adding IDE
virtual disk to VM. Also added check if VM is running as IDE virtual disk cannot be attached
to VM if VM is runnning.If user detaches a virtual disk on lower unit number of controller,
then subsequent attach operation should find free unit number on the controller and attach
the virtual disk there.
commit #6
Let the controllers of existing VMs continue without flip, current busInfo retrieved from
chain_info field of volume record from database would be preferred over
controller settings from all configuration settings.
commit #5
Editing global configuration param vmware.root.disk.controller osdefault value results
in loss of previous root disk controller type. Hence root disk's controller type for legacy
VMs is unknow post that modificaiton by user. If VM is stop/start then we could get this
infromation from bus info of existing volume. But if user resets VM and then try to start VM.
The existing bus info would be lost. Hence existing disk info is not available to depend on.
Using lsilogic or generic scsi controller for ROOT disk of legacy VMs if reset.
commit #4
Avoid adding additional (>1) scsi controllers to system vms. While attaching volume to legacy VM
don't use osdefault optoin which applicable only for VM created with the option enabled, use
legacy data disk controller type (lsilogic)
commit #3
If root disk's controller type is scsi and data disk controller type condenses
to any of scsi sub-types then data disk controller type would fall back to root disk controller itself. This
ensures data volumes would be accessible in all cases as controller of root volume would be reliable
and it means VM has the supported controller. It also avoids mix of scsi controller sub-types in a user instance.
Also translating disk controller type scsi to lsilogic.
commit #2
Support auto detection of recommended virtual disk controller type for specific guest OS.
commit #1
Support granual controller types. Add support for controller types in template registration as well.
Fix white spaces.
Removed stale HEAD merge lines
Removed tail of merge lines
Fixed VmwareResource, removing storage commands that moved to VmwareStorageProcessor.
removed stale code of controller that is present in processor
Fixed check style errors.
Fixed injection.
Tested with Linux and windows templates. Unable to run iso based tests due to few bugs in register iso area.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
(cherry picked from commit a4cc987a6f)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Storage motion of vm across clusters/xenserver-pools fails in a
clustered management server setup. In xen storage motion we have to send
a migrate_receive command to the destination host, followed by
migrate_send command to the source host. The sr and network detials of
the destination host have to be passed in migrate_send command on the
source. While migrating across clusters the source and destination
resources are seperate objects. To pass this information across
resources we have to send seperate migrate with storage receive and send
commands to the resource. In a clustered ms setup these commands may
have to be forwarded to another ms as the resource may be owned by it.
The serilization of the command and answer objects fails in such case as
it doesn't understand the xapi sr and network objects. Made a change to
serialize these objects in the resource layer and pass the around as
strings in the command and answer objects. Reviewed-By: Likitha
Adds disk serial ids based on volume uuids to the virt xml. This may be useful
for appliances/software that needs some serial ids on the VM disks. This does not
impact existing/running VMs, the vm virt xmls will be updates for running VMs
the next time they are stopped/started.
For testing, disk serial (of debian based systemvm) in the virt xml matched that
in /sys/devices/pci0000:00:0000:00:07.0/virtio4/block/vda/serial.
We currently don't support scsi-blcok devices for which serial is not supported,
for this we've added a DeviceType (LUN) which may be used in future and a check
to not add the serial to the xml if disk type is LUN.
Refer: https://libvirt.org/formatdomain.html#elementsDisks
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9062: Improve S3 implementation.The S3 implementation is far from finished, this commit focuses on the bases.
- Upgrade AWS SDK to latest version.
- Rewrite S3 Template downloader.
- Rewrite S3Utils utility class.
- Improve addImageStoreS3 API command.
- Split various classes for convenience.
- Various minor improvements and code optimizations.
A side effect of the new AWS SDK is that it, by default, uses the V4 signature. Therefore I added an option to specify the Signer, so it stays compatible with previous versions.
Please review thoroughly, both code inspection and (automated) integration tests. Currently no integration tests are available specifically for S3. Therefore the implementation is needed to be tested manually, for now...
What I tested:
- Greenfield install -> will download latest systemvm template automatically to S3.
- Upload a template/iso
- Download a template/iso
- Restart of management server -> list available templates -> doesn't download them again if available.
* pr/1083:
CLOUDSTACK-9062: Improve S3 implementation.
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9067 - As I developer I want to remove all the unused router-shell scripts from ACSThis PR removes the unused shell scripts that were present in the ACS project. Those script were replaced by the.
Some of the scripts are used by the HyperV Resource, which were hardcoded. I took the opportunity to use the Java constants over there as well, so the next one touching the code will know they exist and won't hardcode anything.
The following task were applied:
* Remove the shell files and the Java constants that were mapping them;
* Apply the use of the Java constants to the HyperV Resource class;
* Wrap the String.format() method in the StringUtils so we can test the changes in the HyperV Resource class.
The last point was added because I do not have a HyperV test environment. Hence, I wanted to make sure the tiny code I changed is covered at least by unit tests.
* pr/1084:
CLOUDSTACK-9067 - Replaces hardcoded paths with the VRScripts constants.
CLOUDSTACK-9067 - Fomatting the code of HypervDirectConnectResource class
CLOUDSTACK-9067 - Remove old script file from the project
Signed-off-by: Remi Bergsma <github@remi.nl>
The S3 implementation is far from finished, this commit focusses on the bases.
- Upgrade AWS SDK to latest version.
- Rewrite S3 Template downloader.
- Rewrite S3Utils utility class.
- Improve addImageStoreS3 API command.
- Split various classes for convenience.
- Various minor improvements and code optimalisations.
A side effect of the new AWS SDK is that it, by default, uses the V4 signature. Therefore I added an option to specify the Signer, so it stays compatible with previous versions.
Add Unit Tests for Libvirt/KVM storage codeThese classes were not covered by Unit Tests and this commit
adds some tests for their basic functionality.
* pr/986:
Add Unit Tests for Libvirt/KVM storage code
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8964: Ovm3HypervisorGuru answer fix and snapshot to volume implementation.some clean up of code after the fix from #975
* pr/1015:
CLOUDSTACK-8964 side effect isolation extract side effect away to emphasize the main commandDelegation objective
host delegation logging for XenServerGuru.java
Getting volume from snapshot working again... odd
simple change to prevent failure and keep OVM3 snapshots working
This closes#975
Signed-off-by: Remi Bergsma <github@remi.nl>
These were lacking, but this helper is used in various places
inside the KVM code.
Some simple tests to verify the helper is doing what we expect it
to do.
CLOUDSTACK-8826: XenServer - Use device id passed as part of attach volume API properly
If device id passed as part of API and available then use it otherwise fallback on XS to automatically assign one.
For ISO device id used is 3 and it is processed before any other entry to avoid conflict.
Signed-off-by: Koushik Das <koushik@apache.org>
If device id passed as part of API and available then use it otherwise fallback on XS to automatically assign one.
For ISO device id used is 3 and it is processed before any other entry to avoid conflict.
Replaced all occurences of Charset.forName(UTF-8) with StringUtils.getPreferredCharset().
* pr/825:
Replaced all occurences of Charset.forName(UTF-8) with StringUtils.getPreferredCharset().
Signed-off-by: Daan Hoogland <daan@onecht.net>
sysctl: don't modify /etc/sysctl.confTo configure firewall rules, CloudStack modifies `/etc/sysctl.conf` and
execute those modifications. This may be harmful for several reasons:
1. `/etc/sysctl.conf` may be managed by some configuration management
system. Such a system will constantly restore the previous version.
2. `/etc/sysctl.conf` may contain additional properties that have been
changed later by some system administrator (for example, once a
firewall has been configured, forwarding may have been activated
while it is disabled in `/etc/sysctl.conf`). Executing the file
again at a later time may disrupt the system.
3. Entries are added again and again. `/etc/sysctl.conf` will contain
the same directives repeated several times.
Using a configuration file is not needed as `sysctl` is able to directly
modify sysctl values with `-w` flag.
Signed-off-by: Vincent Bernat <Vincent.Bernat@exoscale.ch>
* pr/776:
sysctl: don't modify /etc/sysctl.conf
Signed-off-by: Wido den Hollander <wido@widodh.nl>
To configure firewall rules, CloudStack modifies `/etc/sysctl.conf` and
execute those modifications. This may be harmful for several reasons:
1. `/etc/sysctl.conf` may be managed by some configuration management
system. Such a system will constantly restore the previous version.
2. `/etc/sysctl.conf` may contain additional properties that have been
changed later by some system administrator (for example, once a
firewall has been configured, forwarding may have been activated
while it is disabled in `/etc/sysctl.conf`). Executing the file
again at a later time may disrupt the system.
3. Entries are added again and again. `/etc/sysctl.conf` will contain
the same directives repeated several times.
Using a configuration file is not needed as `sysctl` is able to directly
modify sysctl values with `-w` flag.
Signed-off-by: Vincent Bernat <Vincent.Bernat@exoscale.ch>
Use host.reserved.ram.mb agent property to modify total system RAM
before reporting to management server.
Remove dom0ram variable and its min/max calculation. Just reserve a
default of 1GB, unless overridden by host.reserved.mem.mb property.
CLOUDSTACK-8762: Check to confirm disk activity before starting a VMImplements a VM volume/disk file activity checker that checks if QCOW2 file
has been changed before starting the VM. This is useful as a pessimistic
approach to save VMs that were running on faulty hosts that CloudStack could
try to launch on other hosts while the host was not cleanly fenced. This is
optional and available only if you enable the settings in agent.properties
file, on per-host basis.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* pr/753:
CLOUDSTACK-8762: Check to confirm disk activity before starting a VM
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Guys, can you review it? things need to be discussed:
(1) this supports KVM/QCOW2 only. Anyone want to implement for other Hypervisor/format ?
(2) The original data volume (on primary storage) will be removed.
(3) The script uses the default timeout in libvirtComputingResource. Do we need to add one in global configuration (like copy.volume.wait or backup.snapshot.wait, create.volume.from.snapshot.wait)
(4) In scripts/storage/qcow2/managesnapshot.sh, I use "qemu-img convert -f qcow2 -O qcow2" to copy the snapshot from secondary to primary (hence there is no base image file), instead of "cp -f", this is because convert is faster than cp in my testing.
* pr/732:
CLOUDSTACK-5863: revert volume snapshot for KVM/QCOW2
Signed-off-by: Wei Zhou <w.zhou@tech.leaseweb.com>
Made following fixes in simulator
- Support for ScaleVmCommand/NetworkRulesVmSecondaryIpCommand in resource layer
- Added support for scaling up a running VM in simulator
- Fixed some method names not following convention
In order to test PR #725 using simulator some of these changes are needed.
Based on the way HV check is present in the scale VM API, had to explicitly put simulator related check to allow support. The ideal way would be to remove all these HV specific check from code and made them some configuration (by putting them in hypervisor_capabilities table in DB). But that will be a bigger effort outside the scope of this PR.
Signed-off-by: Koushik Das <koushik@apache.org>
This reverts commit cd7218e241, reversing
changes made to f5a7395cc2.
Reason for Revert:
noredist build failed with the below error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project cloud-plugin-hypervisor-vmware: Compilation failure
[ERROR] /home/jenkins/acs/workspace/build-master-noredist/plugins/hypervisors/vmware/src/com/cloud/hypervisor/guru/VMwareGuru.java:[484,12] error: non-static variable logger cannot be referenced from a static context
[ERROR] -> [Help 1]
even the normal build is broken as reported by @koushik-das on dev list
http://markmail.org/message/nngimssuzkj5gpbz
Implements a VM volume/disk file activity checker that checks if QCOW2 file
has been changed before starting the VM. This is useful as a pessimistic
approach to save VMs that were running on faulty hosts that CloudStack could
try to launch on other hosts while the host was not cleanly fenced. This is
optional and available only if you enable the settings in agent.properties
file, on per-host basis.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
messages: Removed unused agent command AttachVolumeCommand and corresponding answer AttachVolumeAnswer
These types are not used anywhere in the orchestration layer.
Signed-off-by: Koushik Das <koushik@apache.org>
Default to notify only script to handle non-CLVM/QCOW cases.This relates to [CLOUDSTACK-8732](https://issues.apache.org/jira/browse/CLOUDSTACK-8732)
Before this commit the call to `getResizeScriptType` would throw an exception (earlier versions returned `null`, which was fine) - this caused the RBD case to fail. By changing the default to notify only we fix the case for any non-CLVM and non-QCOW cases, too.
This is RBD for now, but this should extend to new storage types supported by Libvirt natively in future.
This is my first attempted contribution: I can see a case for adding RBD logic to the actual getResizeScriptType call, too, but I felt that putting it `LibvirtResizeVolumeCommandWrapper.java` kept the special-casing of RBD (and comments about that) in one place.
### Caveat:
With Libvirt 1.2.2 this actually doesn't do the right thing - but it does do what the documentation *says* should be the right thing, so I'm going to test if this is a Libvirt bug which is fixed in a later version.
(To make it work I need to execute something like:
virsh blockresize --path vda --size 100G i-7-44-VM
where vda is the path as far as the *guest* is concerned, and not an `rbd/` path - which *should* work, but doesn't.)
* pr/696:
Default to notify only script to handle non-CLVM/QCOW cases.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Support for ScaleVmCommand/NetworkRulesVmSecondaryIpCommand in resource layer
- Added support for scaling up a running VM in simulator
- Fixed some method names not following convention
CLOUDSTACK-8443: Support CentOS 7 as KVM hypervisorThis adds support for CentOS 7 as a KVM hypervisor. As discussed in the Jira issue, the root cause of the issue was the `cloudstack-setup-agent` script that added this line to `/etc/libvirt/qemu.conf`:
`cgroup_controllers=["cpu"]`
On CentOS 6 this 'works', but since by default no cgroups are mounted, this setting is ignored by `libvirtd`. On CentOS 7 is does not work, because the `cpu` and `cpuacct` are 'co-mounted'. This simpy means you cannot use one without the order. The config line above instructs `libvirtd` to only use the `cpu` cgroup, which it can't.
There was code added to make sure the `cpu` cgroup was not co-mounted. It did this with a shell script, but after a reboot it was gone. It wasn't reliable and not even a solution at all. I went ahead and removed the code, because the co-mounted cgroups work fine. The real issue was the config line we put in `/etc/libvirt/qemu.conf` so much better to fix that instead.
The default for `libvirtd` is to use only cgroups that are mounted. When you remove the 'cgroup_controllers' line, the default kicks in. According to the `qemu.conf` file, the default is:
`cgroup_controllers = ["cpu", "devices", "memory", "blkio", "cpuacct", "net_cls"]`
Then again, if it's not mounted, it's not used. So, that's a nice way to control it.
This works just fine for both RHEL/CentOS 6 and RHEL/CentOS 7.
Along the debugging, I came across some small issues here and there which I also fixed in this PR.
If you test this, be sure to build RPMs from this branch and install those on the KVM hypervisor.
At Schuberg Philis we run KVM hypervisors on CentOS 7 for months with these fixes. Now I took the time to fix the issues in CloudStack, rather than work around them.
@bhaisaab @karuturi could you please review this? Thanks!
Please note:
The next step is to fix CLOUDSTACK-8625 (Systemd profile for KVM Agent) as currently a sysvinit script is still used. Although it works, we do not have any systemd benefits. As it is a separate issue, I'll try to address it in a separate PR.
* pr/731:
CLOUDSTACK-8443: detect CentOS 7.x as RHEL 7 alike
CLOUDSTACK-8443: mention the correct logfile
CLOUDSTACK-8443: display the right hypervisor type
CLOUDSTACK-8443: don't try to fix co-mounted cgroups
Signed-off-by: Remi Bergsma <github@remi.nl>
This setting works on CentOS 6 / RHEL 6 but does nothing, as
"cpu" cgroup is not mounted. On CentOS 7 / RHEL 7 systemd does
mount cgroups and "cpu" is co-mounted with "cpuacc". Hence, if
we specify "cpu" then this results in an error because it can
only use them both, or none.
By removing the setting, we rely on the default of qemu, which
is:
cgroup_controllers = ["cpu", "devices", "memory", "blkio", "cpuacct", "net_cls"]
Only if they are really mounted, they will be used. So, this will
work on both version 6 and 7.
The 'fix script' didn't work well, as after a reboot you'd still have qemu
throwing errors. Now we can handle the co-mountedcgroups.
Removed duplicate code in CitrixResourceBase and its subclassesRemoved unnecessary duplicated code by having the body of the getPatchFiles method only in the CitrixResourceBase superclass. Given that all of its implementations consisted of the same code except for the path which contains the necessary file for that implementation. An abstract method getPatchFilePath was created in the CitrixResourceBase superclass so that each implementation may return the path containing the specific file needed by that implementation.
Test cases were created for each implementation, simple as they may be. One assert is made to verify that the path returned by each implementation corresponds to the path that was previously specified on each getPatchFiles implementation.
* pr/700:
Removed duplicate code in CitrixResourceBase.getPatchFiles
Signed-off-by: Remi Bergsma <github@remi.nl>
This tries to avoid cleaning by a device name.
This closes#718
(cherry picked from commit 74f697a2dd)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Changed methodnames according to Nic.java refactor.
Fixed NicVO.java due to regression from Nic.java refactor.
Fixed VmWareGuru.java after Nic.java refactor.
See issue CLOUDSTACK-8736 for ongoing effort to clean up network code.
@cristofolini comments:
Removed unnecessary duplicated code by having the body of the
getPatchFiles method only in the
CitrixResourceBase superclass. Given that all of its implementations
consisted of the same code
except for the path which contains the necessary file for that
implementation. An abstract method
getPatchFilePath was created in the CitrixResourceBase superclass so
that each implementation may
return the path containing the specific file needed by that
implementation.
Test cases were created for each implementation, simple as they may be.
One assert is made to verify
that the path returned by each implementation corresponds to the path
that was previously specified
on each getPatchFiles implementation.
removed trailing whitespace
Cloudstack 8656: do away with more silently ignoring exceptions.a lot of messages added.
some restructuring for test exception assertions and try-with-resource blocks
* pr/654: (29 commits)
CLOUDSTACK-8656: more logging instead of sysout
CLOUDSTACK-8656: use catch block for validation
CLOUDSTACK-8656: class in json specified not found
CLOUDSTACK-8656: removed unused classes
CLOUDSTACK-8656: restructure of tests
CLOUDSTACK-8656: reorganise sychronized block
CLOUDSTACK-8656: restructure tests to ensure exception throwing
CLOUDSTACK-8656: validate the throwing of ServerApiException
CLOUDSTACK-8656: logging ignored exceptions
CLOUDSTACK-8656: try-w-r removes need for empty catch block
CLOUDSTACK-8656: try-w-r instead of clunckey close-except
CLOUDSTACK-8656: deal with empty SQLException catch block by try-w-r
CLOUDSTACK-8656: unnecessary close construct removed
CLOUDSTACK-8656: message about timed buffer logging
CLOUDSTACK-8656: message about invalid number from store
CLOUDSTACK-8656: move cli test tool to separate file
CLOUDSTACK-8656: exception is the rule for some tests
CLOUDSTACK-8656: network related exception logging
CLOUDSTACK-8656: reporting ignored exceptions in server
CLOUDSTACK-8656: log in case we are on a platform not supporting UTF8
...
Signed-off-by: Remi Bergsma <github@remi.nl>
Before this commit the call to `getResizeScriptType` would throw
an exception (earlier versions returned `null`, which was fine) -
this caused the RBD case to fail. By changing the default to
notify only we fix the case for any non-CLVM and non-QCOW cases.
This is RBD for now, but this should extend to new storage
types suppored by Libvirt natively in future.
When dumping XML use appropriate flags:
1, VIR_DOMAIN_XML_SECURE (dump security sensitive information too)
8, VIR_DOMAIN_XML_MIGRATABLE (dump XML suitable for migration)
Source:
https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainXMLFlags
This fixes CVE 2015-3252: VNC password lost during VM migration across KVM
hosts. The issue is also seen when a VM is rebooted.
(cherry picked from commit cb2aca7516)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* pr/547:
CLOUDSTACK-8601. VMFS storage added as local storage can be re-added as shared storage. Fail addition of a VMFS shared storage pool in case it has already been added as local storage in CS.
Signed-off-by: Mike Tutkowski <mike.tutkowski@solidfire.com>
On NFS we write a heartbeat, but without those we can not safely
fence off a host.
If we fence without knowing about a heartbeat we can cause a split-brain situation.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
- Make sure the new password replaces the old one in the queue
- Updated the patch files for XenServer
- Updated the script path on LibvirtComputing class
- Adding update_host_passwd to VRScripts
- Add implementation to CitrixUpdateHostPasswordCommandWrapper
- Improve testUpdateHostPasswordCommand() unit test on CitrixRequestWrapperTest
- Adding update_host_passwd.sh script
- Adding the host IP address as an instance variable on UpdateHostPasswordCommand
- Improving the Unit Test (LibvirtComputingResourceTest) to get it covering the new code
- Make sure doUpdateHostPassword() doesn't get called if flag is set to false
- Do not update XenServer hosts if the cluster ID is not informed
This closes#559
- Adding more unit tests as suggested by Daan and Rajani.
- Mocking SshHelper and the Script classes, used by Citrix and Libvirt resources respectively.
- Changed location of the update_host_passwd script
- Updated the patch files for XenServer
- Updated the script path on LibvirtComputing class
- Removed the hostIP from the LibvirtUpdateHostPasswordCommandWrapper execute() method
- Adding update_host_passwd to VRScripts
- Add accessor method to host password on CitrixResourceBase
- Add implementation to CitrixUpdateHostPasswordCommandWrapper
- Improve testUpdateHostPasswordCommand() unit test on CitrixRequestWrapperTest
- Add line to patch files on xenserver directory
Concerning the LibVirt change:
- I forgot to assing the return of the getDefaultHypervisorScriptsDir() method to the hypervisorScriptsDir variable
- Modifying the LibvirtUpdateHostPasswordCommandWrapper in order to execute the script on the host
- Adding the script path to LibvirtComputingResource
- Adding the host IP address as an instance variable on UpdateHostPasswordCommand
- Improving the Unit Test (LibvirtComputingResourceTest) to get it covering the new code
- Make sure the doUpdateHostPassword() method returns the result from the answer, not always true
- Added the LibvirtUpdateHostPasswordCommandWrapper class, which will handle the changes in the KVM agent
That's will be further implemented in the future. The command will avoid the agent to complain about Unsupported command
- Added a test to make sure the current implementation of the comment works
- If changes in the future, it will also require changes. In that way, we make sure nobody will break it
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
This closes#527
CLOUDSTACK-3317 - DVS does not support management\storage network
Added support for Management and Storage Network traffic over VMware DVS in CloudStack deployments. Also added support for storage VLAN over dvPortGroup.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
CLOUDSTACK-3317 - DVS does not support management\storage network
Use non-zero dvport count while updating dvportgroups of system traffic.
Updated configuration compare logic to avoid update dvportgroup operation unless required.
This would help improve speed in vm/network deployment as the update calls would reduce. Also improved logging.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Adding unit tests in class HypervisorHostHelperTest
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Added license header to new file being added to repo/branch.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
Assertion is not used in runtime, correct way is throw and handle exception without killing app
Signed-off-by: Daan Hoogland <daan.hoogland@gmail.com>
This closes#362
VmwareResource.java:315, MS_SHOULD_BE_FINAL, Priority: High
com.cloud.hypervisor.vmware.resource.VmwareResource.s_serviceContext isn't final but should be
VmwareResource.java:331, MS_SHOULD_BE_FINAL, Priority: High
com.cloud.hypervisor.vmware.resource.VmwareResource.s_powerStatesTable isn't final but should be
Signed-off-by: Daan Hoogland <daan.hoogland@gmail.com>
This closes#363
Remi Bergsma
Daan Hoogland
nvazquez
Nitin Kumar Maharana
cirstofolini
Boris Schrijver
SudharmaJain
Wido den Hollander
Koushik Das
Abhinandan Prateek
Wei Zhou
Nick Livens
Rohit Yadav
Anshul Gangwar
Josh Harshman
weingartner
pedro-martins
Sateesh Chodapuneedi
Devdeep Singh
Wilder Rodrigues
Funs Kessen
David Mabry
Satoru Nakaya
Vincent Bernat
Kevin McCormick
Mike Tutkowski
Rajani Karuturi
Likitha Shetty
Miguel Ferreira
manuiiit
Aaron Brady
Daan Hoogland
Funs
Wido den Hollander
Jayapal
Kishan Kavala
Rafael da Fonseca
Sanjay Tripathi
