The S3 implementation is far from finished, this commit focusses on the bases.
- Upgrade AWS SDK to latest version.
- Rewrite S3 Template downloader.
- Rewrite S3Utils utility class.
- Improve addImageStoreS3 API command.
- Split various classes for convenience.
- Various minor improvements and code optimalisations.
A side effect of the new AWS SDK is that it, by default, uses the V4 signature. Therefore I added an option to specify the Signer, so it stays compatible with previous versions.
Add Unit Tests for Libvirt/KVM storage codeThese classes were not covered by Unit Tests and this commit
adds some tests for their basic functionality.
* pr/986:
Add Unit Tests for Libvirt/KVM storage code
Signed-off-by: Remi Bergsma <github@remi.nl>
These were lacking, but this helper is used in various places
inside the KVM code.
Some simple tests to verify the helper is doing what we expect it
to do.
Use host.reserved.ram.mb agent property to modify total system RAM
before reporting to management server.
Remove dom0ram variable and its min/max calculation. Just reserve a
default of 1GB, unless overridden by host.reserved.mem.mb property.
CLOUDSTACK-8762: Check to confirm disk activity before starting a VMImplements a VM volume/disk file activity checker that checks if QCOW2 file
has been changed before starting the VM. This is useful as a pessimistic
approach to save VMs that were running on faulty hosts that CloudStack could
try to launch on other hosts while the host was not cleanly fenced. This is
optional and available only if you enable the settings in agent.properties
file, on per-host basis.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* pr/753:
CLOUDSTACK-8762: Check to confirm disk activity before starting a VM
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Guys, can you review it? things need to be discussed:
(1) this supports KVM/QCOW2 only. Anyone want to implement for other Hypervisor/format ?
(2) The original data volume (on primary storage) will be removed.
(3) The script uses the default timeout in libvirtComputingResource. Do we need to add one in global configuration (like copy.volume.wait or backup.snapshot.wait, create.volume.from.snapshot.wait)
(4) In scripts/storage/qcow2/managesnapshot.sh, I use "qemu-img convert -f qcow2 -O qcow2" to copy the snapshot from secondary to primary (hence there is no base image file), instead of "cp -f", this is because convert is faster than cp in my testing.
* pr/732:
CLOUDSTACK-5863: revert volume snapshot for KVM/QCOW2
Signed-off-by: Wei Zhou <w.zhou@tech.leaseweb.com>
This reverts commit cd7218e241, reversing
changes made to f5a7395cc2.
Reason for Revert:
noredist build failed with the below error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project cloud-plugin-hypervisor-vmware: Compilation failure
[ERROR] /home/jenkins/acs/workspace/build-master-noredist/plugins/hypervisors/vmware/src/com/cloud/hypervisor/guru/VMwareGuru.java:[484,12] error: non-static variable logger cannot be referenced from a static context
[ERROR] -> [Help 1]
even the normal build is broken as reported by @koushik-das on dev list
http://markmail.org/message/nngimssuzkj5gpbz
Implements a VM volume/disk file activity checker that checks if QCOW2 file
has been changed before starting the VM. This is useful as a pessimistic
approach to save VMs that were running on faulty hosts that CloudStack could
try to launch on other hosts while the host was not cleanly fenced. This is
optional and available only if you enable the settings in agent.properties
file, on per-host basis.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
messages: Removed unused agent command AttachVolumeCommand and corresponding answer AttachVolumeAnswer
These types are not used anywhere in the orchestration layer.
Signed-off-by: Koushik Das <koushik@apache.org>
Default to notify only script to handle non-CLVM/QCOW cases.This relates to [CLOUDSTACK-8732](https://issues.apache.org/jira/browse/CLOUDSTACK-8732)
Before this commit the call to `getResizeScriptType` would throw an exception (earlier versions returned `null`, which was fine) - this caused the RBD case to fail. By changing the default to notify only we fix the case for any non-CLVM and non-QCOW cases, too.
This is RBD for now, but this should extend to new storage types supported by Libvirt natively in future.
This is my first attempted contribution: I can see a case for adding RBD logic to the actual getResizeScriptType call, too, but I felt that putting it `LibvirtResizeVolumeCommandWrapper.java` kept the special-casing of RBD (and comments about that) in one place.
### Caveat:
With Libvirt 1.2.2 this actually doesn't do the right thing - but it does do what the documentation *says* should be the right thing, so I'm going to test if this is a Libvirt bug which is fixed in a later version.
(To make it work I need to execute something like:
virsh blockresize --path vda --size 100G i-7-44-VM
where vda is the path as far as the *guest* is concerned, and not an `rbd/` path - which *should* work, but doesn't.)
* pr/696:
Default to notify only script to handle non-CLVM/QCOW cases.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This setting works on CentOS 6 / RHEL 6 but does nothing, as
"cpu" cgroup is not mounted. On CentOS 7 / RHEL 7 systemd does
mount cgroups and "cpu" is co-mounted with "cpuacc". Hence, if
we specify "cpu" then this results in an error because it can
only use them both, or none.
By removing the setting, we rely on the default of qemu, which
is:
cgroup_controllers = ["cpu", "devices", "memory", "blkio", "cpuacct", "net_cls"]
Only if they are really mounted, they will be used. So, this will
work on both version 6 and 7.
The 'fix script' didn't work well, as after a reboot you'd still have qemu
throwing errors. Now we can handle the co-mountedcgroups.
This tries to avoid cleaning by a device name.
This closes#718
(cherry picked from commit 74f697a2dd)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Before this commit the call to `getResizeScriptType` would throw
an exception (earlier versions returned `null`, which was fine) -
this caused the RBD case to fail. By changing the default to
notify only we fix the case for any non-CLVM and non-QCOW cases.
This is RBD for now, but this should extend to new storage
types suppored by Libvirt natively in future.
When dumping XML use appropriate flags:
1, VIR_DOMAIN_XML_SECURE (dump security sensitive information too)
8, VIR_DOMAIN_XML_MIGRATABLE (dump XML suitable for migration)
Source:
https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainXMLFlags
This fixes CVE 2015-3252: VNC password lost during VM migration across KVM
hosts. The issue is also seen when a VM is rebooted.
(cherry picked from commit cb2aca7516)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
On NFS we write a heartbeat, but without those we can not safely
fence off a host.
If we fence without knowing about a heartbeat we can cause a split-brain situation.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
- Make sure the new password replaces the old one in the queue
- Updated the patch files for XenServer
- Updated the script path on LibvirtComputing class
- Adding update_host_passwd to VRScripts
- Add implementation to CitrixUpdateHostPasswordCommandWrapper
- Improve testUpdateHostPasswordCommand() unit test on CitrixRequestWrapperTest
- Adding update_host_passwd.sh script
- Adding the host IP address as an instance variable on UpdateHostPasswordCommand
- Improving the Unit Test (LibvirtComputingResourceTest) to get it covering the new code
- Make sure doUpdateHostPassword() doesn't get called if flag is set to false
- Do not update XenServer hosts if the cluster ID is not informed
This closes#559
- Adding more unit tests as suggested by Daan and Rajani.
- Mocking SshHelper and the Script classes, used by Citrix and Libvirt resources respectively.
- Changed location of the update_host_passwd script
- Updated the patch files for XenServer
- Updated the script path on LibvirtComputing class
- Removed the hostIP from the LibvirtUpdateHostPasswordCommandWrapper execute() method
- Adding update_host_passwd to VRScripts
- Add accessor method to host password on CitrixResourceBase
- Add implementation to CitrixUpdateHostPasswordCommandWrapper
- Improve testUpdateHostPasswordCommand() unit test on CitrixRequestWrapperTest
- Add line to patch files on xenserver directory
Concerning the LibVirt change:
- I forgot to assing the return of the getDefaultHypervisorScriptsDir() method to the hypervisorScriptsDir variable
- Modifying the LibvirtUpdateHostPasswordCommandWrapper in order to execute the script on the host
- Adding the script path to LibvirtComputingResource
- Adding the host IP address as an instance variable on UpdateHostPasswordCommand
- Improving the Unit Test (LibvirtComputingResourceTest) to get it covering the new code
- Make sure the doUpdateHostPassword() method returns the result from the answer, not always true
- Added the LibvirtUpdateHostPasswordCommandWrapper class, which will handle the changes in the KVM agent
That's will be further implemented in the future. The command will avoid the agent to complain about Unsupported command
- Added a test to make sure the current implementation of the comment works
- If changes in the future, it will also require changes. In that way, we make sure nobody will break it
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
This closes#527
KVM hosts which are actuall up, but if their agents are shutdown should be put
in disconnected state. This would avoid getting the VMs HA'd and other commands
such as deploying a VM will exclude that host and save us from errors.
The improvement is that, we first try to contact the KVM host itself. If it fails
we assume that it's disconnected, and then ask its KVM neighbours if they can
check its status. If all of the KVM neighbours tell us that it's Down and we're
unable to reach the KVM host, then the host is possibly down. In case any of the
KVM neighbours tell us that it's Up but we're unable to reach the KVM host then
we can be sure that the agent is offline but the host is running.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
This closes#340
When executing the tests in an environment where Libvirt is also installed, it
caused errors.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#342
Pull average Cpu util report between polling intervals instead of since boot
instead of using values since uptime
(cherry picked from commit 04176eaf17)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
This closes#297
Passing the file argument to the xml break for EL 7.1, the fix removes
the argument as just passing rombar='off' with its file arg to be empty string.
This closes#290
(cherry picked from commit aafa0c80b3)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
EL7 has a different output to 'free', use /proc/meminfo instead of a tool to be
more consistent across distros
(cherry picked from commit 212a05a345)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
Removing real IPs from the tests because they cause a long running time for LibvirtComputingResourceTest
- In a local machine it takes 1.977s, but in a KVM test environment it's taking 257.879 sec
Fixing typo on LibvirtRequestWrapper
- Replace linbvirtCommands by libvirtCommands on LibvirtRequestWrapper
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#255
- The test was okay, but when running in an environment where a /root/.ssh/id_rsa existed, it would return true then fail
- We now mock the calls to methods that return the key paths, instead of relying in the static variables
- Adding LibvirtNetworkElementCommandWrapper and LibvirtStorageSubSystemCommandWrapper
- 2 unit tests added
- KVM hypervisor plugin with 22.2% coverage
I also refactored the StorageSubSystemCommand interface into an abstract class
- Remove the pseudo-multiple-inheritance implementation
- The StorageSubSystemCommand was an interface, not related to the Command class
and its implementation were extending the Command class anyway. The whole structure is better now.
- Addin LibvirtPvlanSetupCommandWrapper
- 6 unit tests added
- KVM hypervisor plugin with 21% coverage
From the 6 tests added, 2 were extra tests to increase the coverage of the LibvirtStopCommandWrapper
- Increased from 35% to 78.7%
- Adding LibvirtCopyVolumeCommandWrapper
Refactoring the LibvirtUtilitiesHelper
- Changing method name
Did not add any test to this commit due to the refactor mentioned abot.
Will proceed and add the tests
i# Please enter the commit message for your changes. Lines starting
- Gave it a better, more suggestive, name since I now added other methods to the class.
- It makes easier to mock objects and get a better coverage of the classes
- Adding LibvirtBackupSnapshotCommandWrapper, LibvirtCreatePrivateTemplateFromVolumeCommandWrapper and LibvirtManageSnapshotCommandWrapper
- 3 unit tests added
- KVM hypervisor plugin with 18.3% coverage
Less tests added to those classes because the code is quite complex and way too long.
The tests added are just covering the new flow, to make sure it works fine. I will come back to those classes later.
- Adding LibvirtOvsDestroyBridgeCommandWrapper, LibvirtOvsSetupBridgeCommandWrapper
- 4 unit tests added
- KVM hypervisor plugin with 13.9% coverage
More tests added to cover LibvirtPrepareForMigrationCommandWrapper
- Coverage of this wrapper broght from 37% to 90.6%
- 4 new tests added
- Adding LibvirtCheckConsoleProxyLoadCommandWrapper, LibvirtConsoleProxyLoadCommandWrapper, LibvirtWatchConsoleProxyLoadCommandWrapperand CitrixConsoleProxyLoadCommandWrapper
- 2 unit tests added
- KVM hypervisor plugin with 12% coverage
Refactored the CommandWrapper interface in order to remove the esecuteProxyLoadScan, which is now
implemented bu subclasses.
- Adding LibvirtGetHosStatsCommandWrapper
- 1 unit test added
- KVM hypervisor with 10.5% coverage
Tests are a bit limited on this one becuause of the current implementation. Would clean it up later in a separate branch
- Adding LibvirtGetVmStatsCommandWrapper
- 3 unit tests
Refactored the LibvirtConnectiobn by surrounding it with an wrapper.
- Make it easier to cover the static/native calls
- Added better coverage to StopCommand tests
- Adding LibvirtStopCommandWrapper
- LibvirtRequestWrapper
- 1 unit tests
Refactored the RequestWrapper to make it better.
- Changes also applied to the CitrixRequestWrapper
Linux kernel supports vmxnet3, allowing it in KVM plugin would allow us to
run ESX hosts on KVM hosts using CloudStack with vmxnet3 nic which can be
passed as VM's nicAdapter detail
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit e02d787f30)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This improvements checks for "guest.cpu.features" property which is a space
separated list of cpu features that is specific for a host. When added, it
will add <feature policy='require' name='{{feature-you-listed}}'/> in the
<cpu> section of the generated vm spec xml.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit ea7fd37783)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The only artifact resolved from libvirt.org was org.libvirt:libvirt:0.5.1
this artifact is now available from maven's default central repository
This closes#180
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
As suggested by Wido on the dev ML changing the repo to eu.ceph.com to avoid
build failures. Will revert if ceph.com is up again.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit c9fd57fff3)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CentOS 7 does not ship with ifconfig anymore. We should use ip commands instead.
This also works on older versions, like CentOS 6 and Ubuntu 12.x/14.x, that we
support.
This closes#165
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
1. provide compatibility with the Big Cloud Fabric (BCF) controller
L2 Connectivity Service in both VPC and non-VPC modes
2. virtual network terminology updates: VNS --> BCF_SEGMENT
3. uses HTTPS with trust-always certificate handling
4. topology sync support with BCF controller
5. support multiple (two) BCF controllers with HA
6. support VM migration
7. support Firewall, Static NAT, and Source NAT with NAT enabled option
8. add VifDriver for Indigo Virtual Switch (IVS)
This closes#151
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Refactored to use the XPatch expressions to check the generated domain xml rathern than string comparison.
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
Earlier host addition of multiple hosts with local storage failed due to
same local storage UUID being used where the storage path is same.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit bf17f640c6)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
KVMStoragePoolManager is a singleton in practice, any plugin
or extension of LibvirtComputingResource will need to act on
the specific instance of KVMStoragePoolManager that LibvirtComputingResource
has initialized. Therefore, expose this variable for those who
wish to call storage commands from plugins or extensions.
Conflicts:
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
Clearly show if a volume is found and if not, that the pool is being refreshed
and the fetch is tried again.
Due to my commit b53a9dcc9f the chance of a volume
not being found is slightly bigger, but the performance gain is enormous on larger
deployments.
This is why we clearly have to log that we are refreshing the pool information
when a volume is not found.
It could be that a volume is created on host A and a few seconds later host B tries
to access the volume. In that case host B's libvirt doesn't know about the volume
yet and has to refresh the pool before it does.
On larger (especially RBD) storage pools this can take a lot of
time slowing operations like creating volumes down.
The getStorageStats command will still ask a pool to be refreshed so
that the management server has accurate information about the storage pools.
On larger deployments, with thousands of volumes in one pool, this should
significantly improve storage related operations
For ResizeVolume API command -
1. If hypervisor resource throws an exception, handle the NPE thrown by the job framework.
2. Improve user error message in case of RuntimeException by throwing the exception instead of 'Unexpected Exception'.
We don't need an external script to investigate the format of the RBD volume,
we only have to ask Libvirt to resize the volume and that will ask librbd to
do so.
In situations where libvirt lost the storage pool the KVM Agent will re-create the
storage pool in libvirt.
This could be then libvirt is restarted for example.
The object returned internally was missing essential information like the sourceDir
aka the Ceph pool, the monitor IPs, cephx information and such.
In this case the first operation on this newly created pool would fail. All operations
afterwards would succeed.
We used to create the snapshot after the copy from Secondary Storage,
but it could be that we never use the snapshot.
Now we check if the snapshot exists prior to performing the cloning operation
Since we use qemu-img to copy from RBD to Secondary Storage we no
longer have to force to RAW images, but can stick with QCOW2
When the snapshot backups are QCOW2 format they can easily be deployed
again when restoring from a backup
The KVMStorageProcessor no longer has a hardcoded if-statement which sets
RBD volumes to RAW, this is now handled in the LibvirtStorageAdapter
The Management Server still sends QCOW2 as format. That's a fix for later.
fix mismatch of ovs-host-setup, ovs_host_setup used Libvirt resource and
scripts
plug the nic to OVS bridges created for the tunnel network.
Conflicts:
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/OvsVifDriver.java
Added a new flag 'checkBeforeCleanup' to StopCommand based on which check is done to see if VM is running in HV host.
If VM is running then in this case it is not stopped and the operation bails out.
Also modified the MS code to call the StopCommand with appropriate value for the flag based on the context.
Currently it is only set to 'true' when called from the new vmsync logic based on powerstate of VM. For rest it
is set to 'false' meaning no change in behaviour.
This reduces the amount of time and storage it takes dramatically. We no longer
do a full copy, but a sparse copy. The destination image is still in RAW
format, but we only copy over used blocks.
Qemu is also better in doing this then us doing it in Java code.
Otherwise a RBDException will be thrown with the message that the snapshot
isn't protected.
modified: plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java
This saves the step of writing to a temporary image in /tmp first before
writing to RBD.
This is possible due to a new version in librbd. With the rbd_default_format
setting we can now force qemu-img to create format 2 RBD images.
This is available since Ceph version 0.67.5 (Dumpling).
Add executeInVR() with timeout interface to VirtualRouterDeployer
AggregationControlCommand with Action.Finish may take longer than normal command
since it would execute all the commands in one execution, and it may result in
SSH timeout for SshHelper or other mechanism communicate with VR.
Introduce an new executeInVR() interface with added timeout period for waiting
FinishAggregationCommand to complete execution.
- get the hosts on which VPC spans given vpc id
- get the VM's in the VPC
- get the hosts on which a network spans
- get the VPC's to which a hosts is part of
- get VM's of a VPC on a hosts
introduces capability to build a physical toplogy representation of a
VPC. This json file is encapsulated in
OvsVpcPhysicalTopologyConfigCommand, and is used to send full topology
to hypervisor hosts. On hypervisor this json config can be used to setup
tunnels, configure bridge, add flow rules etc
Ovs GURU, to use different broasdcast scheme VS://vpcid.gerkey for the
networks in VPC that use distributed routing
each VIF and tunnel interface to carry the network UUID in other/options
config
2) Corrected some logging in MidoNetPublicNetworkGuru - removed .toString method call on the objects in the log body as toString is called on the object by default when use log4j
With VirtIO enabled on KVM. FreeBSD 10 supports VirtIO for both the
network and the disks. This frees us from IDE and E1000 which should
also improve performance.
By default all network disks are in RAW format. Gluster works fine with
QCOW2 which has some advantages.
Disks are by default in QCOW2 format. It is possible to run into
a mismatch, where the disk is in QCOW2 format, but QEMU gets started
with format=raw. This causes the virtual machines to lockup on boot.
Failures to start a virtual machine can be verified by checking the log
of the virtual machine, and compare the output of 'qemu-img info'.
In /var/log/libvirt/qemu/<VM>.log find the URL for the drive:
-drive file=gluster+tcp://...,format=raw,..
Compare this with the 'qemu-img info' output of the same file, mounted
under /mnt/<pool-uuid>/<img-uuid>:
# qemu-img info /mnt/<pool-uuid>/<img-uuid>
...
file format: qcow2
...
This change makes passes the format when creating a disk located on RBD
(RAW only) and Gluster (QCOW2).
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The support for Gluster as Primary Storage is mostly based on the
implementation for NFS. Like NFS, libvirt can address a Gluster environment
through the 'netfs' pool-type.
PrepareForMigrationCommand, so that destination hypervisor can
mount pool. This further exposed an issue for KVM where iso
was not getting cleaned up upon successful migration, fixed as well.
By default only the Integers between -128..127 are cached (unless overridden by java.lang.Integer.IntegerCache.high system property)
If the inbound or outbound values are higher, the reference comparison won't work.
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
- minor resource leak cleaned up
- cpu-speed reading method extracted
- test added
- logging added in case of exception
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
This saves us a lot of code and libvirt is probably a better
place to do this.
libvirt-java now has the support we want, so we can now resize volumes
with libvirt.
(C)LVM volumes can't be resized using libvirt, so we have to
invoke a resize script for that.
By default the client_mount_timeout setting in librados is 300 seconds,
but that causes the connect to the Ceph cluster to block for 5 minutes
if the Ceph cluster is not available.
This patch is not ideal, but it mitigates the problem for now.
At a later point all this librados/librbd code should go back to libvirt
again, but the current versions of libvirt in the distributions are
to old for all the features we require.
For now this should prevent the CloudStack agent blocking for 5 minutes
when the Ceph cluster isn't available.
This is also tracked at the Ceph tracker: http://tracker.ceph.com/issues/6507
brought back up after being down for few hours,snapshot jobs do not get
triggered with reason "there is other active snapshot tasks on the
instance to which the volume is attached".
- the result of dividing long with long resulted in loss of precision both for network and IO
- unit tests included
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
replace vlanid wih broadcast uri to support vxlan to identify whether id is VLAN ID or VNI
Signed-off-by: ynojima <mail@ynojima.net>
Signed-off-by: Hugo Trippaers <htrippaers@schubergphilis.com>
Detail: getPhysicalDisk() was not matching on volumes with .raw, so
instead setting disk format to QCOW2.
BUG-ID: CLOUDSTACK-5018
Bugfix-for:
Reviewed-by:
Reported-by:
Signed-off-by: John Kinsella <jlk@stratosec.co> 1383287538 -0700
1) vxlan will use bridge scheme 'brvx-<vni>'. Multiple physical networks can host guest
traffic type with vxlan isolation, so long as they don't use the same VNI range.
2) Guest traffic labels can be physical interface if bridge by given name is not found.
Normally we take traffic label name, find the matching bridge, then resolve that to a
physical interface. Then we create guest bridges on that interface. Now we can just
specify the interface.
xs 6.1/6.2 introduce the new virtual platform, so there are two virtual platforms, windows PV driver version must match virtual platforms,
this patch tracks PV driver versions in vm details and template details.
Anthony
Detail: Checks for other Ethernet interface names uses startsWith(),
whereas the p1p1 style interface uses a regex that doesn't allow for
tailing characters, and so blocks vlan IDs. Fixed.
BUG-ID: CLOUDSTACK-4884
Bugfix-for: 4.2.1
Reviewed-by:
Reported-by:
Signed-off-by: John Kinsella <jlk@stratosec.co> 1381965250 -0700
I don't think host kernel version has any bearing on it. Original code
was tested with CentOS 6.3 and 6.4, but it seems to succeed or fail per-host,
e.g. a fast host might work and a slow host might not. I was getting intermittent
failures with ubuntu 12.04.3 prior to this patch.
These changes are a joint effort between Edison and I to refactor some
of the code around snapshotting VM volumes and creating
templates/volumes from VM volume snapshots. In general, we were working
towards allowing PrimaryDataStoreDrivers to create snapshots on primary
storage and not requiring the snapshots to be transferred to secondary
storage.
High level changes:
-Added uuid to NfsTO, SwiftTO & S3TO to cut down on the requirement of
PrimaryDataStoreTO and ImageStoreTO which don't really serve much of a
purpose
-Initial work towards enable reverting VM volume from snapshots
-Added hypervisor commands for introducing and forgetting new hypervisor
objects (snapshots, templates & volumes)
Signed-off-by: Edison Su <sudison@gmail.com>
ACS is now comprised of a hierarchy of spring application contexts.
Each plugin can contribute configuration files to add to an existing
module or create it's own module.
Additionally, for the mgmt server, ACS custom AOP is no longer used
and instead we use Spring AOP to manage interceptors.
The managed context framework provides a simple way to add logic
to ACS at the various entry points of the system. As threads are
launched and ran listeners can be registered for onEntry or onLeave
of the managed context. This framework will be used specifically
to handle DB transaction checking and setting up the CallContext.
This framework is need to transition away from ACS custom AOP to
Spring AOP.
Initial patch for VXLAN support.
Fully functional, hopefully, for GuestNetwork - AdvancedZone.
Patch Note:
in cloudstack-server
- Add isolation method VXLAN
- Add VxlanGuestNetworkGuru as plugin for VXLAN isolation
- Modify NetworkServiceImpl to handle extended vNet range for VXLAN isolation
- Add VXLAN isolation option in zoneWizard UI
in cloudstack-agent (kvm)
- Add modifyvxlan.sh script that handle bridge/vxlan interface manipulation script
-- Usage is exactly same to modifyvlan.sh
- BridgeVifDriver will call modifyvxlan.sh instead of modifyvlan.sh when VXLAN is used for isolation
Database changes:
- No change in database structure.
- VXLAN isolation uses same tables that VLAN uses to store vNet allocation status.
Known Issue and/or TODO:
- Some resource still says 'VLAN' in log even if VXLAN is used
- in UI, "Network - GuestNetworks" dosen't display VNI
-- VLAN ID field displays "N/A"
- Documentation!
Signed-off-by : Toshiaki Hatano <haeena@haeena.net>
Libvirt reports:
org.libvirt.LibvirtException: Storage volume not found: no storage vol
with matching name
in some cases, if the volume is created on one kvm host, while accessed
from other host.
It's possible due to concurrent access(read/write) storage.
The current fix is to try serveral times, and wait for 30 seconds for
each retry.
If the issue still there, then need to sync the storage pool access
CLOUDSTACK-4457:
CLOUDSTACK-4459:
harden kvm getvolume. It's possible that one volume created on other kvm host, won't show up on another host, try more times to refresh storage pool if volume won't shown up
Conflicts:
engine/storage/integration-test/test/org/apache/cloudstack/storage/test/FakeDriverTestConfiguration.java
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java
There still exist two issues after Edison's commits.
(1) Migration from new hosts to old hosts failed.
The bridge name on old host is set to cloudVirBr* if network.bridge.name.schema is set to 3.0 in /etc/cloudstack/agent/agent.properties, but the actual bridge name is breth*-* after running cloudstack-agent-upgrade.
(2) all ports of vms (Basic zone, or Advanced zone with security groups) on old hosts are open, because the iptables rules are binding to device (bridge) name which is changed by cloudstack-agent-upgrade.
After this, the KVM upgrade steps :
a. Install 4.2 cloudstack agent on each kvm host
b. Run "cloudstack-agent-upgrade". This script will upgrade all the existing bridge name to new bridge name, and update related firewall rules.
c. install a libvirt hook:
c1. mkdir /etc/libvirt/hooks
c2. cp /usr/share/cloudstack-agent/lib/libvirtqemuhook /etc/libvirt/hooks/qemu
c3. chmod +x /etc/libvirt/hooks/qemu
c4. service libvirtd restart
c5. service cloudstack-agent restart
Signed-off-by: Wei Zhou <w.zhou@leaseweb.com>
Boris Schrijver
Wilder Rodrigues
Daan Hoogland
Remi Bergsma
Wido den Hollander
David Mabry
Satoru Nakaya
Kevin McCormick
Rohit Yadav
Wei Zhou
Mike Tutkowski
Rajani Karuturi
Rafael Weingartner
Koushik Das
manuiiit
Aaron Brady
Daan Hoogland
Wido den Hollander
Kishan Kavala
Rafael da Fonseca
Laszlo Hornyak
Daan Hoogland
Rajani Karuturi
KC Wang
Sanjay Tripathi
Marcus Sorensen
Wei Zhou
Santhosh Edukulla
Logan Barfield
punith-cloudbyte
Hugo Trippaers
Edison Su
Kishan Kavala
Likitha Shetty
Anthony Xu
amoghvk
Suresh Ramamurthy
Marcus Sorensen
ynojima
Murali Reddy
Kishan Kavala
Sheng Yang
Alena Prokharchyk
Kelven Yang
Jayapal
Min Chen
Niels de Vos
Jayapal
Alex Huang
tuna
Harikrishna Patnala
John Kinsella
Darren Shepherd
Chris Suich
Darren Shepherd