brought back up after being down for few hours,snapshot jobs do not get
triggered with reason "there is other active snapshot tasks on the
instance to which the volume is attached".
- the result of dividing long with long resulted in loss of precision both for network and IO
- unit tests included
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
replace vlanid wih broadcast uri to support vxlan to identify whether id is VLAN ID or VNI
Signed-off-by: ynojima <mail@ynojima.net>
Signed-off-by: Hugo Trippaers <htrippaers@schubergphilis.com>
Detail: getPhysicalDisk() was not matching on volumes with .raw, so
instead setting disk format to QCOW2.
BUG-ID: CLOUDSTACK-5018
Bugfix-for:
Reviewed-by:
Reported-by:
Signed-off-by: John Kinsella <jlk@stratosec.co> 1383287538 -0700
1) vxlan will use bridge scheme 'brvx-<vni>'. Multiple physical networks can host guest
traffic type with vxlan isolation, so long as they don't use the same VNI range.
2) Guest traffic labels can be physical interface if bridge by given name is not found.
Normally we take traffic label name, find the matching bridge, then resolve that to a
physical interface. Then we create guest bridges on that interface. Now we can just
specify the interface.
xs 6.1/6.2 introduce the new virtual platform, so there are two virtual platforms, windows PV driver version must match virtual platforms,
this patch tracks PV driver versions in vm details and template details.
Anthony
Detail: Checks for other Ethernet interface names uses startsWith(),
whereas the p1p1 style interface uses a regex that doesn't allow for
tailing characters, and so blocks vlan IDs. Fixed.
BUG-ID: CLOUDSTACK-4884
Bugfix-for: 4.2.1
Reviewed-by:
Reported-by:
Signed-off-by: John Kinsella <jlk@stratosec.co> 1381965250 -0700
I don't think host kernel version has any bearing on it. Original code
was tested with CentOS 6.3 and 6.4, but it seems to succeed or fail per-host,
e.g. a fast host might work and a slow host might not. I was getting intermittent
failures with ubuntu 12.04.3 prior to this patch.
These changes are a joint effort between Edison and I to refactor some
of the code around snapshotting VM volumes and creating
templates/volumes from VM volume snapshots. In general, we were working
towards allowing PrimaryDataStoreDrivers to create snapshots on primary
storage and not requiring the snapshots to be transferred to secondary
storage.
High level changes:
-Added uuid to NfsTO, SwiftTO & S3TO to cut down on the requirement of
PrimaryDataStoreTO and ImageStoreTO which don't really serve much of a
purpose
-Initial work towards enable reverting VM volume from snapshots
-Added hypervisor commands for introducing and forgetting new hypervisor
objects (snapshots, templates & volumes)
Signed-off-by: Edison Su <sudison@gmail.com>
ACS is now comprised of a hierarchy of spring application contexts.
Each plugin can contribute configuration files to add to an existing
module or create it's own module.
Additionally, for the mgmt server, ACS custom AOP is no longer used
and instead we use Spring AOP to manage interceptors.
The managed context framework provides a simple way to add logic
to ACS at the various entry points of the system. As threads are
launched and ran listeners can be registered for onEntry or onLeave
of the managed context. This framework will be used specifically
to handle DB transaction checking and setting up the CallContext.
This framework is need to transition away from ACS custom AOP to
Spring AOP.
Initial patch for VXLAN support.
Fully functional, hopefully, for GuestNetwork - AdvancedZone.
Patch Note:
in cloudstack-server
- Add isolation method VXLAN
- Add VxlanGuestNetworkGuru as plugin for VXLAN isolation
- Modify NetworkServiceImpl to handle extended vNet range for VXLAN isolation
- Add VXLAN isolation option in zoneWizard UI
in cloudstack-agent (kvm)
- Add modifyvxlan.sh script that handle bridge/vxlan interface manipulation script
-- Usage is exactly same to modifyvlan.sh
- BridgeVifDriver will call modifyvxlan.sh instead of modifyvlan.sh when VXLAN is used for isolation
Database changes:
- No change in database structure.
- VXLAN isolation uses same tables that VLAN uses to store vNet allocation status.
Known Issue and/or TODO:
- Some resource still says 'VLAN' in log even if VXLAN is used
- in UI, "Network - GuestNetworks" dosen't display VNI
-- VLAN ID field displays "N/A"
- Documentation!
Signed-off-by : Toshiaki Hatano <haeena@haeena.net>
Libvirt reports:
org.libvirt.LibvirtException: Storage volume not found: no storage vol
with matching name
in some cases, if the volume is created on one kvm host, while accessed
from other host.
It's possible due to concurrent access(read/write) storage.
The current fix is to try serveral times, and wait for 30 seconds for
each retry.
If the issue still there, then need to sync the storage pool access
CLOUDSTACK-4457:
CLOUDSTACK-4459:
harden kvm getvolume. It's possible that one volume created on other kvm host, won't show up on another host, try more times to refresh storage pool if volume won't shown up
Conflicts:
engine/storage/integration-test/test/org/apache/cloudstack/storage/test/FakeDriverTestConfiguration.java
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java
There still exist two issues after Edison's commits.
(1) Migration from new hosts to old hosts failed.
The bridge name on old host is set to cloudVirBr* if network.bridge.name.schema is set to 3.0 in /etc/cloudstack/agent/agent.properties, but the actual bridge name is breth*-* after running cloudstack-agent-upgrade.
(2) all ports of vms (Basic zone, or Advanced zone with security groups) on old hosts are open, because the iptables rules are binding to device (bridge) name which is changed by cloudstack-agent-upgrade.
After this, the KVM upgrade steps :
a. Install 4.2 cloudstack agent on each kvm host
b. Run "cloudstack-agent-upgrade". This script will upgrade all the existing bridge name to new bridge name, and update related firewall rules.
c. install a libvirt hook:
c1. mkdir /etc/libvirt/hooks
c2. cp /usr/share/cloudstack-agent/lib/libvirtqemuhook /etc/libvirt/hooks/qemu
c3. chmod +x /etc/libvirt/hooks/qemu
c4. service libvirtd restart
c5. service cloudstack-agent restart
Signed-off-by: Wei Zhou <w.zhou@leaseweb.com>
The migrate method from libvirt supports passing down a different XML for running
the instance of the target hypervisor.
This enables the VNC to bind to the private IP Address of the hypervisor and during
migration this will be changed to the private IP address of the target host.
This way VNC doesn't listen world wide and is much safer.
Initial patch for VXLAN support.
Fully functional, hopefully, for GuestNetwork - AdvancedZone.
Patch Note:
in cloudstack-server
- Add isolation method VXLAN
- Add VxlanGuestNetworkGuru as plugin for VXLAN isolation
- Modify NetworkServiceImpl to handle extended vNet range for VXLAN isolation
- Add VXLAN isolation option in zoneWizard UI
in cloudstack-agent (kvm)
- Add modifyvxlan.sh script that handle bridge/vxlan interface manipulation script
-- Usage is exactly same to modifyvlan.sh
- BridgeVifDriver will call modifyvxlan.sh instead of modifyvlan.sh when VXLAN is used for isolation
Database changes:
- No change in database structure.
- VXLAN isolation uses same tables that VLAN uses to store vNet allocation status.
Known Issue:
- Some resource still says 'VLAN' in log even if VXLAN is used
- in UI, "Network - GuestNetworks" dosen't display VNI
-- VLAN ID field displays "N/A"
This failed due to a RAW -> QCOW2 conversion (again).
The current code still makes to much assumptions about everything always
being QCOW2 while that is not always true.
KVM - Create template from volume
Vmware - Create template from volume / Create template from snapshot
send the physical size in the copycommand which accordingly will populate template store ref and the usage_event tables with the right physical size
Signed off by : nitin mehta<nitin.mehta@citrix.com>
Although libvirt supports resizing RBD volumes (and other formats) the
Java bindings (libvirt-java) don't.
Right now we use the Java bindings for librbd to handle the resizing for us,
but in the future this should be done by libvirt rather then these
Java bindings.
Detail: Cloudstack tries to stop VMs all the time, for all sorts of reasons,
but usually just to get into a known state. Libvirt throws an exception of
'Domain not found' when attempting to stop a VM that doesn't exist. This causes
problems for troubleshooting real issues. Domain not found should equate
to success if trying to stop.
BUG-ID: CLOUDSTACK-4011
Bugfix-for: 4.2
Signed-off-by: Marcus Sorensen <marcus@betterservers.com> 1375825281 -0600
- Move vnetBridge clean up function from LibvirtComputingResource to BridgeVifDriver
-- since only BridgeVifDriver have to handle this event
- LibvirtComputingResource now properly call VifDriver.unplug() when it receives UnPlugCommand
- Remove not working and no longer used method getVnet(String) from VirtualMachineName
- Remove not working and no longer used method getVnet() from StopCommand
- Remove unused constructer StopCommand(VirtualMachine, String, boolean) from StopCommand
- Remove unused member vnet from StopCommand
- Remove unused member _modifyVlanPath from OvsVifDriver
Tested with 2 KVM hosts and confirmed it correctly manipulate vnetBridge with start, stop, migrate, plug, and unplug event
Signed-off-by: Hugo Trippaers <htrippaers@schubergphilis.com>
Description:
a) Fixing NPE when wrong path is provided for primary datastore.
b) No error dialog shows up in GUI when wrong path is provided,
after NPE fix - propagating exception upward.
c) If the KVM agent is down, an invalid datastore gets logged in
storage_pool table and doesn't get removed, so it shows up
in the GUI in the list of datastores - fixing this as well.
- writeToFile removed since no references to it
- readFileAsString replaced with FileUtils.readFileToString
- minor code duplication removed in dependent method getNicStats
- unit test added
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
Renaming the method in the command objects to be uniform with
PlugNicCommand/UnplugNicCommand.getVmName
Signed-off-by: Prasanna Santhanam <tsp@apache.org>
inactive VM definitions block a new VM starting. Definitions aren't supposed to
be persistent, but sometimes a crash or failed migration can leave behind a
definition.
Signed-off-by: Marcus Sorensen <marcus@betterservers.com> 1370299734 -0600
RBD format 2 supports cloning (aka layering) where one base image can serve
as a parent image for multiple child images.
This enables fast deployment of a large amount of virtual machines, but it also
saves spaces on the Ceph cluster and improves performance due to better caching.
Qemu-img doesn't support RBD format 2 (yet), so to enable these functions the
RADOS/RBD Java bindings are required.
This patch also enables deployment of System VMs on RBD storage pools. Since we
no longer require a patchdisk for passing the boot arguments we are able to deploy
these VMs on RBD.
Detail: We do two strange things, #1, when a vm is created, we create the uuid
for the domain by converting the name into a uuid, then subsequently, any time
we look for a domain, we convert its name to a uuid and domainLookupByUUID. This
is an unnecessary obfuscation, so instead we just do a domainLookupByName. As
a bonus, we are now setting the domain's uuid to be the cloudstack uuid. It is
no longer used anywhere, but will be less confusing to admins. This shouldn't
affect upgrades, since we can always lookup existing VMs by name.
Signed-off-by: Marcus Sorensen <marcus@betterservers.com> 1369287049 -0600
Detail: Undefine VM after migration. Lingering domain definitions cause
migrations back to the original host to fail, since domain already exists.
BUG-ID: CLOUDSTACK-2640
Bugfix-for: 4.1.0,4.2.0
Signed-off-by: Marcus Sorensen <marcus@betterservers.com> 1369285950 -0600
so that callers of startVM in LibvirtComputingResource know that a vm failed
to start
Signed-off-by: Marcus Sorensen <marcus@betterservers.com> 1366224775 -0600
pools the proper way won't cause problems for KVM HA Monitor, this patch closes
holes. Call the KVMStoragePool deleteStoragePool that properly removes it from
the KVMHA hashmap, instead of the pools direct delete() call.
Signed-off-by: Marcus Sorensen <marcus@betterservers.com> 1366172318 -0600
- Supports DHCP, Source NAT, Static NAT, Firewall rules, Port Forwarding
- Renamed MidokuraMidonet to MidoNet
- Related Jira ticket is CLOUDSTACK-996
Signed-off-by: Dave Cahill <dcahill@midokura.com>
Signed-off-by: Hugo Trippaers <htrippaers@schubergphilis.com>
deleted NFS pools, causing failures when defining new storage pools. Sometimes
a storage pool has never been used on a host, and getStoragePool fails when
copying templates or in storage migration. deleteStoragePool(pool) often fails
silently, leaving no pool defined in libvirt, but a mountpoint left behind.
This patch handles some of these exceptions and brings forward any issues via
logging.
Signed-off-by: Marcus Sorensen <marcus@betterservers.com> 1364603486 -0600
KVM to manager. This adds collection of available storage to KVM, not
just used.
Bugfix-for: 4.0.2, 4.1, master
Submitted-by: Ted Smith <darnoth@gmail.com>
Signed-off-by: Marcus Sorensen <marcus@betterservers.com> 1363966235 -0600
The collection of network usage from VPC virtual router on KVM does not work,
because there is no corresponding procedure to deal with VPC virtual router
(cmd.isForVpc() = true).
Reviewed-by: Kishan Kavala <kishan@apache.org>
Reported-by: Wei Zhou <w.zhou@leaseweb.com>
Signed-off-by: Prasanna Santhanam <tsp@apache.org>
cloud-defined resources on the host has caused various problems. As a backward
compatible fix, if an existing pool with a different name collides with a pool
being created (by path), the pool will be redefined with the name cloudstack
knows about. This is actually what brought up the bug, a persisted storage pool
cloudstack wasn't managing.
Signed-off-by: Marcus Sorensen <marcus@betterservers.com> 1363210149 -0600
Detail: When we stop a VM, it's definition is no longer valid. Therefore, we
need to catch the exception thrown from libvirt in trying to lookup a
non-existent domain by UUID while trying to check if it's shut down.
BUG-ID:CLOUDSTACK-600
Signed-off-by: Marcus Sorensen <marcus@betterservers.com> 1363201066 -0600
Detail: A previous patch fixed an issue where we are defining VMs to persist
locally on KVM hosts, which can cause issues if the agent isn't running and
libvirt decides to start the VM unbeknownst to cloudstack. The previous patch
stopped defining VMs as persistent. This patch adds compatibility for existing
cloudstack environments, removing the persistent definition on stop if needed.
BUG-ID: CLOUDSTACK-600
Signed-off-by: Marcus Sorensen <marcus@betterservers.com> 1363194656 -0600
Detail: This gets rid of the patchdisk method of passing cmdline and
authorized_keys to KVM system VMs. It instead passes them to a virtio socket,
which the KVM guest reads from the character device /dev/vport0p1 during
cloud-early-config. Tested to work on CentOS 6.3 and Ubuntu 12.04. Should
work with even older versions of libvirt.
Signed-off-by: Marcus Sorensen <marcus@betterservers.com> 1362691685 -0700
Current kvm agent will silently ignore many exception, and there's no
way to see what really happened. This patch will log in trace level log
that was silently ignored. And also, it will fix huge bare Exception
catch, which is very harmful because it also catches RuntimeException.
Detail: This device can be used for remotely controlling the system vms through
a local socket on the host. We will attempt to replace the KVM patchdisk with
it. Tested, successfully deploys VM, and if system vm has proper driver it
will create a /dev/vport0p1 device within the VM. We will be updating the
system VM in 4.2/5.0 and will support this.
Signed-off-by: Marcus Sorensen <marcus@betterservers.com> 1362527352 -0700
Libvirt-java 0.4.9 works just fine with JNA 3.2.4 which is in
all distributions.
Future libvirt version require at least JNA 3.5.1 due to new methods
and memory management, but that is not our concern now.
By depending on the JNA in the distribution and adding it to the classpath
we can work just fine.
Mike Tutkowski
Marcus Sorensen
Hugo Trippaers
Min Chen
Alex Huang
tuna
Wido den Hollander
Wei Zhou
Edison Su
Kelven Yang
Laszlo Hornyak
ynojima
Jayapal
Harikrishna Patnala
John Kinsella
Anthony Xu
Darren Shepherd
Chris Suich
Darren Shepherd
Daan Hoogland
Toshiaki Hatano
Wido den Hollander
Rajesh Battala
Nitin Mehta
Edison Su
JijunLiu
Chip Childers
Kishan Kavala
Vijayendra Bhamidipati
Prasanna Santhanam
Hiroaki KAWAI
Fang Wang
Sheng Yang
Jayapal Reddy
Alex Huang
Dave Cahill
Phong Nguyen
Bharat Kumar