Due to PR #1054 this patch fixes the dynamic-roles migration script
to use the mysql-connector-python dependency.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Removes bundling of systemvm.zip in cloudstack-common rpms. This is not
done in debian packaging either there we remove for rpms as well, as this
file is not used by any subsystem but systemvm.iso is used.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
In case of rpms, the commands.properties file is bundled at
/usr/share/cloudstack-management/webapps/client/WEB-INF/classes/commands.properties
In case of a rpm upgrade, new rpms won't ship with commands.properties file. For
existing installations this copies the commands.properties file to
/etc/cloudstack/management
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9362: Skip VXLANs when rewriting the bridge name for migrations (4.8-2)From the [JIRA issue](https://issues.apache.org/jira/browse/CLOUDSTACK-9362):
> bb8f7c652e
>
> The above commit introduces rewriting of bridge device names when migrating a virtual machine from one host to another. However, it also matches bridges called "brvx-1234" and rewrites them to (in my case) "brem1-1234" - this doesn't match the bridge name on the destination and causes the migration to fail with the error:
>
> error : virNetDevGetMTU:397 : Cannot get interface MTU on 'brem1-1234': No such device
>
> I have flagged this as major because it's not possible to migrate VMs using VXLANs for maintenance, which seems important (it's certainly important to me!).
This is a version of #1508 based against 4.8 (sorry!)
* pr/1513:
Skip VXLANs when rewriting the bridge name for migrations
Signed-off-by: Will Stevens <williamstevens@gmail.com>
* 4.7:
Fix Sync of template.properties in Swift
Configure rVPC for router.redundant.vrrp.interval advert_int setting
Have rVPCs use the router.redundant.vrrp.interval setting
Resolve conflict as forceencap is already in master
Split the cidr lists so we won't hit the iptables-resture limits
Check the existence of 'forceencap' parameter before use
Do not load previous firewall rules as we replace everyhing anyway
Wait for dnsmasq to finish restart
Remove duplicate spaces, and thus duplicate rules.
Restore iptables at once using iptables-restore instead of calling iptables numerous times
Add iptables copnversion script.
Fix Sync of template.properties in SwiftWhen using Swift as a secondary storage, we create a template.properties file which stores the metadata about the template. Currently the data which is present in the file is incorrect which leads to templates becoming unavailable after they are downloaded. This fix makes sure that the template.properties has the correct "path" set so that templates are available.
I've also done a bit of cleanup and made the code bit more clean.
* pr/1331:
Fix Sync of template.properties in Swift
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Reimplement router.redundant.vrrp.interval settingGlobal setting `router.redundant.vrrp.interval` is not used any more and it is now set to a hardcoded 1.
This results in a failover from master->backup when the backup doesn't hear from the master in ~3.6sec. This is a bit too tight, as we've seen failovers during live migrations. We could reproduce it in about half of the cases. Setting this to setting to 2 (tested it by hardcoding it in the systemvms) gives twice as much time and we didn't see issues any more. Instead of updating the hardcoded setting from 1 to 2, I reimplemented the global setting by sending it to the router with the cmd_line, as the non-VPC router also does.
Background:
Why is the maximum failover time in the example 3.6 seconds? This comes from the advertisement interval and the skew time. The default advertisement interval is 1 second (configurable in keepalived.conf). The skew time helps to keep everyone from trying to transition at once. It is a number between 0 and 1, based on the formula (256 - priority) / 256
As defined in the RFC, the backup must receive an advertisement from the master every (3 * advert_int) + skew_time seconds. If it doesn't hear anything from the master, it takes over. With a backup router priority of 100 (as in the example), the failover will happen at most 3.6 seconds after the master goes down.
Source: http://www.hollenback.net/KeepalivedForNetworkReliability
* pr/1486:
Configure rVPC for router.redundant.vrrp.interval advert_int setting
Have rVPCs use the router.redundant.vrrp.interval setting
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Restore iptables at once using iptables-restore instead of calling iptables numerous timesThis makes handling the firewall rules about 50-60 times faster because it is generated in memory and then loaded once. It's work by @borisroman see PR #1400. Reopened it here because I think this is a great improvement.
* pr/1482:
Resolve conflict as forceencap is already in master
Split the cidr lists so we won't hit the iptables-resture limits
Check the existence of 'forceencap' parameter before use
Do not load previous firewall rules as we replace everyhing anyway
Wait for dnsmasq to finish restart
Remove duplicate spaces, and thus duplicate rules.
Restore iptables at once using iptables-restore instead of calling iptables numerous times
Add iptables copnversion script.
Signed-off-by: Will Stevens <williamstevens@gmail.com>
On some MySQL server envs, this may cause a SQL statement error, though
I was unable to reproduce it. Since it's not needed, an order by 'sort_order'
is enough, we can safely remove it.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This reverts commit 7ce0e10fbc, reversing
changes made to 29ba71f2db.
This was reverted because it seemed to be related to an issue
when doing a DeployDC, causing an `addHost` error.
This reverts commit 9f970f28b1, reversing
changes made to 6d0c92be72.
This was reverted because it seemed to be related to an issue
when doing a DeployDC, causing an `addHost` error.
This reverts commit f88cb88097, reversing
changes made to 688522ecd4.
This was reverted because it seemed to be related to an issue
when doing a DeployDC, causing an `addHost` error.
Add perl-modules as install dependency for cloudstack-agentRequired to run perl scripts that configure networking for VMs.
* pr/1495:
Add perl-modules as install dependency for cloudstack-agent
Signed-off-by: Will Stevens <williamstevens@gmail.com>
DAO: Hit the cache for entity flagged as removed tooI came along this part of the code and I don't see any reason why the cache should not be used when fetching with the "removed" ones. It will help decrease the number of DB queries.
*It can be merged in many CS versions*
* pr/1532:
DAO: Rewrite change for method findByIdIncludingRemoved(ID id)
dao: Hit the cache for entity flagged as removed too since they are put in cache afterwards.
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CPU socket count reporting correctionCPU socket count reporting correction
From https://github.com/MissionCriticalCloud/cosmic-plugin-hypervisor-kvm/pull/16
* pr/1520:
Remove empty spaces causing the build to fail
CPU socket count reporting correction
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Fix Nio/CPU issue and CI failures- Reverts ea2286 that introduced a wakeup on each connection loop run.
- In SSL handshake code removes delegated tasks to be run in separate threads.
/cc @kiwiflyer @swill @jburwell and others for review
@kiwiflyer please help me test this fix and share if it makes the NioConnection robust now, without having the selector consume a lot of CPU. Thanks.
* pr/1543:
test: fix cleanup sequence for test_acl_listvolume test
CLOUDSTACK-9299: Fix test failures on CI
CLOUDSTACK-9348: Make NioConnectio loop less aggressive
Signed-off-by: Will Stevens <williamstevens@gmail.com>
It defaults to 1, which is hardcoded in the template:
./cosmic/cosmic-core/systemvm/patches/debian/config/opt/cloud/templates/keepalived.conf.templ
As non-VPC redundant routers use this setting, I think it makes sense to use it for rVPCs as well.
We also need a change to pickup the cmd_line parameter and use it in the Python code that configures the router.
- Fixes oobm integration test to skip if known ipmitool bug is hit
- Fixes ProcessTest unit test case to use sleep
- Removes redundant unit test that covers code in ProcessTest
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Reverts ea2286 that introduced a wakeup on each connection loop run.
- In SSL handshake code removes delegated tasks to be run in separate threads.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Honour GS use_ext_dns and redundant VR VIPThis patch addresses two issues:
On redundant VR setups, the primary resolver being handed out to instances is the guest_ip (primary IP for the VR). This might lead to problems upon failover, at least while the DHCP lease doesn't update (because the primary resolver will be checked first until times out, however it'll be gone upon failover).
If Global Setting use_ext_dns is true, we don't want the VR to be the primary resolver at all.
* pr/1536:
This patch addresses two issues:
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CLOUDSTACK-9334: Support jenv and pyenv to manage Java and Python versions * Adds .java-version to specify Java 1.7 as the JDK version to use
for CloudStack development. jenv does not support comments in
this file. Therefore, no license header is present.
* Adds .python-vresion to specify Python 2.7.11 as the Python to
use for CloudStack development. pyenv does not support comments
in this file. Therefore, no license header is present.
* Adds requirements.txt to specify the default dependencies used
for CloudStack development. It does not include Marvin's
dependencies because it's bundle already defines the dependencies
which are installed with Marvin.
* Updates INSTALL.md to explain how to utilize jenv and pyenv for
CloudStack development
* pr/1460:
CLOUDSTACK-9334: Support jenv and pyenv to manage Java and Python versions
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CLOUDSTACK-8800 : Improved the listVirtualMachines API call to include memory utilization information for a VMThis PR introduces the changes proposed in PR #780 with some work to make the code null safe.
During this PR, I have also removed some unused code.
* pr/1444:
Removed unnecessary check when creating the “userVmResponse” object.
Fixed issues from CLOUDSTACK-8800 that were introduced in PR 780
CLOUDSTACK-8800 : Improved the listVirtualMachines API call to include memory utilization information for a VM for xenserver,kvm and for vmware.
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Remove classes with no referencesI used UCDetector (http://www.ucdetector.org/) as a plugin for Eclipse. With this tool, I discovered a lot of code without any reference (variables, methods and classes).
Following the work that was done at [https://github.com/apache/cloudstack/pull/1448]; this pull request had the goal of removing some of these classes. To check if I wasn't missing anything I searched for any file that could reference some of those classes. As I haven't found any way of these classes being used, they were removed. Note that some of them I found other references, but references such as commented lines or tests, nothing that could indicate their use (as XML files configuring beans or another class instantiating an object with "new").
Waiting for tests. Please tell me if I am missing something.
Removed Classes:
- org.apache.cloudstack.framework.jobs.JobCancellationException (**Note:** removed
variable JobCancellationException in com.cloud.utils.SerialVersionUID)
- org.apache.cloudstack.ldap.NoSuchLdapUserException (**Note:** removed test file
/cloud-plugin-user-authenticator-ldap/test/groovy/org/apache/cloudstack/ldap/NoSuchLdapUserExceptionSpec.groovy)
- com.cloud.agent.api.storage.CreateVolumeOVAAnswer
- com.cloud.exception.MissingParameterValueException
- org.apache.cloudstack.api.response.StatusResponse
- org.apache.cloudstack.api.response.VolumeDetailResponse
- org.apache.cloudstack.api.response.UpgradeVmResponse
- org.apache.cloudstack.api.response.AddIpToVmNicResponse
- org.apache.cloudstack.api.response.TemplateZoneResponse (**Note:** at
org.apache.cloudstack.api.response.TemplateResponse, there is this
comment "To avoid breaking backwards compatibility, we still treat a
template at different zones as different templates, so not embedding
template_zone information in this TemplateZoneResponse set. `private
Set<TemplateZoneResponse> zones;`" but right now it is not used)
- org.apache.cloudstack.api.response.NicDetailResponse
* pr/1453:
Removed classes with no reference
Signed-off-by: Will Stevens <williamstevens@gmail.com>
writeIfNotHere requires an array of strings, not a string
* pr/1456:
writeIfNotHere requires an array of strings, not a string
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Disable NioTestHistorically NioTest has caused issue in CI environment and several developer machines due to network requirements which could be disabled by firewall or security enforcers such as selinux. This disables the test once again using a historic commit 881a6e1
Signed-off-by: Rohit Yadav <rohit.yada
To build and just run this test: mvn clean install -pl utils -Dtest=NioTest
* pr/1538:
utils: Disable NioTest
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CLOUDSTACK-9265 cleanup around httpclient versionssome cleanup done
- replaced HttpStatus from org.apache.commons.httpclient with that from org.apache.http
- removed unthrown HttpException
- left auto reformat in place
* pr/1385:
CLOUDSTACK-9265 cleanup around httpclient versions
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CLOUDSTACK-9373: Class methods over-shawdowing instance methodsWe have some methods in base.py that are named the same.
Per my findings below, Python methods in a class should not be named the same even if one is a class method and the other is an instance method.
The solution discussed on dev@ is to remove the instance versions (reason listed in e-mail text, which is listed in JIRA ticket).
https://issues.apache.org/jira/browse/CLOUDSTACK-9373
* pr/1528:
CLOUDSTACK-9373: Removing a few instance methods where there are class methods that are overshadowing them
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Notify listeners when a host has been added to a cluster, is about to be removed from a cluster, or has been removed from a cluster
This PR addresses the following JIRA ticket:
https://issues.apache.org/jira/browse/CLOUDSTACK-8813
The problem is that there needs to be notifications sent when a host is added to, about to be removed from, and removed from a cluster.
Such notifications can be used for many purposes. For example, it can allow storage plug-ins to update ACLs on their storage systems. Also, it can allow us to clean up IQNs from ESXi hosts that are no longer needed.
* pr/816:
CLOUDSTACK-8813: Notify listeners when a host has been added to a cluster, is about to be removed from a cluster, or has been removed from a cluster
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Rohit Yadav
Romain Dartigues
Will Stevens
Wei Zhou
Aaron Brady
SudharmaJain
Mike Tutkowski
Syed
Remi Bergsma
Marc-Aurèle Brothier
Milamber