Uses listViewDataProvider to implement pagination for listing templates and ISOs
in the zones tab. Dedupes isos and templates in the list views.
This closes#740
(cherry picked from commit 26700fbe76)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-8758: Handle redirects in communication with NXS controller (a.k.a. Nicira NVP)When an NSX controller node is part of a cluster it will redirect API calls to the master controller. Because the ACS management server does not follow such redirects, if there is a change of master within a NSX cluster, the NSX device (a.k.a. NiciraNvp) needs to be reconfigured (via the management server DB).
The goal of this PR is to enable ACS management server to follow HTTP redirects sent by NSX controllers. However, other changes were made to the cloud-utils module that provides the REST client that the NSX plugin uses.
Cosmetic changes:
* Upgrade maven module structure for cloud-utils and cloud-plugin-netowkr-nvp to comply with maven default
* Several refactorings on both modules to consistently format the code, remove unused code, declare final when possible, remove auto generated comments, etc
Other changes:
* Upgrade HTTP library used in REST client to version 4.5 of Apache HTTP Components
* Implement generic REST client that supports HTTP redirects
* Implement NSX specific REST client
* Simplify NSX api implementation
* Previously existing unit tests for both the REST client and NSX api were either maintained in the same test classes, moved to new test classes (because code under tests also moved), or removed (because code under tests was also removed)
* New Marvin tests for NSX controllers
Testing:
* Ran all unit tests present in the project
* Ran Java Integration tests for NSX api targeting both a master and a slave controller
* Ran new Marvin test for NSX controller
* Manual inspection of logs to confirm redirection is taking place
* pr/737:
Use NSX specific RestClient in API implementation (NiciraNvpApi)
Refactor NSX api implementation (NiciraNvpApi)
Add NSX specific RestClient implementation
Delegate HTTP protocol activity in RESTServiceConnector to RestClient
Add basic RestClient implentation based on HTTP Components 4.5
Code clean up in cloud-plugin-network-nvp project
Code clean up in cloud-utils project
Refactor cloud-plugin-network-nvp project into Maven default structure
Refactor cloud-utils project into Maven default structure
Fix unit-test library dependencies
Add Marvin test for Nicira NVP plugin
Create Nicira NVP devices and enable plugin at deploy
Signed-off-by: Remi Bergsma <github@remi.nl>
- Add -noverify JVM arg to surefire plugin, to allow Powermockito to
de-encapsulate private methods
- Add dependency on cloud-utils test-jar to use custom HttpRequest
matchers
- All HTTP protocol activities are now handled by RestClient
- This service is now only responsible for creating requests, and
dispatching them to the client
- Provides a Simple API for creating, updating, retrieving and deleting
objects
- Upgrade version of HTTP Components to 4.5
- Add helper class to create Http clients
- Add helper class to build http requests
- Add enum with the different Http Methods
- Add constants class for HTTP related values
- XML formatting
- Fix license header
- Reorder hamcrest, junit, mockito and powermock dependencies
* Since several libraries include a subset of hamcrest classes,
hamcrest should be declared first in the pom, inorder for its
classes to be loaded first by the JVM
- Depdend on hamcrest-all and exclude hamcrest classes from other
libraries
- Support for ScaleVmCommand/NetworkRulesVmSecondaryIpCommand in resource layer
- Added support for scaling up a running VM in simulator
- Fixed some method names not following convention
CLOUDSTACK-8685: Fix for missing defaultroute on VPC router https://issues.apache.org/jira/browse/CLOUDSTACK-8685
Refactored CsRoute and CsAddress to:
- add methods to add and verify the defaultroute
- renamed methods flush, add, and routeTable to flush_table, add_route, add_table respectively.
Added some suboptimal unit tests to TestCsRoutes
* pr/738:
Fixed defaultroute on VPC routers in CsRoute and CsAddress Refactored CsRoute
pep8 cleanup
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8443: Support CentOS 7 as KVM hypervisorThis adds support for CentOS 7 as a KVM hypervisor. As discussed in the Jira issue, the root cause of the issue was the `cloudstack-setup-agent` script that added this line to `/etc/libvirt/qemu.conf`:
`cgroup_controllers=["cpu"]`
On CentOS 6 this 'works', but since by default no cgroups are mounted, this setting is ignored by `libvirtd`. On CentOS 7 is does not work, because the `cpu` and `cpuacct` are 'co-mounted'. This simpy means you cannot use one without the order. The config line above instructs `libvirtd` to only use the `cpu` cgroup, which it can't.
There was code added to make sure the `cpu` cgroup was not co-mounted. It did this with a shell script, but after a reboot it was gone. It wasn't reliable and not even a solution at all. I went ahead and removed the code, because the co-mounted cgroups work fine. The real issue was the config line we put in `/etc/libvirt/qemu.conf` so much better to fix that instead.
The default for `libvirtd` is to use only cgroups that are mounted. When you remove the 'cgroup_controllers' line, the default kicks in. According to the `qemu.conf` file, the default is:
`cgroup_controllers = ["cpu", "devices", "memory", "blkio", "cpuacct", "net_cls"]`
Then again, if it's not mounted, it's not used. So, that's a nice way to control it.
This works just fine for both RHEL/CentOS 6 and RHEL/CentOS 7.
Along the debugging, I came across some small issues here and there which I also fixed in this PR.
If you test this, be sure to build RPMs from this branch and install those on the KVM hypervisor.
At Schuberg Philis we run KVM hypervisors on CentOS 7 for months with these fixes. Now I took the time to fix the issues in CloudStack, rather than work around them.
@bhaisaab @karuturi could you please review this? Thanks!
Please note:
The next step is to fix CLOUDSTACK-8625 (Systemd profile for KVM Agent) as currently a sysvinit script is still used. Although it works, we do not have any systemd benefits. As it is a separate issue, I'll try to address it in a separate PR.
* pr/731:
CLOUDSTACK-8443: detect CentOS 7.x as RHEL 7 alike
CLOUDSTACK-8443: mention the correct logfile
CLOUDSTACK-8443: display the right hypervisor type
CLOUDSTACK-8443: don't try to fix co-mounted cgroups
Signed-off-by: Remi Bergsma <github@remi.nl>
Git PR merge script improvementsProcessed feedback from @bhaisaab and (a colleague of) @karuturi
- make the help message better (I also updated Confluence)
- make sure we can use it on other repo's as well (Documentation and such)
Tested it and works fine for me.
* pr/734:
fixed hardcoded merge message location
fix bug where the countdown would be on multiple lines
make it work with any Apache repository mirrored on GitHub
Improve error/help message in case of missing git remote
Signed-off-by: Remi Bergsma <github@remi.nl>
Example of problem:
ATTENTION: Merging pull request #731 from remibergsma/centos7-kvm into 'master' branch in 5 seconds. CTRL+c to abort..
-n 5
-n 4
-n 3
-n 2
-n 1
-n 0
Should be compatible with more environments if printf is used instead.
This setting works on CentOS 6 / RHEL 6 but does nothing, as
"cpu" cgroup is not mounted. On CentOS 7 / RHEL 7 systemd does
mount cgroups and "cpu" is co-mounted with "cpuacc". Hence, if
we specify "cpu" then this results in an error because it can
only use them both, or none.
By removing the setting, we rely on the default of qemu, which
is:
cgroup_controllers = ["cpu", "devices", "memory", "blkio", "cpuacct", "net_cls"]
Only if they are really mounted, they will be used. So, this will
work on both version 6 and 7.
The 'fix script' didn't work well, as after a reboot you'd still have qemu
throwing errors. Now we can handle the co-mountedcgroups.
This patch makes it possible to expose VM UUID to subsystems, this can be
useful for implementing VM Snapshots for KVM in future.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Fix for the NicVO.java regression.Renamed set*() methods to correct naming.
* pr/726:
Fix for the NicVO.java regression.
Signed-off-by: Remi Bergsma <github@remi.nl>
This is caused by serialization failure for VmWorkMigrateForScale object. Replaced DeployDestination member
present in VmWorkMigrateForScale with serializable types.
Removed duplicate code in CitrixResourceBase and its subclassesRemoved unnecessary duplicated code by having the body of the getPatchFiles method only in the CitrixResourceBase superclass. Given that all of its implementations consisted of the same code except for the path which contains the necessary file for that implementation. An abstract method getPatchFilePath was created in the CitrixResourceBase superclass so that each implementation may return the path containing the specific file needed by that implementation.
Test cases were created for each implementation, simple as they may be. One assert is made to verify that the path returned by each implementation corresponds to the path that was previously specified on each getPatchFiles implementation.
* pr/700:
Removed duplicate code in CitrixResourceBase.getPatchFiles
Signed-off-by: Remi Bergsma <github@remi.nl>
Add scripts to merge PRs and release fixes forwardAs it is now, additions to CloudStack need to go through a GitHub PR and, if accepted, merged to the Apache remote (not in GitHub). This means that merging a PR for CloudStack is a bit more involved than merging a normal GitHub PR. In addition to that, a PR that fixes something in an older release needs to be merged against a release branch, and then forward merged up to master (if applicable).
To help with these two kinds of merges, @remibergsma, @karuturi, @DaanHoogland and myself, have created two scripts (contributed in this PR).
One script (**git-pr**) helps in merging a PR from GitHub to the Apache remote. The script will read the PR info using GitHub's API and will create a merge commit that contains the same commit message a normal GitHub PR merge commit, plus, as extra info, a list of commits that are being merged.
The other script (**git-fwd-merge**) will create a specific merge commit message that indicates a forward merge, and lists the commits being merged.
This Apache wiki [page](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61311655) describes how to use these two scripts, and points to an even more detailed [page](https://github.com/schubergphilis/cloudstack-git-tree-manipulation/wiki) that explains the merge scenarios step-by-step.
By the way, beacause both scripts follow the naming convention for git tools, as long as they are in your "path" you can execute them as simply as "git pr" and "git fwd-merge". But be aware that, in order to use these scripts, you should make sure to remove any aliases that can cause a name clash.
* pr/721:
Add git merge scripts
Signed-off-by: Remi Bergsma <github@remi.nl>
This reverts commit 180afe52e5.
This broke the build on master:
master build broken with the below error http://jenkins.buildacloud.org/job/build-master-slowbuild/2101/consoleText
```
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/jenkins/acs/workspace/build-master-slowbuild/plugins/hypervisors/xenserver/test/com/cloud/hypervisor/xenserver/resource/wrapper/xenbase/CitrixRequestWrapperTest.java:[1581,51] error: constructor CreateVMSnapshotCommand in class CreateVMSnapshotCommand cannot be applied to given types;
[ERROR] required: String,String,VMSnapshotTO,List<VolumeObjectTO>,String
found: String,VMSnapshotTO,List<VolumeObjectTO>,String
reason: actual and formal argument lists differ in length
/home/jenkins/acs/workspace/build-master-slowbuild/plugins/hypervisors/xenserver/test/com/cloud/hypervisor/xenserver/resource/wrapper/xenbase/CitrixRequestWrapperTest.java:[1623,53] error: no suitable constructor found for RevertToVMSnapshotCommand(String,VMSnapshotTO,List<VolumeObjectTO>,String)
[INFO] 2 errors
[INFO] -------------------------------------------------------------
```
This was PR #717 towards 4.5
This patch makes it possible to expose VM UUID to subsystems, this can be
useful for implementing VM Snapshots for KVM in future.
This was PR #717 towards 4.5
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 0062ff2672)
Signed-off-by: Remi Bergsma <github@remi.nl>
Rohit Yadav
Wei Zhou
Koushik Das
Remi Bergsma
Miguel Ferreira
Michael Andersen
shweta agarwal
Rajani Karuturi
Boris Schrijver
radu-stefanache