Use available context to filter a metrics view based on zone, cluster, host
in the context object. This fixes metrics view filtering when metrics view is
viewed via Zone->Compute and Storage-> for a resource.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9535: [API] listVMSnapshots improvement### Improvements
- Include missing fields in response: `account`, `domain`, `domainid`, `zoneid`
- Display total count of snapshots, not depending on page size
### Example
After creating 2 vm snapshots for a given vm, and making this API call: `command=listVMSnapshot&listAll=true&virtualmachineid=c8531ef8-8502-4b42-b1c5-c52ace0e7801&_=1475516598524&pagesize=1&page=1` we get this response:
```
<listvmsnapshotresponse cloud-stack-version="4.9.1.0-SNAPSHOT">
<count>2</count>
<vmSnapshot>
<id>88f7416a-8799-4245-99c6-c707cfbe6f47</id>
<name>i-2-10482-VM_VS_20161003174340</name>
<state>Ready</state>
<description>2</description>
<displayname>testsnap2</displayname>
<zoneid>0d074f25-ed31-482f-8bc5-44c9314fc417</zoneid>
<virtualmachineid>c8531ef8-8502-4b42-b1c5-c52ace0e7801</virtualmachineid>
<parent>24e44fe5-5f2e-4d35-a8f8-109b644a04e0</parent>
<parentName>testsnap</parentName>
<current>true</current>
<type>Disk</type>
<created>2016-10-03T10:43:40-0700</created>
<account>admin</account>
<domainid>5a7ffa07-3fca-11e5-9c45-005056ad45b7</domainid>
<domain>ROOT</domain>
</vmSnapshot>
</listvmsnapshotresponse>
```
**NOTES:** As in `listVirtualMachines`, despite `pagesize=1`, `count` field shows total snapshots count for given vm. Also, `account`, `domain`, `domainid`, `zoneid` fields are listed
* pr/1702:
CLOUDSTACK-9535: [API] listVMSnapshots improvement
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
This allows admins/users to destroy a VR that is in running state from the UI.
This action is possible via cloudmonkey/APIs.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Switches to macchinina as template for VM in the tests
- Modifies the ostype of the macchinina template to 'Other Linux (64-bit)'
- Check template download status, fixes Nonetype iterable issue
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
As per previous discussions and ticket, a template deletion may result in failure
(exception thrown) for templates that are not properly downloaded. The tearDown
method, a template may be tried for deletion but on failure we may ignore it
as account deletion/tearDown would retry to cleanup resource owned by the account.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- test_01_test_vm_volume_snapshot not supported for Xen, tests keep failing
- Skip snapshot tests for centos6/kvm as snapshot is not supported by older
qemu-img versions
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
SSH to VR for vmware, goes via the mgmt server and uses ssh keys at
/var/cloudstack path. Add suitable checks to tests failing on vmware.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Handle case where physical network instance does not have vlan attribute
- Handle case where listIso response may not have status attribute
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
resulting in internal LB vm not come up
parsing cmd_line to create 'ips' data bag, never handled internal lb vm, but still
worked due to another bug. support for internal lb vm is added with this fix
The quota integration test requires special setup and is moved to plugins
directory as in 4.9 and master branch.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
[4.9/lts] CLOUDSTACK-9364: Add Ubuntu 16.04 as a guest OSAdds Ubuntu 16.04 as a supported guest os. This allows users to select
the OS when creating a template etc.
SQL queries tested and adapted from https://issues.apache.org/jira/browse/CLOUDSTACK-9364
* pr/1696:
CLOUDSTACK-9364: Add Ubuntu 16.04 as a guest OS
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Unhides snapshot.backup.rightafter from global configuration
If snapshot.backup.rightafter is set to false (defaults to true), snapshots are
not backed up to secondary storage
Adds Ubuntu 16.04 as a supported guest os. This allows users to select
the OS when creating a template etc.
Note: As XenServer 6.5 does not have 16.04 in its list of known Ubuntu releases,
as a workaround 16.04 guest os refers to 14.04 for XenServer 6.5.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Both host and path could have been NULL which causes the StatsCollector
no to start properly.
By checking if the Strings are not Empty or Null we make sure the StatsCollector
always runs and does not prevent the Management Server from starting.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
Conflicts:
server/src/com/cloud/server/StatsCollector.java
CLOUDSTACK-9480, CLOUDSTACK-9495 fix egress rule incorrect behaviorWhen 'default egress policy' is set to 'allow' in the network offering, any egress rule that is added will 'deny' the traffic overriding the default behaviour.
Conversely, when 'default egress policy' is set to 'deny' in the network offering, any egress rule that is added will 'allow' the traffic overriding the default behaviour.
While this works for 'tcp', 'udp' as expected, for 'icmp' protocol its always set to ALLOW. This patch keeps all protocols behaviour consistent.
Results of running test/integration/component/test_egress_fw_rules.py. With out the patch test_02_egress_fr2 test was failing. This patch fixes the test_02_egress_fr2 scenario.
-----------------------------------------------------------------------------------------------------
Test By-default the communication from guest n/w to public n/w is NOT allowed. ... === TestName: test_01_1_egress_fr1 | Status : SUCCESS ===
ok
Test By-default the communication from guest n/w to public n/w is allowed. ... === TestName: test_01_egress_fr1 | Status : SUCCESS ===
ok
Test Allow Communication using Egress rule with CIDR + Port Range + Protocol. ... === TestName: test_02_1_egress_fr2 | Status : SUCCESS ===
ok
Test Allow Communication using Egress rule with CIDR + Port Range + Protocol. ... === TestName: test_02_egress_fr2 | Status : SUCCESS ===
ok
Test Communication blocked with network that is other than specified ... === TestName: test_03_1_egress_fr3 | Status : SUCCESS ===
ok
Test Communication blocked with network that is other than specified ... === TestName: test_03_egress_fr3 | Status : SUCCESS ===
ok
Test Create Egress rule and check the Firewall_Rules DB table ... === TestName: test_04_1_egress_fr4 | Status : SUCCESS ===
ok
Test Create Egress rule and check the Firewall_Rules DB table ... === TestName: test_04_egress_fr4 | Status : SUCCESS ===
ok
Test Create Egress rule and check the IP tables ... SKIP: Skip
Test Create Egress rule and check the IP tables ... SKIP: Skip
Test Create Egress rule without CIDR ... === TestName: test_06_1_egress_fr6 | Status : SUCCESS ===
ok
Test Create Egress rule without CIDR ... === TestName: test_06_egress_fr6 | Status : SUCCESS ===
ok
Test Create Egress rule without End Port ... === TestName: test_07_1_egress_fr7 | Status : EXCEPTION ===
ERROR
Test Create Egress rule without End Port ... === TestName: test_07_egress_fr7 | Status : SUCCESS ===
ok
Test Port Forwarding and Egress Conflict ... SKIP: Skip
Test Port Forwarding and Egress Conflict ... SKIP: Skip
Test Delete Egress rule ... === TestName: test_09_1_egress_fr9 | Status : SUCCESS ===
ok
Test Delete Egress rule ... === TestName: test_09_egress_fr9 | Status : SUCCESS ===
ok
Test Invalid CIDR and Invalid Port ranges ... === TestName: test_10_1_egress_fr10 | Status : SUCCESS ===
ok
Test Invalid CIDR and Invalid Port ranges ... === TestName: test_10_egress_fr10 | Status : SUCCESS ===
ok
Test Regression on Firewall + PF + LB + SNAT ... === TestName: test_11_1_egress_fr11 | Status : SUCCESS ===
ok
Test Regression on Firewall + PF + LB + SNAT ... === TestName: test_11_egress_fr11 | Status : SUCCESS ===
ok
Test Reboot Router ... === TestName: test_12_1_egress_fr12 | Status : SUCCESS ===
ok
Test Reboot Router ... === TestName: test_12_egress_fr12 | Status : EXCEPTION ===
ERROR
Test Redundant Router : Master failover ... === TestName: test_13_1_egress_fr13 | Status : SUCCESS ===
ok
Test Redundant Router : Master failover ... === TestName: test_13_egress_fr13 | Status : SUCCESS ===
ok
-----------------------------------------------------------------------------------------------------
* pr/1666:
fix egress rule incorrect behavior
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
CLOUDSTACK-9480: Egress Firewall: Incorrect use of Allow/Deny for ICMP
fix ensures, ICMP, TCP, UDP are handled similalry w.r.t egress rule action
CLOUDSTACK-9495: Egress rules functionalty broken when protocol=all specified
when protocol=all specified, CIDR was ignored. Fix ensures if CIDR is specified
its always used in configuring iptable rules
2 new test cased to test /32 CIDR
Changes database upgrade script names to be consistent for the 4.9.1.0 release * Changes the names of the schema-490to491* scripts to
schema-490to4910*
* Changes the name of the Upgrade490to491 class to Upgrade490to4910
* Modifies the Marvin setup.py script to use version 4.9.1.0-SNAPSHOT
/cc @rhtyd @karuturi
* pr/1665:
Renames of 4.9.0->4.9.1.0 upgrade scripts to match the four position version scheme
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* Renames schema-490to491*.sql to schema490to4910*.sql
* Renames the Upgrade490to491 class to Upgrade490to4910
* Removes the unused s_logger contant from Upgrade490to4910
* Updates the version in tools/marvin/setup to 4.9.1.0-SNAPSHOT
Fix a quote issue with Spanish L10N (from transifex translation)This fix is for the 4.8 branch.
* pr/1636:
Fix a quote issue with Spanish L10N (from transifex translation)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Updating pom.xml version numbers for release 4.8.2.0-SNAPSHOTOften, patch and security releases do not require schema migrations or
data migrations. However, if an empty upgrade class and associated
scripts are not defined, the upgrade process will break. With this
change, if a release does not have an upgrade, a noop DbUpgrade is added
to the upgrade path. This approach allows the upgrade to proceed and
for the database to properly reflect the installed version. This change
should make the release process simpler as RMs no longer need to
rememeber to create this boilerplate code when starting a new release.
Beginning with the 4.8.2.0 and 4.9.1.0 releases, the project will
formally adopt a four (4) position release number to properly accomodate
rekeases that contain only CVE fixes. The DatabaseUpgradeChecker and
Version classes made assumptions that they would always parse and
compare three (3) position version numbers. This change adds the
CloudStackVersion value object that supports both three (3) and four (4)
version numbers. It encapsulates version comparsion logic, as well as,
the rules to allow three (3) and four (4) to interoperate.
* Modifies DatabaseUpgradeChecker to handle derive an upgrade path for
a version that was not explicitly specified. It determines the
releases the first release before it with database migrations and uses
that list as the basis for the list for version being calculated. A
noop upgrade is then added to the list which causes no schema changes
or data migrations, but will update the database to the version.
* Adds unit tests for the upgrade path calculation logic in
DatabaseUpgradeChecker
* Removes dummy upgrade logic for the 4.8.2.0 introduced in previous
versions of this patch
* Introduces the CloudStackVersion value object which parses and
compares three (3) and four (4) position version numbers. This class
is intended to replace com.cloud.maint.Version.
* Adds the junit-dataprovider dependency -- allowing test data to be
concisely generated separately from the execution of a test case.
Used extensively in the CloudStackVersionTest.
Signed-off-by: John Burwell <meaux@cockamamy.net>
/cc @rhtyd @karuturi
* pr/1654:
Adds support for four position versions and optional db upgrades
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
[LTS/blocker] CLOUDSTACK-6432: Prevent DNS reflection attacksCLOUDSTACK-6432: Prevent DNS reflection attacks
DNS on VR should not be publically accessible as it may be prone to DNS
amplification/reflection attacks. This fixes the issue by only allowing VR
DNS (port 53) to be accessible from guest network cidr, as per the fix in:
https://issues.apache.org/jira/browse/CLOUDSTACK-6432
- Only allows guest network cidrs to query VR DNS on port 53.
- Includes marvin smoke test that checks the VR DNS accessibility checks from
guest and non-guest network.
- Fixes Marvin sshClient to avoid using ssh agent when password is provided,
previous some environments may have seen 'No existing session' exception without
this fix.
- Adds a new dnspython dependency that is used to perform dns resolutions in the
tests.
Due to repository commit issues I've created this PR, based on #1653 .
/cc @jburwell @karuturi @NuxRo @ustcweizhou @wido and others
* pr/1663:
CLOUDSTACK-6432: Prevent DNS reflection attacks
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
DNS on VR should not be publically accessible as it may be prone to DNS
amplification/reflection attacks. This fixes the issue by only allowing VR
DNS (port 53) to be accessible from guest network cidr, as per the fix in:
https://issues.apache.org/jira/browse/CLOUDSTACK-6432
- Only allows guest network cidrs to query VR DNS on port 53.
- Includes marvin smoke test that checks the VR DNS accessibility checks from
guest and non-guest network.
- Fixes Marvin sshClient to avoid using ssh agent when password is provided,
previous some environments may have seen 'No existing session' exception without
this fix.
- Adds a new dnspython dependency that is used to perform dns resolutions in the
tests.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
[CLOUDSTACK-9444] Fix a little issue from PR1610 if the db.properties file hasn't EOL character at the end of file
And some improvements about the dir/file using variables
cc @wido @rhtyd
* pr/1621:
Fix a little issue from PR1610 if the db.properties file hasn't EOL character at the end of file And some improvements about the dir/file using variables
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Often, patch and security releases do not require schema migrations or
data migrations. However, if an empty upgrade class and associated
scripts are not defined, the upgrade process will break. With this
change, if a release does not have an upgrade, a noop DbUpgrade is added
to the upgrade path. This approach allows the upgrade to proceed and
for the database to properly reflect the installed version. This change
should make the release process simpler as RMs no longer need to
rememeber to create this boilerplate code when starting a new release.
Beginning with the 4.8.2.0 and 4.9.1.0 releases, the project will
formally adopt a four (4) position release number to properly accomodate
rekeases that contain only CVE fixes. The DatabaseUpgradeChecker and
Version classes made assumptions that they would always parse and
compare three (3) position version numbers. This change adds the
CloudStackVersion value object that supports both three (3) and four (4)
version numbers. It encapsulates version comparsion logic, as well as,
the rules to allow three (3) and four (4) to interoperate.
* Modifies DatabaseUpgradeChecker to handle derive an upgrade path for
a version that was not explicitly specified. It determines the
releases the first release before it with database migrations and uses
that list as the basis for the list for version being calculated. A
noop upgrade is then added to the list which causes no schema changes
or data migrations, but will update the database to the version.
* Adds unit tests for the upgrade path calculation logic in
DatabaseUpgradeChecker
* Removes dummy upgrade logic for the 4.8.2.0 introduced in previous
versions of this patch
* Introduces the CloudStackVersion value object which parses and
compares three (3) and four (4) position version numbers. This class
is intended to replace com.cloud.maint.Version.
* Adds the junit-dataprovider dependency -- allowing test data to be
concisely generated separately from the execution of a test case.
Used extensively in the CloudStackVersionTest.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
test/integration: fix tearDown order in list_acl_ teststest/integration: fix tearDown order in list_acl_ tests
In several of the list_acl_tests, the tests run for simulator only where
in the (class) setup domains and accounts are created for the test. When the
tests end the (class) teardown methods would delete and remove these resources.
Due to dependence of one of the resources on the other, domain2 on domain1,
domain2 needs to be removed/cleaned up before domain1. Due to this issue,
several Travis test runs have failed in the past such as:
https://travis-ci.org/apache/cloudstack/jobs/152610967https://travis-ci.org/apache/cloudstack/jobs/152610968
Changing the order of cleanup fixes the tests.
/cc @jburwell @karuturi
The fix is specific to tests that run 'only' on simulator with Travis. A passing Travis run should be enough to validate the changes.
* pr/1648:
test/integration: fix tearDown order in list_acl_ tests
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
[lts] CLOUDSTACK-9462: Systemd support for Ubuntu 16.04Created this based on @wido 's origin PR #1541 .
Requesting for review and testing -- @jburwell @karuturi @wido @vincentbernat
@wido I think this change only brings systemd support to agent and usage packages, or does cloudstack-management pkg has systemd support too?
@blueorangutan package
- systemd: Add a /etc/sysconfig/cloudstack-* file
This allows users to easily override variables passed to Java when
starting up.
It also creates a foundation for sharing the systemd service profile
between CentOS and Ubuntu since it only requires the environment file
to be changed.
- deb: Add Ubuntu 16.04 support
Ubuntu 16.04 differs from Ubuntu 14.04 in a few ways:
- systemd instead of sysvinit / upstart
- Java 8 support
The packaging now detects on which distribution it is being
build and based on that it installs different files in the
packages, but it also changes the Dependencies.
* pr/1647:
CLOUDSTACK-9462: Refactor systemd scripts
CLOUDSTACK-9462: Systemd support for Ubuntu 16.04
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Refactors and unifies usage of systemd script and default files across
CentOS and Ubuntu/Debian packaging system.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Rohit Yadav
Wei Zhou
Rajani Karuturi
Murali Reddy
Boris
nvazquez
Paul Angus
Nathan Johnson
Wido den Hollander
Abhinandan Prateek
Daniel Carbone
John Burwell
nvazquez
Milamber