Commit Graph

1682 Commits

Author SHA1 Message Date
Nick Livens 8d4dc81223 CLOUDSTACK-9402 : Support for underlay features (Source & Static NAT to underlay) in Nuage VSP plugin
CLOUDSTACK-9402 : Marvin tests for Source NAT and Static NAT features verification with NuageVsp (both overlay and underlay infra).

Co-Authored-By: Prashanth Manthena <prashanth.manthena@nuagenetworks.net>, Frank Maximus <frank.maximus@nuagenetworks.net>
2016-11-24 21:33:02 +01:00
Wido den Hollander 2a5f37c1b1
CLOUDSTACK-8715: Add channel to Instances for Qemu Guest Agent
This commit adds a additional VirtIO channel with the name
'org.qemu.guest_agent.0' to all Instances.

With the Qemu Guest Agent the Hypervisor gains more control over the Instance if
these tools are present inside the Instance, for example:

* Power control
* Flushing filesystems
* Fetching Network information

In the future this should allow safer snapshots on KVM since we can instruct the
Instance to flush the filesystems prior to snapshotting the disk.

More information: http://wiki.qemu.org/Features/QAPI/GuestAgent

Keep in mind that on Ubuntu AppArmor still needs to be disabled since the default
AppArmor profile doesn't allow libvirt to write into /var/lib/libvirt/qemu

This commit does not add any communication methods through API-calls, it merely
adds the channel to the Instances and installs the Guest Agent in the SSVMs.

With the addition of the Qemu Guest Agent channel a second channel appears in /dev
on a SSVM as a VirtIO port.

The order in which the ports are defined in the XML matters for the naming inside
the SSVM VM and by not relying on /dev/vportXX but looking for a static name the
SSVM still boots properly if the order in the XML definition is changed.

A SSVM with both ports attached will have something like this:

  root@v-215-VM:~# ls -l /dev/virtio-ports
  total 0
  lrwxrwxrwx 1 root root 11 May 13 21:41 org.qemu.guest_agent.0 -> ../vport0p2
  lrwxrwxrwx 1 root root 11 May 13 21:41 v-215-VM.vport -> ../vport0p1
  root@v-215-VM:~# ls -l /dev/vport*
  crw------- 1 root root 251, 1 May 13 21:41 /dev/vport0p1
  crw------- 1 root root 251, 2 May 13 21:41 /dev/vport0p2
  root@v-215-VM:~#

In this case the SSVM port points to /dev/vport0p1, but if the order in the XML
is different it might point to /dev/vport0p2

By looking for a portname with a pre-defined pattern in /dev/virtio-ports we
do not rely on the order in the XML definition.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2016-11-23 16:01:08 +01:00
Nick Livens 62e858131f CLOUDSTACK-9321 : Multiple Internal LB rules (more than one Internal LB rule with same source IP address) are not getting resolved in the corresponding InternalLbVm instance's haproxy.cfg file
CLOUDSTACK-9321 : Adding component tests for VPC Network functionality - Internal LB rules

CLOUDSTACK-9321 : Extending Nuage VSP Internal LB Marvin tests

Co-Authored-By: Prashanth Manthena <prashanth.manthena@nuagenetworks.net>, Frank Maximus <frank.maximus@nuagenetworks.net>
2016-11-22 15:51:22 +01:00
Prashanth Manthena a97d54f306 CLOUDSTACK-9401 : Marvin tests for Internal DNS verification with NuageVsp 2016-11-15 16:53:11 +01:00
Syed f46651e672 Support Backup of Snapshots for Managed Storage
This PR adds an ability to Pass a new parameter, locationType,
    to the “createSnapshot” API command. Depending on the locationType,
    we decide where the snapshot should go in case of managed storage.

    There are two possible values for the locationType param

    1) `Standard`: The standard operation for managed storage is to
    keep the snapshot on the device. For non-managed storage, this will
    be to upload it to secondary storage. This option will be the
    default.

    2) `Archive`: Applicable only to managed storage. This will
    keep the snapshot on the secondary storage. For non-managed
    storage, this will result in an error.

    The reason for implementing this feature is to avoid a single
    point of failure for primary storage. Right now in case of managed
    storage, if the primary storage goes down, there is no easy way
    to recover data as all snapshots are also stored on the primary.
    This features allows us to mitigate that risk.
2016-10-30 23:19:58 -06:00
Rohit Yadav ccd1734dc4 Merge branch '4.9' 2016-10-21 10:15:29 +05:30
Rohit Yadav 9b9b49e10b Merge branch '4.8' into 4.9 2016-10-21 10:14:16 +05:30
Rajani Karuturi 8d1f1a69a0 Merge pull request #1464 from sanju1010/vcenter
[CLOUDSTACK-9337]Enhance vcenter.py to create data center in vcenter server automaticallyThese changes have been made to support vmware deployments in CI.
For CI to create cloudstack setup with vmware, it is required to create datacenter, cluster and hosts in vcenter server before adding in cloudstack. Added few methods in vcenter.py to perform these tasks.

Please refer to CLOUDSTACK-9337 for more details.

* pr/1464:
  [CLOUDSTACK-9337]Enhance vcenter.py to created data center in vcenter server automatically (Programmatically)

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2016-10-13 14:50:53 +05:30
Rohit Yadav 0c146e14c6 CLOUDSTACK-9532: Use macchinina as a template for failing tests
- Switches to macchinina as template for VM in the tests
- Modifies the ostype of the macchinina template to 'Other Linux (64-bit)'
- Check template download status, fixes Nonetype iterable issue

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-10-06 16:41:22 +05:30
Murali Reddy 1f27874eaf CLOUDSTACK-9522: Check for available attribute in marvin response
- Handle case where physical network instance does not have vlan attribute
- Handle case where listIso response may not have status attribute

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-10-05 15:11:00 +05:30
Milamber 60edc8af46 CLOUDSTACK-9513 Migrate transifex workflow and format to json
Move the localization resource files from Java Properties format to JSON Key-Value format
Change the Transifex sync script to handle JSON resource files instead of Properties files
Update the README
Remove old version from the Transifex configuration file
Remove unused gen-l10n.py script and update the ui/pom.xml to remove the execution of this script
2016-10-03 19:47:06 +01:00
Rohit Yadav fbf6a9df18 Merge pull request #1619 from milamberspace/L10N-update-Master-20160730
Add the Transifex config for next version of CS (4.10)

* pr/1619:
  Add the Transifex config for next version of CS (4.10)

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-09-29 18:33:57 +05:30
Rohit Yadav 16913a9822 CLOUDSTACK-9842: Make UI JSP Free
We use some JSP file just for translation of strings in the UI. This is
achievable purely in JavaScript. This removes those JSPs, simplifies
translation usage and workflow (purely JS based). The l10n js (dictionary)
files are generated from existing messages.properties files during client-ui
code generation phase.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-09-28 12:26:43 +05:30
Milamber a66ebbec61 Add the Transifex config for next version of CS (4.10) 2016-09-20 13:13:45 +01:00
John Burwell eabda0eb55 Renames of 4.9.0->4.9.1.0 upgrade scripts to match the four position version scheme
* Renames schema-490to491*.sql to schema490to4910*.sql
    * Renames the Upgrade490to491 class to Upgrade490to4910
    * Removes the unused s_logger contant from Upgrade490to4910
    * Updates the version in tools/marvin/setup to 4.9.1.0-SNAPSHOT
2016-09-01 21:19:29 -04:00
Rohit Yadav 1d9735c346
Merge branch '4.9' 2016-08-30 22:44:33 +05:30
Rohit Yadav 14504dc7e3 CLOUDSTACK-6432: Prevent DNS reflection attacks
DNS on VR should not be publically accessible as it may be prone to DNS
amplification/reflection attacks. This fixes the issue by only allowing VR
DNS (port 53) to be accessible from guest network cidr, as per the fix in:
https://issues.apache.org/jira/browse/CLOUDSTACK-6432

- Only allows guest network cidrs to query VR DNS on port 53.
- Includes marvin smoke test that checks the VR DNS accessibility checks from
  guest and non-guest network.
- Fixes Marvin sshClient to avoid using ssh agent when password is provided,
  previous some environments may have seen 'No existing session' exception without
  this fix.
- Adds a new dnspython dependency that is used to perform dns resolutions in the
  tests.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-30 22:39:33 +05:30
John Burwell 8d11511b1f Adds support for four position versions and optional db upgrades
Often, patch and security releases do not require schema migrations or
data migrations.  However, if an empty upgrade class and associated
scripts are not defined, the upgrade process will break.  With this
change, if a release does not have an upgrade, a noop DbUpgrade is added
to the upgrade path.  This approach allows the upgrade to proceed and
for the database to properly reflect the installed version.  This change
should make the release process simpler as RMs no longer need to
rememeber to create this boilerplate code when starting a new release.

Beginning with the 4.8.2.0 and 4.9.1.0 releases, the project will
formally adopt a four (4) position release number to properly accomodate
rekeases that contain only CVE fixes.  The DatabaseUpgradeChecker and
Version classes made assumptions that they would always parse and
compare three (3) position version numbers.  This change adds the
CloudStackVersion value object that supports both three (3) and four (4)
version numbers.   It encapsulates version comparsion logic, as well as,
the rules to allow three (3) and four (4) to interoperate.

  * Modifies DatabaseUpgradeChecker to handle derive an upgrade path for
  a version that was not explicitly specified.  It determines the
  releases the first release before it with database migrations and uses
  that list as the basis for the list for version being calculated.  A
  noop upgrade is then added to the list which causes no schema changes
  or data migrations, but will update the database to the version.
  * Adds unit tests for the upgrade path calculation logic in
  DatabaseUpgradeChecker
  * Removes dummy upgrade logic for the 4.8.2.0 introduced in previous
  versions of this patch
  * Introduces the CloudStackVersion value object which parses and
  compares three (3) and four (4) position version numbers.  This class
  is intended to replace com.cloud.maint.Version.
  * Adds the junit-dataprovider dependency -- allowing test data to be
  concisely generated separately from the execution of a test case.
  Used extensively in the CloudStackVersionTest.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-30 13:32:32 +05:30
Rohit Yadav 11c90dfb3b Merge pull request #1599 from shapeblue/independent-marvin
Marvin: Fix codegenerator to work with API discoveryThis fixes Marvin cloudstackAPI generator to work with a live running mgmt server's api discovery.

* pr/1599:
  marvin: fix codegeneration against API discovery endpoint

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-23 15:35:52 +05:30
Rohit Yadav 9555492b4d Merge branch '4.9' 2016-08-23 14:16:53 +05:30
Rohit Yadav f13c224da1 Updating pom.xml version numbers for release 4.9.1.0-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-19 13:53:39 +05:30
Rohit Yadav 7a53feee22 marvin: fix codegeneration against API discovery endpoint
This makes the commands.xml based codegeneration equivalent to the
API discovery end point based discovery.

This fixes the fields that the (api discovery based) codegenerator should
produce in the generated python classes (cmd and response classes per
api/module). The issue was that the autogenerated cloudstackAPI differed between
api-based and apidocs-based code generation. With this fix the generated classes
match exactly thereby allowing us to go with either methods to generate
cloudstackAPI.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-19 13:43:48 +05:30
Will Stevens 740bd45be6 Updating pom.xml version numbers for release 4.8.2-SNAPSHOT
Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-08-15 10:31:09 -04:00
Will Stevens 007c9f0d3a Merge branch '4.8.1-RC20160808T1006' into 4.8 2016-08-15 10:28:33 -04:00
Rajani Karuturi 56a3526502 Merge release branch 4.9 to master
* 4.9:
  CLOUDSTACK-9447: fix build and upgrade to debian 7.11 iso
2016-08-09 15:51:15 +05:30
Rajani Karuturi bdc409c7a2 Merge pull request #1626 from shapeblue/systemvmtemplate-4dot9
[blocker] Fix systemvm template buildPrevious PR: https://github.com/apache/cloudstack/pull/1531

Fixes failing systemvmtemplate build.

* pr/1626:
  CLOUDSTACK-9447: fix build and upgrade to debian 7.11 iso

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2016-08-09 15:49:04 +05:30
Will Stevens a63db21d16 Updating pom.xml version numbers for release 4.8.1
Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-08-08 10:06:21 -04:00
Rajani Karuturi 2875af7112 Merge release branch 4.9 to master
* 4.9:
  server: give more memory to tests
  packaging: Marvin and integration-tests packages
2016-08-05 14:41:31 +05:30
Rajani Karuturi c03d035c8b Merge release branch 4.8 to 4.9
* 4.8:
  server: give more memory to tests
  packaging: Marvin and integration-tests packages
2016-08-05 14:30:45 +05:30
Rajani Karuturi a54a3b5cd5 Merge release branch 4.7 to 4.8
* 4.7:
  server: give more memory to tests
  packaging: Marvin and integration-tests packages
2016-08-05 14:29:16 +05:30
Rohit Yadav 2cddaf3d36 CLOUDSTACK-9447: fix build and upgrade to debian 7.11 iso
- Update base debian iso to version 7.11
- Upgrade ruby version to 2.3.0 (latest/stable)
- Fix Gemfile
- Update README
- Fix openswan pkg name with the same version
- Remove cloud-cleanup it's not available

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-05 12:40:07 +05:30
Will Stevens 62aa3b2bfa Updating pom.xml version numbers for release 4.10.0-SNAPSHOT
Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-07-29 10:11:34 -04:00
Will Stevens 227ff3884d Updating pom.xml version numbers for release 4.9.0
Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-07-25 16:56:04 -04:00
Rohit Yadav de041df74d packaging: Marvin and integration-tests packages
This introduces two new cloudstack packages: marvin and integration-tests.
The two packages will make it easier for CI systems to install Marvin for a
specific cloudstack release/build and run integration tests that are specific
for that version/build.

- maven: add explicit juniper-contrail-api maven repository
- marvin: build source distribution for both install and package mvn phases

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-07-20 16:31:23 +05:30
Ronald van Zantvoort 58bdb44135 SysVM: Cleanup and removal of old (and dangerous) config files
* ports.conf
* default & default-ssl sites
* SSL config in httpd.conf
* deprecated & dead setup_redundant_router in cloud-early-config
2016-06-07 13:03:10 +02:00
Ronald van Zantvoort e32cd1303a VR: consistent SSL setup, vhost is not an example, but a template 2016-06-07 13:03:10 +02:00
Rohit Yadav 91c9fc9f6e travis: use ipmitool from ubuntu repository
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-29 16:14:28 +05:30
Will Stevens b5606dcce8 Merge pull request #1553 from nlivens/mysql_driver_issue
Dynamically load drivers before creating our DB connectionsSolution to the mailing thread titled "MySQL : No suitable driver found for jdbc:mysql".
It doesn't harm that we explicitely load the MySQL driver, and for those which would use a commons-dbcp version < 1.4 this would fix it as well. Since JDBC 4.0, the JDBC driver can auto-register itself, but for some weird cases (like ours), it's not working. Therefore we need to explicitly load the JDBC driver.

* pr/1553:
  Dynamic loading of DB driver + support for other DB providers

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-25 23:00:42 -04:00
Prashanth Manthena 2ca0d9b0e1 Nuage VSP : Extending Marvin test coverage 2016-05-24 11:12:19 +02:00
Nick Livens 02c5d44f0b Dynamic loading of DB driver + support for other DB providers 2016-05-23 15:50:28 +02:00
Will Stevens 3343f1229c Merge pull request #1376 from milamberspace/L10N-update-Master-20160127
L10n update master 20160127cc @remibergsma @bhaisaab @DaanHoogland

* pr/1376:
  Update L10N resource files with 4.9 strings from Transifex (20160511) Force "translator" mode with the transifex client. Update Transifex client config file (regenerated by Tx client)

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-13 17:00:00 -04:00
Milamber 9b93989032 Update L10N resource files with 4.9 strings from Transifex (20160511)
Force "translator" mode with the transifex client.
Update Transifex client config file (regenerated by Tx client)
2016-05-12 22:30:40 +01:00
Will Stevens c79affaefb Merge pull request #1528 from mike-tutkowski/marvin_methods
CLOUDSTACK-9373: Class methods over-shawdowing instance methodsWe have some methods in base.py that are named the same.

Per my findings below, Python methods in a class should not be named the same even if one is a class method and the other is an instance method.

The solution discussed on dev@ is to remove the instance versions (reason listed in e-mail text, which is listed in JIRA ticket).

https://issues.apache.org/jira/browse/CLOUDSTACK-9373

* pr/1528:
  CLOUDSTACK-9373: Removing a few instance methods where there are class methods that are overshadowing them

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-12 11:08:53 -04:00
Mike Tutkowski 1ad0da0390 CLOUDSTACK-9373: Removing a few instance methods where there are class methods that are overshadowing them 2016-05-11 23:57:09 -06:00
Rohit Yadav 4d5e8df2f9 travis: Use patched version of ipmitool for tests
- For out-of-band management feature (CLOUDSTACK-9299) use patched version of
  ipmitool that would work on trusty travis machines
- The ipmitool used is from xenial/16.04 release with patch from RedHat
  https://bugzilla.redhat.com/show_bug.cgi?id=1286035
- Installs ipmitool from xenial repositories to get all the dependencies
  and then install patched deb version
- Skip test if the known failure occurs

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-12 11:05:40 +05:30
Rohit Yadav 07564469e9 CLOUDSTACK-9299: Out-of-band Management for CloudStack
Support access to a host’s out-of-band management interface (e.g. IPMI, iLO,
DRAC, etc.) to manage host power operations (on/off etc.) and querying current
power state in CloudStack.

Given the wide range of out-of-band management interfaces such as iLO and iDRA,
the service implementation allows for development of separate drivers as plugins.
This feature comes with a ipmitool based driver that uses the
ipmitool (http://linux.die.net/man/1/ipmitool) to communicate with any
out-of-band management interface that support IPMI 2.0.

This feature allows following common use-cases:
- Restarting stalled/failed hosts
- Powering off under-utilised hosts
- Powering on hosts for provisioning or to increase capacity
- Allowing system administrators to see the current power state of the host

For testing this feature `ipmisim` can be used:
https://pypi.python.org/pypi/ipmisim

FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Out-of-band+Management+for+CloudStack

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-12 11:05:40 +05:30
Rohit Yadav afcbd270b2 CLOUDSTACK-8562: Deprecate commands.properties
- Removes commands.properties file
- Fixes apidocs and marvin to be independent of commands.properties usage
- Removes bundling of commands.properties in deb/rpm packaging
- Removes file references across codebase

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-11 09:45:19 +05:30
Rohit Yadav 4347776ac6 CLOUDSTACK-8562: DB-Backed Dynamic Role Based API Access Checker
This feature allows root administrators to define new roles and associate API
permissions to them.

A limited form of role-based access control for the CloudStack management server
API is provided through a properties file, commands.properties, embedded in the
WAR distribution. Therefore, customizing API permissions requires unpacking the
distribution and modifying this file consistently on all servers. The old system
also does not permit the specification of additional roles.

FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+Role+Based+API+Access+Checker+for+CloudStack

DB-Backed Dynamic Role Based API Access Checker for CloudStack brings following
changes, features and use-cases:
- Moves the API access definitions from commands.properties to the mgmt server DB
- Allows defining custom roles (such as a read-only ROOT admin) beyond the
  current set of four (4) roles
- All roles will resolve to one of the four known roles types (Admin, Resource
  Admin, Domain Admin and User) which maintains this association by requiring
  all new defined roles to specify a role type.
- Allows changes to roles and API permissions per role at runtime including additions or
  removal of roles and/or modifications of permissions, without the need
  of restarting management server(s)

Upgrade/installation notes:
- The feature will be enabled by default for new installations, existing
  deployments will continue to use the older static role based api access checker
  with an option to enable this feature
- During fresh installation or upgrade, the upgrade paths will add four default
  roles based on the four default role types
- For ease of migration, at the time of upgrade commands.properties will be used
  to add existing set of permissions to the default roles. cloud.account
  will have a new role_id column which will be populated based on default roles
  as well

Dynamic-roles migration tool: scripts/util/migrate-dynamicroles.py
- Allows admins to migrate to the dynamic role based checker at a future date
- Performs a harder one-way migrate and update
- Migrates rules from existing commands.properties file into db and deprecates it
- Enables an internal hidden switch to enable dynamic role based checker feature

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-11 09:45:19 +05:30
Will Stevens 10bcf2955a Merge pull request #1529 from mike-tutkowski/marvin_replace_sleep
Marvin: Replace a timer.sleep(30) with pulling logichttps://issues.apache.org/jira/browse/CLOUDSTACK-9374

From the ticket:

In the base.py file, there is a Host class with a delete instance method.

This method first attempts to transition the host into the maintenance resource state.

The first step in this process is to transition the host into the prepare-for-maintenance resource state.

A while later, the host can be transitioned completely into the maintenance resource state.

In an attempt to wait for this transition to occur, the delete method has a timer.sleep(30) call.

The hope is that the host will have transitioned from the prepare-for-maintenance resource state to the maintenance resource state within 30 seconds, but this does not always happen.

We should correct this problem by putting in logic to query the management server for the resource state of the host. If it's in the expected state, move on; else, sleep for a bit and try again (up to a certain limit).

* pr/1529:
  Replace a timer.sleep(30) with pulling logic

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-11 00:03:59 -04:00
Will Stevens fa3bce5a83 Merge pull request #1496 from shapeblue/kvm-ha
CLOUDSTACK-9350: KVM-HA- Fix CheckOnHost for Local storage- KVM-HA- Fix CheckOnHost for Local storage
 - Also skip HA on VMs that are using local storage

* pr/1496:
  CLOUDSTACK-9350: KVM-HA- Fix CheckOnHost for Local storage     - Also skip HA on VMs that are using local storage

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-10 23:49:04 -04:00
Will Stevens a368cb703f Merge release branch 4.8 to master
* 4.8:
  Update L10N resource files with 4.8 strings from Transifex (20160504) Force "translator" mode with the transifex client. Update Transifex client config file for 4.8 resources/L10N ref. (generated by Tx client)
2016-05-10 23:45:43 -04:00
Mike Tutkowski eeb3373e41 Replace a timer.sleep(30) with pulling logic 2016-05-06 19:20:54 -06:00
Milamber e81a6b270e Update L10N resource files with 4.8 strings from Transifex (20160504)
Force "translator" mode with the transifex client.
Update Transifex client config file for 4.8 resources/L10N ref. (generated by Tx client)
2016-05-04 21:47:37 +01:00
Will Stevens 6d0c92be72 Merge release branch 4.8 to master
* 4.8:
  Update L10N resource files with 4.7 strings from Transifex (20160502) Force "translator" mode with the transifex client.
2016-05-04 10:48:18 -04:00
Will Stevens 38c9e9f336 Merge release branch 4.7 to 4.8
* 4.7:
  Update L10N resource files with 4.7 strings from Transifex (20160502) Force "translator" mode with the transifex client.
2016-05-04 10:47:33 -04:00
Milamber 6460cde8b5 Update L10N resource files with 4.7 strings from Transifex (20160502)
Force "translator" mode with the transifex client.
2016-05-02 22:32:14 +01:00
Abhinandan Prateek 3b89cbe733 CLOUDSTACK-9350: KVM-HA- Fix CheckOnHost for Local storage
- Also skip HA on VMs that are using local storage
2016-04-29 12:38:33 +05:30
Will Stevens 12d8e53dd5 Merge pull request #1454 from shapeblue/host-maint
CLOUDSTACK-9323: Fix cancel host maintenance canFix cancel host maintenance so that if maintenance is cancelled the host come back to normal state gracefully.

Added marvin tests for host maintennace.

* pr/1454:
  CLOUDSTACK-9323: Fix Cancel maintenance so that if maintenance is cancelled the host come back to normal state gracefully. Added marvin tests for host maintennace.

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-28 16:05:10 -04:00
Will Stevens a5ee4432e7 Merge release branch 4.8 to master
* 4.8:
  Removed sleeps and used validateList as requested.
  Added required_hardware="false" attr above test_02_root_volume_attach_detach
  Modified test_volumes.py to include a hypervisor test for root attach/detach testing
  Let hypervisor type KVM and Simulator detach root volumes. Updated test_volumes.py to include a test for detaching and reattaching a root volume from a vm. I also had to update base.py to allow attach_volume to have the parameter deviceid to be passed as needed.
2016-04-28 16:04:20 -04:00
Will Stevens d11194abec Merge release branch 4.7 to 4.8
* 4.7:
  Removed sleeps and used validateList as requested.
  Added required_hardware="false" attr above test_02_root_volume_attach_detach
  Modified test_volumes.py to include a hypervisor test for root attach/detach testing
  Let hypervisor type KVM and Simulator detach root volumes. Updated test_volumes.py to include a test for detaching and reattaching a root volume from a vm. I also had to update base.py to allow attach_volume to have the parameter deviceid to be passed as needed.
2016-04-28 16:03:46 -04:00
Will Stevens 4db5e48f80 Merge pull request #1510 from shapeblue/4.9-mvn-version-safeupgradeonly
4.9 mvn version safeupgradeonlyUpgrades maven dependencies versions that can be safely upgraded without breaking console-proxy/crypto usage.

Bisected changes from: https://github.com/apache/cloudstack/pull/1397

cc @swill @DaanHoogland

* pr/1510:
  maven: fix dependency version support by JDK7
  further maven dependency updates from Daan
  framework/quota: fix checkstyle issue
  maven: Upgrade dependency versions

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-28 15:59:17 -04:00
Will Stevens 939e448227 Merge pull request #1458 from cloudsadhu/cloudsadhulocal
BUG-ID:CLOUDSTACK-9331:added code in marvin frame&new config file for advBaremetal support  Added code  in marvin framework&new config file  to support baremetal advanced testcases

* pr/1458:
  BUG-ID:CLOUDSTACK-9331:added code in marvin frame&new config file to support baremetal advanced testcases

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-25 15:55:09 -04:00
Will Stevens 9b525f51a2 Merge pull request #1452 from prashanthvarma/master
CLOUDSTACK-9322: Support for Internal LB fuctionality with Nuage VSP SDN Plugin including Marvin test coverageTask: https://issues.apache.org/jira/browse/CLOUDSTACK-9322

PR contents:
1) UI changes to support LB provider InternalLbVm during VPC offering creation.
2) Bug fix for CLOUDSTACK-9320.
3) Nuage VSP SDN Plugin related enhancements for VPC network functionality.
4) Marvin test coverage for Internal LB support on master with Nuage VSP SDN Plugin.
5) Enhancements on our exiting Marvin test code (nuagevsp plugins directory).
6) PEP8 & PyFlakes compliance with our Marvin test code.

Test run:
CloudStack$ nosetests --with-marvin --marvin-config=nuage_ant.cfg test/integration/plugins/nuagevsp/ -a tags=nuagevsp

Test results:
Test user data and password reset functionality with Nuage VSP SDN plugin ... === TestName: test_nuage_UserDataPasswordReset | Status : SUCCESS ===
ok
Test Nuage VSP VPC Offering with different combinations of LB service providers ... === TestName: test_01_nuage_internallb_vpc_Offering | Status : SUCCESS ===
ok
Test Nuage VSP VPC Network Offering with and without Internal LB service ... === TestName: test_02_nuage_internallb_vpc_network_offering | Status : SUCCESS ===
ok
Test Nuage VSP VPC Networks with and without Internal LB service ... === TestName: test_03_nuage_internallb_vpc_networks | Status : SUCCESS ===
ok
Test Nuage VSP VPC Internal LB functionality with different combinations of Internal LB rules ... === TestName: test_04_nuage_internallb_rules | Status : SUCCESS ===
ok
Test Nuage VSP VPC Internal LB functionality by performing (wget) traffic tests within a VPC ... === TestName: test_05_nuage_internallb_traffic | Status : SUCCESS ===
ok
Test Nuage VSP VPC Internal LB functionality with different LB algorithms by performing (wget) traffic tests ... === TestName: test_06_nuage_internallb_algorithms_traffic | Status : SUCCESS ===
ok
Test Nuage VSP VPC Internal LB functionality with restarts of VPC network components by performing (wget) ... === TestName: test_07_nuage_internallb_vpc_network_restarts_traffic | Status : SUCCESS ===
ok
Test Nuage VSP VPC Internal LB functionality with InternalLbVm appliance operations by performing (wget) ... === TestName: test_08_nuage_internallb_appliance_operations_traffic | Status : SUCCESS ===
ok
Test Basic VPC Network Functionality with Nuage VSP SDN plugin ... === TestName: test_nuage_vpc_network | Status : SUCCESS ===
ok
Test Nuage VSP SDN plugin with basic Isolated Network functionality ... === TestName: test_nuage_vsp | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 11 tests in 12094.705s

OK

Test run logs:
[results.txt](https://github.com/apache/cloudstack/files/187587/results.txt)
[runinfo.txt](https://github.com/apache/cloudstack/files/187588/runinfo.txt)

Test config file:
[nuage_ant.txt](https://github.com/apache/cloudstack/files/222711/nuage_ant.txt)

Note: Attached the Marvin config file as .txt instead of .cfg.

PEP8 & PyFlakes Compliance:
CloudStack$ pep8 --max-line-length=150 test/integration/plugins/nuagevsp/*.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/nuageTestCase.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/test_nuage_password_reset.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/test_nuage_vpc_internal_lb.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/test_nuage_vpc_network.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/test_nuage_vsp.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/*.py

#CLOUDSTACK-9322

* pr/1452:
  CLOUDSTACK-9322 : Marvin tests for Internal Lb with Nuage VSP
  CLOUDSTACK-9320 : InternalLBVM is not getting destroyed when the last Internal Load Balancer rule is removed for the corresponding source IP address
  CLOUDSTACK-9322 : Changes to support InternalLbVm with Nuage VSP plugin

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-25 15:53:58 -04:00
Abhinandan Prateek 182ab648b5 CLOUDSTACK-9323: Fix Cancel maintenance so that if maintenance is cancelled the host come back to normal state gracefully.
Added marvin tests for host maintennace.
2016-04-25 15:28:32 +05:30
sadhu 5f16199813 BUG-ID:CLOUDSTACK-9331:added code in marvin frame&new config file to support baremetal advanced testcases 2016-04-23 11:42:28 +05:30
Daan Hoogland 101668994d further maven dependency updates from Daan
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-04-22 16:36:37 +05:30
Rohit Yadav bb29b1d063 maven: Upgrade dependency versions
Updated most dependencies to latest minor releases, EXCEPT:

- Gson 2.x
- Major spring framework version
- Servlet version
- Embedded jetty version
- Mockito version (beta)
- Mysql lib minor version upgrade (breaks mysql-ha plugin)

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-04-22 16:20:16 +05:30
Mike Tutkowski ba9a61a302 CLOUDSTACK-9354 - Fixing an issue in Marvin around creating a template from a snapshot (if “is public” is not provided, there was a problem) 2016-04-19 14:13:24 -06:00
David Mabry 0b7cc087f8 Let hypervisor type KVM and Simulator detach root volumes.
Updated test_volumes.py to include a test for detaching and reattaching a root volume from a vm. I also had to update base.py to allow attach_volume to have the parameter deviceid to be passed as needed.
2016-04-18 15:40:36 -05:00
Rohit Yadav 41e3b61245 travis: increase build verbosity
Outputs for modules that fail or succeed with unit tests results

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-04-09 03:12:12 +05:30
Rohit Yadav 87b431c0f3 travis: Fix simulator tests and optimize default global configs
- Migrate to trusty based Travis VMs
- Increase tests across five build matrices
- Fix xunit-reader output, include time
- Fix pip/python usage, pkg installation
- Build CloudStack in parallel with -T4
- Deploy database with optimized global settings

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-04-07 02:19:52 +05:30
sanjeevn 8daaa30fc8 [CLOUDSTACK-9337]Enhance vcenter.py to created data center in vcenter server automatically
(Programmatically)
2016-04-05 17:43:28 +05:30
Prashanth Manthena de86c0cb17 CLOUDSTACK-9322 : Marvin tests for Internal Lb with Nuage VSP 2016-03-29 11:24:54 +02:00
Sowmya 3c8d3d0ee1 Add nuagevsp userdata testcase (Cloudstack-9095) & Refactor existing testcases 2016-03-08 18:13:24 -08:00
Remi Bergsma 3d9919ecfb Revert "Merge pull request #1011 from shwetaag/coreos"
This reverts commit 56859c9904, reversing
changes made to cd71e05b1e.
2016-02-10 10:28:18 +01:00
sanjeev 56859c9904 Merge pull request #1011 from shwetaag/coreos
CLOUDSTACK-9012 :automation of cores feature test pathhttps://issues.apache.org/jira/browse/CLOUDSTACK-9012
Automated a full scenario of coreos guest OS support:
it includes registering coreos templates present at http://dl.openvm.eu/cloudstack/coreos/x86_64/
1. based on hypervisor types of zone
2. creating ssh key pair
3. creating a sample user data
4. creating a coreos virtual machine using this ssh keypair and userdata
5. verifying ssh access to coreo os machine using keypair and core username
6. verifying userdata is applied on virtual machine and the service asked in sample data is actually running
7. Verifying userdata in router vm as well

* pr/1011:
  added suggested changes to coreos automation
  automation of cores feature test path

Signed-off-by: sanjeev <sanjeev@apache.org>
2016-02-10 11:03:52 +05:30
shweta agarwal 94281f952d added suggested changes to coreos automation 2016-02-04 16:57:08 +05:30
Rohit Yadav 06fe7844db Merge pull request #1240 from shapeblue/master-9161
CLOUDSTACK-9161: fix the quota marvin test  1. Create a dummy user, as existing user may already have stale quota
  data
  2. fix the tests to use the dummy user
  3. a boundary condition was revealed and fixed for a new user where
  quota service has never run and created bootstrap entries

* pr/1240:
  CLOUDSTACK-9161: fix the quota marvin test

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-27 12:03:28 +01:00
Remi Bergsma 43ab98d823 Updating pom.xml version numbers for release 4.9.0-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2016-01-26 15:12:20 +01:00
Remi Bergsma 32fcc47117 Updating pom.xml version numbers for release 4.8.1-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2016-01-26 09:39:00 +01:00
Remi Bergsma a4ac9d3fff Updating pom.xml version numbers for release 4.7.2-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2016-01-26 09:33:56 +01:00
Remi Bergsma 62f218b7bd Updating pom.xml version numbers for release 4.8.0
Signed-off-by: Remi Bergsma <github@remi.nl>
2016-01-20 23:43:35 +01:00
Remi Bergsma ad420bb1df Merge pull request #1041 from pritisarap12/CLOUDSTACK-9041-Modifying-template-creation-from-snapshot-function-in-base.py
CLOUDSTACK-9041: Modifying template creation from snapshot function In create_from_snapshot function of Template class there is no parameter to accept if the template is public hence default it is creating private templates
Hence adding this parameter.

* pr/1041:
  CLOUDSTACK-9041: Modifying template creation from snapshot function in base.py

Signed-off-by: Remi Bergsma <github@remi.nl>
2016-01-17 13:37:03 +01:00
Remi Bergsma 75b68c6829 Merge release branch 4.7 to master
* 4.7:
  Fix unable to setup more than one Site2Site VPN Connection
  FIX S2S VPN rVPC: Check only redundant routers in state MASTER
  PEP8 of integration/smoke/test_vpc_vpn
  Add S2S VPN test for Redundant VPC
  Make integration/smoke/test_vpc_vpn Hypervisor independant
  FIX VPN: non-working ipsec commands
  [UI] MADNESS
  [DB] Add force_encap field to s2s_customer_gateway table
  [ROUTER] Add forceencaps field to python router ipsec config method
  [TEST] unittest needs rework
  [MARVIN] Add forceencap field to VpnCustomerGateway class in marvin base
  [CORE] Add Force UDP Encapsulation option to Site2Site VPN
  CLOUDSTACK-9186: Root admin cannot see VPC created by Domain admin user
  CLOUDSTACK-9192: UpdateVpnCustomerGateway is failing
  CLOUDSTACK-6485 prevent ip asignment of private gw iface
  CLOUDSTACK-9204 Do not error when staticroute is already gone
  make both check lines consistent
  CLOUDSTACK-9181 Prevent syntax error in checkrouter.sh
  CLOUDSTACK-9202 Bump ssh timeout
2016-01-16 19:54:41 +01:00
Michael Andersen 0b54871fa3 [MARVIN] Add forceencap field to VpnCustomerGateway class in marvin base 2016-01-07 19:27:42 +01:00
Miguel Ferreira 3dba689f31 Add helper method to migrate router vms 2015-12-24 16:07:06 -03:00
Miguel Ferreira 88774a93e8 Only set L2 Gateway in NSX device if defined 2015-12-24 16:07:06 -03:00
nvazquez 4ac2737754 CLOUDSTACK-9074: Marvin tests for NSX Shared Networks Support 2015-12-24 16:07:06 -03:00
Rohit Yadav 1a194e6745 README: happy holidays!
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-12-23 13:27:40 +05:30
Remi Bergsma 8f5a2920e8 Updating pom.xml version numbers for release 4.8.0-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-21 22:09:31 +01:00
Remi Bergsma 5b112408c7 Updating pom.xml version numbers for release 4.7.1-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-21 19:55:04 +01:00
Abhinandan Prateek ccd0f22061 CLOUDSTACK-9161: fix the quota marvin test
1. Create a dummy user, as existing user may already have stale quota
  data
  2. fix the tests to use the dummy user
  3. a boundary condition was revealed and fixed for a new user where
  quota service has never run and created bootstrap entries

Quota Marvin: If the quota plugin is not enabled skip tests

Quota Service: Enable quota plugin in zone setup configuration

Quota: Moving test_quota.py the test to test/integration/plugins
    In most automated environment this test case will not run as it
    requires a mangement server restart to enable the plugin. Due to
    this requirement moving it to plugin folder. This condition is
    already documented in the test case.
2015-12-16 16:22:31 +05:30
Remi Bergsma 2f26a859a9 Updating pom.xml version numbers for release 4.7.0
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-13 21:09:53 +01:00
Remi Bergsma c1533a1918 Merge pull request #1239 from remibergsma/fix-debian-changelog
build_asf.sh: fix debian changelog alteringThe script already adds a new entry so we shouldn't replace the
previous one. That is done only in the next version script.

As discussed with @DaanHoogland

* pr/1239:
  build_asf.sh: fix debian changelog altering

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-13 20:47:51 +01:00
Remi Bergsma 97b5e12704 build_asf.sh: fix debian changelog altering
The script already adds a new entry so we shouldn't replace the
previous one. That is done only in the next version script.
2015-12-13 20:41:56 +01:00
Remi Bergsma 9862b2c780 Merge pull request #1217 from milamberspace/L10N-update-before-4.7.0RC1
L10N update before 4.7.0 RC1Some sentences (mainly for the new quota plugin) needs to be translated before the 4.7.0 release candidate (planned for 14th december)
https://www.transifex.com/ke4qqq/CloudStack_UI/47xmessagesproperties/

Stats:
German (Germany)            100%
French (France)             100%
Portuguese (Brazil)          98%
Norwegian Bokml (Norway)    98%
Dutch (Netherlands)          98%
Chinese (China)              97%
Japanese (Japan)             97%
Hungarian                    96%
(only languages over 90%)

Ping me after adding the L10N strings into transifex, I will update this PR.

Thanks

cc @remibergsma @agneya2001 @DaanHoogland @nvazquez @barunhalder1234 @K0zka

* pr/1217:
  Update L10N resource files with 4.7 strings from Transifex (20151211)

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-13 19:41:44 +01:00
Remi Bergsma e7030285a2 Merge pull request #1219 from DaanHoogland/CLOUDSTACK-9139
CLOUDSTACK-9139 make zwps default when defined on zone levelWhen a marvin definition contains a primary storage definition at zone level it will default to zone scope. A test run is underway.
using marvin config:
```
{
    "zones": [
        {
            "name": "MCCT-SHARED-1",
            "guestcidraddress": "10.1.1.0/24",
            "dns1": "8.8.8.8",
            "physical_networks": [
                {
                    "broadcastdomainrange": "Zone",
                    "vlan": "100-200",
                    "name": "mcct-pnet",
                    "traffictypes": [
                        {
                            "typ": "Guest"
                        },
                        {
                            "typ": "Management"
                        },
                        {
                            "typ": "Public"
                        }
                    ],
                    "providers": [
                        {
                            "broadcastdomainrange": "ZONE",
                            "name": "VirtualRouter"
                        },
                        {
                            "broadcastdomainrange": "ZONE",
                            "name": "VpcVirtualRouter"
                        },
                        {
                            "broadcastdomainrange": "ZONE",
                            "name": "InternalLbVm"
                        }
                    ],
                    "isolationmethods": [
                             "VLAN"
                    ]
                }
            ],
            "ipranges": [
                {
                    "startip": "192.168.23.2",
                    "endip": "192.168.23.20",
                    "netmask": "255.255.255.0",
                    "vlan": "50",
                    "gateway": "192.168.23.1"
                }
            ],
            "networktype": "Advanced",
            "pods": [
                {
                    "endip": "192.168.22.150",
                    "name": "MCCT-POD",
                    "startip": "192.168.22.130",
                    "netmask": "255.255.255.0",
                    "clusters": [
                    ],
                    "clusters": [
                        {
                            "clustername": "MCCT-KVM-1",
                            "hypervisor": "KVM",
                            "hosts": [
                                {
                                    "username": "root",
                                    "url": "http://kvm1",
                                    "password": "password"
                                },
                                {
                                    "username": "root",
                                    "url": "http://kvm2",
                                    "password": "password"
                                }
                            ],
                            "clustertype": "CloudManaged"
                        }
                    ],
                    "gateway": "192.168.22.1"
                }
            ],
            "internaldns1": "8.8.4.4",
            "secondaryStorages": [
                {
                    "url": "nfs://192.168.22.1:/data/storage/secondary/MCCT-SHARED-1",
                    "provider" : "NFS"
                }
            ],
            "primaryStorages": [
                {
                    "url": "nfs://192.168.22.1:/data/storage/primary/MCCT-KVM-1",
                    "name": "MCCT-KVM-1-primary",
                    "hypervisor": "kvm"
                }
            ]
        }
    ],
    "dbSvr": {
        "dbSvr": "localhost",
        "passwd": "cloud",
        "db": "cloud",
        "port": 3306,
        "user": "cloud"
    },
    "logger":
        {
            "LogFolderPath": "/tmp/"
        },
    "globalConfig": [
        {
            "name": "network.gc.wait",
            "value": "60"
        },
        {
            "name": "storage.cleanup.interval",
            "value": "300"
        },
        {
            "name": "vm.op.wait.interval",
            "value": "5"
        },
        {
            "name": "default.page.size",
            "value": "10000"
        },
        {
            "name": "network.gc.interval",
            "value": "60"
        },
        {
            "name": "workers",
            "value": "10"
        },
        {
            "name": "account.cleanup.interval",
            "value": "600"
        },
        {
            "name": "guest.domain.suffix",
            "value": "cloud"
        },
        {
            "name": "expunge.delay",
            "value": "60"
        },
        {
            "name": "vm.allocation.algorithm",
            "value": "firstfitleastconsumed"
        },
        {
            "name": "expunge.interval",
            "value": "60"
        },
        {
            "name": "expunge.workers",
            "value": "3"
        },
        {
            "name": "check.pod.cidrs",
            "value": "true"
        },
        {
            "name": "secstorage.allowed.internal.sites",
            "value": "192.168.22.0/24"
        },
        {
            "name": "direct.agent.load.size",
            "value": "1000"
        }
    ],
    "mgtSvr": [
        {
            "mgtSvrIp": "localhost",
            "passwd": "password",
            "user": "root",
            "port": 8096,
            "hypervisor": "KVM",
            "useHttps": "False",
            "certCAPath":  "NA",
            "certPath":  "NA"
        }
    ]
}
```
deploys succesful:
```
Deploy data center..

==== Log Folder Path: /tmp//MarvinLogs//DeployDataCenter__Dec_11_2015_16_27_55_AM5LVR. All logs will be available here ====

==== Deploy DC Started ====

=== Data Center Settings are dumped to /tmp//MarvinLogs//DeployDataCenter__Dec_11_2015_16_27_55_AM5LVR/dc_entries.obj===

====Deploy DC Successful=====
Fri Dec 11 16:32:02 GMT 2015
```

![screen shot 2015-12-11 at 17 37 12](https://cloud.githubusercontent.com/assets/2486961/11749410/e7d99db4-a02d-11e5-9a5b-e5d554a45945.png)

* pr/1219:
  CLOUDSTACK-9139 make zwps default when defined on zone level

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-12 21:02:08 +01:00
Milamber 6947afee3a Update L10N resource files with 4.7 strings from Transifex (20151211) 2015-12-12 18:57:10 +00:00
Remi Bergsma bdaa60fd6c Merge release branch 4.6 to master
* 4.6:
  Show actual diff in commits after merge
  CLOUDSTACK-9113: skip vm with inconsistent state when getVmStats/getVmDiskStats
2015-12-12 14:03:43 +01:00
Remi Bergsma 52edbbb6cc Show actual diff in commits after merge 2015-12-12 11:22:32 +01:00