Commit Graph

25557 Commits

Author SHA1 Message Date
Rohit Yadav f30c52a16c CLOUDSTACK-8562: DB-Backed Dynamic Role Based API Access Checker
This feature allows root administrators to define new roles and associate API
permissions to them.

A limited form of role-based access control for the CloudStack management server
API is provided through a properties file, commands.properties, embedded in the
WAR distribution. Therefore, customizing API permissions requires unpacking the
distribution and modifying this file consistently on all servers. The old system
also does not permit the specification of additional roles.

FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+Role+Based+API+Access+Checker+for+CloudStack

DB-Backed Dynamic Role Based API Access Checker for CloudStack brings following
changes, features and use-cases:
- Moves the API access definitions from commands.properties to the mgmt server DB
- Allows defining custom roles (such as a read-only ROOT admin) beyond the
  current set of four (4) roles
- All roles will resolve to one of the four known roles types (Admin, Resource
  Admin, Domain Admin and User) which maintains this association by requiring
  all new defined roles to specify a role type.
- Allows changes to roles and API permissions per role at runtime including additions or
  removal of roles and/or modifications of permissions, without the need
  of restarting management server(s)

Upgrade/installation notes:
- The feature will be enabled by default for new installations, existing
  deployments will continue to use the older static role based api access checker
  with an option to enable this feature
- During fresh installation or upgrade, the upgrade paths will add four default
  roles based on the four default role types
- For ease of migration, at the time of upgrade commands.properties will be used
  to add existing set of permissions to the default roles. cloud.account
  will have a new role_id column which will be populated based on default roles
  as well

Dynamic-roles migration tool: scripts/util/migrate-dynamicroles.py
- Allows admins to migrate to the dynamic role based checker at a future date
- Performs a harder one-way migrate and update
- Migrates rules from existing commands.properties file into db and deprecates it
- Enables an internal hidden switch to enable dynamic role based checker feature

Deprecate commands.properties

- Fixes apidocs and marvin to be independent of commands.properties usage
- Removes bundling of commands.properties in deb/rpm packaging
- Removes file references across codebase

Reviewed-by: John Burwell <john.burwell@shapeblue.com>
QA-by: Boris Stoyanov <boris.stoyanov@shapeblue.com>

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-04-25 14:52:02 +05:30
Rohit Yadav d0416e3524 metrics: Fix issues, implement search feature
- Fix threshold calculations for host and storage metric views
- Implement advance search box for all metrics views
- Save and pass context through navigation
- Add complex nav support, fix context usage

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-20 12:58:59 +01:00
Rohit Yadav c164ffee3b metrics: Increase UI width, fix metrics ui issues
- Add instances count in host view, ip address in instances view
- Add ipaddress in instances view
- Increase UI container width by 200px
- Allow all users to see resource metrics
- Fix instance count issue on host metrics view
- Fix sorting issue in metrics tables
- Fix instances null check regression from 2f250e2

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-20 12:57:09 +01:00
Rohit Yadav 938f1d2a7d CLOUDSTACK-9020: Metrics views for CloudStack UI
Implements following:
- A metrics table widget that is:
  - vertically and horizontally scrollable with pagination/infinite scrolling
  - sortable columns (client side)
  - groupable/collapsible columns
  - alternate row coloring
  - refresh button to refresh views
  - threshold table cell coloring
  - panel/breadcrumb navigation
  - quick view action column
  - translatable labels
- Sortable column for all CloudStack tables (client side)
- Configurable UI pagesize for list API calls, 'default.ui.page.size'
- Metrics views: Zones, Clusters, Hosts, Instances, Storage pools, Volumes
- Resource filtering/navigation: Zones->Clusters->Hosts->Instances->Volumes, Storage Pool->Volumes

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-20 12:54:06 +01:00
ramamurtis 173e2462b3 CLOUDSTACK-9006 - ListTemplates API returns result in inconsistent order when called concurrently
Adding filter test to verify addOrderBy method.

(cherry picked from commit 9d1a469ae5)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-20 12:54:06 +01:00
ramamurtis d82a373604 CLOUDSTACK-9006 - ListTemplates API returns result in inconsistent order when called concurrently
(cherry picked from commit c90723f491)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-20 12:54:05 +01:00
Rohit Yadav 88e1645d0b CLOUDSTACK-9019: Add storage network offering in ssvm only if storage network is defined
During creation of SSVM, checks and adds NetworkOffering.SystemStorageNetwork to
offerings only if storage network exists for the target datacenter

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-20 12:54:05 +01:00
Nitin Kumar Maharana 3a6054e549 CLOUDSTACK-8913: Search box in Templates tab out of alignment
Increased the margin and padding to accomodate all the boxes inside toolbar.

(cherry picked from commit febaeb2a15)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-20 12:54:05 +01:00
Rohit Yadav 847d69edfd CLOUDSTACK-9000: logrotate cloudstack-agent out and err logs
Adds logrotate rules for cloudstack-agent.{err,out} log files

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-20 12:54:04 +01:00
Rohit Yadav 8348a237b8 CLOUDSTACK-9000: logrotate cloudstack-agent out and err logs
Adds logrotate rules for cloudstack-agent.{err,out} log files

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-20 12:54:04 +01:00
Daan Hoogland 84e9ff746b CLOUDSTACK-8848: added null pointer guard to new public method 2016-01-20 12:54:04 +01:00
Rene Moser f7e1029cfc CLOUDSTACK-8848: ensure power state is up to date when handling missing VMs in powerReport
There 2 things which has been changed.

* We look on power_state_update_time instead of update_time. Didn't make sense to me at all to look at update_time.
* Due DB update optimisation, powerState will only be updated if < MAX_CONSECUTIVE_SAME_STATE_UPDATE_COUNT. That is why we can not rely on these information unless we make sure these are up to date.
2016-01-20 12:54:03 +01:00
Pierre-Luc Dion 6902dac319 Fix dockerbuild because hardcoded filename for marvin
Signed-off-by: Pierre-Luc Dion <pdion891@apache.org>
2016-01-20 12:54:03 +01:00
Boris Schrijver db771cd0a4 Added Unit Tests for QCOW2Utils. 2016-01-20 12:54:03 +01:00
Boris Schrijver 8a5aa0faaa Fixed Findbugs issue introduced by 1c6378ec00 PR #795. 2016-01-20 12:54:02 +01:00
Boris Schrijver decaef6cb4 Added QCOW2 virtual size checking for S3.
- Cleaned up S3TemplateDownloader
- Created static QCOW2 utils class.
- Reformatted some parts of DownloadManagerImpl
2016-01-20 12:54:02 +01:00
Rohit Yadav 11abfb8766 tools/appliance: update Debian 7 iso URLs based on 615261
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-16 12:56:25 +05:30
Suresh Kumar Anaparti 52e23996b8 CLOUDSTACK-8820: Updated the code for vCenter6 data center support.
(cherry picked from commit 1d73418c2c)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-16 12:52:46 +05:30
Suresh Kumar Anaparti 5f95f1688e CLOUDSTACK-8820: Showing error when try to add advance zone using VMWare ESXi 6.0 host
Summary: In vCenter 6.0, response headers need to be fetched after service login for server cookie unlike previous versions of vCenter.

(cherry picked from commit 4a6e2cdeee)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-16 12:52:18 +05:30
Rohit Yadav 17166eb631 schema: add 4.5.2 to 4.5.3 upgrade path stubs
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-15 09:57:48 +05:30
Rohit Yadav 284ce1fb88 engine/schema: add 4.5.2 to 4.5.3 upgrade path
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-15 09:57:12 +05:30
Rohit Yadav af38c492d8 debian: add 4.5.3-snapshot to changelog
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-14 10:01:58 +05:30
Rohit Yadav a4f552d46d cloudstack: set next version to 4.5.3-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-14 09:59:40 +05:30
Rohit Yadav d58dfbf21b Merge pull request #814 from wido/CLOUDSTACK-8443
CLOUDSTACK-8443: Support CentOS 7 for 4.5This is based on two PRs:
- 731
- 757

This commit is based on the 4.5 branch for a future 4.5 release.

* pr/814:
  CLOUDSTACK-8443: Support CentOS 7 for 4.5

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-14 09:52:06 +05:30
Wido den Hollander 2b8fd2469f CLOUDSTACK-8443: Support CentOS 7 for 4.5
This is based on two PRs:
- 731
- 757

This commit is based on the 4.5 branch for a future 4.5 release.
2015-09-13 15:30:20 +02:00
Rene Moser 15ef2b114c config: doc: add default value 10 in description for vm.network.nic.max.secondary.ipaddresses
value hardcoded by commit 4925b9f6a1

(cherry picked from commit 0b956bbc1f)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-08 16:56:15 +05:30
Rohit Yadav 047d759a73 Merge pull request #754 from shapeblue/4.5-CLOUDSTACK-8762
CLOUDSTACK-8762: Check to confirm disk activity before starting a VMImplements a VM volume/disk file activity checker that checks if QCOW2 file
has been changed before starting the VM. This is useful as a pessimistic
approach to save VMs that were running on faulty hosts that CloudStack could
try to launch on other hosts while the host was not cleanly fenced. This is
optional and available only if you enable the settings in agent.properties
file, on per-host basis.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* pr/754:
  CLOUDSTACK-8762: Confirm disk activity before starting a VM

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-02 14:54:58 +05:30
Rohit Yadav a6c630c54b Merge pull request #771 from mike-tutkowski/4.5
Support live migration on older version of Libvirthttps://issues.apache.org/jira/browse/CLOUDSTACK-8792

A flag being passed to Libvirt assumes v1.0.0 or later.

We need to put a check in the code to pass in a different flag if the version of Libvirt is < 1.0.0.

* pr/771:
  Support live migration on older version of Libvirt

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-02 10:38:48 +05:30
Rohit Yadav 6cdde0feb4 Merge pull request #770 from milamberspace/L10N-update-4.5-20150901
Update L10N resource files on 4.5 branch with 4.5 translation strings from Transifex (20150901)

Fix an issue with the key message.admin.guide.read on pt_BR L10N in web UI at the root cause

* pr/770:
  Update L10N resource files on 4.5 branch with 4.5 translation strings from Transifex (20150901) Fix an issue with the key message.admin.guide.read on pt_BR L10N in web UI at the root cause

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-02 10:36:56 +05:30
Mike Tutkowski f5b5197048 Support live migration on older version of Libvirt 2015-09-01 16:00:08 -06:00
Milamber a71df257b6 Update L10N resource files on 4.5 branch with 4.5 translation strings from Transifex (20150901)
Fix an issue with the key message.admin.guide.read on pt_BR L10N in web UI at the root cause
2015-09-01 14:01:08 +01:00
Rohit Yadav 5a6928c679 Merge pull request #750 from shapeblue/4.5-CLOUDSTACK-8766
CLOUDSTACK-8766: Fix infinite scrolling pagination for zonal iso/template listing

Due to aggregation of templates and isos on the UI/client side, it could result
that for each page we could end up having lesser templates/isos listed to have
the scroll shown that triggers infinite scrolling. The fix is to use similar
approach as in projects.js, to use the maximum pagesize of 500. In theory, there
is still a chance if there are 500 zones with the same template being listed
resulting in only one aggregated template which could cause the scroll to not
get shown; but in practice I believe this fix should work for most users.

* pr/750:
  CLOUDSTACK-8766: Fix infinite scrolling pagination for zonal iso/template listing

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-31 14:29:25 +05:30
Rohit Yadav 711acfa93d CLOUDSTACK-8762: Confirm disk activity before starting a VM
Implements a VM volume/disk file activity checker that checks if QCOW2 file
has been changed before starting the VM. This is useful as a pessimistic
approach to save VMs that were running on faulty hosts that CloudStack could
try to launch on other hosts while the host was not cleanly fenced. This is
optional and available only if you enable the settings in agent.properties
file, on per-host basis.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-28 15:52:06 +05:30
Rohit Yadav 069e3f92ad CLOUDSTACK-8766: Fix infinite scrolling pagination for zonal iso/template listing
Due to aggregation of templates and isos on the UI/client side, it could result
that for each page we could end up having lesser templates/isos listed to have
the scroll shown that triggers infinite scrolling. In theory, there
is still a chance if there are several zones with the same template being listed
resulting in only one aggregated template which could cause the scroll to not
get shown; but in practice I believe this fix should work for most users.

Page size set based on experimental data:
https://github.com/apache/cloudstack/pull/751#issuecomment-135661968

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-28 13:06:13 +05:30
Rohit Yadav e967df9efa Merge pull request #748 from shapeblue/4.5-br-i18n-fix
messages: fix string in pt_BR translation fileThe string error caused JS error that failed translation on the UI

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* pr/748:
  messages: fix string in pt_BR translation file

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-27 14:12:09 +05:30
Rohit Yadav fad723df1b messages: fix string in pt_BR translation file
The string error caused JS error that failed translation on the UI

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-27 10:44:43 +05:30
Remi Bergsma 5522120b0e Merge pull request #739 from shapeblue/CLOUDSTACK-8766
CLOUDSTACK-8766: Fix infinite scrolling pagination for zonal template listing

Uses listViewDataProvider to implement pagination on listTemplates API
requests in the UI, when a template's zone tab is viewed.

* pr/739:
  CLOUDSTACK-8766: Fix infinite scrolling pagination for zonal template listing

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-08-27 01:11:51 +02:00
Rohit Yadav acf1baf1a3 Merge pull request #733 from shapeblue/kvm-linkbr-checks
CLOUDSTACK-8749: Add checks to prevent malformed/unexpected inputBased on @jburwell's comment on PR #718

* pr/733:
  CLOUDSTACK-8749: Add checks to prevent malformed/unexpected input

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-26 14:20:13 +05:30
Rohit Yadav 2a382e000b CLOUDSTACK-8749: Add checks to prevent malformed/unexpected input
Based on @jburwell's comment on PR #718

This closes #733

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-26 13:36:22 +05:30
Rohit Yadav 26700fbe76 CLOUDSTACK-8766: Fix infinite scrolling pagination for zonal template listing
Uses listViewDataProvider to implement pagination for listing templates and ISOs
in the zones tab. Dedupes isos and templates in the list views.

This closes #739

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-26 12:41:31 +05:30
Koushik Das 3d9cbf0bcb CLOUDSTACK-8733: Host stuck in rebalancing state during agent LB
This is happening as ClusterServiceServletAdapter is started after ClusteredAgentManagerImpl.
Fix is to start ClusterServiceServletAdapter before ClusteredAgentManagerImpl.

(cherry-picked from c989921fb7)
2015-08-25 09:37:50 +05:30
Rohit Yadav 0b4f97251a Merge remote-tracking branch 'origin/4.5-RC20150819T1442' into 4.5 2015-08-23 20:47:31 +05:30
Remi Bergsma 363cc7bc1f Improve cloud-install-sys-tmplt to work in dev environment again
Backported PR #678 to 4.5

This I changed:
 ``jasypt='/usr/share/cloudstack-common/lib/jasypt-1.9.0.jar'``
in master it is 1.9.2. I changed it to 1.9.0 here, to match the original script.

Original commit IDs:
  2f858a7d08
  ee9b644e28
  8a1e79f518

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-08-19 19:59:24 +02:00
Remi Bergsma 43dabb611d RHEL 7 and CentOS 7 need the same fix
(cherry picked from commit d1cb4c7d50)
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-08-19 19:35:38 +02:00
Remi Bergsma 91cfb6068a fixing white space and formatting
(cherry picked from commit 14013d5d1b)
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-08-19 19:35:24 +02:00
Remi Bergsma c0a801a98a Merge pull request #717 from shapeblue/vm-snapshots-uuid-expose
CLOUDSTACK-8748: VM UUID accessible in CreateVMSnapshotCommand and RevertToVMSnapshotCommand

This patch makes it possible to expose VM UUID to subsystems, this can be
useful for implementing VM Snapshots for KVM in future.

* pr/717:
  CLOUDSTACK-8748: VM UUID accessible in CreateVMSnapshotCommand and RevertToVMSnapshotCommand

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-08-19 19:05:57 +02:00
Rohit Yadav 74f697a2dd CLOUDSTACK-8749: KVM - cleanup linklocal interface based on names
This tries to avoid cleaning by a device name.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

This closes #718
2015-08-19 21:42:47 +05:30
Rohit Yadav 0062ff2672 CLOUDSTACK-8748: VM UUID accessible in CreateVMSnapshotCommand and RevertToVMSnapshotCommand
This patch makes it possible to expose VM UUID to subsystems, this can be
useful for implementing VM Snapshots for KVM in future.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-19 15:31:37 +05:30
Rohit Yadav 7385441807 Updating pom.xml version numbers for release 4.5.2
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-19 14:42:54 +05:30
Rajani Karuturi 1a106ec16f Merge pull request #697 from milamberspace/update-4.5-20150814
Update L10N resource files on 4.5 branch with 4.5 translation strings from Transifex (20150814)

* pr/697:
  Update L10N resource files on 4.5 branch with 4.5 translation strings from Transifex (20150814)

Signed-off-by: Rajani Karuturi <rajanikaruturi@gmail.com>
2015-08-19 14:08:37 +05:30