Commit Graph

25565 Commits

Author SHA1 Message Date
John Burwell fe03b91bb8 Merge pull request #11 from shapeblue/nio-fix-4.5
CLOUDSTACK-9348: Use non-blocking SSL handshake
2016-05-06 12:57:37 -04:00
Rohit Yadav ff2729d411 CLOUDSTACK-9348: Optimize NioTest and NioConnection main loop
- Reduces SSL handshake timeout to 15s, previously this was only 10s in
  commit debfcdef78
- Adds an aggresive explicit wakeup to save the Nio main IO loop/handler from
  getting blocked
- Fix NioTest to fail/succeed in about 60s, previously this was 300s
- Due to aggresive wakeup usage, NioTest should complete in less than 5s on most
  systems. On virtualized environment this may slightly increase due to thread,
  CPU burst/scheduling delays.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-06 10:15:00 +05:30
Rohit Yadav e51f524039 CLOUDSTACK-9348: Use non-blocking SSL handshake
- Uses non-blocking SSL handshake and non-blocking connections
- Uses 60s as timeout for both client/server to guard against indefinitely
  blocking clients
- Unit test to prove fix, client and malicious clients trying to connect to server

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-06 10:07:47 +05:30
Rohit Yadav 460a56fdbf Merge pull request #12 from apple/dynamicrbac-4.5 into apple-base-4.5 2016-04-26 13:29:07 +05:30
Rohit Yadav ff9db9cadc dynamic roles: fix role_id for default system users
In case of new deployments, roleid will be set to NULL for system/admin accounts
causing admin accounts to be not able to log in.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-04-26 13:11:59 +05:30
Rohit Yadav 0856820802 client: remove commands.properties file
We don't need commands.properties file now, remove it

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-04-26 13:11:51 +05:30
John Burwell f1040024af Merge pull request #7 from shapeblue/dynamicrbac-4.5
Dynamic RBAC roles for CloudStack
2016-04-25 14:02:28 -04:00
Rohit Yadav 16323f83f3 engine: fix upgrade path to work with mysql 5.7
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-04-25 14:52:08 +05:30
Rohit Yadav f30c52a16c CLOUDSTACK-8562: DB-Backed Dynamic Role Based API Access Checker
This feature allows root administrators to define new roles and associate API
permissions to them.

A limited form of role-based access control for the CloudStack management server
API is provided through a properties file, commands.properties, embedded in the
WAR distribution. Therefore, customizing API permissions requires unpacking the
distribution and modifying this file consistently on all servers. The old system
also does not permit the specification of additional roles.

FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+Role+Based+API+Access+Checker+for+CloudStack

DB-Backed Dynamic Role Based API Access Checker for CloudStack brings following
changes, features and use-cases:
- Moves the API access definitions from commands.properties to the mgmt server DB
- Allows defining custom roles (such as a read-only ROOT admin) beyond the
  current set of four (4) roles
- All roles will resolve to one of the four known roles types (Admin, Resource
  Admin, Domain Admin and User) which maintains this association by requiring
  all new defined roles to specify a role type.
- Allows changes to roles and API permissions per role at runtime including additions or
  removal of roles and/or modifications of permissions, without the need
  of restarting management server(s)

Upgrade/installation notes:
- The feature will be enabled by default for new installations, existing
  deployments will continue to use the older static role based api access checker
  with an option to enable this feature
- During fresh installation or upgrade, the upgrade paths will add four default
  roles based on the four default role types
- For ease of migration, at the time of upgrade commands.properties will be used
  to add existing set of permissions to the default roles. cloud.account
  will have a new role_id column which will be populated based on default roles
  as well

Dynamic-roles migration tool: scripts/util/migrate-dynamicroles.py
- Allows admins to migrate to the dynamic role based checker at a future date
- Performs a harder one-way migrate and update
- Migrates rules from existing commands.properties file into db and deprecates it
- Enables an internal hidden switch to enable dynamic role based checker feature

Deprecate commands.properties

- Fixes apidocs and marvin to be independent of commands.properties usage
- Removes bundling of commands.properties in deb/rpm packaging
- Removes file references across codebase

Reviewed-by: John Burwell <john.burwell@shapeblue.com>
QA-by: Boris Stoyanov <boris.stoyanov@shapeblue.com>

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-04-25 14:52:02 +05:30
Rohit Yadav d0416e3524 metrics: Fix issues, implement search feature
- Fix threshold calculations for host and storage metric views
- Implement advance search box for all metrics views
- Save and pass context through navigation
- Add complex nav support, fix context usage

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-20 12:58:59 +01:00
Rohit Yadav c164ffee3b metrics: Increase UI width, fix metrics ui issues
- Add instances count in host view, ip address in instances view
- Add ipaddress in instances view
- Increase UI container width by 200px
- Allow all users to see resource metrics
- Fix instance count issue on host metrics view
- Fix sorting issue in metrics tables
- Fix instances null check regression from 2f250e2

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-20 12:57:09 +01:00
Rohit Yadav 938f1d2a7d CLOUDSTACK-9020: Metrics views for CloudStack UI
Implements following:
- A metrics table widget that is:
  - vertically and horizontally scrollable with pagination/infinite scrolling
  - sortable columns (client side)
  - groupable/collapsible columns
  - alternate row coloring
  - refresh button to refresh views
  - threshold table cell coloring
  - panel/breadcrumb navigation
  - quick view action column
  - translatable labels
- Sortable column for all CloudStack tables (client side)
- Configurable UI pagesize for list API calls, 'default.ui.page.size'
- Metrics views: Zones, Clusters, Hosts, Instances, Storage pools, Volumes
- Resource filtering/navigation: Zones->Clusters->Hosts->Instances->Volumes, Storage Pool->Volumes

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-20 12:54:06 +01:00
ramamurtis 173e2462b3 CLOUDSTACK-9006 - ListTemplates API returns result in inconsistent order when called concurrently
Adding filter test to verify addOrderBy method.

(cherry picked from commit 9d1a469ae5)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-20 12:54:06 +01:00
ramamurtis d82a373604 CLOUDSTACK-9006 - ListTemplates API returns result in inconsistent order when called concurrently
(cherry picked from commit c90723f491)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-20 12:54:05 +01:00
Rohit Yadav 88e1645d0b CLOUDSTACK-9019: Add storage network offering in ssvm only if storage network is defined
During creation of SSVM, checks and adds NetworkOffering.SystemStorageNetwork to
offerings only if storage network exists for the target datacenter

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-20 12:54:05 +01:00
Nitin Kumar Maharana 3a6054e549 CLOUDSTACK-8913: Search box in Templates tab out of alignment
Increased the margin and padding to accomodate all the boxes inside toolbar.

(cherry picked from commit febaeb2a15)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-20 12:54:05 +01:00
Rohit Yadav 847d69edfd CLOUDSTACK-9000: logrotate cloudstack-agent out and err logs
Adds logrotate rules for cloudstack-agent.{err,out} log files

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-20 12:54:04 +01:00
Rohit Yadav 8348a237b8 CLOUDSTACK-9000: logrotate cloudstack-agent out and err logs
Adds logrotate rules for cloudstack-agent.{err,out} log files

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-20 12:54:04 +01:00
Daan Hoogland 84e9ff746b CLOUDSTACK-8848: added null pointer guard to new public method 2016-01-20 12:54:04 +01:00
Rene Moser f7e1029cfc CLOUDSTACK-8848: ensure power state is up to date when handling missing VMs in powerReport
There 2 things which has been changed.

* We look on power_state_update_time instead of update_time. Didn't make sense to me at all to look at update_time.
* Due DB update optimisation, powerState will only be updated if < MAX_CONSECUTIVE_SAME_STATE_UPDATE_COUNT. That is why we can not rely on these information unless we make sure these are up to date.
2016-01-20 12:54:03 +01:00
Pierre-Luc Dion 6902dac319 Fix dockerbuild because hardcoded filename for marvin
Signed-off-by: Pierre-Luc Dion <pdion891@apache.org>
2016-01-20 12:54:03 +01:00
Boris Schrijver db771cd0a4 Added Unit Tests for QCOW2Utils. 2016-01-20 12:54:03 +01:00
Boris Schrijver 8a5aa0faaa Fixed Findbugs issue introduced by 1c6378ec00 PR #795. 2016-01-20 12:54:02 +01:00
Boris Schrijver decaef6cb4 Added QCOW2 virtual size checking for S3.
- Cleaned up S3TemplateDownloader
- Created static QCOW2 utils class.
- Reformatted some parts of DownloadManagerImpl
2016-01-20 12:54:02 +01:00
Rohit Yadav 11abfb8766 tools/appliance: update Debian 7 iso URLs based on 615261
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-16 12:56:25 +05:30
Suresh Kumar Anaparti 52e23996b8 CLOUDSTACK-8820: Updated the code for vCenter6 data center support.
(cherry picked from commit 1d73418c2c)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-16 12:52:46 +05:30
Suresh Kumar Anaparti 5f95f1688e CLOUDSTACK-8820: Showing error when try to add advance zone using VMWare ESXi 6.0 host
Summary: In vCenter 6.0, response headers need to be fetched after service login for server cookie unlike previous versions of vCenter.

(cherry picked from commit 4a6e2cdeee)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-16 12:52:18 +05:30
Rohit Yadav 17166eb631 schema: add 4.5.2 to 4.5.3 upgrade path stubs
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-15 09:57:48 +05:30
Rohit Yadav 284ce1fb88 engine/schema: add 4.5.2 to 4.5.3 upgrade path
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-15 09:57:12 +05:30
Rohit Yadav af38c492d8 debian: add 4.5.3-snapshot to changelog
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-14 10:01:58 +05:30
Rohit Yadav a4f552d46d cloudstack: set next version to 4.5.3-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-14 09:59:40 +05:30
Rohit Yadav d58dfbf21b Merge pull request #814 from wido/CLOUDSTACK-8443
CLOUDSTACK-8443: Support CentOS 7 for 4.5This is based on two PRs:
- 731
- 757

This commit is based on the 4.5 branch for a future 4.5 release.

* pr/814:
  CLOUDSTACK-8443: Support CentOS 7 for 4.5

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-14 09:52:06 +05:30
Wido den Hollander 2b8fd2469f CLOUDSTACK-8443: Support CentOS 7 for 4.5
This is based on two PRs:
- 731
- 757

This commit is based on the 4.5 branch for a future 4.5 release.
2015-09-13 15:30:20 +02:00
Rene Moser 15ef2b114c config: doc: add default value 10 in description for vm.network.nic.max.secondary.ipaddresses
value hardcoded by commit 4925b9f6a1

(cherry picked from commit 0b956bbc1f)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-08 16:56:15 +05:30
Rohit Yadav 047d759a73 Merge pull request #754 from shapeblue/4.5-CLOUDSTACK-8762
CLOUDSTACK-8762: Check to confirm disk activity before starting a VMImplements a VM volume/disk file activity checker that checks if QCOW2 file
has been changed before starting the VM. This is useful as a pessimistic
approach to save VMs that were running on faulty hosts that CloudStack could
try to launch on other hosts while the host was not cleanly fenced. This is
optional and available only if you enable the settings in agent.properties
file, on per-host basis.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* pr/754:
  CLOUDSTACK-8762: Confirm disk activity before starting a VM

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-02 14:54:58 +05:30
Rohit Yadav a6c630c54b Merge pull request #771 from mike-tutkowski/4.5
Support live migration on older version of Libvirthttps://issues.apache.org/jira/browse/CLOUDSTACK-8792

A flag being passed to Libvirt assumes v1.0.0 or later.

We need to put a check in the code to pass in a different flag if the version of Libvirt is < 1.0.0.

* pr/771:
  Support live migration on older version of Libvirt

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-02 10:38:48 +05:30
Rohit Yadav 6cdde0feb4 Merge pull request #770 from milamberspace/L10N-update-4.5-20150901
Update L10N resource files on 4.5 branch with 4.5 translation strings from Transifex (20150901)

Fix an issue with the key message.admin.guide.read on pt_BR L10N in web UI at the root cause

* pr/770:
  Update L10N resource files on 4.5 branch with 4.5 translation strings from Transifex (20150901) Fix an issue with the key message.admin.guide.read on pt_BR L10N in web UI at the root cause

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-02 10:36:56 +05:30
Mike Tutkowski f5b5197048 Support live migration on older version of Libvirt 2015-09-01 16:00:08 -06:00
Milamber a71df257b6 Update L10N resource files on 4.5 branch with 4.5 translation strings from Transifex (20150901)
Fix an issue with the key message.admin.guide.read on pt_BR L10N in web UI at the root cause
2015-09-01 14:01:08 +01:00
Rohit Yadav 5a6928c679 Merge pull request #750 from shapeblue/4.5-CLOUDSTACK-8766
CLOUDSTACK-8766: Fix infinite scrolling pagination for zonal iso/template listing

Due to aggregation of templates and isos on the UI/client side, it could result
that for each page we could end up having lesser templates/isos listed to have
the scroll shown that triggers infinite scrolling. The fix is to use similar
approach as in projects.js, to use the maximum pagesize of 500. In theory, there
is still a chance if there are 500 zones with the same template being listed
resulting in only one aggregated template which could cause the scroll to not
get shown; but in practice I believe this fix should work for most users.

* pr/750:
  CLOUDSTACK-8766: Fix infinite scrolling pagination for zonal iso/template listing

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-31 14:29:25 +05:30
Rohit Yadav 711acfa93d CLOUDSTACK-8762: Confirm disk activity before starting a VM
Implements a VM volume/disk file activity checker that checks if QCOW2 file
has been changed before starting the VM. This is useful as a pessimistic
approach to save VMs that were running on faulty hosts that CloudStack could
try to launch on other hosts while the host was not cleanly fenced. This is
optional and available only if you enable the settings in agent.properties
file, on per-host basis.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-28 15:52:06 +05:30
Rohit Yadav 069e3f92ad CLOUDSTACK-8766: Fix infinite scrolling pagination for zonal iso/template listing
Due to aggregation of templates and isos on the UI/client side, it could result
that for each page we could end up having lesser templates/isos listed to have
the scroll shown that triggers infinite scrolling. In theory, there
is still a chance if there are several zones with the same template being listed
resulting in only one aggregated template which could cause the scroll to not
get shown; but in practice I believe this fix should work for most users.

Page size set based on experimental data:
https://github.com/apache/cloudstack/pull/751#issuecomment-135661968

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-28 13:06:13 +05:30
Rohit Yadav e967df9efa Merge pull request #748 from shapeblue/4.5-br-i18n-fix
messages: fix string in pt_BR translation fileThe string error caused JS error that failed translation on the UI

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* pr/748:
  messages: fix string in pt_BR translation file

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-27 14:12:09 +05:30
Rohit Yadav fad723df1b messages: fix string in pt_BR translation file
The string error caused JS error that failed translation on the UI

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-27 10:44:43 +05:30
Remi Bergsma 5522120b0e Merge pull request #739 from shapeblue/CLOUDSTACK-8766
CLOUDSTACK-8766: Fix infinite scrolling pagination for zonal template listing

Uses listViewDataProvider to implement pagination on listTemplates API
requests in the UI, when a template's zone tab is viewed.

* pr/739:
  CLOUDSTACK-8766: Fix infinite scrolling pagination for zonal template listing

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-08-27 01:11:51 +02:00
Rohit Yadav acf1baf1a3 Merge pull request #733 from shapeblue/kvm-linkbr-checks
CLOUDSTACK-8749: Add checks to prevent malformed/unexpected inputBased on @jburwell's comment on PR #718

* pr/733:
  CLOUDSTACK-8749: Add checks to prevent malformed/unexpected input

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-26 14:20:13 +05:30
Rohit Yadav 2a382e000b CLOUDSTACK-8749: Add checks to prevent malformed/unexpected input
Based on @jburwell's comment on PR #718

This closes #733

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-26 13:36:22 +05:30
Rohit Yadav 26700fbe76 CLOUDSTACK-8766: Fix infinite scrolling pagination for zonal template listing
Uses listViewDataProvider to implement pagination for listing templates and ISOs
in the zones tab. Dedupes isos and templates in the list views.

This closes #739

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-26 12:41:31 +05:30
Koushik Das 3d9cbf0bcb CLOUDSTACK-8733: Host stuck in rebalancing state during agent LB
This is happening as ClusterServiceServletAdapter is started after ClusteredAgentManagerImpl.
Fix is to start ClusterServiceServletAdapter before ClusteredAgentManagerImpl.

(cherry-picked from c989921fb7)
2015-08-25 09:37:50 +05:30
Rohit Yadav 0b4f97251a Merge remote-tracking branch 'origin/4.5-RC20150819T1442' into 4.5 2015-08-23 20:47:31 +05:30