Commit Graph

346 Commits

Author SHA1 Message Date
sureshanaparti 89111110cb
server: Remove the rule(s) validation with api names while importing role, to be in sync with the create role permission behavior (#4840)
This PR removes the rule(s) validation with api names while importing a role. This will be in sync with the current create role permission behavior.
2021-03-29 16:23:43 +05:30
Rohit Yadav c8f7c0d823 Merge remote-tracking branch 'origin/4.15'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-03-11 13:33:44 +05:30
Rohit Yadav 3f84b9ab42 Merge remote-tracking branch 'origin/4.14' into 4.15 2021-03-11 13:31:46 +05:30
Rohit Yadav 057ad2b7d9
usage: return guest OS uuid, guest OS name, category ID and name (#4755)
This fixes the ostype ID returned in listUsageRecords API response to
be uuid instead of internal DB ID and also returns the os category ID
(uuid) and name.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-03-11 13:08:55 +05:30
Abhishek Kumar 025b654b5d
server: use network details from nic network (#4772)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-03-11 13:06:44 +05:30
Rohit Yadav a1be9b02a6 Merge remote-tracking branch 'origin/4.15' 2021-03-06 16:02:09 +05:30
Wei Zhou df4103f0d1
novnc: Add source IP check (#4736)
* novnc: Add client IP check for novnc console in cloudstack 4.16

* novnc ip check : Fix restart CPVM or mgt server does not update novnc param

* novnc ip check: move to method
2021-03-06 15:08:34 +05:30
Rohit Yadav 1d8a4978e9 Merge remote-tracking branch 'origin/4.15'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-02-24 17:39:12 +05:30
Pearl Dsilva ccbcb08fbf
server: Increase leniency to list templates on secondary stores that have been marked deleted by updating the db (#4713)
This PR aims at restoring the previous level of leniency in listing templates on stores that have been marked as deleted / removed by updating the DB.
While Cloudstack doesn't allow deleting stores that have resources on them, it may so happen that users may mimic a deletion of a store by merely updating the DB. Under such a case, listing of templates is hampered due to an NPE that is caused. (as seen in #4606)

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-02-24 16:30:32 +05:30
Pearl Dsilva a896760a40
storage: Secondary storage allow store deletion after successful data migration (#4615)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-02-24 16:27:39 +05:30
sureshanaparti eba186aa40
storage: New Dell EMC PowerFlex Plugin (formerly ScaleIO, VxFlexOS) (#4304)
Added support for PowerFlex/ScaleIO (v3.5 onwards) storage pool as a primary storage in CloudStack (for KVM hypervisor) and enabled VM/Volume operations on that pool (using pool tag).
Please find more details in the FS here:
https://cwiki.apache.org/confluence/x/cDl4CQ

Documentation PR: apache/cloudstack-documentation#169

This enables support for PowerFlex/ScaleIO (v3.5 onwards) storage pool as a primary storage in CloudStack

Other improvements addressed in addition to PowerFlex/ScaleIO support:

- Added support for config drives in host cache for KVM
	=> Changed configuration "vm.configdrive.primarypool.enabled" scope from Global to Zone level
	=> Introduced new zone level configuration "vm.configdrive.force.host.cache.use" (default: false) to force host cache for config drives
	=> Introduced new zone level configuration "vm.configdrive.use.host.cache.on.unsupported.pool" (default: true) to use host cache for config drives when storage pool doesn't support config drive
	=> Added new parameter "host.cache.location" (default: /var/cache/cloud) in KVM agent.properties for specifying the host cache path and create config drives on the "/config" directory on the host cache path
	=> Maintain the config drive location and use it when required on any config drive operation (migrate, delete)

- Detect virtual size from the template URL while registering direct download qcow2 (of KVM hypervisor) templates

- Updated full deployment destination for preparing the network(s) on VM start

- Propagate the direct download certificates uploaded to the newly added KVM hosts

- Discover the template size for direct download templates using any available host from the zones specified on template registration
	=> When zones are not specified while registering template, template size discovery is performed using any available host, which is picked up randomly from one of the available zones

- Release the VM resources when VM is sync-ed to Stopped state on PowerReportMissing (after graceful period)

- Retry VM deployment/start when the host cannot grant access to volume/template

- Mark never-used or downloaded templates as Destroyed on deletion, without sending any DeleteCommand
	=> Do not trigger any DeleteCommand for never-used or downloaded templates as these doesn't exist and cannot be deleted from the datastore

- Check the router filesystem is writable or not, before performing health checks
	=> Introduce a new test "filesystem.writable.test" to check the filesystem is writable or not
	=> The router health checks keeps the config info at "/var/cache/cloud" and updates the monitor results at "/root" for health checks, both are different partitions. So, test at both the locations.
	=> Added new script: "filesystem_writable_check.py" at /opt/cloud/bin/ to check the filesystem is writable or not

- Fixed NPE issue, template is null for DATA disks. Copy template to target storage for ROOT disk (with template id), skip DATA disk(s)

* Addressed some issues for few operations on PowerFlex storage pool.

- Updated migration volume operation to sync the status and wait for migration to complete.

- Updated VM Snapshot naming, for uniqueness in ScaleIO volume name when more than one volume exists in the VM.

- Added sync lock while spooling managed storage template before volume creation from the template (non-direct download).

- Updated resize volume error message string.

- Blocked the below operations on PowerFlex storage pool:
  -> Extract Volume
  -> Create Snapshot for VMSnapshot

* Added the PowerFlex/ScaleIO client connection pool to manage the ScaleIO gateway clients, which uses a single gateway client per Powerflex/ScaleIO storage pool and renews it when the session token expires.

- The token is valid for 8 hours from the time it was created, unless there has been no activity for 10 minutes.
  Reference: https://cpsdocs.dellemc.com/bundle/PF_REST_API_RG/page/GUID-92430F19-9F44-42B6-B898-87D5307AE59B.html

Other fixes included:

- Fail the VM deployment when the host specified in the deployVirtualMachine cmd is not in the right state (i.e. either Resource State is not Enabled or Status is not Up)

- Use the physical file size of the template to check the free space availability on the host, while downloading the direct download templates.

- Perform basic tests (for connectivity and file system) on router before updating the health check config data
	=> Validate the basic tests (connectivity and file system check) on router
	=> Cleanup the health check results when router is destroyed

* Updated PowerFlex/ScaleIO storage plugin version to 4.16.0.0

* UI Changes to support storage plugin for PowerFlex/ScaleIO storage pool.
- PowerFlex pool URL generated from the UI inputs(Gateway, Username, Password, Storage Pool) when adding "PowerFlex" Primary Storage
- Updated protocol to "custom" for PowerFlex provider
- Allow VM Snapshot for stopped VM on KVM hypervisor and PowerFlex/ScaleIO storage pool

and Minor improvements in PowerFlex/ScaleIO storage plugin code

* Added support for PowerFlex/ScaleIO volume migration across different PowerFlex storage instances.

- findStoragePoolsForMigration API returns PowerFlex pool(s) of different instance as suitable pool(s), for volume(s) on PowerFlex storage pool.
- Volume(s) with snapshots are not allowed to migrate to different PowerFlex instance.
- Volume(s) of running VM are not allowed to migrate to other PowerFlex storage pools.
- Volume migration from PowerFlex pool to Non-PowerFlex pool, and vice versa are not supported.

* Fixed change service offering smoke tests in test_service_offerings.py, test_vm_snapshots.py

* Added the PowerFlex/ScaleIO volume/snapshot name to the paths of respective CloudStack resources (Templates, Volumes, Snapshots and VM Snapshots)

* Added new response parameter “supportsStorageSnapshot” (true/false) to volume response, and Updated UI to hide the async backup option while taking snapshot for volume(s) with storage snapshot support.

* Fix to remove the duplicate zone wide pools listed while finding storage pools for migration

* Updated PowerFlex/ScaleIO volume migration checks and rollback migration on failure

* Fixed the PowerFlex/ScaleIO volume name inconsistency issue in the volume path after migration, due to rename failure
2021-02-24 14:58:33 +05:30
Rohit Yadav 6634382f69 Merge remote-tracking branch 'origin/4.15'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-02-19 15:30:21 +05:30
Abhishek Kumar f489439a3a
api: add zone, vm name params in listVmSnapshot response (#4604)
* api: add zone, vm name params in listVmSnaphots response

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-02-19 14:52:48 +05:30
Sina Kashipazha 543f9827ff
add creation date as a value for domains and accounts. (#4649) 2021-02-13 10:07:53 +00:00
Abhishek Kumar d6e8b53736
vmware: vm migration improvements (#4385)
- Fixes inter-cluster migration of VMs
- Allows migration of stopped VM with disks attached to different and suitable pools
- Improves inter-cluster detached volume migration
- Allows inter-cluster migration (clusters of same Pod) for system VMs, VRs on VMware
- Allows storage migration for stopped system VMs, VRs on VMware within same Pod if StoragePool cluster scopetype

Linked Primate PR: https://github.com/apache/cloudstack-primate/pull/789 [Changes merged in this PR after new UI merge]
Documentation PR: https://github.com/apache/cloudstack-documentation/pull/170

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-02-12 12:41:41 +05:30
Daan Hoogland b6b778f003 Merge release branch 4.14 to 4.15
* 4.14:
  server: select root disk based on user input during vm import (#4591)
  kvm: Use Q35 chipset for UEFI x86_64 (#4576)
  server: fix wrong error message when create isolated network without SourceNat (#4624)
  server: add possibility to scale vm to current customer offerings (#4622)
  server: keep networks order and ips while move a vm with multiple networks (#4602)
  server: throw exception when update vm nic on L2 network (#4625)
  doc: fix typo in install notes (#4633)
2021-02-01 09:57:35 +00:00
Wei Zhou a44fb11a02
server: add possibility to scale vm to current customer offerings (#4622)
We can use cloudmonkey to scale a vm with dynamic offering, to same offering but with different cpunumber or memory.
Enable it on UI to improve user experience.
2021-02-01 14:14:48 +05:30
davidjumani 93ff156222
Adding zone name to physicalnetworkresponse (#4510) 2020-12-02 14:01:22 +05:30
Rohit Yadav df07e27921 Merge remote-tracking branch 'origin/4.14' 2020-12-01 14:04:53 +05:30
davidjumani afebfd5bbf
Adding cpuallocated percentage and value to host and hostsformigrationresponse (#4499)
* Adding cpuallocatedwithoverprovisoning to hostresponse and hostsformigrationresponse

* Adding cpuallocatedpercentage and cpuallocatedvalue
2020-12-01 13:56:19 +05:30
Rakesh beb1edcdbc
api: Add vpcid in usage network response (#4361)
* Add vpcid in usage network response

Currently vpcid is displayed in listUsageNetworks response.
Add the vpcid so that we can see to which vpc, the network belongs

* use new function to get removed
2020-11-23 14:02:25 +05:30
Rakesh 0a2a54aeba
api: Display VPC name to which the network belongs to (#4483)
* Display VPC name to which the network belongs to

If an isolated network is created in VPC then display
its name along with vpc id which is used for UI

* Change description
2020-11-21 02:20:40 +05:30
Daan Hoogland 492962238e Merge branch '4.14' 2020-11-20 11:43:20 +00:00
davidjumani d79d24261a
Adding memoryallocatedpercentage & memoryallocatedbytes to HostsResponse & HostsForMigrationResponse (#4478) 2020-11-20 11:27:47 +00:00
Rakesh 2f5ed9eee4
Consider other conditions while listing templates with id (#4452) 2020-11-17 20:03:49 +01:00
Pearl Dsilva 7f408ec967
Fix: Listing projects comprising of only the user's on listAll=true (#4469)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-11-16 13:37:11 +01:00
Pearl Dsilva fb78fb24c7
fix login issue post upgrade (#4465)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-11-12 13:09:25 +00:00
Daan Hoogland 3ab43e2edb Merge branch '4.14' 2020-10-30 15:58:12 +00:00
Gabriel Beims Bräscher b3a1cb41c8
Allow to configure root disk size via Service Offering (diskoffering of type Service). (#4341) 2020-10-30 15:56:11 +00:00
Daan Hoogland 46ead2df71 Merge branch '4.13' into 4.14 2020-10-30 15:54:26 +00:00
slavkap 8afb451c1c
fix NPE in volumes statistics (#4388) 2020-10-30 15:53:05 +00:00
Daan Hoogland 2b220b5624 Merge branch '4.14' 2020-10-30 10:05:24 +01:00
Daan Hoogland b0d300c681 Merge branch '4.13' into 4.14 2020-10-30 10:03:56 +01:00
Wei Zhou dfd59321b1
server: fix format error with memorywithoverprovisioning in list hosts response (#4435) 2020-10-30 09:38:59 +01:00
Rohit Yadav 6de8e012e8 Merge remote-tracking branch 'origin/4.14'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-10-28 15:56:14 +05:30
Wei Zhou dbb66850b2
server: update template to another template type (#3945)
this contains other changes
(1) add isrouting field for vm templates on UI
(2) show register URL of template/iso on UI
(3) set 'Bootable' field to changable for existing ISO
2020-10-28 15:41:36 +05:30
Pearl Dsilva 963d603ede
Fix usage record count (#4193)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-10-21 19:15:34 +02:00
Harikrishna Patnala 1e4e5cf4c0 Disk offering response: added vsphere storage policy to response if exists 2020-10-19 15:05:58 +05:30
Harikrishna Patnala a54436b790 service offering response: added vsphere storage policy to response if exists 2020-10-19 15:05:58 +05:30
nvazquez 94bebe8792 Revert back deploy as is column on templates but keep it as default for new templates 2020-10-19 15:05:57 +05:30
nvazquez 08c0b07b59 Remove deployasis parameter from register template API 2020-10-19 15:05:57 +05:30
Harikrishna Patnala 1638d7243c Get allocated size for datastoreCluster from its child datastores to show the accumalated value for Datastore cluster 2020-10-19 15:05:57 +05:30
Harikrishna Patnala 33ae2afc89 Removed few duplicate imports during rebase with master 2020-10-19 15:05:57 +05:30
nvazquez f73830acbb Refactor deploy as is constants 2020-10-19 15:05:57 +05:30
nvazquez bb4ce2118d Add new template and vm deploy as is details table and refactor 2020-10-19 15:05:57 +05:30
nvazquez d864e9dc39 [VMware] Full OVF properties support 2020-10-19 15:05:56 +05:30
Harikrishna Patnala c16b40e337 Fix list volumes cmd and update volume cmd to accept datastore cluster as storgeid 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 97ae846c1e Removed unused imports caused due to some merge conflicts 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 85d440d973 Fix list volume response to list datastore cluster name and id instead of child datastore 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 1244fca367 Handle listsystemvms cmd in case of datastore clusters 2020-10-19 14:57:16 +05:30
Harikrishna Patnala fc05d3168f Fix ListVMsCmd with storageid as datastore cluster 2020-10-19 14:57:16 +05:30
Harikrishna Patnala c9c38c0f37 Handle datastore clusters in list storage pools to list only parent not the children in the Datastore cluster 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 6df819028e UI changes and accept any type of datastore as presetup in vmware 2020-10-19 14:57:15 +05:30
Gabriel Beims Bräscher 4c936b4690
Re-add affinity group (#4405) 2020-10-16 09:16:19 +02:00
Rakesh 2333d97098
plugins: Consider maintenance mode as offline for promethues stats (#4366)
If the resource state of hypervisor in "Maintenance" then it
should be considered as offline even though the agent state
is "Up". Since its in maintenance mode, it cant be used to
allocate VM's and hence can't be considered towards resource
allocation
2020-10-14 15:42:02 +05:30
Gabriel Beims Bräscher 2e32a3a94a
List VMs by Security Group & HA (#4397) 2020-10-14 09:26:47 +02:00
davidjumani aab8df09aa
Fixing searchAndCount searchAndDistinctCount when sc is null (#4374) 2020-10-13 11:30:04 +02:00
Pearl Dsilva 82b6971258
server: Handle listProjects API to list projects with user as members when listAll=true (#4316)
* added defensive checks for avoiding NPE and list projects API fix

* list projects with account name provided to not include users in the account in response

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-09-17 10:20:34 +05:30
davidjumani 44bc134162
Adding acl name to several responses (#4315) 2020-09-17 10:13:14 +05:30
Pearl Dsilva b464fe41c6
server: Secondary Storage Usage Improvements (#4053)
This feature enables the following:
Balanced migration of data objects from source Image store to destination Image store(s)
Complete migration of data
setting an image store to read-only
viewing download progress of templates across all data stores
Related Primate PR: apache/cloudstack-primate#326
2020-09-17 10:12:10 +05:30
Rakesh 43a25c78f6
Display acl name in listNetworks response (#4317)
* Display acl name in listNetworks response

Display acl name along with its id so that we
dont need to make extra api call to get acl name

* Add since tag
2020-09-11 14:36:20 +02:00
Greg Goodrich 9b72e5dfb8
Adding os type id to the usage record response for virtual machines (#4266) 2020-08-25 16:54:08 +05:30
Rohit Yadav 961f3ece41 Merge remote-tracking branch 'origin/4.14' 2020-08-21 14:31:52 +05:30
Rakesh 9681a28c7f
api: Display hypervisor type for VM snapshot (#4275)
The "hypervisor" field in listvmsnapshot response will
be used in primate to enable/disable creating snapshot
from vm snapshot functionality.

Creating snpashot from vm snapshot will be enabled only if
hypervisor is KVM
2020-08-20 14:13:49 +05:30
Spaceman1984 b586eb22f1
Human readable sizes in logs (#4207)
This PR adds outputting human readable byte sizes in the management server logs, agent logs, and usage records. A non-dynamic global variable is added (display.human.readable.sizes) to control switching this feature on and off. This setting is sent to the agent on connection and is only read from the database when the management server is started up. The setting is kept in memory by the use of a static field on the NumbersUtil class and is available throughout the codebase.

Instead of seeing things like:
2020-07-23 15:31:58,593 DEBUG [c.c.a.t.Request] (AgentManager-Handler-12:null) (logid:) Seq 8-1863645820801253428: Processing: { Ans: , MgmtId: 52238089807, via: 8, Ver: v1, Flags: 10, [{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-224-VM","bytesSent":"106496","bytesReceived":"0","result":"true","details":"","wait":"0",}}] }

The KB MB and GB values will be printed out:

2020-07-23 15:31:58,593 DEBUG [c.c.a.t.Request] (AgentManager-Handler-12:null) (logid:) Seq 8-1863645820801253428: Processing: { Ans: , MgmtId: 52238089807, via: 8, Ver: v1, Flags: 10, [{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-224-VM","bytesSent":"(104.00 KB) 106496","bytesReceived":"(0 bytes) 0","result":"true","details":"","wait":"0",}}] }

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Human+Readable+Byte+sizes
2020-08-13 15:55:16 +05:30
Pearl Dsilva c578004fe5
projects: Role based users in Projects (#4128)
Enabling Role Based users in projects
Primate PR related to the FR: apache/cloudstack-primate#382
Doc PR: https://github.com/apache/cloudstack-documentation/pull/145

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2020-08-13 15:45:39 +05:30
Rohit Yadav db665fa8df Merge remote-tracking branch 'origin/4.14'
Conflicts:
	engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-05 14:46:13 +05:30
Rakesh a529470b16
server: Fix cpuallocated value in findHostsForMIgration api (#4220)
The findHostsForMigration api displays 0% always for
cpuallocated field which is wrong.

Fixes #4221
2020-08-05 14:02:09 +05:30
Rohit Yadav 2fce8b733e
vmware: volume utilisation is always zero (#4225)
This fixes issues of virtual size to be twice in case the disk is a
linked-clone root disk. The virtual size of root disk (first in chain)
must be used.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-05 14:01:16 +05:30
Rohit Yadav 36ef85012f Merge remote-tracking branch 'origin/4.14'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-07-29 14:08:02 +05:30
davidjumani eec56025c8
api: Bug fixes for primate (#4214)
Adding the following fixes so primate can work without issues :
- Adding pagination for listNetworkAclLists
- Adding pagination for listRoles
- Returning mshost uuid rather than msid in list hosts response
- Allowing listVirtualMachinesMetrics to respect hostid
- Fixing return all details in template response
2020-07-29 12:26:39 +05:30
Rohit Yadav dfc76e0278 Merge remote-tracking branch 'origin/4.14' 2020-07-08 11:37:14 +05:30
Rohit Yadav ba767783bd Merge remote-tracking branch 'origin/4.13' into 4.14 2020-07-08 11:36:30 +05:30
Rohit Yadav 139aa13e6a
server: Purge all cookies on logout, set /client path on login (#4176)
This will purge all the cookies on logout including multiple sessionkey
cookies if passed. On login, this will restrict sessionkey cookie
(httponly) to the / path.

Fixes #4136

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-07-08 08:03:51 +05:30
sureshanaparti 5040283db9
server: Dynamic roles improvements. Add-on functionality below. (#4071)
- Create a role from any of the existing role, using new parameter roleid in createRole API
- Import a role with its rules, using a new importRole API
- New default roles for Read-Only and Support Admin & User
- No modifications allowed for Default roles
- Cleaned up old NetApp APIs from role_permissions table.
2020-07-07 10:56:43 +05:30
Rohit Yadav 3ede1eaa49 Merge remote-tracking branch 'origin/4.14' 2020-06-25 10:48:02 +05:30
Rohit Yadav 26f4edff34 Merge remote-tracking branch 'origin/4.13' into 4.14
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-06-25 10:42:15 +05:30
davidjumani b79407c50b
api: Adding missing fields to API responses (#4167)
Adding missing fields in the following APIs
osdisplayname in listVirtualMachines
vpcofferingname in listVpcs
vpcname in listPublicIpAddresses
vpcname in listPrivateGateways
vpcname in listVpnGateways
templatename, podname in listRouters
templatename, podname in listSystemVms

Fixes: #4161
2020-06-25 10:05:30 +05:30
Rohit Yadav 5471802f86 Merge remote-tracking branch 'origin/4.14' 2020-06-24 08:24:29 +05:30
Rohit Yadav 615ab4ef2c Merge remote-tracking branch 'origin/4.13' into 4.14 2020-06-24 08:23:43 +05:30
Abhishek Kumar 8010718878
server: fix for wrong affinity group count (#4154)
Fixes wrong count in listAffinityGroup API.
API was returning the count of AffinityGroupJoinVO records. 

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-06-24 07:02:56 +05:30
Rohit Yadav 5f695c910d Merge remote-tracking branch 'origin/4.14' 2020-06-18 19:48:45 +05:30
Rohit Yadav 148fab4e43 Merge remote-tracking branch 'origin/4.13' into 4.14 2020-06-18 19:48:12 +05:30
davidjumani 06f3ff0b04
api: listVirtualMachinesMetrics should extend ListVMsCmd instead of ListVMsCmdByAdmin (#4145)
Fixes #4143

Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-06-18 19:47:13 +05:30
Rohit Yadav a2c92029f5 Merge remote-tracking branch 'origin/4.14' 2020-06-18 09:50:54 +05:30
Rohit Yadav 88c5e6e6df Merge remote-tracking branch 'origin/4.13' into 4.14 2020-06-18 09:50:20 +05:30
davidjumani e9f59e2fd3
server: Adding showunique parameter to list templates and isos (#4140)
Adds a new parameter showunique to listTemplate and listIsos to return only unique templates / isos across all zones

Fixes #4041
2020-06-18 09:05:36 +05:30
Rakesh 20f9639842
api: Display network name for IP in shared networks (#4073)
In the list publicipaddress api call, display the network
name if ip is associated to shared network

Co-authored-by: Rakesh Venkatesh <r.venkatesh@global.leaseweb.com>
2020-06-13 06:00:46 +05:30
Rakesh 1fe96cc426
server: Search VR using redundant state (#4075)
Allow VR's to be searched using its redundant state
Under Infrastructure -> Virtual Routers -> Search box
we can search using "MASTER", "BACKUP" and this will display
the VR's matching the state.

Co-authored-by: Rakesh Venkatesh <r.venkatesh@global.leaseweb.com>
2020-06-12 12:22:31 +05:30
Rakesh f4f238e18e
server: Disable searching by instance name for customers (#4077)
Only admins should be able to search VM by instance name
Customers should not see or serach VM's using the instance name (i-)

Co-authored-by: Rakesh Venkatesh <r.venkatesh@global.leaseweb.com>
2020-06-11 17:22:23 +05:30
Rakesh 6bf3769c09
Display image store disk size used and total disk size in (#4046)
list imagestore api response
2020-06-08 07:05:58 +05:30
Daan Hoogland b984184b7a Merge release branch 4.13 to master
* 4.13:
  Snapshot deletion issues (#3969)
  server: Cannot list affinity group if there are hosts dedicated… (#4025)
  server: Search zone-wide storage pool when allocation algothrim is firstfitleastconsumed (#4002)
2020-04-11 16:45:00 +02:00
Wei Zhou e0b67a4c68
server: Cannot list affinity group if there are hosts dedicated… (#4025) 2020-04-10 09:10:51 +02:00
Rohit Yadav db2e2136dd Merge remote-tracking branch 'origin/4.13' 2020-03-25 12:54:21 +05:30
Rohit Yadav 2e3390f06e
server: export full response view for zones response when caller is root admin (#3989)
The listZonesMetrics does not return same keys are listZones as the
default response view is restricted. This fixes that by ensuring that
for root admin full response view is used.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-03-25 12:53:41 +05:30
pavanaravapalli d4b537efa7
UEFI Implementation: Enabled UEFI Support for Guest VM's on Hypervisor KVM,VMware. enabled boot modes [Legacy,Secure] support for UEFI boot with known caveats. (#3638)
Co-authored-by: Pavan Kumar Aravapalli <pavan_aravapalli@accelerite.com>
Co-authored-by: dahn <daan.hoogland@shapeblue.com>
2020-03-13 20:56:26 +01:00
Nicolas Vazquez efe00aa7e0
[KVM] Rolling maintenance (#3610) 2020-03-12 16:59:46 +01:00
Radu Todirica d549f3bdc8
Add cache mode param properly (#3925) 2020-03-09 13:17:21 +01:00
Pearl Dsilva 51794deb62
Regression Fix: Allow full response view to Admin user (#3940)
Change Response view to Full for Admin user

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-03-07 02:40:33 +05:30
Abhishek Kumar 8cc70c7d87
CloudStack Kubernetes Service (#3680) 2020-03-06 08:51:23 +01:00
Rohit Yadav 318924d801
CloudStack Backup & Recovery Framework (#3553) 2020-03-03 13:27:58 +01:00
Daan Hoogland a62a10c814 Merge branch '4.13' 2020-02-26 16:18:41 +01:00
Pearl Dsilva 4d8a2da133
api: Fix count and item issues returned by list APIs (#3894) 2020-02-26 15:14:23 +00:00
Rohit Yadav ccda5fb776 Merge remote-tracking branch 'origin/4.13' 2020-02-10 19:02:09 +05:30
Rohit Yadav 78cc0a44c1
server: use host record related to a ssvm/cpvm (#3876)
This implements the systemvm list API response creator to find and use
the host record for a ssvm/cpvm to get the agent status and other
details like last disconnected date and agent version.

Fixes 3875

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-02-10 19:01:34 +05:30
Rohit Yadav 70d1535df4 Merge remote-tracking branch 'origin/4.13' 2020-02-07 18:05:39 +05:30
Rohit Yadav afcbbc4b3e
systemvm: list systemvm does not return agent state and version (#3870)
This makes the listSystemVms API to return the host status (agent state),
version and last pinged information. This makes it possible for UIs
to call a single API to get this information.
2020-02-07 13:19:35 +01:00
Wei Zhou fd5bea838b
New feature: Add support to destroy/recover volumes (#3688)
* server: fix resource count of primary storage if some volumes are Expunged but not removed

Steps to reproduce the issue
(1) create a vm and stop it. check resource count of primary storage
(2) download volume. resource count of primary storage is not changed.
(3) expunge the vm, the volume will be Expunged state as there is a volume snapshot on secondary storage. The resource count of primary storage decreased.
(4) update resource count of the account (or domain), the resource count of primary storage is reset to the value in step (2).

* New feature: Add support to destroy/recover volumes

* Add integration test for volume destroy/recover

* marvin: check resource count of more types

* messages translate to JP

* Update messages for CN

* translate message for NL

* fix two issues per Daan's comments

Co-authored-by: Andrija Panic <45762285+andrijapanicsb@users.noreply.github.com>
2020-02-07 11:25:10 +01:00
Anurag Awasthi c0abfce8fa
Health check feature for virtual router (#3575) 2020-01-30 12:39:03 +01:00
Rohit Yadav 518ed5379c Merge remote-tracking branch 'origin/4.13' 2020-01-30 11:13:14 +05:30
davidjumani 7a25e40d5a
api: allow listing management server by id and name (#3840)
The List Management Server api returns a list of all the management servers but fails when trying to list by id or name. This ensures that it fetches the details as per the parameters passed.
Fixes: #3833
2020-01-30 10:38:25 +05:30
Pearl Dsilva 1c130a5dd4
api: metrics API response is not super-set of resources response keys (#3834)
The metrics API has few properties missing that are present in the corresponding resource. 

Fixes #3831

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Rohit Yadav <rohit@apache.org>
2020-01-30 08:49:45 +05:30
dahn 5ff932eb86
Ldap fixes (#3694)
* pass domainid for list users

* passing arg in wizzard

* adding userfilter to list ldap users and usersource to response

  port of list ldap users tests to java

* assertion of differnt junit ldap methods

* broken test for directory server (and others)

* embedded context loading

* add user and query test

* UI: filter options passing filter and domain and onchange trigger

* disable tests that only work in ide

prereqs for domain-linkage fixed

move trigger to the right location in code

trigger for changing domain

* logging, comments and refactor

implement search users per domain

retrieve appropriate list of users to filter

get domain specific ldap provider

* query cloudstack users with now db filter

* recreate ldap linked account should succeed

* disable auto import users that don't exist

* ui choice and text

* import filter and potential remove from list bug fixed

* fix rights for domain admins

* list only member of linked groups not of principle group

* Do not show ldap user filter if not importing from ldap
  do not delete un-needed items from dialog permanently
  delete from temp object not from global one

* localdomain should not filterout users not imported from ldap

* several types of authentication handling errors fixed and unit tested

* conflict in output name

* add conflict source field to generic import dialog

* replace reflextion by enum member call

* conflict is now called conflict 🎉
2020-01-20 16:02:33 +01:00
Daan Hoogland 4fa7fadd21 Merge release branch 4.13 to master
* 4.13:
  break session only on illegal origin (#3715)
2019-12-24 10:24:09 +01:00
dahn 8844f59247
break session only on illegal origin (#3715) 2019-12-24 10:22:34 +01:00
dahn 482d0fd85e
4.13 (#3782)
* create template from snapshot regression (partly reverted) (#3767)

* Once again allow a VM to be on multiple networks from VPCs (#3754)

to once again allow a VM to be on multiple networks from VPCs

* convert protocal names to be found as labels (#3747)

* convert protocal names to be found as labels

* format

* filter hosts to query on zone wide storage (#3733)

* config: add isdynamic flag in configuration response (#3729)

Co-authored-by: Wei Zhou <ustcweizhou@gmail.com>
2019-12-20 16:43:09 +01:00
Wei Zhou 2712decf50 config: add isdynamic flag in configuration response (#3729) 2019-12-20 15:17:21 +01:00
Dennis Konrad 2e8c069dd2 Load Average for KVM (#3738)
* Avgload (#2)

* Adding avgload for kvm

* Fix coding style issue

* Add getter/setter

* Fix several small errors

* Add override

* Uncomment getAverageLoad

* Override getAverageLoad()

* Checkstyle bug?

* Delete trailing spaces

* Renaming function

* Change interface to match

* Rename method in GetHostStatsAnswer

* Change method call name

* Convert double to long

* Remove trailing whitespace

* Change names around

* Make load visible to return it

* Parse string to double

* Change Long to Double

* Fix getter

* Unify naming to cpuloadaverage

* Change cpuloadaverage String to Double in listHostsMetrics
Remove some unnecessary whitespaces

* Add CPU_LOAD_AVERAGE to ApiConstants
2019-12-17 10:37:05 +01:00
Rohit Yadav 7f91964aa9
refactoring: Remove duplicate code (by Frank/Nuage) (#3538)
Refactor: Cleanup duplicate code

Make use of Java 8 default implementation in interfaces,
to remove code duplication between XxxCmd and XxxCmdAsAdmin.
Refactor checkFormat by pre-calculating the supported
extensions. Also make use of this in ImageStoreUtil.
Makes it easier to add new file and compression formats.
2019-09-25 10:21:35 +05:30
Nicolas Vazquez bfc08715cc Display VM snapshot tags on usage records (#3560)
* Refactor usage helper tables to include VM snapshot id

* Fix resource type and resource id while listing usage records

* Add defensive checks
2019-08-20 14:20:23 +01:00
Nicolas Vazquez 3c2af55d81 vmware: allow configuring appliances on the VM instance wizard when OVF properties are available (#3271)
Problem: In Vmware, appliances that have options that are required to be answered before deployments are configurable through vSphere vCenter user interface but it is not possible from the CloudStack user interface.

Root cause: CloudStack does not handle vApp configuration options during deployments if the appliance contains configurable options. These configurations are mandatory for VM deployment from the appliance on Vmware vSphere vCenter. As shown in the image below, Vmware detects there are mandatory configurations that the administrator must set before deploy the VM from the appliance (in red on the image below):

Solution:
On template registration, after it is downloaded to secondary storage, the OVF file is examined and OVF properties are extracted from the file when available.
OVF properties extracted from templates after being downloaded to secondary storage are stored on the new table 'template_ovf_properties'.
A new optional section is added to the VM deployment wizard in the UI:
If the selected template does not contain OVF properties, then the optional section is not displayed on the wizard.
If the selected template contains OVF properties, then the optional new section is displayed. Each OVF property is displayed and the user must complete every property before proceeding to the next section.
If any configuration property is empty, then a dialog is displayed indicating that there are empty properties which must be set before proceeding
image
The specific OVF properties set on deployment are stored on the 'user_vm_details' table with the prefix: 'ovfproperties-'.
The VM is configured with the vApp configuration section containing the values that the user provided on the wizard.
2019-08-09 16:14:46 +05:30
Rohit Yadav 263b404386 Revert "Refactoring to remove duplicate code. (#3152)"
This reverts commit 7a27e35a61.

We're near 4.13 RC1, we've low confidence if the changes from #3152
would cause other regressions so reverting this. The author may send a
PR again towards 4.14.

Regressions found are all related to template and iso registration and
upload.
2019-08-02 13:04:12 +05:30
Rohit Yadav d3b292ace9
server: Fixes API permissions, global settings and template UX (#3534)
Fixes:

- This allows getUploadParamsForIso for all user role types, also fixes
authorised field for getUploadParamsForTemplate API.
- Fix global setting description to say what is used when value is empty/blank.
- For VM running/allocated usage description, use parenthesis to return the instance name and ID.
- Display template download progress when template is added to a project

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-08-01 13:30:34 +05:30
Anurag Awasthi cf0649def1 server: Add size to listUsageRecords response for VMSnapShotOnPrimary (type 27) (#3529)
Add size to list usage records for VMSnapShotOnPrimary (type 27).
The size was included in description but missing from response.
2019-07-31 12:39:10 +05:30
Frank Maximus 7a27e35a61 Refactoring to remove duplicate code. (#3152)
Make use of Java 8 default implementation in interfaces,
to remove code duplication between XxxCmd and XxxCmdAsAdmin.

Refactor checkFormat by pre-calculating the supported
extensions. Also make use of this in ImageStoreUtil.
Makes it easier to add new file and compression formats.
2019-07-27 20:21:43 +05:30
Gabriel Beims Bräscher 6a336f8bc1 server: disable unauthenticated integration.api.port by default (#3504)
Set integration.api.port to (0) zero as default. CloudStack provides CloudStack API Unauthenticated Access through port 8096. It should not be open to the Internet in any case.
2019-07-22 17:22:02 +05:30
Rohit Yadav 5d8157422d Allow users to share templates with Accounts or Projects through the UI
* Allow users to share templates with Accounts or Projects through the
updateTemplate permissions API

* Change behaviour to show only supported projects and accounts with update template permissions

* Allow admins to see accounts dropdown and only hide lists for users

* Don't allow sharing project owned templates as you cannot retrieve them in list api calls
2019-07-18 17:42:55 +01:00
Paul Angus e15c3112e4
Merge pull request #3248 from shapeblue/storage-offering-domains-zones
Enable service offerings to be scoped to domain(s) and zone(s)
2019-07-16 10:25:35 +01:00
Rohit Yadav 97df52956c
master: travis and trillian smoketests fixes and stabilisation (#3476)
Fix failing test, add more component tests, optimise and refactor test jobs.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-12 17:27:49 +05:30
Abhishek Kumar 7010f855b5 Merge branch 'master' into storage-offering-domains-zones 2019-07-12 10:21:51 +05:30
Wei Zhou 6780930ecc server: Add option 'details' to listProjects and listAccounts (#3331)
If there are many projects and accounts, listing projects/accounts will take long time getting the resource limitation and resource count in the process. However resource count/limitation are not needed sometimes.

Add an option 'details' to listProjects and listAccounts. If you do not need the resource count/limitation, please add details=min to api call. The api execution time will be reduced significantly.
2019-07-04 16:26:04 +05:30
Wei Zhou 452b48ea0c server: reduce execution time while listing project if projects have many resource tags (#3306)
If projects have many resource tags, it will take a long time to list projects.
Remove resource tags information from project_view will fix it the issue.

Fixes #3178
2019-07-03 11:41:53 +05:30
Abhishek Kumar fc8381549d Merge branch 'master' into storage-offering-domains-zones 2019-06-28 17:33:41 +05:30
Abhishek Kumar 58474530f6 api: snapshot, snapshotpolicy tag support (#3228)
Problem: Currently tags cannot be applied to snapshot when it is being created but through separate “create tags” API calls. For snapshot policies tags cannot be set either at creation or through “create tags” API.

Root Cause: The “create snapshots” API does not support adding tags during creation and it can only be done through “create tags” API. Snapshot policy as a resource does not support tags and no tags can be set for them through any API.

Solution: Tag support for snapshot policy has been added. Snapshot policy with tags when executed will produce snapshots containing the same tags from snapshot policy.

Following APIs have been updated:

Both “create snapshotpolicy” and “create snapshot” now accepts “tags” as a new parameter. The expected format for “tags” parameter is similar to parameter “tags” in “create tags“ API.
Deletion support for tags associated with snapshots policy has been added to “delete snapshotpolicies” API.
Tags set for snapshot policies are added to the Response of “list snapshotpolicies“ API.
UI support for setting tags to snapshots and snapshot policy is provided through the corresponding menus with a new section in each form to set tags.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-06-27 09:21:08 +05:30
Rohit Yadav 14bff7bd03
server: export granular volume bytes and iops metrics (#3259)
Problem: The VM metrics has aggregated volume bytes read/write and iops metrics but not on per volume basis.
Root Cause: The volume stats sub-system is not used to export the metrics, the support is not available for VMware.
Solution: Use the volume stats sub-system and DB table to export the metrics via the listVolumes and listVolumeMetrics API, and implement support for VMware and fix issue with network and disk metrics in the VM metrics view.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-27 09:18:10 +05:30
Rohit Yadav 9f4f2c5348
api: instance and template details are free text (#3240)
Problem: Users don't know what keys/values to enter for template and VM details.
Root Cause: The feature does not exist that can list possible details and options.
Solution: Based on the possible VM and template details handled by the
codebase, those details were refactored and a list API is introduced
that can return users those details along with possible values. When
users add details now, they will be presented with a list of key details
and their possible options if any.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-27 09:14:47 +05:30
Abhishek Kumar 12218e398d Merge branch 'master' into storage-offering-domains-zones
schema and server changes for VPC offering to honour changes from f653e61 in master
2019-06-26 13:27:00 +05:30
Abhishek Kumar 8cff58d3b8 Merge branch 'master' into storage-offering-domains-zones 2019-06-26 02:03:25 +05:30
Rohit Yadav 2c32d60e44
server: return usage description with resource names and UUIDs (#3344)
The usage record descriptions have CloudStack's internal integer IDs
which makes it difficult for users to read their usages. This PRs
introduces a new API boolean flag `oldformat` which when set to true
would return the older description format, otherwise by default
listUsageRecords will process and return description with names and
UUIDs of resources.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-21 03:48:33 +05:30
Rohit Yadav 61ebc6a4a6
api: Set network name as part of the network usage response (#3234)
Problem: Network name is not part of the network usage response
Root Cause: Code does not set the network name
Solution: Set the network name for network usage type usage records in the API response

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-20 21:39:15 +05:30
Rohit Yadav 26a5d50760
Merge branch 'master' into storage-offering-domains-zones 2019-06-20 15:09:48 +05:30
Rohit Yadav 90cd8aa73d
server: add support for sorting zones in UI/API (#3242)
Problem: Not able to configure a sort order for the zones that are listed in various views in the UI.

Root Cause: There is no mechanism to accept sort key for existing zones or UI widget, that would allow to listing zones in the UI in a certain order.

Solution: The order of zones in listed in various views in the UI can now be configured through the newly added “sort_key” field added for the zone. It can be set using updateZone API by providing “sort_key” parameter for a zone, or by reordering the items in the zones list in the UI. UI has been updated to show ordering controls in zones list view. Database changes include updating table “data_center” by adding “sort_key” column (containing integer values and defaults to zero).

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-19 18:03:56 +05:30
Dingane Hlaluku 0d6cae6339 volume: fix volume metrics view from returning sensitive info to end user (#3222)
Problem: The listVolumeMetrics API response does not honor the volume detail visibility restrictions set for normal users and returns sensitive information which should only be visible to the root admin.

Root Cause: The listVolumeMetrics API response extends the ListVolumesByAdmin API internally and this results in a full display view response that is only meant for the root admin.

Solution: This has been fixed by rectifying the API response to not show ‘physical size’, 'storage type', and ‘storage pool’ information. The UI has also been fixed to hide these columns for normal users.
2019-06-19 17:04:26 +05:30
Rohit Yadav 0e87040eb2
network: allow ability to specify if network's ipaddress usage need to be hidden (#3235)
Problem: Admins don’t want to charge for IP address usage on certain (shared) networks.
Root Cause: There is no flag or detail for admins to provide using UI or API when creating networks to specify if they want IP address usage of the network hidden.
Solution: A new boolean hideipaddressusage flag is added to the createNetwork API and a checkbox in the ‘Add guest network’ UI for the root admins to specify if they want the shared network’s IP address usage to be hidden in the listUsageRecords API response. The provided flag is saved as the ‘hideIpAddressUsage’ detail in the cloud.network_details table for the network. For existing (shared) networks, root admins can also specify the same boolean API parameter hideipaddressusage with the updateNetwork API request to configure the behaviour for an existing network. When the detail/flag is true, the IP address usage for the (shared) network is not exported in the listUsageRecords API response. The listNetworks API response will include the details of a network for root admin only. (note usage is still recorded in the usage database but not return by the listUsageRecords API)

The API flag works for any kind of network via the API, but the checkbox is only shown while creating shared networks in the UI.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-19 10:25:59 +05:30
Abhishek Kumar cf347c89ea Merge branch 'master' into storage-offering-domains-zones 2019-06-18 12:52:34 +05:30
Anurag Awasthi c4f6c43852 api: Add to listRouters response the VR scripts version (#3372)
Add missing scriptsversion field to list router api call response.

Fixes #3357
2019-06-10 22:50:56 +05:30
Abhishek Kumar f78b99cc8f refactoring: code review comment changes
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-05-30 16:42:14 +05:30
Rohit Yadav 6d82e63e53
Merge branch 'master' into storage-offering-domains-zones 2019-05-30 15:04:32 +05:30
Abhishek Kumar 46a32795bc ui: instance settings visibility (#3244)
This change allows instance Settings tab to be visible but inaccessible when instance is running. A warning is shown when user tries to access Settings for a running instance and tab content is greyed out.
It also allows some admin defined instance settings/details to be made static for user. User will be able to see them in instance settings tab but cannot change their values as action buttons are disabled and greyed out. This can be achieved by providing a comma-separated list details for global settings key 'user.vm.readonly.ui.details'. A new value 'readonlyuidetails' has been added in UserVMResponse for UI manipulate editing functionality of settings/details.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-05-24 12:28:15 +02:00
Abhishek Kumar 8e999adc4f server: fix for failing searchForNetworkOfferings unit test
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-24 10:18:05 +05:30
Abhishek Kumar 366b253bf2 server: create, update VPC offering for domain(s) & zone(s)
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-24 10:11:00 +05:30
Abhishek Kumar c671e07c18 server: update network offering for specified domain(s) & zone(s)
Added checkAccess while creating network from offering.
Response fixes for networkoffering APIs.

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-24 10:11:00 +05:30
Abhishek Kumar dc589a442d server: create network offering for specified domain(s) and zone(s)
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-24 10:11:00 +05:30
Abhishek Kumar 2102888116 server: changes for domain, zone specified service offerings
Added changes for creating service offerings for specified domain(s) and zone(s).
Fixed checkAccess for disk offerings.
Fixed list APIs for disk and service offerings.
UI changes for creating disk, service offerings for specified domain(s) and zone(s).

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-24 10:09:52 +05:30
Rohit Yadav dd0294e163 add helper method for findings offerings by zone id
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 10:04:10 +05:30
Rohit Yadav d04c6ea276 remove getDomainId and write the workaround code
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 10:04:10 +05:30
Rohit Yadav e3da9ec645 fix db issues
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Rohit Yadav 6a902460af find in set implementation
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Rohit Yadav 8cabbcea92 wip domainId removal stuff
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Rohit Yadav 5988884bd1 wip changes
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Rohit Yadav 7721126076 remove domain id usage
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Rohit Yadav 9c117f2781 DB upgrade path and UI fixes
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Rohit Yadav 728afba5d4 refactorings
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Abhishek Kumar c85b3e597a server: ability to create disk offerings for domain(s) and zone(s)
Allows creating storage offerings associated with particular domain(s) and zone(s). In create disk/storage offfering form UI, a mult-select control has been addded to select desired zone(s) and domain select element has been made multi-select.
createDiskOffering API has been modified to allow passing list of domain and zone IDs with keys domainids and zoneids respectively. These lists are stored in DB in cloud.disk_offering_details table with 'domainids' and 'zoneids' key as string of comma separated list of IDs. Response for create, update and list disk offering APIs will return domainids, domainnames, zoneids and zonenames in details object of offering.
listDiskOfferings API has been modified to allow passing zoneid to return only offerings which are associated with the zone.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-05-24 09:56:23 +05:30
Frank Maximus e11f7ee1ba RIP Nuage Cloudstack Plugin (#3146)
may it rest in peaces
2019-05-14 10:58:24 +02:00
Rohit Yadav 67160478a6
server: allow admins to blacklist vm details that users should not see (#3213)
This introduces a new global setting `user.vm.blacklisted.details` that
allows admins to blacklist VM details that non-admin users should not
see via the VM's settings tab.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-10 23:39:13 +05:30
Rohit Yadav bfad334117
api: include tags in listvmsnapshots response (#3216)
This ensures that tags of a VM snapshot are listed in the UI, available
in the list vmsnapshots API response.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-10 23:37:18 +05:30
Abhishek Kumar 6bb8d977ac ui: show complete domain for accounts (#2994) (#3256)
This allows showing complete domain, ie, domain path for accounts list view and account detail.
Added a new key, domainpath, in AccountResponse.

Fixes #2994

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-04-10 17:18:01 +05:30
Dingane Hlaluku 0d21f0c1e7 api: rename ListUsageRecords file name to ListUsageRecordsCmd (#3151)
* Rename ListUsageRecords API command file name to ListUsageRecordsCmd

* Refactor to use APINAME variable and remove unused s_logger field

* Remove unused import
2019-03-25 13:10:53 +05:30
Nathan Johnson 637cc6ec4e feature: add libvirt / qemu io bursting (#3133)
* feature: add libvirt / qemu io bursting

Adds the ability to set bursting features from libvirt / qemu

This allows you to utilize the iops and bytes temporary "burst" mode
introduced with libvirt 2.4 and improved upon with libvirt 2.6.

https://blogs.igalia.com/berto/2016/05/24/io-bursts-with-qemu-2-6/

* updates per rafael et al
2019-02-04 19:47:44 -02:00
dahn 58466c8954 api: add command to list management servers (#3150)
* api: add command to list management servers

* api: add number of mangement servers in listInfrastructure command

* ui: add block for mangement servers on infra page

* api name resolution method cleanup
2019-02-01 16:23:39 -02:00
dahn b363fd49f7 Vmware offline migration (#2848)
* - Offline VM and Volume migration on Vmware hypervisor hosts
- Also add VM disk consolidation call on successful VM migrations

* Fix indentation of marvin test file and reformat against PEP8

* * Fix few comment typos
* Refactor debug messages to use String.format() when debug log level is enabled.

* Send list of commands returned by hypervisor Guru instead of explicitly selecting the first one

* Fix unhandled NPE during VM migration

* Revert back to distinct event descriptions for VM to host or storage pool migration

* Reformat test_primary_storage file against PEP-8 and Remove unused imports

* Revert back the deprecation messages in the custom StringUtils class to favour the use of the ApacheUtils
2019-01-25 10:05:13 -02:00
Malcolm Taylor 97ddd8dffd Fix some LGTM alerts (#3143)
* Fix some LGTM alerts

* address review comments from Gabriel Brascher and Rafael Weingartner
2019-01-24 20:22:39 -02:00
Anurag Awasthi 1a6eb4b856 server: use resource UUID instead of resource ID in API response (#2527) (#3099)
List resourcedetails was adding DB ID instead of UUID. This is a security risk and needs fix.

Fixes: #2527
2019-01-15 14:05:16 +05:30
Anurag Awasthi 3279707030 api: Throw InvalidParameterValueException for failing ApiArgValidator (#3108)
This is important because it helps in communicating back the exact
error to the API callee.

Current behavior is that ParamProcessWorker#processParameters catches
the exception and returns an incorrect type exception without the
proper message.
2018-12-24 23:50:55 +05:30
Anurag Awasthi 6a4a4daa46 api: Allow password enabled for iso (#2745) (#3042)
Both register and update iso commands can allow for allowing password
enabled.
Fixes: #2745

Fixe: #2745
2018-11-28 00:09:20 +05:30
Yoan Blanc 17c164d59a api: signature v3 to accept more formats (#2893)
It does it by reusing the DateUtil helpers. DateUtil uses java.time.* as that one knows how to deal
with timezones correctly.

The format expected by signatureVersion=3&expires=.... is quite limited.

It should accept the following formats that are containing a timezone and/or milliseconds.

2018-10-01T08:12:14Z
2018-10-01T08:12:14+01:00
2018-10-01T08:12:14+0100
2018-10-01T08:12:14.000Z
2018-10-01T08:12:14.000+01:00
2018-10-01T08:12:14.000+0100
afaik only 2018-10-01T08:12:14+0100 is accepted by the current codebase.

This PR echoes other pull requests I made earlier this year. #2392 and #2867

Signed-off-by: Yoan Blanc <yoan.blanc@exoscale.ch>
2018-10-31 16:57:48 +05:30
Abhishek 6f44a8d7ba template: Return requireshvm key as part of list template response (#2921)
Added requireshvm key in the response of list template API.
Fixes #2821

Signed-off-by: Abhishek Kumar <abhishek.mrt@gmail.com>
2018-10-24 16:20:27 +05:30
Rafael Weingärtner 71e5a88fc1 Forward merge client: mgmt server listen default to 0.0.0.0 (#2907) 2018-10-22 16:32:06 -03:00
Pierre-Yves Ritschard 58b4e71b0e
params: allow signatureversion and expires without logging
This patch considers the new expires and signatureversion parameters
valid. Without this, all calls log when using the V3 signature scheme.
2018-10-16 09:01:46 +02:00
Kui LIU d53fc94485 CLOUDSTACK-10365: Change the "getXXX" boolean method names to "isXXX" (#2847)
These boolean-return methods are named as "getXXX".
Other boolean-return methods are named as "isXXX".
Considering there methods will return boolean values, it should be more clear and consistent to rename them as "isXXX".
(rebase #2602 and #2816)
2018-09-22 17:20:48 +02:00
Gabriel Beims Bräscher fbf488497f Support IPv6 address in addIpToNic (#2773)
The admin will manually need to add the address to the Instance, but the
Security Grouping should allow it.
2018-09-11 12:03:19 -03:00
Rohit Yadav 5e48c0b4c9 Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-08-08 12:20:56 +05:30
ernjvr 542d4da16c asyncjobs: add endtime to async jobs (#2739)
There is currently no functional mechanism that captures or persists the end time of when an asynchronous job has finished. As a result, users are not able to do any reporting about the duration of various asynchronous jobs in Cloudstack.
Link to FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Add+End+Time+To+Asynchronous+Jobs
2018-07-25 14:48:01 +05:30
Mike Tutkowski c7d6376964 Removing an old, unused NetApp plug-in 2018-06-08 12:55:39 -06:00
Rohit Yadav ada1e730c8 Merge branch '4.11' 2018-05-22 17:03:24 +05:30
Rohit Yadav 65511c4335 Merge branch '4.11': Reduce VR downtime during network restart (#2508)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-05-11 13:09:03 +05:30
Rohit Yadav f77b1a7d00 Merge branch '4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-05-09 18:04:53 +05:30
Marc-Aurèle Brothier d94a5bea3c api: Fixes #2528 move ostypeid from id to uuid (#2608)
This moves the ostypeid in api response to uuid instead of integer id.
2018-05-03 16:30:26 +05:30
Marc-Aurèle Brothier 586edec7ac CLOUDSTACK-10253: JSON response for SuccessResponse as boolean instead of string (#2428)
* JSON response for SuccessResponse as boolean instead of string

Signed-off-by: Marc-Aurèle Brothier <m@brothier.org>

* Fixing test cases
2018-04-27 21:07:51 -03:00
Rohit Yadav 5ff2f172cb Merge branch '4.11' 2018-04-24 15:05:38 +02:00
lzh3636 c1c587fffe CLOUDSTACK-10334: Fix inadequate information for handling catch clauses (#2510)
* Add log stack traces

* Change log stack traces

Add original stack trace to the new throw and delete the error logs
2018-03-28 07:02:15 -03:00
Khosrow Moossavi 1708838518 CLOUDSTACK-10268: Fix and enhance package script (#2433)
- new flag `-T, --use-timestamp` to use `timestamp` when POM version contains SNAPSHOT
  - in the final artifacts (jar) name
  - in the final package (rpm, deb) name
  - in `/etc/cloudstack-release` file of SystemVMs
  - in the Management Server > About dialog
- if there's a "branding" string in the POM version (e.g. `x.y.z.a-NAME[-SNAPSHOT]`),
the branding name will be used in the final generated pacakge name such as following:
  - `cloudstack-management-x.y.z.a-NAME.NUMBER.el7.centos.x86_64`
  - `cloudstack-management_x.y.z.a-NAME-NUMBER~xenial_all.deb`
- branding string can be overriden with newly added `-b, --brand` flag
- handle the new format version for VR version
- fix long opts (they were broken)
- tolerate and show a warning message for unrecognized flags
- usage help reformat

* Deprecate Version class in favor or CloudStackVersion
2018-03-15 10:55:29 +00:00
Rafael Weingärtner 7efdaa65f7
[CLOUDSTACK-10314] Add Text-Field to each ACL Rule (#2475)
* [CLOUDSTACK-10314] Add Text-Field to each ACL Rule

It is interesting to have a text field (e.g. CHAR-256) added to each ACL rule, which allows to enter a "reason" for each FW Rule created. This is valuable for customer documentation, as well as best practice for an evidence towards auditing the system

* Formatting to make check style happy and code clean ups
2018-03-13 11:07:35 -03:00
Rafael Weingärtner 73251bf830 [CLOUDSTACK-10259] Missing float part of secondary storage data in listAccounts 2018-01-30 19:52:02 -02:00
Marc-Aurèle Brothier 893a88d225 CLOUDSTACK-10105: Use maven standard project structure in all projects (#2283)
Remove maven standard module (which only a few were using) and get ride of maven customization for the projects structure.

- moved all directories to src/main/java, src/main/resources, src/main/scripts, src/test/java, src/test/resources
- grep scan to search for src/com and src/org left over
- grep for <project>/scripts to fix pom.xml configuration
- remove custom <build> configuration in pom.xml

Signed-off-by: Marc-Aurèle Brothier <m@brothier.org>
2018-01-20 03:19:27 +05:30