Commit Graph

31881 Commits

Author SHA1 Message Date
Abhishek Kumar 2020bfb6a3 server: allows compute offering with or without constraints (#3245)
Problem: Custom compute offering does not allow setting min and max values for CPU and VRAM for custom VMs.

Root Cause: Custom compute offerings cannot be created with a given range of CPU number and memory instead it allows only fixed values.

Solution: createServiceOffering API has been modified to allow setting a defined range for CPU number and memory. Also, UI form for compute offering creation is provided with a new field named 'compute offering type’ with values - Fixed, Custom Constrained, Custom Constrained. It will allow the creation of compute offerings either with a fixed CPU speed and memory for fixed compute offering, or with a range of CPU number and memory for custom constrained compute offering or without predefined CPU number, CPU speed and memory for custom unconstrained compute offering.

To allow the user to set CPU number, CPU speed and memory during VM deployment, UI form for VM deployment has been modified to provide controls to change these values. These controls are depicted in screenshots below for custom constrained and custom unconstrained compute offering types.

Sample API calls using cmk to create a constrained service offering and deploying a VM using it,

create serviceoffering name=Constrained displaytext=Constrained customized=true mincpunumber=2 maxcpunumber=4 cpuspeed=400 minmemory=256 maxmemory=1024

deploy virtualmachine displayname=ConstrainedVM serviceofferingid=60f3e500-6559-40b2-9a61-2192891c2bd6 templateid=8e0f4a3e-601b-11e9-9df4-a0afbd4a2d60 zoneid=9612a0c6-ed28-4fae-9a48-6eb207af29e3 details[0].cpuNumber=3 details[0].memory=800

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-23 11:47:53 +05:30
dahn 2ead7359d1 maven: refactor usage of slf4j version (#3325) 2019-05-23 11:23:18 +05:30
dahn f6ecac42ac ui: reverse character replacements (#3260)
In the cloudstack UI fields get '<', '>' and '&' replaced by xml-entities
these are generic for all fields and hurt us in the case of userdata
this fix calls the existing method to reverse character replacements.
it also removes the ccs class that pretends to prevent special chars

Fixes #3202
2019-05-23 11:20:42 +05:30
Anurag Awasthi 3e642852a7 ui: Bug fix for zone names not appearing in dashboard (#3326)
3 years ago when we moved from JSP this translate was not wrapped
in label tags and consequently the following span wasn't captured
when dashboard.js mapped the data.

Fixes: #3314
2019-05-15 09:52:38 +05:30
Rohit Yadav 0700d91a68 Merge branch '4.12'
- Fixes PR #3146 db cleanup to the correct 4.12->4.13 upgrade path
- Fixes failing unit test due to jdk specific changes after forward
  merging

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-14 15:15:17 +05:30
Frank Maximus e11f7ee1ba RIP Nuage Cloudstack Plugin (#3146)
may it rest in peaces
2019-05-14 10:58:24 +02:00
Rohit Yadav 00ff536f81 Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-14 14:26:11 +05:30
skattoju4 4c60a5b1ff Fix slow vm creation when large sf snapshot count (#3282)
* skip geting used bytes for volumes that are not in Ready state
* updated log message
* filter snapshots by state backedup
* removed * import
* filter templates by state 'DOWNLOADED'
* refactored getUsedBytes to use O(1) queries
* querying for ready volumes instead filtering in memory
* make listByStoreIdInReadyState more generic ex listByStoreIdAndState
* updated snapshot search criteria for listByStoreIdAndState
* updated template search criteria for listByPoolIdAndState
* fixed typo in search criteria for listByTemplateAndState
* fixed typo in search criteria for templates in listByPoolIdAndState
2019-05-11 16:02:52 +02:00
Rohit Yadav 9ff819da2c
systemvm: new qemu-guest-agent based patching for KVM (#3278)
This introduces a new patching script for patching systemvms on KVM
using qemu-guest-agent that runs inside the systemvm on startup. This
also removes the vport device which was previously used by the legacy
patching script and instead uses the modern and new uniform guest
agent vport for host-guest communication.

Also updates the sytemvmtemplate build config to use the latest Debian
9.9.0 iso.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-10 23:42:19 +05:30
Anurag Awasthi f9b61bc737 orchestration: Allow VM that has never started to have volumes attached (#3276)
With this patch b766bf7
we started tracking disks in attaching state so that other attach request can fail gracefully. However this missed the case where disks were in allocated state but attach was requested.

For the use case where users want to attach disk in allocated state but not ready, we need to have allocated-attaching transition as well. We must take care of returning to the original state - allocated or ready - when attach request has completed.

For the use case of unstarted vm's the disk must proceed as follows - "Allocated" -> Attaching -> Allocated. When VM is started, the disk is "created" and pool is assigned. For the use case of started VMs it's more trivial and disk proceeds as follows - Ready -> Attaching -> Ready.

Test this by creating a VM with "startvm=false", create a disk and try attaching it in allocated state. It would give an exception on latest 4.11 but will be fixed on this patch.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-10 23:40:38 +05:30
Rohit Yadav 67160478a6
server: allow admins to blacklist vm details that users should not see (#3213)
This introduces a new global setting `user.vm.blacklisted.details` that
allows admins to blacklist VM details that non-admin users should not
see via the VM's settings tab.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-10 23:39:13 +05:30
Rohit Yadav bfad334117
api: include tags in listvmsnapshots response (#3216)
This ensures that tags of a VM snapshot are listed in the UI, available
in the list vmsnapshots API response.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-10 23:37:18 +05:30
Dennis Konrad a75292e701
Merge pull request #3307 from webermaximilian92/feature_add_scss_to_css_compiler
All change requests were taken care of.
Multiple approvement are there
Merging this onto master
2019-05-10 16:41:06 +02:00
ustcweizhou 3729511c37 kvm: Fix three issues with Ubuntu 16.04 hosts (#3227)
* ubuntu16: fix unable to add host if cloudbrX is not configured

while add a ubuntu16.04 host with native eth0 (cloudbrX is not configured),
the operation failed and I got the following error in /var/log/cloudstack/agent/setup.log

```
DEBUG:root:execute:ifconfig eth0
DEBUG:root:[Errno 2] No such file or directory
  File "/usr/lib/python2.7/dist-packages/cloudutils/serviceConfig.py", line 38, in configration
    result = self.config()
  File "/usr/lib/python2.7/dist-packages/cloudutils/serviceConfig.py", line 211, in config
    super(networkConfigUbuntu, self).cfgNetwork()
  File "/usr/lib/python2.7/dist-packages/cloudutils/serviceConfig.py", line 108, in cfgNetwork
    device = self.netcfg.getDefaultNetwork()
  File "/usr/lib/python2.7/dist-packages/cloudutils/networkConfig.py", line 53, in getDefaultNetwork
    pdi = networkConfig.getDevInfo(dev)
  File "/usr/lib/python2.7/dist-packages/cloudutils/networkConfig.py", line 157, in getDevInfo
    elif networkConfig.isBridge(dev) or networkConfig.isOvsBridge(dev):
```

The issue is caused by commit 9c7cd8c248
2017-09-19 16:45 Sigert Goeminne ● CLOUDSTACK-10081: CloudUtils getDevInfo function will now return "bridge" instead o

* ubuntu16: Stop service libvirt-bin.socket while add a host

service libvirt-bin.socket will be started when add a ubuntu 16.04 host
DEBUG:root:execute:sudo /usr/sbin/service libvirt-bin start

However, libvirt-bin service will be broken by it after restarting
Stopping service libvirt-bin.socket will fix the issue.

An example is given as below.

```
root@node32:~# /etc/init.d/libvirt-bin restart
[ ok ] Restarting libvirt-bin (via systemctl): libvirt-bin.service.
root@node32:~# virsh list
error: failed to connect to the hypervisor
error: no valid connection
error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory

root@node32:~# systemctl stop libvirt-bin.socket

root@node32:~# /etc/init.d/libvirt-bin restart
[ ok ] Restarting libvirt-bin (via systemctl): libvirt-bin.service.
root@node32:~# virsh list
 Id    Name                           State
----------------------------------------------------

```

* ubuntu16: Diable libvirt default network

By default, libvirt will create default network virbr0 on kvm hypervisors.
If vm uses the same ip range 192.168.122.0/24, there will be some issues.

In some cases, if we run tcpdump inside vm, we will see the ip of kvm hypervisor as source ip.
2019-05-05 13:57:58 +05:30
Maximilian Weber 5f5ff7ed0b Add facebook scss file to apache rat plugin list 2019-05-02 11:47:33 +02:00
Gerd Müller de545229f9 Include 'removed' async jobs to check recurring snapshots (#3190) 2019-05-01 16:49:09 +02:00
Maximilian Weber 5343a190a0 Compile new scss files to css 2019-04-30 16:16:37 +02:00
Maximilian Weber f621b1a3a8 Copy css files to scss, lint auto fix them 2019-04-30 16:16:20 +02:00
Maximilian Weber 134408c18e Add new scss to css compiler with npm and gulp 2019-04-30 16:14:29 +02:00
Abhishek Kumar 6dad46b7ed server: sync templates on adding new secondary storage (#3302)
Fixes #2689

With the current code, existing templates were not downloaded to the new secondary storage when it is added. SSVM needed to be restarted to start the download process. This PR starts templates sync for the new secondary storage when it is added.

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-04-29 23:37:15 +05:30
Gabriel Beims Bräscher d6437d5aca server: use latest InfluxDB (2.15), add support to Batch Mode (#3289)
The InfluxDB Java client supports Batch Mode at versions 2.9+ [1]. Thus, this PR updated to the latest InfluxDB (2.15), adding support to Batch Mode

[1] https://github.com/influxdata/influxdb-java
2019-04-29 23:35:37 +05:30
ustcweizhou b60daf7142 server: Fix exception while update domain resource count (#3204) 2019-04-29 08:51:08 +02:00
Nico Wohlfarth 89d548052b Improvements after jquery update (#3183)
* Add jQuery ui stylings to the project

* Adapt stylings to jquery ui stylings

- remove unnecessary (empty) style definitions

* Check if jquery autocomplete is initialized before destroying it

* Use dialog modal option instead of setting z-index styling

- set minimum heigth of dialog

* Swap removeAttr() for prop() since it's not longer recommended as of jQuery 3.0

* Fix reccuring snapshots modal view

* Further style improvements

* More improvements on the instance-wizard (and multi-wizard in general)

* Raise input hints over modal z-index

* Adapt/fix stylings

- center buttons of create form modals
- fix width of select menu in service offerings
- fix width of modal in Network -> VPC -> configure -> static NAT -> aquire new Ip -> enable static nat

* Adapt width of dynamic-input container
2019-04-25 16:09:43 -03:00
Gabriel Beims Bräscher 8f7b27bbdc Mock Scanner, instead of scan the computer running the test. (#3173)
* Mock Scanner, instead of scan the computer running the test.

This allows non linux machines to run the tests without scanning for a
non existing /proc/meminfo.

* test fixes on 'other' platforms libvirt wrapper unit tests (#3)
2019-04-24 13:33:06 +02:00
Rohit Yadav 379d779a72
css: fix computer diagram margin that blocks down arrow (#3225)
Problem: Down arrow key of the vertical scroll bar in the VM deployment wizard does not work.
Root Cause: The computer diagram on the right side in the VM deployment wizard spills its margin on the left causing the arrow key to be blocked.
Solution: Fix the background-position to get the correct position than margin-left for the zone-panel. With the fix, the left size arrow keys and vertical scrollbars are no longer under the diagram image and press-able now.
2019-04-11 12:46:41 +05:30
Abhishek Kumar 6bb8d977ac ui: show complete domain for accounts (#2994) (#3256)
This allows showing complete domain, ie, domain path for accounts list view and account detail.
Added a new key, domainpath, in AccountResponse.

Fixes #2994

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-04-10 17:18:01 +05:30
Rohit Yadav 491a10be0c
storage: publish delete usage event for snapshot deletion (#3212)
Problem: Users are billed for destroyed VMs with VM snapshots because usage records don't get that the VM and VM snapshots are removed.
Root Cause: The destroyVirtualMachine and expungeVirtualMachine APIs were removing VM snapshots but not generating VMSNAPSHOT.DELETE usage event due to which the VM snapshots were not marked as removed in the usage_vmsnapshot table.
Solution: The issue was fixed by emitting the proper usage event for all the VM snapshots of a VM that is destroyed.
2019-04-10 17:12:55 +05:30
Rohit Yadav 1af5dedfb1
ui: don't ignore project when listing templates (#3233)
Problem: When uploading template or iso in a project, the progress is not seen
Root Cause: When users upload a template/iso in project view, the template/iso is not visible in the all filter. This creates confusion that template/iso has not been uploaded.
Solution: Since the api listtemplates with a projectid acts is a superset of list templates without a project id, we should not ignore the project id when listing templates in all filter.
2019-04-09 16:40:38 +05:30
Rohit Yadav ac6c84ec25 Merge remote-tracking branch 'origin/4.12' 2019-04-09 14:22:49 +05:30
Rohit Yadav 9488c6dcc5 Merge remote-tracking branch 'origin/4.11' into 4.12 2019-04-09 14:22:18 +05:30
Rohit Yadav 96611fc640
packaging: systemctl daemon-reload after agent install or upgrade (#3269)
This runs systemctl daemon-reload after cloudstack-agent is installed
or upgraded.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-04-09 14:21:09 +05:30
Abhishek Kumar 9b14e17a58 server: fix for vm snapshot search (#3208) (#3257)
Using db column instead of VO variable name was causing issue with SQL select statement.
This change fixes the problem by using VO variable for adding conditional.

Fixes #3208

Using db column instead of VO variable name was causing issue with SQL select statement.
This PR fixes the problem by using VO variable for adding conditional.
Additionally in UI listAll parameter was being sent twice in the listVMSnapshot API call. It is fixed with this PR.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-04-08 14:18:00 +05:30
Abhishek Kumar f86a7a058b utils: removed port check for url validation (#2802) (#3254)
This allows urls with ports other thatn 80 and 443

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-04-08 14:16:14 +05:30
Rohit Yadav cc24801c87 Merge remote-tracking branch 'origin/4.12' 2019-04-08 13:52:57 +05:30
Rohit Yadav 9171942565 Merge remote-tracking branch 'origin/4.11' into 4.12 2019-04-08 13:52:14 +05:30
Rohit Yadav 55efaf14d9
packaging: don't skip unit tests while building packages (#3266)
This may slow down CI and release, but ensures that unit tests always
run as part of the packaging build process.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-04-08 13:51:30 +05:30
Daan Hoogland 0237ee7cd5 Merge release branch 4.12 to master
* 4.12:
  fix bugs that may cause program crash, change mkdir to mkdirs (#3249)
2019-04-05 11:32:57 +02:00
bd2019us 09045bd3e5 fix bugs that may cause program crash, change mkdir to mkdirs (#3249)
Fixes: [#CLOUDSTACK-10406](https://issues.apache.org/jira/browse/CLOUDSTACK-10406)
2019-04-05 11:14:04 +02:00
skattoju3 beb0422dd5 fix incorrect iscsi path stat for managed storage (#3181)
## Description
The issue was that an Incorrect iscsi path was being passed for managed storage pools when collecting volume stats. Storage pools normally have a UUID based path while managed storage pools require an IQN based path
2019-03-29 10:25:01 +01:00
Rohit Yadav 4985e57f30
ui: use executable template filter for users (#3214)
Problem: When reinstalling a VM, the UI only shows featured templates and not all possible list of allowed templates.
Root Cause: The list of templates for reinstall used featured as filter in ‘listTemplates’ API which did not include all available options.
Solution: The issue is fixed by using the executable template filter of the ‘listTemplates’ API to list all
possible templates that a user is allowed to use to deploy a VM.
2019-03-29 12:13:26 +05:30
Gabriel Beims Bräscher 6c841bf69b apidocs: Make the API documentation version not *hardcoded* to v4.9.0 (#3247)
This commit makes the API documentation automatically updated to the
current project version. If one wants to set another version, it is
possible by just editing the ACS_RELEASE variable in 'build-apidoc.sh'.
2019-03-29 11:55:45 +05:30
Rohit Yadav fd541efe80 Merge remote-tracking branch 'origin/4.12' 2019-03-29 11:50:04 +05:30
Boris c2bd32fa41 Merge remote-tracking branch 'upstream/4.11' into 4.12 2019-03-27 12:04:15 +02:00
Boris Stoyanov - a.k.a Bobby c4b06ffe19
Merge pull request #3238 from shapeblue/tls-vmware-issue-fix
client: don't disable TLSv1, TLSv1.1 by default that breaks VMware env
2019-03-27 11:39:54 +02:00
Rohit Yadav 671a70a9a5
schema: add empty DB upgrade path from 4.12.0.0 to 4.13.0.0 (#3236)
This adds empty empty upgrade path from 4.12.0.0 to 4.13.0.0.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-03-27 00:00:47 +05:30
Rohit Yadav 6e51bde228 client: don't disable TLSv1, TLSv1.1 by default that breaks VMware env
This fixes the issue that TLSv1 and TLSv1.1 are still used by CloudStack
management server to communicate with VMware vCenter server. With the
current defaults, the setup/deployment on VMware fails. Users/admins
can however setup the security file according to their env needs to
disable TLSv1 and TLSv1.1 for server sockets (8250/agent service for
example).

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-03-26 18:15:20 +05:30
Rohit Yadav 5ee7a9c530 Merge remote-tracking branch 'origin/4.12' 2019-03-26 15:35:17 +05:30
Rohit Yadav 12c0d88e31 Merge remote-tracking branch 'origin/4.11' into 4.12 2019-03-26 15:34:41 +05:30
Paul Angus 8b25fdf520 tests: fix some Marvin smoketests (#2869)
Fixes intermittent failures:
- Add a pause to avoid  tests restarting before VRs recovering from HA have fully booted.
- Add some pauses to allow services to restart and hosts to reconnect before continuing tests.
- Adding a loop around this method because sometimes VRs are overloaded and just respond with exception, so it'll catch it and try 5 times with 30sec cooldown. If that fails as well it'll fail the test.
- wait until host is up using explicit check
2019-03-26 15:33:40 +05:30
Philipp Bankonier 2e2b6700a5 ui: Fix behavior of multiselect in list view (#3161)
Hide multiselectaction buttons when selection is resetted after a action is performed
Use checked prop instead of attr to show/hide triggered selection when box was clicked by a user before.

How to reproduce:

Select one entry in a multiSelect listView (e.g. instance list) and then deselect it again. Now click on the "select all" checkbox. The entries which were clicked previously are not displayed as selected.

Select at least one entry. Use the multiSelectAction buttons to trigger an action. Wait for completion. The list refreshes and no entry is selected but the multiSelectAction buttons are displayed.

Fix:

Use "checked" property instead of the attribute to select all entries. This is necessary because when you checked an entry by clicking on it the property gets set and the browser is preferably using this. So setting the attribute by clicking on the "select all" checkbox had no impact anymore on the checkBox of the previously clicked entries.

Hide the multiSelectAction buttons when the list is refreshed after an action is performed and no element is selected anymore.
2019-03-26 13:11:41 +05:30