Commit Graph

32712 Commits

Author SHA1 Message Date
Abhishek Kumar c671e07c18 server: update network offering for specified domain(s) & zone(s)
Added checkAccess while creating network from offering.
Response fixes for networkoffering APIs.

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-24 10:11:00 +05:30
Abhishek Kumar eaa759209a test fix and temporary skip for search test
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-24 10:11:00 +05:30
Abhishek Kumar dc589a442d server: create network offering for specified domain(s) and zone(s)
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-24 10:11:00 +05:30
Abhishek Kumar e2bb43a392 refactoring
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-24 10:10:59 +05:30
Abhishek Kumar 977c5b7614 schema: domain and zone IDs in offering details hidden for user in upgrade
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-24 10:10:59 +05:30
Abhishek Kumar 98df2c2197 server: zone, domain IDs in service, disk offering details hidden for user
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-24 10:10:59 +05:30
Abhishek Kumar 9f46efef02 server update service, disk offerings for domain(s) and zone(s)
updateServiceOffering and updateDiskOffering API has been modified to allow updating domain(s) and zone(s) for the offering.

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-24 10:10:59 +05:30
Abhishek Kumar 2102888116 server: changes for domain, zone specified service offerings
Added changes for creating service offerings for specified domain(s) and zone(s).
Fixed checkAccess for disk offerings.
Fixed list APIs for disk and service offerings.
UI changes for creating disk, service offerings for specified domain(s) and zone(s).

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-24 10:09:52 +05:30
Abhishek Kumar b8f1cf2648 schema: update service_offering details constraint
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-24 10:04:10 +05:30
Rohit Yadav dd0294e163 add helper method for findings offerings by zone id
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 10:04:10 +05:30
Rohit Yadav 94f425f6b1 refactored changes
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 10:04:10 +05:30
Rohit Yadav f431711922 sql changes for service offering
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 10:04:10 +05:30
Rohit Yadav d04c6ea276 remove getDomainId and write the workaround code
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 10:04:10 +05:30
Rohit Yadav e3da9ec645 fix db issues
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Rohit Yadav 6a902460af find in set implementation
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Rohit Yadav 02913e9ea1 server: fix failing unit test issue
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Rohit Yadav 8cabbcea92 wip domainId removal stuff
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Rohit Yadav 5988884bd1 wip changes
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Rohit Yadav 7721126076 remove domain id usage
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Rohit Yadav 9c117f2781 DB upgrade path and UI fixes
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Rohit Yadav 728afba5d4 refactorings
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-24 09:56:23 +05:30
Abhishek Kumar c85b3e597a server: ability to create disk offerings for domain(s) and zone(s)
Allows creating storage offerings associated with particular domain(s) and zone(s). In create disk/storage offfering form UI, a mult-select control has been addded to select desired zone(s) and domain select element has been made multi-select.
createDiskOffering API has been modified to allow passing list of domain and zone IDs with keys domainids and zoneids respectively. These lists are stored in DB in cloud.disk_offering_details table with 'domainids' and 'zoneids' key as string of comma separated list of IDs. Response for create, update and list disk offering APIs will return domainids, domainnames, zoneids and zonenames in details object of offering.
listDiskOfferings API has been modified to allow passing zoneid to return only offerings which are associated with the zone.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-05-24 09:56:23 +05:30
Rohit Yadav 2aa4842a94 Merge remote-tracking branch 'origin/4.12' 2019-05-23 18:17:42 +05:30
Rohit Yadav d77e69a2f2 Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-23 18:16:45 +05:30
Rohit Yadav 0929866956
server: ssh-keygen in PEM format and reduce main systemvm patching script (#3333)
On first startup, the management server creates and saves a random
ssh keypair using ssh-keygen in the database. The command does
not specify keys in PEM format which is not the default as generated
by latest ssh-keygen tool.

The systemvmtemplate always needs re-building whenever there is a change
in the cloud-early-config file. This also tries to fix that by introducing a
stage 2 bootstrap.sh where the changes specific to hypervisor detection
etc are refactored/moved. The initial cloud-early-config only patches
before the other scripts are called.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-23 18:08:00 +05:30
Daan Hoogland 8b75d61ffc Merge release branch 4.12 to master
* 4.12:
  KVM: Fix agents dont reconnect post maintenance (#3239)
2019-05-23 14:31:21 +02:00
Daan Hoogland 29918e25e3 Merge release branch 4.11 to 4.12
* 4.11:
  KVM: Fix agents dont reconnect post maintenance (#3239)
2019-05-23 14:29:41 +02:00
Nicolas Vazquez e86f671c8e KVM: Fix agents dont reconnect post maintenance (#3239)
* Keep connection alive when on maintenance

* Refactor cancel maintenance and unit tests

* Add marvin tests

* Refactor

* Changing the way we get ssh credentials

* Add check on SSH restart and improve marvin tests
2019-05-23 14:13:17 +02:00
Philipp Bankonier 62c8a1110c Fix iops values when creating a compute offering (#3345)
@mike-tutkowski @syed If there's something more that needs to be added/changed, we'll just open another PR for this. 

For now this seems to be a very straightforward fix for the UI problem with managed storage.

* Fix iops values when creating a compute offering
* Fix iops values when creating a disk offering
2019-05-23 13:25:48 +02:00
Rohit Yadav 4f356392ab vmware: don't use redundant worker VM to extract volume (#3218)
This fixes the issue that VM with VMsnapshots fails to start after
extract volume is done on a stopped VM, on VMware.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-23 13:18:29 +02:00
M. Weber a5da389b15 Enhancement scss refactoring (#3328)
* Introduce jsbeautifyrc and sass linter for automated unified scss code syntax

* Introduce new z-index scss file to manage different z-index correctly

* Use beautify tool for scss files and sass lint fix all scss files

* Add vscode folder to gitignore

* Add more new files to global gitignore

* Refactor scss files and rework custom styles implementation strategy

* Remove outdated ie7 styles

* Fix typo of facebook input token include

* Fix apache licences for new lint files

* Splitt massive cloudstack.scss into many modular and smaller files

* Refactor scss language files

* Change and move apache licence css file name
2019-05-23 11:09:31 +02:00
Abhishek Kumar 2020bfb6a3 server: allows compute offering with or without constraints (#3245)
Problem: Custom compute offering does not allow setting min and max values for CPU and VRAM for custom VMs.

Root Cause: Custom compute offerings cannot be created with a given range of CPU number and memory instead it allows only fixed values.

Solution: createServiceOffering API has been modified to allow setting a defined range for CPU number and memory. Also, UI form for compute offering creation is provided with a new field named 'compute offering type’ with values - Fixed, Custom Constrained, Custom Constrained. It will allow the creation of compute offerings either with a fixed CPU speed and memory for fixed compute offering, or with a range of CPU number and memory for custom constrained compute offering or without predefined CPU number, CPU speed and memory for custom unconstrained compute offering.

To allow the user to set CPU number, CPU speed and memory during VM deployment, UI form for VM deployment has been modified to provide controls to change these values. These controls are depicted in screenshots below for custom constrained and custom unconstrained compute offering types.

Sample API calls using cmk to create a constrained service offering and deploying a VM using it,

create serviceoffering name=Constrained displaytext=Constrained customized=true mincpunumber=2 maxcpunumber=4 cpuspeed=400 minmemory=256 maxmemory=1024

deploy virtualmachine displayname=ConstrainedVM serviceofferingid=60f3e500-6559-40b2-9a61-2192891c2bd6 templateid=8e0f4a3e-601b-11e9-9df4-a0afbd4a2d60 zoneid=9612a0c6-ed28-4fae-9a48-6eb207af29e3 details[0].cpuNumber=3 details[0].memory=800

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-23 11:47:53 +05:30
dahn 2ead7359d1 maven: refactor usage of slf4j version (#3325) 2019-05-23 11:23:18 +05:30
dahn f6ecac42ac ui: reverse character replacements (#3260)
In the cloudstack UI fields get '<', '>' and '&' replaced by xml-entities
these are generic for all fields and hurt us in the case of userdata
this fix calls the existing method to reverse character replacements.
it also removes the ccs class that pretends to prevent special chars

Fixes #3202
2019-05-23 11:20:42 +05:30
Anurag Awasthi 3e642852a7 ui: Bug fix for zone names not appearing in dashboard (#3326)
3 years ago when we moved from JSP this translate was not wrapped
in label tags and consequently the following span wasn't captured
when dashboard.js mapped the data.

Fixes: #3314
2019-05-15 09:52:38 +05:30
Rohit Yadav 0700d91a68 Merge branch '4.12'
- Fixes PR #3146 db cleanup to the correct 4.12->4.13 upgrade path
- Fixes failing unit test due to jdk specific changes after forward
  merging

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-14 15:15:17 +05:30
Frank Maximus e11f7ee1ba RIP Nuage Cloudstack Plugin (#3146)
may it rest in peaces
2019-05-14 10:58:24 +02:00
Rohit Yadav 00ff536f81 Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-14 14:26:11 +05:30
skattoju4 4c60a5b1ff Fix slow vm creation when large sf snapshot count (#3282)
* skip geting used bytes for volumes that are not in Ready state
* updated log message
* filter snapshots by state backedup
* removed * import
* filter templates by state 'DOWNLOADED'
* refactored getUsedBytes to use O(1) queries
* querying for ready volumes instead filtering in memory
* make listByStoreIdInReadyState more generic ex listByStoreIdAndState
* updated snapshot search criteria for listByStoreIdAndState
* updated template search criteria for listByPoolIdAndState
* fixed typo in search criteria for listByTemplateAndState
* fixed typo in search criteria for templates in listByPoolIdAndState
2019-05-11 16:02:52 +02:00
Rohit Yadav 9ff819da2c
systemvm: new qemu-guest-agent based patching for KVM (#3278)
This introduces a new patching script for patching systemvms on KVM
using qemu-guest-agent that runs inside the systemvm on startup. This
also removes the vport device which was previously used by the legacy
patching script and instead uses the modern and new uniform guest
agent vport for host-guest communication.

Also updates the sytemvmtemplate build config to use the latest Debian
9.9.0 iso.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-10 23:42:19 +05:30
Anurag Awasthi f9b61bc737 orchestration: Allow VM that has never started to have volumes attached (#3276)
With this patch b766bf7
we started tracking disks in attaching state so that other attach request can fail gracefully. However this missed the case where disks were in allocated state but attach was requested.

For the use case where users want to attach disk in allocated state but not ready, we need to have allocated-attaching transition as well. We must take care of returning to the original state - allocated or ready - when attach request has completed.

For the use case of unstarted vm's the disk must proceed as follows - "Allocated" -> Attaching -> Allocated. When VM is started, the disk is "created" and pool is assigned. For the use case of started VMs it's more trivial and disk proceeds as follows - Ready -> Attaching -> Ready.

Test this by creating a VM with "startvm=false", create a disk and try attaching it in allocated state. It would give an exception on latest 4.11 but will be fixed on this patch.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-10 23:40:38 +05:30
Rohit Yadav 67160478a6
server: allow admins to blacklist vm details that users should not see (#3213)
This introduces a new global setting `user.vm.blacklisted.details` that
allows admins to blacklist VM details that non-admin users should not
see via the VM's settings tab.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-10 23:39:13 +05:30
Rohit Yadav bfad334117
api: include tags in listvmsnapshots response (#3216)
This ensures that tags of a VM snapshot are listed in the UI, available
in the list vmsnapshots API response.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-10 23:37:18 +05:30
Dennis Konrad a75292e701
Merge pull request #3307 from webermaximilian92/feature_add_scss_to_css_compiler
All change requests were taken care of.
Multiple approvement are there
Merging this onto master
2019-05-10 16:41:06 +02:00
ustcweizhou 3729511c37 kvm: Fix three issues with Ubuntu 16.04 hosts (#3227)
* ubuntu16: fix unable to add host if cloudbrX is not configured

while add a ubuntu16.04 host with native eth0 (cloudbrX is not configured),
the operation failed and I got the following error in /var/log/cloudstack/agent/setup.log

```
DEBUG:root:execute:ifconfig eth0
DEBUG:root:[Errno 2] No such file or directory
  File "/usr/lib/python2.7/dist-packages/cloudutils/serviceConfig.py", line 38, in configration
    result = self.config()
  File "/usr/lib/python2.7/dist-packages/cloudutils/serviceConfig.py", line 211, in config
    super(networkConfigUbuntu, self).cfgNetwork()
  File "/usr/lib/python2.7/dist-packages/cloudutils/serviceConfig.py", line 108, in cfgNetwork
    device = self.netcfg.getDefaultNetwork()
  File "/usr/lib/python2.7/dist-packages/cloudutils/networkConfig.py", line 53, in getDefaultNetwork
    pdi = networkConfig.getDevInfo(dev)
  File "/usr/lib/python2.7/dist-packages/cloudutils/networkConfig.py", line 157, in getDevInfo
    elif networkConfig.isBridge(dev) or networkConfig.isOvsBridge(dev):
```

The issue is caused by commit 9c7cd8c248
2017-09-19 16:45 Sigert Goeminne ● CLOUDSTACK-10081: CloudUtils getDevInfo function will now return "bridge" instead o

* ubuntu16: Stop service libvirt-bin.socket while add a host

service libvirt-bin.socket will be started when add a ubuntu 16.04 host
DEBUG:root:execute:sudo /usr/sbin/service libvirt-bin start

However, libvirt-bin service will be broken by it after restarting
Stopping service libvirt-bin.socket will fix the issue.

An example is given as below.

```
root@node32:~# /etc/init.d/libvirt-bin restart
[ ok ] Restarting libvirt-bin (via systemctl): libvirt-bin.service.
root@node32:~# virsh list
error: failed to connect to the hypervisor
error: no valid connection
error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory

root@node32:~# systemctl stop libvirt-bin.socket

root@node32:~# /etc/init.d/libvirt-bin restart
[ ok ] Restarting libvirt-bin (via systemctl): libvirt-bin.service.
root@node32:~# virsh list
 Id    Name                           State
----------------------------------------------------

```

* ubuntu16: Diable libvirt default network

By default, libvirt will create default network virbr0 on kvm hypervisors.
If vm uses the same ip range 192.168.122.0/24, there will be some issues.

In some cases, if we run tcpdump inside vm, we will see the ip of kvm hypervisor as source ip.
2019-05-05 13:57:58 +05:30
Maximilian Weber 5f5ff7ed0b Add facebook scss file to apache rat plugin list 2019-05-02 11:47:33 +02:00
Gerd Müller de545229f9 Include 'removed' async jobs to check recurring snapshots (#3190) 2019-05-01 16:49:09 +02:00
Maximilian Weber 5343a190a0 Compile new scss files to css 2019-04-30 16:16:37 +02:00
Maximilian Weber f621b1a3a8 Copy css files to scss, lint auto fix them 2019-04-30 16:16:20 +02:00
Maximilian Weber 134408c18e Add new scss to css compiler with npm and gulp 2019-04-30 16:14:29 +02:00