Commit Graph

2778 Commits

Author SHA1 Message Date
Daan Hoogland 01b3e361c7 Updating pom.xml version numbers for release 4.15.0.0
Signed-off-by: Daan Hoogland <dahn@onecht.net>
2020-12-23 16:32:25 +00:00
Harikrishna b1ddd7c2e6
vmware: Fix for mapping guest OS type read from OVF to existing guest OS in C… (#4553)
* Fix for mapping guest OS type read from OVF to existing guest OS in CloudStack database  while registering VMware template

* Added unit tests to String Utils methods and updated the code

* Updated the java doc section

* Updated os description logic to keep equals ignore match with guest os display name
2020-12-23 19:37:21 +05:30
Nicolas Vazquez 4617be4583
vmware: Fix template upload from local (#4555)
Update the guest OS from the OVF file after upload is completed
This PR fixes the template upload from local on VMware

Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2020-12-23 15:13:39 +05:30
Rohit Yadav fdf9573266 Merge remote-tracking branch 'origin/4.14' 2020-12-14 12:05:52 +05:30
Alexandru Bagu fdb2ee3165
storage: Fix hypervisor type cast to string (#4516)
This PR addresses an error that appears when you try to add a new host. I don't even understand why there was a cast to String in the first place. I will assume some classes send HypervisorType and some send a string (empty or otherwise). Shouldn't this be addressed to use the same type everywhere? With this fix adding a new xenserver host works fine.

Co-authored-by: dahn <daan.hoogland@gmail.com>
2020-12-14 11:56:44 +05:30
Wei Zhou 96cfe27504
db upgrade: fix sql exception: Access denied; you need (at least one of) the SUPER privilege(s) for this operation (#4533) 2020-12-14 11:40:33 +05:30
Wei Zhou 9de679ea1c
db: Fix description of volume.stats.interval which is in milliseconds not seconds (#4526) 2020-12-10 14:32:16 +05:30
Pearl Dsilva e4a504b084
Make global setting non-dynamic (#4505)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-12-01 14:00:35 +05:30
Wei Zhou 4fb69e695c
sql: Fix Zones are returned in a random order (#3934) (#4494) 2020-11-25 15:30:43 +05:30
Spaceman1984 dfa09fc856
server: Setting snapshot removed on timeout (#4425)
* Setting snapshot state to error on timeout

* Setting removed field so snapshot record is ignored by garbage collection

* Removed explicitly setting error status, renamed method from markFailed to markRemoved

* Renamed method, moved code a few lines down

* Moved remove logic

* Removed unused service

* Moved removed logic - last time, promise
2020-11-21 02:20:16 +05:30
Rakesh 735b6de296
Cleanup download urls when SSVM destroyed (#4078)
Co-authored-by: Rakesh Venkatesh <r.venkatesh@global.leaseweb.com>
2020-11-18 14:01:31 +01:00
Spaceman1984 acee15a530
Moved dedicated hosts to the end of the resultset when selecting an e… (#4428) 2020-11-18 12:07:14 +00:00
Pearl Dsilva 1dbb76f64b
Fix: Data migration (#4475)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-11-18 09:45:53 +01:00
Daan Hoogland 79e46f50ca Merge branch '4.14' 2020-11-13 13:25:07 +00:00
Daan Hoogland db3c12d6cf Merge branch '4.13' into 4.14 2020-11-13 13:24:39 +00:00
slavkap cc8bee7223
Fix IndexOutOfBoundsException when creating basic network (#4464)
For Basic network isolation methods are not provided, and exception is
thrown when trying to encode the Vlan id. That's why we have to check
before encoding that the list with isolation methods is not empty
2020-11-13 08:07:40 +00:00
Olivier Lemasle 5f8289ffe9
Re-enable IP address usage hiding (#4327) 2020-11-07 10:42:44 +01:00
Nicolas Vazquez 63fbbe7506
[Bug fix] VMware: Fix for SSVM recreation on deployasis systemVM templates (#4437) 2020-11-07 09:46:54 +01:00
Gabriel Beims Bräscher b3a1cb41c8
Allow to configure root disk size via Service Offering (diskoffering of type Service). (#4341) 2020-10-30 15:56:11 +00:00
Wei Zhou e447764d66
DB: fix wrong category id of guest os 'Other PV Virtio-SCSI (64-bit)' (#4426) 2020-10-29 09:08:02 +01:00
Pearl Dsilva 25dd7aad51
Modify alter view to drop/create view (#4417)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-10-28 09:53:56 +01:00
nvazquez d119a5d19f Merge branch 'master' into ovfprops-and-vsphere-adv-together 2020-10-24 13:02:31 -03:00
Rakesh 71c5dbcf49
server: Update use_bytes of storage pools (#4360)
Update the used_bytes for all default primary storage pools
Also get used_bytes of storage pool from database instead of
memory
2020-10-21 19:18:03 +02:00
Andrija Panic 5797cf5e17
guest-os-GC (#67) 2020-10-20 10:40:03 -03:00
nvazquez 50ab1b2fbe Add missing guest OS entries for VMware 2020-10-19 15:05:58 +05:30
nvazquez ee5b8763a6 Fix remove VM and its volumes for deploy-as-is if have previously failed - restore cpu flags in nested virtualization test 2020-10-19 15:05:58 +05:30
nvazquez 3600b3c7bc Reconcile root disk controller fix 2020-10-19 15:05:58 +05:30
Harikrishna Patnala 5fdabc1cb0 Added storage policy details to disk while creating disk and restricted migration of volumes to storage pools which are not storage policy compliance 2020-10-19 15:05:58 +05:30
Harikrishna Patnala 46b5322d9b Adding vSphere storage policy to disk on start command and attach volume command 2020-10-19 15:05:58 +05:30
Harikrishna Patnala 1e4e5cf4c0 Disk offering response: added vsphere storage policy to response if exists 2020-10-19 15:05:58 +05:30
Harikrishna Patnala a54436b790 service offering response: added vsphere storage policy to response if exists 2020-10-19 15:05:58 +05:30
Harikrishna Patnala 07abcf5705 During migrate volume command, when operation timed out exception or any exception is occured it is not handled properly to clean the volume_store_ref entry.
Fixed it to clean the volume_store_ref entry upon on any exception
2020-10-19 15:05:57 +05:30
Harikrishna Patnala 9b923ba9dc Handle detach volume of datastore cluster if the volume name has changed at vCenter level and reconsile the chaininfo 2020-10-19 15:05:57 +05:30
nvazquez 94bebe8792 Revert back deploy as is column on templates but keep it as default for new templates 2020-10-19 15:05:57 +05:30
nvazquez 46d412d998 Fix for system VMs 2020-10-19 15:05:57 +05:30
nvazquez 9b51a706db Set deploy-as-is to default on VMware 2020-10-19 15:05:57 +05:30
Harikrishna Patnala 9543fd6e6a Fix startcommand on Datastore cluster when the volume datastore in CloudStack mismatches with vCenter datastore. Volume could have migrated with in datastore cluster which caused the mismatch
Fix dettach volume when volume is not on CloudStack intended datastore
2020-10-19 15:05:57 +05:30
nvazquez b0d3168e0b Fail template registration when guest OS not found 2020-10-19 15:05:57 +05:30
nvazquez 32d85b0fa2 Display storage on logging when not deploy-as-is and guest OS small refactor 2020-10-19 15:05:57 +05:30
nvazquez 41354227e2 Handle guest OS read from deploy-as-is OVF descriptor 2020-10-19 15:05:57 +05:30
nvazquez edfbed34ad Use network adapter from OVF on deploy-as-is 2020-10-19 15:05:57 +05:30
nvazquez 0f5a6ee589 Remove deploy as-is details when removing templates or vms 2020-10-19 15:05:57 +05:30
nvazquez a43d0a6085 Fix VM power sync NPE on system VMs 2020-10-19 15:05:57 +05:30
Harikrishna Patnala 745dd00c9a Remove duplicate schema changes 2020-10-19 15:05:57 +05:30
Harikrishna Patnala 33ae2afc89 Removed few duplicate imports during rebase with master 2020-10-19 15:05:57 +05:30
Harikrishna Patnala 44dc0c6072 Fixed rat failure on new class DeployAsIsHelper.java
Also removed some unused imports during rebase
2020-10-19 15:05:57 +05:30
nvazquez 818981d8a7 Fix typo 2020-10-19 15:05:57 +05:30
nvazquez 44bae98664 Fix daos issue 2020-10-19 15:05:57 +05:30
nvazquez f73830acbb Refactor deploy as is constants 2020-10-19 15:05:57 +05:30
nvazquez bb4ce2118d Add new template and vm deploy as is details table and refactor 2020-10-19 15:05:57 +05:30
nvazquez d864e9dc39 [VMware] Full OVF properties support 2020-10-19 15:05:56 +05:30
Harikrishna Patnala 19745ea049 Fix enable primary datastore maintenance command seriliaztion on it 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 38b9618e4b Fix destination hostname for stopped VMs in case of reset ssh keys 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 201ebe8868 Simulator failures fixing 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 295ea50282 Fixed unexpected token Exception 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 48dd0dae07 Adding Storagepolicies to the service offering for root disks 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 61dd85876b Fix migrate vm and volume APIs in case if datastore cluster 2020-10-19 14:57:16 +05:30
Pearl Dsilva 4178a7cd90 inter-cluster migration support 2020-10-19 14:57:16 +05:30
Harikrishna Patnala fef3926469 Datastore cluster: update tag to the child storagepools also 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 873f9dd9ac Datastore Clusters operations on putting into maintenance mode, update storage pool with tags, cancelling mantenance mode and deleting storage pool 2020-10-19 14:57:16 +05:30
Harikrishna Patnala 75fb1d91ee Fix adding Datastore clusters and listing 2020-10-19 14:57:15 +05:30
Harikrishna Patnala c9c38c0f37 Handle datastore clusters in list storage pools to list only parent not the children in the Datastore cluster 2020-10-19 14:57:15 +05:30
Harikrishna Patnala b4a23ea5f6 Allocation logic to skip datastore cluster and consider only storagepools inside the datastore cluster 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 41b3fc19d6 Add Datastore cluster and the child entities which are datastores in the cluster into CloudStack
Setting scope is still pending.
2020-10-19 14:57:15 +05:30
Harikrishna Patnala 48786b2d31 DataStore Clusters addition as a storage pool 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 586fa7eeaf Added listVsphereStoragePoliciesCmd 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 6df819028e UI changes and accept any type of datastore as presetup in vmware 2020-10-19 14:57:15 +05:30
Harikrishna Patnala fb0a96e7fb Check if datastore is complaince with the storagepolicy provided in the disk offering.
Added corresponding manager objects from PBM sdk to do the job.
Made dao layer changes to read the storage policy in diskoffering
2020-10-19 14:57:15 +05:30
Harikrishna Patnala 27e5dfa633 Updated code to handle sync of storage policies when importVsphereStoragePolicies API is called multiple times 2020-10-19 14:57:15 +05:30
Harikrishna Patnala 8b23ea90b8 Moved Dao and VO files to different packages 2020-10-19 14:57:15 +05:30
Harikrishna Patnala b8b4f0f9ad Added importVsphereStoragePolicies API and scheme changes 2020-10-19 14:57:15 +05:30
Pearl Dsilva 0d487fc8c9
support for data migration of incremental snaps on xen (#4395)
* support for handling incremental snaps (on DB entries) on xen

* Addressed comments

* Update NfsSecondaryStorageResource.java

adjusted space in comment/ log

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-10-18 02:15:10 +05:30
Rodrigo D. Lopez c222d0bf60
enable update tags on disk offerings (#4194) 2020-10-16 09:22:42 +02:00
Wei Zhou 2acd87c41e
server: Add global configuration vm.serviceoffering.cpu.cores.max and vm.serviceoffering.ram.size.max (#4379)
vm.serviceoffering.cpu.cores.max and vm.serviceoffering.ram.size.max
2020-10-14 15:48:35 +05:30
davidjumani aab8df09aa
Fixing searchAndCount searchAndDistinctCount when sc is null (#4374) 2020-10-13 11:30:04 +02:00
Rohit Yadav b3bafffff3 Merge remote-tracking branch 'origin/4.14' 2020-09-29 14:33:58 +05:30
Abhishek Kumar 9391fa9b6b
cks: assorted fixes, test refactoring (#4269)
Fixes #4265
Fixes for:

template selection
network cleanup
capacity check
specific patch version during deployment

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-09-29 14:31:41 +05:30
Rohit Yadav 86fcb14238 Merge remote-tracking branch 'origin/4.14'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-09-24 12:20:29 +05:30
Rohit Yadav 1efe6e2df0 Merge remote-tracking branch 'origin/4.13' into 4.14
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-09-24 12:13:23 +05:30
Gabriel Beims Bräscher d6152b37ad
server: Broadcast URI not set to vxlan, but vlan (Fix #3040) (#4190)
This PR sets properly Broadcast URI to vxlan://vxlan_id when the physical network is of VXLAN.

Fixes: #3040
2020-09-24 11:09:50 +05:30
Wei Zhou 534dd475d8
agent: Compare indirect agent lb algorithm when cloudstack agent connects (#4335)
Compare not only the list of management servers but also the lb algorithm when agent connects.

Fixes: #3895
2020-09-23 12:48:16 +05:30
Pearl Dsilva cfbb4ff3dd
schema: change upgrade path to 4.14 (from 4.13) and intensify check (#4331)
* change upgrade path to 4.14 (from 4.13) and intensify check

* extracted check

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-09-22 09:40:51 +05:30
Pearl Dsilva b464fe41c6
server: Secondary Storage Usage Improvements (#4053)
This feature enables the following:
Balanced migration of data objects from source Image store to destination Image store(s)
Complete migration of data
setting an image store to read-only
viewing download progress of templates across all data stores
Related Primate PR: apache/cloudstack-primate#326
2020-09-17 10:12:10 +05:30
Pearl Dsilva 37c7a2b851
Incorrect md5sums for systemVM templates results in failure to download templates to other image stores (#4297)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-09-09 15:57:49 +05:30
Spaceman1984 cb717741fc
server: Fixed delayed power state update after vm shutdown (#4284)
After a vm is shutdown, the power state isn't updated immediately. This prevents changing the service offering.
This PR updates the power state immediately after the vm is confirmed to be shutdown.

Fixes: #3159
2020-09-01 15:53:52 +05:30
Wei Zhou 4746c8c726
server: move UpdateDefaultNic to vm work job queue (#4020)
While remove secondary nic from a Running vm, if update the default nic to the secondary nic before the nic is removed, the vm will not have default nic (and cannot be started) when both operations are completed.

It is because UpdateDefaultNic api is not handled as a vm work job (AddNicToVMCmd and RemoveNicFromVMCmd are), it is processed before nic is removed. The result is that secondary nic becomes default nic and got removed.
2020-09-01 13:54:48 +05:30
Rohit Yadav 6d6e4025f5 Merge remote-tracking branch 'origin/4.14' 2020-08-25 17:02:19 +05:30
Rohit Yadav 9c7c22eb84
engine: honour bypass VLAN id/range for L2 networks (#4274)
* engine: honour bypass VLAN id/range for L2 networks

Commit e894238d904a9c49c1140371f612a51d251efc1 (#3899) allowed private
gateways to bypass vlan check while refactoring it did not cover the
case for L2 but only shared network. This fix will re-enable honouring
the bypass vlan check option for L2 guest network (in addition to the
Shared networks).

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* Update NetworkOrchestrator.java
2020-08-25 15:19:02 +05:30
davidjumani 3872bf1ff9
kvm: Enable PVLAN support on L2 networks (#4040)
This is an extention of #3732 for kvm.
This is restricted to ovs > 2.9.2
Since Xen uses ovs 2.6, pvlan is unsupported.
This also fixes issues of vms on the same pvlan unable to communicate if they're on the same host
2020-08-20 15:46:34 +05:30
Paul Angus 400641b1cf Revert "Revert "move power sync messages to debug""
This reverts commit e06cf1a642.
2020-08-20 10:51:05 +01:00
Paul Angus e06cf1a642 Revert "move power sync messages to debug"
This reverts commit 1954ff32b7.
2020-08-20 10:50:58 +01:00
Paul Angus 1954ff32b7 move power sync messages to debug 2020-08-20 10:50:35 +01:00
Spaceman1984 d57aa83517
server: Added nfs minor version support (#4180)
This PR adds minor version support when mounting nfs on the SSVM as requested in #2861

The global setting "secstorage.nfs.version" has been changed to use the String data type which allows any minor version to be specified.
2020-08-19 14:53:38 +05:30
davidjumani d949302d0f
packaging: Adding Centos8, Ubuntu 20.04, XCPNG8.1 Support (#4068)
* DB : Add support for MySQL 8

- Splits commands to create user and grant access on database, the old
statement is no longer supported by MySQL 8.x
- `NO_AUTO_CREATE_USER` is no longer supported by MySQL 8.x so remove
that from db.properties conn parameters

For mysql-server 8.x setup the following changes were added/tested to
make it work with CloudStack in /etc/mysql/mysql.conf.d/mysqld.cnf and
then restart the mysql-server process:

    server_id = 1
    sql-mode="STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION,ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE,NO_ENGINE_SUBSTITUTION"
    innodb_rollback_on_timeout=1
    innodb_lock_wait_timeout=600
    max_connections=1000
    log-bin=mysql-bin
    binlog-format = 'ROW'

    default-authentication-plugin=mysql_native_password

Notice the last line above, this is to reset the old password based
authentication used by MySQL 5.x.

Developers can set empty password as follows:

    > sudo mysql -u root
    ALTER USER 'root'@'localhost' IDENTIFIED BY '';

In libvirt repository, there are two related commits

2019-08-23 13:13 Daniel P. Berrangé            ● rpm: don't enable socket activation in upgrade if --listen present
2019-08-22 14:52 Daniel P. Berrangé            ● remote: forbid the --listen arg when systemd socket activation

In libvirt.spec.in

        /bin/systemctl mask libvirtd.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-ro.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-admin.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-tls.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-tcp.socket >/dev/null 2>&1 || :

Co-authored-by: Wei Zhou <w.zhou@global.leaseweb.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-17 16:28:30 +05:30
Spaceman1984 b586eb22f1
Human readable sizes in logs (#4207)
This PR adds outputting human readable byte sizes in the management server logs, agent logs, and usage records. A non-dynamic global variable is added (display.human.readable.sizes) to control switching this feature on and off. This setting is sent to the agent on connection and is only read from the database when the management server is started up. The setting is kept in memory by the use of a static field on the NumbersUtil class and is available throughout the codebase.

Instead of seeing things like:
2020-07-23 15:31:58,593 DEBUG [c.c.a.t.Request] (AgentManager-Handler-12:null) (logid:) Seq 8-1863645820801253428: Processing: { Ans: , MgmtId: 52238089807, via: 8, Ver: v1, Flags: 10, [{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-224-VM","bytesSent":"106496","bytesReceived":"0","result":"true","details":"","wait":"0",}}] }

The KB MB and GB values will be printed out:

2020-07-23 15:31:58,593 DEBUG [c.c.a.t.Request] (AgentManager-Handler-12:null) (logid:) Seq 8-1863645820801253428: Processing: { Ans: , MgmtId: 52238089807, via: 8, Ver: v1, Flags: 10, [{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-224-VM","bytesSent":"(104.00 KB) 106496","bytesReceived":"(0 bytes) 0","result":"true","details":"","wait":"0",}}] }

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Human+Readable+Byte+sizes
2020-08-13 15:55:16 +05:30
Pearl Dsilva c578004fe5
projects: Role based users in Projects (#4128)
Enabling Role Based users in projects
Primate PR related to the FR: apache/cloudstack-primate#382
Doc PR: https://github.com/apache/cloudstack-documentation/pull/145

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2020-08-13 15:45:39 +05:30
Rohit Yadav 7bbb4d4bef Merge remote-tracking branch 'origin/4.14' 2020-08-12 13:46:13 +05:30
Rohit Yadav 6b6cc71be9 Merge remote-tracking branch 'origin/4.13' into 4.14 2020-08-12 13:44:41 +05:30
Spaceman1984 86939e7f9d
server: Fixed private gateway can't be deleted (#4016)
When the static route service is not available on the VPC and a static route is created, the static route is created in a revoked state.

Currently, the UI doesn't distinguish between active or revoked static routes.

This PR adds the missing state filter to the list routes command and only lists active routes in the UI.
It also ignores revoked routes when the private gateway is being removed but clears out the inactive routes before the gateway is removed.

Fixes #2908
2020-08-12 13:29:12 +05:30
Rohit Yadav 7707d3fd92 engine: fix NPE from fwd-merge 562a7db8df
This fixes NPE caused due to merge conflict fix from the forward merge
commit 562a7db8df and fixes travis test
regression:

    === TestName: test_01_reset_vm_on_reboot | Status : SUCCESS ===

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-07 15:31:43 +05:30