Commit Graph

32712 Commits

Author SHA1 Message Date
Rohit Yadav 2088f0ad73 Merge pull request #1783 from jayapalu/CLOUDSTACK-9615
CLOUDSTACK-9615: Fixd applying ingress rules without portsWhen ingress rule is applied without ports (port start and port end params are not passed) then API/UI is showing rule got applied but in the VR, iptables rule not got applied.

Fixed this issue in the VR script.

* pr/1783:
  CLOUDSTACK-9615: Fixed applying ingress rules without ports

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-22 12:37:13 +05:30
Rohit Yadav 342162bad7 Merge branch '4.9'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-22 11:59:02 +05:30
Rohit Yadav a0e36b73ae Merge pull request #1711 from syed/xenserver7
XenServer 7 SupportThis PR adds support for XenServer 7. I have manually done the following tests
- Create a new cluster with XenServer7
- Add Primary storage: Should create an SR on XS7
- Add another XS7 host to the Pool
- Add host2 to Cloudstack
- Create VM1 from template
- Create VM2 from template
- Ping/SSH VM1 to VM2 and vice-versa
- Stop/Delete/Expunge VM2
- Create Data disk
- Attach it to VM1
- Create VM snaphsot of VM1
- Restore VM snapshot of VM1
- Delete VM snapshot of VM1
- Create Volume snapshot of Datadisk
- Create volume snapshot of Root disk
- Create new template from snapshot of root disk
- Create volume from snapshot of datadisk
- Detach datadisk volume
- Delete datadisk volume
- Aquire a public IP
- Create a static nat to VM1
- Live migrate VM1 while traffic on VM
- Delete VM1

* pr/1711:
  [CLOUDSTACK-9662] Add support for XenServer 7

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-22 11:18:57 +05:30
Rohit Yadav 38a955e8ec Merge pull request #1851 from shapeblue/4910to4920upgradepath
schema: Upgrade path from 4.9.1.0 to 4.9.2.0Upgrade paths added so PRs such as #1711 can use it.

* pr/1851:
  schema: Upgrade path from 4.9.1.0 to 4.9.2.0

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-22 11:13:47 +05:30
Rohit Yadav f1cc4d1f5d schema: Upgrade path from 4.9.1.0 to 4.9.2.0
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-22 11:12:45 +05:30
Rohit Yadav 5b7f35c39f
Merge branch '4.9' 2016-12-22 10:02:24 +05:30
Rohit Yadav 743dd4249c Merge pull request #1839 from shapeblue/CLOUDSTACK-9683
CLOUDSTACK-9683: system.vm.default.hypervisor will pin the hypervisor for VR too with this fix

* pr/1839:
  CLOUDSTACK-9683: system.vm.default.hypervisor will pin the hypervisor for VR too with this fix

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-22 10:01:52 +05:30
Syed eabf862ba9 [CLOUDSTACK-9662] Add support for XenServer 7 2016-12-21 16:58:10 -05:00
Sateesh Chodapuneedi 674bb064a3 CLOUDSTACK-9676 Start instance fails after reverting to a VM snapshot, when there are child VM snapshots
Issue
====
Start instance fails after reverting to a VM snapshot, when there is 1 or more child VM snapshots in the snapshot tree of the VM.
Per the code that detects the presence of a snapshot, we are checking for only current snapshot instead of checking presence of any snapshot in the snapshot tree.
The failure to detect all snapshots means ACP reconfigures the VM in wrong way assuming there are no snapshots for the VM.
This results in start failure.

Fix
===
Ensure correct detection of VM snapshots in the VM snapshot tree

Signed-off-by: Sateesh Chodapuneedi <sateesh.chodapuneedi@accelerite.com>
2016-12-22 01:41:33 +05:30
Sateesh Chodapuneedi 32e9e29a17 CLOUDSTACK-9673 Exception occured while creating the CPVM in the VmWare Setup over standard vSwitches
Issue
====
Exception occured while creating the CPVM in the VmWare Setup using standard vswitches.
StartCommand failed due to Exception: com.vmware.vim25.AlreadyExists
message: [] com.vmware.vim25.AlreadyExistsFaultMsg: The specified key, name, or identifier already exists

Fix
===
Ensure synchronization while attempting to create port group such that simultaneous attempts are not made with same port group name on same ESXi host.

Signed-off-by: Sateesh Chodapuneedi <sateesh.chodapuneedi@accelerite.com>
2016-12-22 01:35:10 +05:30
Rohit Yadav d63c74a7d9
Merge branch '4.9' 2016-12-22 00:16:41 +05:30
Rohit Yadav f68b6e8312 Merge pull request #1846 from shapeblue/49smoketest-fixes
CLOUDSTACK-9688: Fix failing smoke testsFixes failing smoke tests due to enviroment issues or corner cases:
- Fixes NPE in Template Manager

@blueorangutan package

* pr/1846:
  CLOUDSTACK-9688: Fix failing smoke tests

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-22 00:16:21 +05:30
Rohit Yadav 6f98fcd3a5 CLOUDSTACK-9688: Fix failing smoke tests
Fixes failing smoke tests due to enviroment issues or corner cases:
- Fixes NPE in Template Manager

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-21 11:45:20 +05:30
Anshul Gangwar 929595c114 CLOUDSTACK-9686: Fixed multiple entires for builtin template in template
store ref table so builtin template is never downloaded completely
 In handleSysTemplateDownload method creating template only if there exists no entry
handleTemplateSync will take care of other scenario
2016-12-21 10:08:33 +05:30
Anshul Gangwar 336df84f17 CLOUDSTACK-9685: delete snapshot on primary associated with a volume when that volume is deleted
as that snapshot will never be going to use again and also it will fill up primary storage
2016-12-20 15:21:04 +05:30
Abhinandan Prateek ca8a00442c CLOUDSTACK-9683: system.vm.default.hypervisor will pin the hypervisor for VR too with this fix 2016-12-19 13:06:02 +05:30
Rohit Yadav 0dce1c50c1 CLOUDSTACK-9456: Update Spring version in maven poms
- Bump spring-framework version to 4.x and Jetty to version that runs with JDK8
- Bump servet dependency version
- Migrate spring xmls to version 4, fixes schema locations that are 3.0
  dependent in various xmls.
- Fix failing tests due to spring upgrade
  (Thanks @marcaurele Marc-Aurèle Brothier for fixing them)
    * Fix test DeploymentPlanningManagerImplTest
    * Fix GloboDNS test

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-16 21:21:20 +05:30
Rohit Yadav 61073ea511
Merge branch '4.9' 2016-12-16 21:00:23 +05:30
Rohit Yadav 5e19e64f2f Updating pom.xml version numbers for release 4.9.2.0-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-16 20:48:16 +05:30
pdumbre 87849c3964 CLOUDSTACK-9679:Allow master user to manage subordinate user uploaded template 2016-12-16 19:00:26 +05:30
Syed 966814b07d Fix HVM VM restart bug in XenServer 2016-12-15 14:12:59 -05:00
Rohit Yadav f850b381ad
Merge pull request #1831 from shapeblue/fix-sql-pr-1615-410
CLOUDSTACK-9671: Fix sql change to corresponding version paths- Fixes issue of failing upgrade paths
- Moves schema changes from PR #1615 in the 4.9.1.0 to 4.10.0.0 sql path

* pr/1831:
  CLOUDSTACK-9671: Fix sql change to corresponding version paths

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-15 12:10:05 +05:30
Rohit Yadav dd321a6de3 CLOUDSTACK-9671: Fix sql change to corresponding version paths
- Fixes issue of failing upgrade paths
- Moves schema changes from PR #1615 in the 4.9.1.0 to 4.10.0.0 sql path

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-15 12:00:47 +05:30
Bharat Kumar c6e9d82595 CLOUDSTACK-8857 listProjects doesn't return tags vmstopped or vmrunning when their value is zero 2016-12-12 13:36:46 +05:30
Jayapalu d2ca30a133 CLOUDSTACK-9617: Fixed enabling remote access after PF or LB configured on vpn tcp ports 2016-12-12 11:59:33 +05:30
Jayapalu fed58eebdd CLOUDSTACK-9615: Fixed applying ingress rules without ports 2016-12-12 11:11:02 +05:30
Rohit Yadav af2679959b Updating pom.xml version numbers for release 4.9.1.0
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-10 08:38:03 +05:30
Syed 79f4177212 CLOUDSTACK-9628: Use correct virtualsize with Swift as secondary storage 2016-12-09 15:48:08 -05:00
Rohit Yadav 17653a86fa
Merge branch '4.9' 2016-12-10 01:29:16 +05:30
Rohit Yadav 7da95172a4
Merge branch '4.8' into 4.9 2016-12-10 01:28:14 +05:30
Rohit Yadav a13ee852af Merge pull request #896 from yvsubhash/CLOUDSTACK-8908
CLOUDSTACK-8908 After copying the template charging for that template is getting stoppedThis is happening as the zone id is not part of the query.  Zone id is added to the query and unit tests are also added

* pr/896:
  CLOUDSTACK-8908 After copying the template charging for that template is stopped

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-10 00:15:35 +05:30
Rohit Yadav c89198f4c6 Merge pull request #1796 from sudhansu7/CLOUDSTACK-9626
CLOUDSTACK-9626: Instance fails to start after unsuccesful computeISSUE
============
Instance fails to start after unsuccesful compute offering upgrade.

TROUBLESHOOTING
==================
We observed VM instance get compute values "cpuNumber","cpuSpeed","memory" removed from table "user_vm_details", which cause instance fail to startup next time on XenServer

`mysql> select * from user_vm_details where vm_id=10;
--------------------------------------------------------------------------------------------------
id 	vm_id 	name 	value 	display

--------------------------------------------------------------------------------------------------
218 	10 	platform 	viridian:true;acpi:1;apic:true;pae:true;nx:true 	1
219 	10 	hypervisortoolsversion 	xenserver56 	1
220 	10 	Message.ReservedCapacityFreed.Flag 	true 	1

--------------------------------------------------------------------------------------------------
3 rows in set (0.00 sec)`

`2016-11-29 06:49:03,667 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-12:ctx-49c25b1d job-125) (logid:114a2f1b) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.ScaleVMCmdByAdmin
java.lang.NullPointerException
	at com.cloud.vm.UserVmManagerImpl.upgradeRunningVirtualMachine(UserVmManagerImpl.java:1664)
	at com.cloud.vm.UserVmManagerImpl.upgradeVirtualMachine(UserVmManagerImpl.java:1631)
	at com.cloud.vm.UserVmManagerImpl.upgradeVirtualMachine(UserVmManagerImpl.java:1561)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
	at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:106)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
	at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at com.sun.proxy.$Proxy197.upgradeVirtualMachine(Unknown Source)
	at org.apache.cloudstack.api.command.admin.vm.ScaleVMCmdByAdmin.execute(ScaleVMCmdByAdmin.java:48)
	at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:150)
	at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:554)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:502)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)`

REPRO STEPS
==================
1. Set global setting enable.dynamic.scale.vm to true
2. Create a custom Compute Offerings A
3. Create a VM instance apply A, ie. cpuNumber=1,cpuSpeed=1000,memory=512M
4. Create another custom Compute Offerings B
5. Change service offering to B, ie. cpuNumber=2,cpuSpeed=2000,memory=4096M (ensure 4 times over previous memory size), then you will encounter scaling failed
6. Stop VM instance , you will never startup again

EXPECTED BEHAVIOR
==================
Succeed Startup VM instance

ACTUAL BEHAVIOR
==================
Fail to start instance

RCA:
The ROLLBACK does not take care of restoring old service offering details. In case failure we are removing the new service offering details but restoring old service offering details is missing.

Before Fix:
`user_vm_details before upgrade.
mysql> select * from user_vm_details where vm_id =9;
+-----+-------+------------------------------------+-------------------------------------------------+---------+
| id  | vm_id | name                               | value                                           | display |
+-----+-------+------------------------------------+-------------------------------------------------+---------+
| 118 |     9 | platform                           | viridian:true;acpi:1;apic:true;pae:true;nx:true |       1 |
| 119 |     9 | hypervisortoolsversion             | xenserver56                                     |       1 |
| 120 |     9 | Message.ReservedCapacityFreed.Flag | false                                           |       1 |
| 121 |     9 | cpuNumber                          | 1                                               |       1 |
| 122 |     9 | cpuSpeed                           | 1000                                            |       1 |
| 123 |     9 | memory                             | 256                                             |       1 |
+-----+-------+------------------------------------+-------------------------------------------------+---------+
6 rows in set (0.00 sec)

user_vm_details after unsuccessful upgrade.
mysql> select * from user_vm_details where vm_id =9;
+-----+-------+------------------------------------+-------------------------------------------------+---------+
| id  | vm_id | name                               | value                                           | display |
+-----+-------+------------------------------------+-------------------------------------------------+---------+
| 133 |     9 | platform                           | viridian:true;acpi:1;apic:true;pae:true;nx:true |       1 |
| 134 |     9 | hypervisortoolsversion             | xenserver56                                     |       1 |
| 135 |     9 | Message.ReservedCapacityFreed.Flag | false                                           |       1 |
+-----+-------+------------------------------------+-------------------------------------------------+---------+
3 rows in set (0.00 sec)`

After fix:
`
mysql> select * from user_vm_details where vm_id =9;
+-----+-------+------------------------------------+-------------------------------------------------+---------+
| id  | vm_id | name                               | value                                           | display |
+-----+-------+------------------------------------+-------------------------------------------------+---------+
| 166 |     9 | cpuNumber                          | 1                                               |       1 |
| 167 |     9 | platform                           | viridian:true;acpi:1;apic:true;pae:true;nx:true |       1 |
| 168 |     9 | cpuSpeed                           | 1000                                            |       1 |
| 169 |     9 | Message.ReservedCapacityFreed.Flag | false                                           |       1 |
| 170 |     9 | memory                             | 256                                             |       1 |
| 171 |     9 | hypervisortoolsversion             | xenserver56                                     |       1 |
+-----+-------+------------------------------------+-------------------------------------------------+---------+
6 rows in set (0.00 sec)
`

* pr/1796:
  CLOUDSTACK-9626: Instance fails to start after unsuccesful compute offering  upgrade.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-09 23:52:47 +05:30
Sudhansu 542a2112ba CLOUDSTACK-9626: Instance fails to start after unsuccesful compute
offering  upgrade.
2016-12-09 17:06:13 +05:30
Koushik Das d6b41d9ac2 CLOUDSTACK-9660: NPE while destroying volumes during 1000 VMs deploy and destroy tests
NPE is seen as VM destroy and storage cleanup threads try to remove the same root volume. Fix is to handle
only non-root volumes in storage cleanup thread, root volumes will be handled as part of VM destroy.
2016-12-09 15:49:39 +05:30
Bharat Kumar bddd7c61a7 CLOUDSTACK-8856 Primary Storage Used(type tag with value 2) related tag is not showing in listCapacity api response.
Conflicts:
	server/src/com/cloud/server/ManagementServerImpl.java
2016-12-09 15:42:22 +05:30
Rohit Yadav f2798403b5 Merge pull request #1815 from pdion891/dockerfile4.10.0
Fix docker image build for cloudstack-managementCLOUDSTACK-9651

Fix Docker images build for 4.10.0.0:
- simulator
- marvin
- management_centos6

Squashed version of #1435.

* pr/1815:
  CLOUDSTACK-9651; fix docker image build for cloudstack-management, simulator and marvin for 4.10.0.0 close #1435

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-09 13:34:27 +05:30
Rohit Yadav fac8c5fc2a
Merge branch '4.9' 2016-12-09 13:32:43 +05:30
Rohit Yadav b931b798f8
Merge branch '4.8' into 4.9 2016-12-09 13:32:21 +05:30
Rohit Yadav f8c9f454d2 Merge pull request #1823 from shapeblue/4.8-travis-templatefix
CLOUDSTACK-9594: Fix regression in test_templatesFixes regression in component test `test_templates.py`

* pr/1823:
  CLOUDSTACK-9594: Fix regression in test_templates

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-09 13:31:51 +05:30
Nitin Kumar Maharana 25efc1acd6 CLOUDSTACK-9639: Unable to create shared network with vLan isolation 2016-12-09 12:55:31 +05:30
Rohit Yadav ae32aa13ed CLOUDSTACK-9594: Fix regression in test_templates
Fixes regression in component test `test_templates.py`

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-09 11:57:40 +05:30
Rohit Yadav 7df21f9cce
Merge branch '4.9' 2016-12-09 00:40:05 +05:30
Rohit Yadav f03015b25b Merge pull request #1817 from sateesh-chodapuneedi/pr-cloudstack-9654
CLOUDSTACK-9654 Missing hypervisor mapping of various SUSE Linux gues**Jira**
CLOUDSTACK-9654 Missing hypervisor mapping of various SUSE Linux guest os versions on VMware 6.0

**Issue:**
Currently many versions of SUSE Linux does not have any hypervisor mapping entry in guest_os_hypervisor table in cloud database for VMware 6.0. Also observed that the guest_os_name field is incorrect for some SUSE Linux variants, which results in deployed instance (with SUSE Linux) set to guest OS type as "Other (64-bit)" on vCenter, which would not represent the guest OS accurately on hypervisor.

**Fix:**
Add the missing hypervisor mappings
Signed-off-by: Sateesh Chodapuneedi <sateesh.chodapuneedi@accelerite.com>

The current (4.9) list of SUSE Linux guest os in database looks as below,

```
> mysql> select id,display_name from guest_os where display_name like '%suse%';
 +-----+----------------------------------------------+
 | id  | display_name                                 |
 +-----+----------------------------------------------+
 |  40 | SUSE Linux Enterprise Server 9 SP4 (32-bit)  |
 |  41 | SUSE Linux Enterprise Server 10 SP1 (32-bit) |
 |  42 | SUSE Linux Enterprise Server 10 SP1 (64-bit) |
 |  43 | SUSE Linux Enterprise Server 10 SP2 (32-bit) |
 |  44 | SUSE Linux Enterprise Server 10 SP2 (64-bit) |
 |  45 | SUSE Linux Enterprise Server 10 SP3 (64-bit) |
 |  46 | SUSE Linux Enterprise Server 11 (32-bit)     |
 |  47 | SUSE Linux Enterprise Server 11 (64-bit)     |
 |  96 | SUSE Linux Enterprise 8(32-bit)              |
 |  97 | SUSE Linux Enterprise 8(64-bit)              |
 | 107 | SUSE Linux Enterprise 9(32-bit)              |
 | 108 | SUSE Linux Enterprise 9(64-bit)              |
 | 109 | SUSE Linux Enterprise 10(32-bit)             |
 | 110 | SUSE Linux Enterprise 10(64-bit)             |
 | 151 | SUSE Linux Enterprise Server 10 SP3 (32-bit) |
 | 152 | SUSE Linux Enterprise Server 10 SP4 (64-bit) |
 | 153 | SUSE Linux Enterprise Server 10 SP4 (32-bit) |
 | 154 | SUSE Linux Enterprise Server 11 SP1 (64-bit) |
 | 155 | SUSE Linux Enterprise Server 11 SP1 (32-bit) |
 | 185 | SUSE Linux Enterprise Server 11 SP2 (64-bit) |
 | 186 | SUSE Linux Enterprise Server 11 SP2 (32-bit) |
 | 187 | SUSE Linux Enterprise Server 11 SP3 (64-bit) |
 | 188 | SUSE Linux Enterprise Server 11 SP3 (32-bit) |
 | 202 | Other SUSE Linux(32-bit)                     |
 | 203 | Other SUSE Linux(64-bit)                     |
 | 244 | SUSE Linux Enterprise Server 12 (64-bit)     |
 +-----+----------------------------------------------+
 26 rows in set (0.00 sec)

```

The current (4.9) hypervisor mappings for SUSE Linux guest os over VMware 6.0 in database looks as below. We can observe in the below query result, which lists all hypervisor mappings for SUSE Linux guest OS over VMware 6.0, many guest os listed in above query result are missing their mappings for VMware 6.0. Hence the need to add the missing hypervisor mappings.

```
mysql> select o.id,o.display_name, h.guest_os_name, h.hypervisor_version from guest_os as o, guest_os_hypervisor as h where o.id=h.guest_os_id and h.hypervisor_version='6.0' and h.hypervisor_type='vmware' and o.display_name like '%SUSE%';
+-----+----------------------------------+---------------+--------------------+
| id  | display_name                     | guest_os_name | hypervisor_version |
+-----+----------------------------------+---------------+--------------------+
|  96 | SUSE Linux Enterprise 8(32-bit)  | suseGuest     | 6.0                |
|  97 | SUSE Linux Enterprise 8(64-bit)  | suse64Guest   | 6.0                |
| 107 | SUSE Linux Enterprise 9(32-bit)  | suseGuest     | 6.0                |
| 108 | SUSE Linux Enterprise 9(64-bit)  | suse64Guest   | 6.0                |
| 109 | SUSE Linux Enterprise 10(32-bit) | suseGuest     | 6.0                |
| 110 | SUSE Linux Enterprise 10(64-bit) | suse64Guest   | 6.0                |
| 202 | Other SUSE Linux(32-bit)         | suseGuest     | 6.0                |
| 203 | Other SUSE Linux(64-bit)         | suse64Guest   | 6.0                |
+-----+----------------------------------+---------------+--------------------+
8 rows in set (0.00 sec)

```

* pr/1817:
  CLOUDSTACK-9654 Missing hypervisor mapping of various SUSE Linux guest os versions on VMware 6.0

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-09 00:38:41 +05:30
Rohit Yadav 6bb019ab63
Merge branch '4.9' 2016-12-09 00:34:26 +05:30
Rohit Yadav 7c27bf3c90
Merge branch '4.8' into 4.9 2016-12-09 00:33:46 +05:30
Rohit Yadav 8769597523 Merge pull request #1763 from sudhansu7/CLOUDSTACK-9594
CLOUDSTACK-9594: API "list templates templatefilter=all" reveals allAPI "list templates templatefilter=all" reveals all templates.
Using a "list templates templatefilter=all" API call any domain admin can see all templates of all domains in ACS. Information returned includes the account and domain of the template's owner.

The template data shows what that VM is using and any hints from the label. This would give an advantage in what attack vectors to use. The account and domain can possibly be used in brute force attack to guess the password and login information.

Test Scenario:

created two accounts in different domain.

```
mysql> select account_id,username,api_key from user where id in (4,5);
+------------+-----------+----------------------------------------------------------------------------------------+
| account_id | username  | api_key                                                                                |
+------------+-----------+----------------------------------------------------------------------------------------+
|          4 | sudadmin1 | 3qeSuWadNzUFZ_i6c6zbwafjM3Eo0TWpkHw3En9jNsg5Ditk2N18DnbbL2quBYQ7FsdXQ8rwxbyFlE8vyUTwEg |
|          5 | sudadmin  | N5uHVOrg1Ek1F1a_5OXTz4WpLG3ewHqcbPUSBjQ-2CTJdxmUe2go0S8fyqH4Np0scYiehYg2KqthZXCWEyKx1A |
+------------+-----------+----------------------------------------------------------------------------------------+
2 rows in set (0.00 sec)

mysql> select account_name,domain_id from account where id in (4,5);
+--------------+-----------+
| account_name | domain_id |
+--------------+-----------+
| sudadmin     |         2 |
| sudadmin1    |         3 |
+--------------+-----------+
2 rows in set (0.00 sec)
```

User sudadmin registered a private template named 'Debian'.

http://10.147.59.107:8080/client/api?apikey=N5uHVOrg1Ek1F1a_5OXTz4WpLG3ewHqcbPUSBjQ-2CTJdxmUe2go0S8fyqH4Np0scYiehYg2KqthZXCWEyKx1A&command=listTemplates&templatefilter=self&signature=ODt7zEWCLL20z1FT%2FIkd1molRaM%3D

listTemplate with "templatefilter=self", lists the newly registered template.

```
<listtemplatesresponse cloud-stack-version="4.8.0">
<count>1</count>
	<template>
		<id>51026d32-60ee-4e25-8ffd-3fa3c57fc14c</id>
		<name>Debian</name>
		<displaytext>Debian</displaytext>
		<ispublic>false</ispublic>
		<created>2016-11-10T17:18:00-0500</created>
		<isready>true</isready>
		<passwordenabled>false</passwordenabled>
		<format>VHD</format>
		<isfeatured>false</isfeatured>
		<crossZones>false</crossZones>
		<ostypeid>38c1fc84-a687-11e6-a8c8-06f654000053</ostypeid>
		<ostypename>Debian GNU/Linux 7(64-bit)</ostypename>
		<account>sudadmin</account>
		<zoneid>25fa5b74-d4c2-4bad-8e3a-ceffcd10985e</zoneid>
		<zonename>z1</zonename>
		<status>Download Complete</status>
		<size>2621440000</size>
		<templatetype>USER</templatetype>
		<hypervisor>XenServer</hypervisor>
		<domain>SUDDOMAIN</domain>
		<domainid>a350c00d-4048-4876-ae09-74ad4b7bb28c</domainid>
		<isextractable>false</isextractable>
		<checksum>e87a6d7291b999c92baa9623c9c3c207</checksum>
		<details>{hypervisortoolsversion=xenserver61}</details>
		<sshkeyenabled>false</sshkeyenabled>
		<isdynamicallyscalable>false</isdynamicallyscalable>
	</template>
</listtemplatesresponse>

```

User: sudadmin1
listTemplate with "templatefilter=self" does not list any template.

http://10.147.59.107:8080/client/api?apikey=3qeSuWadNzUFZ_i6c6zbwafjM3Eo0TWpkHw3En9jNsg5Ditk2N18DnbbL2quBYQ7FsdXQ8rwxbyFlE8vyUTwEg&command=listTemplates&templatefilter=self&signature=RfKsdg3RxDkqJotbTlHU2RdbdPA%3D

`<listtemplatesresponse cloud-stack-version="4.8.0"/>
`

NO TEMPLATES

**listTemplate with "templatefilter=all" lists all templates**

http://10.147.59.107:8080/client/api?apikey=3qeSuWadNzUFZ_i6c6zbwafjM3Eo0TWpkHw3En9jNsg5Ditk2N18DnbbL2quBYQ7FsdXQ8rwxbyFlE8vyUTwEg&command=listTemplates&templatefilter=all&signature=l5tubfyABT67d1jY702dvtZODbc%3D

Result:

```
<listtemplatesresponse cloud-stack-version="4.8.0">
<count>3</count>
	<template>
		<id>38451a02-a687-11e6-a8c8-06f654000053</id>
		<name>CentOS 5.6(64-bit) no GUI (XenServer)</name>
		<displaytext>CentOS 5.6(64-bit) no GUI (XenServer)</displaytext>
		<ispublic>true</ispublic>
		....
	</template>
	<template>
		<id>51026d32-60ee-4e25-8ffd-3fa3c57fc14c</id>
		<name>Debian</name>
		<displaytext>Debian</displaytext>
		<ispublic>false</ispublic>
		<created>2016-11-10T17:18:00-0500</created>
		<isready>true</isready>
		<passwordenabled>false</passwordenabled>
		<format>VHD</format>
		<isfeatured>false</isfeatured>
		<crossZones>false</crossZones>
		<ostypeid>38c1fc84-a687-11e6-a8c8-06f654000053</ostypeid>
		<ostypename>Debian GNU/Linux 7(64-bit)</ostypename>
		**<account>sudadmin</account>**
		<zoneid>25fa5b74-d4c2-4bad-8e3a-ceffcd10985e</zoneid>
		<zonename>z1</zonename>
		<size>2621440000</size>
		<templatetype>USER</templatetype>
		<hypervisor>XenServer</hypervisor>
		<domain>SUDDOMAIN</domain>
		<domainid>a350c00d-4048-4876-ae09-74ad4b7bb28c</domainid>
		<isextractable>false</isextractable>
		<checksum>e87a6d7291b999c92baa9623c9c3c207</checksum>
		<details>{hypervisortoolsversion=xenserver61}</details>
		<sshkeyenabled>false</sshkeyenabled>
		<isdynamicallyscalable>false</isdynamicallyscalable>
	</template>
	<template>
		<id>5f6af7bb-d965-4b9b-ab45-6d455b0d6bbe</id>
		<name>SystemVM Template (XenServer)</name>
		<displaytext>SystemVM Template (XenServer)</displaytext>
		<ispublic>false</ispublic>
		.....
	</template>
</listtemplatesresponse>

```

**After Fix:**

http://10.147.59.107:8080/client/api?apikey=3qeSuWadNzUFZ_i6c6zbwafjM3Eo0TWpkHw3En9jNsg5Ditk2N18DnbbL2quBYQ7FsdXQ8rwxbyFlE8vyUTwEg&command=listTemplates&templatefilter=all&signature=l5tubfyABT67d1jY702dvtZODbc%3D

```
<listtemplatesresponse cloud-stack-version="4.8.0">
<count>1</count>
	<template>
		<id>38451a02-a687-11e6-a8c8-06f654000053</id>
		<name>CentOS 5.6(64-bit) no GUI (XenServer)</name>
		<displaytext>CentOS 5.6(64-bit) no GUI (XenServer)</displaytext>
		<ispublic>true</ispublic>
		<created>2016-11-10T09:32:44-0500</created>
		<isready>true</isready>
		<passwordenabled>false</passwordenabled>
		<format>VHD</format>
		<isfeatured>true</isfeatured>
		<crossZones>true</crossZones>
		<ostypeid>38a2bfd6-a687-11e6-a8c8-06f654000053</ostypeid>
		<ostypename>CentOS 5.6 (64-bit)</ostypename>
		<account>system</account>
		<zoneid>25fa5b74-d4c2-4bad-8e3a-ceffcd10985e</zoneid>
		<zonename>z1</zonename>
		<size>21474836480</size>
		<templatetype>BUILTIN</templatetype>
		<hypervisor>XenServer</hypervisor>
		<domain>ROOT</domain>
		<domainid>383e0ea6-a687-11e6-a8c8-06f654000053</domainid>
		<isextractable>true</isextractable>
		<checksum>905cec879afd9c9d22ecc8036131a180</checksum>
		<sshkeyenabled>false</sshkeyenabled>
		<isdynamicallyscalable>true</isdynamicallyscalable>
	</template>
</listtemplatesresponse>

```

Bug has been fixed considering below points
1. templatefilter=all or isofilter=all is applicable only to admin and domain admin.
2. With templatefilter=all or isofilter=all below are the visiblity of templates in system.
- admin should be able to see all templates/iso in system.
- domain admin should be able to see all public template and templates under its domain tree (including sub domain).
- domain admin in a project context should be able to see all public templates and templates registered
as project account and templates which are shared(using updateTemplatePermission api) with project account.

Also Modified "test/integration/component/test_escalation_listTemplateDomainAdmin.py"
This marvin test was written for this scenario but for the second account "templatefilter=all" is not used.

* pr/1763:
  CLOUDSTACK-9594: reverted changes  introduced in  CLOUDSTACK-9376
  CLOUDSTACK-9594: API "list templates templatefilter=all" reveals all templates of all domains

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-09 00:31:09 +05:30
Rohit Yadav 01ec8e67b1
Merge branch '4.9' 2016-12-09 00:27:11 +05:30
Rohit Yadav 80724b01bc Merge pull request #1396 from nitin-maharana/CloudStack-Nitin26_4.7
CLOUDSTACK-9269: Missing field for Switch type for Management and Storage traffic types# Repro Steps:

Create an Advanced zone(VMware).
Configure physical network.
Edit traffic type (Management or Storage)
Observe the switch type dropdown list is missing.

If we choose Guest or Public the dropdown is visible. See the below snapshots.
# Expected Result:

The list should be shown in case of Management and storage.
# Actual Result:

The list is missing in case of management and storage.
# Fix:

Showing vswitchtype for all traffic types in case of VMware.
# Traffic Type - Guest:

![public-nitin](https://cloud.githubusercontent.com/assets/12583725/12759158/aec4d38e-ca05-11e5-911f-d8aa8cb995b7.jpg)
# Traffic Type - Management (Issue):

![management-nitin](https://cloud.githubusercontent.com/assets/12583725/12759171/bff00e08-ca05-11e5-8051-c24f0952a321.jpg)

* pr/1396:
  CLOUDSTACK-9269: Missing field for Switch type for Management and Storage traffic types

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-09 00:26:30 +05:30
Rohit Yadav 227c088d77 Merge pull request #1820 from pdube/CLOUDSTACK-9656-usage-exception
CLOUDSTACK-9656 Preventing autoboxing NPE in Usage by setting a default role when not foundhttps://issues.apache.org/jira/browse/CLOUDSTACK-9656

This is a workaround to avoid NPE when using the usage server with Projects

To reproduce bug:

Create project
Add account to project
Create VM within that project
Run the usage server

Same steps to test resolution

* pr/1820:
  CLOUDSTACK-9656: Preventing autoboxing NPE in Usage by setting a default role when not found

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-09 00:25:48 +05:30
Patrick Dube 1a034d168d CLOUDSTACK-9656: Preventing autoboxing NPE in Usage by setting a default role when not found 2016-12-08 13:26:46 -05:00