Commit Graph

9713 Commits

Author SHA1 Message Date
Rohit Yadav 743dd4249c Merge pull request #1839 from shapeblue/CLOUDSTACK-9683
CLOUDSTACK-9683: system.vm.default.hypervisor will pin the hypervisor for VR too with this fix

* pr/1839:
  CLOUDSTACK-9683: system.vm.default.hypervisor will pin the hypervisor for VR too with this fix

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-22 10:01:52 +05:30
Rohit Yadav 6f98fcd3a5 CLOUDSTACK-9688: Fix failing smoke tests
Fixes failing smoke tests due to enviroment issues or corner cases:
- Fixes NPE in Template Manager

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-21 11:45:20 +05:30
Abhinandan Prateek ca8a00442c CLOUDSTACK-9683: system.vm.default.hypervisor will pin the hypervisor for VR too with this fix 2016-12-19 13:06:02 +05:30
Rohit Yadav 5e19e64f2f Updating pom.xml version numbers for release 4.9.2.0-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-16 20:48:16 +05:30
Rohit Yadav af2679959b Updating pom.xml version numbers for release 4.9.1.0
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-10 08:38:03 +05:30
Rohit Yadav c89198f4c6 Merge pull request #1796 from sudhansu7/CLOUDSTACK-9626
CLOUDSTACK-9626: Instance fails to start after unsuccesful computeISSUE
============
Instance fails to start after unsuccesful compute offering upgrade.

TROUBLESHOOTING
==================
We observed VM instance get compute values "cpuNumber","cpuSpeed","memory" removed from table "user_vm_details", which cause instance fail to startup next time on XenServer

`mysql> select * from user_vm_details where vm_id=10;
--------------------------------------------------------------------------------------------------
id 	vm_id 	name 	value 	display

--------------------------------------------------------------------------------------------------
218 	10 	platform 	viridian:true;acpi:1;apic:true;pae:true;nx:true 	1
219 	10 	hypervisortoolsversion 	xenserver56 	1
220 	10 	Message.ReservedCapacityFreed.Flag 	true 	1

--------------------------------------------------------------------------------------------------
3 rows in set (0.00 sec)`

`2016-11-29 06:49:03,667 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-12:ctx-49c25b1d job-125) (logid:114a2f1b) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.ScaleVMCmdByAdmin
java.lang.NullPointerException
	at com.cloud.vm.UserVmManagerImpl.upgradeRunningVirtualMachine(UserVmManagerImpl.java:1664)
	at com.cloud.vm.UserVmManagerImpl.upgradeVirtualMachine(UserVmManagerImpl.java:1631)
	at com.cloud.vm.UserVmManagerImpl.upgradeVirtualMachine(UserVmManagerImpl.java:1561)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
	at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:106)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
	at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at com.sun.proxy.$Proxy197.upgradeVirtualMachine(Unknown Source)
	at org.apache.cloudstack.api.command.admin.vm.ScaleVMCmdByAdmin.execute(ScaleVMCmdByAdmin.java:48)
	at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:150)
	at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:554)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:502)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)`

REPRO STEPS
==================
1. Set global setting enable.dynamic.scale.vm to true
2. Create a custom Compute Offerings A
3. Create a VM instance apply A, ie. cpuNumber=1,cpuSpeed=1000,memory=512M
4. Create another custom Compute Offerings B
5. Change service offering to B, ie. cpuNumber=2,cpuSpeed=2000,memory=4096M (ensure 4 times over previous memory size), then you will encounter scaling failed
6. Stop VM instance , you will never startup again

EXPECTED BEHAVIOR
==================
Succeed Startup VM instance

ACTUAL BEHAVIOR
==================
Fail to start instance

RCA:
The ROLLBACK does not take care of restoring old service offering details. In case failure we are removing the new service offering details but restoring old service offering details is missing.

Before Fix:
`user_vm_details before upgrade.
mysql> select * from user_vm_details where vm_id =9;
+-----+-------+------------------------------------+-------------------------------------------------+---------+
| id  | vm_id | name                               | value                                           | display |
+-----+-------+------------------------------------+-------------------------------------------------+---------+
| 118 |     9 | platform                           | viridian:true;acpi:1;apic:true;pae:true;nx:true |       1 |
| 119 |     9 | hypervisortoolsversion             | xenserver56                                     |       1 |
| 120 |     9 | Message.ReservedCapacityFreed.Flag | false                                           |       1 |
| 121 |     9 | cpuNumber                          | 1                                               |       1 |
| 122 |     9 | cpuSpeed                           | 1000                                            |       1 |
| 123 |     9 | memory                             | 256                                             |       1 |
+-----+-------+------------------------------------+-------------------------------------------------+---------+
6 rows in set (0.00 sec)

user_vm_details after unsuccessful upgrade.
mysql> select * from user_vm_details where vm_id =9;
+-----+-------+------------------------------------+-------------------------------------------------+---------+
| id  | vm_id | name                               | value                                           | display |
+-----+-------+------------------------------------+-------------------------------------------------+---------+
| 133 |     9 | platform                           | viridian:true;acpi:1;apic:true;pae:true;nx:true |       1 |
| 134 |     9 | hypervisortoolsversion             | xenserver56                                     |       1 |
| 135 |     9 | Message.ReservedCapacityFreed.Flag | false                                           |       1 |
+-----+-------+------------------------------------+-------------------------------------------------+---------+
3 rows in set (0.00 sec)`

After fix:
`
mysql> select * from user_vm_details where vm_id =9;
+-----+-------+------------------------------------+-------------------------------------------------+---------+
| id  | vm_id | name                               | value                                           | display |
+-----+-------+------------------------------------+-------------------------------------------------+---------+
| 166 |     9 | cpuNumber                          | 1                                               |       1 |
| 167 |     9 | platform                           | viridian:true;acpi:1;apic:true;pae:true;nx:true |       1 |
| 168 |     9 | cpuSpeed                           | 1000                                            |       1 |
| 169 |     9 | Message.ReservedCapacityFreed.Flag | false                                           |       1 |
| 170 |     9 | memory                             | 256                                             |       1 |
| 171 |     9 | hypervisortoolsversion             | xenserver56                                     |       1 |
+-----+-------+------------------------------------+-------------------------------------------------+---------+
6 rows in set (0.00 sec)
`

* pr/1796:
  CLOUDSTACK-9626: Instance fails to start after unsuccesful compute offering  upgrade.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-09 23:52:47 +05:30
Sudhansu 542a2112ba CLOUDSTACK-9626: Instance fails to start after unsuccesful compute
offering  upgrade.
2016-12-09 17:06:13 +05:30
Rohit Yadav 7c27bf3c90
Merge branch '4.8' into 4.9 2016-12-09 00:33:46 +05:30
Rohit Yadav 8769597523 Merge pull request #1763 from sudhansu7/CLOUDSTACK-9594
CLOUDSTACK-9594: API "list templates templatefilter=all" reveals allAPI "list templates templatefilter=all" reveals all templates.
Using a "list templates templatefilter=all" API call any domain admin can see all templates of all domains in ACS. Information returned includes the account and domain of the template's owner.

The template data shows what that VM is using and any hints from the label. This would give an advantage in what attack vectors to use. The account and domain can possibly be used in brute force attack to guess the password and login information.

Test Scenario:

created two accounts in different domain.

```
mysql> select account_id,username,api_key from user where id in (4,5);
+------------+-----------+----------------------------------------------------------------------------------------+
| account_id | username  | api_key                                                                                |
+------------+-----------+----------------------------------------------------------------------------------------+
|          4 | sudadmin1 | 3qeSuWadNzUFZ_i6c6zbwafjM3Eo0TWpkHw3En9jNsg5Ditk2N18DnbbL2quBYQ7FsdXQ8rwxbyFlE8vyUTwEg |
|          5 | sudadmin  | N5uHVOrg1Ek1F1a_5OXTz4WpLG3ewHqcbPUSBjQ-2CTJdxmUe2go0S8fyqH4Np0scYiehYg2KqthZXCWEyKx1A |
+------------+-----------+----------------------------------------------------------------------------------------+
2 rows in set (0.00 sec)

mysql> select account_name,domain_id from account where id in (4,5);
+--------------+-----------+
| account_name | domain_id |
+--------------+-----------+
| sudadmin     |         2 |
| sudadmin1    |         3 |
+--------------+-----------+
2 rows in set (0.00 sec)
```

User sudadmin registered a private template named 'Debian'.

http://10.147.59.107:8080/client/api?apikey=N5uHVOrg1Ek1F1a_5OXTz4WpLG3ewHqcbPUSBjQ-2CTJdxmUe2go0S8fyqH4Np0scYiehYg2KqthZXCWEyKx1A&command=listTemplates&templatefilter=self&signature=ODt7zEWCLL20z1FT%2FIkd1molRaM%3D

listTemplate with "templatefilter=self", lists the newly registered template.

```
<listtemplatesresponse cloud-stack-version="4.8.0">
<count>1</count>
	<template>
		<id>51026d32-60ee-4e25-8ffd-3fa3c57fc14c</id>
		<name>Debian</name>
		<displaytext>Debian</displaytext>
		<ispublic>false</ispublic>
		<created>2016-11-10T17:18:00-0500</created>
		<isready>true</isready>
		<passwordenabled>false</passwordenabled>
		<format>VHD</format>
		<isfeatured>false</isfeatured>
		<crossZones>false</crossZones>
		<ostypeid>38c1fc84-a687-11e6-a8c8-06f654000053</ostypeid>
		<ostypename>Debian GNU/Linux 7(64-bit)</ostypename>
		<account>sudadmin</account>
		<zoneid>25fa5b74-d4c2-4bad-8e3a-ceffcd10985e</zoneid>
		<zonename>z1</zonename>
		<status>Download Complete</status>
		<size>2621440000</size>
		<templatetype>USER</templatetype>
		<hypervisor>XenServer</hypervisor>
		<domain>SUDDOMAIN</domain>
		<domainid>a350c00d-4048-4876-ae09-74ad4b7bb28c</domainid>
		<isextractable>false</isextractable>
		<checksum>e87a6d7291b999c92baa9623c9c3c207</checksum>
		<details>{hypervisortoolsversion=xenserver61}</details>
		<sshkeyenabled>false</sshkeyenabled>
		<isdynamicallyscalable>false</isdynamicallyscalable>
	</template>
</listtemplatesresponse>

```

User: sudadmin1
listTemplate with "templatefilter=self" does not list any template.

http://10.147.59.107:8080/client/api?apikey=3qeSuWadNzUFZ_i6c6zbwafjM3Eo0TWpkHw3En9jNsg5Ditk2N18DnbbL2quBYQ7FsdXQ8rwxbyFlE8vyUTwEg&command=listTemplates&templatefilter=self&signature=RfKsdg3RxDkqJotbTlHU2RdbdPA%3D

`<listtemplatesresponse cloud-stack-version="4.8.0"/>
`

NO TEMPLATES

**listTemplate with "templatefilter=all" lists all templates**

http://10.147.59.107:8080/client/api?apikey=3qeSuWadNzUFZ_i6c6zbwafjM3Eo0TWpkHw3En9jNsg5Ditk2N18DnbbL2quBYQ7FsdXQ8rwxbyFlE8vyUTwEg&command=listTemplates&templatefilter=all&signature=l5tubfyABT67d1jY702dvtZODbc%3D

Result:

```
<listtemplatesresponse cloud-stack-version="4.8.0">
<count>3</count>
	<template>
		<id>38451a02-a687-11e6-a8c8-06f654000053</id>
		<name>CentOS 5.6(64-bit) no GUI (XenServer)</name>
		<displaytext>CentOS 5.6(64-bit) no GUI (XenServer)</displaytext>
		<ispublic>true</ispublic>
		....
	</template>
	<template>
		<id>51026d32-60ee-4e25-8ffd-3fa3c57fc14c</id>
		<name>Debian</name>
		<displaytext>Debian</displaytext>
		<ispublic>false</ispublic>
		<created>2016-11-10T17:18:00-0500</created>
		<isready>true</isready>
		<passwordenabled>false</passwordenabled>
		<format>VHD</format>
		<isfeatured>false</isfeatured>
		<crossZones>false</crossZones>
		<ostypeid>38c1fc84-a687-11e6-a8c8-06f654000053</ostypeid>
		<ostypename>Debian GNU/Linux 7(64-bit)</ostypename>
		**<account>sudadmin</account>**
		<zoneid>25fa5b74-d4c2-4bad-8e3a-ceffcd10985e</zoneid>
		<zonename>z1</zonename>
		<size>2621440000</size>
		<templatetype>USER</templatetype>
		<hypervisor>XenServer</hypervisor>
		<domain>SUDDOMAIN</domain>
		<domainid>a350c00d-4048-4876-ae09-74ad4b7bb28c</domainid>
		<isextractable>false</isextractable>
		<checksum>e87a6d7291b999c92baa9623c9c3c207</checksum>
		<details>{hypervisortoolsversion=xenserver61}</details>
		<sshkeyenabled>false</sshkeyenabled>
		<isdynamicallyscalable>false</isdynamicallyscalable>
	</template>
	<template>
		<id>5f6af7bb-d965-4b9b-ab45-6d455b0d6bbe</id>
		<name>SystemVM Template (XenServer)</name>
		<displaytext>SystemVM Template (XenServer)</displaytext>
		<ispublic>false</ispublic>
		.....
	</template>
</listtemplatesresponse>

```

**After Fix:**

http://10.147.59.107:8080/client/api?apikey=3qeSuWadNzUFZ_i6c6zbwafjM3Eo0TWpkHw3En9jNsg5Ditk2N18DnbbL2quBYQ7FsdXQ8rwxbyFlE8vyUTwEg&command=listTemplates&templatefilter=all&signature=l5tubfyABT67d1jY702dvtZODbc%3D

```
<listtemplatesresponse cloud-stack-version="4.8.0">
<count>1</count>
	<template>
		<id>38451a02-a687-11e6-a8c8-06f654000053</id>
		<name>CentOS 5.6(64-bit) no GUI (XenServer)</name>
		<displaytext>CentOS 5.6(64-bit) no GUI (XenServer)</displaytext>
		<ispublic>true</ispublic>
		<created>2016-11-10T09:32:44-0500</created>
		<isready>true</isready>
		<passwordenabled>false</passwordenabled>
		<format>VHD</format>
		<isfeatured>true</isfeatured>
		<crossZones>true</crossZones>
		<ostypeid>38a2bfd6-a687-11e6-a8c8-06f654000053</ostypeid>
		<ostypename>CentOS 5.6 (64-bit)</ostypename>
		<account>system</account>
		<zoneid>25fa5b74-d4c2-4bad-8e3a-ceffcd10985e</zoneid>
		<zonename>z1</zonename>
		<size>21474836480</size>
		<templatetype>BUILTIN</templatetype>
		<hypervisor>XenServer</hypervisor>
		<domain>ROOT</domain>
		<domainid>383e0ea6-a687-11e6-a8c8-06f654000053</domainid>
		<isextractable>true</isextractable>
		<checksum>905cec879afd9c9d22ecc8036131a180</checksum>
		<sshkeyenabled>false</sshkeyenabled>
		<isdynamicallyscalable>true</isdynamicallyscalable>
	</template>
</listtemplatesresponse>

```

Bug has been fixed considering below points
1. templatefilter=all or isofilter=all is applicable only to admin and domain admin.
2. With templatefilter=all or isofilter=all below are the visiblity of templates in system.
- admin should be able to see all templates/iso in system.
- domain admin should be able to see all public template and templates under its domain tree (including sub domain).
- domain admin in a project context should be able to see all public templates and templates registered
as project account and templates which are shared(using updateTemplatePermission api) with project account.

Also Modified "test/integration/component/test_escalation_listTemplateDomainAdmin.py"
This marvin test was written for this scenario but for the second account "templatefilter=all" is not used.

* pr/1763:
  CLOUDSTACK-9594: reverted changes  introduced in  CLOUDSTACK-9376
  CLOUDSTACK-9594: API "list templates templatefilter=all" reveals all templates of all domains

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-09 00:31:09 +05:30
Rohit Yadav f70428a232 Merge pull request #1805 from sudhansu7/CLOUDSTACK-9637
CLOUDSTACK-9637: Template create from snapshot does not populate vm_t**ISSUE**
============
Template create from snapshot does not populate vm_template_details

**REPRO STEPS**
==================
1. Register a template A and specify property:
Root disk controller: scsi
NIC adapter type: E1000
Keyboard type: us

2. Create a vm instance from template A

3. Take volume snapshot for vm instance

4. Delete VM instance

5. Switch to "Storage->Snapshots", convert snapshot to a template B

6. Observe template B does not inherit property from template A, the table vm_template_details is empty

**SOLUTION**: Retrieve and add source template details to VMTemplateVO.

Before Fix:

```

mysql> select id,name,source_template_id from vm_template where id=202;
+-----+--------+--------------------+
| id  | name   | source_template_id |
+-----+--------+--------------------+
| 202 | Debian |               NULL |
+-----+--------+--------------------+
1 row in set (0.00 sec)

mysql> select * from vm_template_details where template_id=202;
+----+-------------+--------------------+-------+---------+
| id | template_id | name               | value | display |
+----+-------------+--------------------+-------+---------+
|  1 |         202 | keyboard           | us    |       1 |
|  2 |         202 | nicAdapter         | E1000 |       1 |
|  3 |         202 | rootDiskController | scsi  |       1 |
+----+-------------+--------------------+-------+---------+
3 rows in set (0.00 sec)

mysql> select id,name,source_template_id from vm_template where source_template_id=202;
+-----+----------------+--------------------+
| id  | name           | source_template_id |
+-----+----------------+--------------------+
| 203 | derived-debian |                202 |
+-----+----------------+--------------------+
1 row in set (0.00 sec)

mysql> select * from vm_template_details where template_id=203;
Empty set (0.00 sec)

After Fix:

mysql> select id,name,source_template_id from vm_template where source_template_id=202;
+-----+--------------------------+--------------------+
| id  | name                     | source_template_id |
+-----+--------------------------+--------------------+
| 203 | derived-debian           |                202 |
| 204 | debian-derived-after-fix |                202 |
+-----+--------------------------+--------------------+
2 rows in set (0.00 sec)

mysql> select * from vm_template_details where template_id=204;
+----+-------------+--------------------+-------+---------+
| id | template_id | name               | value | display |
+----+-------------+--------------------+-------+---------+
|  4 |         204 | keyboard           | us    |       1 |
|  5 |         204 | nicAdapter         | E1000 |       1 |
|  6 |         204 | rootDiskController | scsi  |       1 |
+----+-------------+--------------------+-------+---------+
3 rows in set (0.00 sec)

```

**Marvin Test :** test_template_from_snapshot_with_template_details.py

**Result:**
```
test_01_create_template_snampshot (integration.component.test_template_from_snapshot_with_template_details.TestCreateTemplate) ... === TestName: test_01_create_template_snampshot | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 1 test in 864.523s

OK
```

* pr/1805:
  CLOUDSTACK-9637: Template create from snapshot does not populate vm_template_details

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-08 17:14:52 +05:30
Rohit Yadav de7a06efec Merge pull request #1809 from Accelerite/CLOUDSTACK-9646
CLOUDSTACK-9646: No usage is generated for uploaded templates/volumes from localpublished usage events on successful upload of template or volume.

* pr/1809:
  CLOUDSTACK-9646: No usage is generated for uploaded templates/volumes from local

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-08 00:17:40 +05:30
Sudhansu 7059f9e12d CLOUDSTACK-9594: reverted changes introduced in CLOUDSTACK-9376 2016-12-06 23:17:37 +05:30
Rohit Yadav abfcd5b95f CLOUDSTACK-9632: Upgrade bouncy castle to version 1.55
- Upgrades Maven dependency version to v1.55
- Fixes bountycastle usages and issues
- Adds timeout to jetty/annotation scanning
- Fixes servlet issue, uses servlet 3.1.0
- Downgrade javassist used by reflections to fix annotation process errors
- Make console-proxy-rdp bc dependency same as rest of the codebase
- Picks up PR #1510 by Daan

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-06 17:24:24 +05:30
Rajani Karuturi aed59433a2 CLOUDSTACK-9646: No usage is generated for uploaded templates/volumes from local
published usage events on successful upload of template or volume.
2016-12-05 10:51:23 +05:30
Sudhansu 545a214a42 CLOUDSTACK-9637: Template create from snapshot does not populate vm_template_details
Summary: Retrieve and add source template details to VMTemplateVO.
2016-12-03 18:45:08 +05:30
Rohit Yadav 50f80cc2a0
Merge branch '4.8' into 4.9 2016-11-25 13:03:04 +05:30
Rohit Yadav 5811d33658 Merge pull request #1635 from myENA/feature/honor_force_stop_vm
CLOUDSTACK-9451https://issues.apache.org/jira/browse/CLOUDSTACK-9451

Re-doing against 4.8 since this is a bug, not a feature.

* pr/1635:
  CLOUDSTACK-9451

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-11-25 12:56:23 +05:30
John Burwell dc1a7b4338 Merge release branch 4.8 to 4.9
* 4.8:
  CLOUDSTACK-9071: Properly parse stats.output.uri in StatsCollector
2016-11-17 00:46:12 -05:00
John Burwell 293ec4f3fc Merge pull request #1673 from wido/CLOUDSTACK-9071
CLOUDSTACK-9071: Properly parse stats.output.uri in StatsCollectorBoth host and path could have been NULL which causes the StatsCollector
no to start properly.

By checking if the Strings are not Empty or Null we make sure the StatsCollector
always runs and does not prevent the Management Server from starting.

* pr/1673:
  CLOUDSTACK-9071: Properly parse stats.output.uri in StatsCollector

Signed-off-by: John Burwell <meaux@cockamamy.net>
2016-11-17 00:45:21 -05:00
Sudhansu a13231226c CLOUDSTACK-9594: API "list templates templatefilter=all" reveals all
templates of all domains

Bug has been fixed considering below points
1. templatefilter=all or isofilter=all is applicable only to admin
and domain admin.
2. With templatefilter=all or isofilter=all below are the visiblity
of templates in system.
a. admin should be able to see all templates/iso in system.
b. domain admin should be able to see all public template and
templates under its domain tree (including sub domain).
c. domain admin in a project context should be able to see all public
 templates and templates registered as project account and templates
 which are shared(using updateTemplatePermission api) with project account.

Modified
"test/integration/component/test_escalation_listTemplateDomainAdmin.py"
This marvin test is written for this scenario but for the second account
 "templatefilter=all" is not used.
2016-11-11 19:56:35 +05:30
Rohit Yadav 240f44392c
Merge branch '4.8' into 4.9 2016-11-03 16:55:47 +05:30
Rohit Yadav c0a6fc2bfb
Merge pull request #1714 from yvsubhash/CLOUDSTACK-9553
CLOUDSTACK-9553 Usage event is not getting recorded for snapshots in a specific scenario

* pr/1714:
  CLOUDSTACK-9553 Usage event is not getting recorded for snapshots in a specific scenario

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-11-03 16:55:21 +05:30
Rohit Yadav 25b6990e22
server: Fix build regression caused by 32a39
A constructor signature has changed between 4.8 and 4.9+ branches which caused
failure in a unit test introduced by PR #1694. This fixes the unit test by
passing null as the additional parameter (the test does not need instantiated
object).

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-11-03 09:14:35 +05:30
Rohit Yadav 19951346ad
Merge branch '4.8' into 4.9 2016-11-02 23:03:31 +05:30
Rohit Yadav 68f22e2a43
Merge pull request #1694 from shapeblue/kvm-no-storage-failfast
CLOUDSTACK-9509: Host Connects Without StorageKVM hosts on shared storage failure was accepted by mgmt server with the
host state as Up, even though there was no primary/shared storage available on
it. This patch offers a quick fix by throwing an exception in the storage monitor
which connects storage pool on host. The failure is trapped by agent manager
that disconnects the agent without any investigation.

Based on Lab tests, KVM agent may take upto 2 minutes to attempt NFS mount when
the storage is inaccessible (firewalled, or shutdown) before returning back with
an error. It is safe to assume that this won't add pressure on mgmt server due to
several reconnection attempts, and KVM agent would retry reconnection every 2
minutes.

For such KVM hosts, where failure happens due to storage issues; they will be
briefly put in Alert state but will be mostly be in Connecting state during which
the KVM host attempts to mount/reconfigure NFS storage pool.

/cc @jburwell @karuturi
@blueorangutan package

* pr/1694:
  CLOUDSTACK-9509: Host Connects Without Storage

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-11-02 22:57:51 +05:30
Rohit Yadav fcee71f35b Merge branch '4.8' into 4.9 2016-10-28 11:43:16 +05:30
Marc-Aurèle Brothier 158497d68a CLOUDSTACK-9544: Check access on account trying to generate user API keys
This fixes CVE-2016-6813

Signed-off-by: Marc-Aurèle Brothier <m@brothier.org>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-10-27 22:15:49 +05:30
subhash_y 3987b8c6b4 CLOUDSTACK-9553 Usage event is not getting recorded for snapshots in a specific scenario 2016-10-24 11:47:53 +05:30
Rajani Karuturi 72612f79cd Merge release branch 4.8 to 4.9
* 4.8:
  Emit a VOLUME_DELETE usage event when account deletion destroys an instance.
2016-10-24 11:13:06 +05:30
nnesic d989c5d8be Emit a VOLUME_DELETE usage event when account deletion destroys an instance.
Currently the logic about volume deletion seems to be that an event
should be emitted when the volume delete is requested, not when the
deletion completes.

The VolumeStateListener specifically ignores destroy events for ROOT
volumes, assuming that the ROOT volume only gets deleted when the
instance is destroyed and the UserVmManager should take care of it.

When deleting an account, all of its resources get destroyed, but the
instance expunging circumvents the UserVmManager, and thus we miss the
VOLUME_DESTROY usage event. The account manager now attempts to
propperly destroy the vm before expunging it. This way the destroy
logic is respected, including the event emission.
2016-10-21 09:26:59 +00:00
Rohit Yadav 32a397aa93 CLOUDSTACK-9509: Host Connects Without Storage
KVM hosts on shared storage failure was accepted by mgmt server with the
host state as Up, even though there was no primary/shared storage available on
it. This patch offers a quick fix by throwing an exception in the storage monitor
which connects storage pool on host. The failure is trapped by agent manager
that disconnects the agent without any investigation.

Based on Lab tests, KVM agent may take upto 2 minutes to attempt NFS mount when
the storage is inaccessible (firewalled, or shutdown) before returning back with
an error. It is safe to assume that this won't add pressure on mgmt server due to
several reconnection attempts, and KVM agent would retry reconnection every 2
minutes.

For such KVM hosts, where failure happens due to storage issues; they will be
briefly put in Alert state but will be mostly be in Connecting state during which
the KVM host attempts to mount/reconfigure NFS storage pool.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-10-21 10:22:32 +05:30
nvazquez f4a42ab0d7 CLOUDSTACK-9535: [API] listVMSnapshots improvement 2016-10-05 17:42:14 -03:00
Wido den Hollander c1997a1705
CLOUDSTACK-9071: Properly parse stats.output.uri in StatsCollector
Both host and path could have been NULL which causes the StatsCollector
no to start properly.

By checking if the Strings are not Empty or Null we make sure the StatsCollector
always runs and does not prevent the Management Server from starting.

Signed-off-by: Wido den Hollander <wido@widodh.nl>

Conflicts:
	server/src/com/cloud/server/StatsCollector.java
2016-09-29 16:00:38 +02:00
Nathan Johnson 53fd4a7997 CLOUDSTACK-9451
Honor the forced parameter to stop virtual machine api call.
2016-09-09 13:00:17 -05:00
John Burwell 8d11511b1f Adds support for four position versions and optional db upgrades
Often, patch and security releases do not require schema migrations or
data migrations.  However, if an empty upgrade class and associated
scripts are not defined, the upgrade process will break.  With this
change, if a release does not have an upgrade, a noop DbUpgrade is added
to the upgrade path.  This approach allows the upgrade to proceed and
for the database to properly reflect the installed version.  This change
should make the release process simpler as RMs no longer need to
rememeber to create this boilerplate code when starting a new release.

Beginning with the 4.8.2.0 and 4.9.1.0 releases, the project will
formally adopt a four (4) position release number to properly accomodate
rekeases that contain only CVE fixes.  The DatabaseUpgradeChecker and
Version classes made assumptions that they would always parse and
compare three (3) position version numbers.  This change adds the
CloudStackVersion value object that supports both three (3) and four (4)
version numbers.   It encapsulates version comparsion logic, as well as,
the rules to allow three (3) and four (4) to interoperate.

  * Modifies DatabaseUpgradeChecker to handle derive an upgrade path for
  a version that was not explicitly specified.  It determines the
  releases the first release before it with database migrations and uses
  that list as the basis for the list for version being calculated.  A
  noop upgrade is then added to the list which causes no schema changes
  or data migrations, but will update the database to the version.
  * Adds unit tests for the upgrade path calculation logic in
  DatabaseUpgradeChecker
  * Removes dummy upgrade logic for the 4.8.2.0 introduced in previous
  versions of this patch
  * Introduces the CloudStackVersion value object which parses and
  compares three (3) and four (4) position version numbers.  This class
  is intended to replace com.cloud.maint.Version.
  * Adds the junit-dataprovider dependency -- allowing test data to be
  concisely generated separately from the execution of a test case.
  Used extensively in the CloudStackVersionTest.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-30 13:32:32 +05:30
Rohit Yadav e49ca1ecd2 Merge pull request #1646 from shapeblue/4.9-491upgradepath
[4.9/LTS] Add upgrade path from 4.9.0 to 4.9.1, change version to 4.9.1.0-SNAPSHOTThis adds db upgrade path from 4.9.0 to 4.9.1 and fixes a typo in default user role description (CLOUDSTACK-9449)

/cc @karuturi @jburwell  -- this will cause issues when fwd-merged to master, I can do the fwd-merging if you would like to avoid fixing the conflicts yourself

@blueorangutan package

* pr/1646:
  Updating pom.xml version numbers for release 4.9.1.0-SNAPSHOT
  cloudstack: upgrade path from 4.9.0 to 4.9.1

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-23 13:48:30 +05:30
Rohit Yadav 1060dc47e5 Merge pull request #1620 from shapeblue/4.9-oobm-password-fix
oobm: simply change password transactional logic- Simplifies change password transactional logic without using pessmistic locks
- Adds a re-enter password field in the UI to valid ipmi/oobm password

* pr/1620:
  oobm: simply change password transactional logic

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-22 15:52:14 +05:30
Rohit Yadav f13c224da1 Updating pom.xml version numbers for release 4.9.1.0-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-19 13:53:39 +05:30
Will Stevens 740bd45be6 Updating pom.xml version numbers for release 4.8.2-SNAPSHOT
Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-08-15 10:31:09 -04:00
Will Stevens 007c9f0d3a Merge branch '4.8.1-RC20160808T1006' into 4.8 2016-08-15 10:28:33 -04:00
Will Stevens a63db21d16 Updating pom.xml version numbers for release 4.8.1
Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-08-08 10:06:21 -04:00
Rajani Karuturi c03d035c8b Merge release branch 4.8 to 4.9
* 4.8:
  server: give more memory to tests
  packaging: Marvin and integration-tests packages
2016-08-05 14:30:45 +05:30
Rajani Karuturi a54a3b5cd5 Merge release branch 4.7 to 4.8
* 4.7:
  server: give more memory to tests
  packaging: Marvin and integration-tests packages
2016-08-05 14:29:16 +05:30
Rohit Yadav e5750b3331 server: give more memory to tests
Increases allowed max and permgen memory flags to maven-surefire plugins.
This fixes unit test failures in cloud-server.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit fd7273b446)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-04 15:52:29 +05:30
Rohit Yadav 8ac3c883cd oobm: simply change password transactional logic
- Simplifies change password transactional logic without using pessmistic locks
- Adds a re-enter password field in the UI to valid ipmi/oobm password

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-02 12:47:15 +05:30
Will Stevens 227ff3884d Updating pom.xml version numbers for release 4.9.0
Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-07-25 16:56:04 -04:00
nvazquez 148e974482 CLOUDSTACK-9436: Release network resources on expunge command 2016-07-21 12:25:22 -03:00
Will Stevens 46a6530e70 Revert "Merge pull request #1594 from nvazquez/vmnetworkmapissue"
This reverts commit 9be93c6e90, reversing
changes made to 8d45d711bf.
2016-07-21 11:04:10 -04:00
nvazquez c754a0cf30 CLOUDSTACK-9407: Refactor 2016-06-21 13:49:55 -03:00
nvazquez d3f3fb0590 CLOUDSTACK-9407: Release network resources on expunge command 2016-06-16 12:35:42 -03:00