[CLOUDSTACK-9645] Followup fix for #1162: Add support for not (re)starting server after cloud-setup-management.New version against #1330, opened against master.
```
Moving stop of management in config inside if statement.
It will only run the stop command when --no-start is specified.
```
* pr/1566:
CLOUDSTACK-9645: Moving stop of management in config inside if statement.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9646: No usage is generated for uploaded templates/volumes from localpublished usage events on successful upload of template or volume.
* pr/1809:
CLOUDSTACK-9646: No usage is generated for uploaded templates/volumes from local
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9403 : Support for shared networks in Nuage VSP pluginThis is first phase of support of Shared Network in cloudstack through NuageVsp Network Plugin. A shared network is a type of virtual network that is shared between multiple accounts i.e. a shared network can be accessed by virtual machines that belong to many different accounts. This basic functionality will be supported with the below common use case:
- shared network can be used for monitoring purposes. A shared network can be assigned to a domain and can be used for monitoring VMs belonging to all accounts in that domain.
With the current implementation with NuageVsp plugin, Each shared network needs its unique IP address range, and can not overlap with another shared network.
In VSD, it is implemented in below manner:
- In order to have tenant isolation for shared networks, we will have to create a Shared L3 Subnet for each shared network, and instantiate it across the relevant enterprises. A shared network will only exist under an enterprise when it is needed, so when the first VM is spinned under that ACS domain inside that shared network.
PR contents:
1) Support for shared networks with tenant isolation on master with Nuage VSP SDN Plugin.
2) Marvin test coverage for shared networks on master with Nuage VSP SDN Plugin.
3) Enhancements on our exiting Marvin test code (nuagevsp plugins directory).
4) PEP8 & PyFlakes compliance with our Marvin test code.
* pr/1579:
CLOUDSTACK-9403: Support for shared networks in Nuage VSP plugin
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9339 Virtual Routers don't handle Multiple Public Interfaces correctlyAs pointed out in CLOUDSTACK-9339, in case of multiple public IP's from different public IP ranges are associated with VR, VR functionality is broken from 4.6. Below are the brief list of problems specific to non-VPC networks addressed in the PR. This PR handles both VPC and non-VPC scenarios.
- reverse traffic for the connections accepted on the eth3 and above public interfaces are getting blocked. Need a rule for e.g "-A FORWARD -i eth3 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT" in the FORWARD chain of filter table to permit reverse path traffic for established connections.
- outbound public traffic from eth0 to eth3 (or for interfaces above like eth4 eth5 etc) needs rule to run through FW_OUTBOUND chain in the filter table
- network stats on public interfaces eth3 are getting gathered
- default gateway is missing in the device specific routing table, resulting in traffic to be looked up in main routing table
- creating a device specific route table is generating "from all lookup Table_eth3" in the
ip rules, resulting in rest of the traffic getting blocked.
Picked few commits from #1519 from dsclose (https://github.com/apache/cloudstack/pull/1519) submitted for 4.7
Marvin tests are added to test below
- Static NAT works on the public interfaces above eth2, in case non-vpc networks
- Portforwarding works on the public interfaces above eth2, in case non-vpc networks
- Route tables are configured as expected for the device specific table for the public interfaces above eth2, in case non-vpc networks
- IP tables rules are as expected for the traffic from and to the public interfaces above eth2, in case non-vpc networks
* pr/1659:
CLOUDSTACK-9339 Virtual Routers don't handle Multiple Public Interfaces correctly
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
-when processing static nat rule, add a mangle table rule, to mark the traffic
from the guest vm when it has associated static nat rule so that traffic gets
routed using the route tabe of the device which has public ip associated
-fix the case where nic_device_id is empty when ip is getting disassociated
resulting in empty deviceid in ips.json
-add utility methods in CsRule, and CsRoute to add 'ip rule' and 'ip route' rules respectivley
-ensure traffic from all public interfaces are connection marked with device number, and restored
for the reverse traffic. use the connection marked number to do device specific routing table lookup
fill the device specific routing table with default route
-component tests for testing multiple public interfaces of VR
CLOUDSTACK-9632: Upgrade bouncy castle to version 1.55- Upgrades Maven dependency version to v1.55
- Fixes bountycastle usages and issues
- Adds timeout to jetty/annotation scanning
- Picks up PR #1510 by Daan
* pr/1799:
CLOUDSTACK-9632: Upgrade bouncy castle to version 1.55
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Upgrades Maven dependency version to v1.55
- Fixes bountycastle usages and issues
- Adds timeout to jetty/annotation scanning
- Fixes servlet issue, uses servlet 3.1.0
- Downgrade javassist used by reflections to fix annotation process errors
- Make console-proxy-rdp bc dependency same as rest of the codebase
- Picks up PR #1510 by Daan
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9564: Fix NPE due to intermittent test assertionThe test assertion on a pool object may return a null object, as objects
can be randomly expired/tombstoned. This will fix a NPE sometimes seen due
to recently merge for the fix for CLOUDSTACK-9564.
(we can merge this if Travis passes)
/cc @abhinandanprateek @murali-reddy
* pr/1816:
CLOUDSTACK-9564: Fix NPE due to intermittent test assertion
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The test assertion on a pool object may return a null object, as objects
can be randomly expired/tombstoned. This will fix a NPE sometimes seen due
to recently merge for the fix for CLOUDSTACK-9564.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Cloudstack 9586: When using local storage with Xenserver prepareTemplate does not work with multiple primary storeThe race condition will happen whenever there are multiple primary storages and the CS tries to mount the secondary store to xenserver host simultaneously.
Due to synchronised block one mount will be successful and other thread will get the already mounted SR. Without the fix the two thread will try to mount it parallely and one will fail on Xenserver.
* pr/1765:
Cloudstack 9586: When using local storage with Xenserver prepareTemplate does not work with multiple primary store The race condition will happen whenever there are multiple primary storages and the CS tries to mount the secondary store to xenserver host simultaneously. Due to synchronised block one mount will be successful and other thread will get the already mounted SR. Without the fix the two thread will try to mount it parallely and one will fail on Xenserver.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9564: Fix memory leaks in VmwareContextPoolIn a recent management server crash, it was found that the largest contributor
to memory leak was in VmwareContextPool where a registry is held (arraylist)
that grows indefinitely. The list itself is not used anywhere or consumed. There
exists a hashmap (pool) that returns a list of contexts for existing poolkey
(address/username) that is used instead.
This fixes the issue by removing the arraylist registry, and limiting the
length of the context list for a given poolkey.
@blueorangutan package
* pr/1729:
CLOUDSTACK-9564: Fix memory leaks in VmwareContextPool
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9635: fix test_privategw_acl.pyensure VLAN used for createPrivateGateway is determined after the guest
networks in the VPC is created, so that we skip VLAN allocated for guest
network for the private network of vpc gateway
* pr/1802:
CLOUDSTACK-9635: fix test_privategw_acl.py
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
ensure VLAN used for createPrivateGateway is determined after the guest
networks in the VPC is created, so that we skip VLAN allocated for guest
network for the private network of vpc gateway
The race condition will happen whenever there are multiple primary storages and the CS tries to mount the secondary store to xenserver host simultaneously.
Due to synchronised block one mount will be successful and other thread will get the already mounted SR. Without the fix the two thread will try to mount it parallely and one will fail on Xenserver.
This fixes build_asf.sh release script to update checkstyle pom.xml with the
provided new version.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9636: The host alerts box should be named as hosts in Alerts.The host Alerts box shows hosts in Alerts. The name host Alerts is misleading,
it should be changed to hosts in alerts.
For rest of the languages, it should be modified accordingly.
As I am not familiar with other languages, contributors familiar with other languages can suggest the change. or Open a new PR.
* pr/1803:
CLOUDSTACK-9636: The host alerts box should be named as hosts in Alerts.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9633:test_snapshot is failing due to incorrect string construction in utils.py
* pr/1800:
CLOUDSTACK-9633:test_snapshot is failing due to incorrect string construction in utils.py
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-8854: Apple Mac OS/X VM get created without USB controller in ESXi hypervisorsCLOUDSTACK-8854: Apple Mac OS/X VM get created without USB controller in ESXi hypervisors
Problem Description: CloudStack doesnt add a USB controller to the Apple Mac OS X VMs created in ESXi hypervisors. But, vSphere Client, by default, adds a USB Controller to the Mac OS VMs. Mac OS X machines require USB Controller for USB mouse and keyboard access.
Root Cause: The Guest OS details are specified in the Virtual Machine Configuration Spec for creating the VM (using the SDK API) in the EXSi hypervisor. No USB Controller is added to the Virtual Machine Configuration Spec. As the guest OS Identification details are specified in the VM Configuration Spec, It is assumed that the Create VM SDK API would create the defaults in the VM same as vSphere Client. But, as per the observation, USB Controller is not added to the Guest OS - Mac OS VM created through the SDK API.
Resolution: When the Guest OS is Apple Mac OS, Add the USB Controller (EHCI+UHCI - Mac supported) to the Virtual Machine Configuration Spec before Creating or Starting the VM. For any existing Mac OS VMs, Stop and Start to add the USB Controller. For new VMs with Mac OS, USB Controller is added automatically.
* pr/828:
CLOUDSTACK-8854: Apple Mac OS/X VM get created without USB controller in ESXi hypervisors
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
In a recent management server crash, it was found that the largest contributor
to memory leak was in VmwareContextPool where a registry is held (arraylist)
that grows indefinitely. The list itself is not used anywhere or consumed. There
exists a hashmap (pool) that returns a list of contexts for existing poolkey
(address/username) that is used instead.
This fixes the issue by removing the arraylist registry, and limiting the
length of the context list for a given poolkey.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
fix marvin test failure test_router_dhcp_optsmarvin, VirtualMachine object's, nic attribute does not have nic's in any
particualr order in the array. so check isdefault attribute to the get non-default nic
earlier test made assumption that nic[0] is default nic, which is not true always
* pr/1801:
CLOUDSTACK-9634: fix marvin test failure test_router_dhcp_opts
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
marvin, VirtualMachine object, nic attribute does not have nic's in any
particualr order in the array. soi check isdefault attribute to the get non-default nic
earlier test made assumption that nic[0] is default nic, which is not true always
CLOUDSTACK-9624 Incorrect hypervisor mapping of guest os Windows 2008 Server R2 (64-bit) for VMware**JIRA ticket**
CLOUDSTACK-9624 Incorrect hypervisor mapping of guest os Windows 2008 Server R2 (64-bit) for VMware
**Issue**
Guest OS Windows Server 2008 R2 (64-bit) is being mapped to incorrect guest os at hypervisor, which is winLonghorn64Guest, same as that of Windows Server 2008 (64-bit).
Due to this the VM's guest os type was set to "Other (64-bit)", which would not represent the guest OS accurately on hypervisor.
**Solution**
Fix is to update incorrect guest_os_name field value in DB table cloud.guest_os_hypervisor.
Th query is,
UPDATE IGNORE `cloud`.`guest_os_hypervisor` SET guest_os_name = 'windows7Server64Guest' WHERE guest_os_id IN (SELECT id FROM guest_os WHERE display_name LIKE 'windows%2008%r2%64%') AND hypervisor_type = 'VMware' AND hypervisor_version != 'default';
After running above query, the 6 updated rows looks like
UPDATE IGNORE `cloud`.`guest_os_hypervisor` SET guest_os_name = 'windows7Server64Guest' WHERE guest_os_id IN (SELECT id FROM guest_os WHERE display_name LIKE 'windows%2008%r2%64%') AND hypervisor_type = 'VMware' AND hypervisor_version != 'default';
Query OK, 6 rows affected (0.01 sec)
Rows matched: 6 Changed: 6 Warnings: 0
mysql> select * from guest_os_hypervisor where guest_os_id in (select id from guest_os where display_name like 'windows%2008%r2%64%') and hypervisor_type = 'VMware' and hypervisor_version != 'default';
+------+-----------------+-----------------------+-------------+--------------------+--------------------------------------+---------------------+---------+-----------------+
| id | hypervisor_type | guest_os_name | guest_os_id | hypervisor_version | uuid | created | removed | is_user_defined |
+------+-----------------+-----------------------+-------------+--------------------+--------------------------------------+---------------------+---------+-----------------+
| 1307 | VMware | windows7Server64Guest | 54 | 4.0 | 98fce372-b271-11e6-b56b-4e61adb7c6b1 | 2016-11-24 23:42:44 | NULL | 0 |
| 1448 | VMware | windows7Server64Guest | 54 | 4.1 | 990abdcc-b271-11e6-b56b-4e61adb7c6b1 | 2016-11-24 23:42:45 | NULL | 0 |
| 1589 | VMware | windows7Server64Guest | 54 | 5.0 | 99166f75-b271-11e6-b56b-4e61adb7c6b1 | 2016-11-24 23:42:45 | NULL | 0 |
| 1730 | VMware | windows7Server64Guest | 54 | 5.1 | 9930ff30-b271-11e6-b56b-4e61adb7c6b1 | 2016-11-24 23:42:45 | NULL | 0 |
| 1871 | VMware | windows7Server64Guest | 54 | 5.5 | 993acb18-b271-11e6-b56b-4e61adb7c6b1 | 2016-11-24 23:42:45 | NULL | 0 |
| 2381 | VMware | windows7Server64Guest | 54 | 6.0 | 9cb53675-b271-11e6-b56b-4e61adb7c6b1 | 2016-11-24 18:12:51 | NULL | 0 |
+------+-----------------+-----------------------+-------------+--------------------+--------------------------------------+---------------------+---------+-----------------+
6 rows in set (0.01 sec)
**Tests**
Registered a template with Windows 2008 R2 (64-bit) guest OS and deployed an instance from the template. Found that the VM appeared in vCenter with valid guest OS type instead of "Other (64-bit)" shown up before the fix.
* pr/1793:
CLOUDSTACK-9624 Incorrect hypervisor mapping of guest os Windows 2008 Server R2 (64-bit) for VMware
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9538: FIX failure in Deleting Snapshot From Primary Storage RBD Storage if vm has been removed
* pr/1710:
CLOUDSTACK-9538: FIX failure in Deleting Snapshot From Primary Storage RBD Storage if vm has been removed
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Rohit Yadav
Frank Maximus
Murali Reddy
Rajani Karuturi
Abhinandan Prateek
Pall Helgason
Boris
Suresh Kumar Anaparti
Nitin Kumar Maharana