fix marvin test failure test_router_dhcp_optsmarvin, VirtualMachine object's, nic attribute does not have nic's in any
particualr order in the array. so check isdefault attribute to the get non-default nic
earlier test made assumption that nic[0] is default nic, which is not true always
* pr/1801:
CLOUDSTACK-9634: fix marvin test failure test_router_dhcp_opts
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
marvin, VirtualMachine object, nic attribute does not have nic's in any
particualr order in the array. soi check isdefault attribute to the get non-default nic
earlier test made assumption that nic[0] is default nic, which is not true always
CLOUDSTACK-9624 Incorrect hypervisor mapping of guest os Windows 2008 Server R2 (64-bit) for VMware**JIRA ticket**
CLOUDSTACK-9624 Incorrect hypervisor mapping of guest os Windows 2008 Server R2 (64-bit) for VMware
**Issue**
Guest OS Windows Server 2008 R2 (64-bit) is being mapped to incorrect guest os at hypervisor, which is winLonghorn64Guest, same as that of Windows Server 2008 (64-bit).
Due to this the VM's guest os type was set to "Other (64-bit)", which would not represent the guest OS accurately on hypervisor.
**Solution**
Fix is to update incorrect guest_os_name field value in DB table cloud.guest_os_hypervisor.
Th query is,
UPDATE IGNORE `cloud`.`guest_os_hypervisor` SET guest_os_name = 'windows7Server64Guest' WHERE guest_os_id IN (SELECT id FROM guest_os WHERE display_name LIKE 'windows%2008%r2%64%') AND hypervisor_type = 'VMware' AND hypervisor_version != 'default';
After running above query, the 6 updated rows looks like
UPDATE IGNORE `cloud`.`guest_os_hypervisor` SET guest_os_name = 'windows7Server64Guest' WHERE guest_os_id IN (SELECT id FROM guest_os WHERE display_name LIKE 'windows%2008%r2%64%') AND hypervisor_type = 'VMware' AND hypervisor_version != 'default';
Query OK, 6 rows affected (0.01 sec)
Rows matched: 6 Changed: 6 Warnings: 0
mysql> select * from guest_os_hypervisor where guest_os_id in (select id from guest_os where display_name like 'windows%2008%r2%64%') and hypervisor_type = 'VMware' and hypervisor_version != 'default';
+------+-----------------+-----------------------+-------------+--------------------+--------------------------------------+---------------------+---------+-----------------+
| id | hypervisor_type | guest_os_name | guest_os_id | hypervisor_version | uuid | created | removed | is_user_defined |
+------+-----------------+-----------------------+-------------+--------------------+--------------------------------------+---------------------+---------+-----------------+
| 1307 | VMware | windows7Server64Guest | 54 | 4.0 | 98fce372-b271-11e6-b56b-4e61adb7c6b1 | 2016-11-24 23:42:44 | NULL | 0 |
| 1448 | VMware | windows7Server64Guest | 54 | 4.1 | 990abdcc-b271-11e6-b56b-4e61adb7c6b1 | 2016-11-24 23:42:45 | NULL | 0 |
| 1589 | VMware | windows7Server64Guest | 54 | 5.0 | 99166f75-b271-11e6-b56b-4e61adb7c6b1 | 2016-11-24 23:42:45 | NULL | 0 |
| 1730 | VMware | windows7Server64Guest | 54 | 5.1 | 9930ff30-b271-11e6-b56b-4e61adb7c6b1 | 2016-11-24 23:42:45 | NULL | 0 |
| 1871 | VMware | windows7Server64Guest | 54 | 5.5 | 993acb18-b271-11e6-b56b-4e61adb7c6b1 | 2016-11-24 23:42:45 | NULL | 0 |
| 2381 | VMware | windows7Server64Guest | 54 | 6.0 | 9cb53675-b271-11e6-b56b-4e61adb7c6b1 | 2016-11-24 18:12:51 | NULL | 0 |
+------+-----------------+-----------------------+-------------+--------------------+--------------------------------------+---------------------+---------+-----------------+
6 rows in set (0.01 sec)
**Tests**
Registered a template with Windows 2008 R2 (64-bit) guest OS and deployed an instance from the template. Found that the VM appeared in vCenter with valid guest OS type instead of "Other (64-bit)" shown up before the fix.
* pr/1793:
CLOUDSTACK-9624 Incorrect hypervisor mapping of guest os Windows 2008 Server R2 (64-bit) for VMware
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9538: FIX failure in Deleting Snapshot From Primary Storage RBD Storage if vm has been removed
* pr/1710:
CLOUDSTACK-9538: FIX failure in Deleting Snapshot From Primary Storage RBD Storage if vm has been removed
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9584: run component tests in Travis runThis would run additional component tests in Travis run.
* pr/1755:
CLOUDSTACK-9584: run component tests in Travis run
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Issue
Guest OS Windows Server 2008 R2 (64-bit) is being mapped to incorrect guest os at hypervisor, which is winLonghorn64Guest, same as that of Windows Server 2008 (64-bit).
Due to this the VM's guest os type was set to "Other (64-bit)", which would not represent the guest OS accurately on hypervisor.
Solution
Fix is to update incorrect guest_os_name field value in DB table cloud.guest_os_hypervisor.
Th query is,
UPDATE IGNORE cloud.guest_os_hypervisor SET guest_os_name = 'windows7Server64Guest' WHERE guest_os_id IN (SELECT id FROM guest_os WHERE display_name LIKE 'windows%2008%r2%64%') AND hypervisor_type = 'VMware' AND hypervisor_version != 'default';
Update L10N files from Transifex (2016-11-27) for the new release 4.10.0.0I've check manually the correct display of the UI foreach language.
Don't forget to merge this before 4.10 release.
cc @jburwell @rhtyd
* pr/1789:
CLOUDSTACK-9621: Update L10N resource files with 4.10 strings from Transifex (20161127)
CLOUDSTACK-9621: Improve conversion Transifex's JSON format to CloudStack JS
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9416 : Enabling Static NAT on an associated Public IP to one of the NICs (networks) of a multi-NIC VM fails due to a wrong (default) Guest VM IP being selected in the GUIBug in ACS master GUI: Enabling Static NAT on an associated Public IP to one of the NICs (networks) of a multi-NIC VM fails due to a wrong (default) Guest VM IP being selected in the GUI.
ERROR:
INFO [c.c.a.ApiServer](catalina-exec-7:ctx-83926837 ctx-fc7aa5ed) VM ip 10.10.2.163 address not belongs to the vm
ACS GUI before the bug fix:
As you can see in the above attached ACS GUI screen shots, same (default) guest VM IP is being selected in the ACS GUI for both the NICs (networks) of the multi-NIC VM while enabling Static NAT on the corresponding associated Public IPs. Thus, we hit this issue.
ACS GUI after this bug fix:
As you can see in the above attached ACS GUI screen shots, this bug fix resolves the above mentioned issue.
* pr/1785:
CLOUDSTACK-9416 : Enabling Static NAT on an associated Public IP to one of the NICs (networks) of a multi-NIC VM fails due to a wrong (default) Guest VM IP being selected in the GUI
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9321 : Multiple Internal LB rules (more than one Internal LB rule with same source IP address) are not getting resolved in the corresponding InternalLbVm instance's haproxy.cfg fileMultiple Internal LB rules (more than one Internal LB rule with same source IP address) are not getting resolved in the corresponding InternalLbVm instance's haproxy.cfg file. Moreover, each time a new Internal LB rule is added to the corresponding InternalLbVm instance, it replaces the existing one. Thus, traffic corresponding to these un-resolved (old) Internal LB rules are getting dropped by the InternalLbVm instance.
PR contents:
1) Fix for this bug.
2) Marvin test coverage for Internal LB feature on master with native ACS setup (component directory) including validations for this bug fix.
3) Enhancements on our exiting Internal LB Marvin test code (nuagevsp plugins directory) to validate this bug fix.
4) PEP8 & PyFlakes compliance with the added Marvin test code.
* pr/1577:
CLOUDSTACK-9321 : Multiple Internal LB rules (more than one Internal LB rule with same source IP address) are not getting resolved in the corresponding InternalLbVm instance's haproxy.cfg file
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9402 : Support for underlay features (Source & Static NAT to underlay) in Nuage VSP pluginSupport for underlay features (Source & Static NAT to underlay) with Nuage VSP SDN Plugin including Marvin test coverage for corresponding Source & Static NAT features on master. Moreover, our Marvin tests are written in such a way that they can validate our supported feature set with both Nuage VSP SDN platform's overlay and underlay infra.
PR contents:
1) Support for Source NAT to underlay feature on master with Nuage VSP SDN Plugin.
2) Support for Static NAT to underlay feature on master with Nuage VSP SDN Plugin.
3) Marvin test coverage for Source & Static NAT to underlay on master with Nuage VSP SDN Plugin.
4) Enhancements on our exiting Marvin test code (nuagevsp plugins directory).
5) PEP8 & PyFlakes compliance with our Marvin test code.
* pr/1580:
CLOUDSTACK-9402 : Support for underlay features (Source & Static NAT to underlay) in Nuage VSP plugin
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9566 instance-id metadata for baremetal VM returns IDThere is difference in instance-id metadata across baremetal and other hypervisors.
On Baremetal
[root@ip-172-17-0-144 ~]# curl http://8.37.203.221/latest/meta-data/instance-id
6021
on Xen
[root@ip-172-17-2-103 ~]# curl http://172.17.0.252/latest/meta-data/instance-id
cbeb517a-e833-4a0c-b1e8-9ed70200fbbf
In both cases it should be vm's uuid.
* pr/1738:
CLOUDSTACK-9566 instance-id metadata for baremetal VM returns ID
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9402 : Marvin tests for Source NAT and Static NAT features verification with NuageVsp (both overlay and underlay infra).
Co-Authored-By: Prashanth Manthena <prashanth.manthena@nuagenetworks.net>, Frank Maximus <frank.maximus@nuagenetworks.net>
CLOUDSTACK-9561 After domain/account deletion, snapshot taken by the
domain/account remains undeleted
While deleting the UserAccount Cleanup for the removed VMs/volumes are not
happening. For the removed VMs, snapshots doesn't get cleaned. Only for running
VMs(volumes in ready state) the cleanup happens.
When the VM is desroyed, the volume is marked as destroyed and later storage
garbage collector perform the cleanup. But if we try delete domain/account
before storage garbage collector runs, then it fails.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9598: wrong defaut gateway for the nic in non-default network when guest VM has nic's in more than one guest network set the tag for each host in /etc/dhcphosts.txt, and use the tag to add exception in /etc/dhcpopts.txt to prevent sending default route, dns server in case if the nic is in non-default network
this was the behaviour with edithosts.sh prior to 4.6
* pr/1766:
CLOUDSTACK-9598: wrong defaut gateway for the nic in non-default network
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This commit adds a additional VirtIO channel with the name
'org.qemu.guest_agent.0' to all Instances.
With the Qemu Guest Agent the Hypervisor gains more control over the Instance if
these tools are present inside the Instance, for example:
* Power control
* Flushing filesystems
* Fetching Network information
In the future this should allow safer snapshots on KVM since we can instruct the
Instance to flush the filesystems prior to snapshotting the disk.
More information: http://wiki.qemu.org/Features/QAPI/GuestAgent
Keep in mind that on Ubuntu AppArmor still needs to be disabled since the default
AppArmor profile doesn't allow libvirt to write into /var/lib/libvirt/qemu
This commit does not add any communication methods through API-calls, it merely
adds the channel to the Instances and installs the Guest Agent in the SSVMs.
With the addition of the Qemu Guest Agent channel a second channel appears in /dev
on a SSVM as a VirtIO port.
The order in which the ports are defined in the XML matters for the naming inside
the SSVM VM and by not relying on /dev/vportXX but looking for a static name the
SSVM still boots properly if the order in the XML definition is changed.
A SSVM with both ports attached will have something like this:
root@v-215-VM:~# ls -l /dev/virtio-ports
total 0
lrwxrwxrwx 1 root root 11 May 13 21:41 org.qemu.guest_agent.0 -> ../vport0p2
lrwxrwxrwx 1 root root 11 May 13 21:41 v-215-VM.vport -> ../vport0p1
root@v-215-VM:~# ls -l /dev/vport*
crw------- 1 root root 251, 1 May 13 21:41 /dev/vport0p1
crw------- 1 root root 251, 2 May 13 21:41 /dev/vport0p2
root@v-215-VM:~#
In this case the SSVM port points to /dev/vport0p1, but if the order in the XML
is different it might point to /dev/vport0p2
By looking for a portname with a pre-defined pattern in /dev/virtio-ports we
do not rely on the order in the XML definition.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
CLOUDSTACK-8781 : Superfluous field during VPC creation
* pr/756:
CLOUDSTACK-8781 : Superfluous field during VPC creation
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9379: Support nested virtualization at VM level on VMware Hypervisor## Introduction
[JIRA TICKET](https://issues.apache.org/jira/browse/CLOUDSTACK-9379)
It is desired to support nested virtualization at VM level for VMware hypervisor. Current behaviour supports enabling/desabling global nested virtualization by modifying global config `'vmware.nested.virtualization'`. It is wished to improve this feature, having control at VM level instead of a global control only.
A new global configuration is added, to enable/disable VM nested virtualization control: `'vmware.nested.virtualization.perVM'`. Default value=false
After a vm deployment or start command, vm params include `'nestedVirtualizationFlag'` key and its value is:
- true -> nested virtualization enabled
- false -> nested virtualization disabled
**We will determinate nested virtualization enabled/disabled by examining this 3 values:**
- **(1)** global configuration `'vmware.nested.virtualization'` value
- **(2)** global configuration `'vmware.nested.virtualization.perVM'` value
- **(3)** `'nestedVirtualizationFlag'` value in `user_vm_details` if present, `null` if not.
Using this 3 values, there are different use cases:
- **(1)** = TRUE, **(2)** = TRUE, **(3)** is null -> _ENABLED_
- **(1)** = TRUE, **(2)** = TRUE, **(3)** = TRUE -> _ENABLED_
- **(1)** = TRUE, **(2)** = TRUE, **(3)** = FALSE -> _DISABLED_
- **(1)** = TRUE, **(2)** = FALSE, **(3)** indifferent -> _ENABLED_
- **(1)** = FALSE, **(2)** = TRUE, **(3)** is null -> _DISABLED_
- **(1)** = FALSE, **(2)** = TRUE, **(3)** = TRUE -> _ENABLED_
- **(1)** = FALSE, **(2)** = TRUE, **(3)** = FALSE -> _DISABLED_
- **(1)** = FALSE, **(2)** = FALSE, **(3)** indifferent -> _DISABLED_
* pr/1542:
CLOUDSTACK-9379: Support nested virtualization at VM level on VMware Hypervisor
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9498: VR CsFile search utility methods fail when search stThere is no real use of python 're' module in CsFile.py utility methods searchString, deleteLine. Regular string search is sufficient. These methods are used only for VPN user add/delete. Since VPN user password can have python 're' module meta characters, it interfere with search functionality.
Replacing re.search() with regular string search instead.
Change is confined to VPN add/delete users. Have run the test/integration/component/test_vpn_users.py
VPN remote access user limit tests ... === TestName: test_01_VPN_user_limit | Status : SUCCESS ===
ok
Test create VPN when L2TP port in use ... === TestName: test_02_use_vpn_port | Status : SUCCESS ===
ok
Test create NAT rule when VPN when L2TP enabled ... === TestName: test_03_enable_vpn_use_port | Status : SUCCESS ===
ok
Test add new users to existing VPN ... === TestName: test_04_add_new_users | Status : SUCCESS ===
ok
Test add duplicate user to existing VPN ... === TestName: test_05_add_duplicate_user | Status : SUCCESS ===
ok
Test as global admin, add a new VPN user to an existing VPN entry ... === TestName: test_06_add_VPN_user_global_admin | Status : SUCCESS ===
ok
Test as domain admin, add a new VPN user to an existing VPN entry ... === TestName: test_07_add_VPN_user_domain_admin | Status : SUCCESS ===
ok
* pr/1680:
CLOUDSTACK-9498: VR CsFile search utility methods fail when search string has 're' meta chars, and causing VPN user add/deelte to fail
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9491: incorrect parsing of device list to find ethernet index of plugged NICIn VmwareResource, findRouterEthDeviceIndex() method find ethernet interface index given
the mac address. This method is used, once a nic is plugged to determine ethernet interface.
"/proc/sys/net/ipv4/conf" from the VR and looped through the devices to find the right
ethernet interface. Howver current logic read it once, and loops through the device list.
Its observerd device may not show up '/proc/sys/net/ipv4/conf' immediatly once NIC is plugged
in the VM from vCenter.
Fix ensured, while waiting for 15 sec in the loop, read the latest content from /proc/sys/net/ipv4/conf
, so that right device list is processed.
Manual tested VPC scenarios of adding new tiers which uses findRouterEthDeviceIndex, to find the guest/public network ethernet index.
* pr/1681:
CLOUDSTACK-9491: incorrect parsing of device list to find ethernet index of plugged NIC
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9503: Increased the VR script timeout. Most of the changes are about converting int/long time values to joda Duration.
* pr/1745:
CLOUDSTACK-9503: Increased the VR script timeout. Most of the changes are about converting int/long time values to joda Duration.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Pall Helgason
Rohit Yadav
Murali Reddy
Sateesh Chodapuneedi
Milamber
Nick Livens
Wido den Hollander
