debian: allow rules to pick ACS_BUILD_OPTS from envOnly now debian builds can be noredist etc.
Old PR: https://github.com/apache/cloudstack/pull/1149
cc @remibergsma
* pr/1169:
debian: allow rules to pick ACS_BUILD_OPTS from env
Signed-off-by: Remi Bergsma <github@remi.nl>
Enables forkmode for surefire plugin and increases memory opts for JVM
for rate-limit tests. This tries to fix intermittent Jenkins failures
which look like:
multipleClientsCanAccessWithoutBlocking(org.apache.cloudstack.ratelimit.ApiRateLimitTest):
unable to create new native thread
(cherry picked from commit 421bc00cd6)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-6276 Fixing affinity groups for projectsWith some contributions from @resmo and @ustcweizhou.
This closes https://github.com/apache/cloudstack/pull/508
To test manually (need at least 2 hosts):
Create a project
Create an affinity group in that project
Deploy a vm with that affinity group
Deploy a second vm with that affinity group
They should be on different hosts
Ran old and new tests for affinity groups on the simulator
Test create affinity group as admin in project ... === TestName: test_01_admin_create_aff_grp_for_project | Status : SUCCESS ===
ok
Test create affinity group as domain admin for projects ... === TestName: test_02_doadmin_create_aff_grp_for_project | Status : SUCCESS ===
ok
Test create affinity group as user for projects ... === TestName: test_03_user_create_aff_grp_for_project | Status : SUCCESS ===
ok
Test create affinity group that exists (same name) for projects ... === TestName: test_4_user_create_aff_grp_existing_name_for_project | Status : SUCCESS ===
ok
#Delete Affinity Group by id. ... === TestName: test_01_delete_aff_grp_by_id | Status : SUCCESS ===
ok
#Delete Affinity Group by id should fail for user not in project ... === TestName: test_02_delete_aff_grp_by_id_another_user | Status : SUCCESS ===
ok
test DeployVM in anti-affinity groups ... === TestName: test_01_deploy_vm_anti_affinity_group | Status : SUCCESS ===
ok
test DeployVM in anti-affinity groups with more vms than hosts. ... === TestName: test_02_deploy_vm_anti_affinity_group_fail_on_not_enough_hosts | Status : SUCCESS ===
ok
List affinity group for a vm for projects ... === TestName: test_01_list_aff_grps_for_vm | Status : SUCCESS ===
ok
List multiple affinity groups associated with a vm for projects ... === TestName: test_02_list_multiple_aff_grps_for_vm | Status : SUCCESS ===
ok
List affinity groups by id for projects ... === TestName: test_03_list_aff_grps_by_id | Status : SUCCESS ===
ok
List Affinity Groups by name for projects ... === TestName: test_04_list_aff_grps_by_name | Status : SUCCESS ===
ok
List Affinity Groups by non-existing id for projects ... === TestName: test_05_list_aff_grps_by_non_existing_id | Status : SUCCESS ===
ok
List Affinity Groups by non-existing name for projects ... === TestName: test_06_list_aff_grps_by_non_existing_name | Status : SUCCESS ===
ok
List affinity group should list all for a vms associated with that group for projects ... === TestName: test_07_list_all_vms_in_aff_grp | Status : SUCCESS ===
ok
Update the list of affinityGroups by using affinity groupids ... === TestName: test_01_update_aff_grp_by_ids | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 16 tests in 581.706s
OK
Deploy vm as Admin in Affinity Group belonging to regular user (should fail) ... === TestName: test_01_deploy_vm_another_user | Status : SUCCESS ===
ok
Create Affinity Group as admin for regular user ... === TestName: test_02_create_aff_grp_user | Status : SUCCESS ===
ok
List Affinity Groups as admin for all the users ... === TestName: test_03_list_aff_grp_all_users | Status : SUCCESS ===
ok
List Affinity Groups belonging to admin user ... === TestName: test_04_list_all_admin_aff_grp | Status : SUCCESS ===
ok
List Affinity Groups belonging to regular user passing account id and domain id ... === TestName: test_05_list_all_users_aff_grp | Status : SUCCESS ===
ok
List Affinity Groups belonging to regular user passing group id ... === TestName: test_06_list_all_users_aff_grp_by_id | Status : SUCCESS ===
ok
Delete Affinity Group belonging to regular user ... === TestName: test_07_delete_aff_grp_of_other_user | Status : SUCCESS ===
ok
Test create affinity group as admin ... === TestName: test_01_admin_create_aff_grp | Status : SUCCESS ===
ok
Test create affinity group as domain admin ... === TestName: test_02_doadmin_create_aff_grp | Status : SUCCESS ===
ok
Test create affinity group as user ... === TestName: test_03_user_create_aff_grp | Status : SUCCESS ===
ok
Test create affinity group that exists (same name) ... === TestName: test_04_user_create_aff_grp_existing_name | Status : SUCCESS ===
ok
Test create affinity group with existing name but within different account ... === TestName: test_05_create_aff_grp_same_name_diff_acc | Status : SUCCESS ===
ok
Test create affinity group of non-existing type ... === TestName: test_06_create_aff_grp_nonexisting_type | Status : SUCCESS ===
ok
Delete Affinity Group by name ... === TestName: test_01_delete_aff_grp_by_name | Status : SUCCESS ===
ok
Delete Affinity Group as admin for an account ... === TestName: test_02_delete_aff_grp_for_acc | Status : SUCCESS ===
ok
Delete Affinity Group which has vms in it ... === TestName: test_03_delete_aff_grp_with_vms | Status : SUCCESS ===
ok
Delete Affinity Group with id which does not belong to this user ... === TestName: test_05_delete_aff_grp_id | Status : SUCCESS ===
ok
Delete Affinity Group by name which does not belong to this user ... === TestName: test_06_delete_aff_grp_name | Status : SUCCESS ===
ok
Delete Affinity Group by id. ... === TestName: test_08_delete_aff_grp_by_id | Status : SUCCESS ===
ok
Root admin should be able to delete affinity group of other users ... === TestName: test_09_delete_aff_grp_root_admin | Status : SUCCESS ===
ok
Deploy VM without affinity group ... === TestName: test_01_deploy_vm_without_aff_grp | Status : SUCCESS ===
ok
Deploy VM by aff grp name ... === TestName: test_02_deploy_vm_by_aff_grp_name | Status : SUCCESS ===
ok
Deploy VM by aff grp id ... === TestName: test_03_deploy_vm_by_aff_grp_id | Status : SUCCESS ===
ok
test DeployVM in anti-affinity groups ... === TestName: test_04_deploy_vm_anti_affinity_group | Status : SUCCESS ===
ok
Deploy vms by affinity group id ... === TestName: test_05_deploy_vm_by_id | Status : SUCCESS ===
ok
Deploy vm in affinity group of another user by name ... === TestName: test_06_deploy_vm_aff_grp_of_other_user_by_name | Status : SUCCESS ===
ok
Deploy vm in affinity group of another user by id ... === TestName: test_07_deploy_vm_aff_grp_of_other_user_by_id | Status : SUCCESS ===
ok
Deploy vm in multiple affinity groups ... === TestName: test_08_deploy_vm_multiple_aff_grps | Status : SUCCESS ===
ok
Deploy multiple vms in multiple affinity groups ... === TestName: test_09_deploy_vm_multiple_aff_grps | Status : SUCCESS ===
ok
Deploy VM by aff grp name and id ... === TestName: test_10_deploy_vm_by_aff_grp_name_and_id | Status : SUCCESS ===
ok
List affinity group for a vm ... === TestName: test_01_list_aff_grps_for_vm | Status : SUCCESS ===
ok
List multiple affinity groups associated with a vm ... === TestName: test_02_list_multiple_aff_grps_for_vm | Status : SUCCESS ===
ok
List affinity groups by id ... === TestName: test_03_list_aff_grps_by_id | Status : SUCCESS ===
ok
List Affinity Groups by name ... === TestName: test_04_list_aff_grps_by_name | Status : SUCCESS ===
ok
List Affinity Groups by non-existing id ... === TestName: test_05_list_aff_grps_by_non_existing_id | Status : SUCCESS ===
ok
List Affinity Groups by non-existing name ... === TestName: test_06_list_aff_grps_by_non_existing_name | Status : SUCCESS ===
ok
List affinity group should list all for a vms associated with that group ... === TestName: test_07_list_all_vms_in_aff_grp | Status : SUCCESS ===
ok
Update the list of affinityGroups by using affinity groupids ... === TestName: test_01_update_aff_grp_by_ids | Status : SUCCESS ===
ok
Update the list of affinityGroups by using affinity groupnames ... === TestName: test_02_update_aff_grp_by_names | Status : SUCCESS ===
ok
Update the list of affinityGroups for vm which is not associated ... === TestName: test_03_update_aff_grp_for_vm_with_no_aff_grp | Status : SUCCESS ===
ok
Update the list of Affinity Groups to empty list ... SKIP: Skip - Failing - work in progress
Update the list of Affinity Groups on running vm ... === TestName: test_05_update_aff_grp_on_running_vm | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 42 tests in 976.432s
OK (SKIP=1)
* pr/1134:
CLOUDSTACK-6276 Removing unused parameter in integration test for projects
CLOUDSTACK-6276 Removing unused parameter in integration test
CLOUDSTACK-6276 Fixing affinity groups for projects
Signed-off-by: Remi Bergsma <github@remi.nl>
[4.6] CLOUDSTACK-9075 - As a Developer I want the Private GW feature fixed on single VPCsThis PR fixes the issue we faced with Private Gateways on single VPC when using ACS 4.6.0 and onwards.
The root cause: during the VR refactor, the static routes configuration was left unimplemented.
This PR also improves the existing Replace ACL test and adds a new test, that cover the Private Gateway in a more complete way.
The new test does the following:
1. Create 2 VPCs
2. Create 2 Tiers - 1 per VPC
3. Deploy 2 VMs - 1 per Tier
4. Acquire 2 pub IPs - 1 per VPC
5. Create 2 PF rules - 1 per pub IP
6. Create 2 ACLs + rules - 1 per VPC
7. Assign new ACLs to Tiers
8. Create 2 Private GWs - 1 per VPC
9. Replace the Pvt GWs ACLs
10. Create 2 Static routes - 1 per Pvt GW
11. SSH into VM1 (VPC1) and from there ping VM2 (VPC2)
There is also a test for Private Gateways on Redundant VPCs. But I found out that the feature is broken in when used with rVPCs. It will be addressed in a separate Issue/PR.
I'm running the tests. Will post results as soon as they are ready.
* pr/1151:
CLOUDSTACK-9075 - Uses the same vlan since it should have been already released
CLOUDSTACK-9075 - Adds VPC static routes test
CLOUDSTACK-9075 - Covers Private GW ACL with Redundant VPCs
CLOUDSTACK-9075 - Add method to get list of Physical Networks per zone
Signed-off-by: Remi Bergsma <github@remi.nl>
We need to send an Unsolicited ARP to the gateway, instead of our own address. We now encounter problems when people deploy/destroy/deploy and get the same public ip.
[4.6.1] CLOUDSTACK-9022: keep Destroyed volumes for sometimefor now, the Destroyed volumes will be expunged in Storage cleanup thread, no matter when they are destroyed.
In Expunging vm thread, we expunge the Destroyed vms which have been destroyed at least 'expunge.delay' seconds. We add the similar configuration for volumes.
same to #1029 , for 4.6
* pr/1145:
CLOUDSTACK-9022: move storage.cleanup related global configurations to StorageManager
CLOUDSTACK-9022: keep Destroyed volumes for sometime
This closes#1029
Signed-off-by: Remi Bergsma <github@remi.nl>
make sure all files are updates with new versionDuring the previous release I had to edit these manually. This will fix that so all versions get updated.
* pr/1146:
Use version for RC branch name instead of branch
make sure all files are updates with new version
Signed-off-by: Remi Bergsma <github@remi.nl>
Update L10N resource files with 4.6 strings from Transifex (20151129)Small update of L10N files before the 4.6.1 release candidate
cc @remibergsma
* pr/1138:
Update L10N resource files with 4.6 strings from Transifex (20151129)
Signed-off-by: Remi Bergsma <github@remi.nl>
Fix secondary storage not working with swiftOriginal PR and discussion at #1112
* pr/1133:
Fix secondary storage not working with swift
Signed-off-by: Remi Bergsma <github@remi.nl>
[4.6] CLOUDSTACK-9083: Add disk serial to kvm virt xmlAdds disk serial ids based on volume uuids to the virt xml. This may be useful
for appliances/software that needs some serial ids on the VM disks. This does not
impact existing/running VMs, the vm virt xmls will be updates for running VMs
the next time they are stopped/started.
For testing, disk serial (of debian based systemvm) in the virt xml matched that
in /sys/devices/pci0000:00:0000:00:07.0/virtio4/block/vda/serial.
We currently don't support scsi-blcok devices for which serial is not supported,
for this we've added a DeviceType (LUN) which may be used in future and a check
to not add the serial to the xml if disk type is LUN.
Refer: https://libvirt.org/formatdomain.html#elementsDisks
* pr/1121:
CLOUDSTACK-9083: Add disk serial to kvm virt xml
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
[4.6] Cannot list vlanipranges by keywordBefore change:
cloudmonkey> list vlanipranges keyword=118
: Caught: com.mysql.jdbc.JDBC4PreparedStatement@18f36b6e: SELECT vlan.id, vlan.vlan_id, vlan.vlan_gateway, vlan.vlan_netmask, vlan.ip6_gateway, vlan.ip6_cidr, vlan.data_center_id, vlan.description, vlan.ip6_range, vlan.network_id, vlan.physical_network_id, vlan.vlan_type, vlan.uuid, vlan.removed, vlan.created FROM vlan WHERE ( OR vlan.description LIKE ** NOT SPECIFIED ** ) AND vlan.removed IS NULL ORDER BY vlan.id ASC LIMIT 0, 500
After change:
cloudmonkey> list vlanipranges keyword='118'
count = 1
vlaniprange:
id = 0d80fd9c-cd6b-4f99-96c6-261420e75f58
account = system
domain = ROOT
domainid = 2044762d-c4a5-11e3-8379-005056ac4490
......
* pr/1085:
Cannot list vlanipranges by keyword
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9064: The users should be able to create multiple Guest Shhttps://issues.apache.org/jira/browse/CLOUDSTACK-9064
CLOUDSTACK-9064: The users should be able to create multiple Guest Shared Networks in same Vlan ID, same Physical Network and same network, just with a different IP ranges.
* pr/1116:
CLOUDSTACK-9064: The users should be able to create multiple Guest Shared Networks in same Vlan ID, same Physical Network and same network, just with a different IP ranges.
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-9068: Listing Port Forwarding Rules take too much time to loadFor setting the width of each data item for each row of Port Forwarding rules, it was processing all rules.
Basically for each data item, it was searching in all rules, which is un-necessary.
If there are N-Rules, It was processing N-times.
Now, it only processes one time by taking all N-rules at a time.
The previous solution was of O(NxN). Now its changed to O(N).
* pr/1081:
CLOUDSTACK-9068: Listing Port Forwarding Rules take too much time to load
Signed-off-by: Remi Bergsma <github@remi.nl>
Fix event UUIDS missing on event busSame as pull request #1111, but this time on the 4.6 branch for forward merging in accordance with [the wiki](https://cwiki.apache.org/confluence/display/CLOUDSTACK/Release+principles+for+Apache+CloudStack+4.6+and+up#ReleaseprinciplesforApacheCloudStack4.6andup-HowtomergeaPullRequest?).
The fixing of CLOUDSTACK-8816 introduced a regression that removed the first class entities in the event bus description property. This is because everything was changed to use the Class as a key... Everything but the populateFirstClassEntities method in ActionEventUtils.
This commit tries to load the class key instead of the String key, which re-enables the populateFirstClassEntities method.
Likely this was not caught because of the trace exception handling... maybe some better logging/unit tests would be good for this PR.
* pr/1127:
Fix event UUIDS missing on event bus
Signed-off-by: Daan Hoogland <daan@onecht.net>
CLOUDSTACK-9077 Fix injectkeys.sh to work on CentOS7Fix regression from commit 3381154faf
The error seen on CentOS 7:
```
2015-11-20 21:51:16,161 DEBUG [c.c.s.ConfigurationServerImpl] (localhost-startStop-1:null) No loop device found, skipping ssh key insertion in systemvm.iso
```
Instead of detecting `/dev/loop0` this checks if we run inside Docker.
Tested on CentOS 7 and that now works again as expected.
```
2015-11-23 16:20:16,777 DEBUG [c.c.s.ConfigurationServerImpl] (localhost-startStop-1:null) Executing: /bin/bash /var/lib/tomcat/webapps/client/WEB-INF/classes/scripts/vm/systemvm/injectkeys.sh /home/cloud/.ssh/id_rsa.pub /home/cl
oud/.ssh/id_rsa /var/lib/tomcat/webapps/client/WEB-INF/classes/vms/systemvm.iso
2015-11-23 16:20:16,821 DEBUG [c.c.s.ConfigurationServerImpl] (localhost-startStop-1:null) Execution is successful.
2015-11-23 16:20:16,827 INFO [c.c.s.ConfigurationServerImpl] (localhost-startStop-1:null) Injected public and private keys into systemvm iso with result : null
```
Pinging @pdion891 to have a look.
* pr/1109:
CLOUDSTACK-9077 Fix injectkeys.sh to work on CentOS7
Signed-off-by: Remi Bergsma <github@remi.nl>
- Adds new controller types in the UI, for selecting root disk controller while
registering templates
- Fixes bug to not override disk controller type if provided in the details (either
vm details or from template details)
(cherry picked from commit c7d67628b3)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
commit #7
So far only 1 controller (scsi or ide) is supported in Cloudstack for ide or
scsi, this is existing limitation. Added support for 2nd IDE controller. Support adding IDE
virtual disk to VM. Also added check if VM is running as IDE virtual disk cannot be attached
to VM if VM is runnning.If user detaches a virtual disk on lower unit number of controller,
then subsequent attach operation should find free unit number on the controller and attach
the virtual disk there.
commit #6
Let the controllers of existing VMs continue without flip, current busInfo retrieved from
chain_info field of volume record from database would be preferred over
controller settings from all configuration settings.
commit #5
Editing global configuration param vmware.root.disk.controller osdefault value results
in loss of previous root disk controller type. Hence root disk's controller type for legacy
VMs is unknow post that modificaiton by user. If VM is stop/start then we could get this
infromation from bus info of existing volume. But if user resets VM and then try to start VM.
The existing bus info would be lost. Hence existing disk info is not available to depend on.
Using lsilogic or generic scsi controller for ROOT disk of legacy VMs if reset.
commit #4
Avoid adding additional (>1) scsi controllers to system vms. While attaching volume to legacy VM
don't use osdefault optoin which applicable only for VM created with the option enabled, use
legacy data disk controller type (lsilogic)
commit #3
If root disk's controller type is scsi and data disk controller type condenses
to any of scsi sub-types then data disk controller type would fall back to root disk controller itself. This
ensures data volumes would be accessible in all cases as controller of root volume would be reliable
and it means VM has the supported controller. It also avoids mix of scsi controller sub-types in a user instance.
Also translating disk controller type scsi to lsilogic.
commit #2
Support auto detection of recommended virtual disk controller type for specific guest OS.
commit #1
Support granual controller types. Add support for controller types in template registration as well.
Fix white spaces.
Removed stale HEAD merge lines
Removed tail of merge lines
Fixed VmwareResource, removing storage commands that moved to VmwareStorageProcessor.
removed stale code of controller that is present in processor
Fixed check style errors.
Fixed injection.
Tested with Linux and windows templates. Unable to run iso based tests due to few bugs in register iso area.
Signed-off-by: Sateesh Chodapuneedi <sateesh@apache.org>
(cherry picked from commit a4cc987a6f)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The fixing of CLOUDSTACK-8816 introduced a regression that removed the
first class entities in the event bus description property. This is
because everything was changed to use the Class as a key... Everything
but the populateFirstClassEntities method in ActionEventUtils.
Adds disk serial ids based on volume uuids to the virt xml. This may be useful
for appliances/software that needs some serial ids on the VM disks. This does not
impact existing/running VMs, the vm virt xmls will be updates for running VMs
the next time they are stopped/started.
For testing, disk serial (of debian based systemvm) in the virt xml matched that
in /sys/devices/pci0000:00:0000:00:07.0/virtio4/block/vda/serial.
We currently don't support scsi-blcok devices for which serial is not supported,
for this we've added a DeviceType (LUN) which may be used in future and a check
to not add the serial to the xml if disk type is LUN.
Refer: https://libvirt.org/formatdomain.html#elementsDisks
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Sortable columns
- Logos with cloudmonkey
- Wider UI
- CSS fixes and new status icons
- Configurable UI list api requests page size
- IP address column on instances page
- Listview widget enhancements
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Remi Bergsma
Rohit Yadav
Wei Zhou
Wilder Rodrigues
Milamber
Patrick Dube
Daan Hoogland
Syed
Sateesh Chodapuneedi
jeff
Srikanteswararao Talluri
Abhinandan Prateek
David Amorim Faria
Boris Schrijver