Commit Graph

31471 Commits

Author SHA1 Message Date
Paul Angus a5201746e9 individually disable guest services and check for failures 2018-12-18 07:50:19 +00:00
Paul Angus b01582cb94 allow systemd based guest agents to start regardless of systemd detect virtualisation type 2018-12-12 22:10:31 +00:00
Paul Angus 2183212dcd Merge branch '41120-memory' of https://github.com/ShapeBlue/cloudstack into 41120-memory
# Conflicts:
#	tools/appliance/systemvmtemplate/scripts/configure_systemvm_services.sh
2018-12-09 17:03:34 +00:00
Paul Angus 2a10fc1e1d move dedication 2018-12-09 17:00:37 +00:00
Paul Angus 198efd44c9 force true for testing 2018-12-09 09:34:47 +00:00
Paul Angus cd38191599 /lib/systemd/systemd-sysv-install disable qemu-guest-agent 2018-12-09 08:15:37 +00:00
Paul Angus ea6f0de8f0 base attempt to move disable services and selectively start them 2018-12-08 21:10:04 +00:00
Paul Angus 5aac32d2b2 put xe-guest-tools back 2018-12-05 21:49:48 +00:00
Paul Angus 5e0fad00f8 switch rng-tools for haveged (better for cloud instances) and use repo version of xen-tools instead of outdated version 2018-12-05 14:59:28 +00:00
Paul Angus 6bad3b8cec remove no longer required aria2 adn add rng-tools 2018-12-03 21:50:38 +00:00
Paul Angus f716510bae move changes to postinit 2018-11-30 13:43:41 +00:00
Paul Angus 1d996807a4 workaround for startup race condition. 2018-11-30 10:31:36 +00:00
Paul Angus c673284d9b restart vmware tools if it fails
tune apache2 mpm_event worker
remove ntp - will conflict with vmtools
2018-11-29 12:35:15 +00:00
Paul Angus aceb81ce21 alternate way of wrting 2018-11-28 08:30:01 +00:00
Paul Angus 7a5b167f90 further tweaks - add vm.min_free_kbytes=20480 2018-11-27 17:28:38 +00:00
Paul Angus 7aa12dbc62 adjust cache back pressure dynamically on startup - reduce to default on 'normal' memory size (4GB+) 2018-11-26 09:36:12 +00:00
Paul Angus b7c73ee9ba fix typo in chmod 2018-11-24 07:18:39 +00:00
Paul Angus 968a1e16d6 fix blank tools.conf
increase swap backpressure
periodically sync and clear cache
disable vgauth
2018-11-23 20:56:07 +00:00
Paul Angus 90f4cb9369 add cache backpressure 2018-11-23 15:16:17 +00:00
Paul Angus 5aae410dfc Updating pom.xml version numbers for release 4.11.2.0
Signed-off-by: Paul Angus <paul.angus@shapeblue.com>
2018-11-13 09:24:27 +00:00
Rohit Yadav 4d8e75cec8
systemvmtemplate: update debian 9.6 iso url and checksum (#3022)
This fixes the failing systemvmtemplate build due to 404 on old ISO url.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-11-13 06:30:09 +05:30
Bitworks LLC f6e600e4d8 CLOUDSTACK-3009: Fix resource calculation CPU, RAM for accounts. (#3012)
The view "service_offering_view" doesn't include removed SOs, as a result when SO is removed, the bug happens. The PR introduces a change for resource calculation changing "service_offering_view" to "service_offering" table which has all service offerings.

Must be fixed in:

4.12
4.11
Fixes: #3009
2018-11-13 06:29:08 +05:30
Paul Angus f95aec4a84
Merge pull request #3018 from shapeblue/fixrouterfilecreation
Prevent error on GroupAnswers on VR creation
2018-11-12 13:25:08 +00:00
Nicolas Vazquez bb7493ad4b configdrive: Add missing ConfigDrive entries on existing zones after upgrade (#3007)
After upgrade existing environments to 4.11, ConfigDrive cannot be enabled for existing zones due to missing entry on 'physical_network_service_providers' table.
2018-11-12 11:30:00 +05:30
nvazquez dea0b3eb78 Prevent error on GroupAnswers on VR creation 2018-11-09 15:30:57 -03:00
Nicolas Vazquez 7d8eb37924 [4.11] Fix set initial reservation on public IP ranges (#2980)
* Fix initial reservation on public IP ranges

* Do not allow dedicating a system VM IP range
2018-11-07 10:48:07 -02:00
Nicolas Vazquez af0c1e48cf Fix DirectNetworkGuru canHandle checks for lowercase isolation methods (#3010) 2018-11-07 09:53:01 -02:00
Rohit Yadav c6e53f6cc6
kvm: reset KVM host on heartbeat failure (#2984)
On actual testing, I could see that kvmheartbeat.sh script fails on NFS
server failure and stops the agent only. Any HA VMs could be launched
in different hosts, and recovery of NFS server could lead to a state
where a HA enabled VM runs on two hosts and can potentially cause
disk corruptions. In most cases, VM disk corruption will be worse than
VM downtime. I've kept the sleep interval between check/rounds but
reduced it to 10s. The change in behaviour was introduced in #2722.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-30 15:13:59 +05:30
Rene Diepstraten f7bc5807a3 vr: defer was broken in VR because of json name change
Committed at f0491d5c72 (#2979).

After upgrade from CS 4.10 to CS 4.11, multiple VRs did not start through.
It did not properly defer the finalize config in update_config.py.
Apparently, the json files are now called differently: where it used to
be vm_dhcp_entry.json it now has a uuid added, for example
vm_metadata.json.4d727b6e-2b48-49df-81c3-b8532f3d6745.
The if statement that checks if the finalize can be safely deferred
therefore no longer matches. This PR contains a fix so finalize is
defered again.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-29 16:19:33 +05:30
Nicolas Vazquez dffb430975 kvm: Fix migrating VM from ISO failures (#2928)
Prevents errors while migrating VM from ISO:

Test 1: Deploy VM from ISO -> Live migrate VM to another host -> ERROR
Test 2: Register ISO using Direct Download on KVM -> Deploy VM from ISO -> Live migrate VM to another host -> ERROR

- Prevent NullPointerException migrating VM from ISO
- Prevent mount secondary storage on ISO direct downloads on KVM
2018-10-29 16:14:20 +05:30
Rohit Yadav f0491d5c72
vr: defer was broken in VR because of json name change (#2979)
After upgrade from CS 4.10 to CS 4.11, multiple VRs did not start through.
It did not properly defer the finalize config in update_config.py.
Apparently, the json files are now called differently: where it used to
be vm_dhcp_entry.json it now has a uuid added, for example
vm_metadata.json.4d727b6e-2b48-49df-81c3-b8532f3d6745.
The if statement that checks if the finalize can be safely deferred
therefore no longer matches. This PR contains a fix so finalize is
defered again.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-29 16:11:43 +05:30
Rohit Yadav e2ba934c19
server: fix unwanted txn commit warning messages (#2927)
This fixes unwanted transaction commit warning messages such:

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-29 02:49:54 +05:30
alexanderbazhenoff a87acf93d8 kvm: improved performance on creating VM (#2923)
Improved performance on creating VM for KVM virtualization.

On a huge hosts every "ifconfig | grep" takes a lot of time (about 2.5-3 minutes on hosts with 500 machines). For example: ip link show dev $vlanDev > /dev/null is faster than ifconfig |grep -w $vlanDev > /dev/null. But using ip command is much better. Using this patch you can create 500s machine in 10 seconds. You don't need slow ifconfig prints anymore.
2018-10-25 16:28:13 +05:30
Rohit Yadav 9cf57d2568
network: on rolling restart force stop old routers (#2926)
This force stops old VRs when performing rolling restart with
cleanup=true. This will ensure that VRs are powered off quickly than
wait longer for the normal ACPI shutdown. During testing, it was found
on VMware where VM stops are slow compared to XenServer and KVM.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-25 09:20:39 +05:30
Rohit Yadav 9b35b64b3c
packaging: install plugins at /usr/share/cloudstack-management/lib (#2915)
Install any additional plugin jars in the lib directory to be picked up
by the classpath builder, otherwise one has to manually add the jar
to /etc/default/cloudstack-management after installation. This fixes
the issue for `mysql-ha` plugin.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-24 18:21:03 +05:30
Rohit Yadav e092529c98
systemvm: Ensure cloud service reboots after failure (#2916)
This fixes an issue for systemvms (CPVM and SSVM) on VMware, as eth0
is not programmed (link-local) the networking.service fails to start
which is a dependency for cloud-postinit service. When cloud-postinit
service fails to start/run, it fails to start the agent (cloud) process.
This fixes the smoketest failures we saw in case of VMware 6.5 with
4.11.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-23 23:33:08 +05:30
Rohit Yadav 47c9c1cb58
client: mgmt server listen default to 0.0.0.0 (#2907)
This makes the management server listen on all interfaces by default.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-22 20:00:51 +05:30
Nicolas Vazquez 5cf163d888 server: Unify templates/ISOs checksum API output (#2911)
Unify checksum API output for templates and ISOs: not list the checksum algorithm on:
KVM direct downloads

On in progress normal template downloads. The algorithm is shown on the listtemplates API, but after it is downloaded it is not shown anymore.
2018-10-21 22:33:04 +05:30
Rohit Yadav 5ce14df31f
network: Allow ability to disable rolling restart feature (#2900)
This adds a global setting for admins who may not want the rolling
restart of routers or are seeing any issues around it. In future, this
setting may be removed.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-17 20:27:08 +05:30
Rohit Yadav 1904a70512
agent: on shutdown don't allow server reconnection (#2904)
When agent is stopped, don't allow reconnection. Previously this would
send a shutdown command to the management server which would put the
host state to Disconnected but then agent's reconnection logic may kick
in sometimes which would connect the agent to the management server
but then the agent process would terminate causing the host to be
put in Alert state (due to ping timeout or it waiting too long).

This fixes the issue by ensuring that when the agent is stopped, it
does not reconnect to the management server.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-17 06:31:13 +05:30
Nicolas Vazquez 9003c7bfdc Add checksum sanity validation on template registration (#2902)
* Add checksum sanity validation on template registration

* Refactor

* Rename checksum sanity method
2018-10-16 10:21:20 -03:00
Nicolas Vazquez 11d83fab43 agent: set log level to INFO as default for http wire (#2903)
Avoid logging bytes on direct download on KVM.
2018-10-16 10:32:03 +05:30
Rohit Yadav 933ee23104
vr: memory and swap optimizations (#2892)
This tries to provide a threshold based fix for #2873 where swappinness of VR is not used until last resort. By limiting swappiness unless actually needed, the VR system degradation can be avoided for most cases. The other change is around not starting baremetal-vr by default on all VRs, according to the spec https://cwiki.apache.org/confluence/display/CLOUDSTACK/Baremetal+Advanced+Networking+Support only vmware VRs need to run it and that too only as the last step of the setup/completion, so we don't need to run it all the time.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-16 10:29:48 +05:30
Rohit Yadav 63f4d852d5 PULL_REQUEST_TEMPLATE: simplify and remove unpopular sections (#2876)
This removes the section from the pull request template that is not very
popular or filled by the PR author.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-15 15:21:13 -03:00
Rohit Yadav ea771cfda4
router: Fixes #2719 program VR nics by device id order for VPC (#2888)
This fixes #2719 where private gateway IP might be incorrectly
programmed on a guest network nic. The VR would now check ipassoc
requests by mac addresses than provided nic/device id in case they are
wrong.

The root cause is that the device id information is lost when aggregated
commands are created upon starting of a new VPC VR, without the correct
device id in ip_associations json it mis-programs the VR.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-10 15:20:36 +05:30
Frank Maximus a6196b0a60 Fixes: #2881 Improve Exception message (#2889)
Network.Service and Network.Provider were missing a toString() method.
Added this so appending (a list of) them will be understandable.
2018-10-09 15:43:48 +05:30
Paul Angus 37ecfe2d28
Merge pull request #2884 from shapeblue/usage-server-timstamp
add date to usage server logs

Merged based on 2x LGTM and checking errors in smoke tests - none in any way related to the logging output change.
2018-10-08 15:20:51 -04:00
Rohit Yadav f430f41edd
ca: Fixes #2877 mgmt server cert should have all addrs of default nic (#2879)
This fixes the default RootCA provider implementation to initiate
and issue certificate for mgmt server on startup for all the IP addresses
on the default nic of that host.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-07 21:07:10 +05:30
Paul Angus 35656553ca add date to usage server logs 2018-10-06 17:20:17 +01:00
Simon Weller 5db65a6363 kvm: Fixes #2868 libvirt resize notify failure (#2878)
Incorrect diskpath information was being sent to virsh blockresize, so the block device size was never refreshed to reflect the new disk size.
Fixes #2868
2018-10-05 18:35:09 +05:30