Commit Graph

34046 Commits

Author SHA1 Message Date
alexanderbazhenoff a87acf93d8 kvm: improved performance on creating VM (#2923)
Improved performance on creating VM for KVM virtualization.

On a huge hosts every "ifconfig | grep" takes a lot of time (about 2.5-3 minutes on hosts with 500 machines). For example: ip link show dev $vlanDev > /dev/null is faster than ifconfig |grep -w $vlanDev > /dev/null. But using ip command is much better. Using this patch you can create 500s machine in 10 seconds. You don't need slow ifconfig prints anymore.
2018-10-25 16:28:13 +05:30
Rohit Yadav 9cf57d2568
network: on rolling restart force stop old routers (#2926)
This force stops old VRs when performing rolling restart with
cleanup=true. This will ensure that VRs are powered off quickly than
wait longer for the normal ACPI shutdown. During testing, it was found
on VMware where VM stops are slow compared to XenServer and KVM.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-25 09:20:39 +05:30
Rohit Yadav b8ed159f47 Merge remote-tracking branch 'origin/4.11' 2018-10-25 08:14:49 +05:30
Simon Weller c4b621a418 kvm: HyperV Enlightment for Improved Windows Server 2008+ Performance (#2870)
Windows has support for several paravirt features that it will use when running on Hyper-V, Microsoft's hypervisor. These features are called enlightenments. Many of the features are similar to paravirt functionality that exists with Linux on KVM (virtio, kvmclock, PV EOI, etc.)

Nowadays QEMU/KVM can also enable support for several Hyper-V enlightenments. When enabled, Windows VMs running on KVM will use many of the same paravirt optimizations they would use when running on Hyper-V.

A number of years ago, a PR was introduced that added a good portion of the code to enable this feature set, but it was never completed. This PR enables the existing features. The previous patch set detailed in #1013 also included the tests.

By selecting Windows PV, the enlightenment additions will be applied to the libvirt configuration. This is support on Windows Server 2008 and beyond, so all currently supported versions of Windows Server.

In our testing, we've seen benchmark improvements of around 20-25% running on Centos 7 hosts and it is also supported on Centos/RHEL 6.5 and later. Testing on Ubuntu would be appreciated.
2018-10-25 06:54:13 +05:30
Rohit Yadav 9b35b64b3c
packaging: install plugins at /usr/share/cloudstack-management/lib (#2915)
Install any additional plugin jars in the lib directory to be picked up
by the classpath builder, otherwise one has to manually add the jar
to /etc/default/cloudstack-management after installation. This fixes
the issue for `mysql-ha` plugin.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-24 18:21:03 +05:30
Abhishek 6f44a8d7ba template: Return requireshvm key as part of list template response (#2921)
Added requireshvm key in the response of list template API.
Fixes #2821

Signed-off-by: Abhishek Kumar <abhishek.mrt@gmail.com>
2018-10-24 16:20:27 +05:30
Rohit Yadav 8d31024a60 Merge remote-tracking branch 'origin/4.11' 2018-10-24 11:08:00 +05:30
Abhishek c8ca9e2fa0 template: create/updateTemplate should allow to set/change sshKeyEnabled (#2922)
Added sshKeyEnabled key in createTemplate and updateTemplate API.
Fixes #2822.

Signed-off-by: Abhishek Kumar <abhishek.mrt@gmail.com>
2018-10-24 11:07:03 +05:30
Rohit Yadav e092529c98
systemvm: Ensure cloud service reboots after failure (#2916)
This fixes an issue for systemvms (CPVM and SSVM) on VMware, as eth0
is not programmed (link-local) the networking.service fails to start
which is a dependency for cloud-postinit service. When cloud-postinit
service fails to start/run, it fails to start the agent (cloud) process.
This fixes the smoketest failures we saw in case of VMware 6.5 with
4.11.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-23 23:33:08 +05:30
Rafael Weingärtner 71e5a88fc1 Forward merge client: mgmt server listen default to 0.0.0.0 (#2907) 2018-10-22 16:32:06 -03:00
Abhishek d0fbd00e9b tools: Added '-B' flag for mvn current version cmd (#2910) (#2914)
maven command for finding current version might need to download packages and without batch mode it will end in variable. Added '-B' to enable batch mode.
2018-10-22 22:04:42 +05:30
Rohit Yadav 47c9c1cb58
client: mgmt server listen default to 0.0.0.0 (#2907)
This makes the management server listen on all interfaces by default.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-22 20:00:51 +05:30
Abhishek f5d52421b9 kvm: Remove libvirt-bin check for Debian (#2909) (#2912)
Debian does not have libvirt-bin package. Therefore, only for Ubuntu host distro process name libvirt-bin is used, otherwise libvirtd will be used
2018-10-22 17:48:11 +05:30
Nicolas Vazquez 5cf163d888 server: Unify templates/ISOs checksum API output (#2911)
Unify checksum API output for templates and ISOs: not list the checksum algorithm on:
KVM direct downloads

On in progress normal template downloads. The algorithm is shown on the listtemplates API, but after it is downloaded it is not shown anymore.
2018-10-21 22:33:04 +05:30
Rohit Yadav 233f46c94b Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-17 20:29:58 +05:30
Rohit Yadav 5ce14df31f
network: Allow ability to disable rolling restart feature (#2900)
This adds a global setting for admins who may not want the rolling
restart of routers or are seeing any issues around it. In future, this
setting may be removed.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-17 20:27:08 +05:30
Rohit Yadav e871638dc8 Merge remote-tracking branch 'origin/4.11' 2018-10-17 13:25:06 +05:30
Rohit Yadav 1904a70512
agent: on shutdown don't allow server reconnection (#2904)
When agent is stopped, don't allow reconnection. Previously this would
send a shutdown command to the management server which would put the
host state to Disconnected but then agent's reconnection logic may kick
in sometimes which would connect the agent to the management server
but then the agent process would terminate causing the host to be
put in Alert state (due to ping timeout or it waiting too long).

This fixes the issue by ensuring that when the agent is stopped, it
does not reconnect to the management server.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-17 06:31:13 +05:30
Gabriel Beims Bräscher 143fe5375c Allow enable the Java remote Debug for CloudStack Agent (#2897)
The idea is to create a property in /etc/defaults/cloudstack-agent that
can be enabled/disabled (uncommented/commented) to control the enabling
of Java remote debug in CloudStack's JVM.
2018-10-16 15:57:13 -03:00
Rafael Weingärtner 92f8cda64a Merge forward #2902 (Add checksum sanity validation on template registration) 2018-10-16 10:25:50 -03:00
Nicolas Vazquez 9003c7bfdc Add checksum sanity validation on template registration (#2902)
* Add checksum sanity validation on template registration

* Refactor

* Rename checksum sanity method
2018-10-16 10:21:20 -03:00
Pierre-Yves Ritschard 58b4e71b0e
params: allow signatureversion and expires without logging
This patch considers the new expires and signatureversion parameters
valid. Without this, all calls log when using the V3 signature scheme.
2018-10-16 09:01:46 +02:00
Rohit Yadav 84994c841f Merge remote-tracking branch 'origin/4.11' 2018-10-16 10:54:39 +05:30
Nicolas Vazquez 11d83fab43 agent: set log level to INFO as default for http wire (#2903)
Avoid logging bytes on direct download on KVM.
2018-10-16 10:32:03 +05:30
Rohit Yadav 933ee23104
vr: memory and swap optimizations (#2892)
This tries to provide a threshold based fix for #2873 where swappinness of VR is not used until last resort. By limiting swappiness unless actually needed, the VR system degradation can be avoided for most cases. The other change is around not starting baremetal-vr by default on all VRs, according to the spec https://cwiki.apache.org/confluence/display/CLOUDSTACK/Baremetal+Advanced+Networking+Support only vmware VRs need to run it and that too only as the last step of the setup/completion, so we don't need to run it all the time.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-16 10:29:48 +05:30
Rafael Weingärtner 272167c3b3 merge forward #2876 -- PULL_REQUEST_TEMPLATE simplification 2018-10-15 15:33:13 -03:00
Rohit Yadav 63f4d852d5 PULL_REQUEST_TEMPLATE: simplify and remove unpopular sections (#2876)
This removes the section from the pull request template that is not very
popular or filled by the PR author.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-15 15:21:13 -03:00
Rafael Weingärtner ebc6b2f706 Remove push force test file 2018-10-13 09:40:23 -03:00
Rafael Weingärtner 1758a75a54 test force push 2018-10-12 10:58:41 -03:00
Rohit Yadav bd9880003f Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-10 16:10:26 +05:30
Rohit Yadav ea771cfda4
router: Fixes #2719 program VR nics by device id order for VPC (#2888)
This fixes #2719 where private gateway IP might be incorrectly
programmed on a guest network nic. The VR would now check ipassoc
requests by mac addresses than provided nic/device id in case they are
wrong.

The root cause is that the device id information is lost when aggregated
commands are created upon starting of a new VPC VR, without the correct
device id in ip_associations json it mis-programs the VR.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-10 15:20:36 +05:30
Frank Maximus a6196b0a60 Fixes: #2881 Improve Exception message (#2889)
Network.Service and Network.Provider were missing a toString() method.
Added this so appending (a list of) them will be understandable.
2018-10-09 15:43:48 +05:30
Paul Angus e73fb765d2 ui: Display i18n language name in the same language (#2883)
This update changes how the languages are shown on the UI's login page. Previously all languages were shown in the current language of the login page. This meant for instance, that a Japanese speaker would have to know English to find their language. Likewise if the language had been changed, a German speaker might need to speak Korean in order to find their language.

Therefore the UI has been altered to always show the name of each language in the appropriate language.
2018-10-09 05:57:10 +05:30
Rohit Yadav 80debcb409 Merge remote-tracking branch 'origin/4.11' 2018-10-09 05:44:10 +05:30
Paul Angus 37ecfe2d28
Merge pull request #2884 from shapeblue/usage-server-timstamp
add date to usage server logs

Merged based on 2x LGTM and checking errors in smoke tests - none in any way related to the logging output change.
2018-10-08 15:20:51 -04:00
Rohit Yadav 1fa4f10a9e Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-07 21:12:20 +05:30
Rohit Yadav f430f41edd
ca: Fixes #2877 mgmt server cert should have all addrs of default nic (#2879)
This fixes the default RootCA provider implementation to initiate
and issue certificate for mgmt server on startup for all the IP addresses
on the default nic of that host.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-07 21:07:10 +05:30
Paul Angus 35656553ca add date to usage server logs 2018-10-06 17:20:17 +01:00
Rohit Yadav 5313c04b21 Merge remote-tracking branch 'origin/4.11' 2018-10-05 18:36:37 +05:30
Simon Weller 5db65a6363 kvm: Fixes #2868 libvirt resize notify failure (#2878)
Incorrect diskpath information was being sent to virsh blockresize, so the block device size was never refreshed to reflect the new disk size.
Fixes #2868
2018-10-05 18:35:09 +05:30
Rohit Yadav b6302d4e90 Merge remote-tracking branch 'origin/4.11'
Conflicts resolved for:
	engine/orchestration/src/org/apache/cloudstack/engine/orchestration/NetworkOrchestrator.java
	engine/schema/src/com/cloud/vm/dao/UserVmDaoImpl.java
server/src/com/cloud/network/element/VirtualRouterElement.java
server/src/com/cloud/vm/UserVmManagerImpl.java
tools/marvin/setup.py

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-05 05:18:42 +05:30
Rohit Yadav 0c943ab1f0
CertUtils: export private key to pem format correctly (#2875)
This makes openssl rsa -in <file> -check pass, due to "RSA" string the
validate of private key (pem file) by openssl fails. Also removes
a commented import.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-05 04:45:47 +05:30
Khosrow Moossavi 769a0085be tools: Remove deprecated buildw and wix plugin (#2849)
Fixes #2771
This PR removes deprecated and obsolete wix plugin and corresponding buildw to build Windows related artifacts.
2018-10-04 06:12:33 +05:30
René Moser 8c0b9d6202 systemvm: baremetal-vr: reduce memory usage (#2866)
We see a suspicious continuous increase in memory usage. Kind of looks like a memory leak.

One thing noted during debugging is that flask is started in debug mode. This is not best practice for a production system.
2018-10-03 16:38:32 +05:30
Paul Angus fe10e684f9
Merge pull request #2743 from nuagenetworks/bugfix/marvin_config_drive
CLOUDSTACK-10380: Fix startvm giving another pw after pw reset
2018-09-26 10:21:52 -04:00
Rohit Yadav c2f4b3653d
packaging: Fixes #2857 don't overwrite agent logrotate config (#2860)
This makes the agent logrotate config to `noreplace` so on upgrade
any changes to the file are not lost.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-09-25 11:25:06 -04:00
Rohit Yadav 1f9811db8d Merge remote-tracking branch 'origin/4.11' 2018-09-24 20:05:08 +05:30
Kui LIU d53fc94485 CLOUDSTACK-10365: Change the "getXXX" boolean method names to "isXXX" (#2847)
These boolean-return methods are named as "getXXX".
Other boolean-return methods are named as "isXXX".
Considering there methods will return boolean values, it should be more clear and consistent to rename them as "isXXX".
(rebase #2602 and #2816)
2018-09-22 17:20:48 +02:00
Rohit Yadav 6f1c5551fc
agent: Fixes #2858 agent LB not working (#2859)
This fixes the issue that on reconnection, agent LB feature will fail
and only the first ms-host will be tried reconnection again and again.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-09-22 14:40:18 +05:30
Frank Maximus cca25055fa Handle review comments 2018-09-21 14:01:35 +02:00