We do not need to build a tarball before building the DEB packages
Saves a few minutes on building DEB packages
Signed-off-by: Wido den Hollander <wido@widodh.nl>
MySQL 5.7 has a more strict SQL mode by default with which CloudStack
is not compatible.
By setting the SQL Mode to a more relaxed mode on run-time we can
run without changing any SQL server settings.
Admins could also apply this to the [mysqld] section of their my.cnf:
sql_mode = 'STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION'
Signed-off-by: Wido den Hollander <wido@widodh.nl>
This commit implements basic Security Grouping for KVM in
Basic Networking.
It does not implement full Security Grouping yet, but it does:
- Prevent IP-Address source spoofing
- Allow DHCPv6 clients, but disallow DHCPv6 servers
- Disallow Instances to send out Router Advertisements
The Security Grouping allows ICMPv6 packets as described by RFC4890
as they are essential for IPv6 connectivity.
Following RFC4890 it allows:
- Router Solicitations
- Router Advertisements (incoming only)
- Neighbor Advertisements
- Neighbor Solicitations
- Packet Too Big
- Time Exceeded
- Destination Unreachable
- Parameter Problem
- Echo Request
ICMPv6 is a essential part of IPv6, without it connectivity will break or be very
unreliable.
For now it allows any UDP and TCP packet to be send in to the Instance which
effectively opens up the firewall completely.
Future commits will implement Security Grouping further which allows controlling UDP and TCP
ports for IPv6 like can be done with IPv4.
Regardless of the egress filtering (which can't be done yet) it will always allow outbound DNS
to port 53 over UDP or TCP.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
- Switches Travis to use jdk1.8
- Changes java-version to 1.8
- Change jdk/maven version to 1.8
- Switch to F5/java8 compatible library release
- Switch packaging to use jdk 1.8, and jre 1.8 in init/systemd scripts
- Switch systemvm to openjdk-8-jre
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
While forward merging PR #1728, and resolving merge issues a semi-colon was
not added causing cloudstack-agent to fail to start. This fixes the
issue of running agent on centos7.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9551: Move java tmp dir to cloudstack-agent's path to avoidMove java tmp dir to cloudstack-agent's path to avoid noexec on /tmp
* pr/1728:
CLOUDSTACK-9551: Move java tmp dir to cloudstack-agent's path to avoid noexec on /tmp
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
We use some JSP file just for translation of strings in the UI. This is
achievable purely in JavaScript. This removes those JSPs, simplifies
translation usage and workflow (purely JS based). The l10n js (dictionary)
files are generated from existing messages.properties files during client-ui
code generation phase.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
[CLOUDSTACK-9444] Fix a little issue from PR1610 if the db.properties file hasn't EOL character at the end of file
And some improvements about the dir/file using variables
cc @wido @rhtyd
* pr/1621:
Fix a little issue from PR1610 if the db.properties file hasn't EOL character at the end of file And some improvements about the dir/file using variables
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Refactors and unifies usage of systemd script and default files across
CentOS and Ubuntu/Debian packaging system.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- systemd: Add a /etc/sysconfig/cloudstack-* file
This allows users to easily override variables passed to Java when
starting up.
It also creates a foundation for sharing the systemd service profile
between CentOS and Ubuntu since it only requires the environment file
to be changed.
- deb: Add Ubuntu 16.04 support
Ubuntu 16.04 differs from Ubuntu 14.04 in a few ways:
- systemd instead of sysvinit / upstart
- Java 8 support
The packaging now detects on which distribution it is being
build and based on that it installs different files in the
packages, but it also changes the Dependencies.
Packages for Ubuntu 16.04 will require Java 8 as a JRE
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
On fresh installation, the usage server fails to start if the `key` file does
not exist in its classpath. The issue is reproducible in environments where
the usage server is installed before cloudstack-setup-databases has been called.
Before the cloudstack db has been setup, the key file does not exist at its
default location and installation of usage-server fails to add a symlink to the
key file.
This fix adds a default symlink to `/etc/cloudstack/management/key` if a
symlink/file does not already exist in the /etc/cloudstack/usage directory.
On new installation, in the post-installation steps it checks if the symlink
or file exists, and adds a symlink if it does not exist. On existing
installations, if symlink or file exists then it will skip adding symlink.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The patchviasocket script was rewritten in Python from PR #1533 and made
assumptions that Python 2.7 would be available. In case of CentOS, python 2.7
may not be available or installed. This change ensures that python-argparse
is installed which is used by this script.
Expose cmd error in the logs when patch command fails.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This introduces two new cloudstack packages: marvin and integration-tests.
The two packages will make it easier for CI systems to install Marvin for a
specific cloudstack release/build and run integration tests that are specific
for that version/build.
- maven: add explicit juniper-contrail-api maven repository
- marvin: build source distribution for both install and package mvn phases
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Removes bundling of systemvm.zip in cloudstack-common rpms. This is not
done in debian packaging either there we remove for rpms as well, as this
file is not used by any subsystem but systemvm.iso is used.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
In case of rpms, the commands.properties file is bundled at
/usr/share/cloudstack-management/webapps/client/WEB-INF/classes/commands.properties
In case of a rpm upgrade, new rpms won't ship with commands.properties file. For
existing installations this copies the commands.properties file to
/etc/cloudstack/management
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-8818: Use MySQL native connector with PythonMySQLdb has been deprecated and is also not supported in Python 3.
mysql.connector is a connector written in Python which talks the
native MySQL protocol without any external code.
https://dev.mysql.com/doc/connector-python/en/
* pr/1054:
CLOUDSTACK-8818: Use MySQL native connector with Python
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Addresses CLOUDSTACK-9300 where the MySQL HA StaticStrategy class fai
* pr/1428:
Addresses CLOUDSTACK-9300 where the MySQL HA StaticStrategy class fails to load successfully
Signed-off-by: Will Stevens <williamstevens@gmail.com>
MySQLdb has been deprecated and is also not supported in Python 3.
mysql.connector is a connector written in Python which talks the
native MySQL protocol without any external code.
https://dev.mysql.com/doc/connector-python/en/
CLOUDSTACK-9305: Cloudstack Usage Breaks if DB HA enabledWith DB HA enabled in db.properties, the cloudstack-usage service restarts every 10 seconds. Making the suggested change has fixed it for me. Cloudstack 4.8 on Centos7
* pr/1433:
Cloudstack Usage Breaks if DB HA enabled
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CLOUDSTACK-9283: add pid to java arguments in cloudstack-usage.servicecloudstack-usage fails to start throwing Integer exception during PID retrieval, and the service keeps restarting after 10s (as defined in the systemd service definition).
Adding the pid to the java arguments in the systemd service definition makes it stop looping in centos7
* pr/1409:
CLOUDSTACK-9283: add pid to java arguments in systemd/cloudstack-usage.service
Signed-off-by: Will Stevens <williamstevens@gmail.com>
With DB HA enabled in db.properties, the cloudstack-usage service restarts every 10 seconds. Making the suggested change has fixed it for me. Cloudstack 4.8 on Centos7
Update cloudstack-usage.service
The default umask of 0022 is set in Ubuntu and other packages. Set the same
in case of CentOS startup scripts. Use umask 022 in the injectkeys.sh script
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* 4.6:
Revert "Change references of people.apache.org to home.apache.org in the test code"
Change references of people.apache.org to home.apache.org in the test code This closes#1123 Signed-off-by: SrikanteswaraRao Talluri <talluri@apache.org>
CLOUDSTACK-9077 Fix injectkeys.sh to work on CentOS7
CLOUDSTACK-9065: fix bug when creating packaging with noredist flag
* 4.6:
CLOUDSTACK-9052 Shuffling the password to avoid having a subset of characters in fixed positions.
Refactor package.sh: * lint * adjust exit codes (1 for usage, 2 for maven, 3 for rpmbuild) * variable naming consistency * add option for package release version * revise synopsis and usage
The S3 implementation is far from finished, this commit focusses on the bases.
- Upgrade AWS SDK to latest version.
- Rewrite S3 Template downloader.
- Rewrite S3Utils utility class.
- Improve addImageStoreS3 API command.
- Split various classes for convenience.
- Various minor improvements and code optimalisations.
A side effect of the new AWS SDK is that it, by default, uses the V4 signature. Therefore I added an option to specify the Signer, so it stays compatible with previous versions.
CLOUDSTACK-9049: fix Centos7 with Tomcat7 packaging and python libsCLOUDSTACK-9049: Fix Centos7 with Tomcat7 packaging and python libs
* pr/1052:
CLOUDSTACK-9049: fix Centos7 with Tomcat7 packaging and python libs * adjust library to support tomcat7 config files * adjust centos7 spec to use tomcat7 config files from deployment * add option to use tomcat7 files in management server setup
reorder content to match original
Signed-off-by: Remi Bergsma <github@remi.nl>
* adjust library to support tomcat7 config files
* adjust centos7 spec to use tomcat7 config files from deployment
* add option to use tomcat7 files in management server setup
CLOUDSTACK-8812 / CLOUDSTACK-9010: adjust packaging for centos7
Here are a few adjustments for the packaging in centos7. With these changes I was able to start the service. Please review. Thank you.
* pr/1008:
CLOUDSTACK-9010: adjust packaging for centos7
This closes#888
Signed-off-by: Remi Bergsma <github@remi.nl>
* Adjust systemd service to match tomcat7 startup,
and change service type to simple.
* Adjust sysconfig to only have one JAVA_OPTS
due to behaviour change in tomcat7.
* Adjust spec to remove some config files
from WEB-INF since they are placed in /etc.
This is a similar behaviour to the centos6 spec.
CLOUDSTACK-8840: Systemd service for the Usage ServerThere already was a uncompleted systemd service file for the Usage
Server.
This new one replaces sysvinit and the old systemd service file.
* pr/820:
CLOUDSTACK-8840: Do not include old systemd wrapper
CLOUDSTACK-8840: Fix the source path of the service file
CLOUDSTACK-8840: Systemd service for the Usage Server
Signed-off-by: Wido den Hollander <wido@widodh.nl>
With CentOS 7 and Ubuntu 16.04 (to be released) using systemd
it is preferred that CloudStack's Agent is also being started using
systemd.
This commit includes a service file for the CloudStack Agent with
a wrapper script which actually executes Java
It no longer uses jsvc for daemonizing and thus this requirement
has also been dropped for CentOS 7 packaging.
The Agent log output to stdout has also been modified to no longer
include the timestamp as this is done by journalctl.
This has been tested on a CentOS 7.1 machine and the Agent starts,
stops and restarts properly.
Normal users would require /sbin in their paths to access lsmod, this adds
a profile.d script that adds /sbin, for centos7, f20 and f21 packages.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This allows non-root users to add KVM hosts, the user should be an admin or
added to sudoers to execute sudo cloudstack-setup-agent.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Signed-off-by: Remi Bergsma <apache@remi.nl>
This closes#288
(cherry picked from commit d2b0c1a32b)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Removes awsapi packaging rules for debian, centos63, centos7, fedora 20/21
- Removes catalina port 7080 service configs
- Fixes build replace properties for AWSAPILOG
- Removes maven profile for building awsapi and deploying db in developer profile
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Following actions from discussions on dev ML regarding removing awsapi and
prefering ec2stack. Reference from last PR:
https://github.com/apache/cloudstack/pull/44
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-7460: mgmt server package should not create agent directory
Revert "CLOUDSTACK-8402: Depend on openjdk 1.7 for both CentOS 6 and 7"
Revert "CLOUDSTACK-8404: uninstall/conflict if java-1.8.0-openjdk is installed"
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This would force to uninstall openjdk 1.8.0 and only install 1.7 in case of ACS
4.5.x releases. On master/4.6, we might support java 1.8.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This commit forces rpms to depend on java-1.7.0-openjdk which is available
on both CentOS 6 and CentOS 7, also the version that ACS 4.5 supports.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Depend on java-1.7.0-openjdk for EL7 rpms as agent fails with openjdk8. This
fix needs to be reverted/removed to use openjdk8 once we start supporting it.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
rpmbuild complained about the date "Fri Oct 03 2012"
according to the calendar, Oct 3 2012 was Wednesday
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
The initial commit (f96c65416a) missed part of the change to package.sh, so we were not actually passing through the simulator build option to the rpmbuild call. This patch completes the support.
(cherry picked from commit e717450e0e)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The -Xms value specifies the minimum heap size the JVM should start with and
-Xmx is the maximum heap size it can grow. The previous fix imposed minimum
limit of 1G which is unreasonably for small deployments. The fix is to start
with 256MB and limit to 2G for cloudstack-agent process. This was tested on
DevCloud/KVM and then again on a ACS/KVM deployment on real hardware.
With these values, it's possible for the agent to work in a DevCloud/KVM
environment and if JVM needs it can increase the heap size to 2G. The fix also
ports these settings to Debian cloud-agent init.d script as well.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit bb81082e58)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The -Xms value specifies the minimum heap size the JVM should start with and
-Xmx is the maximum heap size it can grow. The previous fix imposed minimum
limit of 1G which is unreasonably for small deployments. The fix is to start
with 256MB and limit to 2G for cloudstack-agent process. This was tested on
DevCloud/KVM and then again on a ACS/KVM deployment on real hardware.
With these values, it's possible for the agent to work in a DevCloud/KVM
environment and if JVM needs it can increase the heap size to 2G. The fix also
ports these settings to Debian cloud-agent init.d script as well.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit bb81082e58)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The initial commit (f96c65416a) missed part of the change to package.sh, so we were not actually passing through the simulator build option to the rpmbuild call. This patch completes the support.
In init.d scripts, the LSB header may specify what kind of service is
provided by an init script. If spaces are used, this means the init
script is providing several boot facilities. We fix that by using an
hyphen.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 2401eb927b)
In init.d scripts, the LSB header may specify what kind of service is
provided by an init script. If spaces are used, this means the init
script is providing several boot facilities. We fix that by using an
hyphen.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
changed the order of preference to check for java first.
Usage server rpm installs JRE 1.7. In the case where JDK 1.6 is already
installed, java version would be 1.7 but, javac would be 1.6
If javac is given preference, usage server fails to start in this case.
On a secured environment (selinux w/ env_reset enabled in sudoers), the
runuser command that is invoked by the daemon() function does not pass
along environment variables, so $JAVA_HOME is empty, and JSVC falls
back to its default behavior, which may not find java or may not find
the intended java.
This fix simply passes $JAVA_HOME explicitly using the -home argument to
JSVC.
Signed-off-by: Rajani Karuturi <rajanikaruturi@gmail.com>
Since we've agreed to use JDK/JRE 1.7, this enforces that for Ubuntu builds
- this fix remove usage of 1.6 paths in JDK_DIR for cloud-{agent, management, usage}.
- adds oracle jdk 1.7 path (in case a user is using that)
- adds mysql-connector-java path to CLASSPATH for usage server
- adds libmysql-java pkg dependency (tested and available for precise and trusty)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 96d6a2a037)
Conflicts:
packaging/debian/init/cloud-usage
Adds pessimistic logic to try the hard coded paths if Rajani's logic fails
Replacing whatami with $0 which is how UNIX shell scripts should get the
script's name.
BUG-ID: CLOUDSTACK-6129
Bugfix-for:
Reviewed-by:
Reported-by:
Signed-off-by: John Kinsella <jlk@stratosec.co> 1392660036 -0800
We now require at least Java 7 to build and run CloudStack.
Both the DEB and RPM packaging now also require Java 7 during installation
of the packages.
All of the code changes for the Spring Modularization will work in
a modularized context or a non-modularized context. This commit
is the final commit to turn modularization on. Revert this commit
to disable the modularization and go back to monolithic Spring
configuration.
Including following steps:
b. Run "cloudstack-agent-upgrade". This script will upgrade all the existing bridge name to new bridge name, and update related firewall rules.
c. install a libvirt hook:
c1. mkdir /etc/libvirt/hooks
c2. cp /usr/share/cloudstack-agent/lib/libvirtqemuhook /etc/libvirt/hooks/qemu
c3. chmod +x /etc/libvirt/hooks/qemu
c4. service libvirtd restart
(cherry picked from commit a0988780ad)
Signed-off-by: Wei Zhou <w.zhou@leaseweb.com>
[upgrade][2.2.13 -> 2.2.14 -> 4.2][KVM] When we try to upgrade the KVM agent from 2.2.14 to 4.2 using the "U" option in install.sh script, management server also gets installed!
[upgrade][2.2.13 -> 2.2.14 -> 4.2][KVM] When we try to upgrade the KVM agent from 2.2.14 to 4.2 using the "U" option in install.sh script, management server also gets installed!
Retains the systevm.iso from the previous run causing systemvm.iso to
never update itself. Do a complete clean install for all profiles
specified in mvn.
Signed-off-by: Prasanna Santhanam <tsp@apache.org>
Wido den Hollander
Wei Zhou
Abhinandan Prateek
Rajani Karuturi
Rene Moser
Rohit Yadav
Abhinandan Prateek
Milamber
Nick Livens
Paul Angus
Will Stevens
Slair1
Simon Weller
David Amorim Faria
Remi Bergsma
David Amorim Faria
Boris Schrijver
Sudhansu
sanjeev
Rafael da Fonseca
pdion891
Daan Hoogland
Daan Hoogland
Satoru Nakaya
Rajani Karuturi
Harikrishna Patnala
Star Guo
Laszlo Hornyak
Daan Hoogland
Alex Brett
Hugo Trippaers
Keiichi Yusa
Damodar
rayeesn
Nitin Mehta
Vincent Bernat
Leo Simons
Frank.Zhang
David Bierce
ynojima
Rajesh Battala
Marcus Sorensen
John Kinsella
Alex Hitchins
Wei Zhou
Darren Shepherd
Kishan Kavala
Saksham Srivastava
Wido den Hollander
Edison Su
Rene Diepstraten
David Nalley
Hiroaki KAWAI
Prasanna Santhanam