Merge pull request #1547 from PCextreme/vrouter-fixes

Fixes for VirtualRouters in Basic Networking, especially with mutliple ranges in VLANsDuring the last few modifications on the SystemVM scripts, it turns out quite a lot of stuff broke in our setups.

This PR fixes a number of things:
* Multiple IP's per VLAN interface are now supported & working again, including DNS, DHCP ranges, password and metadata services
* `useextdns` fixed (I had a small merge conflict with an attempt to fix this at 4.7, but these fixes are more comprehensive)
*  CLOUDSTACK-8303
* Apache configs better in line with best-practices and distro-expected locations
* Added a few more helper functions & getters & setters for utility
* some minor cleanup & fixes

* pr/1547:
  Remove /etc/apache2/sites-enabled/000-default in cloud-early-config
  SysVM various fixes to previous refactorings * make CORS include a regular glob-matched one * fix NameVirtualHost in CsApp.py as well * even moar cleanups
  SysVM: Cleanup and removal of old (and dangerous) config files * ports.conf * default & default-ssl sites * SSL config in httpd.conf * deprecated & dead setup_redundant_router in cloud-early-config
  SysVM cloud-early-config: Intermediate fix for SecStore & CORS * Take setup from vhost.template rather than default(-ssl)   * should move into Python CS code as well * Move CORS setup to separate conf * Modify vhost template to Optionally include the cors file * Add NameVirtualHost to vhost template for feature parity with ports.conf * Take setup from vhost.template rather than default(-ssl)
  VR cloud-early-config: Commonize Apache2 common setup
  VR cloud-early-config: Fix Apache2 alias cleanup
  VR: consistent SSL setup, vhost is not an example, but a template
  VR CsConfig: reintroduce old get_dns() behaviour for redundant non-VPC's
  VR CsAddress fixes: * cleanup imports, * fix to_str(), * improve & fix service post_config logic * don't arpPing when there's no gateway
  VR CsApp: Expose config to classes, move vhost confs to proper location, allow for multiple IP's per intf, sanitize servername, don't open port 53 if no DNS is foreseen
  VR CsConfig: Add is_router(), is_dns(), has_dns(), has_metadata(), use_extdns(), fix get_dns() with use_extdns()
  VR CsDhcp: allow multiple ranges & finite lease time (fixes CLOUDSTACK-8303)
  VR CsGuestNetwork obey useextdns
  VR merge.py ipalias fix & dhcpconfig stub notification

Signed-off-by: Will Stevens <williamstevens@gmail.com>
This commit is contained in:
Will Stevens 2016-06-28 11:28:06 -04:00
commit e4ba640a28
17 changed files with 190 additions and 436 deletions

View File

@ -221,10 +221,7 @@ Within the patches/systemvm/debian/config/etc/apache2 directory
Copyright (c) 2012 The Apache Software Foundation
from The Apache Software Foundation http://www.apache.org/
httpd.conf
ports.conf
sites-available/default
sites-available/default-ssl
vhostexample.conf
vhost.template
Within the patches/systemvm/debian/config/etc/ssh/ directory
licensed under the BSD (2-clause) http://www.opensource.org/licenses/BSD-2-Clause (as follows)

View File

@ -892,10 +892,7 @@
<exclude>systemvm/patches/debian/systemvm.vmx</exclude>
<exclude>systemvm/patches/debian/config/root/.ssh/authorized_keys</exclude>
<exclude>systemvm/patches/debian/config/etc/apache2/httpd.conf</exclude>
<exclude>systemvm/patches/debian/config/etc/apache2/ports.conf</exclude>
<exclude>systemvm/patches/debian/config/etc/apache2/sites-available/default</exclude>
<exclude>systemvm/patches/debian/config/etc/apache2/sites-available/default-ssl</exclude>
<exclude>systemvm/patches/debian/config/etc/apache2/vhostexample.conf</exclude>
<exclude>systemvm/patches/debian/config/etc/apache2/vhost.template</exclude>
<exclude>systemvm/patches/debian/config/etc/dnsmasq.conf.tmpl</exclude>
<exclude>systemvm/patches/debian/config/etc/vpcdnsmasq.conf</exclude>
<exclude>systemvm/patches/debian/config/etc/ssh/sshd_config</exclude>

View File

@ -1,3 +1 @@
SSLProtocol all -SSLv2 -SSLv3
SSLCipherSuite ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA
SSLHonorCipherOrder on
# Empty

View File

@ -1,23 +0,0 @@
# If you just change the port or add more ports here, you will likely also
# have to change the VirtualHost statement in
# /etc/apache2/sites-enabled/000-default
# This is also true if you have upgraded from before 2.2.9-3 (i.e. from
# Debian etch). See /usr/share/doc/apache2.2-common/NEWS.Debian.gz and
# README.Debian.gz
NameVirtualHost 10.1.1.1:80
Listen 10.1.1.1:80
<IfModule mod_ssl.c>
# If you add NameVirtualHost *:443 here, you will also have to change
# the VirtualHost statement in /etc/apache2/sites-available/default-ssl
# to <VirtualHost *:443>
# Server Name Indication for SSL named virtual hosts is currently not
# supported by MSIE on Windows XP.
Listen 10.1.1.1:443
</IfModule>
<IfModule mod_gnutls.c>
Listen 10.1.1.1:443
</IfModule>

View File

@ -1,41 +0,0 @@
<VirtualHost 10.1.1.1:80>
ServerAdmin webmaster@localhost
DocumentRoot /var/www/html
<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>
<Directory /var/www/html>
Options Indexes FollowSymLinks MultiViews
AllowOverride All
Order allow,deny
allow from all
</Directory>
ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
<Directory "/usr/lib/cgi-bin">
AllowOverride None
Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
Order allow,deny
Allow from all
</Directory>
ErrorLog ${APACHE_LOG_DIR}/error.log
# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn
CustomLog ${APACHE_LOG_DIR}/access.log combined
Alias /doc/ "/usr/share/doc/"
<Directory "/usr/share/doc/">
Options Indexes MultiViews FollowSymLinks
AllowOverride None
Order deny,allow
Deny from all
Allow from 127.0.0.0/255.0.0.0 ::1/128
</Directory>
</VirtualHost>

View File

@ -1,175 +0,0 @@
<IfModule mod_ssl.c>
<VirtualHost 10.1.1.1:443>
ServerAdmin webmaster@localhost
DocumentRoot /var/www/html
<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>
<Directory /var/www/html>
Options Indexes FollowSymLinks MultiViews
AllowOverride all
Order allow,deny
allow from all
</Directory>
ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
<Directory "/usr/lib/cgi-bin">
AllowOverride None
Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
Order allow,deny
Allow from all
</Directory>
ErrorLog ${APACHE_LOG_DIR}/error.log
# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn
CustomLog ${APACHE_LOG_DIR}/ssl_access.log combined
Alias /doc/ "/usr/share/doc/"
<Directory "/usr/share/doc/">
Options Indexes MultiViews FollowSymLinks
AllowOverride None
Order deny,allow
Deny from all
Allow from 127.0.0.0/255.0.0.0 ::1/128
</Directory>
# SSL Engine Switch:
# Enable/Disable SSL for this virtual host.
SSLEngine on
SSLProtocol all -SSLv2 -SSLv3
SSLCipherSuite ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA
SSLHonorCipherOrder on
# A self-signed (snakeoil) certificate can be created by installing
# the ssl-cert package. See
# /usr/share/doc/apache2.2-common/README.Debian.gz for more info.
# If both key and certificate are stored in the same file, only the
# SSLCertificateFile directive is needed.
SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem
SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
# Server Certificate Chain:
# Point SSLCertificateChainFile at a file containing the
# concatenation of PEM encoded CA certificates which form the
# certificate chain for the server certificate. Alternatively
# the referenced file can be the same as SSLCertificateFile
# when the CA certificates are directly appended to the server
# certificate for convinience.
#SSLCertificateChainFile /etc/apache2/ssl.crt/server-ca.crt
# Certificate Authority (CA):
# Set the CA certificate verification path where to find CA
# certificates for client authentication or alternatively one
# huge file containing all of them (file must be PEM encoded)
# Note: Inside SSLCACertificatePath you need hash symlinks
# to point to the certificate files. Use the provided
# Makefile to update the hash symlinks after changes.
#SSLCACertificatePath /etc/ssl/certs/
#SSLCACertificateFile /etc/apache2/ssl.crt/ca-bundle.crt
# Certificate Revocation Lists (CRL):
# Set the CA revocation path where to find CA CRLs for client
# authentication or alternatively one huge file containing all
# of them (file must be PEM encoded)
# Note: Inside SSLCARevocationPath you need hash symlinks
# to point to the certificate files. Use the provided
# Makefile to update the hash symlinks after changes.
#SSLCARevocationPath /etc/apache2/ssl.crl/
#SSLCARevocationFile /etc/apache2/ssl.crl/ca-bundle.crl
# Client Authentication (Type):
# Client certificate verification type and depth. Types are
# none, optional, require and optional_no_ca. Depth is a
# number which specifies how deeply to verify the certificate
# issuer chain before deciding the certificate is not valid.
#SSLVerifyClient require
#SSLVerifyDepth 10
# Access Control:
# With SSLRequire you can do per-directory access control based
# on arbitrary complex boolean expressions containing server
# variable checks and other lookup directives. The syntax is a
# mixture between C and Perl. See the mod_ssl documentation
# for more details.
#<Location />
#SSLRequire ( %{SSL_CIPHER} !~ m/^(EXP|NULL)/ \
# and %{SSL_CLIENT_S_DN_O} eq "Snake Oil, Ltd." \
# and %{SSL_CLIENT_S_DN_OU} in {"Staff", "CA", "Dev"} \
# and %{TIME_WDAY} >= 1 and %{TIME_WDAY} <= 5 \
# and %{TIME_HOUR} >= 8 and %{TIME_HOUR} <= 20 ) \
# or %{REMOTE_ADDR} =~ m/^192\.76\.162\.[0-9]+$/
#</Location>
# SSL Engine Options:
# Set various options for the SSL engine.
# o FakeBasicAuth:
# Translate the client X.509 into a Basic Authorisation. This means that
# the standard Auth/DBMAuth methods can be used for access control. The
# user name is the `one line' version of the client's X.509 certificate.
# Note that no password is obtained from the user. Every entry in the user
# file needs this password: `xxj31ZMTZzkVA'.
# o ExportCertData:
# This exports two additional environment variables: SSL_CLIENT_CERT and
# SSL_SERVER_CERT. These contain the PEM-encoded certificates of the
# server (always existing) and the client (only existing when client
# authentication is used). This can be used to import the certificates
# into CGI scripts.
# o StdEnvVars:
# This exports the standard SSL/TLS related `SSL_*' environment variables.
# Per default this exportation is switched off for performance reasons,
# because the extraction step is an expensive operation and is usually
# useless for serving static content. So one usually enables the
# exportation for CGI and SSI requests only.
# o StrictRequire:
# This denies access when "SSLRequireSSL" or "SSLRequire" applied even
# under a "Satisfy any" situation, i.e. when it applies access is denied
# and no other module can change it.
# o OptRenegotiate:
# This enables optimized SSL connection renegotiation handling when SSL
# directives are used in per-directory context.
#SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire
<FilesMatch "\.(cgi|shtml|phtml|php)$">
SSLOptions +StdEnvVars
</FilesMatch>
<Directory /usr/lib/cgi-bin>
SSLOptions +StdEnvVars
</Directory>
# SSL Protocol Adjustments:
# The safe and default but still SSL/TLS standard compliant shutdown
# approach is that mod_ssl sends the close notify alert but doesn't wait for
# the close notify alert from client. When you need a different shutdown
# approach you can use one of the following variables:
# o ssl-unclean-shutdown:
# This forces an unclean shutdown when the connection is closed, i.e. no
# SSL close notify alert is send or allowed to received. This violates
# the SSL/TLS standard but is needed for some brain-dead browsers. Use
# this when you receive I/O errors because of the standard approach where
# mod_ssl sends the close notify alert.
# o ssl-accurate-shutdown:
# This forces an accurate shutdown when the connection is closed, i.e. a
# SSL close notify alert is send and mod_ssl waits for the close notify
# alert of the client. This is 100% SSL/TLS standard compliant, but in
# practice often causes hanging connections with brain-dead browsers. Use
# this only for browsers where you know that their SSL implementation
# works correctly.
# Notice: Most problems of broken clients are also related to the HTTP
# keep-alive facility, so you usually additionally want to disable
# keep-alive for those clients, too. Use variable "nokeepalive" for this.
# Similarly, one has to force some clients to use HTTP/1.0 to workaround
# their broken HTTP/1.1 implementation. Use variables "downgrade-1.0" and
# "force-response-1.0" for this.
BrowserMatch "MSIE [2-6]" \
nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0
# MSIE 7 and newer should be able to use keepalive
BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown
</VirtualHost>
</IfModule>

View File

@ -83,10 +83,15 @@
Allow from 127.0.0.0/255.0.0.0 ::1/128
</Directory>
# Include CORS configuration **IF SET**
Include /etc/apache2/[cC][oO][rR][sS].conf
# SSL Engine Switch:
# Enable/Disable SSL for this virtual host.
SSLEngine on
SSLProtocol all -SSLv2 -SSLv3
SSLCipherSuite ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA
SSLHonorCipherOrder on
# A self-signed (snakeoil) certificate can be created by installing
# the ssl-cert package. See
@ -223,6 +228,7 @@
# README.Debian.gz
Listen 10.1.1.1:80
NameVirtualHost 10.1.1.1:80
<IfModule mod_ssl.c>
# If you add NameVirtualHost *:443 here, you will also have to change

View File

@ -342,6 +342,7 @@ cat << "EOF" > /usr/local/cloud/systemvm/conf/temp.xml
EOF
mv /usr/local/cloud/systemvm/conf/temp.xml /usr/local/cloud/systemvm/conf/log4j-cloud.xml
}
setup_interface() {
local intfnum=$1
local ip=$2
@ -801,38 +802,33 @@ setup_sshd(){
setup_vpc_apache2() {
log_it "Setting up apache web server for VPC"
chkconfig apache2 off
rm -f /etc/apache2/conf.d/vhost*.conf
[ -f /etc/apache2/sites-available/default ] && echo "" >/etc/apache2/sites-available/default
[ -f /etc/apache2/sites-available/default-ssl ] && echo "">/etc/apache2/sites-available/default-ssl
[ -f /etc/apache2/ports.conf ] && echo "">/etc/apache2/ports.conf
[ -f /etc/apache2/ports.conf ] && echo "">/etc/apache2/ports.conf
[ -f /etc/apache2/ports.conf ] && echo "">/etc/apache2/ports.conf
[ -f /etc/apache2/conf.d/security ] && sed -i -e "s/^ServerTokens .*/ServerTokens Prod/g" /etc/apache2/conf.d/security
[ -f /etc/apache2/conf.d/security ] && sed -i -e "s/^ServerSignature .*/ServerSignature Off/g" /etc/apache2/conf.d/security
# Disable listing of http://SSVM-IP/icons folder for security issue. see article http://www.i-lateral.com/tutorials/disabling-the-icons-folder-on-an-ubuntu-web-server/
[ -f /etc/apache2/mods-available/alias.conf ] && sed -i s/"Options Indexes MultiViews"/"Options -Indexes MultiViews"/ /etc/apache2/mods-available/alias.conf
echo "Options -Indexes" > /var/www/html/.htaccess
clean_ipalias_config
setup_apache2_common
}
clean_ipalias_config() {
rm -f /etc/apache2/conf.d/ports.*.meta-data.conf
rm -f /etc/apache2/sites-available/ipAlias*
rm -f /etc/apache2/sites-enabled/ipAlias*
rm -rf /etc/failure_config
# Old
rm -f /etc/apache2/conf.d/ports.*.meta-data.conf
rm -f /etc/apache2/sites-available/ipAlias*
rm -f /etc/apache2/sites-enabled/ipAlias*
rm -f /etc/apache2/conf.d/vhost*.conf
rm -f /etc/apache2/ports.conf
rm -f /etc/apache2/vhostexample.conf
rm -f /etc/apache2/sites-available/default
rm -f /etc/apache2/sites-available/default-ssl
rm -f /etc/apache2/sites-enabled/default
rm -f /etc/apache2/sites-enabled/default-ssl
# New
rm -f /etc/apache2/sites-enabled/vhost-*.conf
rm -f /etc/apache2/sites-enabled/000-default
rm -rf /etc/failure_config
}
setup_apache2() {
clean_ipalias_config
log_it "Setting up apache web server"
local ip=$1
[ -f /etc/apache2/sites-available/default ] && sed -i -e "s/<VirtualHost.*>/<VirtualHost $ip:80>/" /etc/apache2/sites-available/default
[ -f /etc/apache2/sites-available/default-ssl ] && sed -i -e "s/<VirtualHost.*>/<VirtualHost $ip:443>/" /etc/apache2/sites-available/default-ssl
[ -f /etc/apache2/ports.conf ] && sed -i -e "s/Listen .*:80/Listen $ip:80/g" /etc/apache2/ports.conf
[ -f /etc/apache2/ports.conf ] && sed -i -e "s/Listen .*:443/Listen $ip:443/g" /etc/apache2/ports.conf
[ -f /etc/apache2/ports.conf ] && sed -i -e "s/NameVirtualHost .*:80/NameVirtualHost $ip:80/g" /etc/apache2/ports.conf
setup_apache2_common() {
sed -i 's/^Include ports.conf.*/# CS: Done by Python CsApp config\n#Include ports.conf/g' /etc/apache2/apache2.conf
[ -f /etc/apache2/conf.d/security ] && sed -i -e "s/^ServerTokens .*/ServerTokens Prod/g" /etc/apache2/conf.d/security
[ -f /etc/apache2/conf.d/security ] && sed -i -e "s/^ServerSignature .*/ServerSignature Off/g" /etc/apache2/conf.d/security
@ -842,72 +838,18 @@ setup_apache2() {
echo "Options -Indexes" > /var/www/html/.htaccess
}
setup_redundant_router() {
rrouter_bin_path="/ramdisk/rrouter"
rrouter_log="/ramdisk/rrouter/keepalived.log"
rrouter_bin_path_str="\/ramdisk\/rrouter"
rrouter_log_str="\/ramdisk\/rrouter\/keepalived.log"
mkdir -p /ramdisk
mount tmpfs /ramdisk -t tmpfs
mkdir -p /ramdisk/rrouter
ip route delete default
cp /root/redundant_router/keepalived.conf.templ /etc/keepalived/keepalived.conf
cp /root/redundant_router/conntrackd.conf.templ /etc/conntrackd/conntrackd.conf
cp /root/redundant_router/enable_pubip.sh.templ $rrouter_bin_path/enable_pubip.sh
cp /root/redundant_router/master.sh.templ $rrouter_bin_path/master.sh
cp /root/redundant_router/backup.sh.templ $rrouter_bin_path/backup.sh
cp /root/redundant_router/fault.sh.templ $rrouter_bin_path/fault.sh
cp /root/redundant_router/primary-backup.sh.templ $rrouter_bin_path/primary-backup.sh
cp /root/redundant_router/heartbeat.sh.templ $rrouter_bin_path/heartbeat.sh
cp /root/redundant_router/check_heartbeat.sh.templ $rrouter_bin_path/check_heartbeat.sh
cp /root/redundant_router/arping_gateways.sh.templ $rrouter_bin_path/arping_gateways.sh
cp /root/redundant_router/check_bumpup.sh $rrouter_bin_path/
cp /root/redundant_router/disable_pubip.sh $rrouter_bin_path/
cp /root/redundant_router/checkrouter.sh.templ /opt/cloud/bin/checkrouter.sh
cp /root/redundant_router/services.sh $rrouter_bin_path/
sed -i "s/\[ROUTER_ID\]/$NAME/g" /etc/keepalived/keepalived.conf
sed -i "s/\[ROUTER_IP\]/$GUEST_GW\/$GUEST_CIDR_SIZE/g" /etc/keepalived/keepalived.conf
sed -i "s/\[BOARDCAST\]/$GUEST_BRD/g" /etc/keepalived/keepalived.conf
sed -i "s/\[PRIORITY\]/$ROUTER_PR/g" /etc/keepalived/keepalived.conf
sed -i "s/\[RROUTER_BIN_PATH\]/$rrouter_bin_path_str/g" /etc/keepalived/keepalived.conf
sed -i "s/\[DELTA\]/2/g" /etc/keepalived/keepalived.conf
sed -i "s/\[LINK_IF\]/eth0/g" /etc/conntrackd/conntrackd.conf
sed -i "s/\[LINK_IP\]/$ETH0_IP/g" /etc/conntrackd/conntrackd.conf
sed -i "s/\[IGNORE_IP1\]/$GUEST_GW/g" /etc/conntrackd/conntrackd.conf
sed -i "s/\[IGNORE_IP2\]/$ETH0_IP/g" /etc/conntrackd/conntrackd.conf
sed -i "s/\[IGNORE_IP3\]/$ETH1_IP/g" /etc/conntrackd/conntrackd.conf
sed -i "s/\[ETH2IP\]/$ETH2_IP/g" $rrouter_bin_path/enable_pubip.sh
sed -i "s/\[ETH2MASK\]/$ETH2_MASK/g" $rrouter_bin_path/enable_pubip.sh
sed -i "s/\[GATEWAY\]/$GW/g" $rrouter_bin_path/enable_pubip.sh
sed -i "s/\[GATEWAY\]/$GW/g" $rrouter_bin_path/master.sh
setup_apache2() {
log_it "Setting up apache web server"
clean_ipalias_config
setup_apache2_common
local ip=$1
sed -i "s/\[RROUTER_BIN_PATH\]/$rrouter_bin_path_str/g" $rrouter_bin_path/master.sh
sed -i "s/\[RROUTER_BIN_PATH\]/$rrouter_bin_path_str/g" $rrouter_bin_path/backup.sh
sed -i "s/\[RROUTER_BIN_PATH\]/$rrouter_bin_path_str/g" $rrouter_bin_path/fault.sh
sed -i "s/\[RROUTER_BIN_PATH\]/$rrouter_bin_path_str/g" $rrouter_bin_path/heartbeat.sh
sed -i "s/\[RROUTER_BIN_PATH\]/$rrouter_bin_path_str/g" $rrouter_bin_path/check_heartbeat.sh
sed -i "s/\[RROUTER_LOG\]/$rrouter_log_str/g" $rrouter_bin_path/master.sh
sed -i "s/\[RROUTER_LOG\]/$rrouter_log_str/g" $rrouter_bin_path/backup.sh
sed -i "s/\[RROUTER_LOG\]/$rrouter_log_str/g" $rrouter_bin_path/fault.sh
sed -i "s/\[RROUTER_LOG\]/$rrouter_log_str/g" $rrouter_bin_path/primary-backup.sh
sed -i "s/\[RROUTER_LOG\]/$rrouter_log_str/g" $rrouter_bin_path/check_heartbeat.sh
sed -i "s/\[RROUTER_LOG\]/$rrouter_log_str/g" $rrouter_bin_path/arping_gateways.sh
sed -i "s/\[RROUTER_LOG\]/$rrouter_log_str/g" /opt/cloud/bin/checkrouter.sh
if [ $ADVERT_INT ]
then
sed -i "s/advert_int 1/advert_int $ADVERT_INT/g" /etc/keepalived/keepalived.conf
fi
chmod a+x $rrouter_bin_path/*.sh
sed -i "s/--exec\ \$DAEMON;/--exec\ \$DAEMON\ --\ --vrrp;/g" /etc/init.d/keepalived
crontab -l|grep "check_heartbeat.sh"
if [ $? -ne 0 ]
then
(crontab -l; echo -e "SHELL=/bin/bash\nPATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin\n*/1 * * * * $rrouter_bin_path/check_heartbeat.sh 2>&1 > /dev/null") | crontab
fi
load_modules
# Deprecated, functionality moved to Cs Python code
# [ -f /etc/apache2/sites-available/default ] && sed -i -e "s/<VirtualHost.*>/<VirtualHost $ip:80>/" /etc/apache2/sites-available/default
# [ -f /etc/apache2/sites-available/default-ssl ] && sed -i -e "s/<VirtualHost.*>/<VirtualHost $ip:443>/" /etc/apache2/sites-available/default-ssl
# [ -f /etc/apache2/ports.conf ] && sed -i -e "s/Listen .*:80/Listen $ip:80/g" /etc/apache2/ports.conf
# [ -f /etc/apache2/ports.conf ] && sed -i -e "s/Listen .*:443/Listen $ip:443/g" /etc/apache2/ports.conf
# [ -f /etc/apache2/ports.conf ] && sed -i -e "s/NameVirtualHost .*:80/NameVirtualHost $ip:80/g" /etc/apache2/ports.conf
}
setup_aesni() {
@ -1207,33 +1149,27 @@ setup_secstorage() {
fi
setup_apache2 $ETH2_IP
# Deprecated, should move to Cs Python all of it
sed -e "s/<VirtualHost .*:80>/<VirtualHost $ETH2_IP:80>/" \
-e "s/<VirtualHost .*:443>/<VirtualHost $ETH2_IP:443>/" \
-e "s/Listen .*:80/Listen $ETH2_IP:80/g" \
-e "s/Listen .*:443/Listen $ETH2_IP:443/g" \
-e "s/NameVirtualHost .*:80/NameVirtualHost $ETH2_IP:80/g" /etc/apache2/vhost.template > /etc/apache2/sites-enabled/vhost-${ETH2_IP}.conf
log_it "setting up apache2 for post upload of volume/template"
a2enmod proxy
a2enmod proxy_http
a2enmod headers
SSL_FILE="/etc/apache2/sites-available/default-ssl"
PATTERN="RewriteRule ^\/upload\/(.*)"
CORS_PATTERN="Header set Access-Control-Allow-Origin"
if [ -f $SSL_FILE ]; then
if grep -q "$PATTERN" $SSL_FILE ; then
log_it "rewrite rules already exist in file $SSL_FILE"
else
log_it "adding rewrite rules to file: $SSL_FILE"
sed -i -e "s/<\/VirtualHost>/RewriteEngine On \n&/" $SSL_FILE
sed -i -e "s/<\/VirtualHost>/RewriteCond %{HTTPS} =on \n&/" $SSL_FILE
sed -i -e "s/<\/VirtualHost>/RewriteCond %{REQUEST_METHOD} =POST \n&/" $SSL_FILE
sed -i -e "s/<\/VirtualHost>/RewriteRule ^\/upload\/(.*) http:\/\/127.0.0.1:8210\/upload?uuid=\$1 [P,L] \n&/" $SSL_FILE
fi
if grep -q "$CORS_PATTERN" $SSL_FILE ; then
log_it "cors rules already exist in file $SSL_FILE"
else
log_it "adding cors rules to file: $SSL_FILE"
sed -i -e "s/<\/VirtualHost>/Header always set Access-Control-Allow-Origin \"*\" \n&/" $SSL_FILE
sed -i -e "s/<\/VirtualHost>/Header always set Access-Control-Allow-Methods \"POST, OPTIONS\" \n&/" $SSL_FILE
sed -i -e "s/<\/VirtualHost>/Header always set Access-Control-Allow-Headers \"x-requested-with, Content-Type, origin, authorization, accept, client-security-token, x-signature, x-metadata, x-expires\" \n&/" $SSL_FILE
fi
fi
cat >/etc/apache2/cors.conf <<CORS
RewriteEngine On
RewriteCond %{HTTPS} =on
RewriteCond %{REQUEST_METHOD} =POST
RewriteRule ^/upload/(.*) http://127.0.0.1:8210/upload?uuid=\$1 [P,L]
Header always set Access-Control-Allow-Origin "*"
Header always set Access-Control-Allow-Methods "POST, OPTIONS"
Header always set Access-Control-Allow-Headers "x-requested-with, Content-Type, origin, authorization, accept, client-security-token, x-signature, x-metadata, x-expires"
CORS
service apache2 restart

View File

@ -15,15 +15,13 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
from CsDatabag import CsDataBag, CsCmdLine
from CsApp import CsApache, CsDnsmasq, CsPasswdSvc
import CsHelper
import logging
from netaddr import IPAddress, IPNetwork
import CsHelper
import subprocess
import time
import CsHelper
from CsDatabag import CsDataBag
from CsApp import CsApache, CsDnsmasq, CsPasswdSvc
from CsRoute import CsRoute
from CsRule import CsRule
@ -198,7 +196,7 @@ class CsInterface:
return self.get_attr("add")
def to_str(self):
pprint(self.address)
return self.address
class CsDevice:
@ -293,9 +291,10 @@ class CsIP:
interfaces = [CsInterface(address, self.config)]
CsHelper.reconfigure_interfaces(self.cl, interfaces)
self.set_mark()
self.arpPing()
if 'gateway' in self.address:
self.arpPing()
CsRpsrfs(self.dev).enable()
self.post_config_change("add")
@ -514,16 +513,19 @@ class CsIP:
self.fw_vpcrouter()
# On deletion nw_type will no longer be known
if self.get_type() in ["guest"] and self.config.is_vpc():
if self.get_type() in ('guest'):
if self.config.is_vpc() or self.config.is_router():
CsDevice(self.dev, self.config).configure_rp()
logging.error(
"Not able to setup source-nat for a regular router yet")
CsDevice(self.dev, self.config).configure_rp()
if self.config.has_dns() or self.config.is_dhcp():
dns = CsDnsmasq(self)
dns.add_firewall_rules()
logging.error(
"Not able to setup source-nat for a regular router yet")
dns = CsDnsmasq(self)
dns.add_firewall_rules()
app = CsApache(self)
app.setup()
if self.config.has_metadata():
app = CsApache(self)
app.setup()
cmdline = self.config.cmdline()
# If redundant then this is dealt with by the master backup functions
@ -685,3 +687,4 @@ class CsRpsrfs:
if count < 2:
logging.debug("Single CPU machine")
return count

View File

@ -28,35 +28,42 @@ class CsApp:
self.ip = ip.get_ip_address()
self.type = ip.get_type()
self.fw = ip.fw
self.config = ip.config
class CsApache(CsApp):
""" Set up Apache """
def remove(self):
file = "/etc/apache2/conf.d/vhost%s.conf" % self.dev
file = "/etc/apache2/sites-enabled/vhost-%s.conf" % self.dev
if os.path.isfile(file):
os.remove(file)
CsHelper.service("apache2", "restart")
def setup(self):
CsHelper.copy_if_needed("/etc/apache2/vhostexample.conf",
"/etc/apache2/conf.d/vhost%s.conf" % self.dev)
CsHelper.copy_if_needed("/etc/apache2/vhost.template",
"/etc/apache2/sites-enabled/vhost-%s.conf" % self.ip)
file = CsFile("/etc/apache2/conf.d/vhost%s.conf" % (self.dev))
file.search("<VirtualHost.*:80>", "\t<VirtualHost %s:80>" % (self.ip))
file = CsFile("/etc/apache2/sites-enabled/vhost-%s.conf" % (self.ip))
file.search("<VirtualHost.*:80>", "\t<VirtualHost %s:80>" % (self.ip))
file.search("<VirtualHost.*:443>", "\t<VirtualHost %s:443>" % (self.ip))
file.search("Listen .*:80", "Listen %s:80" % (self.ip))
file.search("Listen .*:443", "Listen %s:443" % (self.ip))
file.search("ServerName.*", "\tServerName vhost%s.cloudinternal.com" % (self.dev))
file.search("NameVirtualHost .*:80", "NameVirtualHost %s:80" % (self.ip))
file.search("ServerName.*", "\tServerName %s.%s" % (self.config.cl.get_type(), self.config.get_domain()))
if file.is_changed():
file.commit()
CsHelper.service("apache2", "restart")
self.fw.append(["", "front",
"-A INPUT -i %s -d %s/32 -p tcp -m tcp -m state --state NEW --dport 80 -j ACCEPT" % (self.dev, self.ip)
])
self.fw.append([
"", "front",
"-A INPUT -i %s -d %s/32 -p tcp -m tcp -m state --state NEW --dport 80 -j ACCEPT" % (self.dev, self.ip)
])
self.fw.append([
"", "front",
"-A INPUT -i %s -d %s/32 -p tcp -m tcp -m state --state NEW --dport 443 -j ACCEPT" % (self.dev, self.ip)
])
class CsPasswdSvc():
@ -94,10 +101,13 @@ class CsDnsmasq(CsApp):
"-A INPUT -i %s -p udp -m udp --dport 67 -j ACCEPT" % self.dev
])
self.fw.append(["", "front",
"-A INPUT -i %s -d %s/32 -p udp -m udp --dport 53 -j ACCEPT" % (self.dev, self.ip)
])
if self.config.has_dns():
self.fw.append([
"", "front",
"-A INPUT -i %s -d %s/32 -p udp -m udp --dport 53 -j ACCEPT" % (self.dev, self.ip)
])
self.fw.append(["", "front",
"-A INPUT -i %s -d %s/32 -p tcp -m tcp --dport 53 -j ACCEPT" % (self.dev, self.ip)
])
self.fw.append([
"", "front",
"-A INPUT -i %s -d %s/32 -p tcp -m tcp --dport 53 -j ACCEPT" % (self.dev, self.ip)
])

View File

@ -58,25 +58,38 @@ class CsConfig(object):
return self.__LOG_LEVEL
def is_vpc(self):
return self.cl.get_type() == "vpcrouter"
return self.cl.get_type() == 'vpcrouter'
def is_router(self):
return self.cl.get_type() == "router"
return self.cl.get_type() == 'router'
def is_dhcp(self):
return self.cl.get_type() == 'dhcpsrvr'
def has_dns(self):
return not self.use_extdns()
def has_metadata(self):
return any((self.is_vpc(), self.is_router(), self.is_dhcp()))
def get_domain(self):
return self.cl.get_domain()
def use_extdns(self):
return self.cmdline().idata().get('useextdns', 'false') == 'true'
def get_dns(self):
conf = self.cmdline().idata()
dns = []
if not self.cl.get_use_ext_dns():
if not self.is_vpc() and self.cl.is_redundant():
if not self.use_extdns():
if not self.is_vpc() and self.cl.is_redundant() and self.cl.get_guest_gw():
dns.append(self.cl.get_guest_gw())
else:
dns.append(self.address().get_guest_ip())
names = ["dns1", "dns2"]
for name in names:
if name in self.cmdline().idata():
dns.append(self.cmdline().idata()[name])
for name in ('dns1', 'dns2'):
if name in conf:
dns.append(conf[name])
return dns
def get_format(self):

View File

@ -17,6 +17,7 @@
import CsHelper
import logging
from netaddr import *
from random import randint
from CsGuestNetwork import CsGuestNetwork
from cs.CsDatabag import CsDataBag
from cs.CsFile import CsFile
@ -44,7 +45,7 @@ class CsDhcp(CsDataBag):
continue
self.add(self.dbag[item])
self.write_hosts()
if self.cloud.is_changed():
self.delete_leases()
@ -59,23 +60,24 @@ class CsDhcp(CsDataBag):
def configure_server(self):
# self.conf.addeq("dhcp-hostsfile=%s" % DHCP_HOSTS)
idx = 0
for i in self.devinfo:
if not i['dnsmasq']:
continue
device = i['dev']
ip = i['ip'].split('/')[0]
sline = "dhcp-range=interface:%s,set:interface" % (device)
line = "dhcp-range=interface:%s,set:interface-%s,%s,static" % (device, device, ip)
sline = "dhcp-range=interface:%s,set:interface-%s-%s" % (device, device, idx)
line = "dhcp-range=interface:%s,set:interface-%s-%s,%s,static" % (device, device, idx, ip)
self.conf.search(sline, line)
gn = CsGuestNetwork(device, self.config)
sline = "dhcp-option=tag:interface-%s,15" % device
line = "dhcp-option=tag:interface-%s,15,%s" % (device, gn.get_domain())
sline = "dhcp-option=tag:interface-%s-%s,15" % (device, idx)
line = "dhcp-option=tag:interface-%s-%s,15,%s" % (device, idx, gn.get_domain())
self.conf.search(sline, line)
# DNS search order
if gn.get_dns() and device:
sline = "dhcp-option=tag:interface-%s,6" % device
sline = "dhcp-option=tag:interface-%s-%s,6" % (device, idx)
dns_list = [x for x in gn.get_dns() if x is not None]
line = "dhcp-option=tag:interface-%s,6,%s" % (device, ','.join(dns_list))
line = "dhcp-option=tag:interface-%s-%s,6,%s" % (device, idx, ','.join(dns_list))
self.conf.search(sline, line)
# Gateway
gateway = ''
@ -83,8 +85,8 @@ class CsDhcp(CsDataBag):
gateway = gn.get_gateway()
else:
gateway = i['gateway']
sline = "dhcp-option=tag:interface-%s,3," % device
line = "dhcp-option=tag:interface-%s,3,%s" % (device, gateway)
sline = "dhcp-option=tag:interface-%s-%s,3," % (device, idx)
line = "dhcp-option=tag:interface-%s-%s,3,%s" % (device, idx, gateway)
self.conf.search(sline, line)
# Netmask
netmask = ''
@ -92,9 +94,10 @@ class CsDhcp(CsDataBag):
netmask = gn.get_netmask()
else:
netmask = self.config.address().get_guest_netmask()
sline = "dhcp-option=tag:interface-%s,1," % device
line = "dhcp-option=tag:interface-%s,1,%s" % (device, netmask)
sline = "dhcp-option=tag:interface-%s-%s,1," % (device, idx)
line = "dhcp-option=tag:interface-%s-%s,1,%s" % (device, idx, netmask)
self.conf.search(sline, line)
idx += 1
def delete_leases(self):
try:
@ -104,7 +107,7 @@ class CsDhcp(CsDataBag):
def preseed(self):
self.add_host("127.0.0.1", "localhost")
self.add_host("::1", "localhost ip6-localhost ip6-loopback")
self.add_host("::1", "localhost ip6-localhost ip6-loopback")
self.add_host("ff02::1", "ip6-allnodes")
self.add_host("ff02::2", "ip6-allrouters")
if self.config.is_vpc():
@ -125,9 +128,15 @@ class CsDhcp(CsDataBag):
def add(self, entry):
self.add_host(entry['ipv4_adress'], entry['host_name'])
self.cloud.add("%s,%s,%s,infinite" % (entry['mac_address'],
entry['ipv4_adress'],
entry['host_name']))
# lease time boils down to once a month
# with a splay of 60 hours to prevent storms
lease = randint(700, 760)
self.cloud.add("%s,%s,%s,%sh" % (entry['mac_address'],
entry['ipv4_adress'],
entry['host_name'],
lease
))
i = IPAddress(entry['ipv4_adress'])
# Calculate the device
for v in self.devinfo:

View File

@ -38,13 +38,15 @@ class CsGuestNetwork:
def get_dns(self):
if not self.guest:
return self.config.get_dns()
# Can a router provide dhcp but not dns?
if 'dns' in self.data and 'router_guest_gateway' in self.data:
return [self.data['router_guest_gateway']] + self.data['dns'].split(',')
elif "router_guest_gateway" in self.data:
return [self.data['router_guest_gateway']]
else:
return [""]
dns = []
if not self.config.use_extdns() and 'router_guest_gateway' in self.data:
dns.append(self.data['router_guest_gateway'])
if 'dns' in self.data:
dns.extend(self.data['dns'].split(','))
return dns or ['']
def set_dns(self, val):
self.data['dns'] = val

View File

@ -129,6 +129,13 @@ class updateDataBag:
dbag = self.process_vpnusers(self.db.getDataBag())
elif self.qFile.type == 'staticroutes':
dbag = self.process_staticroutes(self.db.getDataBag())
elif self.qFile.type == 'ipaliases':
self.db.setKey('ips')
self.db.load()
dbag = self.process_ipaliases(self.db.getDataBag())
elif self.qFile.type == 'dhcpconfig':
logging.error("I don't think I need %s anymore", self.qFile.type)
return
else:
logging.error("Error I do not know what to do with file of type %s", self.qFile.type)
return
@ -231,6 +238,30 @@ class updateDataBag:
cs_vmdata.merge(dbag, self.qFile.data)
return dbag
def process_ipaliases(self, dbag):
nic_dev = None
# Should be a way to deal with this better
for intf, data in dbag.items():
if intf == 'id':
continue
elif any([net['nw_type'] == 'guest' for net in data]):
nic_dev = intf
break
assert nic_dev is not None, 'Unable to determine Guest interface'
nic_dev_id = nic_dev[3:]
for alias in self.qFile.data['aliases']:
ip = {
'add': not alias['revoke'],
'nw_type': 'guest',
'public_ip': alias['ip_address'],
'netmask': alias['netmask'],
'nic_dev_id': nic_dev_id
}
dbag = cs_ip.merge(dbag, ip)
return dbag
class QueueFile:

View File

@ -2771,10 +2771,7 @@ Within the patches/systemvm/debian/config/etc/apache2 directory
Copyright (c) 2012 The Apache Software Foundation
from The Apache Software Foundation http://www.apache.org/
httpd.conf
ports.conf
sites-available/default
sites-available/default-ssl
vhostexample.conf
vhost.template
Within the patches/systemvm/debian/config/etc/ssh/ directory
licensed under the BSD (2-clause) http://www.opensource.org/licenses/BSD-2-Clause (as follows)

View File

@ -2459,10 +2459,7 @@ Copyright (c) 2012 The Apache Software Foundation
</copyright-notice>
<by-organisation id='apache.org.2'>
<resource name='httpd.conf' />
<resource name='ports.conf' />
<resource name='vhostexample.conf' />
<resource name='sites-available/default' />
<resource name='sites-available/default-ssl' />
<resource name='vhost.template' />
</by-organisation>
</with-license>
</within>

View File

@ -2443,10 +2443,7 @@ Copyright (c) 2012 The Apache Software Foundation
</copyright-notice>
<by-organisation id='apache.org.2'>
<resource name='httpd.conf' />
<resource name='ports.conf' />
<resource name='vhostexample.conf' />
<resource name='sites-available/default' />
<resource name='sites-available/default-ssl' />
<resource name='vhost.template' />
</by-organisation>
</with-license>
</within>