CLOUDSTACK-8710: Fixed applying iptables rules for s2s vpn
@remibergsma @wilderrodrigues
Moved applying iptables rules apply after vpn configuration so that vpn specific rules also get applied
* pr/690:
CLOUDSTACK-8710: Fixed applying iptables rules for s2s vpn
This closes#690
Signed-off-by: Remi Bergsma <github@remi.nl>
Logging before:
2015-08-12 16:30:07,126 Searching for 192.168.23.6 and replacing with 192.168.23.6 192.168.23.5: PSK "preSharedKey"
Logging after:
2015-08-12 16:30:07,126 Searching for 192.168.23.6 and replacing with 192.168.23.6 192.168.23.5: PSK "****"
- For package installation/update please refer to: cloud-tools/appliance/definitions/systemvmtemplate/install_systemvm_packages.sh
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
This closes#587
- preempt delay reverted on version 1.2.13 - from the backports
- vrrp : Revert "Honor preempt_delay setting on startup.".
- See changelog: http://www.keepalived.org/changelog.html
- Refactoring some variable names to avoid misunderstanding
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
- We use no preempt mode with state set as EQUAL to both nodes, no need to have Priotities setup
- Do not add IPs as comments to the configuration. If a new guest interface is added, the file will change anyway.
- This was used in the past when keepalived would restart for each new interface added
- Removed the long sleep form the tests: we now sleep 5 seconds per PF rule added
CLOUDSTACK-8616 - Fix keepalived.ts/2 files comparison
- Add call to set_fault() in case of router transits to that state
- Removing commented out code
CLOUDSTACK-8616 - Fixing check_heartbeat.sh.templ
CLOUDSTACK-8616 - Call set_fault from the check_heartbeat.sh script
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
- After configuration save the ipdated in files
* /etc/iptables/router_rules.v4 and /etc/iptables/router_rules.v6
* Reload the configuration on reboot via the /etc/rc.local using iptables-restore
In 6ac06e5e5e logrotate was changed to run hourly.
Some logrotate configs still have set `daily` only which results in logs not
rotated hourly. The only way to ensure the log is rotated is to use size.
This closes#162
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Pub IP port forwarding and static NAT fixed for single VPCs
- Pub IP port forwarding fixed for redundant VPCs
[wip] fix static NAT for redundant VPCs
This closes#150
- With the changes added by the rVPC work, the bump priority became deprecated.
This commit includes a refactor to get it removed from the following resources:
* Java classes
* domain_router table - removing the is_priority_bumpup column
* Fixing unit tests
All changes were tested with:
XenServer 6.2 running under our VMWare zone
CloudStack Management Server running on MacBook Pro
MySql running on MackBook Pro
Storage Type: Local
This is a known issue in irqbalance 1.0.3 and was partially fixed in 1.0.4.
Using the package 1.0.6 from wheezy backports helped on many VRs balancing the
interrupts, but not on all.
It seems only 1.0.7 fixes all issues regarding this, also see thread in user ML
http://mail-archives.apache.org/mod_mbox/cloudstack-users/201503.mbox/%3C5508540E.4090302%40renemoser.net%3E
This closes#119
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit a45f0e29e2)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The logic is same as passwd_server_ip script which runs password server on all
IPs on eth0 interface.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 294503a025)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Fixed failure on using eth10 (which CS now supports)
Refactored CSForward to us CsInterface object instead of teh way it was kludged together
Fixed hex conversion for device numbers and iptables marks
- If for some reason the cmdLine json doesn't contain the password key, which is almost impossible to happen,
we generate a password based on other unique data per VPC
Customise the messages showed during VPC restart based on redundancy
Do not show "Make Redundant" option if VPC is already redundant
Show on VPC details if a VPC is redundant
De Morgans laws chaps
The negation of a conjunction is the disjunction of the negations.
The negation of a disjunction is the conjunction of the negations.
Fix the router state. do not show UNKNOW, but MASTER or BACKUP depending on the type of router
Implement the virtual_router_id to be passed as a boot parameter to the router
- it is needed for the keepalived configuration
Adding kill_all() method on the CsProcess.py file (not being used yet)
Changing a bit the logic in the merge.py
The changes in the logic might be reverted, but they are causing no harm now.
Fix: RTNETLINK errors
- Management Server health check trying to create already existing interface
- Changes on update_config.py, cs_guestnetwork.py, merger.py
Fix: replace RRouTER_LOG in the CsRedundant.py per log file location
Fix: Guest Net address association during Router restart
- Changes on NicProfileHelper, NicProfileHelperImpl
Fix: aggregationExecution() method on VirtualNetworkApplianceManagerImpl
- Do not send an AggregationControlCommand to a non-configured router
Some classes have been formatted.
Fixing assign IPv4 on GuestNetworkGuru.java
Fixing getRouters() on VpcVirtualRouterElement.java
Fixing Fixing paths on keepalived.conf.templ
Refactor on the other files
* removed the "is redundant" flag form the addVpcRouterToGuestNetwork() method
* removed the "is redundant" flag from the removeVpcRouterFromGuestNetwork() method
* changed the path of the master.py file in the keepalived.conf.temp file
* the call to routerDao.addRouterToGuestNetwork() in the VpcRouterDeploymentDefinition is not needed. That step will be performed once a VM is created
- In addition, when restarting a VPC the routers will have the guest net configured, if any exists.
* Pushing the POM.xml as well, to use the old Jetty for now. Could not fix the logging problem. Will replace the POM with master version after VPC is done.
firewallrules implemented sperately from networkacl (ugly but best that can be done ATM)
Some refactoring in CsDHCP as it made no sense to do it that way anymore
processmonitoring implemented
Domain not correctly sourced in VR
Some refactoring to make class loading a little more logical
Removed fw global (does not persist accross modules)
Added first tests
Added command line call to set and disable redundancy
Added command line call to set master
The dhcp server will only start listening after a host has been added
In this condition, HUP is not enough
Changed so that if it is adding a first host it will restart the server
The problem is that teh convergence model does not works as it currently is
Firewall rules will get added but not deleted
This builds a table of firewall rules that need to be present
The new CsNetfilter class will then do:
1. Add rules that are not present
2. Delete any that were not explicitly in the add list
WIP
cloud-early-config now saves its input file to /var/cache/cloud
Execute load cmd_line.json in update_config
Old way of working still in there to stop and possible clash until the base image is made
Also found condition inw hich apache would be miscobfigured and failed to run (I love tests!!)
Fixed configure.py to cover this case
Added a test to provoke this case!
Fixed tests to use /var/cache/cloud
Added some test files but will remove them when tests are properly completed
Fixed a bug in configure that did not deal well with databags with empty dev sections
Added the last pieces of vpc guest network
password server and associated FW rules
Added apache2 config and start of server
Corrected a bug in CsFile
Fixed some of the many pyLint bleats
Replace chef with a python script
configure.py will read the bags and (hopefully) create the desired state
At this stage this is ipassociation
This code should work for both VR and VPCrs
TODO:
iptables
ip route throw (present in VR but not in VPCr
Determine default route
Unit tests
----
Author: Ian Southam <isoutham@schubergphilis.com>
First commit towards moving systemvm to chef based configuration
In this commit
1. cmdline json databag is created
2. ip association data bag is created
3. Basic chef cookbook to manage ips and routes
Conflicts:
systemvm/patches/debian/config/etc/init.d/cloud-early-config
systemvm/patches/debian/config/var/chef/cookbooks/README
tools/appliance/definitions/systemvm64template/postinstall.sh
----
Because we've refactored the systemvm template the change to
postinstall.sh now gets its own chef.sh file.
- VRs are single CPU, so Threading based implementation favoured than Forking based
- Implements a Python based password server that does not use file based locks
- Saving password mechanism is provided by using secure token only to VR (localhost)
- Old serve_password implementation is removed
- Runs with Python 2.6+ with no external dependencies
- Locks used within threads for extra safety
This closes#106
(cherry picked from commit 4b45d25152)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* No offline mirrors
* No out of date mirrors
* New mirrors are used
* Load-balancing
* Faster downloads (when APT is used, via parallel downloads)
* Great on mobility
Also see http://http.debian.net/ for more information.
This closes#103
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
removing nio server as it is currently handling only https connections
and the parsing logic is also specific to agent communication.
current limitation of httpcomponents server is that the entire file is
read in memory. need figure out how to read it in chunks and send it
through a inputstreamreader to save on secondary storage.
When adding a VM, it adds an entry to /etc/hosts file on the VR but does not
clear up any older entries for the VM with a same name. The fix uncomments the
command that removes any old entries in the VM.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 63298d9b74)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Added destination and source definition. Flag -S can be used
to ignore this. It's the new default as it is more secure
and does not impact the way things work (backwords compatible).
If connecting the VPN takes some time, for example because
the other end is not (yet) up, CloudStack will delete
the VPN because the ipsectunnel.sh does not return in time.
The VPN connection then enters the Error state.
This change makes sure ipsectunnel.sh returns in time,
and lets ipsec connect in the background. If it all fails,
the connection enters Disconnected.
Changed default to no, as the other side may not be up yet.
If this check fails, the VPN enters Error state and will not
work. It's safe to just let it connect on its own so it will
connect when it can.
Changed 'auto=add' to 'auto=start' to make sure the tunnel starts.
When both sides are there they will connect. This resolves the
issue that there is only a small time frame in which the VPN
would connect.
Biglock breaks creating VPN's when other scripts run at the
same time that also use the same biglock. These other scripts
do nothing that could harm our deployment and even multiple
vpn's can safely be created simultaniously.
On default iptables rules are updated to add ACCEPT egress traffic.
If the network egress default policy is false, CS remove ACCEPT and adds the DROP rule which
is egress default rule when there are no other egress rules.
If the CS network egress default policy is true, CS won't configure any default rule for egress because
router already came up to accept egress traffic. If there are already egress rules for network then the
egress rules get applied on VR.
For isolated network with out firewall service, VR default allows egress traffic (guestnetwork --> public network)
This reverts commit 83656a6ea8.
The systemvm/routers will reboot automatically if the systemvm.iso changes.
They will stuck during the startup due to this commit because the virtio-port are empty.
The booting sequence result in change of IPv6 related sysctl options was
overrided by sysctl.conf which is loaded later.
So this patch would patch sysctl.conf in VR as well, ensure IPv6 would be
enabled during booting period otherwise the network setup may not work, result
in IPv6 VM deployment failure.
The booting sequence result in change of IPv6 related sysctl options was
overrided by sysctl.conf which is loaded later.
So this patch would patch sysctl.conf in VR as well, ensure IPv6 would be
enabled during booting period otherwise the network setup may not work, result
in IPv6 VM deployment failure.
The old way would disconnect all the existing connections through haproxy when
reload the config.
This new way would ensure that all the existing connections would still alive
after reload the config.
The old way would disconnect all the existing connections through haproxy when
reload the config.
This new way would ensure that all the existing connections would still alive
after reload the config.
Just prefer TLS over SSL in apache configuration in systemvm
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 88acc9bd53)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
OSX always declaims it's behind NAT no matter it's true or not, thus result in
confusion of openswan.
Add parameter "forceencaps=yes" to openswan to make sure non NAT VPN connection
from OSX can pass through.
OSX always declaims it's behind NAT no matter it's true or not, thus result in
confusion of openswan.
Add parameter "forceencaps=yes" to openswan to make sure non NAT VPN connection
from OSX can pass through.
This fix is to correct the JP keyboard mapping for VMs with windows and centOS GUI
and CLI OS on VMware hypervisor. Also fixed some known issues on centOS CLI on XS
hypervisor. Fix is not causing any regression.
Sometime in VR ntpd would move time backward to keep sync with NTP server, which
can result in false alarm of keepalived monitering process.
This patch adds 3 strikes for keepalived process dead detection to avoid falsely
shutdown keepalived process due to time adjustment for only once.
Made changes so that uploading custom certificate works for ssvm.
1. Reboot ssvm only when private key is passed meaning the server cert is passed. This is because while uploading the server cert is the last to be uploaded. And we want to propagate the entire chain once uploading is done.
2. Change the SecStorageSetupCommand sent to ssvm so that it also carries the root cert apart from having the chain and the server cert and key.
3. Change ssvm agent code to be able to configure root cert to the java key store.
4. Change ssvm configure ssl script to insert the chain certs correctly.
5. Fix order of chain certificates for apache webserver in SSVM
6. Remove double encoding and decoding for uploadCustomCertificate API from UI and server code respectively, so that API call without UI works fine
7. Java 1.7 - disable using SNI since copyTemplate doesnt work for SSL.
In some network environment, 1*3 seconds by default make RvR setup too
sensitive. A configurable parameter would be better for fitting different
network environments.
Moving default transport for console proxy, SSVM to http.
See
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Realhost+IP+changes
for more info.
jlk ported Amogh's patch for 4.3 to master - code base is different
enough that patch has multiple issues.
Author: Amogh Vasekar <Amogh Vasekar <amogh.vasekar@citrix.com>
Signed-off-by: John Kinsella <jlk@stratosec.co> 1394398017 -0700
The orignal "dhcp-range=xxx,static" would only prohibit DHCPREQUEST from unknown
hosts, but not DHCPINFORM which can request routing information. This caused
trouble for some bridged networks outside CloudStack.
This patch would fix the issue, prevent dnsmasq from acking any unknown request.
Signed-off-by: Sheng Yang <sheng.yang@citrix.com>
Replacing whatami with $0 which is how UNIX shell scripts should get the
script's name.
BUG-ID: CLOUDSTACK-6129
Bugfix-for:
Reviewed-by:
Reported-by:
Signed-off-by: John Kinsella <jlk@stratosec.co> 1392660036 -0800
The original issue has been exposed due to CloudStack VR would modify the
dnsmasq.leases, thus make it unsync with dnsmasq's memory lease.
Make the modification to let dnsmasq handle the lease file if dhcp_release is
available.
Implemented commands that are required for VR to bootup and Vm deployment to work
Modified hyperv agent code, to deploy VR with Boot Args, boot args passed to VR using KVP Exchange Component.
Fix for VR to boot up and get configured with boot args, Fixed issue in VolumeOrchestrator
Implemented SetFirewallRulesCommand in HyperV Resource
Implemented VR network commands to provide the necessary services from VR
Fixed hyperv localstorage path encode url issue. encode is converting space to '+'
Now VPN connection can be created as "passive", which would enable the ability
of remote peer initiate the connection. So it's possible for VPC VR to
establish the connection to another VPC VR of CloudStack.
Test case also included.
The test case would create 2 vpcs and using VPN to connect them.
This patch would reset the priority in such condition:
1. All redundant routers are stopped, e.g. due to network GC
2. User start one VM in the network
3. The routers would be brought up with reseted priority(100 & 99).
This would resolve the issue of network GC result in lower limit of redundant router priority reached.
All (almost) files belonging to the systemvm aer now centralize in the systemvm directory. The code for the separate functions is still in the services directory. This will make the code easier to understand and makes it clear that the systemvm is a separate item. It alos means that it can be excluded from the build entirely by not adding the systemvm profile, this will speed up the compiles somewhat.
Michael Andersen
Remi Bergsma
Jayapal
Kishan Kavala
wilderrodrigues
Ian Southam
David Bierce
Abhinandan Prateek
Rajani Karuturi
Rohit Yadav
Rene Moser
Rene Moser
Hugo Trippaers
Leo Simons
René Moser
ramamurtis
Remi Bergsma
Wei Zhou
Axel Delahaye
Sheng Yang
vetrivelc
Sanjay Tripathi
Joris van Lieshout
Edison Su
Daan Hoogland
Saksham Srivastava
David Nalley
Frank Zhang
Bharat Kumar
Fred Clift
Wido den Hollander
Saurav Lahiri
Nitin Mehta
Marcus Sorensen
Harikrishna Patnala
Rajesh Battala
John Kinsella
Koushik Das
Alex Huang
Darren Shepherd