We need to use ifup/ifdown to bring up the interfaces, because ifconfig don't
know the ip of the interface after we modify cloud-early-config to avoid
first start up of public interface.
Reviewed-by: Edison
In order to make sure next time, booting process would use cloud-early-config's
setup, rather than networking scripts to bring up interfaces.
Reviewed-by: Kelven Yang
I can't see why we set eth0 to dhcp by default. It would result in eth0 want to
get a DHCP address from outside. We should always assign ip through
cloud-early-config for it.
But one point is, the priority of cloud-early-config and networking script is
the same. So even networking got some ip from outside, cloud-early-config
should able to override it(if cloud-early-config runs after networking) or
networking script won't get dhcp (if cloud-early-config runs before networking),
so I am not quite understand why router would get DHCP address in fact. Maybe
there are other issues.
The routing table with two nics may be messed up, due to we sent same
router(gateway) information from different DHCP server, in order to specify
default gateway. E.g.
Network A: 192.168.1.0/24, gw 192.168.1.1
Network B: 192.168.2.0/24, gw 192.168.2.1
User VM: Nic 1 connect to network A, get ip 192.168.1.10; nic 2 connect to
network B, get ip 192.168.2.10.
Set network A as the default network of user VM.
Currently we would send this information to user VM through DHCP offer:
In network A: dhcp-option:router 192.168.1.1
In network B: dhcp-option:router 192.168.1.1
So both NIC in the guest VM would receive 192.168.1.1 as router(gateway).
But, in CentOS 5.6, dhclient-scripts try to tell if the gateway is reachable
for current subnet.
So when we try to enable nic 2(eth1) of user VM, dhclient would receive:
IP: 192.168.2.10
Mask: 255.255.255.0
Router: 192.168.1.1
Then it would found that the specified gateway(router) is not within its own
subnet(192.168.2.0/24). But since we send out this ip(192.168.1.1) as the
gateway for it, dhclient thought that it should got someway to access the
network through this IP. So it would execute:
ip route add 192.168.1.1 dev eth1
ip route replace default via 192.168.1.1 dev eth1
But it can never reach 192.168.1.1(which is in the eth0's subnet and the
gateway of eth0) by go through eth1 interface. So it is messed up.
We've tested Windows 2008 R2, CentOS 5.3, CentOS 5.6 and Ubuntu 10.04. Windows
and Ubuntu are fine with above policy.
To solve this, we send different dhcp:router option according to the guest OS
type now.
We may need expand this list later, but for now we only know that CentOS and
RHEL would behavior in this way.
status 14042: resolved fixed
Summary of changes:
- Fix the order of source nat ip's : Static Nat IP's will be on top of Router source nat IP's. means Static NAT ip will take higher preference when compare to router ip while picking ip for source nat.
Reviewed-by: Abhi
Summary of changes: Added Hairpin Nat.
- defined Harpin NAT function.
- Called Hairpin NAT while adding/deleting port forwading and Static NAT rules.
- added rules in IPtables config file, this will be iniated during bootup to forward New/established connectons from eth0 to eth0.
Summary of Changes: Using multiple routing tables to send the packets on the public NIC's based on source IP for the following type of connections:
- Inbound connections of Static NAT ip .
- Outbound connections of static-NAT (using static NAT-ip for SNAT).
The problem is remove_first_ip() in ipassoc.sh can't be called more than one.
The call after the first time would result in iptable and ip command failure,
thus result in failure of execution of IpAssocCommand.
Use the same way to detect already disassociated ip address of non-first
IP(remove_an_ip()) to fix the issue.
reviewed-by: Edison Su
status 13606: resolved fixed
Revert "bug 11056: Add backported kernel and discard customized kernel module"
This reverts commit 857e817cfc707f4280f295a91642ded861c5aa68.
Bug 13403 is due to new kernel fail to suppose hot-unplug of xen vnif.
Notice the module is only backported for kernel 2.6.32-5-686-bigmem. That's why
I hardcode the kernel version here.
status 13403: resolved fixed
Summary of changes :
- Added a new flag -s to ipassoc command to carry if the ip address is
used for SNAT or not.
- SNAT is completly decoupled from the first flag. first flag is used
to decide if the ip address is first ip address of the interface.
- -s and -f are independent, SNAT can be enabled on the non-first ip
also.
Summary of changes:
- Mutiple routing table for each public interface is added (previously there is only one routing table ). when the packet is send out of public interface corresponding per-interface routing table will be used. per-interface routing table will modified when ever ip/interface added/deleted.
- New parameter is added to ipassoc command to include the default gateway for every interface/ip. prevously it is using only one public interface to send out, default gateway is obtained at the boot up time.
- In the DNAT case. In the revese path(from guest vm to outside, or when DNAT packet receives from the eth0) the public ip/source ip will not be available till POSTROUTING. to overcome this, DNAT connection are marked with routing table number at the time of connection creation, in the reverse path the routing table# from DNAT connection is used to detect per-interface routing table.
But still keep customized iptables because the newer version of iptables would
result in very big range of package upgrading due to dependence relationship.
Also add newer version of "virt-what" from debian testing, otherwise it would
provide wrong information for xen-domU(reported as xen-dom0 in previous version
for 2.6.39). This one have no dependency issue and can be added easily.
status 11056: resolved fixed
reviewed-by: Abhi
Summary of Changes:
while adding a primary address to the domR interface, previous primaray addresses(ip) are removed and added as with 32-bit netmask.
This is to avoid two same ip's with different netmask attached to the interface.
The following are summary of changes:
1) when network.disable.rpfilter is set to true, then rp_filter will be disadbled(set to 0) on all the public interfaces and also default setting of the system.
2) when network.disable.rpfilter is set to false, then rp_filter will be enabled(set to 1) on all the public interfaces and also default setting of the system.
3) here public public interface means , eth2 ... ethN. default setting means (/proc/sys/net/ipv4/conf/default/rp_filter).
4) Default setting change will have impact on non-public interface. Due to these, rp_filter is always enabled on Non-public interfaces(eth0,eth1 and lo).
5) when a new public interface is created, new interface will take rp_filter value from the default setting.
The following are summary of changes:
1) when network.disable.rpfilter is set to true, then rp_filter will be disadbled(set to 0) on all the public interfaces and also default setting of the system.
2) when network.disable.rpfilter is set to false, then rp_filter will be enabled(set to 1) on all the public interfaces and also default setting of the system.
2) here public public interface means , eth2 ... ethN. default setting means (/proc/sys/net/ipv4/conf/default/rp_filter).
3) Default setting change will have impact on non-public interface.if there is no specific setting for other interfaces in /etc/sysctl.conf or otherexplict setting , they will follow this default settings. currently non-public interface like eth0 ,eth1 does not have any specific setting in sysctl.conf, due to this there rp_filters will be changed when ever network.disable.rpfilter setting is changed.
4) default setting is required to changes beacuse when a new public interface is created, new interface will take rp_filter value from the default setting.
Because currently the lock in the script is retried every 1 second, and it's a
quite a long time that it's possible for some other active script can be
executed and retain the lock again. So it's possible that the first one request
the lock is always being preemptted by others, then finally got timeout.
To fix this issue, the retry interval is reduced to 0.1 seconds, which would
provide more retry times. And each process want to get the lock would create a
file named lockname-PID.lock, and only the first one(judged by timestamp) would
get the lock. The remaining ones would retry every 0.1 seconds to see if it can
get the lock.
Also timeout time is extended to 30 seconds.
And add testcase for it.
status 11772: resolved fixed
To solve password file is destroyed along with restartNetwork command issue. If
the password is not set in fact, user can use "ResetPassword" to try again. But
it won't happen mostly, because it's only possible if the restartNetwork
happened between user start up VM and set the new password.
Reviewed-by: Keshav
status 11518: resolved fixed
Added New value "link-local" to global config network.loadbalancer.haproxy.stats.visibility . With this change it can take new parameter "link-local" value apart from the existing 3 values global,guest-network,disabled.
global - stats visible from public network
guest-network - stats visible only to guestnetwork.
link-local - stats visible only to link local network
disabled - stats disabled.
Added global config to enable/disable rp_filter for domR.
previous commit: d966906374d4a0cb8fa57326a1f7625c871f64fd
Test Case-1 :
1) Set network.disable.rpfilter global config to true
2) Restart the domR
3) check the settings reflected in proc filesystem
- for public interface like eth2,eth3 : /proc/sys/net/ipv4/conf/eth2/rp_filter should have 0 , and rest other interfaces should have value of 1
Test Case-2 :
1) set network.disable.rpfilter global config to false
2) Restart the domR
3) check the settings reflected in proc filesystem
- for public interface like eth2,eth3 : /proc/sys/net/ipv4/conf/eth2/rp_filter should have 1 , and rest other interfaces should also have value of 1
This message may show during redundant router start up:
FAULT (Restarting DNS forwarder and DHCP server: dnsmasq failed!)
This caused by edithost.sh is racy with keepalived process. They both want to
restart dnsmasq.
Even in normal condition, it's very hard to reproduce this bug. Add file lock
for edithost.sh should solve it.
This message may show during redundant router start up:
FAULT (Restarting DNS forwarder and DHCP server: dnsmasq failed!)
This caused by edithost.sh is racy with keepalived process. They both want to
restart dnsmasq.
Even in normal condition, it's very hard to reproduce this bug. Add file lock
for edithost.sh should solve it.
The reason is:
1. In redundant router, we won't enable eth2(public network interface) until
keepalived determine the router is MASTER.
2. ipassoc.sh normally kick in before keepalived process running. And it would
set eth2's IP address using "ip addr add $dev $ip"
3. "ip addr add $dev $ip" won't add mask for the device, then there is no way to
update broadcast address for eth2. Then broadcast address is 0.0.0.0.
4. As long as "ip addr add $dev $ip" executed, later executed "ifconfig $dev $ip
netmask $mask" won't calculated the broadcast address from $ip and $mask.
To fix this, we enable and configure eth2 temporaily when cloud-early-config
executed, then disable eth2 interface. By this way, broadcast address of should
be calculated and set correctly.
status 11083: resolved fixed
The reason is:
1. In redundant router, we won't enable eth2(public network interface) until
keepalived determine the router is MASTER.
2. ipassoc.sh normally kick in before keepalived process running. And it would
set eth2's IP address using "ip addr add $dev $ip"
3. "ip addr add $dev $ip" won't add mask for the device, then there is no way to
update broadcast address for eth2. Then broadcast address is 0.0.0.0.
4. As long as "ip addr add $dev $ip" executed, later executed "ifconfig $dev $ip
netmask $mask" won't calculated the broadcast address from $ip and $mask.
To fix this, we enable and configure eth2 temporaily when cloud-early-config
executed, then disable eth2 interface. By this way, broadcast address of should
be calculated and set correctly.
status 11083: resolved fixed
commit e4fe14a9ce19fbbdb15bbfaad586d80031ca9fbc break redundant router, because
at time of ping, the network is not up for redundant router.
Add timout for ping
The issue happened quite rare, but indeed can show.
And when the issue happen, the status of redundant router would be "Status:
FAULT".
It's due to ipassoc.sh wasn't executed before the system bring eth2 up and go to
master mode, then eth2 wasn't configured correctly. Then "ip route add default
xx" can't complete.
This commit should fixes the issue.
The issue happened quite rare, but indeed can show.
And when the issue happen, the status of redundant router would be "Status:
FAULT".
It's due to ipassoc.sh wasn't executed before the system bring eth2 up and go to
master mode, then eth2 wasn't configured correctly. Then "ip route add default
xx" can't complete.
This commit should fixes the issue.
commit e4fe14a9ce19fbbdb15bbfaad586d80031ca9fbc break redundant router, because
at time of ping, the network is not up for redundant router.
Add timout for ping
Cahnges:
1) putting back the changes(bug 10800 and 10557) that had been reverted during merging of Elb/nectarine.
2) 10800 Upgrade from previous release also added: Upgrade from Previous release will leave iptable rules in the INPUT ipchain, this is fixed.
Part 1
This backport contained:
commit 52317c718c25111c2535657139b541db0c9d1e1f
bug 9154: Initial check in for enabling redundant virtual router
commit 54199112055d754371bfb141168fb5538bf6d6ea
Add host verification for CheckRouterCommand
commit cef978a228c90056ead9be10cbc4de74c2b8de76
Fix CheckRouterAnswer's isMaster report
commit 4072f0a6991ac3b63601a1764fbe14188965f62f
Some build fixes and code refactoring for redundant router
commit 4d3350b7cd8ee2706a9bace4437fc194e36c8dd5
Redundant Router: Fix OVS
commit 6a228830e7c46d819fa0c3317e159e041337e887
Fix findByNetwork()/findByNetworkAndPod()'s return
commit c627777b3d5bdbcd60db4032cebd349a5b1ecd83
Redundant Router: Fix isVmAlive()
commit e1275d2514adc41f8744f5107d4069c38be195f1
Only issue CheckRouterCommand to redundant routers
And all modification to the scripts till
commit 4e3942462ed3fde3a3d7011e95839e2128fba514
logging changes
in the master branch.
Sometime when keepalived start up(during system boot up period), it would fail
to(likely due to unable to receive the packet), and think itself is the only
router, then make itself master.
Add 10 seconds delay after start up to work around the issue.
anthony
Sheng Yang
Jayapal
Naredula Janardhana Reddy
frank
Sheng Yang
Chiradeep Vittal
Edison Su
Edison Su
Kelven Yang
kishan
frank
Abhinandan Prateek
alena
Edison Su