CLOUDSTACK-6989: Add 3 strikes rule for RvR freezing detection

Sometime in VR ntpd would move time backward to keep sync with NTP server, which
can result in false alarm of keepalived monitering process.

This patch adds 3 strikes for keepalived process dead detection to avoid falsely
shutdown keepalived process due to time adjustment for only once.

(cherry picked from commit 75c9a20c77)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This commit is contained in:
Sheng Yang 2014-06-24 16:58:55 -07:00 committed by Rohit Yadav
parent 96e14f309e
commit 7f1dd86443
1 changed files with 15 additions and 0 deletions

View File

@ -17,12 +17,27 @@
# under the License.
STRIKE_FILE="[RROUTER_BIN_PATH]/keepalived.strikes"
if [ -e [RROUTER_BIN_PATH]/keepalived.ts2 ]
then
lasttime=$(cat [RROUTER_BIN_PATH]/keepalived.ts2)
thistime=$(cat [RROUTER_BIN_PATH]/keepalived.ts)
diff=$(($thistime - $lasttime))
s=0
if [ $diff -lt 30 ]
then
if [ -e $STRIKE_FILE ]
then
s=`cat $STRIKE_FILE 2>/dev/null`
fi
s=$(($s+1))
echo $s > $STRIKE_FILE
else
rm $STRIKE_FILE
fi
#3 strikes rule
if [ $s -gt 2 ]
then
echo Keepalived process is dead! >> [RROUTER_LOG]
[RROUTER_BIN_PATH]/services.sh stop >> [RROUTER_LOG] 2>&1