Date: Tue, 01 Mar 2005 19:30:52 -0600 From: Matthew Grooms <mgrooms@seton.org> To: Gleb Smirnoff <glebius@FreeBSD.org> Cc: freebsd-pf@FreeBSD.org Subject: Re: Fwd: pf + pfsync + carp testing ... Message-ID: <4225174C.801@seton.org> In-Reply-To: <20050301185431.GA81982@cell.sick.ru> References: <200502282232.17646.max@love2party.net> <4223931C.9000607@seton.org> <200502282326.41760.max@love2party.net> <4224B078.9020301@seton.org> <20050301185431.GA81982@cell.sick.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
Gleb,
Thanks for the response. I have net.inet.carp.preempt=1 set but
only one carp interface changes state to master at a time. The second
host always retains the master for the other two carp interfaces. I am
able to manually fail over the remaining carp interfaces by changing
either the carp or parent em[n] interface to down which quickly brings
the other hosts corresponding interface from backup to master. After a
firewall holds a master status for all carp devices, I can start to talk
again through the firewall out to the internet or into the DMZ from my
test pc.
I have two Dell SMP boxes running dual amd64 compatible intel processors
with two on board intel ports ( em0 & em1 ) and a quad port intel pro
1000 MT ( em2, em3, em4 & em5 ). I am using RELENG_5 amd64 SMP builds on
both hosts. Here is my config info ...
--- both firewalls ---
em0 -> carp0 -> External
em1 -> carp1 -> Internal
em2 -> carp2 -> DMZ
--- fw1 sysctl.conf ---
net.inet.carp.preempt=1
--- fw1 rc.conf ---
hostname="fw1.seton.org"
cloned_interfaces="carp0 carp1 carp2"
ifconfig_em0="inet 192.168.253.2 netmask 255.255.255.0"
ifconfig_em1="inet 192.168.254.2 netmask 255.255.255.0"
ifconfig_em2="inet 192.168.251.2 netmask 255.255.255.0"
ifconfig_em5="inet 192.168.252.2 netmask 255.255.255.0"
ifconfig_carp0="up vhid 1 advskew 0 pass blah 192.168.253.1 255.255.255.0"
ifconfig_carp1="up vhid 2 advskew 0 pass blah 192.168.254.1 255.255.255.0"
ifconfig_carp2="up vhid 3 advskew 0 pass blah 192.168.251.1 255.255.255.0"
ifconfig_pfsync0="up syncif em5"
defaultrouter="192.168.253.252"
pf_enable="YES"
gateway_enable="YES"
sshd_enable="YES"
--- fw1 pf.conf ---
nat on em0 from ! em0 to any -> carp0
pass from any to any keep state
--- fw2 sysctl.conf ---
net.inet.carp.preempt=1
--- fw2 rc.conf ---
hostname="fw2.seton.org"
cloned_interfaces="carp0 carp1 carp2"
ifconfig_em0="inet 192.168.253.3 netmask 255.255.255.0"
ifconfig_em1="inet 192.168.254.3 netmask 255.255.255.0"
ifconfig_em2="inet 192.168.251.3 netmask 255.255.255.0"
ifconfig_em5="inet 192.168.252.3 netmask 255.255.255.0"
ifconfig_carp0="up vhid 1 advskew 100 pass blah 192.168.253.1 255.255.255.0"
ifconfig_carp1="up vhid 2 advskew 100 pass blah 192.168.254.1 255.255.255.0"
ifconfig_carp2="up vhid 3 advskew 100 pass blah 192.168.251.1 255.255.255.0"
ifconfig_pfsync0="up syncif em5"
defaultrouter="192.168.253.252"
pf_enable="YES"
gateway_enable="YES"
sshd_enable="YES"
--- fw2 pf.conf ---
nat on em0 from ! em0 to any -> carp0
pass from any to any keep state
The other thing I should mention is that I cannot get any of the carp
interfaces to change to an up state using ifconfig_carp[n] entries in
the rc.conf file no matter what I try. When the box comes up, they
always have a flags value of zero and I have to manually set them to up
after gaining remote console access. Here is ifconfig output from both
firewalls after a boot using the above configs ...
--- fw1 ifconfig ---
carp0: flags=0<> mtu 1500
inet 192.168.253.1 netmask 0xffffff00
carp: INIT vhid 1 advbase 1 advskew 0
carp1: flags=0<> mtu 1500
inet 192.168.254.1 netmask 0xffffff00
carp: INIT vhid 2 advbase 1 advskew 0
carp2: flags=0<> mtu 1500
inet 192.168.251.1 netmask 0xffffff00
carp: INIT vhid 3 advbase 1 advskew 0
--- fw2 ifconfig ---
carp0: flags=0<> mtu 1500
inet 192.168.253.1 netmask 0xffffff00
carp: INIT vhid 1 advbase 1 advskew 100
carp1: flags=0<> mtu 1500
inet 192.168.254.1 netmask 0xffffff00
carp: INIT vhid 2 advbase 1 advskew 100
carp2: flags=0<> mtu 1500
inet 192.168.251.1 netmask 0xffffff00
carp: INIT vhid 3 advbase 1 advskew 100
Please let me know what other information I can provide or what else I
can do to help test this out.
P.S. - I have had the preempt flag set all along but I thought it did
something else entirely due to the way it is described at ...
http://www.countersiege.com/doc/pfsync-carp/
Sorry for not being able to read the carp man page earlier to get the
updated definition. It actually didn't install for me when I applied the
patch and did the buildinstall. Probably something I goofed. After
manually copying it over from /usr/src/share/man/man4 to
/usr/share/man/man4 I was able to read it. Thanks again.
Matthew Grooms
Gleb Smirnoff wrote:
> On Tue, Mar 01, 2005 at 12:12:08PM -0600, Matthew Grooms wrote:
> M> Thanks Max and Gleb. You have been a great help. The patch applied
> M> cleanly and compiled fine. After configuring a few carp interfaces, they
> M> seem to fail over well. I am curious though, is CARP designed to have
> M> interfaces fail over individually or as a group?
>
> To make backup router preempt the master, when at least one
> interface fail you need to set sysctl net.inet.carp.preempt to one.
>
> This is also described in carp(4).
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4225174C.801>