From owner-freebsd-pf@FreeBSD.ORG Wed Mar 2 01:27:09 2005 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8F51616A4CE for ; Wed, 2 Mar 2005 01:27:09 +0000 (GMT) Received: from zixvpm01.seton.org (zixvpm01.seton.org [207.193.126.161]) by mx1.FreeBSD.org (Postfix) with ESMTP id 78EC843D46 for ; Wed, 2 Mar 2005 01:27:08 +0000 (GMT) (envelope-from mgrooms@seton.org) Received: from zixvpm01.seton.org (ZixVPM [127.0.0.1]) by Outbound.seton.org (Proprietary) with ESMTP id AFD11360076 for ; Tue, 1 Mar 2005 19:27:07 -0600 (CST) Received: from smtp-out.seton.org (unknown [10.21.254.249]) by zixvpm01.seton.org (Proprietary) with ESMTP id 6480D330061; Tue, 1 Mar 2005 19:27:07 -0600 (CST) Received: from localhost (unknown [127.0.0.1]) by smtp-out.seton.org (Postfix) with ESMTP id 589728014E24; Tue, 1 Mar 2005 19:27:07 -0600 (CST) Received: from smtp-out.seton.org ([10.21.254.249]) by localhost (mail [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 03233-45; Tue, 1 Mar 2005 19:27:07 -0600 (CST) Received: from ausexfe02.seton.org (ausexfe02.seton.org [10.20.10.185]) by smtp-out.seton.org (Postfix) with ESMTP id 3594D8014E23; Tue, 1 Mar 2005 19:27:07 -0600 (CST) Received: from [10.20.160.190] ([10.20.160.190]) by ausexfe02.seton.org with Microsoft SMTPSVC(6.0.3790.211); Tue, 1 Mar 2005 19:27:06 -0600 Message-ID: <4225174C.801@seton.org> Date: Tue, 01 Mar 2005 19:30:52 -0600 From: Matthew Grooms Organization: Seton Healthcare Network User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Gleb Smirnoff References: <200502282232.17646.max@love2party.net> <4223931C.9000607@seton.org> <200502282326.41760.max@love2party.net> <4224B078.9020301@seton.org> <20050301185431.GA81982@cell.sick.ru> In-Reply-To: <20050301185431.GA81982@cell.sick.ru> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 02 Mar 2005 01:27:06.0984 (UTC) FILETIME=[F271F680:01C51EC6] X-Virus-Scanned: by amavisd-new at seton.org cc: freebsd-pf@FreeBSD.org Subject: Re: Fwd: pf + pfsync + carp testing ... X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical discussion and general questions about packet filter (pf) List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Mar 2005 01:27:09 -0000 Gleb, Thanks for the response. I have net.inet.carp.preempt=1 set but only one carp interface changes state to master at a time. The second host always retains the master for the other two carp interfaces. I am able to manually fail over the remaining carp interfaces by changing either the carp or parent em[n] interface to down which quickly brings the other hosts corresponding interface from backup to master. After a firewall holds a master status for all carp devices, I can start to talk again through the firewall out to the internet or into the DMZ from my test pc. I have two Dell SMP boxes running dual amd64 compatible intel processors with two on board intel ports ( em0 & em1 ) and a quad port intel pro 1000 MT ( em2, em3, em4 & em5 ). I am using RELENG_5 amd64 SMP builds on both hosts. Here is my config info ... --- both firewalls --- em0 -> carp0 -> External em1 -> carp1 -> Internal em2 -> carp2 -> DMZ --- fw1 sysctl.conf --- net.inet.carp.preempt=1 --- fw1 rc.conf --- hostname="fw1.seton.org" cloned_interfaces="carp0 carp1 carp2" ifconfig_em0="inet 192.168.253.2 netmask 255.255.255.0" ifconfig_em1="inet 192.168.254.2 netmask 255.255.255.0" ifconfig_em2="inet 192.168.251.2 netmask 255.255.255.0" ifconfig_em5="inet 192.168.252.2 netmask 255.255.255.0" ifconfig_carp0="up vhid 1 advskew 0 pass blah 192.168.253.1 255.255.255.0" ifconfig_carp1="up vhid 2 advskew 0 pass blah 192.168.254.1 255.255.255.0" ifconfig_carp2="up vhid 3 advskew 0 pass blah 192.168.251.1 255.255.255.0" ifconfig_pfsync0="up syncif em5" defaultrouter="192.168.253.252" pf_enable="YES" gateway_enable="YES" sshd_enable="YES" --- fw1 pf.conf --- nat on em0 from ! em0 to any -> carp0 pass from any to any keep state --- fw2 sysctl.conf --- net.inet.carp.preempt=1 --- fw2 rc.conf --- hostname="fw2.seton.org" cloned_interfaces="carp0 carp1 carp2" ifconfig_em0="inet 192.168.253.3 netmask 255.255.255.0" ifconfig_em1="inet 192.168.254.3 netmask 255.255.255.0" ifconfig_em2="inet 192.168.251.3 netmask 255.255.255.0" ifconfig_em5="inet 192.168.252.3 netmask 255.255.255.0" ifconfig_carp0="up vhid 1 advskew 100 pass blah 192.168.253.1 255.255.255.0" ifconfig_carp1="up vhid 2 advskew 100 pass blah 192.168.254.1 255.255.255.0" ifconfig_carp2="up vhid 3 advskew 100 pass blah 192.168.251.1 255.255.255.0" ifconfig_pfsync0="up syncif em5" defaultrouter="192.168.253.252" pf_enable="YES" gateway_enable="YES" sshd_enable="YES" --- fw2 pf.conf --- nat on em0 from ! em0 to any -> carp0 pass from any to any keep state The other thing I should mention is that I cannot get any of the carp interfaces to change to an up state using ifconfig_carp[n] entries in the rc.conf file no matter what I try. When the box comes up, they always have a flags value of zero and I have to manually set them to up after gaining remote console access. Here is ifconfig output from both firewalls after a boot using the above configs ... --- fw1 ifconfig --- carp0: flags=0<> mtu 1500 inet 192.168.253.1 netmask 0xffffff00 carp: INIT vhid 1 advbase 1 advskew 0 carp1: flags=0<> mtu 1500 inet 192.168.254.1 netmask 0xffffff00 carp: INIT vhid 2 advbase 1 advskew 0 carp2: flags=0<> mtu 1500 inet 192.168.251.1 netmask 0xffffff00 carp: INIT vhid 3 advbase 1 advskew 0 --- fw2 ifconfig --- carp0: flags=0<> mtu 1500 inet 192.168.253.1 netmask 0xffffff00 carp: INIT vhid 1 advbase 1 advskew 100 carp1: flags=0<> mtu 1500 inet 192.168.254.1 netmask 0xffffff00 carp: INIT vhid 2 advbase 1 advskew 100 carp2: flags=0<> mtu 1500 inet 192.168.251.1 netmask 0xffffff00 carp: INIT vhid 3 advbase 1 advskew 100 Please let me know what other information I can provide or what else I can do to help test this out. P.S. - I have had the preempt flag set all along but I thought it did something else entirely due to the way it is described at ... http://www.countersiege.com/doc/pfsync-carp/ Sorry for not being able to read the carp man page earlier to get the updated definition. It actually didn't install for me when I applied the patch and did the buildinstall. Probably something I goofed. After manually copying it over from /usr/src/share/man/man4 to /usr/share/man/man4 I was able to read it. Thanks again. Matthew Grooms Gleb Smirnoff wrote: > On Tue, Mar 01, 2005 at 12:12:08PM -0600, Matthew Grooms wrote: > M> Thanks Max and Gleb. You have been a great help. The patch applied > M> cleanly and compiled fine. After configuring a few carp interfaces, they > M> seem to fail over well. I am curious though, is CARP designed to have > M> interfaces fail over individually or as a group? > > To make backup router preempt the master, when at least one > interface fail you need to set sysctl net.inet.carp.preempt to one. > > This is also described in carp(4). >