From owner-freebsd-fs@FreeBSD.ORG Sat Mar 12 22:42:57 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AA556106564A for ; Sat, 12 Mar 2011 22:42:57 +0000 (UTC) (envelope-from peppe.maniscalco@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 419408FC16 for ; Sat, 12 Mar 2011 22:42:56 +0000 (UTC) Received: by eyg7 with SMTP id 7so1279931eyg.13 for ; Sat, 12 Mar 2011 14:42:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:date:message-id:subject:from:to :content-type; bh=3rAM7VGM4byhJ7xVSJ4so8cn4SBSpVXZcOsAZTi7wfo=; b=cFNtpkX7dm/PyfVokmjKgpW5F13cBd/2XVDPbl1uiMF3tFIQzzK3RRB77rHTyCqHPq uqcYwdG9y2Te+kfMduBm6BLNar/m6g+6VblE5KtlU82BYm5SD4dyI3BCLKYq1SfuI4RQ r9N4LohKLkc9A28qO/SOiTRR1ih6Q3+zUHMEE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=eCLC2F8QbYiUQ8YsLV5Bs8xAZuhauZTBdHHU9aV8BpJJq/kr5nO1ejNg15aXs+Tz1R oAmi53GxTJwaEsQy1hNq+DkAWili7T8F1gZa4gdbE2FtVFflm2VPGdb+1WIEU8YmT/J9 fTEhybzZlfSrkV68nWFHAhxFVjX/pvarkmAao= MIME-Version: 1.0 Received: by 10.14.126.205 with SMTP id b53mr3689318eei.41.1299968282781; Sat, 12 Mar 2011 14:18:02 -0800 (PST) Received: by 10.14.122.202 with HTTP; Sat, 12 Mar 2011 14:18:02 -0800 (PST) Date: Sat, 12 Mar 2011 23:18:02 +0100 Message-ID: From: Giuseppe Maniscalco To: freebsd-fs Content-Type: text/plain; charset=ISO-8859-1 Subject: FreeBSD7, pf, carp... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Mar 2011 22:42:57 -0000 Hi List! I need your help!!! I've two firewalls configured in parallel (connected with a crossover cable) and I use pfsync+carp to failover. So one firewall (A) handles all traffic as MASTER and, if it dies or if some NIC interface go down, the second firewall (B) takes over automatically. Well... As usually everything works properly, but since a few days ago "B" takes control and "A" become backup. But "A" cannot return to be master until rebooting! After reboot, "A" is the master for a while, then I've the same problem... I identified a problem here: fwA# sysctl -a | grep arp net.inet.ip.same_prefix_carp_only: 0 net.inet.carp.allow: 1 net.inet.carp.preempt: 1 net.inet.carp.log: 1 net.inet.carp.arpbalance: 0 net.inet.carp.suppress_preempt: 1 >From man carp: net.inet.carp.suppress_preempt: A read only value showing the status of preemption suppression. Preemption can be suppressed if link on an interface is down or when pfsync(4) interface is not synchronized. Value of 0 means that preemption is not suppressed, since no problems are detected. Every problem increments suppression counter. All my interfaces are UP... now I don't know how to check if pfsync is synched or not... Meanwhile, in B node: fwB# sysctl -a | grep arp net.inet.ip.same_prefix_carp_only: 0 net.inet.carp.allow: 1 net.inet.carp.preempt: 1 net.inet.carp.log: 1 net.inet.carp.arpbalance: 0 net.inet.carp.suppress_preempt: 0 I tried with a tcpdump on the interfaces, but I see just the change of condition (master/backup) with the advskew modification... This is the only strange thing on DMZ interface... : 17:01:32.397429 01:80:c2:00:00:01 (oui Unknown) > 01:80:c2:00:00:01 (oui Unknown), ethertype Unknown (0x8808), length 60: 0x0000: 0001 ffff 0000 0000 0000 0000 0000 0000 ................ 0x0010: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0x0020: 0000 0000 0000 0000 0000 0000 0000 .............. I just tried to change the NIC, but nothing! "A" continue to lose control in 30/45 minutes... I read somewhere that the result of "pfctl -ss" must give the same result on both nodes: fwA# pfctl -ss | wc -l 5833 fwB# pfctl -ss | wc -l 5507 Could it be important? Some additional information: fwA# more /etc/rc.conf ifconfig_em0="inet a.a.a.12 netmask 255.255.255.0 polling" ### DMZ ### ifconfig_em1="inet b.b.b.2 netmask 255.255.0.0 polling" ### CROSSOVER ### ifconfig_em2="inet c.c.c.189 netmask 255.255.255.224 polling" ### ISP1 ### ifconfig_em3="inet d.d.d.249 netmask 255.255.255.0 polling" ### ISP2 ### defaultrouter="a.a.a.1" #Firewall pf_enable="YES" pf_rules="/etc/pf.conf" pf_flags="" pflog_enable="YES" pflog_logfile="/var/log/pflog" #Failover pfsync_enable="YES" pfsync_syncdev="em1" cloned_interfaces="carp0 carp1 carp2" ifconfig_carp0="a.a.a.1/24 vhid 1 pass foo" ifconfig_carp0_alias0="a.a.a.11/24 vhid 1 pass foo" ifconfig_carp1="d.d.d.14/24 vhid 2 pass bar" ifconfig_carp1_alias0="d.d.d.2/24 vhid 2 pass bar" ifconfig_carp2="c.c.c.188/27 vhid 3 pass jack" ifconfig_carp2_alias0="c.c.c.165/27 vhid 3 pass jack" fwB# more /etc/rc.conf ifconfig_ste0="inet a.a.a.13 netmask 255.255.255.0 polling" ifconfig_ste1="inet b.b.b.3 netmask 255.255.0.0 polling" ifconfig_em0="inet c.c.c.190 netmask 255.255.255.224 polling" ifconfig_em1="inet d.d.d.250 netmask 255.255.255.0 polling" defaultrouter="c.c.c.1" #Firewall pf_enable="YES" pf_rules="/etc/pf.conf" pf_flags="" pflog_enable="YES" pflog_logfile="/var/log/pflog" #Failover pfsync_enable="YES" pfsync_syncdev="ste1" cloned_interfaces="carp0 carp1 carp2" ifconfig_carp0="a.a.a.1/24 vhid 1 advskew 128 pass foo" ifconfig_carp0_alias0="a.a.a.11/24 vhid 1 advskew 128 pass foo" ifconfig_carp1="d.d.d.14/24 vhid 2 advskew 64 pass bar" ifconfig_carp1_alias0="d.d.d.2/24 vhid 2 advskew 64 pass bar" ifconfig_carp2="c.c.c.188/27 vhid 3 advskew 100 pass jack" ifconfig_carp2_alias0="c.c.c.165/27 vhid 3 advskew 100 pass jack" In each node pf.conf I added: fwA# more pf.conf | grep failover pass quick on { em1 } proto pfsync # failover pass on { em0 em2 em3 } proto carp # failover fwB# more pf.conf | grep failover pass quick on { ste1 } proto pfsync # failover pass on { em0 ste0 em1 } proto carp # failover I hope that someone can give me a solution please, or maybe just an idea, cause I'm getting crazy!!! Please ask me, if you need further information... Thank you all!