From owner-freebsd-questions@FreeBSD.ORG Fri Mar 7 17:45:50 2008 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8BDCB1065677 for ; Fri, 7 Mar 2008 17:45:50 +0000 (UTC) (envelope-from maddog2k@maddog2k.net) Received: from smtp.fiqz.com (smtp.fiqz.com [IPv6:2001:898:2000:8::132]) by mx1.freebsd.org (Postfix) with ESMTP id 09BA68FC43 for ; Fri, 7 Mar 2008 17:45:50 +0000 (UTC) (envelope-from maddog2k@maddog2k.net) Received: from WOUTERWIDEXS (we-are.widexs.nl [213.206.125.242]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.fiqz.com (Postfix) with ESMTP id DB28E2F7251 for ; Fri, 7 Mar 2008 18:43:04 +0100 (CET) Message-ID: <11C1AB18B24648D9A2B5A83B78A46218@intra.widexs.nl> From: "Wouter de Jong" To: Date: Fri, 7 Mar 2008 18:45:47 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Windows Mail 6.0.6000.16480 X-MimeOLE: Produced By Microsoft MimeOLE V6.0.6000.16545 Subject: strange issue with carp interface aliases X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2008 17:45:50 -0000 Hi, We have 2 FreeBSD machines running as a firewall in a CARP+pf+pfsync setup. Worked great, however ..... today I noticed something weird. I had to reboot the master machine, and when it came back ... one of the CARP addresses no longer worked. Looking in the logs, I got carp4: incorrect hash And looking at the carp interface .... both machines were running MASTER for this interface. Looking closer, I noticed my primary machine had this configuration : carp4: flags=49 metric 0 mtu 1500 inet 213.206.xx.62 netmask 0xfffffff0 inet 213.206.xx.49 netmask 0xfffffff0 carp: MASTER vhid 4 advbase 1 advskew 100 and my secondary : carp4: flags=49 metric 0 mtu 1500 inet 213.206.xx.49 netmask 0xfffffff0 inet 213.206.xx.62 netmask 0xfffffff0 carp: MASTER vhid 4 advbase 1 advskew 100 It swapped the carp alias alias (213.206.xx.62) to be the first address on the interface. This was the only interface it happened. The config : primary: ############################## defaultrouter="213.206.yy.193" hostname="fw01.xxx.yyy" cloned_interfaces="carp0 carp1 carp2 carp3 carp4 carp5 carp6 carp7 carp8 carp9 carp10 carp11 carp12 carp13 carp14 carp15 carp16 carp17 carp18 carp19 carp20 carp21 carp22 carp23 carp24 carp25 carp26 carp27 carp28" ifconfig_bge0="inet 213.206.yy.194 netmask 255.255.255.240" ifconfig_bge1="inet 213.206.xx.2 netmask 255.255.255.240" ifconfig_bge1_alias0="inet 213.206.xx.18 netmask 255.255.255.240" ifconfig_bge1_alias1="inet 213.206.xx.34 netmask 255.255.255.240" ifconfig_bge1_alias2="inet 213.206.xx.50 netmask 255.255.255.240" ifconfig_bge1_alias3="inet 213.206.xx.66 netmask 255.255.255.240" ifconfig_bge1_alias4="inet 213.206.xx.82 netmask 255.255.255.240" ifconfig_carp0="vhid 255 pass blubVIP0255 213.206.yy.206/28" ifconfig_carp1="vhid 1 pass blubVIP0001 213.206.xx.1/28" ifconfig_carp2="vhid 2 pass blubVIP0002 213.206.xx.17/28" ifconfig_carp2_alias0="vhid 2 pass blubVIP0002 213.206.xx.30/28" ifconfig_carp3="vhid 3 pass blubVIP0003 213.206.xx.33/28" ifconfig_carp4="vhid 4 pass blubVIP0004 213.206.xx.49/28" ifconfig_carp4_alias0="vhid 4 pass blubVIP0004 213.206.xx.62/28" ifconfig_carp5="vhid 5 pass blubVIP0005 213.206.xx.65/28" ifconfig_carp6="vhid 6 pass blubVIP0006 213.206.xx.81/28" ############################## secondary: ############################## defaultrouter="213.206.yy.193" hostname="fw02.xxx.yyy" cloned_interfaces="carp0 carp1 carp2 carp3 carp4 carp5 carp6 carp7 carp8 carp9 carp10 carp11 carp12 carp13 carp14 carp15 carp16 carp17 carp18 carp19 carp20 carp21 carp22 carp23 carp24 carp25 carp26 carp27 carp28" ifconfig_bge0="inet 213.206.yy.195 netmask 255.255.255.240" ifconfig_bge1="inet 213.206.xx.3 netmask 255.255.255.240" ifconfig_bge1_alias0="inet 213.206.xx.19 netmask 255.255.255.240" ifconfig_bge1_alias1="inet 213.206.xx.35 netmask 255.255.255.240" ifconfig_bge1_alias2="inet 213.206.xx.51 netmask 255.255.255.240" ifconfig_bge1_alias3="inet 213.206.xx.67 netmask 255.255.255.240" ifconfig_bge1_alias4="inet 213.206.xx.83 netmask 255.255.255.240" ifconfig_carp0="vhid 255 advskew 100 pass blubVIP0255 213.206.yy.206/28" ifconfig_carp1="vhid 1 advskew 100 pass blubVIP0001 213.206.xx.1/28" ifconfig_carp2="vhid 2 advskew 100 pass blubVIP0002 213.206.xx.17/28" ifconfig_carp2_alias0="vhid 2 advskew 100 pass blubVIP0002 213.206.xx.30/28" ifconfig_carp3="vhid 3 advskew 100 pass blubVIP0003 213.206.xx.33/28" ifconfig_carp4="vhid 4 advskew 100 pass blubVIP0004 213.206.xx.49/28" ifconfig_carp4_alias0="vhid 4 advskew 100 pass blubVIP0004 213.206.xx.62/28" ifconfig_carp5="vhid 5 advskew 100 pass blubVIP0005 213.206.xx.65/28" ifconfig_carp6="vhid 6 advskew 100 pass blubVIP0006 213.206.xx.81/28" ############################## After rebooting the secondary, it still gave me incorrect hash. But, it gave me the same thing on carp2 now. ... however, here the secondary had the carp2_alias0 listed as first, where as the primary had the carp2 as first, and the carp2_alias0 as second address. How can this ever happen ? Now I'm redundant .... but I must pray that the addresses will come up in the same order. Never had this issue on FreeBSD 6.x(p*)-RELEASE, but now I'm running FreeBSD 7.0-RELEASE. Help ! :) Kind regards, Wouter de Jong The Netherlands