From owner-freebsd-net@FreeBSD.ORG Thu Feb 18 17:04:53 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1D1AD106566C for ; Thu, 18 Feb 2010 17:04:53 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [128.238.9.200]) by mx1.freebsd.org (Postfix) with ESMTP id B422A8FC16 for ; Thu, 18 Feb 2010 17:04:52 +0000 (UTC) Received: (qmail 16517 invoked from network); 18 Feb 2010 17:04:51 -0000 Received: from unknown (HELO ?10.0.0.158?) (spawk@128.238.64.31) by acm.poly.edu with AES256-SHA encrypted SMTP; 18 Feb 2010 17:04:51 -0000 Message-ID: <4B7D72BF.1040104@acm.poly.edu> Date: Thu, 18 Feb 2010 12:02:55 -0500 From: Boris Kochergin User-Agent: Thunderbird 2.0.0.23 (X11/20091021) MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: CARP vs. if_bridge X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 17:04:53 -0000 Ahoy. I'm seeing what appears to be erroneous interaction between CARP and if_bridge on multiple machines with a variety of Ethernet controllers and architectures. I've observed it on 7.2-R and 8.0-R. The test setup is simple enough: CARP master: FreeBSD t30 8.0-RELEASE-p1 FreeBSD 8.0-RELEASE-p1 #5: Sun Feb 14 20:22:41 EST 2010 root@t30:/usr/obj/usr/src/sys/T30 i386 lo0: flags=8049 metric 0 mtu 16384 options=3 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet6 ::1 prefixlen 128 inet 127.0.0.1 netmask 0xff000000 dc0: flags=8943 metric 0 mtu 1500 options=8 ether 00:04:5a:a8:e0:bf inet 192.168.0.2 netmask 0xffffff00 broadcast 192.168.0.255 media: Ethernet autoselect (100baseTX ) status: active carp0: flags=49 metric 0 mtu 1500 inet 192.168.0.1 netmask 0xffffff00 carp: MASTER vhid 1 advbase 1 advskew 0 CARP backup: FreeBSD ultra5 8.0-RELEASE FreeBSD 8.0-RELEASE #0: Thu Feb 18 15:19:39 UTC 2010 boris@ultra5:/usr/obj/usr/src/sys/GENERIC.carp sparc64 hme0: flags=8802 metric 0 mtu 1500 options=b ether 08:00:20:f5:65:d4 media: Ethernet autoselect xl0: flags=8943 metric 0 mtu 1500 options=9 ether 00:01:03:2c:06:6d inet 192.168.0.3 netmask 0xffffff00 broadcast 192.168.0.255 media: Ethernet autoselect (100baseTX ) status: active lo0: flags=8049 metric 0 mtu 16384 options=3 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 inet6 ::1 prefixlen 128 inet 127.0.0.1 netmask 0xff000000 carp0: flags=49 metric 0 mtu 1500 inet 192.168.0.1 netmask 0xffffff00 carp: MASTER vhid 1 advbase 1 advskew 100 bridge0: flags=8843 metric 0 mtu 1500 ether 3a:e6:09:2d:da:bc id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: xl0 flags=143 ifmaxaddr 0 port 2 priority 128 path cost 200000 member: hme0 flags=8 ifmaxaddr 0 port 1 priority 128 path cost 200000 In summary, I have a basic CARP configuration and, on the backup CARP machine, a bridge with the CARP device's physical interface in it. The purpose of this setup is the ability to monitor traffic passing through that interface using another machine. If the master CARP machine is disconnected from the network, the CARP interface on the backup machine correctly changes to the MASTER state, but does not act on traffic bound for the shared IP address--192.168.0.1. tcpdump shows the traffic coming in on the correct physical interface, but it is never replied to, or, in the case of routing, forwarded. Removing xl0 from the bridge on the backup machine instantly fixes this, and the shared IP address behaves as expected. Adding xl0 back to the bridge while the backup CARP interface is in the MASTER state keeps things running correctly, so the problem is only observed when xl0 is part of the bridge during the CARP transition from BACKUP to MASTER. Thoughts? -Boris