From owner-freebsd-net@FreeBSD.ORG Wed Jul 15 08:07:28 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B07201065670; Wed, 15 Jul 2009 08:07:28 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.delphij.net (delphij-pt.tunnel.tserv2.fmt.ipv6.he.net [IPv6:2001:470:1f03:2c9::2]) by mx1.freebsd.org (Postfix) with ESMTP id 235DB8FC21; Wed, 15 Jul 2009 08:07:28 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [211.166.10.233]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.delphij.net (Postfix) with ESMTPS id C36ED5C024; Wed, 15 Jul 2009 16:07:26 +0800 (CST) Received: from localhost (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 8BB4555CD816; Wed, 15 Jul 2009 16:07:26 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by localhost (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with ESMTP id qPfPZeVjOrPr; Wed, 15 Jul 2009 16:06:33 +0800 (CST) Received: from charlie.delphij.net (c-67-188-2-183.hsd1.ca.comcast.net [67.188.2.183]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 701F755CD7F5; Wed, 15 Jul 2009 16:06:26 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type; b=Ad4du1AWWyelHx/nx1jvhlE+MvJ8RA021+zjc7Wfud7LPJDpVEj6bZFnAQxge7wtp oQorrEgKTtcn8SEJ0+lHA== Message-ID: <4A5D8DF0.4010708@delphij.net> Date: Wed, 15 Jul 2009 01:06:08 -0700 From: Xin LI Organization: The FreeBSD Project User-Agent: Thunderbird 2.0.0.22 (X11/20090701) MIME-Version: 1.0 To: Glen Barber References: <4A5BD40E.9080108@delphij.net> <4ad871310907131747g6798c0b9j96a8ce5540f42289@mail.gmail.com> In-Reply-To: <4ad871310907131747g6798c0b9j96a8ce5540f42289@mail.gmail.com> X-Enigmail-Version: 0.95.7 OpenPGP: id=18EDEBA0; url=http://www.delphij.net/delphij.asc Content-Type: multipart/mixed; boundary="------------080001040707000106080403" Cc: freebsd-net@freebsd.org, glebius@FreeBSD.org, d@delphij.net Subject: Re: [LOR] carp vs bridge X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Jul 2009 08:07:29 -0000 This is a multi-part message in MIME format. --------------080001040707000106080403 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Here is a brute-force workaround for the issue, basically this would convert the hard deadlock into another problem (no multicast/broadcast being done on bridge, if CARP is enabled). It's good enough for my usage but is of course far from ideal. As I am not sure how much I could step further on a real fix due to $REALJOB, I'd like to share some further analysis for the problem. Basically, it was caused by CARP's use of locking and bridge's locking, the former would acquire CARP interface's mutex, while holding the mutex, it tries to indirectly call ether_output which, in turn, calls bridge_output if the CARP enabled interface is also part of a bridge. Since the bridge_output wants to acquire lock for the bridge's member interface, which is potentially parent for CARP interface, the order constraint is violated and thus a deadlock could happen. I think there is no obvious clean solution to break the dependency without weakening the realtimeness of sending the required multicast at this point. Will think again about it when I got some spare time. Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (FreeBSD) iEYEARECAAYFAkpdjfAACgkQi+vbBBjt66BxsACgggT8vhXo62V7Sh+2uAA0re2c dmEAnix+ax16obT2+neW7Iw0/P12cBVp =IM/r -----END PGP SIGNATURE----- --------------080001040707000106080403 Content-Type: text/plain; name="carp_bridge.diff" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="carp_bridge.diff" Index: if_ethersubr.c =================================================================== --- if_ethersubr.c (revision 195705) +++ if_ethersubr.c (working copy) @@ -394,8 +394,18 @@ * Bridges require special output handling. */ if (ifp->if_bridge) { +#if defined(INET) || defined(INET6) +#ifdef DEV_CARP + if ((m->m_flags & ~(M_MCAST | M_BCAST)) == m->m_flags) { +#endif +#endif BRIDGE_OUTPUT(ifp, m, error); return (error); +#if defined(INET) || defined(INET6) +#ifdef DEV_CARP + } +#endif +#endif } #if defined(INET) || defined(INET6) --------------080001040707000106080403--