From owner-freebsd-net@FreeBSD.ORG Thu Jan 25 21:06:04 2007 Return-Path: X-Original-To: net@FreeBSD.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0AD0B16A404; Thu, 25 Jan 2007 21:06:04 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 9CFB013C44C; Thu, 25 Jan 2007 21:06:03 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 624134B89D; Thu, 25 Jan 2007 15:40:52 -0500 (EST) Date: Thu, 25 Jan 2007 20:40:52 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Gleb Smirnoff In-Reply-To: <20070125183720.GB7922@cell.sick.ru> Message-ID: <20070125203807.S13293@fledge.watson.org> References: <20070125162422.GA7922@bestcom.ru> <45B8EB23.705@FreeBSD.org> <20070125183720.GB7922@cell.sick.ru> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: "Bruce M. Simpson" , net@FreeBSD.org Subject: Re: rev. 1.94 of netinet/in.c broke CARP X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jan 2007 21:06:04 -0000 On Thu, 25 Jan 2007, Gleb Smirnoff wrote: > B> > Before this change, most of the subsystems, that allocated multicast > B> >membership instances had freed is theirselves. I don't know about others, > B> >but at least CARP is broken now. It attempts to free a memory, that > B> >already has been freed. > B> > > B> I would suggest that the correct fix, for now, would be for carp(4) to > B> now *not* perform its own cleanup for the IPv4 groups it joins on member > B> interfaces. > > Unfortunately, this won't be a correct fix. In a scenario when the parent > interface stays on its place, but you are creating, attaching and destroying > a CARP interface, the multicast membership would not be left and memory > won't be freed. So, after the following sequence > > ifconfig fxp0 10.0.0.1/24 > ifconfig carp0 create > ifconfig carp0 vhid 1 10.0.0.2/24 > ifconfig carp0 destroy > > , we would still have a multicast membership on fxp0. > > B> The symptom here is that carp(4) needs to join a multicast group on its > B> member interface. When the interface goes away, the group membership is > B> now destroyed, at the netinet global level, by the netinet detach path B> > first. B> B> However, carp(4) is keeping its own imo_membership vector of > the B> addresses it joined on its member interfaces (rather than using the > one B> which netinet assigns to it in its attach path), and later tries to > free B> these memberships. B> B> netinet6 does not have the same problem > because in6 memberships are B> reference counted. B> B> The root problem is > that we should be using consistent semantics for B> both the IPv4 and IPv6 > paths, and the kernel APIs where soft-ifnets B> (such as carp(4)) and > routing code (such as MROUTING) need to manipulate B> multicast group > memberships. Architecturally, the right fix is that CARP needs to have a handler for ifnet destruction that always runs before the multicast address garbage collection. I'm pretty preoccupied for the next few days due to an impending paper deadline, so can't investigate further currently, but one way or the other that ordering dependency needs to be expressed. If done properly, CARP will always have released its multicast address before they are forceably removed. Having the reference count is good too, but what I describe should be sufficient regardless of the refcount. Robert N M Watson Computer Laboratory University of Cambridge