From owner-freebsd-net@FreeBSD.ORG Mon Jul 25 08:59:56 2011 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 949E4106564A; Mon, 25 Jul 2011 08:59:56 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.64.117]) by mx1.freebsd.org (Postfix) with ESMTP id 1A0E58FC0A; Mon, 25 Jul 2011 08:59:55 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.4/8.14.4) with ESMTP id p6P8xsNJ002167; Mon, 25 Jul 2011 12:59:54 +0400 (MSD) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.4/8.14.4/Submit) id p6P8xsw3002166; Mon, 25 Jul 2011 12:59:54 +0400 (MSD) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Mon, 25 Jul 2011 12:59:54 +0400 From: Gleb Smirnoff To: "Robert N. M. Watson" Message-ID: <20110725085954.GR63969@glebius.int.ru> References: <20110714154457.GI70776@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: gnn@FreeBSD.org, bz@FreeBSD.org, Ryan Stone , net@FreeBSD.org Subject: Re: m_pkthdr.rcvif dangling pointer problem X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 08:59:56 -0000 On Sun, Jul 24, 2011 at 09:43:59AM +0100, Robert N. M. Watson wrote: R> Instead, I think we should go for a more radical notion, which is a bit = harder to implement in our stack: the network stack needs a race-free way t= o "drain" all mbufs referring to a particular ifnet, which does not cause e= xisting processing to become more expensive. This is easy in some subsystem= s, but more complex in others -- and the composition of subsystems makes it= all much harder since we need to know that (to be 100% correct) packets ar= en't getting passed between subsystems (and hence belong to neither) in a w= ay that races with a sweep through the subsystems. It may be possible to ge= t this 99.9% right simply by providing a series of callbacks into subsystem= s that cause queues to be walked and drained of packets matching the doomed= ifnet. It may also be quite cheap to have subsystems that "hold" packets o= utside of explicit queues for some period (i.e., in a thread-local pointer = out of the stack) add explicit invalidation tests (i.e., for IFF_DYING) be= fore handing off to prevent those packets from traversing into other subsys= tems -- which can be done synchronisation-free, but still wouldn't 100% pre= vent the race R>=20 R> Just to give an example: netisr should offer a method for netisr_drain_i= fnet(struct ifnet *) that causes netisr to walk all of its queues to find m= atching packets and free them. Due to direct dispatch and thread-local queu= es during processing, netisr should also check IFF_DYING before handing off. R>=20 R> If we do that, I wonder how robust the system then becomes...? This may = not be too hard to test. But I'd rather we penalise ifnet removal than, say= , the IP input path when it needs to check a source interface property. What if some thread (e.g. netisr) have taken an mbuf off the queue, stored = pointer to it on stack, and got rescheduled. Then an ifnet has departured, all call= backs were called, all queues cleaned up from pointers. Then a thread with that m= buf on=20 stack continues to process. ? --=20 Totus tuus, Glebius.