From owner-freebsd-net@FreeBSD.ORG Fri Dec 14 15:28:14 2007 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 43B5B16A417; Fri, 14 Dec 2007 15:28:14 +0000 (UTC) (envelope-from mux@freebsd.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 30B8113C455; Fri, 14 Dec 2007 15:28:13 +0000 (UTC) (envelope-from mux@freebsd.org) Received: by elvis.mu.org (Postfix, from userid 1920) id B7B691A4D86; Fri, 14 Dec 2007 07:27:17 -0800 (PST) Date: Fri, 14 Dec 2007 16:27:17 +0100 From: Maxime Henrion To: Gleb Smirnoff Message-ID: <20071214152717.GG71713@elvis.mu.org> References: <20071213133817.GC71713@elvis.mu.org> <47617AF5.7070701@elischer.org> <20071214092539.GB14339@glebius.int.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20071214092539.GB14339@glebius.int.ru> User-Agent: Mutt/1.4.2.3i Cc: Julian Elischer , net@FreeBSD.org Subject: Re: Deadlock in the routing code X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Dec 2007 15:28:14 -0000 Gleb Smirnoff wrote: > On Thu, Dec 13, 2007 at 10:33:25AM -0800, Julian Elischer wrote: > J> Maxime Henrion wrote: > J> > Replying to myself on this one, sorry about that. > J> > I said in my previous mail that I didn't know yet what process was > J> > holding the lock of the rtentry that the routed process is dealing > J> > with in rt_setgate(), and I just could verify that it is held by > J> > the swi1: net thread. > J> > So, in a nutshell: > J> > - The routed process does its business on the routing socket, that ends up > J> > calling rt_setgate(). While in rt_setgate() it drops the lock on its > J> > rtentry in order to call rtalloc1(). At this point, the routed > J> > process hold the gateway route (rtalloc1() returns it locked), and it > J> > now tries to re-lock the original rtentry. > J> > - At the same time, the swi net thread calls arpresolve() which ends up > J> > calling rt_check(). Then rt_check() locks the rtentry, and tries to > J> > lock the gateway route. > J> > A classical case of deadlock with mutexes because of different locking > J> > order. Now, it's not obvious to me how to fix it :-). > J> > J> On failure to re-lock, the routed call to rt_setgate should completely abort > J> and restart from scratch, releasing all locks it has on the way out. > > Do you suggest mtx_trylock? I actually have the beginning of a patch that uses mtx_trylock(), wrapped into a RT_TRYLOCK() macro. It certainly isn't very pretty, but if that can help me having a workaround, that'd still be useful. It really seems like the real fix would invovle a fair amount of rewrite and analysis of the current code, so... I have yet to find time to finish it, build-test it, and run-test it. Did you got any farther in the rt_check() cleanup you've been telling me about? Cheers, Maxime