From owner-freebsd-ipfw@FreeBSD.ORG  Sun Mar  2 15:10:13 2008
Return-Path: <owner-freebsd-ipfw@FreeBSD.ORG>
Delivered-To: freebsd-ipfw@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 4CCD21065673;
	Sun,  2 Mar 2008 15:10:13 +0000 (UTC)
	(envelope-from piso@southcross.wired.org)
Received: from mail.oltrelinux.com (krisma.oltrelinux.com [194.242.226.43])
	by mx1.freebsd.org (Postfix) with ESMTP id E97BC8FC12;
	Sun,  2 Mar 2008 15:10:12 +0000 (UTC)
	(envelope-from piso@southcross.wired.org)
Received: from southcross.wired.org (host-62-10-34-34.cust-adsl.tiscali.it
	[62.10.34.34])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mail.oltrelinux.com (Postfix) with ESMTP id 4C52011AE68;
	Sun,  2 Mar 2008 16:10:12 +0100 (CET)
Received: (from piso@localhost)
	by southcross.wired.org (8.14.2/8.14.1/Submit) id m22FDBa0023669;
	Sun, 2 Mar 2008 16:13:11 +0100 (CET) (envelope-from piso)
Date: Sun, 2 Mar 2008 16:13:10 +0100
From: Paolo Pisati <piso@freebsd.org>
To: Paolo Pisati <piso@freebsd.org>
Message-ID: <20080302151310.GB23353@tin.it>
References: <20080302144939.GA23353@tin.it>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20080302144939.GA23353@tin.it>
User-Agent: Mutt/1.5.17 (2007-11-01)
X-Virus-Scanned: by amavisd-new-20030616-p10 (Debian) at krisma.oltrelinux.com
Cc: freebsd-ipfw@freebsd.org, freebsd-net@freebsd.org
Subject: Re: ipfw initialization: SI_ORDER_ANY -> SI_ORDER_MIDDLE?
X-BeenThere: freebsd-ipfw@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: IPFW Technical Discussions <freebsd-ipfw.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw>,
	<mailto:freebsd-ipfw-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-ipfw>
List-Post: <mailto:freebsd-ipfw@freebsd.org>
List-Help: <mailto:freebsd-ipfw-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw>,
	<mailto:freebsd-ipfw-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 02 Mar 2008 15:10:13 -0000

On Sun, Mar 02, 2008 at 03:49:39PM +0100, Paolo Pisati wrote:
> Hi,
> 
> i just found out that depending on a KLD doesn't imply any
> initialization order, thus depending on a lock initialized in the ipfw
> init path is _really_ a bad idea from another KLD init path (see
> ip_fw_nat.c::ipfw_nat_init()).
> 
> A fix would be to move ipfw init priority from SI_ORDER_ANY to
> SI_ORDER_MIDDLE, but i guess there are side effects that i'm
> unaware in this modification...
> 
> On the other hand, if we keep ipfw at SI_ORDER_ANY, i don't know how
> to build stuff that relies on it without opening race conditions:
> see ip_fw_nat.c::flush_nat_ptrs() called in rule deletion and
> rule configuration paths.

as the problem arises only during ip_fw_nat initialization, another
viable solution would be to fix ipfw_nat_init() this way:

static void
ipfw_nat_init(void)
{

-        IPFW_WLOCK(&layer3_chain);
         /* init ipfw hooks */
-        ipfw_nat_ptr = ipfw_nat;
         ipfw_nat_cfg_ptr = ipfw_nat_cfg;
         ipfw_nat_del_ptr = ipfw_nat_del;
         ipfw_nat_get_cfg_ptr = ipfw_nat_get_cfg;
         ipfw_nat_get_log_ptr = ipfw_nat_get_log;
-        IPFW_WUNLOCK(&layer3_chain);
+	 ipfw_nat_ptr = ipfw_nat;
         ifaddr_event_tag = EVENTHANDLER_REGISTER(ifaddr_event, ifaddr_change, 
             NULL, EVENTHANDLER_PRI_ANY);
}

avoid grabbing the lock at all during init, and exploit orders of
hooks initialization: as the presence of ipfw_nat in ipfw is checked
via ipfw_nat_ptr, i can narrow/close the race window initializing
ipfw_nat_ptr at the end of the function, but can i trust the order of
variables assignment? i guess no without some sort of memory barriers,
and are memory barriers available in all archs? and are memory
barriers enough?

bye,
P.