From owner-freebsd-arch@FreeBSD.ORG  Sat Feb  8 01:05:58 2014
Return-Path: <owner-freebsd-arch@FreeBSD.ORG>
Delivered-To: freebsd-arch@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id CE638CF3;
 Sat,  8 Feb 2014 01:05:58 +0000 (UTC)
Received: from mail-oa0-x22e.google.com (mail-oa0-x22e.google.com
 [IPv6:2607:f8b0:4003:c02::22e])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id 7DD521BD7;
 Sat,  8 Feb 2014 01:05:58 +0000 (UTC)
Received: by mail-oa0-f46.google.com with SMTP id n16so5145615oag.33
 for <multiple recipients>; Fri, 07 Feb 2014 17:05:57 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:in-reply-to:references:date:message-id:subject:from:to
 :cc:content-type;
 bh=X/E1R8j29/YJMQ0lt+NFFkfYeCo4xQyRTjCfcZ1UOKo=;
 b=0sBtERLIcX6RnMwD6GItOfhud47wuXL0TRTBQIqao8Cy48z+sdfyYKP9YQY2Nz5tP9
 2A6EulqE6I6iyC4S/LdxeJFzB8u/k18yzAYBrvYbkpnuQkGR3nZ1x7CwZr/UU2ZXeaRq
 hiOkIpGiPEtYyyuoZO7nnUW+Ucedae7ZjKJV6zOJkn6J+VbsHRGuQoT3dTXWsHljcpJQ
 tUNeKz2IW7TIsW0GliOQXKWQ12oVrxLXj5IAcxrNCaHvI/76Vyg4xIJNLlwT86DjpkNS
 R7Zd2yMCLQ/RuD+THPBJ2aJwXs8p8psINaalTx4z+8o+06IYtfuFCKzn9L1v9r07M1xB
 gexw==
MIME-Version: 1.0
X-Received: by 10.182.225.137 with SMTP id rk9mr14688254obc.51.1391821557788; 
 Fri, 07 Feb 2014 17:05:57 -0800 (PST)
Received: by 10.76.130.196 with HTTP; Fri, 7 Feb 2014 17:05:57 -0800 (PST)
In-Reply-To: <CAJ-VmonNCzFED=20_C2fV1g1jvFNRE=N-H+09Wb2OdxdzHp9JQ@mail.gmail.com>
References: <CAJ-VmonNCzFED=20_C2fV1g1jvFNRE=N-H+09Wb2OdxdzHp9JQ@mail.gmail.com>
Date: Fri, 7 Feb 2014 20:05:57 -0500
Message-ID: <CAFMmRNyYFte0Q5Jhw0A3Fkya9d8hQ-22pRHYZE62pB_wKb9Caw@mail.gmail.com>
Subject: Re: flowtable, collisions, locking and CPU affinity
From: Ryan Stone <rysto32@gmail.com>
To: Adrian Chadd <adrian@freebsd.org>
Content-Type: text/plain; charset=ISO-8859-1
Cc: FreeBSD Net <freebsd-net@freebsd.org>,
 "freebsd-arch@freebsd.org" <freebsd-arch@freebsd.org>
X-BeenThere: freebsd-arch@freebsd.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Discussion related to FreeBSD architecture <freebsd-arch.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-arch>,
 <mailto:freebsd-arch-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-arch/>
List-Post: <mailto:freebsd-arch@freebsd.org>
List-Help: <mailto:freebsd-arch-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-arch>,
 <mailto:freebsd-arch-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 08 Feb 2014 01:05:58 -0000

On Fri, Feb 7, 2014 at 7:12 PM, Adrian Chadd <adrian@freebsd.org> wrote:
> In any case - the reason it's happening above is because there's no
> actual lock held over the whole lookup/insert path. It's a per-CPU
> critical enter/exit path, so the only way to guarantee consistency is
> to use sched_pin() for the entirety of the function.

sched_pin seems like a very heavy hammer for what has to be a very
rare and mostly harmless race.  Why not redo the lookup after you have
reacquired the lock, and if you don't have to do the insert anymore
then don't and move on?