From owner-freebsd-net@FreeBSD.ORG Sat Feb 8 01:13:02 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 864CFF4A; Sat, 8 Feb 2014 01:13:02 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5D6A41C60; Sat, 8 Feb 2014 01:13:02 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s181CugE082530 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 7 Feb 2014 17:12:56 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s181Cubw082529; Fri, 7 Feb 2014 17:12:56 -0800 (PST) (envelope-from jmg) Date: Fri, 7 Feb 2014 17:12:56 -0800 From: John-Mark Gurney To: Ryan Stone Subject: Re: flowtable, collisions, locking and CPU affinity Message-ID: <20140208011256.GH89104@funkthat.com> Mail-Followup-To: Ryan Stone , Adrian Chadd , FreeBSD Net , "freebsd-arch@freebsd.org" References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Fri, 07 Feb 2014 17:12:56 -0800 (PST) Cc: FreeBSD Net , Adrian Chadd , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Feb 2014 01:13:02 -0000 Ryan Stone wrote this message on Fri, Feb 07, 2014 at 20:05 -0500: > On Fri, Feb 7, 2014 at 7:12 PM, Adrian Chadd wrote: > > In any case - the reason it's happening above is because there's no > > actual lock held over the whole lookup/insert path. It's a per-CPU > > critical enter/exit path, so the only way to guarantee consistency is > > to use sched_pin() for the entirety of the function. > > sched_pin seems like a very heavy hammer for what has to be a very > rare and mostly harmless race. Why not redo the lookup after you have > reacquired the lock, and if you don't have to do the insert anymore > then don't and move on? Why not drop the work since the current CPU has the results? It sucks to throw away work, but the other option is to remeber what cpu you were on, and put it there, but that would also be expensive... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."