From owner-freebsd-net@FreeBSD.ORG  Thu Aug 15 03:19:02 2013
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTP id A205399;
 Thu, 15 Aug 2013 03:19:02 +0000 (UTC)
 (envelope-from luigi@onelab2.iet.unipi.it)
Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238])
 by mx1.freebsd.org (Postfix) with ESMTP id 602D32DF0;
 Thu, 15 Aug 2013 03:19:02 +0000 (UTC)
Received: by onelab2.iet.unipi.it (Postfix, from userid 275)
 id 576EA7300A; Thu, 15 Aug 2013 05:23:38 +0200 (CEST)
Date: Thu, 15 Aug 2013 05:23:38 +0200
From: Luigi Rizzo <rizzo@iet.unipi.it>
To: Peter Wemm <peter@wemm.org>
Subject: Re: it's the output, not ack coalescing (Re: TSO and FreeBSD vs Linux)
Message-ID: <20130815032338.GA74257@onelab2.iet.unipi.it>
References: <520A6D07.5080106@freebsd.org> <520AFBE8.1090109@freebsd.org>
 <520B24A0.4000706@freebsd.org> <520B3056.1000804@freebsd.org>
 <20130814102109.GA63246@onelab2.iet.unipi.it>
 <587579055.20130814154713@serebryakov.spb.ru>
 <CAJ-VmokxgWKeuPFGEE-wzLcYpyt3Ua90JZH2BBTj39LoTSUsMg@mail.gmail.com>
 <CAGE5yCrdptEAK7y_qBM3Og9U45o1_0ZtN0ANXg6WwXNTqX-1GQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAGE5yCrdptEAK7y_qBM3Og9U45o1_0ZtN0ANXg6WwXNTqX-1GQ@mail.gmail.com>
User-Agent: Mutt/1.5.20 (2009-06-14)
Cc: Lawrence Stewart <lstewart@freebsd.org>, Adrian Chadd <adrian@freebsd.org>,
 Lev Serebryakov <lev@freebsd.org>, FreeBSD Net <net@freebsd.org>
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 15 Aug 2013 03:19:02 -0000

On Wed, Aug 14, 2013 at 12:40:19PM -0700, Peter Wemm wrote:
> On Wed, Aug 14, 2013 at 11:11 AM, Adrian Chadd <adrian@freebsd.org> wrote:
> > On 14 August 2013 04:47, Lev Serebryakov <lev@freebsd.org> wrote:
> >
> >
> >>   And we should invalidate this info on ARP/route changes, or connection
> >>  will be lost in such cases, am I right?.. So, on each such event code
> >>  should look into all sockets and check, if routing/ARP information is
> >> still
> >>  valid for them. Or we should store lists of sockets in routing and ARP
> >>  tables... I don't know, what is worse.
> >>
> >
> > .. or per-CPU copies of the ARP table.. ?
> 
> Local cache at each consumer and check a generation number to see if
> it needs to be re-validated before using.  The obvious problem with
> this though is that big networks tend to kill your caches.

if you expect this to be problematic you can partition the entries
and use a different generation number per cluster.
Anyways if you really want to be guaranteed you need atomic
reads on the generation numbers (or ticks), which I have heard
are expensive on !i386/amd64 machines.

This is why I would probably try to live with races (which for
arp are a non problem).

	cheers
	luigi