From owner-freebsd-net@FreeBSD.ORG Sat Aug 24 17:04:35 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A6C671EC; Sat, 24 Aug 2013 17:04:35 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-we0-x229.google.com (mail-we0-x229.google.com [IPv6:2a00:1450:400c:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0269228F2; Sat, 24 Aug 2013 17:04:34 +0000 (UTC) Received: by mail-we0-f169.google.com with SMTP id t61so1563724wes.28 for ; Sat, 24 Aug 2013 10:04:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=rei4Wwr4OIgvQo2wybC1i+Az3jxGshCn2mkt5ZWHf28=; b=MHhgHOhiPpSGOV5NWf5ys1WzoFPzETfQwxOPPQO9wM4/P4RfhkK3y/BBipkCSArz83 s7GvowkAD8uV9LUd25LXM4ZnChPntLJyFq2tJ//BQCxyKOfu8aWUOwYPzrHUuT4z7hHs qYOwtVLupb2RGs1KaUCoI0BAjDnBZCusNYDMzuDBc5AHzGZFc3Ts9ak86Kw2nWgVKC4C v5/c+/6TJQQ29Qc1jZukPsb9Eii+YSuA9ymhl8/8YfHmoH7gDHPbR67GVxZWlGLtyfLf /1VCS5Cp8uM8fYXc9dsjWiatAPhtm8GUac4HViAF8eMxhvm9iDgWtw0uWXjz4/j53z+q bpoA== MIME-Version: 1.0 X-Received: by 10.194.20.170 with SMTP id o10mr3922186wje.4.1377363873200; Sat, 24 Aug 2013 10:04:33 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.217.116.136 with HTTP; Sat, 24 Aug 2013 10:04:33 -0700 (PDT) In-Reply-To: <5218ABB4.5070601@ipfw.ru> References: <520A6D07.5080106@freebsd.org> <520B74DD.1060102@ipfw.ru> <20130814124024.GA64548@onelab2.iet.unipi.it> <201308141740.28779.zec@fer.hr> <20130814154853.GA66341@onelab2.iet.unipi.it> <521204A9.7080607@ipfw.ru> <52152837.9010101@freebsd.org> <5218ABB4.5070601@ipfw.ru> Date: Sat, 24 Aug 2013 10:04:33 -0700 X-Google-Sender-Auth: XPTVFUcPmvoSzQCW_iTprUOYriU Message-ID: Subject: Re: route/arp lifetime (Re: it's the output, not ack coalescing (Re: TSO and FreeBSD vs Linux)) From: Adrian Chadd To: "Alexander V. Chernikov" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Net , Andre Oppermann X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Aug 2013 17:04:35 -0000 Hi! Can you give us actual numbers? Something that we can easily graph? Do you have a diff against -HEAD? I'd like to stare at it a bunch and think about merging some stuff in. :-) How are you testing it? Is it something I can set up in our lab and thoroughly thrash? I'm very close to starting an mbuf batching thing to use in a few places like receive, transmit and transmit completion -> free path. I'd be interested in your review/feedback and testing as it sounds like something you can easily stress test there. :) Thanks, -Adrian On 24 August 2013 05:48, Alexander V. Chernikov wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 22.08.2013 00:51, Andre Oppermann wrote: > > On 19.08.2013 13:42, Alexander V. Chernikov wrote: > >> On 14.08.2013 19:48, Luigi Rizzo wrote: > >>> On Wed, Aug 14, 2013 at 05:40:28PM +0200, Marko Zec wrote: > >>>> On Wednesday 14 August 2013 14:40:24 Luigi Rizzo wrote: > >>>>> On Wed, Aug 14, 2013 at 04:15:25PM +0400, Alexander V. > >>>>> Chernikov wrote: > >>> ... > >>>> FWIW, apparently we already have that infrastrucure in place > >>>> - if_rele() calls if_free_internal() only when the last > >>>> reference to the ifnet is dropped, so with little care this > >>>> should be usable for caching ifp pointers w/o fears for > >>>> kernel crashes mentioned above. > >>> maybe Alexander was referring to holding references to the rte > >>> entries returned as a result of the lookup. The rte holds a > >>> reference to the ifp. > >> > >> Yes. Since there is the only refcount which is protected (and is > >> also a huge performance killer). > >> > >> Btw, there is a picture describing IPv4 packet flow from my > >> still-not-written post related network stack performance, maybe > >> it can be useful: > >> http://static.ipfw.ru/images/freebsd_ipv4_flow.png > > > > Wow, that's really cool. Please note that a rmlock doesn't cost > > anything for the read case (unless contended of course). Whereas > > normal rlocks or > We're running this entire stack without singe rwlock (everything is > either converted to rmlock or using lockless data copies with delayed > GC (in_adrr_local and other similar)). It really is fasters, but, > however, due to current process-to-completion routing architecture > this is limited to 5-6MPPS for 12 cores on 2xE5645. > > > rwlocks write to the lock memory location and cause atomic bus lock > > cycles as well as a lot of cache line invalidations across cores. > > The same is true for refcounts. > > > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.20 (FreeBSD) > Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ > > iEYEARECAAYFAlIYq7QACgkQwcJ4iSZ1q2nFZwCfZLckg4b/iny2CK+bYJa20XxE > y7UAnRZHVr4AZRYnB8acrN54KtRMpvNQ > =0kPb > -----END PGP SIGNATURE----- > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >