From owner-freebsd-net@FreeBSD.ORG Fri May 30 04:12:28 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AD8B8663 for ; Fri, 30 May 2014 04:12:28 +0000 (UTC) Received: from mail-pb0-x229.google.com (mail-pb0-x229.google.com [IPv6:2607:f8b0:400e:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 817512A62 for ; Fri, 30 May 2014 04:12:28 +0000 (UTC) Received: by mail-pb0-f41.google.com with SMTP id uo5so1271616pbc.0 for ; Thu, 29 May 2014 21:12:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=CTAiAj3QanUqGgcwuksGu4rvTNqBiBS5/MttvsQuizw=; b=z4DnJEO4QXwXhYv9raQPPmkVikxsEVLp8n3lBoRW0Bc55dxoFOQo9Tk7I6vR2e/Aul x9bQJpKNK/eEB5XPYq6HhRXoctnboInsIGLymTQuiL6cOcHMIAikrWJ7To8BHyat2vyU UFY5QAP7BVr+T0VEJAJzTs38JZuz8yOsY++BjwvWgibAj8WNfUq7bxwvnuA6Te9WmB35 mbVCvT0zU/8XmB3ThQozL/sIQHCRqJcN0L6mxc6cakFKH6fmSyKM85HfHxDDQH10lK0+ q69i2wmZ2xiux+xSgrGGVkxv4zno+fgpHSraUu3rstOgzhNqTA1ts73WENZZtxY8oJBV Bq9g== X-Received: by 10.68.197.195 with SMTP id iw3mr14755570pbc.139.1401423148059; Thu, 29 May 2014 21:12:28 -0700 (PDT) Received: from kmatoMacBook-Pro.local ([27.24.140.240]) by mx.google.com with ESMTPSA id i10sm12233536pat.36.2014.05.29.21.12.25 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 29 May 2014 21:12:27 -0700 (PDT) Message-ID: <53880525.6000203@gmail.com> Date: Fri, 30 May 2014 12:12:21 +0800 From: k simon User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: TCP stack lock contention with short-lived connections References: <537F39DF.1090900@verisign.com> <537FB51D.2060401@verisign.com> <53861209.2000306@verisign.com> In-Reply-To: <53861209.2000306@verisign.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 May 2014 04:12:28 -0000 Hi, Does any plan commit and MFC to the 10-stable ? Regards Simon 于 14-5-29 0:42, Julien Charbon 写道: > > Hi, > > On 23/05/14 22:52, Julien Charbon wrote: >> On 23/05/14 14:06, Julien Charbon wrote: >>> On 27/02/14 11:32, Julien Charbon wrote: >>>> On 07/11/13 14:55, Julien Charbon wrote: >>>>> On Mon, 04 Nov 2013 22:21:04 +0100, Julien Charbon >>>>> wrote: >>>>>> I have put technical and how-to-repeat details in below PR: >>>>>> >>>>>> kern/183659: TCP stack lock contention with short-lived connections >>>>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=183659 >>>>>> >>>>>> We are currently working on this performance improvement >>>>>> effort; it >>>>>> will impact only the TCP locking strategy not the TCP stack logic >>>>>> itself. We will share on freebsd-net the patches we made for >>>>>> reviewing and improvement propositions; anyway this change might >>>>>> also >>>>>> require enough eyeballs to avoid tricky race conditions introduction >>>>>> in TCP stack. >> >> Joined the two cumulative patches (tcp-scale-inp-list-v1.patch and >> tcp-scale-pcbinfo-rlock-v1.patch) we discussed the most at BSDCan 2014. > > At BSDCan 2014 we were also asked to provide flame graph [1][2] to > highlight impacts of these TCP changes. The Dtrace sampling was done on > a NIC receive queue IRQ bound core. > > o First CPU flame graph on 10.0-RELENG at 40k TCP connection/secs: > > https://googledrive.com/host/0BwwgoN552srvQi1JWG42TklfQ28/releng10-40k.html > > Note: > > - __rw_wlock_hard on ipi_lock contention is clear as usual. > > o Second, same test with all our patches applied (thus from 10.0-next > branch [3]): > > https://googledrive.com/host/0BwwgoN552srvQi1JWG42TklfQ28/tcp-scale-40k.html > > > Note: > > - Almost all __rw_wlock_hard on ipi_lock contention is converted in > idle time. > > o Third, still using 10.0-next branch, the flame graph when doubling > the rate to 80k TCP connection/sec: > > https://googledrive.com/host/0BwwgoN552srvQi1JWG42TklfQ28/tcp-scale-80k.html > > > My 2 cents. > > [1] http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html > [2] https://wiki.freebsd.org/201405DevSummit/NetworkStack > [3] https://github.com/verisign/freebsd/commits/share/10.0-next > > -- > Julien > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"