From owner-svn-src-all@freebsd.org Fri Aug 26 21:42:29 2016 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5BA98B76A44; Fri, 26 Aug 2016 21:42:29 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-it0-x232.google.com (mail-it0-x232.google.com [IPv6:2607:f8b0:4001:c0b::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 18F64DE3; Fri, 26 Aug 2016 21:42:29 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-it0-x232.google.com with SMTP id g62so13574753ith.1; Fri, 26 Aug 2016 14:42:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=UZEs+/nkExghu/GnwI3muBgOshHjMbyqlyXbomvnpvU=; b=qH1Ot8FvZRD2dEUF72Q5I66TT14WE46BrJzsEnuYJINEfjYjuP/QlDohLhVj6iwxb6 QBEW55XWpyEoshahIEYboKG5lGVAQc3KmIUX8hQQMcWtpv7OiyrLIlwV46jgmfQUNQMw 0J6DRfJqLtTafxhIbwEthDwKq115SbH6j4bErgnkPp/AcjHf4E52xKVoi6F87wZHG0zg x3n0F/nqN5of5tLBX+ymx0XC7UjMkvIUJVIn/3rAwPwEGc6hYxbEHydVrszUi0Sk7L2V toHWD4LKaewY0folu9HqaDYQhOut/zCH/sMyz4GptRVMG0H2yLWOVCvlXgF5QAjF25IV nkYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=UZEs+/nkExghu/GnwI3muBgOshHjMbyqlyXbomvnpvU=; b=K42h63YTslchm3mOdHkoVj+jZ/ZCgRdbBvKg8BXHqvpntHj6+jSEqAksp2VpmptuLT Wif15RH2pFPMp9L9pAeI1Aau9AwIBM0KJFIhUgaWI/vQqJGof3Gyg+1LE7QY+BkcV4AN 7hCYCrgYgOhJFoq27LE5I0dfUMaGlj2jG4tX2VTPDYBPE5m7uCCZUue4/o6BP6pnC94c GS5d3k5jwPdhK8eUsmO+ZxoQSKkLfNDQ7tY/XDWVshcbKjwkz6+67U8MuC0J8cFyAYF3 B/nRtzVaZ6S5JERS6xjgjBNZwFuHk8atcjTkwoxRAsJNVn9PP0Nwozx0uGvGDz/g7l16 Srag== X-Gm-Message-State: AE9vXwMGXSEoCP9BIG+cJWIe+y2tGyw2W4/MgNMXkTHW9FW4CZOPhFdSZ8fVsiSY/ot0XidM3DpFLS4YDVsEOg== X-Received: by 10.107.15.229 with SMTP id 98mr6449397iop.123.1472247748515; Fri, 26 Aug 2016 14:42:28 -0700 (PDT) MIME-Version: 1.0 Sender: adrian.chadd@gmail.com Received: by 10.36.141.129 with HTTP; Fri, 26 Aug 2016 14:42:27 -0700 (PDT) In-Reply-To: <20160826213613.GH88122@zxy.spb.ru> References: <0f42c5fb-f930-c6e3-75d6-df97f67c201d@fastmail.net> <20160820204106.GW8192@zxy.spb.ru> <0acba141-4701-d9c2-0ddb-46d1f60ff55b@fastmail.net> <20160820220510.GX8192@zxy.spb.ru> <8ac23bd1-dcb3-7c64-f195-5039f9af0eaf@fastmail.net> <20160821000400.GY8192@zxy.spb.ru> <20160826144926.GE88122@zxy.spb.ru> <3dba1b70-54cc-0bb1-5cc8-8c56cd750bec@fastmail.net> <20160826151324.GF88122@zxy.spb.ru> <20160826213613.GH88122@zxy.spb.ru> From: Adrian Chadd Date: Fri, 26 Aug 2016 14:42:27 -0700 X-Google-Sender-Auth: glQGm2oRtk38f40WrA2sVTme230 Message-ID: Subject: Re: svn commit: r304436 - in head: . sys/netinet To: Slawa Olhovchenkov Cc: Bruce Simpson , Ryan Stone , "svn-src-head@freebsd.org" , Ryan Stone , "src-committers@freebsd.org" , "svn-src-all@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2016 21:42:29 -0000 On 26 August 2016 at 14:36, Slawa Olhovchenkov wrote: > On Fri, Aug 26, 2016 at 02:32:00PM -0700, Adrian Chadd wrote: > >> Hi, >> >> It's pcb lock contention. > > Not sure: only 5% of all time. > And same 5% for tcbhashsize = 65K and 256K. > Or you talk about some more thin effect? You're in the inpcb lock from multiple places. the tcbhashsize doesnt influence the pcb lock contention - it just affects how long you take doing lookups. iF your hash table is too small then you end up doing lots of O(n) walks of a hash bucket to find a pcb entry. :) -adrian >> >> On 26 August 2016 at 08:13, Slawa Olhovchenkov wrote: >> > On Fri, Aug 26, 2016 at 04:01:14PM +0100, Bruce Simpson wrote: >> > >> >> Slawa, >> >> >> >> I'm afraid this may be a bit of a non-sequitur. Sorry.. I seem to be >> >> missing something. As I understand it this thread is about Ryan's change >> >> to netinet for broadcast. >> >> >> >> On 26/08/16 15:49, Slawa Olhovchenkov wrote: >> >> > On Sun, Aug 21, 2016 at 03:04:00AM +0300, Slawa Olhovchenkov wrote: >> >> >> On Sun, Aug 21, 2016 at 12:25:46AM +0100, Bruce Simpson wrote: >> >> >>> Whilst I agree with your concerns about multipoint, I support the >> >> >>> motivation behind Ryan's original change: optimize the common case. >> >> >> >> >> >> Oh, common case... >> >> >> I am have pmc profiling for TCP output and see on this SVG picture and >> >> >> don't find any simple way. >> >> >> You want to watch too? >> >> > >> >> > At time peak network traffic (more then 25K connections, about 20Gbit >> >> > total traffic) half of cores fully utilised by network stack. >> >> > >> >> > This is flamegraph from one core: http://zxy.spb.ru/cpu10.svg >> >> > This is same, but stack cut of at ixgbe_rxeof for more unified >> >> > tcp/ip stack view http://zxy.spb.ru/cpu10u.svg >> >> ... >> >> >> >> I appreciate that you've taken the time to post a flamegraph (a >> >> fashionable visualization) of relative performance in the FreeBSD >> >> networking stack. >> >> >> >> Sadly, I am mostly out of my depth for looking at stack wide performance >> >> for the moment; for the things I look at involving FreeBSD at work just >> >> at the moment, I would not generally go down there except for specific >> >> performance issues (e.g. with IEEE 1588). >> >> >> >> It sounds as though perhaps you should raise a wider discussion about >> >> your results on -net. I would caution you however that the Function >> >> Boundary Trace (FBT) provider for DTrace can introduce a fair amount of >> >> noise to the raw performance data because of the trap mechanism it uses. >> >> This ruled it out for one of my own studies requiring packet-level accuracy. >> >> >> >> Whilst raw pmc(4) profiles may require more post-processing, they will >> >> provide less equivocal data (and a better fix) on the hot path, due also >> >> to being sampled effectively on a PMC interrupt (a gather stage- poll >> >> core+uncore MSRs), not purely a software timer interrupt. >> > >> > Thanks for answer, I am now try to start discussion on -net.