From owner-svn-src-head@freebsd.org Sun Jan 5 08:35:00 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1E7231F339C for ; Sun, 5 Jan 2020 08:35:00 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47rBlk6LwJz43K2 for ; Sun, 5 Jan 2020 08:34:58 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: by mail-pj1-x102a.google.com with SMTP id a6so6295705pjh.2 for ; Sun, 05 Jan 2020 00:34:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jroberson-net.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version; bh=+vfTMhoycwapEZ+lyJ5+TT8Ek8FG4ALeeIW9XxUxs+k=; b=R5cvxoU4vP5AapabaLT8+X0jHXFCL40V+0t6WDcq0kZADothMVIm5KH5V4enuoV0P0 A04KCrZ+g6BqZRqnShVZLKI3KdSsrNoMFzlrTZHRjmd34RwRVeo06CmYqHBLxxOOsjWB fXQp62zZ8C0ZFO0B3lL1XZvWAGBH9CcvTh988Aka0YuMQx/rBRCTZCqkaLnWcYSy8dGl ykT+bRVHNeC724zNcE+4wi9uLz657TXGZYP8KQHyBdD7ssJZLJdMGBeelZDj48yq/R+G EiN+wPRiiCWauR7cKTJbgXUwGV4t6764a8itWf39oAah2vbHFdVvgQlUMNV3o3mCpSlu 6rZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:in-reply-to:message-id :references:user-agent:mime-version; bh=+vfTMhoycwapEZ+lyJ5+TT8Ek8FG4ALeeIW9XxUxs+k=; b=EUGWrCyXXcEJTZ8EITu9NWxvhNEaIt8RivMU0xI1GJ2lf/ln99ZlcBkWNJ9z5tmsxr 9b71LLKJwa+bGYW7FObLCmEYJvTTcSf9KELF1zv0f0PJzvrDMWaQttr07cG8I3oXZsU5 s10NRausoLE5kQ0+/eWZyB3B7BEV7X5Aj5YTcDSuOgbST9XE7n26QPsCmxQtSU6qvvZ4 tyC1FIMRLi0Y1TOpQ7HUsUA1pKnnbTkswYAs1H537h6LwZf1WJNLlwPjmJMahON3cT0M 2Twrgvn8Y8BLsKMJqxiRJU96MaUnIMQ8hAJOhYu17QxOtMbWzUBhI2uZGOBZNVJiCOb0 KqxQ== X-Gm-Message-State: APjAAAUwTthI1kTiZtwECg0zPj+V0J4FzW51m5e+0qOOfRUaNUS0RhBh vku6jLAwYqS23hSH5TVuJ5sYog== X-Google-Smtp-Source: APXvYqx+OvnGcaE4sqlsW4Xr4o72rQVubY1zFW+htY8g08jrgEy6ez1T0Y30lUZn/QQtAYn243LK7A== X-Received: by 2002:a17:90a:33ad:: with SMTP id n42mr36177086pjb.23.1578213291975; Sun, 05 Jan 2020 00:34:51 -0800 (PST) Received: from rrcs-76-81-105-82.west.biz.rr.com (rrcs-76-81-105-82.west.biz.rr.com. [76.81.105.82]) by smtp.gmail.com with ESMTPSA id 2sm66910766pgo.79.2020.01.05.00.34.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 05 Jan 2020 00:34:51 -0800 (PST) Date: Sat, 4 Jan 2020 22:34:49 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: Mark Linimon cc: Jeff Roberson , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r356348 - in head/sys: kern vm In-Reply-To: <20200105013314.GA3681@lonesome.com> Message-ID: References: <202001040315.0043FYhn047977@repo.freebsd.org> <20200105013314.GA3681@lonesome.com> User-Agent: Alpine 2.21.9999 (BSF 287 2018-06-16) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Rspamd-Queue-Id: 47rBlk6LwJz43K2 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=jroberson-net.20150623.gappssmtp.com header.s=20150623 header.b=R5cvxoU4; dmarc=none; spf=none (mx1.freebsd.org: domain of jroberson@jroberson.net has no SPF policy when checking 2607:f8b0:4864:20::102a) smtp.mailfrom=jroberson@jroberson.net X-Spamd-Result: default: False [-2.61 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[jroberson-net.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[svn-src-head@freebsd.org]; DMARC_NA(0.00)[jroberson.net]; RCPT_COUNT_FIVE(0.00)[5]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[jroberson-net.20150623.gappssmtp.com:+]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_TLS_ALL(0.00)[]; IP_SCORE(-0.81)[ipnet: 2607:f8b0::/32(-2.13), asn: 15169(-1.86), country: US(-0.05)] X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Jan 2020 08:35:00 -0000 On Sun, 5 Jan 2020, Mark Linimon wrote: > On Sat, Jan 04, 2020 at 03:15:34AM +0000, Jeff Roberson wrote: >> Use a separate lock for the zone and keg. > > Out of curiosity, will there be measurable real-world speedups from > this an similar work, or will this mostly apply to edge cases, or ... ? It depends on which real world. A lot of workloads don't really show much allocator activity. For very high speed networking, and especially very high speed networking on big NUMA machines, the speedup is considerable. Netflix reported the earlier round of work cut the time spent in uma by about 30%. For non-numa machines the last ~6 patches cut another 30% off of that in my tests. Even for Netflix, uma was not in the top 5 of their profile before this work. The major perf upshot was somewhere around an 8x improvement when freeing on a different NUMA domain than you allocated from when the allocation policy is first-touch. This is called a cross-domain or 'xdomain' free in the code. This made it possible to enable first-touch for UMA by default on all NUMA machines. I wrote a simple allocator perf test that loops allocating 2k mbufs and appending them to a random remote core's queue after which it drains its local queue. 10 million iterations across 32 cores in two numa domains gives 320,000,000 packets allocated and freed. The time within the same domain was about 4 seconds, before this patch series going to a different domain was around 40 seconds and after it was around 5 seconds. So only a ~25% penalty when doing 2 million packets-per-second-per-core. Many of the recent changes were really as much about code organization and readability as performance. After 18 years of features coming and going, reorganizations, etc. it was getting a bit crufty. Jeff > > mcl >