From owner-svn-src-all@freebsd.org Sun Jan 5 08:34:55 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 404AE1F3389 for ; Sun, 5 Jan 2020 08:34:55 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from mail-pl1-x630.google.com (mail-pl1-x630.google.com [IPv6:2607:f8b0:4864:20::630]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47rBld6lBQz43Hw for ; Sun, 5 Jan 2020 08:34:53 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: by mail-pl1-x630.google.com with SMTP id b22so20639849pls.12 for ; Sun, 05 Jan 2020 00:34:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jroberson-net.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version; bh=+vfTMhoycwapEZ+lyJ5+TT8Ek8FG4ALeeIW9XxUxs+k=; b=R5cvxoU4vP5AapabaLT8+X0jHXFCL40V+0t6WDcq0kZADothMVIm5KH5V4enuoV0P0 A04KCrZ+g6BqZRqnShVZLKI3KdSsrNoMFzlrTZHRjmd34RwRVeo06CmYqHBLxxOOsjWB fXQp62zZ8C0ZFO0B3lL1XZvWAGBH9CcvTh988Aka0YuMQx/rBRCTZCqkaLnWcYSy8dGl ykT+bRVHNeC724zNcE+4wi9uLz657TXGZYP8KQHyBdD7ssJZLJdMGBeelZDj48yq/R+G EiN+wPRiiCWauR7cKTJbgXUwGV4t6764a8itWf39oAah2vbHFdVvgQlUMNV3o3mCpSlu 6rZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:in-reply-to:message-id :references:user-agent:mime-version; bh=+vfTMhoycwapEZ+lyJ5+TT8Ek8FG4ALeeIW9XxUxs+k=; b=Jq0tzekpYqycPScK5ebo6dQNxr2xPkzdEr6jqtmQm400bLJnhIleVMNz//YYTfgoiK 0NO95U78YX3IdGpzPhEek9Ikr6grSUdeZ2QAGQ+Zz/EosNlP/KzDXgJZzfFn86hHoX/M PZdC84vqoA4U/di6C9QY8NjlSOZCZis3LY81POEcbHh7uo88Ax/l1LGOsozqqJYOW679 vlispZIYUjqLknPoXT6OA1G74rxlsncVh6RuE3GbKJLcVUzIWT50tSR4OWd03mctCCjv uQYAT0UVEFEo08SM1M8Qx2QVEUm0ebbe6J67ys8E5qsFUika3E7EZ+ri+Dx+NmlzF7Yv Rh9w== X-Gm-Message-State: APjAAAWCibI6ABLG+fFeoccRNRyncc3+mDdDvTo6vGQb6mxkaF8pGN9W paGACHY53V/0YPSJCVrFuudMZQ== X-Google-Smtp-Source: APXvYqx+OvnGcaE4sqlsW4Xr4o72rQVubY1zFW+htY8g08jrgEy6ez1T0Y30lUZn/QQtAYn243LK7A== X-Received: by 2002:a17:90a:33ad:: with SMTP id n42mr36177086pjb.23.1578213291975; Sun, 05 Jan 2020 00:34:51 -0800 (PST) Received: from rrcs-76-81-105-82.west.biz.rr.com (rrcs-76-81-105-82.west.biz.rr.com. [76.81.105.82]) by smtp.gmail.com with ESMTPSA id 2sm66910766pgo.79.2020.01.05.00.34.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 05 Jan 2020 00:34:51 -0800 (PST) Date: Sat, 4 Jan 2020 22:34:49 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: Mark Linimon cc: Jeff Roberson , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r356348 - in head/sys: kern vm In-Reply-To: <20200105013314.GA3681@lonesome.com> Message-ID: References: <202001040315.0043FYhn047977@repo.freebsd.org> <20200105013314.GA3681@lonesome.com> User-Agent: Alpine 2.21.9999 (BSF 287 2018-06-16) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Rspamd-Queue-Id: 47rBld6lBQz43Hw X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=jroberson-net.20150623.gappssmtp.com header.s=20150623 header.b=R5cvxoU4; dmarc=none; spf=none (mx1.freebsd.org: domain of jroberson@jroberson.net has no SPF policy when checking 2607:f8b0:4864:20::630) smtp.mailfrom=jroberson@jroberson.net X-Spamd-Result: default: False [-4.40 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[jroberson-net.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[svn-src-all@freebsd.org]; DMARC_NA(0.00)[jroberson.net]; RCPT_COUNT_FIVE(0.00)[5]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[jroberson-net.20150623.gappssmtp.com:+]; RCVD_IN_DNSWL_NONE(0.00)[0.3.6.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_TLS_ALL(0.00)[]; IP_SCORE(-2.60)[ip: (-8.94), ipnet: 2607:f8b0::/32(-2.13), asn: 15169(-1.86), country: US(-0.05)] X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Jan 2020 08:34:55 -0000 On Sun, 5 Jan 2020, Mark Linimon wrote: > On Sat, Jan 04, 2020 at 03:15:34AM +0000, Jeff Roberson wrote: >> Use a separate lock for the zone and keg. > > Out of curiosity, will there be measurable real-world speedups from > this an similar work, or will this mostly apply to edge cases, or ... ? It depends on which real world. A lot of workloads don't really show much allocator activity. For very high speed networking, and especially very high speed networking on big NUMA machines, the speedup is considerable. Netflix reported the earlier round of work cut the time spent in uma by about 30%. For non-numa machines the last ~6 patches cut another 30% off of that in my tests. Even for Netflix, uma was not in the top 5 of their profile before this work. The major perf upshot was somewhere around an 8x improvement when freeing on a different NUMA domain than you allocated from when the allocation policy is first-touch. This is called a cross-domain or 'xdomain' free in the code. This made it possible to enable first-touch for UMA by default on all NUMA machines. I wrote a simple allocator perf test that loops allocating 2k mbufs and appending them to a random remote core's queue after which it drains its local queue. 10 million iterations across 32 cores in two numa domains gives 320,000,000 packets allocated and freed. The time within the same domain was about 4 seconds, before this patch series going to a different domain was around 40 seconds and after it was around 5 seconds. So only a ~25% penalty when doing 2 million packets-per-second-per-core. Many of the recent changes were really as much about code organization and readability as performance. After 18 years of features coming and going, reorganizations, etc. it was getting a bit crufty. Jeff > > mcl >