From owner-freebsd-arch@FreeBSD.ORG Sun Jul 13 03:55:05 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C359333D; Sun, 13 Jul 2014 03:55:05 +0000 (UTC) Received: from mail-we0-x236.google.com (mail-we0-x236.google.com [IPv6:2a00:1450:400c:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 15B7A2F9E; Sun, 13 Jul 2014 03:55:04 +0000 (UTC) Received: by mail-we0-f182.google.com with SMTP id q59so2704202wes.41 for ; Sat, 12 Jul 2014 20:55:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=GjcEfkGqiwGMwdsFdJLn3AXEFv0zBBhc6W03vsTupdQ=; b=UwXW+ojYeLhjPEMzAaCa65UhRtBNBbgQeHj016esOmyGHFMtIQN/UsP4e0XHfskZhp NuZTJlrnpnxA9Y0vuyUzhuDlyGipb6VDCWItAGdTrwFV6a8DQ/6c9AXazJtfTjd1fYj7 thI3cAQ7lPs83VwvrrbV6EHI2zSu+vCSDzZ7aFm9mf1NiH+WGf+KoVTcBpI07ENbZ9UL LB/RVLuB+wiBNEirVeQWM3a6C0MNRw4gzTvXJOlMWRSH+A/4otCtOHr7icf7LCl4RGrN IgA/3gwCn5iYkBmyULocqrJW+DFfrnA0iL7F4n2921chagywEdvm1HJ+7Y1PX5UUSJbt yh/A== X-Received: by 10.194.20.230 with SMTP id q6mr10054470wje.43.1405223703093; Sat, 12 Jul 2014 20:55:03 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id jb16sm14004315wic.10.2014.07.12.20.55.01 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Sat, 12 Jul 2014 20:55:02 -0700 (PDT) Date: Sun, 13 Jul 2014 05:55:00 +0200 From: Mateusz Guzik To: freebsd-arch@freebsd.org Subject: Getting rid of atomic_load_acq_int(&fdp->fd_nfiles)) from fget_unlocked Message-ID: <20140713035500.GC16884@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Cc: kib@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Jul 2014 03:55:05 -0000 Currently: /* * Avoid reads reordering and then a first access to the * fdp->fd_ofiles table which could result in OOB operation. */ if (fd < 0 || fd >= atomic_load_acq_int(&fdp->fd_nfiles)) return (EBADF); However, if we put fd_nfiles and fd_otable into one atomically replaced structure the only need to: 1. make sure the pointer is read once 2. issue a data dependency barrier - this is a noop on all supported architectures and we don't even have approprate macro, so doing nothing seems fine The motivation is to boost performance to amortize for seqlock cost, in case it hits the tree. This has no impact on races with capability lookup. In a microbenchmark of 16 threads reading from the same pipe fd immediately returning EAGAIN the numbers are: x vanilla-readpipe-run-sum + noacq-readpipe-run-sum [..] N Min Max Median Avg Stddev x 20 13133671 14900364 13893331 13827075 471500.82 + 20 59479718 59527286 59496714 59499504 13752.968 Difference at 95.0% confidence 4.56724e+07 +/- 213483 330.312% +/- 1.54395% There are 3 steps: 1. tidy up capsicum to accept fde: http://people.freebsd.org/~mjg/patches/single-fdtable-read-capsicum.patch 2. add __READ_ONCE: http://people.freebsd.org/~mjg/patches/read-once.patch 3. put stuff into one structure: http://people.freebsd.org/~mjg/patches/filedescenttable.patch Comments? -- Mateusz Guzik