From owner-svn-src-all@FreeBSD.ORG Thu Dec 27 12:55:31 2012 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4A974919; Thu, 27 Dec 2012 12:55:31 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-la0-f43.google.com (mail-la0-f43.google.com [209.85.215.43]) by mx1.freebsd.org (Postfix) with ESMTP id 3113F8FC12; Thu, 27 Dec 2012 12:55:29 +0000 (UTC) Received: by mail-la0-f43.google.com with SMTP id z14so12247791lag.30 for ; Thu, 27 Dec 2012 04:55:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=r6Fel6LeOL247Dpn4oQuu0O2m7t3TJIl6RV0VhWKorE=; b=NQzw+bNHnVCRcn5itOPU9loxhHVU8Anw+YqvTs0aU/vumbBVDL5VPruQtjAI+EgKHM C9eTcOgl65boDNTzNJjGavBBPfEp5nVjUVVlTyE4OSGv14z6Rzf/gEgcB4MALrYLonNh Po2EOOq1jtesUQvA1RLWKLx+ZclNFTQxEE4FJhDnjxrdsfPl4eATipLZNFSGFBDynKPm vmUDUrt38/wsSFlTIJNVGT5JhaZtaqBKH4R20oCGZkYc0HZVt3F1fmy0GW4idDX9HiHv fIUyDVEu/vpAOXgFuopEg3qdiHjskYSPKempsMT8CjPxThHsYmG98EayHG/MKTxpZwpN tjoA== MIME-Version: 1.0 Received: by 10.112.87.40 with SMTP id u8mr12139761lbz.50.1356612922749; Thu, 27 Dec 2012 04:55:22 -0800 (PST) Sender: asmrookie@gmail.com Received: by 10.112.84.193 with HTTP; Thu, 27 Dec 2012 04:55:22 -0800 (PST) In-Reply-To: <20121227124657.GX80310@FreeBSD.org> References: <201212271236.qBRCawuU078203@svn.freebsd.org> <20121227124657.GX80310@FreeBSD.org> Date: Thu, 27 Dec 2012 04:55:22 -0800 X-Google-Sender-Auth: 97SeYChrGgRPsoI2sPT1O4IdO3g Message-ID: Subject: Re: svn commit: r244732 - head/sys/sys From: Attilio Rao To: Gleb Smirnoff Content-Type: text/plain; charset=UTF-8 Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: attilio@FreeBSD.org List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Dec 2012 12:55:31 -0000 On Thu, Dec 27, 2012 at 4:46 AM, Gleb Smirnoff wrote: > Attilio, > > On Thu, Dec 27, 2012 at 12:36:58PM +0000, Attilio Rao wrote: > A> Author: attilio > A> Date: Thu Dec 27 12:36:58 2012 > A> New Revision: 244732 > A> URL: http://svnweb.freebsd.org/changeset/base/244732 > A> > A> Log: > A> br_prod_tail and br_cons_tail members are used as barrier to > A> signal bug_ring ownership. However, instructions can be reordered > A> around members write leading to stale values for ie. br_prod_bufs. > A> > A> Use correct memory barriers to ensure proper ordering of the > A> ownership tokens updates. > A> > A> Sponsored by: EMC / Isilon storage division > A> MFC after: 2 weeks > > Have you profiled this? > > After this change the buf_ring actually gains its own hand-rolled > mutex: > > while (atomic_load_acq_32(&br->br_prod_tail) != prod_head) > cpu_spinwait(); > > The only difference with mutex(9) is that this one isn't monitored > by WITNESS. I think you are not correct. It doesn't disable interrupts (as spinlock do) and it doesn't sleep. So your analogy is completely off. Also, on x86 atomic_store_rel_*() is a simple write. The only thing that really changes is the atomic_load_acq_*() that introduces a locked instruction. > The idea behind buf_ring was lockless storing and lockless fetching > from a ring and now this vanished. > > What does your change actually fixes except precise accounting of > br_prod_bufs that are actually unused and should be better garbage > collected rather than fixed? The write of br_prod_tail must happens as very last thing, also after the whole buf setup. The only way you can enforce this is with a memory barrier. I can double-check if we can garbage collect br_prod_bufs but this should not be enough yet. Attilio -- Peace can only be achieved by understanding - A. Einstein