From owner-freebsd-arch@FreeBSD.ORG Tue Jun 4 15:09:21 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 2C7A78A5 for ; Tue, 4 Jun 2013 15:09:21 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ie0-x22d.google.com (mail-ie0-x22d.google.com [IPv6:2607:f8b0:4001:c03::22d]) by mx1.freebsd.org (Postfix) with ESMTP id EE23E1825 for ; Tue, 4 Jun 2013 15:09:20 +0000 (UTC) Received: by mail-ie0-f173.google.com with SMTP id k13so688950iea.4 for ; Tue, 04 Jun 2013 08:09:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=mQjEh5F3H8ANLTXokocrCCP3P7W6+e2Rh72FQ4MymME=; b=aVQtrvDWUsRcOanK5JetmOo+Sxhma3vxumKNs7taDOp+dbueoVDhCEzY+MJeakOd2G msRH/Ou6Qe0UqhFmu9tAffPZ8BbsTE6eRuGRDeWNSoGfiTb/Chh398kVv6taNJ80ngI0 c35xbLBjCMLURpoU08m6o2i/S79DDXQf9Si2ng4Ze1Wf0A75RNDgzWZ8AiKgeBjD+mLY bNWVYB+3sqJWq5R3X7Colvz5EfRImEEpPZ10h6g39Gw6L0rdZUlpdNr6BTJ8pM6SdGyt k7xuMiu7XAxlOnd7P2HnFLUPdPzZNltl5UtKu2jrRA08MbRGDXqnh/Ta/9mlSaWJN8R8 RbTw== X-Received: by 10.50.18.17 with SMTP id s17mr1027482igd.80.1370358560526; Tue, 04 Jun 2013 08:09:20 -0700 (PDT) Received: from 53.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id f6sm2261850igz.1.2013.06.04.08.09.18 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 04 Jun 2013 08:09:19 -0700 (PDT) Sender: Warner Losh Subject: Re: Kernelspace C11 atomics for MIPS Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <477C1270D3E5484DA2303CEBE274C9E13210A1C0@CH1PRD0510MB392.namprd05.prod.outlook.com> Date: Tue, 4 Jun 2013 09:09:17 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <2EE5CF35-1E46-44B8-83B3-6923FC6FA854@bsdimp.com> References: <477C1270D3E5484DA2303CEBE274C9E13210A1C0@CH1PRD0510MB392.namprd05.prod.outlook.com> To: Andrew Duane X-Mailer: Apple Mail (2.1085) X-Gm-Message-State: ALoCoQkZBQwj0BoelMl+KMUU9fl0C0wjeCnLPSNhHUcOuA5spnybpy3TjxOcLH843c4/W2pV2yM5 Cc: Juli Mallett , Ed Schouten , Adrian Chadd , "freebsd-mips@FreeBSD.org" , FreeBSD-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jun 2013 15:09:21 -0000 On Jun 4, 2013, at 6:22 AM, Andrew Duane wrote: >=20 >> -----Original Message----- >> From: owner-freebsd-mips@freebsd.org [mailto:owner-freebsd- >> mips@freebsd.org] On Behalf Of Juli Mallett >> Sent: Monday, June 03, 2013 11:55 PM >> To: Adrian Chadd >> Cc: Ed Schouten; freebsd-mips@FreeBSD.org; FreeBSD-arch >> Subject: Re: Kernelspace C11 atomics for MIPS >>=20 >> On Mon, Jun 3, 2013 at 7:45 PM, Adrian Chadd = wrote: >>=20 >>> Speaking of this; any idea why the SYNC operators have 8 NOPs = following >>> them? >>>=20 >>> I noticed that when going through disassemblies of various mips24k = .o >>> files. >>>=20 >>=20 >> To drain the pipeline on certain deficient (and mostly older) CPUs by = way >> of guesswork and a little vague magic. Most CPUs we support, I would >> guess, do not need this, and it continues to exist solely for >> hysterical reasons. >>=20 >> I've certainly gotten rid of them and some other cargo cult = synchronization >> on Octeon for testing and had it survive under considerable load, and >> occasionally with some slight speedups (for some more commonly-used = or >> slower things than Just a Bunch Of NOPs.) >>=20 >> The trouble is that proving they aren't necessary requires being = rigorous >> and careful in understanding documentation and errata, and FUD about = their >> possible necessity is somewhat-intimidating. It's not an easy kind = of >> corruption/unreliability/etc., to prove the lack of empirically. >=20 > The various CPU types are supposed to specify exactly how many NOPs = are needed, what kind of barrier is needed and where, and which type of = NOP is needed (Alpha had at least two). The barriers are designed to = insure correct operation ordering across the memory architectures = including write buffers, DMA hardware, L1/L2[/L3] caches and their = connections to the cores. The CPU should specify exactly what is needed = where, and why. It should never be "superstition". There is an exact = hardware reason for every sync/barrier operation, and every NOP needed, = just like the COP0 hazards. Except that none of the examples in the ISA manual have them, and = there's no mention of them at all, unlike COP0 hazards. I know of only = one case in the Linux tree where it is done (for the au1xxxx cores). = There's another place where it is defined in a function, but that = function is never called for older Broadcom MIPS. In NetBSD, extra NOPs are not inserted at all. They do do two syncs for = the SB1250 PASS 1. None of the docs I've seen for latter-day MIPS CPUs document the need = for NOPs. In fact, the only place I think that I've seen them was on one = or two of the older R8000, R10000 and R120000 errata that stated they = were needed there. Since these were the first complicated multi-issue = designs, it wasn't surprising that the NOPs were needed as work arounds. = I can't find these errata with a google search now, so I can't confirm = this dim memory that I have. The reason the NOPs are there today likely is superstition. A cargo-cult = workaround from the past whose days have come and gone, leaving nothing = but an echo in the code. > Given that, Juli's last paragraph is right on point. The documentation = can be dense and difficult to understand, since it's usually written by = hardware engineers :-) And since getting it wrong can make for some = really subtle, intermittent, and incredibly hard to diagnose problems, = it's easier to err on the side of caution. It also happens that = different CPUs included in a certain compile switch may have different = requirements, so you have to use worst case. I'll agree about the dense documentation. But usually that's around all = the crazy cache effects that one must understand to cope with the design = that puts half of the cache management in software. I'm all for ditching them unless a specific reason for keeping them can = be found. >> Juli. >> _______________________________________________ >> freebsd-mips@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-mips >> To unsubscribe, send any mail to = "freebsd-mips-unsubscribe@freebsd.org" >>=20 >=20 >=20 > .................................... > Andrew L. Duane > Resident Architect - AT&T Technical Lead > m +1 603.770.7088 > o +1 408.933.6944 (2-6944) > skype: andrewlduane > aduane@juniper.net >=20 >=20 >=20 > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org"