Date: Tue, 4 Jun 2013 09:09:17 -0600 From: Warner Losh <imp@bsdimp.com> To: Andrew Duane <aduane@juniper.net> Cc: Juli Mallett <jmallett@freebsd.org>, Ed Schouten <ed@80386.nl>, Adrian Chadd <adrian@freebsd.org>, "freebsd-mips@FreeBSD.org" <freebsd-mips@freebsd.org>, FreeBSD-arch <freebsd-arch@freebsd.org> Subject: Re: Kernelspace C11 atomics for MIPS Message-ID: <2EE5CF35-1E46-44B8-83B3-6923FC6FA854@bsdimp.com> In-Reply-To: <477C1270D3E5484DA2303CEBE274C9E13210A1C0@CH1PRD0510MB392.namprd05.prod.outlook.com> References: <CAJOYFBD502MYbkVR2hnVDTYWOvOUr15=OPyjotNvv%2BZ09vQ1OQ@mail.gmail.com> <D02AF210-5129-40AB-9481-3F0A44575E98@bsdimp.com> <CAJ-Vmo=vNbT9majPCZ8ugzPsNzh46DTD4mMDX-cuxx9Og91ptw@mail.gmail.com> <CACVs6=_X5vOfR%2BQOgvz6P-j3jUoNoK9hCFvz80fGRL3-PgBf5g@mail.gmail.com> <477C1270D3E5484DA2303CEBE274C9E13210A1C0@CH1PRD0510MB392.namprd05.prod.outlook.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Jun 4, 2013, at 6:22 AM, Andrew Duane wrote: >=20 >> -----Original Message----- >> From: owner-freebsd-mips@freebsd.org [mailto:owner-freebsd- >> mips@freebsd.org] On Behalf Of Juli Mallett >> Sent: Monday, June 03, 2013 11:55 PM >> To: Adrian Chadd >> Cc: Ed Schouten; freebsd-mips@FreeBSD.org; FreeBSD-arch >> Subject: Re: Kernelspace C11 atomics for MIPS >>=20 >> On Mon, Jun 3, 2013 at 7:45 PM, Adrian Chadd <adrian@freebsd.org> = wrote: >>=20 >>> Speaking of this; any idea why the SYNC operators have 8 NOPs = following >>> them? >>>=20 >>> I noticed that when going through disassemblies of various mips24k = .o >>> files. >>>=20 >>=20 >> To drain the pipeline on certain deficient (and mostly older) CPUs by = way >> of guesswork and a little vague magic. Most CPUs we support, I would >> guess, do not need this, and it continues to exist solely for >> hysterical reasons. >>=20 >> I've certainly gotten rid of them and some other cargo cult = synchronization >> on Octeon for testing and had it survive under considerable load, and >> occasionally with some slight speedups (for some more commonly-used = or >> slower things than Just a Bunch Of NOPs.) >>=20 >> The trouble is that proving they aren't necessary requires being = rigorous >> and careful in understanding documentation and errata, and FUD about = their >> possible necessity is somewhat-intimidating. It's not an easy kind = of >> corruption/unreliability/etc., to prove the lack of empirically. >=20 > The various CPU types are supposed to specify exactly how many NOPs = are needed, what kind of barrier is needed and where, and which type of = NOP is needed (Alpha had at least two). The barriers are designed to = insure correct operation ordering across the memory architectures = including write buffers, DMA hardware, L1/L2[/L3] caches and their = connections to the cores. The CPU should specify exactly what is needed = where, and why. It should never be "superstition". There is an exact = hardware reason for every sync/barrier operation, and every NOP needed, = just like the COP0 hazards. Except that none of the examples in the ISA manual have them, and = there's no mention of them at all, unlike COP0 hazards. I know of only = one case in the Linux tree where it is done (for the au1xxxx cores). = There's another place where it is defined in a function, but that = function is never called for older Broadcom MIPS. In NetBSD, extra NOPs are not inserted at all. They do do two syncs for = the SB1250 PASS 1. None of the docs I've seen for latter-day MIPS CPUs document the need = for NOPs. In fact, the only place I think that I've seen them was on one = or two of the older R8000, R10000 and R120000 errata that stated they = were needed there. Since these were the first complicated multi-issue = designs, it wasn't surprising that the NOPs were needed as work arounds. = I can't find these errata with a google search now, so I can't confirm = this dim memory that I have. The reason the NOPs are there today likely is superstition. A cargo-cult = workaround from the past whose days have come and gone, leaving nothing = but an echo in the code. > Given that, Juli's last paragraph is right on point. The documentation = can be dense and difficult to understand, since it's usually written by = hardware engineers :-) And since getting it wrong can make for some = really subtle, intermittent, and incredibly hard to diagnose problems, = it's easier to err on the side of caution. It also happens that = different CPUs included in a certain compile switch may have different = requirements, so you have to use worst case. I'll agree about the dense documentation. But usually that's around all = the crazy cache effects that one must understand to cope with the design = that puts half of the cache management in software. I'm all for ditching them unless a specific reason for keeping them can = be found. >> Juli. >> _______________________________________________ >> freebsd-mips@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-mips >> To unsubscribe, send any mail to = "freebsd-mips-unsubscribe@freebsd.org" >>=20 >=20 >=20 > .................................... > Andrew L. Duane > Resident Architect - AT&T Technical Lead > m +1 603.770.7088 > o +1 408.933.6944 (2-6944) > skype: andrewlduane > aduane@juniper.net >=20 >=20 >=20 > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2EE5CF35-1E46-44B8-83B3-6923FC6FA854>