From owner-svn-src-all@freebsd.org Tue Dec 11 04:57:14 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 45B411318C3E; Tue, 11 Dec 2018 04:57:14 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5FA098307F; Tue, 11 Dec 2018 04:57:12 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 7198E1247; Mon, 10 Dec 2018 23:57:10 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Mon, 10 Dec 2018 23:57:11 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsco.org; h= content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s=fm2; bh=R hwREypnUcOO9txwAgiZSMmQ7uxdRsFRrBlPfekG2DE=; b=E+EjLkTV7mJ0Va0Rw LQDhUrcGYPpoXHSzVJF+ljNm4EeJp0h/8YDzMtZIzVdcPTLGMxJsOzpNMXmymTNR X/wxw+/wGsgZKvqFH3KMAU88PoTfRFsI++Pzmt7E5i0zzzJqgCa+AN6iu35N91Qk EEM1x0OiYcCAQWmrUK+rr0Tl1GTstNPJyHmRDTJ0zrSgl3U1xmIg4EiPvMLI2ODd 0nu4CWS87b84QWPNoJg2b39u6bp+J5UioNfN6UF5VqiZlUuDBZwfzeKv1EO6R9H2 NAp9NigaZQfZotA6VuhHl89pLEJ8ox052aK1MVn1ty0IwtoWUltnkNQHMnsZ1U+7 EEk4A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=RhwREypnUcOO9txwAgiZSMmQ7uxdRsFRrBlPfekG2 DE=; b=isST6UXiK9Qe0jEyj6fZokUfvZIm6mFR6RPx5oJUzDZPuo7rhiTrEbUAC CyXpt73Mg8OQ3Pj6XjmeDhCDATD4EsApEpHS6XLfw2tGFpx2vzMaqdu+ylHFWO5e ECtKYGZgnOzo7bFjco2bUaka3xjlg33OBIwQqywGbP051XOLQZsCGAgaF6JNS1TU xYdu4xUlXL9p8R8JPLRRrrxfvE6eewmy8SckVKKntboE3byzlVp74n2RPyqBSGWb NMan08nmzAzUUeOMmslvkjs9SsyNkXPvK+MOo7biRXx51Q5jLVnmMQ7xOoJvfI8n ZWfrGcFp/gGhVqBU10QJISCfM5eIA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedtkedrudegiedgjeekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfquhhtnecuuegrihhlohhuthemucef tddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjughrpegtggfuhfgjff fgkfhfvffosehtqhhmtdhhtdejnecuhfhrohhmpefutghothhtucfnohhnghcuoehstgho thhtlhesshgrmhhstghordhorhhgqeenucfkphepkedrgeeirdekledrvddufeenucfrrg hrrghmpehmrghilhhfrhhomhepshgtohhtthhlsehsrghmshgtohdrohhrghenucevlhhu shhtvghrufhiiigvpedt X-ME-Proxy: Received: from [192.168.0.137] (unknown [8.46.89.213]) by mail.messagingengine.com (Postfix) with ESMTPA id B473D102EE; Mon, 10 Dec 2018 23:57:08 -0500 (EST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.0 \(3445.100.39\)) Subject: Re: svn commit: r341682 - head/sys/sys From: Scott Long In-Reply-To: <20181210234754.GD60291@kib.kiev.ua> Date: Mon, 10 Dec 2018 21:57:08 -0700 Cc: John Baldwin , Warner Losh , Kevin Bowling , Mateusz Guzik , Ian Lepore , svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers , scottl@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <201812071205.wB7C5BvA038350@repo.freebsd.org> <1544206201.1860.288.camel@freebsd.org> <45f85061-2633-852c-3cc0-41f64d51e4f0@FreeBSD.org> <20181210234754.GD60291@kib.kiev.ua> To: Konstantin Belousov X-Mailer: Apple Mail (2.3445.100.39) X-Rspamd-Queue-Id: 5FA098307F X-Spamd-Result: default: False [-6.44 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[samsco.org,messagingengine.com]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.25]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[samsco.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[samsco.org:+,messagingengine.com:+]; MX_GOOD(-0.01)[in2-smtp.messagingengine.com,in1-smtp.messagingengine.com,in2-smtp.messagingengine.com,in1-smtp.messagingengine.com,in2-smtp.messagingengine.com,in1-smtp.messagingengine.com,in2-smtp.messagingengine.com,in1-smtp.messagingengine.com,in2-smtp.messagingengine.com,in1-smtp.messagingengine.com,in2-smtp.messagingengine.com,in1-smtp.messagingengine.com]; RCPT_COUNT_SEVEN(0.00)[10]; NEURAL_HAM_SHORT(-0.86)[-0.862,0]; IP_SCORE(-3.47)[ip: (-8.97), ipnet: 64.147.123.0/24(-4.48), asn: 11403(-3.82), country: US(-0.09)]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_IN_DNSWL_LOW(-0.10)[25.123.147.64.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:11403, ipnet:64.147.123.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Server: mx1.freebsd.org X-Mailman-Approved-At: Tue, 11 Dec 2018 05:31:35 +0000 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Dec 2018 04:57:14 -0000 > On Dec 10, 2018, at 4:47 PM, Konstantin Belousov = wrote: >=20 > On Mon, Dec 10, 2018 at 02:15:20PM -0800, John Baldwin wrote: >> On 12/8/18 7:43 PM, Warner Losh wrote: >>>=20 >>>=20 >>> On Sat, Dec 8, 2018, 8:36 PM Kevin Bowling wrote: >>>=20 >>> On Sat, Dec 8, 2018 at 12:09 AM Mateusz Guzik > wrote: >>>=20 >>>>=20 >>>> Fully satisfying solution would be that all architectures get = 64-bit >>>> ops, even if in the worst case they end up taking a lock. Then >>>> subsystems would not have to ifdef on anything. However, there >>>> was some opposition to this proposal and I don't think this is >>>> important enough to push. >>>=20 >>> Mateusz, >>>=20 >>> Who is opposing this particular polyfill solution? Scott Long = brought >>> up a situation in driver development where this would be useful = as >>> well. The polyfills lower the cognitive load and #ifdef soup = which >>> are the right call here regardless of performance on toy ports. >>>=20 >>>=20 >>> I don't recall seeing the opposition either. It would have to be a = global lock for all 64bit atomics.... but I think it would only be 2 = atomics on those architectures.=20 >>=20 >> It would have to be a spin lock, so in the case of unrl you would be = trading >> an operation on one of N regular mutexes for a single spin lock that = was >> also contested by other things. This would be pretty crappy. For = drivers >> that aren't actually used on platforms without 32-bit atomics we can = simply >> not build them in sys/modules/Makefile or not put them in GENERIC. = For >> something in the core kernel like unrl I think we will have to do = what >> Mateusz has done here. >=20 > It is worse. All atomics that acess the same location must use the = same > lock. Otherwise, you could observe torn writes and out of thin air > values. Since you cannot know in advance which locations are acceses > by the locked variant, all freebsd atomics ops have to be switched to > locked variant on the architecture. 64bit atomics on I486 already suffer the risk of torn reads; the = implementation merely does a CLI to protect against local preemption (though you could = still get unlucky with an NMI). I suppose you could argue that SMP isn=E2=80=99= t really viable on I486 and therefore this fact is irrelevant, but it does = illustrate precedence for having API completeness in a platform. Really, this isn=E2=80=99t that hard. Part of the existing contract of = using atomics is that you carefully evaluate all uses of the variable and decide when to = use an atomic instruction. Arguing that we can=E2=80=99t make this process = automatic and foolproof for 64bit quantities, especially for a subset of subset of platforms/architectures, and therefore we should be even more of a = difficult landmine, is not=E2=80=A6. I don=E2=80=99t know what to say=E2=80=A6 = sensical? 64bit operations are a reality for MI code in a modern OS, and I=E2=80=99m= tired of having to tip-toe around them due to incomplete MD implementations. The instructions have been available on Intel CPUs for 25 years! My very strong preference is to have a complete and functional = implementation of atomic.h for any architecture that is hooked up to the build. We can = then tackle the details of optimization and edge case refinement, just like = we do with every other API and service that we work on. It doesn=E2=80=99t = have to be perfect to be useful, and at this point we=E2=80=99re providing neither = perfection nor utility, just =E2=80=9Cbuts=E2=80=9D and =E2=80=9Cwhat ifs=E2=80=9D. Going forward, I=E2=80=99m going to start using 64bit atomics where = they=E2=80=99re prudent, instead of avoiding them due to this niche 32bit argument. If that = means more and more of what I do no longer compiles on a mips or a ppc32, then that=E2=80=99s a sacrifice that is fine with me. It still creates extra = development work, and having a uniformly available implementation would be much nicer. Scott