From owner-freebsd-numerics@freebsd.org Wed Nov 13 22:43:13 2019 Return-Path: Delivered-To: freebsd-numerics@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D501A1BE611 for ; Wed, 13 Nov 2019 22:43:13 +0000 (UTC) (envelope-from jwalden@mit.edu) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47D04w3Wjpz3Cmp for ; Wed, 13 Nov 2019 22:43:11 +0000 (UTC) (envelope-from jwalden@mit.edu) Received: from [10.252.25.189] (corp-nat.fw1.untrust.mtv2.mozilla.net [63.245.221.198]) (authenticated bits=0) (User authenticated as jwalden@ATHENA.MIT.EDU) by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id xADMh5S5023770 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT) for ; Wed, 13 Nov 2019 17:43:08 -0500 To: freebsd-numerics@freebsd.org From: Jeff Walden Subject: UB in various hypot() implementations (left-shifting a negative number) Openpgp: preference=signencrypt Message-ID: Date: Wed, 13 Nov 2019 14:43:04 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 47D04w3Wjpz3Cmp X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of jwalden@mit.edu designates 18.9.28.11 as permitted sender) smtp.mailfrom=jwalden@mit.edu X-Spamd-Result: default: False [-5.34 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:18.9.28.0/24]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-numerics@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_MED(-0.20)[11.28.9.18.list.dnswl.org : 127.0.11.2]; DMARC_NA(0.00)[mit.edu]; IP_SCORE(-2.84)[ip: (-9.55), ipnet: 18.9.0.0/16(-4.62), asn: 3(0.01), country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:3, ipnet:18.9.0.0/16, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-numerics@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussions of high quality implementation of libm functions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Nov 2019 22:43:13 -0000 Hi all, Just wanted to note here that I filed https://reviews.freebsd.org/D22354 to fix a bit of undefined behavior in the hypot() function implementations. `hypot(x, y)` computes `sqrt(x*x + y*y)`, with IEEE-754-aware precision. For very large or small `x` or `y`, the naive implementation would lose precision -- so in such cases the calculation is performed after multiplying the numbers by a very large (or very small) power of two, then the multiplication is undone at end. Undoing the multiplication involves multiplying a quantity `w` by `2**k`, where `k` (which may be positive or negative) was computed depending on the particular `x` and `y` values provided. Current algorithms generally take `t1=0.0`, extract the high word, add `k<<20` or `k<<23` to it to appropriately bump the exponent, then overwrite `t1`'s high word. But it seems equally effective to take `t1=0.0`, then write `(1023+k)<<20` or `(127+k)<<23` to it for identical effect -- and `k` is never so negative to compute a negative value. My changes do this. (I wish there were named constants I could use for all these numbers, but as there don't seem to be any, magic numbers seems like the best I can do.) Errors in these changes would most likely produce a power of two off by a factor of two, so *probably* testing any inputs that would happen to invoke these code paths should be adequate testing. I'm fixing this in order to upstream a likely fix in the SpiderMonkey JavaScript engine for this, so the (double, double) algorithm/change is the only one I have (purely manually) tested. Eyeballs on the other functions' changes especially appreciated! Jeff