From owner-freebsd-numerics@freebsd.org Tue Nov 19 05:55:09 2019 Return-Path: Delivered-To: freebsd-numerics@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 090DC17E092 for ; Tue, 19 Nov 2019 05:55:09 +0000 (UTC) (envelope-from jwalden@mit.edu) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47HFQz6wpJz43Qp; Tue, 19 Nov 2019 05:55:07 +0000 (UTC) (envelope-from jwalden@mit.edu) Received: from [10.252.25.189] (corp-nat.fw1.untrust.mtv2.mozilla.net [63.245.221.198]) (authenticated bits=0) (User authenticated as jwalden@ATHENA.MIT.EDU) by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id xAJ5t3lB009713 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 19 Nov 2019 00:55:05 -0500 Subject: Re: UB in various hypot() implementations (left-shifting a negative, number) To: sgk@troutmask.apl.washington.edu, Dimitry Andric Cc: freebsd-numerics@freebsd.org References: <20191116211419.GA40056@troutmask.apl.washington.edu> From: Jeff Walden Openpgp: preference=signencrypt Message-ID: <11f139aa-708a-4799-dfee-1a5b988a3c0b@mit.edu> Date: Mon, 18 Nov 2019 21:55:03 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.1 MIME-Version: 1.0 In-Reply-To: <20191116211419.GA40056@troutmask.apl.washington.edu> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 47HFQz6wpJz43Qp X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of jwalden@mit.edu designates 18.9.28.11 as permitted sender) smtp.mailfrom=jwalden@mit.edu X-Spamd-Result: default: False [-5.35 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:18.9.28.0/24]; NEURAL_HAM_LONG(-1.00)[-0.998,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[mit.edu]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_MED(-0.20)[11.28.9.18.list.dnswl.org : 127.0.11.2]; IP_SCORE(-2.85)[ip: (-9.57), ipnet: 18.9.0.0/16(-4.64), asn: 3(0.01), country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:3, ipnet:18.9.0.0/16, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-numerics@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussions of high quality implementation of libm functions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Nov 2019 05:55:09 -0000 On 11/16/19 1:14 PM, Steve Kargl wrote: > Well, clearly, the patch to e_hypotl.c is wrong. It > clear the significand when t1 = 0 whereas t1 = 1 leaves > one bit set in the significand. Simply looking at the > value of t1 under a poor man's debugger shows the > difference. Adding "printf("%Le %La\n", t1, t1);" after > the SET_HIGH_WORD gives > > 2.962347e-2493 0x1p-8280 <-- t1 = 1 > 0.000000e+00 0x1p-8280 <-- t1 = 0 > > for hypotl(ldexpl(1.1,-16000), ldexpl(2.1, -16000)). Hrm. My understanding based on an earlier line in e_hypotl.c was that that change should be identical, but something mildly subtle is going on and my understanding of |long double| is not quite as good as I thought it was. I've removed that file's changes and left just the original two in, then resubmitted the diff. Jeff