Date: Wed, 20 Nov 2019 12:06:20 -0800 From: Steve Kargl <sgk@troutmask.apl.washington.edu> To: Jeff Walden <jwalden@mit.edu> Cc: Dimitry Andric <dim@freebsd.org>, freebsd-numerics@freebsd.org Subject: Re: UB in various hypot() implementations (left-shifting a negative, number) Message-ID: <20191120200620.GA61176@troutmask.apl.washington.edu> In-Reply-To: <11f139aa-708a-4799-dfee-1a5b988a3c0b@mit.edu> References: <a3fe6fb4-8f33-6d10-64d4-e722bfebcc85@FreeBSD.org> <20191116211419.GA40056@troutmask.apl.washington.edu> <11f139aa-708a-4799-dfee-1a5b988a3c0b@mit.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Nov 18, 2019 at 09:55:03PM -0800, Jeff Walden wrote:
> On 11/16/19 1:14 PM, Steve Kargl wrote:
> > Well, clearly, the patch to e_hypotl.c is wrong. It
> > clear the significand when t1 = 0 whereas t1 = 1 leaves
> > one bit set in the significand. Simply looking at the
> > value of t1 under a poor man's debugger shows the
> > difference. Adding "printf("%Le %La\n", t1, t1);" after
> > the SET_HIGH_WORD gives
> >
> > 2.962347e-2493 0x1p-8280 <-- t1 = 1
> > 0.000000e+00 0x1p-8280 <-- t1 = 0
> >
> > for hypotl(ldexpl(1.1,-16000), ldexpl(2.1, -16000)).
>
> Hrm. My understanding based on an earlier line in e_hypotl.c
> was that that change should be identical, but something mildly
> subtle is going on and my understanding of |long double| is not
> quite as good as I thought it was. I've removed that file's
> changes and left just the original two in, then resubmitted the diff.
>
> Jeff
In general, float, double, and long double all use the same
general approach where a value is scaled if required. The
details of unraveling the scaling do differ. float operates
on the 32-bit integer representation of the float value. double
operates on the 32-bit integer portion of the double that contains
the exponent. For at least ld80 long double, the macro SET_HIGH_WORD
assigns directly to the bitfield of struct within a union. The bitfield
represents the sign and exponent. So, setting t1=0 clears the significand,
which has an undesirable effect.
--
Steve
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20191120200620.GA61176>