Date: Thu, 28 Feb 2019 05:06:23 -0800 From: Mark Millard <marklmi@yahoo.com> To: FreeBSD PowerPC ML <freebsd-ppc@freebsd.org> Cc: freebsd-hackers Hackers <freebsd-hackers@freebsd.org> Subject: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes Message-ID: <D3D7E9F4-9A5E-4320-B3C8-EC5CEF4A2764@yahoo.com>
next in thread | raw e-mail | index | archive | help
Basic context:
The code for sleeps of various forms depends
on calls to:
static __inline sbintime_t
sbinuptime(void)
{
struct bintime _bt;
binuptime(&_bt);
return (bttosbt(_bt));
}
and comparisons with the return values, such
as checking for timeouts. The upper 32 bits
of the unsigned 64 bit result has the seconds
and the lower 32 bits has the fraction as
a multiplier of 1sec/(2**64).
An observed problem is that later sbinuptime calls
sometimes end up with smaller values than earlier
ones. (Past lisy freebsd-ppc messages give details.)
This makes for problems with checking for timeouts
when using later sbinuptime() calls after a timeout
was initially detected against an earlier value:
A.0) timercb getting the earlier sbinuptime() value
A.1) callout_process using that to detect a timeout,
B) sleepq_timeout checking the timeout again,
using a separate sbinuptime() call.
Some details about example values, overflows, and such follow.
I used the following sort of hacked code to report values when
overflows happen:
#if defined(__powerpc64__) && defined(AIM)
void
binuptime(struct bintime *bt)
{
struct timehands *th;
u_int gen;
struct timecounter *tc; // HACK!!!
u_int tim_cnt, tim_offset, tim_diff; // HACK!!!
uint64_t freq, scale_factor, diff_scaled; // HACK!!!
do {
th =3D timehands;
tc =3D th->th_counter; // HACK!!!
gen =3D atomic_load_acq_int(&th->th_generation);
*bt =3D th->th_offset;
tim_cnt=3D tc->tc_get_timecount(tc); // HACK!!! (steps =
of tc_diff with values recorded)
tim_offset=3D th->th_offset_count; // HACK!!!
tim_diff=3D (tim_cnt - tim_offset) & =
tc->tc_counter_mask; // HACK!!!
scale_factor=3D th->th_scale; // HACK!!!
diff_scaled=3D scale_factor * tim_diff; // HACK!!!
bintime_addx(bt, diff_scaled); // HACK!!!
freq=3D tc->tc_frequency; // HACK!!!
atomic_thread_fence_acq();
} while (gen =3D=3D 0 || gen !=3D th->th_generation);
if (*(volatile uint64_t*)0xc0000000000000f0=3D=3D0u && =
(0xffffffffffffffffull/scale_factor)<tim_diff) { // HACK!!!
*(volatile uint64_t*)0xc0000000000000d0=3D freq;
*(volatile uint64_t*)0xc0000000000000d8=3D scale_factor;
*(volatile u_int*)0xc0000000000000e0=3D tim_offset;
*(volatile u_int*)0xc0000000000000e4=3D tim_cnt;
*(volatile u_int*)0xc0000000000000e8=3D tim_diff;
*(volatile uint64_t*)0xc0000000000000f0=3D diff_scaled;
*(volatile uint64_t*)0xc0000000000000f8=3D =
scale_factor*freq;
__asm__ ("sync");
}
}
#else
. . .
#endif
(mtfb() is used to provide the tc->tc_get_timecount(tc)
value --but only the lower 32 bits are extracted and
returned.)
Basically whenever tim_diff is such that:
(0xffffffffffffffff/scale_factor)<tim_dif
then diff_scaled overflows an unsigned, 64 bit representation,
ending up with just the least 64 bits. This truncated value
ends up being used in:
bintime_addx(bt, diff_scaled);
Observed consistently for tc->tc_frequency:
tc->tc_frequency =3D=3D 0x1fca055 (i.e., 33333333)
( tc->tc_counter_mask is 0xfffffffful as well. )
An example observation of diff_scaled having an overflowed
value is:
scale_factor =3D=3D 0x80da2067ac
scale_factor*freq overflows unsigned, 64 bit representation.
tim_offset =3D=3D 0x3da0eaeb
tim_cnt =3D=3D 0x42dea3c4
tim_diff =3D=3D 0x53db8d9
For reference: 0x1fc9d43 =3D=3D =
0xffffffffffffffffull/scale_factor
scaled_diff =3D=3D 0xA353A5BF3FF780CC (truncated to 64 bits)
So scale_factor * tim_diff leaves diff_scaled truncated to
the least significant 64 bits, which does not preserve
ordering properties.
Another example:
scale_factor =3D=3D 0x80d95962c0
scale_factor*freq =3D=3D 0xfffffffffd65c9c0
tim_offset =3D=3D 0x4d1fb8e2
tim_cnt =3D=3D 0x4d1fb8e1
tim_diff =3D=3D 0xffffffff
For reference: 0x1fca055 =3D=3D =
0xffffffffffffffffull/scale_factor
scaled_diff =3D=3D 0xD959623F26A69D40 (truncated to 64 bits)
Again the diff_scaled holds a truncated value from
scale_factor * tim_diff .
Another example:
scale_factor =3D=3D 0x80da20c940
scale_factor*freq overflows unsigned, 64 bit representation.
tim_offset =3D=3D 0x9a7f5cdb
tim_cnt =3D=3D 0xb26bbd5
tim_diff =3D=3D 0x70a75efa
For reference: 0x1fc9d41 =3D=3D =
0xffffffffffffffffull/scale_factor
scaled_diff =3D=3D 0xB3AC715C56AA0880 (truncated to 64 bits)
Again the diff_scaled holds a truncated value from
scale_factor * tim_diff .
Note that the scale_factor does vary.
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?D3D7E9F4-9A5E-4320-B3C8-EC5CEF4A2764>