From owner-freebsd-hackers@freebsd.org Thu Feb 28 13:06:30 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 49A051506B0A for ; Thu, 28 Feb 2019 13:06:30 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic313-19.consmr.mail.gq1.yahoo.com (sonic313-19.consmr.mail.gq1.yahoo.com [98.137.65.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0284C6A03E for ; Thu, 28 Feb 2019 13:06:28 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: uS5nA9MVM1kgQOAg.OjvK9ohSvowwPCWEH6z6nkRCCbAtSGxWRzU21GZFDeY.to vy76VoNkeSGzQVh3mE0ao1t5uPYaxRZw3V18KZjQTCej4MovNEmKPdqfPcmz014J8CjMh0050Fdw YBcrNXcIrZfdfKldTC41dPrPQD4LtPf3POGVdQnbt5NPif9xpQnLIHy4wgdKbPFSDq6VeobOykEv pZAxijmxTDvU.Ove_piQ3gCM3r0GHI6F_AuZ.wWP7RIKiH9RRdwoitB.K1jfCWOaBhSHH8X7bxAp pmBQRiR5KudKoWvTCq2aXNX3PtXGwYfpBCou_rVjU11_eYjywNbbjy6yXqjc0fZmB1oYm60z1m6z owghgL0y7EerB0zwLebae6TpelC_Iv8Wa533DHtN8I5kVMem3JKlt10C.YxYBk00U41_SjDkN69e OjRcBJj8iBKc29tFoCJyhVh4NtalXSy1eyAEgxGcIuXC78kFNGpTeiKIyFQmqh_FD5KZx_o7LHFB C81K2a6UglMDl8p_NpnzuifeqCTMwVGmPIBx9xEfZhLVO8KsVFRXnWoqcmznh7PZHLEXX3GCYrEG YgbXv99F4mDYemczik_3M.WtCwdS1jVQzAKGVpdQy_yxqbYatk8VavJ7HRnHl7U05oSEmNdrlHHL SR33J98IAywWjF4PDndC7yezqyq2XOVwrSf0sXc1QLbBE_VPvnhkSH6e8WZhXSGEg81pg8BHs5vZ XygZmGJZUCrRThdu8Sa.xAQmb0mnvRjb9BtKCdSLdN7dm8XDHI.3EAFpZJZe6S1nzeh_LEd7dBCM d0rx.fr.Owq.K9g0QH6dsb.hX1LN5HpyNMlgQcajUSOhBgjJiXZoNyX1e1Rn8vJFK9mMgAuM6Kel _yXJky5nwV4cTvy6b_L_mAMWT1H1iOjWxNqoALuKCiXCTNZ.lg4hxicogP7dXyK8v5Wbw1guER.u acb6WRfSIPMAGRUdPMJPmLQtJDFVJwnhjUqWgIk7z0vds3K_W8g5xgljdmFxKdBsg6qrjGMmzBXs hxynt3cPlxvncCJJ6aabDL_ztJ5BVp_aZK7t6MlxM4xNc0iBkus_sTetaeSUG7NlveHIJMdiO Received: from sonic.gate.mail.ne1.yahoo.com by sonic313.consmr.mail.gq1.yahoo.com with HTTP; Thu, 28 Feb 2019 13:06:26 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.115]) ([67.170.167.181]) by smtp405.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 9137d10ee3f5edb79ba565dfae703c50; Thu, 28 Feb 2019 13:06:24 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes Message-Id: Date: Thu, 28 Feb 2019 05:06:23 -0800 Cc: freebsd-hackers Hackers To: FreeBSD PowerPC ML X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: 0284C6A03E X-Spamd-Bar: +++ X-Spamd-Result: default: False [3.80 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.97)[0.970,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.77)[ip: (7.09), ipnet: 98.137.64.0/21(1.01), asn: 36647(0.81), country: US(-0.07)]; NEURAL_SPAM_MEDIUM(0.66)[0.664,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.91)[0.911,0]; RCVD_IN_DNSWL_NONE(0.00)[82.65.137.98.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Feb 2019 13:06:30 -0000 Basic context: The code for sleeps of various forms depends on calls to: static __inline sbintime_t sbinuptime(void) { struct bintime _bt; binuptime(&_bt); return (bttosbt(_bt)); } and comparisons with the return values, such as checking for timeouts. The upper 32 bits of the unsigned 64 bit result has the seconds and the lower 32 bits has the fraction as a multiplier of 1sec/(2**64). An observed problem is that later sbinuptime calls sometimes end up with smaller values than earlier ones. (Past lisy freebsd-ppc messages give details.) This makes for problems with checking for timeouts when using later sbinuptime() calls after a timeout was initially detected against an earlier value: A.0) timercb getting the earlier sbinuptime() value A.1) callout_process using that to detect a timeout, B) sleepq_timeout checking the timeout again, using a separate sbinuptime() call. Some details about example values, overflows, and such follow. I used the following sort of hacked code to report values when overflows happen: #if defined(__powerpc64__) && defined(AIM) void binuptime(struct bintime *bt) { struct timehands *th; u_int gen; struct timecounter *tc; // HACK!!! u_int tim_cnt, tim_offset, tim_diff; // HACK!!! uint64_t freq, scale_factor, diff_scaled; // HACK!!! do { th =3D timehands; tc =3D th->th_counter; // HACK!!! gen =3D atomic_load_acq_int(&th->th_generation); *bt =3D th->th_offset; tim_cnt=3D tc->tc_get_timecount(tc); // HACK!!! (steps = of tc_diff with values recorded) tim_offset=3D th->th_offset_count; // HACK!!! tim_diff=3D (tim_cnt - tim_offset) & = tc->tc_counter_mask; // HACK!!! scale_factor=3D th->th_scale; // HACK!!! diff_scaled=3D scale_factor * tim_diff; // HACK!!! bintime_addx(bt, diff_scaled); // HACK!!! freq=3D tc->tc_frequency; // HACK!!! atomic_thread_fence_acq(); } while (gen =3D=3D 0 || gen !=3D th->th_generation); if (*(volatile uint64_t*)0xc0000000000000f0=3D=3D0u && = (0xffffffffffffffffull/scale_factor)tc_get_timecount(tc) value --but only the lower 32 bits are extracted and returned.) Basically whenever tim_diff is such that: (0xffffffffffffffff/scale_factor)tc_frequency: tc->tc_frequency =3D=3D 0x1fca055 (i.e., 33333333) ( tc->tc_counter_mask is 0xfffffffful as well. ) An example observation of diff_scaled having an overflowed value is: scale_factor =3D=3D 0x80da2067ac scale_factor*freq overflows unsigned, 64 bit representation. tim_offset =3D=3D 0x3da0eaeb tim_cnt =3D=3D 0x42dea3c4 tim_diff =3D=3D 0x53db8d9 For reference: 0x1fc9d43 =3D=3D = 0xffffffffffffffffull/scale_factor scaled_diff =3D=3D 0xA353A5BF3FF780CC (truncated to 64 bits) So scale_factor * tim_diff leaves diff_scaled truncated to the least significant 64 bits, which does not preserve ordering properties. Another example: scale_factor =3D=3D 0x80d95962c0 scale_factor*freq =3D=3D 0xfffffffffd65c9c0 tim_offset =3D=3D 0x4d1fb8e2 tim_cnt =3D=3D 0x4d1fb8e1 tim_diff =3D=3D 0xffffffff For reference: 0x1fca055 =3D=3D = 0xffffffffffffffffull/scale_factor scaled_diff =3D=3D 0xD959623F26A69D40 (truncated to 64 bits) Again the diff_scaled holds a truncated value from scale_factor * tim_diff . Another example: scale_factor =3D=3D 0x80da20c940 scale_factor*freq overflows unsigned, 64 bit representation. tim_offset =3D=3D 0x9a7f5cdb tim_cnt =3D=3D 0xb26bbd5 tim_diff =3D=3D 0x70a75efa For reference: 0x1fc9d41 =3D=3D = 0xffffffffffffffffull/scale_factor scaled_diff =3D=3D 0xB3AC715C56AA0880 (truncated to 64 bits) Again the diff_scaled holds a truncated value from scale_factor * tim_diff . Note that the scale_factor does vary. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)