From owner-freebsd-ppc@freebsd.org Sun Mar 24 11:41:53 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C69461556471 for ; Sun, 24 Mar 2019 11:41:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 5602B80932 for ; Sun, 24 Mar 2019 11:41:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 150F41556470; Sun, 24 Mar 2019 11:41:53 +0000 (UTC) Delivered-To: ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E50CD155646F for ; Sun, 24 Mar 2019 11:41:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 41F4F8092C for ; Sun, 24 Mar 2019 11:41:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 56042C737 for ; Sun, 24 Mar 2019 11:41:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x2OBfp4f088294 for ; Sun, 24 Mar 2019 11:41:51 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x2OBfppE088291 for ppc@FreeBSD.org; Sun, 24 Mar 2019 11:41:51 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: ppc@FreeBSD.org Subject: [Bug 233863] r345425 on PowerMac G5 may require kern.smp.disabled=1 and must set usefdt=1 which causes net interface reorder Date: Sun, 24 Mar 2019 11:41:51 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: dclarke@blastwave.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ppc@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_severity short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Mar 2019 11:41:54 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D233863 Dennis Clarke changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|Affects Only Me |Affects Some People Summary|r341705 on PowerMac G5 may |r345425 on PowerMac G5 may |require kern.smp.disabled=3D1 |require kern.smp.disable= d=3D1 |and must set usefdt=3D1 which |and must set usefdt=3D1 = which |causes net interface |causes net interface |reorder |reorder --- Comment #5 from Dennis Clarke --- 123456789+123456789+123456789+123456789+123456789+123456789+123456789+12 This feels like the bug that simply won't go away however I did a test build of r345425 and no amount of coaxing nor kern.smp.disabled would get past fans roaring after seeing a wake up call to CPU3. A step back to r344744 seems to work fine provided that one provide the usefdt=3D1 and then boot -sv : hydra$ uname -a=20 FreeBSD hydra 13.0-CURRENT FreeBSD 13.0-CURRENT r344744 GENERIC powerpc hydra$ sysctl -a kern.smp.cpus kern.smp.cpus: 4 hydra$=20 Also bge0 and bge1 network interfaces are reversed however this is=20 merely an annoyance :=20 bge0: flags=3D8843 metric 0 mtu 1500 =20=20=20=20=20=20=20 options=3D8009b ether 00:14:51:64:67:11 inet 172.16.35.8 netmask 0xffffffc0 broadcast 172.16.35.63 nd6 options=3D29 media: Ethernet autoselect (1000baseT ) status: active bge1: flags=3D8802 metric 0 mtu 1500 =20=20=20=20=20=20=20 options=3D8009b ether 00:14:51:64:67:10 nd6 options=3D29 media: Ethernet autoselect hydra$ date -u Sun Mar 24 11:41:28 UTC 2019 --=20 Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken GreyBeard and suspenders optional --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-ppc@freebsd.org Sun Mar 24 11:01:48 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4F0811554ECD; Sun, 24 Mar 2019 11:01:48 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8787777643; Sun, 24 Mar 2019 11:01:47 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x2OB1cmi085645 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 24 Mar 2019 13:01:41 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x2OB1cmi085645 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x2OB1cIF085644; Sun, 24 Mar 2019 13:01:38 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 24 Mar 2019 13:01:38 +0200 From: Konstantin Belousov To: Bruce Evans Cc: Mark Millard , freebsd-hackers Hackers , FreeBSD PowerPC ML Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] Message-ID: <20190324110138.GR1923@kib.kiev.ua> References: <20190303111931.GI68879@kib.kiev.ua> <20190303223100.B3572@besplex.bde.org> <20190303161635.GJ68879@kib.kiev.ua> <20190304043416.V5640@besplex.bde.org> <20190304114150.GM68879@kib.kiev.ua> <20190305031010.I4610@besplex.bde.org> <20190306172003.GD2492@kib.kiev.ua> <20190308001005.M2756@besplex.bde.org> <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190309144844.K1166@besplex.bde.org> User-Agent: Mutt/1.11.4 (2019-03-13) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Mar 2019 11:01:48 -0000 On Sat, Mar 09, 2019 at 06:00:14PM +1100, Bruce Evans wrote: > I more strongly disclike (sic) the more complete merge. The central APIs > have even more parameters and reduced type safety to describe objects as > (offset, size) pairs. I changed the patch to be type-safe. Now I like it even more. It provides 1. internal 2. concise 3. type-safe API to fetch data from timehands. The implementation needs to be read only once. > Small delicate loops are ideal for duplicating. They are easier to > understand individually and short enough to compare without using diff > to see gratuitous and substantive differences. Multiple instances are > only hard to write and maintain. Since these multiple instances are > already written, they are only harder to maintain. This is a good argument to have bintime_off and getthmember unmerged (there are two small but delicate loops). The API is internal, so it is only matter for maintainer, which means that the absence of duplication is important. More, all that arguments clearly explain why there should be not twenty similar loops scattered over the source. > > >> XX void > >> XX binuptime(struct bintime *bt) > >> XX { > >> XX @@ -361,7 +383,7 @@ > >> XX th = timehands; > >> XX gen = atomic_load_acq_int(&th->th_generation); > >> XX *bt = th->th_offset; > >> XX - bintime_addx(bt, th->th_scale * tc_delta(th)); > >> XX + bintime_adddelta(bt, th); > >> XX atomic_thread_fence_acq(); > >> XX } while (gen == 0 || gen != th->th_generation); > >> XX } > >> > >> This is the kind of non-churning change that I like. > > Ok. I made all cases where timehands are read, more uniform by > > moving calculations after the generation loop. This makes the > > atomic part of the functions easier to see, and loop body has lower > > chance to hit generation reset. > > I think this change is slightly worse: > - it increases register pressure. 'scale' and 'delta' must be read in a > alost program program before the loop exit test. The above order uses > them and stores the results to memory, so more registers are free for > the exit test. i386 certainly runs out of registers. IIRC, i386 now > spills 'gen'. It would have to spill something to load 'gen' or 'th' > for the test. Which does not matter on any modern architecture anyway. > - it enlarges the window between reading 'scale' and 'delta' and the > caller seeing the results. Preemption in this window gives results > that may be far in the past. My opinion is that quickly exiting the code and avoid retry is more important (as in performance) than making an impression that we protect against preemption. If preemption is important for the caller, then the calling place must use some measures like interrupt disabling and re-checking the time after the operation. Preemption can occur after the loop exit with the same consequences. > The 'get' name is another problem. I would like all the get*time > functions and not add new names starting with 'get'. The library > implementation already doesn't bother optimizing the get*time functions, > but always uses the hardware timecounter. > > getfoo() is a more natural name than foo_get() for the action of getting > foo, but the latter is better for consistency, especially in code that > puts the subsystem name first in nearby code. > > The get*time functions would be better if they were more like > time_second. Note that time_second is racy if time_t is too larger > for the arch so that accesses to it are not atomic, as happens on > 32-bit arches with premature 64-bit time_t. However, in this 32/64 > case, the race is only run every 136 years, with the next event > scheduled in 2038, so this race is even less important now than other > events scheduled in 2038. Bintimes are 96 or 128 bits, so directly > copying a global like time_second for them would race every 1/2**32 > second on 2-bit arches or every 1 second on 64-bit arches. Most of > the loops on the generation count are for fixing these races, but > perhaps a simpler method would work. On 64-bit arches with atomic > 64 accesses on 32-bit boundaries, the following would work: > - set the lower 32 bits of the fraction to 0, or ignore them > - load the higher 32 bits of the fraction and the lower 32 bits of the > seconds > - race once every 136 years starting in 2038 reading the higher 32 bits > of the seconds non-atomically. > - alternatively, break instead of racing in 2038 by setting the higher > 32 bits to 0. This is the same as using sbintimes instead of bintimes. > - drop a few more lower bits by storing a right-shifted value. Right > shifting by just 1 gives a race frequency of once per 272 years, with > the next one in 2006. It would make sense if the functions were written from scratch, but since we already have the generation counts, it is not obvious that such change is useful. But if we decide to go that route (later), my current patch only requires exactly one location getthmember() to experiment with and to change after. So you effectively made yet another, and perhaps most convincing, argument, for me. > > > diff --git a/sys/kern/kern_tc.c b/sys/kern/kern_tc.c > > index 2656fb4d22f..8d12847f2cd 100644 > > --- a/sys/kern/kern_tc.c > > +++ b/sys/kern/kern_tc.c > > @@ -200,20 +202,56 @@ tc_delta(struct timehands *th) > > * the comment in for a description of these 12 functions. > > */ > > > > -#ifdef FFCLOCK > > -void > > -fbclock_binuptime(struct bintime *bt) > > +static __inline void > > +bintime_off(struct bintime *bt, u_int off) > > { > > struct timehands *th; > > - unsigned int gen; > > + struct bintime *btp; > > + uint64_t scale, x; > > + u_int delta, gen, large_delta; > > > > do { > > th = timehands; > > gen = atomic_load_acq_int(&th->th_generation); > > - *bt = th->th_offset; > > - bintime_addx(bt, th->th_scale * tc_delta(th)); > > You didn't fully obfuscate this by combinining this function with > getthmember() so as to deduplicate the loop. > > > + btp = (struct bintime *)((vm_offset_t)th + off); > > Ugly conversion to share code. This is technically incorrect. Improving > the casts gives: > > btp = (void *)(uintptr_t)((uintptr_t)(void *)th + off); > > but this assumes that arithmetic on the intermediate integer does what > is espected. uintptr_t is only guaranteed to work when the intermediate > representation held in it is not adjusted. vm_offset_t has the semantic that is needed for the arithmetic. It is better uintptr_t for kernel, where we know that all object pointers are compatible with vm_offset_t (AKA flat tag-less memory model). > > Fixing the API gives > > static __inline void > bintime_off(struct bintime *btp, struct bintime *base_btp) > > where base_btp is &th->th_bintime or &th->th_offset. > > (th_offset and th_bintime are badly named. th_offset is really a base > time and the offset is tc_delta(). th_bintime is also a base time. > It is the same as th_offset with another actual offset (the difference > between UTC and local time) already added to it as an optimization. In > old versions, th_bintime didn't exist, but the related struct members > th_nanotime and th_microtime existed, since these benefit more from > not converting on every call. How could it be &th->th_offset, when th is calculated inside the call ? But I did modified the API in this spirit, indeed. It takes the member name directly as an argument. > > My old version even documents the struct members, while -current still > has no comments. The comments were lost to staticization. My version > mostly adds "duh" to the banal comments after recovering them: > > XX /* > XX * XXX rotted comment cloned from . > XX * > XX * th_counter is undocumented (duh). > XX * > XX * th_adjustment [PPM << 16] which means that the smallest unit of correction > XX * you can apply amounts to 481.5 usec/year. > XX * > XX * th_scale is undocumented (duh). > XX * > XX * th_offset_count is the contents of the counter which corresponds to the > XX * > XX * rest of the offset_* values. > XX * > XX * th_offset is undocumented (duh). > XX * > XX * th_microtime is undocumented (duh). > XX * > XX * th_nanotime is undocumented (duh). > XX * > XX * XXX especially massive bitrot here. "three" is now "many"... > XX * Each timecounter must supply an array of three timecounters. This is needed > XX * to guarantee atomicity in the code. Index zero is used to transport > XX * modifications, for instance done with sysctl, into the timecounter being > XX * used in a safe way. Such changes may be adopted with a delay of up to 1/HZ. > XX * Index one and two are used alternately for the actual timekeeping. > XX * > XX * th_generation is undocumented (duh). > XX * > XX * th_next is undocumented (duh). > XX */ > > > + *bt = *btp; > > + scale = th->th_scale; > > + delta = tc_delta(th); > > + large_delta = th->th_large_delta; > > I had forgotten that th_scale is so volatile (it may be adjusted on > every windup). th_large_delta is equally volatile. So moving the > calculation outside of the loop gives even more register pressure > than I noticed above. > > > atomic_thread_fence_acq(); > > } while (gen == 0 || gen != th->th_generation); > > + > > + if (__predict_false(delta < large_delta)) { > > + /* Avoid overflow for scale * delta. */ > > + x = (scale >> 32) * delta; > > + bt->sec += x >> 32; > > + bintime_addx(bt, x << 32); > > + bintime_addx(bt, (scale & 0xffffffff) * delta); > > + } else { > > + bintime_addx(bt, scale * delta); > > + } > > +} > > + > > +static __inline void > > +getthmember(void *out, size_t out_size, u_int off) > > +{ > > + struct timehands *th; > > + u_int gen; > > + > > + do { > > + th = timehands; > > + gen = atomic_load_acq_int(&th->th_generation); > > + memcpy(out, (char *)th + off, out_size); > > This isn't so ugly or technically incorrect. Now the object is generic, > so the reference to it should be passed as (void *objp, size_t objsize) > instead of the type-safe (struct bintime *base_bpt). _Generic is what gave me a hint how to make the implementation type-safe. diff --git a/sys/kern/kern_tc.c b/sys/kern/kern_tc.c index 2656fb4d22f..4e94f762026 100644 --- a/sys/kern/kern_tc.c +++ b/sys/kern/kern_tc.c @@ -72,6 +72,7 @@ struct timehands { struct timecounter *th_counter; int64_t th_adjustment; uint64_t th_scale; + u_int th_large_delta; u_int th_offset_count; struct bintime th_offset; struct bintime th_bintime; @@ -90,6 +91,7 @@ static struct timehands th1 = { static struct timehands th0 = { .th_counter = &dummy_timecounter, .th_scale = (uint64_t)-1 / 1000000, + .th_large_delta = 1000000, .th_offset = { .sec = 1 }, .th_generation = 1, .th_next = &th1 @@ -200,20 +202,72 @@ tc_delta(struct timehands *th) * the comment in for a description of these 12 functions. */ -#ifdef FFCLOCK -void -fbclock_binuptime(struct bintime *bt) +static __inline void +bintime_off(struct bintime *bt, u_int off) { struct timehands *th; - unsigned int gen; + struct bintime *btp; + uint64_t scale, x; + u_int delta, gen, large_delta; do { th = timehands; gen = atomic_load_acq_int(&th->th_generation); - *bt = th->th_offset; - bintime_addx(bt, th->th_scale * tc_delta(th)); + btp = (struct bintime *)((vm_offset_t)th + off); + *bt = *btp; + scale = th->th_scale; + delta = tc_delta(th); + large_delta = th->th_large_delta; atomic_thread_fence_acq(); } while (gen == 0 || gen != th->th_generation); + + if (__predict_false(delta >= large_delta)) { + /* Avoid overflow for scale * delta. */ + x = (scale >> 32) * delta; + bt->sec += x >> 32; + bintime_addx(bt, x << 32); + bintime_addx(bt, (scale & 0xffffffff) * delta); + } else { + bintime_addx(bt, scale * delta); + } +} +#define GETTHBINTIME(dst, member) \ +do { \ + _Static_assert(_Generic(((struct timehands *)NULL)->member, \ + struct bintime: 1, default: 0) == 1, \ + "struct timehands member is not of struct bintime type"); \ + bintime_off(dst, __offsetof(struct timehands, member)); \ +} while (0) + +static __inline void +getthmember(void *out, size_t out_size, u_int off) +{ + struct timehands *th; + u_int gen; + + do { + th = timehands; + gen = atomic_load_acq_int(&th->th_generation); + memcpy(out, (char *)th + off, out_size); + atomic_thread_fence_acq(); + } while (gen == 0 || gen != th->th_generation); +} +#define GETTHMEMBER(dst, member) \ +do { \ + _Static_assert(_Generic(*dst, \ + __typeof(((struct timehands *)NULL)->member): 1, \ + default: 0) == 1, \ + "*dst and struct timehands member have different types"); \ + getthmember(dst, sizeof(*dst), __offsetof(struct timehands, \ + member)); \ +} while (0) + +#ifdef FFCLOCK +void +fbclock_binuptime(struct bintime *bt) +{ + + GETTHBINTIME(bt, th_offset); } void @@ -237,16 +291,8 @@ fbclock_microuptime(struct timeval *tvp) void fbclock_bintime(struct bintime *bt) { - struct timehands *th; - unsigned int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *bt = th->th_bintime; - bintime_addx(bt, th->th_scale * tc_delta(th)); - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHBINTIME(bt, th_bintime); } void @@ -270,100 +316,55 @@ fbclock_microtime(struct timeval *tvp) void fbclock_getbinuptime(struct bintime *bt) { - struct timehands *th; - unsigned int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *bt = th->th_offset; - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(bt, th_offset); } void fbclock_getnanouptime(struct timespec *tsp) { - struct timehands *th; - unsigned int gen; + struct bintime bt; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - bintime2timespec(&th->th_offset, tsp); - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(&bt, th_offset); + bintime2timespec(&bt, tsp); } void fbclock_getmicrouptime(struct timeval *tvp) { - struct timehands *th; - unsigned int gen; + struct bintime bt; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - bintime2timeval(&th->th_offset, tvp); - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(&bt, th_offset); + bintime2timeval(&bt, tvp); } void fbclock_getbintime(struct bintime *bt) { - struct timehands *th; - unsigned int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *bt = th->th_bintime; - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(bt, th_bintime); } void fbclock_getnanotime(struct timespec *tsp) { - struct timehands *th; - unsigned int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *tsp = th->th_nanotime; - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(tsp, th_nanotime); } void fbclock_getmicrotime(struct timeval *tvp) { - struct timehands *th; - unsigned int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *tvp = th->th_microtime; - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(tvp, th_microtime); } #else /* !FFCLOCK */ + void binuptime(struct bintime *bt) { - struct timehands *th; - u_int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *bt = th->th_offset; - bintime_addx(bt, th->th_scale * tc_delta(th)); - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHBINTIME(bt, th_offset); } void @@ -387,16 +388,8 @@ microuptime(struct timeval *tvp) void bintime(struct bintime *bt) { - struct timehands *th; - u_int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *bt = th->th_bintime; - bintime_addx(bt, th->th_scale * tc_delta(th)); - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHBINTIME(bt, th_bintime); } void @@ -420,85 +413,47 @@ microtime(struct timeval *tvp) void getbinuptime(struct bintime *bt) { - struct timehands *th; - u_int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *bt = th->th_offset; - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(bt, th_offset); } void getnanouptime(struct timespec *tsp) { - struct timehands *th; - u_int gen; + struct bintime bt; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - bintime2timespec(&th->th_offset, tsp); - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(&bt, th_offset); + bintime2timespec(&bt, tsp); } void getmicrouptime(struct timeval *tvp) { - struct timehands *th; - u_int gen; + struct bintime bt; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - bintime2timeval(&th->th_offset, tvp); - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(&bt, th_offset); + bintime2timeval(&bt, tvp); } void getbintime(struct bintime *bt) { - struct timehands *th; - u_int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *bt = th->th_bintime; - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(bt, th_bintime); } void getnanotime(struct timespec *tsp) { - struct timehands *th; - u_int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *tsp = th->th_nanotime; - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(tsp, th_nanotime); } void getmicrotime(struct timeval *tvp) { - struct timehands *th; - u_int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *tvp = th->th_microtime; - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(tvp, th_microtime); } #endif /* FFCLOCK */ @@ -514,15 +469,8 @@ getboottime(struct timeval *boottime) void getboottimebin(struct bintime *boottimebin) { - struct timehands *th; - u_int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *boottimebin = th->th_boottime; - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(boottimebin, th_boottime); } #ifdef FFCLOCK @@ -1038,15 +986,8 @@ getmicrotime(struct timeval *tvp) void dtrace_getnanotime(struct timespec *tsp) { - struct timehands *th; - u_int gen; - do { - th = timehands; - gen = atomic_load_acq_int(&th->th_generation); - *tsp = th->th_nanotime; - atomic_thread_fence_acq(); - } while (gen == 0 || gen != th->th_generation); + GETTHMEMBER(tsp, th_nanotime); } /* @@ -1464,6 +1405,7 @@ tc_windup(struct bintime *new_boottimebin) scale += (th->th_adjustment / 1024) * 2199; scale /= th->th_counter->tc_frequency; th->th_scale = scale * 2; + th->th_large_delta = MIN(((uint64_t)1 << 63) / scale, UINT_MAX); /* * Now that the struct timehands is again consistent, set the new From owner-freebsd-ppc@freebsd.org Sun Mar 24 19:10:17 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3C34B153324E for ; Sun, 24 Mar 2019 19:10:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id AF96C926CB for ; Sun, 24 Mar 2019 19:10:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 7074F153324C; Sun, 24 Mar 2019 19:10:16 +0000 (UTC) Delivered-To: ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4C7EF153324A for ; Sun, 24 Mar 2019 19:10:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DB840926C4 for ; Sun, 24 Mar 2019 19:10:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 2F72710731 for ; Sun, 24 Mar 2019 19:10:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x2OJAFC8074953 for ; Sun, 24 Mar 2019 19:10:15 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x2OJAFDT074951 for ppc@FreeBSD.org; Sun, 24 Mar 2019 19:10:15 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: ppc@FreeBSD.org Subject: [Bug 233863] r345425 on PowerMac G5 may require kern.smp.disabled=1 and must set usefdt=1 which causes net interface reorder Date: Sun, 24 Mar 2019 19:10:14 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: marklmi26-fbsd@yahoo.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ppc@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Mar 2019 19:10:17 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D233863 --- Comment #6 from Mark Millard --- (In reply to Dennis Clarke from comment #5) The following notes are currently based on variations on head -r335044 . It has usefdt=3D1 enabled always (by changing the code) but does not have smp disabled (even manually). The context does not involve reverting the newer kernel maximum figure that I used to revert. I have a hack that avoid the fans problem on the 2-socket/2cores-each G5 that I currently have access to. It also avoids problems with other processes that also get stuck waiting for sleep. But it is a hack that is not even approximately a general/proper solution to the original problem, it just avoids the consequences. The hack involves an experimentally found approximate bound that it tests against. That bound might vary for related G5 contexts. I've another hack that makes it rare that booting gets hung up: it narrows a race condition's time frame. But I've have observed a few examples of hang-up out of hundreds of boots. The hack basically proves what is going on for the hang-ups based on the large change of behavior for narrowing the race-condition's time frame. There is a patch-review active for llvm's libunwind that makes it handle r2 (the TOC register) correctly for powerpc64 both for system-clang based buildworld's and devel/powerpc64-gcc based buildworld's, both ELFv1 ABI and ELFv2 ABI. This makes thrown C++ exceptions work. (This does not cover WITH_LIB32=3D for exception handling when WITH_LLVM_LIBUNWIND=3D is in use in the buildworld.) --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-ppc@freebsd.org Fri Mar 29 20:25:39 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 312271551A24 for ; Fri, 29 Mar 2019 20:25:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C90258090F for ; Fri, 29 Mar 2019 20:25:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by freefall.freebsd.org (Postfix) id A055914940; Fri, 29 Mar 2019 20:25:38 +0000 (UTC) Delivered-To: powerpc@localmail.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [96.47.72.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client CN "mx1.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by freefall.freebsd.org (Postfix) with ESMTPS id 9D2F81493F for ; Fri, 29 Mar 2019 20:25:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DDB58090D for ; Fri, 29 Mar 2019 20:25:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 78F8E13580 for ; Fri, 29 Mar 2019 20:25:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x2TKPbIq049503 for ; Fri, 29 Mar 2019 20:25:37 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x2TKPb8v049502 for powerpc@FreeBSD.org; Fri, 29 Mar 2019 20:25:37 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: powerpc@FreeBSD.org Subject: [Bug 232387] head -r339076: system crash in vnet_epair_init during kern_jail_set in a kyua test on powerpc64 Date: Fri, 29 Mar 2019 20:25:37 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: vimage X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: marklmi26-fbsd@yahoo.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: powerpc@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: C90258090F X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.99 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.99)[-0.992,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Mar 2019 20:25:39 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D232387 --- Comment #14 from Mark Millard --- I finally have a system-clang devel/powerpc64-binutils based build that I've tested ( head -r345558 based, ELFv1 ABI ). Again I'm off experimental futures. This was with a non-debug kernel. This too fails. Running just: kyua test -k /usr/tests/Kyuafile sys/netipsec/tunnel/aes_cbc_128_hmac_sha1 is sufficient for the crash to happen but it fails when a full Kyua test gets there. Again a data storage interrupt: ( Be warned: hand typed from a picture.) exception =3D 0x300 (data storage interrupt) virtual address=3D 0x860ce198 dsisr =3D 0x42000000 srr0 =3D 0xc0000000007b51a4 (0x7b51a4) srr1 =3D 0x9000000000009032 current msr =3D 0x9000000000009032 lr =3D 0xc0000000007184a8 (0x7184a8) frame =3D 0xe00000008ecb0dd0 curthread =3D 0xc00000000ad6c580 pid =3D 28966, comm =3D ifconfig Panic: data storage interrupt trap cpuid=3D 2 time =3D 1553886317 The frmae, curthread, pid, cpuid, and time can vary. The backtrace information is better this time. Be warned: hand typed from a picture. The example is from a full kyua run. KDB: stack backtrace: 0xe000000008eb0b00: at vpanic+0x1d8 0xe000000008eb0bb0: at panic+0x44 0xe000000008eb0be0: at trap_fatal+0x2f4 0xe000000008eb0c70: at trap+0x698 0xe000000008eb0d50: at powerpc_interrupt+0x1a0 0xe000000008eb0da0: at kernel DSI write trap @ 0x860ce198 by lock_init+0x140: srr1 =3D 0x9000000000009032 r1 =3D 0xe00000008ecb1050 cr =3D 0x22880f488 xer =3D 0 ctr =3D 0xc000000000718438 r2 =3D 0xc000000001370000 sr =3D 0x42000000 frame=3D 0xe00000008ecb0dd0 0xe00000008ecb1050: at __set_sysctl_set_sym_sysctl___net_link_epair_netisr_maxqlen+0x4 0xe00000008ecb1080: at epair_modeevent+0xbc 0xe00000008ecb1140: at module_register_init+0x130 0xe00000008ecb11f0: at linker_load_module+0xd88 0xe00000008ecb1620: at kern_kldload+0x8c 0xe00000008ecb1690: at sys_kldload+0x8c 0xe00000008ecb16e0: at trap+0xb28 0xe00000008ecb17c0: at powerpc_interrupt+0x1a0 0xe00000008ecb1810: at user SC trap by 0x810182da8: srr1 =3D 0x900000000200f032 r1 =3D 0x3fffffffffffcf10 cr =3D 0x24002200 xer =3D 0x20000000 ctr =3D 0x810182da0 r2 =3D 0x81033b900 frame=3D 0xe00000008ecb1840 Note: The 'at __set_sysctl_set_sym_sysctl___net_link_eapir_netisr_maxqlen+0= x4' at other times has shown text such as 'at 0xffffffc'. The kernel stack addresses (0xe000 prefixes) can vary. Otherwise the backtraces agree so far as I've noticed. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-ppc@freebsd.org Fri Mar 29 21:16:32 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1BAAD1552FC7 for ; Fri, 29 Mar 2019 21:16:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B348A82657 for ; Fri, 29 Mar 2019 21:16:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by freefall.freebsd.org (Postfix) id 8A5FA15471; Fri, 29 Mar 2019 21:16:31 +0000 (UTC) Delivered-To: powerpc@localmail.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [96.47.72.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client CN "mx1.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by freefall.freebsd.org (Postfix) with ESMTPS id 8746015470 for ; Fri, 29 Mar 2019 21:16:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 332EA82654 for ; Fri, 29 Mar 2019 21:16:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 44EFF13CD4 for ; Fri, 29 Mar 2019 21:16:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x2TLGUXM021106 for ; Fri, 29 Mar 2019 21:16:30 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x2TLGU6w021105 for powerpc@FreeBSD.org; Fri, 29 Mar 2019 21:16:30 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: powerpc@FreeBSD.org Subject: [Bug 232387] head -r345558 (updated): system crash during kldload if_epair on powerpc64 (for more modern buildworld buildkernel toolchain experiments) Date: Fri, 29 Mar 2019 21:16:30 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: vimage X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: marklmi26-fbsd@yahoo.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: powerpc@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: B348A82657 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.99 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.99)[-0.988,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Mar 2019 21:16:32 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D232387 Mark Millard changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|head -r339076: system crash |head -r345558 (updated): |in vnet_epair_init during |system crash during kldload |kern_jail_set in a kyua |if_epair on powerpc64 (for |test on powerpc64 |more modern buildworld | |buildkernel toolchain | |experiments) --- Comment #15 from Mark Millard --- (In reply to Mark Millard from comment #14) Turns out that just doing: # kldload if_epair is sufficient to cause the crash. By contrast, loading geom_uzip laods without crashing (including loading xz). I'd expect the same for a devel/powerpc-xtoolchain-gcc based buildworld buildkernel that is installed and booted but I've not tried it. (The original report was for that context.) devel/powerpc64-binutils is common to both types of builds but the compilers are not. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-ppc@freebsd.org Sat Mar 30 07:06:02 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6284215681CE for ; Sat, 30 Mar 2019 07:06:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 06F0E72868 for ; Sat, 30 Mar 2019 07:06:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by freefall.freebsd.org (Postfix) id EC1341F2EF; Sat, 30 Mar 2019 07:06:01 +0000 (UTC) Delivered-To: powerpc@localmail.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client CN "mx1.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by freefall.freebsd.org (Postfix) with ESMTPS id CF9991F2EE for ; Sat, 30 Mar 2019 07:06:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7172072866 for ; Sat, 30 Mar 2019 07:06:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id A722C1924D for ; Sat, 30 Mar 2019 07:06:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x2U760Rs055910 for ; Sat, 30 Mar 2019 07:06:00 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x2U7601b055907 for powerpc@FreeBSD.org; Sat, 30 Mar 2019 07:06:00 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: powerpc@FreeBSD.org Subject: [Bug 232387] head -r345558 (updated): system crash during kldload if_epair on powerpc64 (for more modern buildworld buildkernel toolchain experiments) Date: Sat, 30 Mar 2019 07:05:59 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: vimage X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: marklmi26-fbsd@yahoo.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: powerpc@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 06F0E72868 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.99 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.99)[-0.988,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Mar 2019 07:06:02 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D232387 --- Comment #16 from Mark Millard --- (In reply to Mark Millard from comment #15) I added printing &DPCPU_NAME(epair_dpcpu) to: static void epair_dpcpu_init(void) This and ddb use gives the following information for the use of: #define _DPCPU_PTR(b, n) \ (__typeof(DPCPU_NAME(n))*)((b) + (uintptr_t)&DPCPU_NAME(n)) . . . #define DPCPU_ID_PTR(i, n) _DPCPU_PTR(dpcpu_off[(i)], n) (Typed from a picture:) &DPCPU_NAME(epair_dpcpu)=3D0xe00000008fcee810 show dpcpu_off in ddb shows: dpcpu_off[0]=3D0x1ffffffffecf6980 dpcpu_off[1]=3D0x2000000002adc980 dpcpu_off[2]=3D0x2000000002ada980 dpcpu_off[3]=3D0x2000000002ad8980 The failing virtual address was reported as: virtual address =3D 0x8e9e5198 . . . cpuid =3D 0 Then, checking: 0x1ffffffffecf6980+0xe00000008fcee810=3D=3D0x8e9e5190 so 0x8 short. But: addi r3,r22,24 bl <0000001b.plt_call._mtx_init> and: <_mtx_init+0x20> mr r30,r3 . . . <_mtx_init+0x60> addi r3,r30,-24 <_mtx_init+0x64> clrldi r7,r6,32 <_mtx_init+0x68> mr r6,r8 <_mtx_init+0x6c> bl and: stw r4,8(r3) So 0x8e9e5190+24-24+8=3D=3D0x8e9e5198 (the failure address). It appears to me that the dpcpu_off[i] figures are expected to convert 0xc???_????_????_???? type (direct-map) addresses to 0xe???_????_????_???? addresses but &DPCPU_NAME(epair_dpcpu) already was a 0xe???_????_????_???? type of address. The result overflowed/wrapped/truncated and was invalid. This looks likely to be a problem for any kldload'd .ko that uses a DPCPU_DEFINE and DPCPU_ID_PTR similarly to (showing my printf addition as well): struct epair_dpcpu { struct mtx if_epair_mtx; /* Per-CPU locking. */ int epair_drv_flags; /* Per-CPU ``hw'' drv flags= . */ struct eid_list epair_ifp_drain_list; /* Per-CPU list of ifps with * data in the ifq. */ }; DPCPU_DEFINE(struct epair_dpcpu, epair_dpcpu); static void epair_dpcpu_init(void) { struct epair_dpcpu *epair_dpcpu; struct eid_list *s; u_int cpuid; printf("epair_dpcpu_init: &DPCPU_NAME(epair_dpcpu)=3D%p\n", &DPCPU_NAME(epair_dpcpu)); CPU_FOREACH(cpuid) { epair_dpcpu =3D DPCPU_ID_PTR(cpuid, epair_dpcpu); /* Initialize per-cpu lock. */ EPAIR_LOCK_INIT(epair_dpcpu); . . . --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-ppc@freebsd.org Sat Mar 30 10:14:41 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 176DE156DC93 for ; Sat, 30 Mar 2019 10:14:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ABC8181219 for ; Sat, 30 Mar 2019 10:14:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by freefall.freebsd.org (Postfix) id 9AFA31EAC; Sat, 30 Mar 2019 10:14:40 +0000 (UTC) Delivered-To: powerpc@localmail.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [96.47.72.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client CN "mx1.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by freefall.freebsd.org (Postfix) with ESMTPS id 7DFE41EAB for ; Sat, 30 Mar 2019 10:14:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2D3CF81216 for ; Sat, 30 Mar 2019 10:14:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 631F41AD9E for ; Sat, 30 Mar 2019 10:14:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x2UAEddC062316 for ; Sat, 30 Mar 2019 10:14:39 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x2UAEdDk062315 for powerpc@FreeBSD.org; Sat, 30 Mar 2019 10:14:39 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: powerpc@FreeBSD.org Subject: [Bug 232387] head -r345558 (updated): system crash during kldload if_epair on powerpc64 (for more modern buildworld buildkernel toolchain experiments), Probably also: ipfw.ko linuxkpi.ko siftr.ko Possibly: hwpmc.ko Date: Sat, 30 Mar 2019 10:14:39 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: vimage X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: marklmi26-fbsd@yahoo.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: powerpc@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: ABC8181219 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.99 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.99)[-0.989,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Mar 2019 10:14:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D232387 Mark Millard changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|head -r345558 (updated): |head -r345558 (updated): |system crash during kldload |system crash during kldload |if_epair on powerpc64 (for |if_epair on powerpc64 (for |more modern buildworld |more modern buildworld |buildkernel toolchain |buildkernel toolchain |experiments) |experiments), Probably | |also: ipfw.ko linuxkpi.ko | |siftr.ko Possibly: hwpmc.ko --- Comment #17 from Mark Millard --- (In reply to Mark Millard from comment #16) Looking around it appears the .ko files with such pcpu_entry_ content are: /boot/kernel/if_epair.ko: U dpcpu_off 0000000000014810 d pcpu_entry_epair_dpcpu /boot/kernel/ipfw.ko: U dpcpu_off 00000000000454c8 d pcpu_entry_dyn_hp /boot/kernel/linuxkpi.ko: U dpcpu_off 0000000000032400 d pcpu_entry_linux_epoch_record 0000000000032380 d pcpu_entry_linux_idr_cache 0000000000032580 d pcpu_entry_tasklet_worker /boot/kernel/siftr.ko: U dpcpu_off 0000000000015450 d pcpu_entry_ss I expect the following might be okay: /boot/kernel/hwpmc.ko: U dpcpu_off U pcpu_entry_pmc_sampled based on /boot/kernel/kernel having pcpu_entry_pmc_sampled in its list: 00000000016f1450 B dpcpu_off 000000000132b778 d pcpu_entry_decr_state 000000000132a6a8 D pcpu_entry_epoch_cb_count 000000000132a6b0 D pcpu_entry_epoch_cb_task 000000000132a688 d pcpu_entry_exec_args_kva 000000000132b6f0 D pcpu_entry_hardclocktime 000000000132a728 d pcpu_entry_modspace 000000000132af80 D pcpu_entry_nws 000000000132a680 d pcpu_entry_pcputicks 000000000132a690 D pcpu_entry_pmc_sampled 000000000132b480 d pcpu_entry_pqbatch 000000000132a6a4 d pcpu_entry_randomval 000000000132a698 d pcpu_entry_tc_cpu_ticks_base 000000000132a6a0 d pcpu_entry_tc_cpu_ticks_last 000000000132b680 d pcpu_entry_timerstate 000000000132b6f8 d pcpu_entry_xive_cpu_data I'll not try to list the .ko's with vnet_entry_ prefixed names. But this seems to operate via curthread->td_vnet instead of something like dpcpu_off[cpuid] . I've not checked what curthread->td_vnet values are like. --=20 You are receiving this mail because: You are the assignee for the bug.=