Date: Sat, 22 Sep 2012 19:55:19 -0500 From: Alan Cox <alc@rice.edu> To: Ian Lepore <freebsd@damnhippie.dyndns.org> Cc: "arm@freebsd.org" <arm@freebsd.org>, Alan Cox <alc@rice.edu> Subject: Re: arm pmap locking Message-ID: <505E5DF7.9050107@rice.edu> In-Reply-To: <1348346711.15745.3.camel@revolution.hippie.lan> References: <1345315508.27688.260.camel@revolution.hippie.lan> <503D12AE.1050705@rice.edu> <1346350374.1140.525.camel@revolution.hippie.lan> <5045351F.6060201@rice.edu> <1346723041.1140.602.camel@revolution.hippie.lan> <504B85BE.3030101@rice.edu> <1347316458.1137.41.camel@revolution.hippie.lan> <504F8BAC.4040902@rice.edu> <20120915045040.GZ58312@funkthat.com> <5054D69B.40209@rice.edu> <20120917033308.GB58312@funkthat.com> <505DE03D.7050101@rice.edu> <1348333663.5548.24.camel@revolution.hippie.lan> <505E0739.8080802@rice.edu> <1348346711.15745.3.camel@revolution.hippie.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
On 09/22/2012 15:45, Ian Lepore wrote:
> On Sat, 2012-09-22 at 13:45 -0500, Alan Cox wrote:
>> On 09/22/2012 12:07, Ian Lepore wrote:
>>> There has been a boot-time LOR involving the same locks, but with a
>>> different backtrace, for a long time. In case it helps, the following
>>> is from freebsd 8.2 (the latest I can test with right now), the pmap.c
>>> in question is r205956 according to the __FBSDID() in the code. There
>>> are actually two LORs below, the first involves the same locks as above,
>>> the second one is different.
>>>
>>> Trying to mount root from ufs:/dev/mmcsd0s1a
>>> warning: no time-of-day clock registered, system time will not be set accurately
>>> lock order reversal:
>>> 1st 0xc0a8a0b0 pmap (pmap) @ /usr/src/sys/arm/arm/pmap.c:971
>>> 2nd 0xc055b608 PV ENTRY (UMA zone) @ /usr/src/sys/vm/uma_core.c:2055
>>
>> This is actually the correct lock ordering. So, something happened
>> before this that incorrectly trained witness. Are you able to boot a
>> kernel with a modified kern/subr_witness? If so, then you could
>> explicitly set the lock order to be the above, correct ordering, and
>> witness would report the actual case of LOR.
> Yep. Looks like the first one (reversed) is very very early in the
> init. After this one, there were no other LORs during the boot until
> the one that you said is fixed now in -current.
>
> lock order reversal:
> 1st 0xc055bd08 4096 (UMA zone) @ /usr/src/sys/vm/uma_core.c:2011
> 2nd 0xc043c198 pmap (pmap) @ /usr/src/sys/arm/arm/pmap.c:3734
Ah. Notice that this is an "UMA zone" lock, but nonetheless a different
zone and lock from before, i.e., "4096" versus "PV ENTRY". By default,
witness treats all uma zone locks as having the same lock ordering.
This helps to keep the distinct lock types that witness must deal with
and consequently its run-time overhead down. However, sometimes uma
zone locks on zones that are used within the VM system need to be
handled specially. There is an argument to uma_zcreate() for doing
this: UMA_ZONE_MTXCLASS
Try this now. Remove whatever change that you made to witness and apply
a change like this:
Index: arm/arm/pmap.c
===================================================================
--- arm/arm/pmap.c (revision 240803)
+++ arm/arm/pmap.c (working copy)
@@ -1821,7 +1821,7 @@ pmap_init(void)
* Initialize the PV entry allocator.
*/
pvzone = uma_zcreate("PV ENTRY", sizeof (struct pv_entry),
NULL, NULL,
- NULL, NULL, UMA_ALIGN_PTR, UMA_ZONE_VM | UMA_ZONE_NOFREE);
+ NULL, NULL, UMA_ALIGN_PTR, UMA_ZONE_MTXCLASS | UMA_ZONE_VM);
TUNABLE_INT_FETCH("vm.pmap.shpgperproc", &shpgperproc);
pv_entry_max = shpgperproc * maxproc + cnt.v_page_count;
uma_zone_set_obj(pvzone, &pvzone_obj, pv_entry_max);
I've also removed UMA_ZONE_NOFREE from the arguments because it's
redundant. uma_zone_set_obj() itself sets that flag.
Alan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?505E5DF7.9050107>