Date: Sun, 23 Sep 2012 08:30:56 -0600 From: Ian Lepore <freebsd@damnhippie.dyndns.org> To: Alan Cox <alc@rice.edu> Cc: "arm@freebsd.org" <arm@freebsd.org> Subject: Re: arm pmap locking Message-ID: <1348410656.15745.9.camel@revolution.hippie.lan> In-Reply-To: <505E5DF7.9050107@rice.edu> References: <1345315508.27688.260.camel@revolution.hippie.lan> <503D12AE.1050705@rice.edu> <1346350374.1140.525.camel@revolution.hippie.lan> <5045351F.6060201@rice.edu> <1346723041.1140.602.camel@revolution.hippie.lan> <504B85BE.3030101@rice.edu> <1347316458.1137.41.camel@revolution.hippie.lan> <504F8BAC.4040902@rice.edu> <20120915045040.GZ58312@funkthat.com> <5054D69B.40209@rice.edu> <20120917033308.GB58312@funkthat.com> <505DE03D.7050101@rice.edu> <1348333663.5548.24.camel@revolution.hippie.lan> <505E0739.8080802@rice.edu> <1348346711.15745.3.camel@revolution.hippie.lan> <505E5DF7.9050107@rice.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, 2012-09-22 at 19:55 -0500, Alan Cox wrote:
> On 09/22/2012 15:45, Ian Lepore wrote:
> > Yep. Looks like the first one (reversed) is very very early in the
> > init. After this one, there were no other LORs during the boot until
> > the one that you said is fixed now in -current.
> >
> > lock order reversal:
> > 1st 0xc055bd08 4096 (UMA zone) @ /usr/src/sys/vm/uma_core.c:2011
> > 2nd 0xc043c198 pmap (pmap) @ /usr/src/sys/arm/arm/pmap.c:3734
>
> Ah. Notice that this is an "UMA zone" lock, but nonetheless a different
> zone and lock from before, i.e., "4096" versus "PV ENTRY". By default,
> witness treats all uma zone locks as having the same lock ordering.
> This helps to keep the distinct lock types that witness must deal with
> and consequently its run-time overhead down. However, sometimes uma
> zone locks on zones that are used within the VM system need to be
> handled specially. There is an argument to uma_zcreate() for doing
> this: UMA_ZONE_MTXCLASS
>
> Try this now. Remove whatever change that you made to witness and apply
> a change like this:
>
> Index: arm/arm/pmap.c
> ===================================================================
> --- arm/arm/pmap.c (revision 240803)
> +++ arm/arm/pmap.c (working copy)
> @@ -1821,7 +1821,7 @@ pmap_init(void)
> * Initialize the PV entry allocator.
> */
> pvzone = uma_zcreate("PV ENTRY", sizeof (struct pv_entry),
> NULL, NULL,
> - NULL, NULL, UMA_ALIGN_PTR, UMA_ZONE_VM | UMA_ZONE_NOFREE);
> + NULL, NULL, UMA_ALIGN_PTR, UMA_ZONE_MTXCLASS | UMA_ZONE_VM);
> TUNABLE_INT_FETCH("vm.pmap.shpgperproc", &shpgperproc);
> pv_entry_max = shpgperproc * maxproc + cnt.v_page_count;
> uma_zone_set_obj(pvzone, &pvzone_obj, pv_entry_max);
>
> I've also removed UMA_ZONE_NOFREE from the arguments because it's
> redundant. uma_zone_set_obj() itself sets that flag.
>
> Alan
>
Did that, now the original LOR that happens at start_init() time
involves pmap and the malloc 4096 uma zone rather than the PV ENTRY
zone. Putting back in the preset order, it looks like a malloc() in
mtx_pool_create() is now the first call with the wrong order, the bottom
of the call stack looks like this:
malloc() at malloc+0x10
scp=0xc00afe98 rlv=0xc00b28c8 (mtx_pool_create+0x54)
rsp=0xc04ddeb4 rfp=0xc04dded0
r10=0x204fe888 r9=0x00000019
r8=0x20479a10 r7=0x00000000 r6=0xc0254a10 r5=0x00000080
r4=0xc0276a40
mtx_pool_create() at mtx_pool_create+0x10
scp=0xc00b2884 rlv=0xc00b290c (mtx_pool_setup_dynamic+0x1c)
rsp=0xc04dded4 rfp=0xc04ddee0
r7=0x20000124 r6=0x00000004
r5=0x20000130 r4=0xc0276a40
mtx_pool_setup_dynamic() at mtx_pool_setup_dynamic+0x10
scp=0xc00b2900 rlv=0xc0082654 (mi_startup+0xf8)
rsp=0xc04ddee4 rfp=0xc04ddef4
mi_startup() at mi_startup+0x10
scp=0xc008256c rlv=0xc00001cc (virt_done+0x18)
rsp=0xc04ddef8 rfp=0x00000000
r4=0x2000020c
If you need complete backtraces or other details let me know.
-- Ian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1348410656.15745.9.camel>