Date: Mon, 14 Sep 2009 14:47:11 -0700 From: Navdeep Parhar <nparhar@gmail.com> To: Matthew Fleming <matthew.fleming@isilon.com> Cc: freebsd-stable@freebsd.org Subject: Re: cxgb LOR Message-ID: <d04e16b70909141447k2eb7f806oaa6d882059b9990e@mail.gmail.com> In-Reply-To: <06D5F9F6F655AD4C92E28B662F7F853E030C6646@seaxch09.desktop.isilon.com> References: <06D5F9F6F655AD4C92E28B662F7F853E030C6646@seaxch09.desktop.isilon.com>
next in thread | previous in thread | raw e-mail | index | archive | help
A lot of these LORs were fixed in cxgb in FreeBSD 8. You can look at cxgb_main.c in 8 for details. I'll also try and figure out if those changes are easily MFC'able. Regards, Navdeep On Mon, Sep 14, 2009 at 2:29 PM, Matthew Fleming <matthew.fleming@isilon.com> wrote: > We got a cxgb LOR report of: > > 1st 0xffffff8001e37be0 vlan_global (vlan_global) @ > /build/mnt/src/sys/modules/if_vlan/../../net/if_vlan.c:1310 > =A02nd 0xffffff80010892f0 cxgb port lock (cxgb port lock) @ > /build/mnt/src/sys/modules/cxgb/../../dev/cxgb/cxgb_main.c:1956 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a > witness_checkorder() at witness_checkorder+0x9e2 > _sx_xlock() at _sx_xlock+0x55 > cxgb_ioctl() at cxgb_ioctl+0x1e8 > vlan_ioctl() at vlan_ioctl+0x359 > ifhwioctl() at ifhwioctl+0xb1 > ifioctl() at ifioctl+0xb1 > kern_ioctl() at kern_ioctl+0xa3 > ioctl() at ioctl+0xf1 > freebsd32_ioctl() at freebsd32_ioctl+0x13e > isi_syscall() at isi_syscall+0x94 > ia32_syscall() at ia32_syscall+0x1a3 > Xint0x80_syscall() at Xint0x80_syscall+0x60 > --- syscall (54, FreeBSD ELF32, freebsd32_ioctl), rip =3D 0x2868db1b, rsp > =3D 0xffffd4bc, rbp =3D 0xffffda38 --- > > > So we tried changing cxgb to not USE_SX. =A0This resulted in a different > LOR: > > lock order reversal: (sleepable after non-sleepable) > =A01st 0xffffff8000f9d508 cxgb controller lock 0 (cxgb controller lock 0) > @ > /build/mnt/src/sys/modules/cxgb/cxgb/../../../dev/cxgb/cxgb_main.c:1889 > =A02nd 0xffffffff806064e0 ACPI root bus (ACPI root bus) @ > /build/mnt/src/sys/dev/acpica/acpi.c:1040 > KDB: stack backtrace: > [ffffffff8018e9fa] db_trace_self_wrapper+0x2a > [ffffffff80298e89] witness_checkorder+0x719 > [ffffffff8025bf75] _sx_xlock+0x55 > [ffffffff8019678a] acpi_alloc_resource+0x9a > [ffffffff80281714] resource_list_alloc+0x184 > [ffffffff801d9f98] pci_alloc_resource+0x158 > [ffffffff802814b9] bus_alloc_resource+0x89 > [ffffffff81804201] cxgb_setup_interrupts+0x51 > [ffffffff81807f33] cxgb_up+0xa3 > [ffffffff818083c0] cxgb_init_locked+0x1b0 > [ffffffff81808539] cxgb_init+0x39 > [ffffffff81808758] cxgb_ioctl+0x1f8 > [ffffffff8031e9e1] ifhwioctl+0xb1 > [ffffffff8031f720] ifioctl+0xb0 > [ffffffff8029a873] kern_ioctl+0xa3 > [ffffffff8029aad1] ioctl+0xf1 > [ffffffff8041eb93] freebsd32_ioctl+0xb3 > [ffffffff8025d963] isi_syscall+0x83 > [ffffffff8041de63] ia32_syscall+0x1a3 > [ffffffff803efc60] Xint0x80_syscall+0x60 > --- syscall (54, FreeBSD ELF32, freebsd32_ioctl), rip =3D 0x2826ea67, rsp > =3D 0xffffd8ac, rbp =3D 0xffffd928 --- > > (we modified cxgb_ioctl to call cxgb_init because otherwise the cxgb > interface would require an ifconfig up before it detected a link, which > was different behaviour from the em driver. =A0Since the locks in questio= n > are acquired inside cxgb_init() I don't think the rest of the stack is > relevant, but network stack isn't my area of expertise). > > So it seems that with cxgb we're damned if we do, damned if we don't. > Any advice on which LOR is "worse" or if one is harmless, or how to make > it go away? > > Note also that if cxgb uses a mtx then it will do malloc while holding > the mtx in this stack: > > [ffffffff803cc58a] uma_zalloc_arg+0x2da > [ffffffff80241ef9] malloc+0x89 > [ffffffff8023115f] intr_event_add_handler+0x5f > [ffffffff803f26d2] intr_add_handler+0x72 > [ffffffff801dc171] pci_setup_intr+0x41 > [ffffffff801dc171] pci_setup_intr+0x41 > [ffffffff802807e6] bus_setup_intr+0x96 > [ffffffff8180423c] cxgb_setup_interrupts+0x8c > [ffffffff81807f33] cxgb_up+0xa3 > [ffffffff818083c0] cxgb_init_locked+0x1b0 > [ffffffff81808539] cxgb_init+0x39 > > Thanks, > matthew > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?d04e16b70909141447k2eb7f806oaa6d882059b9990e>