Date: Tue, 21 Apr 2009 19:31:14 +0400 From: Ruslan Ermilov <ru@FreeBSD.org> To: Mike Tancsa <mike@sentex.net> Cc: Robert Watson <rwatson@FreeBSD.org>, freebsd-stable@freebsd.org, John Baldwin <jhb@freebsd.org> Subject: Re: RELENG_7 crash Message-ID: <20090421153112.GA47589@edoofus.dev.vega.ru> In-Reply-To: <200904211519.n3LFJFsk090691@lava.sentex.ca> References: <200904210524.n3L5O9YS086865@lava.sentex.ca> <200904211111.57295.jhb@freebsd.org> <200904211519.n3LFJFsk090691@lava.sentex.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Apr 21, 2009 at 11:20:13AM -0400, Mike Tancsa wrote:
> At 11:11 AM 4/21/2009, John Baldwin wrote:
>
> >Can you do 'frame 7' followed by 'l', 'p ifp', and 'p ifp->if_snd'?
>
> Hi,
>
>
> kgdb) bt
> #0 doadump () at pcpu.h:196
> #1 0xc05964d7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
> #2 0xc05967a9 in panic (fmt=Variable "fmt" is not available.
> ) at /usr/src/sys/kern/kern_shutdown.c:574
> #3 0xc07f64ac in trap_fatal (frame=0xe766ea6c, eva=104) at
> /usr/src/sys/i386/i386/trap.c:939
> #4 0xc07f6730 in trap_pfault (frame=0xe766ea6c, usermode=0, eva=104)
> at /usr/src/sys/i386/i386/trap.c:852
> #5 0xc07f70dc in trap (frame=0xe766ea6c) at /usr/src/sys/i386/i386/trap.c:530
> #6 0xc07db7eb in calltrap () at /usr/src/sys/i386/i386/exception.s:159
> #7 0xc0637146 in sysctl_ifdata (oidp=0xc08816a0, arg1=0xe766ec24,
> arg2=2, req=0xe766eba4) at /usr/src/sys/net/if_mib.c:127
> #8 0xc059fd77 in sysctl_root (oidp=Variable "oidp" is not available.
> ) at /usr/src/sys/kern/kern_sysctl.c:1413
> #9 0xc059ff14 in userland_sysctl (td=0xc5374460, name=0xe766ec14,
> namelen=6, old=0x0, oldlenp=0xbfbf8478, inkernel=0, new=0x0,
> newlen=0, retval=0xe766ec10, flags=0) at
> /usr/src/sys/kern/kern_sysctl.c:1506
> #10 0xc05a0064 in __sysctl (td=0xc5374460, uap=0xe766ecfc) at
> /usr/src/sys/kern/kern_sysctl.c:1443
> #11 0xc07f6a85 in syscall (frame=0xe766ed38) at
> /usr/src/sys/i386/i386/trap.c:1090
> #12 0xc07db850 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:255
> #13 0x00000033 in ?? ()
> Previous frame inner to this frame (corrupt stack?)
> (kgdb) frame 7
> #7 0xc0637146 in sysctl_ifdata (oidp=0xc08816a0, arg1=0xe766ec24,
> arg2=2, req=0xe766eba4) at /usr/src/sys/net/if_mib.c:127
> 127 ifp->if_snd.ifq_drops = ifmd.ifmd_snd_drops;
> (kgdb) l
> 122 DONTCOPY(baudrate);
> 123 #undef DONTCOPY
> 124 #define COPY(fld) ifp->if_##fld = ifmd.ifmd_##fld
> 125 COPY(data);
> 126 ifp->if_snd.ifq_maxlen = ifmd.ifmd_snd_maxlen;
> 127 ifp->if_snd.ifq_drops = ifmd.ifmd_snd_drops;
> 128 #undef COPY
> 129 break;
> 130
> 131 case IFDATA_LINKSPECIFIC:
> (kgdb) p ifp
> $1 = (struct ifnet *) 0x0
> (kgdb) p ifp->if_snd
> Cannot access memory at address 0xf4
> (kgdb)
>
>
> Is it possible I am running into some of the interface lock fixes
> rwatson has been working on ? This box has a lot of ng interfaces
> which come and go. Perhaps snmp asking about an interface that just
> went away caused the panic ? I disabled bsnmp since the reboot and
> the box has been up for 10hrs so far.
>
It's a documented bug:
: revision 1.281
: date: 2008/06/26 23:05:28; author: rwatson; state: Exp; lines: +69 -12
: SVN rev 180042 on 2008-06-26 23:05:28Z by rwatson
:
: Introduce locking around use of ifindex_table, whose use was previously
: unsynchronized. While races were extremely rare, we've now had a
: couple of reports of panics in environments involving large numbers of
: IPSEC tunnels being added very quickly on an active system.
:
: - Add accessor functions ifnet_byindex(), ifaddr_byindex(),
: ifdev_byindex() to replace existing accessor macros. These functions
: now acquire the ifnet lock before derefencing the table.
: - Add IFNET_WLOCK_ASSERT().
: - Add static accessor functions ifnet_setbyindex(), ifdev_setbyindex(),
: which set values in the table either asserting of acquiring the ifnet
: lock.
: - Use accessor functions throughout if.c to modify and read
: ifindex_table.
: - Rework ifnet attach/detach to lock around ifindex_table modification.
:
: Note that these changes simply close races around use of ifindex_table,
: and make no attempt to solve the probem of disappearing ifnets. Further
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
: refinement of this work, including with respect to ifindex_table
: resizing, is still required.
:
: In a future change, the ifnet lock should be converted from a mutex to an
: rwlock in order to reduce contention.
:
: Reviewed and tested by: brooks
Cheers,
--
Ruslan Ermilov
ru@FreeBSD.org
FreeBSD committer
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20090421153112.GA47589>