From owner-freebsd-sparc64@FreeBSD.ORG Thu Nov 6 18:43:17 2008 Return-Path: Delivered-To: freebsd-sparc64@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 56F721065674 for ; Thu, 6 Nov 2008 18:43:17 +0000 (UTC) (envelope-from scaron@umich.edu) Received: from deathandthemaiden.mr.itd.umich.edu (deathandthemaiden.mr.itd.umich.edu [141.211.14.22]) by mx1.freebsd.org (Postfix) with ESMTP id D50DF8FC17 for ; Thu, 6 Nov 2008 18:43:16 +0000 (UTC) (envelope-from scaron@umich.edu) Received: FROM mohican.web.itd.umich.edu (mohican.web.itd.umich.edu [141.211.144.149]) BY deathandthemaiden.mr.itd.umich.edu ID 491334AE.719EB.19429 ; 6 Nov 2008 13:17:18 -0500 Received: (from www@localhost) by mohican.web.itd.umich.edu () id mA6IHHdk031059; Thu, 6 Nov 2008 13:17:17 -0500 Received: from host8-99.sph.umich.edu (host8-99.sph.umich.edu [141.211.8.99]) by web.mail.umich.edu (Horde Framework) with HTTP; Thu, 06 Nov 2008 13:17:17 -0500 Message-ID: <20081106131717.331718q66y289lgk@web.mail.umich.edu> Date: Thu, 06 Nov 2008 13:17:17 -0500 From: Sean Thomas Caron To: freebsd-sparc64@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.2) X-Remote-Browser: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0 X-IMP-Server: 141.211.144.203 (mohican) X-Originating-IP: 141.211.8.99 X-Originating-User: scaron Cc: Subject: Occasional kernel panic + reboot on 7.0-RELEASE with fatm driver. X-BeenThere: freebsd-sparc64@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to the Sparc List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Nov 2008 18:43:17 -0000 Hi folks, I'm using fatm on FreeBSD/sparc64 7.0-RELEASE; it generally works well =20 but every couple of weeks the system will kernel panic and reboot. I =20 switched on kernel dumps on panic and here's what I got (this time): sonnet.diablonet.net> kgdb kernel.debug /var/crash/vmcore.0 kgdb: kvm_nlist(_stopped_cpus): kgdb: kvm_nlist(_stoppcbs): GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions= . Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "sparc64-marcel-freebsd". Unread portion of the kernel message buffer: panic: trap: fast data access mmu miss Uptime: 16d13h9m7s Dumping 1024 MB (2 chunks) chunk at 0: 536870912 bytes | #0 0x00000000c0280cd8 in doadump () at /usr/src/sys/kern/kern_shutdown.c:24= 0 240 savectx(&dumppcb); (kgdb) backtrace #0 0x00000000c0280cd8 in doadump () at /usr/src/sys/kern/kern_shutdown.c:24= 0 #1 0x00000000c0281608 in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:409 #2 0x00000000c0281860 in panic (fmt=3D0xc066c6e0 "trap: %s") at /usr/src/sys/kern/kern_shutdown.c:563 #3 0x00000000c0541de4 in trap (tf=3D0xe5390e50) at /usr/src/sys/sparc64/sparc64/trap.c:378 #4 0x00000000c0070fe0 in tl1_trap () #5 0x00000000c02dd1d0 in sbsndptr (sb=3D0xfffff800014be6f0, off=3D0, len=3D= 1390, moff=3D0xe5391064) at /usr/src/sys/kern/uipc_sockbuf.c:939 #6 0x00000000c03edac4 in tcp_output (tp=3D0xfffff800014be6f0) at /usr/src/sys/netinet/tcp_output.c:802 #7 0x00000000c03edac4 in tcp_output (tp=3D0xfffff800014fce38) at /usr/src/sys/netinet/tcp_output.c:802 #8 0x00000000c03eaf98 in tcp_do_segment (m=3D0xfffff8005b354000, th=3D0xfffff8000133283c, so=3D0xfffff800014be570, tp=3D0xfffff800014fce= 38, drop_hdrlen=3D52, tlen=3D0) at /usr/src/sys/netinet/tcp_input.c:2347 #9 0x00000000c03ec214 in tcp_input (m=3D0xfffff8005b354000, =20 off0=3DVariable "off0" is not available. ) at /usr/src/sys/netinet/tcp_input.c:845 #10 0x00000000c0381128 in ip_input (m=3D0xfffff8005b354000) at /usr/src/sys/netinet/ip_input.c:665 #11 0x00000000c0339cd0 in netisr_dispatch (num=3D2, m=3D0xfffff8005b354000) at /usr/src/sys/net/netisr.c:185 #12 0x00000000c032a930 in atm_input (ifp=3D0xfffff8000103c000, ah=3D0xe53916= 2c, m=3D0xfffff8005b354000, rxhand=3D0x0) at /usr/src/sys/net/if_atmsubr.c:= 347 #13 0x00000000c013d410 in fatm_intr (p=3D0xfffff80001173c00) at /usr/src/sys/dev/fatm/if_fatm.c:1573 #14 0x00000000c02615ec in ithread_loop (arg=3D0xfffff800011ce760) at /usr/src/sys/kern/kern_intr.c:1036 #15 0x00000000c025dd54 in fork_exit (callout=3D0xc0261420 , arg=3D0xfffff800011ce760, frame=3D0xe5391880) at /usr/src/sys/kern/kern_fork.c:781 #16 0x00000000c00711d0 in fork_trampoline () #17 0x00000000c00711d0 in fork_trampoline () Previous frame identical to this frame (corrupt stack?) (kgdb) up 15 #15 0x00000000c025dd54 in fork_exit (callout=3D0xc0261420 , arg=3D0xfffff800011ce760, frame=3D0xe5391880) at /usr/src/sys/kern/kern_fork.c:781 781 callout(arg, frame); (kgdb) list 776 * cpu_set_fork_handler intercepts this function call to 777 * have this call a non-return function to stay in =20 kernel mode. 778 * initproc has its own fork handler, but it does return. 779 */ 780 KASSERT(callout !=3D NULL, ("NULL callout in fork_exit")); 781 callout(arg, frame); 782 783 /* 784 * Check if a kernel thread misbehaved and returned =20 from its main 785 * function. (kgdb) down #14 0x00000000c02615ec in ithread_loop (arg=3D0xfffff800011ce760) at /usr/src/sys/kern/kern_intr.c:1036 1036 ih->ih_handler(ih->ih_argument); (kgdb) list 1031 __func__, p->p_pid, (void *)ih->ih_handler, 1032 ih->ih_argument, ih->ih_name, ih->ih_flags); 1033 1034 if (!(ih->ih_flags & IH_MPSAFE)) 1035 mtx_lock(&Giant); 1036 ih->ih_handler(ih->ih_argument); 1037 if (!(ih->ih_flags & IH_MPSAFE)) 1038 mtx_unlock(&Giant); 1039 } 1040 if (!(ie->ie_flags & IE_SOFT)) (kgdb) down #13 0x00000000c013d410 in fatm_intr (p=3D0xfffff80001173c00) at /usr/src/sys/dev/fatm/if_fatm.c:1573 1573 atm_input(ifp, &aph, m0, vc->rxhand); (kgdb) list 1568 ifp->if_ipackets++; 1569 1570 vc->ipackets++; 1571 vc->ibytes +=3D m0->m_pkthdr.len; 1572 1573 atm_input(ifp, &aph, m0, vc->rxhand); 1574 } 1575 1576 H_SETSTAT(q->q.statp, FATM_STAT_FREE); 1577 H_SYNCSTAT_PREWRITE(sc, q->q.statp); (kgdb) down #12 0x00000000c032a930 in atm_input (ifp=3D0xfffff8000103c000, ah=3D0xe53916= 2c, m=3D0xfffff8005b354000, rxhand=3D0x0) at /usr/src/sys/net/if_atmsubr.c:= 347 347 netisr_dispatch(isr, m); (kgdb) list 342 else 343 m_freem(m); 344 return; 345 } 346 } 347 netisr_dispatch(isr, m); 348 } 349 350 /* 351 * Perform common duties while attaching to interface list. (kgdb) down #11 0x00000000c0339cd0 in netisr_dispatch (num=3D2, m=3D0xfffff8005b354000) at /usr/src/sys/net/netisr.c:185 185 ni->ni_handler(m); (kgdb) list 180 * the packet but now do not. Doing so here will 181 * not preserve ordering so instead we fallback to 182 * guaranteeing order only from dispatch points 183 * in the system (see above). 184 */ 185 ni->ni_handler(m); 186 } else { 187 isrstat.isrs_deferred++; 188 if (IF_HANDOFF(ni->ni_queue, m, NULL)) 189 schednetisr(num); (kgdb) down #10 0x00000000c0381128 in ip_input (m=3D0xfffff8005b354000) at /usr/src/sys/netinet/ip_input.c:665 665 (*inetsw[ip_protox[ip->ip_p]].pr_input)(m, hlen); (kgdb) list 660 /* 661 * Switch out to protocol's input routine. 662 */ 663 ipstat.ips_delivered++; 664 665 (*inetsw[ip_protox[ip->ip_p]].pr_input)(m, hlen); 666 return; 667 bad: 668 m_freem(m); 669 } (kgdb) down #9 0x00000000c03ec214 in tcp_input (m=3D0xfffff8005b354000, =20 off0=3DVariable "off0" is not available. ) at /usr/src/sys/netinet/tcp_input.c:845 845 tcp_do_segment(m, th, so, tp, drop_hdrlen, tlen); (kgdb) list 840 /* 841 * Segment belongs to a connection in SYN_SENT, =20 ESTABLISHED or later 842 * state. tcp_do_segment() always consumes the mbuf =20 chain, unlocks 843 * the inpcb, and unlocks pcbinfo. 844 */ 845 tcp_do_segment(m, th, so, tp, drop_hdrlen, tlen); 846 INP_INFO_UNLOCK_ASSERT(&tcbinfo); 847 return; 848 849 dropwithreset: (kgdb) down #8 0x00000000c03eaf98 in tcp_do_segment (m=3D0xfffff8005b354000, th=3D0xfffff8000133283c, so=3D0xfffff800014be570, tp=3D0xfffff800014fce= 38, drop_hdrlen=3D52, tlen=3D0) at /usr/src/sys/netinet/tcp_input.c:2347 2347 (void) tcp_output(tp); (kgdb) list 2342 2343 /* 2344 * Return any desired output. 2345 */ 2346 if (needoutput || (tp->t_flags & TF_ACKNOW)) 2347 (void) tcp_output(tp); 2348 2349 check_delack: 2350 KASSERT(headlocked =3D=3D 0, ("%s: check_delack: head locked= ", 2351 __func__)); (kgdb) down #7 0x00000000c03edac4 in tcp_output (tp=3D0xfffff800014fce38) at /usr/src/sys/netinet/tcp_output.c:802 802 mb =3D sbsndptr(&so->so_snd, off, len, &moff); (kgdb) list 797 798 /* 799 * Start the m_copy functions from the closest mbuf 800 * to the offset in the socket buffer chain. 801 */ 802 mb =3D sbsndptr(&so->so_snd, off, len, &moff); 803 804 if (len <=3D MHLEN - hdrlen - max_linkhdr) { 805 m_copydata(mb, moff, (int)len, 806 mtod(m, caddr_t) + hdrlen); (kgdb) down #6 0x00000000c03edac4 in tcp_output (tp=3D0xfffff800014be6f0) at /usr/src/sys/netinet/tcp_output.c:802 802 mb =3D sbsndptr(&so->so_snd, off, len, &moff); (kgdb) list 797 798 /* 799 * Start the m_copy functions from the closest mbuf 800 * to the offset in the socket buffer chain. 801 */ 802 mb =3D sbsndptr(&so->so_snd, off, len, &moff); 803 804 if (len <=3D MHLEN - hdrlen - max_linkhdr) { 805 m_copydata(mb, moff, (int)len, 806 mtod(m, caddr_t) + hdrlen); (kgdb) down #5 0x00000000c02dd1d0 in sbsndptr (sb=3D0xfffff800014be6f0, off=3D0, len=3D= 1390, moff=3D0xe5391064) at /usr/src/sys/kern/uipc_sockbuf.c:939 939 off > 0 && off >=3D m->m_len; (kgdb) list 934 *moff =3D off - sb->sb_sndptroff; 935 m =3D ret =3D sb->sb_sndptr ? sb->sb_sndptr : sb->sb_mb; 936 937 /* Advance by len to be as close as possible for the =20 next transmit. */ 938 for (off =3D off - sb->sb_sndptroff + len - 1; 939 off > 0 && off >=3D m->m_len; 940 m =3D m->m_next) { 941 sb->sb_sndptroff +=3D m->m_len; 942 off -=3D m->m_len; 943 } (kgdb) down #4 0x00000000c0070fe0 in tl1_trap () (kgdb) list 944 sb->sb_sndptr =3D m; 945 946 return (ret); 947 } 948 949 /* 950 * Drop a record off the front of a sockbuf and move the next =20 record to the 951 * front. 952 */ 953 void (kgdb) quit sonnet.diablonet.net> Please let me know if further information is required and I will =20 furnish, no problem. Thanks, -Sean