From owner-freebsd-sparc64@FreeBSD.ORG  Thu Nov  6 18:43:17 2008
Return-Path: <owner-freebsd-sparc64@FreeBSD.ORG>
Delivered-To: freebsd-sparc64@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 56F721065674
	for <freebsd-sparc64@freebsd.org>; Thu,  6 Nov 2008 18:43:17 +0000 (UTC)
	(envelope-from scaron@umich.edu)
Received: from deathandthemaiden.mr.itd.umich.edu
	(deathandthemaiden.mr.itd.umich.edu [141.211.14.22])
	by mx1.freebsd.org (Postfix) with ESMTP id D50DF8FC17
	for <freebsd-sparc64@freebsd.org>; Thu,  6 Nov 2008 18:43:16 +0000 (UTC)
	(envelope-from scaron@umich.edu)
Received: FROM mohican.web.itd.umich.edu (mohican.web.itd.umich.edu
	[141.211.144.149])
	BY deathandthemaiden.mr.itd.umich.edu ID 491334AE.719EB.19429 ; 
	6 Nov 2008 13:17:18 -0500
Received: (from www@localhost)
	by mohican.web.itd.umich.edu () id mA6IHHdk031059;
	Thu, 6 Nov 2008 13:17:17 -0500
Received: from host8-99.sph.umich.edu (host8-99.sph.umich.edu
	[141.211.8.99]) by web.mail.umich.edu (Horde Framework) with HTTP;
	Thu, 06 Nov 2008 13:17:17 -0500
Message-ID: <20081106131717.331718q66y289lgk@web.mail.umich.edu>
Date: Thu, 06 Nov 2008 13:17:17 -0500
From: Sean Thomas Caron <scaron@umich.edu>
To: freebsd-sparc64@freebsd.org
MIME-Version: 1.0
Content-Type: text/plain;
	charset=ISO-8859-1;
	DelSp="Yes";
	format="flowed"
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
User-Agent: Internet Messaging Program (IMP) H3 (4.2)
X-Remote-Browser: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9)
	Gecko/2008052906 Firefox/3.0
X-IMP-Server: 141.211.144.203 (mohican)
X-Originating-IP: 141.211.8.99
X-Originating-User: scaron
Cc: 
Subject: Occasional kernel panic + reboot on 7.0-RELEASE with fatm driver.
X-BeenThere: freebsd-sparc64@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Porting FreeBSD to the Sparc <freebsd-sparc64.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64>, 
	<mailto:freebsd-sparc64-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-sparc64>
List-Post: <mailto:freebsd-sparc64@freebsd.org>
List-Help: <mailto:freebsd-sparc64-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64>,
	<mailto:freebsd-sparc64-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 06 Nov 2008 18:43:17 -0000

Hi folks,

I'm using fatm on FreeBSD/sparc64 7.0-RELEASE; it generally works well =20
but every couple of weeks the system will kernel panic and reboot. I =20
switched on kernel dumps on panic and here's what I got (this time):

sonnet.diablonet.net> kgdb kernel.debug /var/crash/vmcore.0
kgdb: kvm_nlist(_stopped_cpus):
kgdb: kvm_nlist(_stoppcbs):
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions=
.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "sparc64-marcel-freebsd".

Unread portion of the kernel message buffer:
panic: trap: fast data access mmu miss
Uptime: 16d13h9m7s
Dumping 1024 MB (2 chunks)
   chunk at 0: 536870912 bytes |

#0  0x00000000c0280cd8 in doadump () at /usr/src/sys/kern/kern_shutdown.c:24=
0
240             savectx(&dumppcb);
(kgdb) backtrace
#0  0x00000000c0280cd8 in doadump () at /usr/src/sys/kern/kern_shutdown.c:24=
0
#1  0x00000000c0281608 in boot (howto=3D260)
     at /usr/src/sys/kern/kern_shutdown.c:409
#2  0x00000000c0281860 in panic (fmt=3D0xc066c6e0 "trap: %s")
     at /usr/src/sys/kern/kern_shutdown.c:563
#3  0x00000000c0541de4 in trap (tf=3D0xe5390e50)
     at /usr/src/sys/sparc64/sparc64/trap.c:378
#4  0x00000000c0070fe0 in tl1_trap ()
#5  0x00000000c02dd1d0 in sbsndptr (sb=3D0xfffff800014be6f0, off=3D0, len=3D=
1390,
     moff=3D0xe5391064) at /usr/src/sys/kern/uipc_sockbuf.c:939
#6  0x00000000c03edac4 in tcp_output (tp=3D0xfffff800014be6f0)
     at /usr/src/sys/netinet/tcp_output.c:802
#7  0x00000000c03edac4 in tcp_output (tp=3D0xfffff800014fce38)
     at /usr/src/sys/netinet/tcp_output.c:802
#8  0x00000000c03eaf98 in tcp_do_segment (m=3D0xfffff8005b354000,
     th=3D0xfffff8000133283c, so=3D0xfffff800014be570, tp=3D0xfffff800014fce=
38,
     drop_hdrlen=3D52, tlen=3D0) at /usr/src/sys/netinet/tcp_input.c:2347
#9  0x00000000c03ec214 in tcp_input (m=3D0xfffff8005b354000, =20
off0=3DVariable "off0" is not available.
)
     at /usr/src/sys/netinet/tcp_input.c:845
#10 0x00000000c0381128 in ip_input (m=3D0xfffff8005b354000)
     at /usr/src/sys/netinet/ip_input.c:665
#11 0x00000000c0339cd0 in netisr_dispatch (num=3D2, m=3D0xfffff8005b354000)
     at /usr/src/sys/net/netisr.c:185
#12 0x00000000c032a930 in atm_input (ifp=3D0xfffff8000103c000, ah=3D0xe53916=
2c,
     m=3D0xfffff8005b354000, rxhand=3D0x0) at /usr/src/sys/net/if_atmsubr.c:=
347
#13 0x00000000c013d410 in fatm_intr (p=3D0xfffff80001173c00)
     at /usr/src/sys/dev/fatm/if_fatm.c:1573
#14 0x00000000c02615ec in ithread_loop (arg=3D0xfffff800011ce760)
     at /usr/src/sys/kern/kern_intr.c:1036
#15 0x00000000c025dd54 in fork_exit (callout=3D0xc0261420 <ithread_loop>,
     arg=3D0xfffff800011ce760, frame=3D0xe5391880)
     at /usr/src/sys/kern/kern_fork.c:781
#16 0x00000000c00711d0 in fork_trampoline ()
#17 0x00000000c00711d0 in fork_trampoline ()
Previous frame identical to this frame (corrupt stack?)
(kgdb) up 15
#15 0x00000000c025dd54 in fork_exit (callout=3D0xc0261420 <ithread_loop>,
     arg=3D0xfffff800011ce760, frame=3D0xe5391880)
     at /usr/src/sys/kern/kern_fork.c:781
781             callout(arg, frame);
(kgdb) list
776              * cpu_set_fork_handler intercepts this function call to
777              * have this call a non-return function to stay in =20
kernel mode.
778              * initproc has its own fork handler, but it does return.
779              */
780             KASSERT(callout !=3D NULL, ("NULL callout in fork_exit"));
781             callout(arg, frame);
782
783             /*
784              * Check if a kernel thread misbehaved and returned =20
from its main
785              * function.
(kgdb) down
#14 0x00000000c02615ec in ithread_loop (arg=3D0xfffff800011ce760)
     at /usr/src/sys/kern/kern_intr.c:1036
1036                    ih->ih_handler(ih->ih_argument);
(kgdb) list
1031                        __func__, p->p_pid, (void *)ih->ih_handler,
1032                        ih->ih_argument, ih->ih_name, ih->ih_flags);
1033
1034                    if (!(ih->ih_flags & IH_MPSAFE))
1035                            mtx_lock(&Giant);
1036                    ih->ih_handler(ih->ih_argument);
1037                    if (!(ih->ih_flags & IH_MPSAFE))
1038                            mtx_unlock(&Giant);
1039            }
1040            if (!(ie->ie_flags & IE_SOFT))
(kgdb) down
#13 0x00000000c013d410 in fatm_intr (p=3D0xfffff80001173c00)
     at /usr/src/sys/dev/fatm/if_fatm.c:1573
1573                            atm_input(ifp, &aph, m0, vc->rxhand);
(kgdb) list
1568                            ifp->if_ipackets++;
1569
1570                            vc->ipackets++;
1571                            vc->ibytes +=3D m0->m_pkthdr.len;
1572
1573                            atm_input(ifp, &aph, m0, vc->rxhand);
1574                    }
1575
1576                    H_SETSTAT(q->q.statp, FATM_STAT_FREE);
1577                    H_SYNCSTAT_PREWRITE(sc, q->q.statp);
(kgdb) down
#12 0x00000000c032a930 in atm_input (ifp=3D0xfffff8000103c000, ah=3D0xe53916=
2c,
     m=3D0xfffff8005b354000, rxhand=3D0x0) at /usr/src/sys/net/if_atmsubr.c:=
347
347             netisr_dispatch(isr, m);
(kgdb) list
342                             else
343                                     m_freem(m);
344                             return;
345                     }
346             }
347             netisr_dispatch(isr, m);
348     }
349
350     /*
351      * Perform common duties while attaching to interface list.
(kgdb) down
#11 0x00000000c0339cd0 in netisr_dispatch (num=3D2, m=3D0xfffff8005b354000)
     at /usr/src/sys/net/netisr.c:185
185                     ni->ni_handler(m);
(kgdb) list
180                      * the packet but now do not.  Doing so here will
181                      * not preserve ordering so instead we fallback to
182                      * guaranteeing order only from dispatch points
183                      * in the system (see above).
184                      */
185                     ni->ni_handler(m);
186             } else {
187                     isrstat.isrs_deferred++;
188                     if (IF_HANDOFF(ni->ni_queue, m, NULL))
189                             schednetisr(num);
(kgdb) down
#10 0x00000000c0381128 in ip_input (m=3D0xfffff8005b354000)
     at /usr/src/sys/netinet/ip_input.c:665
665             (*inetsw[ip_protox[ip->ip_p]].pr_input)(m, hlen);
(kgdb) list
660             /*
661              * Switch out to protocol's input routine.
662              */
663             ipstat.ips_delivered++;
664
665             (*inetsw[ip_protox[ip->ip_p]].pr_input)(m, hlen);
666             return;
667     bad:
668             m_freem(m);
669     }
(kgdb) down
#9  0x00000000c03ec214 in tcp_input (m=3D0xfffff8005b354000, =20
off0=3DVariable "off0" is not available.
)
     at /usr/src/sys/netinet/tcp_input.c:845
845             tcp_do_segment(m, th, so, tp, drop_hdrlen, tlen);
(kgdb) list
840             /*
841              * Segment belongs to a connection in SYN_SENT, =20
ESTABLISHED or later
842              * state.  tcp_do_segment() always consumes the mbuf =20
chain, unlocks
843              * the inpcb, and unlocks pcbinfo.
844              */
845             tcp_do_segment(m, th, so, tp, drop_hdrlen, tlen);
846             INP_INFO_UNLOCK_ASSERT(&tcbinfo);
847             return;
848
849     dropwithreset:
(kgdb) down
#8  0x00000000c03eaf98 in tcp_do_segment (m=3D0xfffff8005b354000,
     th=3D0xfffff8000133283c, so=3D0xfffff800014be570, tp=3D0xfffff800014fce=
38,
     drop_hdrlen=3D52, tlen=3D0) at /usr/src/sys/netinet/tcp_input.c:2347
2347                    (void) tcp_output(tp);
(kgdb) list
2342
2343            /*
2344             * Return any desired output.
2345             */
2346            if (needoutput || (tp->t_flags & TF_ACKNOW))
2347                    (void) tcp_output(tp);
2348
2349    check_delack:
2350            KASSERT(headlocked =3D=3D 0, ("%s: check_delack: head locked=
",
2351                __func__));
(kgdb) down
#7  0x00000000c03edac4 in tcp_output (tp=3D0xfffff800014fce38)
     at /usr/src/sys/netinet/tcp_output.c:802
802                     mb =3D sbsndptr(&so->so_snd, off, len, &moff);
(kgdb) list
797
798                     /*
799                      * Start the m_copy functions from the closest mbuf
800                      * to the offset in the socket buffer chain.
801                      */
802                     mb =3D sbsndptr(&so->so_snd, off, len, &moff);
803
804                     if (len <=3D MHLEN - hdrlen - max_linkhdr) {
805                             m_copydata(mb, moff, (int)len,
806                                 mtod(m, caddr_t) + hdrlen);
(kgdb) down
#6  0x00000000c03edac4 in tcp_output (tp=3D0xfffff800014be6f0)
     at /usr/src/sys/netinet/tcp_output.c:802
802                     mb =3D sbsndptr(&so->so_snd, off, len, &moff);
(kgdb) list
797
798                     /*
799                      * Start the m_copy functions from the closest mbuf
800                      * to the offset in the socket buffer chain.
801                      */
802                     mb =3D sbsndptr(&so->so_snd, off, len, &moff);
803
804                     if (len <=3D MHLEN - hdrlen - max_linkhdr) {
805                             m_copydata(mb, moff, (int)len,
806                                 mtod(m, caddr_t) + hdrlen);
(kgdb) down
#5  0x00000000c02dd1d0 in sbsndptr (sb=3D0xfffff800014be6f0, off=3D0, len=3D=
1390,
     moff=3D0xe5391064) at /usr/src/sys/kern/uipc_sockbuf.c:939
939                  off > 0 && off >=3D m->m_len;
(kgdb) list
934             *moff =3D off - sb->sb_sndptroff;
935             m =3D ret =3D sb->sb_sndptr ? sb->sb_sndptr : sb->sb_mb;
936
937             /* Advance by len to be as close as possible for the =20
next transmit. */
938             for (off =3D off - sb->sb_sndptroff + len - 1;
939                  off > 0 && off >=3D m->m_len;
940                  m =3D m->m_next) {
941                     sb->sb_sndptroff +=3D m->m_len;
942                     off -=3D m->m_len;
943             }
(kgdb) down
#4  0x00000000c0070fe0 in tl1_trap ()
(kgdb) list
944             sb->sb_sndptr =3D m;
945
946             return (ret);
947     }
948
949     /*
950      * Drop a record off the front of a sockbuf and move the next =20
record to the
951      * front.
952      */
953     void
(kgdb) quit
sonnet.diablonet.net>

Please let me know if further information is required and I will =20
furnish, no problem.

Thanks,

-Sean