From owner-freebsd-net@FreeBSD.ORG Wed Jul 19 15:17:24 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DA7F216A4DA for ; Wed, 19 Jul 2006 15:17:24 +0000 (UTC) (envelope-from rrs@cisco.com) Received: from sj-iport-3.cisco.com (sj-iport-3-in.cisco.com [171.71.176.72]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3E48043D4C for ; Wed, 19 Jul 2006 15:17:23 +0000 (GMT) (envelope-from rrs@cisco.com) Received: from sj-dkim-4.cisco.com ([171.71.179.196]) by sj-iport-3.cisco.com with ESMTP; 19 Jul 2006 08:17:24 -0700 X-IronPort-AV: i="4.06,260,1149490800"; d="scan'208"; a="435186806:sNHT30312156" Received: from sj-core-2.cisco.com (sj-core-2.cisco.com [171.71.177.254]) by sj-dkim-4.cisco.com (8.12.11.20060308/8.12.11) with ESMTP id k6JFHKWx021723; Wed, 19 Jul 2006 08:17:20 -0700 Received: from xbh-sjc-231.amer.cisco.com (xbh-sjc-231.cisco.com [128.107.191.100]) by sj-core-2.cisco.com (8.12.10/8.12.6) with ESMTP id k6JFHK2D029818; Wed, 19 Jul 2006 08:17:20 -0700 (PDT) Received: from xfe-sjc-211.amer.cisco.com ([171.70.151.174]) by xbh-sjc-231.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.211); Wed, 19 Jul 2006 08:17:20 -0700 Received: from [127.0.0.1] ([171.68.225.134]) by xfe-sjc-211.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 19 Jul 2006 08:17:19 -0700 Message-ID: <44BE4D01.3070300@cisco.com> Date: Wed, 19 Jul 2006 11:17:21 -0400 From: Randall Stewart User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20060223 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Pawel Worach References: <44BB7A92.9080008@cisco.com> In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 19 Jul 2006 15:17:20.0078 (UTC) FILETIME=[6D8EE6E0:01C6AB46] DKIM-Signature: a=rsa-sha1; q=dns; l=9659; t=1153322243; x=1154186243; c=relaxed/simple; s=sjdkim4002; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=rrs@cisco.com; z=From:Randall=20Stewart=20 |Subject:Re=3A=20SCTP; X=v=3Dcisco.com=3B=20h=3DeFwn8yHouLxYzxwWS0cbAORTlr0=3D; b=De1WrPzgaZZmN4N6/5yzHj5vnoyK235lhZOgVsK1cmZILmGLiwNTzElP16b3IkFV1QH4WBlW YDxHuYCK+Zdu30J4g0TzQCOL6HZxBi6tM7jq7fS77HYVw0/4p17FI+QG; Authentication-Results: sj-dkim-4.cisco.com; header.From=rrs@cisco.com; dkim=pass ( sig from cisco.com verified; ); Cc: freebsd-net@freebsd.org Subject: Re: SCTP X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jul 2006 15:17:24 -0000 Pawel: Ok, I have verified that even with my patch I can recreate this one.. have not tried the mac one yet (I will fix this first :-D) I do think its an SCTP problem.. it looks to me like an accounting issue... sb_cc or sb_mbcnt is left with a value (at a guess) when we never use the sb_mb pointers... so it panics.. In theory sb_cc and sb_mbcnt should be zero at cleanup.. so some where a bug crept in from the last time I played with netpipe :-0 Should have a patch later today hopefully.. of course I am now trying to get kgdb to build.. I have not done a buildworld in a while and of course there have been changes and the old kgdb will not read the core.. some nice error... sigh... I hate to go down the build-world rat-hole.. at least after the last time I did this it took me a long time to recover... but I will jump in with both feet :-D R Pawel Worach wrote: > On 7/18/06, Pawel Worach wrote: > >> On 7/17/06, Randall Stewart wrote: >> > All: >> > >> > Just a friendly reminder/prod... if you have started >> > testing SCTP.. thats great (any feedback?).. >> > and if you have not .. please do so :-D >> >> Hi, >> >> I played around a bit with NetPIPE, FreeBSD-CURRENT in one end and >> Linux 2.6.17 in the other over a gigabit crossover cable network, 1500 >> MTU. FreeBSD crashes after a while. I do have MAC enabled (no policy >> modules loaded at the time), it looks like it is involved. I think I >> can reproduce this, made it happen on both attempts. >> > > And I removed MAC and I get another panic. No idea if directly related > to SCTP. > > panic: sbdrop > KDB: stack backtrace: > kdb_backtrace(c0761224,c07bd400,c07649ee,d5b14a10,100,...) at > kdb_backtrace+0x2e > panic(c07649ee,c2b42d24,0,0,0,...) at panic+0xb7 > sbdrop_locked(c2aaf9d0,10c19,0,c05250f9,c2aaf9d0,...) at sbdrop_locked+0x46 > sbflush_locked(c2aaf9d0,c2b42b70,0,0,3041,...) at sbflush_locked+0x54 > sbrelease_locked(c2aaf9d0,c2aaf914,d5b14a88,c05eb388,c2b42b70,...) at > sbrelease_locked+0x12 > sofree(c2aaf914,d5b14ae4,4,d5b14ab4,c073ce8c,...) at sofree+0x2a3 > soclose(c2aaf914,d5b14b24,0,d5b14ae0,c059d652,...) at soclose+0x3ff > soo_close(c27305a0,c275bbd0,d5b14af8,c05a9798,c27305a0,...) at > soo_close+0x7d > fdrop_locked(c27305a0,c275bbd0,c0722f76,d5b14be0,4) at fdrop_locked+0xf0 > fdrop(c27305a0,c275bbd0,c2720924,0,c275bbd0,d5b14be0,0,0,0,c06d1e2b,c2720924,c2b41374,c175b040,d5b14b68,c06e2c12,4,c29b7514,28111000,d5b14c5c,c06d39f2,d5b14be0,80,c2b41374,5,0) > > at fdrop+0x5f > closef(c27305a0,c275bbd0,c25c4a00,d5b14c64,0,...) at closef+0x542 > fdfree(c275bbd0,c078e240,2,c053e2e6,c275bbd0,...) at fdfree+0x756 > exit1(c275bbd0,19100,d5b14d30,c0727af3,c275bbd0,...) at exit1+0x660 > sys_exit(c275bbd0,d5b14d04,4,c275bbd0,d5b14d30,...) at sys_exit+0x1d > syscall(bfbf003b,82b003b,bfbc003b,bfbfe910,82c4000,...) at syscall+0x3d3 > Xint0x80_syscall() at Xint0x80_syscall+0x1f > --- syscall (1, FreeBSD ELF32, sys_exit), eip = 0x280e87ef, esp = > 0xbfbc002c, ebp = 0xbfbc0038 --- > Uptime: 10m15s > Physical memory: 502 MB > Dumping 84 MB: 69 53 37 21 5 > > #0 doadump () at pcpu.h:166 > 166 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) bt > #0 doadump () at pcpu.h:166 > #1 0xc0535b44 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 > #2 0xc0535ebd in panic (fmt=0xc07649ee "sbdrop") > at /usr/src/sys/kern/kern_shutdown.c:565 > #3 0xc05876b6 in sbdrop_locked (sb=0xc2aaf9d0, len=68633) > at /usr/src/sys/kern/uipc_socket2.c:1002 > #4 0xc0587574 in sbflush_locked (sb=0xc2aaf9d0) > at /usr/src/sys/kern/uipc_socket2.c:969 > #5 0xc0586952 in sbrelease_locked (sb=0xc2aaf9d0, so=0x0) > at /usr/src/sys/kern/uipc_socket2.c:469 > #6 0xc0581603 in sofree (so=0xc2aaf914) at > /usr/src/sys/kern/uipc_socket.c:587 > #7 0xc0581a8f in soclose (so=0xc2aaf914) > at /usr/src/sys/kern/uipc_socket.c:662 > #8 0xc056b9bd in soo_close (fp=0xc27305a0, td=0xc275bbd0) > at /usr/src/sys/kern/sys_socket.c:317 > #9 0xc050be00 in fdrop_locked (fp=0xc27305a0, td=0x0) at file.h:296 > #10 0xc050bcff in fdrop (fp=0xc27305a0, td=0x0) > at /usr/src/sys/kern/kern_descrip.c:2156 > #11 0xc0509b42 in closef (fp=0xc27305a0, td=0xc275bbd0) > at /usr/src/sys/kern/kern_descrip.c:1971 > #12 0xc0508396 in fdfree (td=0xc275bbd0) > at /usr/src/sys/kern/kern_descrip.c:1653 > #13 0xc0514620 in exit1 (td=0xc275bbd0, rv=102656) > ---Type to continue, or q to quit--- > at /usr/src/sys/kern/kern_exit.c:280 > #14 0xc0513fbd in sys_exit (td=0x0, uap=0x0) > at /usr/src/sys/kern/kern_exit.c:101 > #15 0xc0727af3 in syscall (frame= > {tf_fs = -1078001605, tf_es = 137035835, tf_ds = -1078198213, > tf_edi = -1077942000, tf_esi = 137117696, tf_ebp = -1078198216, tf_isp > = -709800604, tf_ebx = 672598776, tf_edx = 0, tf_ecx = 3, tf_eax = 1, > tf_trapno = 12, tf_err = 2, tf_eip = 672040943, tf_cs = 51, tf_eflags > = 646, tf_esp = -1078198228, tf_ss = 59}) at > /usr/src/sys/i386/i386/trap.c:1015 > #16 0xc071613f in Xint0x80_syscall () at > /usr/src/sys/i386/i386/exception.s:191 > #17 0x00000033 in ?? () > Previous frame inner to this frame (corrupt stack?) > (kgdb) f 3 > #3 0xc05876b6 in sbdrop_locked (sb=0xc2aaf9d0, len=68633) > at /usr/src/sys/kern/uipc_socket2.c:1002 > 1002 panic("sbdrop"); > (kgdb) list > 997 > 998 next = (m = sb->sb_mb) ? m->m_nextpkt : 0; > 999 while (len > 0) { > 1000 if (m == 0) { > 1001 if (next == 0) > 1002 panic("sbdrop"); > 1003 m = next; > 1004 next = m->m_nextpkt; > 1005 continue; > 1006 } > (kgdb) > > I also ran across this in the middle of things... this likely should > have a new thread. > > Fatal trap 12: page fault while in kernel mode > fault virtual address = 0xa0 > fault code = supervisor write, page not present > instruction pointer = 0x20:0xc06444be > stack pointer = 0x28:0xd5b3bc24 > frame pointer = 0x28:0xd5b3bc40 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, def32 1, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 1169 (ssh) > trap number = 12 > panic: page fault > KDB: stack backtrace: > kdb_backtrace(c0761224,c07bd400,c0755b24,d5b3bad8,100,...) at > kdb_backtrace+0x2e > panic(c0755b24,c0779cd1,c2a9e3d4,1,1,...) at panic+0xb7 > trap_fatal(d5b3bbe4,a0,2,8,0,...) at trap_fatal+0x342 > trap_pfault(d5b3bbe4,0,a0,c29e5434,a0,...) at trap_pfault+0x245 > trap(8,c0760028,c0760028,c079aed4,c29e53e4,...) at trap+0x3e3 > calltrap() at calltrap+0x5 > --- trap 0xc, eip = 0xc06444be, esp = 0xd5b3bc24, ebp = 0xd5b3bc40 --- > udp_shutdown(c29e53e4,0,d5b3bd04,c29c3a20,d5b3bc84,...) at > udp_shutdown+0x1e > soshutdown(c29e53e4,2,d5b3bc74,0,d5b3bdd0,...) at soshutdown+0x41 > shutdown(c29c3a20,d5b3bd04,8,16,d5b3bd30,...) at shutdown+0xad > syscall(82a003b,bfbf003b,bfbf003b,0,2,...) at syscall+0x3d3 > Xint0x80_syscall() at Xint0x80_syscall+0x1f > --- syscall (134, FreeBSD ELF32, shutdown), eip = 0x2826c327, esp = > 0xbfbfe9fc, ebp = 0xbfbfea18 --- > Uptime: 13m47s > Physical memory: 502 MB > Dumping 46 MB: 31 (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to > abort) 15 > > #0 doadump () at pcpu.h:166 > 166 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) bt > #0 doadump () at pcpu.h:166 > #1 0xc0535b44 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 > #2 0xc0535ebd in panic (fmt=0xc0755b24 "%s") > at /usr/src/sys/kern/kern_shutdown.c:565 > #3 0xc07276b2 in trap_fatal (frame=0xd5b3bbe4, eva=160) > at /usr/src/sys/i386/i386/trap.c:869 > #4 0xc0727345 in trap_pfault (frame=0xd5b3bbe4, usermode=0, eva=160) > at /usr/src/sys/i386/i386/trap.c:778 > #5 0xc0726e93 in trap (frame= > {tf_fs = 8, tf_es = -1066008536, tf_ds = -1066008536, tf_edi = > -1065767212, tf_esi = -1029811228, tf_ebp = -709641152, tf_isp = > -709641200, tf_ebx = 0, tf_edx = -1029948896, tf_ecx = 4, tf_eax = 4, > tf_trapno = 12, tf_err = 2, tf_eip = -1067170626, tf_cs = 32, > tf_eflags = 66198, tf_esp = 0, tf_ss = 0}) > at /usr/src/sys/i386/i386/trap.c:463 > #6 0xc07160ea in calltrap () at /usr/src/sys/i386/i386/exception.s:138 > #7 0xc06444be in udp_shutdown (so=0xc29e53e4) at atomic.h:146 > #8 0xc0583bf1 in soshutdown (so=0xc29e53e4, how=2) > at /usr/src/sys/kern/uipc_socket.c:1810 > #9 0xc058a07d in shutdown (td=0xc29c3a20, uap=0xd5b3bd04) > at /usr/src/sys/kern/uipc_syscalls.c:1328 > #10 0xc0727af3 in syscall (frame= > {tf_fs = 136970299, tf_es = -1078001605, tf_ds = -1078001605, > tf_edi = 0, tf_esi = 2, tf_ebp = -1077941736, tf_isp = -709640860, > tf_ebx = 671968624, tf_ed---Type to continue, or q > to quit--- > x = 0, tf_ecx = 2, tf_eax = 134, tf_trapno = 22, tf_err = 2, tf_eip = > 673628967, tf_cs = 51, tf_eflags = 582, tf_esp = -1077941764, tf_ss = > 59}) > at /usr/src/sys/i386/i386/trap.c:1015 > #11 0xc071613f in Xint0x80_syscall () at > /usr/src/sys/i386/i386/exception.s:191 > #12 0x00000033 in ?? () > Previous frame inner to this frame (corrupt stack?) > -- Randall Stewart NSSTG - Cisco Systems Inc. 803-345-0369 815-342-5222 (cell)