From owner-freebsd-bugs@FreeBSD.ORG Sat Jul 10 10:30:22 2004 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5206816A4CE for ; Sat, 10 Jul 2004 10:30:22 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4084A43D2D for ; Sat, 10 Jul 2004 10:30:22 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.11/8.12.11) with ESMTP id i6AAUMpv022475 for ; Sat, 10 Jul 2004 10:30:22 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.12.11/8.12.11/Submit) id i6AAUMUk022474; Sat, 10 Jul 2004 10:30:22 GMT (envelope-from gnats) Resent-Date: Sat, 10 Jul 2004 10:30:22 GMT Resent-Message-Id: <200407101030.i6AAUMUk022474@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Daniel Lang Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ADAAD16A4CE for ; Sat, 10 Jul 2004 10:24:19 +0000 (GMT) Received: from mailout1.informatik.tu-muenchen.de (mailout1.informatik.tu-muenchen.de [131.159.0.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id C906243D1F for ; Sat, 10 Jul 2004 10:24:18 +0000 (GMT) (envelope-from langd@informatik.tu-muenchen.de) Message-Id: <20040710102417.8C54428465@atrbg11.informatik.tu-muenchen.de> Date: Sat, 10 Jul 2004 12:24:17 +0200 (CEST) From: Daniel Lang To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 Subject: kern/68889: panic: m_copym, length > size of mbuf chain X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Daniel Lang List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2004 10:30:22 -0000 >Number: 68889 >Category: kern >Synopsis: panic: m_copym, length > size of mbuf chain >Confidential: no >Severity: critical >Priority: high >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Sat Jul 10 10:30:19 GMT 2004 >Closed-Date: >Last-Modified: >Originator: Daniel Lang >Release: FreeBSD 5.2-CURRENT i386 >Organization: LEO >Environment: System: FreeBSD atleo6.leo.org 5.2-CURRENT FreeBSD 5.2-CURRENT #8: Fri Jul 9 13:40:23 CEST 2004 root@atleo6.leo.org:/usr/obj/usr/src/sys/ATLEO6 i386 See PR kern/68779 for kernel config, dmesg and tunables. >Description: kgdb session: GNU gdb 20040615 [GDB v6.x for FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-portbld-freebsd5.2"... panic: m_copym, length > size of mbuf chain panic messages: --- panic: m_copym, length > size of mbuf chain cpuid = 2; Stack backtrace: backtrace(100,c3510000,ca90c300,0,ca90c300) at 0xc0521c36 = backtrace+0x12 panic(c06b358b,0,c3cbea00,0,1) at 0xc0521d56 = panic+0x11e m_copym(0,2e1,4c6,1,c06ad10b) at 0xc0551805 = m_copym+0xa1 tcp_output(c3f50000,0,0,0,1) at 0xc059ed5a = tcp_output+0xa4a tcp_input(c862f400,14,0,14,17489f83) at 0xc059c8a9 = tcp_input+0x1d9d ip_input(c862f400) at 0xc059571e = ip_input+0x832 netisr_processqueue(c074b358,c3522640,c351e880,e1c15d1c,c0510004) at 0xc05866aa = netisr_processqueue+0x6e swi_net(0) at 0xc0586a11 = swi_net+0x85 ithread_loop(c351e880,e1c15d48,c351e880,c050fed0,0) at 0xc0510004 = ithread_loop+0x134 fork_exit(c050fed0,c351e880,e1c15d48) at 0xc050f460 = fork_exit+0x98 fork_trampoline() at 0xc06611dc = fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe1c15d7c, ebp = 0 --- Debugger("panic") Dumping 2047 MB 16 32 48 64 80 96 112 128 144 160 176 192 208 224 240 256 272 288 304 320 336 352 368 384 400 416 432 448 464 480 496 512 528 544 560 576 592 608 624 640 656 672 688 704 720 736 752 768 784 800 816 832 848 864 880 896 912 928 944 960 976 992 1008 1024 1040 1056 1072 1088 1104 1120 1136 1152 1168 1184 1200 1216 1232 1248 1264 1280 1296 1312 1328 1344 1360 1376 1392 1408 1424 1440 1456 1472 1488 1504 1520 1536 1552 1568 1584 1600 1616 1632 1648 1664 1680 1696 1712 1728 1744 1760 1776 1792 1808 1824 1840 1856 1872 1888 1904 1920 1936 1952 1968 1984 2000 2016 2032 --- #0 doadump () at /usr/src/sys/kern/kern_shutdown.c:236 236 dumping++; doadump () at /usr/src/sys/kern/kern_shutdown.c:236 236 dumping++; (kgdb) bt #0 doadump () at /usr/src/sys/kern/kern_shutdown.c:236 #1 0xc04524e2 in db_fncall (dummy1=0, dummy2=0, dummy3=-1066031552, dummy4=0xe1c158ec "\bYÁáX¢QÀF") at /usr/src/sys/ddb/db_command.c:551 #2 0xc04522f0 in db_command (last_cmdp=0xc0716a30, cmd_table=0x0, aux_cmd_tablep=0xc06cea88, aux_cmd_tablep_end=0xc06ceaa0) at /usr/src/sys/ddb/db_command.c:348 #3 0xc04523c8 in db_command_loop () at /usr/src/sys/ddb/db_command.c:475 #4 0xc0454b4d in db_trap (type=3, code=0) at /usr/src/sys/ddb/db_trap.c:73 #5 0xc065fbe9 in kdb_trap (type=3, code=0, regs=0xe1c15a18) at /usr/src/sys/i386/i386/db_interface.c:159 #6 0xc06720c8 in trap (frame= {tf_fs = -1067057128, tf_es = 16, tf_ds = 16, tf_edi = -1066715765, tf_esi = 1, tf_ebp = -507422116, tf_isp = -507422140, tf_ebx = 0, tf_edx = 0, tf_ecx = -1056882688, tf_eax = 18, tf_trapno = 3, tf_err = 0, tf_eip = -1067057498, tf_cs = 8, tf_eflags = 642, tf_esp = -507422072, tf_ss = -507422084}) at /usr/src/sys/i386/i386/trap.c:579 #7 0xc066117a in calltrap () at /usr/src/sys/i386/i386/exception.s:140 #8 0xc0660018 in decode_syscall (number=0, p=0x1) at /usr/src/sys/i386/i386/db_trace.c:190 #9 0xc0521d69 in panic (fmt=0xc06b358b "m_copym, length > size of mbuf chain") at /usr/src/sys/kern/kern_shutdown.c:543 #10 0xc0551805 in m_copym (m=0x0, off0=737, len=1222, wait=1) at /usr/src/sys/kern/uipc_mbuf.c:380 ---Type to continue, or q to quit--- #11 0xc059ed5a in tcp_output (tp=0xc3f50000) at /usr/src/sys/netinet/tcp_output.c:748 #12 0xc059c8a9 in tcp_input (m=0xc862f400, off0=20) at /usr/src/sys/netinet/tcp_input.c:1929 #13 0xc059571e in ip_input (m=0xc862f400) at /usr/src/sys/netinet/ip_input.c:946 #14 0xc05866aa in netisr_processqueue (ni=0xc074b358) at /usr/src/sys/net/netisr.c:152 #15 0xc0586a11 in swi_net (dummy=0x0) at /usr/src/sys/net/netisr.c:257 #16 0xc0510004 in ithread_loop (arg=0xc351e880) at /usr/src/sys/kern/kern_intr.c:544 #17 0xc050f460 in fork_exit (callout=0xc050fed0 , arg=0xc351e880, frame=0xe1c15d48) at /usr/src/sys/kern/kern_fork.c:815 #18 0xc06611dc in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:209 (kgdb) up 10 #10 0xc0551805 in m_copym (m=0x0, off0=737, len=1222, wait=1) at /usr/src/sys/kern/uipc_mbuf.c:380 380 KASSERT(len == M_COPYALL, (kgdb) l 375 } 376 np = ⊤ 377 top = 0; 378 while (len > 0) { 379 if (m == NULL) { 380 KASSERT(len == M_COPYALL, 381 ("m_copym, length > size of mbuf chain")); 382 break; 383 } 384 if (copyhdr) (kgdb) p len $1 = 1222 (kgdb) p np $2 = (struct mbuf **) 0xca90c300 (kgdb) p *np $3 = (struct mbuf *) 0x0 (kgdb) p m $4 = (struct mbuf *) 0x0 (kgdb) quit # grep M_COPYALL sys/mbuf.h sys/mbuf.h:#define M_COPYALL 1000000000 Hmm I don't get it, why requires the KASSERT len to be _equal_ M_COPYALL. Ah because m is NULL. Clearly that's the problem, right? Ok, I fired up gdb once more (the following is cut & pasted): (kgdb) frame #11 0xc059ed5a in tcp_output (tp=0xc3f50000) at /usr/src/sys/netinet/tcp_output.c:748 748 m->m_next = m_copy(so->so_snd.sb_mb, off, (int) len); (kgdb) p *m $3 = {m_hdr = {mh_next = 0x0, mh_nextpkt = 0x0, mh_data = 0xc3cbe340 "H¬\005Ä\003", mh_len = 40, mh_flags = 2, mh_type = 2}, M_dat = {MH = {MH_pkthdr = {rcvif = 0x0, len = 52, header = 0x264187ab, csum_flags = 0, csum_data = 65535, tags = { slh_first = 0x0}}, MH_dat = {MH_ext = {ext_buf = 0x0, ext_free = 0, ext_args = 0x0, ext_size = 2048, ref_cnt = 0xc405ac48, ext_type = 3}, MH_databuf = '\0' , "\b\000\000H¬\005Ä\003\000\000\000@\006\177ü\203\237H\027Ø¢|\232ÿ¶\004êr E\210#Ñ\177\017\200\020\202\030&Â\000\000\001\001\b\n\000\004ÓÄ\000\002ë ¼Éfce¸ÊiZSÕ\b©0Jî<\027éôУò'\226¢Ð«Ól\2022,ÝQ¦", '*' , "\r\n- --------\r\n220- * about OpenOffice at\r\n\001\000\001\000\000\r\235\000\004EÀÓ&\000\000"}}, M_databuf = "\000\000\000\0004\000\000\000«\207A&\000\000\000\000ÿÿ", '\0' , "\b\000\000H¬\005Ä\003\000\000\000@\006\177ü\203\237H\027Ø¢|\232ÿ¶\004êr E\210#Ñ\177\017\200\020\202\030&Â\000\000\001\001\b\n\000\004ÓÄ\000\002ë ¼Éfce¸ÊiZSÕ\b©0Jî<\027éôУò'\226¢Ð«Ól\2022,ÝQ¦", '*' , "\r\n- --------\r\n220- * about OpenOffice at\r\n\001\000\001\000\000\r\235\000\004EÀÓ&\000\000"}} (kgdb) p *so $4 = {so_count = 1, so_type = 1, so_options = 260, so_linger = 0, so_state = 2, so_qstate = 0, so_pcb = 0xc4d592d0, so_proto = 0xc0700d08, so_head = 0x0, so_incomp = {tqh_first = 0x0, tqh_last = 0x0}, so_comp = { tqh_first = 0x0, tqh_last = 0x0}, so_list = {tqe_next = 0xc58194f0, tqe_prev = 0xc4095c7c}, so_qlen = 0, so_incqlen = 0, so_qlimit = 0, so_timeo = 0, so_error = 0, so_sigio = 0xc4266400, so_oobmark = 0, so_aiojobq = {tqh_first = 0x0, tqh_last = 0xc409f538}, so_rcv = {sb_sel = { si_thrlist = {tqe_next = 0x0, tqe_prev = 0x0}, si_thread = 0x0, si_note = {slh_first = 0x0}, si_flags = 0}, sb_mtx = {mtx_object = { lo_class = 0xc06f6bbc, lo_name = 0xc06b0d7b "so_rcv", lo_type = 0xc06b0d7b "so_rcv", lo_flags = 196608, lo_list = { tqe_next = 0xc4d59360, tqe_prev = 0xc409f5c8}, lo_witness = 0xc07275e8}, mtx_lock = 4, mtx_recurse = 0}, sb_mb = 0x0, sb_mbtail = 0x0, sb_lastrecord = 0x0, sb_cc = 0, sb_hiwat = 65700, sb_mbcnt = 0, sb_mbmax = 262144, sb_ctl = 0, sb_lowat = 1, sb_timeo = 0, sb_flags = 0, sb_state = 32}, so_snd = {sb_sel = {si_thrlist = { tqe_next = 0x0, tqe_prev = 0x0}, si_thread = 0x0, si_note = { slh_first = 0x0}, si_flags = 0}, sb_mtx = {mtx_object = { lo_class = 0xc06f6bbc, lo_name = 0xc06b0d74 "so_snd", lo_type = 0xc06b0d74 "so_snd", lo_flags = 196608, lo_list = { tqe_next = 0xc409f554, tqe_prev = 0xc88d0a10}, lo_witness = 0xc0727610}, mtx_lock = 4, mtx_recurse = 0}, sb_mb = 0xca22f400, sb_mbtail = 0xca423300, sb_lastrecord = 0xca22f400, sb_cc = 975, sb_hiwat = 33580, sb_mbcnt = 1536, sb_mbmax = 262144, sb_ctl = 0, sb_lowat = 2048, sb_timeo = 0, sb_flags = 0, sb_state = 0}, so_upcall = 0, so_upcallarg = 0x0, so_cred = 0xc3461600, so_label = 0x0, so_peerlabel = 0x0, so_gencnt = 44889, so_emuldata = 0x0, so_accf = 0x0} (kgdb) p so->so_snd $5 = {sb_sel = {si_thrlist = {tqe_next = 0x0, tqe_prev = 0x0}, si_thread = 0x0, si_note = {slh_first = 0x0}, si_flags = 0}, sb_mtx = { mtx_object = {lo_class = 0xc06f6bbc, lo_name = 0xc06b0d74 "so_snd", lo_type = 0xc06b0d74 "so_snd", lo_flags = 196608, lo_list = { tqe_next = 0xc409f554, tqe_prev = 0xc88d0a10}, lo_witness = 0xc0727610}, mtx_lock = 4, mtx_recurse = 0}, sb_mb = 0xca22f400, sb_mbtail = 0xca423300, sb_lastrecord = 0xca22f400, sb_cc = 975, sb_hiwat = 33580, sb_mbcnt = 1536, sb_mbmax = 262144, sb_ctl = 0, sb_lowat = 2048, sb_timeo = 0, sb_flags = 0, sb_state = 0} (kgdb) p so->so_snd.sb_mb $6 = (struct mbuf *) 0xca22f400 (kgdb) p *so->so_snd.sb_mb $7 = {m_hdr = {mh_next = 0xc3e42d00, mh_nextpkt = 0x0, mh_data = 0xca22f430 "220- ", '*' , "\r\n220- Welcome to LEO.ORG. Please login as `ftp' to access our archive.\r\n220- \r\n4", mh_len = 161, mh_flags = 2, mh_type = 1}, M_dat = {MH = {MH_pkthdr = { rcvif = 0x0, len = 83, header = 0x64617074, csum_flags = 0, csum_data = 16, tags = {slh_first = 0x0}}, MH_dat = {MH_ext = { ext_buf = 0x2d303232---Can't read userspace from dump, or kernel process--- Soooo, since the argument to m_copy is not 0x0 in the previous frame, but it is on entering m_copy, this looks like a trashed stack? How can I proceed now? >How-To-Repeat: >Fix: >Release-Note: >Audit-Trail: >Unformatted: