From owner-freebsd-net@freebsd.org Wed Jan 6 20:50:53 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 77C44A66DDE for ; Wed, 6 Jan 2016 20:50:53 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-ig0-x22f.google.com (mail-ig0-x22f.google.com [IPv6:2607:f8b0:4001:c05::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B214194F; Wed, 6 Jan 2016 20:50:53 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-ig0-x22f.google.com with SMTP id ik10so40548656igb.1; Wed, 06 Jan 2016 12:50:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ceKy5t2gyhkMrcIC1sAieJNzIoMSFXuiKf5V6rWkJTc=; b=z4fsJ+Nt4/gfEXkHNFf8s3xcpPGEPnbuKmOlzAKP851uUJeG0WHvgIsP1YURIU6JAq GNf5/g0JaaR9HAItdKIPTSB041NI5xXe0uCVhNZi38r1Hl957S1R1evcJotzcLm7Rq3l RURq7xV7r0L/oAF8FRfihdB43KNVCwqeiDVyRw1im7xwfd0MUnugqjLsYKUdGUeD9JZz QiM59JClg7GXV00wkJC6JfWdFXMX0Uyky7trF/FO/6ShPbivMlaiApccfBVBLA0qv+4+ 9BoXvFIea5bh6+NnbeV9e9pB2VQTdCEIIGueaX/12xJ5Gd0i4v+T72Bgh1pWMAsn8Jkc jAiA== MIME-Version: 1.0 X-Received: by 10.50.143.39 with SMTP id sb7mr2769017igb.37.1452113452831; Wed, 06 Jan 2016 12:50:52 -0800 (PST) Received: by 10.36.121.202 with HTTP; Wed, 6 Jan 2016 12:50:52 -0800 (PST) In-Reply-To: <20160106180008.GA1363@chd.heemeyer.club> References: <20160106180008.GA1363@chd.heemeyer.club> Date: Wed, 6 Jan 2016 12:50:52 -0800 Message-ID: Subject: Re: long network segment call stack can fail From: Adrian Chadd To: Chagin Dmitry Cc: FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jan 2016 20:50:53 -0000 hah, that's very pretty. The downside of a completely direct-dispatch network stack. :) -a On 6 January 2016 at 10:00, Chagin Dmitry wrote: > Hi, all. > > Here's a panic that occurs during the massive network loads, > as far as I can see network segment has eaten the whole stack, so > only 0xeb0 bytes left for interrupt hanlder. > > > #0 doadump (textdump=0x8151cd90) at /home/git/head/sys/kern/kern_shutdown.c:296 > #1 0xffffffff803d4715 in db_fncall_generic (addr=0xffffffff80763f00, rv=0xffffffff8151cd60 , > nargs=0x0, args=0xffffffff8151cd70 ) at /home/git/head/sys/ddb/db_command.c:568 > #2 0xffffffff803d3e94 in db_fncall (dummy1=0xffffffff8151cea0, dummy2=0x0, dummy3=0x0, > dummy4=0xffffffff8151cea0 "\n") at /home/git/head/sys/ddb/db_command.c:616 > #3 0xffffffff803d371e in db_command (last_cmdp=0xffffffff812e3838 , cmd_table=0x0, dopager=0x1) > at /home/git/head/sys/ddb/db_command.c:440 > #4 0xffffffff803d32be in db_command_loop () at /home/git/head/sys/ddb/db_command.c:493 > #5 0xffffffff803d7b53 in db_trap (type=0x3, code=0x0) at /home/git/head/sys/ddb/db_main.c:251 > #6 0xffffffff807c568f in kdb_trap (type=0x3, code=0x0, tf=0xffffffff8151d430 ) > at /home/git/head/sys/kern/subr_kdb.c:654 > #7 0xffffffff80c7a9f1 in trap (frame=0xffffffff8151d430 ) > at /home/git/head/sys/amd64/amd64/trap.c:549 > #8 0xffffffff80c7bb5a in trap_check (frame=0xffffffff8151d430 ) > at /home/git/head/sys/amd64/amd64/trap.c:628 > #9 0xffffffff80c53f47 in calltrap () at /home/git/head/sys/amd64/amd64/exception.S:234 > #10 0xffffffff807c4ee5 in breakpoint () at ./machine/cpufunc.h:63 > #11 0xffffffff807c4ae7 in kdb_enter (why=0xffffffff80db94e3 "panic", msg=0xffffffff80db94e3 "panic") > at /home/git/head/sys/kern/subr_kdb.c:443 > #12 0xffffffff8076461b in vpanic (fmt=0xffffffff80e1067f "double fault", ap=0xffffffff8151d630 ) > at /home/git/head/sys/kern/kern_shutdown.c:750 > #13 0xffffffff807646c0 in panic (fmt=0xffffffff80e1067f "double fault") at /home/git/head/sys/kern/kern_shutdown.c:688 > #14 0xffffffff80c7bc6a in dblfault_handler (frame=0xffffffff8151d6b0 ) > at /home/git/head/sys/amd64/amd64/trap.c:861 > #15 0xffffffff80c5403c in Xdblfault () at /home/git/head/sys/amd64/amd64/exception.S:290 > #16 0xffffffff807a26c5 in cpu_search (cg=0xffffffff8146b148 , low=0xfffffe02d858b298, high=0x0, match=0x1) > at /home/git/head/sys/kern/sched_ule.c:689 > #17 cpu_search_lowest (cg=0xffffffff8146b148 , low=0xfffffe02d858b298) > at /home/git/head/sys/kern/sched_ule.c:781 > #18 0xffffffff807a286b in cpu_search (cg=0xffffffff8146b068 , low=0xfffffe02d858b418, high=0x0, match=0x1) > at /home/git/head/sys/kern/sched_ule.c:708 > #19 cpu_search_lowest (cg=0xffffffff8146b068 , low=0xfffffe02d858b418) > at /home/git/head/sys/kern/sched_ule.c:781 > #20 0xffffffff807a286b in cpu_search (cg=0xffffffff8146b030 , low=0xfffffe02d858b490, high=0x0, match=0x1) > at /home/git/head/sys/kern/sched_ule.c:708 > #21 cpu_search_lowest (cg=0xffffffff8146b030 , low=0xfffffe02d858b490) > at /home/git/head/sys/kern/sched_ule.c:781 > #22 0xffffffff807a9496 in sched_lowest (cg=0xffffffff8146b030 , mask=..., pri=0x78, maxload=0x7fffffff, > prefer=0x0) at /home/git/head/sys/kern/sched_ule.c:813 > #23 0xffffffff807a511b in sched_pickcpu (td=0xfffff8000bb3a4f0, flags=0x0) at /home/git/head/sys/kern/sched_ule.c:1290 > #24 0xffffffff807a61c8 in sched_add (td=0xfffff8000bb3a4f0, flags=0x0) at /home/git/head/sys/kern/sched_ule.c:2426 > #25 0xffffffff807a5e78 in sched_wakeup (td=0xfffff8000bb3a4f0) at /home/git/head/sys/kern/sched_ule.c:2058 > #26 0xffffffff80777eb4 in setrunnable (td=0xfffff8000bb3a4f0) at /home/git/head/sys/kern/kern_synch.c:522 > #27 0xffffffff807dcf99 in sleepq_resume_thread (sq=0xfffff800036fb680, td=0xfffff8000bb3a4f0, pri=0x0) > at /home/git/head/sys/kern/subr_sleepqueue.c:776 > #28 0xffffffff807db303 in sleepq_timeout (arg=0xfffff8000bb3a4f0) at /home/git/head/sys/kern/subr_sleepqueue.c:915 > #29 0xffffffff8078bfac in softclock_call_cc (c=0xfffff8000bb3a8a8, cc=0xffffffff81589180 , direct=0x1) > at /home/git/head/sys/kern/kern_timeout.c:723 > #30 0xffffffff8078b78a in callout_process (now=0x393bab69d8fb) at /home/git/head/sys/kern/kern_timeout.c:493 > #31 0xffffffff80cc8932 in handleevents (now=0x393bab69d8fb, fake=0x0) at /home/git/head/sys/kern/kern_clocksource.c:212 > #32 0xffffffff80cc941c in timercb (et=0xffffffff81562d98 , arg=0x0) > at /home/git/head/sys/kern/kern_clocksource.c:345 > #33 0xffffffff80d1e9e8 in lapic_handle_timer (frame=0xfffffe02d858bdf0) at /home/git/head/sys/x86/x86/local_apic.c:1056 > #34 0xffffffff80c54b3c in Xtimerint () at /home/git/head/sys/amd64/amd64/apic_vector.S:133 > #35 0xffffffff80c0cc36 in uma_dbg_getslab (zone=0xfffff80003e15000, item=0xfffff801f0e4b800) > at /home/git/head/sys/vm/uma_dbg.c:219 > #36 0xffffffff80c0ca34 in uma_dbg_alloc (zone=0xfffff80003e15000, slab=0x0, item=0xfffff801f0e4b800) > at /home/git/head/sys/vm/uma_dbg.c:242 > #37 0xffffffff80c04e58 in uma_zalloc_arg (zone=0xfffff80003e15000, udata=0xfffffe02d858c070, flags=0x1) > at /home/git/head/sys/vm/uma_core.c:2220 > #38 0xffffffff8095ea38 in m_gethdr (how=0x1, type=0x1) at /home/git/head/sys/sys/mbuf.h:672 > #39 0xffffffff8095e8b6 in ieee80211_mbuf_adjust (vap=0xfffff8000f2b1000, hdrsize=0x1a, key=0xfffffe0001817138, > m=0xfffff8000f744400) at /home/git/head/sys/net80211/ieee80211_output.c:1113 > #40 0xffffffff8095add2 in ieee80211_encap (vap=0xfffff8000f2b1000, ni=0xfffffe0001817000, m=0xfffff8000f744400) > at /home/git/head/sys/net80211/ieee80211_output.c:1354 > #41 0xffffffff8095a234 in ieee80211_vap_pkt_send_dest (vap=0xfffff8000f2b1000, m=0xfffff8000f744400, > ni=0xfffffe0001817000) at /home/git/head/sys/net80211/ieee80211_output.c:258 > #42 0xffffffff8095c91e in ieee80211_start_pkt (vap=0xfffff8000f2b1000, m=0xfffff8000f744400) > at /home/git/head/sys/net80211/ieee80211_output.c:426 > #43 0xffffffff8095c27b in ieee80211_vap_transmit (ifp=0xfffff8000f2b7000, m=0xfffff8000f744400) > at /home/git/head/sys/net80211/ieee80211_output.c:486 > #44 0xffffffff808db57b in bridge_enqueue (sc=0xfffff8000f2c5c00, dst_ifp=0xfffff8000f2b7000, m=0xfffff8000f744400) > at /home/git/head/sys/net/if_bridge.c:1919 > #45 0xffffffff808e02be in bridge_output (ifp=0xfffff8000f2b7000, m=0xfffff8000f744400, sa=0x0, rt=0x0) > at /home/git/head/sys/net/if_bridge.c:2085 > #46 0xffffffff808e4b52 in ether_output (ifp=0xfffff8000f2b7000, m=0xfffff8000f744400, dst=0xfffff8000befe6b0, > ro=0xfffffe02d858d248) at /home/git/head/sys/net/if_ethersubr.c:376 > #47 0xffffffff8095cab7 in ieee80211_output (ifp=0xfffff8000f2b7000, m=0xfffff8000f744400, dst=0xfffff8000befe6b0, > ro=0xfffffe02d858d248) at /home/git/head/sys/net80211/ieee80211_output.c:576 > #48 0xffffffff809fd851 in ip_output (m=0xfffff8000f744400, opt=0x0, ro=0xfffffe02d858d248, flags=0x0, imo=0x0, > inp=0xfffff801f786c7d0) at /home/git/head/sys/netinet/ip_output.c:638 > #49 0xffffffff80ade593 in tcp_output (tp=0xfffff8016b2bc418) at /home/git/head/sys/netinet/tcp_output.c:1414 > #50 0xffffffff80ad59ee in tcp_do_segment (m=0xfffff8000f71d100, th=0xfffff800071f804a, so=0xfffff8016b5c92e8, > tp=0xfffff8016b2bc418, drop_hdrlen=0x34, tlen=0x544, iptos=0x0, ti_locked=0x1) > at /home/git/head/sys/netinet/tcp_input.c:1904 > #51 0xffffffff80ad3c9f in tcp_input (mp=0xfffffe02d858de28, offp=0xfffffe02d858de00, proto=0x6) > at /home/git/head/sys/netinet/tcp_input.c:1417 > #52 0xffffffff809f8b91 in ip_input (m=0x0) at /home/git/head/sys/netinet/ip_input.c:786 > #53 0xffffffff808f6b4e in netisr_dispatch_src (proto=0x1, source=0x0, m=0xfffff8000f71d100) > at /home/git/head/sys/net/netisr.c:972 > #54 0xffffffff808f7157 in netisr_dispatch (proto=0x1, m=0xfffff8000f71d100) at /home/git/head/sys/net/netisr.c:1063 > #55 0xffffffff808e55b0 in ether_demux (ifp=0xfffff8000f2b7000, m=0xfffff8000f71d100) > at /home/git/head/sys/net/if_ethersubr.c:805 > #56 0xffffffff808e7231 in ether_input_internal (ifp=0xfffff8000f2b7000, m=0xfffff8000f71d100) > at /home/git/head/sys/net/if_ethersubr.c:611 > #57 0xffffffff808e6a3f in ether_nh_input (m=0xfffff8000f71d100) at /home/git/head/sys/net/if_ethersubr.c:641 > #58 0xffffffff808f6b4e in netisr_dispatch_src (proto=0x5, source=0x0, m=0xfffff8000f71d100) > at /home/git/head/sys/net/netisr.c:972 > #59 0xffffffff808f7157 in netisr_dispatch (proto=0x5, m=0xfffff8000f71d100) at /home/git/head/sys/net/netisr.c:1063 > #60 0xffffffff808e5b6c in ether_input (ifp=0xfffff8000f2b7000, m=0xfffff8000f71d100) > at /home/git/head/sys/net/if_ethersubr.c:715 > #61 0xffffffff80938d6a in ieee80211_deliver_data (vap=0xfffff8000f2b1000, ni=0xfffffe0001817000, m=0xfffff8000f71d100) > at /home/git/head/sys/net80211/ieee80211_input.c:298 > #62 0xffffffff80974e77 in sta_input (ni=0xfffffe0001817000, m=0xfffff8000f71d100, rxs=0xfffffe02d858e440, rssi=0x82, > nf=0xb0) at /home/git/head/sys/net80211/ieee80211_sta.c:868 > #63 0xffffffff809383b8 in ieee80211_input_mimo (ni=0xfffffe0001817000, m=0xfffff8000f71d100, rx=0xfffffe02d858e560) > at /home/git/head/sys/net80211/ieee80211_input.c:103 > #64 0xffffffff804c158f in iwm_mvm_rx_rx_mpdu (sc=0xfffffe000108f000, pkt=0xfffff800071f8000, data=0xfffffe00010ce928) > at /home/git/head/sys/dev/iwm/if_iwm.c:2391 > #65 0xffffffff804bfe88 in iwm_notif_intr (sc=0xfffffe000108f000) at /home/git/head/sys/dev/iwm/if_iwm.c:4092 > #66 0xffffffff804bf55e in iwm_intr (arg=0xfffffe000108f000) at /home/git/head/sys/dev/iwm/if_iwm.c:4416 > #67 0xffffffff8070f7c8 in intr_event_execute_handlers (p=0xfffff80003df7aa0, ie=0xfffff80003def900) > at /home/git/head/sys/kern/kern_intr.c:1262 > #68 0xffffffff807105b7 in ithread_execute_handlers (p=0xfffff80003df7aa0, ie=0xfffff80003def900) > at /home/git/head/sys/kern/kern_intr.c:1275 > #69 0xffffffff807103e6 in ithread_loop (arg=0xfffff80007002500) at /home/git/head/sys/kern/kern_intr.c:1356 > #70 0xffffffff8070ad89 in fork_exit (callout=0xffffffff80710270 , arg=0xfffff80007002500, > frame=0xfffffe02d858eac0) at /home/git/head/sys/kern/kern_fork.c:1010 > #71 0xffffffff80c5447e in fork_trampoline () at /home/git/head/sys/amd64/amd64/exception.S:609 > #71 0xffffffff80c5447e in fork_trampoline () at /home/git/head/sys/amd64/amd64/exception.S:609 > > up 71 > 609 call fork_exit > > print $sp > $1 = (void *) 0xfffffe02d858eac0 > > > up 35 > #35 0xffffffff80c0cc36 in uma_dbg_getslab (zone=0xfffff80003e15000, item=0xfffff801f0e4b800) > at /home/git/head/sys/vm/uma_dbg.c:219 > 219 if (keg->uk_flags & UMA_ZONE_HASH) > > print $sp > $37 = (void *) 0xfffffe02d858beb0 > > so, 0x0eb0 bytes left on stack before fault at 0xfffffe02d858b000 > > > up 16 > #16 0xffffffff807a26c5 in cpu_search (cg=0xffffffff8146b148 , low=0xfffffe02d858b298, high=0x0, match=0x1) > at /home/git/head/sys/kern/sched_ule.c:689 > 689 cpu = CPU_FFS(&cpumask) - 1; > > print $sp > $56 = (void *) 0xfffffe02d858b000 > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"