From owner-freebsd-wireless@FreeBSD.ORG Wed Jan 29 21:29:43 2014 Return-Path: Delivered-To: freebsd-wireless@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 09F91E2E; Wed, 29 Jan 2014 21:29:43 +0000 (UTC) Received: from mail-qc0-x231.google.com (mail-qc0-x231.google.com [IPv6:2607:f8b0:400d:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 988F5174E; Wed, 29 Jan 2014 21:29:42 +0000 (UTC) Received: by mail-qc0-f177.google.com with SMTP id i8so3643479qcq.36 for ; Wed, 29 Jan 2014 13:29:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=gc7/15eIjiN5YGODvevvmh6MwtdvfoKTABS2eehQ76c=; b=TS7Wm8m2OEgfHhVNcn3d07P+67r0djn9/XO4ePmZplbgMJXE6qr6R25tn6tTzX6cDg 015nMkUJYEIaOeczNhhIfGIPAxQ8x7SZiHrqaw9z8cv/DYEnQA/6T1sNHppoT2+kxW22 2MECUYHNTVsvJin+UfJekkrgdYHJWrr6V2rESnMQQR4kf1ZuYrigAT3vq/wquiuNJarO csrIs5hZhk6jv3m+sPkxosl+r4rqXTI0yLeQacDhcPS109FWeJVS6onCSE0RPBpiYiM5 jjFqIW5jqh6FhjfVd7vaicM5N79byPG1QSraylrl/lFkSoyT7LS7gijOm/q3lYrGiVCc 3S9A== MIME-Version: 1.0 X-Received: by 10.224.103.131 with SMTP id k3mr16096293qao.102.1391030981665; Wed, 29 Jan 2014 13:29:41 -0800 (PST) Received: by 10.140.27.151 with HTTP; Wed, 29 Jan 2014 13:29:41 -0800 (PST) In-Reply-To: References: Date: Wed, 29 Jan 2014 19:29:41 -0200 Message-ID: Subject: Re: FreeBSD 10.0: hostapd crash with Ralink 3070 From: Pedro Flynn To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-wireless@freebsd.org" X-BeenThere: freebsd-wireless@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussions of 802.11 stack, tools device driver development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2014 21:29:43 -0000 Hi! As stated yesterday, I built a kernel with debug symbols and generated a second crash dump based on this kernel. Files are vmcore.1.xz and core.txt.1: https://drive.google.com/folderview?id=0B0sVwxI7RI7oc3R2bjVQR0pXWG8&usp=sharing Thanks for any suggestion, pflynn On Wed, Jan 29, 2014 at 8:35 AM, Pedro Flynn wrote: > Hmmm... > Where did you see the NULL value? I could not figure it out. > > (Yesterday I built a kernel with debugging symbols enabled and I will > generate a new crash dump tonight. I hope this one will have much more > information). > > Thanks, > > pflynn > > > On Tue, Jan 28, 2014 at 9:54 PM, Adrian Chadd wrote: > >> Yup. Is it? >> >> Adrian >> On Jan 28, 2014 6:10 PM, "Pedro Flynn" wrote: >> >>> You mean rvp->beacon_mbuf is null? >>> >>> Thanks, >>> >>> pflynn >>> >>> >>> On Tue, Jan 28, 2014 at 9:06 PM, Pedro Flynn wrote: >>> >>>> Just to bring to our attention frame 8: >>>> >>>> (kgdb) frame 8 >>>> #8 0xffffffff81a198bc in run_update_beacon (vap=0xfffff8000e8dd000, >>>> item=2) >>>> at /usr/src/sys/modules/usb/run/../../../dev/usb/wlan/if_run.c:3974 >>>> 3974 ieee80211_beacon_update(vap->iv_bss, &rvp->bo, rvp->beacon_mbuf, >>>> mcast); >>>> Current language: auto; currently minimal >>>> (kgdb) print run_update_beacon >>>> $23 = {void (struct ieee80211vap *, >>>> int)} 0xffffffff81a19750 >>>> (kgdb) >>>> >>>> thanks, >>>> >>>> pflynn >>>> >>>> >>>> On Tue, Jan 28, 2014 at 9:04 PM, Adrian Chadd wrote: >>>> >>>>> Right, frame 8 (the run beacon update) is passing a NULL mbuf into >>>>> net80211. Why's it doing that. >>>>> >>>>> >>>>> >>>>> -a >>>>> >>>>> >>>>> On 28 January 2014 15:02, Pedro Flynn wrote: >>>>> > Here we go (this output is not beautiful...). Please, let me know if >>>>> I >>>>> > missed something or if I did something wrong: >>>>> > >>>>> > bt output: >>>>> > >>>>> > #0 doadump (textdump=) at pcpu.h:219 >>>>> > #1 0xffffffff808af530 in kern_reboot (howto=260) >>>>> > at /usr/src/sys/kern/kern_shutdown.c:447 >>>>> > #2 0xffffffff808af8f4 in panic (fmt=) >>>>> > at /usr/src/sys/kern/kern_shutdown.c:754 >>>>> > #3 0xffffffff80c8e692 in trap_fatal (frame=, >>>>> > eva=) at /usr/src/sys/amd64/amd64/trap.c:882 >>>>> > #4 0xffffffff80c8e969 in trap_pfault (frame=0xfffffe009695f720, >>>>> usermode=0) >>>>> > at /usr/src/sys/amd64/amd64/trap.c:699 >>>>> > #5 0xffffffff80c8e0f6 in trap (frame=0xfffffe009695f720) >>>>> > at /usr/src/sys/amd64/amd64/trap.c:463 >>>>> > #6 0xffffffff80c75392 in calltrap () >>>>> > at /usr/src/sys/amd64/amd64/exception.S:232 >>>>> > #7 0xffffffff809b1163 in ieee80211_beacon_update >>>>> (ni=0xfffffe0000ffc000, >>>>> > bo=0xfffff8000e8dd9e8, m=0x0, mcast=0) at atomic.h:161 >>>>> > #8 0xffffffff81a198bc in run_update_beacon (vap=0xfffff8000e8dd000, >>>>> item=2) >>>>> > at >>>>> /usr/src/sys/modules/usb/run/../../../dev/usb/wlan/if_run.c:3974 >>>>> > #9 0xffffffff809b42bd in ieee80211_wme_updateparams_locked ( >>>>> > vap=0xfffff8000e8dd000) at ieee80211_var.h:814 >>>>> > #10 0xffffffff809b437a in ieee80211_wme_updateparams >>>>> > (vap=0xfffff8000e8dd000) >>>>> > at /usr/src/sys/net80211/ieee80211_proto.c:1150 >>>>> > #11 0xffffffff809b3f43 in ieee80211_wme_initparams (vap=>>>> optimized >>>>> > out>) >>>>> > at /usr/src/sys/net80211/ieee80211_proto.c:955 >>>>> > #12 0xffffffff809a9aec in ieee80211_sta_join1 () >>>>> > at /usr/src/sys/net80211/ieee80211_node.c:741 >>>>> > #13 0xffffffff8099047b in hostap_newstate (vap=0xfffff8000e8dd000, >>>>> > nstate=, arg=) >>>>> > at /usr/src/sys/net80211/ieee80211_hostap.c:274 >>>>> > #14 0xffffffff81a1a36a in run_newstate (vap=, >>>>> > nstate=IEEE80211_S_RUN, arg=-1) >>>>> > at >>>>> /usr/src/sys/modules/usb/run/../../../dev/usb/wlan/if_run.c:1881 >>>>> > #15 0xffffffff809b2edf in ieee80211_newstate_cb >>>>> (xvap=0xfffff8000e8dd000, >>>>> > npending=) >>>>> > at /usr/src/sys/net80211/ieee80211_proto.c:1756 >>>>> > #16 0xffffffff808f5b66 in taskqueue_run_locked >>>>> (queue=0xfffff8000e8e4600) >>>>> > at /usr/src/sys/kern/subr_taskqueue.c:333 >>>>> > #17 0xffffffff808f63e8 in taskqueue_thread_loop (arg=>>>> optimized out>) >>>>> > at /usr/src/sys/kern/subr_taskqueue.c:535 >>>>> > #18 0xffffffff8088198a in fork_exit ( >>>>> > callout=0xffffffff808f6340 , >>>>> > arg=0xfffffe0000ff60f0, frame=0xfffffe009695fc00) >>>>> > at /usr/src/sys/kern/kern_fork.c:995 >>>>> > #19 0xffffffff80c758ce in fork_trampoline () >>>>> > at /usr/src/sys/amd64/amd64/exception.S:606 >>>>> > #20 0x0000000000000000 in ?? () >>>>> > >>>>> > frame 0 >>>>> > #0 doadump (textdump=) at pcpu.h:219 >>>>> > 219 pcpu.h: No such file or directory. >>>>> > in pcpu.h >>>>> > print doadump >>>>> > $1 = {int (boolean_t)} 0xffffffff808af6f0 >>>>> > >>>>> > frame 1: >>>>> > #1 0xffffffff808af530 in kern_reboot (howto=260) >>>>> > at /usr/src/sys/kern/kern_shutdown.c:447 >>>>> > 447 doadump(TRUE); >>>>> > print kern_reboot >>>>> > print kern_reboot >>>>> > $3 = {void (int)} 0xffffffff808aedf0 >>>>> > >>>>> > frame 2 >>>>> > #2 0xffffffff808af8f4 in panic (fmt=) >>>>> > at /usr/src/sys/kern/kern_shutdown.c:754 >>>>> > 754 kern_reboot(bootopt); >>>>> > (kgdb) print panic >>>>> > $4 = {void (const char *)} 0xffffffff808af760 >>>>> > >>>>> > frame 3 >>>>> > #3 0xffffffff80c8e692 in trap_fatal (frame=, >>>>> > eva=) at /usr/src/sys/amd64/amd64/trap.c:882 >>>>> > 882 panic("%s", trap_msg[type]); >>>>> > (kgdb) print trap_fatal >>>>> > $5 = {void (struct trapframe *, vm_offset_t)} 0xffffffff80c8e2f0 >>>>> > >>>>> > (kgdb) frame 4 >>>>> > #4 0xffffffff80c8e969 in trap_pfault (frame=0xfffffe009695f720, >>>>> usermode=0) >>>>> > at /usr/src/sys/amd64/amd64/trap.c:699 >>>>> > 699 trap_fatal(frame, eva); >>>>> > (kgdb) print trap_pfault >>>>> > $6 = {int (struct trapframe *, int)} 0xffffffff80c8e6a0 >>>>> > (kgdb) frame 5 >>>>> > #5 0xffffffff80c8e0f6 in trap (frame=0xfffffe009695f720) >>>>> > at /usr/src/sys/amd64/amd64/trap.c:463 >>>>> > 463 (void) trap_pfault(frame, FALSE); >>>>> > (kgdb) print trap >>>>> > $7 = {void (struct trapframe *)} 0xffffffff80c8db10 >>>>> > >>>>> > frame 6 >>>>> > #6 0xffffffff80c75392 in calltrap () >>>>> > at /usr/src/sys/amd64/amd64/exception.S:232 >>>>> > 232 call trap >>>>> > Current language: auto; currently asm >>>>> > (kgdb) print calltrap >>>>> > $8 = {} 0xffffffff80c7538a >>>>> > (kgdb) frame 7 >>>>> > #7 0xffffffff809b1163 in ieee80211_beacon_update >>>>> (ni=0xfffffe0000ffc000, >>>>> > bo=0xfffff8000e8dd9e8, m=0x0, mcast=0) at atomic.h:161 >>>>> > 161 atomic.h: No such file or directory. >>>>> > in atomic.h >>>>> > Current language: auto; currently minimal >>>>> > (kgdb) print ieee80211_beacon_update >>>>> > $9 = {int (struct ieee80211_node *, struct ieee80211_beacon_offsets >>>>> *, >>>>> > struct mbuf *, int)} 0xffffffff809b1090 >>>>> > >>>>> > frame 8 >>>>> > #8 0xffffffff81a198bc in run_update_beacon (vap=0xfffff8000e8dd000, >>>>> item=2) >>>>> > at >>>>> /usr/src/sys/modules/usb/run/../../../dev/usb/wlan/if_run.c:3974 >>>>> > 3974 ieee80211_beacon_update(vap->iv_bss, &rvp->bo, rvp->beacon_mbuf, >>>>> > mcast); >>>>> > (kgdb) print run_update_beacon >>>>> > $10 = {void (struct ieee80211vap *, >>>>> > int)} 0xffffffff81a19750 >>>>> > (kgdb) frame 9 >>>>> > #9 0xffffffff809b42bd in ieee80211_wme_updateparams_locked ( >>>>> > vap=0xfffff8000e8dd000) at ieee80211_var.h:814 >>>>> > 814 vap->iv_update_beacon(vap, what); >>>>> > (kgdb) print ieee80211_wme_updateparams_locked >>>>> > $11 = {void (struct ieee80211vap >>>>> > *)} 0xffffffff809b3f90 >>>>> > (kgdb) frame 10 >>>>> > #10 0xffffffff809b437a in ieee80211_wme_updateparams >>>>> > (vap=0xfffff8000e8dd000) >>>>> > at /usr/src/sys/net80211/ieee80211_proto.c:1150 >>>>> > 1150 ieee80211_wme_updateparams_locked(vap); >>>>> > (kgdb) print ieee80211_wme_updateparams >>>>> > $12 = {void (struct ieee80211vap >>>>> > *)} 0xffffffff809b4320 >>>>> > >>>>> > frame 11 >>>>> > #11 0xffffffff809b3f43 in ieee80211_wme_initparams (vap=>>>> optimized >>>>> > out>) >>>>> > at /usr/src/sys/net80211/ieee80211_proto.c:955 >>>>> > 955 ieee80211_wme_updateparams(vap); >>>>> > (kgdb) print ieee80211_wme_initparams >>>>> > $13 = {void (struct ieee80211vap >>>>> > *)} 0xffffffff809b3ca0 >>>>> > (kgdb) frame 12 >>>>> > #12 0xffffffff809a9aec in ieee80211_sta_join1 () >>>>> > at /usr/src/sys/net80211/ieee80211_node.c:741 >>>>> > 741 ieee80211_wme_initparams(vap); >>>>> > (kgdb) print ieee80211_sta_join1 >>>>> > $14 = {int (struct ieee80211_node *)} 0xffffffff809a9a10 >>>>> > >>>>> > (kgdb) frame 13 >>>>> > #13 0xffffffff8099047b in hostap_newstate (vap=0xfffff8000e8dd000, >>>>> > nstate=, arg=) >>>>> > at /usr/src/sys/net80211/ieee80211_hostap.c:274 >>>>> > 274 ieee80211_ht_adjust_channel(ic, >>>>> > (kgdb) print hostap_newstate >>>>> > $15 = {int (struct ieee80211vap *, enum ieee80211_state, >>>>> > int)} 0xffffffff80990190 >>>>> > frame 14 >>>>> > #14 0xffffffff81a1a36a in run_newstate (vap=, >>>>> > nstate=IEEE80211_S_RUN, arg=-1) >>>>> > at >>>>> /usr/src/sys/modules/usb/run/../../../dev/usb/wlan/if_run.c:1881 >>>>> > 1881 return(rvp->newstate(vap, nstate, arg)); >>>>> > (kgdb) print run_newstate >>>>> > $16 = {int (struct ieee80211vap *, enum ieee80211_state, >>>>> > int)} 0xffffffff81a19b30 >>>>> > (kgdb) frame 15 >>>>> > #15 0xffffffff809b2edf in ieee80211_newstate_cb >>>>> (xvap=0xfffff8000e8dd000, >>>>> > npending=) >>>>> > at /usr/src/sys/net80211/ieee80211_proto.c:1756 >>>>> > 1756 rc = vap->iv_newstate(vap, nstate, arg); >>>>> > (kgdb) print ieee80211_newstate_cb >>>>> > $17 = {void (void *, int)} 0xffffffff809b2d90 >>>>> > (kgdb) frame 16 >>>>> > #16 0xffffffff808f5b66 in taskqueue_run_locked >>>>> (queue=0xfffff8000e8e4600) >>>>> > at /usr/src/sys/kern/subr_taskqueue.c:333 >>>>> > 333 task->ta_func(task->ta_context, pending); >>>>> > (kgdb) print taskqueue_run_locked >>>>> > $18 = {void (struct taskqueue *)} 0xffffffff808f5a80 >>>>> >>>>> > frame 17 >>>>> > #17 0xffffffff808f63e8 in taskqueue_thread_loop (arg=>>>> optimized out>) >>>>> > at /usr/src/sys/kern/subr_taskqueue.c:535 >>>>> > 535 taskqueue_run_locked(tq); >>>>> > (kgdb) print taskqueue_thread_loop >>>>> > $19 = {void (void *)} 0xffffffff808f6340 >>>>> > (kgdb) frame 18 >>>>> > #18 0xffffffff8088198a in fork_exit ( >>>>> > callout=0xffffffff808f6340 , >>>>> > arg=0xfffffe0000ff60f0, frame=0xfffffe009695fc00) >>>>> > at /usr/src/sys/kern/kern_fork.c:995 >>>>> > 995 callout(arg, frame); >>>>> > (kgdb) print fork_exit >>>>> > $20 = {void (void (*)(void *, struct trapframe *), void *, struct >>>>> trapframe >>>>> > *)} 0xffffffff808818f0 >>>>> > (kgdb) frame 19 >>>>> > #19 0xffffffff80c758ce in fork_trampoline () >>>>> > at /usr/src/sys/amd64/amd64/exception.S:606 >>>>> > 606 call fork_exit >>>>> > Current language: auto; currently asm >>>>> > (kgdb) print fork_trampoline >>>>> > $21 = {} 0xffffffff80c758c0 >>>>> >>>>> > frame 20 >>>>> > #20 0x0000000000000000 in ?? () >>>>> > >>>>> > Thanks, >>>>> > >>>>> > pflynn >>>>> > >>>>> > >>>>> > On Tue, Jan 28, 2014 at 8:47 PM, Adrian Chadd >>>>> wrote: >>>>> >> >>>>> >> ok, do 'bt', and see what's being passed into >>>>> ieee80211_beacon_update. >>>>> >> Use 'frame X' to switch to frame X, and 'print VARIABLE_NAME' to >>>>> print >>>>> >> out the contents of the given variable name. >>>>> >> >>>>> >> That mbuf looks like it's NULL, which is odd. >>>>> >> >>>>> >> Thanks! >>>>> >> >>>>> >> >>>>> >> -a >>>>> >> >>>>> >> >>>>> >> On 28 January 2014 14:45, Pedro Flynn >>>>> wrote: >>>>> >> > OK! This is what I have: >>>>> >> > >>>>> >> > list * (0xffffffff809b1163) >>>>> >> > Undefined command: "". Try "help". >>>>> >> > (kgdb) list * (0xffffffff809b1163) >>>>> >> > 0xffffffff809b1163 is in ieee80211_beacon_update >>>>> >> > (/usr/src/sys/net80211/ieee80211_output.c:3099). >>>>> >> > 3094 /* XXX do WME aggressive mode processing? */ >>>>> >> > 3095 IEEE80211_UNLOCK(ic); >>>>> >> > 3096 return 1; /* just assume length changed */ >>>>> >> > 3097 } >>>>> >> > 3098 >>>>> >> > 3099 wh = mtod(m, struct ieee80211_frame *); >>>>> >> > 3100 seqno = ni->ni_txseqs[IEEE80211_NONQOS_TID]++; >>>>> >> > 3101 *(uint16_t *)&wh->i_seq[0] = >>>>> >> > 3102 htole16(seqno << IEEE80211_SEQ_SEQ_SHIFT); >>>>> >> > 3103 M_SEQNO_SET(m, seqno); >>>>> >> > Current language: auto; currently minimal >>>>> >> > (kgdb) >>>>> >> > >>>>> >> > >>>>> >> > (by the way, I'm building a kernel with debug symbols) >>>>> >> > >>>>> >> > Thanks, >>>>> >> > >>>>> >> > pflynn >>>>> >> > >>>>> >> > >>>>> >> > >>>>> >> > On Tue, Jan 28, 2014 at 8:34 PM, Adrian Chadd >>>> > >>>>> >> > wrote: >>>>> >> >> >>>>> >> >> Ok, fire up kgdb >>>>> >> >> >>>>> >> >> # kgdb /boot/kernel/kernel /var/crash/vmcore.0 >>>>> >> >> >>>>> >> >> then >>>>> >> >> >>>>> >> >> (gdb) list * (0xffffffff809b1163) >>>>> >> >> >>>>> >> >> (.. that's the "instruction pointer" at the time of the panic.) >>>>> >> >> >>>>> >> >> I bet it's iv_bss. >>>>> >> >> >>>>> >> >> >>>>> >> >> >>>>> >> >> -a >>>>> >> > >>>>> >> > >>>>> > >>>>> > >>>>> >>>> >>>> >>> >