From owner-freebsd-stable@freebsd.org Thu Aug 11 23:37:57 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5A6D0BB6B29 for ; Thu, 11 Aug 2016 23:37:57 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 362211721 for ; Thu, 11 Aug 2016 23:37:57 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 35661BB6B24; Thu, 11 Aug 2016 23:37:57 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 35193BB6B23 for ; Thu, 11 Aug 2016 23:37:57 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-io0-x229.google.com (mail-io0-x229.google.com [IPv6:2607:f8b0:4001:c06::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F3129171F; Thu, 11 Aug 2016 23:37:56 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-io0-x229.google.com with SMTP id m101so10740638ioi.2; Thu, 11 Aug 2016 16:37:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=tzWJD+MzbrkW6r3MWKzrwkNSLzW2NEl6aCV3WN16eIY=; b=rEB3FFssM4Lxpi6nbYz1MfOfuCChKfZDIGwFeMuOQ4kCI5mle6dBHQD2b/TycsMwQF aXh6wJ/r5jekhQtK5/zADwqTV3pZp+oHLPPeX1nkj/UgBgtHuMRqNn8ECySAO/SQxzO6 rlbe2i5lC8Dc6IH2k+ToSVQnmOKdaRPIOcEfI3Z1VxON9sQ7Sz+sxMnNHK9FbGopSnPe BJ/j5ZMcyQ/Vt+3468WKHvShAlzx+DYA5bncUqSabf8Un49hRHC/XYo+N5aU+j+/g3j9 KraYxlnE4nl9pya2l/7b9xeTUp5DO+fRJ6e/T+VP1Vraz9Nff/Z+9QFcLMvLeAcU5Igd 4q1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=tzWJD+MzbrkW6r3MWKzrwkNSLzW2NEl6aCV3WN16eIY=; b=PcsZjzswjchr9mIcshHE6gkwGjZYhXphoSLqasLuYhNMnQwvlgkDXbsl2KJFDxi8PC RESFAkY4aIVgLyg4n+tLO9GhHKQ02kffg4qROOeBar1X5m2OknH40+W4R6a6ubLvhf+T WHvKZbTBNSIUDN2JSh8bp2AjkeU2CWXKBcXGTCnU4HlQTVMmSALAi5QAxvNi1Ae4pCaE 4zINxhelRLKRde+ZSjG0kaDvoFDIVCI48G02HNWY9aEnxatJd41R0TecX8m23AwLOd7e 8yhmYEm8LWVtHzFXgVcEBXqNriyFFBlJFOFFYrKFRCx7j2T18gf+vnzTvsfEdSuy9uog nsRw== X-Gm-Message-State: AEkoouv+4w7GlHYuNVb9EqZ6DYwvYqAJyS2/CA8rF37jJ0Oqr4rGmJIERyxirWIfkjil/RGtd/6dn0XcTKorUQ== X-Received: by 10.107.15.229 with SMTP id 98mr14251559iop.123.1470958675872; Thu, 11 Aug 2016 16:37:55 -0700 (PDT) MIME-Version: 1.0 Received: by 10.36.141.129 with HTTP; Thu, 11 Aug 2016 16:37:55 -0700 (PDT) In-Reply-To: References: <20160810165458.GB1112@albert.catwhisker.org> <570bda1e-d4d7-42dc-6037-7c321ba9e97d@FreeBSD.org> From: Adrian Chadd Date: Thu, 11 Aug 2016 16:37:55 -0700 Message-ID: Subject: Re: Panic in stable/11 (amd64) @r303903: page fault while in kernel mode To: Bryan Drewery Cc: "stable@freebsd.org" , Andriy Voskoboinyk Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Aug 2016 23:37:57 -0000 .. and maybe we should revert or comment out the code until we figure out what to do about LLADDR checks. (I see this in the detach path too; same kind of race. Sigh.) -adrian On 11 August 2016 at 16:37, Adrian Chadd wrote: > Eep. Is this anotehr case where there's a race and ifp is NULL or the > ll pointer for ifp is NULL or use-after-free'd? > > I remember bumping into these here and there because we don't seem to > have a well defined lifecycle for lladdr access. ;( > > > -adrian > > > On 10 August 2016 at 12:10, Bryan Drewery wrote: >> On 8/10/16 9:54 AM, David Wolfskill wrote: >>> Happened after a few iterations of {"pkill dhclient" followed by >>> "dhclient wlan0"}. >>> >>> Gory details (both "normal" and gzipped, and including the crash >>> dump and crashinfo) are in >>> . >>> >>> Summary: >>> Wed Aug 10 15:56:26 UTC 2016 >>> >>> FreeBSD 11.0-BETA4 FreeBSD 11.0-BETA4 #69 r303902M/303903:1100120: Wed Aug 10 04:00:09 PDT 2016 root@g1-252.catwhisker.org:/common/S3/obj/usr/src/sys/CANARY amd64 >>> >>> panic: page fault >>> >>> GNU gdb 6.1.1 [FreeBSD] >>> Copyright 2004 Free Software Foundation, Inc. >>> GDB is free software, covered by the GNU General Public License, and you are >>> welcome to change it and/or distribute copies of it under certain conditions. >>> Type "show copying" to see the conditions. >>> There is absolutely no warranty for GDB. Type "show warranty" for details. >>> This GDB was configured as "amd64-marcel-freebsd"... >>> >>> Unread portion of the kernel message buffer: >>> >>> >>> Fatal trap 12: page fault while in kernel mode >>> cpuid = 7; apic id = 07 >>> fault virtual address = 0x0 >>> fault code = supervisor read data, page not present >>> instruction pointer = 0x20:0xffffffff80bdaaa1 >>> stack pointer = 0x28:0xfffffe060bc956e0 >>> frame pointer = 0x28:0xfffffe060bc957b0 >>> code segment = base 0x0, limit 0xfffff, type 0x1b >>> = DPL 0, pres 1, long 1, def32 0, gran 1 >>> processor eflags = interrupt enabled, resume, IOPL = 0 >>> current process = 20685 (wpa_supplicant) >>> trap number = 12 >>> panic: page fault >>> cpuid = 7 >>> KDB: stack backtrace: >>> #0 0xffffffff80add787 at kdb_backtrace+0x67 >>> #1 0xffffffff80a950e2 at vpanic+0x182 >>> #2 0xffffffff80a94f53 at panic+0x43 >>> #3 0xffffffff80eead51 at trap_fatal+0x351 >>> #4 0xffffffff80eeaf43 at trap_pfault+0x1e3 >>> #5 0xffffffff80eea4ec at trap+0x26c >>> #6 0xffffffff80ece0d1 at calltrap+0x8 >>> #7 0xffffffff80b9811c at ifioctl+0x133c >>> #8 0xffffffff80afc914 at kern_ioctl+0x2d4 >>> #9 0xffffffff80afc5d1 at sys_ioctl+0x171 >>> #10 0xffffffff80eeb6c9 at amd64_syscall+0x4e9 >>> #11 0xffffffff80ece3bb at Xfast_syscall+0xfb >>> Uptime: 3h0m4s >>> ... >>> Reading symbols from /boot/kernel/linux64.ko...Reading symbols from /usr/lib/debug//boot/kernel/linux64.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/linux64.ko >>> #0 doadump (textdump=) at pcpu.h:221 >>> 221 pcpu.h: No such file or directory. >>> in pcpu.h >>> (kgdb) #0 doadump (textdump=) at pcpu.h:221 >>> #1 0xffffffff80a94b69 in kern_reboot (howto=260) >>> at /usr/src/sys/kern/kern_shutdown.c:366 >>> #2 0xffffffff80a9511b in vpanic (fmt=, >>> ap=) at /usr/src/sys/kern/kern_shutdown.c:759 >>> #3 0xffffffff80a94f53 in panic (fmt=0x0) >>> at /usr/src/sys/kern/kern_shutdown.c:690 >>> #4 0xffffffff80eead51 in trap_fatal (frame=0xfffffe060bc95630, eva=0) >>> at /usr/src/sys/amd64/amd64/trap.c:841 >>> #5 0xffffffff80eeaf43 in trap_pfault (frame=0xfffffe060bc95630, usermode=0) >>> at /usr/src/sys/amd64/amd64/trap.c:691 >>> #6 0xffffffff80eea4ec in trap (frame=0xfffffe060bc95630) >>> at /usr/src/sys/amd64/amd64/trap.c:442 >>> #7 0xffffffff80ece0d1 in calltrap () >>> at /usr/src/sys/amd64/amd64/exception.S:236 >>> #8 0xffffffff80bdaaa1 in ieee80211_ioctl (ifp=0xfffff80007991800, >>> cmd=, data=) >>> at /usr/src/sys/net80211/ieee80211_ioctl.c:3398 >> >> The code crashing is quite recent: >> >>> commit c6321695321bae43c0cd024db564c5207a7e8e31 >>> Author: avos >>> Date: Mon May 2 20:46:05 2016 +0000 >>> >>> net80211: fix MAC address change via SIOCSIFLLADDR ioctl. >>> >>> Recheck MAC address on SIOCSIFFLAGS; as a result, >>> 'ifconfig wlan0 ether ' can be used after interface startup. >>> >>> PR: 208933 >>> >>> >>> git-svn-id: svn+ssh://svn.freebsd.org/base/head@298941 ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f >>> >>> diff --git sys/net80211/ieee80211_ioctl.c sys/net80211/ieee80211_ioctl.c >>> index c3b02e8..823906b 100644 >>> --- sys/net80211/ieee80211_ioctl.c >>> +++ sys/net80211/ieee80211_ioctl.c >>> @@ -3382,8 +3382,18 @@ ieee80211_ioctl(struct ifnet *ifp, u_long cmd, caddr_t data) >>> } >>> IEEE80211_UNLOCK(ic); >>> /* Wait for parent ioctl handler if it was queued */ >>> - if (wait) >>> + if (wait) { >>> ieee80211_waitfor_parent(ic); >>> + >>> + /* >>> + * Check if the MAC address was changed >>> + * via SIOCSIFLLADDR ioctl. >>> + */ >>> + if ((ifp->if_flags & IFF_UP) == 0 && >>> + !IEEE80211_ADDR_EQ(vap->iv_myaddr, IF_LLADDR(ifp))) >>> + IEEE80211_ADDR_COPY(vap->iv_myaddr, >>> + IF_LLADDR(ifp)); >>> + } >>> break; >>> case SIOCADDMULTI: >>> case SIOCDELMULTI: >> >> >>> #9 0xffffffff80b9811c in ifioctl (so=, >>> cmd=, data=, >>> td=) at /usr/src/sys/net/if.c:2447 >>> #10 0xffffffff80afc914 in kern_ioctl (td=, >>> fd=, com=2149607696, data=0xfffffe060bc958e0 "wlan0") >>> at file.h:327 >>> #11 0xffffffff80afc5d1 in sys_ioctl (td=, >>> uap=0xfffffe060bc95a40) at /usr/src/sys/kern/sys_generic.c:743 >>> #12 0xffffffff80eeb6c9 in amd64_syscall (td=, >>> traced=) at subr_syscall.c:135 >>> #13 0xffffffff80ece3bb in Xfast_syscall () >>> at /usr/src/sys/amd64/amd64/exception.S:396 >>> #14 0x00000008015c448a in ?? () >>> Previous frame inner to this frame (corrupt stack?) >>> Current language: auto; currently minimal >>> (kgdb) >>> >>> This was on my laptop, which I'm actively using at work as I type >>> -- though it's now connected via wired NIC (em0). I had experienced >>> no trouble with wlan0 at home (before coming in to work) or on the >>> bus (en route to work). (I didn't attempt it while cycling to the >>> bus stop. :-}) >>> >>> Also, I had no issues running stable/11 (amd64) @303870 -- either >>> at home or at work -- yesterday. On the other hand, this is (so >>> far) a one-off, so alleging a "pattern" at this point is not something >>> I'm willing to do. >>> >>> Peace, >>> david >>> >> >> >> -- >> Regards, >> Bryan Drewery >>