From owner-freebsd-current@freebsd.org Mon Jan 29 03:19:01 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E963FED0EB1 for ; Mon, 29 Jan 2018 03:19:00 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x243.google.com (mail-it0-x243.google.com [IPv6:2607:f8b0:4001:c0b::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 79BBC7C06F for ; Mon, 29 Jan 2018 03:19:00 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x243.google.com with SMTP id k131so6595013ith.4 for ; Sun, 28 Jan 2018 19:19:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=pAF+Pf9iIpI8tH+GB+K+AsYmKB5evlPFs8Or4EqAEcc=; b=UCZC5vtWI2WeUJqiMFAQ3x5T19+27i5Zs+C9Okb30nAouLVxonBOxbBlDC81SNldMI /0VD1br3wGhO1ZLDLz7VXYCMIOWnlTBhn20IIVKGMJxCn6eTxa5kckvr6z9VkvkCIRVj wMP8hX5uz1bb/WeUYaj8vDV0yC2okI9qVFREXqNRi314PvCohyRY95NV8QqigKKQB47p o9/FLDoFi80mbZd/KbNS5GioRBlcKOKUuBLzzUIV0VbwLesUGbF4rJ+A4pz0wahFjlxD yeT+hCrEF9j5fUt3al+wp+eeYcZzR5Ww0F94R23hqtz3kZ3priSsdGLNR8Pf4GMt9u2N 5lNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=pAF+Pf9iIpI8tH+GB+K+AsYmKB5evlPFs8Or4EqAEcc=; b=WYUwDf36kG3ET9vQCJTRF/Bc9KT6WH2UgFbtjPe/9wFVYqIhrNF9CP26L9maE3qFsE yHlM/IdjHZ58tITo8Uiw5XLMP+vDAkJxVszZ1Ln9GTYORy0PMyUpEsNJ3aRDJ3Zjbc0K y+1cDQuNyrhF/wills/0Y7c4XkqT0xgBDI5yaMZifU/nVXXZZ3ZxkRiiKOjXxn/znYnF DkNuexyvXB5il/I5V1zrbO/3b8a1ZT0vxhk6fTY7qg6Af3ZV4Twcry3+pDRx0YsAGDKi AumgZxr4jE3rLMv9UX21S+Pf191OhOITUwKKLx+qIXFuFFD3DMnFRUv5lZOe8I/9my6t FpVw== X-Gm-Message-State: AKwxytdhHLsVWfc0uujddftWR0nd4uqDLBEOYGXhNhFPo5Jx16g3IUz9 TAuGdPGzci+SVXOT9FGZCj+vqVjW2hz6Xy95xIaZ5w== X-Google-Smtp-Source: AH8x225eVnwbUKPBIL0+go4ZzmrZr/duvXyki26rxtRF+vWCyElGmFpBjv5X406ZvivhUKISIoOJc4gVkbvHcXTU52o= X-Received: by 10.36.238.68 with SMTP id b65mr25327679iti.97.1517195939693; Sun, 28 Jan 2018 19:18:59 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.201.67 with HTTP; Sun, 28 Jan 2018 19:18:59 -0800 (PST) X-Originating-IP: [2603:300b:6:5100:18a2:a4f7:170:8dd9] In-Reply-To: <201801290238.w0T2co4M053082@slippy.cwsent.com> References: <2effa324-c428-6135-371b-acb00c803d29@freebsd.org> <201801290238.w0T2co4M053082@slippy.cwsent.com> From: Warner Losh Date: Sun, 28 Jan 2018 20:18:59 -0700 X-Google-Sender-Auth: ocAmQNbgyUEYgSMUp47B0W8EQY0 Message-ID: Subject: Re: Panic on shutdown @r328436: "Unholding 6 with cnt = -559038242" To: Cy Schubert Cc: Allan Jude , FreeBSD Current Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jan 2018 03:19:01 -0000 On Sun, Jan 28, 2018 at 7:38 PM, Cy Schubert wrote: > In message <2effa324-c428-6135-371b-acb00c803d29@freebsd.org>, Allan > Jude write > s: > > On 2018-01-28 16:28, Warner Losh wrote: > > > On Sun, Jan 28, 2018 at 2:22 PM, thomas masper < > thomas.masper@gmail.com> > > > wrote: > > > > > >> Hi, > > >> similar panic happen to me when extracting a pendrive from laptop US= B > port > > >> (I tried 3 different pendrive). > > >> No issue if I reboot or shutdown. I don't know if those two issues a= re > > >> related. > > >> > > > > > > Do you have a reproducible test case? Ideally, it would be 'insert an= d > > > remove usb thumb drive' but maybe there's more steps between insert a= nd > > > removal. > > > > > > Warner > > > > > > > > > > > >> panic: Releasing 6 with cnt =3D -559038242 > > Converting this to hex we get DEADC0DE. vm/uma_dbg.c:static const uint32_t uma_junk =3D 0xdeadc0de; Use after free it is then... Warner > > >> > > >> GNU gdb (GDB) 8.0.1 [GDB v8.0.1 for FreeBSD] > > >> Copyright (C) 2017 Free Software Foundation, Inc. > > >> License GPLv3+: GNU GPL version 3 or later < > http://gnu.org/licenses/gpl. > > >> html > > >>> > > >> This is free software: you are free to change and redistribute it. > > >> There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > > >> and "show warranty" for details. > > >> This GDB was configured as "x86_64-portbld-freebsd12.0". > > >> Type "show configuration" for configuration details. > > >> For bug reporting instructions, please see: > > >> . > > >> Find the GDB manual and other documentation resources online at: > > >> . > > >> For help, type "help". > > >> Type "apropos word" to search for commands related to "word"... > > >> Reading symbols from /boot/kernel/kernel...Reading symbols from > > >> /usr/lib/debug//boot/kernel/kernel.debug...done. > > >> done. > > >> > > >> Unread portion of the kernel message buffer: > > >> da0 at umass-sim0 bus 0 scbus4 target 0 lun 0 > > >> da0: s/n 30E47C20 detached > > >> (da0:umass-sim0:0:0:0): Periph destroyed > > >> panic: Releasing 6 with cnt =3D -559038242 > > >> cpuid =3D 0 > > >> time =3D 1517158352 > > >> KDB: stack backtrace: > > >> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > > >> 0xfffffe00593838c0 > > >> vpanic() at vpanic+0x18d/frame 0xfffffe0059383920 > > >> panic() at panic+0x43/frame 0xfffffe0059383980 > > >> dadiskgonecb() at dadiskgonecb+0x42/frame 0xfffffe00593839a0 > > >> g_disk_providergone() at g_disk_providergone+0x25/frame > 0xfffffe00593839d0 > > >> g_destroy_provider() at g_destroy_provider+0xae/frame > 0xfffffe00593839f0 > > >> g_wither_washer() at g_wither_washer+0x87/frame 0xfffffe0059383a30 > > >> g_run_events() at g_run_events+0x3ca/frame 0xfffffe0059383a70 > > >> fork_exit() at fork_exit+0x84/frame 0xfffffe0059383ab0 > > >> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0059383ab0 > > >> --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- > > >> KDB: enter: panic > > >> > > >> __curthread () at ./machine/pcpu.h:229 > > >> 229 __asm("movq %%gs:%1,%0" : "=3Dr" (td) > > >> (kgdb) #0 __curthread () at ./machine/pcpu.h:229 > > >> #1 doadump (textdump=3D0) at /usr/src/sys/kern/kern_shutdown.c:346 > > >> #2 0xffffffff8040a08b in db_dump (dummy=3D, > > >> dummy2=3D, dummy3=3D, dummy4=3D) > > >> at /usr/src/sys/ddb/db_command.c:574 > > >> #3 0xffffffff80409e59 in db_command (last_cmdp=3D, > > >> cmd_table=3D, dopager=3D) > > >> at /usr/src/sys/ddb/db_command.c:481 > > >> #4 0xffffffff80409bd4 in db_command_loop () > > >> at /usr/src/sys/ddb/db_command.c:534 > > >> #5 0xffffffff8040cdff in db_trap (type=3D, > code=3D > >> out>) > > >> at /usr/src/sys/ddb/db_main.c:250 > > >> #6 0xffffffff80b0d923 in kdb_trap (type=3D3, code=3D-61456, tf=3D > >> out>) > > >> at /usr/src/sys/kern/subr_kdb.c:697 > > >> #7 0xffffffff80f7b498 in trap (frame=3D0xfffffe00593837f0) > > >> at /usr/src/sys/amd64/amd64/trap.c:547 > > >> #8 > > >> #9 kdb_enter (why=3D0xffffffff811f101e "panic", msg=3D) > > >> at /usr/src/sys/kern/subr_kdb.c:479 > > >> #10 0xffffffff80ac8d3a in vpanic (fmt=3D, > > >> ap=3D0xfffffe0059383960) > > >> at /usr/src/sys/kern/kern_shutdown.c:800 > > >> #11 0xffffffff80ac8dc3 in panic ( > > >> fmt=3D0xffffffff81b1bbd8 "\257\257\033\201\377\377\= 377\ > > >> 377") > > >> at /usr/src/sys/kern/kern_shutdown.c:738 > > >> #12 0xffffffff80368bb2 in da_periph_release (periph=3D, > > >> token=3DDA_REF_GEOM) at /usr/src/sys/cam/scsi/scsi_da.c:1591 > > >> #13 dadiskgonecb (dp=3D) at > > >> /usr/src/sys/cam/scsi/scsi_da.c:1904 > > >> #14 0xffffffff80a0fdd5 in g_disk_providergone (pp=3D0xfffff80003e8b7= 00) > > >> at /usr/src/sys/geom/geom_disk.c:783 > > >> #15 0xffffffff80a15f9e in g_destroy_provider (pp=3D0xfffff80003e8b70= 0) > > >> at /usr/src/sys/geom/geom_subr.c:746 > > >> #16 0xffffffff80a15e17 in g_wither_washer () > > >> at /usr/src/sys/geom/geom_subr.c:461 > > >> #17 0xffffffff80a112da in g_run_events () > > >> at /usr/src/sys/geom/geom_event.c:297 > > >> #18 0xffffffff80a89444 in fork_exit ( > > >> callout=3D0xffffffff80a138c0 , arg=3D0x0, > > >> frame=3D0xfffffe0059383ac0) at /usr/src/sys/kern/kern_fork.c:103= 9 > > >> #19 > > >> (kgdb) > > >> > > >> > > >> uname -a > > >> FreeBSD laptopW530.tommyBSD.org 12.0-CURRENT FreeBSD 12.0-CURRENT #1= 3 > > >> r328509M: Sun Jan 28 15:38:35 CET 2018 > > >> tommy@laptopW530.tommyBSD.org:/usr/obj/usr/src/amd64.amd64/ > sys/GENERIC > > >> amd64 > > >> > > >> Regards, > > >> thomas > > >> > > >> > > >> On Fri, Jan 26, 2018 at 4:07 PM, David Wolfskill < > david@catwhisker.org> > > >> wrote: > > >> > > >>> On Fri, Jan 26, 2018 at 07:47:48AM -0700, Warner Losh wrote: > > >>>> On Fri, Jan 26, 2018 at 5:29 AM, David Wolfskill < > david@catwhisker.org > > >>> > > >>>> wrote: > > >>>> > > >>>>> This is on my "build machine" (laptop is still building updated > ports > > >>>>> for today, so I don't know yet whether or not it encounters this.= ) > > >>>>> > > >>>> > > >>>> Running a kernel with INVARIANTS, right? > > >>> > > >>> Yes -- GENERIC. > > >>> > > >>>>> I had performed a source-based update from r328393 to r328436, > > >>>>> rebooted, performed "make delete-old-libs", and all seemed well. > > >>>>> > > >>>> > > >>>> This has my change 328415 in it. > > >>> > > >>> :-) > > >>> > > >>>>> I then issued "sudo shutdown -p now", and serial console shows: > > >>>>> panic: Unholding 6 with cnt =3D -559038242 > > >>>>> cpuid =3D 3 > > >>>>> time =3D 1516968697 > > >>>>> KDB: stack backtrace: > > >>>>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > > >>>>> 0xfffffe00004288c0 > > >>>>> vpanic() at vpanic+0x18d/frame 0xfffffe0000428920 > > >>>>> panic() at panic+0x43/frame 0xfffffe0000428980 > > >>>>> dadiskgonecb() at dadiskgonecb+0x42/frame 0xfffffe00004289a0 > > >>>>> g_disk_providergone() at g_disk_providergone+0x25/frame > > >>> 0xfffffe00004289d0 > > >>>>> g_destroy_provider() at g_destroy_provider+0xae/frame > > >>> 0xfffffe00004289f0 > > >>>>> g_wither_washer() at g_wither_washer+0x87/frame 0xfffffe0000428a3= 0 > > >>>>> g_run_events() at g_run_events+0x3ca/frame 0xfffffe0000428a70 > > >>>>> fork_exit() at fork_exit+0x84/frame 0xfffffe0000428ab0 > > >>>>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000428ab0 > > >>>>> --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- > > >>>>> KDB: enter: panic > > >>>>> [ thread pid 13 tid 100044 ] > > >>>>> Stopped at kdb_enter+0x3b: movq $0,kdb_why > > >>>>> db> > > >>>>> > > >>>> > > >>>> That's no good. We're releasing a reference to the da peripheral > > >> because > > >>>> geom has finished with the disk and is giving us a final callback > so we > > >>> can > > >>>> drop the reference we took when we created the geom. Trouble is, c= nt > > >>> should > > >>>> be like 1 always for this code, but it's not. It looks like it may > be > > >>> bytes > > >>>> to a pointer :( > > >>>> > > >>>> > > >>>>> As noted, this is a build machine, and it was to be powered off f= or > > >>>>> the rest of the day anyway, so I don't need to get it up & runnin= g > > >>>>> immediately: I can poke at the ddb prompt, given some clues. > > >>>>> > > >>>> > > >>>> I don't suppose you can attach kgdb to this machine? I'd be > interested > > >> to > > >>>> see what the contents of the softc are...a > > >>> > > >>> Pointer to how to do that? > > >>> > > >>> I do have ddb right now.... > > >>> > > >>>> .... > > >>>> Thanks for the report. This is quite troubling. > > >>> > > >>> Well, let's get it fixed, then! :-) > > >>> > > >>>> Warner > > >>>> .... > > >>> > > >>> I should still have access to the serial console after I get in to > the > > >>> office (heading out shortly). > > >>> > > >>> Peace, > > >>> david > > >>> -- > > >>> David H. Wolfskill david@catwhisker.or= g > > >>> "unfortunately, no trust!=E2=80=9D -- well, of course! You reap wh= at you > sow. > > >>> > > >>> See http://www.catwhisker.org/~david/publickey.gpg for my public > key. > > >>> > > >> _______________________________________________ > > >> freebsd-current@freebsd.org mailing list > > >> https://lists.freebsd.org/mailman/listinfo/freebsd-current > > >> To unsubscribe, send any mail to "freebsd-current-unsubscribe@ > freebsd.org" > > >> > > > _______________________________________________ > > > freebsd-current@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > > To unsubscribe, send any mail to "freebsd-current-unsubscribe@ > freebsd.org" > > > > > > > I've been seeing this today while working on my laptop. > > > > 1) insert USB stick. > > 2) mount UFS partition to /mnt > > 3) copy a file off > > 4) umount /mnt > > 5) remove usb stick > > 6) instant panic > > > > Oddly, it is the same negative number every time (-559038242), so it > > isn't random/memory corruption. > > > > > > -- > > Allan Jude > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to "freebsd-current-unsubscribe@ > freebsd.org" > > > > > -- > Cheers, > Cy Schubert > FreeBSD UNIX: Web: http://www.FreeBSD.org > > The need of the many outweighs the greed of the few. > > > > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org= " > >