From owner-freebsd-current@freebsd.org Tue Mar 9 23:05:09 2021 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D165D574E21 for ; Tue, 9 Mar 2021 23:05:09 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4Dw9ln3mbFz3Qny for ; Tue, 9 Mar 2021 23:05:09 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mailman.nyi.freebsd.org (Postfix) id 814F2574E20; Tue, 9 Mar 2021 23:05:09 +0000 (UTC) Delivered-To: current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 810F1574DAC for ; Tue, 9 Mar 2021 23:05:09 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com [IPv6:2607:f8b0:4864:20::732]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Dw9ln33MSz3R6x for ; Tue, 9 Mar 2021 23:05:09 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x732.google.com with SMTP id z128so14909247qkc.12 for ; Tue, 09 Mar 2021 15:05:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=aJb3Kc0/xCq3QoBp9qEuPhNdVGxHwGiL1GPhrE1/WJE=; b=HOwtszIvHOgSu/JT7Slwvz2ti5jmX5ykN9KDS1oLjZem7RY0j8FO2XdOAzBnsdMDTL V7XlnjSpMXd7gIJGkBW+9oAchu5FRqEORH0JFFKawmKMcTIqn8BFbEuEQX1IBm6v13hq sN82/uDOoSDdHYiHRCR9PvBYNsPKs841D0BbcPg0BAc4n1/2ttronQfXATY9347CKluq 4svp98Dy+5pn7GSw28iCQVyN6G6JoKzoxiOlFhkYR9a30xKMDRTnPOQln1obS4rc+N85 L77BR0ZRCUR0B+CTWLo6yw7w8WBrXm4FtCxpixboF59DVSHYWSJGayKtw6ogll3OSThF ZFbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=aJb3Kc0/xCq3QoBp9qEuPhNdVGxHwGiL1GPhrE1/WJE=; b=B8MVAgLEJy3Ql4WDQhefPM8Zd1norgjPeN5wEsHN5PCNPoKemuBERe/gJShCB7OanI yHL1JHuP2cP5D+bOdWPaqyabGRaK8M0SSwp1wsIB8timDlDhAhFib4u2OUZElFPlpgnU ciBRYPX9fFzH5ANdd6xR0yxfQ7N4KdZ5X9ppiUfpfPO07+P3G7NvV/BSpw1dA+BNmHNY FnWDTgDMG/j9Vq2TlQRp9ebLyOYzsHV4P+1UQxaEbHhAlJx7Jzq1Lt1yN7M6aI/y8XQV AlHEgjumPFnn3Oirt+hwneunxvUEqb6CeGDfOclXr8mx4j2V4EnE7BhIibd7qYwUi9zw JiBw== X-Gm-Message-State: AOAM532SIwa2lKXAQPbRTNp5ioS00P1GtUmt1q/gqhm0cwgwpVcyL+3G C6BxVc/S04AfZxSSGg1vho1XRCBMe6dAGOngzTmdrIyBOuD7ng== X-Google-Smtp-Source: ABdhPJwmnbN9LHdmw6Yziezqc/T8N2pbPeH85dCackSrcoygzXFmamxxGGpdaNze1hmCftE//Zx0ZtG2i081ANYd1oc= X-Received: by 2002:a37:a085:: with SMTP id j127mr48867qke.206.1615331108229; Tue, 09 Mar 2021 15:05:08 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Tue, 9 Mar 2021 16:04:56 -0700 Message-ID: Subject: Re: "panic: malloc(M_WAITOK) with sleeping prohibited" at main-n245363-b3dac3913dc9 To: David Wolfskill , Warner Losh , FreeBSD Current X-Rspamd-Queue-Id: 4Dw9ln33MSz3R6x X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2021 23:05:09 -0000 On Tue, Mar 9, 2021 at 2:46 PM David Wolfskill wrote: > On Tue, Mar 09, 2021 at 01:23:16PM -0800, David Wolfskill wrote: > > On Tue, Mar 09, 2021 at 01:53:37PM -0700, Warner Losh wrote: > > > ... > > > The following reviews should fix this. It introduces a no-wait variant > for > > > disk_alloc(), provides a way to free allocated, but not created, > disks and > > > changes CAM to use the new routines and take some care for not leaking > when > > > an allocation fails. > > > > > > https://reviews.freebsd.org/D29161 > > > https://reviews.freebsd.org/D29162 > > > https://reviews.freebsd.org/D29163 > > > > > > Maybe you can try it? I got similar tracebacks when I booted w/o these > > > changes, but not a peep with them... > > > ... > > > > Thanks! > > > > They applied cleanly; building now -- both on the build machine (which > > failed earlier) and on the newer laptop (which did not fail earlier, as > > it's good to find out if a change has broken somehing that had been > > working). > > .... > > The laptop still works: > > FreeBSD g1-48.catwhisker.org 14.0-CURRENT FreeBSD 14.0-CURRENT #169 > main-n245338-221622ec0c8e-dirty: Mon Mar 8 03:50:50 PST 2021 > root@g1-48.catwhisker.org:/common/S4/obj/usr/src/amd64.amd64/sys/CANARY > amd64 1400005 1400005 > > FreeBSD g1-48.catwhisker.org 14.0-CURRENT FreeBSD 14.0-CURRENT #170 > main-n245363-b3dac3913dc9-dirty: Tue Mar 9 05:06:34 PST 2021 > root@g1-48.catwhisker.org:/common/S4/obj/usr/src/amd64.amd64/sys/CANARY > amd64 1400005 1400005 > > FreeBSD g1-48.catwhisker.org 14.0-CURRENT FreeBSD 14.0-CURRENT #170 > main-n245363-b3dac3913dc9-dirty: Tue Mar 9 05:06:34 PST 2021 > root@g1-48.catwhisker.org:/common/S4/obj/usr/src/amd64.amd64/sys/CANARY > amd64 1400005 1400005 > > > The build nmachine (still?) panics: > > ... > pass3: 600.000MB/s transfers (SATA 3.x, UDMA5, PIO 8192bytes) > pass3: Command Queueing enabled > pass4 at ahcich4 bus 0 scbus4 target 0 lun 0 > pass4: ACS-2 ATA SATA 3.x device > pass4: Serial Number 000000001242091982C2 > pass4: 600.000MB/s transfers (SATA 3.x, ugen2.2: 0x8000> at usbus2 > UDMA5, PIO 8192bytes) > pass4: Command Queueing enabled > uhub4 on uhub1 > pass5 at ahciem0 bus 0 scbus5 target 0 lun 0 > uhub4: on > usbus2 > Root mount waiting for:pass5: usbus0 > usbus1 usbus2ugen0.2: at usbus0 > CAM SEMB S-E-S 2.00 device > > uma_zalloc_debug: zone "malloc-1024"umass0 on uhub2 > with the following non-sleepable locks held: > umass0: on usbus0 > exclusive sleep mutex CAM device lockumass0: SCSI over Bulk-Only; quirks > = 0x4000 > (CAM device lock) r = 0 (0xfffff800122c9cd0) locked @ > /usr/src/sys/cam/cam_xpt.c:2333 > umass0:6:0: Attached to scbus6 > stack backtrace: > (probe0:umass-sim0:0:0:0): Down reving Protocol Version from 2 to 0? > #0 0xffffffff80c7cce1 at witnesuhub3: 6 ports with 6 removable, self > powered > s_debugger+0x71 > pass6 at umass-sim0 bus 0 scbus6 target 0 lun 0 > #1 0xffffffff80pass6: uhub4: 8 ports with 8 removable, self powered > c7ddfd at witness_warn+0x40d > #2 Removable Direct Access SCSI device > 0xffffffff80f42fe6 at uma_zallpass6: Serial Number 20100818841300000 > oc_arg+0x46 > #3 0xffffffff80be34pass6: 40.000MB/s transfers > panic: malloc(M_WAITOK) with sleeping prohibited > cpuid = 1 > time = 22 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > 0xfffffe00e157a2d0 > vpanic() at vpanic+0x181/frame 0xfffffe00e157a320 > panic() at panic+0x43/frame 0xfffffe00e157a380 > malloc_dbg() at malloc_dbg+0xd4/frame 0xfffffe00e157a3a0 > malloc() at malloc+0x34/frame 0xfffffe00e157a400 > g_post_event_x() at g_post_event_x+0x5a/frame 0xfffffe00e157a450 > g_post_event() at g_post_event+0x48/frame 0xfffffe00e157a4b0 > disk_create() at disk_create+0x16f/frame 0xfffffe00e157a600 > daregister() at daregister+0x70a/frame 0xfffffe00e157a880 > cam_periph_alloc() at cam_periph_alloc+0x57b/frame 0xfffffe00e157a950 > daasync() at daasync+0x2c2/frame 0xfffffe00e157a9c0 > xpt_async_process_dev() at xpt_async_process_dev+0x152/frame > 0xfffffe00e157aa10 > xpt_async_process() at xpt_async_process+0x334/frame 0xfffffe00e157ab20 > xpt_done_process() at xpt_done_process+0x3a3/frame 0xfffffe00e157ab60 > xpt_done_td() at xpt_done_td+0xf5/frame 0xfffffe00e157abb0 > fork_exit() at fork_exit+0x80/frame 0xfffffe00e157abf0 > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e157abf0 > --- trap 0, rip = 0, rsp = 0, rbp = 0 --- > KDB: enter: panic > [ thread pid 17 tid 100095 ] > Stopped at kdb_enter+0x37: movq $0,0x128b8ce(%rip) > db> > I'm willing to "poke at it" a bit, given a hint or two... > OK. I know what's happening here. disk_create() posts an event and also expects to sleep to get memory. I can fix that too, but then that's one more 'no memory' hole. I'm unsure if we should keep playing whack-a-mole, or just defer this creation to the taskqueue we already have hanging around... Warner > Peace, > david > -- > David H. Wolfskill david@catwhisker.org > It is supremely disingenuous to claim a lack of jurisdiction, then > proceed to participate in a decision on the same matter. > > See https://www.catwhisker.org/~david/publickey.gpg for my public key. >