From owner-freebsd-net@freebsd.org Mon Apr 6 21:29:12 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CAE1D27CA6D for ; Mon, 6 Apr 2020 21:29:12 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qk1-x730.google.com (mail-qk1-x730.google.com [IPv6:2607:f8b0:4864:20::730]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48x3Zc4yDPz4ccc; Mon, 6 Apr 2020 21:29:12 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-qk1-x730.google.com with SMTP id 130so2783089qke.4; Mon, 06 Apr 2020 14:29:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=RjN4jiC2V+JPfRA1yEM2LLUKc8p9VVBZFRpbztpvLnk=; b=lObHRdti5rJ7UCuFRhfJjCfNB1qojWTWREwN/+zhMAIfio/dBqgZ65suNKuvJ7sVBr U+8KkH8PPQlaOlD1Qk7kwpQX2tLLbtZuvnWqJQo4a4nkZPS7eH0jd8QI3Cxy+e5QNFjV +UXgUqT15LKByuf+xAnhcUE+FXWlv0EiK8Q2nSMm+6s+NXY5jzjB8qxOuQFxlayakwq2 PJ6Xi5DI6uIVaCUdvrSt4XsaMHdmknA5dRR9MbgET2dHdpOAlzivzypTJhCMkxgohjlv wmi3lGD9w8iOBn0wOy7ypTnahXjjiitd+xIXqHCmJEmLzBna9SgumCfeRAtyNOWN7nlC whSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=RjN4jiC2V+JPfRA1yEM2LLUKc8p9VVBZFRpbztpvLnk=; b=hZsSlfNfw4bikpbEqXdU4eV0SsES2DrOhYbY0tdSPf5xrG4EkzZcuQ6ob96QOrC5Ns DflSAjKeL5VGKWVRGECQ+eZ7bFEEuKy7ew+eeNpJk/iSbU9WTutLehfY2f3JGcKn5Ag8 my5GbQw+/nmB5NzCJVIvzG2dmJxnH9NWmVoa84FKdk0JPzOz2vhvGqsOuhR9RiHCx4Hz 3c4i72+mVokNcApuQAiYarAnFq4rwgap2n0FUboWQJBpwzliVDJXaZ2LFCmVUKDkwgWG Y2Zob/hlqoaCTLrSWKtmixLf40laDGp92y3ffj5hLxsPxsIa8O1+H2uRbWvMbD/kW/8E NwEA== X-Gm-Message-State: AGi0PuYJr6OmHl/P3NmFszvy1+m2vsvenrj7Dp+bRv8/hxFFLYeVbjgf SsarAqggrvW8m66P5FSE77dCwTAUDSM= X-Google-Smtp-Source: APiQypLcxfGO0AL58BrsR7GYeb1gHRRPXlI+ryWi7WKE+PNUaHUTUjXuDbDfNWZU2SsEZwhcKwDGfg== X-Received: by 2002:a37:4548:: with SMTP id s69mr24223387qka.63.1586208551159; Mon, 06 Apr 2020 14:29:11 -0700 (PDT) Received: from raichu (toroon0560w-lp130-10-174-94-17-182.dsl.bell.ca. [174.94.17.182]) by smtp.gmail.com with ESMTPSA id x43sm4791448qtj.65.2020.04.06.14.29.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Apr 2020 14:29:10 -0700 (PDT) Sender: Mark Johnston Date: Mon, 6 Apr 2020 17:29:03 -0400 From: Mark Johnston To: Eric Joyner Cc: Hans Petter Selasky , John Baldwin , Drew Gallatin , freebsd-net@freebsd.org, shurd Subject: Re: Issue with epoch_drain_callbacks and unloading iavf(4) [using iflib] Message-ID: <20200406212903.GA55712@raichu> References: <20200130030911.GA15281@spy> <20200212222219.GE83892@raichu> <20200328225150.GA82767@raichu> <20200331192024.GE97238@raichu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 48x3Zc4yDPz4ccc X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-5.99 / 15.00]; NEURAL_HAM_MEDIUM(-0.99)[-0.993,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2020 21:29:12 -0000 On Mon, Apr 06, 2020 at 02:19:25PM -0700, Eric Joyner wrote: > Mark, > > I think I was mistaken about the backtrace looking the same. I was looking > at it from within ddb, and I think I focused on the > epoch_block_handler_preempt line and didn't notice that it only stopped > there this time. Here's the new one I've got from kgdb: Thanks. Could you try to print "td->td_name" from frame 4? It should also be available as er->er_blockedtd. Basically, I'm trying to verify that the interrupt thread itself isn't the one that we're waiting for, else there is another bug to be fixed. If you can provide kernel symbols and vmcore, I'd be happy to look at it myself. > #0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1448 > #1 0xffffffff80ff2f79 in ipi_nmi_handler () at > /usr/src/sys/x86/x86/mp_x86.c:1405 > #2 0xffffffff810294a6 in trap (frame=0xfffffe003b9b6f30) at > /usr/src/sys/amd64/amd64/trap.c:201 > #3 > #4 epoch_block_handler_preempt (global=0xfffff80003de0100, > cr=0xfffffe00dee85900, arg=0x0) at /usr/src/sys/kern/subr_epoch.c:507 > #5 0xffffffff803b576d in epoch_block (global=0xfffff80003de0100, > cr=0xfffffe00dee85900, cb=0xffffffff80bcf190 , > ct=0x0) at /usr/src/sys/contrib/ck/src/ck_epoch.c:416 > #6 ck_epoch_synchronize_wait (global=0xfffff80003de0100, cb= out>, ct=) at /usr/src/sys/contrib/ck/src/ck_epoch.c:465 > #7 0xffffffff80bcf03c in epoch_wait_preempt (epoch=0xfffff80003de0100) at > /usr/src/sys/kern/subr_epoch.c:529 > #8 0xffffffff80c9410a in if_detach_internal (ifp=0xfffff80067ed4000, > vmove=0, ifcp=0x0) at /usr/src/sys/net/if.c:1123 > #9 0xffffffff80c93ebd in if_detach (ifp=0xfffff80003de0100) at > /usr/src/sys/net/if.c:1063 > #10 0xffffffff80cafa56 in iflib_device_deregister (ctx=0xfffff80088c91800) > at /usr/src/sys/net/iflib.c:5104 > #11 0xffffffff80bc1e2e in DEVICE_DETACH (dev=0xfffff80004706a00) at > ./device_if.h:234 > #12 device_detach (dev=0xfffff80004706a00) at > /usr/src/sys/kern/subr_bus.c:3049 > #13 0xffffffff80bc13fd in devclass_driver_deleted > (busclass=0xfffff80004352900, dc=0xfffff80004385a00, > driver=0xffffffff823329e0 ) at > /usr/src/sys/kern/subr_bus.c:1235 > #14 0xffffffff80bc12ef in devclass_delete_driver > (busclass=0xfffff80004352900, driver=0xffffffff823329e0 > ) at /usr/src/sys/kern/subr_bus.c:1310 > #15 0xffffffff80bc721c in driver_module_handler (mod=0xfffff80015cd8680, > what=1, arg=0xffffffff823329b0 ) at > /usr/src/sys/kern/subr_bus.c:5229 > #16 0xffffffff80b67b82 in module_unload (mod=0xfffff80015cd8680) at > /usr/src/sys/kern/kern_module.c:261 > #17 0xffffffff80b5895b in linker_file_unload (file=0xfffff8016da69a00, > flags=0) at /usr/src/sys/kern/kern_linker.c:700 > #18 0xffffffff80b59dad in kern_kldunload (td=, fileid=5, > flags=0) at /usr/src/sys/kern/kern_linker.c:1153 > #19 0xffffffff8102aa40 in syscallenter (td=) at > /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:162 > #20 amd64_syscall (td=0xfffffe00e839f100, traced=0) at > /usr/src/sys/amd64/amd64/trap.c:1161 > #21 > #22 0x00000008002ddcba in ?? () > Backtrace stopped: Cannot access memory at address 0x7fffffffe188 > > - Eric