From owner-freebsd-net@freebsd.org Fri Feb 21 04:08:03 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0B98D24F93C for ; Fri, 21 Feb 2020 04:08:03 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from cell.glebi.us (glebi.us [162.251.186.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "cell.glebi.us", Issuer "cell.glebi.us" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 48Nyc16GcGz4v77; Fri, 21 Feb 2020 04:08:01 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from cell.glebi.us (localhost [127.0.0.1]) by cell.glebi.us (8.15.2/8.15.2) with ESMTPS id 01L47nNm054022 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 20 Feb 2020 20:07:49 -0800 (PST) (envelope-from glebius@freebsd.org) Received: (from glebius@localhost) by cell.glebi.us (8.15.2/8.15.2/Submit) id 01L47njH054021; Thu, 20 Feb 2020 20:07:49 -0800 (PST) (envelope-from glebius@freebsd.org) X-Authentication-Warning: cell.glebi.us: glebius set sender to glebius@freebsd.org using -f Date: Thu, 20 Feb 2020 20:07:49 -0800 From: Gleb Smirnoff To: Kristof Provost Cc: freebsd-net Subject: Re: vtnet IFF_NEEDSEPOCH? Message-ID: <20200221040749.GM5741@FreeBSD.org> References: <20200218193708.GH5741@FreeBSD.org> <5520BD42-7D17-4561-A2CD-C690B159D15E@FreeBSD.org> <20200219224845.GI5741@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Queue-Id: 48Nyc16GcGz4v77 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.995,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Feb 2020 04:08:03 -0000 On Thu, Feb 20, 2020 at 12:16:06PM +0100, Kristof Provost wrote: K> On 19 Feb 2020, at 23:48, Gleb Smirnoff wrote: K> > On Tue, Feb 18, 2020 at 10:08:50PM +0100, Kristof Provost wrote: K> > K> So I suspect our call stack is something like virtqueue_intr() -> K> > K> vtnet_rx_vq_intr() -> vtnet_rxq_eof() -> vtnet_rxq_input() -> K> > K> ether_input(). K> > K> I don’t see anything entering epoch in that path, which presumably K> > explains K> > K> the panic, but I still don’t understand why my bhyve current vm K> > doesn’t K> > K> panic in the same way. K> > K> > On bhyve we enter it through interrupt handler, and this is where we K> > enter the epoch. Does RISC-V has interrupt handling by the MI code K> > in sys/kern/kern_intr.c as other platforms? K> > K> It does, yes, but that was the hint I needed. I didn’t know that we entered K> net_epoch automagically based on the interrupt type. K> K> The difference between the two is that Bhyve uses virtio_pci, and in my K> qemu case we run through the virtio_mmio path. In that path we always set K> INTR_TYPE_MISC, so we never set INTR_TYPE_NET, even for if_vtnet, so we K> never entered epoch. K> K> This is the correct fix: K> K> diff --git a/sys/dev/virtio/mmio/virtio_mmio.c K> b/sys/dev/virtio/mmio/virtio_mmio.c K> index 95eb8647052..ccafe326868 100644 K> --- a/sys/dev/virtio/mmio/virtio_mmio.c K> +++ b/sys/dev/virtio/mmio/virtio_mmio.c K> @@ -196,7 +196,7 @@ vtmmio_setup_intr(device_t dev, enum intr_type type) K> return (ENXIO); K> } K> K> - if (bus_setup_intr(dev, sc->res[1], INTR_TYPE_MISC | INTR_MPSAFE, K> + if (bus_setup_intr(dev, sc->res[1], type | INTR_MPSAFE, K> NULL, vtmmio_vq_intr, sc, &sc->ih)) { K> device_printf(dev, "Can't setup the interrupt\n"); K> return (ENXIO); Thanks a lot for the fix, Kristof! -- Gleb Smirnoff