Date: Sun, 14 Jun 2020 23:05:42 +0100 From: Jessica Clarke <jrtc27@freebsd.org> To: Tom Jones <thj@freebsd.org>, Vincenzo Maffione <vmaffione@freebsd.org> Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r361944 - in head/sys/dev/virtio: . network Message-ID: <0FD9E443-1B40-4495-B2C0-4803121EF911@freebsd.org> In-Reply-To: <20200614212230.GC68578@tom-desk.erg.abdn.ac.uk> References: <202006082151.058LpabU003001@repo.freebsd.org> <20200614195126.GB68578@tom-desk.erg.abdn.ac.uk> <97EEF019-16A4-4626-A484-A00979B52A74@freebsd.org> <20200614212230.GC68578@tom-desk.erg.abdn.ac.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
On 14 Jun 2020, at 22:22, Tom Jones <thj@freebsd.org> wrote: > On Sun, Jun 14, 2020 at 09:56:03PM +0100, Jessica Clarke wrote: >> On 14 Jun 2020, at 20:51, Tom Jones <thj@freebsd.org> wrote: >>> On Mon, Jun 08, 2020 at 09:51:36PM +0000, Jessica Clarke wrote: >>>> Author: jrtc27 >>>> Date: Mon Jun 8 21:51:36 2020 >>>> New Revision: 361944 >>>> URL: https://svnweb.freebsd.org/changeset/base/361944 >>>>=20 >>>> Log: >>>> virtio: Support non-legacy network device and queue >>>>=20 >>>> The non-legacy interface always defines num_buffers in the header, >>>> regardless of whether VIRTIO_NET_F_MRG_RXBUF, just leaving it = unused. We >>>> also need to ensure our virtqueue doesn't filter out = VIRTIO_F_VERSION_1 >>>> during negotiation, as it supports non-legacy transports just fine. = This >>>> fixes network packet transmission on TinyEMU. >>>>=20 >>>> Reviewed by: br, brooks (mentor), jhb (mentor) >>>> Approved by: br, brooks (mentor), jhb (mentor) >>>> Differential Revision: https://reviews.freebsd.org/D25132 >>>>=20 >>>> Modified: >>>> head/sys/dev/virtio/network/if_vtnet.c >>>> head/sys/dev/virtio/network/if_vtnetvar.h >>>> head/sys/dev/virtio/virtio.c >>>> head/sys/dev/virtio/virtqueue.c >>>>=20 >>>=20 >>> Hi Jessica, >>>=20 >>> After updating my current bhyve vm today (on a 12.1 host), = networking no longer >>> works. Reverting this commit seems to resolve the issue. I think = vtnet is not >>> passing enough data up to the ip layer. >>>=20 >>> If I capture on the tap interface for the vm I see arp requests and = arp >>> replies, however kern.msgbuf is full of:=20 >>>=20 >>> <5>arp: short packet received on vtnet0 >>>=20 >>> and netstat does not see any replies to arp requests: >>>=20 >>> root@freebsd-current:~ # netstat -s -p arp >>> arp: >>> 11 ARP requests sent >>> 0 ARP requests failed to sent >>> 0 ARP replies sent >>> 0 ARP requests received >>> 0 ARP replies received >>> 0 ARP packets received >>> 24 total packets dropped due to no ARP entry >>> 2 ARP entrys timed out >>> 0 Duplicate IPs seen >>>=20 >>> If I set up an arp entry manually I can see ICMP echo requests and = responses on >>> the tap interface, but the vm does not see the responses.=20 >>>=20 >>> root@freebsd-current:~ # netstat -s -p ip >>> ip: >>> 7 total packets received >>> 0 bad header checksums >>> 0 with size smaller than minimum >>> 7 with data size < data length >>> 0 with ip length > max ip packet size >>> 0 with header length < data size >>> 0 with data length < header length >>>=20 >>> The line >>>=20 >>> 7 with data size < data length >>>=20 >>> makes me think that vtnet is truncating packets.=20 >>>=20 >>> markj pointed me at this bug in irc which might also be related: >>>=20 >>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D247242 >>=20 >> Hi Tom, >> Sorry about that; it seems bhyve hits the "legacy and no MrgRxBuf" >> case. Could you please try the patch below? >>=20 >> Jess >>=20 >=20 > This changed fixed the issue for me. Please feel free to add=20 >=20 > Tested By: thj=20 >=20 > when you commit. Great, thanks for the report. > In testing I this lor went by, I wonder if this is something you care = about: >=20 > acquiring duplicate lock of same type: "vtnet0-rx0" > 1st vtnet0-rx0 @ = /usr/home/tj/code/freebsd/projects/review-D25220/sys/dev/virtio/network/if= _vtnet.c:1780 > 2nd vtnet0-rx0 @ = /usr/home/tj/code/freebsd/projects/review-D25220/sys/kern/subr_taskqueue.c= :281 > stack backtrace: > #0 0xffffffff80c32881 at witness_debugger+0x71 > #1 0xffffffff80ba3e54 at __mtx_lock_flags+0x94 > #2 0xffffffff80c24bd2 at taskqueue_enqueue+0x42 > #3 0xffffffff80a1af99 at vtnet_rxq_tq_intr+0xb9 > #4 0xffffffff80c2520a at taskqueue_run_locked+0xaa > #5 0xffffffff80c26284 at taskqueue_thread_loop+0x94 > #6 0xffffffff80b830e0 at fork_exit+0x80 > #7 0xffffffff81040eae at fork_trampoline+0xe Hm, I think that's just a false-positive, because if_vtnet constructs the taskqueue using the same name as its own internal mutexes. Though the locking around vtnet_rx_vq_intr and vtnet_rxq_tq_intr is a bit fishy given they're rather similar yet inconsistent. I would imagine rxq->vtnrx_stats.vrxs_rescheduled is supposed to be protected by that mutex, but wouldn't like to say whether taskqueue_enqueue needs to be. Vincenzo, you recently touched code around there, perhaps you could be persuaded to have a quick look?.. Jess
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?0FD9E443-1B40-4495-B2C0-4803121EF911>