From owner-freebsd-hackers@FreeBSD.ORG Thu Aug 28 21:28:15 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 173AC966 for ; Thu, 28 Aug 2014 21:28:15 +0000 (UTC) Received: from mail.choopa.net (mail.choopa.net [216.155.136.52]) by mx1.freebsd.org (Postfix) with ESMTP id C922E1350 for ; Thu, 28 Aug 2014 21:28:14 +0000 (UTC) Received: from localhost (unknown [127.0.0.1]) by mail.choopa.net (iRedMail) with ESMTP id 79EA44AF41B; Thu, 28 Aug 2014 17:28:13 -0400 (EDT) X-Virus-Scanned: amavisd-new at mail.choopa.net Received: from mail.choopa.net ([127.0.0.1]) by localhost (mail.choopa.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3izn5OJ8p4gD; Thu, 28 Aug 2014 17:28:07 -0400 (EDT) Received: from [10.5.5.117] (office-nat.choopa.net [208.167.225.40]) by mail.choopa.net (iRedMail) with ESMTPSA id C6CBB4AF3F6; Thu, 28 Aug 2014 17:28:07 -0400 (EDT) Message-ID: <53FF9EE7.8070501@gameservers.com> Date: Thu, 28 Aug 2014 17:28:07 -0400 From: Brian Rak User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Bryan Venteicher Subject: Re: Bug in FreeBSD VirtIO network driver (only with pf enabled) References: <53FE2B37.7050309@gameservers.com> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Aug 2014 21:28:15 -0000 Actually, is this related to https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192013 ? That looks awfully similar to what I imagine the fix for this issue would look like. On 8/28/2014 12:35 AM, Bryan Venteicher wrote: > > > > On Wed, Aug 27, 2014 at 2:02 PM, Brian Rak > wrote: > > I have a FreeBSD 10 x64 guest installed inside a KVM instance on > Linux. When pf is active, and the server is sending data it > causes the Linux host to report warnings related to GSO. > > I've talked to some Linux developers, and they believe it to be a > bug inside the FreeBSD VirtIO drivers. Based on what I'm seeing, > I'm inclined to agree with them. > > Reproduction is mildly annoying, you need: > 1) A KVM guest setup with bridged networking, with FreeBSD running > side using the VirtIO network interface. (We tested with CentOS, > but the exact distribution should not matter) > 2) pf enabled (I'm using a single rule: "scrub in all") > 3) Some method of sending a bit of traffic from the guest (I use > netcat) > > So, when I do this: > > # service pf start > # cat /root/test | nc vultr.com 80 > > The Linux kernel on the host will report: > > kernel: WARNING: CPU: 7 PID: 7772 at net/core/dev.c:2246 > skb_warn_bad_offload+0xc3/0xd0() > kernel: igb: caps=(0x0000000640114bb3, 0x0000000000000000) > len=1498 data_len=0 gso_size=1380 gso_type=5 ip_summed=0 > > If I do: > > # service pf stop > # cat /root/test | nc vultr.com 80 > > No such warning is reported. I can only reproduce this with pf > enabled. The contents of the /root/test don't matter, I'm using > 4k of data from /dev/urandom. The test file just needs to be > bigger then the MTU of the host's network interface. > > I was able to track this down to the virtio_net_hdr being sent by > the FreeBSD guest. With pf enabled, this outbound traffic has the > following header: > > flags = 0 > gso_type = VIRTIO_NET_HDR_GSO_TCPV4 > hdr_len = 66 > gso_size = 1440 > csum_start = 0 > csum_offset = 0 > > But, this is not a valid configuration. With > VIRTIO_NET_HDR_GSO_TCPV4 enabled, you should also be setting > VIRTIO_NET_HDR_F_NEEDS_CSUM and populating the csum_start and > csum_offset fields. > > > > ​It would appear that somewhere in pf the CSUM_TCP flag is getting > clear for CSUM_TSO packets. I don't believe the stack otherwise sets > CSUM_TSO without CSUM_TCP. Now, it is probably safe to assume CSUM_TCP > when CSUM_TSO is set: the mbuf's m_pkthdr.csum_data better be valid in > such cases though. > > http://www.spinics.net/lists/netdev/msg293976.html gives more > detail on this. I don't fully understand this, so I'd probably > mangle the explanation if I tried to give more detail. I can > reproduce this at will now, but fixing it is beyond my abilities. > Is there a better place to report this? I'm not entirely sure who > is responsible for maintaining the virtio driver. > _______________________________________________ > freebsd-hackers@freebsd.org > mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org > " > >