From owner-freebsd-net@freebsd.org Thu Nov 30 21:03:18 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B5075DBA779 for ; Thu, 30 Nov 2017 21:03:18 +0000 (UTC) (envelope-from aspam@cox.net) Received: from fed1rmwml114.cox.net (fed1wml22.cox.net [68.230.241.21]) by mx1.freebsd.org (Postfix) with ESMTP id 970C575948 for ; Thu, 30 Nov 2017 21:03:17 +0000 (UTC) (envelope-from aspam@cox.net) Received: from eastrmimpo210.cox.net ([68.230.241.225]) by eastrmfepo202.cox.net (InterMail vM.8.01.05.28 201-2260-151-171-20160122) with ESMTP id <20171130205315.HPPC3211.eastrmfepo202.cox.net@eastrmimpo210.cox.net> for ; Thu, 30 Nov 2017 15:53:15 -0500 Received: from thunder.sweets ([68.100.138.62]) by eastrmimpo210.cox.net with cox id gLtF1w0061LxgH801LtFY2; Thu, 30 Nov 2017 15:53:15 -0500 X-CT-Class: Clean X-CT-Score: 0.00 X-CT-RefID: str=0001.0A020205.5A206FBB.0090, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0 X-CT-Spam: 0 X-Authority-Analysis: v=2.2 cv=NN6lwwyg c=1 sm=1 tr=0 a=3mkzfl4ircflX6G+lDqBYw==:117 a=3mkzfl4ircflX6G+lDqBYw==:17 a=8nJEP1OIZ-IA:10 a=x7bEGLp0ZPQA:10 a=sC3jslCIGhcA:10 a=e9ASbk4n0QUA:10 a=iJhTRe1rzglwqlNLiJIA:9 a=wPNLvfGTeEIA:10 X-CM-Score: 0.00 Authentication-Results: cox.net; none Received: from [10.10.10.15] (thunder.sweets [10.10.10.15]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by thunder.sweets (Postfix) with ESMTP id 9F9CA10E40 for ; Thu, 30 Nov 2017 15:53:14 -0500 (EST) Message-ID: <5A206FB9.3010106@cox.net> Date: Thu, 30 Nov 2017 15:53:13 -0500 From: Joe Buehler User-Agent: Thunderbird 1.5.0.12 (X11/20120201) MIME-Version: 1.0 To: "freebsd-net@freebsd.org" Subject: netmap / LINUX realtime / ixgbevf: huge RX latencies Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Nov 2017 21:03:18 -0000 I am using the LINUX 4.4.86 realtime kernel patch with netmap and the ixgbevf driver (SRIOV in a VM) and having some serious RX latency issues. The ixbgevf driver built by the netmap build against the kernel source does not work. I haven't tried to debug it -- the netmap driver complains about bad ring index values and resets (over and over again as my app runs). The non-netmap version of ixgbevf that comes with the kernel works but appears to have serious RX latency issues due to frame batching. It appears that the kernel's NAPI polling reads frames from the NIC, then netmap processes the frames, then epoll exits and my app can work on the frames. The delay between start of NAPI to my app getting frames can be milliseconds!!! Would the netmap-modified ixgbevf driver eliminate this latency? If so, any ideas on what might be going wrong with the modified driver? Perhaps an older version of the driver might be better? The kernel ixgbevf version is 2.12.1-k while netmap is trying to use 4.2.1. Looking at the RT patch for LINUX there are only a few network driver patches, none for the ixgbe* cards. Joe Buehler