From owner-freebsd-net@freebsd.org Thu Oct 26 18:05:07 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CC367E4FFB1 for ; Thu, 26 Oct 2017 18:05:07 +0000 (UTC) (envelope-from aspam@cox.net) Received: from fed1rmwml205.cox.net (fed1rmwml205.cox.net [68.230.241.179]) by mx1.freebsd.org (Postfix) with ESMTP id A7C8663614 for ; Thu, 26 Oct 2017 18:05:07 +0000 (UTC) (envelope-from aspam@cox.net) Received: from eastrmimpo209.cox.net ([68.230.241.224]) by eastrmfepo101.cox.net (InterMail vM.8.01.05.28 201-2260-151-171-20160122) with ESMTP id <20171026180452.FOFI4075.eastrmfepo101.cox.net@eastrmimpo209.cox.net> for ; Thu, 26 Oct 2017 14:04:52 -0400 Received: from thunder.sweets ([68.100.138.62]) by eastrmimpo209.cox.net with cox id SJ4s1w00F1LxgH801J4s3d; Thu, 26 Oct 2017 14:04:52 -0400 X-CT-Class: Clean X-CT-Score: 0.00 X-CT-RefID: str=0001.0A020205.59F223C4.010D, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0 X-CT-Spam: 0 X-Authority-Analysis: v=2.2 cv=G/psK5s5 c=1 sm=1 tr=0 a=3mkzfl4ircflX6G+lDqBYw==:117 a=3mkzfl4ircflX6G+lDqBYw==:17 a=IkcTkHD0fZMA:10 a=x7bEGLp0ZPQA:10 a=02M-m0pO-4AA:10 a=e9ASbk4n0QUA:10 a=YuaIYVqPCbt3uL0FN-4A:9 a=QEXdDO2ut3YA:10 X-CM-Score: 0.00 Authentication-Results: cox.net; none Received: from [10.10.10.15] (thunder.sweets [10.10.10.15]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by thunder.sweets (Postfix) with ESMTP id D03C5113FC; Thu, 26 Oct 2017 14:04:51 -0400 (EDT) Message-ID: <59F223C3.5040907@cox.net> Date: Thu, 26 Oct 2017 14:04:51 -0400 From: Joe Buehler User-Agent: Thunderbird 1.5.0.12 (X11/20120201) MIME-Version: 1.0 To: Vincenzo Maffione CC: FreeBSD Net Subject: Re: virtio_net / netmap RX dropping frames References: <59F0FBEE.6030008@cox.net> <59F1F1BA.3020608@cox.net> <59F214F6.9060905@cox.net> In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Oct 2017 18:05:07 -0000 Vincenzo Maffione wrote: > You can have how many threads and processes you want. The constraint is > that there must not be two threads accessing the same ring at the same > time. In this case each pktgen is accessing different rings. Thanks that was very useful info. I had run this before and got large frame drop so assumed it was a violation of threading constraints. So now I can remove a mutex from my app, which has an RX and TX thread. Running the two pkt-gen instances, I am getting a lot of RX frame drop. Based on counters, the TX frames are making it to the external loopback device, about 1 million frames/sec, which is looping them all back, but the macvtap interface on the host that feeds into the ixgbe/vhost/virtio_net/netmap interface in the VM shows about 80-90% of them as dropped. CPU usage is low in the VM, very roughly 25% for the TX thread and 5% for the RX thread. The frame rate displayed by pkt-gen and the CPU displayed by top is bouncing around. Joe Buehler