From owner-freebsd-net@freebsd.org Thu Oct 26 15:12:25 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A3DF8E4BE86 for ; Thu, 26 Oct 2017 15:12:25 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mail-io0-x232.google.com (mail-io0-x232.google.com [IPv6:2607:f8b0:4001:c06::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6A4D081739 for ; Thu, 26 Oct 2017 15:12:25 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mail-io0-x232.google.com with SMTP id 189so6094015iow.10 for ; Thu, 26 Oct 2017 08:12:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=gQ2N++ierR6cM+Xp9MZgrx0zH3f6v1/Cq+MpFn1x/9Y=; b=t29JMZstmElM99xZu50R1uhwswxyNT7q1HwV1HefLN18Nl3TQdLcd/CX4g1iIp4t3h z17zRPCN3d2pN2kbvw1iwOYqSj1gqt4+RSb+QHQDvsS00ilAZGorDKQpc1QHJF8Leo/z a8UvRs/S/LySWGbGiPynjbkaiEh4q/q0srM9RHncrJyp3zQZaEt9EmoLBsd6gbU8LxK8 28ZE5usCoIclkwCOPQy+lyC0b6lO4x7M4tCZ/U2K778/oz0qwuWmChv5OPr94Gb2fSNs NolaCzs90L4WQR7cik4zb6j+Q7wxQj4Pky21gma8OnYFPI/tMQ9lKsOz10/02d5XlxeS PidA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=gQ2N++ierR6cM+Xp9MZgrx0zH3f6v1/Cq+MpFn1x/9Y=; b=nv+lEmmIg6WNKkk9bA8Vt0HB5MPXeZUmHc4nmpn25mJjwWgpFUH+Bm/AIkBJ/Gh6S4 vNGv0eR+QaoqzqU2iwUhwMUKX7Mqeyg+GbKvsGInXPEmVX7FnVbz3/8aFfx5g2Ex7rBD eWikN5HwiADi+okU22jH6Qo8v0NTqziO5lmmNu1OaJsu4FJ8OagX58ud5PleBk7hK0Kt ld0fR59bGa8wNfUXl6gENd2x9+9/YiFWKP9zz2pSLYqvBB0xTjPR02/IKdBzkE4CdQRe xNkNfcVHyg7vjf23GeAex1jWnFric5J0FnaAhJpj3nHmChSwIhPn3YUCl0KBiYEkZ/eV dd/A== X-Gm-Message-State: AMCzsaUWaJYStKQYqm8RCg0mX3SkP0lrcqEvjWsuurOQQsVAU3R3NdXm mWUhga5KavBDVylln+noqBCB+sdm/RSgbXJc1nw= X-Google-Smtp-Source: ABhQp+Rm62tS8kDhZnvW3pkZdaMDvYJY5MPel/qYK8VNx3CTcUisHdGODI9vl7mKudqgA3FncEXbDU68f+ztJqLEf0E= X-Received: by 10.107.59.18 with SMTP id i18mr28723286ioa.97.1509030744664; Thu, 26 Oct 2017 08:12:24 -0700 (PDT) MIME-Version: 1.0 Received: by 10.2.176.143 with HTTP; Thu, 26 Oct 2017 08:12:24 -0700 (PDT) In-Reply-To: <59F1F1BA.3020608@cox.net> References: <59F0FBEE.6030008@cox.net> <59F1F1BA.3020608@cox.net> From: Vincenzo Maffione Date: Thu, 26 Oct 2017 17:12:24 +0200 Message-ID: Subject: Re: virtio_net / netmap RX dropping frames To: Joe Buehler Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Oct 2017 15:12:25 -0000 So you are using netmap only in the guest (and not in the host). And you are running a sender and a receiver inside the VM, both on the VM interface. Something like this # pkt-gen -i eth1 -f rx # pkt-gen -i eth1 -f tx ? What happens if you use pkt-gen rather than your application? 2017-10-26 16:31 GMT+02:00 Joe Buehler : > Vincenzo Maffione wrote: > > I guess you are using a FreeBSD guest. Is this the case? If you have the > > Sorry, I am using LINUX, ubuntu 16.04 LTS for both host and VM. I am > posting here at standing request of netmap driver author. > > The host has 24 CPUs @ 2.5 GHz and 128G of memory and is *idle* so I am > a bit disappointed > > > chance, try a linux guest to check if virtio-net works better there > > (I've used netmap on the netmap-patched virtio-net in Linux guests, > > never tried on FreeBSD). > > The netmap ring size is just the NIC ring size. If you change the > > virtio-net NIC ring size (sysctl on FreeBSD, I guess). > > OK I'll look into that. I increased the ring size on the host ixgbe but > that had no effect so I guess it must be virtio_net. > > > Anyway, for your specific use-case (VM accessing the physical 10G NIC) > > there is a way better solution, which is the netmap passthrough. > > Unfortunately I don't have control of the host, just the VM, so pt > netmap is not an option. > > My initial query regarded frame drops but the latency is also pretty > bad. The LINUX ping utility inside the VM says 0.2 mS consistently > without netmap in use. My app sees that value for almost all frames but > it spikes (up to 0.6 mS!) for a few frames, which is not acceptable for > this application -- was expecting much better due to network stack > bypass. And this is just 100 frames/sec... > > Joe Buehler > -- Vincenzo Maffione