From owner-freebsd-net@FreeBSD.ORG Fri Jan 23 10:53:28 2015 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5CF8CCB9 for ; Fri, 23 Jan 2015 10:53:28 +0000 (UTC) Received: from mail-we0-f179.google.com (mail-we0-f179.google.com [74.125.82.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ED06314D for ; Fri, 23 Jan 2015 10:53:27 +0000 (UTC) Received: by mail-we0-f179.google.com with SMTP id q59so6843172wes.10 for ; Fri, 23 Jan 2015 02:53:20 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=G2qcTnMn9qlZz6GwYbUX4yooocS78za+qXQgOKjEzkw=; b=iBVxSvknyWerH+smqsqAiWfqh5DP0HKhGH3qqqalbl+YmYxTE0SNtYUGuuy6BcWVxb gyGLEFOD5ItY/+cgMBSIah6YgYcIJshnNV5GXNWQEDDneSPYi+gv3pD2sALnJDtEgh2o 1dwc/ZRoaLuYObOXT7WqJxJqswh5zx+04SrCazCt5/+rBXyAcj+aqKJoZ55tFX8Pxuji ZA8bVzbMk091p+UquxYCpypJGXJd2HeNEMrtNP7Ze/fV3mWQ2/6RZOOcSC9I6Bvi0b1f nreY6tqoqpFnA+aOIgqUNMgruhYJCs22wUm4WBSYaqDbMDgKg4TKLSjEmZCurkBtukyi 4qLw== X-Gm-Message-State: ALoCoQll5Eh/P2q2SPCe2bCJ105MAN7q+Ml+ZL4dG7VWY7OCVYvZ234j1BN+IlA2PpObRK4rBZOw MIME-Version: 1.0 X-Received: by 10.181.12.112 with SMTP id ep16mr2603314wid.38.1422010400533; Fri, 23 Jan 2015 02:53:20 -0800 (PST) Received: by 10.180.97.161 with HTTP; Fri, 23 Jan 2015 02:53:20 -0800 (PST) In-Reply-To: References: Date: Fri, 23 Jan 2015 12:53:20 +0200 Message-ID: Subject: Re: Netmap mempool From: Ciprian Barbu To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jan 2015 10:53:28 -0000 On Thu, Jan 22, 2015 at 9:03 PM, Luigi Rizzo wrote: > On Thu, Jan 22, 2015 at 10:12 AM, Ciprian Barbu > wrote: >> Hello, >> >> I have some questions related to OpenDataPlane. I have implemented the >> I/O access using netmap for ODP but currently it's suffering from low >> performance due to packets being copied between the netmap slots and >> ODP buffers. >> >> I want to create ODP buffer pools on top of netmap memory but I'm >> concerned with three things: >> 1. how to keep a buffer for as long as needed without affecting the >> receive process; I guess I could use the NS_BUF_CHANGED flag in some >> way >> 2. the netmap memory may not be large enough to accommodate a complex >> ODP application >> 3. the ODP application may occasionally hold on to buffers for longer time >> > > there is a way to hold buffers without returning them to the ring > using the 'cur' pointer in the ring, see the netmap manpage how this > is implemented and how it works on the tx and rx path. > > However, before embarking in zero copy i'd like you to elaborate on what > you mean for "low performance" (how many pps you see) and what are > your expectations. In my experience low performance almost always comes > from insufficient batching or other application bottlenecks. I didn't do exact measurements, but there seemed to be a bottleneck somewhere, I will have another look, maybe I missed something. > > Zero copy tricks come with great complications to track who owns what, > sometimes with additional hidden overhead (e.g. if you change buffers the driver > has to reprogram the IOMMU) and often have a very limited effect > in real world applications. > > Sure the numbers do look good on benchmarks or academic papers, but there > people don't worry about robustness and code maintainability. > > cheers > luigi