From owner-freebsd-current@FreeBSD.ORG Wed Nov 12 09:50:26 2014 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8BD52910; Wed, 12 Nov 2014 09:50:26 +0000 (UTC) Received: from host64.kissl.de (host64.kissl.de [213.239.241.64]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.shmhost.net", Issuer "COMODO RSA Domain Validation Secure Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 452C17DC; Wed, 12 Nov 2014 09:50:26 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by host64.kissl.de (Postfix) with ESMTP id 85C2BB02DCA; Wed, 12 Nov 2014 10:50:23 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at host64.kissl.de Received: from host64.kissl.de ([127.0.0.1]) by localhost (host64.kissl.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ntT3B2uGblyQ; Wed, 12 Nov 2014 10:50:23 +0100 (CET) Received: from francos-mbp.hq.packetwerk.com (p50993595.dip0.t-ipconnect.de [80.153.53.149]) (Authenticated sender: web104p1) by host64.kissl.de (Postfix) with ESMTPSA id 26E4BB02DC9; Wed, 12 Nov 2014 10:50:23 +0100 (CET) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: netmap: extension to store user data per packet/slot? From: Franco Fichtner In-Reply-To: Date: Wed, 12 Nov 2014 10:50:21 +0100 Content-Transfer-Encoding: 7bit Message-Id: <649AF052-F51A-48A7-BAA9-F85B36DD0C12@lastsummer.de> References: <560E07EA-2506-487E-88DD-E2B9F7ED9792@lastsummer.de> To: Luigi Rizzo X-Mailer: Apple Mail (2.1878.6) Cc: Adrian Chadd , freebsd-current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Nov 2014 09:50:26 -0000 Hi Luigi, On 12 Nov 2014, at 00:00, Luigi Rizzo wrote: > apparently you want some user-defined metadata to move > along with the packet, but i do not think it is > reasonable to put it in the slots. > If we do that, what about timestamps, flow IDs, > interface and queue index and all the rest of the things that > we normally find in an mbuf/skbuf ? This is not > going to scale. that's true. I'm only suggesting a small extension to be used freely, but would never consider increasing the slot size beyond 32 bytes in any case. Keeping it sleek is obviously important. > Also consider that at some point you may use a different > arrangement (with packets passed along VALE switches > or physical interfaces etc.) i believe the most > reasonable place to put the extra info is at the end > of the packet and possibly bump the length in the slot > so you are safe in case the packet is copied. I dont believe dirtying the cache lines in the actual packet buffer is a wise choice, but it certainly works. > There is no timestamp appended to the packet at the moment, > it was a feature i thought somebody may want to have, > but between the relative scarcity of hardware that provides > per-packet timestamps, and the questionable usefulness > of the same, i doubt it will be available. It is a useful feature to have receive timestamps per packet for better accounting, but I can see it being too mystical in its current form inside the packet buffer. It's still in my TODO list to investigate the impact, but a system certainly works without that extra bit of resolution. For now, I'll go with Adrians suggestion and keep track of the buffer index inside the first process away from netmap(4) itself. This setup breaks for non-circular pipe arrangements, but the load-balancing use case at hand is alright. Cheers, Franco