From owner-freebsd-current@FreeBSD.ORG Tue Nov 11 21:20:19 2014 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E01526E1 for ; Tue, 11 Nov 2014 21:20:19 +0000 (UTC) Received: from host64.kissl.de (host64.kissl.de [213.239.241.64]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.shmhost.net", Issuer "COMODO RSA Domain Validation Secure Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A00FEE6A for ; Tue, 11 Nov 2014 21:20:19 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by host64.kissl.de (Postfix) with ESMTP id F1A42B010B7; Tue, 11 Nov 2014 22:13:56 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at host64.kissl.de Received: from host64.kissl.de ([127.0.0.1]) by localhost (host64.kissl.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id j9tZTeckNu2L; Tue, 11 Nov 2014 22:13:56 +0100 (CET) Received: from [192.168.0.11] (unknown [95.91.254.245]) (Authenticated sender: web104p1) by host64.kissl.de (Postfix) with ESMTPSA id B3A7EB010B6; Tue, 11 Nov 2014 22:13:56 +0100 (CET) From: Franco Fichtner Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: netmap: extension to store user data per packet/slot? Date: Tue, 11 Nov 2014 22:13:54 +0100 Message-Id: <560E07EA-2506-487E-88DD-E2B9F7ED9792@lastsummer.de> To: Luigi Rizzo Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Nov 2014 21:20:20 -0000 Hi Luigi, hi all, so I was running into logistics issues with netmap(4) with regard to zero-copy and redirection through pipes: working on a load-balancing framework revealed that it is very hard to track a packet's origins to later move it onward to the respective outgoing interface, be it another device or the host stack. Long story short: user data needs to be stored for the packet buffer or slot. There are three ways that I can see so far: (1) Allocate a netmap pipe pair for each interface, in case of transparent mode also a pipe for the host stack each. That's a lot of pipes and most likely insane, but it won't extend the ABI. (2) Store the additional data in the actual buffer. That is sort of ok, but seems sluggish WRT cache behaviour -- maybe the buffer won't be read but it needs to be written. Sure, we can store it at the end, but there already resides the packet timestamp if enabled (if I recall correctly). Wouldn't extend the ABI per se, but might collide with the timestamping.... (3) Make room in struct netmap_slot itself like this: diff --git a/sys/net/netmap.h b/sys/net/netmap.h index 15ebf73..d0a9c0e 100644 --- a/sys/net/netmap.h +++ b/sys/net/netmap.h @@ -147,6 +147,7 @@ struct netmap_slot { uint16_t len; /* length for this slot */ uint16_t flags; /* buf changed, etc. */ uint64_t ptr; /* pointer for indirect buffers */ + uint64_t userdata; /* reserved storage for caller */ }; It could also be broken down in two fields with uint32_t each; not sure what would be more sensible. This of course requires an API bump, although it should be backwards compatible. Any feedback on this is highly appreciated. Cheers, Franco