From owner-freebsd-net@freebsd.org Mon Jun 29 16:05:44 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3325898FAC8 for ; Mon, 29 Jun 2015 16:05:44 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mail-la0-x232.google.com (mail-la0-x232.google.com [IPv6:2a00:1450:4010:c03::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 95FA0198B for ; Mon, 29 Jun 2015 16:05:43 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: by lagc2 with SMTP id c2so15748351lag.3 for ; Mon, 29 Jun 2015 09:05:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=vklNA+UB93MQTSQbHa12G/heaWyioqWWYNi3oZlj9mo=; b=NfEmAT3fOiRmXZjgIYz3C/EWqpAuSngmndkm63ffFXxxUIJ3W1ux2daBAUrzraCw8m f46st35gQoglFAjfyIeOtVfz+a/X4z3B+RblnJvIK9vwIrz8vBnPK4028J61ji1TXHGl NlHY1+Ut5W+hv1/EdEx9dc8UFYXoCbTUqCZWv5ts8zTi4PFu2hnEL21Qyo/gS8I16GH4 t450bZB3NsLBXBAY71Jl1huKB8g56tNiEFYifkV6YM7T/vJ3byshE91Uiw/G5p5wzj1/ e+EUMoT1Ln4gmB0t8yplj8Pncyign5gvQZA6dybJfUF0JNGapWOdqVVG8ZEvawLU7+SS YqhA== MIME-Version: 1.0 X-Received: by 10.112.125.166 with SMTP id mr6mr14782151lbb.83.1435593941620; Mon, 29 Jun 2015 09:05:41 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.230.103 with HTTP; Mon, 29 Jun 2015 09:05:41 -0700 (PDT) In-Reply-To: <20150629151750.GD1647@zxy.spb.ru> References: <20150629151750.GD1647@zxy.spb.ru> Date: Mon, 29 Jun 2015 18:05:41 +0200 X-Google-Sender-Auth: aqE__GiLKHVNLQ3QKWPiSzwajBk Message-ID: Subject: Re: netmap custom RSS and custom packet info From: Luigi Rizzo To: Slawa Olhovchenkov Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jun 2015 16:05:44 -0000 On Mon, Jun 29, 2015 at 5:17 PM, Slawa Olhovchenkov wrote: > Working with netmap and modern hardware I am lacking some features: > > a) some spare space before packet (64/128/192/256 bytes) for > application data. For example: application do some pre-analysig > packet, filled structure in this space and routed packet (via NETMAP > pipe) to other thread. Received thread got packet and linked > inforamtion about this packet for processing w/o additional overhead. > =E2=80=8Bspare space in front of the packet is something we have been considering for a different purpose, namely better support for encapsulation/decapsulation and things like vhost-net header. =E2=80=8BNote though that the annotation is transferred for free only in the case of pipes or ports sharing the same memory region; vale ports would have to explicitly copy the extra=E2=80=8B bytes which is (moderately) expensive. A quick and dirty way to support what you want is the following: - in the kernel code, modify NMB(), PNMB() and the offset between the netmap_ring and the first buffer to add the extra space you want in front of the packet. You can possibly make this offset a sysctl-controlled value - in netmap_vale.c, make a small change to the code that copies buffers so that it includes also the space before the actual packet. That should be all. > b) custom RSS. Modern NIC have RSS poorly interoperable with packet > analysing: packets from same flow, but different direction placed in > different queue, PPPoE encapsulated packets placed in queue 0, > different tunneling don't recognised and etc. May be NETMAP can be > used custom RSS hashing from loadable kernel module, provideng by > user? Function frm this module can be packet analysing, tunnel > removing, custom RSS hashnig with direction-independly maner, filled > some structure prepended to buffer (see above) and pass this > information to application. > =E2=80=8BRSS is completely orthogonal to=E2=80=8B =E2=80=8B netmap and I strongly suggest to keep it this way, using either use the NIC-specific tools to control RSS or some generic mechanism (on linux there is ethtool, and we should implement something similar also on freebsd). =E2=80=8Bcheers luigi =E2=80=8B > This is possible? This is useful not only to me? > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > --=20 -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2217533 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+-------------------------------