From owner-freebsd-net@freebsd.org Thu Jun 16 03:51:23 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5A168A47B8D for ; Thu, 16 Jun 2016 03:51:23 +0000 (UTC) (envelope-from jim@netgate.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 3D96D18B5 for ; Thu, 16 Jun 2016 03:51:23 +0000 (UTC) (envelope-from jim@netgate.com) Received: by mailman.ysv.freebsd.org (Postfix) id 3CE9AA47B8C; Thu, 16 Jun 2016 03:51:23 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3C980A47B8B for ; Thu, 16 Jun 2016 03:51:23 +0000 (UTC) (envelope-from jim@netgate.com) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 19CE618B3 for ; Thu, 16 Jun 2016 03:51:23 +0000 (UTC) (envelope-from jim@netgate.com) Received: by mail-it0-x236.google.com with SMTP id e5so37463103ith.0 for ; Wed, 15 Jun 2016 20:51:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netgate.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=mNsyQ47o6jOd63/ioIGUb5k19w/rM5QZd22PNcbPY8E=; b=AVCdmlJrL4RypYU/Ddl6KLzr7mWRDmfpQxDo9BC3gZDlxSswqvREYiwtQWG0agGnsu ctmZN2ubBIOCx+HiGNFYuSNJQa3LqDW2AWHuYzTNlKA5c+gWdiZdQsfY1aD3DsljvuC8 22fEsnUBgi56wxEWn3XNcJ5qbggx5gJ8yiIFs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=mNsyQ47o6jOd63/ioIGUb5k19w/rM5QZd22PNcbPY8E=; b=VjMV2XrRy1fkLGKg8o4gWmK2d4HnSxYS459TupIX+dnUT1yRTSs/W08yZZFA+bUbQh B/5gLMYAB1efSmJOX1un7ueTiKQCWnSNHhGIvNKIUcZFUov2tYHu78YALzLLJLxLKpkJ IrexWdnnXYf9enAD9Jtc2nFoz4wG3xSHaXnOw+vSZPtHSGZ2S/J94/muAQRbj1evdkIU 38Gg1n94+4kQ931wTZ4IIlH7gmXvjMgY4vO2tDIiyq3Bn3uUhJPrqjDhNKFQu3x6L/+8 DnJ2sZ/IpcGKiLWknf/i/1YGnvyjKztyhR/EtTLYHDPN62CQFT0xE6p6U/OeT2y6fGaV 0eGQ== X-Gm-Message-State: ALyK8tIXZYWnks+aayCl4yDkDV068lzzqED/P65ZeYLSi1J+33PT7lOflAfdHXmH1GMkDYupQ/9D+UzjbklOnDWW X-Received: by 10.36.2.2 with SMTP id 2mr4552246itu.34.1466049082265; Wed, 15 Jun 2016 20:51:22 -0700 (PDT) MIME-Version: 1.0 Received: by 10.50.156.167 with HTTP; Wed, 15 Jun 2016 20:51:21 -0700 (PDT) In-Reply-To: <57601DFA.2040008@in.tum.de> References: <57601DFA.2040008@in.tum.de> From: Jim Thompson Date: Wed, 15 Jun 2016 22:51:21 -0500 Message-ID: Subject: Re: Netmap Checksum Offloading To: Dominik Schoeffmann Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Jun 2016 03:51:23 -0000 We've focused on just the IP header checksum, but it's possible to add L4 checksum offload as well. I asked Luigi why he hadn't included checksum offload (with a library in software for devices that don't offer a hw offload), and his answer was that when he wrote netmap, he wanted a fast path to the wire. On Tue, Jun 14, 2016 at 10:08 AM, Dominik Schoeffmann wrote: > Dear Netmap Developers, > > during the course of my bachelor's thesis, I modified a packet generator > called MoonGen [1] in order to utilize netmap. > One key component was to flexibly offload checksums for different kinds > of packets (IPv4, UDP, TCP). > The ixgbe netmap patch was modified [2] in order to construct context > descriptors and suitable data descriptors. This is implemented in less > than 250 LoC (including pseudo-header calculations). > The man page states, that checksum offloading is available via ethtool, > although a solution inside the netmap API might be a cleaner way for > applications to actually use these features. > Attached is a graph showing the performance implication of using > offloading in the current implementation. > As can be seen, offloading has only a minor impact. > When regarding this data (and comparing it to other frameworks), please > keep in mind, that internally a lot of per-packet effort is needed due > to the software architecture of the packet generator. > > The question being: > Would it not make sense to include checksum offloading inside of netmap > in order to accomodate applications operating on layer 3 and above? > As these programs need to calculate the checksums in software, it would > be just as fast to move these calculations to the kernel for NICs > without checksum offloading support (and the kernel would act as a library). > The problem which currently is imposed by the fact, that netmap exports > the complete ring, is that context descriptors disrupt the data > descriptors, which is unpleasant for the application. > But you may find the data interesting nevertheless. > > > Best Regards, > Dominik Schoeffmann > > > > [1] https://github.com/dschoeffm/MoonGen/tree/netmap > [2] https://github.com/dschoeffm/netmap/tree/mg-chksum-offloading >