From owner-svn-src-head@freebsd.org Tue Jan 19 20:47:19 2016 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5B440A89A1D; Tue, 19 Jan 2016 20:47:19 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-io0-x234.google.com (mail-io0-x234.google.com [IPv6:2607:f8b0:4001:c06::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 242D71D35; Tue, 19 Jan 2016 20:47:19 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-io0-x234.google.com with SMTP id 1so547274314ion.1; Tue, 19 Jan 2016 12:47:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=EJZ9pUBiXrXBOt2U5zOnxK8Ig5fnsmd+e2TETYSjjDk=; b=ykstaB9sbSfVfxQKC6m1WjM8GXy2Z82k2Mb8caCj+DdDFNVvPVarMOSmfUDAhAFh/n 2eHidv6ACoXheYuK4RMzi1jbfi/8MBlUpwYkMjZb+aFaV1dVNbN9GM4RmFaRdZ694dxH DfNYReMKKcjRremIxbP9T86WD0/YpgZeNVtIEeDighs5yedO9qZ1ou2GOLNGOkkHGuDO Q+FUZDsCdoHNVqNWbHregc1bTSZp001Q6T2rpuBmxHKqXTTHsaAHhCmKZSxU+z+979k5 h/XqPKYwAJPI1j7mGzC75GJ+RQB0HBSarwzp3+W/wVKN2tKVWz5UytTrcsKN012q6Wwj z15g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=EJZ9pUBiXrXBOt2U5zOnxK8Ig5fnsmd+e2TETYSjjDk=; b=di0UGucGx7WMOx3n55LUWNYuCNHlYgtJlzUBuSqMQu0MylG97p1pWKLH60L5vBgOaK m26nzouZcBlXvvsb3ql/uMS8XuPb/n2kqs/JzIT6dfMW5cVRpmxZjWYJWVoW9kszIN2y 9w/NmxGK44BlAcgZMxJfOH9Sd/etCzG4dMlgQ1KKYQFps6/R4s11L9rW9t0522/wSG9u hk84kn2E11nhpSuhuFr8pSjahpOGYq9twTf8RbJvTMiKbQeXQFBQtnCbdkFfSEimfTWX tybdXKoScowN9M/Rj3xgZkVDEc9bL0++cC3sVwJj4hVGbhBZu8tfDdjrhGEjM9bjJO1q l+tA== X-Gm-Message-State: ALoCoQl4f5hqqkXzXRYE7LdgmLQAkKuRM/i/xasKCU5ol1b/d5iUjDKcdfjq/CQMy7rCRkozsBnXxAmKYOov480guzefCm2KLQ== MIME-Version: 1.0 X-Received: by 10.107.11.162 with SMTP id 34mr27078906iol.165.1453236438603; Tue, 19 Jan 2016 12:47:18 -0800 (PST) Received: by 10.36.121.16 with HTTP; Tue, 19 Jan 2016 12:47:18 -0800 (PST) In-Reply-To: <569EA026.5020906@selasky.org> References: <201601191533.u0JFXSxf037804@repo.freebsd.org> <569E6A38.8080108@selasky.org> <569E909B.60506@selasky.org> <569E9B66.1070200@selasky.org> <569EA026.5020906@selasky.org> Date: Tue, 19 Jan 2016 12:47:18 -0800 Message-ID: Subject: Re: svn commit: r294327 - in head/sys: dev/cxgb dev/cxgbe dev/e1000 dev/hyperv/netvsc dev/ixgbe dev/mxge netinet sys From: Adrian Chadd To: Hans Petter Selasky Cc: Ryan Stone , "src-committers@freebsd.org" , "svn-src-all@freebsd.org" , "svn-src-head@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Jan 2016 20:47:19 -0000 On 19 January 2016 at 12:44, Hans Petter Selasky wrote: > On 01/19/16 21:23, Adrian Chadd wrote: >> >> On 19 January 2016 at 12:24, Hans Petter Selasky wrote: >>> >>> On 01/19/16 21:05, Adrian Chadd wrote: >>>> >>>> >>>> Erm, ok. So I'm confused about the state of how the streams are >>>> tracked. So not all mbufs are in a stream, but they're in some single >>>> LRO mbuf list? >>>> >>> >>> Hi, >>> >>> The streams are typically tracked using the hardware generated Toeplitz >>> hash >>> value. The mbufs are sorted according to the hash value and the hash >>> type. >>> The list of mbufs is scanned, flushing the LRO entries every time we see >>> a >>> change in the hash value or hash type. OK? >> >> > > Hi, > >> Sure, but TCP fragments and non-fragments can hash to different >> values, so suddenly you get interleaved packets. > > > Fragmented TCP packets should not be subject to LRO. Depending on the NIC we > will get the same hash value or not. I think that's fine. If you want to use > this feature the NIC should not hash the TCP port numbers when it sees a > fragmented packet including the starting fragment. That will give the best > result. > > What does the RSS code expect currently? The NIC doesn't mis-tag something, so it knows to rehash things in the netisr input path. And yes, I've seen streams with both fragments and non-fragments at the same time. Then you end up with some packets with different hash types. The IP input path will reassemble fragments, and then you'll rehash the packet with the correct L3 or L4 calculation as appropriate. In this setup you may have the fragments show up at a different hash value to the non-fragments, so you'll handle all the non-fragments first, then the fragments show up later. So hopefully the LRO code handles seeing that hole in the TCP stream and will eject the whole stream up. >> You can't rely on the toeplitz hash 100% hashing /all packets in a >> stream/ reliably, because it's highly dependent upon the NIC config. > > >> This is why I did all that effort in the RSS code to handle things. > > --HPS > >