From owner-freebsd-arch@FreeBSD.ORG Tue Jul 15 14:38:55 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5D84684A for ; Tue, 15 Jul 2014 14:38:55 +0000 (UTC) Received: from mail-qa0-x22b.google.com (mail-qa0-x22b.google.com [IPv6:2607:f8b0:400d:c00::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1C5292CAC for ; Tue, 15 Jul 2014 14:38:55 +0000 (UTC) Received: by mail-qa0-f43.google.com with SMTP id w8so4563868qac.2 for ; Tue, 15 Jul 2014 07:38:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=u16qXV0oIWEQzZjGLvJop6jlbyvwEHiOMxh/RMZSOzc=; b=pMTLtZtZfXsW9gpDMxVDiA95UNcscQ3e4ypxLyLt67WvYA1dhKCHYBKsGatPnV4W+E QnxuM13Woq/yEtz/lpCdnczEtqOsvqfBWDn2OBO3P85z3ICs8annN8/jQMuUJ9JOaqoO TFEA3nCT8vb4vCe364uYDZv90zZCXneNCPGefbQ22TVVx0u9Qe33T9rVhyW9OGZh3Mpp O3o/J4m2pK0IlNzm85LIb58rOUCglIOWznX4tnVM3jSXZzNfBkNaCdwnR2rAtyy4nHY0 VkPZrBnF/kPVNBECuiI/3cWFyyyy2hpJvZVEclAj1PHWKsIwD5ojMhFiAJpnkghLyeSa xCQw== MIME-Version: 1.0 X-Received: by 10.140.80.19 with SMTP id b19mr34103547qgd.102.1405435134351; Tue, 15 Jul 2014 07:38:54 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Tue, 15 Jul 2014 07:38:54 -0700 (PDT) In-Reply-To: <20140715143635.GA5178@zxy.spb.ru> References: <53C4EE00.5090705@gmail.com> <20140715093125.GA89128@zxy.spb.ru> <53C507F2.2090604@gmail.com> <20140715113319.GA96254@zxy.spb.ru> <20140715143635.GA5178@zxy.spb.ru> Date: Tue, 15 Jul 2014 07:38:54 -0700 X-Google-Sender-Auth: QOx5XpbUkrD-V37QXDsWZXLbtA8 Message-ID: Subject: Re: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing From: Adrian Chadd To: Slawa Olhovchenkov Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 14:38:55 -0000 On 15 July 2014 07:36, Slawa Olhovchenkov wrote: > On Tue, Jul 15, 2014 at 07:33:57AM -0700, Adrian Chadd wrote: > >> On 15 July 2014 04:33, Slawa Olhovchenkov wrote: >> > On Tue, Jul 15, 2014 at 03:22:34PM +0430, Hooman Fazaeli wrote: >> > >> >> On 7/15/2014 2:01 PM, Slawa Olhovchenkov wrote: >> >> > On Tue, Jul 15, 2014 at 01:31:52PM +0430, Hooman Fazaeli wrote: >> >> > >> >> >> Doesn't the problem applies to TCP too? >> >> >> TCP may be fragmented too but is less likely because of MSS. >> >> > Don't forget GRE, IPIP, ESP and AH! >> >> > >> >> These protocols don't use port numbers and the RSS hash is computed based on the >> >> (srcip,dstip) tuple, so the problem does not apply to them. >> > >> > And all flows go to one queue? Bad. >> >> All the packets between two IP addresses for non-TCP, non-UDP get >> hashed to the same CPU core, yes. >> >> If you have 1000 IP tunnels to different end-hosts, they'll be on >> different CPUs. > > What about two host, 10G link and 1000 TCP connections over IPsec? If the IP data is encrypted, including the TCP header, then yes, receive occurs on one CPU for now. Transmit occurs from multiple CPUs but it'll likely be serialised through one transmit queue. If the TCP data is encrypted, but the TCP header is in clear-text and it's a normal TCP frame, then hardware will hash it across multiple CPUs. -a