From owner-freebsd-net@FreeBSD.ORG  Tue Mar  4 01:17:09 2014
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id CCA3FD93;
 Tue,  4 Mar 2014 01:17:09 +0000 (UTC)
Received: from mail-qa0-x234.google.com (mail-qa0-x234.google.com
 [IPv6:2607:f8b0:400d:c00::234])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id 790E6B36;
 Tue,  4 Mar 2014 01:17:09 +0000 (UTC)
Received: by mail-qa0-f52.google.com with SMTP id m5so4256646qaj.39
 for <multiple recipients>; Mon, 03 Mar 2014 17:17:08 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:date:message-id:subject:from:to:content-type;
 bh=/BSvHUXI/1mzcKfA02fxoe/7E4W8rPdRl519zAHMYvI=;
 b=VILdh2p3V1Tnetbgmee2b/7R2ceKK5LeZBFUsAYfaaTFB8Vb0r4bokQgMsc1XenZ0L
 SfAR+gqP+hk19O6awjdsAG3R9oLN4kUEMaShq4xYMEtStnn5sMEk9GmzQbqmkO0L2PoU
 DUFF4q2R0yXZp+j4zCvrTyY7V+8+LkFdR8RLrS+z0v/gUJcOtl3MLlJD0qiwFbf/bXbl
 PqaFLFMoTZXRJKW7uL34nxJIOhJ7CLkL25SHALGvrJRX3OQ6II2nnPdoKqVsTBqLt5Rt
 EloINpoVTN+SwBCtyBOrTw1gEq0NqFURX9n6ed3JgfTs1PKTgjUJGrDKzBVJMBl6W8+P
 AWOw==
MIME-Version: 1.0
X-Received: by 10.140.42.138 with SMTP id c10mr26749023qga.24.1393895828549;
 Mon, 03 Mar 2014 17:17:08 -0800 (PST)
Received: by 10.224.16.10 with HTTP; Mon, 3 Mar 2014 17:17:08 -0800 (PST)
Date: Mon, 3 Mar 2014 17:17:08 -0800
Message-ID: <CAJ-VmomNsR5n8K7k+qcTAaAn0HiUcxmFVDxJUw8AD5ZyKHTn5w@mail.gmail.com>
Subject: UDP transmit and no flowid
From: Adrian Chadd <adrian.chadd@gmail.com>
To: FreeBSD Net <freebsd-net@freebsd.org>, 
 "freebsd-arch@freebsd.org" <freebsd-arch@freebsd.org>
Content-Type: text/plain; charset=ISO-8859-1
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 04 Mar 2014 01:17:09 -0000

Hi,

So something I was told about whilst investigating RSS at Netflix was
that the UDP path doesn't actually get set by anything unless you're
using flowtable. So, if one doesn't define  flowtable, the transmit
path congests quite heavily through one TX queue.

I was about to do up a simple hack to do a toeplitz hash in the UDP
send path before it gets bumped up to ip_output() - that way it can
set the flowid correctly.

The other option is to do a topelitz hash in ip_output() if m_flowid
isn't set. The downside there is that we'd have to check whether it's
actually a TCP, UDP, or IP flow before we assign it a flowid.

In any case, this would likely decongest the transmit path quite a bit
for UDP send. It's still not completely RSS-y (we would still need to
line up transmit and receive paths to minimise lock contention) but it
seems like a necessary first step in that direction.

What do people think?

I'll try this out in the next week or two once I've sorted out my
employment situation and I can reserve some time on the netperf
cluster.

(before people ask "why udp?" - I'm kinda fed up with memcached
performing much worse on freebsd than linux. Since fixing the UDP side
of things requires a subset of the same work needed for TCP, I figure
we should tackle UDP first.)

Thanks,


-adrian