From owner-freebsd-stable@FreeBSD.ORG Mon Sep 22 06:29:26 2014 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7495E78D; Mon, 22 Sep 2014 06:29:26 +0000 (UTC) Received: from mail-wg0-x234.google.com (mail-wg0-x234.google.com [IPv6:2a00:1450:400c:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BA4DD18F; Mon, 22 Sep 2014 06:29:25 +0000 (UTC) Received: by mail-wg0-f52.google.com with SMTP id n12so372274wgh.11 for ; Sun, 21 Sep 2014 23:29:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=a157o+BUVJu1m4KnfEP55AShoehOTzsefYrtyn8m/yU=; b=NNBej8EJx89FIHqMa5Gg38fCTvCdglNxionLqAjBk6YfkaNESjxy2bCWy5HAMwdlkQ GbwnacCu4QUQR4S8S9xz0VRXN68UAF8xO6jxvJGj6vEaqxRGc2L9g0IzisvE3Byj8Q1v hQXth2okTLQICcfqoQb/TTNCFBT/s91BfFvAs2ihm4O9jHU2LjSJLf5o8pJpqTz3DYBG diUdAQrJ+ppUhElETDSit8iciG9L/ne0DrcVdVCqDzcEj30YGrMmcNG4Wfrwod4jmiKp RLSU/CDL2g8vlK3K/eAywZDYxRQ8wGV0BMOgUlxM1XS51PJUlxYS3r6UsuPG92rRFXbx n0sw== MIME-Version: 1.0 X-Received: by 10.180.9.144 with SMTP id z16mr13022764wia.26.1411367363952; Sun, 21 Sep 2014 23:29:23 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.216.106.199 with HTTP; Sun, 21 Sep 2014 23:29:23 -0700 (PDT) In-Reply-To: References: <1411259605.674669006.339g4pd4@frv35.fwdcdn.com> Date: Sun, 21 Sep 2014 23:29:23 -0700 X-Google-Sender-Auth: qmF0m72FZf7JGMAKdyLsWxHQCTc Message-ID: Subject: Re: FreeBSD 10 network performance problems From: Adrian Chadd To: "K. Macy" Content-Type: text/plain; charset=UTF-8 Cc: Rumen Telbizov , Tom Elite , "freebsd-stable@freebsd.org" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Sep 2014 06:29:26 -0000 Hi! On 21 September 2014 15:08, K. Macy wrote: > What you're dealing with is hardly an edge case. Most people don't need to > push more than a couple of Gbps in production. > > Flowtable is hardly "untested." However, it has been a source of friction > at times because it can be somewhat brittle, having limits on the number of > cache entries that it can store that are frequently too low for people with > very large numbers of active flows. Without raising this limit > substantially these systems will fail in a rather spectacular fashion. > Additionally, flowtable was not written with the intent of being a routing > cache. It was developed to support stateful multipath routing for load > balancing. In its current incarnation, stripped of much of the code for its > initial purpose, it's really just a band-aid around locking problems in > routing. That said, the handful of commercial users of FreeBSD that do have > large amounts of traffic (10s of Gbps) per system that I personally know of > all have flowtable enabled. > > Unfortunately, at least in terms of what is in HEAD, little has been done > to fix the contention that flowtable works around. For your purposes the > response that Adrian gave you is the closest to "optimal." Gleb and I spent a bunch of time late last year and early this year finding and fixing a lot of the corner cases with Flowtable handling. I'm pretty sure that it'll behave predictably and reliably for people now. If it doesn't then please file PRs. It's no longer some corner of the codebase that isn't well understood. At least two people besides the author (Gleb and I) understand what it is, how it works and how it ties into things. What's missing is someone sitting down and adding flowtable support to the rest of the forwarding path(s). It shouldn't be too hard - as long as you have an mbuf that has the IPv4/IPv6 header setup as that's what flowtable currently expects when doing lookups - but it has to be done. So I thoroughly recommend someone who has the test setup and the desire to do so and post results. I have enough equipment now to test this out and develop it but I'm really busy doing work, wireless RSS stuff. So I just don't have the spare cycles to do it. I do think it'll be a pretty simple task. In theory - once you have the flowtable code working correctly in the forwarding path you shouldn't see any rtentry lock contention except during route changes (which will invalidate flowtable entries and cause normal routing table lookups until the flowtable has all the route entries in question.) -a