From owner-freebsd-net@freebsd.org Tue Aug 16 01:45:09 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A6A58BBB744 for ; Tue, 16 Aug 2016 01:45:09 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-io0-x230.google.com (mail-io0-x230.google.com [IPv6:2607:f8b0:4001:c06::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6C5FB19F8 for ; Tue, 16 Aug 2016 01:45:09 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-io0-x230.google.com with SMTP id 38so95145081iol.0 for ; Mon, 15 Aug 2016 18:45:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=CQS8SI196iWuDE7M/ELGSLRcxLZ7tBWty6aMxqbwMJk=; b=LyIqgO57GsiAuFYpOSSP/z6ny1nru5geakf9AUNMRlF7DncFB7Y+mOIWSBl1h8Xcyc ZtiM1ZEy/okl+DzSeNqUzIKL+rtaQ/Aj8rnWdR1hxDFR6TGxvmc41/oGbMFdzUIMFFHx T6jOx6RiDlAYWcRD7mAr5Ei2eckUJJ0u5Kh+Ft5r/1go4Hsx+I7487InNUSbk697wDLh 89Hi7sk39LvLT/HpU1vRhIWw4b2ZVa2nnQRWyWCFK36t6SMjKiCYKGHJ/ahachvFlQOV jdJe7wFDuj66pnDk/yqCINI2Qz5mgbqiTdQvNF9/bQzSmrk0WWkWO4Kl1STW2Nb1F82W +J2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=CQS8SI196iWuDE7M/ELGSLRcxLZ7tBWty6aMxqbwMJk=; b=P+jUKsY2ztWlTksXjSNMatY2J37KeerQ4yCCX4YLHLVDeDUR46f+Z3UszugNAQB7I5 3ptof3B7QCbhEG2lM0V4iaxPjRXKAir7/OjpxTbzFDKGthER0K4g8oJWpxduQEtouQAD 7kaZAYwHlqatX240CpPQejVYfHmPSThFXbp01PZ+C38JQYvBLKsQyNurLba2B7wfog6x hlGZA3wWnBE8cXodHPHpwt+3tMLohqLy0oJBK+Ohx6S+C8GeZMgJ+6zcnQy4ZCjHuqvC PM2EbxHo31NvDtdeF3xa4yBZtyN/Au8F1Tf2AbbxqYxtTzHR+33+HQnZrwq0YQhliB9y eBoQ== X-Gm-Message-State: AEkoout8w2MZgVBNgpeEcyRtLeZAZ6gLkxXy1OhqOQb/mcnV3JoYu4lb1RBDlkUE9xMWqpgyVOhSOxF7YzFTpw== X-Received: by 10.107.144.10 with SMTP id s10mr35362409iod.165.1471311908895; Mon, 15 Aug 2016 18:45:08 -0700 (PDT) MIME-Version: 1.0 Received: by 10.36.141.129 with HTTP; Mon, 15 Aug 2016 18:45:07 -0700 (PDT) In-Reply-To: <91AEB1BD-44EA-43AD-A9A1-6DEBF367DF9B@gmail.com> References: <3C0D892F-2BE8-4650-B9FC-93C8EE0443E1@gmail.com> <3B164B7B-CBFB-4518-B57D-A96EABB71647@gmail.com> <5D6DF8EA-D9AA-4617-8561-2D7E22A738C3@gmail.com> <7DD30CE7-32E6-4D26-91D4-C1D4F2319655@gmail.com> <91AEB1BD-44EA-43AD-A9A1-6DEBF367DF9B@gmail.com> From: Adrian Chadd Date: Mon, 15 Aug 2016 18:45:07 -0700 Message-ID: Subject: Re: Unstable local network throughput To: Ben RUBSON Cc: FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 01:45:09 -0000 Hi, ok, can you try 5) but also running with the interrupt threads pinned to CPU 1? It looks like the interrupt threads are running on CPU 0, and my /guess/ (looking at the CPU usage distributions) that sometimes the userland bits run on the same CPU or numa domain as the interrupt bits, and it likely decreases some latency -> increasing throughput slightly. Thanks, -adrian