From owner-freebsd-net@freebsd.org Mon Feb 27 01:31:36 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8B5C8CEFEAD for ; Mon, 27 Feb 2017 01:31:36 +0000 (UTC) (envelope-from sepherosa@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 695F5AE2 for ; Mon, 27 Feb 2017 01:31:36 +0000 (UTC) (envelope-from sepherosa@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 65B72CEFEAC; Mon, 27 Feb 2017 01:31:36 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65603CEFEAB for ; Mon, 27 Feb 2017 01:31:36 +0000 (UTC) (envelope-from sepherosa@gmail.com) Received: from mail-ua0-x234.google.com (mail-ua0-x234.google.com [IPv6:2607:f8b0:400c:c08::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1EE12AE1; Mon, 27 Feb 2017 01:31:36 +0000 (UTC) (envelope-from sepherosa@gmail.com) Received: by mail-ua0-x234.google.com with SMTP id 72so19849057uaf.3; Sun, 26 Feb 2017 17:31:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Wx14uybWlxnToZGQEBw7ygcOTtP7fwtb0ic+Nalq/k8=; b=rg5v9ZVwCt3tP4UYodtZ7sYZlu2hT+8jhK1G4goCpu3vKiRdCtPGWn9OAnpkH/YnXC E8xfGHmSIDkYZ072EKQd6n3qRIQYHS4KBPWRSfLWioXntlQ2XhVrUlHmqZvG6/1BxOtP ZE+Ynq73bg+zrelTKmeR8B7fRjBmDpwJDnNV2q+MZSD0K+lAu64qC410sIkW4iR4JK5S dlGzz6DjfEVn/Hsh6nIPEo7Jzjeo5vYDWOThJP5HlWWdHsbiTFBIdEXFBPxn/Hv4gV0B lwvJ3L0G4sQt2QExmJ1gVPuvzqW6HL/JmW1MR7Ys2eaPETBr5GcTKKWLCwgsqplvd9mc C+Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Wx14uybWlxnToZGQEBw7ygcOTtP7fwtb0ic+Nalq/k8=; b=uM+CLodahlyqiClvIQBd4bYKrvPLgMlmPFFj6NfAjP9IzO5QoQCw5HyeRznNbttaP3 rKD7+iVonx2jdKNS2mBAAJA3xXufhsdrxn5yQdgFPgnqjupFIarByJVZEt3gGGtachD2 MBz/jtYLanj/Hb2Bj6DwiRjVDGgL+APOXcsq0zIZadgOq0urzuywJB7RM6kIbz17S3Vl +HbZ1arGhNHRcF+ww0CtU9C5D6Q3+9LLfixCgdvfmbpsV8Qa5cDMSZKhOMcXpC98Q3nn whimZxTGv1BTRQUAVJaPUT25ahZSuQndlG6zJMrMD+AzJoRMUTb/EvASOc2mNN+sX3yr jRNQ== X-Gm-Message-State: AMke39mg0QFw2XkROHRuLoN33k5Kt16KQJmejZe93qACX14GkQkNmq/PpaXZSg5IKn4HgktfdESM/+zkXVDPXQ== X-Received: by 10.31.109.71 with SMTP id i68mr6061088vkc.89.1488159095220; Sun, 26 Feb 2017 17:31:35 -0800 (PST) MIME-Version: 1.0 Received: by 10.176.66.193 with HTTP; Sun, 26 Feb 2017 17:31:34 -0800 (PST) In-Reply-To: <20170226173744.GH8899@FreeBSD.org> References: <20170127005251.GM2611@FreeBSD.org> <20170210063024.GE1973@FreeBSD.org> <20170216184903.GF58829@FreeBSD.org> <0858647a-ec3c-1a78-053f-d04397a82d8a@freebsd.org> <20170222232704.GJ8899@FreeBSD.org> <20170226173744.GH8899@FreeBSD.org> From: Sepherosa Ziehau Date: Mon, 27 Feb 2017 09:31:34 +0800 Message-ID: Subject: Re: listening sockets as non sockets To: Gleb Smirnoff Cc: Julien Charbon , Jason Eggleston , "freebsd-net@freebsd.org" , hiren@freebsd.org, jtl@freebsd.org, rrs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Feb 2017 01:31:36 -0000 On Mon, Feb 27, 2017 at 1:37 AM, Gleb Smirnoff wrote: > On Sun, Feb 26, 2017 at 11:37:59PM +0800, Sepherosa Ziehau wrote: > S> Just got enough time to do some benchmark. The configuration is > S> pretty simple, one server box w/ 2x82599, two client boxes w/ 1x82599, > S> connected through DAC. nginx-devel is installed through pkg; access > S> log is disabled, 24 workers, each worker w/ 16384 max connections. I > S> use the wrk I mentioned to you. HTTP/1.1 and 1 request/connection, > S> 15K concurrent connections from each client boxes, so 30K concurrent > S> connections total. Each client has been verified to be able to do > S> 160Kreqs/s w/ 1KB web object w/ 1 request/connection. 4 different > S> sized staic web objects are benched, see below. The solisten was > S> checked out on Feb 25th. Several GENERIC options (NFS and SCTP > S> related) have to be commented out and kernel is built w/ -DNO_MODULES > S> (CAM ctl module does not compile) and GENERIC-NODEBUG on the solisten > S> branch. Each test lasts 2 minutes. MSL on the server side has been > S> changed to 10ms, though I don't think it matters here. Server hw: > S> 2x2620v2 (HT enabled), 32GB ddr3-1600. Client hw: i7-3770 (HT > S> enabled), 16GB ddr3-1600. > S> > S> r314268 -> solisten > S> > S> 1KB: > S> Performance (reqs/s) > S> 77916.71 -> 26240.37 > S> Latency average > S> 121ms -> 294ms > S> > S> 8KB: > S> Performance (reqs/s) > S> 77803.72 -> 25968.87 > S> Latency average > S> 121ms -> 251ms > S> > S> 16KB: > S> Performance (reqs/s) > S> 75698.77 -> 25047.23 > S> Latency average > S> 173ms -> 377ms > S> > S> 24KB: > S> Performance (reqs/s) > S> 73736.80 -> 24946.36 > S> Latency average > S> 180ms -> 379ms > S> > S> So what I have seen is solisten's performance is 1/3 of r314268, and > S> average latency doubles. > > Thanks a lot! Looks like I've put something on the accept path that degrades > performance. :) I will look at this tomorrow and post an update. > > Can you please share exact wrk command you used? On my testing client (8 HT). wrk -c 15000 -t 8 -d 120s --delay --latency --connreqs 1 http://server/1K.bin > Do I understand it correct that there was only 1 listening socket on the > server side? Yep, only port 80 is listened on the server side, as a normal website will do. Thanks, sephe -- Tomorrow Will Never Die