Date: Sun, 26 Feb 2017 23:37:59 +0800 From: Sepherosa Ziehau <sepherosa@gmail.com> To: Gleb Smirnoff <glebius@freebsd.org> Cc: Julien Charbon <jch@freebsd.org>, Jason Eggleston <jeggleston@llnw.com>, "freebsd-net@freebsd.org" <net@freebsd.org>, hiren@freebsd.org, jtl@freebsd.org, rrs@freebsd.org Subject: Re: listening sockets as non sockets Message-ID: <CAMOc5cyNA3cWjagQ6VHz5fy8F=j6J1_xGNxkr-GbqUxP6phLpg@mail.gmail.com> In-Reply-To: <20170222232704.GJ8899@FreeBSD.org> References: <20170127005251.GM2611@FreeBSD.org> <20170210063024.GE1973@FreeBSD.org> <20170216184903.GF58829@FreeBSD.org> <0858647a-ec3c-1a78-053f-d04397a82d8a@freebsd.org> <20170222232704.GJ8899@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Feb 23, 2017 at 7:27 AM, Gleb Smirnoff <glebius@freebsd.org> wrote: > On Tue, Feb 21, 2017 at 09:58:45AM +0100, Julien Charbon wrote: > J> > T> 1) The patch worked pretty okay, but the idea of separate file type is > J> > T> abandoned. With current filedescriptor code it is almost impossible > J> > T> to racelessly switch fileops and f_data. > J> > T> For more details read the commit message, URL below. > J> > T> > J> > T> 2) I moved the work to git. It allows for easier code sharing. > J> > T> > J> > T> https://github.com/glebius/FreeBSD/commit/3a52df429889ea9c6e61013f6913aad95939f159 > J> > > J> > The current 'solisten' branch at https://github.com/glebius/FreeBSD has > J> > been running successfully for 48 hours at Netflix. But as mentioned before, > J> > our connection rate is pretty low. > J> > > J> > So, testing from Julien and Hiren will be much appreciated. > J> > J> Sound good, I will have spare time to get performance results on our > J> side w/ and w/o your change before the end of this week. > > Thanks, Julien! Looking forward for your results. Meanwhile, the code has > been running for a week on a single Netflix box. And for 24 hours on a > set of boxes as A/B test against non-patched code. Since our connection > rate is low, and we never seen accept_mtx to be a major problem in our > profiling, the A/B test didn't show any statistically significant > improvements. But my biased eye of course sees some improvements in the > graphs :) Just got enough time to do some benchmark. The configuration is pretty simple, one server box w/ 2x82599, two client boxes w/ 1x82599, connected through DAC. nginx-devel is installed through pkg; access log is disabled, 24 workers, each worker w/ 16384 max connections. I use the wrk I mentioned to you. HTTP/1.1 and 1 request/connection, 15K concurrent connections from each client boxes, so 30K concurrent connections total. Each client has been verified to be able to do 160Kreqs/s w/ 1KB web object w/ 1 request/connection. 4 different sized staic web objects are benched, see below. The solisten was checked out on Feb 25th. Several GENERIC options (NFS and SCTP related) have to be commented out and kernel is built w/ -DNO_MODULES (CAM ctl module does not compile) and GENERIC-NODEBUG on the solisten branch. Each test lasts 2 minutes. MSL on the server side has been changed to 10ms, though I don't think it matters here. Server hw: 2x2620v2 (HT enabled), 32GB ddr3-1600. Client hw: i7-3770 (HT enabled), 16GB ddr3-1600. r314268 -> solisten 1KB: Performance (reqs/s) 77916.71 -> 26240.37 Latency average 121ms -> 294ms 8KB: Performance (reqs/s) 77803.72 -> 25968.87 Latency average 121ms -> 251ms 16KB: Performance (reqs/s) 75698.77 -> 25047.23 Latency average 173ms -> 377ms 24KB: Performance (reqs/s) 73736.80 -> 24946.36 Latency average 180ms -> 379ms So what I have seen is solisten's performance is 1/3 of r314268, and average latency doubles. Thanks, sephe -- Tomorrow Will Never Die
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAMOc5cyNA3cWjagQ6VHz5fy8F=j6J1_xGNxkr-GbqUxP6phLpg>