Date: Mon, 8 Aug 2016 09:57:49 +0800 From: Sepherosa Ziehau <sepherosa@gmail.com> To: Eugene Grosbein <eugen@grosbein.net> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: 40Gbps http client benchmark Message-ID: <CAMOc5cyHsLZ93R9jQncBgReii0nCTnW0ZoBFbA4RkHpTcwHcnw@mail.gmail.com> In-Reply-To: <57A62668.7020309@grosbein.net> References: <57A62668.7020309@grosbein.net>
next in thread | previous in thread | raw e-mail | index | archive | help
We have an optimized wrk here: https://github.com/sepherosa/wrk It greatly reduces the # of kqueue syscalls and avoids unnecessary setsockopt etc. BTW, how many concurrent connections and threads are you testing w/? Did you reduce the MSL on your client machines? Default local port range probably should be ok, but it still worth checking. Thanks, sephe On Sun, Aug 7, 2016 at 2:03 AM, Eugene Grosbein <eugen@grosbein.net> wrote: > Hi! > > Is there any high performance benchmark acting as http client for outer http > server > capable to receive 40Gbps without overwhelming CPU with insane number of > syscalls? > > I've tried benchmarks/wrk version 4.0.2 and it works just fine upto 20Gbps > for my hardware: two 6-core (HT disabled) Intel(R) Xeon(R) CPU E5-2620 v3 @ > 2.40GHz > with two dual-port ix(4) 82599ES 10-Gigabit SFI/SFP+ Network Connection > combined to single lagg interface (lagghash l4). > > But each worker pthread of wrk generates too many kqueue() system calls > polling for incoming data and eats 100% of its CPU core and cannot receive > more. > Or, it may be some kqueue() kernel level lock contention, I do not know. > More worker threads, more overloaded CPU cores, no increase of transfer over > about 20Gbps. > > I transfer 1MByte-sized files for the benchmark (tried 8MB-sized too). > I set sysctl net.inet.tcp.recvspace=4194304 (4MB). > I've even patched wrk to to use read buffer sized 4MB instead of its > compiled-in > default 8k, as it does not change default socket options other than > TCP_NODELAY: > > --- src/wrk.h.orig 2016-08-06 23:20:16.205906000 +0700 > +++ src/wrk.h 2016-08-06 23:20:20.460579000 +0700 > @@ -18,7 +18,7 @@ > #include "ae.h" > #include "http_parser.h" > > -#define RECVBUF 8192 > +#define RECVBUF 4194304 > > #define MAX_THREAD_RATE_S 10000000 > #define SOCKET_TIMEOUT_MS 2000 > > Nothing helps to decrease consumed system time and ktrace(1) assures > that's because of kevent() calls. > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" -- Tomorrow Will Never Die
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAMOc5cyHsLZ93R9jQncBgReii0nCTnW0ZoBFbA4RkHpTcwHcnw>