From owner-freebsd-current@freebsd.org Wed Aug 15 16:52:27 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9D68410677AF for ; Wed, 15 Aug 2018 16:52:27 +0000 (UTC) (envelope-from micchie@sfc.wide.ad.jp) Received: from mail.sfc.wide.ad.jp (mail.sfc.wide.ad.jp [IPv6:2001:200:0:8803:203:178:142:146]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2EB3C96490; Wed, 15 Aug 2018 16:52:27 +0000 (UTC) (envelope-from micchie@sfc.wide.ad.jp) Received: from n-1059.office.hd (mito.neclab.eu [195.37.70.39]) by mail.sfc.wide.ad.jp (Postfix) with ESMTPSA id 4749E2799BD; Thu, 16 Aug 2018 01:52:22 +0900 (JST) Subject: Re: TCP server app performance To: Navdeep Parhar , freebsd-current@freebsd.org References: From: Michio Honda Message-ID: <9e67c516-799c-8a1e-0cc5-64ab1e582a98@sfc.wide.ad.jp> Date: Wed, 15 Aug 2018 18:52:17 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Aug 2018 16:52:27 -0000 On 08/14/2018 09:30 PM, Navdeep Parhar wrote: > On 8/12/18 9:50 AM, Honda Michio wrote: >> Hi, >> >> I'm measuring TCP server app performance using my toy web server. >> It just accept TCP connections and responds back HTTP OK to the clients. >> It monitors sockets using kqueue, and processes each ready descriptor using >> a pair of read() and write(). (in more detail, it's >> https://github.com/micchie/netmap/tree/paste/apps/phttpd) >> >> Using 100 persistent TCP connections (the client sends 44 B HTTP GET and >> the server responds with 151 B of HTTP OK) and a single CPU core, I only >> get 152K requests per second, which is 2.5x slower than Linux that runs the >> same app (except that it uses epoll instead of kqueue). >> I cannot justify this by myself. Does anybody has some intuition about how >> much FreeBSD would get with such workloads? >> I tried disabling TCP delayed ack and changing interrupt rates, but no >> significant difference was observed. >> >> I use FreeBSD-CURRENT with GENERIC-NODEBUG (git commit hash: 3015145c3aa4b). >> For hardware, the server has Xeon Silver 4110 and Intel X540 NIC (activate >> only a single queue as I test with a single CPU core). All the offloadings >> are disabled. > > I hope hw L3/L4 checksumming is still on? They are off but exchanged messages are too small to benefit from these offloads anyways. > > Are your results similar to what you get with 100 (same number as your > test clients) netperf's doing TCP_RR on this setup, or wildly different? I cannot find any netperf option that sends multiple connections with TCP_RR. I also don't find any equivalent in iperf. But single-connection performance is similar to what I get with wrk and my HTTP server. Running 100 netperf processes does not show aggregate throughput, and that also means running 100 kevent() loops, which is quite different behaviour from what I want. Cheers, - Michio > > Regards, > Navdeep >