From owner-freebsd-current@freebsd.org Tue Aug 14 19:30:11 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 05F17105582F for ; Tue, 14 Aug 2018 19:30:11 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 70E2083B53 for ; Tue, 14 Aug 2018 19:30:10 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: by mail-pf1-x42c.google.com with SMTP id k19-v6so9702535pfi.1 for ; Tue, 14 Aug 2018 12:30:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:references:from:openpgp:autocrypt:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=XbWP14XG0K1PIQfwLDS8ghB0M8U9hrDdPQW7DjN2oi0=; b=QsccPVHTTIuOQQIRNPSx5JzOPqkVKiIl0yfCBmuK+L/z2azQNqi1EXQZCmWY/gS4Gg fV58XiXI24xlRaqqt0TZ7Bp3cXBY+1728EKtizs/6HgkDSa2tyvFhd0931f3/pJi33xD aT5/tx9U3xWS1MZLp22gqPa5EzCYcIdnrKfs/o6O+HdiqNGH/t7xKRUPDojN3eT1mxan xMS9d/cey/xxSFPebI5r++tV81rg5z7D2AQvHVWFVnvAXgA+emRY5s/QlLjUQ+p6Gw7g 406gi3AlsCwY4QmNn1lVSe+OzaKVtfsP8IEcW1PZbIBaPo/XkDKOfoEgc6gaAu5GB0hy nALQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:references:from:openpgp :autocrypt:message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=XbWP14XG0K1PIQfwLDS8ghB0M8U9hrDdPQW7DjN2oi0=; b=p4ZV7jWO13FxXlx0ywpHpgtFGFjYzzi4mJXljE8qBv4aEbwH6JylxKgxQtsMMv0vco TFGTcQbK8+XXig1lHWRRj62VLAepMoniSpgoziJXfr6JqxNmYCtnGmi7Bc9sfyXLmmLe /IMb6ZoFY/zlnPcg3DuHsWCUZeAgy/aMSJvlMFPReNvaZbfuUoDR4+NrT7m4MI7bZ/Gs /iwr9E7Y9umEFZWWVPjc7QcUWLZWswXCCDleJODxXrkN/oCiMSt7An5WrsBx1MpUUJPM /Wt/SxHjg0sLAuQ8Y4BHGtgWwHkHxuP6f63BNJSbLX9jc1gN8iJGLr+PRg3dzJeRjdYX gQKQ== X-Gm-Message-State: AOUpUlHb1AhdrmRgIdSgq27q9iUytRL9xzmDQ17i+x6GaRMJ6tPEL6yA LiWLg0nPz+Z6auetG7PGe7rIxk/O X-Google-Smtp-Source: AA+uWPx6HEyNz41rS9e2x1Z1AdCrNOgghMD6Llks4cCDB+faaOetB011ZJl7Wi+3AkWKR/8TKn6eAQ== X-Received: by 2002:a63:f18:: with SMTP id e24-v6mr22509382pgl.320.1534275009246; Tue, 14 Aug 2018 12:30:09 -0700 (PDT) Received: from [10.192.166.0] (stargate.chelsio.com. [12.32.117.8]) by smtp.googlemail.com with ESMTPSA id x184-v6sm26485272pfd.169.2018.08.14.12.30.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 14 Aug 2018 12:30:08 -0700 (PDT) Sender: Navdeep Parhar Subject: Re: TCP server app performance To: Honda Michio , freebsd-current@freebsd.org References: From: Navdeep Parhar Openpgp: preference=signencrypt Autocrypt: addr=np@FreeBSD.org; prefer-encrypt=mutual; keydata= xsDiBEosaGcRBACOXnXquGEW53BjpMt2jViod/TUf1xgjMekcbDxqOODPeX7eYfrwJ8G6BCN OpGjBmWDu/JcNj4Z+gmTilJ6WLZQ7ecFZfEeO91pt6ys0cyWh0xfO+/mT83D7W81S/kqrJBk QbBIdV6LumevdErHo272r8RcMELC4Ru87eRtX3hmEwCgnnGNJMpQFUfYTt5XE7nY0yQoeV8D /0OcWmJbEZWxX9O7AuliCe3zd2Dw0B4LB9SZ2Dis7+gpVd3xVgYnt5wRE9kM+ThgrMA/wqr8 07qmEG6bcfUsfwwGN9YUtNF3xAN07cXTs026sCIFNZK816PrThBzCgkwR7pDpkMzGWIBr8Wi XXy0eB+JlQ6UV4PEiXuZ5ulzP0b1A/9CZm3wJfrNC0r1gMyrfVedg4zwKU997bmPLGcYs+rW XDTI9CvMseOUYn4CoDZQCp/9zxuHK+VU7Y/w0c/hVE5ERACSn4SjN2unEDstK9njZBMHEPVk Ae/YvSG5cmc97SHlVE+eu/bbLKcvFb6rRLPOaVFQJMJA2VJEGWtYhvP7Zc0fTmF2ZGVlcCBQ YXJoYXIgPG5wQEZyZWVCU0Qub3JnPsJgBBMRAgAgBQJKLGhnAhsDBgsJCAcDAgQVAggDBBYC AwECHgECF4AACgkQyrIrk6yriBL0MQCfUJOiS2PbJFDeiav1ylcXXwfpggAAoJRoS7GDENGy M4BzjJ4b0ptZqTLRzsFNBEosaGcQCACFCWs47SL4DQA6bNDlVJu4w8wLf8uVOyatuGmdXX8Y /OTVQJgA3vS+ODNVJCxhKVlvhcn7bhBdGdWKS9K+lr8+eEvr4hf2bQpesoHC+uFgKyILkCBN L8raixbhysyq0pfZWWDJMyn+G42BG1yJJi+bykygdpYnbIVA8dYHmBibI8mkPKOHSohjXT1S RfGGn+l1w54OO4NlJhCXMkjTA/Z9Bt4XeaiR85uJi0UUfV8FGZHhgSvT+/P1xIvz+nytuehS P/QLXl13CtAG/nKVkAcZnsT/3NrJ4Z2r45k+c50Wrf210scAaBogrrV5eIHfNGgOANApN8+8 vj+aXO4pXRuXAAMFB/44ea8rd+P5N3OMrfuM8i91Qe1bJ+BIoroKPOr8jvCry0h3QpdfLKUN IgaqbS3JZeBJ8HHnWSGCF+o6H5gzRe1hvylPEclLPDCuPe7T746h9Mzejf2hNDJvOg+BuweD ZW4KhovVbdS+syJEvpGF4bO8qgHT2CKgruXSHbFetdQWbkM0rfMmTuo0GcR2BEVrPb/SPFv6 4ZZyAZzmnGO4vT1bzClnTzJixrDpH74M3vSEYegMB4KdbLYBi8Jx4QUKgVEhJHjJubKWX4et yU/uuehOC3xYrmr1UXvsom3U8r36Dvdo77Yr3dgDVXa7bolNx0TIhdWxZI+R4z9E75QY+/wg wkkEGBECAAkFAkosaGcCGwwACgkQyrIrk6yriBI+JQCfUxgyqGtzZvLh5Al7gsTmRc11PLwA niD3NfWGRcO2+9uxSSQqRH1ywC4n Message-ID: Date: Tue, 14 Aug 2018 12:30:07 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Aug 2018 19:30:11 -0000 On 8/12/18 9:50 AM, Honda Michio wrote: > Hi, > > I'm measuring TCP server app performance using my toy web server. > It just accept TCP connections and responds back HTTP OK to the clients. > It monitors sockets using kqueue, and processes each ready descriptor using > a pair of read() and write(). (in more detail, it's > https://github.com/micchie/netmap/tree/paste/apps/phttpd) > > Using 100 persistent TCP connections (the client sends 44 B HTTP GET and > the server responds with 151 B of HTTP OK) and a single CPU core, I only > get 152K requests per second, which is 2.5x slower than Linux that runs the > same app (except that it uses epoll instead of kqueue). > I cannot justify this by myself. Does anybody has some intuition about how > much FreeBSD would get with such workloads? > I tried disabling TCP delayed ack and changing interrupt rates, but no > significant difference was observed. > > I use FreeBSD-CURRENT with GENERIC-NODEBUG (git commit hash: 3015145c3aa4b). > For hardware, the server has Xeon Silver 4110 and Intel X540 NIC (activate > only a single queue as I test with a single CPU core). All the offloadings > are disabled. I hope hw L3/L4 checksumming is still on? Are your results similar to what you get with 100 (same number as your test clients) netperf's doing TCP_RR on this setup, or wildly different? Regards, Navdeep