From owner-freebsd-net@FreeBSD.ORG Wed May 20 13:27:53 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2AA5E568 for ; Wed, 20 May 2015 13:27:53 +0000 (UTC) Received: from mail-wi0-f178.google.com (mail-wi0-f178.google.com [209.85.212.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B7766185E for ; Wed, 20 May 2015 13:27:52 +0000 (UTC) Received: by wicmx19 with SMTP id mx19so154942458wic.0 for ; Wed, 20 May 2015 06:27:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type; bh=R321FTqNbOzUgyyAH1ABZe2+KMr+RAg8lfCv6vRmI+w=; b=EW5EV8Ym5DBxPHJBmVO0wvPV3Y0UfEHe5I/DFvukyRZO/Kznmwp8SG03rriF7kt/u8 +ubg0bp+AhpJBhzr6QHwEWGm076eMBJzGKPFLTrYAHmNAlsAd2lRHM0rOcdzUIHGqUuv iqiRZW6E1B3fpiFL1ck9UzGkbzgQEfLbm/8ZBqJlMbDVK9hA24MfudK+rLrjn5ouYVnh GqjVSnx1g+XwV9BdpA8mPuJVSafhOhCaBgemBzVxoVFhpl2/FQuS7+QRWWdsYhH/7cit /N5vmgdN/ME6sJWluEoYg45Vcs1Dgv+FxcoQCelDhjZh1S9L1kHUbN+ymR9OyiHWKsdH YVNQ== X-Gm-Message-State: ALoCoQk1KWX1jl/qr3fkO8duDB/YJ6LYOLTfVR70lPLr0YP91zvpc9OhgJXNT1Wl3HjRwWr7//9B X-Received: by 10.194.109.97 with SMTP id hr1mr62950828wjb.10.1432128465424; Wed, 20 May 2015 06:27:45 -0700 (PDT) Received: from FRI2JCHARBON-M1.local ([217.30.88.7]) by mx.google.com with ESMTPSA id gw7sm3493433wib.15.2015.05.20.06.27.44 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 20 May 2015 06:27:44 -0700 (PDT) Message-ID: <555C8BC9.2080304@freebsd.org> Date: Wed, 20 May 2015 15:27:37 +0200 From: Julien Charbon User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Adrian Chadd , freebsd-net@freebsd.org Subject: Re: TCP stack lock contention with short-lived connections References: <537F39DF.1090900@verisign.com> <537FB51D.2060401@verisign.com> <537FBFA4.1010902@FreeBSD.org> <53834368.6070103@verisign.com> In-Reply-To: <53834368.6070103@verisign.com> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="epFRD7QcQ1FxABumOFJwiRdXf3pSI0Al1" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 May 2015 13:27:53 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --epFRD7QcQ1FxABumOFJwiRdXf3pSI0Al1 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi, On 26/05/14 15:36, Julien Charbon wrote: > On 23/05/14 23:37, Navdeep Parhar wrote: >> On 05/23/14 13:52, Julien Charbon wrote: >>> On 23/05/14 14:06, Julien Charbon wrote: >>>> On 27/02/14 11:32, Julien Charbon wrote: >>>>> On 07/11/13 14:55, Julien Charbon wrote: >>>>>> On Mon, 04 Nov 2013 22:21:04 +0100, Julien Charbon >>>>>> wrote: >>>>>>> I have put technical and how-to-repeat details in below >>>>>>> PR: >>>>>>> >>>>>>> kern/183659: TCP stack lock contention with short-lived >>>>>>> connections >>>>>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=3D183659 >>>>>>> >>>>>>> We are currently working on this performance improvement >>>>>>> effort; it will impact only the TCP locking strategy not >>>>>>> the TCP stack logic itself. We will share on freebsd-net >>>>>>> the patches we made for reviewing and improvement >>>>>>> propositions; anyway this change might also require enough >>>>>>> eyeballs to avoid tricky race conditions introduction in >>>>>>> TCP stack. >>> >>> [...] >=20 > Below, just for your information, more details on context of these > changes: >=20 > o The rough consensus at BSDCan was that there is a shared interest fo= r > scalability improvement of TCP workloads with potential high rate of > connections establishment and tear-down. >=20 > o Our requirements for this task were: > - Achieve more than 100k TCP connections per second without dropping = a > single packet in reception > - Use a strategy that does not require to change all network stacks i= n > a row (TCP, UDP, RAW, etc.) > - Be able to progressively introduce better performance, leveraging > already in place mutex strategy > - Keep the TCP stack stable (obviously) For people interested about this short-lived TCP connection scalability effort, you can subscribe to the review of our latest (and biggest so far) change: Decompose TCP INP_INFO lock to increase short-lived connections scalabili= ty https://reviews.freebsd.org/D2599 The main goal of this review is ideally to start the rough discussion before BSDCan (and then discuss details in person at BSDCan), and ease tests by other people with more exotic configurations (thanks Adrian for your early tests). This patch still improves the short-lived TCP connection rate (setup and teardown) from 60k/sec to 150k/sec. My 2 cents. -- Julien --epFRD7QcQ1FxABumOFJwiRdXf3pSI0Al1 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJVXIvPAAoJEKVlQ5Je6dhx57MIAOvKPS+Y+6HFOIoxobowKu6C kmwNwEmgwhCPuQK58carrEkXZXgjaYmfnhiHii02qzwanwope2V+UrpfkZ4eBzdQ TT2U0U6XWbB/iOCWcaQjJKNZlryAUsFZG6U/jNj9xOK/MTF5lfeTq6l6/N6sTJ/R HZhqLcHwC3eTlEuWfiYrw6X6iKxiTbNMWCaBqT+K5CJY+rk9YGnAYKKBP/IhBRoP oP8LXo0XCcptmGzIFzAjsE3lxmLZGj/Kuk0GW1EqIhOIuqXKJF45wRSfDYxgcMZG sOabDj+242O1YQDtL+Cxj9a6UWrHFzV64oTyaA5b1pdM76p28+XdMFy0yIPd1aI= =CF03 -----END PGP SIGNATURE----- --epFRD7QcQ1FxABumOFJwiRdXf3pSI0Al1--