Date: Mon, 04 Nov 2013 22:21:04 +0100 From: "Julien Charbon" <jcharbon@verisign.com> To: freebsd-net@freebsd.org Subject: TCP stack lock contention with short-lived connections Message-ID: <op.w51mxed6ak5tgc@fri2jcharbon-m1.local>
next in thread | raw e-mail | index | archive | help
Hi list, just a follow-up of vBSDCon discussions about FreeBSD TCP performances with short-lived connections. In summary: Using a single NIC receive queue, the TCP stack performance results with short-lived connections on FreeBSD are great: We got 52,000 (52k) TCP queries per second (QPS) before being CPU bound, we achieved the same result on Linux. However, when configuring 4 NIC receive queues each bound to a different core of the same CPU, results are lower than expected: We got 56k QPS, where we reached 200k QPS on Linux on same hardware. Basically, FreeBSD TCP stack does not scale well across multiple receive queues/CPU cores due to a lock contention. I have put technical and how-to-repeat details in below PR: kern/183659: TCP stack lock contention with short-lived connections http://www.freebsd.org/cgi/query-pr.cgi?pr=183659 We are currently working on this performance improvement effort; it will impact only the TCP locking strategy not the TCP stack logic itself. We will share on freebsd-net the patches we made for reviewing and improvement propositions; anyway this change might also require enough eyeballs to avoid tricky race conditions introduction in TCP stack. Thanks. -- Julien
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?op.w51mxed6ak5tgc>