Date: Sat, 24 Feb 2007 11:47:34 +0000 (GMT) From: Robert Watson <rwatson@FreeBSD.org> To: current@FreeBSD.org Cc: performance@FreeBSD.org Subject: Fine-grained locking for UNIX domain sockets: patch updated Message-ID: <20070224095324.K36322@fledge.watson.org>
next in thread | raw e-mail | index | archive | help
As part of Kris and Jeff's recent work on improving MySQL scalability on FreeBSD, I've updated my fine-grained locking patch for UNIX domain sockets to a more recent 7-CURRENT: http://www.watson.org/~robert/freebsd/netperf/20070224-uds-fine-grain.diff This patch replaces the global mutex with a global rwlock, and introduces per-unpcb locks to protect the state of each socket at the UNIX domain socket layer. Depending on the protocol operation, either the per-unpcb mutex is sufficient, or global read or write locks may also be required. The role of the global lock is to protect the topology of the connections between sockets, protect vnode v_socket pointers, and to prevent lock order issues in the event the per-unpcb lock of more than one socket must be acquired at a time. For example, exclusive global locking is required during bind() and connect() to protect v_socket and change the topology of connected sockets, whereas recv() requires only the unpcb lock of the receiving socket. send() may require either just an unpcb lock or a global lock if send() is implicitly connecting (sendto()) or disconnecting (PRUS_EOF) a socket. This is basically the same patch that I have been maintaining since 2005 and posted at intermittent intervals when there have been reports of negative performance due to high contention on the UNIX domain socket subsystem lock. I continue to look for performance feedback, especially from workloads that are heavy consumers of UNIX domain sockets (such as MySQL). My experience has been that it gives a small performance hit on workloads and in configurations without much contention/scalability, but a measurable performance benefit with anything more substantial. Decrease in lower end workloads and configurations is at least in part due to the overhead of some additional locking vs. the benefits of reduced contention. Something that I am particularly interested in looking for is little or no performance hit on UP. This is a fairly heavily tested and refined patch, so the issue in deciding whether or not to commit this is its real-world effect on performance. Robert N M Watson Computer Laboratory University of Cambridge
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070224095324.K36322>