Date: Fri, 25 Mar 2011 21:38:10 +0000 From: "Robert N. M. Watson" <rwatson@freebsd.org> To: John Baldwin <jhb@freebsd.org> Cc: freebsd-net@freebsd.org, Jim <uukkhh@gmail.com> Subject: Re: why use INP_WLOCK instead of INP_RLOCK Message-ID: <997B5A3A-8AC7-42F5-BE43-64B6CB4E2A25@freebsd.org> In-Reply-To: <201103251701.34576.jhb@freebsd.org> References: <AANLkTinrVJ4jehNm7xh=ix7k3r0Q0YY38nc59AkgiP_P@mail.gmail.com> <201103251701.34576.jhb@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 25 Mar 2011, at 21:01, John Baldwin wrote: > On Tuesday, February 01, 2011 12:54:33 am Jim wrote: >> I am not sure if anybody has asked it before. I could not find answer = by >> doing rough search on Internet, if it is duplicate question, sorry in >> advance. >>=20 >> My question is that, for getting socket options in tcp_ctloutput() in >> tcp_usrreq.c, why do we need to do lock with INP_WLOCK(inp) as = setting >> socket options does. Why do we just use INP_RLOCK(inp), as it looks = not >> changing anything in tcp control block? >=20 > I think mostly it is just because no one has bothered to change it. =20= > Realistically it probably won't make any noticable difference unless = your=20 > workload consists of doing lots of calls to getsockopt() but not = sending any=20 > actual traffic on the associated sockets. :) (Almost all of the other=20= > operations on a TCP connection require a write lock on the pcb.) Just to reiterate John's point here: the critical performance paths for = TCP both require the inpcb lock to be held exclusively (input and = output), and socket options are typically called from the same user = thread doing I/O, meaning that acquiring read locks instead of write = locks is unlikely to make any measurable difference. However, in = principle I believe most if not all getsockopt()'s in TCP should be fine = with just a read lock, and for socket options used with UDP, there might = well be some benefit to using a read lock, since most UDP operations use = read locks and note write locks on the inpcb. I should further note that Jeff Roberson has some exciting in-progress = work to reduce transmit-input contention on the inpcb that appears to = make quite a noticeable difference in improving TCP performance. We = don't have much global lock contention currently when in the steady = state, but the per-connection locks do get heavily contended. His work = is similar to some work done in the Linux stack a year or two ago to = defer input processing to the user thread rather than contending on the = inpcb lock, if it's already held. Hopefully we'll see the results of = that work in 9.0, and possibly backported to 8.x. I also have a large pending patchset adding connection group support, = and aligning software lookup tables with hardware work distribution via = RSS, which is due to go in before 9.0. Robert=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?997B5A3A-8AC7-42F5-BE43-64B6CB4E2A25>