Date: Fri, 03 Aug 2007 09:30:27 +0200 From: Attilio Rao <attilio@FreeBSD.org> To: Jeff Roberson <jroberson@chesapeake.net> Cc: arch@freebsd.org, Alfred Perlstein <alfred@freebsd.org>, Kris Kennaway <kris@obsecurity.org> Subject: Re: Fine grain select locking. Message-ID: <46B2D993.6070409@FreeBSD.org> In-Reply-To: <20070802190033.J561@10.0.0.1> References: <20070702230728.E552@10.0.0.1> <20070703181242.T552@10.0.0.1> <20070704105525.GU45894@elvis.mu.org> <20070704114005.X552@10.0.0.1> <20070729180722.GB85196@rot26.obsecurity.org> <20070802174819.S561@10.0.0.1> <20070803014445.GS92956@elvis.mu.org> <20070802190033.J561@10.0.0.1>
next in thread | previous in thread | raw e-mail | index | archive | help
Jeff Roberson wrote: > > On Thu, 2 Aug 2007, Alfred Perlstein wrote: > >> * Jeff Roberson <jroberson@chesapeake.net> [070802 17:52] wrote: >>> >>> I believe filedescriptor locking is the place where we are most lacking. >>> The new sx helped tremendously. However, this is still going to be a >>> scalability limiter. I have looked into both linux and solaris's >>> solution >>> to this problem. Briefly, linux uses RCU to protect the list, which is >>> close to ideal as this is certainly a read heavy workload. Solaris >>> on the >>> other hand uses the actual file lock to protect the descriptor slot. So >>> they fetch the file pointer, lock it, and then check to see if they >>> lost a >>> race with the slot being reassigned while they were acquiring the lock. >>> This approach is perhaps better than rcu in many cases except when the >>> descriptor set is expanded. Then they have to lock every file in the >>> set. >> >> Certainly this is an extreme edge case... ? > > Well that may be true, yes. However, there are other problems with this > scheme. For example, flags settings could be done entirely with cmpset, > without using a lock at all. In most cases we're just setting a bit > which can be done with atomic_set. When we're doing multiple operations > we could compute the value and attempt to est it in a loop. So we can > totally eliminate locking the descriptor here. > > We also could use atomic ops to protect the file descriptor reference > count. This would eliminate another use of the FILE_LOCK(). I'm not > sure if it's possible to merge this with an approach that uses the > FILE_LOCK() to protect the descriptor table. Although I've not thought > it all the way through. > > If the ref count and flags were done with atomics the main consumer of > FILE_LOCK would actually be the unix domain socket garbage collection > code. How's that for old unix baggage. Do many programs actually pass > around descriptors these days? inetd? others? It might be worth it to > lock this seperately from the file lock. I'm sure I've alredy implemented it, but later I realized that there is a race with the p_fd field (if I got you right you are referring to the fdesc_mtx here), so we probabilly should better arrange those paths firstly. Thanks, Attilio
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?46B2D993.6070409>