From owner-freebsd-fs@FreeBSD.ORG Mon Oct 22 16:00:41 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CAC1A16A420; Mon, 22 Oct 2007 16:00:41 +0000 (UTC) (envelope-from lulf@stud.ntnu.no) Received: from fri.itea.ntnu.no (fri.itea.ntnu.no [129.241.7.60]) by mx1.freebsd.org (Postfix) with ESMTP id 7E5C813C48E; Mon, 22 Oct 2007 16:00:41 +0000 (UTC) (envelope-from lulf@stud.ntnu.no) Received: from localhost (localhost [127.0.0.1]) by fri.itea.ntnu.no (Postfix) with ESMTP id 241468E09; Mon, 22 Oct 2007 18:00:31 +0200 (CEST) Received: from caracal.stud.ntnu.no (caracal.stud.ntnu.no [129.241.56.185]) by fri.itea.ntnu.no (Postfix) with ESMTP; Mon, 22 Oct 2007 18:00:30 +0200 (CEST) Received: by caracal.stud.ntnu.no (Postfix, from userid 2312) id 50A4C624102; Mon, 22 Oct 2007 18:00:41 +0200 (CEST) Date: Mon, 22 Oct 2007 18:00:41 +0200 From: Ulf Lilleengen To: Kris Kennaway Message-ID: <20071022160041.GA12710@stud.ntnu.no> References: <20070816100526.GA31897@stud.ntnu.no> <20071017220948.GA4279@stud.ntnu.no> <47170C54.6050502@FreeBSD.org> <20071018083448.GA1079@stud.ntnu.no> <47171D89.7030906@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <47171D89.7030906@FreeBSD.org> User-Agent: Mutt/1.5.9i X-Content-Scanned: with sophos and spamassassin at mailgw.ntnu.no. Cc: freebsd-fs@freebsd.org Subject: Re: [PATCH] Make fdescfs MPSAFE X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2007 16:00:41 -0000 On tor, okt 18, 2007 at 10:47:05 +0200, Kris Kennaway wrote: > Ulf Lilleengen wrote: > >On tor, okt 18, 2007 at 09:33:40 +0200, Kris Kennaway wrote: > >>Ulf Lilleengen wrote: > >>>On tor, aug 16, 2007 at 12:05:26 +0200, Ulf Lilleengen wrote: > >>>>Hi, > >>>> [...] > >>>http://folk.ntnu.no/lulf/patches/freebsd/fdescfs/fdescfs_lock.diff > >>This might be OK but you should be aware that rwlocks can be slower than > >>mutexes when there is a suitably mixed read/write workload. We don't do > >>the same adaptive spinning for wlocks as for mutexes when they are held > >>by shared holders (since we don't track who they are so can't track > >>whether they're running), and it is possible for readers to starve > >>writers. > >> > >>If possible some benchmarks trying to find the worst case behaviour > >>would be useful. > >Good point. I guess having a benchmark where one would open #hashentries > >files, and then have #hashentries threads reading (accessing the files) and > >one thread writing (closing perhaps) could produce the starvation? > > Perhaps, give it a try :) > I created the program found here: http://folk.ntnu.no/lulf/dev/freebsd/fdescfs/benchmark.c I compiled it with -lthr, and the hardware I've tested it on is a core duo, as well as a dual-cpu p3 machine. I tried opening X files, then having X "readers" doing a lookup on the filedescriptors for those files in /dev/fd (Doing a lookup for a first time puts it into the hash table (using wlock). Looking it up the second time, it will just use rlock). While these X readers was busy reading, tried to open a new file and doing lookup on it (will try to insert and acquire wlock). I tried with X = 1, 2, 4 and 16. I was unable to produce the starvation with this. I'll try see if I can come up with some other benchmark. I assume it would be easier to reproduce it if I have CPUs I can run threads on to assure I always hold it, but I don't. Also, as a general way to observe behaviour in benchmarks, is there a way I could force/check what CPU a thread is running on from userland? -- Ulf Lilleengen