Date: Mon, 12 Sep 2005 20:51:44 -0400 From: Kris Kennaway <kris@obsecurity.org> To: Nikolai Schupbach <nikolai@net24.co.nz> Cc: freebsd-stable@freebsd.org Subject: Re: Long Format Directory Listing 15x Slower in FreeBSD 5.x Message-ID: <20050913005144.GA46178@xor.obsecurity.org> In-Reply-To: <4326158B.8010502@net24.co.nz> References: <4326158B.8010502@net24.co.nz>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] On Tue, Sep 13, 2005 at 11:55:55AM +1200, Nikolai Schupbach wrote: > Hello, > > We have been trying to migrate to FreeBSD 5.4 from an older 4.x release > for one of our busy mail servers. But we have encountered problems with > directory listings on 5.4. > > Our /var/mail directory contains approximately 8,000 files doing a long > directory listing (ls -l) takes approximately 5 min and during this time > the CPU is running near 100%, on a FSBD 4 box the same directory > contents takes less than 20 seconds to list. Even on a directory with > 200 files, each file with a different owner, it will still take at least > 4-5 seconds to list. The problems only seems to occur when the directory > contains files from many different users. (as in /var/mail). If I chown > all the files in the /var/mail directory to a single user the directory > listing is near instant. > > It appears it has something to do with 'ls' looking up the id's in the > password database, because if I instruct ls to display numeric IDs (ls > -ln), rather than converting to user and group names, the directory with > 8,000 files, with 8,000 different owners will list instantly. > > The reason this concerns me so much is we are also having a problem with > our Washington IMAP / POP3 server on FBSD 5 using a lot of CPU while > operating on small and even 0 byte mailboxes, when there are approx five > or more concurrent POP3 or IMAP sessions. And I can't help thinking that > the two problems are related. > > Does anyone have any ideas? Has anyone else noticed this problem? If I > can't resolve it I'm most likely going to revert to using 4.11, but I'd > really like to know what is going on. (and yes I'm using UFS_DIRHASH) Sounds like the real problem is that you have >8000 users on your machine and lookups are taking a long time. There has been discussion of this problem and how to solve it on freebsd-questions@ several times..I think it involves making a minor change to pwd_mkdb and rebuilding. Please search the archives for that mailing list. Kris [-- Attachment #2 --] -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFDJiKgWry0BWjoQKURAti+AJ9PH0eCgZr0H2im7SOBkdAnd0/LBwCgt27s WnPqJANxqvJg8cIklPGmB60= =+OT5 -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20050913005144.GA46178>