From owner-freebsd-fs@FreeBSD.ORG Mon May 21 18:27:34 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E6D4E16A478 for ; Mon, 21 May 2007 18:27:33 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (grnl-static-02-0046.dsl.iowatelecom.net [69.66.56.110]) by mx1.freebsd.org (Postfix) with ESMTP id 6C5A313C4C4 for ; Mon, 21 May 2007 18:27:33 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.13.8/8.13.8) with ESMTP id l4LHmJfu065632; Mon, 21 May 2007 12:48:19 -0500 (CDT) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.13.8/8.13.8/Submit) id l4LHmJAJ065631; Mon, 21 May 2007 12:48:19 -0500 (CDT) (envelope-from brooks) Date: Mon, 21 May 2007 12:48:19 -0500 From: Brooks Davis To: Gore Jarold Message-ID: <20070521174818.GA64826@lor.one-eyed-alien.net> References: <829849.56057.qm@web63013.mail.re1.yahoo.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="VbJkn9YxBvnuCH5J" Content-Disposition: inline In-Reply-To: <829849.56057.qm@web63013.mail.re1.yahoo.com> User-Agent: Mutt/1.5.15 (2007-04-06) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (lor.one-eyed-alien.net [127.0.0.1]); Mon, 21 May 2007 12:48:19 -0500 (CDT) Cc: freebsd-fs@freebsd.org Subject: Re: VERY frustrated with FreeBSD/UFS stability - please help or comment... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 May 2007 18:27:34 -0000 --VbJkn9YxBvnuCH5J Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, May 21, 2007 at 09:19:02AM -0700, Gore Jarold wrote: >=20 > I have been extremely dissatisfied with the stability > of the FreeBSD UFS/UFS2 implementation throughout all > of 5.x and 6.x. I can take any random release from > this period and halt/lock/crash it with basic and > uninteresting filesystem operations. In early 5.x it > was simply big inode movements from disk to disk.=20 > Later it was multiple snapshots. In 6.x any number of > seemingly benign operations (filling a disk, using > quotas, dense (lots of inodes) deletes and copies) and > each time some or all of these problems are solved in > one release, slightly different versions of the same > problem show up in the next one. For instance, > snapshot stability got a lot better from 6.0 to 6.2, > but now 6.2 has problems just moving and deleting lots > of inodes. >=20 > I am running _nothing interesting_. I don't even run > snapshots anymore ... and as you can see from other > posts, I am crashing/halting/etc. all over the place.=20 > This is on systems that do nothing but TCP (scp, ftp) > file service and some big (rm, cp) movements of inodes > once in a while. In other words, my setup is as > vanilla as it gets. >=20 > It's not the hardware (happens on 3ware, adaptec, > etc.) (different systems). >=20 > It's not my esoteric config (I take generic kernel and > just delete the devices I don't use) ... also I have > no custom sysctls/loader.conf >=20 > So I am at my wits end. Since early 2004 there has > not been a single release version of FreeBSD (well, > except _perhaps_ 6.1-RELEASE ...) that I don't > regularly knock over with _simple, generic movements > of files_. >=20 > So my plea for help is as follows: >=20 > a) am I really the only person in the world that moves > around millions of inodes throughout the day ? Am I > the only person in the world that has ever filled up a > snapshotted FS (or a quota'd FS, for that matter) ?=20 > Am I the only person in the world that does a mass > deletion of several hundred thousand inodes several > times per day ? >=20 > OR: >=20 > b) am I just stupid ? Is everyone doing this, and > there is 3 pages of sysctls and kernel tunes that > everyone does to their system when they are going to > use it this way ? Am I just naive for taking a > release and paring down GENERIC and attempting to run > as-is out of the box without major tuning ? >=20 > If so, can I see those tunes/sysctls ? >=20 > I am _really_ hoping that it is (b) ... I would much > rather look back on all of this frustration as my own > fault than have the burden of proving all of this (as > I will no doubt be called upon to do). (1) >=20 > Thanks. Please add your comments... I'd say it's certaintly (a). Consider that a full source tree contains a few under 85K files so that's a reasionable bound on average workloads. Deliberatly producing a kernel that required tuning to just us the APIs without crashing would be stupid and we wouldn't go it without a very good reason and very large warnings all over the place. Lousy performance might be expected, but crashing wouldn't be. > (1) just load up 6.2 and cp/rm a few million inodes > around. Or turn on quotas and fill your filesystem > up. Kaboom. It's not clear to me what you mean by "cp/rm a few million inodes around." The organization of those inodes into files and directories could conceviably have a major impact on the problem. If you could provide a script that fails for you, that would really help. -- Brooks --VbJkn9YxBvnuCH5J Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFGUdtiXY6L6fI4GtQRAieyAKCCJvWlq6utDMgzvpSz+UpVG4dQ2ACfRNzl Y+PAEfyzR23shzuDLvxMR5w= =h8VM -----END PGP SIGNATURE----- --VbJkn9YxBvnuCH5J--