Date: Mon, 21 May 2007 12:16:33 -0700 (PDT) From: Gore Jarold <gore_jarold@yahoo.com> To: Brooks Davis <brooks@freebsd.org> Cc: freebsd-fs@freebsd.org Subject: Re: VERY frustrated with FreeBSD/UFS stability - please help or comment... Message-ID: <475187.33232.qm@web63006.mail.re1.yahoo.com> In-Reply-To: <20070521174818.GA64826@lor.one-eyed-alien.net>
next in thread | previous in thread | raw e-mail | index | archive | help
--- Brooks Davis <brooks@freebsd.org> wrote: > > a) am I really the only person in the world that > moves > > around millions of inodes throughout the day ? Am > I > > the only person in the world that has ever filled > up a > > snapshotted FS (or a quota'd FS, for that matter) > ? > > Am I the only person in the world that does a mass > > deletion of several hundred thousand inodes > several > > times per day ? > > > > OR: > > > > b) am I just stupid ? Is everyone doing this, and > > there is 3 pages of sysctls and kernel tunes that > > everyone does to their system when they are going > to > > use it this way ? Am I just naive for taking a > > release and paring down GENERIC and attempting to > run > > as-is out of the box without major tuning ? > > > > If so, can I see those tunes/sysctls ? > > > > I am _really_ hoping that it is (b) ... I would > much > > rather look back on all of this frustration as my > own > > fault than have the burden of proving all of this > (as > > I will no doubt be called upon to do). (1) > > > > Thanks. Please add your comments... > > I'd say it's certaintly (a). Consider that a full > source tree contains > a few under 85K files so that's a reasionable bound > on average > workloads. Deliberatly producing a kernel that > required tuning to just > us the APIs without crashing would be stupid and we > wouldn't go it > without a very good reason and very large warnings > all over the place. > Lousy performance might be expected, but crashing > wouldn't be. Ok - your initial comments / impression are reassuring. It's hard to believe that the simple file movements I do are so alien to mainstream use, but I'll accept your judgement. > > (1) just load up 6.2 and cp/rm a few million > inodes > > around. Or turn on quotas and fill your > filesystem > > up. Kaboom. > > It's not clear to me what you mean by "cp/rm a few > million inodes > around." The organization of those inodes into > files and directories > could conceviably have a major impact on the > problem. If you could > provide a script that fails for you, that would > really help. Specifically, I have private departmental fileservers that other fileservers rsync to using Mike Rubel-style rsync snapshots: http://www.mikerubel.org/computers/rsync_snapshots/ This means that the remote system runs a script like this: ssh user@host rm -rf backup.2 ssh user@host mv backup.1 backup.2 ssh user@host cp -al backup.0 backup.1 rsync /files user@host:/backup.0 The /files in question range from .2 to 2.2 million files, all told. This means that when this script runs, it first either deletes OR unlinks up to 2 million items. Then it does a (presumably) zero cost move operation. Then it does a hard-link-creating cp of the same (up to 2 million) items. As I write this, I realize this isn't _totally_ generic, since I am using GNU cp rather than the built-in FreeBSD cp, but that is _truly_ the extent of customization on this system. So that's that. Run that a few times from a few servers and 6.2 locks up. Can't ping. For trivias sake, it does the same thing if you fill up one of its filesystems with quotas turned on. For further trivias sake, 6.1 is (seemingly) not susceptible to this, but it is certainly susceptible to other situations that I regularly produce. What do you think of this (more specific) workload, and are there any tunings that immediately jump to mind ? ____________________________________________________________________________________Yahoo! oneSearch: Finally, mobile search that gives answers, not web links. http://mobile.yahoo.com/mobileweb/onesearch?refer=1ONXIC
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?475187.33232.qm>