From owner-freebsd-questions@FreeBSD.ORG Sun Jul 26 01:19:46 2009 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AE00F106566B for ; Sun, 26 Jul 2009 01:19:46 +0000 (UTC) (envelope-from vogelke@hcst.com) Received: from beta.hcst.com (beta.hcst.com [192.52.183.241]) by mx1.freebsd.org (Postfix) with ESMTP id 6F5728FC14 for ; Sun, 26 Jul 2009 01:19:46 +0000 (UTC) (envelope-from vogelke@hcst.com) Received: from beta.hcst.com (localhost [127.0.0.1]) by beta.hcst.com (8.13.8/8.13.8/Debian-3) with ESMTP id n6Q1Jjf1014880 for ; Sat, 25 Jul 2009 21:19:45 -0400 Received: (from vogelke@localhost) by beta.hcst.com (8.13.8/8.13.8/Submit) id n6Q1JjHF014879; Sat, 25 Jul 2009 21:19:45 -0400 Received: by kev.msw.wpafb.af.mil (Postfix, from userid 32768) id AC51DB7E0; Sat, 25 Jul 2009 18:29:18 -0400 (EDT) To: freebsd-questions@freebsd.org In-reply-to: <200907231025.49916.mel.flynn+fbsd.questions@mailing.thruhere.net> (message from Mel Flynn on Thu, 23 Jul 2009 10:25:49 -0800) Organization: Oasis Systems Inc. X-Disclaimer: I don't speak for the USAF or Oasis. X-GPG-ID: 1024D/711752A0 2006-06-27 Karl Vogel X-GPG-Fingerprint: 56EB 6DBF 4224 C953 F417 CC99 4C7C 7D46 7117 52A0 Message-Id: <20090725222918.AC51DB7E0@kev.msw.wpafb.af.mil> Date: Sat, 25 Jul 2009 18:29:18 -0400 (EDT) From: vogelke+unix@pobox.com (Karl Vogel) Subject: Re: limit to number of files seen by ls? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vogelke+unix@pobox.com List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Jul 2009 01:19:46 -0000 >> On Thursday 23 July 2009 09:41:26 Karl Vogel wrote: K> Every version of Unix I've ever used had an upper limit on the size of K> the argument list you could pass to a program, so it won't just be "ls" K> that's affected here. That's why I use 1,000 as a rule of thumb for the K> maximum number of files I put in a directory. >> On Thu, 23 Jul 2009 10:25:49 -0800, >> Mel Flynn said: M> That arbitrary number works simply because kern.argmax default has been M> raised somewhere in 6.x (before it was 64kB). That arbitrary number has worked very nicely for me for 20 years under Solaris, Linux, and several BSD variants. The main reason I stick with 1000 is because directories are read linearly unless you're using something like ReiserFS, and I get impatient waiting for more than that number of filenames to be sorted when using ls. M> And MAXNAMLEN in sys/dirent.h is 255. That's the maximum length of a single filename in a directory. Since I keep my filenames much shorter, I don't have a problem. M> Knowing your way around maximum arguments length through xargs as M> suggested in this thread is much better solution then trying to exercise M> control over directory sizes, which may or not be under your control in M> the first place. Xargs is very useful, but it's not a substitute for poor design, and it's not something you can drop into any existing pipeline without a little thought first. If your application is trying to create hundreds of thousands or millions of files in any one directory, or you're creating lots of 200-character filenames from hell, then your design is a poor match for most varieties of Unix; small directories perform better than enormous ones, and lots of commonly-used scripts and programs will fall over when handed zillion-file argument lists. I'm sure the latest version of fixes all these objections, but not everyone gets to run the latest and greatest. Don't fight your filesystem, and it won't fight you. -- Karl Vogel I don't speak for the USAF or my company Birds of a feather flock together and usually crap on your car.