From owner-freebsd-questions@FreeBSD.ORG Tue Mar 29 01:25:38 2005 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 85EE616A4CE for ; Tue, 29 Mar 2005 01:25:38 +0000 (GMT) Received: from smtpout.eastlink.ca (smtpout.eastlink.ca [24.222.0.30]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2F98943D41 for ; Tue, 29 Mar 2005 01:25:38 +0000 (GMT) (envelope-from fairwinds@eastlink.ca) Received: from eastlink.ca ([24.224.224.109]) by mx3.eastlink.ca (iPlanet Messaging Server 5.2 Patch 2 (built Jul 14 2004)) with ESMTP id <0IE3009Z8AMPPM@mx3.eastlink.ca> for FreeBSD-Questions@FreeBSD.org; Mon, 28 Mar 2005 21:25:37 -0400 (AST) Date: Mon, 28 Mar 2005 21:25:28 -0400 From: David Pratt In-reply-to: <20050324193456.GB31083@Grumpy.DynDNS.org> To: FreeBSD-Questions@FreeBSD.org Message-id: <6EDD6E31-9FF1-11D9-8CB8-000A27B3B070@eastlink.ca> MIME-version: 1.0 X-Mailer: Apple Mail (2.553) Content-type: text/plain; charset=US-ASCII; format=flowed Content-transfer-encoding: 7BIT Subject: Re: Max files in unix folder from PIL process X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Mar 2005 01:25:38 -0000 Hi. I am creating a python application that uses PIL to generate thumbnails and sized images. It is beginning to look the volume of images will be large. This has got me to thinking. Is there a number that Unix can handle in a single directory. I am using FreeBSD4.x at the moment. I am thinking the number could be as high 500,000 images in a single directory but more likely in the range of 6,000 to 30,000 for most. I did not want to store these in Postgres. I will most likely to break these into directories by size ie. thumbnail, small, medium, large, etc. . That will at least take it down by a factor of the number of sizes used but still the possibility of a very large number (maximum to perhaps 100,000 or more) There is really no other way that I can think of to categorize these at the moment. Should this pose a problem on the filesystem? How will it affect the use of Unix tools? Will there be access problems that affect speed? This is unchartered territory for me so hope someone who has been there, done that can provide some of what they learned from experience. Many thanks. Regards, David