From owner-freebsd-fs Mon May 6 8:40:30 2002 Delivered-To: freebsd-fs@freebsd.org Received: from sluis.cs.vu.nl (sluis.cs.vu.nl [130.37.192.174]) by hub.freebsd.org (Postfix) with ESMTP id 5D93337B408 for ; Mon, 6 May 2002 08:40:16 -0700 (PDT) Received: from centaur.cs.vu.nl (centaur.cs.vu.nl [192.35.191.4]) by sluis.cs.vu.nl with esmtp (Smail #74) id m174kb0-000OesC; Mon, 6 May 2002 17:40 +0200 Received: from cs.vu.nl (localhost [127.0.0.1]) by centaur.cs.vu.nl with esmtp (Smail #74) id m174kb0-0014NkC; Mon, 6 May 2002 17:40 +0200 Message-Id: To: fs@FreeBSD.ORG Subject: Re: Filesystem References: <200205040019.UAA13780@illustrious.cnchost.com> <3CD32F43.327CDA46@mindspring.com> <20020504041936.GA19646@quic.net> <3CD3FB02.3EC1DA29@mindspring.com> <20020505084827.GA3688@quic.net> In-reply-to: Your message of "Sun, 5 May 2002 04:48:27 -0400 ." <20020505084827.GA3688@quic.net> Date: Mon, 06 May 2002 17:40:14 +0200 From: Philip Homburg Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org In your letter dated Sun, 5 May 2002 04:48:27 -0400 utsl@quic.net wrote: >On Sat, May 04, 2002 at 08:15:14AM -0700, Terry Lambert wrote: >> My take on an application that doesn't scale is that "fixing" the >> application by changing the behaviour of the underlying system is >> just propping up bad code. Bad code deserves to lose. So if >> someone wrote an application like that, it's just as well that the >> programmer who failed to consider scaling issues lose out to the >> programmer who considered them. After all, it's very likely that >> the failure to consider scaling issues is more of an "all or nothing" >> thing, and that the failure to consider one means that solving it in >> the OS will just expose the next one. There's really no way you >> can make the OS behave perfectly for all applications. At some >> point, applications programmers will have to learn how to program, >> or all bets are off. > >Yes. Most people that supported the application I described would have >liked to catch the application programmers in a dark alley. People who >put 100,000 files in a single directory deserve what happens to them, >IMHO. > >However, it is nice to have the tools to do things right. Given how >common this particulary problem seems to be, I think a library might be >a good idea. I may write one, I'm working on an application now that >needs to be able to scale to at least 1M files. :( From an O.S. point of view, what's the problem with providing large directories? Some tools, such as ls and find, may need some attention due to scaling issues. Backup software is needed to handle the new filesystem layout. If somebody would like to create 1M files, and you know that, when properly implemented, directories accesses cost O(log(n)), then what's the problem? I know from experience that implementing a directory as a btree is trivial if lower layers of the filesystem have (rudimentary) transaction support. I don't want to think about crash recovery of btrees in a normal FFS filesystem. Philip Homburg To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message