From owner-freebsd-fs@FreeBSD.ORG Sun Feb 19 13:53:30 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9AFDC16A420 for ; Sun, 19 Feb 2006 13:53:30 +0000 (GMT) (envelope-from mirya@matrix.ua) Received: from gw.matrix.kiev.ua (gw.matrix.kiev.ua [213.159.235.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id 21F2B43D46 for ; Sun, 19 Feb 2006 13:53:29 +0000 (GMT) (envelope-from mirya@matrix.ua) Received: by gw.matrix.kiev.ua (Postfix, from userid 426) id 353A114BB2D; Sun, 19 Feb 2006 15:53:28 +0200 (EET) Received: from miryahost.matrix.local (unknown [192.168.1.27]) by gw.matrix.kiev.ua (Postfix) with ESMTP id DE0DA14BB20 for ; Sun, 19 Feb 2006 15:53:27 +0200 (EET) From: Kyryll A Mirnenko aka Mirya Organization: MATIX.ua To: freebsd-fs@freebsd.org Date: Sun, 19 Feb 2006 15:53:25 +0200 User-Agent: KMail/1.8.3 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200602191553.26338.mirya@matrix.ua> X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on gw.matrix.kiev.ua X-Spam-Level: X-Spam-Status: No, score=-100.6 required=5.0 tests=AWL,BAYES_50, USER_IN_WHITELIST autolearn=no version=3.0.2 Subject: RDBMS features into VFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2006 13:53:30 -0000 Consider this a feature request. THE PROBLEM The userland is missing "ORDER BY" and "LIMIT"-like features for directory reading. E.g. The only avaliable API is readdir() interating over the directory in unspecified order. On other hand most filebrowsers utilize file sorting extensively (mc, for, ex, will sort files by name as default option). When working with large directories this slowdowns operation much - e.g. the filebrowser must read huge filelist, then sort it and output only a small part of it - despite the fact some FS have BTREE index for filenames. Other potentional candidates for sorting are: - mtime - size - extattrs THE SOLUTION Add an api into VFS layer allowing to fetch a sorted filelist and limiting the output to a range from N to M, like it does "SELECT ... FROM ... ORDER BY field LIMIT a, b" in MySQL notation. Underlying FS'es are allowed not to add any optimized API for this operation, if so VFS implements it by itself THE USE Someone may criticise the exaple above as huge directories in REAL fs are exception, but the main application comes when talking about syntetic FS. After fusefs was ported to FreeBSD it became possible to represent some business logic as filesystem and "merge" it into the real one w/o digging in the kernel and thinking much about how a bug in your code can make the whole system crash. FS interface allows programmers not to care much about the user iface as the user is free to use any HTML editor he/she prefers to edit a piece of data provided by an application - reading and writing will be forwardrd to aprop. backend via fuse. The issue is most virtual "directories" in such systems will contain hundred thousand of "files" (like customers table) so while underlying RDBMS can effectively sort (the fields not representing standart file attributes are mapped to extattr) and chunk the data, the fs layer is not aware about that. -- Regards, Mirya ICQ #313898202