From owner-freebsd-fs@FreeBSD.ORG Sun Jan 16 21:44:05 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E34D816A4CE for ; Sun, 16 Jan 2005 21:44:05 +0000 (GMT) Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.192]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9226643D4C for ; Sun, 16 Jan 2005 21:44:05 +0000 (GMT) (envelope-from chiahsing@gmail.com) Received: by wproxy.gmail.com with SMTP id 68so675299wri for ; Sun, 16 Jan 2005 13:44:05 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding; b=FmuZbAERh6CNCOt0ffp4KCNwQaSsjVaDDEkP6BfncoNEUeI9cz1u8wn2biS3qfX+u1xpSIKaI1WuzMLoRPwIeDKoFSQ7Ki0nBqnKcuGkEfFG9Q3CNkz1mn711mkKqvNkHr1D/wfWrChV2XX0bnO6HlbHQXVjMlcSuXHoi/1vMnU= Received: by 10.54.32.35 with SMTP id f35mr156378wrf; Sun, 16 Jan 2005 13:44:04 -0800 (PST) Received: by 10.54.31.21 with HTTP; Sun, 16 Jan 2005 13:44:04 -0800 (PST) Message-ID: Date: Sun, 16 Jan 2005 13:44:04 -0800 From: David Yu To: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: NTFS unicode converting problem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: David Yu List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Jan 2005 21:44:06 -0000 Hi, it seems that NTFS in FreeBSD uses a 16-bit long wchar to store filename. When I wanted to convert some Chinese filename into UTF-8, the conversion was failed because a single Chinese character needs 3 bytes in UTF-8. Is anyone already working on this problem? If not, I would like to do something about it. Any suggestions? David From owner-freebsd-fs@FreeBSD.ORG Mon Jan 17 02:25:55 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4EDFD16A4CE for ; Mon, 17 Jan 2005 02:25:55 +0000 (GMT) Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.200]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6429643D3F for ; Mon, 17 Jan 2005 02:25:54 +0000 (GMT) (envelope-from chiahsing@gmail.com) Received: by wproxy.gmail.com with SMTP id 68so690165wri for ; Sun, 16 Jan 2005 18:25:53 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=rBsAioiUN7SUoILomxWUv/zVhcowg1+NR1vSzbQvM2iOGNj0R8w1ePojBG5unW+Xe3gyQLufSdOAvb/8yTIB7Z55JfgPw1ia18izH1de2KpupfsB/3Ggc2UO37OeSJVB3tMMyApW2hvLg1J48Lt2CL23EwPTbe6tvZQ7JWrjhj8= Received: by 10.54.52.40 with SMTP id z40mr71151wrz; Sun, 16 Jan 2005 18:25:52 -0800 (PST) Received: by 10.54.31.21 with HTTP; Sun, 16 Jan 2005 18:25:52 -0800 (PST) Message-ID: Date: Sun, 16 Jan 2005 18:25:52 -0800 From: David Yu To: Christopher Vance In-Reply-To: <20050117020039.GB630@nu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <20050117020039.GB630@nu.org> cc: freebsd-fs@freebsd.org cc: freebsd-current@freebsd.org Subject: Re: NTFS unicode converting problem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: David Yu List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Jan 2005 02:25:55 -0000 In my case, those Chinese filenames are still in UCS-2LE, and all characters are in plane 0. I tried to modify codes so that it can directly store the conversion result into the dirent structure, but the convchr() funtion failed everytime for Chinese character while there is no problem with Ascii characters. I thought converting from UCS-2 to UTF-8 should be very easy? On Mon, 17 Jan 2005 13:00:39 +1100, Christopher Vance wrote: > On Sun, Jan 16, 2005 at 01:44:04PM -0800, David Yu wrote: > >Hi, it seems that NTFS in FreeBSD uses a 16-bit long wchar to store > >filename. When I wanted to convert some Chinese filename into UTF-8, > >the conversion was failed because a single Chinese character needs 3 > >bytes in UTF-8. Is anyone already working on this problem? If not, I > >would like to do something about it. Any suggestions? > > From memory, old Windows used UCS-2, while newer Windows uses UTF-16. > Was the bad character in plane 0 or higher? > > -- > Christopher Vance > From owner-freebsd-fs@FreeBSD.ORG Mon Jan 17 05:28:21 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E970A16A4CE for ; Mon, 17 Jan 2005 05:28:21 +0000 (GMT) Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.205]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7574943D45 for ; Mon, 17 Jan 2005 05:28:21 +0000 (GMT) (envelope-from chiahsing@gmail.com) Received: by wproxy.gmail.com with SMTP id 68so700847wri for ; Sun, 16 Jan 2005 21:28:20 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=CvYPal2nDlp9tRlNbzAI1FNClwPoSA/qUWFqQ7nPQ0y0aBlVYwFXnaO7lSv2wJfERufC76rDz1zNxpDAll1j/BtliHYRPFZWT8garqfVmFuUNi7gWWU9NSPgH1e0E3MeAGQHPkC0+CFJZ3AtixZJeag8rG7LqVrOELKGzy7hCyI= Received: by 10.54.40.34 with SMTP id n34mr131592wrn; Sun, 16 Jan 2005 21:28:20 -0800 (PST) Received: by 10.54.31.21 with HTTP; Sun, 16 Jan 2005 21:28:20 -0800 (PST) Message-ID: Date: Sun, 16 Jan 2005 21:28:20 -0800 From: David Yu To: Christopher Vance In-Reply-To: <20050117032255.GC630@nu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <20050117020039.GB630@nu.org> <20050117032255.GC630@nu.org> cc: freebsd-fs@freebsd.org cc: freebsd-current@freebsd.org Subject: Re: NTFS unicode converting problem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: David Yu List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Jan 2005 05:28:22 -0000 The problem is that I have to default XLAT16_ACCEPT_3BYTE_CHR in both libkiconv and module/libiconv.ko. Though it works, I don't it is a good way to use translation table since UCS-2 to UTF-8 is so straightforward. Maybe we need to write a new kernel side iconv converter for this? I have no idea about those iconv codes in the kernel at all. On Mon, 17 Jan 2005 14:22:55 +1100, Christopher Vance wrote: > On Sun, Jan 16, 2005 at 06:25:52PM -0800, David Yu wrote: > >In my case, those Chinese filenames are still in UCS-2LE, and all > >characters are in plane 0. I tried to modify codes so that it can > >directly store the conversion result into the dirent structure, but > >the convchr() funtion failed everytime for Chinese character while > >there is no problem with Ascii characters. I thought converting from > >UCS-2 to UTF-8 should be very easy? > > Maybe the conversion routine disagrees with you about byte order? > > -- > Christopher Vance > From owner-freebsd-fs@FreeBSD.ORG Mon Jan 17 16:01:35 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D215116A4CE; Mon, 17 Jan 2005 16:01:35 +0000 (GMT) Received: from elrond.anembo.nu.org (dsl-202-173-130-73.nsw.westnet.com.au [202.173.130.73]) by mx1.FreeBSD.org (Postfix) with ESMTP id DC6BE43D2F; Mon, 17 Jan 2005 16:01:33 +0000 (GMT) (envelope-from cjsv@elrond.anembo.nu.org) Received: from elrond.anembo.nu.org (localhost [127.0.0.1]) by elrond (8.13.1/8.13.1) with ESMTP id j0H3Mvvj002376; Mon, 17 Jan 2005 14:22:57 +1100 (EST) (envelope-from cjsv@elrond.anembo.nu.org) Received: (from cjsv@localhost) by elrond.anembo.nu.org (8.13.1/8.13.1/Submit) id j0H3MtlY002375; Mon, 17 Jan 2005 14:22:55 +1100 (EST) (envelope-from cjsv) Date: Mon, 17 Jan 2005 14:22:55 +1100 From: Christopher Vance To: David Yu Message-ID: <20050117032255.GC630@nu.org> References: <20050117020039.GB630@nu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: cc: freebsd-fs@freebsd.org cc: freebsd-current@freebsd.org Subject: Re: NTFS unicode converting problem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Jan 2005 16:01:36 -0000 On Sun, Jan 16, 2005 at 06:25:52PM -0800, David Yu wrote: >In my case, those Chinese filenames are still in UCS-2LE, and all >characters are in plane 0. I tried to modify codes so that it can >directly store the conversion result into the dirent structure, but >the convchr() funtion failed everytime for Chinese character while >there is no problem with Ascii characters. I thought converting from >UCS-2 to UTF-8 should be very easy? Maybe the conversion routine disagrees with you about byte order? -- Christopher Vance From owner-freebsd-fs@FreeBSD.ORG Mon Jan 17 16:01:37 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 177FF16A4CE; Mon, 17 Jan 2005 16:01:37 +0000 (GMT) Received: from elrond.anembo.nu.org (dsl-202-173-130-73.nsw.westnet.com.au [202.173.130.73]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3942543D1D; Mon, 17 Jan 2005 16:01:36 +0000 (GMT) (envelope-from cjsv@elrond.anembo.nu.org) Received: from elrond.anembo.nu.org (localhost [127.0.0.1]) by elrond (8.13.1/8.13.1) with ESMTP id j0H20fJ3002089; Mon, 17 Jan 2005 13:00:41 +1100 (EST) (envelope-from cjsv@elrond.anembo.nu.org) Received: (from cjsv@localhost) by elrond.anembo.nu.org (8.13.1/8.13.1/Submit) id j0H20dml002088; Mon, 17 Jan 2005 13:00:39 +1100 (EST) (envelope-from cjsv) Date: Mon, 17 Jan 2005 13:00:39 +1100 From: Christopher Vance To: David Yu Message-ID: <20050117020039.GB630@nu.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: cc: freebsd-fs@freebsd.org cc: freebsd-current@freebsd.org Subject: Re: NTFS unicode converting problem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Jan 2005 16:01:37 -0000 On Sun, Jan 16, 2005 at 01:44:04PM -0800, David Yu wrote: >Hi, it seems that NTFS in FreeBSD uses a 16-bit long wchar to store >filename. When I wanted to convert some Chinese filename into UTF-8, >the conversion was failed because a single Chinese character needs 3 >bytes in UTF-8. Is anyone already working on this problem? If not, I >would like to do something about it. Any suggestions? >From memory, old Windows used UCS-2, while newer Windows uses UTF-16. Was the bad character in plane 0 or higher? -- Christopher Vance From owner-freebsd-fs@FreeBSD.ORG Tue Jan 18 03:09:14 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3517116A4CE for ; Tue, 18 Jan 2005 03:09:14 +0000 (GMT) Received: from web17208.mail.tpe.yahoo.com (web17208.mail.tpe.yahoo.com [202.43.200.117]) by mx1.FreeBSD.org (Postfix) with SMTP id B5AD243D1D for ; Tue, 18 Jan 2005 03:09:13 +0000 (GMT) (envelope-from laface.tw@yahoo.com.tw) Message-ID: <20050118030922.82458.qmail@web17208.mail.tpe.yahoo.com> Received: from [220.130.160.4] by web17208.mail.tpe.yahoo.com via HTTP; Mon, 17 Jan 2005 19:09:22 PST Date: Mon, 17 Jan 2005 19:09:22 -0800 (PST) From: Edward To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailman-Approved-At: Tue, 18 Jan 2005 13:29:16 +0000 Subject: free block bitmap size in FFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jan 2005 03:09:14 -0000 Dear all, I have a question about free block bitmap in FFS. I can get free block map in cg. But how to compute size of the bitmap ? Thanks you in advance. Best regards, Edward __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-fs@FreeBSD.ORG Wed Jan 19 10:06:14 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3057B16A4CE; Wed, 19 Jan 2005 10:06:14 +0000 (GMT) Received: from as.itesm.mx (as.itesm.mx [200.34.200.251]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7860C43D2D; Wed, 19 Jan 2005 10:06:13 +0000 (GMT) (envelope-from ccardena@itesm.mx) X-Ironport-AV: i="3.88,137,1102312800"; d="scan'208"; a="47693860:sNHT12885628" Received: from [137.194.164.47] by itesm.mx with HTTP; Wed, 19 Jan 2005 04:06:11 -0600 Date: Wed, 19 Jan 2005 11:06:11 +0100 Message-ID: <41EE0B1E00000045@mailserver1.itesm.mx> In-Reply-To: <41E11BD300003B8D@mailserver3.itesm.mx> From: =?ISO-8859-15?Q?C=E9sar=20C=E1rdenas?= To: freebsd-scsi@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-15" Content-Transfer-Encoding: quoted-printable cc: freebsd-fs@freebsd.org cc: freebsd-performance@freebsd.org cc: freebsd-i386@freebsd.org Subject: Turning off a partition X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: ccardena@itesm.mx List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2005 10:06:14 -0000 Dear all, I am using FreeBSD 4.5 Release P20... In order to increase slice space because I have "File System Full Errors"= . I need to turn off (inactiving) device "ad0s1e"... How I can do that? I appreciate any help on this issue, have a nice day, C=E9sar C=E1rdenas (ccardena@itesm.mx) Electronics and Computer Science Department Monterrey Tech, Quer=E9taro Campus http://www.qro.itesm.mx Personal Phone: +(33) 625249469 Office Phone: +(33) 145817146 Office Fax: +(33) 145817158 All phones and fax from abroad France The content of this data transmission is not considered as an offer, prop= osal, understanding, or agreement unless it is confirmed in a document signed by a legal representative of ITESM. The content of this data transmission= is confidential and it is intended to be delivered only to the addresses,= therefore, it shall not be distributed and/or disclosed through any mean without the original sender's previous authorization. If you are not the addressee you are forbidden to use it, either totally or partially, for any purpose. From owner-freebsd-fs@FreeBSD.ORG Thu Jan 20 02:00:00 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F2CB516A4CE for ; Thu, 20 Jan 2005 01:59:59 +0000 (GMT) Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.202]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2BC4943D58 for ; Thu, 20 Jan 2005 01:59:58 +0000 (GMT) (envelope-from chiahsing@gmail.com) Received: by wproxy.gmail.com with SMTP id 70so39830wra for ; Wed, 19 Jan 2005 17:59:57 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=EIXRhWi+s9IRtAVhkk43lGjZKnkMVnYj+yCSOO/7ygBmcFb4NxFsvSChBIkdUN9K1XrDZEGOaj9RgjmgyDhsympXL8Mo0WsLBEtzrJjmNzJJaQx0j9FeavwkDNpLm39bK12C0Uk5WTfOxAHGhgXDt+E5I8Jk7cBHu+SkB2EhPzY= Received: by 10.54.13.43 with SMTP id 43mr287799wrm; Wed, 19 Jan 2005 17:59:57 -0800 (PST) Received: by 10.54.31.21 with HTTP; Wed, 19 Jan 2005 17:59:57 -0800 (PST) Message-ID: Date: Wed, 19 Jan 2005 17:59:57 -0800 From: David Yu To: freebsd-fs@freebsd.org, freebsd-current@freebsd.org In-Reply-To: Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <20050117020039.GB630@nu.org> <20050117032255.GC630@nu.org> Subject: Re: NTFS unicode converting problem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: David Yu List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jan 2005 02:00:00 -0000 I wrote a patch for mounting NTFS as UTF-8. The patch is at http://www.cse.ucsd.edu/~chyu/ntfs-utf8.diff . After patch, you need to recompile the kernel module libiconv and ntfs (or the whole kernel if you do not use kernel module). This patch should solve the problem that the original libiconv in the kernel cannot convert characters to UTF-8 longer than 2 bytes per char. I ported the UTF-8 <-> UCS-2 converter from GNU libiconv into the kernel. However, I don't know how to implement "casetype" in the conversion function, this may cause some problem in case insensitive matching used in msdosfs. I think this patch should just be a temporary workaround. The libiconv in the kernel should be rewritten for a good structure. From owner-freebsd-fs@FreeBSD.ORG Thu Jan 20 05:14:18 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D494316A4CE for ; Thu, 20 Jan 2005 05:14:18 +0000 (GMT) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id B4E0143D2F for ; Thu, 20 Jan 2005 05:14:18 +0000 (GMT) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id A41EE5C9ED; Wed, 19 Jan 2005 21:14:18 -0800 (PST) Date: Wed, 19 Jan 2005 21:14:18 -0800 From: Alfred Perlstein To: "Charles M. Hannum" Message-ID: <20050120051418.GB6174@elvis.mu.org> References: <1106180748.64610.21987.camel@palm.tree.com> <20050120014731.GD16440@netbsd.org> <20050120032433.GO13149@bcd.geek.com.au> <200501200503.13300.abuse@spamalicious.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200501200503.13300.abuse@spamalicious.com> User-Agent: Mutt/1.4.2.1i cc: Daniel Carosone cc: Havard Eidnes cc: Bill Studenmund cc: fs@freebsd.org cc: tech-perform@netbsd.org cc: tech-kern@netbsd.org Subject: Re: namei caching of newly created files? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jan 2005 05:14:18 -0000 cc'd to fs@freebsd.org because I wish someone would do this for us too. :) * Charles M. Hannum [050119 21:03] wrote: > On Thursday 20 January 2005 03:24, Daniel Carosone wrote: > > This is hurting, mostly, because it's being done over and over again > > all the way to the end of the dir. As the simplest possible change, > > before looking at more complex solutions, having done the search to > > the end once and found no free slots, we could remember that fact. We > > would reset the flag on deleting an entry and making a space, or > > perhaps after doing so several times. > > It's not that simple. There are often small pieces of directory blocks > available, but not large enough for the file name you're trying to create. There's two ways to solve this, one is easy but wastes space, the other is hard but more space effecient. Easy way: Keep track of the amount of free space in a directory versus its size. If the amount of free space is below a threshold then just append to the directory. (this should make creating large numbers of files very fast as we'll always be full or near full so we'll just append to the directory) Hard way: Keep an in memory bucket list of available sizes. The bucket list can be N offsets into the directory for sizes that are power of two up to MAXFILELEN. Tune the bucket structure to the size of the directory. Make sure to keep a "negative entry" for sizes not seen. Whenever you delete make sure to update your bucket structures to include the deleted space or mark the bucket as overflowed if all the entries for a particular size have been taken. Then finding a free space is easy. Do a best fit. -- - Alfred Perlstein - Research Engineering Development Inc. - email: bright@mu.org cell: 408-480-4684 From owner-freebsd-fs@FreeBSD.ORG Thu Jan 20 05:19:07 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3D85D16A4CE for ; Thu, 20 Jan 2005 05:19:07 +0000 (GMT) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2155743D53 for ; Thu, 20 Jan 2005 05:19:07 +0000 (GMT) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 160CD5C9F1; Wed, 19 Jan 2005 21:19:07 -0800 (PST) Date: Wed, 19 Jan 2005 21:19:07 -0800 From: Alfred Perlstein To: "Charles M. Hannum" Message-ID: <20050120051907.GC6174@elvis.mu.org> References: <1106180748.64610.21987.camel@palm.tree.com> <20050120014731.GD16440@netbsd.org> <20050120032433.GO13149@bcd.geek.com.au> <200501200503.13300.abuse@spamalicious.com> <20050120051418.GB6174@elvis.mu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20050120051418.GB6174@elvis.mu.org> User-Agent: Mutt/1.4.2.1i cc: Daniel Carosone cc: Havard Eidnes cc: Bill Studenmund cc: fs@freebsd.org cc: tech-perform@netbsd.org cc: tech-kern@netbsd.org Subject: Re: namei caching of newly created files? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jan 2005 05:19:07 -0000 One other trick is just a negative cache: Each in memory directory has a bucket list in it for power of two up until MAXFILELEN. If you have to do a scan, just populate the negative entries. Then only "fix" the negative entries if you delete something. Then you'll avoid repeated scans when there's no space. * Alfred Perlstein [050119 21:14] wrote: > > There's two ways to solve this, one is easy but wastes space, the > other is hard but more space effecient. > > Easy way: > Keep track of the amount of free space in a directory versus its > size. > > If the amount of free space is below a threshold then just > append to the directory. > > (this should make creating large numbers of files very fast > as we'll always be full or near full so we'll just append to > the directory) > > > Hard way: > Keep an in memory bucket list of available sizes. > > The bucket list can be N offsets into the directory for > sizes that are power of two up to MAXFILELEN. > > Tune the bucket structure to the size of the directory. > > Make sure to keep a "negative entry" for sizes not seen. > > Whenever you delete make sure to update your bucket structures to > include the deleted space or mark the bucket as overflowed if all > the entries for a particular size have been taken. > > Then finding a free space is easy. Do a best fit. > > -- > - Alfred Perlstein > - Research Engineering Development Inc. > - email: bright@mu.org cell: 408-480-4684 -- - Alfred Perlstein - Research Engineering Development Inc. - email: bright@mu.org cell: 408-480-4684