From owner-freebsd-questions@FreeBSD.ORG Sun Jul 1 22:19:32 2007 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7CA2716A5B0 for ; Sun, 1 Jul 2007 22:19:32 +0000 (UTC) (envelope-from unixvn@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.179]) by mx1.freebsd.org (Postfix) with ESMTP id 2E17A13C4C4 for ; Sun, 1 Jul 2007 22:19:32 +0000 (UTC) (envelope-from unixvn@gmail.com) Received: by py-out-1112.google.com with SMTP id u77so2697638pyb for ; Sun, 01 Jul 2007 15:19:31 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=a6pnhzdgFTy+9+1lepN+4ejUMtaG8LX9v2ZXGjl7p9HZj92yxQmhFcCumg/LjVdR8S+gLW024jWRYKCinYvZATf/Odf90OuO5kTtkopSO1NUNGJ9y0p9PM3pBuwZTBlrJs3G3LSSAVABfkSh88zczOKqHNLyTA1KugcBdUqPolw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=hRnGRGLzwy4r+O3V10YSNKnfH8AYqIMpGIYYXJvkW0zRjPwgwDX6uJhTgP90PpnJNVaTWqKPnESvVLuM9EkDJYTadjo+AbyzfbYRKvw6kdgIW9UagoflrwYmxSnT3h+pdRi0eIt98ZGmGK8M6ERXCYD369ui8eIcoGYP60qtDHc= Received: by 10.65.186.18 with SMTP id n18mr8147218qbp.1183328371320; Sun, 01 Jul 2007 15:19:31 -0700 (PDT) Received: by 10.65.250.15 with HTTP; Sun, 1 Jul 2007 15:19:31 -0700 (PDT) Message-ID: <64b284310707011519k68ebc35p716cd900d9212234@mail.gmail.com> Date: Mon, 2 Jul 2007 02:19:31 +0400 From: "Nguyen Tam Chinh" To: "Nikolay Pavlov" , "Nguyen Tam Chinh" , FreeBSD-Questions , freebsd-stable@freebsd.org In-Reply-To: <20070701215229.GB10527@zone3000.net> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <64b284310706270311j2a6af2f6i6766b483a4b66a5c@mail.gmail.com> <20070701215229.GB10527@zone3000.net> Cc: Subject: Re: UFS2 optimization for many small files X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2007 22:19:32 -0000 On 7/2/07, Nikolay Pavlov wrote: > On Wednesday, 27 June 2007 at 14:11:19 +0400, Nguyen Tam Chinh wrote: > > Greetings, > > > > We're going to build a server with some 1Tb of over 500 million small > > files with size from 0,5k to 4k. I'm wonder if the ufs2 can handle > > this kind of system well. From newfs(8) the min block size is 4k. This > > is not optimal in our case, a 1k or 0,5k block is more effective IMHO. > > I'd be happy if anyone can suggest what does fragment (block/8) in the > > ufs2 mean and how this parameter works. I know It's better to read the > > full ufs2 specification, but hope that someone here can give a hint. > > Please advice with optimizations or tricks. > > Thank you very much. > > > > -- > > With best regards, | The Power to Serve > > Nguyen Tam Chinh | http://www.FreeBSD.org > > _______________________________________________ > > freebsd-stable@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > > I am not aware of any ZFS results on such tasks, may be you will be the > one who share them ;) However RaiserFS whould be the best choise on such > spesific case. It's not available on FreeBSD currently. > I don't think UFS can handle a huge amount of small files effectively. > Of course gjournal could be an option for fsck problems, but how do you > plan to backup or sync this storage? > I'm aware of the fsck/backup problems. In our case there's no need for backup so i went with ufs2. The current configuration is 4x250Gb disks with bloc/frag ratio 4k/512b. We're generating files with the average size of 6k ('cause the compress procedure does not work as well as we estimated). After a week I think we could collect some statistics in production. Anyway, in this case a 8k/1k would be more effective for us. Hope that I can test this in the next server. -- With best regards, | The Power to Serve Nguyen Tam Chinh | http://www.FreeBSD.org