From owner-freebsd-fs@FreeBSD.ORG Tue Jul 24 13:46:31 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 31AEF106566C for ; Tue, 24 Jul 2012 13:46:31 +0000 (UTC) (envelope-from maurizio.vairani@cloverinformatica.it) Received: from smtplq04.aruba.it (smtplq-out5.aruba.it [62.149.158.25]) by mx1.freebsd.org (Postfix) with SMTP id 77CFB8FC14 for ; Tue, 24 Jul 2012 13:46:30 +0000 (UTC) Received: (qmail 26996 invoked by uid 89); 24 Jul 2012 13:46:28 -0000 Received: from unknown (HELO smtp2.aruba.it) (62.149.158.222) by smtplq04.aruba.it with SMTP; 24 Jul 2012 13:46:28 -0000 Received: (qmail 29329 invoked by uid 89); 24 Jul 2012 13:46:28 -0000 Received: from unknown (HELO cloverinformatica.it) (info@cloverinformatica.it@151.55.125.196) by smtp2.ad.aruba.it with SMTP; 24 Jul 2012 13:46:28 -0000 Received: from [192.168.0.185] (ASUS-TERMINATOR [192.168.0.185]) by cloverinformatica.it (Postfix) with ESMTP id 225775E; Tue, 24 Jul 2012 15:46:28 +0200 (CEST) Message-ID: <500EA732.1070401@cloverinformatica.it> Date: Tue, 24 Jul 2012 15:46:26 +0200 From: Maurizio Vairani User-Agent: Mozilla/5.0 (Windows NT 4.0; rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1 MIME-Version: 1.0 To: Gleb Kurtsou References: <20120621140149.GA59722@reks> In-Reply-To: <20120621140149.GA59722@reks> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Rating: smtp2.ad.aruba.it 1.6.2 0/1000/N X-Spam-Rating: smtplq04.aruba.it 1.6.2 0/1000/N Cc: freebsd-fs@freebsd.org Subject: Re: [RFC] tmpfs RB-Tree for directory entries X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jul 2012 13:46:31 -0000 On 21/06/2012 16.01, Gleb Kurtsou wrote: > Hello, > > Here is patch for CURRENT replacing directory entry linked list with > RB-Tree in tmpfs. Performance improvement varies for different > workloads, it may be negligible for directories with small number of > files or due to VFS name caching. > > http://people.freebsd.org/~gleb/tmpfs-nrbtree.1.patch > > This patch is unrelated to similar changes recently committed to > DragonFly: > https://bugs.dragonflybsd.org/issues/2375 > http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/29ca4fd6da8bb70ae90d8e73ea3c47fda22491a7 > > My patch uses name hashes instead of comparing entries by file name, > moreover it reuses the same hash value as directory entry offset and > eliminates possible issue of duplicate directory offsets on 64-bit > archs. In other words it makes VOP_READDIR on large directories faster > for non-zero offsets. > > I'm willing to commit the patch and would appreciate if people actively > using tmpfs give it a try. > > Thanks, > Gleb. > > > ** file_create test from DragonFly PR > % time ~/file_create 10000 > > x tmpfs-file_create-rb > + tmpfs-file_create-orig > +------------------------------------------------------------------------+ > | + | > |x + | > |xx + | > |xx + +| > |A| | > | |MA_| | > +------------------------------------------------------------------------+ > N Min Max Median Avg Stddev > x 5 0.112 0.14 0.119 0.1234 0.012116105 > + 5 2.551 2.734 2.551 2.5886 0.081309901 > Difference at 95.0% confidence > 2.4652 +/- 0.0847787 > 1997.73% +/- 68.7023% > (Student's t, pooled s = 0.0581296) > > > ** test1 -- create 5000 files, rename some of them, remove files > time sh ~/test1.sh > > x tmpfs-test1-rb > + tmpfs-test1-orig > +------------------------------------------------------------------------+ > | x x x + +++ | > ||___MA_____| | > | |_____________A______M______|| > +------------------------------------------------------------------------+ > N Min Max Median Avg Stddev > x 4 5.893 6.091 5.932 5.9535 0.093289871 > + 4 6.49 7.006 6.987 6.8655 0.25058931 > Difference at 95.0% confidence > 0.912 +/- 0.327153 > 15.3187% +/- 5.49514% > (Student's t, pooled s = 0.189074) > > > test1.sh: > #!/bin/sh > for i in `jot 5000`; do echo $i> longername$i; done > ls>/dev/null > for i in `jot 899 100`; do mv longername$i longername1$i; done > ls>/dev/null > for i in `jot 899 100`; do mv longername2$i longername3$i; done > ls>/dev/null > for i in `jot 5000`; do rm longername$i 2>/dev/null; done > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to"freebsd-fs-unsubscribe@freebsd.org" I applied patch on my test machines on 9-STABLE with 4G RAM and 2G of swap space : # uname -a FreeBSD clover-nas-test 9.0-STABLE FreeBSD 9.0-STABLE #2: Tue Jul 10 17:45:25 CEST 2012 ssh@clover-nas-test:/usr/obj/usr/src/sys/GENERIC amd64 # swapinfo -h Device 1K-blocks Used Avail Capacity /dev/zvol/poolusb/swap 2097152 3.5M 2G 0% tmpfs is used for : # df -h | grep tmpfs tmpfs 2.2G 1.1M 2.2G 0% /tmp tmpfs 2.2G 4.0k 2.2G 0% /var/tmp tmpfs 2.2G 188k 2.2G 0% /var/log tmpfs 2.2G 120k 2.2G 0% /var/run tmpfs 2.2G 36k 2.2G 0% /var/db/entropy tmpfs 2.2G 512k 2.2G 0% /var/db/samba tmpfs 2.2G 32k 2.2G 0% /var/spool No regression for three days even low memory condition. # dmesg | grep swap pid 58480 (ssh), uid 0, was killed: out of swap space