From owner-freebsd-arch@FreeBSD.ORG Fri Jan 1 20:54:06 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 271351065679; Fri, 1 Jan 2010 20:54:06 +0000 (UTC) (envelope-from pieter@degoeje.nl) Received: from mx.utwente.nl (mx1.utsp.utwente.nl [130.89.2.12]) by mx1.freebsd.org (Postfix) with ESMTP id 9CC958FC0A; Fri, 1 Jan 2010 20:54:05 +0000 (UTC) Received: from nox.student.utwente.nl (nox.student.utwente.nl [130.89.165.91]) by mx.utwente.nl (8.12.10/SuSE Linux 0.7) with ESMTP id o01Kripo006131; Fri, 1 Jan 2010 21:53:46 +0100 From: Pieter de Goeje To: "Poul-Henning Kamp" Date: Fri, 1 Jan 2010 21:53:43 +0100 User-Agent: KMail/1.9.10 References: <61364.1262345170@critter.freebsd.dk> In-Reply-To: <61364.1262345170@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <201001012153.44349.pieter@degoeje.nl> X-UTwente-MailScanner-Information: Scanned by MailScanner. Contact icts.servicedesk@utwente.nl for more information. X-UTwente-MailScanner: Found to be clean X-UTwente-MailScanner-From: pieter@degoeje.nl X-Spam-Status: No Cc: Alexander Motin , freebsd-current@freebsd.org, Miroslav Lachman <000.fbsd@quip.cz>, Thomas Backman , freebsd-arch@freebsd.org Subject: Re: File system blocks alignment X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jan 2010 20:54:06 -0000 On Friday 01 January 2010 12:26:10 Poul-Henning Kamp wrote: > In message <200912311712.18347.pieter@degoeje.nl>, Pieter de Goeje writes: > >Ok, as Miroslav wrote, it does not report 4k sectors: > > If you care to, and have the time, try the following: > > > for N in (0...7) > for M in (0..2) > create partition starting at byte offset N*512 > newfs -f 4096 -b 32768 /dev/foo > mount /dev/foo /mnt > time restore(8) a filesystem into it. > unmount /mnt > time fsck_ffs /dev/foo > mount /dev/foo /mnt > time tar cf /dev/null /mnt > unmount /mnt That yielded some pretty spectacular results. When the partition is not correctly aligned, performance is not so good ;-). About the benchmark: - Test was conducted in single user mode, while running gstat and systat to monitor the system. - OS and dump archive were located on a different harddisk. - Partition size was 10GB, dump archive ~550MB (dump of /). The archive easily fitted in main memory (8GB). - I observed L(q) sizes over 600 and ms/w times over 30sec during unaligned restore. - CPU load was very low during entire test. Performance for restore was abysmal in the unaligned case, easily being 10 times slower than aligned restore. Newfs was about 5 times as slow. I also tested various combinations of soft-updates, mounting async and block/fragment size. Some helped mildly as one might expect, but nothing significant was gained. It was somewhat surprising to see restore being faster than tar in the aligned case though. Mounting async or using soft updates resulted in a 6sec restore, while tar took about 8sec. Raw results (and benchmark script) below. Offset is the number of (512byte) sectors. The partition is aligned at offset 40. The test begins at offset 34 because that is the first free sector for GPT partitions. --------------- results --------------- ==> offset: 34 iter: 1 newfs 0.98 real 0.00 user 0.00 sys restore 117.76 real 0.12 user 0.98 sys fsck_ffs 1.83 real 0.02 user 0.00 sys tar 9.83 real 0.06 user 0.34 sys ==> offset: 34 iter: 2 newfs 0.98 real 0.00 user 0.00 sys restore 149.68 real 0.15 user 1.02 sys fsck_ffs 1.75 real 0.02 user 0.00 sys tar 9.56 real 0.02 user 0.37 sys ==> offset: 35 iter: 1 newfs 0.95 real 0.00 user 0.00 sys restore 135.31 real 0.11 user 1.07 sys fsck_ffs 1.82 real 0.02 user 0.00 sys tar 9.74 real 0.04 user 0.35 sys ==> offset: 35 iter: 2 newfs 1.21 real 0.00 user 0.00 sys restore 154.59 real 0.13 user 1.05 sys fsck_ffs 1.91 real 0.02 user 0.00 sys tar 9.58 real 0.04 user 0.35 sys ==> offset: 36 iter: 1 newfs 0.98 real 0.00 user 0.00 sys restore 151.36 real 0.11 user 1.08 sys fsck_ffs 1.77 real 0.02 user 0.00 sys tar 9.86 real 0.07 user 0.32 sys ==> offset: 36 iter: 2 newfs 0.95 real 0.00 user 0.00 sys restore 153.71 real 0.14 user 1.05 sys fsck_ffs 1.90 real 0.02 user 0.00 sys tar 10.02 real 0.03 user 0.37 sys ==> offset: 37 iter: 1 newfs 0.98 real 0.00 user 0.00 sys restore 128.90 real 0.08 user 1.10 sys fsck_ffs 1.76 real 0.03 user 0.00 sys tar 9.76 real 0.08 user 0.31 sys ==> offset: 37 iter: 2 newfs 0.98 real 0.00 user 0.00 sys restore 147.45 real 0.13 user 1.05 sys fsck_ffs 1.83 real 0.02 user 0.00 sys tar 9.75 real 0.06 user 0.33 sys ==> offset: 38 iter: 1 newfs 0.95 real 0.00 user 0.00 sys restore 150.35 real 0.15 user 1.04 sys fsck_ffs 2.04 real 0.02 user 0.01 sys tar 9.42 real 0.04 user 0.35 sys ==> offset: 38 iter: 2 newfs 0.94 real 0.00 user 0.00 sys restore 125.40 real 0.16 user 1.02 sys fsck_ffs 1.69 real 0.02 user 0.00 sys tar 9.74 real 0.09 user 0.30 sys ==> offset: 39 iter: 1 newfs 0.95 real 0.01 user 0.00 sys restore 135.23 real 0.11 user 1.09 sys fsck_ffs 1.93 real 0.02 user 0.01 sys tar 9.82 real 0.05 user 0.35 sys ==> offset: 39 iter: 2 newfs 0.93 real 0.01 user 0.00 sys restore 150.30 real 0.07 user 1.12 sys fsck_ffs 1.86 real 0.02 user 0.01 sys tar 9.61 real 0.08 user 0.32 sys ==> offset: 40 iter: 1 newfs 0.22 real 0.00 user 0.00 sys restore 11.05 real 0.10 user 1.08 sys fsck_ffs 1.97 real 0.02 user 0.00 sys tar 7.86 real 0.05 user 0.34 sys ==> offset: 40 iter: 2 newfs 0.17 real 0.00 user 0.00 sys restore 8.83 real 0.09 user 1.09 sys fsck_ffs 1.67 real 0.01 user 0.01 sys tar 7.86 real 0.08 user 0.31 sys ==> offset: 41 iter: 1 newfs 0.95 real 0.01 user 0.00 sys restore 133.37 real 0.16 user 1.04 sys fsck_ffs 1.71 real 0.02 user 0.00 sys tar 9.43 real 0.07 user 0.31 sys ==> offset: 41 iter: 2 newfs 1.09 real 0.00 user 0.01 sys restore 150.71 real 0.09 user 1.10 sys fsck_ffs 2.01 real 0.01 user 0.01 sys tar 9.25 real 0.06 user 0.33 sys ------------------- bench.sh ---------------- disk=/dev/ada0 fsarchive=/usr/home/pyotr/rootdump results=/usr/home/pyotr/bench-results bench() { offset=$((34 + $1)) echo "==> offset: $offset iter: $2" >> $results dd if=/dev/zero of=$disk bs=1m count=1 2> /dev/null gpart create -s gpt $disk gpart add -b $offset -s 10G -t freebsd-ufs $disk partition=${disk}p1 echo "newfs" >> $results time -ao $results newfs -f 4096 -b 32768 $partition || exit 1 mount $partition /mnt cd /mnt echo "restore" >> $results time -ao $results restore -rf $fsarchive || exit 1 cd / umount /mnt echo "fsck_ffs" >> $results time -ao $results fsck_ffs $partition || exit 1 mount $partition /mnt echo "tar" >> $results time -ao $results tar -cf /dev/null /mnt || exit 1 umount /mnt } true > $results umount /mnt for N in `jot 8 0`; do for M in `jot 2`; do bench $N $M done done -- Pieter de Goeje