From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 16:56:03 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D7008106566B for ; Sat, 12 Feb 2011 16:56:03 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from tower.berklix.org (tower.berklix.org [83.236.223.114]) by mx1.freebsd.org (Postfix) with ESMTP id 5EE5D8FC19 for ; Sat, 12 Feb 2011 16:56:02 +0000 (UTC) Received: from park.js.berklix.net (p5B22D64C.dip.t-dialin.net [91.34.214.76]) (authenticated bits=0) by tower.berklix.org (8.14.2/8.14.2) with ESMTP id p1CGttE1038618; Sat, 12 Feb 2011 16:55:56 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by park.js.berklix.net (8.13.8/8.13.8) with ESMTP id p1CGtn4C016594; Sat, 12 Feb 2011 17:55:49 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.3/8.14.3) with ESMTP id p1CGu8kC024460; Sat, 12 Feb 2011 17:56:13 +0100 (CET) (envelope-from jhs@fire.js.berklix.net) Message-Id: <201102121656.p1CGu8kC024460@fire.js.berklix.net> To: Bruce Cran From: "Julian H. Stacey" Organization: http://www.berklix.com BSD Unix Linux Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://www.berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Sat, 12 Feb 2011 01:01:15 GMT." <20110212010115.000066db@unknown> Date: Sat, 12 Feb 2011 17:56:08 +0100 Sender: jhs@berklix.com Cc: hackers@freebsd.org Subject: Re: memstick.img is bloated with 7% 2K blocks of nulls X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 16:56:03 -0000 Bruce Cran wrote: > On Sat, 12 Feb 2011 01:54:58 +0100 > "Julian H. Stacey" wrote: > > > -O filesystem-type > > Use 1 to specify that a UFS1 format file system be > > built; use 2 to specify that a UFS2 format file system be built. The > > default format is UFS2. > > If anyone fancies looking deeper, please do :-) > > I checked with dumpfs that memstick.img is UFS1. > > Also, mounting /dev/md0 confuses the kernel into ultimately panic'ing, > since /dev/md0a is the proper slice. I'm suprised it mounts. Not suprised it crashes. > For the mfsroot.gz file from the > CD ISOs: > > # mdconfig -a -f mfsroot > md0 > # mount /dev/md0a /mnt > # ls /mnt > ls: /mnt: Bad file descriptor > # cd /mnt > cd: /mnt: Not a directory > # vim /mnt > > panic: ffs_read: type 0 It's not so obvious but in man mdconfig, there's no "[]" around "-t type", I read that as "-t something" is mandatory, (though it starts without for you ... & I suspect -t default is malloc, though manual doesnt say that, but look what manual says re. malloc ... panic ). I use "-t vnode", which doesn't crash here on 8.1-RELEASE i686: mdconfig -a -t vnode -f FreeBSD-8.2-RC3-amd64-memstick.img mount /dev/md0a /mnt cd /mnt tar cf - . | ( cd /pri/FreeBSD/releases/amd64/ISO-IMAGES/8.2\ /FreeBSD-8.2-RC3-amd64-memstick && tar xf - ) dumpfs /dev/md0a # UFS1 mdconfig -a -t vnode -f FreeBSD-8.2-RC3-amd64-livefs.iso mount -t cd9660 /dev/md1 /mnt1 cd /mnt1 tar cf - . | ( cd /pri/FreeBSD/releases/amd64/ISO-IMAGES/8.2\ /FreeBSD-8.2-RC3-amd64-livefs && tar xf - ) mdconfig -a -t vnode -f FreeBSD-8.2-RC3-amd64-disc1.iso xs mount -t cd9660 /dev/md2 /mnt2 cd /mnt2 tar cf - . | ( cd /pri/FreeBSD/releases/amd64/ISO-IMAGES/8.2\ /FreeBSD-8.2-RC3-amd64-disc1 && tar xf - ) cd / ; umount /mnt2 ; umount /mnt1 ; umount /mnt mdconfig -d -u 2 ; mdconfig -d -u 1 ; mdconfig -d -u 0 Analysing sizes: ls -l 723834880 FreeBSD-8.2-RC3-amd64-disc1.iso 348796928 FreeBSD-8.2-RC3-amd64-livefs.iso 1087467520 FreeBSD-8.2-RC3-amd64-memstick.img cd /pri/FreeBSD/releases/amd64/ISO-IMAGES/8.2 ; du -s -k /mnt* * 988708 /mnt # memstick.img 985807 /mnt1 # livefs.iso 705444 /mnt2 # disc1.iso 999340 FreeBSD-8.2-RC3-amd64-memstick 1023032 FreeBSD-8.2-RC3-amd64-livefs 711640 FreeBSD-8.2-RC3-amd64-disc1 df /mnt Filesystem Size Used Avail Capacity Mounted on /dev/md0a 1.1G 1.0G -14M 101% /mnt umount /mnt ; tunefs -m 0 /dev/md0a tunefs: minimum percentage of free space changes from 8% to 0% tunefs: should optimize for space with minfree < 8% df /dev/md0a Filesystem Size Used Avail Capacity Mounted on /dev/md0a 1.1G 1.0G 73M 93% cd /pri/FreeBSD/releases/amd64/ISO-IMAGES/8.2 ; /bin/pwd mount | grep /usra /dev/ad4s4e on /usra (ufs, NFS exported, local, soft-updates) dumpfs /dev/ad4s4e | head -1 # UFS2 Note du on UFS1 stick (988708) is near the same as tree copied by tar to UFS2 hard disc (999340), so its not the differerence between UFS1 & UFS2 that wastes the space (though good guess & thanks for suggesting it pushed me to investigate more :-) We are distributing 7% null space: Mainly because the file system is too big. We probably don't need 7% free space on stick just to install or repair, though that might be useful (if slow) for demos ) + Also each file will have unused bytes at end, (but so will livefs.iso have padding per file & not near so bloated, albeit probably some other fragment size. Per my first post: > > 2K blocks of nulls. > > FreeBSD-8.2-RC3-i386-disc1.iso: 565 > > FreeBSD-8.2-RC3-i386-livefs.iso: 905 > > FreeBSD-8.2-RC3-i386-memstick.img: 34521 As a sample test of what makefs produces: makefs /tmp/bla /usr/share Extent size set to 8192 block size 8192, fragment size 1024 du -s -k /usr/share # 54 108 ls -l /tmp/bla # 54 927 360 df /tmp/bla Filesystem Size Used Avail Capacity Mounted on /dev/ad0s2e 520M 55M 423M 12% /tmp # PS Note df lies 520M - some bug as not a node ? mdconfig -a -t vnode -f /tmp/bla # md2 df /dev/md2* Filesystem 1K-blocks Used Avail Capacity /dev/md2 51175 48723 -1642 103% tunefs -m 2 /dev/md2 minimum percentage of free space changes from 8% to 2% df /dev/md2* Filesystem 1K-blocks Used Avail Capacity /dev/md2 51175 48723 1429 97% Conclusion: makefs leaves 2+3=5% here. memstick.img is created by /usr/src/release/scripts/make-memstick.sh which (as above test) at present calls makefs without any tuning parameters Maybe t should have some parameters ? eg: makefs -m 1030000000 -b 1% ${tempfile} ${1} Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com Mail plain text; Not quoted-printable, Not HTML, Not base 64. Reply below text sections not at top, to avoid breaking cumulative context.