From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 19:09:19 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 276CB106566B for ; Fri, 11 Feb 2011 19:09:19 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from tower.berklix.org (tower.berklix.org [83.236.223.114]) by mx1.freebsd.org (Postfix) with ESMTP id 8D4478FC0C for ; Fri, 11 Feb 2011 19:09:17 +0000 (UTC) Received: from park.js.berklix.net (p5B22F3C7.dip.t-dialin.net [91.34.243.199]) (authenticated bits=0) by tower.berklix.org (8.14.2/8.14.2) with ESMTP id p1BJ9EL8019440 for ; Fri, 11 Feb 2011 19:09:16 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by park.js.berklix.net (8.13.8/8.13.8) with ESMTP id p1BJ9ZCH011451 for ; Fri, 11 Feb 2011 20:09:35 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.3/8.14.3) with ESMTP id p1BJ9UAE097045 for ; Fri, 11 Feb 2011 20:09:35 +0100 (CET) (envelope-from jhs@fire.js.berklix.net) Message-Id: <201102111909.p1BJ9UAE097045@fire.js.berklix.net> To: hackers@freebsd.org From: "Julian H. Stacey" Organization: http://www.berklix.com BSD Linux Unix Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://www.berklix.com/free/ X-URL: http://www.berklix.com/~jhs/cv/ Date: Fri, 11 Feb 2011 20:09:30 +0100 Sender: jhs@berklix.com Cc: Subject: memstick.img is bloated with 7% 2K blocks of nulls X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 19:09:19 -0000 memstick.img wastes 7% with 2K blocks of nulls. shown by: 8f -b 0 -n 2048 -l -f Fr* http://berklix.com/~jhs/src/bsd/jhs/bin/public/8f/ 8f.c 8f.1 FreeBSD-8.1-RELEASE-amd64-bootonly.iso: 543 FreeBSD-8.1-RELEASE-amd64-disc1.iso: 543 FreeBSD-8.1-RELEASE-amd64-dvd1.iso: 1125 FreeBSD-8.1-RELEASE-amd64-livefs.iso: 1109 FreeBSD-8.1-RELEASE-amd64-memstick.img: 38722 FreeBSD-8.1-RELEASE-i386-bootonly.iso: 511 FreeBSD-8.1-RELEASE-i386-disc1.iso: 511 FreeBSD-8.1-RELEASE-i386-dvd1.iso: 865 FreeBSD-8.1-RELEASE-i386-dvd1.iso: 893 FreeBSD-8.1-RELEASE-i386-livefs.iso: 850 FreeBSD-8.1-RELEASE-i386-memstick.img: 34066 FreeBSD-8.2-RC3-amd64-bootonly.iso: 594 FreeBSD-8.2-RC3-amd64-disc1.iso: 594 FreeBSD-8.2-RC3-amd64-dvd1.iso: 1167 FreeBSD-8.2-RC3-amd64-livefs.iso: 1166 FreeBSD-8.2-RC3-amd64-memstick.img: 39216 FreeBSD-8.2-RC3-i386-bootonly.iso: 565 FreeBSD-8.2-RC3-i386-disc1.iso: 565 FreeBSD-8.2-RC3-i386-dvd1.iso: 906 FreeBSD-8.2-RC3-i386-livefs.iso: 905 FreeBSD-8.2-RC3-i386-memstick.img: 34521 It's not just one big block of trailing nulls. od -c FreeBSD-8.2-RC3-i386-memstick.img > tmp.od ; tail tmp.od 7114017760 @ 244 377 344 Q \b * 325 \t 333 y ` & ' \n W 7114020000 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 * 7114050000 050000 - 020000 = 030000 = 0x3000 = 12288 It's a lot of bursts of null blocks, seen by either: grep -n '^*' tmp.od 8f -f -c -b 0 -n 2048 FreeBSD-8.2-RC3-i386-memstick.img The CD & DVD images are not nearly so wasteful, see above. As near 1G ( 959467520 FreeBSD-8.2-RC3-i386-memstick.img ) it will soon not fit on 1G sticks. If its not easy for someone to find & trim, xz compression would at least save 27% of transmission bandwidth, ( dc 702447528 100 * 959467520 / p 73 ) a much higher % than DVD that also uses compression. PS I sent this to hackers@ rather than re@ as I imagine re@ are busy with more urgent matters at present :-) Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com Mail plain text; Not quoted-printable, Not HTML, Not base 64. Reply below text sections not at top, to avoid breaking cumulative context.