From owner-freebsd-embedded@FreeBSD.ORG Fri Nov 30 01:04:11 2012 Return-Path: Delivered-To: freebsd-embedded@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3EEF3DBA for ; Fri, 30 Nov 2012 01:04:11 +0000 (UTC) (envelope-from freebsd@psconsult.nl) Received: from mx1.psconsult.nl (unknown [IPv6:2001:7b8:30f:e0::5059:ee8a]) by mx1.freebsd.org (Postfix) with ESMTP id DCB4A8FC08 for ; Fri, 30 Nov 2012 01:04:10 +0000 (UTC) Received: from mx1.psconsult.nl (mx1.hvnu.psconsult.nl [46.44.189.154]) by mx1.psconsult.nl (8.14.5/8.14.4) with ESMTP id qAU1438j029881 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 30 Nov 2012 02:04:08 +0100 (CET) (envelope-from freebsd@psconsult.nl) Received: (from paul@localhost) by mx1.psconsult.nl (8.14.5/8.14.4/Submit) id qAU1431F029880 for freebsd-embedded@freebsd.org; Fri, 30 Nov 2012 02:04:03 +0100 (CET) (envelope-from freebsd@psconsult.nl) X-Authentication-Warning: mx1.psconsult.nl: paul set sender to freebsd@psconsult.nl using -f Date: Fri, 30 Nov 2012 02:04:03 +0100 From: Paul Schenkeveld To: freebsd-embedded@freebsd.org Subject: Re: NanoBSD: Create image sparsely Message-ID: <20121130010403.GA82618@psconsult.nl> References: <42654855-CF29-488A-A645-2A280FC9D074@van-laarhoven.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <42654855-CF29-488A-A645-2A280FC9D074@van-laarhoven.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Nov 2012 01:04:11 -0000 On Thu, Nov 29, 2012 at 08:42:23AM +0100, Nick Hibma wrote: > In the past I have submitted patches to create images sparsely. There is however one patch missing still which, with the addition of NANO_INIT_IMG2, has become much more useful: > > --- AnyWi/freebsd/nanobsd/nanobsd.sh 2012-11-08 19:48:40.000000000 +0100 > +++ /usr/src/tools/tools/nanobsd/nanobsd.sh 2012-11-06 15:24:52.000000000 +0100 > @@ -540,7 +540,7 @@ > if [ $NANO_IMAGES -gt 1 -a $NANO_INIT_IMG2 -gt 0 ] ; then > # Duplicate to second image (if present) > echo "Duplicating to second image..." > - dd if=/dev/${MD}s1 of=/dev/${MD}s2 bs=64k > + dd conv=sparse if=/dev/${MD}s1 of=/dev/${MD}s2 bs=64k > mount /dev/${MD}s2a ${MNT} > for f in ${MNT}/etc/fstab ${MNT}/conf/base/etc/fstab > do > @@ -569,7 +569,7 @@ > > if ${do_copyout_partition} ; then > echo "Writing out _.disk.image..." > - dd if=/dev/${MD}s1 of=${NANO_DISKIMGDIR}/_.disk.image bs=64k > + dd conv=sparse if=/dev/${MD}s1 of=${NANO_DISKIMGDIR}/_.disk.image bs=64k > fi > mdconfig -d -u $MD > > It copies out the image sparsely (conv=sparse). The big advantage comes when creating many images (I have between 10 and 20 images simultaneously on my disk). > > Any objections to this? (*) When copying sparsely, blocks containing zeroes are not copied over but skipped using lseek. When the output file is a newly created regular file this is OK because sparse blocks read as zeroes. But here nanobsd.sh writes to a md(4) device which does not contain zeroes when created as a swap based device (NANO_MD_BACKING=swap). When a file on ${MD}s1 contains a block that deliberately contains zeroes the resulting copy is not the same as the original which may cause all kinds of problems. Only when ${MD}s1 is created as a file based device, blocks seeked over will read back as zeroes. Earlier this month I started a thread on freebsd-security@ related to this issue (subject: md(4) (swap-base) disks not cleaned on creation) and the patch by Konstantin Belousov will solve this problem but that leaves all systems before this patch vulnerable to this problem. HTH Paul Schenkeveld