From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 16 20:46:55 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 43FBE106564A for ; Sat, 16 Jun 2012 20:46:55 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 031B48FC0A for ; Sat, 16 Jun 2012 20:46:54 +0000 (UTC) Received: by obcni5 with SMTP id ni5so7632509obc.13 for ; Sat, 16 Jun 2012 13:46:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=AQJ82lE+dKZdw4n6wgURKn3wPFu639RwVqYq0RoYqA0=; b=Zoxx+385rE4Hr0H9xShLnZybHOQGysv1nszzDUK1DUqtsR3IOte6uq/AOAHcsoAh9j t0auxZJZil9CPKFPNJLOXSsdoCPlu74TjCLfauG9rTP/tuU2pdmUOyGTfnTcZ/WsrKeH 6t5yXrhievZsS1OtNcaDL3nKuPcH2XxCsNzQF+Z+goJnqJHHUd1q1BYYbX5ElR+B4wkU tqlpRfjuQ7ZpFRzvgUCkVVVnUZrmMrMKipFgDSA+tnVNnU/36FxqrIJy+V+88uOsXwCp pA2mUi0vgVTu4UJtJ4hBpYolPb9z+1hAzz7GqnAXWpqld/EOP49FE3WqpA6BSrSC2NWk FLqA== MIME-Version: 1.0 Received: by 10.60.3.202 with SMTP id e10mr10472758oee.52.1339879614494; Sat, 16 Jun 2012 13:46:54 -0700 (PDT) Received: by 10.76.98.77 with HTTP; Sat, 16 Jun 2012 13:46:54 -0700 (PDT) In-Reply-To: <20120616163450.2d0fa3c7@bhuda.mired.org> References: <4FD94240.6060806@delphij.net> <20120616163450.2d0fa3c7@bhuda.mired.org> Date: Sat, 16 Jun 2012 13:46:54 -0700 Message-ID: From: Garrett Cooper To: Mike Meyer Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: BIO_DELETE equivalent for file on FFS filesystem X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Jun 2012 20:46:55 -0000 On Sat, Jun 16, 2012 at 1:34 PM, Mike Meyer wrote: > On Sat, 16 Jun 2012 21:04:26 +0100 > Chris Rees wrote: > >> On Jun 16, 2012 8:37 PM, "Xin LI" wrote: >> > >> > On Sat, Jun 16, 2012 at 12:01 PM, Chris Rees wrote= : >> > > On Jun 14, 2012 5:49 AM, "Wojciech Puchar" < >> wojtek@wojtek.tensor.gdynia.pl> >> > > wrote: >> > >>>> >> > >>>> file to take 900MB or... can i call some system function to "punc= h" >> > >>>> holes? >> > >>> >> > >>> >> > >>> I think you can only truncate the file at this time, pretty much l= ike >> > >>> brk() works for memory. >> > >> >> > >> >> > >> >> > >> BAD. suppose i keep windoze VM image on filesystem which takes 10GB= but >> > > uses 5GB. >> > >> >> > >> i could write simple program to find out what blocks are unused and >> > > then...do nothing. >> > >> >> > > >> > > What if you cp it? >> > >> > That would be a dd(1) unless we teach cp(1) how to do sparse. =A0I thi= nk >> > what he wanted is to tell the OS "I don't need block XX - YY anymore" >> > and the OS creates a sparse hole, which is not available at this time. >> >> Sorry, I must have misread. =A0I take it cp would take a file with holes= and >> only copy the data part? i.e. take a 10G file of which 5G is a hole, you= 'd >> end up with a 5G file? > > No, cp just does read()s. Reading data from a hole returns a block > full of zeros. A quick test (after writing a program to create the > file) shows this: > > bhuda% df -h . > Filesystem =A0 =A0Size =A0 =A0Used =A0 Avail Capacity =A0Mounted on > /dev/md0 =A0 =A0 =A0123M =A0 =A01.2M =A0 =A0112M =A0 =A0 1% =A0 =A0/tmp > bhuda% ls -lh holey.test > -rwxr-xr-x =A01 mwm =A0wheel =A0 953M Jun 16 16:22 holey.test > > Ok, I've got a file that's 953M on an FS with 1.2 meg used. It's got > holes. > > bhuda% cp holey.test foobar > /tmp: write failed, filesystem is full > cp: foobar: No space left on device > > And doing a cp fails. Use dd conv=3Dsparse to get the effect you want. Our version of cp doesn't support sparsing of files, but Linux's does: http://linux.about.com/od/commands/l/blcmdl1_cp.htm Our copy of dd and rsync are the two common tools I know of that properly handle sparse files if the underlying filesystem supports it (like Xin said), apart from writing a tool to do the necessary calls to lseek to do the right thing in the file (which is possible, but you would need to potentially read the whole file in, or pieces, similar to defrag until Windows might do). Thanks, -Garrett