Date: Fri, 29 Sep 2017 08:26:13 +0200 From: Ben RUBSON <ben.rubson@gmail.com> To: Freebsd fs <freebsd-fs@freebsd.org> Subject: Re: How to create holes in files ? Message-ID: <454261D6-0061-4DB9-8184-BDD7061870FB@gmail.com> In-Reply-To: <1506664468.7204.37.camel@inparadise.se> References: <12ed413d-00d1-4182-bdd5-0e5230d46f01@email.android.com> <4D304428-E4AF-45D4-AB2E-D4B73A7FEE87@gmail.com> <1506664468.7204.37.camel@inparadise.se>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 29 Sep 2017, at 07:54, Karli Sj=C3=B6berg <karli@inparadise.se> = wrote: >=20 > On tor, 2017-09-28 at 22:16 +0200, Ben RUBSON wrote: >>>=20 >>> On 28 Sep 2017, at 20:48, Karli Sj=C3=B6berg <karli@inparadise.se> >>> wrote: >>>=20 >>> Den 28 sep. 2017 6:47 em skrev Ben RUBSON <ben.rubson@gmail.com>: >>>>=20 >>>> On 28 Sep 2017, at 18:34, Bob Eager wrote: >>>>=20 >>>> On Thu, 28 Sep 2017 17:26:09 +0200 >>>> Fabian Keil wrote: >>>>=20 >>>>>=20 >>>>> Ben RUBSON wrote: >>>>>=20 >>>>>>=20 >>>>>> I'm trying to make holes in files in C. >>>>>> Goal is to deallocate huge files on ZFS while (randomly) >>>>>> reading >>>>>> them.=20 >>>>> My interpretation of the above is that you want to create holes >>>>> without changing the file size and without affecting data that >>>>> is located before or after the holes that you want to create. >>>>>=20 >>>>> Otherwise you could simply "deallocate" the content with >>>>> truncate(1). >>>> If he doesn't mind copying the files, dd(1) will do the job. >>>> However, I >>>> expect that doesn't meet his criteria. >>> Thank you Bob for your suggestion. >>> You're right goal is to avoid copying data : free space would not >>> necessarily >>> allow this, and as the files I'm working on are some hundreds of GB >>> in size, >>> it would really be a counterproductive long stressing storage >>> operation. >>>=20 >>> Well, correct me if I'm wrong, but wouldn't dd with "seek" mitigate >>> the issue of writing out all of the data from beginning to end. If >>> you seek from the beginning of the file to the point you want to >>> start writing from and use bs to specify how large of a "hole" you >>> want, the operation wouldn't take long at all. You would, in my >>> opinion, achieve exactly what you want, to "create holes in files". >>> Am I wrong? >> I would have liked to do this in C. >=20 > Yepp, I understand that, I was just bringing up the idea. But hey, if > dd can do it, just look in the source of how it does it? Of course yes, you're right :) >> And the holes I need to create are not necessarily at the beginning >> of a file. >=20 > No, you _do not_ have to create holes in the beginning of a file, you > can do it anywhere in the file you like. E.g. it=C2=B4s the trick I = use to > "clean" hardrives to make them look empty, by only erasing the first > and last 10 MB of the drives, a whole JBOD only takes seconds to = clean. > Say you have a 10 MB large file and you want to make a 1 MB large hole > somewhere in the middle of it, you do it like this: >=20 > # dd if=3D/dev/zero of=3D/foo/bar.bin bs=3D1M seek=3D7 >=20 > It=C2=B4ll make the file look like this inside (hope the ASCII gods = are with > me): > ____________ > |_______|_|__| Unfortunately here storage blocks will be set/written to 0, they will not be freed/recovered, a file hole will not be created. Below are some commands which demonstrate this. The last 3/4 commands show a _newly_ created file with a hole. # zfs userspace home POSIX User root 36.5K none # dd if=3D/dev/random of=3Df bs=3D1M count=3D4 4+0 records out # zfs userspace home POSIX User root 4.04M none # dd if=3D/dev/zero of=3Df bs=3D1M count=3D2 seek=3D2 2+0 records out # zfs userspace home POSIX User root 4.04M none # rm f # dd if=3D/dev/zero of=3Df bs=3D1M count=3D2 seek=3D2 2+0 records out # zfs userspace home POSIX User root 2.04M none # ls -lh f -rw------- 1 root wheel 4.0M 29 Sep 08:19 f
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?454261D6-0061-4DB9-8184-BDD7061870FB>