Date: Fri, 29 Sep 2017 08:54:28 +0200 From: Ben RUBSON <ben.rubson@gmail.com> To: Freebsd fs <freebsd-fs@freebsd.org> Subject: Re: How to create holes in files ? Message-ID: <7F95ECD3-3284-4808-9182-AF3FE5970333@gmail.com> In-Reply-To: <1506666838.7204.39.camel@inparadise.se> References: <12ed413d-00d1-4182-bdd5-0e5230d46f01@email.android.com> <4D304428-E4AF-45D4-AB2E-D4B73A7FEE87@gmail.com> <1506664468.7204.37.camel@inparadise.se> <454261D6-0061-4DB9-8184-BDD7061870FB@gmail.com> <1506666838.7204.39.camel@inparadise.se>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 29 Sep 2017, at 08:33, Karli Sjöberg <karli@inparadise.se> wrote: > > On fre, 2017-09-29 at 08:26 +0200, Ben RUBSON wrote: >>> >>> On 29 Sep 2017, at 07:54, Karli Sjöberg <karli@inparadise.se> >>> wrote: >>> >>> On tor, 2017-09-28 at 22:16 +0200, Ben RUBSON wrote: >>>> >>>>> >>>>> >>>>> On 28 Sep 2017, at 20:48, Karli Sjöberg <karli@inparadise.se> >>>>> wrote: >>>>> >>>>> Den 28 sep. 2017 6:47 em skrev Ben RUBSON <ben.rubson@gmail.com >>>>>> : >>>>>> >>>>>> >>>>>> On 28 Sep 2017, at 18:34, Bob Eager wrote: >>>>>> >>>>>> On Thu, 28 Sep 2017 17:26:09 +0200 >>>>>> Fabian Keil wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> Ben RUBSON wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> I'm trying to make holes in files in C. >>>>>>>> Goal is to deallocate huge files on ZFS while (randomly) >>>>>>>> reading >>>>>>>> them. >>>>>>> My interpretation of the above is that you want to create >>>>>>> holes >>>>>>> without changing the file size and without affecting data >>>>>>> that >>>>>>> is located before or after the holes that you want to >>>>>>> create. >>>>>>> >>>>>>> Otherwise you could simply "deallocate" the content with >>>>>>> truncate(1). >>>>>> If he doesn't mind copying the files, dd(1) will do the job. >>>>>> However, I >>>>>> expect that doesn't meet his criteria. >>>>> Thank you Bob for your suggestion. >>>>> You're right goal is to avoid copying data : free space would >>>>> not >>>>> necessarily >>>>> allow this, and as the files I'm working on are some hundreds >>>>> of GB >>>>> in size, >>>>> it would really be a counterproductive long stressing storage >>>>> operation. >>>>> >>>>> Well, correct me if I'm wrong, but wouldn't dd with "seek" >>>>> mitigate >>>>> the issue of writing out all of the data from beginning to end. >>>>> If >>>>> you seek from the beginning of the file to the point you want >>>>> to >>>>> start writing from and use bs to specify how large of a "hole" >>>>> you >>>>> want, the operation wouldn't take long at all. You would, in my >>>>> opinion, achieve exactly what you want, to "create holes in >>>>> files". >>>>> Am I wrong? >>>> I would have liked to do this in C. >>> Yepp, I understand that, I was just bringing up the idea. But hey, >>> if >>> dd can do it, just look in the source of how it does it? >> Of course yes, you're right :) >> >>> >>>> >>>> And the holes I need to create are not necessarily at the >>>> beginning >>>> of a file. >>> No, you _do not_ have to create holes in the beginning of a file, >>> you >>> can do it anywhere in the file you like. E.g. it´s the trick I use >>> to >>> "clean" hardrives to make them look empty, by only erasing the >>> first >>> and last 10 MB of the drives, a whole JBOD only takes seconds to >>> clean. >>> Say you have a 10 MB large file and you want to make a 1 MB large >>> hole >>> somewhere in the middle of it, you do it like this: >>> >>> # dd if=/dev/zero of=/foo/bar.bin bs=1M seek=7 >>> >>> It´ll make the file look like this inside (hope the ASCII gods are >>> with >>> me): >>> ____________ >>>> >>>> _______|_|__| >> Unfortunately here storage blocks will be set/written to 0, >> they will not be freed/recovered, a file hole will not be created. >> Below are some commands which demonstrate this. >> The last 3/4 commands show a _newly_ created file with a hole. >> >> # zfs userspace home >> POSIX User root 36.5K none >> >> # dd if=/dev/random of=f bs=1M count=4 >> 4+0 records out >> >> # zfs userspace home >> POSIX User root 4.04M none >> >> # dd if=/dev/zero of=f bs=1M count=2 seek=2 >> 2+0 records out >> >> # zfs userspace home >> POSIX User root 4.04M none >> >> # rm f >> >> # dd if=/dev/zero of=f bs=1M count=2 seek=2 >> 2+0 records out >> >> # zfs userspace home >> POSIX User root 2.04M none >> >> # ls -lh f >> -rw------- 1 root wheel 4.0M 29 Sep 08:19 f > > Ah, I see, nice demonstration! But wouldn´t TRIM take of that though > (if that´s available)? Good question ! I'm however not sure TRIM would notify ZFS about space not consumed. In addition I'm mainly on non-SSD drives :-/ Ben
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7F95ECD3-3284-4808-9182-AF3FE5970333>