Date: Thu, 24 Jan 2013 11:37:58 -0500 From: Zaphod Beeblebrox <zbeeble@gmail.com> To: Adam Nowacki <nowakpl@platinum.linux.pl> Cc: freebsd-hackers@freebsd.org Subject: Re: ZFS regimen: scrub, scrub, scrub and scrub again. Message-ID: <CACpH0Mc%2BPDDA%2BVAcqHmcJVPJ7a=cgz1fSMOdp6qx5W62w1RScg@mail.gmail.com> In-Reply-To: <51015523.2060701@platinum.linux.pl> References: <CACpH0Mf6sNb8JOsTzC%2BWSfQRB62%2BZn7VtzEnihEKmEV2aO2p%2Bw@mail.gmail.com> <alpine.BSF.2.00.1301211201570.9447@wojtek.tensor.gdynia.pl> <20130122073641.GH30633@server.rulingia.com> <alpine.BSF.2.00.1301232121430.1659@wojtek.tensor.gdynia.pl> <51013345.8010701@platinum.linux.pl> <CACpH0McdJOrCgNWCsRwqnO_AvzzzDCx5gQxJL1nvF%2B8=ysqwRg@mail.gmail.com> <51015523.2060701@platinum.linux.pl>
next in thread | previous in thread | raw e-mail | index | archive | help
Ok... here's the existing data: There are 3,236,316 files summing to 97,500,008,691 bytes. That puts the "average" file at 30,127 bytes. But for the full breakdown: 512 : 7758 1024 : 139046 2048 : 1468904 4096 : 325375 8192 : 492399 16384 : 324728 32768 : 263210 65536 : 102407 131072 : 43046 262144 : 22259 524288 : 17136 1048576 : 13788 2097152 : 8279 4194304 : 4501 8388608 : 2317 16777216 : 1045 33554432 : 119 67108864 : 2 I produced that list with the output of ls -R's byte counts, sorted and then processed with: (while read num; do count=$[count+1]; if [ $num -gt $size ]; then echo $size : $count;size=$[size*2]; count=0; fi; done) <imapfilesizelist ... now the new machine has two 2T disks in a ZFS mirror --- so I suppose it won't waste as much space as a RAID-Z ZFS --- in that files less than 512 bytes will take 512 bytes? By far the most common case is 2048 bytes ... so that would indicate that a RAID-Z larger than 5 disks would waste much space. Does that go to your recomendations on vdev size, then? To have an 8 or 9 disk vdev, you should be storing at smallest 4k files?
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CACpH0Mc%2BPDDA%2BVAcqHmcJVPJ7a=cgz1fSMOdp6qx5W62w1RScg>