Date: Wed, 06 May 2009 15:22:01 +0200 From: Olivier Mueller <om-lists-bsd@omx.ch> To: Bill Moran <wmoran@potentialtech.com> Cc: freebsd-performance@freebsd.org, freebsd-questions@freebsd.org Subject: Re: filesystem: 12h to delete 32GB of data (4 million files) Message-ID: <1241616121.16418.109.camel@ompc.insign.local> In-Reply-To: <20090506084834.61600c42.wmoran@potentialtech.com> References: <1241610888.16418.64.camel@ompc.insign.local> <20090506084834.61600c42.wmoran@potentialtech.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Thanks for your answer Bill! (and to Will as well), Some more infos I gathered a few minutes ago: [~/templates_c]$ date; du -s -m ; date Wed May 6 13:35:15 CEST 2009 2652 . Wed May 6 13:52:36 CEST 2009 [~/templates_c]$ date ; find . | wc -l ; date Wed May 6 13:52:56 CEST 2009 305461 Wed May 6 14:09:39 CEST 2009 So this is on the system after a complete cache cleanup (at 00h00). 300'000 files and 2.6GB. So this night, there were probably around 3-4 million files to delete. Deletion may take time, but 20 minutes juste to _count_ all the files seems pretty long to me... I think I'll say a word to the developers to let them tune their caching system a bit :) On Wed, 2009-05-06 at 08:48 -0400, Bill Moran wrote: > With lots of small files, the time involved is far less dependent on > the size of data, and much more dependent on the number of files, and > the resultant number of directory entries that need to be updated. > "Lots" isn't a particularly accurate count of the # of files, but if > you're talking web cache files, I'll guess they average 5k each, which > means you had 6 million files. df -i would have been more useful in > the output above. Thanks, noted for next time. Now it looks like that: Filesystem 1M-blocks Used Avail Capacity iused ifree %iused Mounted on /dev/da0s1f 128631 70544 47795 60% 1913875 15114219 11% /usr > This brings a number of questions up: > * Are you _sure_ softupdates is enabled on that partition? That's > going to make the biggest improvement in speed. According to "mount" output, yes. I found no specific message about that in the syslog or dmesg. > * Are these 7200RPM disks or 15,000? Again, going to make a big > difference. HP 146GB 6G SAS 10K SFF DP ENT HDD (15k were not available at the time the servers were ordered) ( http://h18004.www1.hp.com/products/servers/proliantstorage/serial/sas/index.html ) > * If apache was still running, is it possible that it was creating > enough disk activity to slow the activity down? Running > top -m io will show you how much disk IO each process is creating. Yes, apache was still running, but the activity was quite low (it was during the night, and the webpage doesn't get so many hits before 9 am local time) While watching "top -m io", the "du" or "find" takes between 80 and 99%, so I guess it's not the probleme here: PID UID VCSW IVCSW READ WRITE FAULT TOTAL PERCENT COMMAND 87996 1002 59 56 0 0 0 0 0.00% php 45389 1002 35 25 0 0 2 2 0.84% php 3964 1002 0 0 0 0 0 0 0.00% httpd 3822 1002 151 98 0 0 0 0 0.00% httpd 3005 1002 0 0 0 0 0 0 0.00% httpd 4129 1002 0 0 0 0 0 0 0.00% httpd 3971 1002 0 0 0 0 0 0 0.00% httpd 4231 1002 1 0 0 0 0 0 0.00% httpd 4132 0 234 5 234 0 0 234 97.91% find 98862 1002 1 0 0 0 0 0 0.00% top 609 0 0 0 0 0 0 0 0.00% snmpd [...] > * When you compared the speed to your laptop, did you delete 6 million > files from the laptop? If you deleted a single 30G file, then you're > comparing apples to atom bombs. Yes sorry, I know :) > If this is a directory that you blow away on a regular schedule, you'd > do much better to make it a dedicated partition and simply reformat > it. Yes, it is one of the best options. My initial goal was to delete all files older than N days by cron (find | xargs | rm, etc.), but if each cronjob takes 2 hours (and takes so much cpu time), it's probably not the best way. I'll make some more tests on an test-server later this week and speak with the devs. Thanks again for your very constructive feedback! Regards, Olivier
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1241616121.16418.109.camel>