Date: Tue, 10 Jul 2007 08:40:50 -0400 From: Bill Vermillion <bv@wjv.com> To: Gore Jarold <gore_jarold@yahoo.com> Cc: freebsd-fs@freebsd.org Subject: Re: help needed - tuning a filesystem for rm and cp ? (MORE) Message-ID: <20070710124050.GA2184@wjv.com> In-Reply-To: <423307.86822.qm@web63003.mail.re1.yahoo.com> References: <423307.86822.qm@web63003.mail.re1.yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
"Ang utong ko ay sasabog sa sarap!" exclaimed Gore Jarold while reading this message on Mon, Jul 09, 2007 at 21:34 and then responded with: > > Some more information on my question... > The directories on the single mount point that I am > referring to are varied in depth and density - but > some of them have as many as a few million inodes in > them and can go 5-10 levels deep. > But that is not a rule - it is a large multi-user > system (think old school shell server) with hundreds > of users that can populate their home directories with > anything they want. The only thing I can say for sure > is that I am using 2.5 TB of space (out of 8 TB) and > am using 23.8 million inodes. > So it's not that dense with inodes at all, but there > is no telling how even a distribution that is - a > cp/rm target might not be represented well by the > average (ie. they might be very sparse or very dense) > So again, all is well, but I have these long 'cp' and > 'rm' processes that I would like to speed up, if > possible. > All else being equal, how do you optimize a system for > copying from one place to another on the same mount > point ? How do you optimize a system for fast file > deletion ? Are the two mutually exclusive ? I've not done this recently as I've not had to, but when you need to move files from one place to another >on the same filesystem< you don't have to copy them. Use cpio with the -pdlm arguments. This will make hard links from the original file locations to the new file location. Then you can just 'rm' the original directory entries and the new directory will be running just as the old one. -p is passthrough, -d makes directories as needed, -l make links but you must use the -p option, and -v is verbose. I always run this way as I like to see just exactly what is going on. The big upside of this is that you only build directories and it does NOT move the files themselves. Thus both the build new directory and remove the old directories are going to be much faster than copying files. It can be quite fast. Just checking the man pages for FreeBSD's cpio, I see the flags have greatly expanded, as I used to just use the -pdlm flags on System V.3 systems I was running. You may have to go thorugh manual carefuly to make sure of the flags. Bill -- Bill Vermillion - bv @ wjv . com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070710124050.GA2184>