Date: Tue, 10 Apr 2001 14:30:28 -0700 (PDT) From: Matt Dillon <dillon@earth.backplane.com> To: Zhihui Zhang <zzhang@cs.binghamton.edu> Cc: Brian Somers <brian@Awfulhak.org>, Jason DiCioccio <Jason.DiCioccio@Epylon.com>, "'current@freebsd.org'" <current@FreeBSD.ORG> Subject: Re: FW: Filesystem gets a huge performance boost Message-ID: <200104102130.f3ALUSv92398@earth.backplane.com> References: <Pine.SOL.4.21.0104101640100.624-100000@onyx>
next in thread | previous in thread | raw e-mail | index | archive | help
:Why VMIO dir works better if directories are placed close to each other? I
:think it only makes the cache data of an individual directory stay in the
:memory longer. Is there a way to measure the effectiveness of the disk
:drive's cache?
:
:-Zhihui
I wasn't being clear enough. There are two different things going
on. VMIO works better because it is capable of caching many, many
more directories using the VM page cache (all of physical memory) rather
then buffer malloc space (which is limited to a few megabytes). The
buffer cache in general will also operate better with the improved
locality of reference for the inodes underlying the files in the
directories.
The disk drive's cache works better because it will cache things
immediately that the VM system caches across a longer span of time.
If you are scanning a large number of small directories you can wind up
with a situation where each small directory is being held in a
file fragment (1K on disk). Several small directories may wind up
together on-disk, either contiguous or very close by. The VM page
cache cannot cache an entire filesystem block (potentially holdling
8 small directories) in a single operation oweing to the virtual,
file-oriented nature of the cache. However, the disk drive's cache CAN,
and will. The result is that you are suddenly able to read a huge number
of small directories with 1/10 the number of hard seeks on the disk that
you would otherwise have needed.
Prior to this change, all those small directories were not near each
other on-disk. Not only was the VM cache not able to cache the
surrounding blocks, but even if the disk drive did the data would wind
up being thrown away anyway because the surrounding blocks were not where
the other directories resided.
Disk drives are more limited by whatever seeking they have to do then by
the amount of data they are transfering. Anything that reduces seeking
has a huge effect on performance, and this is what is accomplished by
the patch.
-Matt
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200104102130.f3ALUSv92398>