Date: Wed, 23 Apr 2008 10:17:07 -0400 (EDT) From: Andrew Gallatin <gallatin@cs.duke.edu> To: freebsd-current@freebsd.org Subject: ZFS file caching question Message-ID: <18447.17658.759349.720175@grasshopper.cs.duke.edu>
next in thread | raw e-mail | index | archive | help
If I have, say, 512MB RAM and a 1GB file which is written or read sequentially on an otherwise idle system, I'd expect the last 512MB (- epsilon) of the file's pages to be cached in RAM. This is true for UFS, but it does not appear to be the case with ZFS (see example below). Can somebody explain how the arc cache in ZFS relates to the normal page cache used by traditional filesystems? Are ZFS filesystems cached exclusively in the arc cache, and not in the page cache? Is the arc cache per-filesystem, per-pool, or global for ZFS as a whole? Hmm.. Could this be the cause of the problems with ZFS and mmap'ed files? Thanks, Drew ### host has 406MB free before file is written %vmstat procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad8 ad10 in sy cs us sy id 0 0 4 74232K 406M 309 0 1 11 1747 2518 0 0 358 303 3267 0 3 97 %dd if=/dev/zero of=zot bs=1m count=1024 1024+0 records in 1024+0 records out 1073741824 bytes transferred in 21.076331 secs (50945386 bytes/sec) ### host has 26MB free after file is written %vmstat procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad8 ad10 in sy cs us sy id 0 0 6 74232K 26M 303 0 1 11 1707 2565 0 0 359 299 3252 0 3 97 ### at least the last 300MB or so are cached, and the disk is not hit ### when reading them. %dd if=zot of=/dev/null iseek=700 bs=1m 324+0 records in 324+0 records out 339738624 bytes transferred in 0.372665 secs (911646356 bytes/sec) % But with ZFS, this is not true: ### host has 412MB free before file is written %vmstat procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad8 ad10 in sy cs us sy id 0 0 6 84552K 412M 263 0 1 9 1520 2200 0 0 309 263 2854 0 3 97 %dd if=/dev/zero of=zot bs=1m count=1024 1024+0 records in 1024+0 records out 1073741824 bytes transferred in 17.447763 secs (61540372 bytes/sec) % ### host has 254MB free after file is written %vmstat procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad8 ad10 in sy cs us sy id 0 0 6 84552K 254M 249 0 1 9 1441 2069 0 0 299 251 2776 0 3 97 %dd if=zot of=/dev/null iseek=700 bs=1m 324+0 records in 324+0 records out 339738624 bytes transferred in 4.545527 secs (74741306 bytes/sec) ### the last 300MB of the file are not cached, and the disk is ### hit when reading them %vmstat procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad8 ad10 in sy cs us sy id 0 0 6 84552K 254M 249 0 1 9 1441 2069 0 0 299 251 2776 0 3 97 %dd if=zot of=/dev/null iseek=700 bs=1m 324+0 records in 324+0 records out 339738624 bytes transferred in 4.545527 secs (74741306 bytes/sec)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?18447.17658.759349.720175>