Date: Sun, 17 Jan 2010 18:21:20 -0500 From: Garrett Moore <garrettmoore@gmail.com> To: freebsd-stable@freebsd.org Subject: Re: ZFS performance degradation over time Message-ID: <7346c5c61001171521w1ca4738w98e8fcca24643cda@mail.gmail.com> In-Reply-To: <7346c5c61001091706m45a3a2a5k3ca8bb0c4bec5ea8@mail.gmail.com> References: <7346c5c61001030842r7dc76199y51e4c1c90a3eea6e@mail.gmail.com> <hi2nsf$do5$1@ger.gmane.org> <7346c5c61001080831w375d158fu5b1996ee58cb0f8d@mail.gmail.com> <ed91d4a81001080940ybfe1a63nc13bee5bd3136984@mail.gmail.com> <2ae8edf31001091051l156cb57alf549cfe06f1c7197@mail.gmail.com> <7346c5c61001091640w1a0ffb5y3ffe97cb88c76436@mail.gmail.com> <20100110005621.GA45015@icarus.home.lan> <7346c5c61001091706m45a3a2a5k3ca8bb0c4bec5ea8@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
I upgraded my system to 8GB of ram to see if that would help. It hasn't made much of a difference. After having rTorrent running for a while, my performance again tanked. Around 6.5GB of memory was showing as 'Active' according to top. Copying a file from the zpool to another location on the zpool had the following performance (from zpool iostat 1): tank 2.57T 8.31T 11 215 273K 745K tank 2.57T 8.31T 22 97 816K 540K tank 2.57T 8.31T 6 67 13.5K 887K tank 2.57T 8.31T 5 101 639K 190K tank 2.57T 8.31T 14 81 1.14M 154K tank 2.57T 8.31T 6 152 12.5K 897K tank 2.57T 8.31T 11 153 143K 790K tank 2.57T 8.31T 3 172 134K 566K tank 2.57T 8.31T 3 105 7.48K 699K tank 2.57T 8.31T 5 136 138K 383K Combined read/write of <2MB/s -- pretty pathetic. I then tried Artem's trick, running ` perl -e '$x="x"x3000000000'` to force a swapout. This command completed in about 10 seconds, and I then had >5GB of memory showing as 'Free' according to top. Checking zpool iostat 1 again showed: tank 2.57T 8.31T 375 477 46.1M 57.9M tank 2.57T 8.31T 18 472 1.75M 44.8M tank 2.57T 8.31T 129 0 16.1M 0 tank 2.57T 8.31T 428 0 53.2M 0 tank 2.57T 8.31T 262 947 31.8M 103M tank 2.57T 8.30T 80 105 9.61M 196K tank 2.57T 8.30T 612 0 75.8M 0 tank 2.57T 8.30T 155 951 18.4M 103M tank 2.57T 8.30T 662 0 82.1M 0 tank 2.57T 8.30T 176 945 21.1M 103M Which is obviously much better, and a respectable rate of performance. This was all done during the same single `copy` command - I did not stop/restart the copy when running the perl oneliner. So it seems based on this that ZFS is keeping too much data cached and not being smart about when to release 'old' cache entries in favour of new ones. Any suggestions at this point? The only tuning I have done so far is to disable prefetch, since my primary usage is streaming HD media, and prefetch has been known to cause problems in that situation.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7346c5c61001171521w1ca4738w98e8fcca24643cda>