Date: Tue, 6 Dec 2016 10:52:12 +0300 From: Alex Tutubalin <lexa@lexa.ru> To: "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org> Subject: 12-CURRENT: ZFS single stream/large files read performance degradation Message-ID: <477264f9-9263-6df8-4566-2911e84f2ad8@lexa.ru>
next in thread | raw e-mail | index | archive | help
Hi, I'm using freebsd/zfs box to store/access media files (mostly RAW photos /thousands/ and some video), so it is 'single user NAS' (connected via 10G to my workstation). This box has upgraded to 12-current several weeks ago, it was also updated with new disks and after that I see serious read speed performance degradation. This hurts me so much, that I've run tests with separate set of HDDs yesterday, using FreeBSD 10.1, 10.2, 10.3, 11.0, 12-20161130 booted from memstick (so, default settings, no adjustments). In details: Box configuration: Intel i5-2400 (3.1Ghz), 16GB RAM, LSI-9211-8i controller in dumb IT-mode. Disks: WD1003FBYX (6-drive and 7-drive RAIDZ2 were tested) Testing methodology: boot from memstick zpool create -O recordsize=nn ztest raidz2 /dev/gpt/DISK[0-5/6] (for 6 and 7 disk RAIDZ2) zfs create ztest/pool dd if=/dev/zero of=/ztest/pool/100gfile bs=1m count=100k dd if=/ztest/pool/100gfile of=/dev/null bs=1m And write numbers (first free digits :) into excel table. I've tested 5 versions of FreeBSD (all memstick images are from ftp.freebsd.org), two recordsizes (128k and 1m) and two RAIDZ2 disc count: 6 and 7. Results are here in table: http://blog.lexa.ru/2016/12/06/freebsd_zfs_read_speed_oni_ubili_kenni.html My blog post is in Russian, but the table has english captions, so speaks for itself. In short, for 6-drive array and 11.0 against 12-20163011 128k recordsize: 11.0 503Mb/s(read)/462 Mb/s(write) 12-Current: 380/189 1m recordsize: 11.0: 507/521 12-current: 515/203. It is possible to get single-stream read speed back in 12-Current by setting vfs.zfs.zfetch.max_distance to very high number (1 gigabyte or even more, there is significant difference between 800m and 1800m), but this is not acceptable solution because any small read will become very large. For now, I'm downgrading back to 11.0, but it looks like I need to report this degradation to whom it may concern :) -- Alex Tutubalin Web: http://blog.lexa.ru mailto:lexa@lexa.ru
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?477264f9-9263-6df8-4566-2911e84f2ad8>