Date: Mon, 10 Aug 2015 16:58:09 +0100 From: Matthew Seaman <matthew@freebsd.org> To: freebsd-questions@freebsd.org Subject: Re: Poor I/O performance; maybe ZFS? Message-ID: <55C8CA11.4030906@freebsd.org> In-Reply-To: <B94A0CB6-FEDA-41AA-A335-14988E5662CC@freebsd.org> References: <B94A0CB6-FEDA-41AA-A335-14988E5662CC@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --fr17XCTH47WVbBc75g5eW03iiD8WKExDn Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 08/10/15 16:12, Julio Merino wrote: > Hello, >=20 > I have a PowerMac G5 running 11.0-CURRENT (upgraded yesterday). The ma= chine is a dual-CPU 2GHz G5 with 6GB of RAM and two drives: >=20 > ada0 at ata2 bus 0 scbus2 target 0 lun 0cd0 at ata0 bus 0 scbus0 target= 0 lun 0 > ada0: <ST3160023AS 3.05> ATA-6 SATA 1.x device > ada0: Serial Number 3JS478VL > ada0: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes) > ada0: 152627MB (312581808 512 byte sectors: 16H 63S/T 16383C) > ada1 at ata3 bus 0 scbus3 target 0 lun 0 > ada1: <WDC WD6400AAKS-75A7B0 01.03B01> ATA8-ACS SATA 2.x device > ada1: Serial Number WD-WMASY2696897 > ada1: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes) > ada1: 610480MB (1250263728 512 byte sectors: 16H 63S/T 16383C) >=20 > I have a UFS file system on ada0s4 for / and a ZFS pool on ada1s2 with = file systems for /home and poudriere. >=20 > For a while, I have felt that the system is incredibly sluggish. For > example, a simple "make buildworld" with up-to-date obj, which should b= e > I/O bound, takes forever (something like 20-30 minutes). This is runnin= g > with src and obj on ZFS. "svn update" is similarly infuriatingly slow. Beware: 'svn update' is frequently limited by network bandwidth, so not really a good test of your filesystems. > Today, I decided to install bonnie and ran a couple of tests using "-s = 4096". >=20 > -------Sequential Output-------- ---Sequential Input-- --= Random-- > -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --= Seeks--- > Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /= sec %CPU > ada0 4196 43973 62.4 46207 37.7 43259 41.9 89210 96.8 280589 95.9 1= 7038.2 103.1 > tank 4196 54093 72.2 27596 93.0 21478 90.7 64214 83.5 65358 50.8 12= 6.7 3.7 >=20 > Note the vast difference on the numbers, especially random seeks. >=20 > In case this matters, this is how ada1s2 looks like: >=20 > 1. Name: ada1s2 > Mediasize: 640135019520 (596G) > Sectorsize: 512 > Stripesize: 0 > Stripeoffset: 9216 > Mode: r1w1e1 > rawtype: FreeBSD-ZFS > label: 1 > length: 640135019520 > offset: 9216 > type: freebsd-zfs > index: 1 > end: 1250263727 > start: 18 >=20 > Any obvious things I should look at? 6GB is not a great deal when we're talking ZFS. For a mixed read/write workload like compiling, ZFS is going to be working its read cache 'ARC' pretty hard. It's also going to be competing with the traditional buffer cache for the available memory. Try installing 'zfs-stats' from ports and look at the output of 'zfs-stats -A' and 'zfs-stats -E' for clues. You should also *limit* the amount of memory ZFS can dedicate to the ARC -- counter intuitive, but memory dedicated to ARC is memory that the compiler (for instance) cannot be using. I'd be looking at setting vfs.zfs.arc_max=3D"3G" in /boot/loader.conf were I in your shoes -- or possibly even less. Whatever you do, you aren't going to get stellar ZFS performance out of a machine with that small an amount of memory, and pushing all the iops through a single SATA-1 drive, but these sort of measures should at least improve things. Beyond that there's a bunch of settings you can fiddle with documented he= re: https://www.freebsd.org/doc/handbook/zfs-advanced.html https://wiki.freebsd.org/ZFSTuningGuide Cheers, Matthew PS. poudriere doesn't actually need ZFS nowadays. --fr17XCTH47WVbBc75g5eW03iiD8WKExDn Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJVyMoRAAoJEABRPxDgqeTn7BkP/3skqn3rJ8o7NeF7spusHjEO TGznllG+KOJ3YjpC++uN0G3I5dDzblrpMNmx0VJFMY8IwuYnHd68zGSI/+epUG6h ucGLKihLAyEXz3CLAuQ4gaRpi9qRBP1ZRHTMyEGau/ayKFms7+wmxwlv49W7avDR rCD6Y8+rsY0eDgBMgqN1meuKz5wcVPNs0jBMwtN1b4DbATzU4QVLxpBqo3Mdjkp+ 8wZIw/i7E02Ugv6WbcxWRQlOQmsh1XIRaIdVX5ywIRAsPTVZHwni2sw0e7hwVL2S QBa2vamC13aSS/4aUeybMJ5Jr00oaQ56r5ctpPNuZAbmdhfMdPmyOLe7Kkg6dW9h IYvLu0+nfHUllmp7iFze3mszFLyGNDcguxCv3jIX9U+okFjTohXTzyeQhs9OGX5a Ll2aaRAkPYuDAQpIV2G++bvOVA6BTovhzMoaszJzN/pYB4OvpoD2DdZXm1AmVgea YDBWM0wYfFo1LEmlpGjN3g+pTR9L2Mx0S1W2jpJKo605DoQPE7aB0TvwOXJ8tUSW SkBgxRyXOo7BXPCpXEhUc/3+3wBYGEMbswWb77B0aTNZ9mR3ECE25NWs8SsWztkh aY4/VCKNU3mxTP7NWi5ABdbpg6CYTHEAtODrHtSBlba4mgEUYKWVi5bUYCfZQHTV pF8+jBJh/WW1eC33ZCs0 =42sZ -----END PGP SIGNATURE----- --fr17XCTH47WVbBc75g5eW03iiD8WKExDn--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?55C8CA11.4030906>