Date: Wed, 14 Jul 2010 10:14:23 +1000 From: Peter Jeremy <peterjeremy@acm.org> To: Martin Matuska <mm@FreeBSD.org> Cc: freebsd-current@FreeBSD.org Subject: Re: [CFT] ZFS v15 patch (version 3) Message-ID: <20100714001423.GA92530@server.vk2pj.dyndns.org> In-Reply-To: <4C364379.6020608@FreeBSD.org> References: <4C31C71C.2010606@FreeBSD.org> <20100708200446.GA33822@server.vk2pj.dyndns.org> <4C364379.6020608@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--u3/rZRmxL6MmkK24 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2010-Jul-08 23:30:33 +0200, Martin Matuska <mm@FreeBSD.org> wrote: >On 8. 7. 2010 22:04, Peter Jeremy wrote / nap=EDsal(a): >> Without patching arc_memory_throttle(), a system behaves especially >> poorly if it uses ZFS with any of mmap(2), UFS or NFS client - in my >> case, ports/mail/mairix was almost guaranteed to wedge the system. >> This is the problem that the following hack is intended to work around: >> perl -e '$x =3D "x" x 1000000;' >> >> =20 >Regarding ARC, you might want to try the revision 209227 from head that >is scheduled for MFC on 18.7.2010: >http://people.freebsd.org/~mm/patches/zfs/head-12636.patch I have done some testing with 8-STABLE with head-12636.patch and have managed to successfully reproduce a deadlock. The system is amd64 with 2GB RAM running a mixed UFS+ZFS environment. On a freshly booted system, I unmount/remount my ZFS /home and a UFS scratch filesystem that contains a 1.5GB file [ensuring there is no cached data from either FS]. I then dd(1) the 1.5GB UFS file to /dev/null and, once that is finished, start mairix on my ~6GB mail directory (on ZFS /home). After some time, I get the following 'systat -v' output: 4 users Load 9.30 8.97 8.33 Jul 14 09:49 Mem:KB REAL VIRTUAL VN PAGER SWAP PAG= ER Tot Share Tot Share Free in out in o= ut Act 122308 4436 721892 7876 59824 count =20 All 418376 7020 1074594k 38920 pages =20 Proc: Interrupts r p d s w Csw Trp Sys Int Sof Flt cow 4031 total 4 76 133k 3 194 30 135 zfod ata0 = irq14 ozfod 30 bge0 = irq16 99.8%Sys 0.2%Intr 0.0%User 0.0%Nice 0.0%Idle %ozfod atapc= i1 20 | | | | | | | | | | | daefr uhci0= ehci =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= prcfr uhci1 22 dtbuf totfr 2000 cpu0:= time Namei Name-cache Dir-cache 100000 desvn react 2001 cpu1:= time Calls hits % hits % 918 numvn pdwak 273 frevn pdpgs intrn Disks ad0 ad1 540404 wire KB/t 0.00 0.00 297512 act tps 0 0 1122808 inact MB/s 0.00 0.00 57876 cache %busy 0 0 1948 free 218192 buf Apart from normal daemons, the only processes running are vmstat, systat and mairix (via SSH sessions). Note that the system is running at virtually 100%sys with extremely low free memory and extremely high context switches but no obviously useful activity. At this stage, the system is basically unusable (I can't even kill the mairix process). My understanding of the problem is that the VM system sees "available" RAM as the sum of "cache" and "free" - which is reasonably high so there is no pressure to free up "inact" RAM. OTOH, ZFS ARC only counts "free" RAM - which is critically low so it throttles itself but has no way to get the VM system to move RAM onto the "free" list. --=20 Peter Jeremy --u3/rZRmxL6MmkK24 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.15 (FreeBSD) iEYEARECAAYFAkw9AV8ACgkQ/opHv/APuIeJGQCfeXn7XQ7VGWkcZ53X8rxqEr+g GLsAoMTUTWy8T2j7nKxmk6zf6hCQ4NL4 =NhxZ -----END PGP SIGNATURE----- --u3/rZRmxL6MmkK24--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20100714001423.GA92530>