Date: Wed, 16 Jan 2013 14:05:28 +0200 From: Andriy Gapon <avg@FreeBSD.org> To: Nicolas Rachinsky <fbsd-mas-0@ml.turing-complete.org> Cc: freebsd-fs <freebsd-fs@FreeBSD.org> Subject: Re: slowdown of zfs (tx->tx) Message-ID: <50F69788.2040506@FreeBSD.org> In-Reply-To: <FD780217EA4548F187715AF1AAF2B91A@multiplay.co.uk> References: <CAFqOu6gWpMsWN0pTBiv10WfwyGWMfO9GzMLWTtcVxHixr-_i3Q@mail.gmail.com> <20130114094010.GA75529@mid.pc5.i.0x5.de> <CAFqOu6hxfGt_M6Jo9qWeifDz9YnNc_Bd9H-GEe4RYtutaPvH5w@mail.gmail.com> <20130114195148.GA20540@mid.pc5.i.0x5.de> <CAFqOu6jwJ4qhbOovN_NhzusdQJvrbvUC3g93sziR=Uw99SGenw@mail.gmail.com> <20130114214652.GA76779@mid.pc5.i.0x5.de> <CAFqOu6jKX-Ks6C1RK5GwZ51ZVUSnGSe7S99_EfK%2BfwLPjAFFYw@mail.gmail.com> <20130115224556.GA41774@mid.pc5.i.0x5.de> <CAFqOu6jJnWdbikPmE1-UML5i_x7meF%2BiyY=9WBRyv2j7AeOaSg@mail.gmail.com> <50F67551.5020704@FreeBSD.org> <20130116095009.GA36867@mid.pc5.i.0x5.de> <FD780217EA4548F187715AF1AAF2B91A@multiplay.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
on 16/01/2013 12:14 Steven Hartland said the following: > ----- Original Message ----- From: "Nicolas Rachinsky" > >> * Andriy Gapon <avg@FreeBSD.org> [2013-01-16 11:39 +0200]: >>> on 16/01/2013 02:16 Artem Belevich said the following: >>> > It appears that lots of threads are stuck in >>> > metaslab_activate->space_map_load_wait path. >>> >>> another thing to check - is your pool nearly full. >> >> Don't think so: >> NAME USED AVAIL REFER MOUNTPOINT >> pool1 5.52T 697G 11.9M /pool1 > > You only have ~11% free so yer it is pretty full ;-) Nicolas, just in case, Steve is not kidding. Those free hundreds of gigabytes could be spread over the terabytes and could be quite fragmented if the pool has a history of adding and removing lots of files. ZFS could be spending quite a lot of time in that case when it looks for some free space and tries to minimize further fragmentation. Empirical/anecdotal safe limit on pool utilization is said to be about 70-80%. You can test if this guess is true by doing the following: kgdb -w (kgdb) set metaslab_min_alloc_size=4096 If performance noticeably improves after that, then this is your problem indeed. -- Andriy Gapon
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?50F69788.2040506>