Date: Tue, 04 Mar 2014 14:44:51 -0600 From: Karl Denninger <karl@denninger.net> To: freebsd-fs@freebsd.org Subject: Re: Is LZ4 compression of the ZFS L2ARC available in any RELEASE/STABLE? Message-ID: <53163B43.7010009@denninger.net> In-Reply-To: <CAJ7kQyFf19Un_TS=kW=T21HT%2BoabhsUhJij5oixQ2_uh0LvHRA@mail.gmail.com> References: <CAJ7kQyGTOuynOoLukXbP2E6GPKRiBWx8_mLEchk90WDKO%2Bo-SA@mail.gmail.com> <53157CC2.8080107@FreeBSD.org> <CAJ7kQyGQjf_WbY64bLVX=YfmJUfAd8i22kVbVhZhEWPMg7bbQw@mail.gmail.com> <5315D446.3040701@freebsd.org> <CAJ7kQyFf19Un_TS=kW=T21HT%2BoabhsUhJij5oixQ2_uh0LvHRA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] There's all sorts of issues here and as someone who uses Postgres on ZFS/FreeBSD in a heavy mixed-I/O environment I'm pretty aware of them. Getting the L2ARC cache OFF the spindles where the data set is will make a huge difference in performance in any mixed I/O (read and write) environment. The interleaving of that data on the base data set is murder on spinning rust due to the requirement to move the heads. I strongly recommend starting there. UFS *may* be faster, but don't count on it as the small I/O issue still is a serious factor and seek times becomes the overwhelming latency problem very quickly with small I/Os. Remember too that you still need data protection (e.g. RAID, Gmirror, etc.) Benchmark for your workload and see. If that's not enough going to SSDs erases the head-movement penalty completely. You may well see improvements in net I/O under mixed database loads as high as 20 or more *times* (not percent) what you get from rotating media for this reason, especially with the better SSD devices. I have clocked (under synthetic conditions) improvements in I/O latency on "first accesses" for data not in-RAMcache as high as *one hundred times* if the workload includes interleaved writes (e.g. large numbers of clients who both need to read and write at once.) On 3/4/2014 2:33 PM, Olav Gjerde wrote: > I managed to mess up who I replied to and Matthew replied back with a good > answer which I think didn't reach the mailing list. > > I actually have a problem with query performance in one of my databases > related to running PostgreSQL on ZFS. Which is why I'm so interested in > compression for the L2ARC Cache. The problem is random IO read were > creating a report were I aggregate 75000 rows takes 30 minutes!!! The table > that I query has 400 million rows though. > The dataset easily fit in memory, so if I run the same query again it takes > less than a second. > > I'm going to test UFS with my dataset, it may be a lot faster as you said. > Currently I've only tested ZFS with gzip, lz4 and no compression. Gzip and > no compression has about the same performance and LZ4 is about 20% > faster(for both read and write). LZ4 has a compressratio about 2.5 and > gzip9 has a compressratio that is about 4.5 > > Steven Hartland, thank you for your suggestion. I will try the 10-STABLE > then instead of a RELEASE. > > > On Tue, Mar 4, 2014 at 2:25 PM, Matthew Seaman <matthew@freebsd.org> wrote: > >> On 03/04/14 12:17, Olav Gjerde wrote: >>> This is really great, I wonder how well it plays together with PostgreSQL >>> and a SSD. >> You probably *don't* want to turn on any sort of compression for a >> Postgresql cluster's data area (ie. /usr/local/pgsql) -- and there are a >> bunch of other tuning things to make ZFS and Pg play well together, like >> adjusting the ZFS block size. The sort of small random IOs that RDBMSes >> do are hard work for any filesystem, but particularly difficult for ZFS >> due to the copy-on-write semantics it uses. It's a lot easier to get >> good performance on a UFS partition. >> >> On the other hand, ZFS has recently grown TRIM support, which makes it a >> much happier prospect on SSDs. >> >> Cheers, >> >> Matthew >> >> >> >> > -- -- Karl karl@denninger.net [-- Attachment #2 --] 0 *H 01 0 + 0 *H O0K030 *H 01 0 UUS10UFlorida10U Niceville10U Cuda Systems LLC1 0UCuda Systems LLC CA1/0- *H customer-service@cudasystems.net0 130824190344Z 180823190344Z0[1 0 UUS10UFlorida10UKarl Denninger1!0 *H karl@denninger.net0"0 *H 0 bi՞] MNԿawx?`)'ҴcWgR@BlWh+ u}ApdCF JVй~FOL }EW^bچYp3K& ׂ(R lxڝ.xz?6 &nsJ +1v9 v/( kqĪp[vjcK%fϻe?iq]z lyzFO'ppdX//Lw(3 JI A*S#՟H[f|CGqJKoo y.oEuOw$ /섀$삻J9b|A P~8]D1YI<"""Y^T2iQ2 b yH)] Ƶ0y$ _N 6XqMC 9 XgώjGTP"#nˋ"Bk1 00 U 0 0 `HB0 U 0, `HB OpenSSL Generated Certificate0 U |8 ˴d[20 U #0]Af4 U3x&^"408 `HB+)https://cudasystems.net:11443/revoked.crl0 *H gBwH] j\x`( &gW32"Uf^. ^Iϱ k!DQA g{(w /)\N'[ oRW @C HO>)XrTNɘ!u`xt5(=f\-l3<@C6mnhv##1ŃbH͍_Nq aʷ?rk$^9TIa!kh,D -ct1 0001 0 UUS10UFlorida10U Niceville10U Cuda Systems LLC1 0UCuda Systems LLC CA1/0- *H customer-service@cudasystems.net0 + ;0 *H 1 *H 0 *H 1 140304204451Z0# *H 1:ںk˥<y):0l *H 1_0]0 `He*0 `He0 *H 0*H 0 *H @0+0 *H (0 +71001 0 UUS10UFlorida10U Niceville10U Cuda Systems LLC1 0UCuda Systems LLC CA1/0- *H customer-service@cudasystems.net0 *H 101 0 UUS10UFlorida10U Niceville10U Cuda Systems LLC1 0UCuda Systems LLC CA1/0- *H customer-service@cudasystems.net0 *H <[k˲:$Vĭf1BY˱!A kNs2m\ ~u*M\d*@QIx}udYHj@I =8L gRƁZL}߶WkgcN `C6(8 6Q# &9h=f(J=wp tl|}E}9# y7DpAn}& аHv&mIQ A #9j'ouPhS93O73v#ä,Dvj^"]L>:ە\{d"( u "eHDk:/.VG" T/TvAp?W(_ng0K9uZq>9 jD\ϭna$.\Bqi +Ť!C0}FQՒlVnA1] +N~;~ĩTG<vC A m0C| B1
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?53163B43.7010009>