Date: Sat, 3 Sep 2011 18:28:55 -0700 From: Jeremy Chadwick <freebsd@jdc.parodius.com> To: David Brodbeck <brodbd@uw.edu> Cc: freebsd-fs@freebsd.org Subject: Re: ZFSv28+NFSv4 poor file creation performance, "sync=disabled" has no effect Message-ID: <20110904012855.GA79580@icarus.home.lan> In-Reply-To: <CAHHaOuaK9Cedq5LFskQMjAH2qu5ukGRNK1DfHEYmZ_hRUbqdYg@mail.gmail.com> References: <CAHHaOuY=BEMrhYuzXtD5AtXG7niLXEO1yhO5P4EimcsLuTrLXw@mail.gmail.com> <14220705.747900.1315013770239.JavaMail.root@erie.cs.uoguelph.ca> <CAHHaOua8i8ZvRpbtev0knFJCW0m0i9PSD_w3U0J9FgU44oAW=A@mail.gmail.com> <CAHHaOuaK9Cedq5LFskQMjAH2qu5ukGRNK1DfHEYmZ_hRUbqdYg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Sep 03, 2011 at 05:00:05PM -0700, David Brodbeck wrote: > On Fri, Sep 2, 2011 at 6:36 PM, Rick Macklem <rmacklem@uoguelph.ca> wrote: > > > One post explained how disabling the ZIL can result in up to 5seconds worth > > of changes being lost if/when the server crashes. (A lot can change on a > > file > > system in 5sec. The NFS protocol assumes all fs changes related to a file > > creation are done before the server replies to the RPC. As such, disabling > > the > > ZIL does violate the protocol specs and means you are living dangerously.) > > > > ... > > Also, most other filesystems make no guarantees that data has been > flushed to the platters, only that it's been sent to the disk > hardware, so it really isn't a big increase in risk compared to our > pre-ZFS setup. Can you explain where you got this impression from? Please provide actual references/validation/proof for non-network filesystems. Disk-level filesystems are more or less at the mercy of the applications using them; if a programmer calls open(2) without O_FSYNC or O_SYNC, or does not explicitly flush I/O to disk in their software, then is the filesystem really to blame? Furthermore, the world does know of some disks/devices/products where executing BIO_FLUSH (which on ATA disks should get translated into the ATA FLUSH or FLUSH EXT command) does not truly write the data to the platters. Again: the filesystem is innocent, blame the disk/device/manufacturer. > The sync=disabled parameter was, in my understanding, added as a > per-filesystem way to achieve this risk/performance tradeoff, instead of > zil_disable, which was system-wide. I posted the question because I'm not > seeing any changes when I use sync=disabled. pjd@, mm@, and others will need to comment on this. -fs is the best list for this, so I'm a little surprised no key members have chimed in here. What we (the community) need clarification regarding is whether or not OpenSolaris and/or Illumos still provides the zil_disable tunable on those OSes; if they do, FreeBSD should provide the same (which means removable of zil_disable on FreeBSD is effectively a regression). If said OSes do not provide it, then FreeBSD should not provide it. If said OSes moved to using the sync parameter where it works, yet it does not work on FreeBSD, then that's a bug and a PR should be filed + discussion induced. And yes, I'm aware that I have given the "try using the sync parameter instead" advice to others here on FreeBSD lists. > This is a bit of a show-stopper for us because performance with full syncing > is not acceptable. I agree that it's not ideal, but we don't have the extra > drive bays to add SSDs at this point; also it seems like people have had a > lot of problems with SSDs and FreeBSD, so I'd prefer for the technology to > mature a bit before I risk my entire pool by putting a ZIL on one. The risks/complexities and performance concerns with SSDs when used as part of a ZFS filesystem on FreeBSD greatly differ from that of using an SSD as a dedicated intent log (ZIL) device. What exactly do you need ZFS for given your requirements? Is it that you want a simple-to-use LVM-like filesystem that offers a vast in-memory cache but without the data integrity bits? -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20110904012855.GA79580>