Date: Sat, 1 Feb 2014 22:53:48 -0500 (EST) From: Rick Macklem <rmacklem@uoguelph.ca> To: J David <j.david.lists@gmail.com> Cc: freebsd-net@freebsd.org, Garrett Wollman <wollman@freebsd.org> Subject: Re: Terrible NFS performance under 9.2-RELEASE? Message-ID: <251642279.1400386.1391313228315.JavaMail.root@uoguelph.ca> In-Reply-To: <CABXB=RQDDn2AW0tU4hb9mS5hN5B52VFcCGiicv0SCkMB7SEKAg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
J David wrote:
> On Fri, Jan 31, 2014 at 6:16 PM, Rick Macklem <rmacklem@uoguelph.ca>
> wrote:
> > You can certainly try "-o rsize=61440,wsize=61440" (assuming a 4K
> > page size)
> > for the mount, if you'd like.
>
> This has previously been tested with all 4k steps between 16k and
> 32k.
> All of them perform worse than
>
> With 61440, NFS fails outright on the random read test:
>
> $ iozone -e -I -s 1g -r 4k -i 0 -i 2
>
Btw, if you do want to test with O_DIRECT ("-I"), you should enable
direct io in the client.
sysctl vfs.nfs.nfs_directio_enable=1
I just noticed that it is disabled by default. This means that your
"-I" was essentially being ignored by the FreeBSD client.
It also explains why Linux isn't doing a read before write, since
that wouldn't happen for direct I/O. You should test Linux without "-I"
and see if it still doesn't do the read before write, including a "-r 2k"
to avoid the "just happens to be a page size" case.
rick
> Iozone: Performance Test of File I/O
>
> Version $Revision: 3.420 $
>
> Compiled for 64 bit mode.
>
> Build: freebsd
>
> [...]
>
> Include fsync in write timing
>
> O_DIRECT feature enabled
>
> File size set to 1048576 KB
>
> Record Size 4 KB
>
> Command line used: iozone -e -I -s 1g -r 4k -i 0 -i 2
>
> Output is in Kbytes/sec
>
> Time Resolution = 0.000005 seconds.
>
> Processor cache size set to 1024 Kbytes.
>
> Processor cache line size set to 32 bytes.
>
> File stride size set to 17 * record size.
>
> random
> random bkwd record stride
>
> KB reclen write rewrite read reread read
> write read rewrite read fwrite frewrite fread freread
>
> 1048576 4 24688 23891
>
> Error reading block at 1073729536
>
> read: Bad file descriptor
>
>
> Upon using the -w option, which leaves the file intact on exit, it's
> possible to see that it's not even 1gig in length:
>
> $ ls -aln iozone.tmp
>
> -rw-r----- 1 1000 0 1073709056 Feb 1 01:18 iozone.tmp
>
>
> It's 32k short, which is a pretty surprising result.
>
> Thanks!
> _______________________________________________
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to
> "freebsd-net-unsubscribe@freebsd.org"
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?251642279.1400386.1391313228315.JavaMail.root>