Date: Fri, 22 Feb 2019 05:28:35 -0700 From: Rebecca Cran <rebecca@bluestop.org> To: Rajesh Kumar <rajfbsd@gmail.com>, FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: Any ideal way to run FIO benchmarking for NVMEe devices in FreeBSD Message-ID: <e8f62043-3e1d-707b-a496-366e02ffdecf@bluestop.org> In-Reply-To: <CAAO%2BANM34aY4g%2BFjPdt8F2sNo5e6N2dZdTDKavEJwvRbNJz=Gw@mail.gmail.com> References: <CAAO%2BANM34aY4g%2BFjPdt8F2sNo5e6N2dZdTDKavEJwvRbNJz=Gw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2/22/19 1:51 AM, Rajesh Kumar wrote: > 1. Should we use "posixaio" as the ioengine (or) something else? > 2. Should we use single thread (or) multiple threads for test? If > multiple threads, how can we decide on the optimal thread count? > 3. Should we use "raw device files" (Eg: nvme namespace file - > /dev/nvme0ns1) without filesystem (or) use a mounted filesystem with a > regular file (Eg: /mnt/nvme/test1). Looks like raw device files give better > numbers. > 4. Should we use a shared file (or) one file per thread? > 5. I believe 1Job should be fine for benchmarking. (or) should we try > multiple jobs? I just ran a quick test on a filesystem on my machine which has an M.2 NVMe drive, and it seems posixaio performs pretty poorly compared to the sync ioengine: around 700 MB/s vs. 1100 MB/s! I _was_ going to suggest using posixaio and setting iodepth to something like 32, but since it performs badly I'd suggest playing around with the numjobs parameter and seeing where the best performance is achieved - whether that's latency or throughput. On my system, single-threaded achieves ~530 MB/s, 8 jobs/threads 1150 MB/s and 32 1840 MB/s with a 4 KB block size. Bumping the block size from 4 KB to 16 KB makes the throughput more jumpy, but appears to average 2300 MB/s when used with 32 jobs. -- Rebecca Cran
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?e8f62043-3e1d-707b-a496-366e02ffdecf>