From owner-freebsd-hackers@freebsd.org Fri Feb 22 08:51:24 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F0C214E0D09 for ; Fri, 22 Feb 2019 08:51:24 +0000 (UTC) (envelope-from rajfbsd@gmail.com) Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2F8D78A025 for ; Fri, 22 Feb 2019 08:51:23 +0000 (UTC) (envelope-from rajfbsd@gmail.com) Received: by mail-wr1-x42b.google.com with SMTP id f14so1473894wrg.1 for ; Fri, 22 Feb 2019 00:51:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=XhLlGH4dRPH6d2+SAyNEn8CZTBAQz+bqwUqSoGRrAUM=; b=tOfPQ8LD5pcfOteOyyC2a3fGmtISIYU/gN85zdIu+PddIg55cKJyBU2BZ8yD70W/Wy dl4FbY1Z+g+2gt92PLQzmMIY/tEV8YvL6LPlo8qGXlgzEfa3iecfqKoeW/Zv1++fds+R lwqBNF17qEY/Mghvb+X41fF4RPVGOV9iM6Gwzpb2mVIEPIC94zpemtHoN8Pa6o38pekg IEUSjzO99eVIRpWX6TBTkW56hFsgm3hlgjx2kcktx70aHA6OMMoG3/Ck2r4z/Df/j366 pkuoz5RnmQRx0cbsUTHff8dwh/UauJQZ8SlHjQ5R/CAAAsBgKNShMrVhtRiMG8MU0DKg 0q2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=XhLlGH4dRPH6d2+SAyNEn8CZTBAQz+bqwUqSoGRrAUM=; b=Amyuxm/CTMYbayRwXet1wtTR3ekjqLNikG3iNEYsSd+Jp6I1DsJBkkQ/n8psGgOB+1 6W6tcsbaUQoRfPtMJan479Cq5CTcFCeA/uRUF/KLQTQZaxvfoRXosKe+t2piIhG27YaZ isbExOGldFOXGt6s/OmXTQ+o8aSxXXgdtRYYlkACgDaULlkjUT20/ujKTgbwLHqv9mpI nK6ed0/H1bmzRajWM+sg7ZEJXWtcnGxcIDw1q++sN85X/oCGGJrFvtSnhze14Sl792xJ qihPQlkH8H9GfGQctn1901P6Np2tm0k8U1ODbA5Y6VCDoeAl0Egey+kZ3gUkryvtmd8g j+jw== X-Gm-Message-State: AHQUAuZFeM5tB7N2LHpbvdZ9E3ao8OfDriNKSkfRE7D+9PzQlAPXufqz NdtuGM15A74KFylGRtbiMwe14pgxgHoOuGqjIQzp/eRa22Q= X-Google-Smtp-Source: AHgI3IaXo2+XTzCk35RGfwtfle3u916LAa38WSadb3uwmMRBcU2d9OMfWnHLWefhHVs4phCDM0NmE9HxLZKW7OfYON0= X-Received: by 2002:adf:9d14:: with SMTP id k20mr1950782wre.97.1550825481680; Fri, 22 Feb 2019 00:51:21 -0800 (PST) MIME-Version: 1.0 From: Rajesh Kumar Date: Fri, 22 Feb 2019 14:21:10 +0530 Message-ID: Subject: Any ideal way to run FIO benchmarking for NVMEe devices in FreeBSD To: FreeBSD Hackers X-Rspamd-Queue-Id: 2F8D78A025 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=tOfPQ8LD; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rajfbsd@gmail.com designates 2a00:1450:4864:20::42b as permitted sender) smtp.mailfrom=rajfbsd@gmail.com X-Spamd-Result: default: False [-6.23 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[b.2.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; TO_MATCH_ENVRCPT_ALL(0.00)[]; IP_SCORE(-2.74)[ip: (-9.37), ipnet: 2a00:1450::/32(-2.26), asn: 15169(-1.99), country: US(-0.07)]; NEURAL_HAM_SHORT(-0.48)[-0.478,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Feb 2019 08:51:24 -0000 Hi, I am trying to run FIO benchmark test with NVMe devices and see how FreeBSD performs. There are lot of variables and combination. So, can anyone suggest a Ideal way to do FIO benchmarking in FreeBSD? My intent is to check what is the maximum throughput and IOPS the device delivers in FreeBSD. Few questions regarding the same, 1. Should we use "posixaio" as the ioengine (or) something else? 2. Should we use single thread (or) multiple threads for test? If multiple threads, how can we decide on the optimal thread count? 3. Should we use "raw device files" (Eg: nvme namespace file - /dev/nvme0ns1) without filesystem (or) use a mounted filesystem with a regular file (Eg: /mnt/nvme/test1). Looks like raw device files give better numbers. 4. Should we use a shared file (or) one file per thread? 5. I believe 1Job should be fine for benchmarking. (or) should we try multiple jobs? Please let me know your suggestions. Also, please suggest performance tuning methods for NVMe and storage devices in general. Thanks, Rajesh.