Date: Fri, 16 Jan 2015 10:42:28 +0200 From: Mihai Vintila <unixro@gmail.com> To: freebsd-stable@freebsd.org Subject: Re: Poor performance on Intel P3600 NVME driver Message-ID: <54B8CEF4.9040206@gmail.com> In-Reply-To: <54B8C7E9.3030602@gmail.com> References: <54B7F769.40605@gmail.com> <20150115175927.GA19071@zxy.spb.ru> <54B8C7E9.3030602@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Actually seems that i've was wrong vfs.zfs.top_maxinflight does not have any influence and this is confirmed by the nvmecontrol perftests. nvmecontrol perftest -n 32 -o read -s 512 -t30 nvme0ns1 Threads: 32 Size: 512 READ Time: 30 IO/s: 270212 MB/s: 131 nvmecontrol perftest -n 32 -o write -s 512 -t30 nvme0ns1 Threads: 32 Size: 512 WRITE Time: 30 IO/s: 13658 MB/s: 6 I was able to recover from the errors from previous message. They were cause by the fact that i've commented hw.nvme.per_cpu_io_queues=0 in loader.conf . After setting it back things got back to "slow normal" Performance is half of what it should be and issue seems to be the nvme driver. I'll try it on another OS to confirm it's not a hardware setting issue. Best regards, Vintila Mihai Alexandru On 1/16/2015 10:12 AM, Mihai Vintila wrote: > I've tried to increase it and obtained around 90k iops for write, but > now i only get : > > nvme0: WRITE sqid:6 cid:125 nsid:1 lba:3907028496 len:16 > nvme0: WRITE FAULTS (02/80) sqid:6 cid:125 cdw0:0 > nvme0: async event occurred (log page id=0x1) > nvme0: WRITE sqid:6 cid:126 nsid:1 lba:528 len:16 > nvme0: WRITE FAULTS (02/80) sqid:6 cid:126 cdw0:0 > nvme0: async event occurred (log page id=0x1) > nvme0: WRITE sqid:6 cid:127 nsid:1 lba:3907027984 len:16 > nvme0: WRITE FAULTS (02/80) sqid:6 cid:127 cdw0:0 > nvme0: async event occurred (log page id=0x1) > > Best regards, > Vintila Mihai Alexandru > > On 1/15/2015 7:59 PM, Slawa Olhovchenkov wrote: >> On Thu, Jan 15, 2015 at 07:22:49PM +0200, Mihai Vintila wrote: >> >>> /etc/sysctl.conf >>> >>> net.inet.tcp.fast_finwait2_recycle=1 >>> net.inet.ip.portrange.randomized=0 >>> net.inet.ip.portrange.first=1024 >>> net.inet.ip.portrange.last=65535 >>> net.inet.tcp.recvbuf_max=16777216 >>> net.inet.tcp.sendbuf_max=16777216 >>> net.inet.tcp.recvbuf_inc=65536 >>> vfs.zfs.vdev.trim_on_init=0 >>> #close time_wait connections at 2*7500ms >>> net.inet.tcp.msl=7500 >>> kern.ipc.somaxconn=4096 >>> net.inet.icmp.icmplim=2000 >>> >>> #zfs >>> vfs.zfs.txg.timeout=5 >>> vfs.zfs.prefetch_disable=1 >> >>> Any help to bring this device to proper speed will be welcomed. >> Do you try to increase vfs.zfs.top_maxinflight? >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?54B8CEF4.9040206>