Date: Thu, 8 Jun 2023 00:24:55 -0600 From: Warner Losh <imp@bsdimp.com> To: Rebecca Cran <rebecca@bsdio.com> Cc: FreeBSD CURRENT <freebsd-current@freebsd.org> Subject: Re: Seemingly random nvme (nda) write error on new drive (retries exhausted) Message-ID: <CANCZdfrdRN%2BzGkk=V9Sk=uoZYgtEkRx9G5MKaJQ9tPMARDwjEA@mail.gmail.com> In-Reply-To: <5b52fc08-fb5a-900e-b98c-817a4ab79846@bsdio.com> References: <5b52fc08-fb5a-900e-b98c-817a4ab79846@bsdio.com>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] On Wed, Jun 7, 2023 at 11:12 PM Rebecca Cran <rebecca@bsdio.com> wrote: > I got a seemingly random nvme data transfer error on my new arm64 Ampere > Altra machine, which has a Samsung PM1735 PCIe AIC NVMe drive. > > Since it's a new drive and smartctl doesn't show any errors I thought it > might be worth mentioning here. > > I'm running 14.0-CURRENT FreeBSD 14.0-CURRENT #0 main-n263139-baef3a5b585f. > > > dmesg contains: > > nvme0: WRITE sqid:16 cid:126 nsid:1 lba:2550684560 len:8 > nvme0: DATA TRANSFER ERROR (00/04) crd:0 m:0 dnr:0 sqid:16 cid:126 cdw0:0 > (nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 > cdw=98085b90 0 7 0 0 0 > (nda0:nvme0:0:0:1): CAM status: CCB request completed with an error > (nda0:nvme0:0:0:1): Error 5, Retries exhausted > > > nvmecontrol identify nvme0 shows: > > Vendor ID: 144d > Subsystem Vendor ID: 144d > Model Number: SAMSUNG MZPLJ6T4HALA-00007 > Firmware Version: EPK9CB5Q > Recommended Arb Burst: 8 > IEEE OUI Identifier: 00 25 38 > Multi-Path I/O Capabilities: Multiple controllers, Multiple ports > Max Data Transfer Size: 131072 bytes > Sanitize Crypto Erase: Supported > Sanitize Block Erase: Supported > Sanitize Overwrite: Not Supported > Sanitize NDI: Not Supported > Sanitize NODMMAS: Undefined > Controller ID: 0x0041 > Version: 1.3.0 > PCIe 3 or PCIe 4? So the only documented reason for this error is if we setup the memory wrong such that the drive couldn't start a transfer from the specified address. This seems weird to me... But in the prior paragraph it talks about other types of aborts that need software intervention. If this is a transient error, then maybe we should retry it as part of the data recovery. Unless this do not retry bit is set. which it isn't. I wonder this is retried 5 times or not before generating the error... Warner [-- Attachment #2 --] <div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jun 7, 2023 at 11:12 PM Rebecca Cran <<a href="mailto:rebecca@bsdio.com">rebecca@bsdio.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">I got a seemingly random nvme data transfer error on my new arm64 Ampere <br> Altra machine, which has a Samsung PM1735 PCIe AIC NVMe drive.<br> <br> Since it's a new drive and smartctl doesn't show any errors I thought it <br> might be worth mentioning here.<br> <br> I'm running 14.0-CURRENT FreeBSD 14.0-CURRENT #0 main-n263139-baef3a5b585f.<br> <br> <br> dmesg contains:<br> <br> nvme0: WRITE sqid:16 cid:126 nsid:1 lba:2550684560 len:8<br> nvme0: DATA TRANSFER ERROR (00/04) crd:0 m:0 dnr:0 sqid:16 cid:126 cdw0:0<br> (nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 <br> cdw=98085b90 0 7 0 0 0<br> (nda0:nvme0:0:0:1): CAM status: CCB request completed with an error<br> (nda0:nvme0:0:0:1): Error 5, Retries exhausted<br> <br> <br> nvmecontrol identify nvme0 shows:<br> <br> Vendor ID: 144d<br> Subsystem Vendor ID: 144d<br> Model Number: SAMSUNG MZPLJ6T4HALA-00007<br> Firmware Version: EPK9CB5Q<br> Recommended Arb Burst: 8<br> IEEE OUI Identifier: 00 25 38<br> Multi-Path I/O Capabilities: Multiple controllers, Multiple ports<br> Max Data Transfer Size: 131072 bytes<br> Sanitize Crypto Erase: Supported<br> Sanitize Block Erase: Supported<br> Sanitize Overwrite: Not Supported<br> Sanitize NDI: Not Supported<br> Sanitize NODMMAS: Undefined<br> Controller ID: 0x0041<br> Version: 1.3.0<br></blockquote><div><br></div><div>PCIe 3 or PCIe 4?</div><div><br></div><div>So the only documented reason for this error is if we setup the memory wrong</div><div>such that the drive couldn't start a transfer from the specified address. This seems</div><div>weird to me... But in the prior paragraph it talks about other types of aborts that</div><div>need software intervention. If this is a transient error, then maybe we should retry</div><div>it as part of the data recovery. Unless this do not retry bit is set. which it isn't. I wonder</div><div>this is retried 5 times or not before generating the error...</div><div><br></div><div>Warner</div><div> </div></div></div>help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfrdRN%2BzGkk=V9Sk=uoZYgtEkRx9G5MKaJQ9tPMARDwjEA>