Date: Sun, 6 Feb 2022 11:20:51 -0700 From: Warner Losh <imp@bsdimp.com> To: Sean Bruno <sbruno@freebsd.org> Cc: freebsd-current <freebsd-current@freebsd.org> Subject: Re: USB Disk Stalls on -current Message-ID: <CANCZdfqG-%2B9dfFz-%2BeezZaqbPQN5-mQpw%2B214CkiKC%2B_kmW2ig@mail.gmail.com> In-Reply-To: <7e8459e4-d708-7750-402c-cda2adf6199f@freebsd.org> References: <7e8459e4-d708-7750-402c-cda2adf6199f@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000e3225705d75d8b1d Content-Type: text/plain; charset="UTF-8" On Sun, Feb 6, 2022 at 10:15 AM Sean Bruno <sbruno@freebsd.org> wrote: > I'm doing something "gross" with ZFS & Plex on a little Intel NUC that I > have here at the house to provide me with a nice little NAS at home. > I'm using 2x USB2 external disks as the mirror. > > I noted that the two USB2 disks I'm using in a mirror seem to "stall" > from time to time and its not clear to me why. > > I'd like to poke further into the USB system but I'm not sure where I > should start to see if there is something amiss with the hardware (e.g. > the disks suck) or if FreeBSD is losing track of something during I/O > leading to a stall/timeout. > > I'm not seeing data loss or anything, I just note from time to time > during large file transfers that the clanking/grinding sound of the > spinning rust on my desk completely stops, the encoding of the video > files stops (so its waiting for a read to complete) and its gets much > quieter in my office. :-) > So there's some tools you can use. For usb, there's usbdump that can get you the USB transactions. I've not used it enough to give more details here. This will let you know what's going on, and when, on the USB endpoint. You can also enable the CAM_IOSCHED stuff. This will allow you to get latency measurements for 'requests in the sim' which basically will tell you what your latency spread is for the drives. This will tell you if things are getting caught up in the USB layer, or after CAM's da driver completes the I/O request (granted, that's almost certainly not happening, but it will help you figure out what's going on and put numbers to the oddities you are seeing). Also, make sure you have good cables. I've had lots of hicups over the years from dodgy USB cables. Also make sure you have good, high quality enclosures. Many from the USB2 time-period are sketchy at best and I went through several at one point trying to find a good one. I'd be tempted to get USB 3 enclosures. I've had better luck with USB3 gear than USB2 gear here, but you need a USB-3 controller to get USB-3 speeds which might not be compatible with the NUC's built-in stuff (though my NUC has one USB3 port, there's lots of different models). Usually, though, I see weirdness associated with dmesg messages from usb, cam, etc when the hardware is on the sketch end. Warner --000000000000e3225705d75d8b1d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">= <div dir=3D"ltr" class=3D"gmail_attr">On Sun, Feb 6, 2022 at 10:15 AM Sean = Bruno <<a href=3D"mailto:sbruno@freebsd.org">sbruno@freebsd.org</a>> = wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0= px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">I'm d= oing something "gross" with ZFS & Plex on a little Intel NUC = that I <br> have here at the house to provide me with a nice little NAS at home. <br> I'm using 2x USB2 external disks as the mirror.<br> <br> I noted that the two USB2 disks I'm using in a mirror seem to "sta= ll" <br> from time to time and its not clear to me why.<br> <br> I'd like to poke further into the USB system but I'm not sure where= I <br> should start to see if there is something amiss with the hardware (e.g. <br= > the disks suck) or if FreeBSD is losing track of something during I/O <br> leading to a stall/timeout.<br> <br> I'm not seeing data loss or anything, I just note from time to time <br= > during large file transfers that the clanking/grinding sound of the <br> spinning rust on my desk completely stops, the encoding of the video <br> files stops (so its waiting for a read to complete) and its gets much <br> quieter in my office.=C2=A0 :-)<br></blockquote><div><br></div><div>So ther= e's some tools you can use. For usb, there's usbdump that can</div>= <div>get you the USB transactions. I've not used it enough to give more= details</div><div>here. This will let you know what's going on, and wh= en, on the USB endpoint.</div><div><br></div><div>You can also enable the C= AM_IOSCHED stuff. This will allow you to get latency</div><div>measurements= for 'requests in the sim' which basically will tell you what your<= /div><div>latency spread is for the drives. This will tell you if things ar= e getting caught</div><div>up in the USB layer, or after CAM's da drive= r completes the I/O request</div><div>(granted, that's almost certainly= not happening, but it will help you figure out</div><div>what's going = on and put numbers to the oddities you are seeing).</div><div><br></div><di= v>Also, make sure you have good cables. I've had lots of hicups=C2=A0ov= er the</div><div>years from dodgy USB cables. Also make sure you have good,= high quality</div><div>enclosures. Many from the USB2 time-period are sket= chy at best and I</div><div>went through several at one point trying to fin= d a good one. I'd be tempted to</div><div>get USB 3 enclosures. I'v= e had better luck with USB3 gear than USB2 gear</div><div>here, but you nee= d a USB-3 controller to get USB-3 speeds which might not</div><div>be compa= tible with the NUC's built-in stuff (though my NUC has one USB3</div><d= iv>port, there's lots of different models).</div><div><br></div><div>Us= ually, though, I see weirdness associated with dmesg messages from</div><di= v>usb, cam, etc when the hardware is on the sketch end.</div><div><br></div= ><div>Warner</div></div></div> --000000000000e3225705d75d8b1d--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfqG-%2B9dfFz-%2BeezZaqbPQN5-mQpw%2B214CkiKC%2B_kmW2ig>