From owner-freebsd-current@freebsd.org Sun Jan 7 09:36:01 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 95F71E62346 for ; Sun, 7 Jan 2018 09:36:01 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-149.reflexion.net [208.70.210.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5ACFB7EBCC for ; Sun, 7 Jan 2018 09:36:01 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 26540 invoked from network); 7 Jan 2018 09:35:53 -0000 Received: from unknown (HELO rtc-sm-01.app.dca.reflexion.local) (10.81.150.1) by 0 (rfx-qmail) with SMTP; 7 Jan 2018 09:35:53 -0000 Received: by rtc-sm-01.app.dca.reflexion.local (Reflexion email security v8.40.4) with SMTP; Sun, 07 Jan 2018 04:35:53 -0500 (EST) Received: (qmail 9478 invoked from network); 7 Jan 2018 09:35:53 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 7 Jan 2018 09:35:53 -0000 Received: from [192.168.1.25] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id F3817EC8BF3; Sun, 7 Jan 2018 01:35:52 -0800 (PST) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: USB stack Message-Id: <3F9697E3-3C25-45CB-804A-9C3607E434C4@dsl-only.net> Date: Sun, 7 Jan 2018 01:35:52 -0800 To: blubee blubeeme , FreeBSD Current X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Jan 2018 09:36:01 -0000 blubee blubeeme gurenchan at gmail.com wrote on Wed Jan 3 10:31:56 UTC 2018 : > Does FreeBSD current USB stack support usb >=3D 2.0 devices? >=20 > Testing out the USB devices support I get about 7.2-7.8 megabytes per > second which seems odd. FreeBSD machine: Pine64+ 2GB? Ryzen Threadripper 1950X? . . .? It would help to specify the type of system and its relevant properties (not just the processor). What independent channels are in use? Any? Or are the source and destination on the same channel at some stage? > I transferred about 30GB of audio from laptop The 30GB was on what type of device? Plugged in to what? What file system? > to Samsung usb class 10 usb > device connected to LG v30. What file system? And in another message (indicating the other direction of transfer compared to the above?): > I use the phone, LG V30 to record basically ungraded RAW video files = to the > microsd card; they are large files. > I transfer them to my computer copy a backup to the 1TB driver; then = do > edits/ color grading, etc in blender, > then I transfer the finished to another 1TB hdd for backup as well. So the LG v30 was plugged in as a USB device, effectively acting as a media reader/writer? What file system? (It seems unlikely that the LG v30 would use a FreeBSD native file system to record RAW video files.) Going the other direction of providing some examples of files copies for UFS. . . Note: These are based on head -r327485 with non-debug kernel builds. =20 Example performance copying /usr/src/ : (lots of small files on a fairly low-end FreeBSD machine) RPi2B V1.1 (with USB 2.0) One USB 3.0 powered hub (USB 2.0 compatible) with both: A) USB 3.0 SSD stick (USB 2.0 compatible) with the root file system B) 64 GB eMMC on a usdcard adapter, plugged into a USB 3.0 media reader/writer (USB 2.0 compatible). mount -o noatime in use for (A) and (B). UFS file systems. soft-updates enabled. cp -ax /usr/src/ /mnt/root/srccpy_test gstat -pd outputs, a few examples: dT: 1.007s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps = ms/d %busy Name 0 255 255 5501 1.9 0 0 0.0 0 0 = 0.0 48.0| da0 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da1 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da2 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da3 64 426 1 32 221.4 425 6287 140.4 0 0 = 0.0 62.9| da4 Note that the read kBps + write kBps means around 11MiByte/s for r+w. (There is only one USB connection at the RPi2B V1.1 here, not multiple, independent channels.) dT: 1.007s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps = ms/d %busy Name 0 393 393 5295 1.3 0 0 0.0 0 0 = 0.0 50.7| da0 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da1 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da2 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da3 46 102 2 64 2.9 100 2101 116.9 0 0 = 0.0 19.5| da4 The above last shows a period with around 7 MiBytes/s for r+w. dT: 1.007s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps = ms/d %busy Name 16 245 245 9761 37.4 0 0 0.0 0 0 = 0.0 77.4| da0 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da1 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da2 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da3 28 481 0 0 0.0 481 10809 95.1 0 0 = 0.0 93.7| da4 That last shows a period with around 20 MiBytes/s for r+w. (Probably copying fewer, bigger files at the time.) This might be around 8 MiBytes/s being written (mean rate overall for the copy). Example high end machine copying /usr/src/ to fast USB 3.0 SSD stick over USB 3.0, all UFS file system based: Ryzen Threadripper 1950X Running FreeBSD under Hyper-V under Windows 10 Pro Samsung 960 Pro 1TB NVMe root root UFS file system USB 3.0 SSD stick (USB 2.0 compatible) on a USB 3.0 connection (UFS) (These are Hyper-V "Physical disk drive" bindings, not NTFS files used as a virtual file system.) mount -o noatime in use for both. UFS file systems. soft-updates. cp -ax /usr/src/ /mnt/root/srccpy_test gstat -pd outputs, a couple of examples: dT: 1.023s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps = ms/d %busy Name 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| fd0 0 6519 6519 103339 0.1 0 0 0.0 0 0 = 0.0 35.5| da0 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da1 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da2 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| cd0 676 6635 0 0 0.0 6635 119898 43.6 0 0 = 0.0 43.0| da3 dT: 1.058s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps = ms/d %busy Name 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| fd0 0 6839 6839 106968 0.1 0 0 0.0 0 0 = 0.0 34.7| da0 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da1 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da2 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| cd0 1 6967 0 0 0.0 6967 133410 42.4 0 0 = 0.0 46.1| da3 In this context there are 2 independent channels and reading from one and writing from the other can potentially happen at the same time. Vastly faster than 8 MiBytes/s mean rate for writes. Example high end machine copying /usr/src/ from and to fast USB 3.0 SSD sticks over USB 3.0, all UFS file system based: Ryzen Threadripper 1950X Running FreeBSD under Hyper-V under Windows 10 Pro USB 3.0 SSD stick (USB 2.0 compatible) on a USB 3.0 connection (UFS) Another USB 3.0 SSD stick (USB 2.0 compatible) on a USB 3.0 connection = (UFS) (These are Hyper-V "Physical disk drive" bindings, not NTFS files used as a virtual file system.) mount -o noatime in use for both. UFS file systems. soft-updates. cp -ax /mnt/usr/src /media/root/srccpy_test gstat -pd outputs, a couple of examples: dT: 1.008s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps = ms/d %busy Name 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| fd0 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da0 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da1 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da2 1 2388 2388 36317 0.3 0 0 0.0 0 0 = 0.0 80.5| da3 0 2197 0 0 0.0 2197 38206 35.7 0 0 = 0.0 14.8| da4 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| cd0 dT: 1.070s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps = ms/d %busy Name 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| fd0 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da0 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da1 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da2 1 2443 2443 44537 0.3 0 0 0.0 0 0 = 0.0 82.5| da3 0 2309 0 0 0.0 2309 51142 36.0 0 0 = 0.0 18.7| da4 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| cd0 dT: 1.070s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps = ms/d %busy Name 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| fd0 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da0 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da1 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da2 0 2664 2664 65516 0.3 0 0 0.0 0 0 = 0.0 82.8| da3 0 2932 0 0 0.0 2932 84290 34.5 0 0 = 0.0 32.2| da4 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| cd0 dT: 1.047s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps = ms/d %busy Name 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| fd0 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da0 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da1 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da2 1 2542 2542 28571 0.3 0 0 0.0 0 0 = 0.0 77.7| da3 778 1803 13 428 0.4 1789 15985 27.8 0 0 = 0.0 8.0| da4 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| cd0 I doubt that this has independent channels at all stages. Faster than 8 MiBytes/s mean rate for writes. Example high end machine copying /usr/src/ from and to fast USB 3.0 SSD sticks over USB 2.0, all UFS file system based: Ryzen Threadripper 1950X Running FreeBSD under Hyper-V under Windows 10 Pro USB 3.0 SSD stick (USB 2.0 compatible) on a USB 2.0 connection (UFS) Another USB 3.0 SSD stick (USB 2.0 compatible) on a USB 2.0 connection = (UFS) (These are Hyper-V "Physical disk drive" bindings, not NTFS files used as a virtual file system.) mount -o noatime in use for both. UFS file systems. soft-updates. cp -ax /mnt/usr/src /media/root/srccpy_test gstat -pd outputs, a couple of examples: dT: 1.070s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps = ms/d %busy Name 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| fd0 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da0 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da1 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da2 1 812 812 10487 0.8 0 0 0.0 0 0 = 0.0 62.6| da3 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| cd0 0 949 5 150 66.9 944 13853 229.0 0 0 = 0.0 48.6| da4 dT: 1.042s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps = ms/d %busy Name 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| fd0 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da0 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da1 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da2 1 1093 1093 12903 0.7 0 0 0.0 0 0 = 0.0 71.5| da3 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| cd0 180 1003 8 246 94.9 996 9443 242.7 0 0 = 0.0 35.9| da4 dT: 1.068s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps = ms/d %busy Name 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| fd0 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da0 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da1 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| da2 1 617 617 15973 1.4 0 0 0.0 0 0 = 0.0 87.4| da3 0 0 0 0 0.0 0 0 0.0 0 0 = 0.0 0.0| cd0 1 1147 2 60 49.2 1145 17220 187.0 0 0 = 0.0 78.1| da4 Still solidly more than 8 MiBytes/s being written (mean rate). Having a few large files to copy instead should normally be faster in each of the earlier example contexts. But no hard drives or flash drives were involved: all SSDs of one form or another. Hard drive properties and file system fragmentation could make things much slower by comparison. Flash drives also can have issues. =3D=3D=3D Mark Millard markmi at dsl-only.net