Date: Mon, 8 Jan 2018 17:15:42 +0800 From: blubee blubeeme <gurenchan@gmail.com> To: Mark Millard <markmi@dsl-only.net> Cc: FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: USB stack Message-ID: <CALM2mE=iKr0gEqCxM8KWDZYLipVQmzRdGYTv72CiQoemcB9YtA@mail.gmail.com> In-Reply-To: <899567F7-0C02-4A60-ADA0-5F43A6CF594B@dsl-only.net> References: <3F9697E3-3C25-45CB-804A-9C3607E434C4@dsl-only.net> <CALM2mEnaA7zDVfONFQEBtC2WghbRFoFW2iPpmBKohP1pd45CcQ@mail.gmail.com> <0AB4ED58-E01A-4761-B6EF-4D56F8CA21E3@dsl-only.net> <F9A5DBFF-79C8-417D-9B6D-0788976B558C@dsl-only.net> <FA0FA34D-B941-43DB-8885-902B502A5442@dsl-only.net> <CALM2mEnnc4bzpFoRA_F_FayiBuGLwbwzyf-Gb-U6vmQ0RuGuYQ@mail.gmail.com> <1F10CBFE-1AAC-4307-976A-0CDA80EDC616@dsl-only.net> <CALM2mEkxqQMfzKQ7CmBnGmcu11evt129vAjWjm654PG=FJs2kA@mail.gmail.com> <2B2F495A-9CBE-4366-B049-0EC5EF7F9629@dsl-only.net> <899567F7-0C02-4A60-ADA0-5F43A6CF594B@dsl-only.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Jan 8, 2018 at 3:20 PM, Mark Millard <markmi@dsl-only.net> wrote: > [The involvement of sysutils/fusefs-simple-mtpfs > and sysutils/fusefs-libs and multimedia/libmtp > in order to use the Media Transfer Protocol (MTP) > against the LG v30 likely explains much of the > performance difference vs. local UFS file > systems. I provide some related notes.] > > blubee blubeeme gurenchan at gmail.com wrote on > Mon Jan 8 05:17:24 UTC 2018 : > > [Note: the original was in a reply to a Jon Brawn > post. I've merged it back with my post.] > > > On 2018-Jan-7, at 11:46 AM, Mark Millard <markmi at dsl-only.net> wrote: > > > >> On 2018-Jan-7, at 10:23 AM, blubee blubeeme <gurenchan at gmail.com> > wrote: > >> > >> > >> > >>> On Mon, Jan 8, 2018 at 1:41 AM, Mark Millard <markmi at dsl-only.net> > wrote: > >>> > >>>> On 2018-Jan-7, at 7:50 AM, blubee blubeeme <gurenchan at gmail.com> > wrote: > >>>> > >>>>> I ran this test and here's some results. > >>>>> gstat -pd images: > >>>>> > >>>>> 18GB file from laptop to phone: https://imgur.com/a/7iHwv > >>>>> 18GB file from laptop to ssd: https://imgur.com/a/40Q6V > >>>>> multiple small files from laptop to phone: https://imgur.com/a/B4v4y > >>>>> multiple small files from laptop to ssd: https://imgur.com/a/mDiMu > >>>>> > >>>>> The files are missing timestamps but the originals were taken with > scrot and have timestamps available here: https://nofile.io/f/ > mzKnkpM9CyC/stats.tar.gz2 > >>>>> > >>>>> as far as why there's such high deletions? I can't say I'm only > using cp. > >>>> > >>>> I assume that md99 is for a file-based swap-space, such as > >>>> via /var/swap0 file. (As a side note I warn about bugzilla > >>>> 206048 for such contexts.) Otherwise please describe how > >>>> md99 is created. (Below I assume the swap-space usage of > >>>> md99.) > >>>> > >>>> The only other device that your pictures show is your > >>>> NVMe device nvd0. > >>>> > >>>> No picture shows a device for the LG v30 when it is mentioned > >>>> above as being copied to or from. How is it that there is no > >>>> mounted device shown for the LG v30? > >>>> > >>>> No picture shows a device for the SSD when it is mentioned > >>>> above as being copied to or from. How is it that there > >>>> is no mounted device shown for the SSD? > >>>> > >>>> Without a device displayed for the LG-v30/SSD there is nothing > >>>> displayed for its reads or writes. This makes the gstat -pd > >>>> useless. > >>>> > >>>> May be the p needs to be omitted for some reason? gstat -d > >>>> > >>>> === > >>>> Mark Millard > >>>> markmi at dsl-only.net > >>> > >>> You are correct that md99 is a file backed swap disk, I am aware of > the issues but I had to test some things out. > >>> > >>> Those tests earlier was a huge time sink. > >>> Here's the dmesg output from earlier: > >>> . . . > >>> ---------------------------------- > >>> > >>> I don't know why gstat isn't showing the info u are looking for. > >>> > >>> You said earlier that something was getting deleted, > >>> I don't know what could be causing that. > >> > >> Those "deletes" are more commonly called TRIM on SSD's. > >> FreeBSD called them, BIO_DELETE as I remember. > >> > >> Those deletes have nothing to do with umounting, deleteing > >> devices, etc. > >> > >>> I've provided tons of debug out, logs, pciconf, dmesg, etc... > >>> Even say forget the mobile device and go from > >>> zpool > >> > >> From which I've not been able to figure out the kind of > >> information that I'm looking for. > >> > >> Just because a device is present, does not mean that it > >> is available as a file system. I'm more interested in > >> how the file systems are made available --or if some > >> non-file system way of working is involved. > >> > >>> Are you saying that there's something misconfigured on my end? > >>> What other info do you need me to provide to try to figure out > >>> what's going on or why I'm getting these transfer rates? > >> > >> I'm saying I still can not tell how you make the SSD or the > >> LGv 30 available to FreeBSD (mount?). Or why no matching > >> mounted device shows up in gstat's display. > >> > >> If you wish to keep trying to help me help you, . . . > >> > >> Please show how you make the file system on the > >> SSD available to FreeBSD: what FreeBSD commands make > >> the device available for use. (I'd guess that mount > >> is used or that something like /etc/fstab is used > >> to do mounts more implicitly.) > >> > >> Please show how you make the LG v30 available to FreeBSD: > >> what FreeBSD commands make the device available for > >> use. (I'd guess that mount is used or that something like > >> /etc/fstab is used to do mounts more implicitly.) > >> > >> (I would expect these are mount commands, or at least > >> involve mount commands/calls. Some of the following makes > >> that presumption.) > >> > >> For each of those: with the device available show the > >> output of: > >> > >> mount > >> > >> and of: > >> > >> df -m > >> > >> Similarly, show the exact commands used to make the > >> copies to and from the SSD. Show the exact commands > >> used to make the copies to and from the LG v30. > >> > >> (You can for now stop the commands early or just > >> not start any that would take a long time.) > >> > >> I'm looking for a way to get information similar to > >> what I expected gstat to show. I'd expect a mounted > >> file system but may be something else is involved? > > > [Extracted from a reply to a Jon Brawn post.] > > @Mark Millard > > I use sysutils/simple-mtpfs to mount the android device. > > when I mount the phone through USB this is the relevant section: > > /dev/fuse 356311 78912 277398 22% > > /mnt > > > > That's the most complicated mount process that I use, > > for the ssd it's just a simple mount /dev/device /mnt > > relevant output: > > /dev/da0 923913 121450 728550 14% > > /mnt > > > > Can you tell me what information you're looking for so that I can gather > it > > all up and send it. > > > I do not find a /usr/ports/sysutils/simple-mtpfs . But I > do find: > > # ls -lTd /usr/ports/sysutils/*simple* > drwxr-xr-x 3 root wheel 512 Sep 23 19:51:59 2017 > /usr/ports/sysutils/fusefs-simple-mtpfs > > And that was very important to know. > > I had to look it up but this means that you are using > the Media Transfer Protocol (MTP) to transfer files from > the LG v30. It also means that part of that involves use > of the Filesystem in USErspace (FUSE). The two would be > likely to add overheads that slow things down (extra > stages/steps in getting the bytes copied). > > [Apparently MTP is an extension to the Picture Transfer > Protocol (PTP) communications protocol. MTP has been > standardized as a full-fledged USB device class but > originated with Microsoft.] > > [MTP use involves putting the Android device into MTP mode, > frequently referenced in the Android UI via a selection > of "USB for file transfer" someplace in the user > interface (according to what I read).] > > One example of why FreeBSD and sysutils/fusefs-simple-mtpfs > and sysutils/fusefs-libs and multimedia/libmtp might be > slower than some linux software is: > > It appears that some Linux software has implemented "USB > 'Zerocopy' support found in recent Linux kernel" (and, > so, avoid user-space vs. kernel space data copying and > some context switching). > > I doubt that FreeBSD and its sysutils/fusefs-simple-mtpfs > and sysutils/fusefs-libs and multimedia/libmtp combination > happen to do that. This difference likely would make a > notable speed difference for the transfer and save to local > file system (if my doubt is correct). > > It is not surprising that the speeds would be far less > than the experiments that I reported. > > > I do not have a context to experiment with the > consequences of using sysutils/fusefs-simple-mtpfs > and what it requires (fusefs-libs and libmtp). I'm > afraid I'll not be able to be much help relative to > the LG v30 MTP-based transfer performance > investigations. > > > That leaves the SSD context. Is there anything that > you still want to investigate relative to the SSD > usage? Or is there no point because of the LG v30 > status? > > === > Mark Millard > markmi at dsl-only.net > > > OK, forget the android specific usb issues for now; there is quite a few layers of crud between the laptop and the android device. I have this SSD. What are some tests that you'd like to run to test the speeds on my end?
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CALM2mE=iKr0gEqCxM8KWDZYLipVQmzRdGYTv72CiQoemcB9YtA>