Date: Sun, 7 Jan 2018 23:20:19 -0800 From: Mark Millard <markmi@dsl-only.net> To: blubee blubeeme <gurenchan@gmail.com> Cc: FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: USB stack Message-ID: <899567F7-0C02-4A60-ADA0-5F43A6CF594B@dsl-only.net> In-Reply-To: <2B2F495A-9CBE-4366-B049-0EC5EF7F9629@dsl-only.net> References: <3F9697E3-3C25-45CB-804A-9C3607E434C4@dsl-only.net> <CALM2mEnaA7zDVfONFQEBtC2WghbRFoFW2iPpmBKohP1pd45CcQ@mail.gmail.com> <0AB4ED58-E01A-4761-B6EF-4D56F8CA21E3@dsl-only.net> <F9A5DBFF-79C8-417D-9B6D-0788976B558C@dsl-only.net> <FA0FA34D-B941-43DB-8885-902B502A5442@dsl-only.net> <CALM2mEnnc4bzpFoRA_F_FayiBuGLwbwzyf-Gb-U6vmQ0RuGuYQ@mail.gmail.com> <1F10CBFE-1AAC-4307-976A-0CDA80EDC616@dsl-only.net> <CALM2mEkxqQMfzKQ7CmBnGmcu11evt129vAjWjm654PG=FJs2kA@mail.gmail.com> <2B2F495A-9CBE-4366-B049-0EC5EF7F9629@dsl-only.net>
next in thread | previous in thread | raw e-mail | index | archive | help
[The involvement of sysutils/fusefs-simple-mtpfs and sysutils/fusefs-libs and multimedia/libmtp in order to use the Media Transfer Protocol (MTP) against the LG v30 likely explains much of the performance difference vs. local UFS file systems. I provide some related notes.] blubee blubeeme gurenchan at gmail.com wrote on Mon Jan 8 05:17:24 UTC 2018 : [Note: the original was in a reply to a Jon Brawn post. I've merged it back with my post.] > On 2018-Jan-7, at 11:46 AM, Mark Millard <markmi at dsl-only.net> = wrote: >=20 >> On 2018-Jan-7, at 10:23 AM, blubee blubeeme <gurenchan at gmail.com> = wrote: >>=20 >>=20 >>=20 >>> On Mon, Jan 8, 2018 at 1:41 AM, Mark Millard <markmi at = dsl-only.net> wrote: >>>=20 >>>> On 2018-Jan-7, at 7:50 AM, blubee blubeeme <gurenchan at gmail.com> = wrote: >>>>=20 >>>>> I ran this test and here's some results. >>>>> gstat -pd images: >>>>>=20 >>>>> 18GB file from laptop to phone: https://imgur.com/a/7iHwv >>>>> 18GB file from laptop to ssd: https://imgur.com/a/40Q6V >>>>> multiple small files from laptop to phone: = https://imgur.com/a/B4v4y >>>>> multiple small files from laptop to ssd: https://imgur.com/a/mDiMu >>>>>=20 >>>>> The files are missing timestamps but the originals were taken with = scrot and have timestamps available here: = https://nofile.io/f/mzKnkpM9CyC/stats.tar.gz2 >>>>>=20 >>>>> as far as why there's such high deletions? I can't say I'm only = using cp. >>>>=20 >>>> I assume that md99 is for a file-based swap-space, such as >>>> via /var/swap0 file. (As a side note I warn about bugzilla >>>> 206048 for such contexts.) Otherwise please describe how >>>> md99 is created. (Below I assume the swap-space usage of >>>> md99.) >>>>=20 >>>> The only other device that your pictures show is your >>>> NVMe device nvd0. >>>>=20 >>>> No picture shows a device for the LG v30 when it is mentioned >>>> above as being copied to or from. How is it that there is no >>>> mounted device shown for the LG v30? >>>>=20 >>>> No picture shows a device for the SSD when it is mentioned >>>> above as being copied to or from. How is it that there >>>> is no mounted device shown for the SSD? >>>>=20 >>>> Without a device displayed for the LG-v30/SSD there is nothing >>>> displayed for its reads or writes. This makes the gstat -pd >>>> useless. >>>>=20 >>>> May be the p needs to be omitted for some reason? gstat -d >>>>=20 >>>> =3D=3D=3D >>>> Mark Millard >>>> markmi at dsl-only.net >>>=20 >>> You are correct that md99 is a file backed swap disk, I am aware of = the issues but I had to test some things out. >>>=20 >>> Those tests earlier was a huge time sink. >>> Here's the dmesg output from earlier: >>> . . . >>> ---------------------------------- >>>=20 >>> I don't know why gstat isn't showing the info u are looking for. >>>=20 >>> You said earlier that something was getting deleted,=20 >>> I don't know what could be causing that. >>=20 >> Those "deletes" are more commonly called TRIM on SSD's. >> FreeBSD called them, BIO_DELETE as I remember. >>=20 >> Those deletes have nothing to do with umounting, deleteing >> devices, etc. >>=20 >>> I've provided tons of debug out, logs, pciconf, dmesg, etc... >>> Even say forget the mobile device and go from >>> zpool >>=20 >> =46rom which I've not been able to figure out the kind of >> information that I'm looking for. >>=20 >> Just because a device is present, does not mean that it >> is available as a file system. I'm more interested in >> how the file systems are made available --or if some >> non-file system way of working is involved. >>=20 >>> Are you saying that there's something misconfigured on my end? >>> What other info do you need me to provide to try to figure out >>> what's going on or why I'm getting these transfer rates? >>=20 >> I'm saying I still can not tell how you make the SSD or the >> LGv 30 available to FreeBSD (mount?). Or why no matching >> mounted device shows up in gstat's display. >>=20 >> If you wish to keep trying to help me help you, . . . >>=20 >> Please show how you make the file system on the >> SSD available to FreeBSD: what FreeBSD commands make >> the device available for use. (I'd guess that mount >> is used or that something like /etc/fstab is used >> to do mounts more implicitly.) >>=20 >> Please show how you make the LG v30 available to FreeBSD: >> what FreeBSD commands make the device available for >> use. (I'd guess that mount is used or that something like >> /etc/fstab is used to do mounts more implicitly.) >>=20 >> (I would expect these are mount commands, or at least >> involve mount commands/calls. Some of the following makes >> that presumption.) >>=20 >> For each of those: with the device available show the >> output of: >>=20 >> mount >>=20 >> and of: >>=20 >> df -m >>=20 >> Similarly, show the exact commands used to make the >> copies to and from the SSD. Show the exact commands >> used to make the copies to and from the LG v30. >>=20 >> (You can for now stop the commands early or just >> not start any that would take a long time.) >>=20 >> I'm looking for a way to get information similar to >> what I expected gstat to show. I'd expect a mounted >> file system but may be something else is involved? >=20 [Extracted from a reply to a Jon Brawn post.] > @Mark Millard > I use sysutils/simple-mtpfs to mount the android device. > when I mount the phone through USB this is the relevant section: > /dev/fuse 356311 78912 277398 22% > /mnt >=20 > That's the most complicated mount process that I use, > for the ssd it's just a simple mount /dev/device /mnt > relevant output: > /dev/da0 923913 121450 728550 = 14% > /mnt >=20 > Can you tell me what information you're looking for so that I can = gather it > all up and send it. I do not find a /usr/ports/sysutils/simple-mtpfs . But I do find: # ls -lTd /usr/ports/sysutils/*simple* drwxr-xr-x 3 root wheel 512 Sep 23 19:51:59 2017 = /usr/ports/sysutils/fusefs-simple-mtpfs And that was very important to know. I had to look it up but this means that you are using the Media Transfer Protocol (MTP) to transfer files from the LG v30. It also means that part of that involves use of the Filesystem in USErspace (FUSE). The two would be likely to add overheads that slow things down (extra stages/steps in getting the bytes copied). [Apparently MTP is an extension to the Picture Transfer Protocol (PTP) communications protocol. MTP has been standardized as a full-fledged USB device class but originated with Microsoft.] [MTP use involves putting the Android device into MTP mode, frequently referenced in the Android UI via a selection of "USB for file transfer" someplace in the user interface (according to what I read).] One example of why FreeBSD and sysutils/fusefs-simple-mtpfs and sysutils/fusefs-libs and multimedia/libmtp might be slower than some linux software is: It appears that some Linux software has implemented "USB 'Zerocopy' support found in recent Linux kernel" (and, so, avoid user-space vs. kernel space data copying and some context switching). I doubt that FreeBSD and its sysutils/fusefs-simple-mtpfs and sysutils/fusefs-libs and multimedia/libmtp combination happen to do that. This difference likely would make a notable speed difference for the transfer and save to local file system (if my doubt is correct). It is not surprising that the speeds would be far less than the experiments that I reported. I do not have a context to experiment with the consequences of using sysutils/fusefs-simple-mtpfs and what it requires (fusefs-libs and libmtp). I'm afraid I'll not be able to be much help relative to the LG v30 MTP-based transfer performance investigations. That leaves the SSD context. Is there anything that you still want to investigate relative to the SSD usage? Or is there no point because of the LG v30 status? =3D=3D=3D Mark Millard markmi at dsl-only.net
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?899567F7-0C02-4A60-ADA0-5F43A6CF594B>