Date: Tue, 07 Jan 2014 20:55:10 -0700 From: Ian Lepore <ian@FreeBSD.org> To: Steven Lee <ssl.qlt@gmail.com> Cc: freebsd-arm@FreeBSD.org Subject: Re: Fwd: Dreamplug stable/10 usb - sd file transfer corruption error - cache related? Message-ID: <1389153310.1158.349.camel@revolution.hippie.lan> In-Reply-To: <CALD1=PkzBfoPUSYMGnc%2BvkLEeb1T7Xi5pugiNOUq9e%2BV3TP90g@mail.gmail.com> References: <CALD1=P=gWosQsTTYt=hpkasP8y1rH3oH6y0n-y=APP_X0S%2BYFQ@mail.gmail.com> <1382734990.1170.132.camel@revolution.hippie.lan> <CALD1=PkqJAmwzfq4Go_zQrc7nVMMzC=B%2BpTvT9rnCFkQ7rTJew@mail.gmail.com> <CALD1=P=H0PwDh%2B2ix9RkszCuOsv8ucmk=8pSYYRwN__i0veSZw@mail.gmail.com> <1382738462.1170.153.camel@revolution.hippie.lan> <CALD1=PkzBfoPUSYMGnc%2BvkLEeb1T7Xi5pugiNOUq9e%2BV3TP90g@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Oddly enough, I never forgot about this, I just never got to it until today. I put a concerted effort into getting data corruption on usb->usb copying of files and even entire filesystems, and couldn't get a glitch. I used the external sdcard (da1) and a thumb drive. I used 'mtree -ck md5' on both source and destination, then diff'd the results, and the files were all identical. I tested with 11-current (@r260393) and 10-stable (@r260394) on my DreamPlug, compiled with clang-eabi in both cases. -- Ian On Fri, 2013-10-25 at 18:40 -0700, Steven Lee wrote: > Ian, thanks for taking a look to see if you can recreate the problem. In > case it makes any difference in behavior, the > command I used for the transfer was: > > (dump -0Lf - /) | (cd /mnt/flash_root ; restore -rf -) > > with /dev/da2s2 mounted at / and /dev/da0s2 mounted at /mnt/flash_root > > Regards, > > Steve > On Fri, Oct 25, 2013 at 3:01 PM, Ian Lepore <ian@freebsd.org> wrote: > > > That old advice was based on bugs in old code that has been fixed. I've > > just been double-checking that the various fixes are in 10 (some of them > > were things I submitted a PR for before I became a committer). I'm not > > sure about the usb driver, but there have been big changes in the dma > > cache coherency code between 9 and 10, although much of that is the > > fixes I was alluding to. > > > > Basically what you're doing is a usb->usb copy, I'll see if I can > > recreate the corruption on my dreamplug here the same way. > > > > -- Ian > > > > On Fri, 2013-10-25 at 14:38 -0700, Steven Lee wrote: > > > ---------- Forwarded message ---------- > > > From: Steven Lee <ssl.qlt@gmail.com> > > > Date: Fri, Oct 25, 2013 at 2:37 PM > > > Subject: Re: Dreamplug stable/10 usb - sd file transfer corruption error > > - > > > cache related? > > > To: Ian Lepore <ian@freebsd.org> > > > > > > > > > Ian, thanks for your reply. > > > > > > I am using the DREAMPLUG-1001 stock kernel config, and you are right the > > > internal sd card shows up as a usb device, /dev/da0. > > > > > > As I mentioned, I tried modifying pmap.c as per one of your suggestions > > to > > > an earlier poster / problem, but it didn't seem to help the issue. I > > > changed the pmap_pte_init_arm10() function to look like the > > > pmap_pte_init_arm9() function. > > > > > > Any other thoughts about things to try? Since it seems like this is a > > > regression between 9 and 10, are you aware of any specific cache related > > > changes to the usb driver between the branches? > > > > > > - Steve > > > > > > > > > On Fri, Oct 25, 2013 at 2:03 PM, Ian Lepore <ian@freebsd.org> wrote: > > > > > > > On Fri, 2013-10-25 at 13:37 -0700, Steven Lee wrote: > > > > > Hello, > > > > > > > > > > In the process of upgrading my Dreamplug from the stable/9 branch to > > the > > > > > stable/10 branch, I have run into a fairly > > > > > significant file corruption problem. The steps that create this > > problem > > > > > are as follows: > > > > > > > > > > 1. Cross-compile the kernel and buildworld from the stable/10 branch > > on a > > > > > separate host. > > > > > 2. Install the kernel and installworld on a usb stick drive > > > > > 3. Boot the Dreamplug from the usb drive > > > > > > > > > > There are no problems that show up to this point, the Dreamplug boots > > > > with > > > > > no errors. > > > > > > > > > > 4. Copy the kernel from the usb drive to the internal sd card > > > > > 5. Copy the root filesystem from the usb drive to the internal sd > > card > > > > > using (dump | restore) > > > > > > > > > > It is at step 5 that the error manifests itself, as I found that > > > > > approximately 20 shell scripts in the /etc/rc.d directory > > > > > had been randomly corrupted with strings of null characters (in > > groups of > > > > > 32). I assume that the rest of the file system > > > > > was compromised in a similar fashion. The bug is repeatable, > > however the > > > > > corruption is somewhat random, so > > > > > each time different files are corrupted in different places. > > > > > > > > > > When I followed the exact same steps from the stable/9 branch, the > > > > problem > > > > > did not occur, so there is clearly some > > > > > type of regression error between the 9 and 10 branches. > > > > > > > > > > After searching through the arm mailing list, I attempted to work > > around > > > > > the problem by: > > > > > - mounting the file systems with -o noclusterr -o noclusterrw > > > > > - mounting the file systems with -o sync > > > > > - as per comments on bug arm/158950, attempted to modify the pmap.c > > > > > functions to change the cache from > > > > > write-back mode to write-through mode > > > > > > > > > > but all of these attempts were ineffective. > > > > > > > > > > Given that I am relatively new to FreeBSD, I was hoping to get any > > > > insights > > > > > as to any next steps that might make > > > > > sense in terms of either working around the problem, narrowing down > > the > > > > > bug, or any obvious rookie mistakes I > > > > > am making before I give up and revert back to the 9 branch. > > > > > > > > > > Thanks for your help! > > > > > > > > > > Regards, > > > > > > > > On my dreamplug (model 1001) both the internal and external sd cards > > are > > > > actually usb devices (they show up as /dev/da0 and da1, not mmcsd0/1). > > > > I'm not sure if that's true on all models or not. > > > > > > > > Are you using the stock kernel config (DREAMPLUG-1001)? If not, have > > > > you got "option USB_HOST_ALIGN=32" in your kernel config? > > > > > > > > Corruption in 32 byte chunks is almost always partial cacheline flush > > > > problems, but I haven't seen that happen on dreamplug for a long time > > > > (and when I did see it, it was with a sata drive). > > > > > > > > -- Ian > > > > > > > > > > > _______________________________________________ > > > freebsd-arm@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-arm > > > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" > > > > > > > _______________________________________________ > freebsd-arm@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arm > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1389153310.1158.349.camel>