Date: Fri, 14 Aug 1998 12:30:00 -0700 (PDT) From: Matthew Dillon <dillon@backplane.com> To: freebsd-bugs@FreeBSD.ORG Subject: Re: kern/7418 (file corruption on mmap-based-read during file write()) Message-ID: <199808141930.MAA28426@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
The following reply was made to PR kern/7418; it has been noted by GNATS.
From: Matthew Dillon <dillon@backplane.com>
To: Luoqi Chen <luoqi@watermarkgroup.com>
Cc: luoqi@chen.ml.org, freebsd-gnats-submit@freebsd.org
Subject: Re: kern/7418 (file corruption on mmap-based-read during file write())
Date: Fri, 14 Aug 1998 12:21:19 -0700 (PDT)
:>
:> A,B,C,D ... discrete usenet articles stored in file.
:>
:> [AAAAAAA][AAABBBBBB][BBBBBBBB][BBBBCCCCCCdddddd][DDDDDD...]
:> page #1 #2 #3 #4 #5
:>
:>
:I have a very interesting observation of the spool file dump in the PR 7418
:report, article 'C' (<6pip3s$f5b$1@nnrp1.dejanews.com>) was completely lost,
:so it is not just the article that crosses the page boundary, corruption
:could occur for articles precede it and on the same page. I also notice that
:both in PR 7418 and in your illustration above, corruptions take place on
:even numbered pages, i.e., 2nd page of the two-page FFS blocks, is this
:always the case? Are you aware of any other commonalities among all
:corruption cases you had seen?
Ah, looks like it's bounded by the filesystem block size, 8K, rather
then the 4K page size. Here's a bunch of stuff from my spool:
(note: there is supposed to be a single 00 separator between articles,
so the occassional single 00 you see inbetween articles is correct. But
not the to-end-of-block 00 corruption which you can see is obviously
replacing what should have been valid text).
Also, I was once able to ktrace the writing process (I think it's in the
original PR) to ensure that the writing process is not writing the 00's
itself. So it's definitely kernel-generated corruption.
hdump D.00e5a1dc/B.3328 | less
...
0001.96b0 65 69 76 65 20 63 6f 6d 6d 65 72 63 69 61 6c 20 eive commercial
0001.96c0 65 6d 61 69 6c 73 20 6f 72 20 6f 66 66 65 72 69 emails or offeri
0001.96d0 6e 67 73 20 66 72 6f 6d 20 61 6e 79 20 76 65 6e ngs from any ven
0001.96e0 64 6f 72 0a 70 6c 65 61 73 65 20 72 65 70 6c 79 dor.please reply
0001.96f0 20 77 69 74 68 20 27 52 45 4d 4f 56 45 27 20 69 with 'REMOVE' i
0001.9700 6e 20 74 68 65 20 73 75 62 6a 65 63 74 20 66 69 n the subject fi
0001.9710 65 6c 64 2e 0a 0a 0a 00 00 00 00 00 00 00 00 00 eld.............
0001.9720 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0001.9730 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
...
0001.9fc0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0001.9fd0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0001.9fe0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0001.9ff0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0001.a000 74 65 20 75 6e 69 74 3b 20 4d 6f 64 65 6c 20 23 te unit; Model #
0001.a010 20 32 35 30 2d 30 37 31 39 2d 30 30 31 2e 0a 54 250-0719-001..T
0001.a020 68 65 20 75 6e 69 74 20 68 61 73 20 74 68 65 20 he unit has the
...
0002.1010 4f 72 65 67 6f 6e 20 31 39 39 32 2d 39 34 0a 53 Oregon 1992-94.S
0002.1020 75 6e 73 65 74 20 48 53 20 31 39 38 39 2d 39 32 unset HS 1989-92
0002.1030 0a 7b 54 68 65 73 65 20 76 69 65 77 73 20 61 72 .{These views ar
0002.1040 65 20 6d 69 6e 65 20 61 6c 6f 6e 65 2c 20 61 6e e mine alone, an
0002.1050 64 20 64 6f 20 6e 6f 74 20 72 65 70 72 65 73 65 d do not represe
0002.1060 6e 74 20 74 68 65 20 76 69 65 77 73 20 6f 66 20 nt the views of
0002.1070 74 68 65 73 65 20 6f 72 20 61 6e 79 0a 6f 74 68 these or any.oth
0002.1080 65 72 20 6f 72 67 61 6e 69 7a 61 74 69 6f 6e 73 er organizations
0002.1090 7d 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 }...............
0002.10a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0002.10b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0002.10c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0002.10d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
...
0002.1fd0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0002.1fe0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0002.1ff0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0002.2000 69 0a 43 6f 6e 74 65 6e 74 2d 54 72 61 6e 73 66 i.Content-Transf
0002.2010 65 72 2d 45 6e 63 6f 64 69 6e 67 3a 20 37 62 69 er-Encoding: 7bi
0002.2020 74 0a 4c 69 6e 65 73 3a 20 31 36 0a 4d 65 73 73 t.Lines: 16.Mess
**************
hdump D.00e5a204/B.5b40 | less
...
0002.1a60 6f 67 6c 69 6f 6e 6f 20 72 6f 6d 70 65 72 65 20 ogliono rompere
0002.1a70 61 20 6e 6f 69 22 2e 0a 0a 4e 6f 20 63 72 6f 73 a noi"...No cros
0002.1a80 73 70 6f 73 74 2c 20 38 30 20 6c 69 6e 65 73 20 spost, 80 lines
0002.1a90 6f 6e 6c 79 2d 20 4e 6f 73 70 61 6d 20 69 6e 20 only- Nospam in
0002.1aa0 61 64 64 72 65 73 73 0a 2d 2d 2d 2d 2d 2d 2d 2d address.--------
0002.1ab0 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d ----------------
0002.1ac0 2d 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -...............
0002.1ad0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0002.1ae0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0002.1af0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
...
0002.1fa0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0002.1fb0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0002.1fc0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0002.1fd0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0002.1fe0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0002.1ff0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0002.2000 2d 2d 2d 2d 2d 2d 2d 2d 20 20 49 20 6b 6e 6f 77 -------- I know
0002.2010 20 49 27 6d 20 6d 79 20 62 65 73 74 20 66 72 69 I'm my best fri
0002.2020 65 6e 64 20 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d end ------------
0002.2030 2d 2d 2d 2d 2d 2d 2d 2d 0a 00 50 61 74 68 3a 20 --------..Path:
**************
hdump D.00e5a20e/B.7411 | less
...
0002.67f0 53 74 61 6e 64 61 72 64 20 64 69 73 63 6c 61 69 Standard disclai
0002.6800 6d 65 72 3a 20 54 68 65 20 76 69 65 77 73 20 6f mer: The views o
0002.6810 66 20 74 68 69 73 20 75 73 65 72 20 61 72 65 20 f this user are
0002.6820 73 74 72 69 63 74 6c 79 20 68 69 73 2f 68 65 72 strictly his/her
0002.6830 20 6f 77 6e 2e 0a 0a 0a 00 00 00 00 00 00 00 00 own............
0002.6840 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0002.6850 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0002.6860 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
...
0002.7fc0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0002.7fd0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0002.7fe0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0002.7ff0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0002.8000 65 20 64 69 72 65 63 74 20 55 52 4c 20 69 73 3a e direct URL is:
0002.8010 0a 68 74 74 70 3a 2f 2f 77 77 77 2e 62 64 73 6d .http://www.bdsm
0002.8020 74 6f 70 2d 35 30 2e 63 6f 6d 2f 77 65 62 6d 61 top-50.com/webma
0002.8030 73 74 65 72 73 2f 72 65 73 6f 75 72 63 65 73 2f sters/resources/
**************
hdump D.00e5a20e/B.769d | less
0003.1d90 74 68 65 20 43 55 52 52 45 4e 54 20 0a 70 75 62 the CURRENT .pub
0003.1da0 6c 69 73 68 65 64 20 73 74 61 6e 64 61 72 64 2c lished standard,
0003.1db0 20 61 6e 64 20 6c 69 66 65 20 77 69 6c 6c 20 62 and life will b
0003.1dc0 65 20 6d 75 63 68 20 65 61 73 69 65 72 20 66 6f e much easier fo
0003.1dd0 72 20 75 73 20 61 6c 6c 2e 0a 0a 0a 00 00 00 00 r us all........
0003.1de0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0003.1df0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0003.1e00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
...
0003.1fd0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0003.1fe0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0003.1ff0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0003.2000 35 2e 47 41 4d 45 0a 58 2d 46 54 4e 2d 4d 53 47 5.GAME.X-FTN-MSG
0003.2010 49 44 3a 20 32 3a 35 30 31 30 2f 31 34 36 2e 38 ID: 2:5010/146.8
0003.2020 20 33 35 64 34 39 65 61 61 0a 58 2d 46 54 4e 2d 35d49eaa.X-FTN-
**************
hdump D.00e5a22c/B.05e5 | less
...
0000.ff40 20 73 74 6f 72 65 20 74 68 65 20 45 53 50 0a 76 store the ESP.v
0000.ff50 61 6c 75 65 20 69 6e 20 67 6c 6f 62 61 6c 20 6d alue in global m
0000.ff60 65 6d 6f 72 79 20 73 6f 6d 65 77 68 65 72 65 2e emory somewhere.
0000.ff70 20 41 66 74 65 72 20 61 6c 6c 20 73 74 61 63 6b After all stack
0000.ff80 20 77 6f 75 6c 64 20 62 65 20 69 6e 61 63 63 65 would be inacce
0000.ff90 73 73 69 62 6c 65 20 74 6f 0a 72 65 73 74 6f 72 ssible to.restor
0000.ffa0 65 20 69 74 20 77 68 65 72 65 20 49 20 74 6f 20 e it where I to
0000.ffb0 70 75 73 68 20 69 74 2e 2e 2e 0a 0a 4d 69 63 68 push it.....Mich
0000.ffc0 61 65 6c 0a 0a 0a 00 00 00 00 00 00 00 00 00 00 ael.............
0000.ffd0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0000.ffe0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0000.fff0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0001.0000 65 72 6f 6c 73 21 68 6f 77 6c 61 6e 64 2e 65 72 erols!howland.er
0001.0010 6f 6c 73 2e 6e 65 74 21 76 69 78 65 6e 2e 63 73 ols.net!vixen.cs
0001.0020 6f 2e 75 69 75 63 2e 65 64 75 21 73 64 64 2e 68 o.uiuc.edu!sdd.h
0001.0030 70 2e 63 6f 6d 21 6e 69 67 68 74 2e 70 72 69 6d p.com!night.prim
0001.0040 61 74 65 2e 77 69 73 63 2e 65 64 75 21 6e 6e 74 ate.wisc.edu!nnt
0001.0050 70 2e 6d 73 73 74 61 74 65 2e 65 64 75 21 70 61 p.msstate.edu!pa
...
0001.1c90 0a 3e 0a 3e 47 72 65 67 0a 3e 65 6d 61 69 6c 3a .>.>Greg.>email:
0001.1ca0 20 67 72 65 67 6f 72 30 40 70 72 69 76 2e 6f 6e gregor0@priv.on
0001.1cb0 65 74 2e 70 6c 0a 3e 54 68 65 20 4d 61 73 71 75 et.pl.>The Masqu
0001.1cc0 65 72 61 64 65 20 63 6f 6e 74 69 6e 75 65 73 2e erade continues.
0001.1cd0 2e 2e 0a 3e 0a 3e 0a 0a 0a 00 00 00 00 00 00 00 ...>.>..........
0001.1ce0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0001.1cf0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
...
0001.1fd0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0001.1fe0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0001.1ff0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0001.2000 3a 22 49 20 6f 62 6a 65 63 74 2c 20 79 6f 75 72 :"I object, your
0001.2010 20 68 6f 6e 6f 72 2c 20 61 73 6b 65 64 20 61 6e honor, asked an
0001.2020 64 20 61 6e 73 77 65 72 65 64 21 22 0a 48 6f 6e d answered!".Hon
0001.2030 6f 72 3a 22 53 55 53 54 41 49 4e 45 44 21 22 0a or:"SUSTAINED!".
0001.2040 0a 50 6c 65 61 73 65 20 72 65 61 64 20 61 6c 6c .Please read all
:>
:I have written a small test program to simulate what diablo does, and so
:far has not been able to get any corruption. The test was done on an otherwise
:idle machine. So heavy load is definitely a contributing factor here.
:
I've tried writing similar test programs, with no luck either. The
load on the real machine has got to be a significant factor, but since
the memory isn't starved I'm tending towards the disk latency as being
a possible factor.
-Matt
Matthew Dillon Engineering, HiWay Technologies, Inc. & BEST Internet
Communications
<dillon@backplane.com> (Please include original email in any response)
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-bugs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199808141930.MAA28426>