Date: Wed, 19 Aug 1998 13:14:00 -0700 (PDT) From: David Lowe <david@linkexchange.com> To: freebsd-questions@FreeBSD.ORG Subject: mmap() synchronization and locking problems Message-ID: <Pine.BSF.3.96.980819124710.17743A-100000@pootpoot.com>
next in thread | raw e-mail | index | archive | help
People -
I have an interesting problem with mmap(). I'm hoping someone out there
can give me a hand with it.
There is a process which mmap()s a largish (100meg) file, which is used to
store statistical information. There is a custom apache module running in
apache on the same machine, which reads the stats from the mmap()d file,
and generates a graph dynamically based on some information in that file.
The first process is not related to apache i.e. there is no fork/exec, it
is started separately.
The stats process is looping over log data. Basically it reads in a chunk
of log data (about 50k), parses it, and updates the mmap()d stats
structure. This takes place every 20 seconds or so, and works great.
The apache processes also work great. A few seconds after each time
through the loop, though, the disk goes nuts for about 10-15 seconds
(iostat 1 output is included at the end of this message.) During this
time, the machine is pretty much locked up. Worse, the apache processes
all go into a ufslk2 state, and hang for up to 15 seconds.
My assumption has been that this disk thrashing state occurs because of
the way that mmap() keeps in-core data synchronized with the disk backing.
The stats program doesn't call msync() anywhere, so I assume that this is
just the default behavior. It looks like it's postponing all writes for a
little while, then locking the file and rewriting the whole darned thing
at once.
Some thoughts and failed attempts:
1) Periodic msync() calls over portions of the mmap()d area. This solved
the problem but introduced another - the program couldn't keep up with
the log data input rate :(
I could probably hack on the frequency/size of msyncs to find a happy
medium? Maybe?
2) MAP_ANON would solve the I/O problem, clearly, but I can't figure out
how to share an mmap()d MAP_ANON area between unrelated processes.
3) system V shared memory. Too much data :(
4) break it into a few separate mmap()d files or shared memory segments.
Definitely not desirable from a simplicity standpoint, but...
5) make apache and the stats program related, and use MAP_ANON. Again,
not desirable for simplicity's sake.
All but 1 also lose us the disk backing, which is desirable (though not
critical, I suppose...)
So can anyone offer any insight into why this disk thrashing is happening,
whether my assumption(s) are correct, and what (if anything) can be done
to help improve the situation?
Thanks in advance,
David Lowe
iostat 1 output ---------------------------------------------------------
tin tout sps tps msps us ni sy in id
0 116 381 3 0.0 19 0 80 1 0
0 40 127 1 0.0 22 0 76 2 0
0 39 35 4 0.0 55 0 43 1 2
0 40 224 15 0.0 37 0 52 2 9
0 40 16 1 0.0 21 0 79 0 0
0 40 161 4 0.0 21 0 78 2 0
0 39 0 0 0.0 20 0 80 0 0
0 40 127 1 0.0 16 0 82 2 0
0 40 127 1 0.0 22 0 75 3 0
0 40 0 0 0.0 23 0 76 1 0
0 39 378 3 0.0 18 0 80 2 0
0 40 0 0 0.0 16 0 83 1 0
0 40 127 1 0.0 22 0 78 0 0
0 39 337 17 0.0 47 0 40 2 11
0 401689 102 0.0 12 0 87 2 0
0 402604 155 0.0 17 0 81 2 0
0 402604 156 0.0 22 0 77 2 0
0 392505 145 0.0 20 0 78 2 0
0 402683 141 0.0 16 0 83 1 0
0 392584 136 0.0 12 0 88 1 0
tty sd0 cpu
tin tout sps tps msps us ni sy in id
0 1162574 141 0.0 11 0 88 1 0
0 402572 160 0.0 19 0 80 1 0
0 392851 178 0.0 10 0 88 2 0
0 392970 185 0.0 12 0 88 0 0
0 403255 194 0.0 11 0 88 1 0
0 403064 185 0.0 5 0 94 2 0
0 394001 208 0.0 2 0 97 1 0
0 363341 171 0.0 1 0 97 1 0
0 402048 123 0.0 9 0 91 0 0
0 402191 127 0.0 2 0 40 1 57
0 402000 122 0.0 0 0 0 0100
0 402080 125 0.0 0 0 0 0100
0 391922 115 0.0 0 0 0 0100
0 401921 118 0.0 0 0 0 0100
0 401810 112 0.0 0 0 0 0100
0 401873 115 0.0 0 0 0 0100
0 391875 117 0.0 0 0 1 0 99
0 381737 110 0.0 10 0 1 0 89
0 37 24 5 0.0 31 0 69 0 0
0 40 20 3 0.0 25 0 74 1 0
tty sd0 cpu
tin tout sps tps msps us ni sy in id
0 115 126 1 0.0 23 0 73 4 0
0 40 16 1 0.0 24 0 74 2 0
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.3.96.980819124710.17743A-100000>