Date: Fri, 15 Jan 1999 23:10:25 +0000 (GMT) From: Terry Lambert <tlambert@primenet.com> To: dillon@apollo.backplane.com (Matthew Dillon) Cc: tlambert@primenet.com, mike@smith.net.au, rch@iserve.net, freebsd-hackers@FreeBSD.ORG Subject: Re: Problems with 3.0 Message-ID: <199901152310.QAA17271@usr04.primenet.com> In-Reply-To: <199901140307.TAA16614@apollo.backplane.com> from "Matthew Dillon" at Jan 13, 99 07:07:17 pm
next in thread | previous in thread | raw e-mail | index | archive | help
> :Actually, no. All it requires is the VM system reclaiming a clean > :page from an in core object without telling the object about it. > : > :This can't happen for most things, but mmap seems immune from > :correct functioning in a lot of ways, and this happens to be one > :of them. The pages that get zapped are in the libc code section > :for the mmap'ed libc.so image. This probably has something to do > :with the fact that the same libc is mapped into multiple processes > :(e.g., here's some place where having aliases in your VM sucks out). [ ... ] > If the author updated either libc or httpd while the web > server was running, it could certainly have caused the > segfault later on. I see this consistently in 2.2.7 and 2.2.6 systems, after the Dyson patches, but before the fixes to mmap and other areas that have happened subsequently. I have also seen this, in a few limited cases, in a 2.2.8 system, and in a 3.0-current system that is supposedly fixed. Most frequently, it's related to the use of a Cyrix 486 core, or a Cyrix Media GX processor. The 486 core doesn't have an L2 cache on the board, so "bad 486 cache coherency", runored to cause the problem, must remain an unconfirmed rumor. In general, I can see this in sendmail *without* needing to have inetd puke first to cause the problem (e.g., this is *not* a swap reclaim, it is an LRU reclaim). The end result is a page mapping for the libc hung of the vmobject hung off the libc vnode that is pointed to by another object at the same time, apparently as a result of a reclaim of the clean page from the libc vnode's object, but without correctly removing it from the object's list of page references. A subsequent use of the page as a data page results in the correct data for the second reference being in the page, but of course the libc's vnode's object's page appears to be "corrupted". Killing and restarting the daemon, which reinstances the page table for the process, fixes the problem (obviously). I've been considering modifying the procfs so that one can dump region mappings and then represent them graphically so as to quickly locate the problem; there has been some interest in this by the KAFFE people working on the new garbage collector routines, since there are situations where there are memory leaks and/or mappings that are unreferenced, but not recovered, and such a tool would show that pretty dramatically. A similar, but, I believe, unrelated issue occurs in the Vixie cron code. The Vixie cron code modifies the returned pwent entries; this is something that it should not do, since it is based on the incorrect assumption that the content fields are, in fact, static buffers. Given that the returned data is, in fact, in an mmap'ed Berkeley DBM record, what is happening is a copy-on-write fault for the modified data. In an error that appears to be related to the use of a read-only file being mapped copy-on-write instead of private (for the password database page affected), the result is that another file ends up referencing the page that nominally belongs to the password database. In other words, a "read-only" page that has been written is reclaimed, leaving it dirty. The resulting failure is that the modified page from the password database is written back to another file on the system. In general, the file that gets spammed is the crontab; I suspect that this is the result of the file being very active (the system in question runs newsyslog from cron with *absurd* frequency). However, there have been reports of the /etc/aliases.db file (also an mmap'ed Berkeley DB file!) and the /etc/group file. Therefore the problem is not limited to the crontab, and, in effect, all files in the system are subject to damage by "the page sniper". > :Another one recently discovered is that it *appears* that you can > :map (read only) the same file twice letting the system decide where > :to put it (pass NULL as binding location), and the second time the > :system will return the same address as the first time, instead of > :doing the right thing and setting up the mapping at a different > :location. > > I've not seen this either. Do you have an example or PR? I > wrote a quick little test program to map the same file several > times and it seems to map it at different locations as it > should. It was hardly an exhaustive test, though. I'll try to come up with an example. The problem is that the program that causes this problem is a derivative work of a commercial product, and so I will have to do some work to duplicate it in a smaller piece of code. The apparent problem is twofold: 1) Calling munmap(2) with a formerly good mapping address should result in a -1 return and an "EINVAL", not a page fault. I think this is (occasionally) broken, from a truss of the affected program. We recognized the problem existing first by seeing the resulting coredumps, apparently from the munmap(2) call. 2) The mmap(2) is *definitely* returning the same address for the second mapping. This should be pretty trivial to reproduce, but it could be dependent on the other parameters to mmap(2). One would think, though, that it would have to be pretty glaringly obvious, but I believe that there are few programs that actually do this type of weird crap, so it may just be a corner case that was never tested. If worse comes to worse, I'll drop the sample program problem on the lap of the person whose code it was affecting... PS: You missed the BAFUG meeting last night! Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199901152310.QAA17271>