Date: Sun, 25 Feb 2001 02:40:16 -0600 From: seebs@plethora.net (Peter Seebach) To: freebsd-hackers@freebsd.org Subject: Re: Setting memory allocators for library functions. Message-ID: <200102250840.f1P8eG625861@guild.plethora.net> In-Reply-To: Your message of "Sun, 25 Feb 2001 01:29:43 MST." <200102250829.f1P8Thd03268@harmony.village.org>
next in thread | previous in thread | raw e-mail | index | archive | help
In message <200102250829.f1P8Thd03268@harmony.village.org>, Warner Losh writes: >It says can, not must. But if it "can be used", and no one says "undefined behavior", then we're done; no undefined behavior is allowed. >I disagree with you that you can't overcommit. >Also, access is ill defined. It's defined better in C99 than it was in C89. >Does that mean read access? write >access? How about execute access? Within "defined behavior", read and write; all jumps to data are undefined behavior. >The standard makes no promises >that you can do both, and only has the vaguely defined access as to >what you can do with it. Once again, if it had been intended to allow only limited access, it would say so. >Reasonable people can differ as to what this means. Yes. >: Performance characteristics are not at issue; I don't think anyone on the >: committee would complain about an implementation which provided "committed" >: space by backing it all to disk, all the time. >No one would. However, the standard does not preclude the ability to >overcommit. It is certainly read that way by a number of people, and I think we're currently a majority on the committee. No one cares; it's not as though an implementation is required to be in the maximally conforming mode at all times. >Many different systems do this already. Yes, they do. That doesn't mean it's correct. I don't believe there's a single completely correct implementation of the fully ammended C89 standard, let alone C99. >Really, I guess I missed those debates. I've not seen it in the >literature. I've not seen this addressed in the last 20 years that >systems have been overcomitting in any meaningful way. It's an old flamewar on comp.std.c. :) >: That's not what "allocate" means. A resource which is *allocated* is one >: you actually have access to. >Not necessarily. Allocate means, in the absense of resource >shortages, you get the memory. No. To allocate is to get. Allocation *fails* in the case of a resource shortage. >You might get the memory, you might >not. Consider a system that has a quota on dirty pages. If I malloc >a huge array, and then only use part of it, I can stay under my >quota. But if I use most of it by dirtying it, then the system can >kill the process. Sure. Now imagine a quota on file I/O. If I start writing to a file, I'm under my quota, but if I exceed it, then the system can kill the process. If the standard had meant to say "access to allocated space invokes undefined behavior", it would have done so. Would you argue that the system which simply *kills* a process that tries to write to a full disk is correct? If not, can you show how that's any different? >Yes it does. You do not understand. If there's an ECC error, then >you do not have access to the memory without errors. This is no >different, just a different class of error. Hardware errors do, indeed, produce a non-conforming implementation. However, on a system where the hardware works, overcommit-and-kill is *STILL* a conformance problem; killing on ECC errors isn't a conformance problem unless there are some. >That's not necessarily true. There's nothing in the standard that >states that you have to be able to write to all parts of an allocated >region. Sure, and there's nothing in the standard that explicitly says "a call to frwrite with valid arguments should not raise SIGKILL", but everyone knows that would be a mistake. >It is a perfectly reasonable implementation to return the >memory from malloc. It's very reasonable, but it does not provide the environment described in the standard, which is an environment in which the memory returned by malloc is all available for use. >might be 100 right now, but 10 later. This is a host environment >issue. The malloc call gets a resource, but in the future you might >not be able to access that resource due to circumstances beyond the >control of the program. The same argument applies to killing programs when they write to a full disk. >Also, the program does not *HAVE* to succeed. If you have a low CPU >quota, for example, and the program exceeds it, the host environment >will signal SIGXCPU to the process. Another process might send this >one a SIGKILL. The system may crash in the middle of its execution. >There might be an unrecoverable memory parity error. There might be >an unrecoverable disk error on the swap partition. There might be any >number of things that preclude it from reaching completion, beyond >your control. Yes, and *EVERY ONE OF THOSE* is a conformance failure. I'm not saying we can produce an environment which is guaranteed to conform in all cases; it's obviously impossible. We *could* avoid breaking things. >I don't see how this is significantly different than a dirty page >quota or a dirty page limit. Very simple: We don't have control over hardware, and the Unix environment says you're allowed to do all sorts of things which, from the point of view of the spec, magically render the execution environment non-conforming. But that doesn't mean we need to *add* one. >That's what we're talking about here. >There's nothing in the standard that precludes host limits that do not >fit into the standard well. There's nothing that *allows* for them, though. Once again, by the same reasoning, we could defer fopen() until we actually start writing to the file, and *THEN* kill the process if fopen()'s path was invalid. It's just as good an idea, and it's just as correct. >It would be extremely difficult to provide as an option. The >over commit stuff is down in the kernel's vm system. There's no real >way to have a single process get "guaranteed" access to anything. It >is way ugly to change. Okay, so it's hard. I still say that guaranteed memory is a legitimate request for a process to make, which the OS should be able to satisfy in at least some cases. -s To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200102250840.f1P8eG625861>