From owner-freebsd-questions@freebsd.org Wed Jan 8 15:30:16 2020 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A5C581F2491 for ; Wed, 8 Jan 2020 15:30:16 +0000 (UTC) (envelope-from rfg@tristatelogic.com) Received: from outgoing.tristatelogic.com (segfault.tristatelogic.com [69.62.255.118]) by mx1.freebsd.org (Postfix) with ESMTP id 47tCqW3sh4z3DDk for ; Wed, 8 Jan 2020 15:30:15 +0000 (UTC) (envelope-from rfg@tristatelogic.com) Received: by segfault.tristatelogic.com (Postfix, from userid 1237) id 6B5994E71D; Wed, 8 Jan 2020 07:30:12 -0800 (PST) From: "Ronald F. Guilmette" To: freebsd-questions@freebsd.org Subject: Re: Independence of file descriptor flags across forks (or lack thereof) In-Reply-To: <20200108114244.b431a9ae0170ec947e6fb7d8@sohara.org> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <99920.1578497412.1@segfault.tristatelogic.com> Date: Wed, 08 Jan 2020 07:30:12 -0800 Message-ID: <99921.1578497412@segfault.tristatelogic.com> X-Rspamd-Queue-Id: 47tCqW3sh4z3DDk X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of rfg@tristatelogic.com designates 69.62.255.118 as permitted sender) smtp.mailfrom=rfg@tristatelogic.com X-Spamd-Result: default: False [-5.37 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[tristatelogic.com]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:14051, ipnet:69.62.128.0/17, country:US]; IP_SCORE(-3.07)[ip: (-8.05), ipnet: 69.62.128.0/17(-4.03), asn: 14051(-3.20), country: US(-0.05)] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jan 2020 15:30:16 -0000 In message <20200108114244.b431a9ae0170ec947e6fb7d8@sohara.org>, Steve O'Hara-Smith wrote: >> that are currently open in the parent process, the child would also get >> its own independent copy of the system-maintained "flags word" for each >> of those inherited file descriptor copies. > > The fork manpage tells you that the descriptors reference the same >underlying object. It is that object (the open file) which holds the "flags >word". I am not persuaded. I do not wish to be impertinent, but can you point me to the place in the UFS file system specification that will show me where the per-file blocking/ non-blocking bits are stored? >> So dear friends, I must ask you, am I delusional? Is this all just some >> massive misunderstanding on my part? > > Yep. I am sorry to disagree, but I am still not persuaded. Yes, the underyling thing is the same for the child and parent processes, but this fact alone does not imply that both the parent and child have the exact same view of that underlying thing -or- the exact same capabilities with respect to that underyling thing. If one or the other, parent process or child process, closes the file, then is the file also closed as far as the other process is concerned, from that time onward? Of course not. This fact alone proves that a given process' relationship with a file descriptor (or a message queue descriptor) is different and distinct from the underyling file itself. There must be maintained (somewhere) an open/closed bit which is a characteristic not of the underlying file itself, but rather of one particular process' current relationship with and to that underyling file. So there is clearly (a) the underlying thing, and then there is (b) each process' relationship to, and current capabilities in relation to that underlying thing. A file itself is neither blocking nor non-blocking, even if the underyling thing is some FIFO, pipe, or socket. It is still just a thing which can be acted upon by some process. One process can act on the thing, e.g. attempting to read from it, in a non-blocking manner, while the other process may elect, for its own reasons, to read from that some underlying thing with blocking semantics, i.e. waiting for data to be present before returning from the call to read() or mq_receive(). There is no contradition here, and indeed, I have found an instance in which what I habe just described is quite pragmatically useful. Unfortunately, neither of us have so far cited to any authority which might settle this matter definitively. I will now cite to an authority, but if I am to be intellectually honest then I will have to admit that even what I will now cite to may not clearly or definitively settle the matter. All I really have to go on, here with me, is a very old and yellowed hardcopy of IEEE "POSIX" 1003.1b-1993. In this hardcopy document, Section 6.5.2.2 seems most relevant, since it discusses various file attributes that can be read or written with the fcntl() system call. The fcntl operations and their respective descriptions that seem mosty directly relevant are these: F_GETFD F_SETFD F_GETFL F_SETFL Of course, the document that I am looking at dates from 1993, and thus may have been totally superceeded and rewritten by now, but I have also done a search for "fcntl" in the "system interfaces" section at this location: https://pubs.opengroup.org/onlinepubs/9699919799/ and I assume that this gives me more up-to-date information on the current semnatics of "POSIX-conformant" systems. Sadly however, all I get back in this case is something that looks an awful lot like a man page for fcntl() and it largely or entirely just repeats the same text as is present in my old hardcopy copy of the 1993 POSIX standard. Here are relevant passages: F_GETFD Get the file descriptor flags defined in that are associated with the file descriptor fildes. File descriptor flags are associated with a single file descriptor and do not affect other file descriptors that refer to the same file. F_SETFD Set the file descriptor flags defined in , that are associated with fildes, to the third argument, arg, taken as type int. If the FD_CLOEXEC flag in the third argument is 0, the file descriptor shall remain open across the exec functions; otherwise, the file descriptor shall be closed upon successful execution of one of the exec functions. F_GETFL Get the file status flags and file access modes, defined in , for the file description associated with fildes. The file access modes can be extracted from the return value using the mask O_ACCMODE, which is defined in . File status flags and file access modes are associated with the file description and do not affect other file descriptors that refer to the same file with different open file descriptions. The flags returned may include non-standard file status flags which the application did not set, provided that these additional flags do not alter the behavior of a conforming application. F_SETFL Set the file status flags, defined in , for the file description associated with fildes from the corresponding bits in the third argument, arg, taken as type int. Bits corresponding to the file access mode and the file creation flags, as defined in , that are set in arg shall be ignored. If any bits in arg other than those mentioned here are changed by the application, the result is unspecified. If fildes does not support non-blocking operations, it is unspecified whether the O_NONBLOCK flag will be ignored. As I say, all of the above is almost entirely just a reprint/reiteration of almost identical text which is present also in the hardcopy of the 1993 POSIX standard document that I have here. The only possibly relevant difference is that the POSIX document I have here, in its description of F_GETFL and F_SETFL makes explict reference to the attributes listed in table 6-5, where table 6-5, on the preceeding page, includes all of the following: O_APPEND O_DSYNC O_NONBLOCK O_RSYNC O_SYNC I believe that my position, i.e. that flags (such as O_NONBLOCK) must be maintained separately for each separate file descriptor, including even those that are derived from a "parent" file descriptor, is supported by the passage, reproduced above in relation to F_GETFL, that says explicitly: File status flags and file access modes are associated with the file description and do not affect other file descriptors that refer to the same file with different open file descriptions. Why else would the authors of this standard have included such stilted and, admittedly, imprecise language as that if they were NOT attempting to say exactly what I have said? And just to reiterate, what I have said is that the standard appears to me to require that there must exist a unique set of attribute bits, associated with each and every open file descriptor, -and- that POSIX requires that such a set of attributs must be maninated -separately- for each FD in each process, even for a pair of file descriptors that happen to be related by blood (i.e. one descending from the other). That is my reading of it anyway. Furthermore the section of this same standards document that talks about the fork() system call is at pains to say explicitly that the forked child obtains "a copy" of the parent's file descriptors, NOT the parent's file descriptors themselves. Based on that, after a fork(), if the parent had one FD, then there now exist *two* FDs in its place. And likewise, if the parent had two FDs, then after the fork() we can say that there now exist four FDs. This makes clear, I think, the special meaning of the phrasing "... do not affect other file descriptors that refer to the same file ...". After a fork() there are now twice as many FDs as there were before, and they are all separate and distinct things, in and of themselves, and fiddling the flags (such as the O_NONBLOCK) flag on one is *not* supposed to affect the setting of that same flag in any of the others. That is my interpretation anyway. But I welcome any reasonable challenge to the above reasoning. Regards, rfg