From owner-freebsd-standards@FreeBSD.ORG Mon Apr 25 11:02:23 2005 Return-Path: Delivered-To: freebsd-standards@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 631F516A4CE for ; Mon, 25 Apr 2005 11:02:23 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3C55B43D45 for ; Mon, 25 Apr 2005 11:02:23 +0000 (GMT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.3/8.13.3) with ESMTP id j3PB2N4s061546 for ; Mon, 25 Apr 2005 11:02:23 GMT (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.13.3/8.13.1/Submit) id j3PB2MAd061540 for freebsd-standards@freebsd.org; Mon, 25 Apr 2005 11:02:22 GMT (envelope-from owner-bugmaster@freebsd.org) Date: Mon, 25 Apr 2005 11:02:22 GMT Message-Id: <200504251102.j3PB2MAd061540@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: freebsd-standards@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Apr 2005 11:02:23 -0000 Current FreeBSD problem reports Critical problems Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2001/03/05] bin/25542 standards /bin/sh: null char in quoted string p [2002/02/25] standards/35307standards standard include files are not standard c o [2002/12/13] kern/46239 standards posix semaphore implementation errors o [2003/04/21] standards/51209standards [PATCH] add a64l()/l64a/l64a_r functions p [2003/06/05] standards/52972standards /bin/sh arithmetic not POSIX compliant o [2003/06/18] kern/53447 standards poll(2) semantics differ from susV3/POSIX o [2003/07/12] standards/54410standards one-true-awk not POSIX compliant (no exte o [2004/01/01] standards/60772standards _Bool and bool should be unsigned o [2004/11/03] standards/73500standards 'set +o' in /bin/sh does not include unse o [2005/03/03] standards/78357standards getaddrinfo() doesn't appear to support A o [2005/03/09] standards/78650standards ttyname_r() is not standards compliant o [2005/03/16] standards/78907standards does not define pthread typ 12 problems total. Non-critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/09/24] bin/21519 standards sys/dir.h should be deprecated some more o [2001/01/16] bin/24390 standards Replacing old dir-symlinks when using /bi s [2001/01/24] standards/24590standards timezone function not compatible witn Sin s [2001/06/18] kern/28260 standards UIO_MAXIOV needs to be made public p [2001/11/20] standards/32126standards getopt(3) not Unix-98 conformant o [2002/02/27] misc/35381 standards incorrect floating-point display of large s [2002/03/19] standards/36076standards Implementation of POSIX fuser command o [2002/06/14] standards/39256standards [v]snprintf aren't POSIX-conformant for s o [2002/07/09] kern/40378 standards stdlib.h gives needless warnings with -an p [2002/08/12] standards/41576standards POSIX compliance of ln(1) o [2002/10/23] standards/44425standards getcwd() succeeds even if current dir has o [2002/12/09] standards/46119standards Priority problems for SCHED_OTHER using p o [2003/07/24] standards/54809standards pcvt deficits o [2003/07/25] standards/54833standards more pcvt deficits o [2003/07/25] standards/54839standards pcvt deficits o [2003/07/31] standards/55112standards glob.h, glob_t's gl_pathc should be "size o [2003/09/05] standards/56476standards cd9660 unicode support simple hack o [2003/10/29] standards/58676standards grantpt(3) alters storage used by ptsname p [2003/12/26] standards/60597standards FreeBSD's /usr/include lacks of cpio.h s [2004/02/14] standards/62858standards malloc(0) not C99 compliant p [2004/02/21] standards/63173standards Patch to add getopt_long_only(3) to libc o [2004/03/29] kern/64875 standards [patch] add a system call: fdatasync() o [2004/05/07] standards/66357standards make POSIX conformance problem ('sh -e' & o [2004/05/11] standards/66531standards _gettemp uses a far smaller set of filena o [2004/08/22] standards/70813standards [PATCH] ls not Posix compliant o [2004/08/26] docs/70985 standards [patch] sh(1): incomplete documentation o o [2004/09/22] standards/72006standards floating point formating in non-C locales o [2005/03/20] standards/79055standards Add an IFS regression test for shells o [2005/03/20] standards/79056standards regex(3) regression tests o [2005/03/21] standards/79067standards /bin/sh should be more intelligent about 30 problems total. From owner-freebsd-standards@FreeBSD.ORG Mon Apr 25 17:17:11 2005 Return-Path: Delivered-To: freebsd-standards@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 77DA416A4CE; Mon, 25 Apr 2005 17:17:11 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5FD7C43D60; Mon, 25 Apr 2005 17:17:11 +0000 (GMT) (envelope-from bde@FreeBSD.org) Received: from freefall.freebsd.org (bde@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.3/8.13.3) with ESMTP id j3PHHBPi011377; Mon, 25 Apr 2005 17:17:11 GMT (envelope-from bde@freefall.freebsd.org) Received: (from bde@localhost) by freefall.freebsd.org (8.13.3/8.13.1/Submit) id j3PHHBxQ011373; Mon, 25 Apr 2005 17:17:11 GMT (envelope-from bde) Date: Mon, 25 Apr 2005 17:17:11 GMT From: Bruce Evans Message-Id: <200504251717.j3PHHBxQ011373@freefall.freebsd.org> To: bde@FreeBSD.org, freebsd-standards@FreeBSD.org, bde@FreeBSD.org Subject: Re: kern/53447: poll(2) semantics differ from susV3/POSIX X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Apr 2005 17:17:11 -0000 Synopsis: poll(2) semantics differ from susV3/POSIX Responsible-Changed-From-To: freebsd-standards->bde Responsible-Changed-By: bde Responsible-Changed-When: Mon Apr 25 17:04:09 GMT 2005 Responsible-Changed-Why: FreeBSD has changes that are supposed to fix problems in this area, but the changes seem to just make things worse -- there are now at least 2 more PRs about the new misbehaviour. The correct fix seems to be to simply implement POLLHUP, and not ignore EOF on FIFOs like FreeBSD does now. On EOF, applications polling for POLLIN (including via select() on a read descriptor) should get POLLIN returned (or the read descriptor bit set for select()) like they used to. For poll(), POLLHUP is set too, and applications should check this if they don't want to read EOF. For select(), it is not easy to avoid endlessy reading EOF in some cases, but POSIX is very clear (much clearer than for poll()) that select() on a read descriptor must return immediately on EOF. http://www.freebsd.org/cgi/query-pr.cgi?pr=53447 From owner-freebsd-standards@FreeBSD.ORG Tue Apr 26 14:07:02 2005 Return-Path: Delivered-To: freebsd-standards@freebsd.org Received: from green.homeunix.org (freefall.freebsd.org [216.136.204.21]) by hub.freebsd.org (Postfix) with ESMTP id 7DA5A16A4CE; Tue, 26 Apr 2005 14:07:02 +0000 (GMT) Received: from green.homeunix.org (green@localhost [127.0.0.1]) by green.homeunix.org (8.13.3/8.13.1) with ESMTP id j3QE728S017569; Tue, 26 Apr 2005 10:07:02 -0400 (EDT) (envelope-from green@green.homeunix.org) Received: (from green@localhost) by green.homeunix.org (8.13.3/8.13.1/Submit) id j3QE71HK017568; Tue, 26 Apr 2005 10:07:01 -0400 (EDT) (envelope-from green) Date: Tue, 26 Apr 2005 10:07:01 -0400 From: Brian Fundakowski Feldman To: Marc Olzheim Message-ID: <20050426140701.GB5789@green.homeunix.org> References: <20050419151800.GE1157@green.homeunix.org> <20050419160258.GA12287@stack.nl> <20050419160900.GB12287@stack.nl> <20050419161616.GF1157@green.homeunix.org> <20050419204723.GG1157@green.homeunix.org> <20050420140409.GA77731@stack.nl> <20050420142448.GH1157@green.homeunix.org> <20050420143842.GB77731@stack.nl> <16998.36437.809896.936800@khavrinen.csail.mit.edu> <20050420173859.GA99695@stack.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20050420173859.GA99695@stack.nl> User-Agent: Mutt/1.5.6i cc: Garrett Wollman cc: freebsd-standards@freebsd.org Subject: Re: NFS client/buffer cache deadlock X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Apr 2005 14:07:02 -0000 On Wed, Apr 20, 2005 at 07:38:59PM +0200, Marc Olzheim wrote: > On Wed, Apr 20, 2005 at 01:16:05PM -0400, Garrett Wollman wrote: > > < said: > > > > > Btw.: I'm not sure write(),writev() and pwrite() are allowed to do short > > > writes on regular files... ? > > > > I believe it is the intent of the Standard to prohibit this (a > > paragraph in the rationale says that short writes can only happen if > > O_NONBLOCK is set, but this is clearly wrong because the normative > > text says end-of-medium also results in a short write) but there does > > not appear to be any language which requires atomic behavior for > > descriptors other than pipes and FIFOs. > > > > As a quality-of-implementation matter, for writes to regular files not > > to be atomic would be considered surprising. > > > > -GAWollman > > Could someone from standards comment here ? I believe Garrett is > right... > (thread is on -hackers and -current) What prevents you from using O_FSYNC | O_APPEND to get the functionality you desire? The semantics of IO_UNIT -- atomic writes -- are definitely defined and assumed to function properly by the rest of the kernel. Allowing asynchronous unbounded atomic appends is impossible, so something must be done to prevent deadlock. Breaking IO_UNIT really shouldn't be considered as a solution. Automatically turning the write into a synchronous + atomic append if an asynchrous + atomic append is not possible might follow POLA best. -- Brian Fundakowski Feldman \'[ FreeBSD ]''''''''''\ <> green@FreeBSD.org \ The Power to Serve! \ Opinions expressed are my own. \,,,,,,,,,,,,,,,,,,,,,,\ From owner-freebsd-standards@FreeBSD.ORG Tue Apr 26 15:17:53 2005 Return-Path: Delivered-To: freebsd-standards@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3DFB616A4CE; Tue, 26 Apr 2005 15:17:53 +0000 (GMT) Received: from mailhost.stack.nl (vaak.stack.nl [131.155.140.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5490943D2D; Tue, 26 Apr 2005 15:17:52 +0000 (GMT) (envelope-from marcolz@stack.nl) Received: from hammer.stack.nl (hammer.stack.nl [IPv6:2001:610:1108:5010::153]) by mailhost.stack.nl (Postfix) with ESMTP id 5FBE61F031; Tue, 26 Apr 2005 17:17:51 +0200 (CEST) Received: by hammer.stack.nl (Postfix, from userid 333) id 4642C663D; Tue, 26 Apr 2005 17:17:51 +0200 (CEST) Date: Tue, 26 Apr 2005 17:17:51 +0200 From: Marc Olzheim To: Brian Fundakowski Feldman Message-ID: <20050426151751.GB68038@stack.nl> References: <20050419160258.GA12287@stack.nl> <20050419160900.GB12287@stack.nl> <20050419161616.GF1157@green.homeunix.org> <20050419204723.GG1157@green.homeunix.org> <20050420140409.GA77731@stack.nl> <20050420142448.GH1157@green.homeunix.org> <20050420143842.GB77731@stack.nl> <16998.36437.809896.936800@khavrinen.csail.mit.edu> <20050420173859.GA99695@stack.nl> <20050426140701.GB5789@green.homeunix.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="azLHFNyN32YCQGCU" Content-Disposition: inline In-Reply-To: <20050426140701.GB5789@green.homeunix.org> X-Operating-System: FreeBSD hammer.stack.nl 5.4-STABLE FreeBSD 5.4-STABLE X-URL: http://www.stack.nl/~marcolz/ User-Agent: Mutt/1.5.9i cc: Garrett Wollman cc: freebsd-standards@freebsd.org Subject: Re: NFS client/buffer cache deadlock X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Apr 2005 15:17:53 -0000 --azLHFNyN32YCQGCU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Apr 26, 2005 at 10:07:01AM -0400, Brian Fundakowski Feldman wrote: > > Could someone from standards comment here ? I believe Garrett is > > right... > > (thread is on -hackers and -current) >=20 > What prevents you from using O_FSYNC | O_APPEND to get the > functionality you desire? The semantics of IO_UNIT -- atomic writes > -- are definitely defined and assumed to function properly by the rest > of the kernel. Allowing asynchronous unbounded atomic appends is > impossible, so something must be done to prevent deadlock. Breaking > IO_UNIT really shouldn't be considered as a solution. Automatically > turning the write into a synchronous + atomic append if an asynchrous > + atomic append is not possible might follow POLA best. I don't care whether a user application corrupts it's own data by writing simultaneously to the same file from different hosts; that's the choice of the application. What I want is when the application behaves and is the only one writing to the file, that that writev() succeeds. I'm okay with the fact that simultaneous huge writes to the same file over NFS could lead to corruption and that the exact outcome is undefined. This is exactly how it was in FreeBSD 4.x and that's perfectly workable. But that's just my way of looking at it and certainly not ideal. :-/ Marc --azLHFNyN32YCQGCU Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (FreeBSD) iD8DBQFCblufezjnobFOgrERAiaHAJ0ax7p0SXtg5Fh4AULAydrTxZ0HSwCeOiJB bkyOC3BxMT5LBHfB+6ngWl4= =HIYP -----END PGP SIGNATURE----- --azLHFNyN32YCQGCU-- From owner-freebsd-standards@FreeBSD.ORG Tue Apr 26 15:50:45 2005 Return-Path: Delivered-To: freebsd-standards@freebsd.org Received: from green.homeunix.org (freefall.freebsd.org [216.136.204.21]) by hub.freebsd.org (Postfix) with ESMTP id B765F16A4CE; Tue, 26 Apr 2005 15:50:44 +0000 (GMT) Received: from green.homeunix.org (green@localhost [127.0.0.1]) by green.homeunix.org (8.13.3/8.13.1) with ESMTP id j3QFoilE018201; Tue, 26 Apr 2005 11:50:44 -0400 (EDT) (envelope-from green@green.homeunix.org) Received: (from green@localhost) by green.homeunix.org (8.13.3/8.13.1/Submit) id j3QFoiJF018200; Tue, 26 Apr 2005 11:50:44 -0400 (EDT) (envelope-from green) Date: Tue, 26 Apr 2005 11:50:43 -0400 From: Brian Fundakowski Feldman To: Marc Olzheim Message-ID: <20050426155043.GC5789@green.homeunix.org> References: <20050419160900.GB12287@stack.nl> <20050419161616.GF1157@green.homeunix.org> <20050419204723.GG1157@green.homeunix.org> <20050420140409.GA77731@stack.nl> <20050420142448.GH1157@green.homeunix.org> <20050420143842.GB77731@stack.nl> <16998.36437.809896.936800@khavrinen.csail.mit.edu> <20050420173859.GA99695@stack.nl> <20050426140701.GB5789@green.homeunix.org> <20050426151751.GB68038@stack.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20050426151751.GB68038@stack.nl> User-Agent: Mutt/1.5.6i cc: Garrett Wollman cc: freebsd-standards@freebsd.org Subject: Re: NFS client/buffer cache deadlock X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Apr 2005 15:50:45 -0000 On Tue, Apr 26, 2005 at 05:17:51PM +0200, Marc Olzheim wrote: > On Tue, Apr 26, 2005 at 10:07:01AM -0400, Brian Fundakowski Feldman wrote: > > > Could someone from standards comment here ? I believe Garrett is > > > right... > > > (thread is on -hackers and -current) > > > > What prevents you from using O_FSYNC | O_APPEND to get the > > functionality you desire? The semantics of IO_UNIT -- atomic writes > > -- are definitely defined and assumed to function properly by the rest > > of the kernel. Allowing asynchronous unbounded atomic appends is > > impossible, so something must be done to prevent deadlock. Breaking > > IO_UNIT really shouldn't be considered as a solution. Automatically > > turning the write into a synchronous + atomic append if an asynchrous > > + atomic append is not possible might follow POLA best. > > I don't care whether a user application corrupts it's own data by > writing simultaneously to the same file from different hosts; that's the > choice of the application. What I want is when the application behaves > and is the only one writing to the file, that that writev() succeeds. > > I'm okay with the fact that simultaneous huge writes to the same file > over NFS could lead to corruption and that the exact outcome is > undefined. > > This is exactly how it was in FreeBSD 4.x and that's perfectly workable. > > But that's just my way of looking at it and certainly not ideal. :-/ I don't know what you mean. The exact same bug should exists in 4.x, and should cause a system deadlock in exactly the same scenario. Simultaneous huge writes for NFSv3 were and still are atomic and I do not intend to break that -- just make it so they won't deadlock the system. I'm not okay with making applications suddenly start corrupting data. Why can't you use O_FSYNC for your huge writes? I'm willing to bet that its semantics are exactly what you're looking for. -- Brian Fundakowski Feldman \'[ FreeBSD ]''''''''''\ <> green@FreeBSD.org \ The Power to Serve! \ Opinions expressed are my own. \,,,,,,,,,,,,,,,,,,,,,,\ From owner-freebsd-standards@FreeBSD.ORG Tue Apr 26 16:06:11 2005 Return-Path: Delivered-To: freebsd-standards@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3A68D16A4CF; Tue, 26 Apr 2005 16:06:11 +0000 (GMT) Received: from mailhost.stack.nl (vaak.stack.nl [131.155.140.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id 85B2443D45; Tue, 26 Apr 2005 16:06:10 +0000 (GMT) (envelope-from marcolz@stack.nl) Received: from hammer.stack.nl (hammer.stack.nl [IPv6:2001:610:1108:5010::153]) by mailhost.stack.nl (Postfix) with ESMTP id 8AD2A1F038; Tue, 26 Apr 2005 18:06:09 +0200 (CEST) Received: by hammer.stack.nl (Postfix, from userid 333) id 6C6FA663D; Tue, 26 Apr 2005 18:06:09 +0200 (CEST) Date: Tue, 26 Apr 2005 18:06:09 +0200 From: Marc Olzheim To: Brian Fundakowski Feldman Message-ID: <20050426160609.GA68511@stack.nl> References: <20050419161616.GF1157@green.homeunix.org> <20050419204723.GG1157@green.homeunix.org> <20050420140409.GA77731@stack.nl> <20050420142448.GH1157@green.homeunix.org> <20050420143842.GB77731@stack.nl> <16998.36437.809896.936800@khavrinen.csail.mit.edu> <20050420173859.GA99695@stack.nl> <20050426140701.GB5789@green.homeunix.org> <20050426151751.GB68038@stack.nl> <20050426155043.GC5789@green.homeunix.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="9jxsPFA5p3P2qPhR" Content-Disposition: inline In-Reply-To: <20050426155043.GC5789@green.homeunix.org> X-Operating-System: FreeBSD hammer.stack.nl 5.4-STABLE FreeBSD 5.4-STABLE X-URL: http://www.stack.nl/~marcolz/ User-Agent: Mutt/1.5.9i cc: Garrett Wollman cc: freebsd-standards@freebsd.org Subject: Re: NFS client/buffer cache deadlock X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Apr 2005 16:06:11 -0000 --9jxsPFA5p3P2qPhR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Apr 26, 2005 at 11:50:43AM -0400, Brian Fundakowski Feldman wrote: > > I'm okay with the fact that simultaneous huge writes to the same file > > over NFS could lead to corruption and that the exact outcome is > > undefined. > >=20 > > This is exactly how it was in FreeBSD 4.x and that's perfectly workable. > >=20 > > But that's just my way of looking at it and certainly not ideal. :-/ >=20 > I don't know what you mean. The exact same bug should exists in 4.x, > and should cause a system deadlock in exactly the same scenario. I'm not sure you understand the "scenario". All I do is create a new file and writev 600 * 1MB to it. This creates a VFS hangup on FreeBSD 5.x after writing an amount of 2-100 MB (depending on how much memory is in the system), while 4.x just does what it is told and doesn't hangup. I do not have any synchronisation problems. See kern/79208 Marc --9jxsPFA5p3P2qPhR Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (FreeBSD) iD8DBQFCbmbxezjnobFOgrERAn9DAJ944AM1NardYTUU3Gv/QA4ym0Hd3ACcCrQn DwYvjfTwGd3v7iTxdlZczdI= =7qgW -----END PGP SIGNATURE----- --9jxsPFA5p3P2qPhR-- From owner-freebsd-standards@FreeBSD.ORG Tue Apr 26 16:25:50 2005 Return-Path: Delivered-To: freebsd-standards@freebsd.org Received: from green.homeunix.org (freefall.freebsd.org [216.136.204.21]) by hub.freebsd.org (Postfix) with ESMTP id A95DC16A4CE; Tue, 26 Apr 2005 16:25:50 +0000 (GMT) Received: from green.homeunix.org (green@localhost [127.0.0.1]) by green.homeunix.org (8.13.3/8.13.1) with ESMTP id j3QGPokY018497; Tue, 26 Apr 2005 12:25:50 -0400 (EDT) (envelope-from green@green.homeunix.org) Received: (from green@localhost) by green.homeunix.org (8.13.3/8.13.1/Submit) id j3QGPoZJ018496; Tue, 26 Apr 2005 12:25:50 -0400 (EDT) (envelope-from green) Date: Tue, 26 Apr 2005 12:25:49 -0400 From: Brian Fundakowski Feldman To: Marc Olzheim Message-ID: <20050426162549.GD5789@green.homeunix.org> References: <20050419204723.GG1157@green.homeunix.org> <20050420140409.GA77731@stack.nl> <20050420142448.GH1157@green.homeunix.org> <20050420143842.GB77731@stack.nl> <16998.36437.809896.936800@khavrinen.csail.mit.edu> <20050420173859.GA99695@stack.nl> <20050426140701.GB5789@green.homeunix.org> <20050426151751.GB68038@stack.nl> <20050426155043.GC5789@green.homeunix.org> <20050426160609.GA68511@stack.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20050426160609.GA68511@stack.nl> User-Agent: Mutt/1.5.6i cc: freebsd-standards@freebsd.org Subject: Re: NFS client/buffer cache deadlock X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Apr 2005 16:25:51 -0000 On Tue, Apr 26, 2005 at 06:06:09PM +0200, Marc Olzheim wrote: > On Tue, Apr 26, 2005 at 11:50:43AM -0400, Brian Fundakowski Feldman wrote: > > > I'm okay with the fact that simultaneous huge writes to the same file > > > over NFS could lead to corruption and that the exact outcome is > > > undefined. > > > > > > This is exactly how it was in FreeBSD 4.x and that's perfectly workable. > > > > > > But that's just my way of looking at it and certainly not ideal. :-/ > > > > I don't know what you mean. The exact same bug should exists in 4.x, > > and should cause a system deadlock in exactly the same scenario. > > I'm not sure you understand the "scenario". All I do is create a new > file and writev 600 * 1MB to it. This creates a VFS hangup on FreeBSD > 5.x after writing an amount of 2-100 MB (depending on how much memory is > in the system), while 4.x just does what it is told and doesn't hangup. > > I do not have any synchronisation problems. > > See kern/79208 Then it sounds like for whatever reason FreeBSD 4.x isn't negotiating NFSv3 properly and should be fixed. This is fundamentally a deadlock situation. The write is a transaction and requires any part of the write request may be retransmitted. This can only be accomplished by retaining the entire write contents for the duration of the operation. You can assure that this happens in only two ways: 1. Make a complete copy of the data. This is what currently occurs: it gets stuffed into the buffer cache as the write happens. 2. Keep the data around synchronously -- by virtue of the write system call being used synchronously, the thread's VM context is around, and duplication need not occur. I'm trying to fix all situations, not just yours. I think I've changed my mind about short writes being acceptable simply because short writes will cause detectable corruption, but once detected, you have no knowledge of the exact location of the corruption. So it's really only a choice between forcing synchronous operation implicitly, or explicitly by returning an error that says no data at all was written. -- Brian Fundakowski Feldman \'[ FreeBSD ]''''''''''\ <> green@FreeBSD.org \ The Power to Serve! \ Opinions expressed are my own. \,,,,,,,,,,,,,,,,,,,,,,\ From owner-freebsd-standards@FreeBSD.ORG Fri Apr 29 06:54:55 2005 Return-Path: Delivered-To: freebsd-standards@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ECE3016A4CE for ; Fri, 29 Apr 2005 06:54:55 +0000 (GMT) Received: from harmony.village.org (rover.village.org [168.103.84.182]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2F66843D2D for ; Fri, 29 Apr 2005 06:54:55 +0000 (GMT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.13.3/8.13.1) with ESMTP id j3T6qdd4016698; Fri, 29 Apr 2005 00:52:43 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Fri, 29 Apr 2005 00:53:17 -0600 (MDT) Message-Id: <20050429.005317.69580336.imp@bsdimp.com> To: keramida@ceid.upatras.gr From: "M. Warner Losh" In-Reply-To: <20050420162332.GB52948@orion.daedalusnetworks.priv> References: <20050420155407.GA844@falcon.midgard.homeip.net> <84dead720504200910441b9108@mail.gmail.com> <20050420162332.GB52948@orion.daedalusnetworks.priv> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: freebsd-standards@freebsd.org Subject: Re: Standard type for code pointers? X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Apr 2005 06:54:56 -0000 In message: <20050420162332.GB52948@orion.daedalusnetworks.priv> Giorgos Keramidas writes: : > Is 'register_t' guaranteed to be wide enough? : : AFAIK, no. Portable C code cannot assume that a function pointer is : small enough to fit in a single machine register. Some obscure : architecture may choose to represent function entry points with as : many register as it needs. You mean like medium model (64k data, larger code) 8086 :-) Warner From owner-freebsd-standards@FreeBSD.ORG Fri Apr 29 18:04:25 2005 Return-Path: Delivered-To: freebsd-standards@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CCAFE16A4CE; Fri, 29 Apr 2005 18:04:25 +0000 (GMT) Received: from odin.ac.hmc.edu (Odin.AC.HMC.Edu [134.173.32.75]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8563D43D1F; Fri, 29 Apr 2005 18:04:25 +0000 (GMT) (envelope-from brdavis@odin.ac.hmc.edu) Received: from odin.ac.hmc.edu (localhost.localdomain [127.0.0.1]) by odin.ac.hmc.edu (8.13.0/8.13.0) with ESMTP id j3TI4PWo012513; Fri, 29 Apr 2005 11:04:25 -0700 Received: (from brdavis@localhost) by odin.ac.hmc.edu (8.13.0/8.13.0/Submit) id j3TI4PiA012512; Fri, 29 Apr 2005 11:04:25 -0700 Date: Fri, 29 Apr 2005 11:04:25 -0700 From: Brooks Davis To: threads@freebsd.org, standards@freebsd.org Message-ID: <20050429180425.GA11772@odin.ac.hmc.edu> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="pWyiEgJYm5f9v55/" Content-Disposition: inline User-Agent: Mutt/1.4.1i X-Virus-Scanned: by amavisd-new X-Spam-Status: No, hits=0.0 required=8.0 tests=none autolearn=no version=2.63 X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on odin.ac.hmc.edu Subject: standards/78907: does not define pthread types X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Apr 2005 18:04:25 -0000 --pWyiEgJYm5f9v55/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I would like to commit the patch in standards/78907 to HEAD with the intent of MFCing to RELENG_5. This patch is required to compile Sun Grid Engine 6. Are there any objections to it? The summary is that it extracts the pthread types from pthread.h into sys/_pthreadtypes.h and includes that in both sys/types.h and pthread.h. -- Brooks --=20 Any statement of the form "X is the one, true Y" is FALSE. PGP fingerprint 655D 519C 26A7 82E7 2529 9BF0 5D8E 8BE9 F238 1AD4 --pWyiEgJYm5f9v55/ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (GNU/Linux) iD8DBQFCcncoXY6L6fI4GtQRAgHPAJsGPBtA3T3QTACdjJKsmyaNPA9qYgCgj0/r mDr9mSBQhZ4A7ffH2b1nt5s= =Rjvl -----END PGP SIGNATURE----- --pWyiEgJYm5f9v55/-- From owner-freebsd-standards@FreeBSD.ORG Fri Apr 29 18:50:38 2005 Return-Path: Delivered-To: freebsd-standards@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4A8D816A4CF; Fri, 29 Apr 2005 18:50:38 +0000 (GMT) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id BA84943D53; Fri, 29 Apr 2005 18:50:37 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) j3TIoYtq002799; Fri, 29 Apr 2005 14:50:36 -0400 (EDT) Date: Fri, 29 Apr 2005 14:50:34 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Brooks Davis In-Reply-To: <20050429180425.GA11772@odin.ac.hmc.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) cc: threads@freebsd.org cc: standards@freebsd.org Subject: Re: standards/78907: does not define pthread types X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Daniel Eischen List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Apr 2005 18:50:38 -0000 On Fri, 29 Apr 2005, Brooks Davis wrote: > I would like to commit the patch in standards/78907 to HEAD with the > intent of MFCing to RELENG_5. This patch is required to compile Sun > Grid Engine 6. Are there any objections to it? > > The summary is that it extracts the pthread types from pthread.h into > sys/_pthreadtypes.h and includes that in both sys/types.h and pthread.h. Sure, go ahead. -- DE