From owner-freebsd-arch@FreeBSD.ORG Thu Feb 28 08:36:07 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5DBA9106566B; Thu, 28 Feb 2008 08:36:07 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from mail17.syd.optusnet.com.au (mail17.syd.optusnet.com.au [211.29.132.198]) by mx1.freebsd.org (Postfix) with ESMTP id E10468FC1A; Thu, 28 Feb 2008 08:36:06 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from server.vk2pj.dyndns.org (c220-239-20-82.belrs4.nsw.optusnet.com.au [220.239.20.82]) by mail17.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m1S8ZtAk018630 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 28 Feb 2008 19:36:04 +1100 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.2/8.14.1) with ESMTP id m1S8ZtfL075765; Thu, 28 Feb 2008 19:35:55 +1100 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.2/8.14.2/Submit) id m1S8Zta7075764; Thu, 28 Feb 2008 19:35:55 +1100 (EST) (envelope-from peter) Date: Thu, 28 Feb 2008 19:35:55 +1100 From: Peter Jeremy To: John Baldwin Message-ID: <20080228083555.GX83599@server.vk2pj.dyndns.org> References: <200802262251.m1QMp7bV021709@hergotha.csail.mit.edu> <200802262355.16519.jhb@freebsd.org> <20080227065925.GK83599@server.vk2pj.dyndns.org> <200802271138.33979.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="gwtGiOGliFx8mAnm" Content-Disposition: inline In-Reply-To: <200802271138.33979.jhb@freebsd.org> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.17 (2007-11-01) Cc: arch@freebsd.org Subject: Re: Cleaning up FILE in stdio.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Feb 2008 08:36:07 -0000 --gwtGiOGliFx8mAnm Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Feb 27, 2008 at 11:38:33AM -0500, John Baldwin wrote: >> You could change _file from 'short' to 'unsigned short' without breaking >> the ABI - this would allow either 65535 or 65536 file descriptors (I'm >> not sure whether _file =3D=3D -1 is special or not). This would postpone >> the problem for some time. > >-1 is used a lot in the stdio code for file's not backed by an fd. My pro= blem=20 >though is that this doesn't help with existing binaries that are already= =20 >compiled (which is what I have to deal with). Had fileno() not been inlin= ed=20 >I would have been ok, but that's pretty much done for me as far as my curr= ent=20 >problem on 6.x. Had I just been able to change FILE * and not had inlines= ,=20 >then a new fopen would have worked fine in my case. My suggestion was based on short and ushort having the same size and (short)-1 and (ushort)65535 having the same bit pattern. Any code accessing fileno() should not be checking or validating the result but just passing it to low-level I/O routines. This would provide the following: Existing code New code short ushort FD sign-extended zero-extended -1 -1 65535 0..32767 0..32767 0..32767 32768..65534 -32768..-2 [*] 32768..65534 >65534 EMFILE EMFILE [*] This could potentially be fixed using libc or kernel shims. >> e) Don asbestos underwear and re-arrange struct __sFILE to grow _file et= c. > >We can't do e) because thanks to symbol versioning, 8.x and 9.x will have= =20 >libc.so.7, so a 7.0 binary will still use the brand new libc, so it has to= =20 >preserve the ABI of the currently exported fields pretty much forever. Erk. I forgot about that. >think we can get away with renaming '_file' to '_ofile' and adding a new '= int=20 >_file' at the bottom of the struct and making sure '_ofile' is always in s= ync=20 >(when possible, truncated when _file is too bug). Truncation opens up the possibility that old executables could fopen() lots of files (without getting any indication of a problem) and then use fileno() to reference a truncated _ofile - causing it to access some totally unrelated file. Admittedly, that is no worse than would happen today. >Also, I think we can do the new _file in HEAD for 8.0 w/o any worries. I= =20 >don't think waiting until 9.0 buys anything there. I was thinking in terms of changing _file to int without backward compatibility. I'm not sure that that could be done for 8.0 (though, as you have pointed out, it can't be done at all). > Given that, I think I'd=20 >rather just patch the current stable branches to handle the edge case bett= er=20 >and work on making _file an int in HEAD (with the ABI compat _ofile). The EMFILE patch is definitely a good interim step and I support your efforts in removing the limit on the number of open files. --=20 Peter Jeremy Please excuse any delays as the result of my ISP's inability to implement an MTA that is either RFC2821-compliant or matches their claimed behaviour. --gwtGiOGliFx8mAnm Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHxnJr/opHv/APuIcRApt/AJ0Qt+qUxxS2/4DlxAtKZhmoDec/iACfbDW5 5lxxTVS2ZkAeLNjdZ2Pm0QI= =QU7W -----END PGP SIGNATURE----- --gwtGiOGliFx8mAnm--