From owner-freebsd-bugs@FreeBSD.ORG Tue Jul 13 21:30:07 2010 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C002E106566B for ; Tue, 13 Jul 2010 21:30:07 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 82F9D8FC08 for ; Tue, 13 Jul 2010 21:30:07 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o6DLU74d061454 for ; Tue, 13 Jul 2010 21:30:07 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o6DLU7jj061451; Tue, 13 Jul 2010 21:30:07 GMT (envelope-from gnats) Resent-Date: Tue, 13 Jul 2010 21:30:07 GMT Resent-Message-Id: <201007132130.o6DLU7jj061451@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Manish Vachharajani Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 967ED106564A for ; Tue, 13 Jul 2010 21:26:45 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21]) by mx1.freebsd.org (Postfix) with ESMTP id 852658FC1A for ; Tue, 13 Jul 2010 21:26:45 +0000 (UTC) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.14.3/8.14.3) with ESMTP id o6DLQjvV079459 for ; Tue, 13 Jul 2010 21:26:45 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.14.3/8.14.3/Submit) id o6DLQj3q079458; Tue, 13 Jul 2010 21:26:45 GMT (envelope-from nobody) Message-Id: <201007132126.o6DLQj3q079458@www.freebsd.org> Date: Tue, 13 Jul 2010 21:26:45 GMT From: Manish Vachharajani To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 Cc: Subject: misc/148581: fopen fails with EMFILE if there are more than SHORT_MAX fds open X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jul 2010 21:30:07 -0000 >Number: 148581 >Category: misc >Synopsis: fopen fails with EMFILE if there are more than SHORT_MAX fds open >Confidential: no >Severity: non-critical >Priority: medium >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Tue Jul 13 21:30:07 UTC 2010 >Closed-Date: >Last-Modified: >Originator: Manish Vachharajani >Release: 7.3-RELEASE >Organization: LineRate Systems >Environment: FreeBSD wks1.int.lineratesystems.com 7.3-RELEASE FreeBSD 7.3-RELEASE #0: Fri Jul 2 12:56:26 MDT 2010 root@wks1.int.lineratesystems.com:/usr/obj/usr/src/sys/DEV73PMC amd64 >Description: fopen will fail with EMFILE if there are more than SHORT_MAX file descriptors open in the process. It does not matter that these fds were not created by fopen. To make matters worse gcc's libstdc++ uses fopen and friends to implement ofstream so those function mysteriously fail if there are more than 32k fds open in the process. >How-To-Repeat: To reproduce the problem compile and run: #include #include #include #include #define BIGNUM 50000 int main(int argc, char *argv[]) { int i; int fds[BIGNUM]; for(i=0; i < BIGNUM; ++i) { fds[i] = open("/dev/null", O_RDONLY); } FILE *fil = fopen("/dev/null", "r"); if(fil == NULL) { fprintf(stderr, "Could not open /dev/null: %s\n" , strerror(errno)); } for(i=0; i < BIGNUM; ++i) { close(fds[i]); } return 0; } >Fix: The simple fix is to make _file from struct sFILE { ... } FILE; in stdio.h an int instead of a short. However, this will break binary compatibility with anyone compiled with an old libc. A very dirty fix that would not break binary compatibility is, for each architecture, use the open space from the padding and alignment requirements of FILE to stash the other bits of _file and make all users of FILE use an accessor macro that pulls out the right bits. A quick fix to double the threshold at which the problem occurs would be to make _file an unsigned short and use the all 1's value to indicate that this is not a file resource. Not sure if this will work either, though. >Release-Note: >Audit-Trail: >Unformatted: