From owner-freebsd-current@FreeBSD.ORG Fri Jan 11 21:16:10 2013 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id ACEACEE2; Fri, 11 Jan 2013 21:16:10 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-da0-f41.google.com (mail-da0-f41.google.com [209.85.210.41]) by mx1.freebsd.org (Postfix) with ESMTP id 6ED6D1E5; Fri, 11 Jan 2013 21:16:10 +0000 (UTC) Received: by mail-da0-f41.google.com with SMTP id e20so945683dak.0 for ; Fri, 11 Jan 2013 13:16:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=rEuVDI2zABzNtyExBrXusCHr/OC8f50IbU3mrXQsgbE=; b=fj60gIK9m4SosiXYgqqmaa+SbpbEJiZwJdcXCxgnSynwyY6TgiS7GHIQoyOtaRFBeO Od38DmzZrEAJhpiFhbl3xI7vl53vuvJ5BbZd3MFcmfMA2BHVizyBZcA9XftL4n1XCXds eZscX87TbWjB21RgpnUDSfLtILtQeJ9pXLRQs26kEGTHk/U10pgyS9wQt3/PlXaNFGv2 VbFIqvVbJ+iPdSgn/aznDWGvCUZUoAfKF2uE2yeK70xz9TcCvfstT6aqNVO1ryP3r9Tu BHmb97xQv+YLDIyWdTWN+PAlpnxnzhp6y69YEj5eekai1X0LLH48xkUU/WwQGSmLVDBv /QaA== MIME-Version: 1.0 Received: by 10.66.76.8 with SMTP id g8mr210472173paw.40.1357938964398; Fri, 11 Jan 2013 13:16:04 -0800 (PST) Received: by 10.67.2.65 with HTTP; Fri, 11 Jan 2013 13:16:04 -0800 (PST) In-Reply-To: <201301111044.32719.jhb@freebsd.org> References: <201301111044.32719.jhb@freebsd.org> Date: Fri, 11 Jan 2013 13:16:04 -0800 Message-ID: Subject: Re: FILE's _file can only hold a short From: Kevin Oberman To: John Baldwin Content-Type: text/plain; charset=UTF-8 Cc: mdf@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jan 2013 21:16:10 -0000 On Fri, Jan 11, 2013 at 7:44 AM, John Baldwin wrote: > On Wednesday, October 31, 2012 02:12:55 PM mdf@freebsd.org wrote: >> I seem to recall a thread earlier on this limitation, but looking at >> actual libc/stdio sources, the 4 year old check for open(2)'s fd being >> less than SHRT_MAX is still there. I thought I saw a patch to change >> this to an int, but it's not in the tree. Was this in a PR or a >> mailing list thread or am I just imagining things? >> >> We've run into this limitation at work, where some processes have >> around 32k open file descriptors and then try to use the libc FILE >> interface. Since we control ABI we can just change this to int, but I >> had been hoping there was a FreeBSD revision we could pull instead of >> having another diff. > > I had been working on a port-exp run. The problem I have run into is > that perl actually reaches inside of FILE directly to clear out _file > so it can control when the fd is actually closed (really gross). I > have extended my stuff so that old Perl binaries should still work, but > wanted to figure out how to prevent future Perl binaries from growing > the same dependency. Also, I haven't had a chance to do a follow-up > to find what else out in ports-land tries to use _file directly from > FILE. John, I posted a problem I am having with a perl script that opens multiple filehandles with output to a pipe (open $fh,"command|";). The script has worked fine for over a decade, but an upgrade of FreeBSD from 7.2 to 8.3 caused the program to fail to transfer the entire output (between 14 and 30KB of data in lines of between 1 and 80 bytes. (Output of show bgp summary on a Juniper.) The sub-process just hangs until my watchdog sends a sigterm after 15 seconds. When I read the filehadle, I get only a truncated output from these processes. Others complete normally with full output. the failures tend to be ones with larger outputs, but not always. I have worked around this, but I would love to know why it broke in 8.3 after working since at least v5 and probably 4.3. Since this is a common technique or running multiple commands in parallel in perl, it would be nice if I could at least understand what is going on. This looks a bit like what this thread is discussing, but I am far from certain. -- R. Kevin Oberman, Network Engineer E-mail: kob6558@gmail.com