Date: Fri, 11 Jan 2013 13:16:04 -0800 From: Kevin Oberman <kob6558@gmail.com> To: John Baldwin <jhb@freebsd.org> Cc: mdf@freebsd.org, freebsd-current@freebsd.org Subject: Re: FILE's _file can only hold a short Message-ID: <CAN6yY1vB88d2nF1uMbFvPDT3wpo7_zJ7ODOU4KZo4FmTfoXW6A@mail.gmail.com> In-Reply-To: <201301111044.32719.jhb@freebsd.org> References: <CAMBSHm_-5JUGX5nGyOLjMxpQjnh=7%2B5NkPnP1-i0OjSEKe7D6Q@mail.gmail.com> <201301111044.32719.jhb@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Jan 11, 2013 at 7:44 AM, John Baldwin <jhb@freebsd.org> wrote: > On Wednesday, October 31, 2012 02:12:55 PM mdf@freebsd.org wrote: >> I seem to recall a thread earlier on this limitation, but looking at >> actual libc/stdio sources, the 4 year old check for open(2)'s fd being >> less than SHRT_MAX is still there. I thought I saw a patch to change >> this to an int, but it's not in the tree. Was this in a PR or a >> mailing list thread or am I just imagining things? >> >> We've run into this limitation at work, where some processes have >> around 32k open file descriptors and then try to use the libc FILE >> interface. Since we control ABI we can just change this to int, but I >> had been hoping there was a FreeBSD revision we could pull instead of >> having another diff. > > I had been working on a port-exp run. The problem I have run into is > that perl actually reaches inside of FILE directly to clear out _file > so it can control when the fd is actually closed (really gross). I > have extended my stuff so that old Perl binaries should still work, but > wanted to figure out how to prevent future Perl binaries from growing > the same dependency. Also, I haven't had a chance to do a follow-up > to find what else out in ports-land tries to use _file directly from > FILE. John, I posted a problem I am having with a perl script that opens multiple filehandles with output to a pipe (open $fh,"command|";). The script has worked fine for over a decade, but an upgrade of FreeBSD from 7.2 to 8.3 caused the program to fail to transfer the entire output (between 14 and 30KB of data in lines of between 1 and 80 bytes. (Output of show bgp summary on a Juniper.) The sub-process just hangs until my watchdog sends a sigterm after 15 seconds. When I read the filehadle, I get only a truncated output from these processes. Others complete normally with full output. the failures tend to be ones with larger outputs, but not always. I have worked around this, but I would love to know why it broke in 8.3 after working since at least v5 and probably 4.3. Since this is a common technique or running multiple commands in parallel in perl, it would be nice if I could at least understand what is going on. This looks a bit like what this thread is discussing, but I am far from certain. -- R. Kevin Oberman, Network Engineer E-mail: kob6558@gmail.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAN6yY1vB88d2nF1uMbFvPDT3wpo7_zJ7ODOU4KZo4FmTfoXW6A>