Date: Thu, 16 Nov 2000 14:56:41 +0100 From: Sebastiaan van Erk <sebster@sebster.com> To: Zero Sum <count@shalimar.net.au> Cc: freebsd-questions@freebsd.org Subject: Re: argument list too long Message-ID: <20001116145641.A22842@sebster.com> In-Reply-To: <00111700205500.61931@shalimar.net.au>; from count@shalimar.net.au on Fri, Nov 17, 2000 at 12:20:55AM %2B1100 References: <20001116091607.A97857@sebster.com> <00111621362707.00522@shalimar.net.au> <20001116122313.A69018@sebster.com> <00111700205500.61931@shalimar.net.au>
next in thread | previous in thread | raw e-mail | index | archive | help
Zero Sum wrote:
> No, that was not what I was suggesting. But you are describing a common
> problem - not just with tar, so I gave a general solution so that it
> covered similar problems. I then went on to cover your problem specificaly.
This is NOT a general solution. The problem is that if you want to give
a _generic_ number of _ARGUMENTS_ to a program, then you run into the
"argument list too long" error. Xargs does nothing except split something
which you want to do in _ONE_ process into _SEVERAL_. So this is definately
a workaround, and not a solution to the problem. The only real solution
is some _generic_ way to be able to read command line arguments from file.
> No, you miss my point. A construction like
>
> # program `find....`
>
> is going to encounter problems sooner or later because the shell performs
> the expansion not the program. Irrespective of how many parameters the
> program takes you are going to run out of command line space with any
> significant amount of filenames. It is simply a construction to be avoided
> in the same way that one avoids buffer overruns. For the same reason. It
> breaks.
Again you miss my point. First of all, the shell can handle command lines
which are quite long, much longer than the 65536 bytes that the exec call
can handle (as specified by the KERN_ARGMAX system variable, sysctl
kern.argmax, see man execve). Second of all, we have megabytes of memory
and still there's a rediculously SMALL limit on the argument list length.
Fixed buffer size is just ONE way of programming things - it's really not
all that hard to come up with a system which takes some extra memory and
increases a buffer's size when it needs it.
> Look, I was over the top because i've spent most of my day listening to
> FreeBSD vs RedHat arguments, when the question was already settled, just
> recriminations and swipes. So I over reacted severly, and I sincerely
> apologise. Please just forgive me for this.
Forgiven.
> That, said, this is what I mean by a fundamental understanding of the Unix
> system. You missed (1) that the construction you were using was non-viable
> anyway. Any significant number of files would break the system, so it
> didn't matter which broke first, (2) that - is the symbol for stzndard in
> to a program where a filename is required. I'm sure you saw the
> --files-from option in the manpage.
1) I didn't miss that the construction was non-viable. The construction is
NOT non-viable, in the same way that reading from disk or from an argument
list is NOT fundamentally different. It's non-viable because of the
implementation only. And that's too bad, especially with such a small
limit.
2) That - is the name most programs use for standard in is well known to
me. Furthermore, this is a convention which has to be respected by programs
themselves and should be documented in the man page if it is used. For
example, from the man page of ipf:
DESCRIPTION
ipf opens the filenames listed (treating "-" as stdin) and
parses the file for a set of rules which are to be added
This gives some ambiguity as to how to open the file -. A better solution
in my opinion would be to have some reserved system file name for stdin/stdout
which can always be opened with open(2), so this functionality can be
moved away from programs into the standard libraries.
3) I quote from the man page of tar:
--files-from file Get names of files to extract or create from
file, one per line.
I don't recall wanting to CREATE or EXTRACT files. This options seems
to be discussing tar archives and not files to be put in or gotten out of
tar archives. Furhtermore, it does not discuss ADDING files to the tar
file. So forgive me for not seeing that this was the option that I needed.
I think the documentation is rather unclear on this point (also with
respect to the possibility of using the -).
So thanks for the information, because NOW it is complete. The command
works, and that's what I needed.
Greetings,
Sebastiaan van Erk
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20001116145641.A22842>