From owner-freebsd-questions Thu Nov 16 5:56:48 2000 Delivered-To: freebsd-questions@freebsd.org Received: from eeyore.sebster.com (e163161.upc-e.chello.nl [213.93.163.161]) by hub.freebsd.org (Postfix) with SMTP id 0A29E37B4CF for ; Thu, 16 Nov 2000 05:56:44 -0800 (PST) Received: (qmail 38168 invoked by uid 1000); 16 Nov 2000 13:56:42 -0000 Date: Thu, 16 Nov 2000 14:56:41 +0100 From: Sebastiaan van Erk To: Zero Sum Cc: freebsd-questions@freebsd.org Subject: Re: argument list too long Message-ID: <20001116145641.A22842@sebster.com> References: <20001116091607.A97857@sebster.com> <00111621362707.00522@shalimar.net.au> <20001116122313.A69018@sebster.com> <00111700205500.61931@shalimar.net.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <00111700205500.61931@shalimar.net.au>; from count@shalimar.net.au on Fri, Nov 17, 2000 at 12:20:55AM +1100 Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Zero Sum wrote: > No, that was not what I was suggesting. But you are describing a common > problem - not just with tar, so I gave a general solution so that it > covered similar problems. I then went on to cover your problem specificaly. This is NOT a general solution. The problem is that if you want to give a _generic_ number of _ARGUMENTS_ to a program, then you run into the "argument list too long" error. Xargs does nothing except split something which you want to do in _ONE_ process into _SEVERAL_. So this is definately a workaround, and not a solution to the problem. The only real solution is some _generic_ way to be able to read command line arguments from file. > No, you miss my point. A construction like > > # program `find....` > > is going to encounter problems sooner or later because the shell performs > the expansion not the program. Irrespective of how many parameters the > program takes you are going to run out of command line space with any > significant amount of filenames. It is simply a construction to be avoided > in the same way that one avoids buffer overruns. For the same reason. It > breaks. Again you miss my point. First of all, the shell can handle command lines which are quite long, much longer than the 65536 bytes that the exec call can handle (as specified by the KERN_ARGMAX system variable, sysctl kern.argmax, see man execve). Second of all, we have megabytes of memory and still there's a rediculously SMALL limit on the argument list length. Fixed buffer size is just ONE way of programming things - it's really not all that hard to come up with a system which takes some extra memory and increases a buffer's size when it needs it. > Look, I was over the top because i've spent most of my day listening to > FreeBSD vs RedHat arguments, when the question was already settled, just > recriminations and swipes. So I over reacted severly, and I sincerely > apologise. Please just forgive me for this. Forgiven. > That, said, this is what I mean by a fundamental understanding of the Unix > system. You missed (1) that the construction you were using was non-viable > anyway. Any significant number of files would break the system, so it > didn't matter which broke first, (2) that - is the symbol for stzndard in > to a program where a filename is required. I'm sure you saw the > --files-from option in the manpage. 1) I didn't miss that the construction was non-viable. The construction is NOT non-viable, in the same way that reading from disk or from an argument list is NOT fundamentally different. It's non-viable because of the implementation only. And that's too bad, especially with such a small limit. 2) That - is the name most programs use for standard in is well known to me. Furthermore, this is a convention which has to be respected by programs themselves and should be documented in the man page if it is used. For example, from the man page of ipf: DESCRIPTION ipf opens the filenames listed (treating "-" as stdin) and parses the file for a set of rules which are to be added This gives some ambiguity as to how to open the file -. A better solution in my opinion would be to have some reserved system file name for stdin/stdout which can always be opened with open(2), so this functionality can be moved away from programs into the standard libraries. 3) I quote from the man page of tar: --files-from file Get names of files to extract or create from file, one per line. I don't recall wanting to CREATE or EXTRACT files. This options seems to be discussing tar archives and not files to be put in or gotten out of tar archives. Furhtermore, it does not discuss ADDING files to the tar file. So forgive me for not seeing that this was the option that I needed. I think the documentation is rather unclear on this point (also with respect to the possibility of using the -). So thanks for the information, because NOW it is complete. The command works, and that's what I needed. Greetings, Sebastiaan van Erk To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message