Date: Sun, 5 Sep 2010 20:37:57 +0000 From: Alexander Best <arundel@freebsd.org> To: Dag-Erling =?iso-8859-15?Q?Sm=F8rgrav?= <des@des.no> Cc: freebsd-hackers@freebsd.org Subject: Re: expand_number() for fetch'es -B and -S switches Message-ID: <20100905203757.GA11586@freebsd.org> In-Reply-To: <86pqwwjoef.fsf@ds4.des.no> References: <20100831180103.GA92584@freebsd.org> <86fwxt5ng1.fsf@ds4.des.no> <20100901222834.GA66517@freebsd.org> <864oe8mpga.fsf@ds4.des.no> <20100902114655.GA9071@freebsd.org> <8639tsl5q0.fsf@ds4.des.no> <20100902122348.GA38047@freebsd.org> <86pqwwjoef.fsf@ds4.des.no>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu Sep 2 10, Dag-Erling Smørgrav wrote: > Alexander Best <arundel@freebsd.org> writes: > > the current maximum buffer limit of fetch(1) actually is around 1G. i > > think 1M is not enough, because if people are pulling data over fast > > lines they'll have almost constant disk writes. how about 100M then? > > ;) > > Large buffer sizes are *not* better, since fetch(1) will alternate > between filling the buffer and writing it to disk. The buffer should > not be too small, but it should not be too large either; the sweet spot > is somewhere around 128 kB. hmm. so if there should be a maximum buffer size of let's say 1 megabyte, there's no point really in having expand_number() then. so how about forgetting about expand_number() and simply introducing a maximum buffer size of 1 megabyte? everything higher than that will get reduced to 1 megabyte. just like if you want to burn a cd and specify a write speed of 1000 it simply gets reduced to someting like MAX_SPEED. also that would correspond with the semantics of the minimum buffer size. > > > on the other hand why have a maximum limit? if people want to have a > > buffer of 100 gigabyte why shouldn't they? it's their decision > > actually. > > Good point... although if they set it too high, either malloc(3) will > fail - if they're lucky - or fetch(1) will crash when the system runs > out of physical RAM and swap, and they'll have to start over. yeah. users might start using buffer sizes of a few gigabyte which will cause all kinds of problems. don't really wanna go there. > > DES > -- > Dag-Erling Smørgrav - des@des.no cheers. alex -- a13x
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20100905203757.GA11586>