Date: Sun, 03 Feb 2008 16:31:34 +0100 From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= <des@des.no> To: Ed Schouten <ed@fxq.nl> Cc: hackers@freebsd.org Subject: Re: sort(1) memory usage Message-ID: <86lk62kqeh.fsf@ds4.des.no> In-Reply-To: <86prvekqs2.fsf@ds4.des.no> ("Dag-Erling =?utf-8?Q?Sm=C3=B8rg?= =?utf-8?Q?rav=22's?= message of "Sun\, 03 Feb 2008 16\:23\:25 %2B0100") References: <8663x6mc2o.fsf@ds4.des.no> <20080203131322.GK1179@hoeg.nl> <20080203151550.GA67020@owl.midgard.homeip.net> <86prvekqs2.fsf@ds4.des.no>
next in thread | previous in thread | raw e-mail | index | archive | help
Dag-Erling Sm=C3=B8rgrav <des@des.no> writes: > Erik Trulsson <ertr1013@student.uu.se> writes: > > Yep, it seems that GNU sort allocates a quite large buffer by default w= hen > > the size of the input is unknown (such as when it reads input from stdi= n.) > > A quick check in the source code indicates that it tries to size this b= uffer > > according to how much memory the system has (and according to any limit= s set > > on how much memory the process is allowed to use.) > Uh, OK. This scaling doesn't seem to work correctly. It seems to > allocate 27 MB on 32-bit machines and 54 MB on 64-bit machines, > regardless of memory size. Looking at the code, it seems to go to extreme lengths to get it absolutely wrong. For instance, if hw.physmem / 8 > hw.usermem, it will pick the former, which means it's pretty much guaranteed to either fail or hose your system (or both). In the immortal words of Blazing Star: YOU FAIL IT Count this as a vote for ditching GNU sort in favor of a BSD-licensed implementation (from {Net,Open}BSD for instance). DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?86lk62kqeh.fsf>