Date: Sun, 03 Feb 2008 16:31:34 +0100 From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= <des@des.no> To: Ed Schouten <ed@fxq.nl> Cc: hackers@freebsd.org Subject: Re: sort(1) memory usage Message-ID: <86lk62kqeh.fsf@ds4.des.no> In-Reply-To: <86prvekqs2.fsf@ds4.des.no> ("Dag-Erling =?utf-8?Q?Sm=C3=B8rg?= =?utf-8?Q?rav=22's?= message of "Sun\, 03 Feb 2008 16\:23\:25 %2B0100") References: <8663x6mc2o.fsf@ds4.des.no> <20080203131322.GK1179@hoeg.nl> <20080203151550.GA67020@owl.midgard.homeip.net> <86prvekqs2.fsf@ds4.des.no>
next in thread | previous in thread | raw e-mail | index | archive | help
Dag-Erling Smørgrav <des@des.no> writes:
> Erik Trulsson <ertr1013@student.uu.se> writes:
> > Yep, it seems that GNU sort allocates a quite large buffer by default when
> > the size of the input is unknown (such as when it reads input from stdin.)
> > A quick check in the source code indicates that it tries to size this buffer
> > according to how much memory the system has (and according to any limits set
> > on how much memory the process is allowed to use.)
> Uh, OK. This scaling doesn't seem to work correctly. It seems to
> allocate 27 MB on 32-bit machines and 54 MB on 64-bit machines,
> regardless of memory size.
Looking at the code, it seems to go to extreme lengths to get it
absolutely wrong. For instance, if hw.physmem / 8 > hw.usermem, it will
pick the former, which means it's pretty much guaranteed to either fail
or hose your system (or both).
In the immortal words of Blazing Star: YOU FAIL IT
Count this as a vote for ditching GNU sort in favor of a BSD-licensed
implementation (from {Net,Open}BSD for instance).
DES
--
Dag-Erling Smørgrav - des@des.no
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?86lk62kqeh.fsf>