From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 5 20:37:57 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id CFF181065695; Sun, 5 Sep 2010 20:37:57 +0000 (UTC) Date: Sun, 5 Sep 2010 20:37:57 +0000 From: Alexander Best To: Dag-Erling =?iso-8859-15?Q?Sm=F8rgrav?= Message-ID: <20100905203757.GA11586@freebsd.org> References: <20100831180103.GA92584@freebsd.org> <86fwxt5ng1.fsf@ds4.des.no> <20100901222834.GA66517@freebsd.org> <864oe8mpga.fsf@ds4.des.no> <20100902114655.GA9071@freebsd.org> <8639tsl5q0.fsf@ds4.des.no> <20100902122348.GA38047@freebsd.org> <86pqwwjoef.fsf@ds4.des.no> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <86pqwwjoef.fsf@ds4.des.no> Cc: freebsd-hackers@freebsd.org Subject: Re: expand_number() for fetch'es -B and -S switches X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Sep 2010 20:37:57 -0000 On Thu Sep 2 10, Dag-Erling Smørgrav wrote: > Alexander Best writes: > > the current maximum buffer limit of fetch(1) actually is around 1G. i > > think 1M is not enough, because if people are pulling data over fast > > lines they'll have almost constant disk writes. how about 100M then? > > ;) > > Large buffer sizes are *not* better, since fetch(1) will alternate > between filling the buffer and writing it to disk. The buffer should > not be too small, but it should not be too large either; the sweet spot > is somewhere around 128 kB. hmm. so if there should be a maximum buffer size of let's say 1 megabyte, there's no point really in having expand_number() then. so how about forgetting about expand_number() and simply introducing a maximum buffer size of 1 megabyte? everything higher than that will get reduced to 1 megabyte. just like if you want to burn a cd and specify a write speed of 1000 it simply gets reduced to someting like MAX_SPEED. also that would correspond with the semantics of the minimum buffer size. > > > on the other hand why have a maximum limit? if people want to have a > > buffer of 100 gigabyte why shouldn't they? it's their decision > > actually. > > Good point... although if they set it too high, either malloc(3) will > fail - if they're lucky - or fetch(1) will crash when the system runs > out of physical RAM and swap, and they'll have to start over. yeah. users might start using buffer sizes of a few gigabyte which will cause all kinds of problems. don't really wanna go there. > > DES > -- > Dag-Erling Smørgrav - des@des.no cheers. alex -- a13x