From owner-freebsd-hubs  Fri Jan 16 03:44:02 1998
Return-Path: <owner-freebsd-hubs@FreeBSD.ORG>
Received: (from majordom@localhost)
          by hub.freebsd.org (8.8.8/8.8.8) id DAA08426
          for freebsd-hubs-outgoing; Fri, 16 Jan 1998 03:44:02 -0800 (PST)
          (envelope-from owner-freebsd-hubs@FreeBSD.ORG)
Received: from zed.ludd.luth.se (zed.ludd.luth.se [130.240.16.33])
          by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id DAA08409
          for <freebsd-hubs@FreeBSD.ORG>; Fri, 16 Jan 1998 03:43:56 -0800 (PST)
          (envelope-from ragge@ludd.luth.se)
Received: from father.ludd.luth.se (ragge@father.ludd.luth.se [130.240.16.18])
	by zed.ludd.luth.se (8.8.5/8.8.5) with ESMTP id MAA03970;
	Fri, 16 Jan 1998 12:43:51 +0100
From: Anders Magnusson <ragge@ludd.luth.se>
Received: (ragge@localhost) by father.ludd.luth.se (8.6.11/8.6.11) id MAA03242; Fri, 16 Jan 1998 12:43:50 +0100
Message-Id: <199801161143.MAA03242@father.ludd.luth.se>
Subject: Re: mirror keeps crashes
To: keet@mordor.plig.net (Christiaan Keet)
Date: Fri, 16 Jan 1998 12:43:50 +0100 (MET)
Cc: jesper@skriver.dk, freebsd-hubs@FreeBSD.ORG
In-Reply-To: <199801161047.KAA21531@mordor.plig.net> from Christiaan Keet at "Jan 16, 98 10:47:00 am"
X-Mailer: ELM [version 2.4ME+ PL15 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Sender: owner-freebsd-hubs@FreeBSD.ORG
Precedence: bulk
X-Loop: FreeBSD.org

> Jesper Skriver wrote:
> > I run ftp.dk.freebsd.org, and I have a problem, my mirror keeps crashing,
> > probably due to the wast number of files.
> > 
> > Does anybody have a working mirror.conf I can get a copy of? I'm thinking
> > of splitting the mirror in multiple sessions, this way lowering the number
> > of files in each ...
> 
> *nod*   I had the same problem initially when I started up ftp4.uk.freebsd.org.
> 
> mirror.pl runs out of memory on most sane-spec systems.   I wrote a little 
> 'frontend' for mirror.pl that logs onto the remote system, gets a list of 
> remote directories under the root dir and creates a local config file for 
> mirror to use, treating each remote directory as a seperate package.
> 
There are also a rather good mirror program written in C available;
it fetches files continously instead of doing a complete list to 
compare first. It is located at ftp.luth.se:/pub/unix/mirror/spegla-0.92.tar.gz
and is very easy to configure. Actually; a friend of mine wrote it when
we got into trouble with the perl scripts :-)

-- Ragge