From owner-freebsd-questions Wed May 19 22:44:34 1999 Delivered-To: freebsd-questions@freebsd.org Received: from boco.fee.vutbr.cz (boco.fee.vutbr.cz [147.229.9.11]) by hub.freebsd.org (Postfix) with ESMTP id ADEB814C83 for ; Wed, 19 May 1999 22:44:22 -0700 (PDT) (envelope-from cejkar@dcse.fee.vutbr.cz) Received: from kazi.dcse.fee.vutbr.cz (kazi.dcse.fee.vutbr.cz [147.229.8.12]) by boco.fee.vutbr.cz (8.9.3/8.9.3) with ESMTP id HAA08075; Thu, 20 May 1999 07:44:19 +0200 (CEST) Received: (from cejkar@localhost) by kazi.dcse.fee.vutbr.cz (8.9.2/8.9.2) id HAA16116; Thu, 20 May 1999 07:44:18 +0200 (CEST) Date: Thu, 20 May 1999 07:44:18 +0200 From: Rudolf Cejka To: "Robert J. Adams" Cc: freebsd-questions@freebsd.org Subject: Re: Mirroring ftp.freebsd.org Message-ID: <19990520074418.A15827@kazi.dcse.fee.vutbr.cz> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.5i In-Reply-To: ; from Robert J. Adams on Wed, May 19, 1999 at 04:46:36PM -0400 Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Robert J. Adams wrote (1999/05/19): > I don't think it's the connection to the net.. we have a little over > 100Mb/sec to play with.. and ~70ms pings to cdrom.com. I guess I'll play > with it some more.. > > Check this out. > > root 4925 0.1 65.8 173684 170856 p1 S 7:17PM 8:46.15 mirror > freebsd :ftp.cdrom.com :/pub/FreeBSD Got ls-lR.gz 2287659 107 (perl) > > It's using some serious memory. I see... You are trying to use port "mirror"? FreeBSD's site is very very big so you must use another than default algorithm. In your mirroring configuration, use "algorithm=1" (or "use_files=1") and mirror will take less than 6 MB of memory. And now I have another question: Does anybody know about multithreaded or multiprocess mirroring program? My thoughts are: one master process which scans directories and N subprocesses (with arbitrary upper limit) for files downloading. -- Rudolf Cejka (cejkar@dcse.fee.vutbr.cz; http://www.fee.vutbr.cz/~cejkar) Brno University of Technology, Faculty of El. Engineering and Comp. Science Bozetechova 2, 612 66 Brno, Czech Republic To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message