From owner-freebsd-current@FreeBSD.ORG Sat Sep 22 03:49:01 2007 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0EE8016A46C for ; Sat, 22 Sep 2007 03:49:01 +0000 (UTC) (envelope-from jasone@freebsd.org) Received: from canonware.com (canonware.com [64.183.146.166]) by mx1.freebsd.org (Postfix) with ESMTP id 0341E13C459 for ; Sat, 22 Sep 2007 03:49:00 +0000 (UTC) (envelope-from jasone@freebsd.org) Received: from [192.168.168.201] (canonware.com [64.183.146.166]) by canonware.com (Postfix) with ESMTP id 612731298C8; Fri, 21 Sep 2007 20:55:53 -0700 (PDT) Message-ID: <46F490A8.2010004@freebsd.org> Date: Fri, 21 Sep 2007 20:48:56 -0700 From: Jason Evans User-Agent: Thunderbird 1.5.0.12 (X11/20070718) MIME-Version: 1.0 To: Alexandre Biancalana References: <8e10486b0709211457v7771bd66kf3df5ea45ab0d325@mail.gmail.com> <46F45CE2.9080204@freebsd.org> <8e10486b0709211855pbc3b983tf3452890f8d9b9dc@mail.gmail.com> In-Reply-To: <8e10486b0709211855pbc3b983tf3452890f8d9b9dc@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: current@freebsd.org Subject: Re: process size X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Sep 2007 03:49:01 -0000 Alexandre Biancalana wrote: >> You're running your system as i386 rather than amd64, right? It looks >> like rsync is exhausting its address space while trying to reallocate >> the growing (and apparently very large) file list. There's nothing >> surprising here to me. If you use amd64 rather than i386 you won't have >> this problem, though you will still see poor performance due to >> swapping. The most prudent solution is probably to use multiple rsync >> calls to copy portions of your data at a time. > > I agree that this could be the problem... but I´m not running i386. > > # uname -a > FreeBSD 7.0-CURRENT FreeBSD 7.0-CURRENT #2: Fri Sep 21 18:06:33 BRT > 2007 root@:/usr/obj/usr/src/sys/BACKUP amd64 Hmm, okay, so much for that theory. I blame rsync. If you look it its util.c, you will see that it can't allocate over 2^31 bytes per allocation (MALLOC_MAX). Additionally, the code uses (unsigned int) in many places that would have to be changed to (unsigned long) or (size_t) in order for things to work correctly with a larger MALLOC_MAX. Basically, rsync loses. Jason