From owner-freebsd-isp Fri Jan 10 7: 3:54 2003 Delivered-To: freebsd-isp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0C2E937B401 for ; Fri, 10 Jan 2003 07:03:50 -0800 (PST) Received: from rivendell.unixhosts.net (rivendell.unixhosts.net [150.101.60.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id F1D9043EB2 for ; Fri, 10 Jan 2003 07:03:48 -0800 (PST) (envelope-from greg@hurrell.cc) Received: from hurrell.cc (dsl-202-45-118-62.SA.netspace.net.au [202.45.118.62]) (AUTH: LOGIN greg@hurrell.cc) by rivendell.unixhosts.net with esmtp; Sat, 11 Jan 2003 01:33:46 +1030 Date: Sat, 11 Jan 2003 01:33:37 +1030 Subject: Rsync problems (was: SpamAssassin spawned 700 perl processes) Content-Type: text/plain; charset=ISO-8859-1; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Greg Hurrell To: freebsd-isp@FreeBSD.ORG Content-Transfer-Encoding: quoted-printable In-Reply-To: <000101c2b8a1$a66a3960$0201a8c0@scorpio> Message-Id: X-Mailer: Apple Mail (2.551) Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Given today's scare with spamassassin taking my server down, I decided=20= it's time to get some backups over the network happening a little more=20= regularly (one of my databases was corrupted during the crash, but=20 thankfully I was able to repair it). One of the reasons I only backup over the network once per week is that=20= I use rsync to do the backup, but it often gives me error messages: > usr/home/cust42/public_html/service/usage/webalizer.current > 3903 100% 0.00kB/s 0:00:00 > usr/home/cust42/public_html/service/usage/webalizer.hist > 187 100% 0.00kB/s 0:00:00 > total: matches=3D402 tag_hits=3D2424 false_alarms=3D0 data=3D14308139 > wrote 22337 bytes read 11811403 bytes 9198.40 bytes/sec > total size is 123336221 speedup is 10.42 > rsync error: partial transfer (code 23) at=20 > /SourceCache/rsync/rsync-8/rsync/main.c(883) So I try again and it gets a little further this time: > usr/home/cust42/public_html/service/usage/webalizer.hist is uptodate > usr/home/cust42/public_html/spacer.gif is uptodate > usr/home/cust42/public_html/text1.swf is uptodate > usr/home/cust42/public_html/yoursay.htm is uptodate > usr/home/cust39/public_html/resumes/Resume Rosana Garc=EDa.doc > 41472 100% 19.78MB/s 0:00:00 > usr/home/cust39/public_html/resumes/Resum=E92_antonio.doc > 87040 100% 243.55kB/s 0:00:00 > total: matches=3D0 tag_hits=3D0 false_alarms=3D0 data=3D128512 > wrote 69 bytes read 443351 bytes 4243.25 bytes/sec > total size is 123336221 speedup is 278.15 > rsync error: partial transfer (code 23) at=20 > /SourceCache/rsync/rsync-8/rsync/main.c(883) Perhaps this one has a clue to the problem... This time I see a special=20= character (=E9) in one of the filenames... And so on and so forth, until the backup is completely done. Here is the script I use to do the backup. It uses a file called=20 "syncdirs" to know which dirs to back up. The file just contains a list=20= of dirs like this: > /etc > /usr/local/etc > /var/db/mysqldumps > /usr/home/ Anyway, here is the actual script: > #!/bin/sh -e > > # rsync.sh -- remote host backup script > > # list of dirs to sync (stored locally): > syncdirlist=3D"/usr/backups/rsync/host1/syncdirs" > > # remote host to backup from: > remotehost=3D"host1" > > # directory for storing backup on local machine: > localbackupdir=3D"/usr/backups/rsync/host1/store/" > > # check to make sure we are root > if [ $(whoami) !=3D "root" ]; then > echo "You must run this script as root. [Exiting]" > exit > fi > > # read dirs to sync from file, ignoring comments and blank lines > for syncdir in `/bin/cat ${syncdirlist} | /usr/bin/grep -v "#" | \ > /usr/bin/grep -v "^\$"` > do > echo Backing up $syncdir from remote host: > /usr/bin/rsync -avvcRz -e ssh --delete --partial --numeric-ids \ > --modify-window=3D10 --bwlimit=3D10 --progress --exclude = "Maildir/" \ > --exclude "logs/" root@${remotehost}:${syncdir} \ > ${localbackupdir} > echo -e "Backup of $syndir complete.\n\n" > > done Any ideas on what could be causing these failures? Generally if I do=20 the sync again, it will get a little further before failing. After a=20 few tries, the entire backup is done; but I would much prefer it if it=20= would just work the first time! I am running rsync locally on Mac OS X 10.2.3. The output of "rsync=20 --version" is: > rsync version 2.5.2 protocol version 26 > Copyright (C) 1996-2002 by Andrew Tridgell and others > > Capabilities: 64-bit files, socketpairs, hard links, symlinks,=20 > batchfiles, IPv6, > 32-bit system inums, 64-bit internal inums The machine being backed-up is a FreeBSD box (FreeBSD 4.7-RELEASE-p2=20 #6: Thu Nov 14 21:42:32 CST 2002) and the output of "ssh -V" on that=20 machine is: > OpenSSH_3.4p1 FreeBSD-20020702, SSH protocols 1.5/2.0, OpenSSL=20 > 0x0090607f "rsync --version" on the FreeBSD box yields: > rsync version 2.5.5 protocol version 26 > Copyright (C) 1996-2002 by Andrew Tridgell and others > > Capabilities: 64-bit files, socketpairs, hard links, symlinks,=20 > batchfiles, > IPv6, 32-bit system inums, 64-bit internal inums Thanks for any insight you can provide! Cheers :-) Greg El Viernes, 10 enero, 2003, a las 10:43 PM, Jeff Palmer escribi=F3: > Hate to follow up my own post, but I left some information out. > > Currently, calling spamassassin with the -L switch allows=20 > spamassassin > to work, without doing any of the DNS lookups, and thereby not=20 > letting > razor et al affect it's performance. This of course decreases > spamassassins ability to accurately tag spam, but for now seems a very > viable option. At least the mail server will continue running. And = at > least some spam will still be tagged. > > Jeff > > >> -----Original Message----- >> From: owner-freebsd-isp@FreeBSD.ORG > [mailto:owner-freebsd-isp@FreeBSD.ORG] >> On Behalf Of Jeff Palmer >> Sent: Friday, January 10, 2003 7:03 AM >> To: freebsd-isp@FreeBSD.ORG >> Subject: RE: SpamAssassin spawned 700 perl processes >> >> Seems yesterday that razor, razor2 and pyzor were down. >> The current thinking on the SA mailing lists, is that the razor >> timeout setting was uneffective. >> >> There has been a work-around implemented in 2.50 (due any time now) >> which should eliminate this problem in the future should the = situation >> arise again. >> >> >> Jeff Palmer >> scorpio@drkshdw.org >> >> > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-isp" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message