Date: Sat, 11 Jan 2003 01:33:37 +1030 From: Greg Hurrell <greg@hurrell.cc> To: freebsd-isp@FreeBSD.ORG Subject: Rsync problems (was: SpamAssassin spawned 700 perl processes) Message-ID: <B24D68C0-24AC-11D7-93AF-000393BC25EC@hurrell.cc> In-Reply-To: <000101c2b8a1$a66a3960$0201a8c0@scorpio>
next in thread | previous in thread | raw e-mail | index | archive | help
Given today's scare with spamassassin taking my server down, I decided=20=
it's time to get some backups over the network happening a little more=20=
regularly (one of my databases was corrupted during the crash, but=20
thankfully I was able to repair it).
One of the reasons I only backup over the network once per week is that=20=
I use rsync to do the backup, but it often gives me error messages:
> usr/home/cust42/public_html/service/usage/webalizer.current
> 3903 100% 0.00kB/s 0:00:00
> usr/home/cust42/public_html/service/usage/webalizer.hist
> 187 100% 0.00kB/s 0:00:00
> total: matches=3D402 tag_hits=3D2424 false_alarms=3D0 data=3D14308139
> wrote 22337 bytes read 11811403 bytes 9198.40 bytes/sec
> total size is 123336221 speedup is 10.42
> rsync error: partial transfer (code 23) at=20
> /SourceCache/rsync/rsync-8/rsync/main.c(883)
So I try again and it gets a little further this time:
> usr/home/cust42/public_html/service/usage/webalizer.hist is uptodate
> usr/home/cust42/public_html/spacer.gif is uptodate
> usr/home/cust42/public_html/text1.swf is uptodate
> usr/home/cust42/public_html/yoursay.htm is uptodate
> usr/home/cust39/public_html/resumes/Resume Rosana Garc=EDa.doc
> 41472 100% 19.78MB/s 0:00:00
> usr/home/cust39/public_html/resumes/Resum=E92_antonio.doc
> 87040 100% 243.55kB/s 0:00:00
> total: matches=3D0 tag_hits=3D0 false_alarms=3D0 data=3D128512
> wrote 69 bytes read 443351 bytes 4243.25 bytes/sec
> total size is 123336221 speedup is 278.15
> rsync error: partial transfer (code 23) at=20
> /SourceCache/rsync/rsync-8/rsync/main.c(883)
Perhaps this one has a clue to the problem... This time I see a special=20=
character (=E9) in one of the filenames...
And so on and so forth, until the backup is completely done.
Here is the script I use to do the backup. It uses a file called=20
"syncdirs" to know which dirs to back up. The file just contains a list=20=
of dirs like this:
> /etc
> /usr/local/etc
> /var/db/mysqldumps
> /usr/home/
Anyway, here is the actual script:
> #!/bin/sh -e
>
> # rsync.sh -- remote host backup script
>
> # list of dirs to sync (stored locally):
> syncdirlist=3D"/usr/backups/rsync/host1/syncdirs"
>
> # remote host to backup from:
> remotehost=3D"host1"
>
> # directory for storing backup on local machine:
> localbackupdir=3D"/usr/backups/rsync/host1/store/"
>
> # check to make sure we are root
> if [ $(whoami) !=3D "root" ]; then
> echo "You must run this script as root. [Exiting]"
> exit
> fi
>
> # read dirs to sync from file, ignoring comments and blank lines
> for syncdir in `/bin/cat ${syncdirlist} | /usr/bin/grep -v "#" | \
> /usr/bin/grep -v "^\$"`
> do
> echo Backing up $syncdir from remote host:
> /usr/bin/rsync -avvcRz -e ssh --delete --partial --numeric-ids \
> --modify-window=3D10 --bwlimit=3D10 --progress --exclude =
"Maildir/" \
> --exclude "logs/" root@${remotehost}:${syncdir} \
> ${localbackupdir}
> echo -e "Backup of $syndir complete.\n\n"
>
> done
Any ideas on what could be causing these failures? Generally if I do=20
the sync again, it will get a little further before failing. After a=20
few tries, the entire backup is done; but I would much prefer it if it=20=
would just work the first time!
I am running rsync locally on Mac OS X 10.2.3. The output of "rsync=20
--version" is:
> rsync version 2.5.2 protocol version 26
> Copyright (C) 1996-2002 by Andrew Tridgell and others
> <http://rsync.samba.org/>;
> Capabilities: 64-bit files, socketpairs, hard links, symlinks,=20
> batchfiles, IPv6,
> 32-bit system inums, 64-bit internal inums
The machine being backed-up is a FreeBSD box (FreeBSD 4.7-RELEASE-p2=20
#6: Thu Nov 14 21:42:32 CST 2002) and the output of "ssh -V" on that=20
machine is:
> OpenSSH_3.4p1 FreeBSD-20020702, SSH protocols 1.5/2.0, OpenSSL=20
> 0x0090607f
"rsync --version" on the FreeBSD box yields:
> rsync version 2.5.5 protocol version 26
> Copyright (C) 1996-2002 by Andrew Tridgell and others
> <http://rsync.samba.org/>;
> Capabilities: 64-bit files, socketpairs, hard links, symlinks,=20
> batchfiles,
> IPv6, 32-bit system inums, 64-bit internal inums
Thanks for any insight you can provide!
Cheers :-)
Greg
El Viernes, 10 enero, 2003, a las 10:43 PM, Jeff Palmer escribi=F3:
> Hate to follow up my own post, but I left some information out.
>
> Currently, calling spamassassin with the -L switch allows=20
> spamassassin
> to work, without doing any of the DNS lookups, and thereby not=20
> letting
> razor et al affect it's performance. This of course decreases
> spamassassins ability to accurately tag spam, but for now seems a very
> viable option. At least the mail server will continue running. And =
at
> least some spam will still be tagged.
>
> Jeff
>
>
>> -----Original Message-----
>> From: owner-freebsd-isp@FreeBSD.ORG
> [mailto:owner-freebsd-isp@FreeBSD.ORG]
>> On Behalf Of Jeff Palmer
>> Sent: Friday, January 10, 2003 7:03 AM
>> To: freebsd-isp@FreeBSD.ORG
>> Subject: RE: SpamAssassin spawned 700 perl processes
>>
>> Seems yesterday that razor, razor2 and pyzor were down.
>> The current thinking on the SA mailing lists, is that the razor
>> timeout setting was uneffective.
>>
>> There has been a work-around implemented in 2.50 (due any time now)
>> which should eliminate this problem in the future should the =
situation
>> arise again.
>>
>>
>> Jeff Palmer
>> scorpio@drkshdw.org
>>
>>
>
>
>
>
> To Unsubscribe: send mail to majordomo@FreeBSD.org
> with "unsubscribe freebsd-isp" in the body of the message
>
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-isp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?B24D68C0-24AC-11D7-93AF-000393BC25EC>