Date: Tue, 01 Jan 2008 12:31:23 -0200 From: =?ISO-8859-1?Q?Jo=E3o_Carlos_Mendes_Lu=EDs?= <jonny@jonny.eng.br> To: amd64@freebsd.org Subject: Anybody seeing problems with large TCP streams over rsync? Message-ID: <477A4EBB.8090403@jonny.eng.br>
next in thread | raw e-mail | index | archive | help
Hi folks,
I am migrating a backup server from a server running FreeBSD
6.2/i386 to another server running FreeBSD 7/amd64. It is currently
running 7.0-BETA4, as distributed from the CD image.
Both machines have SATA discs and some mix of gmirror and gstripe of
its partitions.
Since it is a Backup server, I am in the process of transferring
large files from one to another. But I always have trouble when
transferring large files (75GB).
The first trouble I had was in the back-to-back Gigabit Ethernet
connection, to speed-up transfers. Somehow, data got quickly screwed,
even above TCP layer!
One side is an onboard nVidia nfe(4), and the other a generic
Realtek re(4). The troubles went on, even after I disable TSO (on fbsd
7), RXCSUM and TXCSUM on both sides. They went on even after I reduced
the speed to 100Mbps.
The copy process is basically rsync+rsh, but I have also tried
rsync+ssh, scp and rcp. If ssh does not detect an error in its stream
first, rsync does.
Assuming it was a problem with the re(4) interface, being it a low
end adapter, I got back to the main network. The new machine is on
(another) onboard nfe(4), and the old one is on a old xl(4) (3Com 905).
Just to be sure, I've also disabled TSO, RXCSUM and TXCSUM on these
interfaces. Also, the copy process is now a raw rsyncd.
I can now copy the files, but the big ones still get screwed.
(Verification by rsync -c and/or md5)
After some days experimenting, I finally could make the copy by
split(1)ing them first and cat(1)ing ater network copy.
This makes me wonder: Where is (are?) the problem(s)?
It's could be rsync, but not only, since scp got screwed too, when
copying over gigabit. But even when working over FastEthernet, there
were errors. These could be at TCP layer and/or at rsync layer.
I have found some reports with rsync failing for large files, but
they were dated from 2004, and fixed since then.
Can anyone help me?
Jonny
PS: Even without old files, the new server is up, running and making new
daily backups, without problems.
--
João Carlos Mendes Luís - Networking Engineer - jonny@jonny.eng.br
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?477A4EBB.8090403>