Date: Tue, 01 Jan 2008 16:08:09 +0100 From: Kris Kennaway <kris@FreeBSD.org> To: =?ISO-8859-1?Q?Jo=E3o_Carlos_Mendes_Lu=EDs?= <jonny@jonny.eng.br> Cc: amd64@freebsd.org Subject: Re: Anybody seeing problems with large TCP streams over rsync? Message-ID: <477A5759.6080906@FreeBSD.org> In-Reply-To: <477A4EBB.8090403@jonny.eng.br> References: <477A4EBB.8090403@jonny.eng.br>
next in thread | previous in thread | raw e-mail | index | archive | help
João Carlos Mendes Luís wrote: > Hi folks, > > I am migrating a backup server from a server running FreeBSD 6.2/i386 > to another server running FreeBSD 7/amd64. It is currently running > 7.0-BETA4, as distributed from the CD image. > > Both machines have SATA discs and some mix of gmirror and gstripe of > its partitions. > > Since it is a Backup server, I am in the process of transferring > large files from one to another. But I always have trouble when > transferring large files (75GB). > > The first trouble I had was in the back-to-back Gigabit Ethernet > connection, to speed-up transfers. Somehow, data got quickly screwed, > even above TCP layer! > > One side is an onboard nVidia nfe(4), and the other a generic Realtek > re(4). The troubles went on, even after I disable TSO (on fbsd 7), > RXCSUM and TXCSUM on both sides. They went on even after I reduced the > speed to 100Mbps. > > The copy process is basically rsync+rsh, but I have also tried > rsync+ssh, scp and rcp. If ssh does not detect an error in its stream > first, rsync does. > > Assuming it was a problem with the re(4) interface, being it a low > end adapter, I got back to the main network. The new machine is on > (another) onboard nfe(4), and the old one is on a old xl(4) (3Com 905). > Just to be sure, I've also disabled TSO, RXCSUM and TXCSUM on these > interfaces. Also, the copy process is now a raw rsyncd. > > I can now copy the files, but the big ones still get screwed. > (Verification by rsync -c and/or md5) > > After some days experimenting, I finally could make the copy by > split(1)ing them first and cat(1)ing ater network copy. > > This makes me wonder: Where is (are?) the problem(s)? > > It's could be rsync, but not only, since scp got screwed too, when > copying over gigabit. But even when working over FastEthernet, there > were errors. These could be at TCP layer and/or at rsync layer. > > I have found some reports with rsync failing for large files, but > they were dated from 2004, and fixed since then. Looks like you didn't rule out nfe. Kris
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?477A5759.6080906>