From owner-freebsd-net@FreeBSD.ORG Wed Nov 21 17:43:38 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E5D3A8E0 for ; Wed, 21 Nov 2012 17:43:37 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Received: from mail-vb0-f54.google.com (mail-vb0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9242F8FC16 for ; Wed, 21 Nov 2012 17:43:37 +0000 (UTC) Received: by mail-vb0-f54.google.com with SMTP id l1so9853537vba.13 for ; Wed, 21 Nov 2012 09:43:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=r62+ltcWOEgFCF4x/Mz6GL6YxOTC4IoDqyqNlfUakdI=; b=bkMEYhRPL5yTn6ZdR1x3canVIXxRwEwS0aTUCMQaKfNCEKWkQV1r8du6kFwM6v2lIp 8eTHZ0AnJPtUcZrbjhu+qUtCGWuCj9kBUXuuY+ST9I7n9gSMsRIRWGxggQKVITNp38fo NEgxOXZblTfAg3I51DyoxbxACYmrSlTNZNTc8AU+jKRv1edoX+7FFSMASBg7ina2nsl5 9ebCCNvcqTJGMoyIqYU6y+3eUq3JjgqLxJnjnzPvgj6KuDAx5kZ3BwlmUaqfzirvFPWB rgKuTiWlPKKM6ZdFu5T13DKrIxhkSB7CSxTxg4uwQve9N7PyO0WVjeoGfdTgVpwz53h3 PeWA== MIME-Version: 1.0 Received: by 10.52.90.212 with SMTP id by20mr26658330vdb.118.1353519817052; Wed, 21 Nov 2012 09:43:37 -0800 (PST) Received: by 10.58.218.35 with HTTP; Wed, 21 Nov 2012 09:43:36 -0800 (PST) In-Reply-To: References: <50ACF62C.8000408@mpeters.org> <50ad087d.1892cc0a.2cce.3bf2@mx.google.com> Date: Wed, 21 Nov 2012 09:43:36 -0800 Message-ID: Subject: Re: Low Bandwidth on intercontinental connections From: Mehmet Erol Sanliturk To: Kevin Oberman Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net@freebsd.org, Benjamin Villain , Marc Peters X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Nov 2012 17:43:38 -0000 On Wed, Nov 21, 2012 at 9:20 AM, Kevin Oberman wrote: > On Wed, Nov 21, 2012 at 8:58 AM, Benjamin Villain > wrote: > > I don't think this is about disk or memory leak as transfering files > locally > > seem to work fine. > > > > Can you test transferring files from (and to) your Linux boxes to (and > from) > > the FreeBSD servers to check that it is not a network issue inside your > DCs. > > > > King regards, > > > > -- > > Ben > > > > > > Mehmet Erol Sanliturk writes: > > > >> On Wed, Nov 21, 2012 at 7:41 AM, Marc Peters wrote: > >> > >> > Hi list, > >> > > >> > we are experiencing low throughput on interncontinental connections > with > >> > our FreeBSD Servers. We made several tests and are wondering, why this > >> > would be. The first tests were on an IPSEC VPN between our datacenter > in > >> > DE and Santa Clara, CA. We are connected with two gigabit uplinks in > >> > each DC. Pushing data by scp between our FreeBSD servers takes ages. > >> > Starting with several MB/s it drops to 60-70KB/s: > >> > > >> > >> > >> > >> ..... > >> > >> > >> I do not have any answer to your question , but I want to share one my > >> experiences . > >> > >> I Linux ( KDE ) I was copying a hard disk contents to another drive by > >> using Dolphin . > >> At the beginning it was very fast , but over time its speed reduced to a > >> few kilobytes per second . > >> It listed completion time left as months . > >> > >> I inspected why this is the case . > >> > >> The reason was the following : > >> > >> On each file it is copied , the Dolphin was producing approximately 1 > >> Kilobyte memory leak . > >> After copying more than one million file , all of the memory exhausted > and > >> it started to swap > >> memory to hard disk swap space which reduced copy speed to a few > kilobytes > >> per second . > >> > >> > >> I stopped the Dolphin and copied small directory groups by restarting > the > >> Dolphin . This cured the problem because on each exit , all of the > leaked > >> memory by Dolphin has been disposed ( where "Undo" item of Dolphin menu > >> was > >> disabled means memory is not reserved for undo ). > >> > >> > >> Please study your data transfer software for such a possibility . It may > >> not be problematic in Linux but FreeBSD version may have some trouble > >> points . > >> > >> > >> There is another possibility : Graceful degradation . > >> > >> http://en.wikipedia.org/wiki/Graceful_degradation > >> http://en.wikipedia.org/wiki/Fail_soft > >> > >> A program part may produce graceful degradation over time or processed > >> data > >> : > >> > >> For example , assume a list is searched by sequentially . When list > length > >> grows , search times > >> also grows linearly and produces a degradation although there is no any > >> error in the process . > >> > >> You may study your system with respect to such a process . > >> > >> > >> These are the possibilities which come to my mind . > > If you have not done so, I suggest you use SIFTR to capture data on > what is happening in TCP. It can often tell you a great deal and is > very easy to work with. Just load the kernel module and use sysctls to > control it. I have used it in conjunction with tcpdump and wireshark > to find performance problems. > > Also, for high performance on bulk data transfers over long, fat > pipes, take a look at http://fasterdata.es.net. It is a detailed guide > on moving data developed by the people who have to deal with the huge > volumes of Large Hadron Collider data moving across the Atlantic from > CERN to researchers in the US. (Note that this is not FreeBSD > specific.) > -- > R. Kevin Oberman, Network Engineer > E-mail: kob6558@gmail.com > A very good link . In the above site , please see the following especially : http://fasterdata.es.net/data-transfer-tools/say-no-to-scp/ Say No to scp Why you should avoid scp over a WAN and http://fasterdata.es.net/data-transfer-tools/scp-and-sftp/ scp and sftp Thank you very much . Mehmet Erol Sanliturk