From owner-freebsd-net@FreeBSD.ORG Mon Jun 27 10:41:20 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 03176106566B for ; Mon, 27 Jun 2011 10:41:20 +0000 (UTC) (envelope-from williamejsalt@googlemail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id C28A28FC0A for ; Mon, 27 Jun 2011 10:41:19 +0000 (UTC) Received: by iwr19 with SMTP id 19so5548093iwr.13 for ; Mon, 27 Jun 2011 03:41:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:date:message-id:subject:from:to :content-type; bh=eYMYnzhKqiiZt4bJ7QzGxsC+hzp8puHBIxLCrUqGM4E=; b=hDY1kj1wGeQrQXAe8NHaaDV3jwEpuModvQrwgDbmEHmJJtwNAf4lvwE7sMI5tVortb YLtHJxTpl6yl5juFB9bZVMjes/YomjbAzXkmeTPddX5nBIKtVW4iW9oRf+lkv1gD8wxr GY9Nl4Nt5DkOhWCNaetsUTE6aP+My0fo03mF8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=K2vutA7QKEA6CmTZE6yRJN3ep+fD1LOOJi0whTzBgIpvHpLb1q4EXrrKNlH8P7Z5F1 ptgmEDmn/rR3kPUrjDmBOk29iwkr9RGncfRjrd0BUgOFbA5OMndmRyDPyAi3Eg1Z3TwQ vT/kDPaWUoszF/rP3xH6wix+b+dCi+d64opXw= MIME-Version: 1.0 Received: by 10.42.108.70 with SMTP id g6mr4763951icp.284.1309169672811; Mon, 27 Jun 2011 03:14:32 -0700 (PDT) Received: by 10.231.31.198 with HTTP; Mon, 27 Jun 2011 03:14:32 -0700 (PDT) Date: Mon, 27 Jun 2011 11:14:32 +0100 Message-ID: From: William Salt To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: 1gbit LFN WAN link - odd tcp behavior X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Jun 2011 10:41:20 -0000 Hi All, For the last couple of months i have been pulling my hair out trying to solve this problem. We have a 1Gbps transatlantic link from the UK to the US, which has successfully passed the RFC2544 test. At either end, we have a media converter, and a supermicro server with an intel quad port NIC running pfsense 2 (freebsd 8.1) with the NIC running on the yandex IGB driver. We can pass 1gbps either way with UDP. However we are experiencing very strange issues with tcp connections. With window scaling enabled, and a max socket buffer set to 16MB, we see no difference. Even disabling window scaling and setting the window to 16MB makes no difference. Each TCP connection starts very slowly, and will max out at around 190mbps, taking nearly 2 minutes to climb to this speed before *plateauing*. We have to initiate many (5+) connections to saturate the link with tcp connections with iperf. I have followed guides like this: http://www.psc.edu/networking/projects/tcptune/#FreeBSD With no luck, and have tweaked, disabled, and enabled nearly every relevant sysctl parameter with no luck. Can anyone shed some light on this? I am now doubting the IGB driver, and am looking to swap out the cards as a last ditch effort. However, we have tried different hardware (L3 switches, media convertes + laptops etc), and the symptoms still persist... The only constant is freebsd 8.1 (or 8.2 for production systems). Cheers in advance Will