From owner-freebsd-net@FreeBSD.ORG Mon May 25 07:36:31 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D9873ABC for ; Mon, 25 May 2015 07:36:30 +0000 (UTC) (envelope-from bimmer@field.hu) Received: from green.field.hu (green.field.hu [217.20.130.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 615DFD3A for ; Mon, 25 May 2015 07:36:30 +0000 (UTC) (envelope-from bimmer@field.hu) Received: from green.field.hu (localhost [127.0.0.1]) by green.field.hu (Postfix) with ESMTP id 918D5250A88 for ; Mon, 25 May 2015 09:36:27 +0200 (CEST) X-Virus-Scanned: by Amavisd-new at field.hu Received: from green.field.hu ([127.0.0.1]) by green.field.hu (green.field.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GpHRGeRwlaHE for ; Mon, 25 May 2015 09:36:24 +0200 (CEST) Received: from [192.168.52.11] (1F2EC6DD.catv.pool.telekom.hu [31.46.198.221]) by green.field.hu (Postfix) with ESMTPA id 6B027250A9A for ; Mon, 25 May 2015 09:36:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=field.hu; s=mail; t=1432539384; bh=cIK0QsojvpCMXEZJYHbu37VCdxAmam63RxEYtjwcl/s=; h=Date:From:To:Subject; b=G6W7W67Y/PabNow8l1fJTDvtX7MbDlGH558CG0j6Gv5l+gIzaR0D73sVQ6zv0viej Q+FLEZD4SBdF8jl9+vSKe96tNDXAsL8ndppPXoIzoUyr9JcLIpDwKtowWTA4X0yZey PFiYfSsywSE/yAAMv/jUdVGJwN+FOXACvBFbiYAI= Message-ID: <5562D0F3.4070408@field.hu> Date: Mon, 25 May 2015 09:36:19 +0200 From: Cs User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: FreeBSD 10.1-REL - network unaccessible after high traffic Content-Type: text/plain; charset=iso-8859-2; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 May 2015 07:36:31 -0000 Hi all, I have two FreeBSd 10.1-RELEASE servers connected to each other. They were connected via cross link, but they are connected to a cisco switch now (the problem was the same with cross link too). When transferring huge files (50-500GB backup files) via Gigabit (it is important!) the network randomly dies. The backup runs every day/week and sometimes the connection is ok for months sometimes it happens twice a week. When the network dies I can log in to the server via IPMI and use the console everything is OK, but can't send anything out on the network. ifconfig em0 down/up doesn't help nor netif restart. The problem never occured when I used 100Mbit connection between them, but it was 3com NIC (xl), gigabit adapter is Intel (em0). When I limit the transfer rate (rsync bandwith limit or ipfw pipe) the problem is much more rare. I tried to set these tuning parameters on both servers with different buffer size but nothing helped: # cat /etc/sysctl.conf security.bsd.see_other_uids=0 net.inet.tcp.recvspace=512000 net.route.netisr_maxqlen=2048 kern.ipc.nmbclusters=1310720 net.inet.tcp.sendbuf_max=16777216 net.inet.tcp.recvbuf_max=16777216 kern.ipc.soacceptqueue=32768 # cat /boot/loader.conf geom_mirror_load="YES" # RAID1 disk driver (see gmirror(8)) ipfw_load="YES" net.inet.ip.fw.default_to_accept=1 kern.maxusers=4096 accf_data_load="YES" The duplex settings are identical on both servers. Server A: em1: flags=8843 metric 0 mtu 9000 options=4219b ether 00:25:90:24:52:66 inet x.x.x.x netmask 0xfffffe00 broadcast x.x.x.x nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active Server B: em0: flags=8843 metric 0 mtu 9000 options=4219b ether 00:30:48:dd:fe:3e inet x.x.x.x netmask 0xfffffe00 broadcast x.x.x.x nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active Today I tried to set mtu to 9000 but in tcpdump I see that during scp it is still 1500: x.x.x.x.222 > x.x.x.x.37612: Flags [.], cksum 0xb6ee (incorrect -> 0xda6f), seq 35749, ack 113701596, win 7986, options [nop,nop,TS val 3103966325 ecr 853712893], length 0 09:27:33.912354 IP (tos 0x8, ttl 64, id 1028, offset 0, flags [DF], proto TCP (6), length 1500) 09:27:33.912358 IP (tos 0x8, ttl 64, id 1029, offset 0, flags [DF], proto TCP (6), length 1500) Any ideas? Thanks guys!