Date: Mon, 25 May 2015 09:36:19 +0200 From: Cs <bimmer@field.hu> To: freebsd-net@freebsd.org Subject: Re: FreeBSD 10.1-REL - network unaccessible after high traffic Message-ID: <5562D0F3.4070408@field.hu>
next in thread | raw e-mail | index | archive | help
Hi all,
I have two FreeBSd 10.1-RELEASE servers connected to each other. They
were connected via cross link, but they are connected to a cisco switch
now (the problem was the same with cross link too). When transferring
huge files (50-500GB backup files) via Gigabit (it is important!) the
network randomly dies. The backup runs every day/week and sometimes the
connection is ok for months sometimes it happens twice a week. When the
network dies I can log in to the server via IPMI and use the console
everything is OK, but can't send anything out on the network. ifconfig
em0 down/up doesn't help nor netif restart. The problem never occured
when I used 100Mbit connection between them, but it was 3com NIC (xl),
gigabit adapter is Intel (em0). When I limit the transfer rate (rsync
bandwith limit or ipfw pipe) the problem is much more rare.
I tried to set these tuning parameters on both servers with different
buffer size but nothing helped:
# cat /etc/sysctl.conf
security.bsd.see_other_uids=0
net.inet.tcp.recvspace=512000
net.route.netisr_maxqlen=2048
kern.ipc.nmbclusters=1310720
net.inet.tcp.sendbuf_max=16777216
net.inet.tcp.recvbuf_max=16777216
kern.ipc.soacceptqueue=32768
# cat /boot/loader.conf
geom_mirror_load="YES" # RAID1 disk driver (see gmirror(8))
ipfw_load="YES"
net.inet.ip.fw.default_to_accept=1
kern.maxusers=4096
accf_data_load="YES"
The duplex settings are identical on both servers.
Server A:
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
ether 00:25:90:24:52:66
inet x.x.x.x netmask 0xfffffe00 broadcast x.x.x.x
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
Server B:
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
ether 00:30:48:dd:fe:3e
inet x.x.x.x netmask 0xfffffe00 broadcast x.x.x.x
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
Today I tried to set mtu to 9000 but in tcpdump I see that during scp it
is still 1500:
x.x.x.x.222 > x.x.x.x.37612: Flags [.], cksum 0xb6ee (incorrect ->
0xda6f), seq 35749, ack 113701596, win 7986, options [nop,nop,TS val
3103966325 ecr 853712893], length 0
09:27:33.912354 IP (tos 0x8, ttl 64, id 1028, offset 0, flags [DF],
proto TCP (6), length 1500)
09:27:33.912358 IP (tos 0x8, ttl 64, id 1029, offset 0, flags [DF],
proto TCP (6), length 1500)
Any ideas? Thanks guys!
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5562D0F3.4070408>