Date: Wed, 3 Dec 2014 16:46:15 +0100 From: Patrick Proniewski <patpro@patpro.net> To: freebsd-net@freebsd.org Subject: Horrendous upload network performance with VLAN (download seems OK) Message-ID: <727AB395-CF83-4D13-A2C3-50969C2969B0@patpro.net>
next in thread | raw e-mail | index | archive | help
Hello, I'm running FreeBSD 9.3-RELEASE up-to-date, on two HP Proliant G6 server = blades in the same enclosure. One with VLANs in the uplink, the other = without VLANs. They use bxe driver. bxe0: <QLogic NetXtreme II BCM57711E 10GbE (A0) BXE v:1.78.78 > mem 0xfb000000-0xfb7fffff,0xfa800000-0xfaffffff irq 28 at = device 0.0 on pci2 bxe0: PCI BAR0 [10] memory allocated: 0xfb000000-0xfb7fffff = (8388608) -> 0xfffffe00fb000000 bxe0: PCI BAR2 [18] memory allocated: 0xfa800000-0xfaffffff = (8388608) -> 0xfffffe00fa800000 bxe0: Found 10GBase-CX4 media. bxe0: Ethernet address: 00:17:a4:77:04:10 bxe0: MSI-X vectors Requested 5 and Allocated 5 ../.. (and so on up to bxe7) Blade A is configured to access the network through a connection without = VLANs (the link provided by Network team comes with no VLAN tagging). = Its transfer rate is perfect, both up and down. ifconfig_bxe0=3D"inet x.y.z.141/24" defaultrouter=3D"x.y.z.1" # ifconfig bxe0 bxe0: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> = metric 0 mtu 1500 = options=3D507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSU= M,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO> ether 00:17:a4:77:04:00 inet x.y.z.141 netmask 0xffffff00 broadcast x.y.z.255 nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> media: Ethernet autoselect (10Gbase-CX4 <full-duplex>) status: active Blade B is configured to access the network through a link sporting = multiple VLANs, so I've created a network interface that uses one of = these VLANs. Ping is OK, I can ssh to this server, transfer rate to the = server (down) is not fantastic but OK, enough to perform pkg = installation or FreeBSD update. Transfer rate from the server to the = rest of the world is abysmal, often stalling after few 100's KB. ifconfig_bxe0=3D"UP" vlans_bxe0=3D"161" ifconfig_bxe0_161=3D"inet x.y.z.142/24" defaultrouter=3D"x.y.z.1" # ifconfig bxe0 bxe0: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> = metric 0 mtu 1500 = options=3D507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSU= M,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO> ether 00:17:a4:77:04:10 nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> media: Ethernet autoselect (10Gbase-CX4 <full-duplex>) status: active # ifconfig bxe0.161 bxe0.161: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> = metric 0 mtu 1500 options=3D303<RXCSUM,TXCSUM,TSO4,TSO6> ether 00:17:a4:77:04:10 inet x.y.z.142 netmask 0xffffff00 broadcast x.y.z.255 inet6 fe80::217:a4ff:fe77:410%bxe0.161 prefixlen 64 scopeid = 0x10 nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> media: Ethernet autoselect (10Gbase-CX4 <full-duplex>) status: active vlan: 161 parent interface: bxe0 The same switch is used by those two blade servers running FreeBSD, and = by about 14 other blade servers running VMware ESXi 5.x. ESXi blades use multiple VLANs and work perfectly. A blade running = FreeBSD with no VLAN in its uplink (i.e. not sharing the same uplink as = ESXi blades) has very good network performances. The only blade with = problem is the one running FreeBSD and sharing the uplink of ESXi = blades. scp of a 347MB file from an ESXi blade server to FreeBSD blade (same = switch, same blade chassis, same uplink): # scp esxi11.domain.tld:/tardisks/FILE /dev/null Password: FILE 100% 347MB 38.6MB/s = 00:09 =20 scp of a 347MB file from an FreeBSD blade to ESXi blade (same switch, = same blade chassis, same uplink): # scp FILE esxi11.domain.tld:/dev/null Password: FILE 0% 400KB 172.2KB/s - = stalled - traceroute from FreeBSD blade to ESXi blade: # traceroute esxi11.domain.tld traceroute to esxi11.domain.tld (x.y.z.151), 64 hops max, 52 = byte packets 1 * * * 2 * * * ^C traceroute from ESXi blade to FreeBSD blade: # traceroute freebsdB.domain.tld traceroute to freebsdB.domain.tld (x.y.z.142), 30 hops max, 40 = byte packets 1 freebsdB (x.y.z.142) 0.123 ms 0.090 ms 0.078 ms I'm quite lost here. Any idea that would explain the problem, and help me resolve it? Original thread posted on = <https://forums.freebsd.org/threads/horrendous-network-performance-with-vl= an.49211/> Patrick
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?727AB395-CF83-4D13-A2C3-50969C2969B0>