From owner-freebsd-net@FreeBSD.ORG Wed Dec 3 15:53:30 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 29B906AB for ; Wed, 3 Dec 2014 15:53:30 +0000 (UTC) Received: from rack.patpro.net (rack.patpro.net [193.30.227.216]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "patpro.net", Issuer "Gandi Standard SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B41A9CF8 for ; Wed, 3 Dec 2014 15:53:29 +0000 (UTC) Received: from patpro.univ-lyon2.fr (patpro.univ-lyon2.fr [159.84.113.250]) by rack.patpro.net (Postfix) with ESMTPSA id 51A1A9DC for ; Wed, 3 Dec 2014 16:46:16 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=patpro.net; s=patpro; t=1417621576; bh=tc+03Jp1F8h028TRSpg4xFiaIxCZEnzcrXPaJHkGGsU=; h=From:Subject:Date:To; b=FEWcEF2QMtWISOYOYP28ErrPrub0nVBD1QR2OxV6MGH49UptSggm6wzrC+49n2+vR RJkSlpJi+mDrUCWsYVGlJcRrVsfwGukRgeTgR3HDv+QO0Z4fWbjJ6JFMomaK+4yVWX GyrgV2K2T2hu0lruw77T8QxY1H0wZWaS1pckJE8g= From: Patrick Proniewski Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Horrendous upload network performance with VLAN (download seems OK) Message-Id: <727AB395-CF83-4D13-A2C3-50969C2969B0@patpro.net> Date: Wed, 3 Dec 2014 16:46:15 +0100 To: freebsd-net@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) X-Mailer: Apple Mail (2.1510) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Dec 2014 15:53:30 -0000 Hello, I'm running FreeBSD 9.3-RELEASE up-to-date, on two HP Proliant G6 server = blades in the same enclosure. One with VLANs in the uplink, the other = without VLANs. They use bxe driver. bxe0: mem 0xfb000000-0xfb7fffff,0xfa800000-0xfaffffff irq 28 at = device 0.0 on pci2 bxe0: PCI BAR0 [10] memory allocated: 0xfb000000-0xfb7fffff = (8388608) -> 0xfffffe00fb000000 bxe0: PCI BAR2 [18] memory allocated: 0xfa800000-0xfaffffff = (8388608) -> 0xfffffe00fa800000 bxe0: Found 10GBase-CX4 media. bxe0: Ethernet address: 00:17:a4:77:04:10 bxe0: MSI-X vectors Requested 5 and Allocated 5 ../.. (and so on up to bxe7) Blade A is configured to access the network through a connection without = VLANs (the link provided by Network team comes with no VLAN tagging). = Its transfer rate is perfect, both up and down. ifconfig_bxe0=3D"inet x.y.z.141/24" defaultrouter=3D"x.y.z.1" # ifconfig bxe0 bxe0: flags=3D8843 = metric 0 mtu 1500 = options=3D507bb ether 00:17:a4:77:04:00 inet x.y.z.141 netmask 0xffffff00 broadcast x.y.z.255 nd6 options=3D29 media: Ethernet autoselect (10Gbase-CX4 ) status: active Blade B is configured to access the network through a link sporting = multiple VLANs, so I've created a network interface that uses one of = these VLANs. Ping is OK, I can ssh to this server, transfer rate to the = server (down) is not fantastic but OK, enough to perform pkg = installation or FreeBSD update. Transfer rate from the server to the = rest of the world is abysmal, often stalling after few 100's KB. ifconfig_bxe0=3D"UP" vlans_bxe0=3D"161" ifconfig_bxe0_161=3D"inet x.y.z.142/24" defaultrouter=3D"x.y.z.1" # ifconfig bxe0 bxe0: flags=3D8843 = metric 0 mtu 1500 = options=3D507bb ether 00:17:a4:77:04:10 nd6 options=3D29 media: Ethernet autoselect (10Gbase-CX4 ) status: active # ifconfig bxe0.161 bxe0.161: flags=3D8843 = metric 0 mtu 1500 options=3D303 ether 00:17:a4:77:04:10 inet x.y.z.142 netmask 0xffffff00 broadcast x.y.z.255 inet6 fe80::217:a4ff:fe77:410%bxe0.161 prefixlen 64 scopeid = 0x10 nd6 options=3D29 media: Ethernet autoselect (10Gbase-CX4 ) status: active vlan: 161 parent interface: bxe0 The same switch is used by those two blade servers running FreeBSD, and = by about 14 other blade servers running VMware ESXi 5.x. ESXi blades use multiple VLANs and work perfectly. A blade running = FreeBSD with no VLAN in its uplink (i.e. not sharing the same uplink as = ESXi blades) has very good network performances. The only blade with = problem is the one running FreeBSD and sharing the uplink of ESXi = blades. scp of a 347MB file from an ESXi blade server to FreeBSD blade (same = switch, same blade chassis, same uplink): # scp esxi11.domain.tld:/tardisks/FILE /dev/null Password: FILE 100% 347MB 38.6MB/s = 00:09 =20 scp of a 347MB file from an FreeBSD blade to ESXi blade (same switch, = same blade chassis, same uplink): # scp FILE esxi11.domain.tld:/dev/null Password: FILE 0% 400KB 172.2KB/s - = stalled - traceroute from FreeBSD blade to ESXi blade: # traceroute esxi11.domain.tld traceroute to esxi11.domain.tld (x.y.z.151), 64 hops max, 52 = byte packets 1 * * * 2 * * * ^C traceroute from ESXi blade to FreeBSD blade: # traceroute freebsdB.domain.tld traceroute to freebsdB.domain.tld (x.y.z.142), 30 hops max, 40 = byte packets 1 freebsdB (x.y.z.142) 0.123 ms 0.090 ms 0.078 ms I'm quite lost here. Any idea that would explain the problem, and help me resolve it? Original thread posted on = Patrick