From owner-freebsd-current@FreeBSD.ORG Tue Aug 27 14:53:07 2013 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 594A4DAB; Tue, 27 Aug 2013 14:53:07 +0000 (UTC) (envelope-from bryanv@daemoninthecloset.org) Received: from torment.daemoninthecloset.org (torment.daemoninthecloset.org [94.242.209.234]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 198E32D1C; Tue, 27 Aug 2013 14:53:06 +0000 (UTC) Received: from sage.daemoninthecloset.org (unknown [70.114.209.60]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "sage.daemoninthecloset.org", Issuer "daemoninthecloset.org" (verified OK)) by torment.daemoninthecloset.org (Postfix) with ESMTPS id B474D42C2632; Tue, 27 Aug 2013 16:58:18 +0200 (CEST) X-Virus-Scanned: amavisd-new at daemoninthecloset.org X-Virus-Scanned: amavisd-new at daemoninthecloset.org Date: Tue, 27 Aug 2013 09:52:52 -0500 (CDT) From: Bryan Venteicher To: Harald Schmalzbauer Message-ID: <1428315986.19291.1377615172231.JavaMail.root@daemoninthecloset.org> In-Reply-To: <521CB7AC.90605@omnilan.de> References: <601099152.721.1375661537866.JavaMail.root@daemoninthecloset.org> <5214D37F.5000307@omnilan.de> <1117188271.19176.1377577117350.JavaMail.root@daemoninthecloset.org> <521CB7AC.90605@omnilan.de> Subject: Re: [CFT] VMware vmxnet3 ethernet driver MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [192.168.10.20] X-Mailer: Zimbra 8.0.2_GA_5569 (ZimbraWebClient - GC20 ([unknown])/8.0.2_GA_5569) Thread-Topic: VMware vmxnet3 ethernet driver Thread-Index: m3nkTpbhAoxSRywSkYqJdsIgyubJ+g== Cc: FreeBSD Stable , current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Aug 2013 14:53:07 -0000 ----- Original Message ----- > Bez=C3=BCglich Bryan Venteicher's Nachricht vom 27.08.2013 06:18 (localti= me): >=20 > ... >=20 > >> It seems if_vmx doesn't support jumbo frames. If I set mtu 9000, I get > >> =C2=BBvmx0: cannot populate Rx queue 0=C2=AB, I have no problems using= jumbo > >> frames with vmxnet3. > >> > > This could fail for two reasons - could not allocate an mbuf cluster, > > or the call to bus_dmamap_load_mbuf_sg() failed. For the former, you > > should check vmstat -z. For the later, the behavior of > > bus_dmamap_load_mbuf_sg() > > changed between 9.1 and 9.2, and I know it was broken for awhile. I don= 't > > recall exactly when I fixed it (I think shortly after I made the origin= al > > announcement). Could you retry with the files from HEAD @ [1]? Also, th= ere > > are new sysctl oids (dev.vmx.X.mbuf_load_failed & dev.vmx.X.mgetcl_fail= ed) > > for these errors. > > > > I just compiled the driver on 9.2-RC2 with the sources from HEAD and wa= s > > able to change the MTU to 9000. > > > > [1]- http://svnweb.freebsd.org/base/head/sys/dev/vmware/vmxnet3/ >=20 > Thanks a lot for your ongoing work! > I can confirm that with recent if_vmx.c from head and compiled for > 9.2-RC3, setting mtu to 9000 works as expected :-) >=20 >=20 > >> I took a oldish host (4x2,8GHz Core2[LGA775]) with recent software: ES= Xi > >> 5.1U1 and FreeBSD-9.2-RC2 > >> Two guests are connected to one MTU9000 "VMware Software Switch". > >> > > I've got a few performance things to still look at. What's the sysctl > > dev.vmx.X output for the if_vmx<->if_vmx tests? >=20 > Just repeated if_vmx simple iperf bench, results vary slightly from > standard 10sec run to run, but still noticable high Intr usage: > The intr usage is higher than the other drivers you compared against because if_vmx does the off-level processing in ithreads where as the others do it in a taskqueue. BTW: if_vmx can to LRO as well. I don't think the emulated e1000 can, but I bet the e1000e does. > if_vmx <-> if_vmx > 1.32 GBits/sec, load: 10-45%Sys 40-48%Intr >=20 > if_vmxJumbo <-> if_vmxJumbo > 5.01 GBits/sec, load: 10-45%Sys 40-48%Intr >=20 > Please find attached the different outputs of dev.vmx.X (the mtu9000 run = was > only 3.47GBits/sec in that case, took the numbers anyway) >=20 > wbr, >=20 > -Harry >=20 >=20