From owner-freebsd-net@FreeBSD.ORG Tue Oct 4 12:01:30 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 857AD106564A for ; Tue, 4 Oct 2011 12:01:30 +0000 (UTC) (envelope-from hoomanfazaeli@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id F0B738FC25 for ; Tue, 4 Oct 2011 12:01:29 +0000 (UTC) Received: by bkbzs8 with SMTP id zs8so685653bkb.13 for ; Tue, 04 Oct 2011 05:01:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject :content-type; bh=rJ4fgeUeJxsS5S7bCEF8AW0CIFc0R+OKrq49jbUeYEo=; b=eDUyGRUhkr6Myzs3DFMh7f1UxTAMgroscroCjdkgwaXWiXFVxfD3pkz1bF4dKnPtCN fqxo8mCjgd/lmYo/znhvj8naHEXjEwcV0LYB2BdQYlqMF0VKXy/cKiutCaNTi3GU5yeX zLV3BHUue1Aop19e3i9nxUVxGB6SVO/FGMHyw= Received: by 10.204.4.71 with SMTP id 7mr666331bkq.106.1317728206106; Tue, 04 Oct 2011 04:36:46 -0700 (PDT) Received: from [127.0.0.1] ([84.241.57.181]) by mx.google.com with ESMTPS id k26sm12587952bks.1.2011.10.04.04.36.43 (version=SSLv3 cipher=OTHER); Tue, 04 Oct 2011 04:36:45 -0700 (PDT) Message-ID: <4E8AEFC5.8050103@gmail.com> Date: Tue, 04 Oct 2011 15:06:37 +0330 From: Hooman Fazaeli User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.23) Gecko/20110920 Thunderbird/3.1.15 MIME-Version: 1.0 To: "freebsd-net@freebsd.org" , Jack Vogel X-Mailman-Approved-At: Tue, 04 Oct 2011 12:34:11 +0000 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Re: em(4) high latency w/o msix X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 12:01:30 -0000 8.2-RELEASE and stable/8 have the same problem. Ping RTT triples when MSIX is disabled. On 10/3/2011 11:50 AM, Jack Vogel wrote: > Can you try the driver in 8.2 and possibly stable/8 to see the behavior there. > > And, just curious, why are you disabling MSIX? > > Jack > > > On Mon, Oct 3, 2011 at 12:51 AM, Hooman Fazaeli > wrote: > > Hi Jack > > The hardware is a PCIe network appliance with 3 port modules. The ports I have > used in the test are 82574L residing on a 4 port module. Anyway, as I noted > in last mail, the stock 7.3-RELEASE driver does not expose this problem on > the same hardware. > > > On 10/2/2011 7:38 PM, Jack Vogel wrote: >> On what hardware? >> >> Jack >> >> >> On Sun, Oct 2, 2011 at 6:42 AM, Hooman Fazaeli > wrote: >> >> >> Latest em(4) driver from HEAD seems to have high latency >> when MSIX is disabled. >> >> With MSIX enabled (hw.em.enable_msix=1): >> >> # ping -c5 192.168.1.83 >> PING 192.168.1.83 (192.168.1.83): 56 data bytes >> 64 bytes from 192.168.1.83 : icmp_seq=0 ttl=64 time=0.055 ms >> 64 bytes from 192.168.1.83 : icmp_seq=1 ttl=64 time=0.076 ms >> 64 bytes from 192.168.1.83 : icmp_seq=2 ttl=64 time=0.066 ms >> 64 bytes from 192.168.1.83 : icmp_seq=3 ttl=64 time=0.051 ms >> 64 bytes from 192.168.1.83 : icmp_seq=4 ttl=64 time=0.063 ms >> >> --- 192.168.1.83 ping statistics --- >> 5 packets transmitted, 5 packets received, 0.0% packet loss >> round-trip min/avg/max/stddev = 0.051/0.062/0.076/0.009 ms >> >> With MSIX disabled: >> >> # ping -c5 192.168.1.83 >> PING 192.168.1.83 (192.168.1.83): 56 data bytes >> 64 bytes from 192.168.1.83 : icmp_seq=0 ttl=64 time=0.180 ms >> 64 bytes from 192.168.1.83 : icmp_seq=1 ttl=64 time=0.164 ms >> 64 bytes from 192.168.1.83 : icmp_seq=2 ttl=64 time=0.169 ms >> 64 bytes from 192.168.1.83 : icmp_seq=3 ttl=64 time=0.172 ms >> 64 bytes from 192.168.1.83 : icmp_seq=4 ttl=64 time=0.167 ms >> >> --- 192.168.1.83 ping statistics --- >> 5 packets transmitted, 5 packets received, 0.0% packet loss >> round-trip min/avg/max/stddev = 0.164/0.170/0.180/0.005 ms >> >> As you see, w/o MSIX, RTT increases by a factor of 3. >> >> I also tested the following drivers: >> - igb(4) from HEAD: OK. >> - Stock 7.3-RELEASE: OK. >> - Stock 7.4-RELEASE: problem exist. >> >> Any ideas? >> >> >> >> >> >