From owner-freebsd-net@FreeBSD.ORG Fri Mar 6 06:44:33 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D0D357B9 for ; Fri, 6 Mar 2015 06:44:33 +0000 (UTC) Received: from mail-la0-x22e.google.com (mail-la0-x22e.google.com [IPv6:2a00:1450:4010:c03::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FF048CE for ; Fri, 6 Mar 2015 06:44:33 +0000 (UTC) Received: by labgm9 with SMTP id gm9so16094959lab.8 for ; Thu, 05 Mar 2015 22:44:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=4DPJKIxNy8UiWPFasYF2ATCIPzUKaRHhCt/LnH+It4s=; b=urXvcsIEZUYgl6NyYiBT6F/ZWQecNySRWvZfxvy1asdcAtilflp/wplnihF/fud74U B7pSVkuXMYGXTUL4IpuTdQ2Otlk3D7WfrlB5O71DjNaxFlZxZ8HqbNa9CfJVJykU/2QO UkX37pJ2Rgq4wiZLVYdNF6o3BtpxmpcBK1AWClUAQQViddnleVBCFHwHIX3wJ3FZXIdf nV07PDl5L2lFG3/1zrFGCEDzSfl+Hi9nenvy/ttKFPWQiWe8lx2VlDW7KdKHLDelT3h5 hdGesoKBtSBZaMnDk2c1OOcR6aCdR+3eN+nzptz60yHNLLW0mZimpuSUGzyqZc5G1WtC 8H6w== MIME-Version: 1.0 X-Received: by 10.112.147.200 with SMTP id tm8mr11125526lbb.71.1425624271375; Thu, 05 Mar 2015 22:44:31 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.114.180.4 with HTTP; Thu, 5 Mar 2015 22:44:31 -0800 (PST) In-Reply-To: References: Date: Fri, 6 Mar 2015 07:44:31 +0100 X-Google-Sender-Auth: bDbNyEwR6eM_EK1YZ8fpG0smkGc Message-ID: Subject: Re: Network interrupt and NAPI in FreeBSD? From: Luigi Rizzo To: Wei Hu Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2015 06:44:34 -0000 On Fri, Mar 6, 2015 at 7:22 AM, Wei Hu wrote: > Hi, > > I am working on network driver performance for Hyper-V. I noticed the net= work interrupt rate on FreeBSD is significantly higher than Linux, in the s= ame Hyper-V environment. The iperf test also shows the FreeBSD performance = is not as good as Linux. Linux has NAPI built in which could avoid a lot of= interrupts on a heavy loaded system. I am wondering if FreeBSD also suppor= t NAP in its network stack? > > Also any thought on the network performance in general? i suppose you refer to network performance in a VM, since the factors that impact performance there are different from those on bare metal. The behaviour of course depends a lot on the NIC and backend that you are u= sing so if you could be more specific (e1000 ? virtio ?), that would help. please talk to me (even privately if you prefer) because we have done a lot of work on enhancing performance in a VM which covers qemu, xen, bhyv= e and surely is applicable to HyperV as well. And while the use of netmap/VAL= E gives up to a 5-10x performance boost, there is another factor of 2-5 that can be gained even without netmap. Details at info.iet.unipi.it/~luigi/research.ht= ml On the specific NAPI request: we do not have NAPI but in some NIC drivers the interrupt service routine will spin until it is out of work, which will contribute reducing load. We often rely on interrupt moderation on the NIC to reduce interrupt rates and give work to the ISR in batches. Unfortunately, moderation is often not emulated in hypervisors (e.g. we pushed it into qemu a couple of years ago for the e1000). An alternative mechanism (supported on some of our network drivers, and trivial to add on others) os "device polling", which i introduced some 15 years ago and finds a new meaning in a VM world because it removes device interrupts and polls the NIC on timer interrupts instead. This circumvents the lack of interrupt moderation and gives surprisingly good results. The caveat is that you need a reasonably high HZ value to avoid excessive latency, and the default HZ=3D1000 is sometimes turned town to 100 in a VM. You should probably override that. Depending on the performance tests you run, there might be other things that cause performance differences, such as support of TSO/LRO offloading on the backend (usually with virtio or whatever is your backend), which lets the guest VM ship large 64k segments through the software switch. cheers luigi > > Thanks, > Wei > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" --=20 -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+-------------------------------