From owner-freebsd-net@FreeBSD.ORG Sat Mar 7 11:47:36 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B04195C7 for ; Sat, 7 Mar 2015 11:47:36 +0000 (UTC) Received: from mail-la0-x233.google.com (mail-la0-x233.google.com [IPv6:2a00:1450:4010:c03::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2CE441B9 for ; Sat, 7 Mar 2015 11:47:36 +0000 (UTC) Received: by labgm9 with SMTP id gm9so22836290lab.8 for ; Sat, 07 Mar 2015 03:47:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=MdT85z/Z88K5UTfu9GujpgnmAlCQEDj8JwtBwVqheHs=; b=Tis1UYhcNaewg2eToYycK9aPbPqqzUOhhM7d0ZNT9DDWmvZ3eE9d1J82DSV9uINmje pHvUBR7Ey1p23fmMwhBqf+wUeSERsAn4lLr5BzJhMryh5uY5PdD++uv480Wo2GZNPkXS fGfoWa6R6+WEbd6IHu5qbtqAEBxmfEi5z0roJL89v8VMM/nxZEQ0B1m80EF895zcNvk5 t7taxzYUxKQx54s15Ja/nlvff6MLJcerJy7Lpgjip87KUpQVPrG8OQWGGpkTCbWBEiO5 +ZzbgqhEJAALv9iS/rcQCkjdMGvD4ZYmYRs2praEdzbm7JeAzkqrIqtY8+ptb14TKy/2 FESw== MIME-Version: 1.0 X-Received: by 10.112.160.10 with SMTP id xg10mr6093294lbb.83.1425728854154; Sat, 07 Mar 2015 03:47:34 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.114.180.4 with HTTP; Sat, 7 Mar 2015 03:47:34 -0800 (PST) In-Reply-To: References: Date: Sat, 7 Mar 2015 12:47:34 +0100 X-Google-Sender-Auth: DCXVFpTI3A3VtnYblXzcYFGeQ1g Message-ID: Subject: Re: Network interrupt and NAPI in FreeBSD? From: Luigi Rizzo To: Wei Hu Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Mar 2015 11:47:36 -0000 On Sat, Mar 7, 2015 at 8:19 AM, Wei Hu wrote: > Many thanks, Luigi! We are measuring the network performance in VM(Hyper-= V), using netvsc virtual NIC device and its own driver. The Linux VM also u= ses the similar virtual device. The driver on both Linux and FreeBSD have T= SO/LRO support. With just one network queue, we found the throughput is hig= her on Linux (around 2.5 - 3 Gbps) than FreeBSD (just around 1.6 Gbps) with= 10GB NIC. If INVARIANT option is disabled, FreeBSD can achieve 2 - 2.3 Gbp= s. The much higher interrupt rate on FreeBSD was observed. > > Thanks for the all suggestions. Do you think netmap could help in this ca= se? netmap per se probably won't help in this case but it is often useful to stress test the datapath and figure out where bottlenecks are. In fact you could try and run the netmap test program, pkt-gen (in tools/tools/netmap) in emulated mode to see how many pps you can send or receive through the netvsc interface. You need to kldload the netmap module (it is already in by default in recent generic) and then do a pkt-gen -i hn0 -f tx # whatever the name is for the hyperv interface I don't know what to expect but unoptimized qemu/kvm using the e1000 driver was as low as 50-100Kpps, and optimized ones went up to 500Kpps and more even without netmap (i believe FreeBSD's virtio is even faster now). Obviously you should disable INVARIANT, but the numbers you cite (2-3 Gbps) are really low, which suggests me that there might some performance problem in the hypervisor itself. I also do not see any TSO reference in the source code for the netvsc guest driver in FreeBSD, so i am not sure whether it is really supported or not. cheers luigi