From owner-freebsd-net@FreeBSD.ORG Wed Apr 8 10:30:57 2015 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DE6ED9ED for ; Wed, 8 Apr 2015 10:30:56 +0000 (UTC) Received: from mail-pa0-x235.google.com (mail-pa0-x235.google.com [IPv6:2607:f8b0:400e:c03::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ABCC4F77 for ; Wed, 8 Apr 2015 10:30:56 +0000 (UTC) Received: by pacyx8 with SMTP id yx8so110276721pac.1 for ; Wed, 08 Apr 2015 03:30:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=uAWg0WlXH0wzgV+TzMNKJxAHcdfRMdWC/MS1uC+jFhk=; b=Dksc36odVbemmv/cqwzQK3oXYDDayeDnk1EWvADo9Y3PFj0m+hanxfieVBLnnfKxS8 LOf7ox4EnazBIUBf1fU8NzUZnqHlcPj6s1iiUfvVMoIwgcX1RNtuHwnyi1WvLHa7OOrw UbIGsIwVumjBVY/8EiZxyGAte0dQUbKciViCg399lH2I5obIbkEaaZ41CiUPpRNPkacw NqBQlrXGgM5OPkiUnHjokOroMOTe5QX9VFAO02vu5I191WdGR7qBl9CGKUuWIuwQMMEB vAInUFcX2N1WO/b7ewRotNWgzF2CcMLpFIkvGiazvsKUAaStQKHbzZo1GV7LhtVwte26 da4w== X-Received: by 10.70.89.237 with SMTP id br13mr45475451pdb.135.1428489056331; Wed, 08 Apr 2015 03:30:56 -0700 (PDT) Received: from [172.29.229.7] ([116.197.184.12]) by mx.google.com with ESMTPSA id t8sm10593875pbs.59.2015.04.08.03.30.54 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 08 Apr 2015 03:30:55 -0700 (PDT) Message-ID: <5525035B.9050200@gmail.com> Date: Wed, 08 Apr 2015 16:00:51 +0530 From: Ashish Gupta User-Agent: Mozilla/5.0 (X11; Linux i686; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: net@freebsd.org Subject: Netmap usage in Freebsd10 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Apr 2015 10:30:57 -0000 Hi Netmap developers, In very simple application, I am using two freebsd10 vm's spawned using Qemu. About my environment, Underlying host os is linux, freebsd10 vm's are guest vms. These two bsd10 vm's are connected using em interface (which again Qemu emulated). |====== 10.10.10.1 (em) =======| | vm1 |<---------------------------------------------->| vm2 | |====== 10.10.10.2(em) ====== | If i start rapid ping between these vm, i see idle cpu remaining in system (bsd10 vm) is just 10-20%. checking further looking all cpu is going in em driver call (interrupt enable/disable) etc. ======= last pid: 8709; load averages: 1.89, 1.57, 1.76 up 0+00:38:44 05:59:46 30 processes: 3 running, 27 sleeping CPU: 2.0% user, 0.0% nice, 72.9% system, 10.2% interrupt, 14.9% idle <<<<<<<<<<<<<<<<<<<<<<<<< Mem: 138M Active, 642M Inact, 91M Wired, 89M Buf, 2624M Free Swap: 1024M Total, 1024M Free PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 8089 root 1 25 0 18880K 3788K RUN 0:15 6.98% sshd 8628 root 1 29 0 14456K 1832K RUN 0:11 6.40% ping ========== I was thinking if i can make use of netmap here to achieve better performance, which will also leave some cpu for other application to run in vm. I enabled netmap related code changes in if_lem.c of bsd10 but looks like these changes also expects change in Qemu side also. ======= /* * Uncomment the following extensions for better performance in a VM, * especially if you have support in the hypervisor. * See http://info.iet.unipi.it/~luigi/netmap/ */ #define BATCH_DISPATCH #define NIC_SEND_COMBINING #define NIC_PARAVIRT /* enable virtio-like synchronization */ ----------- #ifdef NIC_PARAVIRT device_printf(dev, "driver supports paravirt, subdev 0x%x\n", adapter->hw.subsystem_device_id); if (adapter->hw.subsystem_device_id == E1000_PARA_SUBDEV) { <<<<<<<<<< =========== Qemu by default emulated em device with subsystem_device_id as 0x1100. It mean by enabling netmap change alone in bsd10 will not help. we also need changes from Qemu. I checked latest Qemu code but i dont see they emulating em with above subsystem_device_id. Can some one tell me about how to enable Qemu side change, if there is any patch available please provide me that to try. Do we also need application level change (to use /dev/netmap) to gain mentioned performance gain or we can get little bit of gain by just enabling kernel level changes. --Regards, Ashish Gupta