From owner-freebsd-net@freebsd.org Wed Jan 20 15:08:57 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3B5DDA88B99 for ; Wed, 20 Jan 2016 15:08:57 +0000 (UTC) (envelope-from dudu.meyer@gmail.com) Received: from mail-oi0-x22a.google.com (mail-oi0-x22a.google.com [IPv6:2607:f8b0:4003:c06::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 03CBF1C8A for ; Wed, 20 Jan 2016 15:08:57 +0000 (UTC) (envelope-from dudu.meyer@gmail.com) Received: by mail-oi0-x22a.google.com with SMTP id k206so6903257oia.1 for ; Wed, 20 Jan 2016 07:08:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=cvi1P5aEnbE1JmN/h1kPX/qDTe+M/0jyNaxtGI0IXjA=; b=j+7ceJhYfqKXz/gAmvMJOIcPSjyo/1pvL4RyNEi4oPyTmec/O5Ea92iIUEdTf8I4Pb 94GEv8kgGoOSAoMZNE8uIXm02OdSgqvEqRcBAao3GEVxubrcSFNo93eI4gV8Ka8DpVGw +ssKPFG+89V6hSPpluw30zq+lzo+h/GPCDQmDsttjEuL0wwgCFtsTnY6gA6KvAfItxfw V1oJDja6iFRZDfU5S/ZUpOuAcHO8a+K6noNW7cBW++OJviYQqqo5Hwu6SXIFqv5j/jWy O8aqOP5tBzoaT3t1UVAeRhaHe4Ilwnydrs2KRVqOuAQRO0T41hzCot9wYQriaPcunCHP LwqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=cvi1P5aEnbE1JmN/h1kPX/qDTe+M/0jyNaxtGI0IXjA=; b=hXoCBoB56E0bo6bfjpWsyuWmutS0g85eJBdgzUF670Ngha7uY31bAc2QFsWaUoltfs rzXh481hSYJvYAB/J+tyjrvQiZFAiPGmJTyb7nEMNXFZHF533cRUp/56QHAKGF8XOXWr JgJ+fCLROjSjVZ7ypWqL2sncIVJOM7AWw4D+Xd0mIplp+t/UDBUVjcwUNkFhxUJ7pSCb A+CbRuQuqWZ4Si6OTRqRumOdToDhjXam4DuTN9hb5udxo3XAvU6lwJjB5Jmfva5ZCVMH qRAWzjn0nJcoDvaVYzvxx2/6y1VbZ3VlgPwcht1BVSvev8Nxkit4uHCMOs8R4mnetq4O Vr2A== X-Gm-Message-State: ALoCoQlDJvVND8hW4qkOZCN5GVKuv8moGzBmAwt32+YuhnIpI+zdFAXHNebCxnFd5ZcVk4K3QN/EkI7j1PyJy9/bd4Rlwl0S+w== MIME-Version: 1.0 X-Received: by 10.202.199.82 with SMTP id x79mr27317866oif.139.1453302535757; Wed, 20 Jan 2016 07:08:55 -0800 (PST) Received: by 10.182.88.138 with HTTP; Wed, 20 Jan 2016 07:08:55 -0800 (PST) Date: Wed, 20 Jan 2016 13:08:55 -0200 Message-ID: Subject: netmap design question - accessing netmap:X-n individual queues on FreeBSD From: Eduardo Meyer To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Jan 2016 15:08:57 -0000 Hello all, I have some doubts regarding netmap design direct queue usage. If open netmap:ix0 I am opening all 0-7 queues. Are those queues FIFO among themselves? I mean first packeds will be available on netmap:ix0-0 and if this queue fills up the next packets will be on netmap:ix0-1, and via netmap:ix0 I have all queues from 0 to 7, is this understanding correct? Or something else happens, like, only 1 queue is used if I open netmap:ix0? A later question, if I open netmap:ix0-0 and nothing else is it supposed to work? On my tests I can see that "pkt-gen -f tx -i ix0-0" will work, but "pkt-gen -f rx -i ix0-0" will not. I can transmit but cant receive on a given queue. Why is that? And how could I make something like this, work (code change required?): bridge -i netmap:ix0-0 -i netmap:ix1-0 Or should it already work? Mr Pavel Odiltsov, the author from fastnetmon mentioned he can run on Linux and it works: "kipfw netmap:eth0-n netmap:eth1-n" But this or the above bridge example won't work on FreeBSD. Is that any different? (I did not try on Linux). I could also notice performance differences I would like to understand, if I run: pkt-gen -i ix0 -f tx -s 192.168.0.2 -d 192.168.0.1 I have 14.8Mpps (like rate). If I run: pkt-gen -i ix0-1 -f tx -s 192.168.0.2 -d 192.168.0.1 I can have only 11Mpps. In fact I have 11Mpps if I run on ix0-2, ix0-3, ... ix0-7. I can understand if I run all queues I can have better pps rates than only one single queue, sure, however if I run: pkt-gen -i ix0-0 -f tx -s 192.168.0.2 -d 192.168.0.1 I also have 14.8Mpps. So yeah, ix0 or ix0-0 both give me 14.8Mpps while any other queue give me 11Mpps. How should I understand this? I am asking this because I want to hack (for learning) into the bridge code to make it multithreaded, and I want to have a thread opening each one of the 8 available queues allocated on each one of my 8 CPUs. However both opening ix0-1 and ix1-1 on source code or as a parameter to the bridge application, I can't have it working. I also looked on "pkt-gen -p 8 -c 8" and although debug shows I have 8 threads on 8 CPU with 8 queues I still only see 1 thread working, all other threads go IDLE. I expected to see at least 2 threads working, say, for ix0-0 and ix0-1 to fill line rate. Thank you in advance. -- =========== Eduardo Meyer