From owner-freebsd-net@freebsd.org Sun Jan 24 07:31:31 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6CE91A8F6EB for ; Sun, 24 Jan 2016 07:31:31 +0000 (UTC) (envelope-from cenzatti@hush.com) Received: from smtp10.hushmail.com (smtp10a.hushmail.com [65.39.178.239]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.hushmail.com", Issuer "smtp.hushmail.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2E45F10FE for ; Sun, 24 Jan 2016 07:31:30 +0000 (UTC) (envelope-from cenzatti@hush.com) Received: from smtp10.hushmail.com (smtp10a.hushmail.com [65.39.178.239]) by smtp10.hushmail.com (Postfix) with SMTP id 4CF22C012E for ; Sun, 24 Jan 2016 07:31:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=hush.com; h=date:to:subject:from; s=hush; bh=TQ/tVenuIRBdv6SRnDOLKgSxTiqQK5FiCTkheyi/MBc=; b=xzhwWD2TLn0DaMNCstHwJVKSm1eG91R4Lwol8vhCUItxHj3gebiwaT41Hk+rn2+0bUNDP/QR0o4njw2Mb302Fa8dvAUBmRHTLkSbSQBraQm86EjVI9XDtFVu2A2BkmNugnPBaig0c/Mq1uuDR4EM4VSKRV+cydItcncMjoZTAgUN+l5hGZ16YNE9R2t47qWrxhXvPF0/fmZmoMZ2gZM5jYQfDompbOZBnOkBnPQIAtpgvPCuItLcf0u8EHxhguMc8Pv3LaeDliDEO4sro6pQEBfa7Jki0nxlga6jWAr4aKBweOk+S8TszOTfb8GiXmlilfhZoDFmuwFdUpcUYgLgKg== Received: from smtp.hushmail.com (w5.hushmail.com [65.39.178.80]) by smtp10.hushmail.com (Postfix) with ESMTP; Sun, 24 Jan 2016 07:31:28 +0000 (UTC) Received: by smtp.hushmail.com (Postfix, from userid 99) id BE9DFA0126; Sun, 24 Jan 2016 07:31:28 +0000 (UTC) MIME-Version: 1.0 Date: Sun, 24 Jan 2016 05:31:28 -0200 To: "Navdeep Parhar" , "Adrian Chadd" Cc: "Luigi Rizzo" , "FreeBSD Net" Subject: Re: solved: Re: Chelsio T520-SO-CR low performance (netmap tested) for RX From: "Marcus Cenzatti" In-Reply-To: <20160124060706.GA7567@ox> References: <20160124035300.3E533A0126@smtp.hushmail.com> <20160124043050.A4936A0126@smtp.hushmail.com> <20160124060706.GA7567@ox> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="UTF-8" Message-Id: <20160124073128.BE9DFA0126@smtp.hushmail.com> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Jan 2016 07:31:31 -0000 On 1/24/2016 at 4:07 AM, "Navdeep Parhar" wrote: > >On Sat, Jan 23, 2016 at 08:38:24PM -0800, Adrian Chadd wrote: >> ok, that's a discussion to have with navdeep. That /should/ work. >> Someone may have changed it lately. > >Yes this used to work. > >> >> Things should behave very well and predictable once you can >disable >> cxl0 but not ncxl0. :-P > >The plan is to clean all this up by moving the netmap specific >parts to >a driver module of its own. So when you load if_cxgbe you'll get >only >the cxl interfaces. If you want netmap access to the ports you'll >be >able to kldload cxgbe_netmap (or something like that) which will >create >the ncxl ports. These ncxl ports _will_ operate like normal ifnets >hooked to the kernel stack if netmap isn't enabled on them. And >the >cxgbe_netmap driver will attach to PCIe PFs 0-3 so it won't take up >resources (interrupt vectors, etc.) from PF4, which is what the >main >if_cxgbe attaches to. You'll certainly be able to >up/down/whatever all >the interfaces independent of each other. All this will get done >in >time for FreeBSD 11. > >Regards, >Navdeep very nice to read that, this will make tests easier too because when I have this simple test scenario: host1-bridge-host2 - bridge = netmap bridge i should ping host1 to host2 (kernel path) just to make sure bridge is up, and only after that start pkt-gen on both sides since we don't have a netmap-aware ping(1), I can not do simple connectivity tests with ncxl, instead i pkt-gen -R 100 just to check if I get something on the other side, works but is not as simple and "pedagogical" as getting icmp response back, and having cxl vs ncxl logical flow paths may also break kernel-path fallback in a running environment, I dont know just guessing, if_bridge cxl0-cxl1 vs netmap-bridge ncxl0-ncxl1, or if I write a daemon which listens on a given ip address, this IP address must be configured on ncxl or cxl so this application can not work transparently in netmap mode while a sister application runs on kernel-path mode.. since I probably can have the same IP address on both ncxl/cxl (I dont know how switches will behave due to different mac address) but freebsd's routing path will probably always rely on the first addressed interface chelsio# ifconfig cxl0 1.2.3.4/24 chelsio# ifconfig ncxl0 1.2.3.4/24 chelsio# route -n get 1.2.3.1 route to: 1.2.3.1 destination: 1.2.3.0 mask: 255.255.255.0 fib: 0 interface: ncxl0 flags: recvpipe sendpipe ssthresh rtt,msec mtu weight expire 0 0 0 0 1500 1 0 i think this would break many things and make usage scenarios more complex but is it important to have two interface names, mr navdeep, on chelsio hardware? i mean, very nice if I have ncxl0 working both netmap and kernel modes, or cxl0 if netmap support is not compiled or kernel module not loaded but why not simply have cxl as usual w/ intel cards? so we can always have the same interface name, chelsio netmap acceleration and internal working details could happen when the interface enters netmap mode or by a ifconfig (ifconfig cxl0 -netmap, ifconfig cxl0 netmap) or even cxgbetool if by some reason it can not be transparently detected when the application puts the port in netmap mode just wondering why ncxl vs cxl and curious about the internals thank you everyone again