Date: Sun, 24 Jan 2016 05:31:28 -0200 From: "Marcus Cenzatti" <cenzatti@hush.com> To: "Navdeep Parhar" <nparhar@gmail.com>, "Adrian Chadd" <adrian@freebsd.org> Cc: "Luigi Rizzo" <rizzo@iet.unipi.it>, "FreeBSD Net" <freebsd-net@freebsd.org> Subject: Re: solved: Re: Chelsio T520-SO-CR low performance (netmap tested) for RX Message-ID: <20160124073128.BE9DFA0126@smtp.hushmail.com> In-Reply-To: <20160124060706.GA7567@ox> References: <CA%2BhQ2%2Bg7_haaXLFjMuG00ANsUkFdyGzFQyjT4NYVBmPY-vECBg@mail.gmail.com> <CAJ-Vmo=ScLDOEgebffdnFwTDod6q6UngcxChX6TxwVK_MN%2BbjQ@mail.gmail.com> <20160124035300.3E533A0126@smtp.hushmail.com> <CAJ-Vmon=6ZH8f4_W4rT0F7=FqCKwte66VhQKaNPKKU-7h29uyQ@mail.gmail.com> <20160124043050.A4936A0126@smtp.hushmail.com> <CAJ-VmokuRj%2Bs9_o2uTYnXQuEXmPy4MBrNoiHFN5=kjFDW-eQCw@mail.gmail.com> <20160124060706.GA7567@ox>
next in thread | previous in thread | raw e-mail | index | archive | help
On 1/24/2016 at 4:07 AM, "Navdeep Parhar" <nparhar@gmail.com> wrote:
>
>On Sat, Jan 23, 2016 at 08:38:24PM -0800, Adrian Chadd wrote:
>> ok, that's a discussion to have with navdeep. That /should/ work.
>> Someone may have changed it lately.
>
>Yes this used to work.
>
>>
>> Things should behave very well and predictable once you can
>disable
>> cxl0 but not ncxl0. :-P
>
>The plan is to clean all this up by moving the netmap specific
>parts to
>a driver module of its own. So when you load if_cxgbe you'll get
>only
>the cxl interfaces. If you want netmap access to the ports you'll
>be
>able to kldload cxgbe_netmap (or something like that) which will
>create
>the ncxl ports. These ncxl ports _will_ operate like normal ifnets
>hooked to the kernel stack if netmap isn't enabled on them. And
>the
>cxgbe_netmap driver will attach to PCIe PFs 0-3 so it won't take up
>resources (interrupt vectors, etc.) from PF4, which is what the
>main
>if_cxgbe attaches to. You'll certainly be able to
>up/down/whatever all
>the interfaces independent of each other. All this will get done
>in
>time for FreeBSD 11.
>
>Regards,
>Navdeep
very nice to read that, this will make tests easier too because when I have this simple test scenario:
host1-bridge-host2
- bridge = netmap bridge
i should ping host1 to host2 (kernel path) just to make sure bridge is up, and only after that start pkt-gen on both sides
since we don't have a netmap-aware ping(1), I can not do simple connectivity tests with ncxl, instead i pkt-gen -R 100 just to check if I get something on the other side, works but is not as simple and "pedagogical" as getting icmp response back, and having cxl vs ncxl logical flow paths may also break kernel-path fallback in a running environment, I dont know just guessing, if_bridge cxl0-cxl1 vs netmap-bridge ncxl0-ncxl1, or if I write a daemon which listens on a given ip address, this IP address must be configured on ncxl or cxl so this application can not work transparently in netmap mode while a sister application runs on kernel-path mode.. since I probably can have the same IP address on both ncxl/cxl (I dont know how switches will behave due to different mac address) but freebsd's routing path will probably always rely on the first addressed interface
chelsio# ifconfig cxl0 1.2.3.4/24
chelsio# ifconfig ncxl0 1.2.3.4/24
chelsio# route -n get 1.2.3.1
route to: 1.2.3.1
destination: 1.2.3.0
mask: 255.255.255.0
fib: 0
interface: ncxl0
flags: <UP,DONE,PINNED>
recvpipe sendpipe ssthresh rtt,msec mtu weight expire
0 0 0 0 1500 1 0
i think this would break many things and make usage scenarios more complex
but is it important to have two interface names, mr navdeep, on chelsio hardware?
i mean, very nice if I have ncxl0 working both netmap and kernel modes, or cxl0 if netmap support is not compiled or kernel module not loaded
but why not simply have cxl as usual w/ intel cards? so we can always have the same interface name, chelsio netmap acceleration and internal working details could happen when the interface enters netmap mode or by a ifconfig (ifconfig cxl0 -netmap, ifconfig cxl0 netmap) or even cxgbetool if by some reason it can not be transparently detected when the application puts the port in netmap mode
just wondering why ncxl vs cxl and curious about the internals
thank you everyone again
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20160124073128.BE9DFA0126>