Date: Thu, 19 Jan 2017 07:50:15 -0500 From: David Belle-Isle <dbelleisle@gmail.com> To: Vincenzo Maffione <v.maffione@gmail.com> Cc: "freebsd-net@freebsd.org" <net@freebsd.org> Subject: Re: Netmap TX with no impact to host Message-ID: <CAN56noxYyL_Pc_YBoiXcRi-nV5M9k8TOgMEhfE1YYZd1MV9%2BHQ@mail.gmail.com> In-Reply-To: <CA%2B_eA9hausF0N0y7Ff005thOWYREEau4YvJDyT5nC7kK64JhVQ@mail.gmail.com> References: <CAN56nowNx9%2BtpGLbheBoGV==upV=WWmJ4Z8jbH9kprSxb4-0_g@mail.gmail.com> <CA%2B_eA9hausF0N0y7Ff005thOWYREEau4YvJDyT5nC7kK64JhVQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Oh! Thank you soooo much! I was missing the last part about the checksum offloadings and TSO. I disabled them and everything is working perfectly now! Thank you so much!! On Thu, Jan 19, 2017 at 6:00 AM, Vincenzo Maffione <v.maffione@gmail.com> wrote: > Hi, > Before answering to the question, some important disclaimers. > > If you are using FreeBSD-12-current, please be aware that there are IFLIB > developments in progress that affect em devices. Unfortunately, at the > moment netmap does not work on em/lem/igb devices because of the switch to > IFLIB at this commit https://github.com/freebsd/freebsd/commit/ > 4bf01b136dfa554a57f25559e7a848bf32206c66. Netmap on em works correctly > right before this commit. > > If you are using FreeBSD-11 versions, be aware that netmap in emulated > mode is broken in those versions. I have the fix for that, but it is not in > the released code. If you need it, I can give you a patch. Btw, can anyone > point me at the procedure to push this fix to the relevant release/stable > branches? > If you use em devices with DEV_NETMAP enabled, emulated mode is not used, > so it will work. If you use interfaces without DEV_NETMAP support, emulated > netmap is used and you are affected by the broken code. > > Regarding your question: > You can build a netmap program that does what you want: > (1) transmit packets on em0 at high rates using netmap > (2) let the host stack transmit packets on em0 > (3) let packets received on em0 to find their way to the host stack. > > In netmap it is possible to open just a subset of the TX rings and/or a > subset of the RX rings. > In your case you need to open all the TX hw rings (to acheve (1)) and the > RX host ring (a.k.a. RX sw ring) to achieve (2). > You achieve (3) by simply not opening the RX hw rings. > To implement this you need two calls to nm_open() (similarly to what > bridge does): > > pa = nm_open("netmap:em0/T, NULL, 0, NULL); /* open all TX hw rings */ > pb = nm_open("netmap:em0^/R, NULL, NM_OPEN_NO_MMAP, pa); /* open only RX > host ring */ > > Then you can use "pa" to transmit packets on the hw TX rings (1). > To implement (2), you have two options: > (A) your application moves packets from the host RX ring (pb) to the hw > TX rings (pa). In the end this is the logic of the "bridge" program does, > so you could start from there. > (B) you use "transparent mode" in the host->nic direction: applications > marks with the NS_FORWARD flag the host RX ring slots right before they > released to the kernel, and the kernel with take care of forward them to > the TX ring for you. However, be aware this mode is still experimental, and > not well tested yet. > > Regarding the use of bridge, the correct command line for what you want to > do (forward all packets between host stack and NIC, in both directions) is: > > # bridge netmap:em0 > > However, to let TCP/UDP traffic go correctly, you need to disable tso and > checksum offloadings on em0, since netmap will not program the NIC to > perform those offloadings. This is also valid for your custom program above. > > Cheers, > Vincenzo > > 2017-01-19 2:51 GMT+01:00 David Belle-Isle <dbelleisle@gmail.com>: > >> Hi, >> >> I'm trying to open a netmap descriptor to an interface to send packets. >> However, I don't want to prevent the host to send and receive data >> (transparent). I don't think this should be hard but I can't figure out >> how >> to do this. >> >> I tried to run the bridge example in the FreeBSD distribution but even >> that >> I can't get to run without stopping the host's networking completely. I >> tried running ./bridge em0 em0 which, if I understand correctly should >> open >> the NIC and host rings and pass the traffic through. As soon as I start it >> all the networking stops. >> >> I tried testing in a VMware VM and on bare-metal with em cards and got the >> same results with both. >> >> Can someone help me? >> >> Thanks, >> >> David >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > > > -- > Vincenzo Maffione >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAN56noxYyL_Pc_YBoiXcRi-nV5M9k8TOgMEhfE1YYZd1MV9%2BHQ>