Date: Thu, 19 Jan 2017 12:00:10 +0100 From: Vincenzo Maffione <v.maffione@gmail.com> To: David Belle-Isle <dbelleisle@gmail.com> Cc: "freebsd-net@freebsd.org" <net@freebsd.org> Subject: Re: Netmap TX with no impact to host Message-ID: <CA%2B_eA9hausF0N0y7Ff005thOWYREEau4YvJDyT5nC7kK64JhVQ@mail.gmail.com> In-Reply-To: <CAN56nowNx9%2BtpGLbheBoGV==upV=WWmJ4Z8jbH9kprSxb4-0_g@mail.gmail.com> References: <CAN56nowNx9%2BtpGLbheBoGV==upV=WWmJ4Z8jbH9kprSxb4-0_g@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, Before answering to the question, some important disclaimers. If you are using FreeBSD-12-current, please be aware that there are IFLIB developments in progress that affect em devices. Unfortunately, at the moment netmap does not work on em/lem/igb devices because of the switch to IFLIB at this commit https://github.com/freebsd/freebsd/commit/4bf01b136dfa554a57f25559e7a848bf32206c66. Netmap on em works correctly right before this commit. If you are using FreeBSD-11 versions, be aware that netmap in emulated mode is broken in those versions. I have the fix for that, but it is not in the released code. If you need it, I can give you a patch. Btw, can anyone point me at the procedure to push this fix to the relevant release/stable branches? If you use em devices with DEV_NETMAP enabled, emulated mode is not used, so it will work. If you use interfaces without DEV_NETMAP support, emulated netmap is used and you are affected by the broken code. Regarding your question: You can build a netmap program that does what you want: (1) transmit packets on em0 at high rates using netmap (2) let the host stack transmit packets on em0 (3) let packets received on em0 to find their way to the host stack. In netmap it is possible to open just a subset of the TX rings and/or a subset of the RX rings. In your case you need to open all the TX hw rings (to acheve (1)) and the RX host ring (a.k.a. RX sw ring) to achieve (2). You achieve (3) by simply not opening the RX hw rings. To implement this you need two calls to nm_open() (similarly to what bridge does): pa = nm_open("netmap:em0/T, NULL, 0, NULL); /* open all TX hw rings */ pb = nm_open("netmap:em0^/R, NULL, NM_OPEN_NO_MMAP, pa); /* open only RX host ring */ Then you can use "pa" to transmit packets on the hw TX rings (1). To implement (2), you have two options: (A) your application moves packets from the host RX ring (pb) to the hw TX rings (pa). In the end this is the logic of the "bridge" program does, so you could start from there. (B) you use "transparent mode" in the host->nic direction: applications marks with the NS_FORWARD flag the host RX ring slots right before they released to the kernel, and the kernel with take care of forward them to the TX ring for you. However, be aware this mode is still experimental, and not well tested yet. Regarding the use of bridge, the correct command line for what you want to do (forward all packets between host stack and NIC, in both directions) is: # bridge netmap:em0 However, to let TCP/UDP traffic go correctly, you need to disable tso and checksum offloadings on em0, since netmap will not program the NIC to perform those offloadings. This is also valid for your custom program above. Cheers, Vincenzo 2017-01-19 2:51 GMT+01:00 David Belle-Isle <dbelleisle@gmail.com>: > Hi, > > I'm trying to open a netmap descriptor to an interface to send packets. > However, I don't want to prevent the host to send and receive data > (transparent). I don't think this should be hard but I can't figure out how > to do this. > > I tried to run the bridge example in the FreeBSD distribution but even that > I can't get to run without stopping the host's networking completely. I > tried running ./bridge em0 em0 which, if I understand correctly should open > the NIC and host rings and pass the traffic through. As soon as I start it > all the networking stops. > > I tried testing in a VMware VM and on bare-metal with em cards and got the > same results with both. > > Can someone help me? > > Thanks, > > David > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Vincenzo Maffione
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2B_eA9hausF0N0y7Ff005thOWYREEau4YvJDyT5nC7kK64JhVQ>