From owner-freebsd-net@freebsd.org Thu Jan 19 13:51:28 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EEA7ECB7D04 for ; Thu, 19 Jan 2017 13:51:28 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id CBAA2129C for ; Thu, 19 Jan 2017 13:51:28 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id CAEB9CB7D02; Thu, 19 Jan 2017 13:51:28 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CA820CB7D01 for ; Thu, 19 Jan 2017 13:51:28 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mail-ot0-x235.google.com (mail-ot0-x235.google.com [IPv6:2607:f8b0:4003:c0f::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E3E7129B for ; Thu, 19 Jan 2017 13:51:28 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mail-ot0-x235.google.com with SMTP id 65so32180767otq.2 for ; Thu, 19 Jan 2017 05:51:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=p+mmWt8KKLEBzrgNlRaHmpmQk0ikpVnGM7DmzTlNrNg=; b=J2arHMFLTwl63U8E+tlIi6Cz+p1oVlvsij+W5biuq/SI/YMDYYqwNmFBA0vn91IsL4 qtFcYTDRBIWRaOtd2hR+DjmqRFG70vmDL3bMfuS3ryGmfoRkGQjgvolQ7CNbXkdgggzO 5yB32v1EyUF2BTMuucwYG/aJgE9h6H8iF4OWNqUQ3L+q1wIZ+pIelNIO1ACo+PNgEGuv Sc70P/tKCzdQx11z3Rk6rxhxdd2/kH57wqH/hrT78/bIEy0raUDIsHjkKF3qkI2h2J3X gu/UIcPbr4mit+A60PNAkEpkkeYq3gd4Bgh9JDg55A6PQz9tyN5lcR0Hy1WCmGMQSD7z HDWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=p+mmWt8KKLEBzrgNlRaHmpmQk0ikpVnGM7DmzTlNrNg=; b=UdipR2skeANw/WBMsYFj9KTdaoKBrXk3E5nk8INafVi/ob0+by++6xjYOJCYZNICIH PypS0NB6TkFHSfj3p+FgmMSgYn2uxd0sDUjpjjMhjmRT1yjaBAIf5hRPoBQEGoiEiTDz lC0HEe9yAHlumYj6N9mtEO0CRmULlQRhXi+CYTbf4vB9bxvm7v0VL/HTS6n0VwJK7JDO jQ8gb/3ZOHAmi59HZAGNV7wAgSiZjE6NIbsd2MxcXKG7TCemXB8bql49EE+RBC+Serv/ /kvFs+s4DIQuVfVuQ+GIPP1qEL+49kRI8DUGyBfevi7B3wHXAuv2jSfEoRpDTcqcmYs9 6PGw== X-Gm-Message-State: AIkVDXIu2UN/ZR0AW2tYcEgD2wZuTKBMTJ/VZKDHjAkH5+pVzlAZP7P8PNeRo3VyVSktXHFonjZHmEjkhDH7/A== X-Received: by 10.157.24.92 with SMTP id t28mr4704563ott.238.1484833887910; Thu, 19 Jan 2017 05:51:27 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.6.166 with HTTP; Thu, 19 Jan 2017 05:51:27 -0800 (PST) In-Reply-To: References: From: Vincenzo Maffione Date: Thu, 19 Jan 2017 14:51:27 +0100 Message-ID: Subject: Re: Netmap TX with no impact to host To: David Belle-Isle Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2017 13:51:29 -0000 You're welcome! I know it's a bit difficult to notice, but this is written in the very last sentence of the netmap(4) man page. Cheers, Vincenzo 2017-01-19 13:50 GMT+01:00 David Belle-Isle : > Oh! Thank you soooo much! I was missing the last part about the checksum > offloadings and TSO. I disabled them and everything is working perfectly > now! Thank you so much!! > > On Thu, Jan 19, 2017 at 6:00 AM, Vincenzo Maffione > wrote: > >> Hi, >> Before answering to the question, some important disclaimers. >> >> If you are using FreeBSD-12-current, please be aware that there are IFLIB >> developments in progress that affect em devices. Unfortunately, at the >> moment netmap does not work on em/lem/igb devices because of the switch to >> IFLIB at this commit https://github.com/freebsd/fre >> ebsd/commit/4bf01b136dfa554a57f25559e7a848bf32206c66. Netmap on em works >> correctly right before this commit. >> >> If you are using FreeBSD-11 versions, be aware that netmap in emulated >> mode is broken in those versions. I have the fix for that, but it is not in >> the released code. If you need it, I can give you a patch. Btw, can anyone >> point me at the procedure to push this fix to the relevant release/stable >> branches? >> If you use em devices with DEV_NETMAP enabled, emulated mode is not used, >> so it will work. If you use interfaces without DEV_NETMAP support, emulated >> netmap is used and you are affected by the broken code. >> >> Regarding your question: >> You can build a netmap program that does what you want: >> (1) transmit packets on em0 at high rates using netmap >> (2) let the host stack transmit packets on em0 >> (3) let packets received on em0 to find their way to the host stack. >> >> In netmap it is possible to open just a subset of the TX rings and/or a >> subset of the RX rings. >> In your case you need to open all the TX hw rings (to acheve (1)) and the >> RX host ring (a.k.a. RX sw ring) to achieve (2). >> You achieve (3) by simply not opening the RX hw rings. >> To implement this you need two calls to nm_open() (similarly to what >> bridge does): >> >> pa = nm_open("netmap:em0/T, NULL, 0, NULL); /* open all TX hw rings */ >> pb = nm_open("netmap:em0^/R, NULL, NM_OPEN_NO_MMAP, pa); /* open only RX >> host ring */ >> >> Then you can use "pa" to transmit packets on the hw TX rings (1). >> To implement (2), you have two options: >> (A) your application moves packets from the host RX ring (pb) to the hw >> TX rings (pa). In the end this is the logic of the "bridge" program does, >> so you could start from there. >> (B) you use "transparent mode" in the host->nic direction: applications >> marks with the NS_FORWARD flag the host RX ring slots right before they >> released to the kernel, and the kernel with take care of forward them to >> the TX ring for you. However, be aware this mode is still experimental, and >> not well tested yet. >> >> Regarding the use of bridge, the correct command line for what you want >> to do (forward all packets between host stack and NIC, in both directions) >> is: >> >> # bridge netmap:em0 >> >> However, to let TCP/UDP traffic go correctly, you need to disable tso and >> checksum offloadings on em0, since netmap will not program the NIC to >> perform those offloadings. This is also valid for your custom program above. >> >> Cheers, >> Vincenzo >> >> 2017-01-19 2:51 GMT+01:00 David Belle-Isle : >> >>> Hi, >>> >>> I'm trying to open a netmap descriptor to an interface to send packets. >>> However, I don't want to prevent the host to send and receive data >>> (transparent). I don't think this should be hard but I can't figure out >>> how >>> to do this. >>> >>> I tried to run the bridge example in the FreeBSD distribution but even >>> that >>> I can't get to run without stopping the host's networking completely. I >>> tried running ./bridge em0 em0 which, if I understand correctly should >>> open >>> the NIC and host rings and pass the traffic through. As soon as I start >>> it >>> all the networking stops. >>> >>> I tried testing in a VMware VM and on bare-metal with em cards and got >>> the >>> same results with both. >>> >>> Can someone help me? >>> >>> Thanks, >>> >>> David >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >> >> >> >> -- >> Vincenzo Maffione >> > > -- Vincenzo Maffione