From owner-freebsd-net@freebsd.org Thu Jan 19 12:50:37 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 10D7ACB5746 for ; Thu, 19 Jan 2017 12:50:37 +0000 (UTC) (envelope-from dbelleisle@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id E021B1FAF for ; Thu, 19 Jan 2017 12:50:36 +0000 (UTC) (envelope-from dbelleisle@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id DC8F6CB5744; Thu, 19 Jan 2017 12:50:36 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DC291CB5741 for ; Thu, 19 Jan 2017 12:50:36 +0000 (UTC) (envelope-from dbelleisle@gmail.com) Received: from mail-ua0-x236.google.com (mail-ua0-x236.google.com [IPv6:2607:f8b0:400c:c08::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 946291FAD for ; Thu, 19 Jan 2017 12:50:36 +0000 (UTC) (envelope-from dbelleisle@gmail.com) Received: by mail-ua0-x236.google.com with SMTP id 96so32285231uaq.3 for ; Thu, 19 Jan 2017 04:50:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=tKuNSPgtx+F6cWOgMBTgP9EKLSKB8SJy71zto75UW0A=; b=Kwzb3JMML9DcAG/7wdmi/XAtCfqa8c9+P0u+dj4TxRYWSNPiwOqyEPK7M2BRdAQg7S e/vq4iuuj9sJ/kceQwrr9ekk95QQJwqAeHRmw8KOwUGFuFlsk6Bre1XkqClTWgmoDpYU WgR3kcQ6lxAOworf+Gv7TKKW6vcFct+tn2V5sWF5e0Obszkgh2KxMCYe1LH6oI4/IzUL +mqhClZ6V7cIpr5QNC20Yshewt364M5Cee6ID5AZVBaKvdd69Tk0X/44eVAgdnO1u2u8 Ve4QwNi60qcPSd1KaHGGm4jl0iRdpn3s3OpszenDCQzOXmeCy5N9K18BtudaeHtIXmeg Gm2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=tKuNSPgtx+F6cWOgMBTgP9EKLSKB8SJy71zto75UW0A=; b=mwUAPHhh6/MSFQcNdNdEToFuJChP7q6472DXK3AaTL+r9d9OYZhSxuycKeKPnDN4Um HwdQrlGURxkX9MBblAr0a5o8j5N3DJzwDmHS6dx6guDXcIBrFdkYhdVPQi+zDHsbVn4w J+/PgfppMCpDx0mqGeRcdGX0uGL5+YVkIzksveRuCWJ7jKMBMqYJkVEGUB1uIW3ufoXD X/7v8bxT1hj/wigSlrjCdYL7ynyW6PiW8f4zqwirBMjlIjBk1WEg4/Wv9Z5kDyYazZ/R 2XQI9h2y6zwXdGYJOi79rbQlFydiF0cR+nPHwmJF9PywhqI99Wj3BvKQTB5mMgoRHzrL REEg== X-Gm-Message-State: AIkVDXJAIbTBRTSGaNlQiKP8LFEu/v3UpgnnA2347lj+/Q9dB3+hz6HczY3B73pmy48UOvB6nf9xnfeHX2yAvQ== X-Received: by 10.176.7.2 with SMTP id h2mr4585838uah.52.1484830235714; Thu, 19 Jan 2017 04:50:35 -0800 (PST) MIME-Version: 1.0 Received: by 10.176.74.216 with HTTP; Thu, 19 Jan 2017 04:50:15 -0800 (PST) In-Reply-To: References: From: David Belle-Isle Date: Thu, 19 Jan 2017 07:50:15 -0500 Message-ID: Subject: Re: Netmap TX with no impact to host To: Vincenzo Maffione Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2017 12:50:37 -0000 Oh! Thank you soooo much! I was missing the last part about the checksum offloadings and TSO. I disabled them and everything is working perfectly now! Thank you so much!! On Thu, Jan 19, 2017 at 6:00 AM, Vincenzo Maffione wrote: > Hi, > Before answering to the question, some important disclaimers. > > If you are using FreeBSD-12-current, please be aware that there are IFLIB > developments in progress that affect em devices. Unfortunately, at the > moment netmap does not work on em/lem/igb devices because of the switch to > IFLIB at this commit https://github.com/freebsd/freebsd/commit/ > 4bf01b136dfa554a57f25559e7a848bf32206c66. Netmap on em works correctly > right before this commit. > > If you are using FreeBSD-11 versions, be aware that netmap in emulated > mode is broken in those versions. I have the fix for that, but it is not in > the released code. If you need it, I can give you a patch. Btw, can anyone > point me at the procedure to push this fix to the relevant release/stable > branches? > If you use em devices with DEV_NETMAP enabled, emulated mode is not used, > so it will work. If you use interfaces without DEV_NETMAP support, emulated > netmap is used and you are affected by the broken code. > > Regarding your question: > You can build a netmap program that does what you want: > (1) transmit packets on em0 at high rates using netmap > (2) let the host stack transmit packets on em0 > (3) let packets received on em0 to find their way to the host stack. > > In netmap it is possible to open just a subset of the TX rings and/or a > subset of the RX rings. > In your case you need to open all the TX hw rings (to acheve (1)) and the > RX host ring (a.k.a. RX sw ring) to achieve (2). > You achieve (3) by simply not opening the RX hw rings. > To implement this you need two calls to nm_open() (similarly to what > bridge does): > > pa = nm_open("netmap:em0/T, NULL, 0, NULL); /* open all TX hw rings */ > pb = nm_open("netmap:em0^/R, NULL, NM_OPEN_NO_MMAP, pa); /* open only RX > host ring */ > > Then you can use "pa" to transmit packets on the hw TX rings (1). > To implement (2), you have two options: > (A) your application moves packets from the host RX ring (pb) to the hw > TX rings (pa). In the end this is the logic of the "bridge" program does, > so you could start from there. > (B) you use "transparent mode" in the host->nic direction: applications > marks with the NS_FORWARD flag the host RX ring slots right before they > released to the kernel, and the kernel with take care of forward them to > the TX ring for you. However, be aware this mode is still experimental, and > not well tested yet. > > Regarding the use of bridge, the correct command line for what you want to > do (forward all packets between host stack and NIC, in both directions) is: > > # bridge netmap:em0 > > However, to let TCP/UDP traffic go correctly, you need to disable tso and > checksum offloadings on em0, since netmap will not program the NIC to > perform those offloadings. This is also valid for your custom program above. > > Cheers, > Vincenzo > > 2017-01-19 2:51 GMT+01:00 David Belle-Isle : > >> Hi, >> >> I'm trying to open a netmap descriptor to an interface to send packets. >> However, I don't want to prevent the host to send and receive data >> (transparent). I don't think this should be hard but I can't figure out >> how >> to do this. >> >> I tried to run the bridge example in the FreeBSD distribution but even >> that >> I can't get to run without stopping the host's networking completely. I >> tried running ./bridge em0 em0 which, if I understand correctly should >> open >> the NIC and host rings and pass the traffic through. As soon as I start it >> all the networking stops. >> >> I tried testing in a VMware VM and on bare-metal with em cards and got the >> same results with both. >> >> Can someone help me? >> >> Thanks, >> >> David >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > > > -- > Vincenzo Maffione >