From owner-freebsd-net@freebsd.org Thu Jan 19 11:00:12 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 887C4CB8F08 for ; Thu, 19 Jan 2017 11:00:12 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 6635B1244 for ; Thu, 19 Jan 2017 11:00:12 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 6595ACB8F07; Thu, 19 Jan 2017 11:00:12 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 653A5CB8F06 for ; Thu, 19 Jan 2017 11:00:12 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mail-ot0-x229.google.com (mail-ot0-x229.google.com [IPv6:2607:f8b0:4003:c0f::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2A7AC1243 for ; Thu, 19 Jan 2017 11:00:12 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mail-ot0-x229.google.com with SMTP id f9so29287659otd.1 for ; Thu, 19 Jan 2017 03:00:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=COWIBoV+Da7v9wz4g90SWKr8tDwSiD0UFg8ZM0OC8zk=; b=EIIS1odHRvWp/Y35r4Jtv556B3WTG9llhwY2kPdIBIeoQS9G4WXTVJEsdoAXSUQhIM 5g0WvFKKNWj7h6/Mk86pS6+J28Izcd7Wu0xrlDeWb2RBt1nkm3anZEwPbDBmOmcN2XdR 4jSC2XkFgRw4oEMBmcOkgFwDFsLU3Xs0+ZvM/gqx20WU8DgQ7oxRipJir+oLZbWUe81X AjBtpPAN2p4gBogxWOW9sHwVe6NqV1o3elEftQpumMjE0q4Zw0+XTqXSbbovkszOUCue dk5GuxEbOC36xAYK9+EwHDdETGmb9I2W6Uh+OLFbfH5mGo4TvSitkGBK1rcpoc6lmber GoQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=COWIBoV+Da7v9wz4g90SWKr8tDwSiD0UFg8ZM0OC8zk=; b=TkGe8QzbQVpLiQX0sgOUt2iIQUExS/2VoOeW0F0/3WTPc1fKY/o0eSZ+qyks6yfWYc pJ+Vmpcv3NFHkW0qoxgxbDkCFZxUQWoqIv5TAdZgjQYd53ywx8K/rdupOQ7Q8RievJED cuLvqczvmBNnKddkw+uSfcmmL+hoT5n31NohL1tc/weWjfgYJuYEf6MSkXihT2fNSWOs IQefzXkCL4p/Io0DkbOo/OIjrDOejJh9DWotTT+FPTmi/7+5X5+kC1oPawjyG9pvU+0w 6ryWMinIYTPdnGTIFCXEy2VQwJK9WsF24E+48a1ZNIsHNhzycD81pF6kKkASkEfN8n/O yBIg== X-Gm-Message-State: AIkVDXJ+vXlz6PyBXGDf+N5XeJE+g9Pp4ywGxuYIXEcCHbpDvHhD1iZlEsfohTeejeM7ATMHf/qclHPnUKwVog== X-Received: by 10.157.40.46 with SMTP id m43mr4404029otb.213.1484823611306; Thu, 19 Jan 2017 03:00:11 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.6.166 with HTTP; Thu, 19 Jan 2017 03:00:10 -0800 (PST) In-Reply-To: References: From: Vincenzo Maffione Date: Thu, 19 Jan 2017 12:00:10 +0100 Message-ID: Subject: Re: Netmap TX with no impact to host To: David Belle-Isle Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jan 2017 11:00:12 -0000 Hi, Before answering to the question, some important disclaimers. If you are using FreeBSD-12-current, please be aware that there are IFLIB developments in progress that affect em devices. Unfortunately, at the moment netmap does not work on em/lem/igb devices because of the switch to IFLIB at this commit https://github.com/freebsd/freebsd/commit/4bf01b136dfa554a57f25559e7a848bf32206c66. Netmap on em works correctly right before this commit. If you are using FreeBSD-11 versions, be aware that netmap in emulated mode is broken in those versions. I have the fix for that, but it is not in the released code. If you need it, I can give you a patch. Btw, can anyone point me at the procedure to push this fix to the relevant release/stable branches? If you use em devices with DEV_NETMAP enabled, emulated mode is not used, so it will work. If you use interfaces without DEV_NETMAP support, emulated netmap is used and you are affected by the broken code. Regarding your question: You can build a netmap program that does what you want: (1) transmit packets on em0 at high rates using netmap (2) let the host stack transmit packets on em0 (3) let packets received on em0 to find their way to the host stack. In netmap it is possible to open just a subset of the TX rings and/or a subset of the RX rings. In your case you need to open all the TX hw rings (to acheve (1)) and the RX host ring (a.k.a. RX sw ring) to achieve (2). You achieve (3) by simply not opening the RX hw rings. To implement this you need two calls to nm_open() (similarly to what bridge does): pa = nm_open("netmap:em0/T, NULL, 0, NULL); /* open all TX hw rings */ pb = nm_open("netmap:em0^/R, NULL, NM_OPEN_NO_MMAP, pa); /* open only RX host ring */ Then you can use "pa" to transmit packets on the hw TX rings (1). To implement (2), you have two options: (A) your application moves packets from the host RX ring (pb) to the hw TX rings (pa). In the end this is the logic of the "bridge" program does, so you could start from there. (B) you use "transparent mode" in the host->nic direction: applications marks with the NS_FORWARD flag the host RX ring slots right before they released to the kernel, and the kernel with take care of forward them to the TX ring for you. However, be aware this mode is still experimental, and not well tested yet. Regarding the use of bridge, the correct command line for what you want to do (forward all packets between host stack and NIC, in both directions) is: # bridge netmap:em0 However, to let TCP/UDP traffic go correctly, you need to disable tso and checksum offloadings on em0, since netmap will not program the NIC to perform those offloadings. This is also valid for your custom program above. Cheers, Vincenzo 2017-01-19 2:51 GMT+01:00 David Belle-Isle : > Hi, > > I'm trying to open a netmap descriptor to an interface to send packets. > However, I don't want to prevent the host to send and receive data > (transparent). I don't think this should be hard but I can't figure out how > to do this. > > I tried to run the bridge example in the FreeBSD distribution but even that > I can't get to run without stopping the host's networking completely. I > tried running ./bridge em0 em0 which, if I understand correctly should open > the NIC and host rings and pass the traffic through. As soon as I start it > all the networking stops. > > I tried testing in a VMware VM and on bare-metal with em cards and got the > same results with both. > > Can someone help me? > > Thanks, > > David > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Vincenzo Maffione