From owner-freebsd-net@freebsd.org Thu May 25 15:25:01 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 82707D8261B for ; Thu, 25 May 2017 15:25:01 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1C6B61D2F for ; Thu, 25 May 2017 15:25:00 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (mh0.gentlemail.de [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id v4PFOxvp051999; Thu, 25 May 2017 17:24:59 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (titan.inop.mo1.omnilan.net [IPv6:2001:a60:f0bb:1::3:1]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id D2F1D12C; Thu, 25 May 2017 17:24:58 +0200 (CEST) Message-ID: <5926F74A.1000101@omnilan.de> Date: Thu, 25 May 2017 17:24:58 +0200 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Vincenzo Maffione CC: "freebsd-net@freebsd.org" Subject: vale uplink via vlan-if [Was: Re: Are ./valte-ctl and ./bridge friends or competitors?] References: <58CBA727.3040108@omnilan.de> <58CBBF7A.8050604@omnilan.de> <58CC26CF.5050708@omnilan.de> <58CFA606.7090306@omnilan.de> <58D02259.9010101@omnilan.de> In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Thu, 25 May 2017 17:24:59 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 May 2017 15:25:01 -0000 Bezüglich Vincenzo Maffione's Nachricht vom 21.03.2017 19:05 (localtime): > 2017-03-20 19:41 GMT+01:00 Harry Schmalzbauer : > >> Bezüglich Vincenzo Maffione's Nachricht vom 20.03.2017 12:50 (localtime): >> … >>>> So to summarize for newbies exploring netmap(4) world in combination >>>> with physical uplinks and virtual interfaces, it's important to do the >>>> following uplink NIC configuration (ifconfig(8)): >>>> -rxcsum -txcsum -rxcsum6 -txcsum6 -tso -lro promisc >>>> >>> >>> Exactly. This is mentioned at the very end of netmap(4): >>> >>> "netmap does not use features such as checksum offloading, TCP >> segmentation >>> offloading, encryption, VLAN encapsulation/decapsulation, etc. When >> using >>> netmap to exchange packets with the host stack, make sure to disable >> these >>> features." >>> >>> But it is probably a good idea to add these example ifconfig instructions >>> somewhere (man page or at least the README in the netmap repo). >>> >>> >>>> >>>> I guess vlanhwtag, vlanhwfilter and vlanhwtso don't interfere, do they? >>>> >>> >>> Well, I think they interfere: if you receive a tagged packet and the NIC >>> strips the tag and puts it in the packet descriptor, then with netmap you >>> will see the untagged packet, and you wouldn't have a way to see the tag. Sorry picking this up again, but I'm stuck getting vale(4) productive :-( I took lagg(4) out of the game and configured my desired setup using if_bridge(4) at first. The physical uplink NIC is em0. The bridge/vale uplink is em0.232. hostB --switch-- em0-hostA | '- em0.232 | bridge5-vmnet0 | vtnet0-GUESTa <-tcpdump: 17:07:28.423768 00:a0:98:73:9f:42 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 172.21.34.10 tell 172.21.35.1, length 28 17:07:28.424208 00:0c:29:40:3a:dd > 00:a0:98:73:9f:42, ethertype ARP (0x0806), length 60: Reply 172.21.34.10 is-at 00:0c:29:40:3a:dd, length 46 The same is visable on vmnet0 nad em0 of course. Now if I replace bridge5 by vale, leaving everything else unchanged besides using netmap-vtnet with bhyve, I don't get ARP is-at answer. I can see the who-has on all interfaces involved, and also the is-at answer up to em0.232, but not at vtnet0 (the guest, connected via vale). To draw the same picture like with bridge: hostB --switch-- em0-hostA | '- em0.232 | vale232:em0.232-' vale232:GUESTa--vtnet0-GUESTa <-tcpdump: 17:16:00.111868 00:a0:98:73:9f:42 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 172.21.34.10 tell 172.21.35.1, length 28 ... no reply While tcpdump of em0.232 shows: 17:16:01.119537 00:a0:98:73:9f:42 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 172.21.34.10 tell 172.21.35.1, length 46 17:16:01.119849 00:0c:29:40:3a:dd > 00:a0:98:73:9f:42, ethertype ARP (0x0806), length 60: Reply 172.21.34.10 is-at 00:0c:29:40:3a:dd, length 46 The reply made it up to vale's uplink, but not through vale. Am I missing something? Tagging, checksum-disabling etc. seems to be right, since utilizing if_bridge(4) gives the expected result, but I have no idea why I can't get packets via vale(4). Important note: Using em0.232 parent (vlandev em0) for vale uplink does work! So I guess if_em(4)'s native netmap support interferes with the vlan clone. I'm out at this point, far too less knwoledge about the code paths... Can anybody else help here? Thanks, -harry