From owner-freebsd-virtualization@freebsd.org Mon Feb 26 12:13:20 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6A64DF1145D for ; Mon, 26 Feb 2018 12:13:20 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EDD3D7222D for ; Mon, 26 Feb 2018 12:13:19 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id w1QCDIjf092102; Mon, 26 Feb 2018 13:13:18 +0100 (CET) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 8887BB92; Mon, 26 Feb 2018 13:13:18 +0100 (CET) Message-ID: <5A93F9DE.9090908@omnilan.de> Date: Mon, 26 Feb 2018 13:13:18 +0100 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Ruben CC: FreeBSD virtualization Subject: Re: superfluous host interfaces References: <20180225131401.GA3138@v007.zyxst.net> <5A93CEB6.1080406@omnilan.de> <5A93D9D0.4090804@omnilan.de> <54f9019e-6e86-8e10-32d7-9f14d159bb0a@osfux.nl> In-Reply-To: <54f9019e-6e86-8e10-32d7-9f14d159bb0a@osfux.nl> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Mon, 26 Feb 2018 13:13:18 +0100 (CET) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Feb 2018 12:13:20 -0000 Bezüglich Ruben's Nachricht vom 26.02.2018 11:34 (localtime): > On 26/02/2018 10:56, Harry Schmalzbauer wrote: … >> Another, personally very significant, reason is that you'll get a >> superfluous host interface for each if_bridge(4), which makes the output >> of plain ifconfig(8) kind of unreadable. … > > By superflous host interfaces, do you mean the tap interfaces configured > for each vm together with the bridge interfaces they are "bundled" in? Additionally to the if_tap(4) ethernet host interfaces, you also get if_bridge(4) ethernet interfaces, named bridgeX if I remember correctly. The if_bridge(4) host interface is for control purposes only on a VM-SDN host – at least with my setups. I never needed to make use of IP numbered bridges. And I don't need to utilize any if_bridge(4) features like STP, so I consider the bridgeX host interfaces as superfluous in the VM-SDN use case. I'd call the if_tap(4) host interfaces likewise superfluous – you would only need the corresponding character devices – but that's been implemented long before the need for SDN setups, so it is like it is. And using ng_bridge(4) instead of if_bridge(4) doesn't change the need for if_tap(4). Only with vale(4) switches, bhyve(8) was able to provide virtio-net connection wihtout "spamming" the host's ethernet interface list (no tapX, no bridgeX). > Overall I'm very happy with my bhyve setups atm. If there are any > speed-/administrative-advantages that come with bridge_ng however, I'm > very interested in switching to such a setup (or at least play with it). > I'm running my vm's without any helper project so I'm flexible enough to > do some fiddling :P > > Do you know of any documentation on using bridge_ng together with bhyve? > My search-engines don't turn up much Im affraid and I haven't stumbled > on it before. Unfortunately it's not too easy to get started with netgraph. Besides numerous man pages for the different nodes (ng_bridge(4) e.g.), I only know the following source for a good overview: http://www.netbsd.org/gallery/presentations/ast/2012_AsiaBSDCon/Tutorial_NETGRAPH.pdf One convenience disadvantage with ng_bridge(4) is that you have to assign MACs manually, while if_bridge(4) does that itself (adjustable by sysctl net.link.bridge.inherit_mac). And you need to script all setups yourself. Almost all of my setups seem to be awkward enough that I always had to do some local scripting, so that wasn't really a disadvantage for me. If you're happy with your setup, I don't think you gain anything from switching to ng_bridge(4), besides learning to control netgraph(4) (which is very desirable imho). I haven't had time left to do useful benchmarking regarding ng_bridge(4) vs. if_bridge(4). I even don't know if netgraph nodes are still limited to single threads. But rough load comparings on a IvyBride machine showed similar resource usage for both bridges, both easy capable of 1GbE saturation with small frames (while I remember one run with ng_bridge(4) and if_vmnet(4), which couldn't deliver 1GbE speed, and I wanted to falsify for vmnet/tap difference... just ran out of time :-( ). -harry