From owner-freebsd-virtualization@freebsd.org Mon Feb 26 12:28:41 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E9526F1C8A4 for ; Mon, 26 Feb 2018 12:28:40 +0000 (UTC) (envelope-from mail@osfux.nl) Received: from vm1982.vellance.net (vm1982.vellance.net [79.99.187.212]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6940072B0A for ; Mon, 26 Feb 2018 12:28:40 +0000 (UTC) (envelope-from mail@osfux.nl) Received: from vm1982.vellance.net (localhost [127.0.0.1]) by vm1982.vellance.net (Postfix) with ESMTP id B227620157; Mon, 26 Feb 2018 13:28:37 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=osfux.nl; s=default; t=1519648118; bh=JPUCXZ46vQOBkJ4lJxrI+ObT6qXMTKLW7NY+32iTPvM=; h=Subject:To:Cc:References:From:Date:In-Reply-To; b=ORM0Xu7k6W5Oz/MAve1Vd0SvjFDyUlXg55ENBVFV4ugqX8INhoorfxkymP68tGPgh +kzgbUoYvEN3caWZDGMbEvtyq/EZExtofpksyEVbzz0kEdH8x24GCVyGYkMrF57EKt b2XnJB9DoBvpqWtpj0/3fORYkeWHoVbHQFANfZ923GrbWRvr0Qy830WXUEHlmAIemk 93zTzmYOt2yBz+fy1Ry5qJW0ernXBmukRHA27XMUx/Ef0HjOrtSZy+f2LOn6XV6csP Z0LFAPoQOtEUiHScJZsXqZEXbB364D++PL274IJl7BY7uncVokH3b+/3V/xlyOi49L CiKcoCtDOOzVozCoNmRnTmdnbPdmcZPP6eoD8JZhUgOX3/wg4eucr1fcbdeqgGHPeU tJLXnOyP8nVU7Pp1PIpVIcReW8LsVbYKc3gTUHQ4+Do8j5rL/7YOEG7ZJc039oN4oX ie9WJGrILcIOMJcMYiH0G+b9qwBF1NUWEDxCSpXwEIWbiEbiR8Y95Lq5WqBI7zqNJR T7X5AfjW0NsiWFpM4ykILdEJZ5ReATbFb8iF6P3l1xa1Hznhm3mEVG95PTzQqIQTxX 1fKeriszAOhTbmRnf6EhxcpDQZQKDrvqxCtYkh0hY6TCw0P2Fiq+Zj6X6UIUqOprvb dp1jCP7FS9V4LUUPjVnf1ca4= Received: from vm1982.vellance.net (localhost [127.0.0.1]) by vm1982.vellance.net (Postfix) with ESMTP id 8D2DA20126; Mon, 26 Feb 2018 13:28:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=osfux.nl; s=default; t=1519648117; bh=JPUCXZ46vQOBkJ4lJxrI+ObT6qXMTKLW7NY+32iTPvM=; h=Subject:To:Cc:References:From:Date:In-Reply-To; b=jAR2J+8ZcAeD2Dc90vbDPkU0X6LTJmX3nI2LBvS9o5U8B+MZhRc7OLCBS5KieZgnp jA36MKyMgRIfGxnimaPw420kI4QGRLLOd0Mac3dZTpP+iYwwTMe3v7iF7hASMLPntN NgNPOkBj/Z+ZpbU9Xi+8+FoxTezexBUuh26n6nOaUwSrLj7QiXJsQvRg2ODMcwTCWx 8GlsjfH41Jb7eZRF+OnZM16TNhEPaNMz394akeVLeRmV/D6IAAVg5uPPPdGJUJSewc r9yhPKlDSYz94tJs/de/Wm4aY8+6LZcWZqzbMmCQ6BOvO7pVAMk2Pa444mSIlr9Ncj tOPcgd4Tt2VzDQP8HxqBKR68t+K2H/9Vl7ummE4GUc8apw6ctlKX3ZpxUOS2n1kGWf UFLAtUqnv54QItojRKn+l6orhJ8Gm1yH+VWvKJy19d2owhBFwjYETj0pQSInm8CGXi zlNW+a9NNMboBxSQo4yZYYlClpV5rEh1No+vUxv0aC9EXcu1I4KHnFnt/N9nFz+jaw hb4HCRKpHnAAcz31s8+RhDYSKXw5Jn2Rbb1P2OV00mqzuM7yVozvTWRIxXpFjs9+i9 889pqdiD4p7dLNd7RYR80xMRGmHpi2Kwpupz522dtMjGPM1zcP/KhLVQVdR7SYOBL6 pf+uypgyz3Y1boO3EnwSljPs= X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on vm1982.vellance.net X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=ham autolearn_force=no version=3.4.1 Received: from quanza-qhq-dhcp161.q (engineering.quanza.net [91.208.87.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by vm1982.vellance.net (Postfix) with ESMTPSA; Mon, 26 Feb 2018 13:28:25 +0100 (CET) Subject: Re: superfluous host interfaces To: Harry Schmalzbauer Cc: FreeBSD virtualization References: <20180225131401.GA3138@v007.zyxst.net> <5A93CEB6.1080406@omnilan.de> <5A93D9D0.4090804@omnilan.de> <54f9019e-6e86-8e10-32d7-9f14d159bb0a@osfux.nl> <5A93F9DE.9090908@omnilan.de> From: Ruben Message-ID: <745701ca-4351-ae19-2688-ee458e85efa6@osfux.nl> Date: Mon, 26 Feb 2018 13:28:23 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <5A93F9DE.9090908@omnilan.de> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Virus-Scanned: ClamAV using ClamSMTP X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Feb 2018 12:28:41 -0000 Hi Harry, Following some examples in the presentation you linked, Im rather suprised that i'm apparently already utilizing the ng subsystem on some of my machines (ngctl list lists entries and a couple of hooks as well). Ill put a rig together for some playing around with the netgraph subsystem, very interesting! Thank you (again) for your elaboration :) Kind regards, Ruben On 26/02/2018 13:13, Harry Schmalzbauer wrote: > Bezüglich Ruben's Nachricht vom 26.02.2018 11:34 (localtime): >> On 26/02/2018 10:56, Harry Schmalzbauer wrote: > … >>> Another, personally very significant, reason is that you'll get a >>> superfluous host interface for each if_bridge(4), which makes the output >>> of plain ifconfig(8) kind of unreadable. > … >> By superflous host interfaces, do you mean the tap interfaces configured >> for each vm together with the bridge interfaces they are "bundled" in? > Additionally to the if_tap(4) ethernet host interfaces, you also get > if_bridge(4) ethernet interfaces, named bridgeX if I remember correctly. > The if_bridge(4) host interface is for control purposes only on a VM-SDN > host – at least with my setups. I never needed to make use of IP > numbered bridges. And I don't need to utilize any if_bridge(4) features > like STP, so I consider the bridgeX host interfaces as superfluous in > the VM-SDN use case. > > I'd call the if_tap(4) host interfaces likewise superfluous – you would > only need the corresponding character devices – but that's been > implemented long before the need for SDN setups, so it is like it is. > And using ng_bridge(4) instead of if_bridge(4) doesn't change the need > for if_tap(4). Only with vale(4) switches, bhyve(8) was able to provide > virtio-net connection wihtout "spamming" the host's ethernet interface > list (no tapX, no bridgeX). > > >> Overall I'm very happy with my bhyve setups atm. If there are any >> speed-/administrative-advantages that come with bridge_ng however, I'm >> very interested in switching to such a setup (or at least play with it). >> I'm running my vm's without any helper project so I'm flexible enough to >> do some fiddling :P >> >> Do you know of any documentation on using bridge_ng together with bhyve? >> My search-engines don't turn up much Im affraid and I haven't stumbled >> on it before. > Unfortunately it's not too easy to get started with netgraph. > Besides numerous man pages for the different nodes (ng_bridge(4) e.g.), > I only know the following source for a good overview: > http://www.netbsd.org/gallery/presentations/ast/2012_AsiaBSDCon/Tutorial_NETGRAPH.pdf > > One convenience disadvantage with ng_bridge(4) is that you have to > assign MACs manually, while if_bridge(4) does that itself (adjustable by > sysctl net.link.bridge.inherit_mac). > And you need to script all setups yourself. Almost all of my setups > seem to be awkward enough that I always had to do some local scripting, > so that wasn't really a disadvantage for me. > > If you're happy with your setup, I don't think you gain anything from > switching to ng_bridge(4), besides learning to control netgraph(4) > (which is very desirable imho). > I haven't had time left to do useful benchmarking regarding ng_bridge(4) > vs. if_bridge(4). I even don't know if netgraph nodes are still limited > to single threads. But rough load comparings on a IvyBride machine > showed similar resource usage for both bridges, both easy capable of > 1GbE saturation with small frames (while I remember one run with > ng_bridge(4) and if_vmnet(4), which couldn't deliver 1GbE speed, and I > wanted to falsify for vmnet/tap difference... just ran out of time :-( ). > > -harry