From owner-freebsd-virtualization@freebsd.org Tue Feb 27 08:17:24 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 53B82F2EB0A for ; Tue, 27 Feb 2018 08:17:24 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A2DFE686C9 for ; Tue, 27 Feb 2018 08:17:23 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id w1R8HJUH004416; Tue, 27 Feb 2018 09:17:19 +0100 (CET) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 55E61D44; Tue, 27 Feb 2018 09:17:19 +0100 (CET) Message-ID: <5A95140E.8030909@omnilan.de> Date: Tue, 27 Feb 2018 09:17:18 +0100 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Paul Vixie CC: Ruben , FreeBSD virtualization Subject: Re: superfluous host interfaces References: <20180225131401.GA3138@v007.zyxst.net> <5A93CEB6.1080406@omnilan.de> <5A93D9D0.4090804@omnilan.de> <54f9019e-6e86-8e10-32d7-9f14d159bb0a@osfux.nl> <5A93F9DE.9090908@omnilan.de> <5A94F730.7040009@redbarn.org> In-Reply-To: <5A94F730.7040009@redbarn.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Tue, 27 Feb 2018 09:17:20 +0100 (CET) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Feb 2018 08:17:24 -0000 Bezüglich Paul Vixie's Nachricht vom 27.02.2018 07:14 (localtime): > > > Harry Schmalzbauer wrote: >> Bezüglich Ruben's Nachricht vom 26.02.2018 11:34 (localtime): >>> On 26/02/2018 10:56, Harry Schmalzbauer wrote: >> … >>>> Another, personally very significant, reason is that you'll get a >>>> superfluous host interface for each if_bridge(4), which makes the >>>> output >>>> of plain ifconfig(8) kind of unreadable. >> … >>> By superflous host interfaces, do you mean the tap interfaces configured >>> for each vm together with the bridge interfaces they are "bundled" in? >> >> Additionally to the if_tap(4) ethernet host interfaces, you also get >> if_bridge(4) ethernet interfaces, named bridgeX if I remember correctly. > > you do not remember correctly. Please see next para. >> [mm1.redbarn:amd64] egrep 'bridge|tap' /etc/rc.conf >> autobridge_interfaces="bridge0" >> autobridge_bridge0="tap* igb1" >> cloned_interfaces="bridge0 tap0 tap1 tap2 tap3 tap4 tap5 tap6 tap7" >> ifconfig_bridge0="inet 24.104.150.210/27" >> ifconfig_bridge0_ipv6="inet6 2001:559:8000:cd::2/64 auto_linklocal up" >> ifconfig_tap0="up" >> ifconfig_tap1="up" >> ifconfig_tap2="up" >> ifconfig_tap3="up" >> ifconfig_tap4="up" >> ifconfig_tap5="up" >> ifconfig_tap6="up" >> ifconfig_tap7="up" >> [mm1.redbarn:amd64] ifconfig | egrep '^(bridge|tap)' >> bridge0: flags=8843 metric 0 >> mtu 1500 That's what I mean; and it's named bridgeX, so memeory works in that case ;-) If you have only one "LAN" sharing all VMs, the one additional interface is neglectable. But my setups are different. I have almost as many different 802.11q separated ethernet collsion domains (VLANs) as VMs. That's what ESXi's portgroup is used for. I need a separate switch for each VLAN (guests mustn't be able to sniff traffic etc.). … >> And using ng_bridge(4) instead of if_bridge(4) doesn't change the need >> for if_tap(4). Only with vale(4) switches, bhyve(8) was able to provide >> virtio-net connection wihtout "spamming" the host's ethernet interface >> list (no tapX, no bridgeX). > > how did you get bhyve to use the netmap API rather than the tap > character special device? Not me, peter commited the following: https://svnweb.freebsd.org/base/head/usr.sbin/bhyve/pci_virtio_net.c?r1=288470&r2=293459 >From that an, you don't need to add any host ethernet devices/interfaces, simply start your VM with e.g. "-s 1:0,virtio-net,vale0:1,mac=02:03:04:05:06:07". As long as you created vale0, port [:]1 will be created dynamically and also destroyed after shutdown. Again, to mimic ESXi's portgroups, you need one vale(4) switch for each VLAN. And to uplink, you need to utilize if_valn(4), which forces netmap emulated mode. If you don't have/need VLANs, you can uplink a supported NIC via native netmap support, and additionally gain significatn efficiency improvements. Unfortunately, at least if_vlan(4) uplinks, don't work reliably. After some short time, the complete network netmap(4) subsystem locks up. I talked with Vincenco Maffione (a member of Liugi Rizzo's netmap(4) team of University of Pisa) and fixing emulated netpmap mode on FreeBSD doesn't have really high priority there, since such a setup is considered as weak design. For sure, it's a hack/workarround, but we don't have VLAN/portgroup support in vale(4) nor in byhve(8), and writing my own userland filter is beyond my scope. -harry