From owner-freebsd-net@freebsd.org Wed Dec 23 01:20:40 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D02ADA4EDAD; Wed, 23 Dec 2015 01:20:40 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 92C491A0E; Wed, 23 Dec 2015 01:20:40 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (ppp121-45-234-233.lns20.per1.internode.on.net [121.45.234.233]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id tBN1KIwN028503 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Tue, 22 Dec 2015 17:20:21 -0800 (PST) (envelope-from julian@freebsd.org) Subject: Re: Have I got this VIMAGE setup correct? To: Garrett Wollman , freebsd-net@freebsd.org, freebsd-stable@freebsd.org References: <22137.33475.645324.203196@hergotha.csail.mit.edu> From: Julian Elischer Message-ID: <5679F6CD.6020105@freebsd.org> Date: Wed, 23 Dec 2015 09:20:13 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: <22137.33475.645324.203196@hergotha.csail.mit.edu> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Dec 2015 01:20:40 -0000 On 23/12/2015 1:05 AM, Garrett Wollman wrote: > The consensus when I asked seemed to be that VIMAGE+jail was the right > combination to give every container its own private loopback > interface, so I tried to build that. I noticed a few things: > > 1) The kernel prints out a warning message at boot time that VIMAGE is > "highly experimental". Should I be concerned about running this in > production? CYA only If you are not doing much that is super unusual you should be fine. > > 2) Stopping jails with virtual network stacks generates warnings from > UMA about memory being leaked. I haven't any information about that. > > 3) It wasn't clear (or documented anywhere that I could see) how to > get the host network set up properly. Obviously I'm not going to have > a vlan for every single jail, so it seemed like what most people were > doing was "bridge" along with a bunch of "epair" interfaces. I ended > up with the following: there are exapmples in /usr/share/examples/netgraph for some things.. I've never used the build in configuration stuff,, always handcoded it.. It's probably improved a lot since then. > network_interfaces="lo0 bridge0 bce0" > autobridge_interfaces="bridge0" > autobridge_bridge0="bce0 epair0a epair1a" > cloned_interfaces="bridge0 epair0 epair1" > ifconfig_bridge0="inet [deleted] netmask 0xffffff00" > ifconfig_bridge0_ipv6="inet6 [deleted] prefixlen 64 accept_rtadv" > ifconfig_bce0="up" > ifconfig_epair0a="up" > ifconfig_epair1a="up" > > The net.link.bridge.inherit_mac sysctl, which is documented in > bridge(4), doesn't appear to work; I haven't yet verified that I can > create a /etc/start_if.bridge0 to set the MAC address manually without > breaking something else. The IPv6 stack regularly prints > "in6_if2idlen: unknown link type (209)" to the console, which is > annoying, and IPv6 on the host doesn't entirely work -- it accepts > router advertisements but then gives [ENETUNREACH] trying to actually > send packets to the default gateway. (IPv6 to the jails *does* work!) > > In each of the jails I have to manually configure a MAC address using > /etc/start_if.epairNb to ensure that it's globally unique, but then > everything seems to work. > > Does this match up with what other people have been doing? Anything > I've missed? Any patches I should pull up to make this setup more > reliable before I roll it out in production? I haven't used it for a couple of years.. I know others are, so I'll let them pipe up. > > -GAWollman > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >