From owner-freebsd-jail@freebsd.org Sat Feb 10 08:55:57 2018 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7573CF09084; Sat, 10 Feb 2018 08:55:57 +0000 (UTC) (envelope-from zec@fer.hr) Received: from mail.fer.hr (mail.fer.hr [161.53.72.233]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.fer.hr", Issuer "TERENA SSL CA 3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C9DDE73C32; Sat, 10 Feb 2018 08:55:56 +0000 (UTC) (envelope-from zec@fer.hr) Received: from x23 (31.147.122.18) by MAIL.fer.hr (161.53.72.233) with Microsoft SMTP Server (TLS) id 14.3.361.1; Sat, 10 Feb 2018 09:54:43 +0100 Date: Sat, 10 Feb 2018 09:54:49 +0100 From: Marko Zec To: "O. Hartmann" CC: "Bjoern A. Zeeb" , , freebsd-current Subject: Re: VIMAGE: vnet, epair and lots of jails on bridgeX - routing Message-ID: <20180210095449.3117e6d9@x23> In-Reply-To: <20180210085248.7b9af104@thor.intern.walstatt.dynvpn.de> References: <20180208093052.7f5d7a98@freyja.zeit4.iv.bundesimmobilien.de> <20180209172259.1ec9b9f4@thor.intern.walstatt.dynvpn.de> <2D57FE3A-744A-4A44-B572-5338AB9E187D@lists.zabbadoz.net> <20180210085248.7b9af104@thor.intern.walstatt.dynvpn.de> X-Mailer: Claws Mail 3.16.0 (GTK+ 2.24.31; amd64-portbld-freebsd11.1) MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Originating-IP: [31.147.122.18] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Feb 2018 08:55:57 -0000 On Sat, 10 Feb 2018 08:52:21 +0100 "O. Hartmann" wrote: > Am Fri, 09 Feb 2018 16:43:17 +0000 > "Bjoern A. Zeeb" schrieb: > > > On 9 Feb 2018, at 16:22, O. Hartmann wrote: > > > > > Am Thu, 8 Feb 2018 09:31:15 +0100 > > > "O. Hartmann" schrieb: > > > > > > Is this problem to trivial? > > > > I read through it yesterday and found myself in the position that I > > need a whiteboard or paper and pencil or an ASCII art of your > > situation. But by the time I made it to the question I was > > basically lost. Could you massively simplify this and maybe > > produce the ASCII art? > > > > /bz > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to > > "freebsd-current-unsubscribe@freebsd.org" > > All right. > > I'm not much of an artist and at this very moment, I haven't much > experience with neat ASCII art tools. But I'll provide a sketch > later, but I also will simplify the situation. > > Consider three "vswitches", basically based on the creation of > bridges, bridge0, bridge1, bridge2. Create at least three individual > vnet-jails attached to each vbridge. Those jails have epair pseudo > devices. The jail itself owns the "a-part" of the epair and the > b-part is "member of the bridge". Each jail's epairXXXa has an IP > assigned of the network the vswitch is part of. I mention a- and > b-part of the epair here, because I thought it could matter, but I > think for symmetry reasons it doesn't. > > Now consider a further, special jail. This jail is supposed to have > three epair devices, each one is reaching into one of the vbridges. > This jail is the router/routing jail. Later, this jail should filter > via IPFW the traffic between the three vbridges according to rules, > but this doesn't matter here, beacuase the basics are not working as > expected. > > Now the problems. It doesn't matter on which jail of the three > vswitches I login, the moment a vbridge has more than two member > epairs (one is alway member of the routing jail, now consider a > database jail and a webserver jail), pinging each jail or the routing > jail fails. It works sometimes for a couple of ICMP packets and then > stops. > > If each vbridge has only one member jail, I have NO PROBLEMS > traversing accordingly to the static routing rules from one vbridge > to any other, say from vbridge1 to vbridge0 or vbridge2 and any > permutation of that. > > The moment any of the bridges gets an additional member epair > interface (so the bridge has at least three members including the on > reaching into the virtual router jail) the vbridge seems to operate > unpredictable (to me). Pinging jails memeber of that vbridge are > unreachable. > > Technical information: > > The kernel has options IPFIREWALL, VIMAGE. The host's ipfw (kernel) > declines packets by default. Each jail is configured to have ipfw > "open". > > Thanks for the patience. If you could provide a script which sets up the topology you described in two lengthy posts then others could reproduce this, and your chances of getting useful feedback would certainly increase. We also have a graphical tool (https://github.com/imunes/imunes) which can set up a topology like you described in a few clicks of a mouse, albeit using netgraph and ng_eiface instead of epairs, but I assume this is irellevant as long as you are not aiming for maximum packet throughputs. If you attempt to use this tool, note that selecting "stpswitch" will create if_bridge instances, whereas "lanswitch" creates ng_bridge instances. Good luck, Marko > > Kind regards, > > O. Hartmann