From owner-freebsd-questions@freebsd.org Mon Jun 3 08:41:28 2019 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2921D15A8BDF for ; Mon, 3 Jun 2019 08:41:28 +0000 (UTC) (envelope-from SRS0=Nl+t=UC=vega.codepro.be=kp@codepro.be) Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.codepro.be", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3F0916D3E1; Mon, 3 Jun 2019 08:41:27 +0000 (UTC) (envelope-from SRS0=Nl+t=UC=vega.codepro.be=kp@codepro.be) Received: from vega.codepro.be (unknown [172.16.1.3]) by venus.codepro.be (Postfix) with ESMTP id 89CCA36AE1; Mon, 3 Jun 2019 10:41:20 +0200 (CEST) Received: by vega.codepro.be (Postfix, from userid 1001) id 85246E48D; Mon, 3 Jun 2019 10:41:20 +0200 (CEST) Date: Mon, 3 Jun 2019 10:41:20 +0200 From: Kristof Provost To: Matthew Seaman Cc: freebsd-questions@freebsd.org Subject: Re: to jail or not to jail Message-ID: <20190603084120.GA10541@vega.codepro.be> References: <9783db6e-959e-b177-89d5-84af47fd5c3f@FreeBSD.org> <1231820b-830b-4a22-8b08-37242226d276@www.fastmail.com> <5d9af532-45fc-b088-893d-ec413460b2ff@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <5d9af532-45fc-b088-893d-ec413460b2ff@FreeBSD.org> X-Checked-By-NSA: Probably User-Agent: Mutt/1.11.4 (2019-03-13) X-Rspamd-Queue-Id: 3F0916D3E1 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of SRS0=Nl@codepro.be designates 5.9.86.228 as permitted sender) smtp.mailfrom=SRS0=Nl@codepro.be X-Spamd-Result: default: False [-4.94 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:5.9.86.228]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.97)[ip: (-8.47), ipnet: 5.9.0.0/16(0.33), asn: 24940(-1.72), country: DE(-0.00)]; RCVD_IN_DNSWL_MED(-0.20)[228.86.9.5.list.dnswl.org : 127.0.9.2]; RCPT_COUNT_TWO(0.00)[2]; MX_GOOD(-0.01)[mx2.codepro.be,mx1.codepro.be]; NEURAL_HAM_SHORT(-0.76)[-0.755,0]; FORGED_SENDER(0.30)[kp@freebsd.org,SRS0=Nl@codepro.be]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:5.9.0.0/16, country:DE]; FROM_NEQ_ENVFROM(0.00)[kp@freebsd.org,SRS0=Nl@codepro.be]; TAGGED_FROM(0.00)[t=UC=vega.codepro.be=kp] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Jun 2019 08:41:28 -0000 On 2019-06-03 09:33:25 (+0100), Matthew Seaman wrote: > On 02/06/2019 12:41, Dave Cottlehuber wrote: > >> think about using vimage jails on 12.0, as that makes the jails seem a > >> lot more like just regular VMs, and gives you the ability to effectively > >> create a private virtual switch inside your server, rather than having > >> services appear on external interfaces. Beware though that there are > >> currently some quite severe bandwidth limitations on this sort of > >> internally virtualized networking under FreeBSD, so this is not suitable > >> for a high-traffic system. > > > Matthew, anything you can point me to about this limitation? > > Kristof Provost talked about it during his presentation at BSDCAN -- the > video of that is not turning up in my searches, but here's probably a > very similar talk from linux.conf.au: > > https://www.youtube.com/watch?v=2neDPNIcrBk > > In short the problem is that there's a single thread for handling all > the internal traffic. (Possibly a single lock as well?) > if_bridge contends on a single mutex. Expect no more than ~1.2 million packets per second through a bridge, regardless of how many cores you have. That's actually a fairly substantial amount of traffic, it may or may not be a practical concern for you. Note that that's unrelated to vimage as such. It's just a very common way to set things up. If you avoid if_bridge the performance impact of vimage is trivial, and you pay it even if you don't use vimage jails. Best regards, Kristof