Date: Mon, 31 Mar 2014 08:18:37 -0700 From: <dteske@FreeBSD.org> To: "'Palle Girgensohn'" <girgen@FreeBSD.org>, <dteske@freebsd.org> Cc: freebsd-virtualization@FreeBSD.org Subject: RE: VIMAGE, epair/if_bridge or netgraph? Message-ID: <065e01cf4cf4$7e0bd6b0$7a238410$@FreeBSD.org> In-Reply-To: <2E1F87DA-0CC6-4BEE-BF82-2210D49643BF@FreeBSD.org> References: <4FD66519.8030503@FreeBSD.org> <034a01cf4b78$6de95280$49bbf780$@FreeBSD.org> <036601cf4b79$dc61d9c0$95258d40$@FreeBSD.org> <2E1F87DA-0CC6-4BEE-BF82-2210D49643BF@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
> -----Original Message----- > From: Palle Girgensohn [mailto:girgen@FreeBSD.org] > Sent: Monday, March 31, 2014 4:44 AM > To: dteske@freebsd.org > Cc: freebsd-virtualization@FreeBSD.org > Subject: Re: VIMAGE, epair/if_bridge or netgraph? > > > 29 mar 2014 kl. 19:08 skrev dteske@freebsd.org: > > > > > > >> -----Original Message----- > >> From: dteske@FreeBSD.org [mailto:dteske@FreeBSD.org] > >> Sent: Saturday, March 29, 2014 10:58 AM > >> To: 'Palle Girgensohn' > >> Cc: freebsd-virtualization@FreeBSD.org; 'Devin Teske' > >> Subject: RE: VIMAGE, epair/if_bridge or netgraph? > >> > >> > >> > >>> -----Original Message----- > >>> From: owner-freebsd-virtualization@freebsd.org > >>> [mailto:owner-freebsd- virtualization@freebsd.org] On Behalf Of > >>> Palle Girgensohn > >>> Sent: Monday, June 11, 2012 2:37 PM > >>> To: freebsd-virtualization@FreeBSD.org > >>> Subject: VIMAGE, epair/if_bridge or netgraph? > >>> > >>> -----BEGIN PGP SIGNED MESSAGE----- > >>> Hash: SHA1 > >>> > >>> Hi, > >>> > >>> I'm updating some jail servers, and want to use VIMAGE. Compiled it > >>> into the kernel, learned the hard way not to even include PF in the > >>> same kernel [1], so now it works quite well. > >>> > >>> I am setting up many similar jails, some for testing, some for > >>> production. The applications are web servers, som tomcat+apache's, > >>> and some other standard type of services like email and ldap, simple > stuff. > >>> I need no fancy network control, I just need it to work. For each > >>> jail there are two interfaces, one public, connected to a software > >>> bridge (if_bridge or > >>> ng_bridge) acting as a switch, and one internal, for maintenance, > >>> connected to a different software bridge. To each software bridge, I > >>> connect a physical external interface from the jail host. > >>> > >>> I am trying to decide whether to use epair and if_bridge, or to use > >> netgraph. > >>> For netgraph, there is a nice package at DruidBSD [3]. When I found > >>> that, I had already rewritten the standard jail script, using the > >>> v2 patches from polymorf [4]. They work equally fine for my purpose. > >>> > >>> So now I need to know which scales best, is there a difference in > >>> performance or stability between netgraph and epair/if_bridge? > >>> > >>> Cheers, > >>> Palle > >>> > >>> > >>> [1] > >>> https://urldefense.proofpoint.com/v1/url?u=http://forums.freebsd.org > >>> > /showthread.php?t%3D31765&k=%2FbkpAUdJWZuiTILCq%2FFnQg%3D%3D > %0A&r=Mr > >>> > js6vR4%2Faj2Ns9%2FssHJjg%3D%3D%0A&m=55OQxWzTO24ZzksQHz%2Bx0S > %2BolAmp > >>> > ovPuqBDL%2FSJ3eiM%3D%0A&s=14d4e7005de0720881a8a37c21d7738c5efac > 19fd3 > >>> 6a40fd9d86339469412b1c > >>> > >>> [2] > >>> https://urldefense.proofpoint.com/v1/url?u=http://forums.freebsd.org > >>> > /showthread.php?t%3D31949&k=%2FbkpAUdJWZuiTILCq%2FFnQg%3D%3D > %0A&r=Mr > >>> > js6vR4%2Faj2Ns9%2FssHJjg%3D%3D%0A&m=55OQxWzTO24ZzksQHz%2Bx0S > %2BolAmp > >>> > ovPuqBDL%2FSJ3eiM%3D%0A&s=526e98adfe7b28bb2e9387eda1ad4745c142 > 4e8662 > >>> 2109a1b26d53e1ed4526b3 > >>> > >>> [3] > >>> https://urldefense.proofpoint.com/v1/url?u=http://druidbsd.sourcefor > >>> > ge.net/vimage.shtml&k=%2FbkpAUdJWZuiTILCq%2FFnQg%3D%3D%0A&r= > Mrjs6vR4 > >>> > %2Faj2Ns9%2FssHJjg%3D%3D%0A&m=55OQxWzTO24ZzksQHz%2Bx0S%2Bol > AmpovPuqB > >>> > DL%2FSJ3eiM%3D%0A&s=fa628e6b3896b8f1b75b2eda070a9b65375e564e736 > 21da1 > >>> ddf12c18fe56c612 > >>> > >>> [4] > >>> https://urldefense.proofpoint.com/v1/url?u=http://wiki.polymorf.fr/i > >>> > ndex.php?title%3DHowto:FreeBSD_jail_vnet&k=%2FbkpAUdJWZuiTILCq%2F > FnQ > >>> > g%3D%3D%0A&r=Mrjs6vR4%2Faj2Ns9%2FssHJjg%3D%3D%0A&m=55OQxWz > TO24ZzksQH > >>> > z%2Bx0S%2BolAmpovPuqBDL%2FSJ3eiM%3D%0A&s=2762f34c39dd7b58b8b3 > 98d89fa > >>> 0f7fe7e4900978664f25eafb66e1d4aedcdeb > >> > >> [Devin Teske] > >> > >> Never saw a reply to this and I'm locating round-tuits to tackle > >> e-mails that I've marked as "needing reply": > >> > >> I have not profiled > > > > Ugh, that was originally "I have not profiled [epair but I have profiled] > netgraph" > > -- > > Cheers, > > Devin > > > >> netgraph to have a limitation of 65530 eiface devices off a single > >> if_bridge, but are allowed multiple bridges with that many devices. > >> > >> The problems that you run into with that many devices is that if all > >> the interfaces are visible to a single jail or single host... your "ifconfig" > >> command could take several hours (about 4) to enumerate each iface to > >> the screen. > >> > >> I didn't mess much with epair because it failed to produce a > >> situation where I could speak separate subnets over the same wire. > >> Netgraph made it easy by way of being able to enable promiscuous and > >> disable the "autosrc" feature (as you perhaps already found in my code > you linked to above). > >> -- > >> Cheers, > >> Devin > >> > > > Thanks for the response. > > I have since created a setup with epair, only to abandon it and pursue a setup > with netgraph instead. I can't yet say which will best serve my needs, I can > get back to that when I have more data. > > I do know that shutting down a jail that has epairs enabled very likely will > panic the kernel. I'm not certain that netgraph is any different, but I have no > data yey. I do know that some fixes have been made to kernel to avoid > crashes. > > I'll get back with more info as I have more info to reveal. :) > In my experience (which has been with 8.1, 8.3, 8.4, stable/8, 9.0, 9.1, 9.2 and stable/9) is that when you shut down a jail that still has a netgraph eiface in it is that (if the management script didn't reclaim the interface properly using "ifconfig IFACE -vnet JID") the eiface is still active but not visible to any jail nor to the host machine (read: no crash). You can still see the eiface with "ngctl ls -l" and can do "shutdown" commands on it without a panic. You can also (if you know the JID) manually reclaim it (also without panic). -- Devin _____________ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?065e01cf4cf4$7e0bd6b0$7a238410$>