Date: Tue, 09 Aug 2005 14:41:43 +0200 From: Andre Oppermann <andre@freebsd.org> To: Marko Zec <zec@icir.org> Cc: freebsd-net@freebsd.org Subject: Re: running out of mbufs? Message-ID: <42F8A487.67183CA6@freebsd.org> References: <1123040973.95445.TMDA@seddon.ca> <200508081757.47499.zec@icir.org> <42F78C87.5EB79CBC@freebsd.org> <200508091104.06572.zec@icir.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Marko Zec wrote: > > On Monday 08 August 2005 18:47, Andre Oppermann wrote: > > Marko Zec wrote: > > > On Monday 08 August 2005 12:32, Andre Oppermann wrote: > > > > Dave+Seddon wrote: > > > > > BTW, I'd be interested to know people's thoughts on multiple IP > > > > > stacks on FreeBSD. It would be really cool to be able to give > > > > > a jail it's own IP stack bound to a VLAN interface. It could > > > > > then be like a VRF on Cisco. > > > > > > > > There is a patch doing that for FreeBSD 4.x. However while > > > > interesting it is not the way to go. You don't want to have > > > > multiple parallel stacks but just multiple routing tables and > > > > interface groups one per jail. This gives you the same > > > > functionality as Cisco VRF but is far less intrusive to the > > > > kernel. > > > > > > Andre, > > > > > > the stack virtualization framework for 4.x is based precisely on > > > introducing multiple routing tables and interface groups. In order > > > to cleanly implement support for multiple independent interface > > > groups, one has to touch both the link and network layers, not > > > forgetting the ARP stuff... and in no time you have ended up with a > > > huge and intrusive diff against the original network stack code. > > > > While your stack indexing approach is interesting I don't think it is > > the way we should take for the generic FreeBSD. There are better > > ways to implement a less limiting superset of the desired > > functionality. > > Andre, > > there's no doubt almost any idea or particularly software can be > improved. Could you provide a more elaborate argumentation to your > claim the network stack cloning concept is so severely limited that it > has no place to search for in the future of FreeBSD? And what exactly > did you mean by a "stack indexing approach"? I'm not saying your concept is wrong or doesn't have its place. However there are other approaches doing 98% of what people want to do with less intrusive code changes. > > And the ARP is almost done, I have to review the code > > and then it goes into -current. > > While having a per-interface ARP logic is certainly a nice feature, this > alone will not solve much with regards to introducing multiple > independent interface groups. You will still most probably have to > revisit the ARP code once you start introducing non-global interface > lists in the kernel. I don't want to have non-global interface lists in the kernel. What I want to provide is not exactly what your stack virtualizations does. In fact my work does not preclude virtualization like yours on top of it. It's solving a somewhat different problem set in a different, architectual clean way. That's way I said we should wait for my paper before going to too deep into discussions yet. ;) > > > So I see no point in pretending we could get such a functionality > > > for free, i.e. with only a negligible intrusiveness to the kernel > > > code. A more appropriate question would be whether the potential > > > benefits of having multiple stack state instances could outweight > > > the trouble and damage associated with the scope of required > > > modifications to the kernel code tree. Only if we could get an > > > affirmative answer to that question it would make sense to start > > > thinking / debating on the most appropriate methodology to > > > (re)implement the multiple stacks framework. > > > > Having multiple stacks duplicates a lot of structures for each stack > > which don't have to be duplicated. With your approach you need a new > > jail for every new stack. In each jail you have to run a new > > instance of a routing daemon (if you do routing). And it precludes > > having one routing daemon managing multiple routing tables. While > > removing one limitation you create some new ones in addition to the > > complexity. > > Bemusingly, none of the above claims are true. > > A new jail for each network stack instance is NOT required. Inside the > kernel what could be considered "per-jail" and per-network stack > structures are cleanly separated and independent. In fact, one can run > multiple jails bound to a single network stack instance, if desired. Ok. > Furthermore, a single process can simultaneously attach to multiple > network stacks, thus potentially allowing a single routing daemon to > manage multiple separated routing tables and interface groups. The > entity that gets permanently bound to a network stack instance is a > socket and not a process. This translates to the capability of a single > process to open multiple sockets in multiple independent stacks. IMO, > one particular strength of such an approach is that it requires > absolutely no extensions or modifications to the existing routing > socket API. The existing API should be modified, it is pretty out of date. > And finally, I'm wondering what structures exactly are you referring to > when you say that this approach "duplicates a lot of structures for > each stack which don't have to be duplicated"? I absolutely agree that > the virtualization of the network stack should be done as simple and > non-intrusive as possible, but my point is that it just cannot be done > cleanly / properly without taking some sacrifices in terms of the scope > of minimum required modifications. Multiple interface lists, vm zones, etc. as your FAQ spells out. Again, I think we are talking past each other right now and we have different solutions to different problem sets in mind (or already coded). When I have my paper finished my vision and intentions should be more clear and then we can have the discussion on the merits of each approach and whether parts of each are complementary or converse. -- Andre > Cheers, > > Marko > > > What I have in mind is a redesign of some parts of the stack as such. > > I have got funding to work on this through my fundraise of two weeks > > ago. Work is about to start later this week. I will provide a more > > detailed design document for discussion and review by the FreeBSD > > community in a few days. It will include features such as multiple > > hierarchical routing tables (bound to jails or not), interface > > groups, virtual interfaces belonging to one or more interface groups, > > policy routing (based on source address/interface), multi-path > > routing, a reworked routing-socket and some more stuff. > > > > I recommend waiting for the design document for further discussions > > and jumping to any conclusions. :)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?42F8A487.67183CA6>