Date: Mon, 11 Jan 2016 14:25:52 +0200 From: Dan Partelly <dan_partelly@rdsor.ro> To: Jordan Hubbard <jordanhubbard@icloud.com> Cc: Peter Beckman <beckman@angryox.com>, FreeBSD Hackers <freebsd-hackers@freebsd.org>, Jonathan de Boyne Pollard <J.deBoynePollard-newsgroups@ntlworld.com>, Dmitry Sivachenko <trtrmitya@gmail.com>, Mark Heily <mark@heily.com> Subject: Re: relaunchd: a portable clone of launchd Message-ID: <E85C42D4-963B-4632-9182-E591A80D1306@rdsor.ro> In-Reply-To: <70975696-3E07-48B9-BFD1-3C2F51E715BB@icloud.com> References: <5687D3A9.5050400@NTLWorld.com> <CAGfo=8kXzNVKy9gx0jkME4iRRyrgrsfpPnW3nYrZC0gysapPcg@mail.gmail.com> <817860B6-5D67-41A3-ADD7-9757C7E67C35@gmail.com> <alpine.BSF.2.20.1601081020270.34827@nog2.angryox.com> <07D83705-D89F-4125-B57B-920EDEBC8A85@rdsor.ro> <70975696-3E07-48B9-BFD1-3C2F51E715BB@icloud.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> It=E2=80=99s this latter point that makes me roll my eyes a bit = whenever folks use phrases like =E2=80=9Cthe linux way=E2=80=9D or = =E2=80=9Cthe BSD way=E2=80=9D since I=E2=80=99m not entirely sure that = those =E2=80=9Cways=E2=80=9D, at least not as I first heard them = articulated back in the 1990=E2=80=99s, actually exist anymore. I personally use those terms a bit tongue in check. BSDs have very = limited manpower , and so they are forced to use source code from = foreign OSes in the most straightforward way possible, which means = wrappers . (else see the fiasco with the DRM drivers in FreeBSD so far, = there is plainly too much effort to port Linux DRM code to BSD without = wrappers and adapters. Dragonfly did it the right way ) . So , you need = to wrappers and adapters to Linux specific API=E2=80=99s and data = structures to support DRM , OFED and god knows what else.=20 But this is a general problem not only limited to Linux.=20 FreeBSD has NDIS emulation to support some Wifi devices.=20 FreeBSD has Solaris API/ datastruct wrappers and adapters to support = ZFS/ NextBSD has now dragged =E2=80=9Chalf" of mach kernel inside to : 1. = Implement a better and more flexible IPC mechanism than Unix domain = sockets / posix message queues (Truth is,=20 IPC sucks in Linux and BSDs. It is my opinion that Windows LPC/ALPC are = way better designed. and 2: To easily port launchd/notifyd/whateverd=20 Now this is all good and dandy, but who will audit all this code for = bugs and security ? Some bugs will become manifest fast, but others may = lurk in those layers for years. > What problems are we here to solve, and what are the specific details = of those problems?=E2=80=9D It is my oppinion that FreeBSD needs to solve the following techical = problems in the future: =E2=80=94 problem : binary code reuse --- 1. Many utilites from BSD base are monolithic and closed, yet they = provide higher level interfaces to OS configuration. It is my opinion = that=20 all this functionality should be expose as libraries & demon services. = This is the very base upon which enterprise grade tools are built. = Libxo is not a proper solution to this problem. It should not exist in = FreeBSD base, yet somehow it slipped inside, and I seen on some papers = from BSD conferences that some even discussed to put it inside the OS = kernel . I frankly cant imagine what those ppl are/wehre thinking.=20 2. Of secondary importance is to build some demons to allow access to = geom, network , wifi management and so on and implement a proper auto = mounter . Let go to the 80s=20 and not use scaffolding of shell scripts to implement any of those.=20 Those two points also help a great deal towards having softtware = appliances which has only what you need and nothing else on them.=20 =E2=80=94 problem OS configuration and updating in a higly concurrent = world --- 3. Transactional OS wide configuration databases.=20 4. In a world where hundreds of machines, bee-it physical or virtual are = interconnected and talking to each other , safe concurrent access to OS = configuration will become soon very important. =E2=80=94 problem OS wide IPC mechanism to build a high performance = pub/sub system , to cope with the very dynamic world we move forward.=20 4. A new high performance IPC system should be introduced IMO. It should = allow both fast synchronous operation ( like solaris doors) and = asynchronous operations, working with kevent/kqueue. It should allow = kernel endpoints, and should cater to security concerns. It should be a = minimal API , not a full buss 5. a user mode pub / sub message bus. use the IPC api to implement a = high performance , OS wide, message bus. It should present clear = abstractions to clients, to insulate them=20 from using syscalls directly. It should sum some already existing = mechanisms, such as devd. =E2=80=94 problem of service management and fault management: 6. Much needed components of enterprise.=20 =20 Frankly, I like the direction Solaris took with Service management and = faults management.=20 > What I DO know won=E2=80=99t move the ball forward (on any field, in = any game) is arguing about this like it was a religious debate of some = sort, where sweeping generalizations are preferred over far more = pragmatic questions of =E2=80=9CWhat problems are we here to solve, and = what are the specific details of those problems?=E2=80=9D I raise some of those problems in past. Especially the problem of binary = code reuse . My perception was that nobody really gives a damn. The = answers where interesting: libxo is in base because someone coded it , = libxofication of utils proceeds in BSD , no matter that is a half assed = solution which offers a fast hack to appliance vendors like juniper = mostly, Others complained that people talk and nobody contributes code = , yet when you contribute code they dont look at it with months, and so = on. > In my last job, a lot of the questions revolved around =E2=80=9Chow do = we stick this OS into mobile devices with small batteries?=E2=80=9D and = those sorts of pragmatic concerns informed a lot of our engineering = work. =20 Yes. See, One thing I hear for decades is the Unix desktop. And = everybody and their mother started to write Windows managers, desktops , = whatever. We have OSes like PCBSD, which likes to masquerade itself like = a desktop. Yet it seems that nobody realised that a modern desktop , one = which is not a Rupe Goldberg contraption, or even worse, the software = frankenstein=E2=80=99s monster depends heavily on almost all the probem = domains I outlined above. Ironically, the some of the same base = technologies which can be used=20 to driver enteripise features like service management and fault = management, would enable the polar opposite, the user desktop. Yet what we see instead of someone asking those questions is people = jumping into coding layer after layer of half assed solutions, which = only make the situation worse. If people would at least agree on what has to be done, some would surely = study those problems and find some cool solutions to implement.=20 I like your NextBSD effort, save the fact you guys dragged half of Mach = kernel in it. I can understand why you did it, but I dont have to like = it. I think a new slim IPC mechanism=20 should be written , and that the major enterpises depending on BSD = should fork money to cover the development costs. Im happy that they use = BSD techs, but with very few exceptions they contribute almost nothing back.=20 > On 10 Jan 2016, at 20:32, Jordan Hubbard <jordanhubbard@icloud.com> = wrote: >=20 >=20 >> On Jan 10, 2016, at 2:36 AM, Dan Partelly <dan_partelly@rdsor.ro> = wrote: >>=20 >> Copying the linux way of doing things should not be a goal of the = BSDs. >=20 > I=E2=80=99ve been trying to stay out of this since the discussion has, = at least in points, seemed a little on the =E2=80=9Cfnarr! fnarr!=E2=80=9D= polemic side (rather than focusing, as one would hope to see in a group = called =E2=80=9CHackers=E2=80=9D, on the engineering details and = specific technical pros-and-cons of each approach), but I guess I can no = longer resist. As one of the =E2=80=9Claunchd stake-holders=E2=80=9D in = the discussion, and one I hope to have at a conference in the near = future since email is a terrible communications medium for really = getting ones point=E2=80=99s across (and avoiding lots of points that = exist in the mind of the reader but not your own), let me just say that = our goals with NeXTBSD continue to be the following, though not = necessarily always in the following order: >=20 > 1. Stay in sync with FreeBSD-current, just to keep divergence to a = minimum >=20 > 2. Get all of the base technologies we are targeting (launchd et al) = fully working and as faithful in implementation to their upstream = origins (for the exact same reasons as #1) as possible. We=E2=80=99re = not slavishly following Apple, we=E2=80=99ll take any suitably licensed = technology that achieves our goals (see point #4). >=20 > 3. Start migrating all of the older facilities, like /etc/rc and = preferences files, to the new models - this one is really the =E2=80=9Clon= g pole=E2=80=9D and why we=E2=80=99re just staying quiet until this work = is largely completed and ready to show, since otherwise you=E2=80=99re = largely debating the pros and cons of vaporware vs vaporware. >=20 > 4. Use the specific and pragmatic world of Enterprise and Software = Appliance requirements to drive the feature set of the technologies we = choose and the urgency with which we pick them. >=20 > It=E2=80=99s this latter point that makes me roll my eyes a bit = whenever folks use phrases like =E2=80=9Cthe linux way=E2=80=9D or = =E2=80=9Cthe BSD way=E2=80=9D since I=E2=80=99m not entirely sure that = those =E2=80=9Cways=E2=80=9D, at least not as I first heard them = articulated back in the 1990=E2=80=99s, actually exist anymore. >=20 > Yeah, BSD has /usr/src and Linux has collections of packages and a = distro-centric model of looking at the universe, but those are = increasingly superficial distinctions when compared to the far more = pertinent distinctions today around *what* is being packaged and to what = purpose. What do application stacks look like in the year 2016 and = forward? How does mass-deployment work on your OS? Where are the = privilege domains around disparate collections of software drawn, and = how do you manage them? Do you have effective security models for = allowing entirely untrusted software to run on your OS? How are you = managing storage in clustered / hybrid (local + cloud) environments, and = how hard is it to bend the OS to your will as a DevOps person tasked = with any or all of the above? >=20 > I=E2=80=99m not pretending to have answers to even half of those = questions myself, don=E2=80=99t get me wrong, but I think the questions = are important just to staying alive. In my last job, a lot of the = questions revolved around =E2=80=9Chow do we stick this OS into mobile = devices with small batteries?=E2=80=9D and those sorts of pragmatic = concerns informed a lot of our engineering work. My current job asks = questions about Enterprise deployment and how to create Software = Appliances that have everything you need and nothing you don=E2=80=99t = in them, the Linux folks also complicating the picture with = =E2=80=9CContainers=E2=80=9D and =E2=80=9CDocker Apps=E2=80=9D (kudos to = them though - they managed to take a handful of things that had existed = for years and make them suddenly new and sexy again) and now I need = answers for those, too. >=20 > What I DO know won=E2=80=99t move the ball forward (on any field, in = any game) is arguing about this like it was a religious debate of some = sort, where sweeping generalizations are preferred over far more = pragmatic questions of =E2=80=9CWhat problems are we here to solve, and = what are the specific details of those problems?=E2=80=9D >=20 > This is also why I=E2=80=99m doing all of that stuff in a branch = called NextBSD. Who has time for religious debate when you=E2=80=99re = trying to just get code to work and solve a very specific set of = problems? :-) >=20 > - Jordan >=20
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E85C42D4-963B-4632-9182-E591A80D1306>