Date: Fri, 30 Dec 2011 07:39:58 +0100 From: Polytropon <freebsd@edvax.de> To: David Jackson <djackson452@gmail.com> Cc: freebsd-questions@freebsd.org Subject: Re: FreeBSD Kernel Internals Documentation Message-ID: <20111230073958.3949a17b.freebsd@edvax.de> In-Reply-To: <CAGy-%2Bi-NN_SOYrrE6WgHyCBa5VzFexwT_C9UYhO3GyjvfsxpAA@mail.gmail.com> References: <CAGy-%2Bi-NN_SOYrrE6WgHyCBa5VzFexwT_C9UYhO3GyjvfsxpAA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 29 Dec 2011 22:43:16 -0500, David Jackson wrote: > However, My finding is that due to poor documentation, [...] That kind of statement doesn't fit well to FreeBSD which is known for its excellent documentation, often considered superior to other open source projects. > [...] the FreeBSD kernel > is nearly impenetrable to an outsider. I have been able to find no > comprehensive documentation of kernel internals. I have found it nearly > impossible, due to lack of comprehensive documentation, much of any of the > kernel internals. What I see is an internal cliche of developers who are > aware of its myraid of undocumented esoteric secrets, and very little to > actually help anyone else to understand it. The source code itself is very verbose and well documented. Unlike other operating systems, FreeBSD provides man pages that can be accessed offline and provide information about many kernel aspects, such as interfaces and device drivers. Searching "http://www.google.com/search?q=freebsd+kernel+hacker" or something of that kind will also bring further information and and resources. > Any good, well designed software projects will have comprehensive > documentation of the source code, this includes code comments, information > on what every piece of code does, how the entire system fits together, and > descriptions of every variable and function. Any well run project would > insist that code contributors upload full and comprehensive documentation > of how their source code is written, how it works, etc. I think this is what you'll find in the FreeBSD kernel sources. I had a random peek at some files in the /usr/src/sys subtree (where the kernel sources reside) and found that it fits your claim. > Documentation is vital and good practice because it saves time, it prevents > people new to the project having to waste immense amounts of time trying to > figure out a vast and cryptic puzzle. Without good documentation software > can be nearly useless, unmaintainable and difficult for an outsider to > learn, to the point where it may actually take less time to just throw it > out and start from scratch. I couldn't agree more. As a developer, I've worked with many operating systems, and finally I found FreeBSD which provides the documentation in a way I feel comfortable with, unlike its predecessors in my "IT career". :-) However, there are differences in how you judge documentation to be _good_. Talk to a mainframer, and he will tell you a different story. Then talk to a "Windows" person and explain what documentation is, and he'll tell you that you don't need it. :-) > These are reasons that FreeBSD needs better documentation, documentation of > how the entire system fits together, what lines of code do, the purpose of > variables and functions, etc, in descriptive English. This is key to > developing maintainable software. As I said, that _is_ what you already find in the source tree. Allow me to give you one example (out of many): File randomly picked: /usr/src/sys/nfs/nfs_common.c /* * Help break down an mbuf chain by setting the first siz bytes contiguous * pointed to by returned val. * This is used by the macros nfsm_dissect for tough * cases. (The macros use the vars. dpos and dpos2) */ void * nfsm_disct(struct mbuf **mdp, caddr_t *dposp, int siz, int left, int how) { ... and so on. Descriptive english text, usable variable and type names. Or how about /usr/sys/boot/i386/btx/btxldr/btxldr.S /* * BTX program loader for ELF clients. */ start: cld # String ops inc testl $OPT_SET(RBX_MUTE), 4(%esp) # Check first argument setnz muted # for RBX_MUTE, set flag movl $m_logo,%esi # Identify call putstr # ourselves movzwl BDA_MEM,%eax # Get base memory shll $0xa,%eax # in bytes movl %eax,%ebp # Base of user stack #ifdef BTXLDR_VERBOSE movl $m_mem,%esi # Display call hexout # amount of call putstr # base memory #endif lgdt gdtdesc # Load new GDT If that is insufficient, maybe you can give an example for comparison of how you imagine the documentation should be provided. > I saw where someone automatically generated "documentation" with Doxygen. > This is nearly useless, because all it shows is a huge list of functions > and variables but does not include any text on what they do. At best, > Doxygen can only provide a template for documentation that can be filled in > with descriptive English information on what everything does. If properly used Doxygen can help, but it's not the entire solution, I fully agree. Some people need to learn the hard way, just like me when I visited my _own_ source code a decade later and found something like kk[w.q].x = fbx(x,x0+a0+1+aa[q].t,dt,h,dh-1+3); which is totally ugly. :-) > One idea might be to have an official wiki that contains the template > generated by Doxygen which can then be filled in. When changes to the > source code is made, it is good practice for the commiter of such changes > to document their code as it is submitted. Also Wikis aren't for everything. I don't know how _you_ as a developer think about it, but _I_ like to have the documentation available _in_ the code I consult, offline, without the need to use a web browser, everything based on _files_ so I can use the system's powerful tools to work with them (e. g. search for information, filter, create lists and so on). FreeBSD has an official Wiki, and there are several others. Again a question arises: Would you like the documentation to be spread across arbitrary Wikis, user home pages, web forums and blogs? > This allows others who come along who need to maintain the code to more > easily understand what the code does. History teaches that this is achieved by using the code itself. :-) > Another idea which would also improve the useability of FreeBSD would be to > have a wiki which would be updated by kernel contributors whenever they add > support for a certain piece of hardware. This would make finding hardware > compatability information easier from one central, up to date and current > source of information. Those informations will typically be added to the UPDATING information, as well as to the release notes of an upcoming RELEASE. Deltas in the src/ subtree also indicate when something new is present. As FreeBSD is a _quality_ operating system, developers pay attention to write the documentation along with releasing new drivers, so you aren't stuck with an ugly-hacked kernel module full of lines that look like the fbx() example I gave above. :-) > These documentaiton ideas, for commiters to document their code when they > upload it, and document their hardware support additions, are just good > software practices that should be highly recommended and encouraged I fully agree with that statement. If you could, for example, point to documentation you judge _good_, or point at some FreeBSD documentation that you think needs heavy improvement, it would help to back your argumentation. Otherwise, it can be seen as "already done". -- Polytropon Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ...
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20111230073958.3949a17b.freebsd>