From owner-svn-src-vendor@FreeBSD.ORG Sat Feb 27 20:19:06 2010 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9E1CB106564A; Sat, 27 Feb 2010 20:19:06 +0000 (UTC) (envelope-from raj@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 89CFE8FC08; Sat, 27 Feb 2010 20:19:06 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o1RKJ6wZ063038; Sat, 27 Feb 2010 20:19:06 GMT (envelope-from raj@svn.freebsd.org) Received: (from raj@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o1RKJ6wa063032; Sat, 27 Feb 2010 20:19:06 GMT (envelope-from raj@svn.freebsd.org) Message-Id: <201002272019.o1RKJ6wa063032@svn.freebsd.org> From: Rafal Jaworowski Date: Sat, 27 Feb 2010 20:19:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r204431 - in vendor/dtc: . dist dist/Documentation dist/libfdt dist/scripts dist/tests X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 20:19:06 -0000 Author: raj Date: Sat Feb 27 20:19:06 2010 New Revision: 204431 URL: http://svn.freebsd.org/changeset/base/204431 Log: Initial import of the Device Tree Compiler (DTC) package v1.2.0. Obtained from: git://git.jdl.com/software/dtc.git Added: vendor/dtc/ vendor/dtc/dist/ vendor/dtc/dist/Documentation/ vendor/dtc/dist/Documentation/dtc-paper.bib vendor/dtc/dist/Documentation/dtc-paper.tex vendor/dtc/dist/Documentation/dts-format.txt (contents, props changed) vendor/dtc/dist/Documentation/manual.txt (contents, props changed) vendor/dtc/dist/GPL vendor/dtc/dist/Makefile (contents, props changed) vendor/dtc/dist/Makefile.dtc (contents, props changed) vendor/dtc/dist/README.license vendor/dtc/dist/TODO vendor/dtc/dist/checks.c (contents, props changed) vendor/dtc/dist/convert-dtsv0-lexer.l vendor/dtc/dist/data.c (contents, props changed) vendor/dtc/dist/dtc-lexer.l vendor/dtc/dist/dtc-parser.y vendor/dtc/dist/dtc.c (contents, props changed) vendor/dtc/dist/dtc.h (contents, props changed) vendor/dtc/dist/flattree.c (contents, props changed) vendor/dtc/dist/fstree.c (contents, props changed) vendor/dtc/dist/ftdump.c (contents, props changed) vendor/dtc/dist/libfdt/ vendor/dtc/dist/libfdt/Makefile.libfdt (contents, props changed) vendor/dtc/dist/libfdt/TODO vendor/dtc/dist/libfdt/fdt.c (contents, props changed) vendor/dtc/dist/libfdt/fdt.h (contents, props changed) vendor/dtc/dist/libfdt/fdt_ro.c (contents, props changed) vendor/dtc/dist/libfdt/fdt_rw.c (contents, props changed) vendor/dtc/dist/libfdt/fdt_strerror.c (contents, props changed) vendor/dtc/dist/libfdt/fdt_sw.c (contents, props changed) vendor/dtc/dist/libfdt/fdt_wip.c (contents, props changed) vendor/dtc/dist/libfdt/libfdt.h (contents, props changed) vendor/dtc/dist/libfdt/libfdt_env.h (contents, props changed) vendor/dtc/dist/libfdt/libfdt_internal.h (contents, props changed) vendor/dtc/dist/livetree.c (contents, props changed) vendor/dtc/dist/scripts/ vendor/dtc/dist/scripts/setlocalversion (contents, props changed) vendor/dtc/dist/srcpos.c (contents, props changed) vendor/dtc/dist/srcpos.h (contents, props changed) vendor/dtc/dist/tests/ vendor/dtc/dist/tests/Makefile.tests (contents, props changed) vendor/dtc/dist/tests/add_subnode_with_nops.c (contents, props changed) vendor/dtc/dist/tests/bad-empty-ranges.dts vendor/dtc/dist/tests/bad-name-property.dts vendor/dtc/dist/tests/bad-ncells.dts vendor/dtc/dist/tests/bad-reg-ranges.dts vendor/dtc/dist/tests/bad-string-props.dts vendor/dtc/dist/tests/base01.asm vendor/dtc/dist/tests/base01.cmd vendor/dtc/dist/tests/base01.dts vendor/dtc/dist/tests/base01.stderr vendor/dtc/dist/tests/boot-cpuid.c (contents, props changed) vendor/dtc/dist/tests/comments-cmp.dts vendor/dtc/dist/tests/comments.dts vendor/dtc/dist/tests/default-addr-size.dts vendor/dtc/dist/tests/del_node.c (contents, props changed) vendor/dtc/dist/tests/del_property.c (contents, props changed) vendor/dtc/dist/tests/dtbs_equal_ordered.c (contents, props changed) vendor/dtc/dist/tests/dtc-checkfails.sh (contents, props changed) vendor/dtc/dist/tests/dtc-fatal.sh (contents, props changed) vendor/dtc/dist/tests/dumptrees.c (contents, props changed) vendor/dtc/dist/tests/dup-nodename.dts vendor/dtc/dist/tests/dup-phandle.dts vendor/dtc/dist/tests/dup-propname.dts vendor/dtc/dist/tests/empty.dts vendor/dtc/dist/tests/escapes.dts vendor/dtc/dist/tests/find_property.c (contents, props changed) vendor/dtc/dist/tests/get_mem_rsv.c (contents, props changed) vendor/dtc/dist/tests/get_name.c (contents, props changed) vendor/dtc/dist/tests/get_path.c (contents, props changed) vendor/dtc/dist/tests/get_phandle.c (contents, props changed) vendor/dtc/dist/tests/getprop.c (contents, props changed) vendor/dtc/dist/tests/incbin.bin (contents, props changed) vendor/dtc/dist/tests/incbin.c (contents, props changed) vendor/dtc/dist/tests/incbin.dts vendor/dtc/dist/tests/include0.dts vendor/dtc/dist/tests/include1.dts vendor/dtc/dist/tests/include2.dts vendor/dtc/dist/tests/include3.dts vendor/dtc/dist/tests/include4.dts vendor/dtc/dist/tests/include5.dts vendor/dtc/dist/tests/include6.dts vendor/dtc/dist/tests/include7.dts vendor/dtc/dist/tests/include8.dts vendor/dtc/dist/tests/label01.dts vendor/dtc/dist/tests/mangle-layout.c (contents, props changed) vendor/dtc/dist/tests/mangle-layout.supp vendor/dtc/dist/tests/minusone-phandle.dts vendor/dtc/dist/tests/move_and_save.c (contents, props changed) vendor/dtc/dist/tests/node_check_compatible.c (contents, props changed) vendor/dtc/dist/tests/node_offset_by_compatible.c (contents, props changed) vendor/dtc/dist/tests/node_offset_by_phandle.c (contents, props changed) vendor/dtc/dist/tests/node_offset_by_prop_value.c (contents, props changed) vendor/dtc/dist/tests/nonexist-label-ref.dts vendor/dtc/dist/tests/nonexist-node-ref.dts vendor/dtc/dist/tests/nop_node.c (contents, props changed) vendor/dtc/dist/tests/nop_property.c (contents, props changed) vendor/dtc/dist/tests/nopulate.c (contents, props changed) vendor/dtc/dist/tests/notfound.c (contents, props changed) vendor/dtc/dist/tests/obsolete-chosen-interrupt-controller.dts vendor/dtc/dist/tests/open_pack.c (contents, props changed) vendor/dtc/dist/tests/open_pack.supp vendor/dtc/dist/tests/parent_offset.c (contents, props changed) vendor/dtc/dist/tests/path-references.c (contents, props changed) vendor/dtc/dist/tests/path-references.dts vendor/dtc/dist/tests/path_offset.c (contents, props changed) vendor/dtc/dist/tests/prop-after-subnode.dts vendor/dtc/dist/tests/references.c (contents, props changed) vendor/dtc/dist/tests/references.dts vendor/dtc/dist/tests/references_dts0.dts vendor/dtc/dist/tests/reg-ranges-root.dts vendor/dtc/dist/tests/root_node.c (contents, props changed) vendor/dtc/dist/tests/run_tests.sh (contents, props changed) vendor/dtc/dist/tests/rw_tree1.c (contents, props changed) vendor/dtc/dist/tests/set_name.c (contents, props changed) vendor/dtc/dist/tests/setprop.c (contents, props changed) vendor/dtc/dist/tests/setprop_inplace.c (contents, props changed) vendor/dtc/dist/tests/string_escapes.c (contents, props changed) vendor/dtc/dist/tests/subnode_offset.c (contents, props changed) vendor/dtc/dist/tests/supernode_atdepth_offset.c (contents, props changed) vendor/dtc/dist/tests/sw_tree1.c (contents, props changed) vendor/dtc/dist/tests/test01.asm vendor/dtc/dist/tests/test01.dts vendor/dtc/dist/tests/test01.stderr vendor/dtc/dist/tests/test_kernel_dts (contents, props changed) vendor/dtc/dist/tests/test_tree1.dts vendor/dtc/dist/tests/test_tree1_dts0.dts vendor/dtc/dist/tests/testdata.h (contents, props changed) vendor/dtc/dist/tests/tests.h (contents, props changed) vendor/dtc/dist/tests/tests.sh (contents, props changed) vendor/dtc/dist/tests/testutils.c (contents, props changed) vendor/dtc/dist/tests/trees.S (contents, props changed) vendor/dtc/dist/tests/truncated_property.c (contents, props changed) vendor/dtc/dist/tests/zero-phandle.dts vendor/dtc/dist/treesource.c (contents, props changed) Added: vendor/dtc/dist/Documentation/dtc-paper.bib ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/dtc/dist/Documentation/dtc-paper.bib Sat Feb 27 20:19:06 2010 (r204431) @@ -0,0 +1,43 @@ +@STRING{pub-IEEE = "IEEE Computer Society"} +@STRING{pub-IEEE:adr = "345 E. 47th St, New York, NY 10017, USA"} + +@BOOK{IEEE1275, + key = "IEEE1275", + title = "{IEEE} {S}tandard for {B}oot ({I}nitialization {C}onfiguration) {F}irmware: {C}ore {R}equirements and {P}ractices", + publisher = pub-IEEE, + address = pub-IEEE:adr, + series = "IEEE Std 1275-1994", + year = 1994, +} + +@BOOK{IEEE1275-pci, + key = "IEEE1275-pci", + title = "{PCI} {B}us {B}inding to: {IEEE} {S}td 1275-1994 {S}tandard for {B}oot ({I}nitialization {C}onfiguration) {F}irmware", + publisher = pub-IEEE, + address = pub-IEEE:adr, + note = "Revision 2.1", + year = 1998, +} + +@MISC{noof1, + author = "Benjamin Herrenschmidt", + title = "Booting the {L}inux/ppc kernel without {O}pen {F}irmware", + month = may, + year = 2005, + note = "v0.1, \url{http://ozlabs.org/pipermail/linuxppc64-dev/2005-May/004073.html}", +} + +@MISC{noof5, + author = "Benjamin Herrenschmidt", + title = "Booting the {L}inux/ppc kernel without {O}pen {F}irmware", + month = nov, + year = 2005, + note = "v0.5, \url{http://ozlabs.org/pipermail/linuxppc64-dev/2005-December/006994.html}", +} + +@MISC{dtcgit, + author = "David Gibson et al.", + title = "\dtc{}", + howpublished = "git tree", + note = "\url{http://ozlabs.org/~dgibson/dtc/dtc.git}", +} Added: vendor/dtc/dist/Documentation/dtc-paper.tex ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/dtc/dist/Documentation/dtc-paper.tex Sat Feb 27 20:19:06 2010 (r204431) @@ -0,0 +1,597 @@ +\documentclass[a4paper,twocolumn]{article} + +\usepackage{abstract} +\usepackage{xspace} +\usepackage{amssymb} +\usepackage{latexsym} +\usepackage{tabularx} +\usepackage[T1]{fontenc} +\usepackage{calc} +\usepackage{listings} +\usepackage{color} +\usepackage{url} + +\title{Device trees everywhere} + +\author{David Gibson \texttt{<{dwg}{@}{au1.ibm.com}>}\\ + Benjamin Herrenschmidt \texttt{<{benh}{@}{kernel.crashing.org}>}\\ + \emph{OzLabs, IBM Linux Technology Center}} + +\newcommand{\R}{\textsuperscript{\textregistered}\xspace} +\newcommand{\tm}{\textsuperscript{\texttrademark}\xspace} +\newcommand{\tge}{$\geqslant$} +%\newcommand{\ditto}{\textquotedbl\xspace} + +\newcommand{\fixme}[1]{$\bigstar$\emph{\textbf{\large #1}}$\bigstar$\xspace} + +\newcommand{\ppc}{\mbox{PowerPC}\xspace} +\newcommand{\of}{Open Firmware\xspace} +\newcommand{\benh}{Ben Herrenschmidt\xspace} +\newcommand{\kexec}{\texttt{kexec()}\xspace} +\newcommand{\dtbeginnode}{\texttt{OF\_DT\_BEGIN\_NODE\xspace}} +\newcommand{\dtendnode}{\texttt{OF\_DT\_END\_NODE\xspace}} +\newcommand{\dtprop}{\texttt{OF\_DT\_PROP\xspace}} +\newcommand{\dtend}{\texttt{OF\_DT\_END\xspace}} +\newcommand{\dtc}{\texttt{dtc}\xspace} +\newcommand{\phandle}{\texttt{linux,phandle}\xspace} +\begin{document} + +\maketitle + +\begin{abstract} + We present a method for booting a \ppc{}\R Linux\R kernel on an + embedded machine. To do this, we supply the kernel with a compact + flattened-tree representation of the system's hardware based on the + device tree supplied by Open Firmware on IBM\R servers and Apple\R + Power Macintosh\R machines. + + The ``blob'' representing the device tree can be created using \dtc + --- the Device Tree Compiler --- that turns a simple text + representation of the tree into the compact representation used by + the kernel. The compiler can produce either a binary ``blob'' or an + assembler file ready to be built into a firmware or bootwrapper + image. + + This flattened-tree approach is now the only supported method of + booting a \texttt{ppc64} kernel without Open Firmware, and we plan + to make it the only supported method for all \texttt{powerpc} + kernels in the future. +\end{abstract} + +\section{Introduction} + +\subsection{OF and the device tree} + +Historically, ``everyday'' \ppc machines have booted with the help of +\of (OF), a firmware environment defined by IEEE1275 \cite{IEEE1275}. +Among other boot-time services, OF maintains a device tree that +describes all of the system's hardware devices and how they're +connected. During boot, before taking control of memory management, +the Linux kernel uses OF calls to scan the device tree and transfer it +to an internal representation that is used at run time to look up +various device information. + +The device tree consists of nodes representing devices or +buses\footnote{Well, mostly. There are a few special exceptions.}. +Each node contains \emph{properties}, name--value pairs that give +information about the device. The values are arbitrary byte strings, +and for some properties, they contain tables or other structured +information. + +\subsection{The bad old days} + +Embedded systems, by contrast, usually have a minimal firmware that +might supply a few vital system parameters (size of RAM and the like), +but nothing as detailed or complete as the OF device tree. This has +meant that the various 32-bit \ppc embedded ports have required a +variety of hacks spread across the kernel to deal with the lack of +device tree. These vary from specialised boot wrappers to parse +parameters (which are at least reasonably localised) to +CONFIG-dependent hacks in drivers to override normal probe logic with +hardcoded addresses for a particular board. As well as being ugly of +itself, such CONFIG-dependent hacks make it hard to build a single +kernel image that supports multiple embedded machines. + +Until relatively recently, the only 64-bit \ppc machines without OF +were legacy (pre-POWER5\R) iSeries\R machines. iSeries machines often +only have virtual IO devices, which makes it quite simple to work +around the lack of a device tree. Even so, the lack means the iSeries +boot sequence must be quite different from the pSeries or Macintosh, +which is not ideal. + +The device tree also presents a problem for implementing \kexec. When +the kernel boots, it takes over full control of the system from OF, +even re-using OF's memory. So, when \kexec comes to boot another +kernel, OF is no longer around for the second kernel to query. + +\section{The Flattened Tree} + +In May 2005 \benh implemented a new approach to handling the device +tree that addresses all these problems. When booting on OF systems, +the first thing the kernel runs is a small piece of code in +\texttt{prom\_init.c}, which executes in the context of OF. This code +walks the device tree using OF calls, and transcribes it into a +compact, flattened format. The resulting device tree ``blob'' is then +passed to the kernel proper, which eventually unflattens the tree into +its runtime form. This blob is the only data communicated between the +\texttt{prom\_init.c} bootstrap and the rest of the kernel. + +When OF isn't available, either because the machine doesn't have it at +all or because \kexec has been used, the kernel instead starts +directly from the entry point taking a flattened device tree. The +device tree blob must be passed in from outside, rather than generated +by part of the kernel from OF. For \kexec, the userland +\texttt{kexec} tools build the blob from the runtime device tree +before invoking the new kernel. For embedded systems the blob can +come either from the embedded bootloader, or from a specialised +version of the \texttt{zImage} wrapper for the system in question. + +\subsection{Properties of the flattened tree} + +The flattened tree format should be easy to handle, both for the +kernel that parses it and the bootloader that generates it. In +particular, the following properties are desirable: + +\begin{itemize} +\item \emph{relocatable}: the bootloader or kernel should be able to + move the blob around as a whole, without needing to parse or adjust + its internals. In practice that means we must not use pointers + within the blob. +\item \emph{insert and delete}: sometimes the bootloader might want to + make tweaks to the flattened tree, such as deleting or inserting a + node (or whole subtree). It should be possible to do this without + having to effectively regenerate the whole flattened tree. In + practice this means limiting the use of internal offsets in the blob + that need recalculation if a section is inserted or removed with + \texttt{memmove()}. +\item \emph{compact}: embedded systems are frequently short of + resources, particularly RAM and flash memory space. Thus, the tree + representation should be kept as small as conveniently possible. +\end{itemize} + +\subsection{Format of the device tree blob} +\label{sec:format} + +\begin{figure}[htb!] + \centering + \footnotesize + \begin{tabular}{r|c|l} + \multicolumn{1}{r}{\textbf{Offset}}& \multicolumn{1}{c}{\textbf{Contents}} \\\cline{2-2} + \texttt{0x00} & \texttt{0xd00dfeed} & magic number \\\cline{2-2} + \texttt{0x04} & \emph{totalsize} \\\cline{2-2} + \texttt{0x08} & \emph{off\_struct} & \\\cline{2-2} + \texttt{0x0C} & \emph{off\_strs} & \\\cline{2-2} + \texttt{0x10} & \emph{off\_rsvmap} & \\\cline{2-2} + \texttt{0x14} & \emph{version} \\\cline{2-2} + \texttt{0x18} & \emph{last\_comp\_ver} & \\\cline{2-2} + \texttt{0x1C} & \emph{boot\_cpu\_id} & \tge v2 only\\\cline{2-2} + \texttt{0x20} & \emph{size\_strs} & \tge v3 only\\\cline{2-2} + \multicolumn{1}{r}{\vdots} & \multicolumn{1}{c}{\vdots} & \\\cline{2-2} + \emph{off\_rsvmap} & \emph{address0} & memory reserve \\ + + \texttt{0x04} & ...& table \\\cline{2-2} + + \texttt{0x08} & \emph{len0} & \\ + + \texttt{0x0C} & ...& \\\cline{2-2} + \vdots & \multicolumn{1}{c|}{\vdots} & \\\cline{2-2} + & \texttt{0x00000000}- & end marker\\ + & \texttt{00000000} & \\\cline{2-2} + & \texttt{0x00000000}- & \\ + & \texttt{00000000} & \\\cline{2-2} + \multicolumn{1}{r}{\vdots} & \multicolumn{1}{c}{\vdots} & \\\cline{2-2} + \emph{off\_strs} & \texttt{'n' 'a' 'm' 'e'} & strings block \\ + + \texttt{0x04} & \texttt{~0~ 'm' 'o' 'd'} & \\ + + \texttt{0x08} & \texttt{'e' 'l' ~0~ \makebox[\widthof{~~~}]{\textrm{...}}} & \\ + \vdots & \multicolumn{1}{c|}{\vdots} & \\\cline{2-2} + \multicolumn{1}{r}{+ \emph{size\_strs}} \\ + \multicolumn{1}{r}{\vdots} & \multicolumn{1}{c}{\vdots} & \\\cline{2-2} + \emph{off\_struct} & \dtbeginnode & structure block \\\cline{2-2} + + \texttt{0x04} & \texttt{'/' ~0~ ~0~ ~0~} & root node\\\cline{2-2} + + \texttt{0x08} & \dtprop & \\\cline{2-2} + + \texttt{0x0C} & \texttt{0x00000005} & ``\texttt{model}''\\\cline{2-2} + + \texttt{0x10} & \texttt{0x00000008} & \\\cline{2-2} + + \texttt{0x14} & \texttt{'M' 'y' 'B' 'o'} & \\ + + \texttt{0x18} & \texttt{'a' 'r' 'd' ~0~} & \\\cline{2-2} + \vdots & \multicolumn{1}{c|}{\vdots} & \\\cline{2-2} + & \texttt{\dtendnode} \\\cline{2-2} + & \texttt{\dtend} \\\cline{2-2} + \multicolumn{1}{r}{\vdots} & \multicolumn{1}{c}{\vdots} & \\\cline{2-2} + \multicolumn{1}{r}{\emph{totalsize}} \\ + \end{tabular} + \caption{Device tree blob layout} + \label{fig:blob-layout} +\end{figure} + +The format for the blob we devised, was first described on the +\texttt{linuxppc64-dev} mailing list in \cite{noof1}. The format has +since evolved through various revisions, and the current version is +included as part of the \dtc (see \S\ref{sec:dtc}) git tree, +\cite{dtcgit}. + +Figure \ref{fig:blob-layout} shows the layout of the blob of data +containing the device tree. It has three sections of variable size: +the \emph{memory reserve table}, the \emph{structure block} and the +\emph{strings block}. A small header gives the blob's size and +version and the locations of the three sections, plus a handful of +vital parameters used during early boot. + +The memory reserve map section gives a list of regions of memory that +the kernel must not use\footnote{Usually such ranges contain some data +structure initialised by the firmware that must be preserved by the +kernel.}. The list is represented as a simple array of (address, +size) pairs of 64 bit values, terminated by a zero size entry. The +strings block is similarly simple, consisting of a number of +null-terminated strings appended together, which are referenced from +the structure block as described below. + +The structure block contains the device tree proper. Each node is +introduced with a 32-bit \dtbeginnode tag, followed by the node's name +as a null-terminated string, padded to a 32-bit boundary. Then +follows all of the properties of the node, each introduced with a +\dtprop tag, then all of the node's subnodes, each introduced with +their own \dtbeginnode tag. The node ends with an \dtendnode tag, and +after the \dtendnode for the root node is an \dtend tag, indicating +the end of the whole tree\footnote{This is redundant, but included for +ease of parsing.}. The structure block starts with the \dtbeginnode +introducing the description of the root node (named \texttt{/}). + +Each property, after the \dtprop, has a 32-bit value giving an offset +from the beginning of the strings block at which the property name is +stored. Because it's common for many nodes to have properties with +the same name, this approach can substantially reduce the total size +of the blob. The name offset is followed by the length of the +property value (as a 32-bit value) and then the data itself padded to +a 32-bit boundary. + +\subsection{Contents of the tree} +\label{sec:treecontents} + +Having seen how to represent the device tree structure as a flattened +blob, what actually goes into the tree? The short answer is ``the +same as an OF tree''. On OF systems, the flattened tree is +transcribed directly from the OF device tree, so for simplicity we +also use OF conventions for the tree on other systems. + +In many cases a flat tree can be simpler than a typical OF provided +device tree. The flattened tree need only provide those nodes and +properties that the kernel actually requires; the flattened tree +generally need not include devices that the kernel can probe itself. +For example, an OF device tree would normally include nodes for each +PCI device on the system. A flattened tree need only include nodes +for the PCI host bridges; the kernel will scan the buses thus +described to find the subsidiary devices. The device tree can include +nodes for devices where the kernel needs extra information, though: +for example, for ISA devices on a subsidiary PCI/ISA bridge, or for +devices with unusual interrupt routing. + +Where they exist, we follow the IEEE1275 bindings that specify how to +describe various buses in the device tree (for example, +\cite{IEEE1275-pci} describe how to represent PCI devices). The +standard has not been updated for a long time, however, and lacks +bindings for many modern buses and devices. In particular, embedded +specific devices such as the various System-on-Chip buses are not +covered. We intend to create new bindings for such buses, in keeping +with the general conventions of IEEE1275 (a simple such binding for a +System-on-Chip bus was included in \cite{noof5} a revision of +\cite{noof1}). + +One complication arises for representing ``phandles'' in the flattened +tree. In OF, each node in the tree has an associated phandle, a +32-bit integer that uniquely identifies the node\footnote{In practice +usually implemented as a pointer or offset within OF memory.}. This +handle is used by the various OF calls to query and traverse the tree. +Sometimes phandles are also used within the tree to refer to other +nodes in the tree. For example, devices that produce interrupts +generally have an \texttt{interrupt-parent} property giving the +phandle of the interrupt controller that handles interrupts from this +device. Parsing these and other interrupt related properties allows +the kernel to build a complete representation of the system's +interrupt tree, which can be quite different from the tree of bus +connections. + +In the flattened tree, a node's phandle is represented by a special +\phandle property. When the kernel generates a flattened tree from +OF, it adds a \phandle property to each node, containing the phandle +retrieved from OF. When the tree is generated without OF, however, +only nodes that are actually referred to by phandle need to have this +property. + +Another complication arises because nodes in an OF tree have two +names. First they have the ``unit name'', which is how the node is +referred to in an OF path. The unit name generally consists of a +device type followed by an \texttt{@} followed by a \emph{unit +address}. For example \texttt{/memory@0} is the full path of a memory +node at address 0, \texttt{/ht@0,f2000000/pci@1} is the path of a PCI +bus node, which is under a HyperTransport\tm bus node. The form of +the unit address is bus dependent, but is generally derived from the +node's \texttt{reg} property. In addition, nodes have a property, +\texttt{name}, whose value is usually equal to the first path of the +unit name. For example, the nodes in the previous example would have +\texttt{name} properties equal to \texttt{memory} and \texttt{pci}, +respectively. To save space in the blob, the current version of the +flattened tree format only requires the unit names to be present. +When the kernel unflattens the tree, it automatically generates a +\texttt{name} property from the node's path name. + +\section{The Device Tree Compiler} +\label{sec:dtc} + +\begin{figure}[htb!] + \centering + \begin{lstlisting}[frame=single,basicstyle=\footnotesize\ttfamily, + tabsize=3,numbers=left,xleftmargin=2em] +/memreserve/ 0x20000000-0x21FFFFFF; + +/ { + model = "MyBoard"; + compatible = "MyBoardFamily"; + #address-cells = <2>; + #size-cells = <2>; + + cpus { + #address-cells = <1>; + #size-cells = <0>; + PowerPC,970@0 { + device_type = "cpu"; + reg = <0>; + clock-frequency = <5f5e1000>; + timebase-frequency = <1FCA055>; + linux,boot-cpu; + i-cache-size = <10000>; + d-cache-size = <8000>; + }; + }; + + memory@0 { + device_type = "memory"; + memreg: reg = <00000000 00000000 + 00000000 20000000>; + }; + + mpic@0x3fffdd08400 { + /* Interrupt controller */ + /* ... */ + }; + + pci@40000000000000 { + /* PCI host bridge */ + /* ... */ + }; + + chosen { + bootargs = "root=/dev/sda2"; + linux,platform = <00000600>; + interrupt-controller = + < &/mpic@0x3fffdd08400 >; + }; +}; +\end{lstlisting} + \caption{Example \dtc source} + \label{fig:dts} +\end{figure} + +As we've seen, the flattened device tree format provides a convenient +way of communicating device tree information to the kernel. It's +simple for the kernel to parse, and simple for bootloaders to +manipulate. On OF systems, it's easy to generate the flattened tree +by walking the OF maintained tree. However, for embedded systems, the +flattened tree must be generated from scratch. + +Embedded bootloaders are generally built for a particular board. So, +it's usually possible to build the device tree blob at compile time +and include it in the bootloader image. For minor revisions of the +board, the bootloader can contain code to make the necessary tweaks to +the tree before passing it to the booted kernel. + +The device trees for embedded boards are usually quite simple, and +it's possible to hand construct the necessary blob by hand, but doing +so is tedious. The ``device tree compiler'', \dtc{}\footnote{\dtc can +be obtained from \cite{dtcgit}.}, is designed to make creating device +tree blobs easier by converting a text representation of the tree +into the necessary blob. + +\subsection{Input and output formats} + +As well as the normal mode of compiling a device tree blob from text +source, \dtc can convert a device tree between a number of +representations. It can take its input in one of three different +formats: +\begin{itemize} +\item source, the normal case. The device tree is described in a text + form, described in \S\ref{sec:dts}. +\item blob (\texttt{dtb}), the flattened tree format described in + \S\ref{sec:format}. This mode is useful for checking a pre-existing + device tree blob. +\item filesystem (\texttt{fs}), input is a directory tree in the + layout of \texttt{/proc/device-tree} (roughly, a directory for each + node in the device tree, a file for each property). This is useful + for building a blob for the device tree in use by the currently + running kernel. +\end{itemize} + +In addition, \dtc can output the tree in one of three different +formats: +\begin{itemize} +\item blob (\texttt{dtb}), as in \S\ref{sec:format}. The most + straightforward use of \dtc is to compile from ``source'' to + ``blob'' format. +\item source (\texttt{dts}), as in \S\ref{sec:dts}. If used with blob + input, this allows \dtc to act as a ``decompiler''. +\item assembler source (\texttt{asm}). \dtc can produce an assembler + file, which will assemble into a \texttt{.o} file containing the + device tree blob, with symbols giving the beginning of the blob and + its various subsections. This can then be linked directly into a + bootloader or firmware image. +\end{itemize} + +For maximum applicability, \dtc can both read and write any of the +existing revisions of the blob format. When reading, \dtc takes the +version from the blob header, and when writing it takes a command line +option specifying the desired version. It automatically makes any +necessary adjustments to the tree that are necessary for the specified +version. For example, formats before 0x10 require each node to have +an explicit \texttt{name} property. When \dtc creates such a blob, it +will automatically generate \texttt{name} properties from the unit +names. + +\subsection{Source format} +\label{sec:dts} + +The ``source'' format for \dtc is a text description of the device +tree in a vaguely C-like form. Figure \ref{fig:dts} shows an +example. The file starts with \texttt{/memreserve/} directives, which +gives address ranges to add to the output blob's memory reserve table, +then the device tree proper is described. + +Nodes of the tree are introduced with the node name, followed by a +\texttt{\{} ... \texttt{\};} block containing the node's properties +and subnodes. Properties are given as just {\emph{name} \texttt{=} + \emph{value}\texttt{;}}. The property values can be given in any +of three forms: +\begin{itemize} +\item \emph{string} (for example, \texttt{"MyBoard"}). The property + value is the given string, including terminating NULL. C-style + escapes (\verb+\t+, \verb+\n+, \verb+\0+ and so forth) are allowed. +\item \emph{cells} (for example, \texttt{<0 8000 f0000000>}). The + property value is made up of a list of 32-bit ``cells'', each given + as a hex value. +\item \emph{bytestring} (for example, \texttt{[1234abcdef]}). The + property value is given as a hex bytestring. +\end{itemize} + +Cell properties can also contain \emph{references}. Instead of a hex +number, the source can give an ampersand (\texttt{\&}) followed by the +full path to some node in the tree. For example, in Figure +\ref{fig:dts}, the \texttt{/chosen} node has an +\texttt{interrupt-controller} property referring to the interrupt +controller described by the node \texttt{/mpic@0x3fffdd08400}. In the +output tree, the value of the referenced node's phandle is included in +the property. If that node doesn't have an explicit phandle property, +\dtc will automatically create a unique phandle for it. This approach +makes it easy to create interrupt trees without having to explicitly +assign and remember phandles for the various interrupt controller +nodes. + +The \dtc source can also include ``labels'', which are placed on a +particular node or property. For example, Figure \ref{fig:dts} has a +label ``\texttt{memreg}'' on the \texttt{reg} property of the node +\texttt{/memory@0}. When using assembler output, corresponding labels +in the output are generated, which will assemble into symbols +addressing the part of the blob with the node or property in question. +This is useful for the common case where an embedded board has an +essentially fixed device tree with a few variable properties, such as +the size of memory. The bootloader for such a board can have a device +tree linked in, including a symbol referring to the right place in the +blob to update the parameter with the correct value determined at +runtime. + +\subsection{Tree checking} + +Between reading in the device tree and writing it out in the new +format, \dtc performs a number of checks on the tree: +\begin{itemize} +\item \emph{syntactic structure}: \dtc checks that node and property + names contain only allowed characters and meet length restrictions. + It checks that a node does not have multiple properties or subnodes + with the same name. +\item \emph{semantic structure}: In some cases, \dtc checks that + properties whose contents are defined by convention have appropriate + values. For example, it checks that \texttt{reg} properties have a + length that makes sense given the address forms specified by the + \texttt{\#address-cells} and \texttt{\#size-cells} properties. It + checks that properties such as \texttt{interrupt-parent} contain a + valid phandle. +\item \emph{Linux requirements}: \dtc checks that the device tree + contains those nodes and properties that are required by the Linux + kernel to boot correctly. +\end{itemize} + +These checks are useful to catch simple problems with the device tree, +rather than having to debug the results on an embedded kernel. With +the blob input mode, it can also be used for diagnosing problems with +an existing blob. + +\section{Future Work} + +\subsection{Board ports} + +The flattened device tree has always been the only supported way to +boot a \texttt{ppc64} kernel on an embedded system. With the merge of +\texttt{ppc32} and \texttt{ppc64} code it has also become the only +supported way to boot any merged \texttt{powerpc} kernel, 32-bit or +64-bit. In fact, the old \texttt{ppc} architecture exists mainly just +to support the old ppc32 embedded ports that have not been migrated +to the flattened device tree approach. We plan to remove the +\texttt{ppc} architecture eventually, which will mean porting all the +various embedded boards to use the flattened device tree. + +\subsection{\dtc features} + +While it is already quite usable, there are a number of extra features +that \dtc could include to make creating device trees more convenient: +\begin{itemize} +\item \emph{better tree checking}: Although \dtc already performs a + number of checks on the device tree, they are rather haphazard. In + many cases \dtc will give up after detecting a minor error early and + won't pick up more interesting errors later on. There is a + \texttt{-f} parameter that forces \dtc to generate an output tree + even if there are errors. At present, this needs to be used more + often than one might hope, because \dtc is bad at deciding which + errors should really be fatal, and which rate mere warnings. +\item \emph{binary include}: Occasionally, it is useful for the device + tree to incorporate as a property a block of binary data for some + board-specific purpose. For example, many of Apple's device trees + incorporate bytecode drivers for certain platform devices. \dtc's + source format ought to allow this by letting a property's value be + read directly from a binary file. +\item \emph{macros}: it might be useful for \dtc to implement some + sort of macros so that a tree containing a number of similar devices + (for example, multiple identical ethernet controllers or PCI buses) + can be written more quickly. At present, this can be accomplished + in part by running the source file through CPP before compiling with + \dtc. It's not clear whether ``native'' support for macros would be + more useful. +\end{itemize} + +\bibliographystyle{amsplain} +\bibliography{dtc-paper} + +\section*{About the authors} + +David Gibson has been a member of the IBM Linux Technology Center, +working from Canberra, Australia, since 2001. Recently he has worked +on Linux hugepage support and performance counter support for ppc64, +as well as the device tree compiler. In the past, he has worked on +bringup for various ppc and ppc64 embedded systems, the orinoco +wireless driver, ramfs, and a userspace checkpointing system +(\texttt{esky}). + +Benjamin Herrenschmidt was a MacOS developer for about 10 years, but +ultimately saw the light and installed Linux on his Apple PowerPC +machine. After writing a bootloader, BootX, for it in 1998, he +started contributing to the PowerPC Linux port in various areas, +mostly around the support for Apple machines. He became official +PowerMac maintainer in 2001. In 2003, he joined the IBM Linux +Technology Center in Canberra, Australia, where he ported the 64 bit +PowerPC kernel to Apple G5 machines and the Maple embedded board, +among others things. He's a member of the ppc64 development ``team'' +and one of his current goals is to make the integration of embedded +platforms smoother and more maintainable than in the 32-bit PowerPC +kernel. + +\section*{Legal Statement} + +This work represents the view of the author and does not necessarily +represent the view of IBM. + +IBM, \ppc, \ppc Architecture, POWER5, pSeries and iSeries are +trademarks or registered trademarks of International Business Machines +Corporation in the United States and/or other countries. + +Apple and Power Macintosh are a registered trademarks of Apple +Computer Inc. in the United States, other countries, or both. + +Linux is a registered trademark of Linus Torvalds. + +Other company, product, and service names may be trademarks or service +marks of others. + +\end{document} Added: vendor/dtc/dist/Documentation/dts-format.txt ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/dtc/dist/Documentation/dts-format.txt Sat Feb 27 20:19:06 2010 (r204431) @@ -0,0 +1,110 @@ +Device Tree Source Format (version 1) +===================================== + +The Device Tree Source (DTS) format is a textual representation of a +device tree in a form that can be processed by dtc into a binary +device tree in the form expected by the kernel. The description below +is not a formal syntax definition of DTS, but describes the basic +constructs used to represent device trees. + +Node and property definitions +----------------------------- + +Device tree nodes are defined with a node name and unit address with +braces marking the start and end of the node definition. They may be +preceded by a label. + + [label:] node-name[@unit-address] { + [properties definitions] + [child nodes] + } + +Nodes may contain property definitions and/or child node +definitions. If both are present, properties must come before child +nodes. + +Property definitions are name value pairs in the form: + [label:] property-name = value; +except for properties with empty (zero length) value which have the +form: + [label:] property-name; + +Property values may be defined as an array of 32-bit integer cells, as +NUL-terminated strings, as bytestrings or a combination of these. + +* Arrays of cells are represented by angle brackets surrounding a + space separated list of C-style integers + + e.g. interrupts = <17 0xc>; + +* A 64-bit value is represented with two 32-bit cells. + + e.g. clock-frequency = <0x00000001 0x00000000>; + +* A NUL-terminated string value is represented using double quotes + (the property value is considered to include the terminating NUL + character). + + e.g. compatible = "simple-bus"; + +* A bytestring is enclosed in square brackets [] with each byte + represented by two hexadecimal digits. Spaces between each byte are + optional. + + e.g. local-mac-address = [00 00 12 34 56 78]; or equivalently + local-mac-address = [000012345678]; + +* Values may have several comma-separated components, which are + concatenated together. + e.g. compatible = "ns16550", "ns8250"; + example = <0xf00f0000 19>, "a strange property format"; + +* In a cell array a reference to another node will be expanded to that + node's phandle. References may by '&' followed by a node's label: + e.g. interrupt-parent = < &mpic >; + or they may be '&' followed by a node's full path in braces: + e.g. interrupt-parent = < &{/soc/interrupt-controller@40000} >; + +* Outside a cell array, a reference to another node will be expanded + to that node's full path. + e.g. ethernet0 = &EMAC0; + +* Labels may also appear before or after any component of a property + value, or between cells of a cell array, or between bytes of a + bytestring. + e.g. reg = reglabel: <0 sizelabel: 0x1000000>; + e.g. prop = [ab cd ef byte4: 00 ff fe]; + e.g. str = start: "string value" end: ; + + +File layout +----------- + +Version 1 DTS files have the overall layout: + /dts-v1/; + + [memory reservations] + + / { + [property definitions] + [child nodes] + }; + +* The "/dts-v1/;" must be present to identify the file as a version 1 + DTS (dts files without this tag will be treated by dtc as being in + the obsolete "version 0", which uses a different format for integers + amongst other small but incompatible changes). + +* Memory reservations define an entry for the device tree blob's + memory reservation table. They have the form: + e.g. /memreserve/
; + Where
and are 64-bit C-style integers. + +* The / { ... }; section defines the root node of the device tree. + +* C style (/* ... */) and C++ style (// ...) comments are supported. + + + + -- David Gibson + -- Yoder Stuart Added: vendor/dtc/dist/Documentation/manual.txt ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/dtc/dist/Documentation/manual.txt Sat Feb 27 20:19:06 2010 (r204431) @@ -0,0 +1,618 @@ +Device Tree Compiler Manual +=========================== + +I - "dtc", the device tree compiler + 1) Obtaining Sources + 2) Description + 3) Command Line + 4) Source File + 4.1) Overview + 4.2) Properties + 4.3) Labels and References + +II - The DT block format + 1) Header + 2) Device tree generalities + 3) Device tree "structure" block + 4) Device tree "strings" block + + +III - libfdt + + +I - "dtc", the device tree compiler +=================================== + +1) Sources + +Source code for the Device Tree Compiler can be found at jdl.com. +The gitweb interface is: + + http://www.jdl.com/git_repos/ + +The repository is here: + + git://www.jdl.com/software/dtc.git + http://www.jdl.com/software/dtc.git + +Tarballs of the 1.0.0 and latest releases are here: + + http://www.jdl.com/software/dtc-1.0.0.tgz + http://www.jdl.com/software/dtc-latest.tgz + + +2) Description + +The Device Tree Compiler, dtc, takes as input a device-tree in +a given format and outputs a device-tree in another format. +Typically, the input format is "dts", a human readable source +format, and creates a "dtb", or binary format as output. + +The currently supported Input Formats are: + + - "dtb": "blob" format. A flattened device-tree block with + header in one binary blob. + + - "dts": "source" format. A text file containing a "source" + for a device-tree. + + - "fs" format. A representation equivalent to the output of + /proc/device-tree where nodes are directories and + properties are files. + +The currently supported Output Formats are: + + - "dtb": "blob" format + + - "dts": "source" format + + - "asm": assembly language file. A file that can be sourced + by gas to generate a device-tree "blob". That file can + then simply be added to your Makefile. Additionally, the + assembly file exports some symbols that can be used. + + +3) Command Line + +The syntax of the dtc command line is: + + dtc [options] [] + +Options: + + + The name of the input source file. If no + or "-" is given, stdin is used. + + -b + Set the physical boot cpu. + + -f + Force. Try to produce output even if the input tree has errors. + + -h + Emit a brief usage and help message. + + -I + The source input format, as listed above. + + -o + The name of the generated output file. Use "-" for stdout. + + -O + The generated output format, as listed above. + + -q + Quiet: -q suppress warnings, -qq errors, -qqq all + + -R + Make space for reserve map entries + Relevant for dtb and asm output only. + + -S + Ensure the blob at least long, adding additional + space if needed. + + -v + Print DTC version and exit. + + -V + Generate output conforming to the given . + By default the most recent version is generated. + Relevant for dtb and asm output only. + + +The defines what version of the "blob" format will be +generated. Supported versions are 1, 2, 3, 16 and 17. The default is +always the most recent version and is likely the highest number. + +Additionally, dtc performs various sanity checks on the tree. + + +4) Device Tree Source file + +4.1) Overview + +Here is a very rough overview of the layout of a DTS source file: + + + sourcefile: list_of_memreserve devicetree + + memreserve: label 'memreserve' ADDR ADDR ';' + | label 'memreserve' ADDR '-' ADDR ';' + + devicetree: '/' nodedef + + nodedef: '{' list_of_property list_of_subnode '}' ';' + + property: label PROPNAME '=' propdata ';' + + propdata: STRING + | '<' list_of_cells '>' + | '[' list_of_bytes ']' + + subnode: label nodename nodedef + +That structure forms a hierarchical layout of nodes and properties +rooted at an initial node as: + + / { + } + +Both classic C style and C++ style comments are supported. + +Source files may be directly included using the syntax: + + /include/ "filename" + + +4.2) Properties + +Properties are named, possibly labeled, values. Each value +is one of: + + - A null-teminated C-like string, + - A numeric value fitting in 32 bits, + - A list of 32-bit values + - A byte sequence + +Here are some example property definitions: + + - A property containing a 0 terminated string + + property1 = "string_value"; + + - A property containing a numerical 32-bit hexadecimal value + + property2 = <1234abcd>; + + - A property containing 3 numerical 32-bit hexadecimal values + + property3 = <12345678 12345678 deadbeef>; + + - A property whose content is an arbitrary array of bytes + + property4 = [0a 0b 0c 0d de ea ad be ef]; + + +Node may contain sub-nodes to obtain a hierarchical structure. +For example: + + - A child node named "childnode" whose unit name is + "childnode at address". It it turn has a string property + called "childprop". + + childnode@addresss { + childprop = "hello\n"; + }; + + +By default, all numeric values are hexadecimal. Alternate bases +may be specified using a prefix "d#" for decimal, "b#" for binary, +and "o#" for octal. + +Strings support common escape sequences from C: "\n", "\t", "\r", +"\(octal value)", "\x(hex value)". + + +4.3) Labels and References + +Labels may be applied to nodes or properties. Labels appear +before a node name, and are referenced using an ampersand: &label. +Absolute node path names are also allowed in node references. + +In this exmaple, a node is labled "mpic" and then referenced: + + mpic: interrupt-controller@40000 { + ... + }; + + ethernet-phy@3 { + interrupt-parent = <&mpic>; + ... + }; + +And used in properties, lables may appear before or after any value: + + randomnode { + prop: string = data: "mystring\n" data_end: ; *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***