Date: Fri, 19 Jan 2018 21:20:25 +0000 (UTC) From: Kyle Evans <kevans@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r328173 - head/usr.bin/dtc Message-ID: <201801192120.w0JLKPY5013933@repo.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: kevans Date: Fri Jan 19 21:20:24 2018 New Revision: 328173 URL: https://svnweb.freebsd.org/changeset/base/328173 Log: dtc(1): Update to upstream ea3c233 Highlights of this update: - /__local_fixups__ is now generated to be GPL dtc and libfdt compliant - Compiling with -@ will now cause dtc to assign phandles to all labelled nodes - /include/ and /incbin/ now handle absolute paths correctly - The manpage now has information about overlays, including how to apply them and how to generate them - Syntactic sugar for overlays is now supported, allowing an overlay DTS like: = /dts-v1/; /plugin/; &foo { foo,status = "okay"; }; = to generate a fragment targetting <&foo>. Modified: head/usr.bin/dtc/Makefile head/usr.bin/dtc/dtc.1 head/usr.bin/dtc/dtc.cc head/usr.bin/dtc/fdt.cc head/usr.bin/dtc/fdt.hh head/usr.bin/dtc/input_buffer.cc head/usr.bin/dtc/input_buffer.hh Modified: head/usr.bin/dtc/Makefile ============================================================================== --- head/usr.bin/dtc/Makefile Fri Jan 19 21:19:57 2018 (r328172) +++ head/usr.bin/dtc/Makefile Fri Jan 19 21:20:24 2018 (r328173) @@ -4,8 +4,6 @@ PROG_CXX=dtc SRCS= dtc.cc input_buffer.cc string.cc dtb.cc fdt.cc checking.cc MAN= dtc.1 -WARNS?= 3 - CXXFLAGS+= -std=c++11 -fno-rtti -fno-exceptions NO_SHARED?=NO Modified: head/usr.bin/dtc/dtc.1 ============================================================================== --- head/usr.bin/dtc/dtc.1 Fri Jan 19 21:19:57 2018 (r328172) +++ head/usr.bin/dtc/dtc.1 Fri Jan 19 21:20:24 2018 (r328173) @@ -30,7 +30,7 @@ .\" .\" $FreeBSD$ .\"/ -.Dd January 1, 2013 +.Dd January 17, 2018 .Dt DTC 1 .Os .Sh NAME @@ -57,7 +57,7 @@ The .Nm utility converts flattened device tree (FDT) representations. - It is most commonly used to generate device tree blobs (DTB), the binary +It is most commonly used to generate device tree blobs (DTB), the binary representation of an FDT, from device tree sources (DTS), the ASCII text source representation. .Pp @@ -153,9 +153,9 @@ format for property values. These allow property value to be specified on the command line. .It Fl R Ar entries The number of empty reservation table entries to pad the table with. -This is -useful if you are generating a device tree blob for bootloader or similar that -needs to reserve some memory before passing control to the operating system. +This is useful if you are generating a device tree blob for bootloader or +similar that needs to reserve some memory before passing control to the +operating system. .It Fl S Ar bytes The minimum size in bytes of the blob. The blob will be padded after the strings table to ensure that it is the @@ -244,6 +244,54 @@ Checks that all .Va /delete-node/ statements refer to nodes that are merged. .El +.Sh OVERLAYS +The utility provides support for generating overlays, also known as plugins. +Overlays are a method of patching a base device tree that has been compiled with +the +.Fl @ +flag, with some limited support for patching device trees that were not compiled +with the +.Fl @ +flag. +.Pp +To denote that a DTS is intended to be used as an overlay, +.Va /plugin/; +should be included in the header, following any applicable +.Va /dts-v1/; +tag. +.Pp +Conventional overlays are crafted by creating +.Va fragment +nodes in a root. +Each fragment node must have either a +.Va target +property set to a label reference, or a +.Va target-path +string property set to a path. +It must then have an +.Va __overlay__ +child node, whose properties and child nodes are merged into the base device +tree when the overlay is applied. +.Pp +Much simpler syntactic sugar was later invented to simplify generating overlays. +Instead of creating targetted fragments manually, one can instead create a root +node that targets a label in the base node using the +.Va &label +syntax supported in conventional DTS. +This will indicate that a fragment should be generated for the node, with the +given +.Va label +being the target, and the properties and child nodes will be used as the +__overlay__. +.Pp +Both conventional overlays and the later-added syntactic sugar are supported. +.Pp +Overlay blobs can be applied at boot time by setting +.Va fdt_overlays +in +.Xr loader.conf 5 . +Multiple overlays may be specified, and they will be applied in the order given. +.El .Sh EXAMPLES The command: .Pp @@ -254,8 +302,7 @@ will generate a file from the device tree source .Pa device.dts and print errors if any occur during parsing or property checking. -The -resulting file can be assembled and linked into a binary. +The resulting file can be assembled and linked into a binary. .Pp The command: .Pp @@ -265,6 +312,33 @@ will write the device tree source for the device tree .Pa device.dtb to the standard output. This is useful when debugging device trees. +.Pp +The command: +.Pp +.Dl "dtc -@ -O dtb -I dts -o device.dtb device.dts" +.Pp +will generate a +.Pa device.dtb +file from the device tree source +.Pa device.dts +with a __symbols__ node included so that overlays may be applied to it. +.Pp +The command: +.Pp +.Dl "dtc -@ -O dtb -I dts -o device_overlay.dtbo device_overlay.dts" +.Pp +will generate a +.Pa device_overlay.dtbo +file, using the standard extension for a device tree overlay, from the device +tree source +.Pa device_overlay.dts . +A __symbols__ node will be included so that overlays may be applied to it. +The presence of a +.Va /plugin/; +directive in +.Pa device_overlay.dts +will indicate to the utility that it should also generate the underlying +metadata required in overlays. .Sh COMPATIBILITY This utility is intended to be compatible with the device tree compiler provided by elinux.org. Modified: head/usr.bin/dtc/dtc.cc ============================================================================== --- head/usr.bin/dtc/dtc.cc Fri Jan 19 21:19:57 2018 (r328172) +++ head/usr.bin/dtc/dtc.cc Fri Jan 19 21:20:24 2018 (r328173) @@ -49,6 +49,8 @@ using namespace dtc; using std::string; +namespace { + /** * The current major version of the tool. */ @@ -65,7 +67,7 @@ int version_minor_compatible = 4; int version_patch = 0; int version_patch_compatible = 0; -static void usage(const string &argv0) +void usage(const string &argv0) { fprintf(stderr, "Usage:\n" "\t%s\t[-fhsv@] [-b boot_cpu_id] [-d dependency_file]" @@ -80,13 +82,15 @@ static void usage(const string &argv0) /** * Prints the current version of this program.. */ -static void version(const char* progname) +void version(const char* progname) { fprintf(stdout, "Version: %s %d.%d.%d compatible with gpl dtc %d.%d.%d\n", progname, version_major, version_minor, version_patch, version_major_compatible, version_minor_compatible, version_patch_compatible); } + +} // Anonymous namespace using fdt::device_tree; Modified: head/usr.bin/dtc/fdt.cc ============================================================================== --- head/usr.bin/dtc/fdt.cc Fri Jan 19 21:19:57 2018 (r328172) +++ head/usr.bin/dtc/fdt.cc Fri Jan 19 21:20:24 2018 (r328173) @@ -497,6 +497,29 @@ property::property(text_input_buffer &input, return; case '/': { + if (input.consume("/incbin/(\"")) + { + auto loc = input.location(); + std::string filename = input.parse_to('"'); + if (!(valid = input.consume('"'))) + { + loc.report_error("Syntax error, expected '\"' to terminate /incbin/("); + return; + } + property_value v; + if (!(valid = input.read_binary_file(filename, v.byte_data))) + { + input.parse_error("Cannot open binary include file"); + return; + } + if (!(valid &= input.consume(')'))) + { + input.parse_error("Syntax error, expected ')' to terminate /incbin/("); + return; + } + values.push_back(v); + break; + } unsigned long long bits = 0; valid = input.consume("/bits/"); input.next_token(); @@ -999,7 +1022,7 @@ node::get_property(const string &key) } void -node::merge_node(node_ptr other) +node::merge_node(node_ptr &other) { for (auto &l : other->labels) { @@ -1034,7 +1057,7 @@ node::merge_node(node_ptr other) { if (i->name == c->name && i->unit_address == c->unit_address) { - i->merge_node(std::move(c)); + i->merge_node(c); found = true; break; } @@ -1207,9 +1230,68 @@ device_tree::collect_names() collect_names_recursive(root, p); } +property_ptr +device_tree::assign_phandle(node *n, uint32_t &phandle) +{ + // If there is an existing phandle, use it + property_ptr p = n->get_property("phandle"); + if (p == 0) + { + p = n->get_property("linux,phandle"); + } + if (p == 0) + { + // Otherwise insert a new phandle node + property_value v; + while (used_phandles.find(phandle) != used_phandles.end()) + { + // Note that we only don't need to + // store this phandle in the set, + // because we are monotonically + // increasing the value of phandle and + // so will only ever revisit this value + // if we have used 2^32 phandles, at + // which point our blob won't fit in + // any 32-bit system and we've done + // something badly wrong elsewhere + // already. + phandle++; + } + push_big_endian(v.byte_data, phandle++); + if (phandle_node_name == BOTH || phandle_node_name == LINUX) + { + p.reset(new property("linux,phandle")); + p->add_value(v); + n->add_property(p); + } + if (phandle_node_name == BOTH || phandle_node_name == EPAPR) + { + p.reset(new property("phandle")); + p->add_value(v); + n->add_property(p); + } + } + + return (p); +} + void -device_tree::resolve_cross_references() +device_tree::assign_phandles(node_ptr &n, uint32_t &next) { + if (!n->labels.empty()) + { + assign_phandle(n.get(), next); + } + + for (auto &c : n->child_nodes()) + { + assign_phandles(c, next); + } +} + +void +device_tree::resolve_cross_references(uint32_t &phandle) +{ for (auto *pv : cross_references) { node_path path = node_paths[pv->string_data]; @@ -1252,7 +1334,6 @@ device_tree::resolve_cross_references() }); assert(sorted_phandles.size() == fixups.size()); - uint32_t phandle = 1; for (auto &i : sorted_phandles) { string target_name = i.get().val.string_data; @@ -1334,43 +1415,7 @@ device_tree::resolve_cross_references() } } // If there is an existing phandle, use it - property_ptr p = target->get_property("phandle"); - if (p == 0) - { - p = target->get_property("linux,phandle"); - } - if (p == 0) - { - // Otherwise insert a new phandle node - property_value v; - while (used_phandles.find(phandle) != used_phandles.end()) - { - // Note that we only don't need to - // store this phandle in the set, - // because we are monotonically - // increasing the value of phandle and - // so will only ever revisit this value - // if we have used 2^32 phandles, at - // which point our blob won't fit in - // any 32-bit system and we've done - // something badly wrong elsewhere - // already. - phandle++; - } - push_big_endian(v.byte_data, phandle++); - if (phandle_node_name == BOTH || phandle_node_name == LINUX) - { - p.reset(new property("linux,phandle")); - p->add_value(v); - target->add_property(p); - } - if (phandle_node_name == BOTH || phandle_node_name == EPAPR) - { - p.reset(new property("phandle")); - p->add_value(v); - target->add_property(p); - } - } + property_ptr p = assign_phandle(target, phandle); p->begin()->push_to_buffer(i.get().val.byte_data); assert(i.get().val.byte_data.size() == 4); } @@ -1644,7 +1689,73 @@ device_tree::node_path::to_string() const return path; } +node_ptr +device_tree::create_fragment_wrapper(node_ptr &node, int &fragnum) +{ + // In a plugin, we can massage these non-/ root nodes into into a fragment + std::string fragment_address = "fragment@" + std::to_string(fragnum); + ++fragnum; + + std::vector<property_ptr> symbols; + + // Intentionally left empty + node_ptr newroot = node::create_special_node("", symbols); + node_ptr wrapper = node::create_special_node("__overlay__", symbols); + + // Generate the fragment with target = <&name> + property_value v; + v.string_data = node->name; + v.type = property_value::PHANDLE; + auto prop = std::make_shared<property>(std::string("target")); + prop->add_value(v); + symbols.push_back(prop); + + node_ptr fragment = node::create_special_node(fragment_address, symbols); + + wrapper->merge_node(node); + fragment->add_child(std::move(wrapper)); + newroot->add_child(std::move(fragment)); + return newroot; +} + +node_ptr +device_tree::generate_root(node_ptr &node, int &fragnum) +{ + + string name = node->name; + if (name == string()) + { + return std::move(node); + } + else if (!is_plugin) + { + return nullptr; + } + + return create_fragment_wrapper(node, fragnum); +} + void +device_tree::reassign_fragment_numbers(node_ptr &node, int &delta) +{ + + for (auto &c : node->child_nodes()) + { + if (c->name == std::string("fragment")) + { + int current_address = std::stoi(c->unit_address, nullptr, 16); + std::ostringstream new_address; + current_address += delta; + // It's possible that we hopped more than one somewhere, so just reset + // delta to the next in sequence. + delta = current_address + 1; + new_address << std::hex << current_address; + c->unit_address = new_address.str(); + } + } +} + +void device_tree::parse_dts(const string &fn, FILE *depfile) { auto in = input_buffer::buffer_for_file(fn); @@ -1665,6 +1776,7 @@ device_tree::parse_dts(const string &fn, FILE *depfile dirname(fn), depfile); bool read_header = false; + int fragnum = 0; parse_file(input, roots, read_header); switch (roots.size()) { @@ -1673,18 +1785,36 @@ device_tree::parse_dts(const string &fn, FILE *depfile input.parse_error("Failed to find root node /."); return; case 1: - root = std::move(roots[0]); + root = generate_root(roots[0], fragnum); + if (!root) + { + valid = false; + input.parse_error("Failed to find root node /."); + return; + } break; default: { - root = std::move(roots[0]); + root = generate_root(roots[0], fragnum); + if (!root) + { + valid = false; + input.parse_error("Failed to find root node /."); + return; + } for (auto i=++(roots.begin()), e=roots.end() ; i!=e ; ++i) { auto &node = *i; string name = node->name; if (name == string()) { - root->merge_node(std::move(node)); + if (is_plugin) + { + // Re-assign any fragment numbers based on a delta of + // fragnum before we merge it + reassign_fragment_numbers(node, fragnum); + } + root->merge_node(node); } else { @@ -1696,20 +1826,36 @@ device_tree::parse_dts(const string &fn, FILE *depfile } if (existing == node_names.end()) { - fprintf(stderr, "Unable to merge node: %s\n", name.c_str()); + if (is_plugin) + { + auto fragment = create_fragment_wrapper(node, fragnum); + root->merge_node(fragment); + } + else + { + fprintf(stderr, "Unable to merge node: %s\n", name.c_str()); + } } else { - existing->second->merge_node(std::move(node)); + existing->second->merge_node(node); } } } } } collect_names(); - resolve_cross_references(); + uint32_t phandle = 1; + // If we're writing symbols, go ahead and assign phandles to the entire + // tree. We'll do this before we resolve cross references, just to keep + // order semi-predictable and stable. if (write_symbols) { + assign_phandles(root, phandle); + } + resolve_cross_references(phandle); + if (write_symbols) + { std::vector<property_ptr> symbols; // Create a symbol table. Each label in this device tree may be // referenced by other plugins, so we create a __symbols__ node inside @@ -1769,21 +1915,72 @@ device_tree::parse_dts(const string &fn, FILE *depfile } symbols.clear(); // If we have any resolved phandle references in this plugin, then - // we must leave a property in the /__local_fixups__ node whose key - // is 'fixup' and whose value is as described above. + // we must create a child in the __local_fixups__ node whose path + // matches the node path from the root and whose value contains the + // location of the reference within a property. + + // Create a local_fixups node that is initially empty. + node_ptr local_fixups = node::create_special_node("__local_fixups__", symbols); for (auto &i : fixups) { if (!i.val.is_phandle()) { continue; } - symbols.push_back(create_fixup_entry(i, "fixup")); + node *n = local_fixups.get(); + for (auto &p : i.path) + { + // Skip the implicit root + if (p.first.empty()) + { + continue; + } + bool found = false; + for (auto &c : n->child_nodes()) + { + if (c->name == p.first) + { + n = c.get(); + found = true; + break; + } + } + if (!found) + { + n->add_child(node::create_special_node(p.first, symbols)); + n = (--n->child_end())->get(); + } + } + assert(n); + property_value pv; + push_big_endian(pv.byte_data, static_cast<uint32_t>(i.prop->offset_of_value(i.val))); + pv.type = property_value::BINARY; + auto key = i.prop->get_key(); + property_ptr prop = n->get_property(key); + // If we don't have an existing property then create one and + // use this property value + if (!prop) + { + prop = std::make_shared<property>(std::move(key)); + n->add_property(prop); + prop->add_value(pv); + } + else + { + // If we do have an existing property value, try to append + // this value. + property_value &old_val = *(--prop->end()); + if (!old_val.try_to_merge(pv)) + { + prop->add_value(pv); + } + } } // We've iterated over all fixups, but only emit the // __local_fixups__ if we found some that were resolved internally. - if (!symbols.empty()) + if (local_fixups->child_begin() != local_fixups->child_end()) { - root->add_child(node::create_special_node("__local_fixups__", symbols)); + root->add_child(std::move(local_fixups)); } } } Modified: head/usr.bin/dtc/fdt.hh ============================================================================== --- head/usr.bin/dtc/fdt.hh Fri Jan 19 21:19:57 2018 (r328172) +++ head/usr.bin/dtc/fdt.hh Fri Jan 19 21:20:24 2018 (r328173) @@ -370,7 +370,7 @@ class property /** * Returns the key for this property. */ - inline std::string get_key() + inline const std::string &get_key() { return key; } @@ -620,7 +620,7 @@ class node * Merges a node into this one. Any properties present in both are * overridden, any properties present in only one are preserved. */ - void merge_node(node_ptr other); + void merge_node(node_ptr &other); /** * Write this node to the specified output. Although nodes do not * refer to a string table directly, their properties do. The string @@ -676,12 +676,12 @@ class device_tree /** * The format that we should use for writing phandles. */ - phandle_format phandle_node_name; + phandle_format phandle_node_name = EPAPR; /** * Flag indicating that this tree is valid. This will be set to false * on parse errors. */ - bool valid; + bool valid = true; /** * Type used for memory reservations. A reservation is two 64-bit * values indicating a base address and length in memory that the @@ -775,23 +775,23 @@ class device_tree /** * The default boot CPU, specified in the device tree header. */ - uint32_t boot_cpu; + uint32_t boot_cpu = 0; /** * The number of empty reserve map entries to generate in the blob. */ - uint32_t spare_reserve_map_entries; + uint32_t spare_reserve_map_entries = 0; /** * The minimum size in bytes of the blob. */ - uint32_t minimum_blob_size; + uint32_t minimum_blob_size = 0; /** * The number of bytes of padding to add to the end of the blob. */ - uint32_t blob_padding; + uint32_t blob_padding = 0; /** * Is this tree a plugin? */ - bool is_plugin; + bool is_plugin = false; /** * Visit all of the nodes recursively, and if they have labels then add * them to the node_paths and node_names vectors so that they can be @@ -800,6 +800,12 @@ class device_tree */ void collect_names_recursive(node_ptr &n, node_path &path); /** + * Assign a phandle property to a single node. The next parameter + * holds the phandle to be assigned, and will be incremented upon + * assignment. + */ + property_ptr assign_phandle(node *n, uint32_t &next); + /** * Assign phandle properties to all nodes that have been referenced and * require one. This method will recursively visit the tree starting at * the node that it is passed. @@ -812,9 +818,11 @@ class device_tree /** * Resolves all cross references. Any properties that refer to another * node must have their values replaced by either the node path or - * phandle value. + * phandle value. The phandle parameter holds the next phandle to be + * assigned, should the need arise. It will be incremented upon each + * assignment of a phandle. */ - void resolve_cross_references(); + void resolve_cross_references(uint32_t &phandle); /** * Parses a dts file in the given buffer and adds the roots to the parsed * set. The `read_header` argument indicates whether the header has @@ -859,15 +867,33 @@ class device_tree /** * Default constructor. Creates a valid, but empty FDT. */ - device_tree() : phandle_node_name(EPAPR), valid(true), - boot_cpu(0), spare_reserve_map_entries(0), - minimum_blob_size(0), blob_padding(0) {} + device_tree() {} /** * Constructs a device tree from the specified file name, referring to * a file that contains a device tree blob. */ void parse_dtb(const std::string &fn, FILE *depfile); /** + * Construct a fragment wrapper around node. This will assume that node's + * name may be used as the target of the fragment, and the contents are to + * be wrapped in an __overlay__ node. The fragment wrapper will be assigned + * fragnumas its fragment number, and fragment number will be incremented. + */ + node_ptr create_fragment_wrapper(node_ptr &node, int &fragnum); + /** + * Generate a root node from the node passed in. This is sensitive to + * whether we're in a plugin context or not, so that if we're in a plugin we + * can circumvent any errors that might normally arise from a non-/ root. + * fragnum will be assigned to any fragment wrapper generated as a result + * of the call, and fragnum will be incremented. + */ + node_ptr generate_root(node_ptr &node, int &fragnum); + /** + * Reassign any fragment numbers from this new node, based on the given + * delta. + */ + void reassign_fragment_numbers(node_ptr &node, int &delta); + /* * Constructs a device tree from the specified file name, referring to * a file that contains device tree source. */ @@ -906,7 +932,10 @@ class device_tree */ void sort() { - root->sort(); + if (root) + { + root->sort(); + } } /** * Adds a path to search for include files. The argument must be a Modified: head/usr.bin/dtc/input_buffer.cc ============================================================================== --- head/usr.bin/dtc/input_buffer.cc Fri Jan 19 21:19:57 2018 (r328172) +++ head/usr.bin/dtc/input_buffer.cc Fri Jan 19 21:20:24 2018 (r328173) @@ -206,9 +206,9 @@ text_input_buffer::handle_include() { next_token(); string name = parse_property_name(); - if (defines.count(name) > 0) + if (defines.count(name) == 0) { - reallyInclude = true; + reallyInclude = false; } consume('/'); } @@ -250,6 +250,48 @@ text_input_buffer::handle_include() return; } input_stack.push(std::move(include_buffer)); +} + +bool text_input_buffer::read_binary_file(const std::string &filename, byte_buffer &b) +{ + bool try_include_paths = true; + string include_file; + if (filename[0] == '/') + { + include_file = filename; + // Don't try include paths if we're given an absolute path. + // Failing is better so that we don't accidentally do the wrong thing, + // but make it seem like everything is alright. + try_include_paths = false; + } + else + { + include_file = dir + '/' + filename; + } + auto include_buffer = input_buffer::buffer_for_file(include_file, false); + if (include_buffer == 0 && try_include_paths) + { + for (auto i : include_paths) + { + include_file = i + '/' + filename; + include_buffer = input_buffer::buffer_for_file(include_file, false); + if (include_buffer != 0) + { + break; + } + } + } + if (!include_buffer) + { + return false; + } + if (depfile) + { + putc(' ', depfile); + fputs(include_file.c_str(), depfile); + } + b.insert(b.begin(), include_buffer->begin(), include_buffer->end()); + return true; } input_buffer Modified: head/usr.bin/dtc/input_buffer.hh ============================================================================== --- head/usr.bin/dtc/input_buffer.hh Fri Jan 19 21:19:57 2018 (r328172) +++ head/usr.bin/dtc/input_buffer.hh Fri Jan 19 21:20:24 2018 (r328173) @@ -166,6 +166,14 @@ class input_buffer cursor++; return *this; } + const char *begin() + { + return buffer; + } + const char *end() + { + return buffer + size; + } /** * Consumes a character. Moves the cursor one character forward if the * next character matches the argument, returning true. If the current @@ -525,6 +533,13 @@ class text_input_buffer * Prints a message indicating the location of a parse error. */ void parse_error(const char *msg); + /** + * Reads the contents of a binary file into `b`. The file name is assumed + * to be relative to one of the include paths. + * + * Returns true if the file exists and can be read, false otherwise. + */ + bool read_binary_file(const std::string &filename, byte_buffer &b); private: /** * Prints a message indicating the location of a parse error, given a
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201801192120.w0JLKPY5013933>