From owner-freebsd-arm@FreeBSD.ORG Wed Aug 6 14:36:14 2014 Return-Path: Delivered-To: freebsd-arm@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B33DAC01 for ; Wed, 6 Aug 2014 14:36:14 +0000 (UTC) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 743F1248E for ; Wed, 6 Aug 2014 14:36:14 +0000 (UTC) Received: from [73.34.117.227] (helo=ilsoft.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1XF2Jz-000Lhg-0L; Wed, 06 Aug 2014 14:36:07 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by ilsoft.org (8.14.9/8.14.9) with ESMTP id s76Ea5hG017125; Wed, 6 Aug 2014 08:36:06 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 73.34.117.227 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1+yOLsmtTi7qkNkSMeuOFx7 X-Authentication-Warning: paranoia.hippie.lan: Host revolution.hippie.lan [172.22.42.240] claimed to be [172.22.42.240] Subject: Re: Two questions on Flattened Device Tree, newbus and device driver attaching From: Ian Lepore To: =?ISO-8859-1?Q?Mat=EDas?= Perret Cantoni In-Reply-To: References: <1407182403.56408.297.camel@revolution.hippie.lan> Content-Type: text/plain; charset="iso-2022-jp" Date: Wed, 06 Aug 2014 08:36:05 -0600 Message-ID: <1407335765.56408.320.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-arm@freebsd.org X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2014 14:36:14 -0000 On Tue, 2014-08-05 at 22:31 -0300, Matas Perret Cantoni wrote: > 2014-08-04 17:00 GMT-03:00 Ian Lepore : > > > On Sun, 2014-08-03 at 16:41 -0300, Matas Perret Cantoni wrote: > > > Hello everyone! > > > I'm working with FreeBSD on the Zedboard (ported by Thomas Skibo > > > ). Currently I'm trying to fully > > > understand how the Flattened Device Tree (FDT) mechanism works and how it > > > integrates with FreeBSD. What I've already understand (I think) is: > > > > > > (1) how to represent devices, memory mapping/ranges, interrupts, > > etc... > > > in a Device Tree Source (DTS) file, > > > (2) how the newbus framework works, and > > > (3) how the kernel manages resources, devices and drivers. > > > > > > Although I've read all the documents I could find (and some source code) > > > there are still two things I don't understand: > > > > > > *1) The DTS source file and CPUs definition:* > > > > > > The DTS file for the zedboard, > > /release/10.0.0/sys/boot/fdt/dts/zedboard.dts > > > (here > > > < > > https://svnweb.freebsd.org/base/release/10.0.0/sys/boot/fdt/dts/zedboard.dts?revision=260789&view=markup > > >), > > > has the CPU definition all commented out: > > > > > > ... > > > // cpus { > > > // #address-cells = <1>; > > > // #size-cells = <0>; > > > // cpu@0 { > > > // device-type = "cpu"; > > > // model = "ARM Cortex-A9"; > > > // }; > > > // }; > > > ... > > > > > > This sounds really strange to me! How can the system tell the CPU it's > > > running on? I'v found some other DTS files for other boards that *do > > > define* it's > > > CPUs. For example: > > > > > > imx53x.dtsi: (here > > > < > > https://svnweb.freebsd.org/base/release/10.0.0/sys/boot/fdt/dts/imx53x.dtsi?view=markup > > > > > > ) > > > > > > ... > > > cpus { > > > #address-cells = <1>; > > > #size-cells = <0>; > > > > > > cpu@0 { > > > device_type = "cpu"; > > > compatible = "ARM,MCIMX535"; > > > reg = <0x0>; > > > d-cache-line-size = <32>; > > > i-cache-line-size = <32>; > > > d-cache-size = <0x8000>; > > > i-cache-size = <0x8000>; > > > l2-cache-line-size = <32>; > > > l2-cache-line = <0x40000>; > > > timebase-frequency = <0>; > > > bus-frequency = <0>; > > > clock-frequency = <0>; > > > }; > > > ... > > > > > > or: > > > > > > p1020rdb.dts (here > > > < > > https://svnweb.freebsd.org/base/release/10.0.0/sys/boot/fdt/dts/p1020rdb.dts?view=markup > > > > > > ) > > > > > > ... > > > cpus { > > > #address-cells = <1>; > > > #size-cells = <0>; > > > > > > PowerPC,P1020@0 { > > > device_type = "cpu"; > > > reg = <0x0>; > > > next-level-cache = <&L2>; > > > }; > > > > > > PowerPC,P1020@1 { > > > device_type = "cpu"; > > > reg = <0x1>; > > > next-level-cache = <&L2>; > > > }; > > > }; > > > ... > > > > > > *So my first question is: How can the system tell on wich CPU it running > > > on? can I add the CPUs definition in my DTS file?* > > > > > > 2) The 'compatible' property of a node, finding the driver and attaching > > it > > > to the corresponding newbus node: During autoconfiguration the the .dtb > > > (device tree blob) file is parsed and for each node of the device three > > the > > > autoconfiguration systen will create a new newbus node (with > > > device_add_child()) and then it will find a suitable driver for it and > > will > > > attach it: > > > > > > */ > > > * This function is the core of the device autoconfiguration > > > * system. Its purpose is to select a suitable driver for a device and > > > * then call that driver to initialise the hardware appropriately. The > > > * driver is selected by calling the DEVICE_PROBE() method of a set of > > > * candidate drivers and then choosing the driver which returned the > > > * best value. This driver is then attached to the device using > > > * device_attach(). > > > * > > > * The set of suitable drivers is taken from the list of drivers in > > > * the parent device's devclass. If the device was originally created > > > * with a specific class name (see device_add_child()), only drivers > > > * with that name are probed, otherwise all drivers in the devclass > > > * are probed. If no drivers return successful probe values in the > > > * parent devclass, the search continues in the parent of that > > > * devclass (see devclass_get_parent()) if any. > > > * > > > * @param dev the device to initialise > > > */ > > > > > > int device_probe(device_t dev) > > > > > > (I extracted this from here > > > < > > https://svnweb.freebsd.org/base/release/10.0.0/sys/kern/subr_bus.c?revision=260789&view=markup > > > > > > ) > > > > > > I believe that the autoconfiguration system uses the > > > fdt_node_check_compatible() function (from fdtlib > > > < > > https://svnweb.freebsd.org/base/release/10.0.0/sys/contrib/libfdt/libfdt.h?revision=260789&view=markup > > >) > > > to get the "compatible" property out of each node of the .dtb blob and > > then > > > it calls device_probe() to find the best driver and attach it to the > > > corresponding newbus node. (is this correct?). > > > > > > From the ePAPR > > > < > > https://www.power.org/wp-content/uploads/2012/06/Power_ePAPR_APPROVED_v1.1.pdf > > > > > > standard > > > we have this: > > > > > > 2.3.1 compatible > > > Property: compatible > > > Value type: > > > > > > Description: The compatible property value consists of one or more > > strings > > > that define the specific > > > programming model for the device. This list of strings should be used by > > a > > > client program for > > > device driver selection. The property value consists of a concatenated > > list > > > of null terminated > > > strings, from most specific to most general. They allow a device to > > express > > > its compatibility > > > with a family of similar devices, potentially allowing a single device > > > driver to match against > > > several devices. > > > The recommended format is “manufacturer,model”, where manufacturer is a > > > string describing the name of the manufacturer (such as a stock ticker > > > symbol), and model > > > specifies the model number. > > > > > > Example: *compatible=“fsl,mpc8641-uart”, “ns16550";* > > > > > > In this example, an operating system would first try to locate a device > > > driver that supported > > > fsl,mpc8641-uart. If a driver was not found, it would then try to locate > > a > > > driver that supported > > > the more general ns16550 device type. > > > > > > > > > *What I don't understand is how the system locates a device driver based > > on > > > the compatible property. For example the cpu node's compatible property > > > ("ARM,MCIMX535") of the imx53x.dtsi example. More precisely: How can the > > > system relate the string "ARM,MCIMX535" with the actual device driver in > > > the file system.* > > > > > > I hope my questions are clear enough. Many thanks in advance. > > > > > > > For #1, virtually none of our arm code uses the cpu information from the > > fdt data, because we generally compile a custom kernel specific to each > > cpu. We've been slowly (very slowly) moving towards a unified kernel > > that can boot on multiple arm chips (or at least closely related chips > > within a family), and that will make the cpu info more important some > > day. > > > > > Oh, I see. Now It is clearer if I take a look in /10.0.0/sys/arm/std.zynq7: > > ... > # > # std.zynq7 - Generic configuration for Xilinx Zynq-7000 PS. > # > # $FreeBSD$ > > cpu CPU_CORTEXA > machine arm armv6 > ... > > > So #1 is solved. Thank you both! > > For #2, if you're looking for some big master table that maps compatible > > strings to drivers, no such thing exists. > > > > Each driver source has one or more DRIVER_MODULE() macros that provides > > some information about the driver. One of the things it provides is the > > parent. The newbus system builds a metadata hierarchy that tracks which > > drivers have described themselves as potential children on each bus. > > > > Usually a device's parent is some sort of bus such as PCI. In an > > fdt-based system "simplebus" is an abstraction that can represent many > > different types of buses (such as internal on-chip connections between > > the cpu and internal devices). A hardware bus such as PCI has ways to > > query the hardware to see what's connected. In the fdt world, simplebus > > uses the fdt data to do this query... it looks at all the fdt device > > entries that are described as its children in the fdt data. > > > > For each child in the fdt data, simplebus asks newbus to probe all the > > drivers whose DRIVER_MODULE() said they could be children of simplebus. > > The probe() routine of each driver has access to the fdt data for the > > device simplebus is trying to probe. The driver compares the compatible > > strings in that data to the compatible strings that it knows how to > > handle, and returns a success/fail code from probe() to indicate whether > > or not it is the driver for the device. > > > > Sometimes multiple drivers can handle the same hardware, so newbus > > probes every child driver against every device on the bus. For example > > a usb keyboard is a pretty generic thing, but a FooStar1000 keyboard > > might have a special driver that understands extra keys. The generic > > usb keyboard driver would return BUS_PROBE_GENERIC, and the FooStar1000 > > driver would return BUS_PROBE_SPECIFIC. After probing all potential > > devices, newbus chooses the one with the highest return value from > > probe() as being the one most-specific to that hardware. (In reality > > this doesn't happen much; usually only one driver returns success and > > all others return an error.) > > > > -- Ian > > > > #2 is way more clearer now, but I still can't see a few things: > > I can see the function simplebus_attach() calling newbus to create a child, > probe it and attach it for each of the ftd nodes claiming to be simplebus > compatible. Here is a snippet from simplebus.c: > > /* > * Walk simple-bus and add direct subordinates as our children. > */ > dt_node = ofw_bus_get_node(dev); > for (dt_child = OF_child(dt_node); dt_child != 0; > dt_child = OF_peer(dt_child)) { > > ... > /* Add newbus device for this FDT node */ > dev_child = device_add_child(dev, NULL, -1); > } > ... > return (bus_generic_attach(dev)); > } > > > So there are two functions from bus.h being called: device_add_child() > which makes a new newbus child, and bus_generic_attach(dev) which causes > newbus to probe and attach each of the new childs of the simplebus node. > > I can see fdtbus doing the same thing for each node of the fdt's root node. > Here is a snippet from fdtbus.c: > > /* > * Walk the FDT root node and add top-level devices as our children. > */ > for (child = OF_child(root); child != 0; child = OF_peer(child)) { > > ... > newbus_device_from_fdt_node(dev, child); > ... > > return (bus_generic_attach(dev)); > > > _________ > > So until here I think I understand how the newbus nodes (devices) are being > created and how the drivers are being attached to them. > But when I print the information about system device configuration with > devinfo I can see that there is a "nexus0" node and a "ofwbus0" node in the > newbus hierarchy: > > root@zedboard:~ # devinfo > nexus0 > ofwbus0 > simplebus0 > zy7_slcr0 > gic0 > l2cache0 > .... > > > I think that the nexus0 node is created by this function: > > /* > * Determine i/o configuration for a machine. > */ > static void > configure_first(void *dummy) > { > > device_add_child(root_bus, "nexus", 0); > } > > > (which is in the file /10.0.0/sys/arm/arm/autoconf.c and by the way I can't > find any calling function to configure_first). > > And I guess the root node is declared in /10.0.0/sys/kern/subr_bus.c file: > > device_t root_bus; > devclass_t root_devclass; > > > But I can't find out how the "ofwbus0" node is created neither who is doing > so! > > Thanks in advance. > Best regards, Matias.- configure_first() and the other functions in autoconf.c are started by the system init code in mi_startup() in kern/init_main.c. Any part of the kernel or modules that may be loaded by loader(8) can use a SYSINIT() macro to declare that they have init code to run at kernel startup. Linker magic combines all the SYSINIT info into a special section in the kernel binary, and the mi_startup() code walks through that info and calls all the functions. You've already discovered that nexus gets added by configure_first() (I think I knew that years ago, but had completely forgotten). That causes nexus to be probed and attached. The probe always succeeds with nexus, and nexus attach() calls bus_generic_probe(). That causes newbus to call the identify() routine for every driver that said it might be a child of nexus. Most drivers don't have an identify() routine, it's rarely used. I think of it as a sort of pre-probe routine (man 9 DEVICE_IDENTIFY has more info on it.) The ofwbus driver (dev/ofw/ofwbus.c) has a DRIVER_MODULE() macro that declares it to be a potential child of nexus, and it has an identify() routine that calls device_add_child() to add itself as a child of nexus. I think of this as "forced adoption"... normally it is the parent bus that does device_add_child(), but in this case the child attaches itself to the parent. Once ofwbus is added to nexus in this way, its attach() routine gets called when nexus does bus_generic_attach(), and the ofwbus attach() then uses the FDT data to attach simplebus and other children described in the data. -- Ian