From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 13 14:27:57 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 709851065703; Thu, 13 Sep 2012 14:27:57 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 45E278FC0A; Thu, 13 Sep 2012 14:27:57 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 915A7B989; Thu, 13 Sep 2012 10:27:56 -0400 (EDT) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Thu, 13 Sep 2012 10:22:24 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p17; KDE/4.5.5; amd64; ; ) References: <504910CB.8050007@FreeBSD.org> <201209071048.39664.jhb@freebsd.org> In-Reply-To: <201209071048.39664.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201209131022.24898.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 13 Sep 2012 10:27:56 -0400 (EDT) Cc: Navdeep Parhar Subject: Re: Order in which a driver attaches to devices X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Sep 2012 14:27:57 -0000 On Friday, September 07, 2012 10:48:39 am John Baldwin wrote: > On Thursday, September 06, 2012 5:08:27 pm Navdeep Parhar wrote: > > I have a system with multiple cards supported by cxgbe(4). When I build > > a kernel with the driver compiled in, it attaches to the cards in a > > different order from when it's loaded as a module. Why? The network > > interfaces get re-ordered and this is quite annoying. > > Hmmmm. The boot time probe does a depth first walk of the PCI bus. This is > what is suggested by PCI-SIG for enumerating PCI buses (and is normally how > BIOSs walk the bus assigning bus numbers). The walk that is done at kldload > time walks the 'pciX' bus devices in numerical order (rather than walking the > tree). I suspect your BIOS is doing something weird and assigning bus numbers > in a non-depth first ordering so that the two orderings are not consisent as > they are on other machines. BTW, another fix is to stop trying to force unit numbers to patch PCI bus numbers (e.g. change pcib_attach() in pci_pci.c to use -1 instead of sc->secbus). A few other places would need to be changed as well: acpi_pcib_attach(), legacy_pcib_attach(), qpi_pcib_attach(), mptable_hostb_attach(). If we went this route we should probably do it on other platforms as well. (Some, such as sparc64 already do this.) -- John Baldwin