From owner-freebsd-hackers@freebsd.org Sun Jul 30 19:17:03 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 46BB6DC06B9 for ; Sun, 30 Jul 2017 19:17:03 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from mail.metricspace.net (mail.metricspace.net [IPv6:2001:470:1f11:617::107]) by mx1.freebsd.org (Postfix) with ESMTP id 18DB27DEBE for ; Sun, 30 Jul 2017 19:17:03 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from [172.16.0.37] (unknown [172.16.0.37]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: eric) by mail.metricspace.net (Postfix) with ESMTPSA id 0D3B81C62 for ; Sun, 30 Jul 2017 19:16:56 +0000 (UTC) Subject: Re: nested GPT To: freebsd-hackers@freebsd.org References: From: Eric McCorkle Message-ID: Date: Sun, 30 Jul 2017 15:16:57 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Jul 2017 19:17:03 -0000 Please note that the boot loaders are unable to handle nested partition schemes. If you have any data necessary for the initial boot on such a partition, it won't work. On 07/29/2017 08:30, Oliver Pinter wrote: > On 7/29/17, Wojciech Puchar wrote: >> i have GPT partitioned SSD, and some partitions are used for bhyve virtual >> disk containing windows - itself GPT partitioned. >> >> whenbhyve is stopped geom doesn't look at this GPT partitioned partition. >> ggatel is a workaround but can it be changed so nested GPT partitions will >> work? > > Take a look at geom_map(4). > >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" >> > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@freebsd.org Mon Jul 31 10:24:37 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2353EDCF754 for ; Mon, 31 Jul 2017 10:24:37 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-it0-x232.google.com (mail-it0-x232.google.com [IPv6:2607:f8b0:4001:c0b::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DA59771616; Mon, 31 Jul 2017 10:24:36 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-it0-x232.google.com with SMTP id 77so931683itj.1; Mon, 31 Jul 2017 03:24:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=kaxlB+cmQWAxdAN9E5U/z+6atnvts8m5P1eKLOKeBfw=; b=klPX7NtsTEj25JvgAAPyfECQrnkQFp/rjCbSEtPItq9YW2Yv/aiwmcAqa9VkYd0x3x PdmJdhq1899XY2IJf3v1A7CZ1PqDjQwLHMOPvZjovQkqcSTuFprDhNGciI3oVWhvj1ZF FrqJIikAdk6E6ttaDniyYJRYvJLHG5zAgTOnP6gCIIzr+xWHn84+/fNZjk0VHalv1l5m 3rPrra2jXQDscJEaWx7M/7sK/4Z0UBpcMNjDoE7C5zjNO6x9Cll/piKAGzU+cKAiUtk7 ksd8vKt1zdBy8v82z/SvEOg9PRgGTD5sTdaKw0PxLEWtmXQunUfEUY71mG8FE9poa28C FdsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=kaxlB+cmQWAxdAN9E5U/z+6atnvts8m5P1eKLOKeBfw=; b=XZPxa8Vw3bi62Jxcs6Lsc1ooBJkjkUwHCBe0tAHYrV4me6NMLVUtDgPkJbRiOWKsPY +hX354ZJx9+dWi5DY7l7sMA4XFgAwQcuhgs7t/YWDR4NiKD0X6lMK05d7pgRD80/r4Z0 FTtxGbaQBxym1dqJy9gJ2zzFvC1Y5cWQ7tM4Kn9bh03HWswEKz9aAnKNRfl5BrW/DmR2 We5Vx505t7Ax3qHpvMQB1X10zA30c/n8EPGlh1qa/hPV9E43RpvuYC4N53Yd5UQLHlFr 354BFWLgFjORGoqPbrqLvXReKNDrD3t0lcUHnNsnSoFbhgF6yJyw9Hea2emWOQXm85Jc mj0w== X-Gm-Message-State: AIVw113sP44D1uN1a/CRZBVa7LsZ2cOEw/UszVk34VQrHang0+O1uhMn T+fDaZvetxIlNEnzXOk61+cG6yDCvORO X-Received: by 10.36.74.8 with SMTP id k8mr17723870itb.100.1501496676071; Mon, 31 Jul 2017 03:24:36 -0700 (PDT) MIME-Version: 1.0 Sender: carpeddiem@gmail.com Received: by 10.107.24.66 with HTTP; Mon, 31 Jul 2017 03:24:15 -0700 (PDT) In-Reply-To: References: From: Ed Maste Date: Mon, 31 Jul 2017 06:24:15 -0400 X-Google-Sender-Auth: eh-R6sVLu_HIUKCKLEOtuI3gR7Y Message-ID: Subject: Re: CFT: CI for GitHub on FreeBSD To: Alan Somers Cc: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Jul 2017 10:24:37 -0000 On 28 July 2017 at 18:32, Alan Somers wrote: > > Cool! I had actually looked at using pmem at one point, though I > eventually decided against it. Are the build and test instructions > the same for FreeBSD as for Linux? They should be - some recent changes went in to fix the build on FreeBSD. There were a few in progress still when I was preparing to head out to BSDCam. From owner-freebsd-hackers@freebsd.org Wed Aug 2 10:14:11 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6F98EDCC00D; Wed, 2 Aug 2017 10:14:11 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 946B483684; Wed, 2 Aug 2017 10:14:10 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id NAA20807; Wed, 02 Aug 2017 13:14:08 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1dcqfH-000Pzq-V3; Wed, 02 Aug 2017 13:14:07 +0300 To: FreeBSD Hackers , FreeBSD Current From: Andriy Gapon Subject: order of executing MOD_LOAD and registering module sysctl-s Message-ID: <62e7ab4d-8956-545e-b204-4fb63cfe5fbf@FreeBSD.org> Date: Wed, 2 Aug 2017 13:13:11 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Aug 2017 10:14:11 -0000 As far as I understand a module initialization routine is executed via the sysinit mechanism. Specifically, module_register_init is set up as the sysinit function for every module and it calls MOD_EVENT(mod, MOD_LOAD) to invoke the module event handler. In linker_load_file() I see the following code: linker_file_register_sysctls(lf); linker_file_sysinit(lf); I think that this means that any statically declared sysctl-s in the module would be registered before the module receives the MOD_LOAD event. It's possible that some of the sysctl-s could have procedures as handlers and they might access data that is supposed to be initialized by the module event handler. So, for example, running sysctl -a at just the right moment during the loading of a module might end up in an expected behavior (including a crash). Is my interpretation of how the code works correct? Can the order of linker_file_sysinit and linker_file_register_sysctls be changed without a great risk? Thank you! P.S. The same applies to: linker_file_sysuninit(file); linker_file_unregister_sysctls(file); -- Andriy Gapon From owner-freebsd-hackers@freebsd.org Wed Aug 2 10:41:49 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7966DDCCB7C; Wed, 2 Aug 2017 10:41:49 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [IPv6:2a01:4f8:c17:6c4b::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 455B8848B4; Wed, 2 Aug 2017 10:41:49 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 2FF30260179; Wed, 2 Aug 2017 12:41:47 +0200 (CEST) Subject: Re: order of executing MOD_LOAD and registering module sysctl-s To: Andriy Gapon , FreeBSD Hackers , FreeBSD Current References: <62e7ab4d-8956-545e-b204-4fb63cfe5fbf@FreeBSD.org> From: Hans Petter Selasky Message-ID: Date: Wed, 2 Aug 2017 12:39:36 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 In-Reply-To: <62e7ab4d-8956-545e-b204-4fb63cfe5fbf@FreeBSD.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Aug 2017 10:41:49 -0000 On 08/02/17 12:13, Andriy Gapon wrote: > > As far as I understand a module initialization routine is executed via the > sysinit mechanism. Specifically, module_register_init is set up as the sysinit > function for every module and it calls MOD_EVENT(mod, MOD_LOAD) to invoke the > module event handler. > > In linker_load_file() I see the following code: > linker_file_register_sysctls(lf); > linker_file_sysinit(lf); > > I think that this means that any statically declared sysctl-s in the module > would be registered before the module receives the MOD_LOAD event. > It's possible that some of the sysctl-s could have procedures as handlers and > they might access data that is supposed to be initialized by the module event > handler. > > So, for example, running sysctl -a at just the right moment during the loading > of a module might end up in an expected behavior (including a crash). > > Is my interpretation of how the code works correct? > Can the order of linker_file_sysinit and linker_file_register_sysctls be changed > without a great risk? > > Thank you! > > P.S. > The same applies to: > linker_file_sysuninit(file); > linker_file_unregister_sysctls(file); > Hi, Not sure if this answers your question. If a SYSCTL() is TUNABLE, it's procedure can be called when the sysctl is created. Else the SYSCTL() procedure callback might be called right after it's registered. I think there is an own subsystem in sys/kernel.h which takes care of the actual SYSCTL() creation/destruction - after the linker is involved. --HPS From owner-freebsd-hackers@freebsd.org Wed Aug 2 16:14:37 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 71A0DDD43A4; Wed, 2 Aug 2017 16:14:37 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2FB906B337; Wed, 2 Aug 2017 16:14:36 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by mail.baldwin.cx (Postfix) with ESMTPSA id 9949410AF01; Wed, 2 Aug 2017 12:14:27 -0400 (EDT) From: John Baldwin To: freebsd-current@freebsd.org Cc: Hans Petter Selasky , Andriy Gapon , FreeBSD Hackers , FreeBSD Current Subject: Re: order of executing MOD_LOAD and registering module sysctl-s Date: Wed, 02 Aug 2017 08:49:46 -0700 Message-ID: <2718016.8bPh6cqhGc@ralph.baldwin.cx> User-Agent: KMail/4.14.10 (FreeBSD/11.1-STABLE; KDE/4.14.30; amd64; ; ) In-Reply-To: References: <62e7ab4d-8956-545e-b204-4fb63cfe5fbf@FreeBSD.org> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Wed, 02 Aug 2017 12:14:27 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Aug 2017 16:14:37 -0000 On Wednesday, August 02, 2017 12:39:36 PM Hans Petter Selasky wrote: > On 08/02/17 12:13, Andriy Gapon wrote: > > > > As far as I understand a module initialization routine is executed via the > > sysinit mechanism. Specifically, module_register_init is set up as the sysinit > > function for every module and it calls MOD_EVENT(mod, MOD_LOAD) to invoke the > > module event handler. > > > > In linker_load_file() I see the following code: > > linker_file_register_sysctls(lf); > > linker_file_sysinit(lf); > > > > I think that this means that any statically declared sysctl-s in the module > > would be registered before the module receives the MOD_LOAD event. > > It's possible that some of the sysctl-s could have procedures as handlers and > > they might access data that is supposed to be initialized by the module event > > handler. > > > > So, for example, running sysctl -a at just the right moment during the loading > > of a module might end up in an expected behavior (including a crash). > > > > Is my interpretation of how the code works correct? > > Can the order of linker_file_sysinit and linker_file_register_sysctls be changed > > without a great risk? > > > > Thank you! > > > > P.S. > > The same applies to: > > linker_file_sysuninit(file); > > linker_file_unregister_sysctls(file); > > > > Hi, > > Not sure if this answers your question. > > If a SYSCTL() is TUNABLE, it's procedure can be called when the sysctl > is created. Else the SYSCTL() procedure callback might be called right > after it's registered. I think there is an own subsystem in sys/kernel.h > which takes care of the actual SYSCTL() creation/destruction - after the > linker is involved. sysctl nodes are created explicitly via linker_file_register_sysctls, not via SYSINITs, so you can't order them with respect to other init functions. I think Andriy's suggestion of doing sysctls "inside" sysinits (so they are registered last and unregistered first) is probably better than the current state and is a simpler fix than changing all sysctls to use SYSINITs. -- John Baldwin From owner-freebsd-hackers@freebsd.org Wed Aug 2 16:51:01 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8E914DD4E2B for ; Wed, 2 Aug 2017 16:51:01 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1b.ore.mailhop.org (outbound1b.ore.mailhop.org [54.200.247.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7114A6C741 for ; Wed, 2 Aug 2017 16:51:01 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: a4f2c083-77a2-11e7-bfd0-afd4446ba3af X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.ore.mailhop.org (Halon) with ESMTPSA id a4f2c083-77a2-11e7-bfd0-afd4446ba3af; Wed, 02 Aug 2017 16:50:09 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id v72GnpWp006880; Wed, 2 Aug 2017 10:49:51 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <1501692591.90400.142.camel@freebsd.org> Subject: Re: order of executing MOD_LOAD and registering module sysctl-s From: Ian Lepore To: John Baldwin , freebsd-current@freebsd.org Cc: Hans Petter Selasky , Andriy Gapon , FreeBSD Hackers Date: Wed, 02 Aug 2017 10:49:51 -0600 In-Reply-To: <2718016.8bPh6cqhGc@ralph.baldwin.cx> References: <62e7ab4d-8956-545e-b204-4fb63cfe5fbf@FreeBSD.org> <2718016.8bPh6cqhGc@ralph.baldwin.cx> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Aug 2017 16:51:01 -0000 On Wed, 2017-08-02 at 08:49 -0700, John Baldwin wrote: > On Wednesday, August 02, 2017 12:39:36 PM Hans Petter Selasky wrote: > > > > On 08/02/17 12:13, Andriy Gapon wrote: > > > > > > > > > As far as I understand a module initialization routine is > > > executed via the > > > sysinit mechanism.  Specifically, module_register_init is set up > > > as the sysinit > > > function for every module and it calls MOD_EVENT(mod, MOD_LOAD) > > > to invoke the > > > module event handler. > > > > > > In linker_load_file() I see the following code: > > >                          linker_file_register_sysctls(lf); > > >                          linker_file_sysinit(lf); > > > > > > I think that this means that any statically declared sysctl-s in > > > the module > > > would be registered before the module receives the MOD_LOAD > > > event. > > > It's possible that some of the sysctl-s could have procedures as > > > handlers and > > > they might access data that is supposed to be initialized by the > > > module event > > > handler. > > > > > > So, for example, running sysctl -a at just the right moment > > > during the loading > > > of a module might end up in an expected behavior (including a > > > crash). > > > > > > Is my interpretation of how the code works correct? > > > Can the order of linker_file_sysinit and > > > linker_file_register_sysctls be changed > > > without a great risk? > > > > > > Thank you! > > > > > > P.S. > > > The same applies to: > > >                  linker_file_sysuninit(file); > > >                  linker_file_unregister_sysctls(file); > > > > > Hi, > > > > Not sure if this answers your question. > > > > If a SYSCTL() is TUNABLE, it's procedure can be called when the > > sysctl  > > is created. Else the SYSCTL() procedure callback might be called > > right  > > after it's registered. I think there is an own subsystem in > > sys/kernel.h  > > which takes care of the actual SYSCTL() creation/destruction - > > after the  > > linker is involved. > sysctl nodes are created explicitly via linker_file_register_sysctls, > not via > SYSINITs, so you can't order them with respect to other init > functions. > > I think Andriy's suggestion of doing sysctls "inside" sysinits (so > they are > registered last and unregistered first) is probably better than the > current > state and is a simpler fix than changing all sysctls to use SYSINITs. > But if module sysctls are registered last then the ones flagged as CTLFLAG_TUN won't have their tunable values populated before the MOD_LOAD handler runs, is that going to cause trouble?  It would suck to have to handle things differently in a driver depending on whether you're compiled into the kernel or kldload'd interactively. -- Ian From owner-freebsd-hackers@freebsd.org Wed Aug 2 16:56:08 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C822DAB211; Wed, 2 Aug 2017 16:56:08 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [IPv6:2a01:4f8:c17:6c4b::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 25CBB6CCCA; Wed, 2 Aug 2017 16:56:08 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 5AAE3260179; Wed, 2 Aug 2017 18:56:06 +0200 (CEST) Subject: Re: order of executing MOD_LOAD and registering module sysctl-s To: John Baldwin , freebsd-current@freebsd.org Cc: FreeBSD Hackers , Andriy Gapon References: <62e7ab4d-8956-545e-b204-4fb63cfe5fbf@FreeBSD.org> <2718016.8bPh6cqhGc@ralph.baldwin.cx> From: Hans Petter Selasky Message-ID: <1c40d6ef-bfd2-d8ee-e057-47cd8d695544@selasky.org> Date: Wed, 2 Aug 2017 18:53:54 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 In-Reply-To: <2718016.8bPh6cqhGc@ralph.baldwin.cx> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Aug 2017 16:56:08 -0000 On 08/02/17 17:49, John Baldwin wrote: > On Wednesday, August 02, 2017 12:39:36 PM Hans Petter Selasky wrote: >> On 08/02/17 12:13, Andriy Gapon wrote: >>> >>> As far as I understand a module initialization routine is executed via the >>> sysinit mechanism. Specifically, module_register_init is set up as the sysinit >>> function for every module and it calls MOD_EVENT(mod, MOD_LOAD) to invoke the >>> module event handler. >>> >>> In linker_load_file() I see the following code: >>> linker_file_register_sysctls(lf); >>> linker_file_sysinit(lf); >>> >>> I think that this means that any statically declared sysctl-s in the module >>> would be registered before the module receives the MOD_LOAD event. >>> It's possible that some of the sysctl-s could have procedures as handlers and >>> they might access data that is supposed to be initialized by the module event >>> handler. >>> >>> So, for example, running sysctl -a at just the right moment during the loading >>> of a module might end up in an expected behavior (including a crash). >>> >>> Is my interpretation of how the code works correct? >>> Can the order of linker_file_sysinit and linker_file_register_sysctls be changed >>> without a great risk? >>> >>> Thank you! >>> >>> P.S. >>> The same applies to: >>> linker_file_sysuninit(file); >>> linker_file_unregister_sysctls(file); >>> >> >> Hi, >> >> Not sure if this answers your question. >> Hi, >> If a SYSCTL() is TUNABLE, it's procedure can be called when the sysctl >> is created. Else the SYSCTL() procedure callback might be called right >> after it's registered. I think there is an own subsystem in sys/kernel.h >> which takes care of the actual SYSCTL() creation/destruction - after the >> linker is involved. > > sysctl nodes are created explicitly via linker_file_register_sysctls, not via > SYSINITs, so you can't order them with respect to other init functions. For GENERIC (non-modules) the SYSCTLS() are registered by sysctl_register_all() at SYSINIT(sysctl, SI_SUB_KMEM, SI_ORDER_FIRST, sysctl_register_all, 0); > > I think Andriy's suggestion of doing sysctls "inside" sysinits (so they are > registered last and unregistered first) is probably better than the current > state and is a simpler fix than changing all sysctls to use SYSINITs. > If the module provided SYSCTLS's could use the same SI_SUB_KMEM it would be compatible. You have three cases to think about: 1) SYSCTLS's in modules loaded before the kernel is booted 2) SYSCTLS's in modules after the kernel is booted 3) SYSCTLS's in the GENERIC kernel. I'm not 100% sure, but I think 1) and 2) are treated differently. Correct me if I'm wrong. --HPS From owner-freebsd-hackers@freebsd.org Wed Aug 2 18:49:32 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8BE2FDB10BA; Wed, 2 Aug 2017 18:49:32 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 638A672E68; Wed, 2 Aug 2017 18:49:30 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by mail.baldwin.cx (Postfix) with ESMTPSA id 2E8BC10AB01; Wed, 2 Aug 2017 14:49:29 -0400 (EDT) From: John Baldwin To: Hans Petter Selasky Cc: freebsd-current@freebsd.org, FreeBSD Hackers , Andriy Gapon Subject: Re: order of executing MOD_LOAD and registering module sysctl-s Date: Wed, 02 Aug 2017 11:49:16 -0700 Message-ID: <2156888.F9y1Fr7Oe8@ralph.baldwin.cx> User-Agent: KMail/4.14.10 (FreeBSD/11.1-STABLE; KDE/4.14.30; amd64; ; ) In-Reply-To: <1c40d6ef-bfd2-d8ee-e057-47cd8d695544@selasky.org> References: <62e7ab4d-8956-545e-b204-4fb63cfe5fbf@FreeBSD.org> <2718016.8bPh6cqhGc@ralph.baldwin.cx> <1c40d6ef-bfd2-d8ee-e057-47cd8d695544@selasky.org> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Wed, 02 Aug 2017 14:49:29 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Aug 2017 18:49:32 -0000 On Wednesday, August 02, 2017 06:53:54 PM Hans Petter Selasky wrote: > On 08/02/17 17:49, John Baldwin wrote: > > On Wednesday, August 02, 2017 12:39:36 PM Hans Petter Selasky wrote: > >> On 08/02/17 12:13, Andriy Gapon wrote: > >>> > >>> As far as I understand a module initialization routine is executed via the > >>> sysinit mechanism. Specifically, module_register_init is set up as the sysinit > >>> function for every module and it calls MOD_EVENT(mod, MOD_LOAD) to invoke the > >>> module event handler. > >>> > >>> In linker_load_file() I see the following code: > >>> linker_file_register_sysctls(lf); > >>> linker_file_sysinit(lf); > >>> > >>> I think that this means that any statically declared sysctl-s in the module > >>> would be registered before the module receives the MOD_LOAD event. > >>> It's possible that some of the sysctl-s could have procedures as handlers and > >>> they might access data that is supposed to be initialized by the module event > >>> handler. > >>> > >>> So, for example, running sysctl -a at just the right moment during the loading > >>> of a module might end up in an expected behavior (including a crash). > >>> > >>> Is my interpretation of how the code works correct? > >>> Can the order of linker_file_sysinit and linker_file_register_sysctls be changed > >>> without a great risk? > >>> > >>> Thank you! > >>> > >>> P.S. > >>> The same applies to: > >>> linker_file_sysuninit(file); > >>> linker_file_unregister_sysctls(file); > >>> > >> > >> Hi, > >> > >> Not sure if this answers your question. > >> > > Hi, > > >> If a SYSCTL() is TUNABLE, it's procedure can be called when the sysctl > >> is created. Else the SYSCTL() procedure callback might be called right > >> after it's registered. I think there is an own subsystem in sys/kernel.h > >> which takes care of the actual SYSCTL() creation/destruction - after the > >> linker is involved. > > > > sysctl nodes are created explicitly via linker_file_register_sysctls, not via > > SYSINITs, so you can't order them with respect to other init functions. > > For GENERIC (non-modules) the SYSCTLS() are registered by > sysctl_register_all() at SYSINIT(sysctl, SI_SUB_KMEM, SI_ORDER_FIRST, > sysctl_register_all, 0); > > > > > I think Andriy's suggestion of doing sysctls "inside" sysinits (so they are > > registered last and unregistered first) is probably better than the current > > state and is a simpler fix than changing all sysctls to use SYSINITs. > > > > If the module provided SYSCTLS's could use the same SI_SUB_KMEM it would > be compatible. > > You have three cases to think about: > > 1) SYSCTLS's in modules loaded before the kernel is booted > 2) SYSCTLS's in modules after the kernel is booted > 3) SYSCTLS's in the GENERIC kernel. > > I'm not 100% sure, but I think 1) and 2) are treated differently. > Correct me if I'm wrong. 3) sysctls in the kernel are handled at SI_SUB_KMEM. 1) modules loaded by the loader are handled in linker_preload() at SI_SUB_KLD. Their sysctls are all registered when linker_preload() executes. Their SYSINITs are added to the pending sysinit list. Any SYSINITs earlier than SI_SUB_KLD will be executed in sorted order by mi_startup() after linker_preload() returns. Any SYSINITs after SI_SUB_KLD will be kept in the pending list in order and executed as if they were in the kernel. 2) All of the sysctl's are registered first, and afterwards the SYSINITs are run in sorted order. The race Andriy describes has always been present, but it was perhaps easier to fix by just inverting the order when TUNABLE_* were used explicitly as those registered explicit SI_SUB_TUNABLES sysinits. Note that it has always been the case with old TUNABLE_* that handlers could be run before locks were initialized (e.g. via MTX_SYSINIT which runs later at SI_SUB_LOCK), so if you have handlers that need to use locks, etc. you really shouldn't use RDTUN/RWTUN but instead you should use a dedicated SYSINIT to read a tunable and apply the right logic. There are a few different ways to fix Andriy's race while still solving the issue Ian notes that you may have SYSINITs that depend on the tunable fetches having been performed. One observation is that it shouldn't break anything to invert the order on unload and always unregister sysctls before invoking SYSUNINITs, so I think we should probably make that part of the change regardless. The variety comes in how to handle the module loading case: 1) Just extend the SYSCTL_XLOCK and hold it while invoking SYSINITs in modules during post-boot, and/or use some other explicit locking mechanism with sleep/wakeup or a cv to "pause" any sysctl requests until a kld is fully registered. This is a bit of a sledgehammer, but it is simple and kldload isn't a hot path. 2) Use two passes the add sysctl nodes. The first pass registers the sysctl and marks it as "new" which prevents it from being invoked, but the name can be extracted to generate tunable string, etc. SYSINITs are run after the first pass when loading post-boot, and the second pass after SYSINITs goes through and clears "new" from the sysctl nodes. I would probably lean towards 1) as it is simpler. I suspect that holding SYSCTL_XLOCK across SYSINITs won't fly as I know some SYSINIT routines can add dynamic sysctls (e.g. driver_module_handler SYSINIT for if_cxgbe.ko will invoke t4_attach -> t4_sysctls), so you'd have to use the sleep/wakeup or cv route. -- John Baldwin From owner-freebsd-hackers@freebsd.org Thu Aug 3 06:58:15 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 190C7DCD512; Thu, 3 Aug 2017 06:58:15 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 004FC69D0E; Thu, 3 Aug 2017 06:58:13 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id JAA23332; Thu, 03 Aug 2017 09:58:11 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1ddA5D-00011X-Mc; Thu, 03 Aug 2017 09:58:11 +0300 Subject: Re: order of executing MOD_LOAD and registering module sysctl-s To: John Baldwin , freebsd-current@FreeBSD.org Cc: Hans Petter Selasky , FreeBSD Hackers References: <62e7ab4d-8956-545e-b204-4fb63cfe5fbf@FreeBSD.org> <2718016.8bPh6cqhGc@ralph.baldwin.cx> From: Andriy Gapon Message-ID: <158cf433-f2c8-9006-b091-15fe3095c759@FreeBSD.org> Date: Thu, 3 Aug 2017 09:57:15 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <2718016.8bPh6cqhGc@ralph.baldwin.cx> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2017 06:58:15 -0000 On 02/08/2017 18:49, John Baldwin wrote: > sysctl nodes are created explicitly via linker_file_register_sysctls, not via > SYSINITs, so you can't order them with respect to other init functions. > > I think Andriy's suggestion of doing sysctls "inside" sysinits (so they are > registered last and unregistered first) is probably better than the current > state and is a simpler fix than changing all sysctls to use SYSINITs. Kostik (kib) suggested a possible valid use-case that depends on the current order: adding dynamic sysctl-s under static sysctl-s via the module load handler. He also offered an idea for a possible solution: holding the modules lock in the shared mode (MOD_SLOCK) around calls to sysctl-s registered from modules. -- Andriy Gapon From owner-freebsd-hackers@freebsd.org Thu Aug 3 16:34:19 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D28F0DC71DF; Thu, 3 Aug 2017 16:34:19 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AF72514EC; Thu, 3 Aug 2017 16:34:19 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by mail.baldwin.cx (Postfix) with ESMTPSA id AB50810AB01; Thu, 3 Aug 2017 12:34:10 -0400 (EDT) From: John Baldwin To: Andriy Gapon Cc: freebsd-current@freebsd.org, Hans Petter Selasky , FreeBSD Hackers Subject: Re: order of executing MOD_LOAD and registering module sysctl-s Date: Thu, 03 Aug 2017 08:47:48 -0700 Message-ID: <3503074.NZ5sjsRuYH@ralph.baldwin.cx> User-Agent: KMail/4.14.10 (FreeBSD/11.1-STABLE; KDE/4.14.30; amd64; ; ) In-Reply-To: <158cf433-f2c8-9006-b091-15fe3095c759@FreeBSD.org> References: <62e7ab4d-8956-545e-b204-4fb63cfe5fbf@FreeBSD.org> <2718016.8bPh6cqhGc@ralph.baldwin.cx> <158cf433-f2c8-9006-b091-15fe3095c759@FreeBSD.org> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Thu, 03 Aug 2017 12:34:10 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2017 16:34:19 -0000 On Thursday, August 03, 2017 09:57:15 AM Andriy Gapon wrote: > On 02/08/2017 18:49, John Baldwin wrote: > > sysctl nodes are created explicitly via linker_file_register_sysctls, not via > > SYSINITs, so you can't order them with respect to other init functions. > > > > I think Andriy's suggestion of doing sysctls "inside" sysinits (so they are > > registered last and unregistered first) is probably better than the current > > state and is a simpler fix than changing all sysctls to use SYSINITs. > > Kostik (kib) suggested a possible valid use-case that depends on the current > order: adding dynamic sysctl-s under static sysctl-s via the module load handler. > He also offered an idea for a possible solution: holding the modules lock in the > shared mode (MOD_SLOCK) around calls to sysctl-s registered from modules. Yes, that could work. You'd need a way to "tag" sysctls as being module sysctls vs non-module sysctls. Another possiblity would be to make two passes over sysctls when loading/unloading modules where you have a "disabled" flag or some such. During load you would set this flag when doing sysctl_register_oid for the static nodes and then do a second pass after the SYSINITs to clear all the disabled flags. During unload you would do this in reverse with an early pass before SYSUNINITs to set "disabled" on all the static nodes for the kld. -- John Baldwin From owner-freebsd-hackers@freebsd.org Fri Aug 4 02:17:38 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 86464DBEEE9 for ; Fri, 4 Aug 2017 02:17:38 +0000 (UTC) (envelope-from khanzf@gmail.com) Received: from mail-pf0-x232.google.com (mail-pf0-x232.google.com [IPv6:2607:f8b0:400e:c00::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5854C7505B for ; Fri, 4 Aug 2017 02:17:38 +0000 (UTC) (envelope-from khanzf@gmail.com) Received: by mail-pf0-x232.google.com with SMTP id c28so1327886pfe.3 for ; Thu, 03 Aug 2017 19:17:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=Ka9t1+JsjY43jc8fNfMSz7AnMrTIU7QFUF7iSQyHKnc=; b=SmLqBd2P0cQIqTENt2kvIiKdXS2WW6QMyRubgqno1hc4BaQzFW+Z3HlmoTeuPNzDQt P/ujfk2OIocBGGjHVtWRI1UEKyGOBC6aONrP87YBxi7TJF1shVowDHYjQB88fvNSp1u+ 10vLdAjISQjh0xYNIQ+3VRuGtz9TRQn2qUewKiYqI85I8nP9HFxzEv7oPklCF3INp1tG 9Fg9m0wL2Ny6ENCQBD+tUVgcjDz3CGd7vJik/hlhSzCFD3JVtPXXia5sTwiiUxKoM2nf 4YOq4IvkXOn/t50cHEUFdhU2wbaSCEE56zPKI669jpRyhAP90QezGOfggUd876p+Jh/X 19AA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=Ka9t1+JsjY43jc8fNfMSz7AnMrTIU7QFUF7iSQyHKnc=; b=RxXaslqM5SUSlVVEmvcs8e6wxwRf4jrrK0fUSX6mokE9aIPE4x5n7hIINCdrckfm6B 4hytEfhQkkln1SfV+imHT+TYiR4wiUWRvTyK6mLqXDrBhtPPQDDMJAStK92BtaZiAuJd 2xNiIXIiqYJC97sbKPfCqI8YEBk5zmvdE9ZPR4CrFsUSKEelzx1Ddu/+2woVGJvgkKrA 5VUbISXJ8pFEtOi9c6S1e1PJuuruG4F6w7C/Tp8+o4STWG/R5pXVeXnEj/V8TnxHw6VM AdNrf7FXkegJTTz7uLi4rbD5w7aVpldEdOTF64Kr6BzC+Xfmw8oy0CA7gUhIaY0NcYg1 OUyQ== X-Gm-Message-State: AIVw113+ZHok8hyblZ2vq/zGWBVpx6pzzTWWAHsm9/lk7BII+Bk08Qa3 Kk3wcuZmYkY3EUdRctIt28cI6EdcO5NSqfs= X-Received: by 10.98.112.137 with SMTP id l131mr838657pfc.194.1501813057558; Thu, 03 Aug 2017 19:17:37 -0700 (PDT) MIME-Version: 1.0 Received: by 10.100.183.208 with HTTP; Thu, 3 Aug 2017 19:17:17 -0700 (PDT) From: Farhan Khan Date: Thu, 3 Aug 2017 22:17:17 -0400 Message-ID: Subject: PCI bus detaching when driver crashing? To: freebsd-hackers@freebsd.org X-Mailman-Approved-At: Fri, 04 Aug 2017 03:19:09 +0000 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Aug 2017 02:17:38 -0000 Hi all, I am trying to write the Realtek 8188e (pci) driver for FreeBSD. I am making slow incremental progress, and having the driver safely exit out (as opposed to a segfault). I noticed that when the device detaches, due to an error in the driver, the entire pci bus will also detach! For example, here is my kernel output: rtwn0: rtwn_load_firmware: failed to upload firmware rtwn-rtl8188eefw (error 60) rtwn0: detached pci2: detached The first line is where the code crashes: sys/dev/rtwn/if_rtwn_fw.c line 129 https://github.com/khanzf/freebsd/blob/103c05369b1ced770a2cadc9468e0134c8d9421b/sys/dev/rtwn/if_rtwn_fw.c#L129 Why would this result in the entire pci2 bus detatching? Strangely enough, I did not have this problem a few months ago, then it suddenly began. This is significantly slowing down my workflow, and would be nice to have it resolved. Otherwise, I have to reboot to bring pci2 back up. Any insight? Thanks! Farhan Khan