From owner-svn-src-head@freebsd.org Fri Apr 24 16:57:32 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 77B212BB07D for ; Fri, 24 Apr 2020 16:57:32 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qv1-xf44.google.com (mail-qv1-xf44.google.com [IPv6:2607:f8b0:4864:20::f44]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4980hq5cDjz4dvB for ; Fri, 24 Apr 2020 16:57:31 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qv1-xf44.google.com with SMTP id 59so2896364qva.13 for ; Fri, 24 Apr 2020 09:57:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=mJLJTu6RY+jDeuja2uMVKw0ecpr/woAjOG5AEUgQ4Tk=; b=RnGufPJ+63NdHWRizwDkDG1LR5ciHUGcwevmJQDg4d4TUrNmE7Hh+lzDScCxwywAZJ ZZcY7O2ys5+WBvy0HyhFeEuE06Ph3/b8HhQH/Z997MIlW0Xw1eW8mMq+ikKtRd8H4oRp C7Jx1YGSaeP/XdvQFUDnMON5PYYltdk7WAu9CtcKmd6oj5wzVTvOejc3SRsqpYYV3mwt T8gCQlaR+svYl45Qlp1PZkEatlDL0UeM2LqkcQrZSYzEIX2b/zsRPyLOoc7rVJZr3cln TfLCKZg5iKKzO4LPdPbb0sBLpqds7ngvSyQyle0TJOHi0HaVr4H+awFx9EiF1lLwgux2 duBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=mJLJTu6RY+jDeuja2uMVKw0ecpr/woAjOG5AEUgQ4Tk=; b=hi8HcXRtBUIT8I9s3mlQ6bo3xVZeha4SHG7moRtHtF00TnCNvQbkU/wyUdTxH1lDiQ X9hejUIMbFTJ89uBqOocNGthBLnOY0iCiZQEQnveiRdrWlBqUccYTK/FQJWhi34p7wjf b9Z9/QvTxTHWAucryO+1ZOgh7KE7L4puuhtTbRH9z5OqM0v7uXvg9XqsmyOE9/m390cY YOr2hyXdk6L1+ZExRAJlkWa+DdF6WmudgLU6833oX0iWNs550M+3Kpyd3YFqnACCM+OT GlmMSuQbVfNQ88ie7SAUu67m9gl/LTROvvqKhMLymg0W/FNPhYRN0NguJ+d7Ya2Uim2Y j8dQ== X-Gm-Message-State: AGi0PuaJpaPEebKoBnjgivl2raVzD4QZ29Se+UzsgU1PoBDj3K5PGZPE dbGvJSTbALvCAJ0AJZXZ+wlVKPypfaEbvjeTj/o2klJGxrw= X-Google-Smtp-Source: APiQypLZb0ZFHkPbETFgLvUOMiFirY0PZ2d5Ly04e0ELiovvhHKNJ32+kjyYdlTiCEVhfPsZKGWr/EaJauMpGi1dlps= X-Received: by 2002:a0c:8b52:: with SMTP id d18mr10210839qvc.125.1587747450702; Fri, 24 Apr 2020 09:57:30 -0700 (PDT) MIME-Version: 1.0 References: <202004240749.03O7nMSc066344@repo.freebsd.org> <32586115-2c82-2497-ba4c-b3471887518d@FreeBSD.org> <02449f05-6c43-9cb0-b427-34034a3c358f@FreeBSD.org> In-Reply-To: <02449f05-6c43-9cb0-b427-34034a3c358f@FreeBSD.org> From: Warner Losh Date: Fri, 24 Apr 2020 10:57:19 -0600 Message-ID: Subject: Re: svn commit: r360241 - head/sys/dev/ichiic To: Andriy Gapon Cc: John Baldwin , src-committers , svn-src-all , svn-src-head X-Rspamd-Queue-Id: 4980hq5cDjz4dvB X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=RnGufPJ+; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::f44) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-1.13 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.989,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-0.98)[-0.979,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[svn-src-head@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; URI_COUNT_ODD(1.00)[3]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCVD_IN_DNSWL_NONE(0.00)[4.4.f.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-0.16)[ip: (-0.00), ipnet: 2607:f8b0::/32(-0.33), asn: 15169(-0.43), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Apr 2020 16:57:32 -0000 On Fri, Apr 24, 2020 at 10:07 AM Andriy Gapon wrote: > On 24/04/2020 18:11, Warner Losh wrote: > > > > > > On Fri, Apr 24, 2020 at 8:47 AM Andriy Gapon > > wrote: > > > > On 24/04/2020 17:29, Warner Losh wrote: > > > > > > > > > On Fri, Apr 24, 2020 at 1:49 AM Andriy Gapon > > > > >> wrote: > > > > > > Author: avg > > > Date: Fri Apr 24 07:49:21 2020 > > > New Revision: 360241 > > > URL: https://svnweb.freebsd.org/changeset/base/360241 > > > > > > Log: > > > ig4: ensure that drivers always attach in correct order > > > > > > Use DRIVER_MODULE_ORDERED(SI_ORDER_ANY) so that ig4's ACPI > attachment > > > happens after iicbus and acpi_iicbus drivers are registered. > > > > > > I have seen a problem where iicbus attached under ig4 > instead of > > > acpi_iicbus when ig4.ko was loaded with kldload. I believe > that that > > > happened because ig4 driver was a first driver to register, > it attached > > > and created an iicbus child. Then iicbus driver was > registered and, > > > since it was the only driver that could attach to the iicbus > child > > > device, it did exactly that. After that acpi_iicbus driver > was > > > registered. It would be able to attach to the iicbus > device, but it was > > > already attached, so nothing happened. > > > > > > > > > Can you post more details of which devices are affected? From the > description > > > and the patch, I don't see how this could fix things. > > > > I think I listed them all: ig4iic with acpi attachment, iicbus and > acpi_iicbus. > > acpi > > \--- ig4iic > > \---- iicbus vs acpi_iicbus > > > > I tried to explain the problem and the fix in the commit message. > If you want > > to discuss any specifics, let's continue with specifics. If there > is anything > > unclear in my explanation, I can clarify, but I need to understand > what needs to > > be clarified. > > > > > > That won't fix the stated problem. > > It will. It does. You made me write an essay to explain why :) > Ah, it's a workaround for a newbus bug. OK. I'll work on fixing the newbus bug then. > > If changing the module order fixes something, > > it's the wrong fix. Which is why I asked to make sure I understood the > issue (it > > was unclear if it was at the level indicated, or for the children of the > iicbus). > > > > iicbus returns BUS_PROBE_GENERIC (-100) from its probe routine, while > > acpi_iicbus returns BUS_PROBE_DEFAULT (-20). This means that acpi_iicbus > should > > always win. So something else is going on. We don't specify order on > other > > devices except for some weird, special needs drivers (this is not such a > driver). > > This driver, along with all of other I2C controller drivers, has a > particular > quirk, so it can be called weird. > > It does not matter what the probe routines return if one of the drivers is > not > added to "newbus" at all (even if its code is loaded). Its probe method > is not > executed. And that's what I tried to explain in the commit message. > > Now, why this driver as well as all SMBus and I2C controller drivers are > somewhat weird... There is a multitude of drivers for SMBus and I2C > controllers. Those drivers have different names. So, using newbus speak, > smbus > and iicbus drivers can attach to [newbus] devices under many different > [newbus] > buses. For that reason, the corresponding DRIVER_MODULE declarations are > not > consolidated in smbus.c and iicbus.c; instead, they are spread over > individual > controller drivers. So, loading of smbus or iicbus module does not result > in a > call to devclass_add_driver() that would register these drivers under some > bus. > And that's an unusual thing comparing to the most straightforward drivers. > > Let's use ig4 as an example. > Across its source files we had the following DRIVER_MODULE declarations: > DRIVER_MODULE(ig4iic, acpi, ... -- in ig4_acpi.c > DRIVER_MODULE(iicbus, ig4iic, ... -- in ig4_iic.c > DRIVER_MODULE(acpi_iicbus, ig4iic, ... -- in ig4_iic.c > The first one is needed to register ig4iic driver under acpi bus. Other > two are > needed to register iicbus and acpi_iicbus drivers under ig4iic bus. > The order is not explicitly defined, so the corresponding declaration can > be > processed in any order when ig4.ko is loaded and so the corresponding > devclass_add_driver() can be called in any order. > It's a bug in newbus if that matters. I'll grant that it does today, but it shouldn't. In the past we've worked around this issue in a number of different ways (including having 3 different modules with the order encoded into those modules). It also indicates, perhaps, bugs in the iic acpi enumeration stuff, but that's a harder case to make without more careful study since I know that acpi_iic works around the bit of a mismatch between newbus' device model and ACPI's. > So, I observed in practice the scenario where the drivers were added in the > written order. First, ig4iic was added under acpi, it found the > corresponding > device, probed and attached to it. In its attach method it added a new > device > named "iicbus" as a child. Then, iicbus driver was added under ig4iic bus. > That driver found the child device and attached to it. Only then > acpi_iicbus > was added and it was too late to the party. > > So, this is really about an order in which DRIVER_MODULE-s (which > translate to > sysinit-s) _within a kld_ are processed. Essentially, about an order of > objects > in the corresponding section of a loadable ELF object. MODULE_DEPEND does > not > affect that order. > Yes. That's why we rarely put multiple modules into one .ko. One can have a debate about whether or not this is a bug in the loader or not... But we've been talking for a while about deferring the probe/attach phase of the driver registration until after everything is registered, but while the concept is simple, the details can be tricky. There's some support for this with devctl freeze/unfreeze, but maybe that should move into kldload itself. That too would solve this problem. > Just as an aside, for a contrast, gpiobus uses a different model. > There, all controller drivers must have "gpio" as their driver names > (driver->name). > So, a single DRIVER_MODULE(gpiobus, gpio, ...) in the gpiobus.c code is > sufficient for gpiobus driver to be added under all controllers. And that > registration always happens before a controller driver is added because of > MODULE_DEPEND between it and gpiobus. > Yes. This is one of the drivers I was thinking didn't co-locate things. > > Unless I'm mistaken, the real problem is that the following line is > missing instead. > > MODULE_DEPEND(ig4iic, acpi_iicbus, 1, 1, 1); > > which makes the module dependencies explicit. Can you add that to > ig4_acpi.c, > > revert your change and see if that works? It will ensure that the > acpi_iicbus > > driver is loaded first. If things break because it isn't, it sounds like > a hard > > dependency for ig4. Since we're already pulling in acpi, that doesn't > seem > > unreasonable to me. > > Just for clarity, acpi_iicbus is compiled into iicbus.ko (on relevant > platforms, > of course), same as iicbus. It's not a separate kld. > So I was mistaken about what your second message was trying to say. Warner