From owner-freebsd-hackers@freebsd.org Thu Aug 29 14:45:46 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0E025D7CDD for ; Thu, 29 Aug 2019 14:45:46 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x82f.google.com (mail-qt1-x82f.google.com [IPv6:2607:f8b0:4864:20::82f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46K5551snNz3Gk5 for ; Thu, 29 Aug 2019 14:45:45 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x82f.google.com with SMTP id t12so3919553qtp.9 for ; Thu, 29 Aug 2019 07:45:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=3CP8ffJcILN/c9QU/s8UPxyGicn9o+e8qLVU2MG60js=; b=GwYyDB+d5tdVdyXIB1XOlAKgajpGPLaE+/Ec0LX6dyBZl7qAllQvc9BiCE4mYWP0y4 EniMiHBflmzu+CqwS6oMsy9mhPiwybzFfYPmLAWwBVYnIu9YNkg3oOY7wsuEDt5ApePj qgaaD6V3W4aKHJE+K33RGN6GryQ0VJkpJh7NthWjVv5+JYUM7y4A5zhZqnQAzgJ0hbFl fzFQI93LJ+BdjqX5ic046SIqNdIVWL83pxurmFb1OudvZjCVm4A287Qv/B5r/7w+u9Fy 5Ttd8t2EvpyTYa4jdz1mv9Vpv5pvFe+ER7IoNRikJMGhJkMFCahVnlHjGODRhxJinRKR C8dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=3CP8ffJcILN/c9QU/s8UPxyGicn9o+e8qLVU2MG60js=; b=BfGVpyXF2FgU2wV8wy/nOLxZKRvtD1hE3yYWhsnHh3cy55l4NoddOfo6SfETjmB943 Q8+LsXVepE5gTAmnNDsTi1WY2kjUxzV+d0Ar9NN+4L/vEVLrAkJxaJcB3LGUj+euxdoI NNX2l+08BdblzvQjJU4A6gmei7ETm8HGoV4OiX/6Lsl6LvyMCpilPcoInaFUnRNYHR+t tS79sU8w90s1Q+kMhsJ0Iw8IaYFv5/buBu1y7f3IVw4ScoPks2APzZPHC+0+/9OG42rr OjOWBvhOthbctMkUmACrPkfpgQUBiTCcsZ65xaK4vboWGAVhkv5qUGqBKP/DIo9mPjna AusA== X-Gm-Message-State: APjAAAUS27V0X5gpDWMEQno7ddqExXAi2Hc7XNyNPj/xTqGwx1yJFfyy 6gE6jSqbPp1N1z3Ouk9v1587mj22vJoGd1ak5fLtxQ== X-Google-Smtp-Source: APXvYqwcDWcP7/c52zvFbpt9HHldKN8CwQp4lrPjqBgyO/kaSdCvC+n/7X4X1GC0AwR4JmkQItINGlg5bsUglSCII/8= X-Received: by 2002:a0c:f6c6:: with SMTP id d6mr6754679qvo.102.1567089943897; Thu, 29 Aug 2019 07:45:43 -0700 (PDT) MIME-Version: 1.0 References: <7dfebbd3-85d6-c7b7-b83b-fae8b644649e@yuripv.net> <478965aa-5256-e356-5339-de6fb82c3459@selasky.org> <63daa36a-5c22-6b08-3cd7-562fa961ab61@yuripv.net> <7f6de96d-8b56-e242-8950-04a20b197bce@selasky.org> <311a21e3-ed61-8679-b416-b2a4c255c6e7@yuripv.net> <1f91f7a6-050d-d690-d374-6b06950d2ce2@yuripv.net> <4201916c-0f7f-0bf6-2d17-e9f6a1879ba7@yuripv.net> In-Reply-To: From: Warner Losh Date: Thu, 29 Aug 2019 08:45:32 -0600 Message-ID: Subject: Re: ichsmb(4) and msleep() To: Ian Lepore Cc: "Conrad E. Meyer" , Yuri Pankov , FreeBSD Hackers X-Rspamd-Queue-Id: 46K5551snNz3Gk5 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=GwYyDB+d; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::82f) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-5.91 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-0.996,0]; RCVD_IN_DNSWL_NONE(0.00)[f.2.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.92)[ip: (-9.37), ipnet: 2607:f8b0::/32(-2.85), asn: 15169(-2.32), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 14:45:46 -0000 On Thu, Aug 29, 2019 at 8:29 AM Ian Lepore wrote: > On Wed, 2019-08-28 at 20:27 -0700, Conrad Meyer wrote: > > On Wed, Aug 28, 2019 at 6:35 AM Yuri Pankov wrote: > > > > > > Warner Losh wrote: > > > > What's the advantages of doing this instead of deferring attach > until the > > > > interrupts are running? > > > > > > None that I can think of, just going with what was suggested and seeing > > > other drivers doing the same. Could you please name a driver that > > > defers attach until !cold? > > > > I think pretty much all drivers attach when interrupts are enabled > > (not the same as !cold)? At least x86 enables interrupts on BSP at > > SI_SUB_INTR, and DRIVER_MODULE drivers *load* at SI_SUB_DRIVERS, and > > the INTR one is ordered before the other. My skim read is that > > drivers do not actually attach until SI_SUB_CONFIGURE. > > > > I think the panic / test in sleepq_set_timeout_sbt is maybe overly > > strong? !cold indicates the entire autoconfigure process has > > concluded. But interrupts are available long before that. Seems like > > hardclock is started at ~SI_SUB_CLOCKS? Which is admittedly after > > DRIVERS, but still long before !cold. I'm not sure what set of > > interrupt/timer functionality is needed for sleepq, but likely that > > condition can be relaxed. > > > > If it cannot be relaxed enough for your driver, you could expand your > > DRIVER_MODULE() into the expanded macro, replacing SI_SUB_DRIVERS with > > a later stage. > > > > > > Afaik, only x86 enables interrupts before SI_SUB_CONFIGURE. Maybe > sparc64 does too. Other arches do it as the last thing in > SI_SUB_CONFIGURE, usually in a function named configure_final() in > /autoconf.c. > Yes. In fact, I think this idiom is so wide spread, we'd be better off doing 'return (bus_delayed_attach_children(dev));' at the end to signal this and gather together all current uses in the tree under that. Generally, if you need interrupts enabled and scheduling running to configure your children, you should do that anyway. It's a convenient late point, and it's better to do all the late things at the same time with the same idiom than to try to hyper-optimize when they get done. We've cleaned up a number of messes where driver writers invent their own thing due to ignorance of the proper thing (or because there wasn't a recognition that it was similar to what lots of other drivers did). Warner