From owner-svn-src-head@freebsd.org Fri Jul 6 20:51:44 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 24F51102B9CD; Fri, 6 Jul 2018 20:51:44 +0000 (UTC) (envelope-from tijl@freebsd.org) Received: from mailrelay106.isp.belgacom.be (mailrelay106.isp.belgacom.be [195.238.20.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "relay.skynet.be", Issuer "GlobalSign Organization Validation CA - SHA256 - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 37039967BD; Fri, 6 Jul 2018 20:51:43 +0000 (UTC) (envelope-from tijl@freebsd.org) X-Belgacom-Dynamic: yes IronPort-PHdr: =?us-ascii?q?9a23=3AVwPWWh1aCmICjfDnsmDT+DRfVm0co7zxezQtwd?= =?us-ascii?q?8ZsesWLPXxwZ3uMQTl6Ol3ixeRBMOHs6wC07KempujcFRI2YyGvnEGfc4EfD?= =?us-ascii?q?4+ouJSoTYdBtWYA1bwNv/gYn9yNs1DUFh44yPzahANS47xaFLIv3K98yMZFA?= =?us-ascii?q?nhOgppPOT1HZPZg9iq2+yo9JDffwRFiCChbb9uMR67sRjfus4KjIV4N60/0A?= =?us-ascii?q?HJonxGe+RXwWNnO1eelAvi68mz4ZBu7T1et+ou+MBcX6r6eb84TaFDAzQ9L2?= =?us-ascii?q?81/szrugLdQgaJ+3ART38ZkhtMAwjC8RH6QpL8uTb0u+ZhxCWXO9D9QKsqUj?= =?us-ascii?q?q+8ahkVB7oiD8GNzEn9mHXltdwh79frB64uhBz35LYbISTOfFjfK3SYMkaSH?= =?us-ascii?q?JPUMhRSSJPH4CyYIkBD+UOIelWoJLwp0cMoBeiGQWgGP/jxiFOi3Tr3aM6ye?= =?us-ascii?q?MhEQTe0QI+GtAOtGnfocvyNKcVSuC60qzIwi/Fb/NNxDzw75TIchEjofGIRr?= =?us-ascii?q?9+cdDRxlcxGA7Yk1uep5bpPzSP1uQCqmWW6fdrW+yoi24isQ5xoz6vy98iio?= =?us-ascii?q?nTmI0a1EvL9T5kz4ovIt24UkF7bNi5G5VTryGXL4h7T8E4T2xpuSs20KAKtJ?= =?us-ascii?q?2mcCQQ1Zgqxh3SZvqaeIaS+B3jTvyeITJgiXJgf7Kwmgi9/FC7yu35Ssm0yF?= =?us-ascii?q?FKrjdZktXUtnACyRjT6s+fR/t+5Eih3TeP1wXN5eFeJkA4j7bbK58jwr40jJ?= =?us-ascii?q?YcrUPDHijtmEroia+ZbEMk+vOy5+TgeLXmqYeQN45yig7gLqQjgs+yDOYiPg?= =?us-ascii?q?UPXmWX4/mw2b7+8UHjXblHj/47nrHcsJ/AJMQboqC5AxVS0oYm8xu/Ezam0N?= =?us-ascii?q?YcnXQcIlJFYgyIgJbyNFHVPf/0F/C/g06jkDtz3fDJIqXhAonRLnjEiLrhZq?= =?us-ascii?q?h960hFxAoo19BQ+4tYCrEfL/3pR0D8r9LYDgUnPAOq2OnnE8hy2pkZWWKVDa?= =?us-ascii?q?+TKLnSvkOQ5uIzP+mMY5cYuC3jK/gj/vLulmU5lkMEcaaz2ZsXbGu1Hvp8I0?= =?us-ascii?q?qHf3XjmcwBHnoQsgo5Vuzqh0WIUSRPaHaqQ6I8+jY7BZq9DYfZWo+hmaCO3C?= =?us-ascii?q?C+Hp1TZ2BGFkyMHmnyd4WfQPoMZjiSLdF/nTMfTriuVpUt1Ra0tA/107BnNP?= =?us-ascii?q?bb+jUEtZL/09h4//fTlR4o9Tx1CsSSzXqNQnp6nmMSWTA5wrtwoVdgxVuZ1q?= =?us-ascii?q?h4mfNYH8RJ5/xVSgc6KYLcz+tiBtDyQQLOYNOJR0y9QtWlATA8Vdwxw8UQbE?= =?us-ascii?q?ljANqilQ3M0zCtA78PmLzYTKAzp+jm2HT3Ktc19DCO+7MgilQ9CIMbO3eri6?= =?us-ascii?q?Rk+yDLC56PiUXfvID6KOIQ2jXI+33Fy2eS6hJ2Sgl1BJkiWTg0YUzNoNHw4F?= =?us-ascii?q?iKG6OvC7APHBFMxOS5Bu1NcNK/3gYOf+vqJNmLOzH5oGy3Hxvdg+rUNIc=3D?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A2DGCQCR1T9b/6i1QldUCB0BAQUBCwG?= =?us-ascii?q?DSVQOfyiMXYtVAQGCBjEBQ5Y3LoRJAoIsIjcVAQIBAQIBAQIBbBwMgjUkAYJ?= =?us-ascii?q?dAQU6HCMQCxQECSUPKh4GE4MhggMLq32IJCaBNQWIXoIkgQ+CWjWEVYV7Ao0?= =?us-ascii?q?OjEEJhgiJEY1kijWJLSKBUk0wCIMkixSFQD0wAQGPEwEB?= X-IPAS-Result: =?us-ascii?q?A2DGCQCR1T9b/6i1QldUCB0BAQUBCwGDSVQOfyiMXYtVA?= =?us-ascii?q?QGCBjEBQ5Y3LoRJAoIsIjcVAQIBAQIBAQIBbBwMgjUkAYJdAQU6HCMQCxQEC?= =?us-ascii?q?SUPKh4GE4MhggMLq32IJCaBNQWIXoIkgQ+CWjWEVYV7Ao0OjEEJhgiJEY1ki?= =?us-ascii?q?jWJLSKBUk0wCIMkixSFQD0wAQGPEwEB?= Received: from 168.181-66-87.adsl-dyn.isp.belgacom.be (HELO kalimero.tijl.coosemans.org) ([87.66.181.168]) by relay.skynet.be with ESMTP; 06 Jul 2018 22:50:31 +0200 Received: from kalimero.tijl.coosemans.org (kalimero.tijl.coosemans.org [127.0.0.1]) by kalimero.tijl.coosemans.org (8.15.2/8.15.2) with ESMTP id w66KoUvU051137; Fri, 6 Jul 2018 22:50:30 +0200 (CEST) (envelope-from tijl@FreeBSD.org) Date: Fri, 6 Jul 2018 22:50:30 +0200 From: =?UTF-8?B?VMSzbA==?= Coosemans To: "Rodney W. Grimes" Cc: rgrimes@freebsd.org, Warner Losh , Hans Petter Selasky , src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r336025 - in head/sys: amd64/include i386/include Message-ID: <20180706225030.2e689882@kalimero.tijl.coosemans.org> In-Reply-To: <201807061809.w66I9RVR053596@pdx.rh.CN85.dnsmgr.net> References: <201807061809.w66I9RVR053596@pdx.rh.CN85.dnsmgr.net> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2018 20:51:44 -0000 On Fri, 6 Jul 2018 11:09:27 -0700 (PDT) "Rodney W. Grimes" wrote: > > On Fri, Jul 6, 2018, 12:27 PM Rodney W. Grimes < > > freebsd@pdx.rh.cn85.dnsmgr.net> wrote: > > > > > > On Fri, Jul 6, 2018 at 9:52 AM, Rodney W. Grimes < > > > > freebsd@pdx.rh.cn85.dnsmgr.net> wrote: > > > > > > > > > > On Fri, Jul 6, 2018 at 9:32 AM, Rodney W. Grimes < > > > > > > freebsd@pdx.rh.cn85.dnsmgr.net> wrote: > > > > > > > > > > > > > > Author: hselasky > > > > > > > > Date: Fri Jul 6 10:13:42 2018 > > > > > > > > New Revision: 336025 > > > > > > > > URL: https://svnweb.freebsd.org/changeset/base/336025 > > > > > > > > > > > > > > > > Log: > > > > > > > > Make sure kernel modules built by default are portable between > > > UP > > > > > and > > > > > > > > SMP systems by extending defined(SMP) to include > > > > > defined(KLD_MODULE). > > > > > > > > > > > > > > > > This is a regression issue after r335873 . > > > > > > > > > > > > > > > > Discussed with: mmacy@ > > > > > > > > Sponsored by: Mellanox Technologies > > > > > > > > > > > > > > Though this fixes the issue, it also means that now when > > > > > > > anyone intentionally builds a UP kernel with modules > > > > > > > they are getting SMP support in the modules and I am > > > > > > > not sure they would want that. I know I don't. > > > > > > > > > > > > > > > > > > > > > > > > > On UP systems, these additional opcodes are harmless. They take a few > > > > > extra > > > > > > cycles (since they lock an uncontested bus) and add a couple extra > > > memory > > > > > > barriers (which will be NOPs). On MP systems, atomics now work by > > > > > default. > > > > > > Had we not defaulted like this, all modules built outside of a kernel > > > > > build > > > > > > env would have broken atomics. Given that (a) the overwhelming > > > majority > > > > > > (99% or more) is SMP and (b) the MP code merely adds a few cycles to > > > > > what's > > > > > > already a not-too-expensive operation, this was the right choice. > > > > > > > > > > > > It simply doesn't matter for systems that are relevant to the project > > > > > > today. While one could try to optimize this a little (for example, by > > > > > > having SMP defined to be 0 or 1, say, and changing all the ifdef SMP > > > to > > > > > if > > > > > > (defined(SMP) && SMP != 0)), it's likely not going to matter enough > > > for > > > > > > anybody to make the effort. UP on x86 is simply not relevant enough > > > to > > > > > > optimize for it. Even in VMs, people run SMP kernels typically even > > > when > > > > > > they just allocate one CPU to the VM. > > > > > > > > > > > > So while we still support the UP config, and we'll let people build > > > > > > optimized kernels for x86, we've flipped the switch from pessimized > > > for > > > > > SMP > > > > > > modules to pessimized for UP modules, which seems like quite the > > > > > reasonable > > > > > > trade-off. > > > > > > > > > > > > Were it practical to do so, I'd suggest de-orbiting UP on x86. > > > However, > > > > > > it's a lot of work for not much benefit and we'd need to invent much > > > > > crazy > > > > > > to get there. > > > > > > > > > > Trivial to fix this with > > > > > +#if defined(SMP) || !defined(_KERNEL) || defined(KLD_MODULE) || > > > > > !defined(KLD_UP_MODULES) > > > > > > > > > > > > Nope. Not so trivial. Who defines KLD_UP_MODULES? > > > > > > Call it SMP_KLD_MODULES, and it gets defined the same place SMP does. > > > > > > > Not so simple. SMP is defined in the config file, and winds up in one of > No problem, that is where I would be defining this anyway, or in the > latest case removing it and SMP for my UP kernel build. > > > the option files. It will be absent for stand alone builds, > I am ok with that. And it would be reasonable to default to SMP. > > > though. These > > change tweak the default yo be inlined and to include the sequence that > > works everywhere. > > > > > > > > > And really, it's absolutely not worth it unless someone shows up with > > > > numbers to show the old 'function call to optimal routine' is actually > > > > faster than the new 'inline to slightly unoptimal code'. Since I think > > > the > > > > function call overhead is larger than the pessmizations, I'm not sure > > > what > > > > the fuss is about. > > > > > > I have no issues with the SMP converting from function calls to > > > inline locks, I just want to retain the exact same code I had > > > before any of these changes, and that was A UP built system > > > without any SMP locking. Is it too much to ask to keep what > > > already worked? > > > > > > > This doesn't enable or disable locks in the muted sense. It just changes > > the atomic ops for the kernel from a function call to an inlined function. > > The inlining is more efficient than the call, even with the overhead added > > by always inlining the same stuff. It still is faster than before. > > > > And userland has done this forever... > > > > So I honestly think even UP builds are better off, even if it's not hyper > > optimized for UP. The lock instruction prefix is minimal overhead (a cycle > > I think). > > I do not believe, and Bruce seems to have evidence, that LOCK is not > a one cycle cost. And in my head I know that it can not be that > simple as it causes lots of very special things to happen in the > pipeline to ensure you are locked. > > > This is different than the mutexes we optimize for the UP cases > > (and which aren't affected by this change). It's really not a big deal. > > CPU's are not getting any faster, cycles are cycles, and I think we > should at least investigate further before we just start making > assumptions about the lock prefix being a 1 cycle cheap thing to > do. Just install opt_*.h headers already. It's not just about the SMP option. The nvidia-driver ports want to know if PAE is enabled on i386.