From owner-freebsd-hackers@freebsd.org Thu Jan 9 21:14:28 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1EFD91F937D for ; Thu, 9 Jan 2020 21:14:28 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qv1-xf2f.google.com (mail-qv1-xf2f.google.com [IPv6:2607:f8b0:4864:20::f2f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47tzQC31fWz3DV7 for ; Thu, 9 Jan 2020 21:14:27 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qv1-xf2f.google.com with SMTP id dc14so3609225qvb.9 for ; Thu, 09 Jan 2020 13:14:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=mtAFRaiOG6WV2eaFB0PKKC38Rs2qYpxrCSH/9M2G4l0=; b=f1gPARQ/wMZ5jjFvJgIgzJWRdEa78qa0TG233FgFsoZpnzYx+RtVhIkYjPjfWz4umW 5mv7m6x33COFRx2cfSJIMHZfu6N850oRBBhZvZ15nVnaTjENcioPKrG6Gq9ckuDE/FiP R776+lSHVggoUGbE6jWHJG64g0LnljU0Un8LbNeof5Dr3Kjan+V7M8Y7N9tpBwklAVbF 21XxEF7eQuCEfnqFsMRr9abNfoltbu+v1wUCXAL7BETf7awg2szmCJBNdKGQKPyM0fEQ +8a3xfVP5g7Q2JfozX55Wmnr0oGzVYDvldKiTqFIpSXAjD1MuuxljnApO2TNeT1N6a6i P4rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=mtAFRaiOG6WV2eaFB0PKKC38Rs2qYpxrCSH/9M2G4l0=; b=H4g6aOh7PIyw4+AlEvdy3QcSee1VZ7/hA+ekaOtFJyyjS6K29swt8y3o4YxQ5a7KZy L5y78Bl+8RoNq7/slCH9o7MdxbblQg4Cx6jTjvDCG/EVsApVWYA/G3BsGeYlrjzxL70y apCgiU8vO37Hnw9pyihAy9LpswzGlnlCcENFjrn3Gf3xqTNqPYC38CWoyJBVa8y/hiz2 k3wtG45m+vtq/j3nqAHF7YBuTyhvxrmgc8FmmxaCIAO0pA7bJEgztro3FZgGwLA1YkgT P2JTZOdxPG6V6txP/cUK3469Xhtw3JDc06uuND4wtRWbIj2A/hZqxTECjdfrfaJSLV8I KGFg== X-Gm-Message-State: APjAAAVK1SrzhMTgJ1SCl2R3hCzqBAC96JojILXIicx852PcYThauASy XxnJXJWh0DkdzTSXhZpjVOo/DnwiTpanzrGBLX2aTA== X-Google-Smtp-Source: APXvYqzVUhvL0qdANL5Li4tbrWLWVm6/MaTJ54aLFa2lwrfMXIWXhJ3bdZt3ROPOy0R+bm/EJSl4o7HNe+GP+ZCESbA= X-Received: by 2002:ad4:4810:: with SMTP id g16mr10193720qvy.22.1578604460843; Thu, 09 Jan 2020 13:14:20 -0800 (PST) MIME-Version: 1.0 References: <20200108105136.0d54ebce@ernst.home> <20200108141810.GX23031@kib.kiev.ua> <20200109164519.33fc7478@ernst.home> <20200109204943.GC25924@server.rulingia.com> In-Reply-To: <20200109204943.GC25924@server.rulingia.com> From: Warner Losh Date: Thu, 9 Jan 2020 14:14:09 -0700 Message-ID: Subject: Re: maximum MAXBSIZE To: Peter Jeremy Cc: Gary Jennejohn , Hans Petter Selasky , Rick Macklem , Conrad Meyer , "freebsd-hackers@freebsd.org" , Wojciech Puchar , Konstantin Belousov X-Rspamd-Queue-Id: 47tzQC31fWz3DV7 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=f1gPARQ/; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::f2f) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-4.37 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCPT_COUNT_SEVEN(0.00)[8]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.37)[ip: (-7.82), ipnet: 2607:f8b0::/32(-2.12), asn: 15169(-1.85), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; FREEMAIL_CC(0.00)[gmail.com] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Jan 2020 21:14:28 -0000 On Thu, Jan 9, 2020 at 1:55 PM Peter Jeremy wrote: > On 2020-Jan-09 16:45:19 +0100, Gary Jennejohn > wrote: > >On Thu, 9 Jan 2020 15:21:25 +0100 (CET) > >Wojciech Puchar wrote: > >> why FreeBSD default is so completely wrong for modern hardware? > >> > >> i think 4MB is OK for HDDs, more may be optimal for RAID5 arrays. > > > >POLA (principle of least amazement). I certainly don't need a MAXPHYS set > >to 4MB on my desktop machine. > > What are the downsides of running with MAXPHYS set to 4MB (or similar)? > There's two issues. One, it makes every buf and bio 32 times larger. Second, there's a lot of drivers that say their max I/O size is MAXPHYS when really they mean max(128k,MAXPHYS). Newer hardware is better about it, but not perfect (I had to fix a NVMe bug because the format of SG lists we used is limited to 4k which means our NVMe driver can't do more than 1MB I/Os). DFLTPHYS also needs to be raised. There are (or were) some drivers in the tree that bogusly used DFLTPHYS as the maximum I/O, though I think I caught all of those. And once you bump MAXPHYS, there's other limits you'll run into with fast SSDs/NVMe drives (like runningbufs limiting write throughput). > > Not everyone using FreeBSD is running > >servers with large amounts of memory and disk storage. > > Actually, I disagree with this statement. MAXPHYS on x86 was doubled from > 64KB to 128KB in r32724 - 22 years ago. A small, embedded system today has > more RAM than a decent server had disk space then. I think we are well > overdue for an examination of many of the kernel parameters to take into > account that a "typical" user machine today has 3 orders of magnitude more > RAM, disk and performance than it had when most of the kernel parameters > were last tweaked. > Likely 1MB is the right place to have MAXPHYS for most people these days.... But there's a number of other parameters to tweak, and likely a few bugs to hunt... > >It's a trivial change if it's beneficial in a certain use scenario. The > >decision should be left up to the user. > > Actually, I suspect it would benefit most typical use cases - even an > average desktop machine does for more I/O than when the values were last > set. Also, adjusting it isn't quite that easy - it's a compile-time > constant so a user has to build their own kernel and the Project is trying > to get away from requiring users to build from source. > > And, before someone starts, the "it will hurt embedded systems" argument > isn't a good reason for keeping the status quo for two main reasons: > > 1) As I mentioned above, an embedded system today is larger than a decent > server was in 1998. > > 2) People running FreeBSD on embedded systems are going to want to build > from source to cater for their systems' idiosyncracies and needs, so they > can easily tune kernel parameters as needed. > I think we should look at changing these parameters, but some care is needed because there's often interrelated values that aren't obvious to someone new to this tuning. Warner