From owner-freebsd-arch@freebsd.org Sat Nov 14 00:41:01 2020 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 04F6746359E for ; Sat, 14 Nov 2020 00:41:01 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x743.google.com (mail-qk1-x743.google.com [IPv6:2607:f8b0:4864:20::743]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CXxMv4Z4Yz3Nw5 for ; Sat, 14 Nov 2020 00:40:59 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x743.google.com with SMTP id d9so11080678qke.8 for ; Fri, 13 Nov 2020 16:40:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=jZLfUR3aI4oqj6Iy3N+s5NibQ8PEzx0hr49Ii0uYbmw=; b=0bl1jz9ulPutfl1+rwqQwCPW4iOVEZNsh+oPQ2Qkn2+yd0OvbV8zXxyOUt7/Jemb4V oS8oJOB2AaAUUdVT4nqrqDTaYB21puYk0f3oj7fZUw43+aqA+17S8CouSUH5Lp1KR70U fFFskt5l2edq1YHkIoFk1zb5mLoNziv9RH8BOiDSDHgOw9WuRJBPp5thJjC5iH6iPzif ARGW5XwUdbEV9GxgrMIdJTOYSs2kawBfwUqjv5VYxYprgpPA8mTfDYPSg9PRrnoIUS+w 0WgX+63A0h4jZYw8VJpoghai0bXpG9r688A5rp4/zBzGxt2rpLCoS1I+U7Vn4IXifyns mZ/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=jZLfUR3aI4oqj6Iy3N+s5NibQ8PEzx0hr49Ii0uYbmw=; b=NLArrdeRoER8CxJ1oN5NGmYVeum4mvC+qCrOdVwAdJhtKYBQpbDX+XEMU2clE09X91 gWfkPK0R+Y2d99b5rMBivjhKRQO1Wcjzyrihr7kEhTxq6TOImXsEUhnsz9zr7+1af98Y hnbGQvUXnfCVF0HuJWGblUd+MSZWiJ7i1SyiLzuJhmkzhovKjYzklVer1bAt67PurK2m sEbryoYlQbHuG5hEO25i5rJAVAadvuQMBpMctHpGFsyskijB75V4PGBUjRf8y5jwTxHG oeak+gBtcK6j6tJqnhp3HI5VmLOcYL4iiPQWPI7LQKQsveJVXfR+QcVa99BGclfVTozI eh3A== X-Gm-Message-State: AOAM533opIsRaXcY/mm7GZocH4cu7PHvqDIOf/Z3YCi7nRj/Tn6TN549 lfGfR294sI85g+Hx2QLNnrsJwx5f+AAHsc3fUENzWSdFpccGNw== X-Google-Smtp-Source: ABdhPJwmE1u9pjeSGsxCWwF/m7YhwhBDfJMEG0MXqTQ4cS7g4nL28xBoLLlDx1+YDG6E4ymDICwpv7WxaJxM8R7bTwI= X-Received: by 2002:a37:6307:: with SMTP id x7mr4641785qkb.195.1605314458738; Fri, 13 Nov 2020 16:40:58 -0800 (PST) MIME-Version: 1.0 References: <7ff70bea498bf4ec037266ec08b4224c55f76ef3.camel@freebsd.org> In-Reply-To: <7ff70bea498bf4ec037266ec08b4224c55f76ef3.camel@freebsd.org> From: Warner Losh Date: Fri, 13 Nov 2020 17:40:47 -0700 Message-ID: Subject: Re: MAXPHYS bump for FreeBSD 13 To: Ian Lepore Cc: "freebsd-arch@freebsd.org" X-Rspamd-Queue-Id: 4CXxMv4Z4Yz3Nw5 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=0bl1jz9u; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::743) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-3.00 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arch@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::743:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::743:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::743:from]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; MAILMAN_DEST(0.00)[freebsd-arch] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Nov 2020 00:41:01 -0000 On Fri, Nov 13, 2020 at 4:06 PM Ian Lepore wrote: > On Fri, 2020-11-13 at 11:33 -0700, Warner Losh wrote: > > Greetings, > > > > We currently have a MAXPHYS of 128k. This is the maximum size of I/Os > that > > we normally use (though there are exceptions). > > > > I'd like to propose that we bump MAXPHYS to 1MB, as well as bumping > > DFLTPHYS to 1MB. > > > > 128k was good back in the 90s/2000s when memory was smaller, drives did > > smaller I/Os, etc. Now, however, it doesn't make much sense. Modern I/O > > devices can easily do 1MB or more and there's performance benefits from > > scheduling larger I/Os. > > > > Bumping this will mean larger struct buf and struct bio. Without some > > concerted effort, it's hard to make this be a sysctl tunable. While > that's > > desirable, perhaps, it shouldn't gate this bump. The increase in size for > > 1MB is modest enough. > > > > The NVMe driver currently is limited to 1MB transfers due to limitations > in > > the NVMe scatter gather lists and a desire to preallocate as much as > > possible up front. Most NVMe drivers have maximum transfer sizes between > > 128k and 1MB, with larger being the trend. > > > > The mp[rs] drivers can use larger MAXPHYS, though resource limitations on > > some cards hamper bumping it beyond about 2MB. > > > > The AHCI driver is happy with 1MB and larger sizes. > > > > Netflix has run MAXPHYS of 8MB for years, though that's likely 2x too > large > > even for our needs due to limiting factors in the upper layers making it > > hard to schedule I/Os larger than 3-4MB reliably. > > > > So this should be a relatively low risk, and high benefit. > > > > I don't think other kernel tunables need to change, but I always run into > > trouble with runningbufs :) > > > > Comments? Anything I forgot? > > > > Warner > > > > Will this have any negative implications for embedded systems running > slow storage such as sdcard? > It will work. If you have memory pressure, you may need to compile with a smaller MAXPHYS. The savings is about 1700 bytes per struct buf. Warner