From owner-freebsd-arch@freebsd.org Fri Nov 13 23:06:56 2020 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8F97F2EFF99 for ; Fri, 13 Nov 2020 23:06:56 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound5a.ore.mailhop.org (outbound5a.ore.mailhop.org [44.233.67.66]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CXvHM4RjJz3Hsh for ; Fri, 13 Nov 2020 23:06:55 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1605308809; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=nlATxSZY6SSIcDyKdSRo6Z4RAiuBO3sJkFkD5BtO6lk2aH0exI8dpfZewFzHM3+0Ma8n4rWavuY1S ytuzAz/zybKfn8f/8RxWrJJyFi37dL/iFT2zU0Sq8huei7DtWH5IGBFhMhYYWU30ru3fY8e8trUO5m sGcrPzCuR2cgyfYSQsdqNZiCgRf9XchYc9x4HdxieTLW/V2tOkOzPJyDEc84xWf0gBfmi7noD/h8Cv XxxT/DMcfMaZtDbWtHILNVtw2jmtrcMcf2zP+PmIQg5I7rb90MtpqCzafajJQ9uvORgoG7pS1xobuB saAumLu+6+mxtKtqPfKoY9kTB4KQ0WA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:to:from:subject:message-id:dkim-signature:from; bh=yV5Th6jVnQpch4dw8NDr75809TtJ5L9UMZBd/9ftQsQ=; b=LLqINNYTRr/6nPOQgPquNP9JvhSTDcykawj+P1YUaAB+Y+AA+Y5+TopDIkn2oLLRlwdN6tkBGze9F 46OUlZdLYVF3xN02bCBgSjtyDQlLMqxA1kLAVVKVzOcTTvBMiFvcdt3VfjF581o7vlVHlXHi1Ciepg hI7Jnl9PDPcus38f9ibamk/pvHZgps9iB52dCnAk/naFeInfSGl2iz6PWykjPau+1tE+czHBt8xMLt cEzetsyfo5D/WCfw5pQBgKGwiWQWAiDV/pzSr/z376VL8gvbpOqgwWRWJ0fqXB+kw1Ipd/mrjTlVwu BHJi+1izbuBPWaeux394W5dBJcFixlQ== ARC-Authentication-Results: i=1; outbound3.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:to:from:subject:message-id:from; bh=yV5Th6jVnQpch4dw8NDr75809TtJ5L9UMZBd/9ftQsQ=; b=SnjaFgVEOGhAfJXC+eqZSrLSO6shFuuf11h/k/cdZM2OPP0PdAb9OTxObVpPvDII69eJ9qXaJ/XFo JZH/wIJB5fsGRGnu4qnrV1LYCskR0ZWWnQmwfb6RFngwdGEiQAqR44yxtMbhxE9Q3lGFbopfzAfS7w ffxMMcd0JATz+0Pz2M2YaDYNTXMqEY2WMJXoYglVCs5jIomgyLTZKqMbq0rlwXPoF5mg4hvsEDpFML utwuZKwLDB0ytPt/hk9CXOICIp6h0lEK4KLKaP80c0zJhklu3/ubkaxvwo+5Nu5eOIcK+t/XcZEMDZ WGPcBnylvlllZu7m4inBVP0xU5TP0Yg== X-MHO-RoutePath: aGlwcGll X-MHO-User: e737494d-2604-11eb-8b38-614106969e8d X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (c-67-177-211-60.hsd1.co.comcast.net [67.177.211.60]) by outbound3.ore.mailhop.org (Halon) with ESMTPSA id e737494d-2604-11eb-8b38-614106969e8d; Fri, 13 Nov 2020 23:06:47 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id 0ADN6ktN028319; Fri, 13 Nov 2020 16:06:46 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <7ff70bea498bf4ec037266ec08b4224c55f76ef3.camel@freebsd.org> Subject: Re: MAXPHYS bump for FreeBSD 13 From: Ian Lepore To: Warner Losh , "freebsd-arch@freebsd.org" Date: Fri, 13 Nov 2020 16:06:46 -0700 In-Reply-To: References: Content-Type: text/plain; charset="ASCII" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4CXvHM4RjJz3Hsh X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:16509, ipnet:44.224.0.0/11, country:US] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2020 23:06:56 -0000 On Fri, 2020-11-13 at 11:33 -0700, Warner Losh wrote: > Greetings, > > We currently have a MAXPHYS of 128k. This is the maximum size of I/Os that > we normally use (though there are exceptions). > > I'd like to propose that we bump MAXPHYS to 1MB, as well as bumping > DFLTPHYS to 1MB. > > 128k was good back in the 90s/2000s when memory was smaller, drives did > smaller I/Os, etc. Now, however, it doesn't make much sense. Modern I/O > devices can easily do 1MB or more and there's performance benefits from > scheduling larger I/Os. > > Bumping this will mean larger struct buf and struct bio. Without some > concerted effort, it's hard to make this be a sysctl tunable. While that's > desirable, perhaps, it shouldn't gate this bump. The increase in size for > 1MB is modest enough. > > The NVMe driver currently is limited to 1MB transfers due to limitations in > the NVMe scatter gather lists and a desire to preallocate as much as > possible up front. Most NVMe drivers have maximum transfer sizes between > 128k and 1MB, with larger being the trend. > > The mp[rs] drivers can use larger MAXPHYS, though resource limitations on > some cards hamper bumping it beyond about 2MB. > > The AHCI driver is happy with 1MB and larger sizes. > > Netflix has run MAXPHYS of 8MB for years, though that's likely 2x too large > even for our needs due to limiting factors in the upper layers making it > hard to schedule I/Os larger than 3-4MB reliably. > > So this should be a relatively low risk, and high benefit. > > I don't think other kernel tunables need to change, but I always run into > trouble with runningbufs :) > > Comments? Anything I forgot? > > Warner > Will this have any negative implications for embedded systems running slow storage such as sdcard? -- Ian