From owner-freebsd-arch@freebsd.org Sun Nov 15 17:22:13 2020 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E8F012EEA8A for ; Sun, 15 Nov 2020 17:22:13 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CYzXj0CmPz4cGx; Sun, 15 Nov 2020 17:22:12 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 4A54B5C00CD; Sun, 15 Nov 2020 12:22:12 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Sun, 15 Nov 2020 12:22:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsco.org; h= content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s=fm1; bh=G cUGDKkZvbL/zjlHBdsctRvXadbbioks/R2hj+kuuc8=; b=cgsMcpAFsH3pLZq0O +D2LtBM8UgiVF71RkOGL0ybUlR2kk/ie6Ga1aSDtrd14qnwAZ2yrqbE5wa6CRnPW mdeMasaqyKW0tPXo7RXQyN7ivvishOoVfIN/21gDRUcRrkQ+9gBGzJ2x/7CeVwHX OKB1G6ZahdKf93l1QRv2BjcMQFOROWR3DqmHL3P74swQCrS3rj5RR9GTqm2784Ei grumkBIeK8pFq2sX69+1dVNW5sIKSK06eyN++8fbk+oZmxZuTpa/5/tLiYYuuEDR 1A+b5CSBSpovahYNIygzbA/I4buDhwB1GQl9SIV3qcZu827hAhF81ORU8Rpol9m5 XMNUg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=GcUGDKkZvbL/zjlHBdsctRvXadbbioks/R2hj+kuu c8=; b=KZ/FvZI0YBI4mG335yg4JBb/ywI9afk6f+HPNkgKomZKsMpbor0+djTrZ /QUI2wzTDxVps7BVt6VtgddQ8v1fwJyx0wQJLS1GwXVrVZXsKh7P6vGBhSddhaEu lU1QlgOcF2frpd0NZVlJjuPWWoAeY9ISzuZ4T/9EJ2gTP3EDoLV3gz3Cg0ciN0PW skc7bvUWZYS41lRz1LKG2x8LpXmWp2+aYpr17ZJFk6Q07CTQb77UlZSo3Cvm6OFv UdJJRpuPm97yulW6s9fJM0ryw2PkM9q+AqQqQfbEjjF5pQQo7uzeMDyR6GS0G9sA SAmf994NRaQtThbG6C2RXMgVQZdDA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedruddvledguddtfecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurheptggguffhjgffgffkfhfvofesth hqmhdthhdtjeenucfhrhhomhepufgtohhtthcunfhonhhguceoshgtohhtthhlsehsrghm shgtohdrohhrgheqnecuggftrfgrthhtvghrnhepfeejgefgjefhgfdtjeevjeekgeevie elueehjefgudetvefgtdetgffggefgvdegnecukfhppeekrdegiedrkeelrddvudefnecu vehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepshgtohhtth hlsehsrghmshgtohdrohhrgh X-ME-Proxy: Received: from [192.168.0.114] (unknown [8.46.89.213]) by mail.messagingengine.com (Postfix) with ESMTPA id 4F301328005E; Sun, 15 Nov 2020 12:22:11 -0500 (EST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.4\)) Subject: Re: MAXPHYS bump for FreeBSD 13 From: Scott Long In-Reply-To: <46AB39AF-B09A-4694-AA86-30EC82A1EB9E@samsco.org> Date: Sun, 15 Nov 2020 10:22:10 -0700 Cc: Alexander Motin , "freebsd-arch@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <0255CCBE-35C3-499F-80B6-44B57679C403@samsco.org> References: <46AB39AF-B09A-4694-AA86-30EC82A1EB9E@samsco.org> To: Konstantin Belousov X-Mailer: Apple Mail (2.3608.120.23.2.4) X-Rspamd-Queue-Id: 4CYzXj0CmPz4cGx X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=samsco.org header.s=fm1 header.b=cgsMcpAF; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=KZ/FvZI0; dmarc=none; spf=pass (mx1.freebsd.org: domain of scottl@samsco.org designates 66.111.4.29 as permitted sender) smtp.mailfrom=scottl@samsco.org X-Spamd-Result: default: False [-3.10 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.29]; RWL_MAILSPIKE_GOOD(0.00)[66.111.4.29:from]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[samsco.org:+,messagingengine.com:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[66.111.4.29:from]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; MIME_TRACE(0.00)[0:+]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.29:from]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[samsco.org:s=fm1,messagingengine.com:s=fm1]; FREEFALL_USER(0.00)[scottl]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[samsco.org]; SPAMHAUS_ZRD(0.00)[66.111.4.29:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MAILMAN_DEST(0.00)[freebsd-arch] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Nov 2020 17:22:14 -0000 > On Nov 15, 2020, at 10:11 AM, Scott Long wrote: >=20 >=20 >=20 >> On Nov 15, 2020, at 3:58 AM, Konstantin Belousov = wrote: >>=20 >> On Sat, Nov 14, 2020 at 06:40:44PM -0500, Alexander Motin wrote: >>> On 14.11.2020 13:37, Konstantin Belousov wrote: >>>> On Sat, Nov 14, 2020 at 10:01:05AM -0500, Alexander Motin wrote: >>>> 4. My larger concern is, in fact, cam and drivers. >>>=20 >>> I am actually the least concerned about this part. I've already >>> reviewed/cleaned it once, and can do again if needed. We have some >>> drivers unaware about MAXPHYS, and they should safely be limited to >>> DFLTPHYS, the others should properly adapt. And if you like to make >>> MAXPHYS tunable -- I'd be happy to take this part. >> Well, I looked at ahci(4) as a first driver, and it is already = problematic. >> It sizes internal structures, which are really some hardware command >> structures, based on MAXPHYS. Same for siis(4), but this seems to be = even >> worse due to +1. >>=20 >=20 > The MAXPHYS use for AHCI is related to the maximum size of a PRD = needed > for a command. AHCI specifies that you can have up to 65536 PRD = entries, > which translates into 2^16 + 2^12 =3D 256MB maximum I/O size. = However, the > AHCI driver seems inconsistent already in that it defines AHCI_PRD_MAX=20= > (via an indirection through AHCI_PRD_MASK) to be 2^22, or 4MB. This = is > a pretty reasonable max default for AHCI, and I don=E2=80=99t think we = need to jump > through hoops to make it dynamic or make it larger for the future. My > recommendation is to abandon the partial changes you have for AHCI and > use AHCI_PRD_MAX in place of MAXPHYS for the structure sizing and for > the cpi->maxio attribute. >=20 > I have less of an opinion on the SIIS driver since that hardware is = pretty > low-performance and mostly obsolete at this point. I guess we should = do the > same as AHCI, but I don=E2=80=99t know if there are details about SIIS = that diverge > in significant ways. I forgot to add that I=E2=80=99m ok with a static sizing of 4MB for the = PRD array because an AHCI device is only ever going to have 32 outstanding commands. The memory increase here is going to be pretty small compared to NVME or MPR/MPS that has the potential for a much deeper queue, and the size of = a PRD element is much smaller than a vm_page_t element from the buf = discussion. Even a system with a lot of AHCI devices will still not grow the memory = footprint that much. Scott