From owner-freebsd-arch@freebsd.org Sat Nov 14 18:48:36 2020 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0583B4623EC for ; Sat, 14 Nov 2020 18:48:36 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CYPVq6Vwfz3n3T; Sat, 14 Nov 2020 18:48:35 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 6C2BA5C00A4; Sat, 14 Nov 2020 13:48:35 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Sat, 14 Nov 2020 13:48:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsco.org; h= content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s=fm1; bh=n vAXAIETzkZa2C/sm5wBaq+9m/Tv1nAKcXWwdWNFYAk=; b=P95BAkJ37YCSD/dHT Fi1AtT7RfS8u+94geKSKkNQlsIe3K5iPSg5XSAcn/iVcKosmjb7wxnRlVb7MKdpA ef/2QkUHXaZYEXS3bGUhKzUf0xMRAh/CggPBDGNiZk1yKBOjDSPbKcGY9Fl48EOX fgPcm+rsOcrplrfl7Ku0GAcrOEhPqPL73LerUP9LkCSoKTuAwfhwJkcS3toXg+MD S90hadBDSU944bClHUN71sYtrCvq5ltLLjtcbzr3GDUu1yYHx1kfsSjX8YnrElyi rtdBA7Lk6x9Zfvcz5ach1nZhdrq1TPy1daEt9IwPv+sVCuqEtD/5w449oG1H64kf woTcA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=nvAXAIETzkZa2C/sm5wBaq+9m/Tv1nAKcXWwdWNFY Ak=; b=cZQy0tfgXyr/pTVppBDdUeE2DrNfNYHDabYJJhORBtY+QJsegyC1BEWks L0xzN4Ns7wH8VQlXULPNV2K5q4eoLXYg2DYKOyZ/Gp5dluRKai7riGuq1j/Owurt +GctQaqdyq077Svr7ferxHqvnEhlmnU8LHSL4KlXYNc009w5QjNBjUKLqQ9DOmy0 zNWJn/aAQVJTTqMPPEEZIZmJCyS90P9cS2SHe9NJVl8jg2kqfsSmkoxoFZnnLYjQ c2d0VMmw3894zD4ZdBQaQjhwT2r6KBYGnaFaULED2/wRlffC5Sf6Da8nS0Gspluh BOvipQhL2lj3rXL1HxVcHaK/VhDAw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedruddvjedguddukecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurheptggguffhjgffgffkfhfvofesth hqmhdthhdtjeenucfhrhhomhepufgtohhtthcunfhonhhguceoshgtohhtthhlsehsrghm shgtohdrohhrgheqnecuggftrfgrthhtvghrnhepfeejgefgjefhgfdtjeevjeekgeevie elueehjefgudetvefgtdetgffggefgvdegnecukfhppeekrdegiedrkeelrddvudefnecu vehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepshgtohhtth hlsehsrghmshgtohdrohhrgh X-ME-Proxy: Received: from [192.168.0.114] (unknown [8.46.89.213]) by mail.messagingengine.com (Postfix) with ESMTPA id A4B513280060; Sat, 14 Nov 2020 13:48:34 -0500 (EST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.4\)) Subject: Re: MAXPHYS bump for FreeBSD 13 From: Scott Long In-Reply-To: Date: Sat, 14 Nov 2020 11:48:34 -0700 Cc: Alexander Motin , "freebsd-arch@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <634E35E9-9E2B-40CE-9C70-BB130BD9D614@samsco.org> References: To: Konstantin Belousov X-Mailer: Apple Mail (2.3608.120.23.2.4) X-Rspamd-Queue-Id: 4CYPVq6Vwfz3n3T X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Nov 2020 18:48:36 -0000 > On Nov 14, 2020, at 11:37 AM, Konstantin Belousov = wrote: >=20 > On Sat, Nov 14, 2020 at 10:01:05AM -0500, Alexander Motin wrote: >> On Fri, 13 Nov 2020 21:09:37 +0200 Konstantin Belousov wrote: >>> To put the specific numbers, for struct buf it means increase by = 1792 >>> bytes. For bio it does not, because it does not embed vm_page_t[] = into >>> the structure. >>>=20 >>> Worse, typical struct buf addend for excess vm_page pointers is = going >>> to be unused, because normal size of the UFS block is 32K. It is >>> going to be only used by clusters and physbufs. >>>=20 >>> So I object against bumping this value without reworking buffers >>> handling of b_pages[]. Most straightforward approach is stop using >>> MAXPHYS to size this array, and use external array for clusters. >>> Pbufs can embed large array. >>=20 >> I am not very familiar with struct buf usage, so I'd appreciate some >> help there. >>=20 >> Quickly looking on pbuf, it seems trivial to allocate external = b_pages >> array of any size in pbuf_init, that should easily satisfy all of = pbuf >> descendants. Cluster and vnode/swap pagers code are pbuf descendants >> also. Vnode pager I guess may only need replacement for >> nitems(bp->b_pages) in few places. > I planned to look at making MAXPHYS a tunable. >=20 > You are right, we would need: > 1. move b_pages to the end of struct buf and declaring it as flexible. > This would make KBI worse because struct buf depends on some debugging > options, and than b_pages offset depends on config. >=20 > Another option could be to change b_pages to pointer, if we are fine = with > one more indirection. But in my plan, real array is always allocated = past > struct buf, so flexible array is more correct even. >=20 I like this, and I was in the middle of writing up an email that = described it. There could be multiple malloc types or UMA zones of different sizes, depending on the intended i/o size, or just a runtime change to the size = of a single allocation size. > 2. Preallocating both normal bufs and pbufs together with the arrays. >=20 > 3. I considered adding B_SMALLPAGES flag to b_flags and use it to = indicate > that buffer has 'small' b_pages. All buffers rotated through = getnewbuf()/ > buf_alloc() should have it set. >=20 This would work nicely with a variable sized allocator, yes. > 4. There could be some places which either malloc() or allocate struct = buf > on stack (I tend to believe that I converted all later places to = formed). > They would need to get special handling. >=20 I couldn=E2=80=99t find any places that allocated a buf on the stack or = embedded it into another structure. > md(4) uses pbufs. >=20 > 4. My larger concern is, in fact, cam and drivers. >=20 Can you describe your concern? >>=20 >> Could you or somebody help with vfs/ffs code, where I suppose the >> smaller page lists are used? > Do you plan to work on this ? I can help, sure. >=20 > Still, I wanted to make MAXPHYS (and 'small' MAXPHYS, this is not same = as > DFLPHYS), a tunable, in the scope of this work. Sounds great, thank you for looking at it. Scott