From owner-freebsd-fs@freebsd.org Wed Sep 5 03:16:51 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C116FE0E3A for ; Wed, 5 Sep 2018 03:16:51 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-21.consmr.mail.gq1.yahoo.com (sonic304-21.consmr.mail.gq1.yahoo.com [98.137.68.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DA6298E327 for ; Wed, 5 Sep 2018 03:16:50 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: .hf.p_4VM1mNlYQsyB9Us4lXagXaG6Z85YEsvYKIhatRod6i0hfvBg1r1FPVkJG Oxhl9MSor5_OJvu3MTlqCeLxStO8fGgwo66GXLDcs4PqYl5YBPU5zyLd7p0SSAGEAUyJEQ_mJ1Z2 uNJejIAyv_9Y5wh4OL8mLlw5IfkfvWQJ.Hq3W_xFgm1cOKHXYObyDquPlx2BNTrK7uUAdLFR7GY1 80JUgqZ9z9bq7pXhxNraDnYI5wrHKmKdFxU.bY2C2sqftv0uOoVxhOeiD6JSpS134ZVWuPyDSBTr wyMEq6fMLat9CWNIDsFeXcn3ndCo1vGLVVLN5nAzoZVPefdhCaTJRzYXCXYdkeGqfzNt4jFUzsUH VmmvTdoLbGHYuUhipOaKA4VN7Z15LTe8IU1Y4TbKrZ898TdpS9s6RKqmseCuHX2CazljMKybQGiE IaRFlQOja8sNKuSBT4l4kZshFg6cCxSG.J3t0nS7G50L0CMtO44kVfvww4UgxcUuH7TOcxPyH5hg PXiIY_aMXPptWDMMp1R.kA8fgwvTR52hHaCWXmlisamK_OwAI4OiBM7OW7bmB4_FvHkjHDUKitEu Dvd6_MhRH2cGn422L0UcHS1V8XnfYwQMPeh9GpxzDNvPhK0vXftr7xHiwpiTs2smRU5bGgFx5Yl1 5JrBVCOiI9ee9gnHNU8UF9_BMo9H9FuIzroZSVQp1w3UygpMCkbYXzOW8iP4kyh8I92PeOCs7zwK oS6Q9bAkAGT3JdhFVMH9cQ9y3411KnrwIkCEms8NNszG8g3oLhL.GnLWJshzyxg36J4QQQnyyARN CyGeThqlm9T5ue1ivFPKlVjKWaAS5GZHpnq2pYO7tEvhE581mc2GzKDGYWwXdO4DubNpRwIPH4zD 2FEoitCb2L9QJ9feUNRLSmE1G5NUnznsEIP_0.MqhbM.1nfoQx5cBqIInPDVQaTzH0P5gxIXYbkA LrybQL3rKNiL8dp9HPxY.BcPsG77Fn3IgF8zMWem4o32sRvW7LznYBG5yEv2Asxaav6Ct0EXz3Zo rIfU24G7O Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.gq1.yahoo.com with HTTP; Wed, 5 Sep 2018 03:16:42 +0000 Received: from ip70-189-131-151.lv.lv.cox.net (EHLO [192.168.0.105]) ([70.189.131.151]) by smtp424.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID e2f508d1d64b6898544458fba911a409; Wed, 05 Sep 2018 03:16:38 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Re: CFT: TRIM Consolodation on UFS/FFS filesystems From: Mark Millard In-Reply-To: <3F4C9F6F-667A-4E71-A73F-31CF6A0E754A@yahoo.com> Date: Tue, 4 Sep 2018 20:16:37 -0700 Cc: bob prohaska , FreeBSD Filesystems Content-Transfer-Encoding: quoted-printable Message-Id: <373CBC85-7A7C-4EEE-9A50-6AEAFEC05F63@yahoo.com> References: <201808240504.w7O54SPJ067702@chez.mckusick.com> <392FF90D-6D90-457B-94D2-9C4A05B46309@yahoo.com> <3F4C9F6F-667A-4E71-A73F-31CF6A0E754A@yahoo.com> To: Kirk McKusick X-Mailer: Apple Mail (2.3445.9.1) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2018 03:16:51 -0000 [Some results.] On 2018-Sep-4, at 12:06 AM, Mark Millard wrote: > [Test status update. I give some details of the > test context and complications/limitations as > well.] >=20 > On 2018-Aug-23, at 10:58 PM, Mark Millard = wrote: >=20 >> On 2018-Aug-23, at 10:04 PM, Kirk McKusick = wrote: >>=20 >>> So are you in a position to try out the TRIM consolidation = implementation? >>> In particular to see if it helps? >>=20 >> . . . >=20 >=20 > I finally have access to a mmcsdxc card of about the > same capacity as the on-an-adpater e.MMC that I used > to use with the Pine64+ 2GB. I've copied things over, > enabled trim on the (root) file system, and have > "vfs.ffs.dotrimcons: 1". I did a fsck_ffs -E. The > complete UFS (root) file system and the swap partition > are on the microsdxc card, no other storage devices > involved. (Swap not getting trim but the UFS file > system getting it.) >=20 > I'm doing an incremental -j4 buildworld buildkernel > in this environment, jumping from a build of -r337400 > to -r338450, but the Pine64+ 2GB is running -r338341. > The build removes old files as it goes along. We will > see how it goes. It is using "vm.pageout_oom_seq: 120". > I'm using: LDFLAGS.lld +=3D -Wl,--no-threads >=20 > I'll note that Ian Lepore reported in an exchange about > RPI3 testing: >=20 > QUOTE > Trim consolidation at the ufs layer might not have as much effect on > mmcsd as it does on other devices. The mmcsd driver has always > consolidated adjacent trim requests itself even when they arrive in = the > IO queue as a large number of small BIO_DELETE commands. It assembles > blocks of adjacent deletes until they reach the size of a full erase > block, then issue one erase command (I've never been convinced that > doing so is necessary, to me the sd spec implies you can delete > individual sectors). > END QUOTE >=20 > I've no known, good means of isolating and > comparing/contrasting the two consolidations (vs. the > combination of both). >=20 > As stands, at best I'll be able to report if I see any > failures or evidence of any bottlenecks. (Similarly for > a later poudriere-devel bulk.) >=20 > The microsdcx card tested for this is from a: >=20 > Sandisk Ultra 128GB Micro SDXC UHS-I Card with Adapter = SDSQUAR-128G-GN6MA >=20 > It has the application "A1" rating. About 2/3 of the UFS > partition is "Avail". About 5.5 GiByte of the microsdcx card > is not in any partition (but that was dd'd from the e.MCC > too and I do not know how to readily trim those areas). >=20 > Other points of context: >=20 > The Pine64+ 2GB is running a non-debug kernel (but with > symbols), as is my norm unless I'm looking into a > failure. Summary: So far, so good. The incremental -j4 buildworld buildkernel completed fine in somewhat under 13 hr 20 minutes. It included building the llvm related materials. The installkernel . . . installworld . . . reboot worked fine as well. I saw no evidence of bottlenecks or other issues. The context is now -r338450 based. The kernel has Mark Johnston's patch for reporting on long latencies in swapping I/O. They did not report anything. There were no console messages at all during the build, so nor reported I/O warnings or errors. What I saw via "gstat -pd" looked good to me, as did what I was via "top -CawSores". (An RPI3 has only 1 GiByte of RAM instead of 2 GiByte, and so has more I/O required and would be a different test than I ran.) The svnlite update for /usr/ports and the the poudriere bulk with PARALLEL_JOBS=3D2 and ALLOW_MAKE_JOBS=3Dyes has built 25 of 26 ports for its update, the last one being a devel/gcc8 full bootstrap that will likely continue for 10 to 11 hours more. After the pkg update and pkg upgrade I'll add some x11 related ports to the file that I use with -f and start another poudriere bulk, at least that is my current plan. Note: My poudriere-devel use is based on: # poudriere jail -l JAILNAME VERSION ARCH METHOD TIMESTAMP = PATH FBSDcortexA53jail 12.0-CURRENT arm64.aarch64 null 2017-10-20 00:19:15 = /usr/obj/DESTDIRs/clang-cortexA53-installworld-poud The PATH is to an installworld result from the same buildworld used for the Pine64+ 2GB's own installworld, but also with "distrib-dirs distribution DB_FROM_SRC=3D1". Also: # poudriere ports -l PORTSTREE METHOD TIMESTAMP PATH default null 2017-10-20 00:19:48 /usr/ports currently with: # svnlite info /usr/ports/ | grep "Re[plv]" Relative URL: ^/head Repository Root: svn://svn.freebsd.org/ports Repository UUID: 35697150-7ecd-e111-bb59-0022644237b5 Revision: 478753 Last Changed Rev: 478753 =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)