Date: Sat, 15 May 2021 19:24:38 -0400 From: Shawn Webb <shawn.webb@hardenedbsd.org> To: Edward Tomasz Napierala <trasz@FreeBSD.org> Cc: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Subject: Re: git: 3394d4239b85 - main - cam: allocate CCBs from UMA for SCSI and ATA IO Message-ID: <20210515232438.7siob6di7bdvaefs@mutt-hbsd> In-Reply-To: <202105151105.14FB5nv7005915@gitrepo.freebsd.org> References: <202105151105.14FB5nv7005915@gitrepo.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--6k5h3synwn52j6at Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, May 15, 2021 at 11:05:49AM +0000, Edward Tomasz Napierala wrote: > The branch main has been updated by trasz: >=20 > URL: https://cgit.FreeBSD.org/src/commit/?id=3D3394d4239b85b5577845d9e6de= 4e97b18d3dba58 >=20 > commit 3394d4239b85b5577845d9e6de4e97b18d3dba58 > Author: Edward Tomasz Napierala <trasz@FreeBSD.org> > AuthorDate: 2021-05-15 10:17:22 +0000 > Commit: Edward Tomasz Napierala <trasz@FreeBSD.org> > CommitDate: 2021-05-15 11:03:49 +0000 >=20 > cam: allocate CCBs from UMA for SCSI and ATA IO > =20 > This patch makes it possible for CAM to use small CCBs allocated > from an periph-specific UMA zone instead of the usual, huge ones. > The end result is that CCBs issued via da(4) take 544B (size of > ccb_scsiio) instead of the usual 2kB (size of 'union ccb', ~1.5kB, > rounded up by malloc(9)). For ATA it's 272B. We waste less > memory, we avoid zeroing the unused 1kB, and it should be easier > to allocate those CCBs in low memory conditions. It should also > be possible to use uma_zone_reserve(9) to improve behaviour > in low memory conditions even further. > =20 > Note that this does not change the size, or the layout, of CCBs > as such. CCBs get allocated in various different ways, in particular > on the stack, and I don't want to redo all that. Instead, this > provides an opt-in mechanism for the periph to declare "my start() > callback is fine with receiving a CCB allocated from this UMA zone". > In other words, most of the code works exactly as it used to; the > change only happens to IOs issued by xpt_run_allockq(), which > is - conveniently - pretty much all that matters for performance. > =20 > The reason for doing it this way is that it's pretty small, localized > change, and can be implemented gradually and iteratively: take a > periph, make sure its start() callback only casts the CCBs it takes > to a particular type of CCB, for example ccb_scsiio, and that it only > casts CCBs returned by cam_periph_getccb() to that type, then add UMA > zone for that size, and declare it safe to XPT. > =20 > This is disabled by default. Set 'kern.cam.ada.enable_uma_ccbs=3D1' > and 'kern.cam.da.enable_uma_ccbs=3D1' tunables to enable it. Testing > is welcome; I will flip the default to enable in two weeks from now. > =20 > Reviewed By: imp > Sponsored by: NetApp, Inc. > Sponsored by: Klara, Inc. > Differential Revision: https://reviews.freebsd.org/D28674 > --- > sys/cam/ata/ata_da.c | 18 ++++++++++++++++++ > sys/cam/ata/ata_xpt.c | 7 +++++++ > sys/cam/cam_ccb.h | 14 +++++++++++++- > sys/cam/cam_periph.h | 3 +++ > sys/cam/cam_xpt.c | 32 +++++++++++++++++++++++++++++--- > sys/cam/scsi/scsi_da.c | 18 ++++++++++++++++++ > sys/cam/scsi/scsi_xpt.c | 7 +++++++ > 7 files changed, 95 insertions(+), 4 deletions(-) Hey Edward, I suspect this may be the cause of a kernel panic I just had when booting my Softiron Overdrive 1000 arm64 system: panic: ata_action: ccb 0xffff000040388828, func_code 0x15 should not be all= ocated from UMA zone cpuid =3D 1 time =3D 1 __HardenedBSD_version =3D 1400000 __FreeBSD_version =3D 1400013 version =3D FreeBSD 14.0-CURRENT-HBSD #1 hardened/current/master-n190374-a= 541810e5fa: Sat May 15 17:05:42 EDT 2021 shawn@arm64-02:/usr/obj/usr/src/arm64.aarch64/sys/HARDENEDBSD KDB: stack backtrace: db_trace_self() at db_trace_self db_trace_self_wrapper() at db_trace_self_wrapper+0x30 vpanic() at vpanic+0x188 panic() at panic+0x44 ata_action() at ata_action+0x340 _ata_announce_periph() at _ata_announce_periph+0x74 ata_announce_periph_sbuf() at ata_announce_periph_sbuf+0x24 xpt_announce_periph_sbuf() at xpt_announce_periph_sbuf+0x1a8 adaregister() at adaregister+0x420 cam_periph_alloc() at cam_periph_alloc+0x528 adaasync() at adaasync+0xa8 xptsetasyncfunc() at xptsetasyncfunc+0x100 xptdevicetraverse() at xptdevicetraverse+0x9c xpttargettraverse() at xpttargettraverse+0x78 xptbustraverse() at xptbustraverse+0x80 xpt_register_async() at xpt_register_async+0x1fc adainit() at adainit+0x4c periphdriver_init() at periphdriver_init+0x54 xpt_finishconfig_task() at xpt_finishconfig_task+0x18 taskqueue_run_locked() at taskqueue_run_locked+0x188 taskqueue_thread_loop() at taskqueue_thread_loop+0x9c fork_exit() at fork_exit+0x74 fork_trampoline() at fork_trampoline+0x14 KDB: enter: panic [ thread pid 0 tid 100023 ] Stopped at kdb_enter+0x44: undefined f904411f Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A= 4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc --6k5h3synwn52j6at Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAmCgWDMACgkQ/y5nonf4 4fruaQ/+PIkDG/eUlPCAdIrcETOpYDRNCaCK08kg1k6F4OE4H6PUeltr00DQhjH8 NdmIGPlne3Bs2y0qsZLGIMcD5s+NdFyj6bm2d6IX0UP3Si8zSiI8L7bum7jQlv2K Bbk7pAFKBHwYlNm5CotlwoIlDAWztr0N/YNIi4SWX+m2MStoVXZTOcO+3fRB/yFE buWVb1hZlPECmpVnhP/XQtMParxbUd1i9I00oxdYHSVux8CQq73D/Rkm4PT5pj4S iOWouMUGed6fDLS7o1M/wB9t9KOGpvnRqVaolvRfZglTI8ek5d6w9Po0fI4aSUNy LcWZKS0LsQH7zTlevpNvQNDBq1a7xm5UbhMOs3wNiBCgjsJ0anwV9j3/951x/jTS EjqDVfMsa1Vb7s+hXb4Gf9f1RF4HoUeB7P1nOjn+LjZv3kCT/b6mKEMYjrCjeOJz JChaBU2BBY8Q8hsIk4Ndx2Vn3XMLxDjsCsBPQ7f88LX+PZfUiIWtP5aHUuPbQw7l 9zDAkRV7Bb1SeIJeDPvfNbQSHq2waRvd+cZM278X7fM5GmvZG4SfUnEpGpiSA+aD /Eq74lJN35/SdVWH/oRvgpj1zKCwP62Pc112S2Tv6qog2BApWRlPdfMEk+FzMImm B92knlTQDqe1zf2/P5koVWLzLBXteQ9gqpmV30mP8Ki2FLNeoF8= =02Rf -----END PGP SIGNATURE----- --6k5h3synwn52j6at--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20210515232438.7siob6di7bdvaefs>