Date: Sat, 15 May 2021 18:03:00 +0200 From: Gary Jennejohn <gljennjohn@gmail.com> To: Edward Tomasz Napierala <trasz@FreeBSD.org> Cc: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Subject: Re: git: 3394d4239b85 - main - cam: allocate CCBs from UMA for SCSI and ATA IO Message-ID: <20210515160300.6d6d3916@ernst.home> In-Reply-To: <202105151105.14FB5nv7005915@gitrepo.freebsd.org> References: <202105151105.14FB5nv7005915@gitrepo.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, 15 May 2021 11:05:49 GMT Edward Tomasz Napierala <trasz@FreeBSD.org> wrote: > The branch main has been updated by trasz: > > URL: https://cgit.FreeBSD.org/src/commit/?id=3394d4239b85b5577845d9e6de4e97b18d3dba58 > > commit 3394d4239b85b5577845d9e6de4e97b18d3dba58 > Author: Edward Tomasz Napierala <trasz@FreeBSD.org> > AuthorDate: 2021-05-15 10:17:22 +0000 > Commit: Edward Tomasz Napierala <trasz@FreeBSD.org> > CommitDate: 2021-05-15 11:03:49 +0000 > > cam: allocate CCBs from UMA for SCSI and ATA IO > > This patch makes it possible for CAM to use small CCBs allocated > from an periph-specific UMA zone instead of the usual, huge ones. > The end result is that CCBs issued via da(4) take 544B (size of > ccb_scsiio) instead of the usual 2kB (size of 'union ccb', ~1.5kB, > rounded up by malloc(9)). For ATA it's 272B. We waste less > memory, we avoid zeroing the unused 1kB, and it should be easier > to allocate those CCBs in low memory conditions. It should also > be possible to use uma_zone_reserve(9) to improve behaviour > in low memory conditions even further. > > Note that this does not change the size, or the layout, of CCBs > as such. CCBs get allocated in various different ways, in particular > on the stack, and I don't want to redo all that. Instead, this > provides an opt-in mechanism for the periph to declare "my start() > callback is fine with receiving a CCB allocated from this UMA zone". > In other words, most of the code works exactly as it used to; the > change only happens to IOs issued by xpt_run_allockq(), which > is - conveniently - pretty much all that matters for performance. > > The reason for doing it this way is that it's pretty small, localized > change, and can be implemented gradually and iteratively: take a > periph, make sure its start() callback only casts the CCBs it takes > to a particular type of CCB, for example ccb_scsiio, and that it only > casts CCBs returned by cam_periph_getccb() to that type, then add UMA > zone for that size, and declare it safe to XPT. > > This is disabled by default. Set 'kern.cam.ada.enable_uma_ccbs=1' > and 'kern.cam.da.enable_uma_ccbs=1' tunables to enable it. Testing > is welcome; I will flip the default to enable in two weeks from now. > [snip diff] I built and installed world and kernel and enabled both tunables. System's been up for about two hours using both ada and da disks pretty intensely and hasn't trashed any yet. -- Gary Jennejohn
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20210515160300.6d6d3916>