From owner-dev-commits-src-all@freebsd.org Sat May 15 16:03:04 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0251B649BF1; Sat, 15 May 2021 16:03:04 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com [IPv6:2a00:1450:4864:20::52b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fj9Cq6Pj2z50lC; Sat, 15 May 2021 16:03:03 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: by mail-ed1-x52b.google.com with SMTP id di13so1935170edb.2; Sat, 15 May 2021 09:03:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=0pYabJsDvhMWX7/rOW/ty1e+SB90zG7wjhk/p30gv58=; b=JncGC+LTZf8Q6vtIXuMyAADRPjyFs6KNuNpxQMqNXdMRYHvNrQ8kDAJlNUQaYS1CfV GhTRsLx/BYpjUMLTOEHKSsVPROShGSOol2ixQF26D08N3OBu/Tt2ZyHbaYdA3ZvATjmi I1xj17KcsTg0AoE7v5IdMQ2pXm+nlbQL4O82nd8kIFxYBM/lw9l38jpxsWjeqX7JynAR TTZjIFdeaROZdn8pTl7U4n6F4WJzT2irP32xpmZ93dCW+ime13uUulHVjleoJBneCcrL HfMo2I1AoNCyPNOa74k0gVKEJlELtzWjmk8mqNrby33nSVte90Y/K3I6gYqe+CB+IdGV A3pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=0pYabJsDvhMWX7/rOW/ty1e+SB90zG7wjhk/p30gv58=; b=gvPhWAZPqgW8B8eLOsF9sPOshgorgIjtMYGSVhqBbVdeVduGCa71f/kI6gja6FlhQO RoQkdsNYnhAXKRtPY2bm3qGupLEK/NE2juYUZ26uYw4nBp/DgP9rPYcSuqhLv5XPw8jH cN7+AxE2OXQ1eg1NneNO5aIj2MU1wIBWCoDN1z1g3bh1MLKilHe54l06f/UMRkP3L/vo NNNMR5DhcJ8WvwbI8ikvDYhM2cnIQFPsGIPvUc6dXP5sMf7I7kf8BYjsFPFTCJLb1TNV Jvjn8ZfsQ/4bO1HR49c1gMd+1/ZT/CimTSAvaWQVC4B81FIeYw+X/uw3wrfPL642PnkV eA1Q== X-Gm-Message-State: AOAM533OAVFQeLmaTprog2k3xkECpZWp72dxeKHtj5aCv6s7RqIfPIzi M5F7EOgrDXDBWxx7QrMoYndyt+H+djo= X-Google-Smtp-Source: ABdhPJyPO/S76U+6lOZEhLkQhCbf5Lni3h+hXbbVxwmh8D6vQgb7zMHw1PLPnn13osYGDDgtNwTGeQ== X-Received: by 2002:aa7:d455:: with SMTP id q21mr12628496edr.2.1621094581860; Sat, 15 May 2021 09:03:01 -0700 (PDT) Received: from ernst.home (pd9e23d76.dip0.t-ipconnect.de. [217.226.61.118]) by smtp.gmail.com with ESMTPSA id bw26sm5526253ejb.119.2021.05.15.09.03.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 15 May 2021 09:03:00 -0700 (PDT) Date: Sat, 15 May 2021 18:03:00 +0200 From: Gary Jennejohn To: Edward Tomasz Napierala Cc: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Subject: Re: git: 3394d4239b85 - main - cam: allocate CCBs from UMA for SCSI and ATA IO Message-ID: <20210515160300.6d6d3916@ernst.home> In-Reply-To: <202105151105.14FB5nv7005915@gitrepo.freebsd.org> References: <202105151105.14FB5nv7005915@gitrepo.freebsd.org> Reply-To: gljennjohn@gmail.com X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4Fj9Cq6Pj2z50lC X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 May 2021 16:03:04 -0000 On Sat, 15 May 2021 11:05:49 GMT Edward Tomasz Napierala wrote: > The branch main has been updated by trasz: > > URL: https://cgit.FreeBSD.org/src/commit/?id=3394d4239b85b5577845d9e6de4e97b18d3dba58 > > commit 3394d4239b85b5577845d9e6de4e97b18d3dba58 > Author: Edward Tomasz Napierala > AuthorDate: 2021-05-15 10:17:22 +0000 > Commit: Edward Tomasz Napierala > CommitDate: 2021-05-15 11:03:49 +0000 > > cam: allocate CCBs from UMA for SCSI and ATA IO > > This patch makes it possible for CAM to use small CCBs allocated > from an periph-specific UMA zone instead of the usual, huge ones. > The end result is that CCBs issued via da(4) take 544B (size of > ccb_scsiio) instead of the usual 2kB (size of 'union ccb', ~1.5kB, > rounded up by malloc(9)). For ATA it's 272B. We waste less > memory, we avoid zeroing the unused 1kB, and it should be easier > to allocate those CCBs in low memory conditions. It should also > be possible to use uma_zone_reserve(9) to improve behaviour > in low memory conditions even further. > > Note that this does not change the size, or the layout, of CCBs > as such. CCBs get allocated in various different ways, in particular > on the stack, and I don't want to redo all that. Instead, this > provides an opt-in mechanism for the periph to declare "my start() > callback is fine with receiving a CCB allocated from this UMA zone". > In other words, most of the code works exactly as it used to; the > change only happens to IOs issued by xpt_run_allockq(), which > is - conveniently - pretty much all that matters for performance. > > The reason for doing it this way is that it's pretty small, localized > change, and can be implemented gradually and iteratively: take a > periph, make sure its start() callback only casts the CCBs it takes > to a particular type of CCB, for example ccb_scsiio, and that it only > casts CCBs returned by cam_periph_getccb() to that type, then add UMA > zone for that size, and declare it safe to XPT. > > This is disabled by default. Set 'kern.cam.ada.enable_uma_ccbs=1' > and 'kern.cam.da.enable_uma_ccbs=1' tunables to enable it. Testing > is welcome; I will flip the default to enable in two weeks from now. > [snip diff] I built and installed world and kernel and enabled both tunables. System's been up for about two hours using both ada and da disks pretty intensely and hasn't trashed any yet. -- Gary Jennejohn