From nobody Thu Apr 21 19:02:37 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id F320D1992F1E; Thu, 21 Apr 2022 19:02:38 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Kkn3f0XN4z3C7l; Thu, 21 Apr 2022 19:02:38 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1650567758; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Z2LYbS+OXWV/FXYzsuU01rUD9SFgEBXYLrtxK+iMD/s=; b=tfwEdu/zv4ffojizTVLQynBDczfUc0KFkEDVemxdGekPlqgT0ysIFwy4UDtNsQ9ZjSXntC kgkC2rLiLphLy1cN5Xy9aokrKQPBN/vuQya8CrvCyF1Q1I0TKRSmTZzEAaMOBWHYjVQIRo eysk338x3hi60BTOwvS4Amm5vPLfpDp48QI681YIut712XNk21VkJvEEu66QryC1YJXYPd smAJbfGVJ8KMazuZeZeqtuVyZgFphv4igBldj59Ge3hzJNpPxTvEvSwAFkH/gz3vFcRHFx v3fu3uIcJxbllNiGgdAkPxGrnk7minrHyh6af29UJrtx6hFSWUWGw3ZLfeG5JQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CDEDE6736; Thu, 21 Apr 2022 19:02:37 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 23LJ2bhm035893; Thu, 21 Apr 2022 19:02:37 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 23LJ2bug035892; Thu, 21 Apr 2022 19:02:37 GMT (envelope-from git) Date: Thu, 21 Apr 2022 19:02:37 GMT Message-Id: <202204211902.23LJ2bug035892@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: John Baldwin Subject: git: 92e40a9b9241 - main - busdma_bounce: Batch bounce page free operations when possible. List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 92e40a9b9241313b3d16543819ccd8d642bb11a5 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1650567758; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Z2LYbS+OXWV/FXYzsuU01rUD9SFgEBXYLrtxK+iMD/s=; b=FaLrIwKslvfzBqWGIDwFgCpnVsG8YsFq/2J1coxSTeVocwbLpunSpkiy5Xf5w9+7RJYO+7 k4Ww4V3lSJkSN5AaJSo0gkJ0dLB7unUl2Sr7wfkqdFYNXq/xW7ZhPWBApd/BuBe/cctbs8 QE4sxraLVHcBqJm3VFuTPgHQ7uwakvgLqYmvRjIv4qdU5x7R2D1gvSDQ7tHjw1ECeua7Vk j2m9clThXbiycaA8lTbzTbmFxn1w2q7u6FTIhMDT7DoxnK4wfYwWxwH9AbVvNfE1MHQnIn fuAWXRg6wSa4GUc1YrekaG0yCa37CHIesdCLuCQPoRGs5EPoCCyWny+ZF92Esg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1650567758; a=rsa-sha256; cv=none; b=O9F4AlISv+riX6VDJldUtQhN6Bwoq4X1MIs9axRtUZLN6NWlKY/ifTZigCnaHOjtOLwAmv FmA8gaWS9W5p5IBh8n28QfXQvi8TY3ECrVKMLQh5Hs/56j/3xprH+ioK5ralNS0FyE52Yz gTEKeczkXbI72JAt+l8Rth3Lh4/CuMqpN+oXANl4HazymjkeB3+e/8o74Xphsy+8znFFVU ZGlbXYCK6cVSREFyJYaQRhJPr1atcTa8FtWgAEsTkZcb+W5pRkTm7IWseIk08z5axi8N9t V4z/58bPa53dfx8gnt2ugtra1MeUUNbrqq2hl1zkfc/nhTZwW9gBYABracg6lQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=92e40a9b9241313b3d16543819ccd8d642bb11a5 commit 92e40a9b9241313b3d16543819ccd8d642bb11a5 Author: John Baldwin AuthorDate: 2022-04-21 19:01:55 +0000 Commit: John Baldwin CommitDate: 2022-04-21 19:01:55 +0000 busdma_bounce: Batch bounce page free operations when possible. Reviewed by: imp Differential Revision: https://reviews.freebsd.org/D34968 --- sys/kern/subr_busdma_bounce.c | 69 +++++++++++++++++++++---------------------- 1 file changed, 34 insertions(+), 35 deletions(-) diff --git a/sys/kern/subr_busdma_bounce.c b/sys/kern/subr_busdma_bounce.c index 243da8e9487f..f3699cf2ad27 100644 --- a/sys/kern/subr_busdma_bounce.c +++ b/sys/kern/subr_busdma_bounce.c @@ -382,55 +382,54 @@ add_bounce_page(bus_dma_tag_t dmat, bus_dmamap_t map, vm_offset_t vaddr, } static void -free_bounce_page(bus_dma_tag_t dmat, struct bounce_page *bpage) +free_bounce_pages(bus_dma_tag_t dmat, bus_dmamap_t map) { - struct bus_dmamap *map; + struct bounce_page *bpage; struct bounce_zone *bz; bool schedule_thread; + u_int count; + + if (STAILQ_EMPTY(&map->bpages)) + return; bz = dmat->bounce_zone; - bpage->datavaddr = 0; - bpage->datacount = 0; - if (dmat_flags(dmat) & BUS_DMA_KEEP_PG_OFFSET) { - /* - * Reset the bounce page to start at offset 0. Other uses - * of this bounce page may need to store a full page of - * data and/or assume it starts on a page boundary. - */ - bpage->vaddr &= ~PAGE_MASK; - bpage->busaddr &= ~PAGE_MASK; + count = 0; + schedule_thread = false; + STAILQ_FOREACH(bpage, &map->bpages, links) { + bpage->datavaddr = 0; + bpage->datacount = 0; + + if (dmat_flags(dmat) & BUS_DMA_KEEP_PG_OFFSET) { + /* + * Reset the bounce page to start at offset 0. + * Other uses of this bounce page may need to + * store a full page of data and/or assume it + * starts on a page boundary. + */ + bpage->vaddr &= ~PAGE_MASK; + bpage->busaddr &= ~PAGE_MASK; + } + count++; } - schedule_thread = false; mtx_lock(&bounce_lock); - STAILQ_INSERT_HEAD(&bz->bounce_page_list, bpage, links); - bz->free_bpages++; - bz->active_bpages--; - if ((map = STAILQ_FIRST(&bz->bounce_map_waitinglist)) != NULL) { - if (reserve_bounce_pages(map->dmat, map, 1) == 0) { - STAILQ_REMOVE_HEAD(&bz->bounce_map_waitinglist, links); - STAILQ_INSERT_TAIL(&bounce_map_callbacklist, - map, links); - bz->total_deferred++; - schedule_thread = true; - } + STAILQ_CONCAT(&bz->bounce_page_list, &map->bpages); + bz->free_bpages += count; + bz->active_bpages -= count; + while ((map = STAILQ_FIRST(&bz->bounce_map_waitinglist)) != NULL) { + if (reserve_bounce_pages(map->dmat, map, 1) != 0) + break; + + STAILQ_REMOVE_HEAD(&bz->bounce_map_waitinglist, links); + STAILQ_INSERT_TAIL(&bounce_map_callbacklist, map, links); + bz->total_deferred++; + schedule_thread = true; } mtx_unlock(&bounce_lock); if (schedule_thread) wakeup(&bounce_map_callbacklist); } -static void -free_bounce_pages(bus_dma_tag_t dmat, bus_dmamap_t map) -{ - struct bounce_page *bpage; - - while ((bpage = STAILQ_FIRST(&map->bpages)) != NULL) { - STAILQ_REMOVE_HEAD(&map->bpages, links); - free_bounce_page(dmat, bpage); - } -} - static void busdma_thread(void *dummy __unused) {