From owner-dev-commits-src-main@freebsd.org Tue May 25 23:59:35 2021 Return-Path: Delivered-To: dev-commits-src-main@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1B06C656F25; Tue, 25 May 2021 23:59:35 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FqWK305tGz4qfm; Tue, 25 May 2021 23:59:35 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DBEA3263D1; Tue, 25 May 2021 23:59:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 14PNxYCa094862; Tue, 25 May 2021 23:59:34 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 14PNxYlQ094860; Tue, 25 May 2021 23:59:34 GMT (envelope-from git) Date: Tue, 25 May 2021 23:59:34 GMT Message-Id: <202105252359.14PNxYlQ094860@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: John Baldwin Subject: git: 3c7a01d773ac - main - Extend m_apply() to support unmapped mbufs. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 3c7a01d773ac2d128eabb596eed7098f76966cc5 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-main@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for the main branch of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 May 2021 23:59:35 -0000 The branch main has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=3c7a01d773ac2d128eabb596eed7098f76966cc5 commit 3c7a01d773ac2d128eabb596eed7098f76966cc5 Author: John Baldwin AuthorDate: 2021-05-25 23:59:18 +0000 Commit: John Baldwin CommitDate: 2021-05-25 23:59:18 +0000 Extend m_apply() to support unmapped mbufs. m_apply() invokes the callback function separately on each segment of an unmapped mbuf: the TLS header, individual pages, and the TLS trailer. Reviewed by: markj Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D30132 --- sys/kern/uipc_mbuf.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 57 insertions(+), 1 deletion(-) diff --git a/sys/kern/uipc_mbuf.c b/sys/kern/uipc_mbuf.c index f7852bc7dd7f..e5de2286d8f3 100644 --- a/sys/kern/uipc_mbuf.c +++ b/sys/kern/uipc_mbuf.c @@ -1239,6 +1239,62 @@ m_append(struct mbuf *m0, int len, c_caddr_t cp) return (remainder == 0); } +static int +m_apply_extpg_one(struct mbuf *m, int off, int len, + int (*f)(void *, void *, u_int), void *arg) +{ + void *p; + u_int i, count, pgoff, pglen; + int rval; + + KASSERT(PMAP_HAS_DMAP, + ("m_apply_extpg_one does not support unmapped mbufs")); + off += mtod(m, vm_offset_t); + if (off < m->m_epg_hdrlen) { + count = min(m->m_epg_hdrlen - off, len); + rval = f(arg, m->m_epg_hdr + off, count); + if (rval) + return (rval); + len -= count; + off = 0; + } else + off -= m->m_epg_hdrlen; + pgoff = m->m_epg_1st_off; + for (i = 0; i < m->m_epg_npgs && len > 0; i++) { + pglen = m_epg_pagelen(m, i, pgoff); + if (off < pglen) { + count = min(pglen - off, len); + p = (void *)PHYS_TO_DMAP(m->m_epg_pa[i] + pgoff); + rval = f(arg, p, count); + if (rval) + return (rval); + len -= count; + off = 0; + } else + off -= pglen; + pgoff = 0; + } + if (len > 0) { + KASSERT(off < m->m_epg_trllen, + ("m_apply_extpg_one: offset beyond trailer")); + KASSERT(len <= m->m_epg_trllen - off, + ("m_apply_extpg_one: length beyond trailer")); + return (f(arg, m->m_epg_trail + off, len)); + } + return (0); +} + +/* Apply function f to the data in a single mbuf. */ +static int +m_apply_one(struct mbuf *m, int off, int len, + int (*f)(void *, void *, u_int), void *arg) +{ + if ((m->m_flags & M_EXTPG) != 0) + return (m_apply_extpg_one(m, off, len, f, arg)); + else + return (f(arg, mtod(m, caddr_t) + off, len)); +} + /* * Apply function f to the data in an mbuf chain starting "off" bytes from * the beginning, continuing for "len" bytes. @@ -1262,7 +1318,7 @@ m_apply(struct mbuf *m, int off, int len, while (len > 0) { KASSERT(m != NULL, ("m_apply, offset > size of mbuf chain")); count = min(m->m_len - off, len); - rval = (*f)(arg, mtod(m, caddr_t) + off, count); + rval = m_apply_one(m, off, count, f, arg); if (rval) return (rval); len -= count;