From owner-freebsd-net@FreeBSD.ORG Tue Apr 7 06:35:21 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EF868106570C for ; Tue, 7 Apr 2009 06:35:21 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outY.internet-mail-service.net (outy.internet-mail-service.net [216.240.47.248]) by mx1.freebsd.org (Postfix) with ESMTP id CC40F8FC13 for ; Tue, 7 Apr 2009 06:35:21 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id AAB9BB98A2; Mon, 6 Apr 2009 23:35:22 -0700 (PDT) X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (home.elischer.org [216.240.48.38]) by idiom.com (Postfix) with ESMTP id 8F5482D6097; Mon, 6 Apr 2009 23:35:17 -0700 (PDT) Message-ID: <49DAF447.5020407@elischer.org> Date: Mon, 06 Apr 2009 23:35:51 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Sepherosa Ziehau References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Robert Watson , Ivan Voras Subject: Re: Advice on a multithreaded netisr patch? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Apr 2009 06:35:22 -0000 Sepherosa Ziehau wrote: > On Mon, Apr 6, 2009 at 7:59 PM, Robert Watson wrote: >> m_pullup() has to do with mbuf chain memory contiguity during packet >> processing. The usual usage is something along the following lines: >> >> struct whatever *w; >> >> m = m_pullup(m, sizeof(*w)); >> if (m == NULL) >> return; >> w = mtod(m, struct whatever *); while this is true, m_pullup ALWAYS does things so in fact you want to always put it in a test to see if it is really needed.. from memory it is something like: if (m->m_len < headerlen && (m = m_pullup(m, headerlen)) == NULL) { log(LOG_WARNING, "nglmi: m_pullup failed for %d bytes\n", headerlen); return (0); } header = mtod(m, struct header *); >> >> m_pullup() here ensures that the first sizeof(*w) bytes of mbuf data are >> contiguously stored so that the cast of w to m's data will point at a >> complete structure we can use to interpret packet data. In the common case >> in the receipt path, m_pullup() should be a no-op, since almost all drivers >> receive data in a single cluster. >> >> However, there are cases where it might not happen, such as loopback traffic >> where unusual encapsulation is used, leading to a call to M_PREPEND() that >> inserts a new mbuf on the front of the chain, which is later m_defrag()'d >> leading to a higher level header crossing a boundary or the like. >> >> This issue is almost entirely independent from things like the cache line >> miss issue, unless you hit the uncommon case of having to do work in >> m_pullup(), in which case life sucks. >> >> It would be useful to use DTrace to profile a number of the workfull m_foo() >> functions to make sure we're not hitting them in normal workloads, btw. > > I highly suspect m_pullup will take any real effect on RX path, given > how most of drivers allocate the mbuf for RX ring (all RX mbufs should > be mclusters). > > Best Regards, > sephe >