From owner-freebsd-current Fri Dec 14 2:11:13 2001 Delivered-To: freebsd-current@freebsd.org Received: from iguana.aciri.org (iguana.aciri.org [192.150.187.36]) by hub.freebsd.org (Postfix) with ESMTP id 35C1137B405 for ; Fri, 14 Dec 2001 02:11:10 -0800 (PST) Received: (from rizzo@localhost) by iguana.aciri.org (8.11.3/8.11.1) id fBEAB9047674; Fri, 14 Dec 2001 02:11:09 -0800 (PST) (envelope-from rizzo) Date: Fri, 14 Dec 2001 02:11:09 -0800 From: Luigi Rizzo To: current@freebsd.org Subject: Solved (Re: -current vs. -stable network performance) Message-ID: <20011214021109.B46985@iguana.aciri.org> References: <20011212224206.D35108@iguana.aciri.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20011212224206.D35108@iguana.aciri.org> User-Agent: Mutt/1.3.23i Sender: owner-freebsd-current@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In case you are interested, I found why CURRENT performed so badly. It turns out that CURRENT still does not have the fix to M_LEADINGSPACE that permits writing into non-shared mbufs. This caused the header of forwarded packets to be pulled up in a separate buffer, and triggered a known (to me at least!) performance bug in the 21143 which wastes about 5us when a packet is split across two descriptor. With the one-line change shown below, forwarding speed goes up to the same values as STABLE. The change below has been committed to STABLE 7 weeks ago, but did not go into CURRENT because there was some disagreement on the semantics of M_LEADINGSPACE. However I would strongly vote for committing this change to CURRENT as well, given the huge performance implications (even if the 21143 were not buggy, not being able to write into clusters hurts a lot of pieces of the networking stack). cheers luigi Index: mbuf.h =================================================================== RCS file: /home/xorpc/u2/freebsd/src/sys/sys/mbuf.h,v retrieving revision 1.85 diff -u -r1.85 mbuf.h --- mbuf.h 2001/09/30 01:58:35 1.85 +++ mbuf.h 2001/12/14 09:46:28 @@ -360,7 +360,7 @@ */ #define M_LEADINGSPACE(m) \ ((m)->m_flags & M_EXT ? \ - /* (m)->m_data - (m)->m_ext.ext_buf */ 0 : \ + (M_WRITABLE(m) ? (m)->m_data - (m)->m_ext.ext_buf : 0): \ (m)->m_flags & M_PKTHDR ? (m)->m_data - (m)->m_pktdat : \ (m)->m_data - (m)->m_dat) On Wed, Dec 12, 2001 at 10:42:06PM -0800, Luigi Rizzo wrote: > Hi, > I am testing the forwarding performance of CURRENT vs. STABLE > (both more or less up to date, unmodified, with the latest performance > patches to the "dc" driver, which I am using) and I am having some > surprises. > > STABLE can forward approx 125Kpps, whereas CURRENT tops at approx 80Kpps. > > This is on the same hardware, 750MHz Athlon, fastforwarding enabled, > a 4-port 21143 card, one input driven with a stream of up to 148Kpps > (64 bytes each). > > Ability to transmit seems roughly the same (in both cases 138Kpps), > and lack of CPU does not seem to be the problem (at least for > CURRENT), so I am suspecting some difference in the initialization > of PCI parameters, such as burst size etc, but I am unclear on > where to look at. Any ideas ? > > cheers > luigi > ----------------------------------+----------------------------------------- > Luigi RIZZO, luigi@iet.unipi.it . ACIRI/ICSI (on leave from Univ. di Pisa) > http://www.iet.unipi.it/~luigi/ . 1947 Center St, Berkeley CA 94704 > Phone: (510) 666 2927 > ----------------------------------+----------------------------------------- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message