From owner-p4-projects@FreeBSD.ORG Sun Apr 1 21:31:49 2007 Return-Path: X-Original-To: p4-projects@freebsd.org Delivered-To: p4-projects@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 32767) id F011316A407; Sun, 1 Apr 2007 21:31:48 +0000 (UTC) X-Original-To: perforce@freebsd.org Delivered-To: perforce@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9DE9516A406 for ; Sun, 1 Apr 2007 21:31:48 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: from wx-out-0506.google.com (wx-out-0506.google.com [66.249.82.228]) by mx1.freebsd.org (Postfix) with ESMTP id 4A9BC13C455 for ; Sun, 1 Apr 2007 21:31:48 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: by wx-out-0506.google.com with SMTP id s18so993667wxc for ; Sun, 01 Apr 2007 14:31:47 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=DOaI/wfjxqjtRBFr0ZWyx0UbSUskRGhLD08Q+20jUT3EEzFg2uR5k2omC8SEK6xRNbOD1WitrpBocCRJX3eObgNhuQemidKMUvOufS4CI8wdwfMnZRwa9UC9WrGIvHwBgOQK9VbO3Y28LSrmoniAXQwnqXAd5eI/FFISVtW3cDw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=osafUiRsmPkuYieVRWCSzwEsmB0DTvysZCRlwUofmfFToAP3RVApFKJIVQYzlFJOPlGgscRwj6UwRt0n48m//VO/+w9OiAI9j0ujSWVhR3gfKP6A922n4+nGh7ye3+VpztfGt6tJcDsHjF5UrdhUTcqMlfhwJIqbhcoldXXKMqM= Received: by 10.90.106.11 with SMTP id e11mr2762800agc.1175461349416; Sun, 01 Apr 2007 14:02:29 -0700 (PDT) Received: by 10.90.116.6 with HTTP; Sun, 1 Apr 2007 14:02:29 -0700 (PDT) Message-ID: Date: Sun, 1 Apr 2007 14:02:29 -0700 From: "Kip Macy" To: "Andre Oppermann" In-Reply-To: <46101C26.5030306@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <200704012020.l31KKr0O097740@repoman.freebsd.org> <46101C26.5030306@freebsd.org> Cc: Perforce Change Reviews , Kip Macy Subject: Re: PERFORCE change 117123 for review X-BeenThere: p4-projects@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: p4 projects tree changes List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Apr 2007 21:31:49 -0000 > Thanks for using it. This was the idea behind providing this interface. > From a cache busting point of view attaching the mbuf after the cluster > has been filled is very good. The Sandvine guys found that out a long > time ago and it indeed makes a lot of sense. When allocating whole > clusters the mbuf gets touched twice, once at allocation and once when > the driver fills in the information from the RX ring. This way it only > gets touched in the latter case and the former cache pollution is skipped > over. Yup. This is actually only the initial part of what I'm working on. I was expecting < 3% improvement. However, the before and after numbers look more like: before: chaos# netperf -H 10.0.0.150 -tTCP_SENDFILE -F /var/tmp/bigfile -Cc -P0 -l 5 65536 32768 32768 5.00 7682.06 25.36 36.07 1.082 1.539 65536 32768 32768 5.00 7713.27 24.55 36.97 1.043 1.571 65536 32768 32768 5.00 7755.67 26.25 40.62 1.109 1.716 65536 32768 32768 5.00 7593.98 21.03 34.79 0.908 1.501 after: chaos# netperf -H 10.0.0.150 -tTCP_SENDFILE -F /var/tmp/bigfile -Cc -P0 -l 5 65536 32768 32768 5.00 8109.80 33.65 33.22 1.360 1.342 65536 32768 32768 5.00 8649.49 32.89 45.29 1.246 1.716 65536 32768 32768 5.00 8211.80 26.35 34.70 1.051 1.385 65536 32768 32768 5.00 8538.48 29.55 44.05 1.134 1.691 A couple of weeks ago I was getting 8.8 - 9.6 Gbps > > > This change alleviates a good portion of the recent (last 2 weeks) 18% performance drop > > in peak TCP throughput > > Can you attribute any specific change to the drop in performance? Unfortunately, I haven't tracked HEAD the last 2 weeks or so. -Kip