From owner-freebsd-net@FreeBSD.ORG Fri Oct 6 17:05:43 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 80DE116A58D for ; Fri, 6 Oct 2006 17:05:43 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id E331543D5C for ; Fri, 6 Oct 2006 17:05:41 +0000 (GMT) (envelope-from andre@freebsd.org) Received: (qmail 67792 invoked from network); 6 Oct 2006 17:05:36 -0000 Received: from dotat.atdotat.at (HELO [62.48.0.47]) ([62.48.0.47]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 6 Oct 2006 17:05:36 -0000 Message-ID: <45268CE4.2090601@freebsd.org> Date: Fri, 06 Oct 2006 19:05:40 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8b) Gecko/20050217 MIME-Version: 1.0 To: freebsd-current@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, gallatin@cs.duke.edu, phk@freebsd.org Subject: Much improved sendfile(2), sosend_* and soreceive_stream() functions X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Oct 2006 17:05:43 -0000 This is a continuation of the previous postings with similiar titles. New is a specific soreceive_stream() function for stream protocols (primarily TCP) that does only one socket buffer lock per socket read instead of one per data mbuf copied to userland. When doing netperf tests with WITNESS (full lock tracking and validation enabled) the receive performance increases from ~360Mbit/s to ~520Mbit/s. Without WITNESS I could not measure any statistically significant improvement on a otherwise unloaded machine. The reason is two-fold: 1) per packet we do a wakeup and readv() is pretty much as many times as packets come it, thus the general over- head dominates; 2) the packet input path has a pretty high overhead too. On heavily loaded machines which do a lot of high speed receives a performance increase should be measureable. This patch rolls the improved sendfile(2), sosend_*() and soreceive_stream() functions. In general the send path is improved by between 2.8 and 5.7 times. The patch is available here: http://people.freebsd.org/~andre/sendfile+sosend+soreceive-20061006.diff Any testing and heavy (code) beating and reviews welcome. -- Andre