From owner-freebsd-net@FreeBSD.ORG Mon Mar 5 20:44:00 2007 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 588FC16A400; Mon, 5 Mar 2007 20:44:00 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.freebsd.org (Postfix) with ESMTP id 1861113C474; Mon, 5 Mar 2007 20:44:00 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.14.0/8.14.0) with ESMTP id l25Khtf4019410 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 5 Mar 2007 15:43:55 -0500 (EST) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.12.9p2/8.12.9/Submit) id l25KhonE056215; Mon, 5 Mar 2007 15:43:50 -0500 (EST) (envelope-from gallatin) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <17900.33030.751272.890853@grasshopper.cs.duke.edu> Date: Mon, 5 Mar 2007 15:43:50 -0500 (EST) To: John Baldwin In-Reply-To: <200703051352.29939.jhb@freebsd.org> References: <45E8276D.60105@freebsd.org> <17900.24574.751134.397740@grasshopper.cs.duke.edu> <20070305182755.S31701@fledge.watson.org> <200703051352.29939.jhb@freebsd.org> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Cc: Andre Oppermann , Andrew Gallatin , freebsd-net@freebsd.org, freebsd-current@freebsd.org, Robert Watson , kmacy@freebsd.org Subject: Re: New optimized soreceive_stream() for TCP sockets, proof of concept X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Mar 2007 20:44:00 -0000 John Baldwin writes: > > John has a patch that pins interrupt threads, etc, not sure what the status > of > > that is. CC'd. > > Tested and around for over a year. Sent to people several times but no > benchmarking has resulted. It lives in p4 in //depot/user/jhb/intr/... > > I've just regenerated the patch at > http://www.FreeBSD.org/~jhb/patches/intr_bind.patch (same URL as the last N It seems very useful, in conjunction with some sort of CPU binding API. With ULE, and netserver bound via a hack to CPU 1, I can nearly double the bandwidth by binding my ithread to the opposite CPU using your patch. BTW, you need a little sanity checking somehere in the API I tried to remove binding, and guessed at binding to -1 rather than reading the source. That resulted in this panic: # ./ibind/ibind 256 -1 kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 01 fault virtual address = 0x3806778bc fault code = supervisor read data, page not present instruction pointer = 0x8:0xffffffff8040a509 stack pointer = 0x10:0xffffffff91e88ad0 frame pointer = 0x10:0xffffffff91e88b00 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 788 (ibind) [thread pid 788 tid 100047 ] Stopped at intr_bind+0xe9: movl cpu_apic_ids(,%rdi,4),%esi db> bt Tracing pid 788 tid 100047 td 0xffffff001ea1b840 intr_bind() at intr_bind+0xe9 sysarch() at sysarch+0x14a ia32_syscall() at ia32_syscall+0x236 Xint0x80_syscall() at Xint0x80_syscall+0x60 Thanks again! Drew