From owner-freebsd-current@FreeBSD.ORG Fri Nov 16 00:34:13 2007 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A4F3716A475 for ; Fri, 16 Nov 2007 00:34:13 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 13BE513C457 for ; Fri, 16 Nov 2007 00:34:05 +0000 (UTC) (envelope-from andre@freebsd.org) Received: (qmail 67272 invoked from network); 16 Nov 2007 00:08:10 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 16 Nov 2007 00:08:10 -0000 Message-ID: <473CE57E.7010303@freebsd.org> Date: Fri, 16 Nov 2007 01:34:06 +0100 From: Andre Oppermann User-Agent: Thunderbird 1.5.0.13 (Windows/20070809) MIME-Version: 1.0 To: Jack Vogel References: <1B860D81B4F3F44398B9AE84D91C151671D11B@stlex510.dsto.defence.gov.au> <2a41acea0711141712x533bcdbex92df05280311be8e@mail.gmail.com> <473CDE7E.80008@freebsd.org> <2a41acea0711151609i64f59719wcd893d0eb89f4cb0@mail.gmail.com> In-Reply-To: <2a41acea0711151609i64f59719wcd893d0eb89f4cb0@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org, "Wilkinson, Alex" Subject: Re: I/OAT ... Coming Soon ? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Nov 2007 00:34:13 -0000 Jack Vogel wrote: > On Nov 15, 2007 4:04 PM, Andre Oppermann wrote: >> Jack Vogel wrote: >>> On Nov 14, 2007 5:01 PM, Wilkinson, Alex >>> wrote: >>>> Hi all, >>>> >>>> Curious, is I/OAT [http://www.intel.com/go/ioat/] coming to FreeBSD soon >>>> ? >>> LOL, I did a driver for the first version of I/OAT more than a year >>> ago, submitted it and interest was half hearted. >> IMHO the biggest drawback (design failure) is the polling requirement >> for the work queue. Ideally it would be structured like a NIC with >> its DMA engine and rings. > > I know, at the time I did this Chris Leech the Linux developer was having > trouble with an interrupt design, so I figured I'd not bang my head against > the wall unnecessarily and follow what he was doing :) > > Since then however, work has been done and particularly with the CB2 > support I believe Linux now does use interrupts, so if I go back and rework > the driver I will try and follow that path. Actually it shouldn't be too hard. I can provide you with the entry points and a taskqueue entry. The I/OAT support can be done just like a driver serving the taskqueue. The entry points for m_uiotombuf et al would consider offloading if a certain size threshold is met and the copy request is tagged with M_WAITOK. The virtual to physical memory lookup is probably costly though. I don't think it supports the TLB as that is only available within the CPU. How is cache coherency solved? It'd be helpful if the documentation for I/OAT were available. The Core2 and AMD64 CPUs have very high memory bandwidth with low latency. Is I/OAT still a significant enough win compared to the implementation effort required? -- Andre