From owner-svn-src-all@FreeBSD.ORG Wed Mar 17 15:52:30 2010 Return-Path: Delivered-To: svn-src-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B4B26106564A; Wed, 17 Mar 2010 15:52:30 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.freebsd.org (Postfix) with ESMTP id 74C8D8FC12; Wed, 17 Mar 2010 15:52:30 +0000 (UTC) Received: from [172.31.193.10] (rrcs-98-101-145-84.midsouth.biz.rr.com [98.101.145.84]) (authenticated bits=0) by duke.cs.duke.edu (8.14.2/8.14.2) with ESMTP id o2HFqSss002412 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 17 Mar 2010 11:52:29 -0400 (EDT) X-DKIM: Sendmail DKIM Filter v2.8.3 duke.cs.duke.edu o2HFqSss002412 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=cs.duke.edu; s=mail; t=1268841149; bh=4bX8m5mZT2Aa+Q/QHRkZVKgui/PRKzlKH1nHf5ZR2UA=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=XO+bGOR1KrnoobMTnVFccVmOIVK9gPBHqbaYaKvicChGTMHlBOtcBp1Ut1exOE5YZ LefStztkdccBcKudUNhTzquMmHHS/3kXR+5gh21XRm5BYMuKc6yS0ML/ZcFdhs/Cwi egVOKkSGRYElaoIPbX53WR1xLesXK3MWuh5Ixva4= Message-ID: <4BA0FAB7.7060101@cs.duke.edu> Date: Wed, 17 Mar 2010 11:52:23 -0400 From: Andrew Gallatin User-Agent: Thunderbird 2.0.0.23 (X11/20090817) MIME-Version: 1.0 To: Bruce Evans References: <201003161745.o2GHjG3G051630@svn.freebsd.org> <4BA0CF37.2010903@cs.duke.edu> <20100318015744.B26867@delplex.bde.org> In-Reply-To: <20100318015744.B26867@delplex.bde.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: svn-src-head@FreeBSD.org, svn-src-all@FreeBSD.org, src-committers@FreeBSD.org, Pyun YongHyeon Subject: Re: svn commit: r205221 - head/sys/dev/bge X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2010 15:52:30 -0000 Bruce Evans wrote: > On Wed, 17 Mar 2010, Andrew Gallatin wrote: > >> Pyun YongHyeon wrote: >> >>> Revert r205090. >>> It's hard to know when the mail box register write will get flushed to >>> the hardware and it may take longer. >>> Pointed out by: scottl >> >> I may be mis-reading the code, but it looks like the mailbox >> register is in memory space, which should be flushed immediately >> unless write-combining is enabled on the region. The bge >> driver does not seem to be setting up write combining. >> Is the concern that something may enable write combining >> behind your back? In that case, a wmb() could act as a >> serializing instruction and flush the WC buffers. > > We want writes to the PCI bus to be efficient. Normally (?) writes > to bge registers appear to be several times faster than reads. I don't > know if this depends on write combining but think it depends on write > buffering which can delay the write to the hardware by about the > difference between the read time and the time to write to the bufer. > Any forcing of serialization or timing would presumably lose the > benefits of the buffer. What buffer? On the host side of the PCI{e} bus or the NIC side?? For i386/amd64, only write-combining (MTRR or PAT) pio mappings are buffered on the host side. Every other PIO behaves as if implicitly surrounded by a serializing instruction before and after for legacy reasons. On non i386/amd64, there may be buffering; it depends on the platform. Or are you talking about buffering on the bge side of the PCI{e} bus? The point I was trying to make is that PIO reads are very (hopelessly?) slow because the CPU stalls while waiting for the device to return some data, and you're at the mercy of the device. In general, when you're given a choice as to what to use to force a write across the PCI{e} bus (in the case of WC being enabled, or working on a non-x86/amd64 arch), the best choice by far is a serializing instruction (wmb()) and not a PIO read. Unfortunately, it looks like a PIO read is intimately entwined with correct operation of this device, so doing it the old way is probably safest/best. Sorry for the noise.. Drew