From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 27 00:26:31 2009 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 550BD106566B for ; Tue, 27 Oct 2009 00:26:31 +0000 (UTC) (envelope-from gonzo@bluezbox.com) Received: from core.tav.kiev.ua (tavex.colocall.com [62.149.10.42]) by mx1.freebsd.org (Postfix) with ESMTP id EF7A48FC14 for ; Tue, 27 Oct 2009 00:26:30 +0000 (UTC) Received: from [76.77.86.2] (helo=[10.80.5.136]) by core.tav.kiev.ua with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.52 (FreeBSD)) id 1N2Zdp-0005oa-1f for hackers@freebsd.org; Tue, 27 Oct 2009 02:10:25 +0200 Message-ID: <4AE63A64.7010205@bluezbox.com> Date: Mon, 26 Oct 2009 17:10:12 -0700 From: Oleksandr Tymoshenko User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Core-Spam-Level: / X-Core-Spam-Report: Spam detection software, running on the system "core.tav.kiev.ua", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see The administrator of that system for details. Content preview: This problem haunts for a couple of days and I can't find a nice and clean solution so this email is actually a cry for help. The problem: There is a buffer loaded by bus_dmamap_load for use as a DMA buffer. Right before this buffer resides block of vital data structure. Consider following scenario: [...] Content analysis details: (-0.5 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.8 ALL_TRUSTED Passed through trusted hosts only via SMTP 0.0 BAYES_50 BODY: Bayesian spam probability is 40 to 60% [score: 0.5174] 1.3 AWL AWL: From: address is in the auto white-list Cc: Subject: MIPS: bus_dma(9) and cache problems X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Oct 2009 00:26:31 -0000 This problem haunts for a couple of days and I can't find a nice and clean solution so this email is actually a cry for help. The problem: There is a buffer loaded by bus_dmamap_load for use as a DMA buffer. Right before this buffer resides block of vital data structure. Consider following scenario: 1. code modifies data in block and this modification ends up in cache and is not written back to memory 2. right after this code calls bus_dmamap_sync for this buffer and as a result cache invalidation is performed 3. Cache function operates on cache line size-aligned addresses and the block in question happens to share the same cache line with the buffer. So modification made at step (1) is lost. If busdma code controls allocation (bus_dmamem_alloc) this situation can be avoided by forcing pointer alignment. But when address come from the "outer space" via bus_dmamap_load* there is not much to do. There are two solutions I've figured so far: - Create bounce page for not properly aligned memory. Which would reduce performance a lot. - Remap buffer's page(s) as uncached. I haven't succeeded with this one yet and not sure it's always possible. May be someone can suggest better way to clean this mess? Thanks! -- gonzo