From owner-freebsd-mips@FreeBSD.ORG Sun Aug 26 23:18:57 2012 Return-Path: Delivered-To: freebsd-mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D10751065675 for ; Sun, 26 Aug 2012 23:18:57 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 813D78FC16 for ; Sun, 26 Aug 2012 23:18:57 +0000 (UTC) Received: by ialo14 with SMTP id o14so8785190ial.13 for ; Sun, 26 Aug 2012 16:18:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=vV/55IW1m35Nojy+yoYqvhV4JtFulO/qaNQbLw6wfac=; b=Rz3kyYiP8rnLr4XnTxE4N8jp+Z+t7mQOJus40IdW6rXsMTmdU2K2rtcwIDfYLZPLnX VFHJ+SfgnXPFtnflF8OmiBMpUcJeR8qh13lWWEKpTBZBaprV3C7VPAeKOgqeEZ7g55SW mzzMsJpJK2MKBNWWbAyk4/fA3TE7UlhB81xFNF0Q0suQcEphb+Dp9rYBerZWYxWqepD5 5uF3HfeK8BehI0DnpCQw3O3WlxSRL0HaOekCevz8mp7sNjHs8JVPDOpovtl6ZPjS6nFH 5YMqgfhD+WWkZiMuMOuW5rrh60eXlj3/HytNfYzz5iHzSOsjx0Fnh66580su1ZKfZNk1 gC2w== Received: by 10.50.88.229 with SMTP id bj5mr8287211igb.21.1346023136744; Sun, 26 Aug 2012 16:18:56 -0700 (PDT) Received: from 63.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id i2sm4979193igl.8.2012.08.26.16.18.54 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 26 Aug 2012 16:18:55 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <1346011312.1140.107.camel@revolution.hippie.lan> Date: Sun, 26 Aug 2012 17:18:54 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <26F65164-EC99-483E-9FA8-916AD99CBCBC@bsdimp.com> References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> <1346002922.1140.56.camel@revolution.hippie.lan> <1346011312.1140.107.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQmoQfPiY+ZmwnZeJpx8rYO33U7vFyAoBUJAGUcAEMABkvSB7toJ/9nhlYP3gY2qmC1SBavI Cc: Tim Kientzle , freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, "arch@" Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2012 23:18:57 -0000 On Aug 26, 2012, at 2:01 PM, Ian Lepore wrote: > On Sun, 2012-08-26 at 11:53 -0700, Tim Kientzle wrote: >> These rules sound reasonable. Good documentation might >> also give examples of what the PRE/POST operations might entail >> (e.g., from the preceding discussion, it sounds like PREREAD >> and PREWRITE require at least a partial cache flush on ARM). >> That helps folks who are coming to the docs with some hardware >> background. >>=20 >=20 > I agree, I think it would be good to have something like a RATIONALE > section in the manpage that summarizes the issues faced by various > categories of platform (hardware coherency, software-assisted = coherency, > etc) and how they handle it. You don't want people coding to the > implementation (some of which is going on now, and I've been guilty of > it myself); if there were more info in the docs there'd be less > motivation to peek at the implementation. More documentation is good. >>> * Read and write sync operators may be combined in a single = call, >>> PRE and POST operators may not be. E.G., PREREAD|PREWRITE is >>> allowed, PREREAD|POSTREAD is not. We should note that while >>> read and write operations may be combined, on some platforms >>> PREREAD|PREWRITE is needlessly expensive when only a read is >>> being performed. >>=20 >>=20 >> PREREAD|POSTREAD doesn't sound useful to me, but >> why would it be explicitly forbidden? >>=20 >> Would you also forbid POSTREAD|PREWRITE? >> (For a buffer that has just completed a DMA read >> and is going to be immediately used for a DMA write?) >=20 > My thinking on forbidding PREREAD|POSTREAD is at least partly that it > removes some temptation to do the wrong thing: treat the busdma API as > if it were a general cpu cache manipulation library. =20 I think it should be read as POSTREAD + PREREAD, not PREREAD + POSTREAD. > With the new definition of the sequences, a PREREAD|POSTREAD operation > is nonsensical because it leaves no window during which the DMA = hardware > has access to memory; it is in effect a no-op. If you think in terms = of > implementation you might think "This would have to cause a cache > invalidation." If you think in terms of API you should be thinking > something more like "This marks out a time window during which the DMA > hardware has safe access to that memory which has a duration of zero." Acutally, it would allow one to shift the DMA from one device to = another, if read the way I say above. > POSTREAD|PREWRITE is interesting. It is not a no-op in terms of the > API. It closes the hardware access window that was opened by an = earlier > PREREAD, and it opens a new hardware access window with the PREWRITE. > Whether it touches hardware or caches or anything in a given > implementation isn't the point, in terms of the API it's the right = thing > to do for a pair of back to back DMA operations, without intervening = CPU > access, on the same memory. Right. This is useful in many cases. Consider a log structured device = that reads data into memory, and then writes it back to the tail of the = log as a defragging operation. > So PREREAD|POSTREAD and PREWRITE|POSTWRITE make no sense, but all = other > combos should be allowed. Maybe instead of allowing and forbidding > specific combos, we should just advise that these two are effectively > no-ops. I respectfully disagree.. Warner