From owner-freebsd-drivers@freebsd.org Mon Sep 7 19:55:34 2015 Return-Path: Delivered-To: freebsd-drivers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 64DE59CDB65 for ; Mon, 7 Sep 2015 19:55:34 +0000 (UTC) (envelope-from leonardofogel@yahoo.com.br) Received: from nm4-vm6.bullet.mail.ne1.yahoo.com (nm4-vm6.bullet.mail.ne1.yahoo.com [98.138.91.97]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2FBA512B4 for ; Mon, 7 Sep 2015 19:55:33 +0000 (UTC) (envelope-from leonardofogel@yahoo.com.br) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com.br; s=s2048; t=1441655727; bh=5iV4XDLzxY9nfwtGGiiNcBIRtZRH1Reqg2GbJCNn5po=; h=Date:From:Subject:To:Cc:In-Reply-To:From:Subject; b=q6bpmX6DRwI/xg50Qcah+uex1bEafEwVz6BFE2YHBe1033+rhfKk2itJ+Evrpcbg/sjhNqQolcgpncgLZcVr8uBuh5tpuXnb6AnF5uVt5wrneUzePIEWvb/aTyw21PaK/fKzgiYi6LJ/XDyoVmbnj44cH7gpWhqB1cwPlO5VBj5hcqq7hS1+Uog4MzzMqv40N3dhr4G40myQFQn/Ilwl0jcQWaTarJUXmFogqXoXpLaM4MeOQUZph4z6M1GKLfPOJ251qfsuVFJw4whiywL940BBuXgpeHr39eDJ9ffNvkuay02czXVcxUJVlHn37YAlUmJMrYdAho7VfVZII2C+tg== Received: from [98.138.100.112] by nm4.bullet.mail.ne1.yahoo.com with NNFMP; 07 Sep 2015 19:55:27 -0000 Received: from [98.138.87.5] by tm103.bullet.mail.ne1.yahoo.com with NNFMP; 07 Sep 2015 19:55:27 -0000 Received: from [127.0.0.1] by omp1005.mail.ne1.yahoo.com with NNFMP; 07 Sep 2015 19:55:27 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 483469.1119.bm@omp1005.mail.ne1.yahoo.com Received: (qmail 76921 invoked by uid 60001); 7 Sep 2015 19:55:27 -0000 X-YMail-OSG: oqATtNAVM1lU0pejSQ978aNuHWoEA9BBCdDmywl.vo2rQlP zcUbD_g6ugMGZzHYPxc.AgjUvsc8sZxhWWVolpbZCEkbRo8HOsIjAn.UEQAr aAEqobvKmtdTzJicrgY24E.MUSCi0FRysvTINvWS.GTTODDZIkYOjtyqcRDp PJ0duF03rJ06lckLI7oFC940oFChl.lEDXE._a5e.SOFlRjYp2g3oVRT6k_d PYXy.D4sfjS8ZCRJVjf4Zbv8.yEwZHghexkt4O3T63lsOlcnIcF98byT9e9v iLI0wPWJHMMv_qY_YmMioecaebxDqhJoO0FcRDCnaX7reg8QzxRyUJZc7uT0 9C7p.8l8wCvw5IycDTBjkA0jzHKrIFZxkFg3czBhzrk6fsNItgNgeJWSFst_ BCiV.nEqwE3Fi7Th2n6FsQZUTb3bOAHEbdXQMtMqSXGFi0NzLdZAGHFtI7eT PJzDcc7wLG8k7_NMuL71dPcKgR4J5Y1JxeI3Krqe3UucLAQfeKjTxYe86Sv4 8mM0fhckxPF6ti.efwvJK6TkYRbWXp5ekgN6G38He9K7N6M5qNCCpadVrYpH Ylih9UCz9BIhhuaSAYg-- Received: from [189.60.224.236] by web120802.mail.ne1.yahoo.com via HTTP; Mon, 07 Sep 2015 12:55:27 PDT X-Rocket-MIMEInfo: 002.001, PiA.IENhc2UgMToNCj4gPiAgICBidXNfd3JpdGVfMShyZWdpb25fMCwgLi4uKTsNCj4gPiAgICAvKiBiYXJyaWVyIGhlcmUgKi8NCj4gPiAgICBERUxBWShzb21lX3RpbWUpOw0KPiA.DQo.ID4gQ2FzZSAyOg0KPiA.ICAgIGJ1c193cml0ZV8xKHJlZ2lvbl8wLCAuLi4pOw0KPiA.ICAgIC8qIGJhcnJpZXIgaGVyZSAqLw0KPiA.ICAgIGJ1c193cml0ZV8xKHJlZ2lvbl8yLCAuLi4pOw0KPiA.DQo.ID4gSW4gdGhlIGZpcnN0IG9uZSwgSSB3YW50IHRoZSB3cml0ZSB0byByZWFjaCB0aGUgZGV2aWNlIGJlZm9yZSABMAEBAQE- X-Mailer: YahooMailBasic/651 YahooMailWebService/0.8.203.813 Message-ID: <1441655727.36257.YahooMailBasic@web120802.mail.ne1.yahoo.com> Date: Mon, 7 Sep 2015 12:55:27 -0700 From: Leonardo Fogel Subject: Re: Memory barrier To: freebsd-drivers@freebsd.org In-Reply-To: <20150906180311.GS2072@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Sep 2015 19:55:34 -0000 > > Case 1: > > bus_write_1(region_0, ...); > > /* barrier here */ > > DELAY(some_time); > > > > Case 2: > > bus_write_1(region_0, ...); > > /* barrier here */ > > bus_write_1(region_2, ...); > > > > In the first one, I want the write to reach the device before the threa= d busy-waits. > > > > In the second one, I want the write to a device (e.g. power management)= to > > complete before the write to another starts/completes. >=20 > I believe that the bus_write semantic includes the required serialization= . > E.g., on x86 all CPU write buffers are flushed before the write instructi= on > is declared completed, because this is the semantic of the uncacheable > memory. For powerpc, the system automatically inserts powerpc_iomb() aft= er > the write, which is full sync. I am not aware of other architectures. I've found the implementation of the bus_space_barrier for the ARM architec= ture (the one in which I'm interested): generic_bs_barrier(bus_space_tag_t t, bus_space_handle_t bsh, bus_size_t= offset, bus_size_t len, int flags) { /* * dsb() will drain the L1 write buffer and establish a memory a= ccess * barrier point on platforms where that has meaning. On a writ= e we * also need to drain the L2 write buffer, because most on-chip = memory * mapped devices are downstream of the L2 cache. Note that thi= s needs * to be done even for memory mapped as Device type, because whi= le * Device memory is not cached, writes to it are still buffered. */ dsb(); if (flags & BUS_SPACE_BARRIER_WRITE) { cpu_l2cache_drain_writebuf(); } } The ARM architecture specifies two _data_ barrier instructions: DMB and DSB= . The first synchronizes memory accesses, and the second synchronizes both = memory accesses and instruction execution. So, DSB is the answer to Case 1,= and DMB or DSB is the answer to Case 2. The implementation above brings something of which I was not aware: it also= drains the L2 write buffer. Older implementations of the "PL310 Store Buff= er did not have any automatic draining mechanism." (ARM CoreLink Level 2 Ca= che Controller (L2C-310 or PL310), r3 releases, Software Developers Errata = Notice.) In newer implementations, the writes to device memory are "Put in = store buffer, not merged, immediately drained to L3." (CoreLink Level 2 Cac= he Controller L2C-310 Technical Reference Manual=09Revision: r3p3.) Leonardo