From owner-freebsd-arch@freebsd.org Tue Mar 22 17:45:28 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C6475AD8099 for ; Tue, 22 Mar 2016 17:45:28 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A207D34C for ; Tue, 22 Mar 2016 17:45:28 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 2EFCCB9AD for ; Tue, 22 Mar 2016 13:45:27 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: Wrapper API for static bus_dma allocations Date: Tue, 22 Mar 2016 10:45:19 -0700 Message-ID: <1576055.6UjqO6PkRX@ralph.baldwin.cx> User-Agent: KMail/4.14.3 (FreeBSD/10.2-STABLE; KDE/4.14.3; amd64; ; ) In-Reply-To: <2856669.mkVhDvxH7k@ralph.baldwin.cx> References: <2800970.jY4xzTy9Hz@ralph.baldwin.cx> <2856669.mkVhDvxH7k@ralph.baldwin.cx> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 22 Mar 2016 13:45:27 -0400 (EDT) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Mar 2016 17:45:28 -0000 On Friday, February 26, 2016 05:10:53 PM John Baldwin wrote: > On Thursday, January 29, 2015 03:37:19 PM John Baldwin wrote: > > The bus_dma API to allocate a chunk of static DMA'able memory (e.g.= for=20 > > descriptor rings) can be a bit obtuse to use in that it require a b= it of=20 > > boilerplate to create a tag, allocate memory for the tag, then load= it to get=20 > > the bus address. Similarly, freeing it also requires several steps= . In=20 > > addition, some of the steps are a bit subtle (e.g. the need to chec= k for an=20 > > error in the bus_dma callback instead of from bus_dmamap_load()) an= d not all=20 > > drivers get those correct. > >=20 > > To try to make this simpler I've written a little wrapper API that = tries to=20 > > provide a single call to allocate a buffer and a single call to fre= e a buffer. =20 > > Each buffer is described by a structure defined by the API, and if = the call to=20 > > allocate a buffer succeeds, the structure contains both a pointer t= o the=20 > > buffer in the kernel's address space as well as a bus address of th= e buffer. > >=20 > > In the interests of simplicity, this API does not allow the buffer = to be quite=20 > > as fully configured as the underlying bus_dma API, instead it aims = to handle=20 > > the most common cases that are used in most drivers. As such, it a= ssumes that=20 > > the buffer must be one contiguous range of DMA address space, and t= he only > > parameters that can be specified are the parent tag, the alignment = of the=20 > > buffer, the lowaddr parameter, the size of the buffer to allocate, = and the=20 > > flags parameter that is normally passed to bus_dmamem_alloc(). I b= elieve that=20 > > this should be sufficient to cover the vast majority of the drivers= in our=20 > > tree. After some more thinking, I've altered this API to include an 'args' st= ruct similar to the one Konstantin used for make_dev_s() to specify constrai= nts. This would permit most constraints to be specified on an as-needed basi= s without requiring all of them each time. It does still assume 1 contig= uous region, but the majority of our drivers require that anyway. I've forward ported this and converted a more typical driver (xl(4)) as= a demo of the new API. You can find the full diff here: https://reviews.freebsd.org/D5704 (I've not yet written manpage updates) Here's the new code in xl(4) to allocate the TX and RX rings. I think = it highlights the specific constraints (alignment, etc.) more clearly than= the previous version: =09=E2=80=8B /* =09=E2=80=8B * Now allocate a chunk of DMA-able memory for the = DMA =09=E2=80=8B * descriptor lists. All of our lists are allocate= d as a =09=E2=80=8B * contiguous block of memory. =09=E2=80=8B */ =09=E2=80=8B bus_dma_mem_args_init(&args); =09=E2=80=8B args.dma_alignment =3D 8; =09=E2=80=8B args.dma_lowaddr =3D BUS_SPACE_MAXADDR_32BIT; =09=E2=80=8B error =3D bus_dma_mem_alloc(bus_get_dma_tag(dev), X= L_RX_LIST_SZ, 0, &args, =09=E2=80=8B &sc->xl_ldata.xl_rx_ring); =09=E2=80=8B if (error) { =09=E2=80=8B device_printf(dev, "failed to allocate rx r= ing\n"); =09=E2=80=8B goto fail; =09=E2=80=8B } =09=E2=80=8B sc->xl_ldata.xl_rx_list =3D sc->xl_ldata.xl_rx_ring= .dma_vaddr; =09=E2=80=8B =09=E2=80=8B error =3D bus_dma_mem_alloc(bus_get_dma_tag(dev), X= L_TX_LIST_SZ, 0, &args, =09=E2=80=8B &sc->xl_ldata.xl_tx_ring); =09=E2=80=8B if (error) { =09=E2=80=8B device_printf(dev, "failed to allocate tx r= ing\n"); =09=E2=80=8B goto fail; =09=E2=80=8B } =09=E2=80=8B sc->xl_ldata.xl_tx_list =3D sc->xl_ldata.xl_tx_ring= .dma_vaddr; =E2=80=8B --=20 John Baldwin