Date: Mon, 29 Apr 2019 19:15:36 -0400 From: Tycho Nightingale <tychon@freebsd.org> To: Niclas Zeising <zeising@freebsd.org> Cc: "freebsd-x11@freebsd.org" <freebsd-x11@freebsd.org>, Johannes Lundberg <johalun@FreeBSD.org> Subject: Re: dmar, dma_pool, etc Message-ID: <C5E9EE94-00F7-4F9D-AFAC-48609F635C8D@freebsd.org> In-Reply-To: <9E61210C-4939-4D3A-8110-72023B67BBE6@freebsd.org> References: <e0415524-3126-5ea2-c2e2-3d3dccc6832e@FreeBSD.org> <9E2356CF-6483-4C06-B4A8-0120088063FE@freebsd.org> <60b447bb-81da-4c01-e164-bdf10e5560b0@freebsd.org> <594E1E71-6834-431E-B122-005E64EDB1C2@freebsd.org> <3a07ffef-a978-2fdd-8d54-85fc0b6f3a63@freebsd.org> <23fe1183-d12c-b4b8-958f-34cee6e33977@freebsd.org> <9E61210C-4939-4D3A-8110-72023B67BBE6@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, > On Apr 29, 2019, at 4:24 PM, Tycho Nightingale <tychon@freebsd.org> = wrote: >=20 >=20 >> On Apr 29, 2019, at 2:34 PM, Niclas Zeising <zeising@freebsd.org> = wrote: >>=20 >> On 2019-04-29 20:27, Niclas Zeising wrote: >>> On 2019-04-29 18:00, Tycho Nightingale wrote: >>>>> On Apr 29, 2019, at 11:06 AM, Niclas Zeising <zeising@FreeBSD.org> = wrote: >>>>>=20 >>>>> As a side note, I can readily reproduce the hang on a spare = laptop, please let me know if I can help in testing or diagnosing in any = way. >>>>=20 >>>>=20 >>>> If you can readily reproduce the hang, since there are 2 halves = that comprised the fix (the DMA pool and non-pool mappings) it would be = instructive to try reverting either dmapool.h or dma-mapping.h = independently to see if that helps. >>>>=20 >>> Hi! >>> I will test this and report back. Thank you! >>> Regards >>=20 >> Hi! >> Just reverting dmapool.h or dma-mapping.h doesn't work, it won't = build. I need to revert more than that. Can you help me figure out what = to revert in either case, or provide a patch? >=20 > Thanks for trying. I managed to setup my HW and reproduce this and I = see your point that it doesn=E2=80=99t cleave in half perfectly. >=20 > Since I=E2=80=99ve got a (non)-working test case, let me investigate a = bit further. I believe I=E2=80=99ve figured this out. At least in my case I=E2=80=99ve= been able to eliminate the hangs with the patch included at the bottom = of this email. The issue stems from the implementation of dma_map_sg(). According to = the linux DMA documentation[1], entries of the scatter/gather list may = be coalesced: int dma_map_sg(struct device *dev, struct scatterlist *sg, int = nents, enum dma_data_direction direction) Returns: the number of DMA address segments mapped (this may be = shorter than <nents> passed in if some elements of the scatter/gather = list are physically or virtually adjacent and an IOMMU maps them with a = single entry). My implementation of dma_map_sg() does just that. As it turns out there = are several consumers of dma_map_sg(), e.g. i915_gem_map_dma_buf() and = i915_gem_gtt_prepare_pages(), and mock_map_dma_buf() among others that = aren=E2=80=99t compliant with this documented API. Going back to the = non-coalesced version is likely the only path forward as bugs in the = callee redefine this API de facto. If this addresses the hangs you are seeing, I will post this on = Phabricator. Tycho [1] https://www.kernel.org/doc/Documentation/DMA-API.txt Index: sys/compat/linuxkpi/common/src/linux_pci.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sys/compat/linuxkpi/common/src/linux_pci.c (revision 346687) +++ sys/compat/linuxkpi/common/src/linux_pci.c (working copy) @@ -565,10 +565,8 @@ { struct linux_dma_priv *priv; struct linux_dma_obj *obj; - struct scatterlist *dma_sg, *sg; - int dma_nents, error, nseg; - size_t seg_len; - vm_paddr_t seg_phys, prev_phys_end; + struct scatterlist *sg; + int error, i, nseg; bus_dma_segment_t seg; =20 priv =3D dev->dma_priv; @@ -580,25 +578,11 @@ return (0); } =20 - sg =3D sgl; - dma_sg =3D sg; - dma_nents =3D 0; - while (nents > 0) { - seg_phys =3D sg_phys(sg); - seg_len =3D sg->length; - while (--nents > 0) { - prev_phys_end =3D sg_phys(sg) + sg->length; - sg =3D sg_next(sg); - if (prev_phys_end !=3D sg_phys(sg)) - break; - seg_len +=3D sg->length; - } - + for_each_sg(sgl, sg, nents, i) { nseg =3D -1; mtx_lock(&priv->dma_lock); if (_bus_dmamap_load_phys(priv->dmat, obj->dmamap, - seg_phys, seg_len, BUS_DMA_NOWAIT, - &seg, &nseg) !=3D 0) { + sg_phys(sg), sg->length, 0, &seg, &nseg) !=3D 0) { bus_dmamap_unload(priv->dmat, obj->dmamap); bus_dmamap_destroy(priv->dmat, obj->dmamap); mtx_unlock(&priv->dma_lock); @@ -607,14 +591,9 @@ } mtx_unlock(&priv->dma_lock); KASSERT(++nseg =3D=3D 1, ("More than one segment = (nseg=3D%d)", nseg)); + sg_dma_address(sg) =3D seg.ds_addr; + } =20 - sg_dma_address(dma_sg) =3D seg.ds_addr; - sg_dma_len(dma_sg) =3D seg.ds_len; - - dma_sg =3D sg_next(dma_sg); - dma_nents++; - } - obj->dma_addr =3D sg_dma_address(sgl); =20 mtx_lock(&priv->ptree_lock); @@ -629,7 +608,7 @@ return (0); } =20 - return (dma_nents); + return (nents); } =20 void=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?C5E9EE94-00F7-4F9D-AFAC-48609F635C8D>