From owner-freebsd-x11@freebsd.org Mon Apr 29 23:15:44 2019 Return-Path: Delivered-To: freebsd-x11@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 73FD615A0B2C for ; Mon, 29 Apr 2019 23:15:44 +0000 (UTC) (envelope-from tychon@freebsd.org) Received: from pb-smtp21.pobox.com (pb-smtp21.pobox.com [173.228.157.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C28A6822E0; Mon, 29 Apr 2019 23:15:43 +0000 (UTC) (envelope-from tychon@freebsd.org) Received: from pb-smtp21.pobox.com (unknown [127.0.0.1]) by pb-smtp21.pobox.com (Postfix) with ESMTP id 11CE663308; Mon, 29 Apr 2019 19:15:42 -0400 (EDT) (envelope-from tychon@freebsd.org) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=content-type :mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s=sasl; bh= 82/FLxI4z0Eb9a9hsaTEa1NtIFI=; b=jn1JHUhObn4Qkfy6iZV94jjO+1VW9Bc1 h/KzMnGq2UmCZWL0xY1uIGrgU2U1qUx+1J0NQr35avV3ATTyR+hDi6KjyCFU7Gy3 7bAlO05X6HFjMp3hmak0xnJnsGIFi6Bq3KsTvgUB+TOpZwuws5KzoUGmbmRHZLd3 fRSVtfDvLBs= Received: from pb-smtp21.sea.icgroup.com (unknown [127.0.0.1]) by pb-smtp21.pobox.com (Postfix) with ESMTP id 0AAE263306; Mon, 29 Apr 2019 19:15:42 -0400 (EDT) (envelope-from tychon@freebsd.org) Received: from [10.0.1.195] (unknown [146.115.68.244]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by pb-smtp21.pobox.com (Postfix) with ESMTPSA id 3C94A63304; Mon, 29 Apr 2019 19:15:39 -0400 (EDT) (envelope-from tychon@freebsd.org) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Re: dmar, dma_pool, etc From: Tycho Nightingale In-Reply-To: <9E61210C-4939-4D3A-8110-72023B67BBE6@freebsd.org> Date: Mon, 29 Apr 2019 19:15:36 -0400 Cc: "freebsd-x11@freebsd.org" , Johannes Lundberg Content-Transfer-Encoding: quoted-printable Message-Id: References: <9E2356CF-6483-4C06-B4A8-0120088063FE@freebsd.org> <60b447bb-81da-4c01-e164-bdf10e5560b0@freebsd.org> <594E1E71-6834-431E-B122-005E64EDB1C2@freebsd.org> <3a07ffef-a978-2fdd-8d54-85fc0b6f3a63@freebsd.org> <23fe1183-d12c-b4b8-958f-34cee6e33977@freebsd.org> <9E61210C-4939-4D3A-8110-72023B67BBE6@freebsd.org> To: Niclas Zeising X-Mailer: Apple Mail (2.3445.9.1) X-Pobox-Relay-ID: B3E15DBA-6AD4-11E9-96A1-EE24A11ADF13-09779102!pb-smtp21.pobox.com X-Rspamd-Queue-Id: C28A6822E0 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.97)[-0.973,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:11403, ipnet:173.228.157.0/24, country:US] X-BeenThere: freebsd-x11@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: X11 on FreeBSD -- maintaining and support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 23:15:44 -0000 Hi, > On Apr 29, 2019, at 4:24 PM, Tycho Nightingale = wrote: >=20 >=20 >> On Apr 29, 2019, at 2:34 PM, Niclas Zeising = wrote: >>=20 >> On 2019-04-29 20:27, Niclas Zeising wrote: >>> On 2019-04-29 18:00, Tycho Nightingale wrote: >>>>> On Apr 29, 2019, at 11:06 AM, Niclas Zeising = wrote: >>>>>=20 >>>>> As a side note, I can readily reproduce the hang on a spare = laptop, please let me know if I can help in testing or diagnosing in any = way. >>>>=20 >>>>=20 >>>> If you can readily reproduce the hang, since there are 2 halves = that comprised the fix (the DMA pool and non-pool mappings) it would be = instructive to try reverting either dmapool.h or dma-mapping.h = independently to see if that helps. >>>>=20 >>> Hi! >>> I will test this and report back. Thank you! >>> Regards >>=20 >> Hi! >> Just reverting dmapool.h or dma-mapping.h doesn't work, it won't = build. I need to revert more than that. Can you help me figure out what = to revert in either case, or provide a patch? >=20 > Thanks for trying. I managed to setup my HW and reproduce this and I = see your point that it doesn=E2=80=99t cleave in half perfectly. >=20 > Since I=E2=80=99ve got a (non)-working test case, let me investigate a = bit further. I believe I=E2=80=99ve figured this out. At least in my case I=E2=80=99ve= been able to eliminate the hangs with the patch included at the bottom = of this email. The issue stems from the implementation of dma_map_sg(). According to = the linux DMA documentation[1], entries of the scatter/gather list may = be coalesced: int dma_map_sg(struct device *dev, struct scatterlist *sg, int = nents, enum dma_data_direction direction) Returns: the number of DMA address segments mapped (this may be = shorter than passed in if some elements of the scatter/gather = list are physically or virtually adjacent and an IOMMU maps them with a = single entry). My implementation of dma_map_sg() does just that. As it turns out there = are several consumers of dma_map_sg(), e.g. i915_gem_map_dma_buf() and = i915_gem_gtt_prepare_pages(), and mock_map_dma_buf() among others that = aren=E2=80=99t compliant with this documented API. Going back to the = non-coalesced version is likely the only path forward as bugs in the = callee redefine this API de facto. If this addresses the hangs you are seeing, I will post this on = Phabricator. Tycho [1] https://www.kernel.org/doc/Documentation/DMA-API.txt Index: sys/compat/linuxkpi/common/src/linux_pci.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sys/compat/linuxkpi/common/src/linux_pci.c (revision 346687) +++ sys/compat/linuxkpi/common/src/linux_pci.c (working copy) @@ -565,10 +565,8 @@ { struct linux_dma_priv *priv; struct linux_dma_obj *obj; - struct scatterlist *dma_sg, *sg; - int dma_nents, error, nseg; - size_t seg_len; - vm_paddr_t seg_phys, prev_phys_end; + struct scatterlist *sg; + int error, i, nseg; bus_dma_segment_t seg; =20 priv =3D dev->dma_priv; @@ -580,25 +578,11 @@ return (0); } =20 - sg =3D sgl; - dma_sg =3D sg; - dma_nents =3D 0; - while (nents > 0) { - seg_phys =3D sg_phys(sg); - seg_len =3D sg->length; - while (--nents > 0) { - prev_phys_end =3D sg_phys(sg) + sg->length; - sg =3D sg_next(sg); - if (prev_phys_end !=3D sg_phys(sg)) - break; - seg_len +=3D sg->length; - } - + for_each_sg(sgl, sg, nents, i) { nseg =3D -1; mtx_lock(&priv->dma_lock); if (_bus_dmamap_load_phys(priv->dmat, obj->dmamap, - seg_phys, seg_len, BUS_DMA_NOWAIT, - &seg, &nseg) !=3D 0) { + sg_phys(sg), sg->length, 0, &seg, &nseg) !=3D 0) { bus_dmamap_unload(priv->dmat, obj->dmamap); bus_dmamap_destroy(priv->dmat, obj->dmamap); mtx_unlock(&priv->dma_lock); @@ -607,14 +591,9 @@ } mtx_unlock(&priv->dma_lock); KASSERT(++nseg =3D=3D 1, ("More than one segment = (nseg=3D%d)", nseg)); + sg_dma_address(sg) =3D seg.ds_addr; + } =20 - sg_dma_address(dma_sg) =3D seg.ds_addr; - sg_dma_len(dma_sg) =3D seg.ds_len; - - dma_sg =3D sg_next(dma_sg); - dma_nents++; - } - obj->dma_addr =3D sg_dma_address(sgl); =20 mtx_lock(&priv->ptree_lock); @@ -629,7 +608,7 @@ return (0); } =20 - return (dma_nents); + return (nents); } =20 void=