From owner-freebsd-x11@freebsd.org Tue Apr 30 12:26:07 2019 Return-Path: Delivered-To: freebsd-x11@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 98249158F49A for ; Tue, 30 Apr 2019 12:26:07 +0000 (UTC) (envelope-from f0andrey@gmail.com) Received: from mail-wr1-x441.google.com (mail-wr1-x441.google.com [IPv6:2a00:1450:4864:20::441]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A4F2F6DA16; Tue, 30 Apr 2019 12:26:06 +0000 (UTC) (envelope-from f0andrey@gmail.com) Received: by mail-wr1-x441.google.com with SMTP id o4so8846509wra.3; Tue, 30 Apr 2019 05:26:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=Hl9ZjYCkWIsU0V88EilX140wInMoKSLVMxM9hr+9GqA=; b=rUDtwQPUl2w03OoPRU9qCTdJyg6o8WjLymxY+jfHnrM5CS0VszVg3HIovSn+FIpUJ/ Xx+KXhGwC5+DdxbU9zmWfv02rX+cIvs1YtO3cmxhCxnsXG5PnVFxzRk+aeiRsnP8+M0t +CadQGIrWl2JlojL+yOEQEwuGo7/cKvaf980+8Rvt1CWAeKrDLKI3G13chwoi1OQxSIO SjSweEGxmD2bOWgfTiwApOthw+7RhpMeEki+QuE2XL3pFQpSOulS8GjkaBG8DUBPqsAX 0EC9GuF9pkXjFMCrY1KC0PhgPlGtee/WIl4XE2nJ+UpaSA32TKlNX8id/OSx1fYGDvdV 6CZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=Hl9ZjYCkWIsU0V88EilX140wInMoKSLVMxM9hr+9GqA=; b=AtU2Qvg9WPeEu+HQ81f9tXln900m1Z0Cm9tNYhbrmDWowuxBxtXo5iOE7KIObF2cyX 5WUh35On9jgJRK2ivTHuCE7dufpfQJEzyzMqwUlKZFR6y3b90bDcyAUhwtzTcoKnrx4t eNJIloqrW7WLs6fm0vcZlrAH4ZpJz84iZeay9wVwsrtod7MHTFDzKpM6QVrKyi1e2dHB vjrQjGp2nmnjgRN5Kcm7bhTVPSfgqVtF88MIYGd3u2Hghdz0VNMPg/8lDNyAHvEz92aP Z6T2bAiNcKaaAp3n9ORw/mmgQjtXifsD5YeHXSvttVNB9o3/crXqYuIIKYydTLEHa/o+ +CwQ== X-Gm-Message-State: APjAAAXdzdyhlYMG0+6NIXRa8IG0MmMFQbTgC48JdngIka1N6SANYf8y SFd734wy+VBHoKdJLuvqCOaefuB3WisRmc1lAYB6QK+re+us9Q== X-Google-Smtp-Source: APXvYqzcrEgsnDvHJ0Y7Fy3iMUuUM4iPWvvqWyP623irKkzmdkJRU6bDIdn+NyjfOoPn5AJaHubauwTrCGFhtx6nNGw= X-Received: by 2002:adf:dbce:: with SMTP id e14mr9036727wrj.249.1556627165406; Tue, 30 Apr 2019 05:26:05 -0700 (PDT) MIME-Version: 1.0 References: <9E2356CF-6483-4C06-B4A8-0120088063FE@freebsd.org> <60b447bb-81da-4c01-e164-bdf10e5560b0@freebsd.org> <594E1E71-6834-431E-B122-005E64EDB1C2@freebsd.org> <3a07ffef-a978-2fdd-8d54-85fc0b6f3a63@freebsd.org> <23fe1183-d12c-b4b8-958f-34cee6e33977@freebsd.org> <9E61210C-4939-4D3A-8110-72023B67BBE6@freebsd.org> In-Reply-To: From: Andrey Fesenko Date: Tue, 30 Apr 2019 15:25:54 +0300 Message-ID: Subject: Re: dmar, dma_pool, etc To: Tycho Nightingale Cc: Niclas Zeising , "freebsd-x11@freebsd.org" , Johannes Lundberg Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: A4F2F6DA16 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=rUDtwQPU; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of f0andrey@gmail.com designates 2a00:1450:4864:20::441 as permitted sender) smtp.mailfrom=f0andrey@gmail.com X-Spamd-Result: default: False [-4.05 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCVD_IN_DNSWL_NONE(0.00)[1.4.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.55)[-0.551,0]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; IP_SCORE(-0.49)[ip: (2.10), ipnet: 2a00:1450::/32(-2.26), asn: 15169(-2.24), country: US(-0.06)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-x11@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: X11 on FreeBSD -- maintaining and support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 12:26:07 -0000 .On Tue, Apr 30, 2019 at 2:16 AM Tycho Nightingale wro= te: > > > Hi, > > > On Apr 29, 2019, at 4:24 PM, Tycho Nightingale wro= te: > > > > > >> On Apr 29, 2019, at 2:34 PM, Niclas Zeising wrot= e: > >> > >> On 2019-04-29 20:27, Niclas Zeising wrote: > >>> On 2019-04-29 18:00, Tycho Nightingale wrote: > >>>>> On Apr 29, 2019, at 11:06 AM, Niclas Zeising = wrote: > >>>>> > >>>>> As a side note, I can readily reproduce the hang on a spare laptop,= please let me know if I can help in testing or diagnosing in any way. > >>>> > >>>> > >>>> If you can readily reproduce the hang, since there are 2 halves that= comprised the fix (the DMA pool and non-pool mappings) it would be instruc= tive to try reverting either dmapool.h or dma-mapping.h independently to se= e if that helps. > >>>> > >>> Hi! > >>> I will test this and report back. Thank you! > >>> Regards > >> > >> Hi! > >> Just reverting dmapool.h or dma-mapping.h doesn't work, it won't build= . I need to revert more than that. Can you help me figure out what to reve= rt in either case, or provide a patch? > > > > Thanks for trying. I managed to setup my HW and reproduce this and I s= ee your point that it doesn=E2=80=99t cleave in half perfectly. > > > > Since I=E2=80=99ve got a (non)-working test case, let me investigate a = bit further. > > I believe I=E2=80=99ve figured this out. At least in my case I=E2=80=99v= e been able to eliminate the hangs with the patch included at the bottom of= this email. > > The issue stems from the implementation of dma_map_sg(). According to th= e linux DMA documentation[1], entries of the scatter/gather list may be coa= lesced: > > int > dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,= enum dma_data_direction direction) > > Returns: the number of DMA address segments mapped (this may be s= horter > than passed in if some elements of the scatter/gather lis= t are > physically or virtually adjacent and an IOMMU maps them with a si= ngle > entry). > > My implementation of dma_map_sg() does just that. As it turns out there = are several consumers of dma_map_sg(), e.g. i915_gem_map_dma_buf() and i915= _gem_gtt_prepare_pages(), and mock_map_dma_buf() among others that aren=E2= =80=99t compliant with this documented API. Going back to the non-coalesce= d version is likely the only path forward as bugs in the callee redefine th= is API de facto. > > If this addresses the hangs you are seeing, I will post this on Phabricat= or. > > Tycho > > [1] https://www.kernel.org/doc/Documentation/DMA-API.txt > > Index: sys/compat/linuxkpi/common/src/linux_pci.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/compat/linuxkpi/common/src/linux_pci.c (revision 346687) > +++ sys/compat/linuxkpi/common/src/linux_pci.c (working copy) > @@ -565,10 +565,8 @@ > { > struct linux_dma_priv *priv; > struct linux_dma_obj *obj; > - struct scatterlist *dma_sg, *sg; > - int dma_nents, error, nseg; > - size_t seg_len; > - vm_paddr_t seg_phys, prev_phys_end; > + struct scatterlist *sg; > + int error, i, nseg; > bus_dma_segment_t seg; > > priv =3D dev->dma_priv; > @@ -580,25 +578,11 @@ > return (0); > } > > - sg =3D sgl; > - dma_sg =3D sg; > - dma_nents =3D 0; > - while (nents > 0) { > - seg_phys =3D sg_phys(sg); > - seg_len =3D sg->length; > - while (--nents > 0) { > - prev_phys_end =3D sg_phys(sg) + sg->length; > - sg =3D sg_next(sg); > - if (prev_phys_end !=3D sg_phys(sg)) > - break; > - seg_len +=3D sg->length; > - } > - > + for_each_sg(sgl, sg, nents, i) { > nseg =3D -1; > mtx_lock(&priv->dma_lock); > if (_bus_dmamap_load_phys(priv->dmat, obj->dmamap, > - seg_phys, seg_len, BUS_DMA_NOWAIT, > - &seg, &nseg) !=3D 0) { > + sg_phys(sg), sg->length, 0, &seg, &nseg) !=3D 0) { > bus_dmamap_unload(priv->dmat, obj->dmamap); > bus_dmamap_destroy(priv->dmat, obj->dmamap); > mtx_unlock(&priv->dma_lock); > @@ -607,14 +591,9 @@ > } > mtx_unlock(&priv->dma_lock); > KASSERT(++nseg =3D=3D 1, ("More than one segment (nseg=3D= %d)", nseg)); > + sg_dma_address(sg) =3D seg.ds_addr; > + } > > - sg_dma_address(dma_sg) =3D seg.ds_addr; > - sg_dma_len(dma_sg) =3D seg.ds_len; > - > - dma_sg =3D sg_next(dma_sg); > - dma_nents++; > - } > - > obj->dma_addr =3D sg_dma_address(sgl); > > mtx_lock(&priv->ptree_lock); > @@ -629,7 +608,7 @@ > return (0); > } > > - return (dma_nents); > + return (nents); > } > > void Thanks, this patch make system more stable Farefox apparently work infinite= ly. However, a heavy application (3D game in wine) still hangs after a while, once hung after about a minute, about another (after reboot) ten minutes later, video artifacts appear before the crash