From owner-freebsd-x11@freebsd.org Mon Apr 29 16:00:42 2019 Return-Path: Delivered-To: freebsd-x11@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D6F79159508E for ; Mon, 29 Apr 2019 16:00:41 +0000 (UTC) (envelope-from tychon@freebsd.org) Received: from pb-smtp21.pobox.com (pb-smtp21.pobox.com [173.228.157.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4526A8A10A; Mon, 29 Apr 2019 16:00:41 +0000 (UTC) (envelope-from tychon@freebsd.org) Received: from pb-smtp21.pobox.com (unknown [127.0.0.1]) by pb-smtp21.pobox.com (Postfix) with ESMTP id DE44C60652; Mon, 29 Apr 2019 12:00:38 -0400 (EDT) (envelope-from tychon@freebsd.org) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=content-type :mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s=sasl; bh= mLxbR7yZ+FA5mnKQMSNtWtQyIjo=; b=bUgPQr+EbwuM+ntHE2JZTjK46bxyJOl5 KH1fXjqCeSEv8qCgI30MZw6GYRvuBYrdfXX4vDCocn7/ZYfvKnNvZ12JG8J/3Fxr Skx5sHh6cnjJ01RrVkPrXtBZ4Zv/uVJ/M47CShoMryUFKDzGVvymI7NsfWjvZ4T0 vdyCoQ7Xdeo= Received: from pb-smtp21.sea.icgroup.com (unknown [127.0.0.1]) by pb-smtp21.pobox.com (Postfix) with ESMTP id D5F4160651; Mon, 29 Apr 2019 12:00:38 -0400 (EDT) (envelope-from tychon@freebsd.org) Received: from [10.0.1.195] (unknown [146.115.68.244]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by pb-smtp21.pobox.com (Postfix) with ESMTPSA id 108BD6064D; Mon, 29 Apr 2019 12:00:34 -0400 (EDT) (envelope-from tychon@freebsd.org) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Re: dmar, dma_pool, etc From: Tycho Nightingale In-Reply-To: <60b447bb-81da-4c01-e164-bdf10e5560b0@freebsd.org> Date: Mon, 29 Apr 2019 12:00:33 -0400 Cc: Johannes Lundberg , "freebsd-x11@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <594E1E71-6834-431E-B122-005E64EDB1C2@freebsd.org> References: <9E2356CF-6483-4C06-B4A8-0120088063FE@freebsd.org> <60b447bb-81da-4c01-e164-bdf10e5560b0@freebsd.org> To: Niclas Zeising X-Mailer: Apple Mail (2.3445.9.1) X-Pobox-Relay-ID: EC9389AE-6A97-11E9-BD33-EE24A11ADF13-09779102!pb-smtp21.pobox.com X-Rspamd-Queue-Id: 4526A8A10A X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.96)[-0.965,0]; ASN(0.00)[asn:11403, ipnet:173.228.157.0/24, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-x11@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: X11 on FreeBSD -- maintaining and support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 16:00:42 -0000 Hi, > On Apr 29, 2019, at 11:06 AM, Niclas Zeising = wrote: >=20 > On 2019-04-29 16:41, Tycho Nightingale wrote: >> Hi, >>> On Apr 27, 2019, at 2:46 PM, Johannes Lundberg = wrote: >>>=20 >>> Hi >>>=20 >>> Tycho, I'd like to understand what is the goal with the changes to >>> linuxkpi and what you plan to use it for. LinuxKPI base is tightly >>> connected to LinuxkPI GPLv2 and the DRM drivers in ports. We need to >>> make sure that any change to base LinuxKPI is compatible with what = we >>> have in ports, or patch the ports to handle the changes in base. >> Understood. The reports of the recent change to LinuxKPI having = untoward effects are concerning. I=E2=80=99m trying to get more = information to understand them and at the same time setup a machine in = an attempt to reproduce them. >> Stepping back a bit, my addition of bus_dma to LinuxKPI is to make = the mlx4/mlx5 drivers work in an environment where the IOMMU is enabled. = Before physical addresses and dma addresses were not differentiated = making this impossible. The intention of my patch was to get these = devices into compliance and perhaps also other devices; minimally when = the IOMMU is disabled things should behave as they were before. >=20 > How do I disable the IOMMU? Removal of the DMAR ACPI table (this is what typical BIOS knob does) is = the sure way to ensure the IOMMU is off, regardless of system knobs. In = the BIOS ensure that =E2=80=98VT-d=E2=80=99 or =E2=80=98Directed I/O=E2=80= =99 is disabled. >>> For example, there is a CONFIG_INTEL_IOMMU options in Linux code = that >>> enables DMAR. It turns out, ttm has it's own dma_pool = implementation. >>> This is possible since in Linux dma_pool is private to dmapool.c. >>> Enabling this option for us cause a compile error since dma_pool is >>> public in base linuxkpi. I don't know if this is really a problem if >>> CONFIG_INTEL_IOMMU (or CONFIG_SWIOTBL) are options that we'll never = use=E2=80=A6 >> Sounds like there is some redundancy there which can be eliminated. = But with respect to your question of enabling the IOMMU outside of base; = that=E2=80=99s more than what I intended to. >>> Also, we do have a problem with Firefox causing GPU hangs so I'd >>> appreciate it if Tycho could look through linuxkpi_gplv2, drm and >>> i915kms (i915kms does not use ttm so no need to look there for = problems >>> with Intel GPU) to see if there are any places needing patching. I = know >>> there's one vtophys() call in fb_mmap() but IIRC, that is never = used. >>> I'll look into that next. There are also uses of PHYS_TO_DMAP() and >>> VM_PAGE_TO_PHYS(). Would any of these need patching? >>>=20 >>> Use the default branch at https://github.com/FreeBSDDesktop/kms-drm >> I=E2=80=99m planning to have a look, but to get things working as = before nothing further should need patching even if the physical = addresses are treated as dma addresses. >> For the GPU it=E2=80=99s important to note that enabling the IOMMU = didn=E2=80=99t work before, was not a goal of this error, and would be = expected to not work right now. However, it=E2=80=99s possible this = change revealed some cases where some DMAR functionality was enabled in = the BIOS and since the support is incomplete breakage is happening. Or = something else that I am overlooking right now. >=20 > Hi! > I understand that enabling the IOMMU in the drm drivers is not your = priority, which is OK. > Since this breaks graphics drivers, however, is it possible to revert = the change (and related changes if any) until we figure out what's going = on? There are numerous reports of hard lockups or GPU hangs with this = change, and it feels like a resolution of the issue is some time off = still. >=20 > As a side note, I can readily reproduce the hang on a spare laptop, = please let me know if I can help in testing or diagnosing in any way. If you can readily reproduce the hang, since there are 2 halves that = comprised the fix (the DMA pool and non-pool mappings) it would be = instructive to try reverting either dmapool.h or dma-mapping.h = independently to see if that helps. Tycho=