From owner-svn-src-all@freebsd.org Wed May 10 06:18:55 2017 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62C5CD654D7; Wed, 10 May 2017 06:18:55 +0000 (UTC) (envelope-from melounmichal@gmail.com) Received: from mail-wr0-x22d.google.com (mail-wr0-x22d.google.com [IPv6:2a00:1450:400c:c0c::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 049371E76; Wed, 10 May 2017 06:18:55 +0000 (UTC) (envelope-from melounmichal@gmail.com) Received: by mail-wr0-x22d.google.com with SMTP id z52so28407348wrc.2; Tue, 09 May 2017 23:18:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:reply-to:subject:to:cc:references:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding:content-language; bh=rlaJ5nahoC1q4tss2HXr4OLwUVFktswol17qtqc9eaw=; b=JQLp3tfGxgP4zzBl/oQzgQCglpe1fRVF4jsxe9X+JCAeuippZ+aYLjJs7tebQh/Y5k rMgUU9Qz0shKkhaSNIYCMkUjvlnlQI3h7xaofhcPlxjThqoT7+PB2nn4yzBoVCBduppJ B4nHoHtIRe4FEu5FfT72bxoQPs6YF1foYcM8Wscg6Utq0Unrc3caVaoIBMMQNlTSQXb6 2z0LLLUgYbIB6+iE0zNiFZAhLbR6VnJXYqTuvCLfDdawfXf89MHqfDtTNcRLuaqaPxEx FLtd7CiOFRnACKHWCPo9pQABgpTzAlQSBlYlC531TewGNZAKYUcsygjtzAFBF1p46Otd Y9/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:reply-to:subject:to:cc:references :message-id:date:user-agent:mime-version:in-reply-to :content-transfer-encoding:content-language; bh=rlaJ5nahoC1q4tss2HXr4OLwUVFktswol17qtqc9eaw=; b=Hb3hWinvZ9XA77VsDqdl5uVRWj4RgtrYzxATNzN/yxwUsrhGxVs0eV9z+LL0UQqKo6 gq6/WYyacIAUG8wwJisFYtTZ81duYme1L/RJlPyCfT6SIVUqJuCklA0YuXAXG2e4vIFU a2byWVJzpswHGJWvWVQOp6rbEP3Gd9B7+OFws0cLzyDIeTH0UUQLUwbn9WVObl0uzhcF r6AsaahxYj7PXvj0/dZ9jliYkvErXILxXJ0X+/fOe+ANs4ri6er53maXbIUxhRvMu0ux cD6RBbV044n7asSP3LLaUxv4bpUC6jSYoEsXINyb0bTTrilwBN1iBK7OxYPQzXBLB7dJ /5fg== X-Gm-Message-State: AODbwcCmxz0ClUt7mqCa9PdCkdk18pIdCMWeKHyNihYIvcI7pdmmQbOR JnN7mO8zVvjsJEd5Aj/M9Q== X-Received: by 10.28.152.133 with SMTP id a127mr3029472wme.115.1494397132415; Tue, 09 May 2017 23:18:52 -0700 (PDT) Received: from [88.208.79.100] (halouny.humusoft.cz. [88.208.79.100]) by smtp.gmail.com with ESMTPSA id c37sm2102514wra.16.2017.05.09.23.18.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 May 2017 23:18:51 -0700 (PDT) From: Michal Meloun X-Google-Original-From: Michal Meloun Reply-To: mmel@freebsd.org Subject: Re: svn commit: r318021 - in head/sys/arm: arm include To: Andrew Turner Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201705091105.v49B5WAp097952@repo.freebsd.org> <85745E3A-3260-43C7-B134-85BFED786D55@fubar.geek.nz> <8fe20c26-3c0c-98ce-227b-740491253047@freebsd.org> Message-ID: <46e9d14a-87d4-589d-8a1d-97800cdae0cf@freebsd.org> Date: Wed, 10 May 2017 08:18:52 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.1.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 May 2017 06:18:55 -0000 On 09.05.2017 17:09, Andrew Turner wrote: >> On 9 May 2017, at 13:40, Michal Meloun wrote: >> >> >> >> On 09.05.2017 13:34, Andrew Turner wrote: >>>> On 9 May 2017, at 12:05, Michal Meloun wrote: >>>> >>>> Author: mmel >>>> Date: Tue May 9 11:05:32 2017 >>>> New Revision: 318021 >>>> URL: https://svnweb.freebsd.org/changeset/base/318021 >>>> >>>> Log: >>>> Introduce pmap_remap_vm_attr(), >>>> it allows to remap one VM memattr class to another. >>>> >>>> This function is intent to be used as workaround for various SoC bugs, >>>> mainly access ordering/sequencing related bugs in crossbar fabric. >>> This seems quite heavy handed to change the attribute for all memory of a given type. >> Yes, exactly. See comment in D10218 - >> /* >> * Workaround for Marvell Armada38X family HW issue >> * between Cortex-A9 CPUs and on-chip devices that may >> * cause hang on heavy load. >> * To avoid that, map all registers including PCIe IO >> * as strongly ordered instead of device memory. >> */ > I don’t think it’s been answered if this is just for PCIe, or all devices. Well, I have not access to Armada errata, so I can't give you more exact answer. But this kind of bugs (ordering/sequencing problems in crossbar fabric) are not that uncommon, I think that some early versions of NPX QorIQ have exactly same problem (all devices must be mapped as SO), i.MX6 can hangs if devices are not mapped with shared flag... > >>> Other architectures have pmap_change_attr to change the attribute on a specific range of memory. >> Right. Problem is that I don't known any method how we can change >> memory attribute for live memory in SMP system, >> without hitting undefined behavior. > I would expect drivers to change the attributes early, before they access the memory. So then why driver does not request right type of memory at allocation? But yes, current situation around pmap_mapdev(), bus_space_map(), bus_activate_resource() and bus_map_resource() is not an optimal. At least for systems where devices must be mapped with different attributes - e.g. normal devices, PCI config space, memory mapped PCI I/O bars, normal/prefetchable memory bars. I think that we needs more *common* VM_MEMATTR_ classes. > We could also use smp_rendezvous to ensure nothing else is running, this will have a performance code, however I would not expect pmap_change_attr to be used in the fast path. > pmap_change_attr() must work very early, so smp_rendezvous() can be problematic. But, for me, the problematic part is cache - mainly transition from cached to uncached memory. I'm not sure, if I known always working sequence of mapping/cache operations witch ensure this transition in architecturally clean way. Next think is that pmap_change_attr() is used very rarely and probably only as workaround for limited bus_activate_resource() functionality. Michal