From owner-freebsd-mips@freebsd.org Sun Nov 13 06:59:02 2016 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 21DF4C3F4EB; Sun, 13 Nov 2016 06:59:02 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 927FB189E; Sun, 13 Nov 2016 06:59:01 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uAD6wpeD081383 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 13 Nov 2016 08:58:52 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uAD6wpeD081383 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uAD6wpti081382; Sun, 13 Nov 2016 08:58:51 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 13 Nov 2016 08:58:51 +0200 From: Konstantin Belousov To: Adrian Chadd Cc: "freebsd-mips@freebsd.org" , "src-committers@freebsd.org" , "svn-src-all@freebsd.org" , "svn-src-head@freebsd.org" Subject: Re: svn commit: r307626 - head/sys/ufs/ffs Message-ID: <20161113065851.GD54029@kib.kiev.ua> References: <201610191109.u9JB9TTC002727@repo.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.1 (2016-10-04) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Nov 2016 06:59:02 -0000 On Sat, Nov 12, 2016 at 03:19:13PM -0800, Adrian Chadd wrote: > hi! > > This broke freebsd on mips24k. > > BAD_PAGE_FAULT: pid 1 tid 100001 (init), uid 0: pc 0x4002a4 got a read > fault (type 0x2) at 0 > Trapframe Register Dump: > zero: 0 at: 0 v0: 0 v1: 0 > a0: 0x7fffeecc a1: 0 a2: 0 a3: 0 > t0: 0 t1: 0 t2: 0 t3: 0 > t4: 0 t5: 0 t6: 0 t7: 0 > t8: 0 t9: 0x400260 s0: 0x10 s1: 0x2 > s2: 0x7fffeed0 s3: 0 s4: 0 s5: 0 > s6: 0 s7: 0 k0: 0 k1: 0 > gp: 0x4d55d0 sp: 0x7ffeee90 s8: 0 ra: 0 > sr: 0xfc13 mullo: 0 mulhi: 0 badvaddr: 0 > cause: 0x8 pc: 0x4002a4 > Page table info for pc address 0x4002a4: pde = 0x809be000, pte = 0xa001acda > Dumping 4 words starting at pc address 0x4002a4: > 8c420000 14400003 00908021 8f828024 > Page table info for bad address 0: pde = 0, pte = 0 MIPS24k has split I/D caches, and both are VIPT, am I right ? I was not able to find the handling of cache aliasing in mips/pmap.c. Still, I am curious whether setting the loader tunable vfs.buf_pager_relbuf to 1 change anything. > > .. and yes, I've spent three days bisecting everything to get to this > particular commit. From owner-freebsd-mips@freebsd.org Sun Nov 13 07:10:44 2016 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A7864C3F6CC; Sun, 13 Nov 2016 07:10:44 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8123C1DE3; Sun, 13 Nov 2016 07:10:44 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-it0-x236.google.com with SMTP id c20so24451292itb.0; Sat, 12 Nov 2016 23:10:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=e8XXx5i3xTFbb8svbnL6u4XSlfEDmpzN4mQb6amV1o8=; b=Ix8N7OYV81f/jwe57I0PIwf/a1+WhS89U9E98IPwfEIeubT0pue77Uk+hM1md97rF7 Wzby8PjQIXLDQyW7P91cUV03CQULaCPs9cYcl6+zfazrUUgQsi2JbOLBjqO9nEnLERIn oa5oUCd+PH06vFqidJ7J+Wcr3nsT7bjjPT/GvrfOsVTpd7EaW1YJrxb0W++/DV56TX+Z 3Pj3Kqb+1IZIveClnziR3KRF5LOHI2RCd6v8ENeUysFq93xM7ri1y7DUQd/uDirVwUgH CiOPQI6GIAdF2Gjsz1xQGZ4wnHK/FlZYKttR7BJLjiaYc2FjGKOalGmHh/OX12rE9DGo kNyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=e8XXx5i3xTFbb8svbnL6u4XSlfEDmpzN4mQb6amV1o8=; b=eVurqIvyTGUIAuusQeS4xsx+PfDbYDM1MTMVtn8CYYtvh+iCGReGOrcjsTd2wu0Ng7 +h8naOEldrg+LaDZw1cKlQy3Ujo2dBPP1e+al/qBY2XENURP/gbZ3nwEv1B5HGkfI0f5 mp6DIo7jbrfXlNCWuw2YlszrdAbqCen7vRfelghXoJ8tP275c/FpiDuY68SF3we6YdsU DyK9QB0QTpXWGZ3Tg2t77IUtagnnhCfGLU3Psyuz/UOR+iOzRvUCiziEq93ynkDVq2rB e+lX2GQvjrs31eF9z/uDhxtd+LW6PXHxI6ALPjCll6zLV/8PySLhIoJHs00BXZemN7ML 569w== X-Gm-Message-State: ABUngve2D4p0MgQ09bTWxJhHgELLYGCOuZC43TUHnTfcueqTyOafpoVFN/gSvQvjVinqFu+L8Lfl+ToF5QzNPg== X-Received: by 10.36.65.216 with SMTP id b85mr2693568itd.39.1479021043864; Sat, 12 Nov 2016 23:10:43 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.39.134 with HTTP; Sat, 12 Nov 2016 23:10:43 -0800 (PST) In-Reply-To: <20161113065851.GD54029@kib.kiev.ua> References: <201610191109.u9JB9TTC002727@repo.freebsd.org> <20161113065851.GD54029@kib.kiev.ua> From: Adrian Chadd Date: Sat, 12 Nov 2016 23:10:43 -0800 Message-ID: Subject: Re: svn commit: r307626 - head/sys/ufs/ffs To: Konstantin Belousov Cc: "freebsd-mips@freebsd.org" , "src-committers@freebsd.org" , "svn-src-all@freebsd.org" , "svn-src-head@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Nov 2016 07:10:44 -0000 That disappeared from the file in a later commit? -a On 12 November 2016 at 22:58, Konstantin Belousov wrote: > On Sat, Nov 12, 2016 at 03:19:13PM -0800, Adrian Chadd wrote: >> hi! >> >> This broke freebsd on mips24k. >> >> BAD_PAGE_FAULT: pid 1 tid 100001 (init), uid 0: pc 0x4002a4 got a read >> fault (type 0x2) at 0 >> Trapframe Register Dump: >> zero: 0 at: 0 v0: 0 v1: 0 >> a0: 0x7fffeecc a1: 0 a2: 0 a3: 0 >> t0: 0 t1: 0 t2: 0 t3: 0 >> t4: 0 t5: 0 t6: 0 t7: 0 >> t8: 0 t9: 0x400260 s0: 0x10 s1: 0x2 >> s2: 0x7fffeed0 s3: 0 s4: 0 s5: 0 >> s6: 0 s7: 0 k0: 0 k1: 0 >> gp: 0x4d55d0 sp: 0x7ffeee90 s8: 0 ra: 0 >> sr: 0xfc13 mullo: 0 mulhi: 0 badvaddr: 0 >> cause: 0x8 pc: 0x4002a4 >> Page table info for pc address 0x4002a4: pde = 0x809be000, pte = 0xa001acda >> Dumping 4 words starting at pc address 0x4002a4: >> 8c420000 14400003 00908021 8f828024 >> Page table info for bad address 0: pde = 0, pte = 0 > MIPS24k has split I/D caches, and both are VIPT, am I right ? > I was not able to find the handling of cache aliasing in mips/pmap.c. > > Still, I am curious whether setting the loader tunable vfs.buf_pager_relbuf > to 1 change anything. > >> >> .. and yes, I've spent three days bisecting everything to get to this >> particular commit. From owner-freebsd-mips@freebsd.org Sun Nov 13 07:12:03 2016 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E742CC3F885 for ; Sun, 13 Nov 2016 07:12:03 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x243.google.com (mail-it0-x243.google.com [IPv6:2607:f8b0:4001:c0b::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A8DAB1D5 for ; Sun, 13 Nov 2016 07:12:03 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x243.google.com with SMTP id n68so5755859itn.3 for ; Sat, 12 Nov 2016 23:12:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=3tHGgvU3+NGA/go9h/VmsAmB54OHtiv2UaLnDWOlVwQ=; b=vxCDfH1/4PZbj0hl3jFyH3mXGOEgxFtjLJ2wi5HqQ61K/PDWMLzU/qjo0s5uvXnrlC A+XaKJYMWiap3dJChAuzFbSONaH1vHzgyozzb50yhH6AIIUnyb67T/erfzqT16Bm7zpB WbUlywcHBKmGIBFqtrONFiZ69IvMB8T1QMkXDet6LigD1ZfhZR/+/7DDvzI5ooSlwzZB hxmGULfQ4fWuk786EnXeqBEpD9aZVyA9Sx+nT0BUPC0Wdv/9Jn+587esEr/lJ59x4dqT R47Fg4+EqWdL28YX3liqdC0Lnxwhoxw//Yn9ONwPX2Zou3OV2tbBipiQ5S88JWApQ/h3 ewNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=3tHGgvU3+NGA/go9h/VmsAmB54OHtiv2UaLnDWOlVwQ=; b=Stz6QVZyvjPcvZC2jRkkt8EIk+IS5NF5dSBmdRS6AXmknvStJMiIPMznH6CMCRNA5H eM5G5uvmfLmsBMxrJhITzq7kC7d7XAhic0b65Le6syacqjrBUI7BFbJdHb9LqXQjxphU 2z/uWtQJPr3wdFfeR7pE5+EBBK/SCZuXHt97AORbNDBz2RbjRuMpUPeiTqX9ytuoNkqL zQWlX4kStiye1xiEpgeDarnE6vaa32Q2tNC+eF4AY7oUkZKbuLVmUpbsZ0aFKCE3MIyJ Zu/te4rht5SnsXMZ/r5hZ/T+tvGu8tKN6S6IUlzoXXRNCOssm1nPMUZXWHRRDAhCQV9f eNwQ== X-Gm-Message-State: ABUngvdypDMH97+UrrzF1bSbVJKOdniGH9LizmmST0Mmihl3vpX9731lWkWXUMgRM2OrrOpUvHuGcbq0NYzSfg== X-Received: by 10.36.93.193 with SMTP id w184mr2750930ita.85.1479021123080; Sat, 12 Nov 2016 23:12:03 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.134.66 with HTTP; Sat, 12 Nov 2016 23:12:02 -0800 (PST) X-Originating-IP: [50.253.99.174] In-Reply-To: <20161113065851.GD54029@kib.kiev.ua> References: <201610191109.u9JB9TTC002727@repo.freebsd.org> <20161113065851.GD54029@kib.kiev.ua> From: Warner Losh Date: Sun, 13 Nov 2016 00:12:02 -0700 X-Google-Sender-Auth: XlygBA0tK0iZSK9dNQyfFSHmLvs Message-ID: Subject: Re: svn commit: r307626 - head/sys/ufs/ffs To: Konstantin Belousov Cc: Adrian Chadd , "svn-src-head@freebsd.org" , "svn-src-all@freebsd.org" , "src-committers@freebsd.org" , "freebsd-mips@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Nov 2016 07:12:04 -0000 On Sat, Nov 12, 2016 at 11:58 PM, Konstantin Belousov wrote: > On Sat, Nov 12, 2016 at 03:19:13PM -0800, Adrian Chadd wrote: >> hi! >> >> This broke freebsd on mips24k. >> >> BAD_PAGE_FAULT: pid 1 tid 100001 (init), uid 0: pc 0x4002a4 got a read >> fault (type 0x2) at 0 >> Trapframe Register Dump: >> zero: 0 at: 0 v0: 0 v1: 0 >> a0: 0x7fffeecc a1: 0 a2: 0 a3: 0 >> t0: 0 t1: 0 t2: 0 t3: 0 >> t4: 0 t5: 0 t6: 0 t7: 0 >> t8: 0 t9: 0x400260 s0: 0x10 s1: 0x2 >> s2: 0x7fffeed0 s3: 0 s4: 0 s5: 0 >> s6: 0 s7: 0 k0: 0 k1: 0 >> gp: 0x4d55d0 sp: 0x7ffeee90 s8: 0 ra: 0 >> sr: 0xfc13 mullo: 0 mulhi: 0 badvaddr: 0 >> cause: 0x8 pc: 0x4002a4 >> Page table info for pc address 0x4002a4: pde = 0x809be000, pte = 0xa001acda >> Dumping 4 words starting at pc address 0x4002a4: >> 8c420000 14400003 00908021 8f828024 >> Page table info for bad address 0: pde = 0, pte = 0 > MIPS24k has split I/D caches, and both are VIPT, am I right ? > I was not able to find the handling of cache aliasing in mips/pmap.c. > > Still, I am curious whether setting the loader tunable vfs.buf_pager_relbuf > to 1 change anything. MIPS caches are such that creating two virtual mappings to the same physical page will cause corruption. It's simply not allowed, at least for the class of MIPS machines I used to bring up the port originally. Warner From owner-freebsd-mips@freebsd.org Sun Nov 13 07:15:53 2016 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8DA11C3F952; Sun, 13 Nov 2016 07:15:53 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-it0-x231.google.com (mail-it0-x231.google.com [IPv6:2607:f8b0:4001:c0b::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 54D4B679; Sun, 13 Nov 2016 07:15:53 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-it0-x231.google.com with SMTP id q124so46113329itd.1; Sat, 12 Nov 2016 23:15:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=IbaRXHYXAmQnbUBUMwTTb59dAWjnkMpP50MXxqSJt0Y=; b=b6KCgI8IRqwcjTspWHFTLXkaHtk4mpBjE6PdEmV8WSxSUY2WRf3MkK/BNcXyNU732Y qYrgFdCEf42pI7dfL4TsR5iP5TcVMFdNgyLTofy1iR1BwFwp53vyovrFGYMJN6Ne2h/K 7g/aN8NUuIKkyt+XbgdtyWWro+HZ59ClLrkGzuzgMElDh1ueRcboeWUjQIXnJpM3O9ma IQP+CzbcWRInj+CvaPo85VDKUPkFIxLlGGMPQRqaerHamurZgoy2CsRxr9WWX0/NcYQ0 8g5r1xXDQRar3tjqIfIkmUtr6bI2LvPgDSGvX/fmRBINe29tRIMnn21M2TtX3bF9cfqa If+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=IbaRXHYXAmQnbUBUMwTTb59dAWjnkMpP50MXxqSJt0Y=; b=dLudv8SM1ZDyWVs6ptEDGlEj+R3zW0IjTmf6nRCJmyTG+TnuLSQtMTF0WKgHgnT3hx 2WQHZccANbf6b8vxLBuCuzs3waYAeVFRXEWCKhyF0hls/oB+DLriAncwIOQwjeEx8Dxh hf8xBzAvO9Z8bDAHYrIDaw4rtkIf4CT7HtcirwzwYQTTMnIE2B712KzbpelojtmCzbJO GN5crDGm/YFCYcpgKlMoW9ocBrM71iNv9j5EMXbiEgZIriG7ni6/g7tVBcex/bdVwfs5 JzbjBlouncbyLuQtdD5ip846wDa0KrG6F74tvzkx+r7OjVduIUYtm51I+oczEy7fsghY qSqA== X-Gm-Message-State: ABUngvd68FZkqyqFr+a0tZl9KHig0SiROoOYxwaGgSiJcmKSsFfPs8RHWNW8GJUHMDRybXxXPmyOC8bsRtSkjw== X-Received: by 10.36.138.67 with SMTP id v64mr2646348itd.39.1479021352828; Sat, 12 Nov 2016 23:15:52 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.39.134 with HTTP; Sat, 12 Nov 2016 23:15:52 -0800 (PST) In-Reply-To: References: <201610191109.u9JB9TTC002727@repo.freebsd.org> <20161113065851.GD54029@kib.kiev.ua> From: Adrian Chadd Date: Sat, 12 Nov 2016 23:15:52 -0800 Message-ID: Subject: Re: svn commit: r307626 - head/sys/ufs/ffs To: Warner Losh Cc: Konstantin Belousov , "svn-src-head@freebsd.org" , "svn-src-all@freebsd.org" , "src-committers@freebsd.org" , "freebsd-mips@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Nov 2016 07:15:53 -0000 On 12 November 2016 at 23:12, Warner Losh wrote: > On Sat, Nov 12, 2016 at 11:58 PM, Konstantin Belousov > wrote: >> On Sat, Nov 12, 2016 at 03:19:13PM -0800, Adrian Chadd wrote: >>> hi! >>> >>> This broke freebsd on mips24k. >>> >>> BAD_PAGE_FAULT: pid 1 tid 100001 (init), uid 0: pc 0x4002a4 got a read >>> fault (type 0x2) at 0 >>> Trapframe Register Dump: >>> zero: 0 at: 0 v0: 0 v1: 0 >>> a0: 0x7fffeecc a1: 0 a2: 0 a3: 0 >>> t0: 0 t1: 0 t2: 0 t3: 0 >>> t4: 0 t5: 0 t6: 0 t7: 0 >>> t8: 0 t9: 0x400260 s0: 0x10 s1: 0x2 >>> s2: 0x7fffeed0 s3: 0 s4: 0 s5: 0 >>> s6: 0 s7: 0 k0: 0 k1: 0 >>> gp: 0x4d55d0 sp: 0x7ffeee90 s8: 0 ra: 0 >>> sr: 0xfc13 mullo: 0 mulhi: 0 badvaddr: 0 >>> cause: 0x8 pc: 0x4002a4 >>> Page table info for pc address 0x4002a4: pde = 0x809be000, pte = 0xa001acda >>> Dumping 4 words starting at pc address 0x4002a4: >>> 8c420000 14400003 00908021 8f828024 >>> Page table info for bad address 0: pde = 0, pte = 0 >> MIPS24k has split I/D caches, and both are VIPT, am I right ? >> I was not able to find the handling of cache aliasing in mips/pmap.c. >> >> Still, I am curious whether setting the loader tunable vfs.buf_pager_relbuf >> to 1 change anything. > > MIPS caches are such that creating two virtual mappings to the same > physical page will cause corruption. It's simply not allowed, at least > for the class of MIPS machines I used to bring up the port originally. Hi, Right - the bulk of MIPS hardware (including 24k/74k) are VIPT, and we're not allowed to do cache aliasing in that manner. -adrian From owner-freebsd-mips@freebsd.org Sun Nov 13 07:16:10 2016 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7A2FEC3F9A3; Sun, 13 Nov 2016 07:16:10 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E9D8985D; Sun, 13 Nov 2016 07:16:09 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uAD7G58o085733 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 13 Nov 2016 09:16:05 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uAD7G58o085733 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uAD7G5jN085731; Sun, 13 Nov 2016 09:16:05 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 13 Nov 2016 09:16:05 +0200 From: Konstantin Belousov To: Adrian Chadd Cc: "freebsd-mips@freebsd.org" , "src-committers@freebsd.org" , "svn-src-all@freebsd.org" , "svn-src-head@freebsd.org" Subject: Re: svn commit: r307626 - head/sys/ufs/ffs Message-ID: <20161113071605.GE54029@kib.kiev.ua> References: <201610191109.u9JB9TTC002727@repo.freebsd.org> <20161113065851.GD54029@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.1 (2016-10-04) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Nov 2016 07:16:10 -0000 On Sat, Nov 12, 2016 at 11:10:43PM -0800, Adrian Chadd wrote: > That disappeared from the file in a later commit? >From which commit did it disappeared ? The tunable is present since sys/kern/vfs_bio.c r308026. I did not removed it, and did not see any commit removing the code by somebody else. > > > > -a > > > On 12 November 2016 at 22:58, Konstantin Belousov wrote: > > On Sat, Nov 12, 2016 at 03:19:13PM -0800, Adrian Chadd wrote: > >> hi! > >> > >> This broke freebsd on mips24k. > >> > >> BAD_PAGE_FAULT: pid 1 tid 100001 (init), uid 0: pc 0x4002a4 got a read > >> fault (type 0x2) at 0 > >> Trapframe Register Dump: > >> zero: 0 at: 0 v0: 0 v1: 0 > >> a0: 0x7fffeecc a1: 0 a2: 0 a3: 0 > >> t0: 0 t1: 0 t2: 0 t3: 0 > >> t4: 0 t5: 0 t6: 0 t7: 0 > >> t8: 0 t9: 0x400260 s0: 0x10 s1: 0x2 > >> s2: 0x7fffeed0 s3: 0 s4: 0 s5: 0 > >> s6: 0 s7: 0 k0: 0 k1: 0 > >> gp: 0x4d55d0 sp: 0x7ffeee90 s8: 0 ra: 0 > >> sr: 0xfc13 mullo: 0 mulhi: 0 badvaddr: 0 > >> cause: 0x8 pc: 0x4002a4 > >> Page table info for pc address 0x4002a4: pde = 0x809be000, pte = 0xa001acda > >> Dumping 4 words starting at pc address 0x4002a4: > >> 8c420000 14400003 00908021 8f828024 > >> Page table info for bad address 0: pde = 0, pte = 0 > > MIPS24k has split I/D caches, and both are VIPT, am I right ? > > I was not able to find the handling of cache aliasing in mips/pmap.c. > > > > Still, I am curious whether setting the loader tunable vfs.buf_pager_relbuf > > to 1 change anything. > > > >> > >> .. and yes, I've spent three days bisecting everything to get to this > >> particular commit. From owner-freebsd-mips@freebsd.org Sun Nov 13 07:16:55 2016 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 08D08C3FA41; Sun, 13 Nov 2016 07:16:55 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-it0-x22d.google.com (mail-it0-x22d.google.com [IPv6:2607:f8b0:4001:c0b::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C58BCA2A; Sun, 13 Nov 2016 07:16:54 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-it0-x22d.google.com with SMTP id q124so46129614itd.1; Sat, 12 Nov 2016 23:16:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=TPq8IXaoDJM+iHbykObJzBAnYWwTj7qvjk1erWUBpzk=; b=FJApDe3GPV3fjZ2Kb01ILf0N9BP8ljv7WijHVNrQJrYX3ftn7fOh+t/CSetNrnFawf TrifCyn2BRdRhPgplP0uHWK2T0FBDvGCqRKfpJbRbwdOomgXMvdpmWzJNsqaLxMwJW4A Xm7Cb8IlwB7bvOpXF+pxU+PKZLcqzibAaBAvUkfb4Maw77wNNa76kbKTtp5z4P6KvaDJ XDrOpdT8/2JM7blAQXBG60u9d9imLFxfBgynj5DyJ/KUELA2z1Xj6VLdNNeSsjkV35Kn qAfbDlyrOe5A1vcHE0nOKhIZhh51JtAAiAOa9beAYldmevCvhJ5nSys5jmCg3fakf+2q fNHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=TPq8IXaoDJM+iHbykObJzBAnYWwTj7qvjk1erWUBpzk=; b=gRTRBvKd6wTgnN6/isthTBBMtf4+czRSFZTMF0JZ2XvYREZeliCOa20YOO72lOEnyO 5WAitVAejAkhM00NQbYLG0dQDG8o5Q9nJBThnpbHQ4b+TTWkGjPONUfQ3Bz78FcnL5yM BrbqIjJkubl9XPti85juUfYgAnZDav1Wo2ke6ItwnP5MA999n31eHp8SZX3mwhIlhcg4 /1xyiTxVDErPvetgC3dDSFnmx93kI6qHoL7QjslBwquCofOiKGpiW9X29GTmQYi18hwK ByWVdX0s0R/O9Okfz8QuGSl/ytxNc2h6IHAwXim+Ye3ds/yHNaSg0PWWp46LotrwqkLc UkYg== X-Gm-Message-State: ABUngveiFxCtPS7rpgoJALYN3gBF4zdVVOy+fLaWJRi3tVb8Uz7RjmAnNvKUqfs5DSHZnLQEWQ2pYZ0cMdwrzQ== X-Received: by 10.107.136.86 with SMTP id k83mr18258314iod.99.1479021414266; Sat, 12 Nov 2016 23:16:54 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.39.134 with HTTP; Sat, 12 Nov 2016 23:16:53 -0800 (PST) In-Reply-To: <20161113071605.GE54029@kib.kiev.ua> References: <201610191109.u9JB9TTC002727@repo.freebsd.org> <20161113065851.GD54029@kib.kiev.ua> <20161113071605.GE54029@kib.kiev.ua> From: Adrian Chadd Date: Sat, 12 Nov 2016 23:16:53 -0800 Message-ID: Subject: Re: svn commit: r307626 - head/sys/ufs/ffs To: Konstantin Belousov Cc: "freebsd-mips@freebsd.org" , "src-committers@freebsd.org" , "svn-src-all@freebsd.org" , "svn-src-head@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Nov 2016 07:16:55 -0000 On 12 November 2016 at 23:16, Konstantin Belousov wrote: > On Sat, Nov 12, 2016 at 11:10:43PM -0800, Adrian Chadd wrote: >> That disappeared from the file in a later commit? > From which commit did it disappeared ? > > The tunable is present since sys/kern/vfs_bio.c r308026. I did not removed > it, and did not see any commit removing the code by somebody else. I'll go look tomorrow. I could've sworn I only saw one sysctl in there in -head when i did my last bisect (up to head.) If it's there I'll go try it out. -adrian > >> >> >> >> -a >> >> >> On 12 November 2016 at 22:58, Konstantin Belousov wrote: >> > On Sat, Nov 12, 2016 at 03:19:13PM -0800, Adrian Chadd wrote: >> >> hi! >> >> >> >> This broke freebsd on mips24k. >> >> >> >> BAD_PAGE_FAULT: pid 1 tid 100001 (init), uid 0: pc 0x4002a4 got a read >> >> fault (type 0x2) at 0 >> >> Trapframe Register Dump: >> >> zero: 0 at: 0 v0: 0 v1: 0 >> >> a0: 0x7fffeecc a1: 0 a2: 0 a3: 0 >> >> t0: 0 t1: 0 t2: 0 t3: 0 >> >> t4: 0 t5: 0 t6: 0 t7: 0 >> >> t8: 0 t9: 0x400260 s0: 0x10 s1: 0x2 >> >> s2: 0x7fffeed0 s3: 0 s4: 0 s5: 0 >> >> s6: 0 s7: 0 k0: 0 k1: 0 >> >> gp: 0x4d55d0 sp: 0x7ffeee90 s8: 0 ra: 0 >> >> sr: 0xfc13 mullo: 0 mulhi: 0 badvaddr: 0 >> >> cause: 0x8 pc: 0x4002a4 >> >> Page table info for pc address 0x4002a4: pde = 0x809be000, pte = 0xa001acda >> >> Dumping 4 words starting at pc address 0x4002a4: >> >> 8c420000 14400003 00908021 8f828024 >> >> Page table info for bad address 0: pde = 0, pte = 0 >> > MIPS24k has split I/D caches, and both are VIPT, am I right ? >> > I was not able to find the handling of cache aliasing in mips/pmap.c. >> > >> > Still, I am curious whether setting the loader tunable vfs.buf_pager_relbuf >> > to 1 change anything. >> > >> >> >> >> .. and yes, I've spent three days bisecting everything to get to this >> >> particular commit. From owner-freebsd-mips@freebsd.org Sun Nov 13 07:19:17 2016 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B1C49C3FB15; Sun, 13 Nov 2016 07:19:17 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5918BC0F; Sun, 13 Nov 2016 07:19:17 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uAD7JB1Z086203 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 13 Nov 2016 09:19:12 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uAD7JB1Z086203 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uAD7JBtl086202; Sun, 13 Nov 2016 09:19:11 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 13 Nov 2016 09:19:11 +0200 From: Konstantin Belousov To: Warner Losh Cc: Adrian Chadd , "svn-src-head@freebsd.org" , "svn-src-all@freebsd.org" , "src-committers@freebsd.org" , "freebsd-mips@freebsd.org" Subject: Re: svn commit: r307626 - head/sys/ufs/ffs Message-ID: <20161113071911.GF54029@kib.kiev.ua> References: <201610191109.u9JB9TTC002727@repo.freebsd.org> <20161113065851.GD54029@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.1 (2016-10-04) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Nov 2016 07:19:17 -0000 On Sun, Nov 13, 2016 at 12:12:02AM -0700, Warner Losh wrote: > On Sat, Nov 12, 2016 at 11:58 PM, Konstantin Belousov > wrote: > > On Sat, Nov 12, 2016 at 03:19:13PM -0800, Adrian Chadd wrote: > >> hi! > >> > >> This broke freebsd on mips24k. > >> > >> BAD_PAGE_FAULT: pid 1 tid 100001 (init), uid 0: pc 0x4002a4 got a read > >> fault (type 0x2) at 0 > >> Trapframe Register Dump: > >> zero: 0 at: 0 v0: 0 v1: 0 > >> a0: 0x7fffeecc a1: 0 a2: 0 a3: 0 > >> t0: 0 t1: 0 t2: 0 t3: 0 > >> t4: 0 t5: 0 t6: 0 t7: 0 > >> t8: 0 t9: 0x400260 s0: 0x10 s1: 0x2 > >> s2: 0x7fffeed0 s3: 0 s4: 0 s5: 0 > >> s6: 0 s7: 0 k0: 0 k1: 0 > >> gp: 0x4d55d0 sp: 0x7ffeee90 s8: 0 ra: 0 > >> sr: 0xfc13 mullo: 0 mulhi: 0 badvaddr: 0 > >> cause: 0x8 pc: 0x4002a4 > >> Page table info for pc address 0x4002a4: pde = 0x809be000, pte = 0xa001acda > >> Dumping 4 words starting at pc address 0x4002a4: > >> 8c420000 14400003 00908021 8f828024 > >> Page table info for bad address 0: pde = 0, pte = 0 > > MIPS24k has split I/D caches, and both are VIPT, am I right ? > > I was not able to find the handling of cache aliasing in mips/pmap.c. > > > > Still, I am curious whether setting the loader tunable vfs.buf_pager_relbuf > > to 1 change anything. > > MIPS caches are such that creating two virtual mappings to the same > physical page will cause corruption. It's simply not allowed, at least > for the class of MIPS machines I used to bring up the port originally. Yes, caches are VIPT on 24k, according to the "MIPS32(R) 24K(R) Processor Core Family Software User's Manual " rev 3.11. My question is, how is that handled in the mips pmap.c. I was not able to locate the alias detection and prevention code, or e.g. switching to uncached mode for the page when aliasing is detected, after browsing pmap. Does FreeBSD/mips run with the caches enabled ? From owner-freebsd-mips@freebsd.org Sun Nov 13 07:33:58 2016 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D7817C3FEA5 for ; Sun, 13 Nov 2016 07:33:58 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 97BAC1373 for ; Sun, 13 Nov 2016 07:33:58 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x236.google.com with SMTP id c20so24821500itb.0 for ; Sat, 12 Nov 2016 23:33:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=P60tCRo7+0t2WYjzM+DMkCnS6F78O/xVgBJ3ROX7boI=; b=KM+V34Z44+qH20/uOTUSmzeW5P0x6N1nb2WsyNG8e5jTbqLvNb+pTjOcRqONDbh743 MayOPYAtxZjoSCF5Eji+Yz5WzVcTflOfQjrYKaPTjJm/dL3eoFM49jqh5N6QN1OPygzM fGX3cClaSilEPqZ2rMY0THi2Bxy6kRKYO12Bsxz4kbH2LWWjLLRWTq0Z1BkegMbHHhuB hsmfeUjfDzOgMqN9YPkU5NyulFNQnNSYGqW5jrs786BxCozu6niVmDlrFSNoh8LrnP8C 6VW9+seLUB1gFT4bP6odYpARejKexv+Jd4mjo9lOndLC8V5ZmUM9MqTndVI/P/rA1Py0 c64g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=P60tCRo7+0t2WYjzM+DMkCnS6F78O/xVgBJ3ROX7boI=; b=cxNP4A1nNZYWIgo2fUV7C0IzBJn83Vi31yysN7O1mxR/pQeGO7mxkqrz1FildT310H /UbqiAm9rKu0VMDk1VCv3h2P2rJ9ARsqwaQfSp63lEfWSaTLCCeeUCBInnxX16GmD5OA Mwch6bmyMNTwDLew6rjgu2JHPugAPQDQheAXV6Bf5cP5UqzQ1rpSrBAS0xN8OATvaDMo ej0S2kq+phzbm1BJfIAyQEoiS8c4V2JUOG0VQDA4LgqrKu1T/lKPceU+yYu5aMceuThR U3QGBsnR3iyGIJXSvvIyjmCd8+bpY/tiKBB1FuaJFQmYpv7iRLJTDz/5IpiEAITsVEY6 pk9A== X-Gm-Message-State: ABUngvfz3Ow3tm11poZfESNV5O3bGNq3Srat5VEKGcFdZQyE49MIesEXqwozVLcjX34Qsd7yu2/q5vJnqTAKoQ== X-Received: by 10.107.132.74 with SMTP id g71mr18900252iod.19.1479022438017; Sat, 12 Nov 2016 23:33:58 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.134.66 with HTTP; Sat, 12 Nov 2016 23:33:57 -0800 (PST) X-Originating-IP: [50.253.99.174] In-Reply-To: <20161113071911.GF54029@kib.kiev.ua> References: <201610191109.u9JB9TTC002727@repo.freebsd.org> <20161113065851.GD54029@kib.kiev.ua> <20161113071911.GF54029@kib.kiev.ua> From: Warner Losh Date: Sun, 13 Nov 2016 00:33:57 -0700 X-Google-Sender-Auth: QbSIAsv0K34Qsd0xXm2Kd4Wj640 Message-ID: Subject: Re: svn commit: r307626 - head/sys/ufs/ffs To: Konstantin Belousov Cc: Adrian Chadd , "svn-src-head@freebsd.org" , "svn-src-all@freebsd.org" , "src-committers@freebsd.org" , "freebsd-mips@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Nov 2016 07:33:58 -0000 On Sun, Nov 13, 2016 at 12:19 AM, Konstantin Belousov wrote: > On Sun, Nov 13, 2016 at 12:12:02AM -0700, Warner Losh wrote: >> On Sat, Nov 12, 2016 at 11:58 PM, Konstantin Belousov >> wrote: >> > On Sat, Nov 12, 2016 at 03:19:13PM -0800, Adrian Chadd wrote: >> >> hi! >> >> >> >> This broke freebsd on mips24k. >> >> >> >> BAD_PAGE_FAULT: pid 1 tid 100001 (init), uid 0: pc 0x4002a4 got a read >> >> fault (type 0x2) at 0 >> >> Trapframe Register Dump: >> >> zero: 0 at: 0 v0: 0 v1: 0 >> >> a0: 0x7fffeecc a1: 0 a2: 0 a3: 0 >> >> t0: 0 t1: 0 t2: 0 t3: 0 >> >> t4: 0 t5: 0 t6: 0 t7: 0 >> >> t8: 0 t9: 0x400260 s0: 0x10 s1: 0x2 >> >> s2: 0x7fffeed0 s3: 0 s4: 0 s5: 0 >> >> s6: 0 s7: 0 k0: 0 k1: 0 >> >> gp: 0x4d55d0 sp: 0x7ffeee90 s8: 0 ra: 0 >> >> sr: 0xfc13 mullo: 0 mulhi: 0 badvaddr: 0 >> >> cause: 0x8 pc: 0x4002a4 >> >> Page table info for pc address 0x4002a4: pde = 0x809be000, pte = 0xa001acda >> >> Dumping 4 words starting at pc address 0x4002a4: >> >> 8c420000 14400003 00908021 8f828024 >> >> Page table info for bad address 0: pde = 0, pte = 0 >> > MIPS24k has split I/D caches, and both are VIPT, am I right ? >> > I was not able to find the handling of cache aliasing in mips/pmap.c. >> > >> > Still, I am curious whether setting the loader tunable vfs.buf_pager_relbuf >> > to 1 change anything. >> >> MIPS caches are such that creating two virtual mappings to the same >> physical page will cause corruption. It's simply not allowed, at least >> for the class of MIPS machines I used to bring up the port originally. > > Yes, caches are VIPT on 24k, according to the "MIPS32(R) 24K(R) > Processor Core Family Software User's Manual " rev 3.11. My question is, > how is that handled in the mips pmap.c. I was not able to locate > the alias detection and prevention code, or e.g. switching to uncached mode > for the page when aliasing is detected, after browsing pmap. Aliases are not permitted. IIRC, there's no code that detects this condition. One must simply never ever have multiple cached mappings of a page at once. A quick glance at the code doesn't locate anything. > Does FreeBSD/mips run with the caches enabled ? Yes. The implementation is easier w/o caches enabled, but the performance sucks too bad to even contemplate it except for debugging. Warner From owner-freebsd-mips@freebsd.org Sun Nov 13 07:56:03 2016 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BEA27C3F344; Sun, 13 Nov 2016 07:56:03 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BBE11D4E; Sun, 13 Nov 2016 07:56:03 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uAD7tvoN095317 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 13 Nov 2016 09:55:57 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uAD7tvoN095317 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uAD7tvgg095316; Sun, 13 Nov 2016 09:55:57 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 13 Nov 2016 09:55:57 +0200 From: Konstantin Belousov To: Warner Losh Cc: Adrian Chadd , "svn-src-head@freebsd.org" , "svn-src-all@freebsd.org" , "src-committers@freebsd.org" , "freebsd-mips@freebsd.org" Subject: Re: svn commit: r307626 - head/sys/ufs/ffs Message-ID: <20161113075557.GH54029@kib.kiev.ua> References: <201610191109.u9JB9TTC002727@repo.freebsd.org> <20161113065851.GD54029@kib.kiev.ua> <20161113071911.GF54029@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.1 (2016-10-04) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Nov 2016 07:56:03 -0000 On Sun, Nov 13, 2016 at 12:33:57AM -0700, Warner Losh wrote: > On Sun, Nov 13, 2016 at 12:19 AM, Konstantin Belousov > wrote: > > On Sun, Nov 13, 2016 at 12:12:02AM -0700, Warner Losh wrote: > >> On Sat, Nov 12, 2016 at 11:58 PM, Konstantin Belousov > >> wrote: > >> > On Sat, Nov 12, 2016 at 03:19:13PM -0800, Adrian Chadd wrote: > >> >> hi! > >> >> > >> >> This broke freebsd on mips24k. > >> >> > >> >> BAD_PAGE_FAULT: pid 1 tid 100001 (init), uid 0: pc 0x4002a4 got a read > >> >> fault (type 0x2) at 0 > >> >> Trapframe Register Dump: > >> >> zero: 0 at: 0 v0: 0 v1: 0 > >> >> a0: 0x7fffeecc a1: 0 a2: 0 a3: 0 > >> >> t0: 0 t1: 0 t2: 0 t3: 0 > >> >> t4: 0 t5: 0 t6: 0 t7: 0 > >> >> t8: 0 t9: 0x400260 s0: 0x10 s1: 0x2 > >> >> s2: 0x7fffeed0 s3: 0 s4: 0 s5: 0 > >> >> s6: 0 s7: 0 k0: 0 k1: 0 > >> >> gp: 0x4d55d0 sp: 0x7ffeee90 s8: 0 ra: 0 > >> >> sr: 0xfc13 mullo: 0 mulhi: 0 badvaddr: 0 > >> >> cause: 0x8 pc: 0x4002a4 > >> >> Page table info for pc address 0x4002a4: pde = 0x809be000, pte = 0xa001acda > >> >> Dumping 4 words starting at pc address 0x4002a4: > >> >> 8c420000 14400003 00908021 8f828024 > >> >> Page table info for bad address 0: pde = 0, pte = 0 > >> > MIPS24k has split I/D caches, and both are VIPT, am I right ? > >> > I was not able to find the handling of cache aliasing in mips/pmap.c. > >> > > >> > Still, I am curious whether setting the loader tunable vfs.buf_pager_relbuf > >> > to 1 change anything. > >> > >> MIPS caches are such that creating two virtual mappings to the same > >> physical page will cause corruption. It's simply not allowed, at least > >> for the class of MIPS machines I used to bring up the port originally. > > > > Yes, caches are VIPT on 24k, according to the "MIPS32(R) 24K(R) > > Processor Core Family Software User's Manual " rev 3.11. My question is, > > how is that handled in the mips pmap.c. I was not able to locate > > the alias detection and prevention code, or e.g. switching to uncached mode > > for the page when aliasing is detected, after browsing pmap. > > Aliases are not permitted. IIRC, there's no code that detects this > condition. One must simply never ever have multiple cached mappings of > a page at once. A quick glance at the code doesn't locate anything. Then, the obvious next question is what does prevent such aliased mappings ? Not only usermode might establish such situation by double-mapping, but also e.g. our coherent buffer/page cache maps page into KVA and the same page might be mapped into usermode. The later situation is my current thought about possible cause of the reported init(8) fault. > > > Does FreeBSD/mips run with the caches enabled ? > > Yes. The implementation is easier w/o caches enabled, but the > performance sucks too bad to even contemplate it except for debugging. > > Warner From owner-freebsd-mips@freebsd.org Sun Nov 13 08:13:27 2016 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F2A5CC3F963 for ; Sun, 13 Nov 2016 08:13:27 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x243.google.com (mail-it0-x243.google.com [IPv6:2607:f8b0:4001:c0b::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B365E96E for ; Sun, 13 Nov 2016 08:13:27 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x243.google.com with SMTP id q124so5934062itd.1 for ; Sun, 13 Nov 2016 00:13:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=subject:mime-version:from:in-reply-to:date:cc:message-id:references :to; bh=nJ1644C5k7EElCV7NTTLe5hesaNQsySCzegHlm5JHNU=; b=qq0HBl+TcS2qxial7oESELaQh4o4vXi3ChEAo+jKpRjqqFzrO8hhWBrIjld6zo+wNR 09Y0cpp/igauMIKTRSqTTBY8P7bVzfeeLV0tSU8q16dak17F07w0HhqpdEbighR0oNq6 ITdUmwaVNRlwNIFkwUOQcxXbvLSiUhBNMc9awV2UmC7M78O7AJC4OFnk176n6QEKZSun vMjZpNPZnqfjyaZTP4rDKndvb8ly7KFwQRuhD0xWSOo/hQKsPjpbdl+t9fC33WZ+n0IE Jk33Zt0tn0Idk98xYABFwirpuF4ZIBpxQOeSAGm75ppqmlGiV/Uwtuvx5ZaH5sHhYyjb AsuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:mime-version:from:in-reply-to:date:cc :message-id:references:to; bh=nJ1644C5k7EElCV7NTTLe5hesaNQsySCzegHlm5JHNU=; b=SMCETORFpBs+eD/A//kdNBXxLFw9rjhH5fLzKAIio926PcoX/e/vqbCbdt+DWSD3no A5IogO72ngq6RpJpt+DHEUCDu3BLeXSuM0mqIhY6i5Fj6N0gKXJkQSrgDWj/umTIz7z9 q4qvhDf3U/oh6H5vMt8Zohqyt9hYZtufYcBpnu2PiZ6i7hxRFIq9WfuQ34ZCpeaxBHp3 tLTKyxp4cwm+EtpCSZOcwbiUyVDZ+DPg0sB3idrSU2SER7Ig0swO75xcLV4MsLrsoa2W vd15YZXf/W09v8YNU3UNT/kddr2L8cKbkBiec53JOASk//tB6CUQG3ZkTyolhVLpPPYZ GwQQ== X-Gm-Message-State: ABUngvc96W4DGMidwPbXaoADVpoV90shwL53fKf5QLrII6CBiTKJ9TuNN4h+iwl/ifEsmg== X-Received: by 10.36.69.134 with SMTP id c6mr2867188itd.102.1479024807127; Sun, 13 Nov 2016 00:13:27 -0800 (PST) Received: from mac-wired.nflx.bsdimp.com (50-253-99-174-static.hfc.comcastbusiness.net. [50.253.99.174]) by smtp.gmail.com with ESMTPSA id q66sm1900247iof.31.2016.11.13.00.13.26 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 13 Nov 2016 00:13:26 -0800 (PST) Subject: Re: svn commit: r307626 - head/sys/ufs/ffs Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: multipart/signed; boundary="Apple-Mail=_070B6E65-8DF8-45E7-A09D-18C3A65DACF0"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail From: Warner Losh In-Reply-To: <20161113075557.GH54029@kib.kiev.ua> Date: Sun, 13 Nov 2016 01:13:24 -0700 Cc: Warner Losh , Adrian Chadd , "freebsd-mips@freebsd.org" , Juli Mallett Message-Id: <71C512CD-0FB6-40D8-B46C-30467A245693@bsdimp.com> References: <201610191109.u9JB9TTC002727@repo.freebsd.org> <20161113065851.GD54029@kib.kiev.ua> <20161113071911.GF54029@kib.kiev.ua> <20161113075557.GH54029@kib.kiev.ua> To: Konstantin Belousov X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Nov 2016 08:13:28 -0000 --Apple-Mail=_070B6E65-8DF8-45E7-A09D-18C3A65DACF0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 [[ Moved to freebsd-mips and cc=E2=80=99d to Juli ]] > On Nov 13, 2016, at 12:55 AM, Konstantin Belousov = wrote: >=20 > On Sun, Nov 13, 2016 at 12:33:57AM -0700, Warner Losh wrote: >> On Sun, Nov 13, 2016 at 12:19 AM, Konstantin Belousov >> wrote: >>> On Sun, Nov 13, 2016 at 12:12:02AM -0700, Warner Losh wrote: >>>> On Sat, Nov 12, 2016 at 11:58 PM, Konstantin Belousov >>>> wrote: >>>>> On Sat, Nov 12, 2016 at 03:19:13PM -0800, Adrian Chadd wrote: >>>>>> hi! >>>>>>=20 >>>>>> This broke freebsd on mips24k. >>>>>>=20 >>>>>> BAD_PAGE_FAULT: pid 1 tid 100001 (init), uid 0: pc 0x4002a4 got a = read >>>>>> fault (type 0x2) at 0 >>>>>> Trapframe Register Dump: >>>>>> zero: 0 at: 0 v0: 0 v1: 0 >>>>>> a0: 0x7fffeecc a1: 0 a2: 0 a3: 0 >>>>>> t0: 0 t1: 0 t2: 0 t3: 0 >>>>>> t4: 0 t5: 0 t6: 0 t7: 0 >>>>>> t8: 0 t9: 0x400260 s0: 0x10 s1: 0x2 >>>>>> s2: 0x7fffeed0 s3: 0 s4: 0 s5: 0 >>>>>> s6: 0 s7: 0 k0: 0 k1: 0 >>>>>> gp: 0x4d55d0 sp: 0x7ffeee90 s8: 0 ra: 0 >>>>>> sr: 0xfc13 mullo: 0 mulhi: 0 badvaddr: 0 >>>>>> cause: 0x8 pc: 0x4002a4 >>>>>> Page table info for pc address 0x4002a4: pde =3D 0x809be000, pte = =3D 0xa001acda >>>>>> Dumping 4 words starting at pc address 0x4002a4: >>>>>> 8c420000 14400003 00908021 8f828024 >>>>>> Page table info for bad address 0: pde =3D 0, pte =3D 0 >>>>> MIPS24k has split I/D caches, and both are VIPT, am I right ? >>>>> I was not able to find the handling of cache aliasing in = mips/pmap.c. >>>>>=20 >>>>> Still, I am curious whether setting the loader tunable = vfs.buf_pager_relbuf >>>>> to 1 change anything. >>>>=20 >>>> MIPS caches are such that creating two virtual mappings to the same >>>> physical page will cause corruption. It's simply not allowed, at = least >>>> for the class of MIPS machines I used to bring up the port = originally. >>>=20 >>> Yes, caches are VIPT on 24k, according to the "MIPS32(R) 24K(R) >>> Processor Core Family Software User's Manual " rev 3.11. My = question is, >>> how is that handled in the mips pmap.c. I was not able to locate >>> the alias detection and prevention code, or e.g. switching to = uncached mode >>> for the page when aliasing is detected, after browsing pmap. >>=20 >> Aliases are not permitted. IIRC, there's no code that detects this >> condition. One must simply never ever have multiple cached mappings = of >> a page at once. A quick glance at the code doesn't locate anything. > Then, the obvious next question is what does prevent such aliased > mappings ? Not only usermode might establish such situation by > double-mapping, but also e.g. our coherent buffer/page cache maps page > into KVA and the same page might be mapped into usermode. The later > situation is my current thought about possible cause of the reported > init(8) fault. That=E2=80=99s a very good question. Unfortunately my memory here fails = me. I have a recollection that the various cache flushing that we do in = the map ensures that we don=E2=80=99t hit a problem. However, I = couldn=E2=80=99t construct a coherent answer to where this takes place = after a brief look at the code. Multiple mappings will cause data = corruption though. I wouldn=E2=80=99t have thought it causes the fault = in question, but MIPS errors can manifest in strange ways, so perhaps = you are correct. I don=E2=80=99t get how the fault address is 0 though = from that scenario. Maybe Adrian can recall. Or Juli can help. Warner --Apple-Mail=_070B6E65-8DF8-45E7-A09D-18C3A65DACF0 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJYKCClAAoJEGwc0Sh9sBEAIOsQAKWRViXd09yNuv77TI9EKmZi dqnuwrLBy8iuqQMvUQVvmEs/miOBarVZt/4HdrAaij3ruvDwK9XqbRZc8zs2UGI6 Ep5dAzuAXZr7hMJXFSr+5zlCdNaPui85AYCbOqcEEuZ34NNRrE4b3hPUsBd9dcMR ZCDu2wVlWhpZtsCJNohF/4SohU8W2qo7ZCSlpckR04yNsaQX+sxkwRopRGFSs101 wUihLKTto4Kj7Bhvpv8Up79AyIC9HelpZBPhTKOgq777YGrPlJdr1p7yKPfFogKh H+oVbKn+6mPcAYJ0845yz9x2l34WDgxzAKGoDacpPo5+n2W/oFx7uiifHx0j20Ak YYKSDcn/+MjgTGEargqSWxrFORJYhIDVromlMVBq81ENxLbT71xS6NUVczh/EJrr vv3RxBWbk2q05fzCaWzJVc/9f70JXfgwbWMhsn0Bv/NhC3SDrcd2teCTHBZAfCIn j1GlHpmA6pknwVK1lm9NBjeAhpG29jsttizr7C9TiRwOH3aNIBG27/KExSwHDGhz QYht1kp8yYdu6djsTbI+7mTEEWV5fv7qWw6NBHO0sM/GH3GstNNaP4Gw74XHq7Zi pp4tMFi/qacOSazYa+03E8B7Pi5K2vUt7c76kjTYzp3AQnsOglVUzE97LkRNDvIr VVtGxz3BG0UyepWmZoNr =fUMb -----END PGP SIGNATURE----- --Apple-Mail=_070B6E65-8DF8-45E7-A09D-18C3A65DACF0-- From owner-freebsd-mips@freebsd.org Sun Nov 13 15:51:26 2016 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AEA8BC3F831 for ; Sun, 13 Nov 2016 15:51:26 +0000 (UTC) (envelope-from juli@clockworksquid.com) Received: from mail-it0-x235.google.com (mail-it0-x235.google.com [IPv6:2607:f8b0:4001:c0b::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 701D21DB4 for ; Sun, 13 Nov 2016 15:51:26 +0000 (UTC) (envelope-from juli@clockworksquid.com) Received: by mail-it0-x235.google.com with SMTP id c20so33753742itb.0 for ; Sun, 13 Nov 2016 07:51:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=clockworksquid.com; s=google; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-transfer-encoding; bh=ilStorIxegHUnJ/tdviBMRRMv1A6rQRnI2a46P/cF48=; b=LCiQ/j9G/YUDQJR1rbPWZ3m4HOmehkONm3pg0cVR2WSN2+RkxH8lrjDm5QKt00be0j 4fxLluE0sC6UsorQFb9D6npkSscRITyxHnqk0o8EsAaeItzRi8UfP8qv/BLKMDJLNbU8 uD9o/2Yy4vlQu/xx4/twUP8Bsh0LhesoMjLAw= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd-org.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-transfer-encoding; bh=ilStorIxegHUnJ/tdviBMRRMv1A6rQRnI2a46P/cF48=; b=CPFCPR4JQIoZlp+NkO8SpF2N/JZsiqRh+G4ZXrN5Bf9Y5ggH1TrlSgCuit1Txr8k7q 7XTYXxO7DHaLdc5VaS/44v8NQzH87SLHVj1yNA9Jc2LLLkqOflx2L+IF6WDjtQ1UKw7a 2CNcRye/q6gUcvUwfidNTspV2qXsGkaZBKJ3vROILmSv1Cvyfi8xN2coCRZLo8po5Pw8 VmVLN15+0Tyty42SMw1ONcAZoNyNqQQR3phsSgVmexIiWUwc8ssOm41CkjsagEuLdYT1 x7gdkh7s7G1PhEOSnLJgloRY+U428oqUOU8dsrKPHZMm0UvViws8Bh7sMf6lR0w3yNuY iW3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-transfer-encoding; bh=ilStorIxegHUnJ/tdviBMRRMv1A6rQRnI2a46P/cF48=; b=YiEryrmhUftyfi3cntGMS0Zv81wp1+eUcPpJENPc239KWo0I/eCtiQX4Cbf7GlyBbg YTRMfTcegf17Z3Uk0DyhJXjZ0hqvjjv70UszAyXpuCTG0RMs3Da0hUgCp2ysG3UBY+3E p0ZCxKSiitNILKc0MStOxslIDP0p5atO6UGflmhAYffHbK2cMu4DlS+wsI2zGuc8Taor CnqZ3D65KO+Tv/TAo7U3MkSHlevI7sMtYz49agH+po5JWvjad/y8Ibk5ZN6cpYRQgeHY xZ9D0Zq7wx+pzfzw9Nxy87TebNRDt/BxI4kP3M30EtwBKiEkCVdV9TcCflWViLS85S52 EM/A== X-Gm-Message-State: ABUngveD+VvyBeDb8Txlul3EJpqzLUw6MHCmrnZhMYcBW593UrhBi8COrmyQvAP038N3IYt/LE7/ZLCy7K+pzg== X-Received: by 10.107.59.9 with SMTP id i9mr23347047ioa.176.1479052285626; Sun, 13 Nov 2016 07:51:25 -0800 (PST) MIME-Version: 1.0 Sender: juli@clockworksquid.com Received: by 10.79.141.88 with HTTP; Sun, 13 Nov 2016 07:51:05 -0800 (PST) In-Reply-To: <71C512CD-0FB6-40D8-B46C-30467A245693@bsdimp.com> References: <201610191109.u9JB9TTC002727@repo.freebsd.org> <20161113065851.GD54029@kib.kiev.ua> <20161113071911.GF54029@kib.kiev.ua> <20161113075557.GH54029@kib.kiev.ua> <71C512CD-0FB6-40D8-B46C-30467A245693@bsdimp.com> From: Juli Mallett Date: Sun, 13 Nov 2016 07:51:05 -0800 X-Google-Sender-Auth: DuB9-riFV-zkfLGaGs2PpDDjW7s Message-ID: Subject: Re: svn commit: r307626 - head/sys/ufs/ffs To: Warner Losh Cc: Konstantin Belousov , Warner Losh , Adrian Chadd , "freebsd-mips@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Nov 2016 15:51:26 -0000 On Sun, Nov 13, 2016 at 12:13 AM, Warner Losh wrote: > [[ Moved to freebsd-mips and cc=E2=80=99d to Juli ]] > > That=E2=80=99s a very good question. Unfortunately my memory here fails m= e. I have a recollection that the various cache flushing that we do in the = map ensures that we don=E2=80=99t hit a problem. However, I couldn=E2=80=99= t construct a coherent answer to where this takes place after a brief look = at the code. Multiple mappings will cause data corruption though. I wouldn= =E2=80=99t have thought it causes the fault in question, but MIPS errors ca= n manifest in strange ways, so perhaps you are correct. I don=E2=80=99t get= how the fault address is 0 though from that scenario. Maybe Adrian can rec= all. Or Juli can help. As I recall, the "mips2" and Juniper code just punted on this, and the current in-tree code did likewise. In fact, we explicitly use aliasing in some places for short-term things through cached, direct-mapped operations, with probably less than the requisite cache ops to make VIPT processors work safely. There's also no set of cache ops you can put into pmap to make it safe in general; attempts to do so probably haven't understood the scope of the problem. We can end up with aliasing in the same process, so doing it only on pmap activate doesn't prevent aliased accesses. You could, if you want to really churn the TLB, allow only one active mapping at a time and go through a trap to manage the mappings of shared, cacheable pages. It's either that, or we have to, as Adrian suggested, either mappings to uncached when persistent aliasing occurs, or unshare those mappings, depending on whether the aliasing is an optimization or a requirement for correctness. The easier thing is probably to go through the PV mappings, and allow only one active PTE at a time, and to eat the cost of all those traps for aliased mappings, while remembering to be diligent about not allowing sharing without copious cache flushing in the cases in the kernel where we have temporary aliases using the direct map. It'd be neat to see this happen; I'm not sure who would really do it. It's tricky to get right. We may support other architectures that have complete or partial solutions to this, but I deliberately don't know about them. Stare at code dealing with VIPT ARM systems, or pmap on other BSDs that have sunk more effort into all the edge cases of VIPT MIPS systems. Thanks, Juli. From owner-freebsd-mips@freebsd.org Sun Nov 13 16:15:56 2016 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8D5D6C400EF for ; Sun, 13 Nov 2016 16:15:56 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1C61D1B6A; Sun, 13 Nov 2016 16:15:55 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uADGFmSR015585 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 13 Nov 2016 18:15:48 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uADGFmSR015585 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uADGFmii015584; Sun, 13 Nov 2016 18:15:48 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 13 Nov 2016 18:15:48 +0200 From: Konstantin Belousov To: Juli Mallett Cc: Warner Losh , Warner Losh , Adrian Chadd , "freebsd-mips@freebsd.org" Subject: Re: svn commit: r307626 - head/sys/ufs/ffs Message-ID: <20161113161548.GK54029@kib.kiev.ua> References: <201610191109.u9JB9TTC002727@repo.freebsd.org> <20161113065851.GD54029@kib.kiev.ua> <20161113071911.GF54029@kib.kiev.ua> <20161113075557.GH54029@kib.kiev.ua> <71C512CD-0FB6-40D8-B46C-30467A245693@bsdimp.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.1 (2016-10-04) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Nov 2016 16:15:56 -0000 On Sun, Nov 13, 2016 at 07:51:05AM -0800, Juli Mallett wrote: > On Sun, Nov 13, 2016 at 12:13 AM, Warner Losh wrote: > > [[ Moved to freebsd-mips and cc???d to Juli ]] > > > > That???s a very good question. Unfortunately my memory here fails me. I have a recollection that the various cache flushing that we do in the map ensures that we don???t hit a problem. However, I couldn???t construct a coherent answer to where this takes place after a brief look at the code. Multiple mappings will cause data corruption though. I wouldn???t have thought it causes the fault in question, but MIPS errors can manifest in strange ways, so perhaps you are correct. I don???t get how the fault address is 0 though from that scenario. Maybe Adrian can recall. Or Juli can help. > > As I recall, the "mips2" and Juniper code just punted on this, and the > current in-tree code did likewise. In fact, we explicitly use > aliasing in some places for short-term things through cached, > direct-mapped operations, with probably less than the requisite cache > ops to make VIPT processors work safely. > > There's also no set of cache ops you can put into pmap to make it safe > in general; attempts to do so probably haven't understood the scope of > the problem. We can end up with aliasing in the same process, so > doing it only on pmap activate doesn't prevent aliased accesses. You > could, if you want to really churn the TLB, allow only one active > mapping at a time and go through a trap to manage the mappings of > shared, cacheable pages. It's either that, or we have to, as Adrian > suggested, either mappings to uncached when persistent aliasing > occurs, or unshare those mappings, depending on whether the aliasing > is an optimization or a requirement for correctness. The easier thing > is probably to go through the PV mappings, and allow only one active > PTE at a time, and to eat the cost of all those traps for aliased > mappings, while remembering to be diligent about not allowing sharing > without copious cache flushing in the cases in the kernel where we > have temporary aliases using the direct map. Sparc64 also has the aliasing issue, and AFAIR sparc64 pmap does allow more than one mapping of the same page at conflicting ('differently colored') addresses. But indeed, each mapping (not only the managed mappings) are tracked, and all mappings are demoted to uncached if conflict is created. > > It'd be neat to see this happen; I'm not sure who would really do it. > It's tricky to get right. We may support other architectures that > have complete or partial solutions to this, but I deliberately don't > know about them. Stare at code dealing with VIPT ARM systems, or pmap > on other BSDs that have sunk more effort into all the edge cases of > VIPT MIPS systems. For arm, only arm v4 pmap deals with VIPT aliasing. Arm v6 assumes that the system is free from that bug. From owner-freebsd-mips@freebsd.org Sun Nov 13 18:42:46 2016 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9555EC4035B for ; Sun, 13 Nov 2016 18:42:46 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5C18A18F9; Sun, 13 Nov 2016 18:42:46 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-it0-x236.google.com with SMTP id c20so38939915itb.0; Sun, 13 Nov 2016 10:42:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=rxfnVWjcLFOgPlPSu5DaDj4vaI7UqoDAoLsgrH/QhNw=; b=u2D7rsLIGVCYIxMmDBpjPgib4+4gQQNH8ojh/U2cF+Y+1kt+MlKOG7ZSM7xFf2mXNj +lzSy6CFuuNygf5hhAgUV7jPC/UI2qVAX62PLpr0a7fe4f38qDBwssmGvvWhNH/GuEAO fcSO4x70sV3yWcC3Abx6ddvVbtvkCYOiEEs9uz8FnDaLgsYz1XDR1lMBAbMtZtb+flR0 NapW6zDoqz1ltTcDN155RfcoNYzPKEU5XU9W+lc59V3DauIuMtzGF7N/OYEeauuhXZfV ITSYl0Jtes0x2TmX1ldnLYQKnkY7+neMkNCQHTe12BT5LRIXWCdXLCBzLXpM4cZzCBkn Fs7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=rxfnVWjcLFOgPlPSu5DaDj4vaI7UqoDAoLsgrH/QhNw=; b=lBxrfp7Eyw+QP7LEwD6HiJm5D4qKOTEGb7U3Mfmv61DXgRaZ66/5cX/E7f15FEM2rA NwvLDZgBL0zxi6zWZeiPfSgI9s9G/Vf45H+MD/NmTHutwJWQ31md1LI70j8ECfvvj1bi /MPZxSYO+862HEThQZjvAChJ+uzLBJen09P2v0yUmWRz8SsRsTVGi74qCbKBMmgyEUkG fJis2OkDVLt1NzIN73VD1x+UUu7O71dddTE3jNH5PD7+STTz3WRGKl6AhPfbDVsX4+hV mfLtugYtePsSVpO8Dv08dYTXUHnwNd6dbAiWlwL036oyX470BpBZ6O7MnSB+LTwp4Qbp IzYQ== X-Gm-Message-State: ABUngveTjIlMgkF0lne3kFTV0fi7KFCjfFM8Nl5ehvDQv3DV1hFgYeNegcXDOPEaaKZyF9cvXeJ2UqK4F2VNVw== X-Received: by 10.36.58.85 with SMTP id m82mr702194itm.29.1479062565810; Sun, 13 Nov 2016 10:42:45 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.39.134 with HTTP; Sun, 13 Nov 2016 10:42:44 -0800 (PST) In-Reply-To: <20161113161548.GK54029@kib.kiev.ua> References: <201610191109.u9JB9TTC002727@repo.freebsd.org> <20161113065851.GD54029@kib.kiev.ua> <20161113071911.GF54029@kib.kiev.ua> <20161113075557.GH54029@kib.kiev.ua> <71C512CD-0FB6-40D8-B46C-30467A245693@bsdimp.com> <20161113161548.GK54029@kib.kiev.ua> From: Adrian Chadd Date: Sun, 13 Nov 2016 10:42:44 -0800 Message-ID: Subject: Re: svn commit: r307626 - head/sys/ufs/ffs To: Konstantin Belousov Cc: Juli Mallett , Warner Losh , Warner Losh , "freebsd-mips@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Nov 2016 18:42:46 -0000 On 13 November 2016 at 08:15, Konstantin Belousov wrote: > Sparc64 also has the aliasing issue, and AFAIR sparc64 pmap does allow > more than one mapping of the same page at conflicting ('differently > colored') addresses. But indeed, each mapping (not only the managed > mappings) are tracked, and all mappings are demoted to uncached if > conflict is created. Hm, so where's that been happening? Why hasn't this happened in the mips32 world until this commit? >> It'd be neat to see this happen; I'm not sure who would really do it. >> It's tricky to get right. We may support other architectures that >> have complete or partial solutions to this, but I deliberately don't >> know about them. Stare at code dealing with VIPT ARM systems, or pmap >> on other BSDs that have sunk more effort into all the edge cases of >> VIPT MIPS systems. > For arm, only arm v4 pmap deals with VIPT aliasing. Arm v6 assumes that > the system is free from that bug. -adrian From owner-freebsd-mips@freebsd.org Sun Nov 13 18:50:21 2016 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE3DCC4049C for ; Sun, 13 Nov 2016 18:50:21 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-it0-x22f.google.com (mail-it0-x22f.google.com [IPv6:2607:f8b0:4001:c0b::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 83FBA1B5D; Sun, 13 Nov 2016 18:50:21 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-it0-x22f.google.com with SMTP id q124so60811719itd.1; Sun, 13 Nov 2016 10:50:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=e5ya5acOb3EPm3HKBAbyqN4bZbz91tdPii5g8Xk0F90=; b=W8Oed199ZFdYdEp/hzMNxhcg0PWUW0ntzF7hLabvX7BM26xBr9aIEdBsU/wo25Ecw7 hoOP7znWybFnqDI4oH0761KGhxO1zDEOn6TcybryaRl5Inyee65At8i/4Y9MEgbICfMc fZV9yK21KAYLub4O4lHeGVA5hXQbXkWR2YIzHrgYuIiqNyinDXYSw2e/j3+JatBHCqtm AOy0Ckaum59RCRF0TAGts3tNqtXQat862NqFeFaYKuAbakbNBn1lZg2EDMog+UxsSk5D MBcIVYhpERiwxRs/P/xLUH+jKHDS4PZ8Xb1cDS1AVhDlZGPWvJKTBRHHDe3ChQuQK01r I4qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=e5ya5acOb3EPm3HKBAbyqN4bZbz91tdPii5g8Xk0F90=; b=OkouE27YvnAKga4rNB2SMa1/kQsXwcaM9GHXdzai5++xPc2JUcjS7HSXh0COASoH9s 3+VBnPhAsIuvfiVnJXOs/jWQj5aoWq14nxHETBtGLzb3G5RUAzaltKCR8KvD93Bsvzrx xAHs1bg4bpuHU9vZ2exVCFIuJprh1RHPdfLpCFMh122HFPCDFtFHM4NmB78wjhBnV6zY KmKnAWjM6VfZ7yzlFE0X1931mEc78YvAITK5ndb/OC4YkiwG1s+5kSQY40BqSXYNmz0t MMiQzKsAlcmlWGJcEq1pYQ+L7QsmGGrn/vI7cQNx4Z/7r6EpcdNP9U2l0EjkMlpigeM2 ZlTA== X-Gm-Message-State: ABUngvc+TCp9L/0eaknyhKjs/bhtROWtI7BwC41eR3TI3T2ccg5MAqPzVWxo1rCfIe1yCe6Gbq3JoivdjlZAxQ== X-Received: by 10.107.174.157 with SMTP id n29mr16590484ioo.177.1479063021002; Sun, 13 Nov 2016 10:50:21 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.39.134 with HTTP; Sun, 13 Nov 2016 10:50:19 -0800 (PST) In-Reply-To: References: <201610191109.u9JB9TTC002727@repo.freebsd.org> <20161113065851.GD54029@kib.kiev.ua> <20161113071911.GF54029@kib.kiev.ua> <20161113075557.GH54029@kib.kiev.ua> <71C512CD-0FB6-40D8-B46C-30467A245693@bsdimp.com> <20161113161548.GK54029@kib.kiev.ua> From: Adrian Chadd Date: Sun, 13 Nov 2016 10:50:19 -0800 Message-ID: Subject: Re: svn commit: r307626 - head/sys/ufs/ffs To: Konstantin Belousov Cc: Juli Mallett , Warner Losh , Warner Losh , "freebsd-mips@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Nov 2016 18:50:21 -0000 Ok, so after talking with others, my questions are: * I thought our VM was supposed to not be doing double mapping like this. warner's comment on irc was: === 13:39 < bsdimp> adrian the VM isn't supposed to do it at all. 13:39 < bsdimp> adrian that is, double map on purpose. 13:40 < bsdimp> though there's some exceptions to the rule 13:40 < bsdimp> since kernel mappings go away in userland, and userland doesn't execute while you're in kernel mode, you can do the flushing game in busdma to prevent most issues. 13:41 < bsdimp> which is what we do. Generally, though, our VM doesn't do it in-kernel. === * is this still the case? or are there places in the VM where we are doing this? * can we introduce a machdep/pmap capability check to see if aliasing is allowed and if so, turn this feature on? Adding a pmap capability and turning it on for say, i386/amd64/arm64 would allow for this new feature as well as the previous behaviour on older platforms. I don't think I have the time to fix mips pmap to support this new feature, so if you want to turn it on for all features, we should really fix/test pmap on said platforms first. Final comment: I'd really like to see a sort of "tested on" for things like this, because it's not clear which platforms/architectures it was tested on. Thanks, -adrian From owner-freebsd-mips@freebsd.org Sun Nov 13 19:03:51 2016 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C337FC4084C for ; Sun, 13 Nov 2016 19:03:51 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6A6B312FE; Sun, 13 Nov 2016 19:03:51 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uADJ3i8Q062457 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 13 Nov 2016 21:03:45 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uADJ3i8Q062457 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uADJ3i3e062456; Sun, 13 Nov 2016 21:03:44 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 13 Nov 2016 21:03:44 +0200 From: Konstantin Belousov To: Adrian Chadd Cc: Juli Mallett , Warner Losh , Warner Losh , "freebsd-mips@freebsd.org" Subject: Re: svn commit: r307626 - head/sys/ufs/ffs Message-ID: <20161113190344.GM54029@kib.kiev.ua> References: <20161113065851.GD54029@kib.kiev.ua> <20161113071911.GF54029@kib.kiev.ua> <20161113075557.GH54029@kib.kiev.ua> <71C512CD-0FB6-40D8-B46C-30467A245693@bsdimp.com> <20161113161548.GK54029@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.1 (2016-10-04) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Nov 2016 19:03:51 -0000 On Sun, Nov 13, 2016 at 10:50:19AM -0800, Adrian Chadd wrote: > Ok, so after talking with others, my questions are: > > * I thought our VM was supposed to not be doing double mapping like > this. warner's comment on irc was: > > === > 13:39 < bsdimp> adrian the VM isn't supposed to do it at all. > 13:39 < bsdimp> adrian that is, double map on purpose. > 13:40 < bsdimp> though there's some exceptions to the rule > 13:40 < bsdimp> since kernel mappings go away in userland, and > userland doesn't execute while you're in kernel mode, you can do the > flushing > game in busdma to prevent most issues. > 13:41 < bsdimp> which is what we do. Generally, though, our VM doesn't > do it in-kernel. > === > > * is this still the case? or are there places in the VM where we are doing this? > * can we introduce a machdep/pmap capability check to see if aliasing > is allowed and if so, turn this feature on? This never was the case, and never will. VM establishes mappings as requested by the userspace and kernel, and n-times mapping for the same page is always legitimate. It is the pmap duty to handle that. If you cared to read my previous mail, I already explained that besides the userspace asking for n-mapping, there is at least buffer cache which also maps the same page into KVA, from the times when unified buffer cache/page cache was introduced. Same is true for n-mapping of shared anon pages. > > Adding a pmap capability and turning it on for say, i386/amd64/arm64 > would allow for this new feature as well as the previous behaviour on > older platforms. > > I don't think I have the time to fix mips pmap to support this new > feature, so if you want to turn it on for all features, we should > really fix/test pmap on said platforms first. This is not a new feature, this is the old bug on that platform. > > Final comment: I'd really like to see a sort of "tested on" for things > like this, because it's not clear which platforms/architectures it was > tested on. From owner-freebsd-mips@freebsd.org Sun Nov 13 20:55:57 2016 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9B1E7C407B0 for ; Sun, 13 Nov 2016 20:55:57 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x243.google.com (mail-it0-x243.google.com [IPv6:2607:f8b0:4001:c0b::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 61E221300 for ; Sun, 13 Nov 2016 20:55:57 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x243.google.com with SMTP id b123so8804558itb.2 for ; Sun, 13 Nov 2016 12:55:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=subject:mime-version:from:in-reply-to:date:cc:message-id:references :to; bh=QZU1yBd7e4SoNrWVHBDXZ+ETgRmQDdrC1t2HZSGiGWM=; b=go6buCaNh5vnkDUR+ECDW3KRMgPDaf9Gvtgz7l7GvT1qLNKk6EzLMEwlnZ1jH4wj2e am5oLb4JgLvZWF6sjHPIkZYw67UDguFxRm/TDpXTLuhJgnOBzuJHSolGrM5uYVPSFm8+ D4Cd+2jRNRprnm2prm0YU4R9/iXyrEfArolhdxmfQgNAImSQmpEi9rxlH7rGVUPCTp79 pxhDfuAJ+StnCCCdKatNOoDfeldz3EcCgLkZilAk25edl/yBz/XB1vK55D3PkrzSAc2q rFi5dSlnf1dvquBtkGhQR0fg263i9QGgLyVpwA/V0a/rOxwkRqDToruwpsrov5sOXN/a h+lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:mime-version:from:in-reply-to:date:cc :message-id:references:to; bh=QZU1yBd7e4SoNrWVHBDXZ+ETgRmQDdrC1t2HZSGiGWM=; b=NBVrELQnygtioCfx1VDvuvXjrhpumsI2JELq4oZLHAXSoYkRTGw0378u6SFPInD7Up oCE9Ol/Vg5d0y9s7A/1UwZdK/FSmKJwMBCJ8HVgoeAXi/oX5asZ9AD+MetvnOna6BOQ0 peneWz7cRymqQ2zTlKaM44Bng6qdUv+lifSNpUTFW5R4cMN64rCoOAgjN59Ih15TWUw7 UWYfw75OVDmY1RfzBMaY3y/gIIE9hqaIkXwHRz8OzIOIASpOdBNiKhTjQ0jVGpgOB1Vy zeZnGFYx+eN5UbXkGyGO2Br/BV8CzpRVmUk5DiajV+vA6qxqguBAWFAGMUG0Uz3pkeTL YZbA== X-Gm-Message-State: ABUngvcsNLAJWyLeRLMXIE+G6RlTJx7tFQ2zDaLDmpMwMUTB2jaTRriHRwliuhaUSfhgTQ== X-Received: by 10.36.39.85 with SMTP id g82mr4512327ita.52.1479070556640; Sun, 13 Nov 2016 12:55:56 -0800 (PST) Received: from mac-wired.nflx.bsdimp.com (50-253-99-174-static.hfc.comcastbusiness.net. [50.253.99.174]) by smtp.gmail.com with ESMTPSA id o19sm8002551ito.16.2016.11.13.12.55.55 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 13 Nov 2016 12:55:55 -0800 (PST) Subject: Re: svn commit: r307626 - head/sys/ufs/ffs Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: multipart/signed; boundary="Apple-Mail=_F0650D9E-757D-4745-AAE3-E86EB3622C11"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail From: Warner Losh In-Reply-To: <20161113190344.GM54029@kib.kiev.ua> Date: Sun, 13 Nov 2016 13:55:54 -0700 Cc: Adrian Chadd , Juli Mallett , Warner Losh , "freebsd-mips@freebsd.org" Message-Id: <50FE3B7E-8FA4-47DC-BC45-EE75B9FAFC0F@bsdimp.com> References: <20161113065851.GD54029@kib.kiev.ua> <20161113071911.GF54029@kib.kiev.ua> <20161113075557.GH54029@kib.kiev.ua> <71C512CD-0FB6-40D8-B46C-30467A245693@bsdimp.com> <20161113161548.GK54029@kib.kiev.ua> <20161113190344.GM54029@kib.kiev.ua> To: Konstantin Belousov X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Nov 2016 20:55:57 -0000 --Apple-Mail=_F0650D9E-757D-4745-AAE3-E86EB3622C11 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Nov 13, 2016, at 12:03 PM, Konstantin Belousov = wrote: >=20 > On Sun, Nov 13, 2016 at 10:50:19AM -0800, Adrian Chadd wrote: >> Ok, so after talking with others, my questions are: >>=20 >> * I thought our VM was supposed to not be doing double mapping like >> this. warner's comment on irc was: >>=20 >> =3D=3D=3D >> 13:39 < bsdimp> adrian the VM isn't supposed to do it at all. >> 13:39 < bsdimp> adrian that is, double map on purpose. >> 13:40 < bsdimp> though there's some exceptions to the rule >> 13:40 < bsdimp> since kernel mappings go away in userland, and >> userland doesn't execute while you're in kernel mode, you can do the >> flushing >> game in busdma to prevent most issues. >> 13:41 < bsdimp> which is what we do. Generally, though, our VM = doesn't >> do it in-kernel. >> =3D=3D=3D >>=20 >> * is this still the case? or are there places in the VM where we are = doing this? >> * can we introduce a machdep/pmap capability check to see if aliasing >> is allowed and if so, turn this feature on? > This never was the case, and never will. VM establishes mappings as > requested by the userspace and kernel, and n-times mapping for the = same > page is always legitimate. It is the pmap duty to handle that. User space N way mapping is rare. Can you show an actual example of when = this is commonly done? Or being done in init(8)? > If you cared to read my previous mail, I already explained that = besides > the userspace asking for n-mapping, there is at least buffer cache = which > also maps the same page into KVA, from the times when unified buffer > cache/page cache was introduced. Same is true for n-mapping of shared > anon pages. KVA mapping is different, since that mapping goes away. And generally, = the KVA mappings aren=E2=80=99t active while user land mappings are = active. Here =E2=80=98active=E2=80=99 means can change the cache. When = the kernel is executing on these single core 32-bit mips boxes, user = space isn=E2=80=99t creating traffic to the pages. Also, KVA mappings = tend to be for pages that we don=E2=80=99t actually touch in the kernel = once the I/O is complete. This is true for data pages in files. It may = not be true for in-kernel meta-data pages, and that might have changed = with your commit (I haven=E2=80=99t looked in enough detail to know). But that still begs the question: why is the fault address 0 in the = original crash report. If we=E2=80=99re doing multiple mappings, and = that=E2=80=99s creating a cache coherency problem because there=E2=80=99s = two copies of dirty cache data, how does that end up with us doing a = NULL dereference? That=E2=80=99s the part that I=E2=80=99m having = trouble understanding and I worry that arguing over the exact parameters = of the VM mapping issues might be distracting us from a credible theory, = with a specific sequence of events that gets us here. It looks like we use the direct map in kernel for addresses < 256MB. = KSEG0 covers a lot of sins. NetBSD has code to cope with congruent mappings that cause problems. = These look to be only on LONGSOON and MIPS3 processors with mips32 and = newer not affected by it. MIPS3 has VCE exceptions for Instructions and = Data and NetBSD has code there to cope. It also maps the pages uncached = until the multiple mappings go away. The other code it has to cope is = effectively disabled when not on these processors. It is a twisty maze = of #ifdefs, though, so maybe I=E2=80=99m missing something. It does = disable the socket page loaning code as well when there=E2=80=99s = issues. So, we=E2=80=99re back to the basic question: why does this change cause = the observed behavior, exactly? >> Adding a pmap capability and turning it on for say, i386/amd64/arm64 >> would allow for this new feature as well as the previous behaviour on >> older platforms. >>=20 >> I don't think I have the time to fix mips pmap to support this new >> feature, so if you want to turn it on for all features, we should >> really fix/test pmap on said platforms first. > This is not a new feature, this is the old bug on that platform. This sort of argument is pointless: the code does something new, and the = current code doesn=E2=80=99t support it. It=E2=80=99s that simple. = Trying to label new bug / old bug, etc is pointless. It=E2=80=99s a bug, = and it=E2=80=99s gotta get fixed. It isn=E2=80=99t clear to me it=E2=80=99= s even understood why things are happening, so even the old / new finger = pointing is premature given we don=E2=80=99t have a definitive cause, = just a theory. Warner >> Final comment: I'd really like to see a sort of "tested on" for = things >> like this, because it's not clear which platforms/architectures it = was >> tested on. --Apple-Mail=_F0650D9E-757D-4745-AAE3-E86EB3622C11 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJYKNNaAAoJEGwc0Sh9sBEAwrcP/jZwGTBPLiClMsdw/oQYmTK3 2ch2RafvMzVotVTKaRLKapRTYgShEkfSYudRdbQcSjH/p84xluPk14aaioIjJTfE 5jJaAmwKXcxK93oQpVmbW3wDFKY6zl/IeBwVW76/RP2XZkveu/3Lj3ubAUg1O+Jo FFN6gzcgmLYS1ik+4GcRWi4r12BtbwGqITKei0CyWrDGax3aIbmJ3Px5pB+oVa2K t5ONP9GrFUVvyNIof+lSoYiy2GmRK1Y+LovMgyu6lIU8S1wWJdiljuAMY9Al+7fU U/QMcJx4jMkg2HrJJtqwcpaTDCFP24LDgNW+p/DNvyfmllQBjLspptTeflZB0LmB FTNcWEZdSiydPKEhhClXMxbDhSaNQeqmLH+uXhgfgx2AMOQRJIX+GS5jgsrD+WWn +Z/Gi0QkwAgRE5HsCKQxnxf2nvirvosM9LMMDkNGFVfYZIrB6+4saH+On6SewGcB iE8bRplVIvYxqVuytgFdgWRX5k4wg9NrtG37MQ5h+5mhzgvMuSKdV7UiWxHbpSGN MwsP2+GOqKj3p1hFxrjGPQp/UxrUVUejI2tj3d2kbIdJPGPH/mslBJbI16hum3X1 rXfieADkateIhQQjkQIGBIxsnzT9XMUOLzNLOoPlWhr8tmkw0naMYiUQ2r+5IBc+ 2uCcj0enMhkaWPdS8slF =Fczy -----END PGP SIGNATURE----- --Apple-Mail=_F0650D9E-757D-4745-AAE3-E86EB3622C11-- From owner-freebsd-mips@freebsd.org Sun Nov 13 21:31:16 2016 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EAE4CC3F99D for ; Sun, 13 Nov 2016 21:31:16 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 781CA1B43; Sun, 13 Nov 2016 21:31:16 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uADLV8Hv098254 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 13 Nov 2016 23:31:09 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uADLV8Hv098254 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uADLV8xh098253; Sun, 13 Nov 2016 23:31:08 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 13 Nov 2016 23:31:08 +0200 From: Konstantin Belousov To: Warner Losh Cc: Adrian Chadd , Juli Mallett , Warner Losh , "freebsd-mips@freebsd.org" Subject: Re: svn commit: r307626 - head/sys/ufs/ffs Message-ID: <20161113213108.GP54029@kib.kiev.ua> References: <20161113071911.GF54029@kib.kiev.ua> <20161113075557.GH54029@kib.kiev.ua> <71C512CD-0FB6-40D8-B46C-30467A245693@bsdimp.com> <20161113161548.GK54029@kib.kiev.ua> <20161113190344.GM54029@kib.kiev.ua> <50FE3B7E-8FA4-47DC-BC45-EE75B9FAFC0F@bsdimp.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <50FE3B7E-8FA4-47DC-BC45-EE75B9FAFC0F@bsdimp.com> User-Agent: Mutt/1.7.1 (2016-10-04) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Nov 2016 21:31:17 -0000 On Sun, Nov 13, 2016 at 01:55:54PM -0700, Warner Losh wrote: > > > On Nov 13, 2016, at 12:03 PM, Konstantin Belousov wrote: > > > > On Sun, Nov 13, 2016 at 10:50:19AM -0800, Adrian Chadd wrote: > >> Ok, so after talking with others, my questions are: > >> > >> * I thought our VM was supposed to not be doing double mapping like > >> this. warner's comment on irc was: > >> > >> === > >> 13:39 < bsdimp> adrian the VM isn't supposed to do it at all. > >> 13:39 < bsdimp> adrian that is, double map on purpose. > >> 13:40 < bsdimp> though there's some exceptions to the rule > >> 13:40 < bsdimp> since kernel mappings go away in userland, and > >> userland doesn't execute while you're in kernel mode, you can do the > >> flushing > >> game in busdma to prevent most issues. > >> 13:41 < bsdimp> which is what we do. Generally, though, our VM doesn't > >> do it in-kernel. > >> === > >> > >> * is this still the case? or are there places in the VM where we are doing this? > >> * can we introduce a machdep/pmap capability check to see if aliasing > >> is allowed and if so, turn this feature on? > > This never was the case, and never will. VM establishes mappings as > > requested by the userspace and kernel, and n-times mapping for the same > > page is always legitimate. It is the pmap duty to handle that. > > User space N way mapping is rare. Can you show an actual example of when this is commonly done? Or being done in init(8)? > User space N mapping is very common with e.g. shared libraries. Writeable n-way mappings are ubiquitously used by X apps in the form of sysv shared memory segments, by database systems, in particular postgres, by qt c++ framework, and so on. If you use vi on some file, nvi mmaps the content. If any other application reads the file simultaneously with the file being opened in vi, you get double-mapping by userspace and KVA, with all consistency problems. It is rather pointless to argue about the necessity and absolute omnipresent nature of this feature. Even without the new UFS pager, apparently the MIPS port randomly corrupts user data. > > If you cared to read my previous mail, I already explained that besides > > the userspace asking for n-mapping, there is at least buffer cache which > > also maps the same page into KVA, from the times when unified buffer > > cache/page cache was introduced. Same is true for n-mapping of shared > > anon pages. > > KVA mapping is different, since that mapping goes away. And generally, > the KVA mappings aren???t active while user land mappings are active. > Here ???active??? means can change the cache. When the kernel is > executing on these single core 32-bit mips boxes, user space isn???t > creating traffic to the pages. Also, KVA mappings tend to be for pages > that we don???t actually touch in the kernel once the I/O is complete. > This is true for data pages in files. It may not be true for in-kernel > meta-data pages, and that might have changed with your commit (I > haven???t looked in enough detail to know). > This is not how 'mappings' work, at least in FreeBSD. KVA mappings are not switched away when CPU enters usermode, and VI caches for KVA indexes are not flushed on switch to usermode. So if the page is double-mapped and the colors of the mappings are different, it does not matter whether it is kernel or user VA to create problems. > But that still begs the question: why is the fault address 0 in > the original crash report. If we???re doing multiple mappings, and > that???s creating a cache coherency problem because there???s two > copies of dirty cache data, how does that end up with us doing a NULL > dereference? That???s the part that I???m having trouble understanding > and I worry that arguing over the exact parameters of the VM mapping > issues might be distracting us from a credible theory, with a specific > sequence of events that gets us here. > I can't answer exactly why 0 appear there, but a plausible explanation is that kernel faults in the pages mapped at some KVA which is not colored same as the user mapping. Consider what happen: bootstrap mapped init .text, but the pages are not resident. On the first touch, the tlb refill exception is reported, and since mips software pages in pmap are not filled, vm_fault() is called, which eventually called VOP_GETPAGES(). There, the UFS file buffer is bread(9). During bread(9), for mips, the buffer is mapped in KVA, and actual io occurs, which validates the pages. On one hand, the pages constituing the buffer are mapped into KVA, on the other hand, they are mapped into UVA, possibly with different color. The bread(9) operated on KVA, which validated the VIPT cache line for the KVA color. A cache line for UVA is not filled, and its content is probably zeroes. After return from vm_fault()->tlb refill, the not filled UVA line is executed. Most likely, that line is zeroed and either some function pointer is loaded as zero, or MIPS ISA executes something which causes jump to 0. Additional problem for MIPS pmap is that vfs buffer maps pages non-managed non-executable, which does not allow the pmap to even try the dcache/icache consistency trick, but this is secondary. BTW, when I developed the PCID feature for amd64, similar to ASID feature of MIPS TLB, inconsistent cache invalidations also often resulted in attempt to execute @0 or access @0, I believe for similar reasons. > It looks like we use the direct map in kernel for addresses < 256MB. > KSEG0 covers a lot of sins. KSEG0 cannot be used for buffer mappings, since buffers require consequtive mappings of several non-contiguous pages. Typical modern UFS buffer is 32k, meaning b_data is for 8 pages. And, KSEG0 access is cached with all consequences of having different color for physical address of the page and its virtual mapping. > > NetBSD has code to cope with congruent mappings that cause problems. > These look to be only on LONGSOON and MIPS3 processors with mips32 and > newer not affected by it. MIPS3 has VCE exceptions for Instructions > and Data and NetBSD has code there to cope. It also maps the pages > uncached until the multiple mappings go away. The other code it has > to cope is effectively disabled when not on these processors. It is > a twisty maze of #ifdefs, though, so maybe I???m missing something. > It does disable the socket page loaning code as well when there???s > issues. > > So, we???re back to the basic question: why does this change cause the > observed behavior, exactly? I explained above, how new UFS pager work. Old BMAP pager used transient mappings for duration of io, established by pmap_qenter(), and dissolved by pmap_qremove() before VOP_GETPAGES() returns to vm_fault(). From my reading of the code, pmap_qremove() flushed dcache always. > > >> Adding a pmap capability and turning it on for say, > >i386/amd64/arm64 > would allow for this new feature as well as the > >previous behaviour on > older platforms. > > I don't think I have the > >time to fix mips pmap to support this new > feature, so if you want > >to turn it on for all features, we should > really fix/test pmap on > >said platforms first. This is not a new feature, this is the old bug > >on that platform. > > This sort of argument is pointless: the code does something new, and > the current code doesn???t support it. It???s that simple. Trying > to label new bug / old bug, etc is pointless. It???s a bug, and > it???s gotta get fixed. Well, no, this is not the essence of the surrounding noise. > It isn???t clear to me it???s even understood > why things are happening, so even the old / new finger pointing is > premature given we don???t have a definitive cause, just a theory. I have the theory, and asked to fiddle a knob which forces UFS buffer to be dissolved at the end of io in VOP_GETPAGES() in similar manner to the BMAP pager. The result of this experiment will be telling, IMO. I cannot neither stop the discussion until the experiment is done, nor can I conduct the experiment myself. > > Warner > > >> Final comment: I'd really like to see a sort of "tested > >> on" for things like this, because it's not clear which > >> platforms/architectures it was tested on. > From owner-freebsd-mips@freebsd.org Mon Nov 14 01:06:24 2016 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2B554C3E168 for ; Mon, 14 Nov 2016 01:06:24 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-it0-x233.google.com (mail-it0-x233.google.com [IPv6:2607:f8b0:4001:c0b::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DCAFE1DDD; Mon, 14 Nov 2016 01:06:23 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-it0-x233.google.com with SMTP id b123so826763itb.0; Sun, 13 Nov 2016 17:06:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=o1eU9tImGX+ca2C1U3b6VV1+dpHipPziYloHjRwgzFQ=; b=szPpDsMIkt91uFIB3JgVeaoBgyuRy8BLeW4EQCfrwezuq7GAbqyypkizPg6L45zj2D dG++6eUC8rjmxUp4Hbbm8EpHulo7tfkDsY5q+/PKCHlyU8nFC0ELVuNpxZApqtGlTDqP m6qwIKKEsi7LJmv7a5+nKRH2X5E/6+Ipc64h22tvl/V356X26aIsgWXpuX9xCKtWc8Db o306/PPqTTjXX253rOYq1cQXK5xAW5e3ka67IsL2nRPJuqJd3JQPeM1vjMdjto7/kex9 yJFGVidUatp1sPSTrWUB69ZjLpdCNZPuR15rvXi8f0UFat1MTOTmit+3MerdWilYbcWf 6f9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=o1eU9tImGX+ca2C1U3b6VV1+dpHipPziYloHjRwgzFQ=; b=N2SiAXyNLN56x3JUMRmStcsL+AhONgDHnGTjgCZX+X91jvPkELLLjmBdDNu2AGuni7 wRZDwHiMpgP3Fu4HDB/K1FehHXj1YRGnIdIG4tcIPj9hNVOvgiRCQY4avS93TsAyZzIh 5fpQ6iGJZKHmrEMUgwP/Oa9GsN8oQD+YGJdUX5bNQKb535yn+hnk3IbFcvvnQwNuEyh1 pfaOyES8gC8syXW4UWvkbLmbhX4MkvfTCEWwIplEbUxE07pt0gj9BgfsiFLw7PC27vsg dRtuur5wXnaHGC4vjMbOL1Oa3T+GUIL9Yl2u30xmNS6beKb6WrTLHb30j/18ghedu7Ur NcCw== X-Gm-Message-State: ABUngvcAf2lYp7dmGVDJxUY5CFCKHikJVOv6FMMq424JnaFfonekvdJaxGdq+Z1Wzc0F7GZgGQ4VumrDC93bdQ== X-Received: by 10.107.192.194 with SMTP id q185mr21952520iof.129.1479085583340; Sun, 13 Nov 2016 17:06:23 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.39.134 with HTTP; Sun, 13 Nov 2016 17:06:20 -0800 (PST) Received: by 10.36.39.134 with HTTP; Sun, 13 Nov 2016 17:06:20 -0800 (PST) In-Reply-To: References: <20161113071911.GF54029@kib.kiev.ua> <20161113075557.GH54029@kib.kiev.ua> <71C512CD-0FB6-40D8-B46C-30467A245693@bsdimp.com> <20161113161548.GK54029@kib.kiev.ua> <20161113190344.GM54029@kib.kiev.ua> <50FE3B7E-8FA4-47DC-BC45-EE75B9FAFC0F@bsdimp.com> <20161113213108.GP54029@kib.kiev.ua> From: Adrian Chadd Date: Sun, 13 Nov 2016 17:06:20 -0800 Message-ID: Subject: Re: svn commit: r307626 - head/sys/ufs/ffs To: Kostik Belousov Cc: freebsd-mips@freebsd.org, "M. Warner Losh" , Warner Losh , Juli Mallett Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Nov 2016 01:06:24 -0000 I'll try it tonight. It's just unfortunate that it broke a system like this. I understand if it's supposed to just work and hasn't. But typically the onus on fixing things is on the person who broke behaviour, even if it is due to unrelated issues. I'd like to hopefully find some minimal way to keep things working as is until a better longer term solution is found, and I would also like to see some larger scale testing of things on other platforms so we can catch them earlier. Thanks, Adrian On Nov 13, 2016 1:31 PM, "Konstantin Belousov" wrote: On Sun, Nov 13, 2016 at 01:55:54PM -0700, Warner Losh wrote: > > > On Nov 13, 2016, at 12:03 PM, Konstantin Belousov wrote: > > > > On Sun, Nov 13, 2016 at 10:50:19AM -0800, Adrian Chadd wrote: > >> Ok, so after talking with others, my questions are: > >> > >> * I thought our VM was supposed to not be doing double mapping like > >> this. warner's comment on irc was: > >> > >> === > >> 13:39 < bsdimp> adrian the VM isn't supposed to do it at all. > >> 13:39 < bsdimp> adrian that is, double map on purpose. > >> 13:40 < bsdimp> though there's some exceptions to the rule > >> 13:40 < bsdimp> since kernel mappings go away in userland, and > >> userland doesn't execute while you're in kernel mode, you can do the > >> flushing > >> game in busdma to prevent most issues. > >> 13:41 < bsdimp> which is what we do. Generally, though, our VM doesn't > >> do it in-kernel. > >> === > >> > >> * is this still the case? or are there places in the VM where we are doing this? > >> * can we introduce a machdep/pmap capability check to see if aliasing > >> is allowed and if so, turn this feature on? > > This never was the case, and never will. VM establishes mappings as > > requested by the userspace and kernel, and n-times mapping for the same > > page is always legitimate. It is the pmap duty to handle that. > > User space N way mapping is rare. Can you show an actual example of when this is commonly done? Or being done in init(8)? > User space N mapping is very common with e.g. shared libraries. Writeable n-way mappings are ubiquitously used by X apps in the form of sysv shared memory segments, by database systems, in particular postgres, by qt c++ framework, and so on. If you use vi on some file, nvi mmaps the content. If any other application reads the file simultaneously with the file being opened in vi, you get double-mapping by userspace and KVA, with all consistency problems. It is rather pointless to argue about the necessity and absolute omnipresent nature of this feature. Even without the new UFS pager, apparently the MIPS port randomly corrupts user data. > > If you cared to read my previous mail, I already explained that besides > > the userspace asking for n-mapping, there is at least buffer cache which > > also maps the same page into KVA, from the times when unified buffer > > cache/page cache was introduced. Same is true for n-mapping of shared > > anon pages. > > KVA mapping is different, since that mapping goes away. And generally, > the KVA mappings aren???t active while user land mappings are active. > Here ???active??? means can change the cache. When the kernel is > executing on these single core 32-bit mips boxes, user space isn???t > creating traffic to the pages. Also, KVA mappings tend to be for pages > that we don???t actually touch in the kernel once the I/O is complete. > This is true for data pages in files. It may not be true for in-kernel > meta-data pages, and that might have changed with your commit (I > haven???t looked in enough detail to know). > This is not how 'mappings' work, at least in FreeBSD. KVA mappings are not switched away when CPU enters usermode, and VI caches for KVA indexes are not flushed on switch to usermode. So if the page is double-mapped and the colors of the mappings are different, it does not matter whether it is kernel or user VA to create problems. > But that still begs the question: why is the fault address 0 in > the original crash report. If we???re doing multiple mappings, and > that???s creating a cache coherency problem because there???s two > copies of dirty cache data, how does that end up with us doing a NULL > dereference? That???s the part that I???m having trouble understanding > and I worry that arguing over the exact parameters of the VM mapping > issues might be distracting us from a credible theory, with a specific > sequence of events that gets us here. > I can't answer exactly why 0 appear there, but a plausible explanation is that kernel faults in the pages mapped at some KVA which is not colored same as the user mapping. Consider what happen: bootstrap mapped init .text, but the pages are not resident. On the first touch, the tlb refill exception is reported, and since mips software pages in pmap are not filled, vm_fault() is called, which eventually called VOP_GETPAGES(). There, the UFS file buffer is bread(9). During bread(9), for mips, the buffer is mapped in KVA, and actual io occurs, which validates the pages. On one hand, the pages constituing the buffer are mapped into KVA, on the other hand, they are mapped into UVA, possibly with different color. The bread(9) operated on KVA, which validated the VIPT cache line for the KVA color. A cache line for UVA is not filled, and its content is probably zeroes. After return from vm_fault()->tlb refill, the not filled UVA line is executed. Most likely, that line is zeroed and either some function pointer is loaded as zero, or MIPS ISA executes something which causes jump to 0. Additional problem for MIPS pmap is that vfs buffer maps pages non-managed non-executable, which does not allow the pmap to even try the dcache/icache consistency trick, but this is secondary. BTW, when I developed the PCID feature for amd64, similar to ASID feature of MIPS TLB, inconsistent cache invalidations also often resulted in attempt to execute @0 or access @0, I believe for similar reasons. > It looks like we use the direct map in kernel for addresses < 256MB. > KSEG0 covers a lot of sins. KSEG0 cannot be used for buffer mappings, since buffers require consequtive mappings of several non-contiguous pages. Typical modern UFS buffer is 32k, meaning b_data is for 8 pages. And, KSEG0 access is cached with all consequences of having different color for physical address of the page and its virtual mapping. > > NetBSD has code to cope with congruent mappings that cause problems. > These look to be only on LONGSOON and MIPS3 processors with mips32 and > newer not affected by it. MIPS3 has VCE exceptions for Instructions > and Data and NetBSD has code there to cope. It also maps the pages > uncached until the multiple mappings go away. The other code it has > to cope is effectively disabled when not on these processors. It is > a twisty maze of #ifdefs, though, so maybe I???m missing something. > It does disable the socket page loaning code as well when there???s > issues. > > So, we???re back to the basic question: why does this change cause the > observed behavior, exactly? I explained above, how new UFS pager work. Old BMAP pager used transient mappings for duration of io, established by pmap_qenter(), and dissolved by pmap_qremove() before VOP_GETPAGES() returns to vm_fault(). From my reading of the code, pmap_qremove() flushed dcache always. > > >> Adding a pmap capability and turning it on for say, > >i386/amd64/arm64 > would allow for this new feature as well as the > >previous behaviour on > older platforms. > > I don't think I have the > >time to fix mips pmap to support this new > feature, so if you want > >to turn it on for all features, we should > really fix/test pmap on > >said platforms first. This is not a new feature, this is the old bug > >on that platform. > > This sort of argument is pointless: the code does something new, and > the current code doesn???t support it. It???s that simple. Trying > to label new bug / old bug, etc is pointless. It???s a bug, and > it???s gotta get fixed. Well, no, this is not the essence of the surrounding noise. > It isn???t clear to me it???s even understood > why things are happening, so even the old / new finger pointing is > premature given we don???t have a definitive cause, just a theory. I have the theory, and asked to fiddle a knob which forces UFS buffer to be dissolved at the end of io in VOP_GETPAGES() in similar manner to the BMAP pager. The result of this experiment will be telling, IMO. I cannot neither stop the discussion until the experiment is done, nor can I conduct the experiment myself. > > Warner > > >> Final comment: I'd really like to see a sort of "tested > >> on" for things like this, because it's not clear which > >> platforms/architectures it was tested on. > From owner-freebsd-mips@freebsd.org Mon Nov 14 01:11:08 2016 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE823C3E1DC for ; Mon, 14 Nov 2016 01:11:08 +0000 (UTC) (envelope-from juli@clockworksquid.com) Received: from mail-it0-x22b.google.com (mail-it0-x22b.google.com [IPv6:2607:f8b0:4001:c0b::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 758851F76 for ; Mon, 14 Nov 2016 01:11:08 +0000 (UTC) (envelope-from juli@clockworksquid.com) Received: by mail-it0-x22b.google.com with SMTP id q124so73738026itd.1 for ; Sun, 13 Nov 2016 17:11:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=clockworksquid.com; s=google; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=/QO601rrW06fjW4CJ3nk49y+QMmNGsy+7ito1B3y99E=; b=ijPV4kODjjzwJczwWHiEHUbfYbwnk/hcWxCKjq/6FNHyknI/ON7eriRLpjRyed5eY+ A5W6v5+8uYwTJxtKF/jNtI8yx+u5Gygian7NmecK/1A3Z7wixpkYPb0FkJm2K3JWSsNT LDVXTm0fQX4i3oYFfgZ/Ecc0A6+6wwFUtcXXI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd-org.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=/QO601rrW06fjW4CJ3nk49y+QMmNGsy+7ito1B3y99E=; b=MaStqaFmq8FvPoVhhVBs/Ou+4RnGHgwfgivwIgQylD0Vimidv8TfIHMwa5sUs0sjPg okCucq3xx2XBn5LjAX3RXi8d6DMWazBYt+9A+XmIew9g1SFCowVyN7pvuH/OoKsEVq8o n5OYVhSlv7Op+eqOGGhoq6V1Ay6qN/+XvtS2Rjmg/Rst0SGFiX9k6fQ4wXGkw/Mjv2pz zCTB9dYC0wKLu1j8urpruKjvOjQ/1X9ScW8ZTuPo11nvio+C4PAZzfRwYfzC8hJGcc58 DcL/g2uqf+virbQSCGnrcdouDuFJibCsg1M8swNfljZ0NQCzPzzhiKXZf1r7nLZ2YrtE wDtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=/QO601rrW06fjW4CJ3nk49y+QMmNGsy+7ito1B3y99E=; b=e4hnkWbG1ADoTxMC6OA1MKhMfvn2LCtsH0ddN6WVn2HkYYyH9rnA9UcD/WJnpCCN/W 1joT2BEffssQEQ47VWiXz0CjLlIIa8mdZGqiM1EwDdrd8Qi1AJxSIsJill8N+yAOvGod z8TMjck6+9rx5ZBZun5noa1XgU/BzwtGesZCCchHg67MvW5M6GgS7Fjf48YTaqh7zHv2 7n49bMoUhItoAqTLqQnURQYvYim1DxqHwMF4GsShiOhpM5IiNWPk3rNTgoD78UbxayEJ 9u1xOrkvDS1Fswd5TlugK4OGXFa5KPNTExj7x3pbIvtnqxyPl921AbT3ACeWY+8hChbV FjSA== X-Gm-Message-State: ABUngvexqRzFskJh6GUBasxc/qZPpI9+QcHdC7Of5fELEyGS0viII8BJGJBT2XtpdDZBFoolZRarmUywVwCcVw== X-Received: by 10.107.59.9 with SMTP id i9mr25166388ioa.176.1479085867870; Sun, 13 Nov 2016 17:11:07 -0800 (PST) MIME-Version: 1.0 Sender: juli@clockworksquid.com Received: by 10.79.141.88 with HTTP; Sun, 13 Nov 2016 17:10:47 -0800 (PST) In-Reply-To: References: <20161113071911.GF54029@kib.kiev.ua> <20161113075557.GH54029@kib.kiev.ua> <71C512CD-0FB6-40D8-B46C-30467A245693@bsdimp.com> <20161113161548.GK54029@kib.kiev.ua> <20161113190344.GM54029@kib.kiev.ua> <50FE3B7E-8FA4-47DC-BC45-EE75B9FAFC0F@bsdimp.com> <20161113213108.GP54029@kib.kiev.ua> From: Juli Mallett Date: Sun, 13 Nov 2016 17:10:47 -0800 X-Google-Sender-Auth: JKPgoswhOoOZTBA5Oyh9gBroEdI Message-ID: Subject: Re: svn commit: r307626 - head/sys/ufs/ffs To: Adrian Chadd Cc: Kostik Belousov , "freebsd-mips@FreeBSD.org" , "M. Warner Losh" , Warner Losh Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Nov 2016 01:11:08 -0000 On Sun, Nov 13, 2016 at 5:06 PM, Adrian Chadd wrote: > I'll try it tonight. > > It's just unfortunate that it broke a system like this. I understand if it's > supposed to just work and hasn't. But typically the onus on fixing things is > on the person who broke behaviour, even if it is due to unrelated issues. This is not true. Networking crashes caused by NIC driver bugs do not get treated that way. > I'd like to hopefully find some minimal way to keep things working as is > until a better longer term solution is found, and I would also like to see > some larger scale testing of things on other platforms so we can catch them > earlier. On older kernels you should be able to test sharing memory between processes, or multiple mmaps of a file, or similar, I guess? From owner-freebsd-mips@freebsd.org Fri Nov 18 19:59:05 2016 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9EC16C48843 for ; Fri, 18 Nov 2016 19:59:05 +0000 (UTC) (envelope-from jimukyoku@kpa.ees.kyushu-u.ac.jp) Received: from eesserv2.ees.kyushu-u.ac.jp (ckt.ees.kyushu-u.ac.jp [133.5.152.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ED871159B for ; Fri, 18 Nov 2016 19:59:04 +0000 (UTC) (envelope-from jimukyoku@kpa.ees.kyushu-u.ac.jp) Received: from [216.45.54.23] (216.45.54.23.static.greencloudvps.com [216.45.54.23] (may be forged)) (authenticated bits=0) by eesserv2.ees.kyushu-u.ac.jp (8.14.4/8.14.4) with ESMTP id uAIJWGQB012443 for ; Sat, 19 Nov 2016 04:59:02 +0900 Message-Id: <201611181959.uAIJWGQB012443@eesserv2.ees.kyushu-u.ac.jp> MIME-Version: 1.0 Subject: Google Award Notification To: freebsd-mips@freebsd.org From: "Google Inc." Date: Fri, 18 Nov 2016 11:59:06 -0800 Reply-To: mikhaylovichbrinsergey@gmail.com Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Description: Mail message body X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Nov 2016 19:59:05 -0000 Dear Google User, Congratulation for being selected as a winner, find attached email with fur= ther information. Sergey M. Brin, Co-Founder/Foreign Bureau Administrator.