From owner-svn-src-all@FreeBSD.ORG  Thu Nov 27 20:28:19 2014
Return-Path: <owner-svn-src-all@FreeBSD.ORG>
Delivered-To: svn-src-all@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id 5096C1D0;
 Thu, 27 Nov 2014 20:28:19 +0000 (UTC)
Received: from bsdpad.com (xc1.bsdpad.com [195.154.136.64])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 100189A0;
 Thu, 27 Nov 2014 20:28:18 +0000 (UTC)
Received: from localhost ([127.0.0.1] helo=bsdpad.com)
 by bsdpad.com with smtp (Exim 4.83 (FreeBSD))
 (envelope-from <br@bsdpad.com>)
 id 1Xu5d4-000M1a-F5; Thu, 27 Nov 2014 20:25:30 +0000
Received: by bsdpad.com (nbSMTP-1.00) for uid 1001
 br@bsdpad.com; Thu, 27 Nov 2014 20:25:30 +0000 (GMT)
Date: Thu, 27 Nov 2014 20:25:30 +0000
From: Ruslan Bukin <br@freebsd.org>
To: Adrian Chadd <adrian@freebsd.org>
Subject: Re: svn commit: r274914 - in head/sys: kern sys ufs/ffs vm
Message-ID: <20141127202530.GA84608@bsdpad.com>
References: <20141124180356.GA47782@bsdpad.com>
 <20141124192902.GQ47144@FreeBSD.org>
 <20141124211620.GA49975@bsdpad.com>
 <20141125073145.GT47144@FreeBSD.org>
 <20141125122754.GA57252@bsdpad.com>
 <20141125152922.GX47144@FreeBSD.org>
 <20141125173830.GA60905@bsdpad.com>
 <CAJ-Vmo=vaLj6gR7+Z2r_a+__DcsAjBEjgWGMSsQedezhi7LNKg@mail.gmail.com>
 <20141127183629.GA83207@bsdpad.com>
 <CAJ-Vmon9OMxvxAp9saOoCEQL+7npCt--Q4cyvMHFqDoHFY5iUg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <CAJ-Vmon9OMxvxAp9saOoCEQL+7npCt--Q4cyvMHFqDoHFY5iUg@mail.gmail.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
Cc: Konstantin Belousov <kostikbel@gmail.com>,
 "svn-src-head@freebsd.org" <svn-src-head@freebsd.org>,
 "svn-src-all@freebsd.org" <svn-src-all@freebsd.org>,
 Gleb Smirnoff <glebius@freebsd.org>,
 "src-committers@freebsd.org" <src-committers@freebsd.org>
X-BeenThere: svn-src-all@freebsd.org
X-Mailman-Version: 2.1.18-1
Precedence: list
List-Id: "SVN commit messages for the entire src tree \(except for &quot;
 user&quot; and &quot; projects&quot; \)" <svn-src-all.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/svn-src-all>,
 <mailto:svn-src-all-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-all/>
List-Post: <mailto:svn-src-all@freebsd.org>
List-Help: <mailto:svn-src-all-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/svn-src-all>,
 <mailto:svn-src-all-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 27 Nov 2014 20:28:19 -0000

On Thu, Nov 27, 2014 at 11:28:31AM -0800, Adrian Chadd wrote:
> On 27 November 2014 at 10:36, Ruslan Bukin <br@freebsd.org> wrote:
> > Hi,
> >
> > There is a theory that a problem is related to CPU caches, and it can be BERI-cpu
> > specific only.
> >
> > And sorry I didn't realized virtio was still in the kernel when I was trying
> > MDROOT (we do dcache operations in virtio).
> >
> > The workaround I found is to disable all the instruction cache invalidation calls
> > (there are a few calls on cpu startup only).
> >
> 
> Hm, can we test this on real MIPS hardware, just to be sure?
> 
> I can spin this up on MIPS24k/MIPS74k hardware early next week.
> 

It can be helpfull.

It is easy to test, just insert mips_dcache_wbinv_all() call in some driver and ensure mips_cache_ops.mco_sdcache_wbinv_all is not NULL on your hardware, so the dcache invalidation call will invalidate both primary and secondary data caches.

With virtio we are calling mips_dcache_wbinv_all() a few times per second (the good idea here is supposed to be more selective with physical addresses invalidation, but it should not hurt to data consistent as it do both writeback and flush)

Ruslan