From owner-freebsd-sparc64@FreeBSD.ORG Mon Oct 24 20:52:19 2011 Return-Path: Delivered-To: freebsd-sparc64@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 945621065670 for ; Mon, 24 Oct 2011 20:52:19 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) by mx1.freebsd.org (Postfix) with ESMTP id 028E38FC08 for ; Mon, 24 Oct 2011 20:52:18 +0000 (UTC) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.14.4/8.14.4/ALCHEMY.FRANKEN.DE) with ESMTP id p9OKqE2f068335; Mon, 24 Oct 2011 22:52:14 +0200 (CEST) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.14.4/8.14.4/Submit) id p9OKqEMw068334; Mon, 24 Oct 2011 22:52:14 +0200 (CEST) (envelope-from marius) Date: Mon, 24 Oct 2011 22:52:14 +0200 From: Marius Strobl To: Peter Jeremy Message-ID: <20111024205214.GD45230@alchemy.franken.de> References: <20110831212458.GA25926@server.vk2pj.dyndns.org> <20110902153206.GR40781@alchemy.franken.de> <20111006120411.GA903@alchemy.franken.de> <20111011030529.GA4093@server.vk2pj.dyndns.org> <20111011205543.GA81376@alchemy.franken.de> <20111013035648.GA54190@server.vk2pj.dyndns.org> <20111013184224.GG39118@alchemy.franken.de> <20111018042646.GA18863@server.vk2pj.dyndns.org> <20111018172718.GT39118@alchemy.franken.de> <20111021221705.GD45938@server.vk2pj.dyndns.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111021221705.GD45938@server.vk2pj.dyndns.org> User-Agent: Mutt/1.4.2.3i Cc: freebsd-sparc64@freebsd.org Subject: Re: 'make -j16 universe' gives SIReset X-BeenThere: freebsd-sparc64@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to the Sparc List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Oct 2011 20:52:19 -0000 On Sat, Oct 22, 2011 at 09:17:05AM +1100, Peter Jeremy wrote: > On 2011-Oct-18 19:27:18 +0200, Marius Strobl wrote: > >On Tue, Oct 18, 2011 at 03:26:46PM +1100, Peter Jeremy wrote: > >> On 2011-Oct-13 20:42:25 +0200, Marius Strobl wrote: > >> >On Thu, Oct 13, 2011 at 02:56:48PM +1100, Peter Jeremy wrote: > >> >> Unfortunately, I can't get a crashdump because dumpon(8) doesn't like > >> >> my Solaris swap partitions: > >> >> GEOM_PART: Partition 'da0b' not suitable for kernel dumps (wrong type?) > >> >> GEOM_PART: Partition 'da6b' not suitable for kernel dumps (wrong type?) > >> >> No suitable dump device was found. > >> >> > >> >> I did write a patch for that but took it out during some earlier > >> >> testing to get back to stock code. It looks like I didn't PR it > >> >> either so I will do that when I get some time. > >> > >> I've resurrected that patch (and will send-pr it later). > > Thanks for committing it. Thanks for the patch! > > >Hrm, AFAICT this would mean that the _mtx_obtain_lock(), which boils > >down to a atomic_cmpset_acq_ptr(), in _mtx_trylock() didn't work as > >expected, I currently can't think of a good reason why that could > >happen though. The assembly generated for that code also looks just > >fine. Have you run the workload which is triggering this before? It > >would be interesting to know whether it also happens with SCHED_4BSD > >with current sources, pre-r226054 and pre-r225889 if the machine > >previously survived that load. > > It was running 6 parallel -j16 buildworlds. I switched to SCHED_4BSD > and haven't been able to reproduce it - even with a pile of added > "sysctl sysctl vm.vmtotal". I haven't tried rolling back to an > earlier kernel. Well, actually this is good as it means that this problem isn't a regression of r225889 or r226054, which worried me the most. Could you please test whether the following patch makes a difference with SCHED_ULE? http://people.freebsd.org/~marius/sparc64_curthread_preemption.diff > > >Have you enabled PREEMPTION by chance? > > That was using GENERIC and only changing the scheduler. > > >The other thing that worries me is that it could be a silicon bug, > >especially since that machine also has that issue of issuing stale > >vector interrupts along with a state in which it traps even on > >locked TLB entries, which isn't mentioned in the public erratum ... > > I've had a rummage around in the OpenSolaris sources and nothing > jumps out at me. (Actually, I can't find any special case code > that looks like it addresses silicon bugs in Jaguar). Well, as I've learnt the hard way that doesn't mean that much; the OpenSolaris source doesn't resembler a complete OS in the first place as vital parts are missing, workarounds for silicon bugs aren't necessarily marked as such, due to the nature of some CPU bugs it's more likely that they've worked around them in their compiler to not emit the offending instruction (sequence), FreeBSD uses some parts of the hardware differently than (Open)Solaris so they might have never hit it in the first place, etc... > > One other thing is that I'm getting lots of isp watchdog timeouts: > (da4:isp0:0:4:0): first watchdog (handle 0x5cf020f3) timed out- deferring for grace period > (da4:isp0:0:4:0): first watchdog (handle 0x5cf1206d) timed out- deferring for grace period > (da4:isp0:0:4:0): first watchdog (handle 0x5cf2203a) timed out- deferring for grace period > isp0: isp_watchdog: timeout for handle 0x5cad2046 > (da4:isp0:0:4:0): FIN dl16384 resid 0 CDB=0x2a 0x00 0x0f 0xdd 0xe8 0xe0 0x00 0x00 0x20 0x00 STS 0x0 XS_ERR=0xb > isp0: bad request handle 0x5cad2046 (iocb type 0x3) > isp0: isp_watchdog: timeout for handle 0x5cdb20cb > (da4:isp0:0:4:0): FIN dl16384 resid 0 CDB=0x2a 0x00 0x0f 0xe3 0xa8 0x00 0x00 0x00 0x20 0x00 STS 0x0 XS_ERR=0xb > isp0: isp_watchdog: timeout for handle 0x5cdc2059 > (da4:isp0:0:4:0): FIN dl16384 resid 0 CDB=0x2a 0x00 0x0f 0xe3 0xa8 0x20 0x00 0x00 0x20 0x00 STS 0x0 XS_ERR=0xb > isp0: isp_watchdog: timeout for handle 0x5cdd2020 > (da4:isp0:0:4:0): FIN dl16384 resid 0 CDB=0x2a 0x00 0x0f 0xe3 0xa8 0x40 0x00 0x00 0x20 0x00 STS 0x0 XS_ERR=0xb > isp0: bad request handle 0x5cdb20cb (iocb type 0x3) > isp0: bad request handle 0x5cdc2059 (iocb type 0x3) > isp0: bad request handle 0x5cdd2020 (iocb type 0x3) > (da4:isp0:0:4:0): first watchdog (handle 0x6b9520bb) timed out- deferring for grace period > (da4:isp0:0:4:0): first watchdog (handle 0x6b96200e) timed out- deferring for grace period > > Any ideas on that? > Not really; I also see such timeouts on a recently acquired Qlogic HBA based 280R when pushing it but not when using mpt(4), sym(4) etc so I highly doubt that these are caused by a MD problem that causes lost interrupts for example. Also I can't remember having seen such timeouts back when I was testing with B1K/B2K (which basically use the same mainboard as 280R), so this actually _could_ be a regression in isp(4). However, at least in my case no real problem arose from isp(4) timeouts so far. You could try to ask mjacob@ what he thinks about these. Unfortunately, he no longer seems to maintain isp(4) that actively. Marius