From owner-freebsd-mips@freebsd.org Tue Nov 17 07:44:17 2015 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB565A315DD for ; Tue, 17 Nov 2015 07:44:17 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-ig0-x233.google.com (mail-ig0-x233.google.com [IPv6:2607:f8b0:4001:c05::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7E38319C1; Tue, 17 Nov 2015 07:44:17 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by igcph11 with SMTP id ph11so73838254igc.1; Mon, 16 Nov 2015 23:44:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=V9IpKzCaXcUdSDcnSg/wxurNykD9kqzpA+hIrZuk2ho=; b=g+wsYz1LhlnUHAoBEP2nPQFn3TO3eQvfR4Ao0tWn4HXsuGKsr5SFoBSe4hOLQGuMcU 52ek1eWTH9JW5MM+TJoy+ZldfaIbLht0JG2sYpn/v8mUPpPtAeibwZ+SfYBbLmnKl3Sv sho2VOUnDbgKPAzNTFlX54Ykz6s2oEsx6lDJdwXUdtzXG/A2WreKeAscZhh3Q9c2Bwnt 2JJV8zpCjIJdvvY7on9Dif20aTkskNYCxRy2RaJZuAoLrgYaDN5GN6VnZ+NdjF4C8sV8 oxdV9LqRbvsevfX+azQS/yW86o8dQ110Xl4+WKUS5LjtG3MTxR39+A9SiCk6+Nbbthfl lopg== MIME-Version: 1.0 X-Received: by 10.50.6.36 with SMTP id x4mr496408igx.61.1447746257020; Mon, 16 Nov 2015 23:44:17 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.36.217.196 with HTTP; Mon, 16 Nov 2015 23:44:16 -0800 (PST) In-Reply-To: <564512A8.1000006@grosbein.net> References: <56348063.3090508@grosbein.net> <56370E1D.3040801@grosbein.net> <563F5630.2000407@grosbein.net> <563F6F6F.1010807@grosbein.net> <563F91A8.9080702@grosbein.net> <5640C0FD.2040803@grosbein.net> <5640F315.5020303@grosbein.net> <56410214.3070901@grosbein.net> <564231DE.7090308@grosbein.net> <56423D82.5030203@grosbein.net> <564512A8.1000006@grosbein.net> Date: Mon, 16 Nov 2015 23:44:16 -0800 X-Google-Sender-Auth: aSoHbw08lAoh484LpktJVpUrbQY Message-ID: Subject: Re: USB-related problem From: Adrian Chadd To: Eugene Grosbein Cc: benno , Jeff Roberson , Konstantin Belousov , "freebsd-mips@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2015 07:44:17 -0000 hiya, any updates? I caan't trigger this one myself :( -a On 12 November 2015 at 14:28, Eugene Grosbein wrote: > On 11.11.2015 01:57, Adrian Chadd wrote: >> + benno, jeff, kib >> >> cool, so now we know where syncer is hanging. nice work! >> >> That syncer loop that calls it waiting for the list to be empty is >> also a bit suss; it looks like it could also get stuck in a loop and >> never yield. > > I've used new state variables spread all over the kernel > (plus KTR facility) to track this down and came to a result I cannot explain. > > First, here is call sequence leading to kernel hang, always the same: > > sys/kern/vfs_subr.c, sync_vnode() -> VOP_FSYNC(vp, MNT_LAZY, td) -> VOP_FSYNC_APV() > in generated vnode_if.c in the kernel compile directory. > > Here we have KTR_START3(KTR_VOP, ...) that is latest record that DDB shows for > "show ktr /v". There should be KTR_STOP3(KTR_VOP, "VOP", ...) recorded after, but it is not. > In fact, execution proceeds to vop_fsync(a) and never returns, precisely: > > vop_fsync(a) -> sys/fs/devfs/devfs_vnops.c, devfs_fsync() -> > "return (vop_stdfsync(ap))" - never returns. > > It reaches sys/kern/vfs_default.c, vop_stdfsync() and hangs exactly at this KASSERT: > https://svnweb.freebsd.org/base/head/sys/kern/vfs_default.c?annotate=288451#l683 > > I've defined another global variable "volatile unsigned vopsfsyncstate", > set it to 15 just before this KASSERT and to 126 just after the KASSERT > and it always equals to 15 when kernel hangs. I tested kernel compiled > with and without WITNESS (and WITHNESS_SKIPSPIN), no changes. > > I do not understand how and why does it hang here but that's a fact. > Here I'm stuck. > > Eugene Grosbein >