From owner-freebsd-current@FreeBSD.ORG Mon Dec 16 23:48:23 2013 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A7ACFEC2 for ; Mon, 16 Dec 2013 23:48:23 +0000 (UTC) Received: from mail-we0-f172.google.com (mail-we0-f172.google.com [74.125.82.172]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 336991B32 for ; Mon, 16 Dec 2013 23:48:22 +0000 (UTC) Received: by mail-we0-f172.google.com with SMTP id w62so5391535wes.3 for ; Mon, 16 Dec 2013 15:48:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=I0cmIXsoAWvPb7WVFAHv3rwbBYvQ4EhARv2UD5UFwIo=; b=azWuhTsj3ctCGbRB+I6gMRx/0ufPzaY2ueyWbNDSqav6FMgh6wVgTZ1onnEcomlvXd DQ6p/cMZKZ8sEp/QoI1FBni2Q7tCEE5eoJD65lb021wLVFGY8ljbE+WkoLqkf9N2XlKE Yeo2+wHy6x64h7C4Rnd4RNAthF5ufN+pPQ13UKItEKYBNbZrNVGNmTMzZmztdF899oIO mTXlBoGL69k3pSf2B6OHZZkNRK6OapOw3O48oMRemo9nGNEOPeKO/Ar+RODUuFkY0SlZ 4otaKK7xbqjjf4cKyzM2qjyLzrYP1Q3Sxn1LS28GpQJPwGj4HWtm7UyrwA9JBd4yEwZe FG4Q== X-Gm-Message-State: ALoCoQkv/YjyGCdYShbh8QLoentkvWsC29Z4WFfutSubR/Yhyvjsvgbx9v8Qh3mq0rwk4Qnuxhel MIME-Version: 1.0 X-Received: by 10.194.11.38 with SMTP id n6mr15499419wjb.25.1387237306195; Mon, 16 Dec 2013 15:41:46 -0800 (PST) Sender: ray@ddteam.net Received: by 10.216.193.193 with HTTP; Mon, 16 Dec 2013 15:41:46 -0800 (PST) In-Reply-To: <52AEF86B.5080600@FreeBSD.org> References: <52AEF86B.5080600@FreeBSD.org> Date: Tue, 17 Dec 2013 01:41:46 +0200 X-Google-Sender-Auth: Cc1q8vtQTN-KfiCOY1fkCYdzUTQ Message-ID: Subject: Re: r259286 panic From: Aleksandr Rybalko To: Alexander Motin Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Eitan Adler , freebsd-current Current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Dec 2013 23:48:23 -0000 Hi guys! I've investigate problem a bit. And can say that callout initialized with callout_int(), w/o mpsafe flag: callout_init(&vw->vw_proc_dead_timer, 0); [sys/dev/vt/vt_core.c:1714] And callout_init did not set CALLOUT_RETURNUNLOCKED flag, and assign it's lock object to Giant, but where Giant lost on exit from callout I dunno :) seems some bug somewhere much deep. Eitan, do this 100% reproducible? If so, can you please try to replace callout_init(&vw->vw_proc_dead_timer, 0); with callout_init(&vw->vw_proc_dead_timer, CALLOUT_MPSAFE); at [sys/dev/vt/vt_core.c:1714] ? Thanks! 2013/12/16 Alexander Motin > On 15.12.2013 22:22, Eitan Adler wrote: > >> On Sun, Dec 15, 2013 at 3:23 AM, Eitan Adler >> wrote: >> >>> FreeBSD gravity.local 11.0-CURRENT FreeBSD 11.0-CURRENT #7 r259286: >>> Fri Dec 13 00:33:37 EST 2013 >>> eitan@gravity.local:/usr/obj/usr/src/sys/EADLER amd64 >>> >>> Complete textdump here: http://people.freebsd.org/~ >>> eadler/files/core.txt.1 >>> >>> My kernel is built with complete debug symbols, INVARIANTS, ddb, etc. >>> but no WITNESS. >>> >>> I have vt and vt_vga enabled but not sc and vga (newcons, but not >>> syscons). >>> >> >> >> Replying to myself with more data: >> >> gdb$ list kern_timeout.c >> Can't find member of namespace, class, struct, or union named >> "kern_timeout.c" >> Hint: try 'kern_timeout.c or 'kern_timeout.c >> (Note leading single quote.) >> gdb$ list kern_timeout.c:700 >> 695 lastfunc = c_func; >> 696 } >> 697 #endif >> 698 CTR1(KTR_CALLOUT, "callout %p finished", c); >> 699 if ((c_flags & CALLOUT_RETURNUNLOCKED) == 0) >> 700 class->lc_unlock(c_lock); >> 701 skip: >> 702 CC_LOCK(cc); >> 703 KASSERT(cc->cc_exec_entity[direct].cc_curr == c, >> ("mishandled cc_curr")); >> 704 cc->cc_exec_entity[direct].cc_curr = NULL; >> gdb$ p c >> $1 = (struct callout *) 0xffffffff812121f8 >> gdb$ p *c >> $2 = { >> c_links = { >> le = { >> le_next = 0xfffffe0005f00318, >> le_prev = 0xffffffff813aa690 >> }, >> sle = { >> sle_next = 0xfffffe0005f00318 >> }, >> tqe = { >> tqe_next = 0xfffffe0005f00318, >> tqe_prev = 0xffffffff813aa690 >> } >> }, >> c_time = 0x2c0d9170f2f51, >> c_precision = 0xeffffeea, >> c_arg = 0xffffffff81212148, >> c_func = 0xffffffff8067e6e0 , >> c_lock = 0x0, >> c_flags = 0x80, >> c_cpu = 0x0 >> } >> >> >> From the 'vt_switch_timer' function it appears that something is wrong >> with vt. >> >> >> In addition avg@ mentioned that he wonders why class->lc_lock(c_lock, >> ...) is protected by if (c_lock != NULL) but class->lc_unlock(c_lock) >> does not have that guard. >> > > It worked so far because callout_init() sets CALLOUT_RETURNUNLOCKED flag > if there is no callout lock. I am not sure whether we should better add > check or assertion to _callout_init_lock(). > > So either VT passes something odd/NULL pointer to callout_init_mtx(), or > something overwrites the callout structure after scheduling the callout. > > -- > Alexander Motin > -- WBW ------- Rybalko Aleksandr aka Alex RAY D-Link.ua