From owner-freebsd-current@freebsd.org Wed Aug 30 23:54:09 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9C397E0D125 for ; Wed, 30 Aug 2017 23:54:09 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: from mail-oi0-x22a.google.com (mail-oi0-x22a.google.com [IPv6:2607:f8b0:4003:c06::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6ECCD7DA for ; Wed, 30 Aug 2017 23:54:09 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: by mail-oi0-x22a.google.com with SMTP id w10so63498858oie.1 for ; Wed, 30 Aug 2017 16:54:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kev009.com; s=google; h=mime-version:from:date:message-id:subject:to:cc; bh=LOnSg4ftZGdcYCgIWSQ/Z12w/jBnauSSs4EU6dn/cGA=; b=OjiTC0ietVtG3ZIqzJDSRLKJ+0qUxkzLpk1tCGIO2ms9BBv7GzlVLEvrWkBZVjA71S RGYfUpynjW75QAgQS2W0tciPe1DNbtCl3NQmP+ba0DADEU1HXuJ9eEWA2m6RGcyyey5h v1qwPl7TUG4MwcXkgoAQF2UZzGE6ma5K+LrxY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=LOnSg4ftZGdcYCgIWSQ/Z12w/jBnauSSs4EU6dn/cGA=; b=fPBzlic2qZl6PJ3WzWOmZsRgthJvuOEUC6ytWcsXwMdIHN9E6CXsGLtu2DzBVYvngM 2o68R4qFIuHMP1Ti5P85osG/arp8mVJ0tvtzUQC/L+AZ4zxPtclk3nVYKM4Kt/ewnXKn wF5Hws/x523auD1oNnrpO4xMLJF9M4vItlw7QYYzmuZz6OGlmL76IThXxiNBjYFZtMnR OASEGOA2LgwemBFUUsvBw6xrVtrzxArkuc5tbxOWSau5YLrs7/kss5w2A94+uar5uLLl ljuAUddIbDMGNF5jV2UOO4hBs+FojYTkhFJYZC98x4F0uFlQR6SBXsm5CiHv9BaMyjvy Yk1w== X-Gm-Message-State: AHYfb5i4uKb1SupC4WTUlxp5E1Zj+PFE5+wZJjN9tjGh4X9sanRPm4vU AWH+mu6bDPgUqMd34vD4RBb3MBZa5Yv9y5Y= X-Received: by 10.202.252.201 with SMTP id a192mr3401238oii.273.1504137248422; Wed, 30 Aug 2017 16:54:08 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.57.165 with HTTP; Wed, 30 Aug 2017 16:54:07 -0700 (PDT) From: Kevin Bowling Date: Wed, 30 Aug 2017 16:54:07 -0700 Message-ID: Subject: !EARLY_AP_STARTUP and -CURRENT To: freebsd-current@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Aug 2017 23:54:09 -0000 I'm dealing with a shit sandwich right now where the mps(4) or cam_da reorders drives on a few thousand legacy MBR machines I have (and I can't easily install glabel ATM), and !EARLY_AP_STARTUP seems to have regressed. I'd like to be able to run w/o EARLY_AP_STARTUP right quick so I can take a more leisurely approach to fixing mps(4) boot probe correctly (freebsd-scsi@ has that thread). With WITNESS and !EARLY_AP_STARTUP I hit an assert in sched_setpreempt in kern/sched_ule.c 100% of the time. Here are a couple invocations, with oddness around a different CPU holding the curthread lock but somehow a different AP is runnable in the function: Tracing pid 11 tid 100020 td 0xfffff80128cd1560 kdb_enter() at kdb_enter+0x3b/frame 0xfffffe3e653dcc10 vpanic() at vpanic+0x1b9/frame 0xfffffe3e653dcc90 panic() at panic+0x43/frame 0xfffffe3e653dccf0 __mtx_assert() at __mtx_assert+0xb4/frame 0xfffffe3e653dcd00 sched_add() at sched_add+0x152/frame 0xfffffe3e653dcd40 intr_event_schedule_thread() at intr_event_schedule_thread+0xca/frame 0xfffffe3e653dcd80 swi_sched() at swi_sched+0x6c/frame 0xfffffe3e653dcdc0 softclock_call_cc() at softclock_call_cc+0x155/frame 0xfffffe3e653dce70 callout_process() at callout_process+0x1f9/frame 0xfffffe3e653dcef0 handleevents() at handleevents+0x1a4/frame 0xfffffe3e653dcf30 cpu_initclocks_ap() at cpu_initclocks_ap+0xc8/frame 0xfffffe3e653dcf60 init_secondary_tail() at init_secondary_tail+0x1e3/frame 0xfffffe3e653dcf90 init_secondary() at init_secondary+0x2b3/frame 0xfffffe3e653dcff0 db> show thread 0xfffff80128cd1560 Thread 100020 at 0xfffff80128cd1560: proc (pid 11): 0xfffff80128cb5000 name: idle: cpu17 stack: 0xfffffe3e5cd88000-0xfffffe3e5cd8bfff flags: 0x40024 pflags: 0x200000 state: CAN RUN priority: 255 container lock: sched lock 0 (0xffffffff81c39800) db> show lock 0xffffffff81c39800 class: spin mutex name: sched lock 0 flags: {SPIN, RECURSE} state: {OWNED} owner: 0xfffff80128cca000 (tid 100017, pid 11, "idle: cpu14") db> bt Tracing pid 11 tid 100021 td 0xfffff80128cd2000 kdb_enter() at kdb_enter+0x3b/frame 0xfffffe3e655e4c10 vpanic() at vpanic+0x1b9/frame 0xfffffe3e655e4c90 panic() at panic+0x43/frame 0xfffffe3e655e4cf0 __mtx_assert() at __mtx_assert+0xb4/frame 0xfffffe3e655e4d00 sched_add() at sched_add+0x152/frame 0xfffffe3e655e4d40 intr_event_schedule_thread() at intr_event_schedule_thread+0xca/frame 0xfffffe3e655e4d80 swi_sched() at swi_sched+0x6c/frame 0xfffffe3e655e4dc0 softclock_call_cc() at softclock_call_cc+0x155/frame 0xfffffe3e655e4e70 callout_process() at callout_process+0x1f9/frame 0xfffffe3e655e4ef0 handleevents() at handleevents+0x1a4/frame 0xfffffe3e655e4f30 cpu_initclocks_ap() at cpu_initclocks_ap+0xc8/frame 0xfffffe3e655e4f60 init_secondary_tail() at init_secondary_tail+0x1e3/frame 0xfffffe3e655e4f90 init_secondary() at init_secondary+0x2b3/frame 0xfffffe3e655e4ff0 db> show thread 0xfffff80128cd2000 Thread 100021 at 0xfffff80128cd2000: proc (pid 11): 0xfffff80128cb6000 name: idle: cpu18 stack: 0xfffffe3e5cf17000-0xfffffe3e5cf1afff flags: 0x40024 pflags: 0x200000 state: CAN RUN priority: 255 container lock: sched lock 0 (0xffffffff81c39800) db> show lock 0xffffffff81c39800 class: spin mutex name: sched lock 0 flags: {SPIN, RECURSE} state: {OWNED} owner: 0xfffff80128cdb560 (tid 100028, pid 11, "idle: cpu25") Regards, Kevin