From owner-freebsd-stable@FreeBSD.ORG Tue Jun 2 23:10:16 2015 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EBCCF358 for ; Tue, 2 Jun 2015 23:10:16 +0000 (UTC) (envelope-from jim.harris@gmail.com) Received: from mail-qk0-x22e.google.com (mail-qk0-x22e.google.com [IPv6:2607:f8b0:400d:c09::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A70D61B4E for ; Tue, 2 Jun 2015 23:10:16 +0000 (UTC) (envelope-from jim.harris@gmail.com) Received: by qkhg32 with SMTP id g32so110232160qkh.0 for ; Tue, 02 Jun 2015 16:10:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=lux/56UNt9Vf8U67b55I7fPtIGGxMzFhZtFkq5DHiIQ=; b=WpQyn4aWwla+PZAoCusq1nkKgjO93ZFuKLGzLFozusJAlhDS6G1oaRsTZqtaoHPIdk +vM6JOlm4BexEMrY1bN48m9p5aSN0yiMjoB+WkpZ6rnIMtN4sJHiUNbLqcBdgzNK1S+D WA/m8PFQDH1Ygh9ZI/pfPVjgk0Qve8K7ye4/1BqjjFf+zosbv6qzc9QkYHnGYNYTZPTb lthFiGvXUVgVems4Dp3PygWX4ouQlJvDzJmdIpNlP5kVPP3oCuKYCvUJk0K9PudLTTem 88T0I3PUpZOqEcnWGlKL3HwIt09dUCE51ZSpwgBe4PeCG+GSJzYB4so+C9GQJ4+yklre 0JJQ== MIME-Version: 1.0 X-Received: by 10.229.188.137 with SMTP id da9mr33664147qcb.27.1433286615520; Tue, 02 Jun 2015 16:10:15 -0700 (PDT) Received: by 10.229.191.4 with HTTP; Tue, 2 Jun 2015 16:10:15 -0700 (PDT) In-Reply-To: <90B2D392-01FD-415A-B3D9-3CEDFC8373C4@smkelly.org> References: <90B2D392-01FD-415A-B3D9-3CEDFC8373C4@smkelly.org> Date: Tue, 2 Jun 2015 16:10:15 -0700 Message-ID: Subject: Re: 10.1 NVMe kernel panic From: Jim Harris To: Sean Kelly Cc: FreeBSD-STABLE Mailing List Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2015 23:10:17 -0000 On Thu, May 21, 2015 at 8:33 AM, Sean Kelly wrote: > Greetings. > > I have a Dell R630 server with four of Dell=E2=80=99s 800GB NVMe SSDs run= ning > FreeBSD 10.1-p10. According to the PCI vendor, they are some sort of > rebranded Samsung drive. If I boot the system and then load nvme.ko and > nvd.ko from a command line, the drives show up okay. If I put > nvme_load=3D=E2=80=9CYES=E2=80=9D > nvd_load=3D=E2=80=9CYES=E2=80=9D > in /boot/loader.conf, the box panics on boot: > panic: nexus_setup_intr: NULL irq resource! > > If I boot the system with =E2=80=9CSafe Mode: ON=E2=80=9D from the loader= menu, it also > boots successfully and the drives show up. > > You can see a full =E2=80=98boot -v=E2=80=99 here: > http://smkelly.org/stuff/nvme-panic.txt < > http://smkelly.org/stuff/nvme-panic.txt> > > Anyone have any insight into what the issue may be here? Ideally I need t= o > get this working in the next few days or return this thing to Dell. > Hi Sean, Can you try adding hw.nvme.force_intx=3D1 to /boot/loader.conf? I suspect you are able to load the drivers successfully after boot because interrupt assignments are not restricted to CPU0 at that point - see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D199321 for a related issue. Your logs clearly show that vectors were allocated for the first 2 NVMe SSDs, but the third could not get its full allocation. There is a bug in the INTx fallback code that needs to be fixed - you do not hit this bug when loading after boot because bug #199321 only affects interrupt allocation during boot. If the force_intx test works, would you able to upgrade your nvme drivers to the latest on stable/10? There are several patches (one related to interrupt vector allocation) that have been pushed to stable/10 since 10.1 was released, and I will be pushing another patch for the issue you have reported shortly. Thanks, -Jim > > Thanks! > > -- > Sean Kelly > smkelly@smkelly.org > http://smkelly.org > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"