From owner-freebsd-current@freebsd.org Thu May 14 14:29:32 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C80EB2F3616 for ; Thu, 14 May 2020 14:29:32 +0000 (UTC) (envelope-from ctuffli@gmail.com) Received: from mail-pg1-f182.google.com (mail-pg1-f182.google.com [209.85.215.182]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49NDSr0Gcfz4PR3; Thu, 14 May 2020 14:29:31 +0000 (UTC) (envelope-from ctuffli@gmail.com) Received: by mail-pg1-f182.google.com with SMTP id b8so1303429pgi.11; Thu, 14 May 2020 07:29:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=wChxNqs9TtufNlGp5TbWSOfPHSxfolVSZer5edZZb8g=; b=NmdxCTm6FBQtALgleyqIiwM6lbKP42WxnNuUZWnXw7GNSwhp9bMsK7tvHjJTOGzee9 ARayc9F+MIZbGkTwHKfohPcoUPzmkvwuGple3kmU03N+EQ9znSlwZjbM1veitpWZnjyH UBNNleQlu1fVJtiAryG5WV59CdWM/Lu8hJmDShNnEmNIHyikz160mTLScpF2/XJc7TtH LsosxxhUd5YMCvGcXQllxsNMhPW8FTdfM0pFqdh3WJHd8C2QWkO0eEcqGmLx8nE05Ug7 qQrsWaZy2lxhZE8d2AxnywRjYxfz6b9QFNqTGcAcK5ClMsuqxfEXaBpYIYj3LPDSQzJ1 N5+w== X-Gm-Message-State: AOAM533TnL16fHA0+78571xBZs5OJ4W47dqMjE7jd5MXHxSdlVk61sBK KsTMgPPDD+2Lys0FYPThJ6nVInLh X-Google-Smtp-Source: ABdhPJwhq4GvGUmGWdgypJoGRMa3upP8QEO9BNzv3HBdS/XDhYxpRmBlkArKh/c/8iEJy+l3Itn2vQ== X-Received: by 2002:a62:8888:: with SMTP id l130mr4610176pfd.140.1589466570174; Thu, 14 May 2020 07:29:30 -0700 (PDT) Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com. [209.85.214.173]) by smtp.gmail.com with ESMTPSA id ce21sm18216948pjb.51.2020.05.14.07.29.29 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 14 May 2020 07:29:29 -0700 (PDT) Received: by mail-pl1-f173.google.com with SMTP id q16so8854plr.2; Thu, 14 May 2020 07:29:29 -0700 (PDT) X-Received: by 2002:a17:90a:21c9:: with SMTP id q67mr9794374pjc.166.1589466569363; Thu, 14 May 2020 07:29:29 -0700 (PDT) MIME-Version: 1.0 References: <0F8BCB8C-DE60-4A34-A4D8-F1BB4B9F906A@samsco.org> <9EF043C1-FF8F-4997-B59A-EC3BF7D1CEEE@samsco.org> <31E8B2BE-BED2-4084-868D-32C48CB3CD6E@samsco.org> <573f5fab-1ef6-151f-18ca-58d3a4a89c72@quip.cz> <07B6763F-C23B-4B7C-B76A-26267AC35453@samsco.org> <20200417194431.GD39563@home.opsec.eu> <148dcdf7-f185-f14f-52ee-d4df3a2a1dc7@quip.cz> <8D8E1F62-AB66-47E1-8444-3D66F8EADA74@samsco.org> <015c7aa8-9385-4219-1bf1-0137f65ed80d@quip.cz> <90C35FEF-690C-4D04-A0D8-D3E5A448C744@samsco.org> <4736a46d-716d-7860-ff56-6c1d7391dbeb@quip.cz> <6e6461d1-fbc4-c306-f71f-54767f2849cb@quip.cz> In-Reply-To: <6e6461d1-fbc4-c306-f71f-54767f2849cb@quip.cz> From: Chuck Tuffli Date: Thu, 14 May 2020 07:29:18 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: PCIe NVME drives not detected on Dell R6515 To: Miroslav Lachman <000.fbsd@quip.cz> Cc: Scott Long , Kurt Jaeger , Warner Losh , FreeBSD-Current Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 49NDSr0Gcfz4PR3 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of ctuffli@gmail.com designates 209.85.215.182 as permitted sender) smtp.mailfrom=ctuffli@gmail.com X-Spamd-Result: default: False [-4.11 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[182.215.85.209.rep.mailspike.net : 127.0.0.18]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[freebsd.org]; RCPT_COUNT_FIVE(0.00)[5]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[182.215.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.11)[ip: (-9.67), ipnet: 209.85.128.0/17(-0.39), asn: 15169(-0.42), country: US(-0.05)]; FORGED_SENDER(0.30)[chuck@freebsd.org,ctuffli@gmail.com]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[chuck@freebsd.org,ctuffli@gmail.com]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2020 14:29:32 -0000 On Mon, May 4, 2020 at 11:12 AM Miroslav Lachman <000.fbsd@quip.cz> wrote: > > On 2020-04-27 08:02, Miroslav Lachman wrote: > > I don't know what is with Scott. I hope he is well. > > Is there somebody else who can help me with this issue? > > Scott wrote there are hotplug PCIe buses not probed during boot process. > > I am not a developer so I cannot move forward alone. > > The problem is with PCIe Hot Plug. > Hot Plug bus was not enumerated thus no NVME detected. I may have just been bitten by this as well when running FreeBSD under qemu. The q35 machine type with PCIe emulation enables PCIe hot plug on all the root ports, but I am not seeing any downstream devices (either emulated like e1000 or passed through by the host) because of a check in pcib_hotplug_present(): /* * Require the Electromechanical Interlock to be engaged if * present. */ if (sc->pcie_slot_cap & PCIEM_SLOT_CAP_EIP && (sc->pcie_slot_sta & PCIEM_SLOT_STA_EIS) == 0) return (0); Under qemu, the slot indicates an Electromechanical Interlock is Present in the capabilities register, but it does not set the Electromechanical Interlock Status bit. This causes the PCI driver to not probe any children. Commenting out the above code made both emulated PCIe devices as well as host devices passed through appear in FreeBSD. As a data point, I'm not seeing similar checks in the Linux kernel. Miroslav, would it be possible to comment out/delete the above code in your kernel and retest to see if that helps your case as well? --chuck