From owner-freebsd-current@freebsd.org Fri Apr 17 21:39:12 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7C4012ABE1B for ; Fri, 17 Apr 2020 21:39:12 +0000 (UTC) (envelope-from ypankov@fastmail.com) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 493qH30ygbz3K1w; Fri, 17 Apr 2020 21:39:10 +0000 (UTC) (envelope-from ypankov@fastmail.com) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 42E515C01D1; Fri, 17 Apr 2020 17:39:10 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Fri, 17 Apr 2020 17:39:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.com; h= subject:to:cc:references:from:message-id:date:mime-version :in-reply-to:content-type:content-transfer-encoding; s=fm2; bh=p BQ/vpdX3DNBjoLDzovYUi50DaHLqDJeWORqTcABKvE=; b=hpL8tLD59uK5eo7g6 K86hYlXLIW/P+O5AHd5UDmTJEiEViQZq8HlFRqTdSBiNcTsv9+SyQC4bQo/5j7tb t0JmPl/ORX7ljZIHhSIvivnDNNHb0R/da+10qxYTL/rJFTL45rB93asuHSty1mvZ p1WPk5MKHvNDCg4g7rAbyYeb2/V4MHWzD7X1nsN0eL8UzAKPEXd8ECT/RTdk5tIG yU0Wbbnd2Ck6Cz9XiVIXZJHT1ORezICBp6jxLrJ9vcWQRnWFhigUfLWerrim/TxW 1NH2sLU99e7A4Mqy9p/YZ0msmqs3BuNsfok6YIpTb2jKMoAqAAAh9Aw1ODtqX+CD SJM/w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=pBQ/vpdX3DNBjoLDzovYUi50DaHLqDJeWORqTcABK vE=; b=fE2troxDTD+Cc52Bwtlph98phvEYVSXYgT5RpHxuJbKt1TAIA5xlWJTpI S2h4+EUVsgqKlQXySz97GnkRPBSrWAzmR/CSUmG1tM08qpeBEIzkYP91Mtf6uRgI lW6TBZB722HrC5NOI573yreFKQYF8Rau0e9z6v7/PViUW2Fvj+NQySBiumsK5jgL l0BgmCb5jGglgHHf0x7r8CHvPDVXn6rB2b6I/WmmuZST0gHPeV9AFqyYxUmWDV0a eFoO5z/16+qkiBnI20imVeapUwF0fLjL4RhO3ko1kh5c+oHko3EwO3tPOzBVGYQf xXMv2zPfeiu3n+cvqDrYOF/n/2KxA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrfeejgdduieefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne goufhushhpvggtthffohhmrghinhculdegledmnecujfgurhepuffvfhfhkffffgggjggt gfesthekredttdefjeenucfhrhhomhepjghurhhiucfrrghnkhhovhcuoeihphgrnhhkoh hvsehfrghsthhmrghilhdrtghomheqnecuffhomhgrihhnpehisggsrdgtohdpphgrshht vggsihhnrdhplhenucfkphepkeehrddujeehrddufeelrddvhedunecuvehluhhsthgvrh fuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhephihprghnkhhovhesfhgrshht mhgrihhlrdgtohhm X-ME-Proxy: Received: from [192.168.1.6] (unknown [85.175.139.251]) by mail.messagingengine.com (Postfix) with ESMTPA id EAA24328005E; Fri, 17 Apr 2020 17:39:06 -0400 (EDT) Subject: Re: PCIe NVME drives not detected on Dell R6515 To: Scott Long , Miroslav Lachman <000.fbsd@quip.cz> Cc: Kurt Jaeger , Warner Losh , FreeBSD-Current References: <0F8BCB8C-DE60-4A34-A4D8-F1BB4B9F906A@samsco.org> <9EF043C1-FF8F-4997-B59A-EC3BF7D1CEEE@samsco.org> <31E8B2BE-BED2-4084-868D-32C48CB3CD6E@samsco.org> <573f5fab-1ef6-151f-18ca-58d3a4a89c72@quip.cz> <07B6763F-C23B-4B7C-B76A-26267AC35453@samsco.org> <20200417194431.GD39563@home.opsec.eu> <148dcdf7-f185-f14f-52ee-d4df3a2a1dc7@quip.cz> <8D8E1F62-AB66-47E1-8444-3D66F8EADA74@samsco.org> <015c7aa8-9385-4219-1bf1-0137f65ed80d@quip.cz> <90C35FEF-690C-4D04-A0D8-D3E5A448C744@samsco.org> From: Yuri Pankov Message-ID: Date: Sat, 18 Apr 2020 00:39:06 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: <90C35FEF-690C-4D04-A0D8-D3E5A448C744@samsco.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 493qH30ygbz3K1w X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=fastmail.com header.s=fm2 header.b=hpL8tLD5; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=fE2troxD; dmarc=pass (policy=none) header.from=fastmail.com; spf=pass (mx1.freebsd.org: domain of ypankov@fastmail.com designates 66.111.4.27 as permitted sender) smtp.mailfrom=ypankov@fastmail.com X-Spamd-Result: default: False [-3.10 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[fastmail.com:s=fm2,messagingengine.com:s=fm2]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.27]; FREEMAIL_FROM(0.00)[fastmail.com]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_FIVE(0.00)[5]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[fastmail.com:+,messagingengine.com:+]; DMARC_POLICY_ALLOW(-0.50)[fastmail.com,none]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[27.4.111.66.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[fastmail.com]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.00)[ip: (-9.83), ipnet: 66.111.4.0/24(-4.89), asn: 11403(-2.69), country: US(-0.05)]; RECEIVED_SPAMHAUS_PBL(0.00)[251.139.175.85.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Apr 2020 21:39:12 -0000 Scott Long wrote: > > >> On Apr 17, 2020, at 3:07 PM, Miroslav Lachman <000.fbsd@quip.cz> wrote: >> >> Scott Long wrote on 04/17/2020 23:04: >>>> On Apr 17, 2020, at 2:45 PM, Miroslav Lachman <000.fbsd@quip.cz> wrote: >>>> >>>> Scott Long wrote on 04/17/2020 22:17: >>>>>> On Apr 17, 2020, at 1:47 PM, Miroslav Lachman <000.fbsd@quip.cz> wrote: >>>>>> >>>>>> Kurt Jaeger wrote on 04/17/2020 21:44: >>>>>>> Hi! >>>>>>>>> pciconf -lBc pcib12 >>>>>>>>> pciconf -lBc pcib13 >>>>>>>> >>>>>>>> Printscreen attached. >>>>>>> Attachments are stripped from the list -- can you put them somewhere >>>>>>> online ? >>>>>> >>>>>> Here it is https://ibb.co/c1dZrTf >>>>>> >>>>>> Miroslav Lachman >>>>>> >>>>> Ok, the bridges know about their downstream bus numbers, but I see nothing that suggests that they’re being probed. The next step would be bootverbose, but that’s going to be a lot of output to collect in screen captures. >>>> >>>> Over 3000 lines long but I finally managed to make SOL work so I have it as text! >>>> >>>> https://pastebin.pl/view/90fdaafb >>>> >>> This helped a lot, thanks. It looks like these PCIe buses are marked as being hotplug, and for some reason we’re not probing them. At this point, I’d need to feed you some kernel patches that will dump out more info, but you’d have to compile them and get them onto your boot media. Is that a possibility? >> >> Currently I have all machines on 11.3 (where I can rebuild kernel without problem) >> If CURRENT is required I would need to setup some CURRENT VM in VirtualBox. >> >> Can you send me some link to documentation who should I create new ISO after rebuild? >> > > I don’t know of any docs for doing custom releases, and it looks like it’s harder than it used to be to insert custom patches. That said, I recommend doing the following on your 11.x build system: > > 1. Do a clean `make buildworld` with an up-to-date tree > 2. change into the `release` directory that you just did the buildworld from > 3. `sudo make release NOPORTS= NODOC= CHROOTDIR=/usr/tmp/release SRCBRANCH="base/stable/11@rHEAD”` > > You can set CHROOTDIR to whatever you want that has a few GB of space, but remember where you’ve set it for later steps. This will build a release with stock sources. Let it complete, both to prepare for the next step and to ensure that it works. It’ll take an hour or two depending on your machine speed > > 4. Take the patch that I’ll send you shortly and apply it to $CHROOTDIR/usr/src > 5. `sudo make memstick NOPORTS= NODOC= SRC_UPDATE_SKIP= CHROOTDIR=/usr/tmp/release` Just a thought - it could possibly be easier to build just the patched kernel and try booting it (11.3 userland should work with CURRENT kernel, right?), doing the kernel-toolchaing target first, and using INSTKERNNAME with the installkernel target to temporary select it from loader menu.