From owner-svn-src-all@freebsd.org Sat Nov 14 13:39:45 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A7A542EAB39; Sat, 14 Nov 2020 13:39:45 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-ot1-x334.google.com (mail-ot1-x334.google.com [IPv6:2607:f8b0:4864:20::334]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CYGfT3kr7z3CqM; Sat, 14 Nov 2020 13:39:45 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: by mail-ot1-x334.google.com with SMTP id n89so11527161otn.3; Sat, 14 Nov 2020 05:39:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:cc:references:from:autocrypt:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=Fxtvl5QMsbOXqPD+0tevMFm0pwpkdZCHNzXKLV392Oc=; b=myrTMA8AgTOoaJQFp+69m5ypIe9nJ5Znc9yY9HuixmAXHOmeVnri+z69BMHrpdCENl wAeMQSKVAwWCNIgNVHDewe5jaiLtOrFsaWGBRnmynQj/rW8pzz8CFL5q+zg3PPOR7+w4 JhTVJg8rdxhMQQt+6U5ow36uuegS9+iGO4UFB+mld68ohSFKVDdESfwHcVzTIHMss4+o whaKutDUxV85GrW1aN6J2cz2LzqABpkvhM70rHevA6sZZtEoq5hBtMB9W/oJqLUh4ljZ XHBCa2oRH/iL3zwzirmqOly+7GUKDFcdXzrlUB6b4lsvJ/qtEqJbPfpR8JLDf9SXRkk8 9tWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=Fxtvl5QMsbOXqPD+0tevMFm0pwpkdZCHNzXKLV392Oc=; b=DONV5yDmLuzWNeaNxMt73v+paNkHTZYHeOC3Xato4SUsFpC+o5BkkAtIU+ZNC0eeFV kUGEkgdHCBJNeg2IumYgI3buQ4Ds1XWl6RKdUWA/4o9H4AC5F+Us0nVsU0q4fln57jI6 97OKTgQkF/tDuEi4BI3ibi5YI+RfVZHb6NUGycCjMgBfDtyHUNWgfCYbroYOJrymmRoy bxYVaB9zRlz1TX2Z9kUCtODmSrHs9nZbxvDdwXVklWkdWCeWfsx+FY6CVZMksYUiKB/n miNb9PVvelZp0ZjKI/GCpki8remh5G8pnaA1bteVDnHCpqVbTYlcoJl0CnUmsdeec/2L IuYA== X-Gm-Message-State: AOAM533vS4pRSv+RmwAmqjOl8TZpIGZ/0YuNLPzwPvk1+NAJ1322sAIw X76W0lqLvJO4QvxqOZqYHk2UrOVpKvNCEQ== X-Google-Smtp-Source: ABdhPJw7Kx/fkGjzSyIQxYoEXGok3uXhtedeiDHhV1PK47nGXFs0o7lu36EkA3s0CGrsw/NG2+sWdA== X-Received: by 2002:a05:6830:1002:: with SMTP id a2mr5014084otp.316.1605361184222; Sat, 14 Nov 2020 05:39:44 -0800 (PST) Received: from spectre.mavhome.dp.ua ([2600:1700:3580:3560:228:f8ff:fe04:d12]) by smtp.gmail.com with ESMTPSA id a5sm2346651oto.1.2020.11.14.05.39.42 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 14 Nov 2020 05:39:43 -0800 (PST) Sender: Alexander Motin Subject: Re: svn commit: r367359 - stable/12/sys/dev/pci To: martin@vx.sk Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org References: <20201113173821.Horde.jq1A9wp72tHUvM10uhYxhOq@mail.vx.sk> From: Alexander Motin Autocrypt: addr=mav@FreeBSD.org; prefer-encrypt=mutual; keydata= mQENBFOzxAwBCADkPrax0pI2W/ig0CK9nRJJwsHitAGEZ2HZiFEuti+6/4UVxj81yr4ak/4g 9bKUyC7rMEAp/ZHNhd+MFCPAAcHPvtovnfykqE/vuosCS3wlSLloix2iKVLks0CwbLHGAyne 46lTQW74Xl/33c3W1Z6d8jD9gVFT/xaVzZ0U9xdzOmsYAZaAj4ki0tuxO9F7L+ct9grRe7iP g8t9hai7BL4ee3VRwk2JXnKb7UvBiVITKYWKz1jRvZIrjPokgEcCLOSlv7x/1kjuFnj3xWZU 7HSFFT8J93epBbrSSCsYsppIk2fZH41kaaFXsMQfTPH8wkeM6qwrvOh4HiQM08R+9tThABEB AAG0IUFsZXhhbmRlciBNb3RpbiA8bWF2QEZyZWVCU0Qub3JnPokBVwQTAQoAQQIbAwULCQgH AwUVCgkICwUWAwIBAAIeAQIXgAIZARYhBOmM88TmnMPNDledVYMYw5VbqyJ/BQJZYMKuBQkN McyiAAoJEIMYw5VbqyJ/tuUIAOG3ONOSNYqjK4eTZ1TVh9jdUBAhWk5nhDFnODN49Wj0AbYm 7aIqy8O1hnCDSZG5LttjSAo3UfXJZDKQM0BLb0gpRMBnAYqO6tdolLNqAbPGJBnGoPjsh24y 6KcbDaNnis+lD4GwPXwQM+92wZGhCUFElPV9NciZGVS65TNIgk7X+yEjjhD1MSWKKijZ1r9Z zIt4OzUTxxNOvzdlABZS88nNRdJkatOQJPmFdd1mpP6UzTNCiLUo1pIqOEtJgvVVDYq5WHY6 tciWWYdmZG/tIBexJmv2mV2OLVjXR6ZeKmntVH14H72/wRHJuYHQC+r5SVRcWWayrThsY6jZ Yr4+raS5AQ0EU7PEDAEIAOZgWf2cJIu+58IzP2dkXE/urj3tr4OqrB/yHGWUf71Lz6D0Fi6Z AXgDtmcFLGPfMyWuLAvSM+xmoguk7zC4hRBYvQycmIhuqBq1jO1Wp/Z+lpoPM/1cDYLn8Flv mI/c40MhUZh345DA4jYWWaZNjQHUWVQ1fPf595vdVVMPT/abE8E5DaF6fSkRmqFTmfYRkfbt 3ytU8NdUapDcJVY7cEP2nJBVNZPnOIObR/ZIgSxjjrG5o34yXoqeup8JvwEv+/NylzzuyXEZ R1EdEIzQ/a1nh/0j4NXtzZEqKW4aTWlmSqb6wN8jh1OSOOqkYsfnE3nfxcZbxi4IRoNQYlm5 9R8AEQEAAYkBPAQYAQoAJgIbDBYhBOmM88TmnMPNDledVYMYw5VbqyJ/BQJZYMLYBQkNMczM AAoJEIMYw5VbqyJ/TqgH/RQHClkvecE0262lwKoP/m0Mh4I5TLRgoJJn8S7G1BnqohYJkiLq A6xe6urGD7OqdNAl12UbrjWbdJV+zvea3vJoM4MZuYiYrGaXWxzFXqWJcPwMU9sAh8MRghHu uC5vgPb45Tnftw9/+n0i8GfVhQhOqepUGdQg4NPcXviSkoAvig6pp9Lcxisn0groUQKt15Gc sS9YcQWg3j9Hnipc6Mu416HX98Fb113NHJqc2geTHLkRyuBFOoyIqB6N9GKjzOAIzxxsVdl9 TevwGsrp4M4/RFzWbSgsbOnbE7454lmuVZGfReEjnUm8RHp9Q2UWKXlp3exlZjvOp/uVEpCg lz65AQ0EU7PEDAEIAOZgWf2cJIu+58IzP2dkXE/urj3tr4OqrB/yHGWUf71Lz6D0Fi6ZAXgD tmcFLGPfMyWuLAvSM+xmoguk7zC4hRBYvQycmIhuqBq1jO1Wp/Z+lpoPM/1cDYLn8FlvmI/c 40MhUZh345DA4jYWWaZNjQHUWVQ1fPf595vdVVMPT/abE8E5DaF6fSkRmqFTmfYRkfbt3ytU 8NdUapDcJVY7cEP2nJBVNZPnOIObR/ZIgSxjjrG5o34yXoqeup8JvwEv+/NylzzuyXEZR1Ed EIzQ/a1nh/0j4NXtzZEqKW4aTWlmSqb6wN8jh1OSOOqkYsfnE3nfxcZbxi4IRoNQYlm59R8A EQEAAYkBPAQYAQoAJgIbDBYhBOmM88TmnMPNDledVYMYw5VbqyJ/BQJZYMLYBQkNMczMAAoJ EIMYw5VbqyJ/TqgH/RQHClkvecE0262lwKoP/m0Mh4I5TLRgoJJn8S7G1BnqohYJkiLqA6xe 6urGD7OqdNAl12UbrjWbdJV+zvea3vJoM4MZuYiYrGaXWxzFXqWJcPwMU9sAh8MRghHuuC5v gPb45Tnftw9/+n0i8GfVhQhOqepUGdQg4NPcXviSkoAvig6pp9Lcxisn0groUQKt15GcsS9Y cQWg3j9Hnipc6Mu416HX98Fb113NHJqc2geTHLkRyuBFOoyIqB6N9GKjzOAIzxxsVdl9Tevw Gsrp4M4/RFzWbSgsbOnbE7454lmuVZGfReEjnUm8RHp9Q2UWKXlp3exlZjvOp/uVEpCglz4= Message-ID: Date: Sat, 14 Nov 2020 08:39:42 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.0 MIME-Version: 1.0 In-Reply-To: <20201113173821.Horde.jq1A9wp72tHUvM10uhYxhOq@mail.vx.sk> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4CYGfT3kr7z3CqM X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Nov 2020 13:39:45 -0000 Martin, I've never seen such issues. Could you send me verbose `dmesg`, `devinfo -v` and `lspci -vv` outputs with and without the tunable set? I guess your BIOS sets some things differently from all I've tested. On 13.11.2020 11:38, martin@vx.sk wrote: > I am encountering a problem with hw.pci.enable_aspm: > > System: ASUS RS500A-E10-RS12U > CPU: AMD EPYC 7502 32-Core. > NVMe drive model: Micron 9300 PRO > Number of NVMe drives: 10 > > Using X2APIC > > When hw.pci.enable_aspm is enabled, PCIe hot plug of Micron NVMe drives > does not work (including detection on boot) and the system reports the > following errors for the vast majority of the drives: > > Nov 13 15:11:30 xxx kernel: pcib9: Timed out waiting for Data Link Layer > Active > Nov 13 15:12:17 xxx kernel: pcib30: Timed out waiting for Data Link > Layer Active > Nov 13 15:12:44 xxx kernel: pcib29: Timed out waiting for Data Link > Layer Active > Nov 13 15:13:04 xxx kernel: pcib11: Timed out waiting for Data Link > Layer Active > ... > > Flipping the tunable back to 0 workarounds the problem. > > Reproduced both on latest head and stable/12. -- Alexander Motin