From owner-freebsd-stable@freebsd.org Sun Sep 25 00:12:34 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 04A1FBDE19A for ; Sun, 25 Sep 2016 00:12:34 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-72.reflexion.net [208.70.210.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BEAF2687 for ; Sun, 25 Sep 2016 00:12:33 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 20031 invoked from network); 25 Sep 2016 00:12:17 -0000 Received: from unknown (HELO mail-cs-01.app.dca.reflexion.local) (10.81.19.1) by 0 (rfx-qmail) with SMTP; 25 Sep 2016 00:12:17 -0000 Received: by mail-cs-01.app.dca.reflexion.local (Reflexion email security v8.00.0) with SMTP; Sat, 24 Sep 2016 20:12:32 -0400 (EDT) Received: (qmail 28171 invoked from network); 25 Sep 2016 00:12:31 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 25 Sep 2016 00:12:31 -0000 Received: from [192.168.0.105] (ip70-189-131-151.lv.lv.cox.net [70.189.131.151]) by iron2.pdx.net (Postfix) with ESMTPSA id 4D64FEC8FEB; Sat, 24 Sep 2016 17:12:25 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: 11.0-RELEASE tier level for arm64/aaarch64 and the officially built arm/armv6 variants? From: Mark Millard In-Reply-To: Date: Sat, 24 Sep 2016 17:12:24 -0700 Cc: FreeBSD-STABLE Mailing List , freebsd-arm Content-Transfer-Encoding: quoted-printable Message-Id: <4F04B0C0-DAB1-4EEA-A3A0-4FAA89AD93E2@dsl-only.net> References: <4076CFFA-7BE2-4E1B-A7E8-08FD8FC27D21@dsl-only.net> <332FA120-31E5-4D31-B63E-A0DFDD7DEFC7@dsl-only.net> To: Warner Losh X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 00:12:34 -0000 On 2016-Sep-24, at 2:11 PM, Warner Losh wrote: > On Fri, Sep 23, 2016 at 8:29 PM, Mark Millard = wrote: >> [A resend since I forget to list free-arm in the To: the first time.] >>=20 >> =46rom https://www.freebsd.org/platforms/arm.html : >>=20 >>> 32-bit ARM is officially a Tier 2 architecture, as the FreeBSD = project does not provide official releases or pre-built packages for = this platform due to it primarily targeting the embedded arena. However, = FreeBSD/ARM is being actively developed and maintained, is well = supported, and provides an excellent framework for building ARM-based = systems. FreeBSD/arm supports ARMv4 and ARMv5 processors. FreeBSD/armv6 = supports ARMv6 and ARMv7 processors, including SMP on the latter. >>=20 >> "does not provide official releases or pre-built packages"? >>=20 >>> # uname -apKU >>> FreeBSD rpi2 11.0-PRERELEASE FreeBSD 11.0-PRERELEASE #5 r304943M: = Sun Aug 28 03:17:54 PDT 2016 = markmi@FreeBSDx64:/usr/obj/clang/arm.armv6/usr/src/sys/RPI2-NODBG arm = armv6 1100502 1100502 >>=20 >>> # pkg search '.*' | wc >>> 21349 155540 1596736 >>=20 >> Will 11.0-RELEASE change the tier level for any of the specific = arm-armv6 variants that have FreeBSD-11.0-*-arm-armv6-*.img* files = built, such as for RPI2? >>=20 >> Even if all the officially built arm-armv6 variants stay tier 2, the = wording on the web page likely needs to be changed because so much is = built and available that the above quote claims is not available. >=20 > armv6 is basically Tier 1 right now, though not as Tier 1 as i386 or > amd64 due to the fragmented nature of the arm world. On the platforms > we run on and create releases for, however, it's my opinion that it is > Tier 1: it has been running in production a while, things people > expect from a FreeBSD system are present, you can get decent support > if you ask questions, there's no known major gotchas in deploying this > hardware. The only remaining annoying issue is the 'u-boot' problem > where we have to have a different u-boot image for every board and no > standardized way to convert a 'generic' image into one that's specific > for specific boards. For x86 this is all done with the installer since > that boot environment is more standardized. Does this last issue keep > arm from being Tier 1? That's a judgement call, but I think the > project should promote w/o this last issue. Interesting and good to know. Thanks. I might have guessed that going along with the u-boot issue would be the = fanout in: 11.0/sys/arm/ . . . allwinner/a10/ allwinner/a20/ allwinner/a31/ allwinner/a83t/ allwinner/h3/ . . . broadcom/bcm2835/ . . . (Full list not shown.) I was thinking that this might make the tier level specific to the = status of each such directory's content so that it was the combination = of that and the sysutils/u-boot-*/ status that made the difference for = assigning the level. I'd guess that lack of a usable directory in = either place would not be tier 2 even. Similarly until the required = sys/arm/*/* and sysutils/u-boot-*/ directory-tree content have reached a = sufficiently complete status. I'd expect that there will always be a lag for what exists in the world = vs. what has these materials worked out in FreeBSD. >> Also from https://www.freebsd.org/platforms/arm.html : >>=20 >>> Initial support for 64-bit ARM is complete. 64-bit ARM platforms = follow a set of standard conventions, and a single FreeBSD build will = work on hardware from multiple vendors. As a result, FreeBSD will = provide official releases for FreeBSD/arm64 and packages will be = available. FreeBSD/arm64 is on the path to becoming a Tier 1 = architecture. >>=20 >> Will 11.0-RELEASE make arm64/aarch64 Tier 1? >>=20 >> [I will note that, while there are no official builds for the Pine64 = family (A64 based) that are under the Allwinner arm activity, the SOC's = involved are Cortex-A53 64-bit arm based. They likely do not fit in the = "standard conventions" or arm64/aarch64 would be where they would have = been supported. Some rewording might be appropriate for the above quote = as well.] >=20 > No. aarch64 isn't Tier 1 yet. There's many small bits that are > missing. It is quite solidly Tier 2, but we don't have a linker, we > don't have widespread hardware availability, we don't have production > experience with the platform. Most things work, but there's still some > gotchas. There's still the 'u-boot' problem with many arm64 systems > because for systems that use u-boot to bootstrap UEFI, you need a > different image for each board (some closely related board families > can get by with one to be pedantic). All these issues are still > significant barriers to production use. It's not been officially > promoted yet and I don't think the time is quite right yet. Intersting as well. I'd guess that conceptually this probably would = apply to both: sys/arm/allwinner/a64/ and sysutils/u-boot/pine64/ (presuming, contrary to fact, that 11.0 had sys/arm/allwinner/a64/ ) and. . . sys/arm64/cavium/ sys/arm64/cloudabi64/ So just sys/arm/ vs. sys/arm64/ for an aarch64 would not really make a = difference yet for tier level. > Warner Thanks again for the notes. =3D=3D=3D Mark Millard markmi at dsl-only.net From owner-freebsd-stable@freebsd.org Sun Sep 25 04:21:09 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47033BE9707; Sun, 25 Sep 2016 04:21:09 +0000 (UTC) (envelope-from russ.haley@gmail.com) Received: from mail-vk0-x22f.google.com (mail-vk0-x22f.google.com [IPv6:2607:f8b0:400c:c05::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F29CBEA4; Sun, 25 Sep 2016 04:21:08 +0000 (UTC) (envelope-from russ.haley@gmail.com) Received: by mail-vk0-x22f.google.com with SMTP id z126so18820319vkd.0; Sat, 24 Sep 2016 21:21:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=HR5m8jhBvw3EId8GNMY/0AwEF4ng6yUm5Kwml8joCLU=; b=LB9kOPUaP8ZwPG/EbHcn8/lziBYfTj5VpjqVrx9m/1aBVJe0G8qVTFcz/ZhxgLlDF5 NZfZ7bx+ZwOe262U9eITWjy0D3TwjNGvokHhw4jo3nxGGsnmVdDCwcqzvHUepblPHCOD GcrK1Wtr6s2J0paz2PXCjMBfSAHapuKG4KvdZtAVuaOxfX9kSYFZdjPalu146QHZndUp C6PUb5PSscKKkgvY1Y9lEkQ4fpl/V/xZZrvxuiZEnl4LCwsFH5BzK6jJVx+tptqqbBkq 4u+lbHh4jgnzqUIoJRfRNhHQrMx9O9mgTnOfxCmv/SDW8y1IdcBnAiInx09mDxk+kvRs eTzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=HR5m8jhBvw3EId8GNMY/0AwEF4ng6yUm5Kwml8joCLU=; b=aEzA/aY+g0erehnIFD9QWQ3c0G5pXvS/xXpu5j2krjLg5Lo7SKC02kDqCuTAJdrXhH sxXTETEv9mmj5Q0q0xfwdvtVRDpbDbjVcFAB5ZcGfQpzRchO1p/m7bZUthkL8DknaAG4 IyuDvQ1FeC87mtVce0dJ1HdvQ5tPxf7BbSonGiQJ2t8nHQsuJ+C7CZd5lcbWju1A+Rxo l/ZQCaRRV2bpc7fSKGvETKvLd5o+r7bn7al7FGLZlZIVrqy4yHF8xt0Men7fh79UKV26 4EaIX/1y7ajnxGv6wICuKBcEbwxEyUtSIMU06+oI0Y2JydIe3azocEsnJ1NOvGPo/DIi j3jg== X-Gm-Message-State: AA6/9RlTJ6EWZTt2BEQxAGAOu4XwnnpeAlo8NnkrN0Qv8rzb3tRpH43LFI8PZzRqn2mHAnhr1uCytMY6dIcvkA== X-Received: by 10.31.242.206 with SMTP id q197mr3668053vkh.46.1474777268017; Sat, 24 Sep 2016 21:21:08 -0700 (PDT) MIME-Version: 1.0 Received: by 10.31.124.202 with HTTP; Sat, 24 Sep 2016 21:21:07 -0700 (PDT) In-Reply-To: <4F04B0C0-DAB1-4EEA-A3A0-4FAA89AD93E2@dsl-only.net> References: <4076CFFA-7BE2-4E1B-A7E8-08FD8FC27D21@dsl-only.net> <332FA120-31E5-4D31-B63E-A0DFDD7DEFC7@dsl-only.net> <4F04B0C0-DAB1-4EEA-A3A0-4FAA89AD93E2@dsl-only.net> From: Russell Haley Date: Sat, 24 Sep 2016 21:21:07 -0700 Message-ID: Subject: Re: 11.0-RELEASE tier level for arm64/aaarch64 and the officially built arm/armv6 variants? To: Mark Millard Cc: Warner Losh , freebsd-arm , FreeBSD-STABLE Mailing List Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 04:21:09 -0000 On Sat, Sep 24, 2016 at 5:12 PM, Mark Millard wrote: > On 2016-Sep-24, at 2:11 PM, Warner Losh wrote: > >> On Fri, Sep 23, 2016 at 8:29 PM, Mark Millard wrot= e: >>> [A resend since I forget to list free-arm in the To: the first time.] >>> >>> From https://www.freebsd.org/platforms/arm.html : >>> >>>> 32-bit ARM is officially a Tier 2 architecture, as the FreeBSD project= does not provide official releases or pre-built packages for this platform= due to it primarily targeting the embedded arena. However, FreeBSD/ARM is = being actively developed and maintained, is well supported, and provides an= excellent framework for building ARM-based systems. FreeBSD/arm supports A= RMv4 and ARMv5 processors. FreeBSD/armv6 supports ARMv6 and ARMv7 processor= s, including SMP on the latter. >>> >>> "does not provide official releases or pre-built packages"? >>> >>>> # uname -apKU >>>> FreeBSD rpi2 11.0-PRERELEASE FreeBSD 11.0-PRERELEASE #5 r304943M: Sun = Aug 28 03:17:54 PDT 2016 markmi@FreeBSDx64:/usr/obj/clang/arm.armv6/usr= /src/sys/RPI2-NODBG arm armv6 1100502 1100502 >>> >>>> # pkg search '.*' | wc >>>> 21349 155540 1596736 >>> >>> Will 11.0-RELEASE change the tier level for any of the specific arm-arm= v6 variants that have FreeBSD-11.0-*-arm-armv6-*.img* files built, such as = for RPI2? >>> >>> Even if all the officially built arm-armv6 variants stay tier 2, the wo= rding on the web page likely needs to be changed because so much is built a= nd available that the above quote claims is not available. >> >> armv6 is basically Tier 1 right now, though not as Tier 1 as i386 or >> amd64 due to the fragmented nature of the arm world. On the platforms >> we run on and create releases for, however, it's my opinion that it is >> Tier 1: it has been running in production a while, things people >> expect from a FreeBSD system are present, you can get decent support >> if you ask questions, there's no known major gotchas in deploying this >> hardware. The only remaining annoying issue is the 'u-boot' problem >> where we have to have a different u-boot image for every board and no >> standardized way to convert a 'generic' image into one that's specific >> for specific boards. I'll point out again that barebox is an excellent alternative to u-boot (IM= HO): - Supports most, if not all of the boards that FreeBSD supports, plus many that it doesn't - Single source tree for all boards. Specify build time parameters to build one or all the images - Well supported community with central maintainer-ship - Simple, familiar shell interface (*awesome*) - Excellent documentation (u-boots is good too though) - Has support for (U)EFI - Supports quemu aarch64 (not *quite*sure what the means though) To be fair, I'm not saying the problem is the fault of denx, but barebox has a lot going for it. The maintainer was very keen to see it ported top FreeBSD and was willing to support the effort. I ran into some build time linux api requirements, but he didn't think that would be much to overcome (and it wasn't I just kept adding the patches he sent me and the build moved forward. As always, I ran out of time for the really fun stuff). While I am a hopeless dreamer and I'm sure I've over simplified the problem, I thought it would be neat to see FreeBSD upstream support for zfs and ufs to the barebox boot loader and do away with ubldr. We would then have a modern, easy to use, boot loader that supports the standard startup toolchain. Either way, if installers move to a pkgng based method (so cool) then installing u-boot and arm binaries from pkg-static will be the same as x86 (ha ha I said that with a straight face!). >>For x86 this is all done with the installer since >> that boot environment is more standardized. Does this last issue keep >> arm from being Tier 1? That's a judgement call, but I think the >> project should promote w/o this last issue. How does a platform get promoted? Is that something the Core team decides? I see two facts about current Arm support that show platform maturity: a) u-boot is in the ports tree and we have Lego-easy build scripts in crochet that could be called an installation method. Building for arm is not difficult anymore. b)Arm requires images, not installers. Correct me if I'm wrong but, installers are a tool primarily invented for x86 PC type computers. FreeBSD publishes standardized ISOs for all supported Arm platforms that work by simply "xzcat | dd" onto the sd card (or wherever you need it). I'm not sure how standardized or manual that build process is, but I would think that if the Arm platform support is able to keep up with the standard FreeBSD release cycle (i.e. not break every other release) then there would be no reason to NOT call it tier 1? What I don't know about is "official" documentation for building for arm and supporting cross building to Arm. Will someone need to write an "Arm Handbook" to be promoted? Cheers, Russ > > Interesting and good to know. Thanks. > > I might have guessed that going along with the u-boot issue would be the = fanout in: > > 11.0/sys/arm/ . . . > > allwinner/a10/ > allwinner/a20/ > allwinner/a31/ > allwinner/a83t/ > allwinner/h3/ > . . . > broadcom/bcm2835/ > . . . > > (Full list not shown.) > > I was thinking that this might make the tier level specific to the status= of each such directory's content so that it was the combination of that an= d the sysutils/u-boot-*/ status that made the difference for assigning the = level. I'd guess that lack of a usable directory in either place would not= be tier 2 even. Similarly until the required sys/arm/*/* and sysutils/u-bo= ot-*/ directory-tree content have reached a sufficiently complete status. > > I'd expect that there will always be a lag for what exists in the world v= s. what has these materials worked out in FreeBSD. > > >>> Also from https://www.freebsd.org/platforms/arm.html : >>> >>>> Initial support for 64-bit ARM is complete. 64-bit ARM platforms follo= w a set of standard conventions, and a single FreeBSD build will work on ha= rdware from multiple vendors. As a result, FreeBSD will provide official re= leases for FreeBSD/arm64 and packages will be available. FreeBSD/arm64 is o= n the path to becoming a Tier 1 architecture. >>> >>> Will 11.0-RELEASE make arm64/aarch64 Tier 1? >>> >>> [I will note that, while there are no official builds for the Pine64 fa= mily (A64 based) that are under the Allwinner arm activity, the SOC's invol= ved are Cortex-A53 64-bit arm based. They likely do not fit in the "standar= d conventions" or arm64/aarch64 would be where they would have been support= ed. Some rewording might be appropriate for the above quote as well.] >> >> No. aarch64 isn't Tier 1 yet. There's many small bits that are >> missing. It is quite solidly Tier 2, but we don't have a linker, we >> don't have widespread hardware availability, we don't have production >> experience with the platform. Most things work, but there's still some >> gotchas. There's still the 'u-boot' problem with many arm64 systems >> because for systems that use u-boot to bootstrap UEFI, you need a >> different image for each board (some closely related board families >> can get by with one to be pedantic). All these issues are still >> significant barriers to production use. It's not been officially >> promoted yet and I don't think the time is quite right yet. > > Intersting as well. I'd guess that conceptually this probably would apply= to both: > > sys/arm/allwinner/a64/ and sysutils/u-boot/pine64/ > (presuming, contrary to fact, that 11.0 had sys/arm/allwinner/a64/ ) > > and. . . > > sys/arm64/cavium/ > sys/arm64/cloudabi64/ > > So just sys/arm/ vs. sys/arm64/ for an aarch64 would not really make a di= fference yet for tier level. > >> Warner > > Thanks again for the notes. > > =3D=3D=3D > Mark Millard > markmi at dsl-only.net > > _______________________________________________ > freebsd-arm@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arm > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Sun Sep 25 06:10:47 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2AE75BE8B06 for ; Sun, 25 Sep 2016 06:10:47 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E2ACB62A for ; Sun, 25 Sep 2016 06:10:46 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x236.google.com with SMTP id j69so19445873itb.0 for ; Sat, 24 Sep 2016 23:10:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-transfer-encoding; bh=uGkLDd2L62s0iz+iWbFM5qC1zvqnVU/ulEAtpR38sV8=; b=H7VMloZcBldbBRdlpytXeLIIMvgLJH/91BWZd8oQkXvszI7lh1doFw9GGQZIbFan7q z4JHkKp+jdLuLW7XF+7OS+V4ZIkcIfQ9cbiS6G8CAsD2f5UmAtsZUWYtDIx1p7sI6453 AZR4YmpogVpeH1/tS3ZxXPsHGNVtQ7kldce1a4V4fHf+nw4lBbYkdtRNpto6vnY3Pis9 7XJ1gdwwXIWgCDmq3qEWVIGRKnmMUT31OHzWPGAhcaTZMd3Hh4j6aXYfw8C3lNbgA/RY o2a0Z6Rwp+o79MOPbGXfMYktvJ5jgrKAUAdSnFiKsUKl4Ip6y0VAaxFi8TCYT9QKufDO pzLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-transfer-encoding; bh=uGkLDd2L62s0iz+iWbFM5qC1zvqnVU/ulEAtpR38sV8=; b=awczR4XiiLGro3xmpwbIZmi4FaIMpsD8ID1Q2qGYwcoPaD/EmG5fdVPUAeyKnMLfzO COlMTvTJnAsFv5jooOd7n9nxMhCzUxthdGyKlWT83gCsSk095QCBw0A1wg6tZ5qdUeun 7m8eIdMp8GaPaHiQfWICQGTsG5RxLOxZgNxNiytatF6YbsNDuZzGY16wha9R7LdYqwBu uNwW1xTQesQilDDf2Mv1CI4anpUij9AYh7E6OQoClAe3YRKBR3wAGEalkSCBp2BMJztA yM6Ufg/cYkSeZZM8rlEazEO4ul07yB2onfK6UCzlNGJa9dUd/nZW8ZIxeEPE//4xSlDv A1Jw== X-Gm-Message-State: AA6/9RnzWQn7+ECZ1j7LH31x21SrX5QI1+EWQ9/ZjN1TJSGqLhT0zt4Ircw0qKc31P3BQ4DsXcuvlqM0Kcw3Qw== X-Received: by 10.36.58.133 with SMTP id m127mr931821itm.79.1474783845897; Sat, 24 Sep 2016 23:10:45 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.36.65.7 with HTTP; Sat, 24 Sep 2016 23:10:45 -0700 (PDT) X-Originating-IP: [69.53.245.200] In-Reply-To: References: <4076CFFA-7BE2-4E1B-A7E8-08FD8FC27D21@dsl-only.net> <332FA120-31E5-4D31-B63E-A0DFDD7DEFC7@dsl-only.net> <4F04B0C0-DAB1-4EEA-A3A0-4FAA89AD93E2@dsl-only.net> From: Warner Losh Date: Sun, 25 Sep 2016 00:10:45 -0600 X-Google-Sender-Auth: 2ppeaJ_eoltWttM19XyzCBbiCKo Message-ID: Subject: Re: 11.0-RELEASE tier level for arm64/aaarch64 and the officially built arm/armv6 variants? To: Russell Haley Cc: Mark Millard , freebsd-arm , FreeBSD-STABLE Mailing List Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 06:10:47 -0000 On Sat, Sep 24, 2016 at 10:21 PM, Russell Haley wrot= e: > On Sat, Sep 24, 2016 at 5:12 PM, Mark Millard wrote= : >> On 2016-Sep-24, at 2:11 PM, Warner Losh wrote: >> >>> On Fri, Sep 23, 2016 at 8:29 PM, Mark Millard wro= te: >>>> [A resend since I forget to list free-arm in the To: the first time.] >>>> >>>> From https://www.freebsd.org/platforms/arm.html : >>>> >>>>> 32-bit ARM is officially a Tier 2 architecture, as the FreeBSD projec= t does not provide official releases or pre-built packages for this platfor= m due to it primarily targeting the embedded arena. However, FreeBSD/ARM is= being actively developed and maintained, is well supported, and provides a= n excellent framework for building ARM-based systems. FreeBSD/arm supports = ARMv4 and ARMv5 processors. FreeBSD/armv6 supports ARMv6 and ARMv7 processo= rs, including SMP on the latter. >>>> >>>> "does not provide official releases or pre-built packages"? >>>> >>>>> # uname -apKU >>>>> FreeBSD rpi2 11.0-PRERELEASE FreeBSD 11.0-PRERELEASE #5 r304943M: Sun= Aug 28 03:17:54 PDT 2016 markmi@FreeBSDx64:/usr/obj/clang/arm.armv6/us= r/src/sys/RPI2-NODBG arm armv6 1100502 1100502 >>>> >>>>> # pkg search '.*' | wc >>>>> 21349 155540 1596736 >>>> >>>> Will 11.0-RELEASE change the tier level for any of the specific arm-ar= mv6 variants that have FreeBSD-11.0-*-arm-armv6-*.img* files built, such as= for RPI2? >>>> >>>> Even if all the officially built arm-armv6 variants stay tier 2, the w= ording on the web page likely needs to be changed because so much is built = and available that the above quote claims is not available. >>> >>> armv6 is basically Tier 1 right now, though not as Tier 1 as i386 or >>> amd64 due to the fragmented nature of the arm world. On the platforms >>> we run on and create releases for, however, it's my opinion that it is >>> Tier 1: it has been running in production a while, things people >>> expect from a FreeBSD system are present, you can get decent support >>> if you ask questions, there's no known major gotchas in deploying this >>> hardware. The only remaining annoying issue is the 'u-boot' problem >>> where we have to have a different u-boot image for every board and no >>> standardized way to convert a 'generic' image into one that's specific >>> for specific boards. > > I'll point out again that barebox is an excellent alternative to u-boot (= IMHO): Doesn't matter, still has the same issues that u-boot has. But does it support the u-boot ABI? > - Supports most, if not all of the boards that FreeBSD supports, plus > many that it doesn't > - Single source tree for all boards. Specify build time parameters to > build one or all the images > - Well supported community with central maintainer-ship > - Simple, familiar shell interface (*awesome*) > - Excellent documentation (u-boots is good too though) > - Has support for (U)EFI > - Supports quemu aarch64 (not *quite*sure what the means though) Right, u-boot has all these things, except maybe the shell interface (not sure what you mean by that). > To be fair, I'm not saying the problem is the fault of denx, but > barebox has a lot going for it. The maintainer was very keen to see it > ported top FreeBSD and was willing to support the effort. I ran into > some build time linux api requirements, but he didn't think that would > be much to overcome (and it wasn't I just kept adding the patches he > sent me and the build moved forward. As always, I ran out of time for > the really fun stuff). While I am a hopeless dreamer and I'm sure I've > over simplified the problem, I thought it would be neat to see FreeBSD > upstream support for zfs and ufs to the barebox boot loader and do > away with ubldr. We would then have a modern, easy to use, boot loader > that supports the standard startup toolchain. We can't easily do away with ubldr if we want to support tunables, kernel modules loaded at boot time and a few other nifty features like nextboot. > Either way, if installers move to a pkgng based method (so cool) then > installing u-boot and arm binaries from pkg-static will be the same as > x86 (ha ha I said that with a straight face!). Yea, not so much. You have to build the bootable image not on the target system, like you do on x86. We'd have to have something that installs uboot onto a generic image (perhaps with hooks for kernels since those aren't generic on armv6) and then put that into a bootable SD card. >>>For x86 this is all done with the installer since >>> that boot environment is more standardized. Does this last issue keep >>> arm from being Tier 1? That's a judgement call, but I think the >>> project should promote w/o this last issue. > > How does a platform get promoted? Is that something the Core team decides= ? Yes. > I see two facts about current Arm support that show platform maturity: > a) u-boot is in the ports tree and we have Lego-easy build scripts in > crochet that could be called an installation method. Building for arm > is not difficult anymore. Except corchet isn't in the tree, and the solution is horrible. It's a script for each SoC, and those scripts are now scattered about. Plus that's a creation from source model, not a creation from RE produced bits model, which is needed for Tier 1. > b)Arm requires images, not installers. Correct me if I'm wrong but, > installers are a tool primarily invented for x86 PC type computers. > FreeBSD publishes standardized ISOs for all supported Arm platforms > that work by simply "xzcat | dd" onto the sd card (or wherever you > need it). I'm not sure how standardized or manual that build process > is, but I would think that if the Arm platform support is able to keep > up with the standard FreeBSD release cycle (i.e. not break every other > release) then there would be no reason to NOT call it tier 1? Creating the images is currently a pita. That's it. > What I don't know about is "official" documentation for building for > arm and supporting cross building to Arm. Will someone need to write > an "Arm Handbook" to be promoted? No. While useful, most of that already exists. Warner >> Interesting and good to know. Thanks. >> >> I might have guessed that going along with the u-boot issue would be the= fanout in: >> >> 11.0/sys/arm/ . . . >> >> allwinner/a10/ >> allwinner/a20/ >> allwinner/a31/ >> allwinner/a83t/ >> allwinner/h3/ >> . . . >> broadcom/bcm2835/ >> . . . >> >> (Full list not shown.) >> >> I was thinking that this might make the tier level specific to the statu= s of each such directory's content so that it was the combination of that a= nd the sysutils/u-boot-*/ status that made the difference for assigning the= level. I'd guess that lack of a usable directory in either place would no= t be tier 2 even. Similarly until the required sys/arm/*/* and sysutils/u-b= oot-*/ directory-tree content have reached a sufficiently complete status. >> >> I'd expect that there will always be a lag for what exists in the world = vs. what has these materials worked out in FreeBSD. >> >> >>>> Also from https://www.freebsd.org/platforms/arm.html : >>>> >>>>> Initial support for 64-bit ARM is complete. 64-bit ARM platforms foll= ow a set of standard conventions, and a single FreeBSD build will work on h= ardware from multiple vendors. As a result, FreeBSD will provide official r= eleases for FreeBSD/arm64 and packages will be available. FreeBSD/arm64 is = on the path to becoming a Tier 1 architecture. >>>> >>>> Will 11.0-RELEASE make arm64/aarch64 Tier 1? >>>> >>>> [I will note that, while there are no official builds for the Pine64 f= amily (A64 based) that are under the Allwinner arm activity, the SOC's invo= lved are Cortex-A53 64-bit arm based. They likely do not fit in the "standa= rd conventions" or arm64/aarch64 would be where they would have been suppor= ted. Some rewording might be appropriate for the above quote as well.] >>> >>> No. aarch64 isn't Tier 1 yet. There's many small bits that are >>> missing. It is quite solidly Tier 2, but we don't have a linker, we >>> don't have widespread hardware availability, we don't have production >>> experience with the platform. Most things work, but there's still some >>> gotchas. There's still the 'u-boot' problem with many arm64 systems >>> because for systems that use u-boot to bootstrap UEFI, you need a >>> different image for each board (some closely related board families >>> can get by with one to be pedantic). All these issues are still >>> significant barriers to production use. It's not been officially >>> promoted yet and I don't think the time is quite right yet. >> >> Intersting as well. I'd guess that conceptually this probably would appl= y to both: >> >> sys/arm/allwinner/a64/ and sysutils/u-boot/pine64/ >> (presuming, contrary to fact, that 11.0 had sys/arm/allwinner/a64/ ) >> >> and. . . >> >> sys/arm64/cavium/ >> sys/arm64/cloudabi64/ >> >> So just sys/arm/ vs. sys/arm64/ for an aarch64 would not really make a d= ifference yet for tier level. >> >>> Warner >> >> Thanks again for the notes. >> >> =3D=3D=3D >> Mark Millard >> markmi at dsl-only.net >> >> _______________________________________________ >> freebsd-arm@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-arm >> To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Sun Sep 25 06:42:06 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 459E1BE91B3 for ; Sun, 25 Sep 2016 06:42:06 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id 29ED936F; Sun, 25 Sep 2016 06:42:06 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id 3BB4CC6; Sun, 25 Sep 2016 06:42:06 +0000 (UTC) Date: Sun, 25 Sep 2016 06:41:58 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: jenkins-admin@FreeBSD.org, freebsd-stable@FreeBSD.org Message-ID: <735755714.0.1474785726003.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Build failed in Jenkins: FreeBSD_stable_10 #408 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkKKb2VAfYQKfu1t7qk4nR5qzUBEI+UqT4BPec4qHVhqUy0FFdq50sMH+3y9bCDNOufctov6VqTNffZ3YXArnZK95YF0OX97fh+E9txYOUX1adc+TikcKjuYpHmL5dE62eaZTI+4A5jnRonskQ1PaoIFz0Kbu4mWzkFsmdiXTraGzomXq4cHUCATA2+K4eDYgjXEQI30z3GOMmmZ4t/+6QGk1cMb/BqMWHbn80AsRCb4tU7Hpd72XLDpsuO7YRP1Q0CjmNAuBOTj+sFiiOe6U9HpqOlQN+iFUvBdZo/ybuy5Kh71cAaYQNL68cYdZJ6binH/DkG3KY/fS7DFYAeuwjwIDAQAB X-Jenkins-Job: FreeBSD_stable_10 X-Jenkins-Result: FAILURE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 06:42:06 -0000 See ------------------------------------------ [...truncated 265769 lines...] --- radeonkmsfw_RV620_me.ko.debug --- ld -d -warn-common -r -d -o radeonkmsfw_RV620_me.ko.debug RV620_me.bin.fwo= radeonkmsfw_RV620_me.o ctfmerge -L VERSION -g -o radeonkmsfw_RV620_me.ko.debug RV620_me.bin.fwo ra= deonkmsfw_RV620_me.o :> export_syms awk -f /builds/workspace/FreeBSD_stable_10/src/sys/conf/kmod_syms.awk radeo= nkmsfw_RV620_me.ko.debug export_syms | xargs -J% objcopy % radeonkmsfw_RV6= 20_me.ko.debug --- radeonkmsfw_RV620_me.ko.symbols --- objcopy --only-keep-debug radeonkmsfw_RV620_me.ko.debug radeonkmsfw_RV620_m= e.ko.symbols --- radeonkmsfw_RV620_me.ko --- objcopy --strip-debug --add-gnu-debuglink=3Dradeonkmsfw_RV620_me.ko.symbols= radeonkmsfw_RV620_me.ko.debug radeonkmsfw_RV620_me.ko --- all_subdir_RV620_pfp --- =3D=3D=3D> drm2/radeonkmsfw/RV620_pfp (all) --- RV620_pfp.bin --- uudecode -p /builds/workspace/FreeBSD_stable_10/src/sys/modules/drm2/radeon= kmsfw/RV620_pfp/../../../../contrib/dev/drm2/radeonkmsfw/RV620_pfp.bin.uu >= RV620_pfp.bin --- RV620_pfp.bin.fwo --- RV620_pfp.bin RV620_pfp.bin --- radeonkmsfw_RV620_pfp.o --- cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdin= c -DHAVE_KERNEL_OPTION_HEADERS -include /builds/workspace/FreeBSD_stable_= 10/obj/builds/workspace/FreeBSD_stable_10/src/sys/GENERIC/opt_global.h -I. = -I@ -I@/contrib/altq -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-= frame-pointer -I/builds/workspace/FreeBSD_stable_10/obj/builds/workspace/Fr= eeBSD_stable_10/src/sys/GENERIC -mno-aes -mno-avx -mcmodel=3Dkernel -mno-r= ed-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ff= reestanding -fstack-protector -gdwarf-2 -std=3Diso9899:1999 -Qunused-argume= nts -fstack-protector -Wall -Wredundant-decls -Wnested-externs -Wstrict-pr= ototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wunde= f -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnosti= cs-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno= -error-parentheses-equality -Wno-error-unused-function -c radeonkmsfw_RV= 620_pfp.c -o radeonkmsfw_RV620_pfp.o ctfconvert -L VERSION -g radeonkmsfw_RV620_pfp.o --- radeonkmsfw_RV620_pfp.ko.debug --- ld -d -warn-common -r -d -o radeonkmsfw_RV620_pfp.ko.debug RV620_pfp.bin.f= wo radeonkmsfw_RV620_pfp.o ctfmerge -L VERSION -g -o radeonkmsfw_RV620_pfp.ko.debug RV620_pfp.bin.fwo = radeonkmsfw_RV620_pfp.o :> export_syms awk -f /builds/workspace/FreeBSD_stable_10/src/sys/conf/kmod_syms.awk radeo= nkmsfw_RV620_pfp.ko.debug export_syms | xargs -J% objcopy % radeonkmsfw_RV= 620_pfp.ko.debug --- radeonkmsfw_RV620_pfp.ko.symbols --- objcopy --only-keep-debug radeonkmsfw_RV620_pfp.ko.debug radeonkmsfw_RV620_= pfp.ko.symbols --- radeonkmsfw_RV620_pfp.ko --- objcopy --strip-debug --add-gnu-debuglink=3Dradeonkmsfw_RV620_pfp.ko.symbol= s radeonkmsfw_RV620_pfp.ko.debug radeonkmsfw_RV620_pfp.ko --- all_subdir_RV630_me --- =3D=3D=3D> drm2/radeonkmsfw/RV630_me (all) --- RV630_me.bin --- uudecode -p /builds/workspace/FreeBSD_stable_10/src/sys/modules/drm2/radeon= kmsfw/RV630_me/../../../../contrib/dev/drm2/radeonkmsfw/RV630_me.bin.uu > R= V630_me.bin --- RV630_me.bin.fwo --- RV630_me.bin RV630_me.bin --- radeonkmsfw_RV630_me.o --- cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdin= c -DHAVE_KERNEL_OPTION_HEADERS -include /builds/workspace/FreeBSD_stable_= 10/obj/builds/workspace/FreeBSD_stable_10/src/sys/GENERIC/opt_global.h -I. = -I@ -I@/contrib/altq -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-= frame-pointer -I/builds/workspace/FreeBSD_stable_10/obj/builds/workspace/Fr= eeBSD_stable_10/src/sys/GENERIC -mno-aes -mno-avx -mcmodel=3Dkernel -mno-r= ed-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ff= reestanding -fstack-protector -gdwarf-2 -std=3Diso9899:1999 -Qunused-argume= nts -fstack-protector -Wall -Wredundant-decls -Wnested-externs -Wstrict-pr= ototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wunde= f -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnosti= cs-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno= -error-parentheses-equality -Wno-error-unused-function -c radeonkmsfw_RV= 630_me.c -o radeonkmsfw_RV630_me.o --- all_subdir_ext2fs --- ctfconvert -L VERSION -g ext2_alloc.o --- all_subdir_drm2 --- ctfconvert -L VERSION -g radeonkmsfw_RV630_me.o --- radeonkmsfw_RV630_me.ko.debug --- ld -d -warn-common -r -d -o radeonkmsfw_RV630_me.ko.debug RV630_me.bin.fwo= radeonkmsfw_RV630_me.o --- all_subdir_ext2fs --- --- ext2_balloc.o --- --- all_subdir_drm2 --- ctfmerge -L VERSION -g -o radeonkmsfw_RV630_me.ko.debug RV630_me.bin.fwo ra= deonkmsfw_RV630_me.o --- all_subdir_ext2fs --- cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdin= c -DHAVE_KERNEL_OPTION_HEADERS -include /builds/workspace/FreeBSD_stable_= 10/obj/builds/workspace/FreeBSD_stable_10/src/sys/GENERIC/opt_global.h -I. = -I@ -I@/contrib/altq -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-= frame-pointer -I/builds/workspace/FreeBSD_stable_10/obj/builds/workspace/Fr= eeBSD_stable_10/src/sys/GENERIC -mno-aes -mno-avx -mcmodel=3Dkernel -mno-r= ed-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ff= reestanding -fstack-protector -gdwarf-2 -std=3Diso9899:1999 -Qunused-argume= nts -fstack-protector -Wall -Wredundant-decls -Wnested-externs -Wstrict-pr= ototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wunde= f -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnosti= cs-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno= -error-parentheses-equality -Wno-error-unused-function -c /builds/worksp= ace/FreeBSD_stable_10/src/sys/modules/ext2fs/../../fs/ext2fs/ext2_balloc.c = -o ext2_balloc.o --- all_subdir_drm2 --- :> export_syms awk -f /builds/workspace/FreeBSD_stable_10/src/sys/conf/kmod_syms.awk radeo= nkmsfw_RV630_me.ko.debug export_syms | xargs -J% objcopy % radeonkmsfw_RV6= 30_me.ko.debug --- radeonkmsfw_RV630_me.ko.symbols --- objcopy --only-keep-debug radeonkmsfw_RV630_me.ko.debug radeonkmsfw_RV630_m= e.ko.symbols --- radeonkmsfw_RV630_me.ko --- objcopy --strip-debug --add-gnu-debuglink=3Dradeonkmsfw_RV630_me.ko.symbols= radeonkmsfw_RV630_me.ko.debug radeonkmsfw_RV630_me.ko --- all_subdir_em --- --- e1000_ich8lan.o --- ctfconvert -L VERSION -g e1000_ich8lan.o --- all_subdir_drm2 --- --- all_subdir_RV630_pfp --- =3D=3D=3D> drm2/radeonkmsfw/RV630_pfp (all) --- all_subdir_em --- --- e1000_82571.o --- cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdin= c -I/builds/workspace/FreeBSD_stable_10/src/sys/modules/em/../../dev/e1000= -DHAVE_KERNEL_OPTION_HEADERS -include /builds/workspace/FreeBSD_stable_10/= obj/builds/workspace/FreeBSD_stable_10/src/sys/GENERIC/opt_global.h -I. -I@= -I@/contrib/altq -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-fra= me-pointer -I/builds/workspace/FreeBSD_stable_10/obj/builds/workspace/FreeB= SD_stable_10/src/sys/GENERIC -mno-aes -mno-avx -mcmodel=3Dkernel -mno-red-= zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffree= standing -fstack-protector -gdwarf-2 -std=3Diso9899:1999 -Qunused-arguments= -fstack-protector -Wall -Wredundant-decls -Wnested-externs -Wstrict-proto= types -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -= Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-= show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno-er= ror-parentheses-equality -Wno-error-unused-function -c /builds/workspace= /FreeBSD_stable_10/src/sys/modules/em/../../dev/e1000/e1000_82571.c -o e100= 0_82571.o --- all_subdir_drm2 --- --- RV630_pfp.bin --- uudecode -p /builds/workspace/FreeBSD_stable_10/src/sys/modules/drm2/radeon= kmsfw/RV630_pfp/../../../../contrib/dev/drm2/radeonkmsfw/RV630_pfp.bin.uu >= RV630_pfp.bin --- RV630_pfp.bin.fwo --- RV630_pfp.bin RV630_pfp.bin --- radeonkmsfw_RV630_pfp.o --- cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdin= c -DHAVE_KERNEL_OPTION_HEADERS -include /builds/workspace/FreeBSD_stable_= 10/obj/builds/workspace/FreeBSD_stable_10/src/sys/GENERIC/opt_global.h -I. = -I@ -I@/contrib/altq -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-= frame-pointer -I/builds/workspace/FreeBSD_stable_10/obj/builds/workspace/Fr= eeBSD_stable_10/src/sys/GENERIC -mno-aes -mno-avx -mcmodel=3Dkernel -mno-r= ed-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ff= reestanding -fstack-protector -gdwarf-2 -std=3Diso9899:1999 -Qunused-argume= nts -fstack-protector -Wall -Wredundant-decls -Wnested-externs -Wstrict-pr= ototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wunde= f -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnosti= cs-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno= -error-parentheses-equality -Wno-error-unused-function -c radeonkmsfw_RV= 630_pfp.c -o radeonkmsfw_RV630_pfp.o --- all_subdir_radeonkms --- ctfconvert -L VERSION -g radeon_encoders.o --- all_subdir_radeonkmsfw --- ctfconvert -L VERSION -g radeonkmsfw_RV630_pfp.o --- radeonkmsfw_RV630_pfp.ko.debug --- ld -d -warn-common -r -d -o radeonkmsfw_RV630_pfp.ko.debug RV630_pfp.bin.f= wo radeonkmsfw_RV630_pfp.o ctfmerge -L VERSION -g -o radeonkmsfw_RV630_pfp.ko.debug RV630_pfp.bin.fwo = radeonkmsfw_RV630_pfp.o :> export_syms awk -f /builds/workspace/FreeBSD_stable_10/src/sys/conf/kmod_syms.awk radeo= nkmsfw_RV630_pfp.ko.debug export_syms | xargs -J% objcopy % radeonkmsfw_RV= 630_pfp.ko.debug --- radeonkmsfw_RV630_pfp.ko.symbols --- objcopy --only-keep-debug radeonkmsfw_RV630_pfp.ko.debug radeonkmsfw_RV630_= pfp.ko.symbols --- radeonkmsfw_RV630_pfp.ko --- objcopy --strip-debug --add-gnu-debuglink=3Dradeonkmsfw_RV630_pfp.ko.symbol= s radeonkmsfw_RV630_pfp.ko.debug radeonkmsfw_RV630_pfp.ko --- all_subdir_RV635_me --- =3D=3D=3D> drm2/radeonkmsfw/RV635_me (all) --- all_subdir_ext2fs --- ctfconvert -L VERSION -g ext2_balloc.o --- all_subdir_drm2 --- --- RV635_me.bin --- uudecode -p /builds/workspace/FreeBSD_stable_10/src/sys/modules/drm2/radeon= kmsfw/RV635_me/../../../../contrib/dev/drm2/radeonkmsfw/RV635_me.bin.uu > R= V635_me.bin --- all_subdir_radeonkms --- --- radeon_fb.o --- cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdin= c -DHAVE_KERNEL_OPTION_HEADERS -include /builds/workspace/FreeBSD_stable_= 10/obj/builds/workspace/FreeBSD_stable_10/src/sys/GENERIC/opt_global.h -I. = -I@ -I@/contrib/altq -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-= frame-pointer -I/builds/workspace/FreeBSD_stable_10/obj/builds/workspace/Fr= eeBSD_stable_10/src/sys/GENERIC -mno-aes -mno-avx -mcmodel=3Dkernel -mno-r= ed-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ff= reestanding -fstack-protector -gdwarf-2 -std=3Diso9899:1999 -Qunused-argume= nts -fstack-protector -Wall -Wredundant-decls -Wnested-externs -Wstrict-pr= ototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wunde= f -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnosti= cs-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno= -error-parentheses-equality -Wno-error-unused-function -I/builds/worksp= ace/FreeBSD_stable_10/src/sys/modules/drm2/radeonkms/../../../dev/drm2/rade= on -c /builds/workspace/FreeBSD_stable_10/src/sys/modules/drm2/radeonkms/.= ./../../dev/drm2/radeon/radeon_fb.c -o radeon_fb.o --- all_subdir_radeonkmsfw --- --- RV635_me.bin.fwo --- RV635_me.bin RV635_me.bin --- radeonkmsfw_RV635_me.o --- cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdin= c -DHAVE_KERNEL_OPTION_HEADERS -include /builds/workspace/FreeBSD_stable_= 10/obj/builds/workspace/FreeBSD_stable_10/src/sys/GENERIC/opt_global.h -I. = -I@ -I@/contrib/altq -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-= frame-pointer -I/builds/workspace/FreeBSD_stable_10/obj/builds/workspace/Fr= eeBSD_stable_10/src/sys/GENERIC -mno-aes -mno-avx -mcmodel=3Dkernel -mno-r= ed-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ff= reestanding -fstack-protector -gdwarf-2 -std=3Diso9899:1999 -Qunused-argume= nts -fstack-protector -Wall -Wredundant-decls -Wnested-externs -Wstrict-pr= ototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wunde= f -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnosti= cs-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno= -error-parentheses-equality -Wno-error-unused-function -c radeonkmsfw_RV= 635_me.c -o radeonkmsfw_RV635_me.o --- all_subdir_ext2fs --- --- ext2_bmap.o --- cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdin= c -DHAVE_KERNEL_OPTION_HEADERS -include /builds/workspace/FreeBSD_stable_= 10/obj/builds/workspace/FreeBSD_stable_10/src/sys/GENERIC/opt_global.h -I. = -I@ -I@/contrib/altq -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-= frame-pointer -I/builds/workspace/FreeBSD_stable_10/obj/builds/workspace/Fr= eeBSD_stable_10/src/sys/GENERIC -mno-aes -mno-avx -mcmodel=3Dkernel -mno-r= ed-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ff= reestanding -fstack-protector -gdwarf-2 -std=3Diso9899:1999 -Qunused-argume= nts -fstack-protector -Wall -Wredundant-decls -Wnested-externs -Wstrict-pr= ototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wunde= f -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnosti= cs-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno= -error-parentheses-equality -Wno-error-unused-function -c /builds/worksp= ace/FreeBSD_stable_10/src/sys/modules/ext2fs/../../fs/ext2fs/ext2_bmap.c -o= ext2_bmap.o --- all_subdir_drm2 --- ctfconvert -L VERSION -g radeonkmsfw_RV635_me.o --- radeonkmsfw_RV635_me.ko.debug --- ld -d -warn-common -r -d -o radeonkmsfw_RV635_me.ko.debug RV635_me.bin.fwo= radeonkmsfw_RV635_me.o ctfmerge -L VERSION -g -o radeonkmsfw_RV635_me.ko.debug RV635_me.bin.fwo ra= deonkmsfw_RV635_me.o :> export_syms awk -f /builds/workspace/FreeBSD_stable_10/src/sys/conf/kmod_syms.awk radeo= nkmsfw_RV635_me.ko.debug export_syms | xargs -J% objcopy % radeonkmsfw_RV6= 35_me.ko.debug --- radeonkmsfw_RV635_me.ko.symbols --- objcopy --only-keep-debug radeonkmsfw_RV635_me.ko.debug radeonkmsfw_RV635_m= e.ko.symbols --- radeonkmsfw_RV635_me.ko --- objcopy --strip-debug --add-gnu-debuglink=3Dradeonkmsfw_RV635_me.ko.symbols= radeonkmsfw_RV635_me.ko.debug radeonkmsfw_RV635_me.ko --- all_subdir_RV635_pfp --- =3D=3D=3D> drm2/radeonkmsfw/RV635_pfp (all) --- RV635_pfp.bin --- uudecode -p /builds/workspace/FreeBSD_stable_10/src/sys/modules/drm2/radeon= kmsfw/RV635_pfp/../../../../contrib/dev/drm2/radeonkmsfw/RV635_pfp.bin.uu >= RV635_pfp.bin --- RV635_pfp.bin.fwo --- RV635_pfp.bin RV635_pfp.bin --- radeonkmsfw_RV635_pfp.o --- cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdin= c -DHAVE_KERNEL_OPTION_HEADERS -include /builds/workspace/FreeBSD_stable_= 10/obj/builds/workspace/FreeBSD_stable_10/src/sys/GENERIC/opt_global.h -I. = -I@ -I@/contrib/altq -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-= frame-pointer -I/builds/workspace/FreeBSD_stable_10/obj/builds/workspace/Fr= eeBSD_stable_10/src/sys/GENERIC -mno-aes -mno-avx -mcmodel=3Dkernel -mno-r= ed-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ff= reestanding -fstack-protector -gdwarf-2 -std=3Diso9899:1999 -Qunused-argume= nts -fstack-protector -Wall -Wredundant-decls -Wnested-externs -Wstrict-pr= ototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wunde= f -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnosti= cs-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno= -error-parentheses-equality -Wno-error-unused-function -c radeonkmsfw_RV= 635_pfp.c -o radeonkmsfw_RV635_pfp.o ctfconvert -L VERSION -g radeonkmsfw_RV635_pfp.o --- radeonkmsfw_RV635_pfp.ko.debug --- ld -d -warn-common -r -d -o radeonkmsfw_RV635_pfp.ko.debug RV635_pfp.bin.f= wo radeonkmsfw_RV635_pfp.o ctfmerge -L VERSION -g -o radeonkmsfw_RV635_pfp.ko.debug RV635_pfp.bin.fwo = radeonkmsfw_RV635_pfp.o :> export_syms awk -f /builds/workspace/FreeBSD_stable_10/src/sys/conf/kmod_syms.awk radeo= nkmsfw_RV635_pfp.ko.debug export_syms | xargs -J% objcopy % radeonkmsfw_RV= 635_pfp.ko.debug --- radeonkmsfw_RV635_pfp.ko.symbols --- --- all_subdir_ext2fs --- ctfconvert -L VERSION -g ext2_bmap.o --- all_subdir_drm2 --- objcopy --only-keep-debug radeonkmsfw_RV635_pfp.ko.debug radeonkmsfw_RV635_= pfp.ko.symbols --- radeonkmsfw_RV635_pfp.ko --- objcopy --strip-debug --add-gnu-debuglink=3Dradeonkmsfw_RV635_pfp.ko.symbol= s radeonkmsfw_RV635_pfp.ko.debug radeonkmsfw_RV635_pfp.ko --- all_subdir_RV670_me --- =3D=3D=3D> drm2/radeonkmsfw/RV670_me (all) --- all_subdir_ext2fs --- --- ext2_extents.o --- cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdin= c -DHAVE_KERNEL_OPTION_HEADERS -include /builds/workspace/FreeBSD_stable_= 10/obj/builds/workspace/FreeBSD_stable_10/src/sys/GENERIC/opt_global.h -I. = -I@ -I@/contrib/altq -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-= frame-pointer -I/builds/workspace/FreeBSD_stable_10/obj/builds/workspace/Fr= eeBSD_stable_10/src/sys/GENERIC -mno-aes -mno-avx -mcmodel=3Dkernel -mno-r= ed-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ff= reestanding -fstack-protector -gdwarf-2 -std=3Diso9899:1999 -Qunused-argume= nts -fstack-protector -Wall -Wredundant-decls -Wnested-externs -Wstrict-pr= ototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wunde= f -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnosti= cs-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno= -error-parentheses-equality -Wno-error-unused-function -c /builds/worksp= ace/FreeBSD_stable_10/src/sys/modules/ext2fs/../../fs/ext2fs/ext2_extents.c= -o ext2_extents.o --- all_subdir_drm2 --- --- RV670_me.bin --- uudecode -p /builds/workspace/FreeBSD_stable_10/src/sys/modules/drm2/radeon= kmsfw/RV670_me/../../../../contrib/dev/drm2/radeonkmsfw/RV670_me.bin.uu > R= V670_me.bin --- RV670_me.bin.fwo --- RV670_me.bin RV670_me.bin --- radeonkmsfw_RV670_me.o --- cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdin= c -DHAVE_KERNEL_OPTION_HEADERS -include /builds/workspace/FreeBSD_stable_= 10/obj/builds/workspace/FreeBSD_stable_10/src/sys/GENERIC/opt_global.h -I. = -I@ -I@/contrib/altq -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-= frame-pointer -I/builds/workspace/FreeBSD_stable_10/obj/builds/workspace/Fr= eeBSD_stable_10/src/sys/GENERIC -mno-aes -mno-avx -mcmodel=3Dkernel -mno-r= ed-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ff= reestanding -fstack-protector -gdwarf-2 -std=3Diso9899:1999 -Qunused-argume= nts -fstack-protector -Wall -Wredundant-decls -Wnested-externs -Wstrict-pr= ototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wunde= f -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnosti= cs-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno= -error-parentheses-equality -Wno-error-unused-function -c radeonkmsfw_RV= 670_me.c -o radeonkmsfw_RV670_me.o ctfconvert -L VERSION -g radeonkmsfw_RV670_me.o --- radeonkmsfw_RV670_me.ko.debug --- ld -d -warn-common -r -d -o radeonkmsfw_RV670_me.ko.debug RV670_me.bin.fwo= radeonkmsfw_RV670_me.o ctfmerge -L VERSION -g -o radeonkmsfw_RV670_me.ko.debug RV670_me.bin.fwo ra= deonkmsfw_RV670_me.o :> export_syms awk -f /builds/workspace/FreeBSD_stable_10/src/sys/conf/kmod_syms.awk radeo= nkmsfw_RV670_me.ko.debug export_syms | xargs -J% objcopy % radeonkmsfw_RV6= 70_me.ko.debug --- radeonkmsfw_RV670_me.ko.symbols --- objcopy --only-keep-debug radeonkmsfw_RV670_me.ko.debug radeonkmsfw_RV670_m= e.ko.symbols --- radeonkmsfw_RV670_me.ko --- objcopy --strip-debug --add-gnu-debuglink=3Dradeonkmsfw_RV670_me.ko.symbols= radeonkmsfw_RV670_me.ko.debug radeonkmsfw_RV670_me.ko --- all_subdir_RV670_pfp --- =3D=3D=3D> drm2/radeonkmsfw/RV670_pfp (all) --- all_subdir_radeonkms --- ctfconvert -L VERSION -g radeon_fb.o --- all_subdir_em --- ctfconvert -L VERSION -g e1000_82571.o --- e1000_82575.o --- cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdin= c -I/builds/workspace/FreeBSD_stable_10/src/sys/modules/em/../../dev/e1000= -DHAVE_KERNEL_OPTION_HEADERS -include /builds/workspace/FreeBSD_stable_10/= obj/builds/workspace/FreeBSD_stable_10/src/sys/GENERIC/opt_global.h -I. -I@= -I@/contrib/altq -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-fra= me-pointer -I/builds/workspace/FreeBSD_stable_10/obj/builds/workspace/FreeB= SD_stable_10/src/sys/GENERIC -mno-aes -mno-avx -mcmodel=3Dkernel -mno-red-= zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffree= standing -fstack-protector -gdwarf-2 -std=3Diso9899:1999 -Qunused-arguments= -fstack-protector -Wall -Wredundant-decls -Wnested-externs -Wstrict-proto= types -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -= Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-= show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno-er= ror-parentheses-equality -Wno-error-unused-function -c /builds/workspace= /FreeBSD_stable_10/src/sys/modules/em/../../dev/e1000/e1000_82575.c -o e100= 0_82575.o --- all_subdir_ext2fs --- ctfconvert -L VERSION -g ext2_extents.o --- ext2_hash.o --- cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdin= c -DHAVE_KERNEL_OPTION_HEADERS -include /builds/workspace/FreeBSD_stable_= 10/obj/builds/workspace/FreeBSD_stable_10/src/sys/GENERIC/opt_global.h -I. = -I@ -I@/contrib/altq -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-= frame-pointer -I/builds/workspace/FreeBSD_stable_10/obj/builds/workspace/Fr= eeBSD_stable_10/src/sys/GENERIC -mno-aes -mno-avx -mcmodel=3Dkernel -mno-r= ed-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ff= reestanding -fstack-protector -gdwarf-2 -std=3Diso9899:1999 -Qunused-argume= nts -fstack-protector -Wall -Wredundant-decls -Wnested-externs -Wstrict-pr= ototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wunde= f -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnosti= cs-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno= -error-parentheses-equality -Wno-error-unused-function -c /builds/worksp= ace/FreeBSD_stable_10/src/sys/modules/ext2fs/../../fs/ext2fs/ext2_hash.c -o= ext2_hash.o --- all_subdir_drm2 --- --- radeon_fence.o --- cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdin= c -DHAVE_KERNEL_OPTION_HEADERS -include /builds/workspace/FreeBSD_stable_= 10/obj/builds/workspace/FreeBSD_stable_10/src/sys/GENERIC/opt_global.h -I. = -I@ -I@/contrib/altq -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-= frame-pointer -I/builds/workspace/FreeBSD_stable_10/obj/builds/workspace/Fr= eeBSD_stable_10/src/sys/GENERIC -mno-aes -mno-avx -mcmodel=3Dkernel -mno-r= ed-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ff= reestanding -fstack-protector -gdwarf-2 -std=3Diso9899:1999 -Qunused-argume= nts -fstack-protector -Wall -Wredundant-decls -Wnested-externs -Wstrict-pr= ototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wunde= f -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnosti= cs-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno= -error-parentheses-equality -Wno-error-unused-function -I/builds/worksp= ace/FreeBSD_stable_10/src/sys/modules/drm2/radeonkms/../../../dev/drm2/rade= on -c /builds/workspace/FreeBSD_stable_10/src/sys/modules/drm2/radeonkms/.= ./../../dev/drm2/radeon/radeon_fence.c -o radeon_fence.o ctfconvert -L VERSION -g radeon_fence.o --- radeon_gart.o --- cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdin= c -DHAVE_KERNEL_OPTION_HEADERS -include /builds/workspace/FreeBSD_stable_= 10/obj/builds/workspace/FreeBSD_stable_10/src/sys/GENERIC/opt_global.h -I. = -I@ -I@/contrib/altq -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-= frame-pointer -I/builds/workspace/FreeBSD_stable_10/obj/builds/workspace/Fr= eeBSD_stable_10/src/sys/GENERIC -mno-aes -mno-avx -mcmodel=3Dkernel -mno-r= ed-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ff= reestanding -fstack-protector -gdwarf-2 -std=3Diso9899:1999 -Qunused-argume= nts -fstack-protector -Wall -Wredundant-decls -Wnested-externs -Wstrict-pr= ototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wunde= f -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnosti= cs-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno= -error-parentheses-equality -Wno-error-unused-function -I/builds/worksp= ace/FreeBSD_stable_10/src/sys/modules/drm2/radeonkms/../../../dev/drm2/rade= on -c /builds/workspace/FreeBSD_stable_10/src/sys/modules/drm2/radeonkms/.= ./../../dev/drm2/radeon/radeon_gart.c -o radeon_gart.o --- all_subdir_em --- ctfconvert -L VERSION -g e1000_82575.o --- if_em.ko.debug --- ld -d -warn-common -r -d -o if_em.ko.debug if_em.o e1000_osdep.o if_lem.o = e1000_api.o e1000_phy.o e1000_nvm.o e1000_mac.o e1000_manage.o e1000_vf.o e= 1000_mbx.o e1000_i210.o e1000_82540.o e1000_82542.o e1000_82541.o e1000_825= 43.o e1000_80003es2lan.o e1000_ich8lan.o e1000_82571.o e1000_82575.o ctfmerge -L VERSION -g -o if_em.ko.debug if_em.o e1000_osdep.o if_lem.o e10= 00_api.o e1000_phy.o e1000_nvm.o e1000_mac.o e1000_manage.o e1000_vf.o e100= 0_mbx.o e1000_i210.o e1000_82540.o e1000_82542.o e1000_82541.o e1000_82543.= o e1000_80003es2lan.o e1000_ich8lan.o e1000_82571.o e1000_82575.o :> export_syms awk -f /builds/workspace/FreeBSD_stable_10/src/sys/conf/kmod_syms.awk if_em= .ko.debug export_syms | xargs -J% objcopy % if_em.ko.debug --- if_em.ko.symbols --- objcopy --only-keep-debug if_em.ko.debug if_em.ko.symbols --- if_em.ko --- objcopy --strip-debug --add-gnu-debuglink=3Dif_em.ko.symbols if_em.ko.debug= if_em.ko --- all_subdir_fatm --- =3D=3D=3D> fatm (all) --- all_subdir_drm2 --- ctfconvert -L VERSION -g radeon_gart.o --- radeon_gem.o --- cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdin= c -DHAVE_KERNEL_OPTION_HEADERS -include /builds/workspace/FreeBSD_stable_= 10/obj/builds/workspace/FreeBSD_stable_10/src/sys/GENERIC/opt_global.h -I. = -I@ -I@/contrib/altq -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-= frame-pointer -I/builds/workspace/FreeBSD_stable_10/obj/builds/workspace/Fr= eeBSD_stable_10/src/sys/GENERIC -mno-aes -mno-avx -mcmodel=3Dkernel -mno-r= ed-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ff= reestanding -fstack-protector -gdwarf-2 -std=3Diso9899:1999 -Qunused-argume= nts -fstack-protector -Wall -Wredundant-decls -Wnested-externs -Wstrict-pr= ototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wunde= f -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnosti= cs-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno= -error-parentheses-equality -Wno-error-unused-function -I/builds/worksp= ace/FreeBSD_stable_10/src/sys/modules/drm2/radeonkms/../../../dev/drm2/rade= on -c /builds/workspace/FreeBSD_stable_10/src/sys/modules/drm2/radeonkms/.= ./../../dev/drm2/radeon/radeon_gem.c -o radeon_gem.o Resuming build at Sat Sep 24 20:34:15 GMT 2016 after Jenkins restart Waiting to resume part of FreeBSD_stable_10 #408: ??? Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Resuming build at Sun Sep 25 06:06:14 GMT 2016 after Jenkins restart Waiting to resume part of FreeBSD_stable_10 #408: ??? Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Waiting to resume part of FreeBSD_stable_10 #408: jenkins-10.freebsd.org is= offline Ready to run at Sun Sep 25 06:41:56 GMT 2016 [Pipeline] } [Pipeline] // withEnv [Pipeline] } [Pipeline] // dir [Pipeline] } [Pipeline] // node [Pipeline] node Running on master in /usr/local/jenkins/workspace/FreeBSD_stable_10 [Pipeline] { [Pipeline] step From owner-freebsd-stable@freebsd.org Sun Sep 25 07:13:34 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EFEABBE97D4; Sun, 25 Sep 2016 07:13:34 +0000 (UTC) (envelope-from russ.haley@gmail.com) Received: from mail-ua0-x230.google.com (mail-ua0-x230.google.com [IPv6:2607:f8b0:400c:c08::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AA9D71AC; Sun, 25 Sep 2016 07:13:34 +0000 (UTC) (envelope-from russ.haley@gmail.com) Received: by mail-ua0-x230.google.com with SMTP id q42so80503861uaq.1; Sun, 25 Sep 2016 00:13:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=3w18S6CCpMbBbLQxKd8MkMZ4N+6E7eGEY06c06Td6pU=; b=PTl/HldNjG7KiUs5wNtM0cJ+lhvcsw93zL7vgz224BD/yU/MmsPxHAQuv6Iveuv6p0 ra15D6naSj8E53PANRefr/Qxr4JF6YtBYZ+BlNo5vzHAnjOIRwCPrk3t7e5oCkTyLPmC HNcIA7Jvpu2ZYHhy1uvDK3hs2Eeuzh26/GXvrEOQUGx15j/2J5BXTQimUo4J4Kmi8zrJ qCfB7wI0muhzRKfO8j6wNKPJxBSurfrpDRYk4jsP3B65U3JqZZRCfnyT7YzKPJsoGgWc jSeBb08KgsVDvGDrwU0Eya6M4BhLUIGSFX9cWbPXCN9jTvLtUzYMwBHiK+wm+HoPI1NC p7aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=3w18S6CCpMbBbLQxKd8MkMZ4N+6E7eGEY06c06Td6pU=; b=EDjEi3vxlwwptxgugmnxDZP5qruis0vAU0eY2CgMs6kk+Ra4tvvKX30PodvEZArghi 8GiJiuKKHl9L5j67zr+bKk5GUjb2b0Oo3JNFJSP8p3Bac/h5yVkd6o8xoC4RSJPaT4Ko BsvZPl55VBw5VyOtFTc+Qo1MqJ0CUiBrlz4jKXYFSTW96rFEcMax1S1exKx8n3hbbzGV 2tGEixTOJi/pU536ah2Zi85g4h5Ivnk8aG53hn7wB+L7f5fJrtiqcnkE3Fdcbi7WhZOJ uLsPTBLian5nG9CNuFYFRKScCbCrLGey+ymUujfqDWTNudS2PxxhnByEPG5pqqjv3kAM vlOA== X-Gm-Message-State: AA6/9Rnbm1pfW2+SabB+2ADBqYVgn+xQxd4SNag1W4oC3Oc9kI8jPxGfXhMlclM5CiwjwungLRl1EXyT9EIVFA== X-Received: by 10.176.3.197 with SMTP id 63mr7815345uau.149.1474787613540; Sun, 25 Sep 2016 00:13:33 -0700 (PDT) MIME-Version: 1.0 Received: by 10.31.124.202 with HTTP; Sun, 25 Sep 2016 00:13:31 -0700 (PDT) In-Reply-To: References: <4076CFFA-7BE2-4E1B-A7E8-08FD8FC27D21@dsl-only.net> <332FA120-31E5-4D31-B63E-A0DFDD7DEFC7@dsl-only.net> <4F04B0C0-DAB1-4EEA-A3A0-4FAA89AD93E2@dsl-only.net> From: Russell Haley Date: Sun, 25 Sep 2016 00:13:31 -0700 Message-ID: Subject: Re: 11.0-RELEASE tier level for arm64/aaarch64 and the officially built arm/armv6 variants? To: Warner Losh Cc: Mark Millard , freebsd-arm , FreeBSD-STABLE Mailing List Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 07:13:35 -0000 On Sat, Sep 24, 2016 at 11:10 PM, Warner Losh wrote: > On Sat, Sep 24, 2016 at 10:21 PM, Russell Haley wr= ote: >> On Sat, Sep 24, 2016 at 5:12 PM, Mark Millard wrot= e: >>> On 2016-Sep-24, at 2:11 PM, Warner Losh wrote: >>> >>>> On Fri, Sep 23, 2016 at 8:29 PM, Mark Millard wr= ote: >>>>> [A resend since I forget to list free-arm in the To: the first time.] >>>>> >>>>> From https://www.freebsd.org/platforms/arm.html : >>>>> >>>>>> 32-bit ARM is officially a Tier 2 architecture, as the FreeBSD proje= ct does not provide official releases or pre-built packages for this platfo= rm due to it primarily targeting the embedded arena. However, FreeBSD/ARM i= s being actively developed and maintained, is well supported, and provides = an excellent framework for building ARM-based systems. FreeBSD/arm supports= ARMv4 and ARMv5 processors. FreeBSD/armv6 supports ARMv6 and ARMv7 process= ors, including SMP on the latter. >>>>> >>>>> "does not provide official releases or pre-built packages"? >>>>> >>>>>> # uname -apKU >>>>>> FreeBSD rpi2 11.0-PRERELEASE FreeBSD 11.0-PRERELEASE #5 r304943M: Su= n Aug 28 03:17:54 PDT 2016 markmi@FreeBSDx64:/usr/obj/clang/arm.armv6/u= sr/src/sys/RPI2-NODBG arm armv6 1100502 1100502 >>>>> >>>>>> # pkg search '.*' | wc >>>>>> 21349 155540 1596736 >>>>> >>>>> Will 11.0-RELEASE change the tier level for any of the specific arm-a= rmv6 variants that have FreeBSD-11.0-*-arm-armv6-*.img* files built, such a= s for RPI2? >>>>> >>>>> Even if all the officially built arm-armv6 variants stay tier 2, the = wording on the web page likely needs to be changed because so much is built= and available that the above quote claims is not available. >>>> >>>> armv6 is basically Tier 1 right now, though not as Tier 1 as i386 or >>>> amd64 due to the fragmented nature of the arm world. On the platforms >>>> we run on and create releases for, however, it's my opinion that it is >>>> Tier 1: it has been running in production a while, things people >>>> expect from a FreeBSD system are present, you can get decent support >>>> if you ask questions, there's no known major gotchas in deploying this >>>> hardware. The only remaining annoying issue is the 'u-boot' problem >>>> where we have to have a different u-boot image for every board and no >>>> standardized way to convert a 'generic' image into one that's specific >>>> for specific boards. >> >> I'll point out again that barebox is an excellent alternative to u-boot = (IMHO): > > Doesn't matter, still has the same issues that u-boot has. u-boot has a different sources for almost each board we support (due to the usual FOSS issues). That is NOT the case in barebox. There is one source and it's kept up to date by the team, not the vendors. > But does it support the u-boot ABI? I'd have to look into this. > >> - Supports most, if not all of the boards that FreeBSD supports, plus >> many that it doesn't >> - Single source tree for all boards. Specify build time parameters to >> build one or all the images >> - Well supported community with central maintainer-ship >> - Simple, familiar shell interface (*awesome*) >> - Excellent documentation (u-boots is good too though) >> - Has support for (U)EFI >> - Supports quemu aarch64 (not *quite*sure what the means though) > > Right, u-boot has all these things, except maybe the shell interface > (not sure what you mean by that). Instead of stringing together variables and commands, it uses a scripting language like a simplified sh. Want to change how something boots? Update a script. Save it to disk (it has it's own persistence mechanisms) and export it. >> To be fair, I'm not saying the problem is the fault of denx, but >> barebox has a lot going for it. The maintainer was very keen to see it >> ported top FreeBSD and was willing to support the effort. I ran into >> some build time linux api requirements, but he didn't think that would >> be much to overcome (and it wasn't I just kept adding the patches he >> sent me and the build moved forward. As always, I ran out of time for >> the really fun stuff). While I am a hopeless dreamer and I'm sure I've >> over simplified the problem, I thought it would be neat to see FreeBSD >> upstream support for zfs and ufs to the barebox boot loader and do >> away with ubldr. We would then have a modern, easy to use, boot loader >> that supports the standard startup toolchain. > > We can't easily do away with ubldr if we want to support tunables, kernel > modules loaded at boot time and a few other nifty features like nextboot. Are these things not in standard loader? Should they be? >> Either way, if installers move to a pkgng based method (so cool) then >> installing u-boot and arm binaries from pkg-static will be the same as >> x86 (ha ha I said that with a straight face!). > > Yea, not so much. You have to build the bootable image not on the > target system, like you do on x86. Doesn't the current ports and packages cross build everything that's marked= ? > We'd have to have something that > installs uboot onto a generic image (perhaps with hooks for kernels > since those aren't generic on armv6) and then put that into a bootable > SD card. The x86 installer (that I argue is a platform legacy) has to customize the bootloader for each installation. If we HAVE to use an installer on arm, what wrong prompting the user for some input (i.e. what som are you using) while including all the u-boots in the ports tree and all the supported kernels, then just installing the correct ones for the board (with input from the user)? >>>>For x86 this is all done with the installer since >>>> that boot environment is more standardized. Does this last issue keep >>>> arm from being Tier 1? That's a judgement call, but I think the >>>> project should promote w/o this last issue. >> >> How does a platform get promoted? Is that something the Core team decide= s? > > Yes. > >> I see two facts about current Arm support that show platform maturity: >> a) u-boot is in the ports tree and we have Lego-easy build scripts in >> crochet that could be called an installation method. Building for arm >> is not difficult anymore. > > Except corchet isn't in the tree, and the solution is horrible. It's > a script for each SoC, and those scripts are now scattered about. > Plus that's a creation from source model, not a creation from > RE produced bits model, which is needed for Tier 1. Both correctable sins, especially as it's BSD licensed. My point was that even building a SOM specific image is relatively painless with the right scripts. Hell, I've even got a custom build script that could be modified for generic use. >> b)Arm requires images, not installers. Correct me if I'm wrong but, >> installers are a tool primarily invented for x86 PC type computers. >> FreeBSD publishes standardized ISOs for all supported Arm platforms >> that work by simply "xzcat | dd" onto the sd card (or wherever you >> need it). I'm not sure how standardized or manual that build process >> is, but I would think that if the Arm platform support is able to keep >> up with the standard FreeBSD release cycle (i.e. not break every other >> release) then there would be no reason to NOT call it tier 1? > > Creating the images is currently a pita. That's it. I see. So not so mature. >> What I don't know about is "official" documentation for building for >> arm and supporting cross building to Arm. Will someone need to write >> an "Arm Handbook" to be promoted? > > No. While useful, most of that already exists. Thanks for the response Warner, I always appreciate the chance to learn more about FreeBSD. Russ > Warner > >>> Interesting and good to know. Thanks. >>> >>> I might have guessed that going along with the u-boot issue would be th= e fanout in: >>> >>> 11.0/sys/arm/ . . . >>> >>> allwinner/a10/ >>> allwinner/a20/ >>> allwinner/a31/ >>> allwinner/a83t/ >>> allwinner/h3/ >>> . . . >>> broadcom/bcm2835/ >>> . . . >>> >>> (Full list not shown.) >>> >>> I was thinking that this might make the tier level specific to the stat= us of each such directory's content so that it was the combination of that = and the sysutils/u-boot-*/ status that made the difference for assigning th= e level. I'd guess that lack of a usable directory in either place would n= ot be tier 2 even. Similarly until the required sys/arm/*/* and sysutils/u-= boot-*/ directory-tree content have reached a sufficiently complete status. >>> >>> I'd expect that there will always be a lag for what exists in the world= vs. what has these materials worked out in FreeBSD. >>> >>> >>>>> Also from https://www.freebsd.org/platforms/arm.html : >>>>> >>>>>> Initial support for 64-bit ARM is complete. 64-bit ARM platforms fol= low a set of standard conventions, and a single FreeBSD build will work on = hardware from multiple vendors. As a result, FreeBSD will provide official = releases for FreeBSD/arm64 and packages will be available. FreeBSD/arm64 is= on the path to becoming a Tier 1 architecture. >>>>> >>>>> Will 11.0-RELEASE make arm64/aarch64 Tier 1? >>>>> >>>>> [I will note that, while there are no official builds for the Pine64 = family (A64 based) that are under the Allwinner arm activity, the SOC's inv= olved are Cortex-A53 64-bit arm based. They likely do not fit in the "stand= ard conventions" or arm64/aarch64 would be where they would have been suppo= rted. Some rewording might be appropriate for the above quote as well.] >>>> >>>> No. aarch64 isn't Tier 1 yet. There's many small bits that are >>>> missing. It is quite solidly Tier 2, but we don't have a linker, we >>>> don't have widespread hardware availability, we don't have production >>>> experience with the platform. Most things work, but there's still some >>>> gotchas. There's still the 'u-boot' problem with many arm64 systems >>>> because for systems that use u-boot to bootstrap UEFI, you need a >>>> different image for each board (some closely related board families >>>> can get by with one to be pedantic). All these issues are still >>>> significant barriers to production use. It's not been officially >>>> promoted yet and I don't think the time is quite right yet. >>> >>> Intersting as well. I'd guess that conceptually this probably would app= ly to both: >>> >>> sys/arm/allwinner/a64/ and sysutils/u-boot/pine64/ >>> (presuming, contrary to fact, that 11.0 had sys/arm/allwinner/a64/ ) >>> >>> and. . . >>> >>> sys/arm64/cavium/ >>> sys/arm64/cloudabi64/ >>> >>> So just sys/arm/ vs. sys/arm64/ for an aarch64 would not really make a = difference yet for tier level. >>> >>>> Warner >>> >>> Thanks again for the notes. >>> >>> =3D=3D=3D >>> Mark Millard >>> markmi at dsl-only.net >>> >>> _______________________________________________ >>> freebsd-arm@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-arm >>> To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Sun Sep 25 09:06:19 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0BF8FBE97EA; Sun, 25 Sep 2016 09:06:19 +0000 (UTC) (envelope-from manu@bidouilliste.com) Received: from mail.blih.net (mail.blih.net [212.83.177.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.blih.net", Issuer "mail.blih.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6F50161A; Sun, 25 Sep 2016 09:06:17 +0000 (UTC) (envelope-from manu@bidouilliste.com) Received: from mail.blih.net (mail.blih.net [212.83.177.182]) by mail.blih.net (OpenSMTPD) with ESMTP id e1f576f8; Sun, 25 Sep 2016 10:59:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=bidouilliste.com; h=date :from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=mail; bh=Q/AEsWSuJ7fPslsjr+gDPT7sa3I=; b=FsTM63OtxNLGwlKEPGy2I6zlnh0L LFPckQOmuM50VPx2G8iypvuFxdK3UXtRQBD9xqQRmaV4P9V/u6zNwm6yUYPiHI5s f4yhr4cFFZrUwRnhhl5dcK4Od1JPtVEK/TNOI6KexI/YmmcBz+XGvNXCnYHlxg2y CXlianbQqNgvFKs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=bidouilliste.com; h=date :from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; q=dns; s= mail; b=oabMqTLMB2cjHXnrsUiIy/IYmcS/M4+1n9DzCNdZv/vn2MH68kwybjV0 IKthBV+DcUQaxrXJ3fnqVCLGLYrevPc7x0ToPb3da/TqNrT41Z+kO3SczzmNgoTY z9uaMbJNuuq5yv5+fuUTV3E/mI3mxU2Q0oznXhoFpvA+4dLOxrQ= Received: from knuckles.blih.net (free-229-194.mediaworksit.net [109.111.229.194]) by mail.blih.net (OpenSMTPD) with ESMTPSA id e737a702 TLS version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO; Sun, 25 Sep 2016 10:59:33 +0200 (CEST) Date: Sun, 25 Sep 2016 10:59:30 +0200 From: Emmanuel Vadot To: Russell Haley Cc: Warner Losh , freebsd-arm , FreeBSD-STABLE Mailing List , Mark Millard Subject: Re: 11.0-RELEASE tier level for arm64/aaarch64 and the officially built arm/armv6 variants? Message-Id: <20160925105930.2039e17f47a177a7465c9ee7@bidouilliste.com> In-Reply-To: References: <4076CFFA-7BE2-4E1B-A7E8-08FD8FC27D21@dsl-only.net> <332FA120-31E5-4D31-B63E-A0DFDD7DEFC7@dsl-only.net> <4F04B0C0-DAB1-4EEA-A3A0-4FAA89AD93E2@dsl-only.net> X-Mailer: Sylpheed 3.5.1 (GTK+ 2.24.29; amd64-portbld-freebsd12.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 09:06:19 -0000 On Sun, 25 Sep 2016 00:13:31 -0700 Russell Haley wrote: > On Sat, Sep 24, 2016 at 11:10 PM, Warner Losh wrote: > > On Sat, Sep 24, 2016 at 10:21 PM, Russell Haley wrote: > >> On Sat, Sep 24, 2016 at 5:12 PM, Mark Millard wrote: > >>> On 2016-Sep-24, at 2:11 PM, Warner Losh wrote: > >>> > >>>> On Fri, Sep 23, 2016 at 8:29 PM, Mark Millard wrote: > >>>>> [A resend since I forget to list free-arm in the To: the first time.] > >>>>> > >>>>> From https://www.freebsd.org/platforms/arm.html : > >>>>> > >>>>>> 32-bit ARM is officially a Tier 2 architecture, as the FreeBSD project does not provide official releases or pre-built packages for this platform due to it primarily targeting the embedded arena. However, FreeBSD/ARM is being actively developed and maintained, is well supported, and provides an excellent framework for building ARM-based systems. FreeBSD/arm supports ARMv4 and ARMv5 processors. FreeBSD/armv6 supports ARMv6 and ARMv7 processors, including SMP on the latter. > >>>>> > >>>>> "does not provide official releases or pre-built packages"? > >>>>> > >>>>>> # uname -apKU > >>>>>> FreeBSD rpi2 11.0-PRERELEASE FreeBSD 11.0-PRERELEASE #5 r304943M: Sun Aug 28 03:17:54 PDT 2016 markmi@FreeBSDx64:/usr/obj/clang/arm.armv6/usr/src/sys/RPI2-NODBG arm armv6 1100502 1100502 > >>>>> > >>>>>> # pkg search '.*' | wc > >>>>>> 21349 155540 1596736 > >>>>> > >>>>> Will 11.0-RELEASE change the tier level for any of the specific arm-armv6 variants that have FreeBSD-11.0-*-arm-armv6-*.img* files built, such as for RPI2? > >>>>> > >>>>> Even if all the officially built arm-armv6 variants stay tier 2, the wording on the web page likely needs to be changed because so much is built and available that the above quote claims is not available. > >>>> > >>>> armv6 is basically Tier 1 right now, though not as Tier 1 as i386 or > >>>> amd64 due to the fragmented nature of the arm world. On the platforms > >>>> we run on and create releases for, however, it's my opinion that it is > >>>> Tier 1: it has been running in production a while, things people > >>>> expect from a FreeBSD system are present, you can get decent support > >>>> if you ask questions, there's no known major gotchas in deploying this > >>>> hardware. The only remaining annoying issue is the 'u-boot' problem > >>>> where we have to have a different u-boot image for every board and no > >>>> standardized way to convert a 'generic' image into one that's specific > >>>> for specific boards. > >> > >> I'll point out again that barebox is an excellent alternative to u-boot (IMHO): > > > > Doesn't matter, still has the same issues that u-boot has. > u-boot has a different sources for almost each board we support (due > to the usual FOSS issues). That is NOT the case in barebox. There is > one source and it's kept up to date by the team, not the vendors. This is not true, U-Boot support all the platforms we are running on right now in a single source tree. I think that the only ports that is not using the main U-Boot is the wandboard one and I think that Warner have it working now. > > But does it support the u-boot ABI? > I'd have to look into this. > > > >> - Supports most, if not all of the boards that FreeBSD supports, plus > >> many that it doesn't > >> - Single source tree for all boards. Specify build time parameters to > >> build one or all the images > >> - Well supported community with central maintainer-ship > >> - Simple, familiar shell interface (*awesome*) > >> - Excellent documentation (u-boots is good too though) > >> - Has support for (U)EFI > >> - Supports quemu aarch64 (not *quite*sure what the means though) > > > > Right, u-boot has all these things, except maybe the shell interface > > (not sure what you mean by that). > Instead of stringing together variables and commands, it uses a > scripting language like a simplified sh. Want to change how something > boots? Update a script. Save it to disk (it has it's own persistence > mechanisms) and export it. You can do the same with U-Boot. > >> To be fair, I'm not saying the problem is the fault of denx, but > >> barebox has a lot going for it. The maintainer was very keen to see it > >> ported top FreeBSD and was willing to support the effort. I ran into > >> some build time linux api requirements, but he didn't think that would > >> be much to overcome (and it wasn't I just kept adding the patches he > >> sent me and the build moved forward. As always, I ran out of time for > >> the really fun stuff). While I am a hopeless dreamer and I'm sure I've > >> over simplified the problem, I thought it would be neat to see FreeBSD > >> upstream support for zfs and ufs to the barebox boot loader and do > >> away with ubldr. We would then have a modern, easy to use, boot loader > >> that supports the standard startup toolchain. > > > > We can't easily do away with ubldr if we want to support tunables, kernel > > modules loaded at boot time and a few other nifty features like nextboot. > > Are these things not in standard loader? Should they be? They are in the standard loader, using ubldr,loader or loader.efi doesn't matter but we have to use one of them. > >> Either way, if installers move to a pkgng based method (so cool) then > >> installing u-boot and arm binaries from pkg-static will be the same as > >> x86 (ha ha I said that with a straight face!). > > > > Yea, not so much. You have to build the bootable image not on the > > target system, like you do on x86. > > Doesn't the current ports and packages cross build everything that's marked? > > > We'd have to have something that > > installs uboot onto a generic image (perhaps with hooks for kernels > > since those aren't generic on armv6) and then put that into a bootable > > SD card. > > The x86 installer (that I argue is a platform legacy) has to customize > the bootloader for each installation. If we HAVE to use an installer > on arm, what wrong prompting the user for some input (i.e. what som > are you using) while including all the u-boots in the ports tree and > all the supported kernels, then just installing the correct ones for > the board (with input from the user)? > > >>>>For x86 this is all done with the installer since > >>>> that boot environment is more standardized. Does this last issue keep > >>>> arm from being Tier 1? That's a judgement call, but I think the > >>>> project should promote w/o this last issue. > >> > >> How does a platform get promoted? Is that something the Core team decides? > > > > Yes. > > > >> I see two facts about current Arm support that show platform maturity: > >> a) u-boot is in the ports tree and we have Lego-easy build scripts in > >> crochet that could be called an installation method. Building for arm > >> is not difficult anymore. > > > > Except corchet isn't in the tree, and the solution is horrible. It's > > a script for each SoC, and those scripts are now scattered about. > > Plus that's a creation from source model, not a creation from > > RE produced bits model, which is needed for Tier 1. > Both correctable sins, especially as it's BSD licensed. My point was > that even building a SOM specific image is relatively painless with > the right scripts. Hell, I've even got a custom build script that > could be modified for generic use. > > >> b)Arm requires images, not installers. Correct me if I'm wrong but, > >> installers are a tool primarily invented for x86 PC type computers. > >> FreeBSD publishes standardized ISOs for all supported Arm platforms > >> that work by simply "xzcat | dd" onto the sd card (or wherever you > >> need it). I'm not sure how standardized or manual that build process > >> is, but I would think that if the Arm platform support is able to keep > >> up with the standard FreeBSD release cycle (i.e. not break every other > >> release) then there would be no reason to NOT call it tier 1? > > > > Creating the images is currently a pita. That's it. > I see. So not so mature. > > >> What I don't know about is "official" documentation for building for > >> arm and supporting cross building to Arm. Will someone need to write > >> an "Arm Handbook" to be promoted? > > > > No. While useful, most of that already exists. > > Thanks for the response Warner, I always appreciate the chance to > learn more about FreeBSD. > Russ > > > Warner > > > >>> Interesting and good to know. Thanks. > >>> > >>> I might have guessed that going along with the u-boot issue would be the fanout in: > >>> > >>> 11.0/sys/arm/ . . . > >>> > >>> allwinner/a10/ > >>> allwinner/a20/ > >>> allwinner/a31/ > >>> allwinner/a83t/ > >>> allwinner/h3/ > >>> . . . > >>> broadcom/bcm2835/ > >>> . . . > >>> > >>> (Full list not shown.) > >>> > >>> I was thinking that this might make the tier level specific to the status of each such directory's content so that it was the combination of that and the sysutils/u-boot-*/ status that made the difference for assigning the level. I'd guess that lack of a usable directory in either place would not be tier 2 even. Similarly until the required sys/arm/*/* and sysutils/u-boot-*/ directory-tree content have reached a sufficiently complete status. > >>> > >>> I'd expect that there will always be a lag for what exists in the world vs. what has these materials worked out in FreeBSD. > >>> > >>> > >>>>> Also from https://www.freebsd.org/platforms/arm.html : > >>>>> > >>>>>> Initial support for 64-bit ARM is complete. 64-bit ARM platforms follow a set of standard conventions, and a single FreeBSD build will work on hardware from multiple vendors. As a result, FreeBSD will provide official releases for FreeBSD/arm64 and packages will be available. FreeBSD/arm64 is on the path to becoming a Tier 1 architecture. > >>>>> > >>>>> Will 11.0-RELEASE make arm64/aarch64 Tier 1? > >>>>> > >>>>> [I will note that, while there are no official builds for the Pine64 family (A64 based) that are under the Allwinner arm activity, the SOC's involved are Cortex-A53 64-bit arm based. They likely do not fit in the "standard conventions" or arm64/aarch64 would be where they would have been supported. Some rewording might be appropriate for the above quote as well.] > >>>> > >>>> No. aarch64 isn't Tier 1 yet. There's many small bits that are > >>>> missing. It is quite solidly Tier 2, but we don't have a linker, we > >>>> don't have widespread hardware availability, we don't have production > >>>> experience with the platform. Most things work, but there's still some > >>>> gotchas. There's still the 'u-boot' problem with many arm64 systems > >>>> because for systems that use u-boot to bootstrap UEFI, you need a > >>>> different image for each board (some closely related board families > >>>> can get by with one to be pedantic). All these issues are still > >>>> significant barriers to production use. It's not been officially > >>>> promoted yet and I don't think the time is quite right yet. > >>> > >>> Intersting as well. I'd guess that conceptually this probably would apply to both: > >>> > >>> sys/arm/allwinner/a64/ and sysutils/u-boot/pine64/ > >>> (presuming, contrary to fact, that 11.0 had sys/arm/allwinner/a64/ ) > >>> > >>> and. . . > >>> > >>> sys/arm64/cavium/ > >>> sys/arm64/cloudabi64/ > >>> > >>> So just sys/arm/ vs. sys/arm64/ for an aarch64 would not really make a difference yet for tier level. > >>> > >>>> Warner > >>> > >>> Thanks again for the notes. > >>> > >>> === > >>> Mark Millard > >>> markmi at dsl-only.net > >>> > >>> _______________________________________________ > >>> freebsd-arm@freebsd.org mailing list > >>> https://lists.freebsd.org/mailman/listinfo/freebsd-arm > >>> To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" -- Emmanuel Vadot From owner-freebsd-stable@freebsd.org Sun Sep 25 09:55:35 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5392EBE6596 for ; Sun, 25 Sep 2016 09:55:35 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-57.reflexion.net [208.70.210.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 091901C99 for ; Sun, 25 Sep 2016 09:55:34 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 20353 invoked from network); 25 Sep 2016 09:56:18 -0000 Received: from unknown (HELO mail-cs-01.app.dca.reflexion.local) (10.81.19.1) by 0 (rfx-qmail) with SMTP; 25 Sep 2016 09:56:18 -0000 Received: by mail-cs-01.app.dca.reflexion.local (Reflexion email security v8.00.0) with SMTP; Sun, 25 Sep 2016 05:55:32 -0400 (EDT) Received: (qmail 28027 invoked from network); 25 Sep 2016 09:55:32 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 25 Sep 2016 09:55:32 -0000 Received: from [192.168.0.105] (ip70-189-131-151.lv.lv.cox.net [70.189.131.151]) by iron2.pdx.net (Postfix) with ESMTPSA id 48356EC8A8B; Sun, 25 Sep 2016 02:55:26 -0700 (PDT) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Just FYI: FreeBSD-11.0-RELEASE-arm64-aarch64.raw under qemu on odroid-c2 Ubuntu 16.04.1 LTS Message-Id: <27466846-3031-47C7-82FE-808FCE920E69@dsl-only.net> Date: Sun, 25 Sep 2016 02:55:25 -0700 To: FreeBSD-STABLE Mailing List , freebsd-arm Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 09:55:35 -0000 This is just an FYI about my attempt to use = FreeBSD-11.0-RELEASE-arm64-aarch64.raw under Ubuntu's qemu on an = odroid-c2. (This is my first ever use of qemu. I've no clue about how = well the qemu for this context works --or how well for any other = context.) Context of use of FreeBSD 11.0-RELEASE: > Welcome to Ubuntu 16.04.1 LTS (GNU/Linux 3.14.79-83 aarch64) . . . > root@odroid64:~# uname -ap > Linux odroid64 3.14.79-83 #1 SMP PREEMPT Thu Sep 22 13:47:47 BRT 2016 = aarch64 aarch64 aarch64 GNU/Linux > qemu-system-aarch64 -m 1024M -enable-kvm -cpu host -M virt \ > -bios QEMU_EFI.fd -nographic \ > -drive = format=3Draw,if=3Dnone,file=3DFreeBSD-11.0-RELEASE-arm64-aarch64.raw,id=3D= hd0 \ > -device virtio-blk-device,drive=3Dhd0 \ > -device virtio-net-device,netdev=3Dnet0 \ > -netdev user,id=3Dnet0 \ > -smp cpus=3D4 Note the "-nographic". The QEMU_EFI.fd is from: = https://releases.linaro.org/components/kernel/uefi-linaro/15.12/release/qe= mu64/QEMU_EFI.fd > # freebsd-version -ku; uname -apKU > 11.0-RELEASE > 11.0-RELEASE > FreeBSD odroidc2FBSD 11.0-RELEASE FreeBSD 11.0-RELEASE #0 r306211: Fri = Sep 23 11:42:02 UTC 2016 = root@releng2.nyi.freebsd.org:/usr/obj/arm64.aarch64/usr/src/sys/GENERIC = arm64 aarch64 1100122 1100122 Issue #0: "random" illegal instruction faults in FreeBSD use Example *.core file names for an occasional "illegal instruction" (other = times the same command works fine): cron.core fsck_ufs.core ifconfig.core login.core ntpd.core route.core sh.core shutdown.core top.core vi.core All these programs usually/frequently work fine. Some programs tend to not leave .core files and cause me to have to log = in again when they get the illegal instruction fault. ps is a command = that has had this happen. Issue #1: command input/output stops For both the serial console and an ssh into Ubuntu from which I start up = qemu for FreeBSD. . . Special keys, such as up-arrow for command line recall, stop the = input/output for typing commands and such. It is ignoring what is typed = (not just not displaying it). I end up having to kill the = qemu-system-aarch64 process from Ubuntu or use "Control-a x". Sometimes = "Control-a x" does not work. Sometimes pasting text into the window cause such input/output hangups = in FreeBSD use. Usually qemu's "Control-A x" sequence will quit qemu. So usually there = must be some amount of processing of what is typed but at some stage the = text is otherwise being ignored. (I normally kill qemu-system-aarch64 = from another Ubuntu command shell.) Other point(s): I've not managed to get FreeBSD qemu to even ping a numeric address ("No = route to host"). But this may be just my current ignorance of how things = are supposed to be configured, both in Ubuntu and in FreeBSD and on the = command line options to qemu-system-aarch64. I've not put much effort = into figuring such out given the more basic problems above. (The = environment the attempt was done from is dhcp based.) =3D=3D=3D Mark Millard markmi at dsl-only.net From owner-freebsd-stable@freebsd.org Sun Sep 25 10:05:26 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A12FDBE6ABD for ; Sun, 25 Sep 2016 10:05:26 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id 93640260; Sun, 25 Sep 2016 10:05:26 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id B7937CD; Sun, 25 Sep 2016 10:05:26 +0000 (UTC) Date: Sun, 25 Sep 2016 10:05:26 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: jenkins-admin@FreeBSD.org, freebsd-stable@FreeBSD.org Message-ID: <1778604001.2.1474797926632.JavaMail.jenkins@jenkins-9.freebsd.org> In-Reply-To: <735755714.0.1474785726003.JavaMail.jenkins@jenkins-9.freebsd.org> References: <735755714.0.1474785726003.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Jenkins build is back to normal : FreeBSD_stable_10 #409 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkKKb2VAfYQKfu1t7qk4nR5qzUBEI+UqT4BPec4qHVhqUy0FFdq50sMH+3y9bCDNOufctov6VqTNffZ3YXArnZK95YF0OX97fh+E9txYOUX1adc+TikcKjuYpHmL5dE62eaZTI+4A5jnRonskQ1PaoIFz0Kbu4mWzkFsmdiXTraGzomXq4cHUCATA2+K4eDYgjXEQI30z3GOMmmZ4t/+6QGk1cMb/BqMWHbn80AsRCb4tU7Hpd72XLDpsuO7YRP1Q0CjmNAuBOTj+sFiiOe6U9HpqOlQN+iFUvBdZo/ybuy5Kh71cAaYQNL68cYdZJ6binH/DkG3KY/fS7DFYAeuwjwIDAQAB X-Jenkins-Job: FreeBSD_stable_10 X-Jenkins-Result: SUCCESS X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 10:05:26 -0000 See From owner-freebsd-stable@freebsd.org Sun Sep 25 12:46:36 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 07C74BE566A for ; Sun, 25 Sep 2016 12:46:36 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BD07A405 for ; Sun, 25 Sep 2016 12:46:35 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1bo8p8-000L2B-5k; Sun, 25 Sep 2016 15:46:26 +0300 Date: Sun, 25 Sep 2016 15:46:26 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20160925124626.GI2840@zxy.spb.ru> References: <20160916183053.GL9397@strugglingcoder.info> <20160916190330.GG2840@zxy.spb.ru> <78cbcdc9-f565-1046-c157-2ddd8fcccc62@freebsd.org> <20160919204328.GN2840@zxy.spb.ru> <8ba75d6e-4f01-895e-0aed-53c6c6692cb9@freebsd.org> <20160920202633.GQ2840@zxy.spb.ru> <20160921195155.GW2840@zxy.spb.ru> <20160923200143.GG2840@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160923200143.GG2840@zxy.spb.ru> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 12:46:36 -0000 On Fri, Sep 23, 2016 at 11:01:43PM +0300, Slawa Olhovchenkov wrote: > On Wed, Sep 21, 2016 at 11:25:18PM +0200, Julien Charbon wrote: > > > > > Hi Slawa, > > > > On 9/21/16 9:51 PM, Slawa Olhovchenkov wrote: > > > On Wed, Sep 21, 2016 at 09:11:24AM +0200, Julien Charbon wrote: > > >> You can also use Dtrace and lockstat (especially with the lockstat -s > > >> option): > > >> > > >> https://wiki.freebsd.org/DTrace/One-Liners#Kernel_Locks > > >> https://www.freebsd.org/cgi/man.cgi?query=lockstat&manpath=FreeBSD+11.0-RELEASE > > >> > > >> But I am less familiar with Dtrace/lockstat tools. > > > > > > I am still use old kernel and got lockdown again. > > > Try using lockstat (I am save more output), interesting may be next: > > > > > > R/W writer spin on writer: 190019 events in 1.070 seconds (177571 events/sec) > > > > > > ------------------------------------------------------------------------------- > > > Count indv cuml rcnt nsec Lock Caller > > > 140839 74% 74% 0.00 24659 tcpinp tcp_tw_2msl_scan+0xc6 > > > > > > nsec ------ Time Distribution ------ count Stack > > > 4096 | 913 tcp_twstart+0xa3 > > > 8192 |@@@@@@@@@@@@ 58191 tcp_do_segment+0x201f > > > 16384 |@@@@@@ 29594 tcp_input+0xe1c > > > 32768 |@@@@ 23447 ip_input+0x15f > > > 65536 |@@@ 16197 > > > 131072 |@ 8674 > > > 262144 | 3358 > > > 524288 | 456 > > > 1048576 | 9 > > > ------------------------------------------------------------------------------- > > > Count indv cuml rcnt nsec Lock Caller > > > 49180 26% 100% 0.00 15929 tcpinp tcp_tw_2msl_scan+0xc6 > > > > > > nsec ------ Time Distribution ------ count Stack > > > 4096 | 157 pfslowtimo+0x54 > > > 8192 |@@@@@@@@@@@@@@@ 24796 softclock_call_cc+0x179 > > > 16384 |@@@@@@ 11223 softclock+0x44 > > > 32768 |@@@@ 7426 intr_event_execute_handlers+0x95 > > > 65536 |@@ 3918 > > > 131072 | 1363 > > > 262144 | 278 > > > 524288 | 19 > > > ------------------------------------------------------------------------------- > > > > This is interesting, it seems that you have two call paths competing > > for INP locks here: > > > > - pfslowtimo()/tcp_tw_2msl_scan(reuse=0) and > > > > - tcp_input()/tcp_twstart()/tcp_tw_2msl_scan(reuse=1) > > My current hypothesis: > > nginx do write() (or may be close()?) to socket, kernel lock > first inp in V_twq_2msl, happen callout for pfslowtimo() on the same > CPU core and tcp_tw_2msl_scan infinity locked on same inp. > > In this case you modification can't help, before next try we need some > like yeld(). Or may be locks leaks. Or both. From owner-freebsd-stable@freebsd.org Sun Sep 25 15:57:29 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 13351BE8273; Sun, 25 Sep 2016 15:57:29 +0000 (UTC) (envelope-from tim@kientzle.com) Received: from monday.kientzle.com (kientzle.com [142.254.26.11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E3947EE8; Sun, 25 Sep 2016 15:57:28 +0000 (UTC) (envelope-from tim@kientzle.com) Received: (from root@localhost) by monday.kientzle.com (8.14.4/8.14.4) id u8PFvISP062229; Sun, 25 Sep 2016 15:57:18 GMT (envelope-from tim@kientzle.com) Received: from [192.168.2.106] (192.168.1.101 [192.168.1.101]) by kientzle.com with SMTP id w39erfebfkspmgvzbf2vpmx3vi; Sun, 25 Sep 2016 15:57:18 +0000 (UTC) (envelope-from tim@kientzle.com) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.1 \(3243\)) Subject: Re: 11.0-RELEASE tier level for arm64/aaarch64 and the officially built arm/armv6 variants? From: Tim Kientzle In-Reply-To: Date: Sun, 25 Sep 2016 08:57:18 -0700 Cc: Warner Losh , freebsd-arm , FreeBSD-STABLE Mailing List Content-Transfer-Encoding: 7bit Message-Id: <9BFF1495-8142-4C2F-8919-D9A870D3E2BD@kientzle.com> References: <4076CFFA-7BE2-4E1B-A7E8-08FD8FC27D21@dsl-only.net> <332FA120-31E5-4D31-B63E-A0DFDD7DEFC7@dsl-only.net> <4F04B0C0-DAB1-4EEA-A3A0-4FAA89AD93E2@dsl-only.net> To: Russell Haley X-Mailer: Apple Mail (2.3243) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 15:57:29 -0000 > On Sep 25, 2016, at 12:13 AM, Russell Haley wrote: > >> We can't easily do away with ubldr if we want to support tunables, kernel >> modules loaded at boot time and a few other nifty features like nextboot. > > Are these things not in standard loader? Should they be? "ubldr" *is* the standard loader. It's built from (mostly) the same source as loader(8) used on x86. But, where loader(8) uses the BIOS interface to access the disk, "ubldr" uses the U-Boot ABI to access the disk. FreeBSD's current boot sequence for ARM boards looks like this: * U-Boot loads ubldr * ubldr uses U-Boot to access disk and console * ubldr loads the kernel, kernel modules, and sets kernel tunables To replace U-Boot, bare box would either have to duplicate everything ubldr does (which is a lot) or would have to provide the U-Boot ABI (or something similar) so that ubldr can provide the final boot stage. Tim From owner-freebsd-stable@freebsd.org Sun Sep 25 17:19:56 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E15CFBE98DB for ; Sun, 25 Sep 2016 17:19:56 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x22c.google.com (mail-it0-x22c.google.com [IPv6:2607:f8b0:4001:c0b::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A52FAB1A for ; Sun, 25 Sep 2016 17:19:56 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x22c.google.com with SMTP id r192so55402621ita.0 for ; Sun, 25 Sep 2016 10:19:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-transfer-encoding; bh=qGvEMyB29f9K9/MEbbEa5i3Kmfo/gfnVwXX/y/Yv6wc=; b=B+xivf9WzjKeUlCQbMpYfsYB/p0IQ84iTg0G2q5XCW0ihCVYpOl3WjP2x5x6IDFcbw iPRVgakKjp2wVqdNjpMT3HSDubIHJvwxg3dgn4qhmdJl7LtKIRPk/n4yq77amHTUg7AB Wrhvr/kEL9JJyVIuvmb4MFZHNM7aD+iffMPMLzm0dBmQ91Z22+W+WM5xM1R4G9ysXJLx snIF4LaAcPYen9CliUeaYSQ/8iRKBXwQ0cAEmzxwyvQ9FoyKlSYvgWuD1dpmiBBjyMdG pgpVBHKMbbQIRdkAjw8kXG/QHFiNXOWSGO45/qBzxgJg9GBVCnAXPg3GtVQru5HWfy4r uwIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-transfer-encoding; bh=qGvEMyB29f9K9/MEbbEa5i3Kmfo/gfnVwXX/y/Yv6wc=; b=gbbICysEsI/zQXma8HFC3TO2ksx8lr2NHATBk/P8GkbifLC3JOGJY+Nxr8GG57vyE0 DWuALT6DGwkJLLnkaSjxfpwf7Jxm9W46jjFO0RoY8TPNFLTgK6guErmuumfmjeRs4Bt8 eQEqaRsseS0tDjz2Sg9nNODVGml60gXZ4Lzfe7/kxP2J/yDsVRpt5GjirlOtuqo9Efz6 O40SudD+/Xd8Q/YJBZ0FJ/0uVC3Axj+mveyElasN45P5DpHzP97cOC6YJ/feI0np4ugS 6n+KJfwmtTQ23A0eNsg7pCG5mhnsZJ6ui+dcYEFVC8K8CU+d/lWndFPpb6qRkeZ+WS3/ QtIg== X-Gm-Message-State: AA6/9RkR5txE9ACCPhJanKSBs0khTi7n+YFAiEy1QviPoFzPkPcg60ocrfxTBGs2p7CtRqvb+/a24YGQfWmtSA== X-Received: by 10.36.14.68 with SMTP id 65mr12200559ite.99.1474823995988; Sun, 25 Sep 2016 10:19:55 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.36.65.7 with HTTP; Sun, 25 Sep 2016 10:19:55 -0700 (PDT) X-Originating-IP: [50.253.99.174] In-Reply-To: References: <4076CFFA-7BE2-4E1B-A7E8-08FD8FC27D21@dsl-only.net> <332FA120-31E5-4D31-B63E-A0DFDD7DEFC7@dsl-only.net> <4F04B0C0-DAB1-4EEA-A3A0-4FAA89AD93E2@dsl-only.net> From: Warner Losh Date: Sun, 25 Sep 2016 11:19:55 -0600 X-Google-Sender-Auth: LsklrTS85t_g121n76IX46KNqBc Message-ID: Subject: Re: 11.0-RELEASE tier level for arm64/aaarch64 and the officially built arm/armv6 variants? To: Russell Haley Cc: Mark Millard , freebsd-arm , FreeBSD-STABLE Mailing List Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 17:19:57 -0000 On Sun, Sep 25, 2016 at 1:13 AM, Russell Haley wrote= : > On Sat, Sep 24, 2016 at 11:10 PM, Warner Losh wrote: >> On Sat, Sep 24, 2016 at 10:21 PM, Russell Haley w= rote: >>> On Sat, Sep 24, 2016 at 5:12 PM, Mark Millard wro= te: >>>> On 2016-Sep-24, at 2:11 PM, Warner Losh wrote: >>>> >>>>> On Fri, Sep 23, 2016 at 8:29 PM, Mark Millard w= rote: >>>>>> [A resend since I forget to list free-arm in the To: the first time.= ] >>>>>> >>>>>> From https://www.freebsd.org/platforms/arm.html : >>>>>> >>>>>>> 32-bit ARM is officially a Tier 2 architecture, as the FreeBSD proj= ect does not provide official releases or pre-built packages for this platf= orm due to it primarily targeting the embedded arena. However, FreeBSD/ARM = is being actively developed and maintained, is well supported, and provides= an excellent framework for building ARM-based systems. FreeBSD/arm support= s ARMv4 and ARMv5 processors. FreeBSD/armv6 supports ARMv6 and ARMv7 proces= sors, including SMP on the latter. >>>>>> >>>>>> "does not provide official releases or pre-built packages"? >>>>>> >>>>>>> # uname -apKU >>>>>>> FreeBSD rpi2 11.0-PRERELEASE FreeBSD 11.0-PRERELEASE #5 r304943M: S= un Aug 28 03:17:54 PDT 2016 markmi@FreeBSDx64:/usr/obj/clang/arm.armv6/= usr/src/sys/RPI2-NODBG arm armv6 1100502 1100502 >>>>>> >>>>>>> # pkg search '.*' | wc >>>>>>> 21349 155540 1596736 >>>>>> >>>>>> Will 11.0-RELEASE change the tier level for any of the specific arm-= armv6 variants that have FreeBSD-11.0-*-arm-armv6-*.img* files built, such = as for RPI2? >>>>>> >>>>>> Even if all the officially built arm-armv6 variants stay tier 2, the= wording on the web page likely needs to be changed because so much is buil= t and available that the above quote claims is not available. >>>>> >>>>> armv6 is basically Tier 1 right now, though not as Tier 1 as i386 or >>>>> amd64 due to the fragmented nature of the arm world. On the platforms >>>>> we run on and create releases for, however, it's my opinion that it i= s >>>>> Tier 1: it has been running in production a while, things people >>>>> expect from a FreeBSD system are present, you can get decent support >>>>> if you ask questions, there's no known major gotchas in deploying thi= s >>>>> hardware. The only remaining annoying issue is the 'u-boot' problem >>>>> where we have to have a different u-boot image for every board and no >>>>> standardized way to convert a 'generic' image into one that's specifi= c >>>>> for specific boards. >>> >>> I'll point out again that barebox is an excellent alternative to u-boot= (IMHO): >> >> Doesn't matter, still has the same issues that u-boot has. > u-boot has a different sources for almost each board we support (due > to the usual FOSS issues). That is NOT the case in barebox. There is > one source and it's kept up to date by the team, not the vendors. Actually, that's no longer the case. All boards we support have all their bits in u-boot's main repo. But you can't say no one will fork barebox to get their support out faster, which is why we have the current situation in ports... u-boot forked because it was so popular, and the forks rejoined. >> But does it support the u-boot ABI? > I'd have to look into this. >> >>> - Supports most, if not all of the boards that FreeBSD supports, plus >>> many that it doesn't >>> - Single source tree for all boards. Specify build time parameters to >>> build one or all the images >>> - Well supported community with central maintainer-ship >>> - Simple, familiar shell interface (*awesome*) >>> - Excellent documentation (u-boots is good too though) >>> - Has support for (U)EFI >>> - Supports quemu aarch64 (not *quite*sure what the means though) >> >> Right, u-boot has all these things, except maybe the shell interface >> (not sure what you mean by that). > Instead of stringing together variables and commands, it uses a > scripting language like a simplified sh. Want to change how something > boots? Update a script. Save it to disk (it has it's own persistence > mechanisms) and export it. Sounds cool. >>> To be fair, I'm not saying the problem is the fault of denx, but >>> barebox has a lot going for it. The maintainer was very keen to see it >>> ported top FreeBSD and was willing to support the effort. I ran into >>> some build time linux api requirements, but he didn't think that would >>> be much to overcome (and it wasn't I just kept adding the patches he >>> sent me and the build moved forward. As always, I ran out of time for >>> the really fun stuff). While I am a hopeless dreamer and I'm sure I've >>> over simplified the problem, I thought it would be neat to see FreeBSD >>> upstream support for zfs and ufs to the barebox boot loader and do >>> away with ubldr. We would then have a modern, easy to use, boot loader >>> that supports the standard startup toolchain. >> >> We can't easily do away with ubldr if we want to support tunables, kerne= l >> modules loaded at boot time and a few other nifty features like nextboot= . > > Are these things not in standard loader? Should they be? ubldr is the standard loader, just spelled in a funky way and living on a place u-boot can see :). We have different loaders for different environments. even on x86 we have /boot/loader for BIOS boots and /boot/loader.efi for EFI boots. >>> Either way, if installers move to a pkgng based method (so cool) then >>> installing u-boot and arm binaries from pkg-static will be the same as >>> x86 (ha ha I said that with a straight face!). >> >> Yea, not so much. You have to build the bootable image not on the >> target system, like you do on x86. > > Doesn't the current ports and packages cross build everything that's mark= ed? For builds from source, yes. But that's different than installing from FreeBSD's installer. >> We'd have to have something that >> installs uboot onto a generic image (perhaps with hooks for kernels >> since those aren't generic on armv6) and then put that into a bootable >> SD card. > > The x86 installer (that I argue is a platform legacy) has to customize > the bootloader for each installation. If we HAVE to use an installer > on arm, what wrong prompting the user for some input (i.e. what som > are you using) while including all the u-boots in the ports tree and > all the supported kernels, then just installing the correct ones for > the board (with input from the user)? Ummm, you can't boot anything to do that work like you can on x86. >>>>>For x86 this is all done with the installer since >>>>> that boot environment is more standardized. Does this last issue keep >>>>> arm from being Tier 1? That's a judgement call, but I think the >>>>> project should promote w/o this last issue. >>> >>> How does a platform get promoted? Is that something the Core team decid= es? >> >> Yes. >> >>> I see two facts about current Arm support that show platform maturity: >>> a) u-boot is in the ports tree and we have Lego-easy build scripts in >>> crochet that could be called an installation method. Building for arm >>> is not difficult anymore. >> >> Except corchet isn't in the tree, and the solution is horrible. It's >> a script for each SoC, and those scripts are now scattered about. >> Plus that's a creation from source model, not a creation from >> RE produced bits model, which is needed for Tier 1. > Both correctable sins, especially as it's BSD licensed. My point was > that even building a SOM specific image is relatively painless with > the right scripts. Hell, I've even got a custom build script that > could be modified for generic use. Yea, a developer can do it with little pain. However, compared to the pain to boot Linux on these boards, the pain is quite high and you need to know way too much to make it work. BTW, my 'horrible' comment was given the state of things today: we have scattered knowledge of how to make images, custom one-off that work on some boards well, and fail on others. We have crazy, quirky things that aren't well integrated into the tree. Crochet itself solves an interesting problem, but does so a bit in isolation. >>> b)Arm requires images, not installers. Correct me if I'm wrong but, >>> installers are a tool primarily invented for x86 PC type computers. >>> FreeBSD publishes standardized ISOs for all supported Arm platforms >>> that work by simply "xzcat | dd" onto the sd card (or wherever you >>> need it). I'm not sure how standardized or manual that build process >>> is, but I would think that if the Arm platform support is able to keep >>> up with the standard FreeBSD release cycle (i.e. not break every other >>> release) then there would be no reason to NOT call it tier 1? >> >> Creating the images is currently a pita. That's it. > I see. So not so mature. Yes. It's an area I work on when I have time to make better... Warner >>> What I don't know about is "official" documentation for building for >>> arm and supporting cross building to Arm. Will someone need to write >>> an "Arm Handbook" to be promoted? >> >> No. While useful, most of that already exists. > > Thanks for the response Warner, I always appreciate the chance to > learn more about FreeBSD. > Russ > >> Warner >> >>>> Interesting and good to know. Thanks. >>>> >>>> I might have guessed that going along with the u-boot issue would be t= he fanout in: >>>> >>>> 11.0/sys/arm/ . . . >>>> >>>> allwinner/a10/ >>>> allwinner/a20/ >>>> allwinner/a31/ >>>> allwinner/a83t/ >>>> allwinner/h3/ >>>> . . . >>>> broadcom/bcm2835/ >>>> . . . >>>> >>>> (Full list not shown.) >>>> >>>> I was thinking that this might make the tier level specific to the sta= tus of each such directory's content so that it was the combination of that= and the sysutils/u-boot-*/ status that made the difference for assigning t= he level. I'd guess that lack of a usable directory in either place would = not be tier 2 even. Similarly until the required sys/arm/*/* and sysutils/u= -boot-*/ directory-tree content have reached a sufficiently complete status= . >>>> >>>> I'd expect that there will always be a lag for what exists in the worl= d vs. what has these materials worked out in FreeBSD. >>>> >>>> >>>>>> Also from https://www.freebsd.org/platforms/arm.html : >>>>>> >>>>>>> Initial support for 64-bit ARM is complete. 64-bit ARM platforms fo= llow a set of standard conventions, and a single FreeBSD build will work on= hardware from multiple vendors. As a result, FreeBSD will provide official= releases for FreeBSD/arm64 and packages will be available. FreeBSD/arm64 i= s on the path to becoming a Tier 1 architecture. >>>>>> >>>>>> Will 11.0-RELEASE make arm64/aarch64 Tier 1? >>>>>> >>>>>> [I will note that, while there are no official builds for the Pine64= family (A64 based) that are under the Allwinner arm activity, the SOC's in= volved are Cortex-A53 64-bit arm based. They likely do not fit in the "stan= dard conventions" or arm64/aarch64 would be where they would have been supp= orted. Some rewording might be appropriate for the above quote as well.] >>>>> >>>>> No. aarch64 isn't Tier 1 yet. There's many small bits that are >>>>> missing. It is quite solidly Tier 2, but we don't have a linker, we >>>>> don't have widespread hardware availability, we don't have production >>>>> experience with the platform. Most things work, but there's still som= e >>>>> gotchas. There's still the 'u-boot' problem with many arm64 systems >>>>> because for systems that use u-boot to bootstrap UEFI, you need a >>>>> different image for each board (some closely related board families >>>>> can get by with one to be pedantic). All these issues are still >>>>> significant barriers to production use. It's not been officially >>>>> promoted yet and I don't think the time is quite right yet. >>>> >>>> Intersting as well. I'd guess that conceptually this probably would ap= ply to both: >>>> >>>> sys/arm/allwinner/a64/ and sysutils/u-boot/pine64/ >>>> (presuming, contrary to fact, that 11.0 had sys/arm/allwinner/a64/ ) >>>> >>>> and. . . >>>> >>>> sys/arm64/cavium/ >>>> sys/arm64/cloudabi64/ >>>> >>>> So just sys/arm/ vs. sys/arm64/ for an aarch64 would not really make a= difference yet for tier level. >>>> >>>>> Warner >>>> >>>> Thanks again for the notes. >>>> >>>> =3D=3D=3D >>>> Mark Millard >>>> markmi at dsl-only.net >>>> >>>> _______________________________________________ >>>> freebsd-arm@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-arm >>>> To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Sun Sep 25 17:56:40 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E276BBDE2DF for ; Sun, 25 Sep 2016 17:56:40 +0000 (UTC) (envelope-from ef249bb7dc81bccedd80545b6ea8a6b1.user@mail04100.e2b.descontosnet.com.br) Received: from mail04100.e2b.descontosnet.com.br (mail04100.e2b.descontosnet.com.br [149.56.169.100]) by mx1.freebsd.org (Postfix) with SMTP id 7483A1C89 for ; Sun, 25 Sep 2016 17:56:39 +0000 (UTC) (envelope-from ef249bb7dc81bccedd80545b6ea8a6b1.user@mail04100.e2b.descontosnet.com.br) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=e2b.descontosnet.com.br; s=sender; t=1474826197; bh=ond0JnAuS8jdlvos++TNgCQjKpE=; h=From:To:Date:Subject:Message-Id:Reply-To:Errors-To: List-Unsubscribe:X-Report-Abuse:X-Sender:X-Identity:X-Campaign: X-Campaignid:X-RPTags:MIME-Version:Content-Type; b=RgCeMDBQJ+xgMMbhg4AGCYaAsdPbnhnlQAOOaVvsGcNT6OKvICNBSEE6iFufpFByG s+unNOSpWsmbByjtRVUKwfGg1r1YSKNCroPdwa9PUGGH/i85mKTMSN6fAJrhyrCvF5 u4U/YvtL+MY0aQxMXhgd1uwGsJNcAjzpNhQXEPVU= DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; d=e2b.descontosnet.com.br; s=sender; h=From:To:Date:Subject:Message-Id:Reply-To:Errors-To:List-Unsubscribe:X-Report-Abuse:X-Sender:X-Identity:X-Campaign:X-Campaignid:X-RPTags:MIME-Version:Content-Type; b=MiMIokzfFwq0wBGCEK0jDHaaBRc4VFjzqXP5zk7AS2QgpmQOpE8ZL7qt0DFRHe/61C3kjcl9bGG1WmiR02fL8HoibqaOSdAOD8IJOhfGaxVvFh7e/KcFJmDmGWVi1fFk+yuvScrfj/kQnZF3efAESy6SNnMeyfDR96+blLlFRT0=; From: bestdayviagens@e2b.descontosnet.com.br To: freebsd-stable@freebsd.org Date: Sun, 25 Sep 2016 14:56:37 -0300 Subject: =?utf-8?Q?Ultimo=20Dia!=20Cancun=20All=20Inclusive=2061%OFF=20+B=C3=B4nus=20de=20300=20no=20Hotel=20Oasis?= Message-Id: Reply-To: bestdayviagens@descontosnet.com.br Errors-To: ef249bb7dc81bccedd80545b6ea8a6b1-c8f40e6f2da7ede337905ed2d45b7cc8.user@e2b.descontosnet.com.br X-Report-Abuse: Please report abuse for this campaign here: http://WebApp.e2b.descontosnet.com.br/util/Unsubscribe.php?TkEnvio=ef249bb7dc81bccedd80545b6ea8a6b1&Srv=c8f40e6f2da7ede337905ed2d45b7cc8 X-Sender: c8f40e6f2da7ede337905ed2d45b7cc8 X-Identity: ef249bb7dc81bccedd80545b6ea8a6b1 X-Campaign: c21dc439da359878ac7f450ca2ac8f9e X-Campaignid: c21dc439da359878ac7f450ca2ac8f9e X-RPTags: 45beff490a7f6cee79372cc0d3088df9 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 17:56:41 -0000 [logoheader-img](https://www.bestday.com.br/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca-02&utm_term=logo-header-bestday) 0800.522.7826 [Hotéis](https://www.bestday.com.br/hoteis?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca-02&utm_term=menu-hoteis) | [Pacotes](https://www.bestday.com.br/pacotes?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca-02&utm_term=menu-pacotes) | [Passagens](https://www.bestday.com.br/passagens?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca-02&utm_term=menu-passagens) | [Passeios](https://www.bestday.com.br/passeios/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca-02&utm_term=menu-passeios) [bannerprincipal02-img](https://www.bestday.com.br/ofertas/?camp=2518&asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-02-cancun-oasis) [banner-img](https://www.bestday.com.br/passagens/latam/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca-02&utm_term=mini-banner-01-promolatam) [PRAIAS](https://www.bestday.com.br/ofertas/hoteis-de-praia/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-04-praias) [PRAIA](https://www.bestday.com.br/ofertas/nacionais/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-05-destinos-nacionais) [DISFRUTE DAS BELEZAS DO BRASIL!](https://www.bestday.com.br/ofertas/hoteis-de-praia/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-04-praias) [RESERVE AGORA!](https://www.bestday.com.br/ofertas/fim-de-semana/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-06-final-de-semana) [FINAL-DE-SEMANA](https://www.bestday.com.br/ofertas/fim-de-semana/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-06-final-de-semana) [FINAL-DE-SEMANA](https://www.bestday.com.br/ofertas/fim-de-semana/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-06-final-de-semana) [CONHEÇA OS ENCANTOS E APROVEITE!](https://www.bestday.com.br/ofertas/fim-de-semana/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-06-final-de-semana) [RESERVE AGORA!](https://www.bestday.com.br/ofertas/fim-de-semana/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-06-final-de-semana) [DESTINOS NACIONAIS](https://www.bestday.com.br/ofertas/nacionais/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-05-destinos-nacionais) [sub-image](https://www.bestday.com.br/ofertas/nacionais/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-05-destinos-nacionais) [DISFRUTE DAS BELEZAS DO BRASIL!](https://www.bestday.com.br/ofertas/nacionais/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-05-destinos-nacionais) [RESERVE AGORA!](https://www.bestday.com.br/ofertas/nacionais/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-05-destinos-nacionais) [ARGENTINA](https://www.bestday.com.br/ofertas/argentina/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-07-argentina) [sub-image](https://www.bestday.com.br/ofertas/argentina/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-07-argentina) [CONHEÇA OS ENCANTOS E APROVEITE!](https://www.bestday.com.br/ofertas/argentina/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-07-argentina) [RESERVE AGORA!](https://www.bestday.com.br/ofertas/argentina/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-07-argentina) https://www.bestday.com.br/ofertas/descontos/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca-02&utm_term=mini-banner-02-promo12xdesconto https://www.bestday.com.br/ofertas/estados-unidos/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-08-euahttps://www.bestday.com.br/passeios/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca-02&utm_term=mini-banner-05passeios [NATAL EM GRAMADO](https://www.bestday.com.br/gramado-brasil/ofertas/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-03-natal-luz-gramado) [sub-image](https://www.bestday.com.br/gramado-brasil/ofertas/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-03-natal-luz-gramado) [VIVA MOMENTOS INESQUECÍVEIS!](https://www.bestday.com.br/gramado-brasil/ofertas/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-03-natal-luz-gramado) [Reserve Já!](http://%28null%29/Redir?dd43c3911610f08dc711959505efff88&0a28a80961244107a4bd3a86753f5125) [HOTÉIS NO RS](https://www.bestday.com.br/ofertas/?camp=3629&asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-09-rede-laguetto) [sub-image](https://www.bestday.com.br/ofertas/?camp=3629&asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-09-rede-laguetto) [HOSPEDE-SE COM LUXO E CONFORTO!](https://www.bestday.com.br/ofertas/?camp=3629&asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-09-rede-laguetto) [Reserve Já!](https://www.bestday.com.br/ofertas/?camp=3629&asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-09-rede-laguetto) [AH, FLORIPA! - SANTA CATARINA](http://%28null%29/Redir?4dab10b7e6ec2d615f91a6b21642c306&0a28a80961244107a4bd3a86753f5125) Floripa, como é carinhosamente conhecida, é um destino que atende desde os amantes de uma boa gastronomia, até os mais naturebas e apaixonados pelo ecoturismo. A cidade também manda muito bem quando o assunto é agito: muita gente bonita, festas, baladas e animacão [...] Veja mais dicas de Santa Catarina e aproveite sua viagem! [logo-footer](https://www.bestday.com.br/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca-02&utm_term=logo-footer-bestday) Copyright for BestDay Viagens 2016. All rights reserved. Siga nossas redes sociais https://www.facebook.com/pages/bestdaycombr-best-day-travel-brasil/330157133678621?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&;utm_campaign=setembro_2016&;utm_content=peca02&;utm_term=facebook https://twitter.com/#!/bestday_viagens?asoc=crmbdbr&;utm_source=theplus&;utm_medium=mail&;utm_campaign=setembro_2016&;utm_content=peca02&;utm_term=twitter [INSTAGRAM](https://www.instagram.com/bestday_viagens/?asoc=crmbdbr&;utm_source=theplus&;utm_medium=mail&;utm_campaign=setembro_2016&;utm_content=peca02&;utm_term=instagram) https://www.youtube.com/user/bestdaybr?sub_confirmation=1&;asoc=crmbdbr&;utm_source=theplus&;utm_medium=mail&;utm_campaign=setembro_2016&;utm_content=peca02&;utm_term=youtube Saiba de nossas ofertas em primeira mão. E-mail enviado por: BestDay São Paulo - SP, CEP 05804-900 ? Brasil From owner-freebsd-stable@freebsd.org Sun Sep 25 18:19:35 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DE07FBDEDA6 for ; Sun, 25 Sep 2016 18:19:35 +0000 (UTC) (envelope-from e434b6928c3b7f98a1f897183537a694.user@mail04100.e2b.descontosnet.com.br) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id BA526C97 for ; Sun, 25 Sep 2016 18:19:35 +0000 (UTC) (envelope-from e434b6928c3b7f98a1f897183537a694.user@mail04100.e2b.descontosnet.com.br) Received: by mailman.ysv.freebsd.org (Postfix) id B5EE2BDEDA5; Sun, 25 Sep 2016 18:19:35 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B5750BDEDA4 for ; Sun, 25 Sep 2016 18:19:35 +0000 (UTC) (envelope-from e434b6928c3b7f98a1f897183537a694.user@mail04100.e2b.descontosnet.com.br) Received: from mail04100.e2b.descontosnet.com.br (mail04100.e2b.descontosnet.com.br [149.56.169.100]) by mx1.freebsd.org (Postfix) with SMTP id 479FFC96 for ; Sun, 25 Sep 2016 18:19:34 +0000 (UTC) (envelope-from e434b6928c3b7f98a1f897183537a694.user@mail04100.e2b.descontosnet.com.br) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=e2b.descontosnet.com.br; s=sender; t=1474827574; bh=WTs7Ol/fxRqk3vRy7Aq8RmbX4pU=; h=From:To:Date:Subject:Message-Id:Reply-To:Errors-To: List-Unsubscribe:X-Report-Abuse:X-Sender:X-Identity:X-Campaign: X-Campaignid:X-RPTags:MIME-Version:Content-Type; b=PzNrzrc62Xj/9JXZyrwUS7wgAb/0bziGXq/1MkUEZ8RQAtZaGjXHlCaDbJfAKa8dQ LgpV4cgah07n7KoPz+1qk5cDpWiS0kYpiqVeIoebCp+J7ezpm500x+ijFeqfjSpW2J WvffVhyr5vidC76MKMgbqvyjKIxDFbDfOh3s2KII= DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; d=e2b.descontosnet.com.br; s=sender; h=From:To:Date:Subject:Message-Id:Reply-To:Errors-To:List-Unsubscribe:X-Report-Abuse:X-Sender:X-Identity:X-Campaign:X-Campaignid:X-RPTags:MIME-Version:Content-Type; b=qwL/aPer1WwsSy6SabGkJCUg/pmzuAZ0PhNAdp6+bBTgSEGgrMFbwSVbpZvCeqJDlZz/7AYNPeMK6QrDz2Kg7INfLbHvFSkq8RpXd93xiVCEkTVnFVjcFN3h2NIzIRKmpadLg2FndLxa/Tmf/I7KOoAtYkvjfZrm1phYCBWxal8=; From: bestdayviagens@e2b.descontosnet.com.br To: stable@freebsd.org Date: Sun, 25 Sep 2016 15:19:34 -0300 Subject: =?utf-8?Q?Ultimo=20Dia!=20Cancun=20All=20Inclusive=2061%OFF=20+B=C3=B4nus=20de=20300=20no=20Hotel=20Oasis?= Message-Id: Reply-To: bestdayviagens@descontosnet.com.br Errors-To: e434b6928c3b7f98a1f897183537a694-c8f40e6f2da7ede337905ed2d45b7cc8.user@e2b.descontosnet.com.br X-Report-Abuse: Please report abuse for this campaign here: http://WebApp.e2b.descontosnet.com.br/util/Unsubscribe.php?TkEnvio=e434b6928c3b7f98a1f897183537a694&Srv=c8f40e6f2da7ede337905ed2d45b7cc8 X-Sender: c8f40e6f2da7ede337905ed2d45b7cc8 X-Identity: e434b6928c3b7f98a1f897183537a694 X-Campaign: c21dc439da359878ac7f450ca2ac8f9e X-Campaignid: c21dc439da359878ac7f450ca2ac8f9e X-RPTags: 45beff490a7f6cee79372cc0d3088df9 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 18:19:36 -0000 [logoheader-img](https://www.bestday.com.br/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca-02&utm_term=logo-header-bestday) 0800.522.7826 [Hotéis](https://www.bestday.com.br/hoteis?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca-02&utm_term=menu-hoteis) | [Pacotes](https://www.bestday.com.br/pacotes?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca-02&utm_term=menu-pacotes) | [Passagens](https://www.bestday.com.br/passagens?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca-02&utm_term=menu-passagens) | [Passeios](https://www.bestday.com.br/passeios/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca-02&utm_term=menu-passeios) [bannerprincipal02-img](https://www.bestday.com.br/ofertas/?camp=2518&asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-02-cancun-oasis) [banner-img](https://www.bestday.com.br/passagens/latam/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca-02&utm_term=mini-banner-01-promolatam) [PRAIAS](https://www.bestday.com.br/ofertas/hoteis-de-praia/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-04-praias) [PRAIA](https://www.bestday.com.br/ofertas/nacionais/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-05-destinos-nacionais) [DISFRUTE DAS BELEZAS DO BRASIL!](https://www.bestday.com.br/ofertas/hoteis-de-praia/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-04-praias) [RESERVE AGORA!](https://www.bestday.com.br/ofertas/fim-de-semana/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-06-final-de-semana) [FINAL-DE-SEMANA](https://www.bestday.com.br/ofertas/fim-de-semana/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-06-final-de-semana) [FINAL-DE-SEMANA](https://www.bestday.com.br/ofertas/fim-de-semana/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-06-final-de-semana) [CONHEÇA OS ENCANTOS E APROVEITE!](https://www.bestday.com.br/ofertas/fim-de-semana/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-06-final-de-semana) [RESERVE AGORA!](https://www.bestday.com.br/ofertas/fim-de-semana/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-06-final-de-semana) [DESTINOS NACIONAIS](https://www.bestday.com.br/ofertas/nacionais/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-05-destinos-nacionais) [sub-image](https://www.bestday.com.br/ofertas/nacionais/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-05-destinos-nacionais) [DISFRUTE DAS BELEZAS DO BRASIL!](https://www.bestday.com.br/ofertas/nacionais/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-05-destinos-nacionais) [RESERVE AGORA!](https://www.bestday.com.br/ofertas/nacionais/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-05-destinos-nacionais) [ARGENTINA](https://www.bestday.com.br/ofertas/argentina/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-07-argentina) [sub-image](https://www.bestday.com.br/ofertas/argentina/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-07-argentina) [CONHEÇA OS ENCANTOS E APROVEITE!](https://www.bestday.com.br/ofertas/argentina/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-07-argentina) [RESERVE AGORA!](https://www.bestday.com.br/ofertas/argentina/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-07-argentina) https://www.bestday.com.br/ofertas/descontos/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca-02&utm_term=mini-banner-02-promo12xdesconto https://www.bestday.com.br/ofertas/estados-unidos/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-08-euahttps://www.bestday.com.br/passeios/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca-02&utm_term=mini-banner-05passeios [NATAL EM GRAMADO](https://www.bestday.com.br/gramado-brasil/ofertas/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-03-natal-luz-gramado) [sub-image](https://www.bestday.com.br/gramado-brasil/ofertas/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-03-natal-luz-gramado) [VIVA MOMENTOS INESQUECÍVEIS!](https://www.bestday.com.br/gramado-brasil/ofertas/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-03-natal-luz-gramado) [Reserve Já!](http://%28null%29/Redir?dd43c3911610f08dc711959505efff88&0a28a80961244107a4bd3a86753f5125) [HOTÉIS NO RS](https://www.bestday.com.br/ofertas/?camp=3629&asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-09-rede-laguetto) [sub-image](https://www.bestday.com.br/ofertas/?camp=3629&asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-09-rede-laguetto) [HOSPEDE-SE COM LUXO E CONFORTO!](https://www.bestday.com.br/ofertas/?camp=3629&asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-09-rede-laguetto) [Reserve Já!](https://www.bestday.com.br/ofertas/?camp=3629&asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca02&utm_term=banner-principal-09-rede-laguetto) [AH, FLORIPA! - SANTA CATARINA](http://%28null%29/Redir?4dab10b7e6ec2d615f91a6b21642c306&0a28a80961244107a4bd3a86753f5125) Floripa, como é carinhosamente conhecida, é um destino que atende desde os amantes de uma boa gastronomia, até os mais naturebas e apaixonados pelo ecoturismo. A cidade também manda muito bem quando o assunto é agito: muita gente bonita, festas, baladas e animacão [...] Veja mais dicas de Santa Catarina e aproveite sua viagem! [logo-footer](https://www.bestday.com.br/?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&utm_campaign=setembro_2016&utm_content=peca-02&utm_term=logo-footer-bestday) Copyright for BestDay Viagens 2016. All rights reserved. Siga nossas redes sociais https://www.facebook.com/pages/bestdaycombr-best-day-travel-brasil/330157133678621?asoc=crmbdbr&utm_source=theplus&utm_medium=mail&;utm_campaign=setembro_2016&;utm_content=peca02&;utm_term=facebook https://twitter.com/#!/bestday_viagens?asoc=crmbdbr&;utm_source=theplus&;utm_medium=mail&;utm_campaign=setembro_2016&;utm_content=peca02&;utm_term=twitter [INSTAGRAM](https://www.instagram.com/bestday_viagens/?asoc=crmbdbr&;utm_source=theplus&;utm_medium=mail&;utm_campaign=setembro_2016&;utm_content=peca02&;utm_term=instagram) https://www.youtube.com/user/bestdaybr?sub_confirmation=1&;asoc=crmbdbr&;utm_source=theplus&;utm_medium=mail&;utm_campaign=setembro_2016&;utm_content=peca02&;utm_term=youtube Saiba de nossas ofertas em primeira mão. E-mail enviado por: BestDay São Paulo - SP, CEP 05804-900 ? Brasil From owner-freebsd-stable@freebsd.org Sun Sep 25 18:58:19 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 272DCBE9920 for ; Sun, 25 Sep 2016 18:58:19 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DCB4D27A; Sun, 25 Sep 2016 18:58:18 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1boEcx-0003Lh-7l; Sun, 25 Sep 2016 21:58:15 +0300 Date: Sun, 25 Sep 2016 21:58:15 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20160925185815.GA6177@zxy.spb.ru> References: <20160919204328.GN2840@zxy.spb.ru> <8ba75d6e-4f01-895e-0aed-53c6c6692cb9@freebsd.org> <20160920202633.GQ2840@zxy.spb.ru> <20160921195155.GW2840@zxy.spb.ru> <20160922095331.GB2840@zxy.spb.ru> <67862b33-63c0-2f23-d254-5ddc55dbb554@freebsd.org> <20160922102045.GC2840@zxy.spb.ru> <20160923191656.GF2840@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160923191656.GF2840@zxy.spb.ru> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 18:58:19 -0000 On Fri, Sep 23, 2016 at 10:16:56PM +0300, Slawa Olhovchenkov wrote: > On Thu, Sep 22, 2016 at 01:20:45PM +0300, Slawa Olhovchenkov wrote: > > > On Thu, Sep 22, 2016 at 12:04:40PM +0200, Julien Charbon wrote: > > > > > >> These paths can indeed compete for the same INP lock, as both > > > >> tcp_tw_2msl_scan() calls always start with the first inp found in > > > >> twq_2msl list. But in both cases, this first inp should be quickly used > > > >> and its lock released anyway, thus that could explain your situation it > > > >> that the TCP stack is doing that all the time, for example: > > > >> > > > >> - Let say that you are running out completely and constantly of tcptw, > > > >> and then all connections transitioning to TIME_WAIT state are competing > > > >> with the TIME_WAIT timeout scan that tries to free all the expired > > > >> tcptw. If the stack is doing that all the time, it can appear like > > > >> "live" locked. > > > >> > > > >> This is just an hypothesis and as usual might be a red herring. > > > >> Anyway, could you run: > > > >> > > > >> $ vmstat -z | head -2; vmstat -z | grep -E 'tcp|sock' > > > > > > > > ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP > > > > > > > > socket: 864, 4192664, 18604, 25348,49276158, 0, 0 > > > > tcp_inpcb: 464, 4192664, 34226, 18702,49250593, 0, 0 > > > > tcpcb: 1040, 4192665, 18424, 18953,49250593, 0, 0 > > > > tcptw: 88, 16425, 15802, 623,14526919, 8, 0 > > > > tcpreass: 40, 32800, 15, 2285, 632381, 0, 0 > > > > > > > > In normal case tcptw is about 16425/600/900 > > > > > > > > And after `sysctl -a | grep tcp` system stuck on serial console and I am reset it. > > > > > > > >> Ideally, once when everything is ok, and once when you have the issue > > > >> to see the differences (if any). > > > >> > > > >> If it appears your are quite low in tcptw, and if you have enough > > > >> memory, could you try increase the tcptw limit using sysctl > > > > > > > > I think this is not eliminate stuck, just may do it less frequency > > > > > > You are right, it would just be a big hint that the tcp_tw_2msl_scan() > > > contention hypothesis is the right one. As I see you have plenty of > > > memory on your server, thus could you try with: > > > > > > net.inet.tcp.maxtcptw=4192665 > > > > > > And see what happen. Just to validate this hypothesis. > > > > This is bad way for validate, with maxtcptw=16384 happened is random > > and can be waited for month. After maxtcptw=4192665 I am don't know > > how long need to wait for verification this hypothesis. > > > > More frequency (may be 3-5 times per day) happening less traffic drops > > (not to zero for minutes). May be this caused also by contention in > > tcp_tw_2msl_scan, but fast resolved (stochastic process). By eating > > CPU power nginx can't service connection and clients closed > > connections and need more TIME_WAIT and can trigered > > tcp_tw_2msl_scan(reuse=1). After this we can got live lock. > > > > May be after I learning to catch and dignostic this validation is more > > accurately. > > Some more bits: > > socket: 864, 4192664, 30806, 790,28524160, 0, 0 > ipq: 56, 32802, 0, 1278, 1022, 0, 0 > udp_inpcb: 464, 4192664, 44, 364, 14066, 0, 0 > udpcb: 32, 4192750, 44, 3081, 14066, 0, 0 > tcp_inpcb: 464, 4192664, 38558, 378,28476709, 0, 0 > tcpcb: 1040, 4192665, 30690, 738,28476709, 0, 0 > tcptw: 88, 32805, 7868, 772, 8412249, 0, 0 > > last pid: 49575; load averages: 2.00, 2.05, 3.75 up 1+01:12:08 22:13:42 > 853 processes: 15 running, 769 sleeping, 35 waiting, 34 lock > CPU 0: 0.0% user, 0.0% nice, 0.0% system, 100% interrupt, 0.0% idle > CPU 1: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle > CPU 2: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle > CPU 3: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle > CPU 4: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle > CPU 5: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle > CPU 6: 0.0% user, 0.0% nice, 0.4% system, 0.0% interrupt, 99.6% idle > CPU 7: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle > CPU 8: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle > CPU 9: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle > CPU 10: 0.0% user, 0.0% nice, 0.4% system, 0.0% interrupt, 99.6% idle > CPU 11: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle > Mem: 8659M Active, 8385M Inact, 107G Wired, 1325M Free > ARC: 99G Total, 88G MFU, 10G MRU, 32K Anon, 167M Header, 529M Other > Swap: 32G Total, 32G Free I can collect some more info. First: 1046 - - R 125:10.79 nginx: worker process (nginx) 1047 kqread - I 150:11.98 nginx: worker process (nginx) 1049 kqread - I 145:58.35 nginx: worker process (nginx) 1050 kqread - I 136:33.36 nginx: worker process (nginx) 1051 kqread - I 140:59.73 nginx: worker process (nginx) 1052 kqread - I 137:18.12 nginx: worker process (nginx) pid 1046 is nginx running on CPU0 (affinity mask set). # procstat -k -k 1046 PID TID COMM TDNAME KSTACK 1046 100686 nginx - mi_switch+0xd2 critical_exit+0x7e lapic_handle_timer+0xb1 Xtimerint+0x8c __mtx_lock_sleep+0x168 zone_fetch_slab+0x47 zone_import+0x52 zone_alloc_item+0x36 keg_alloc_slab+0x63 keg_fetch_slab+0x16e zone_fetch_slab+0x6e zone_import+0x52 uma_zalloc_arg+0x36e m_getm2+0x14f m_uiotombuf+0x64 sosend_generic+0x356 soo_write+0x42 dofilewrite+0x87 Tracing command nginx pid 1046 tid 100686 td 0xfffff8014485f500 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe20216992a0 /usr/src/sys/kern/sched_ule.c:1973 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe20216992d0 /usr/src/sys/kern/kern_synch.c:465 critical_exit() at 0xffffffff804a6bee = critical_exit+0x7e/frame 0xfffffe20216992f0 /usr/src/sys/kern/kern_switch.c:219 lapic_handle_timer() at 0xffffffff80771701 = lapic_handle_timer+0xb1/frame 0xfffffe2021699330 /usr/src/sys/x86/x86/local_apic.c:1185 Xtimerint() at 0xffffffff806cbbcc = Xtimerint+0x8c/frame 0xfffffe2021699330 /usr/src/sys/amd64/amd64/apic_vector.S:135 --- interrupt, rip = 0xffffffff804de424, rsp = 0xfffffe2021699400, rbp = 0xfffffe2021699420 --- lock_delay() at 0xffffffff804de424 = lock_delay+0x54/frame 0xfffffe2021699420 /usr/src/sys/kern/subr_lock.c:127 __mtx_lock_sleep() at 0xffffffff80484dc8 = __mtx_lock_sleep+0x168/frame 0xfffffe20216994a0 /usr/src/sys/kern/kern_mutex.c:512 zone_fetch_slab() at 0xffffffff806a4257 = zone_fetch_slab+0x47/frame 0xfffffe20216994e0 /usr/src/sys/vm/uma_core.c:2378 zone_import() at 0xffffffff806a4312 = zone_import+0x52/frame 0xfffffe2021699530 /usr/src/sys/vm/uma_core.c:2501 zone_alloc_item() at 0xffffffff806a0986 = zone_alloc_item+0x36/frame 0xfffffe2021699570 /usr/src/sys/vm/uma_core.c:2591 keg_alloc_slab() at 0xffffffff806a2463 = keg_alloc_slab+0x63/frame 0xfffffe20216995d0 /usr/src/sys/vm/uma_core.c:965 keg_fetch_slab() at 0xffffffff806a48ce = keg_fetch_slab+0x16e/frame 0xfffffe2021699620 /usr/src/sys/vm/uma_core.c:2349 zone_fetch_slab() at 0xffffffff806a427e = zone_fetch_slab+0x6e/frame 0xfffffe2021699660 /usr/src/sys/vm/uma_core.c:2375 zone_import() at 0xffffffff806a4312 = zone_import+0x52/frame 0xfffffe20216996b0 /usr/src/sys/vm/uma_core.c:2501 uma_zalloc_arg() at 0xffffffff806a147e = uma_zalloc_arg+0x36e/frame 0xfffffe2021699720 /usr/src/sys/vm/uma_core.c:2531 m_getm2() at 0xffffffff8048231f = m_getm2+0x14f/frame 0xfffffe2021699790 /usr/src/sys/kern/kern_mbuf.c:830 m_uiotombuf() at 0xffffffff80516044 = m_uiotombuf+0x64/frame 0xfffffe20216997e0 /usr/src/sys/kern/uipc_mbuf.c:1535 sosend_generic() at 0xffffffff8051ce56 = sosend_generic+0x356/frame 0xfffffe20216998a0 soo_write() at 0xffffffff804fd872 = soo_write+0x42/frame 0xfffffe20216998d0 dofilewrite() at 0xffffffff804f5c97 = dofilewrite+0x87/frame 0xfffffe2021699920 kern_writev() at 0xffffffff804f5978 = kern_writev+0x68/frame 0xfffffe2021699970 sys_writev() at 0xffffffff804f5be6 = sys_writev+0x36/frame 0xfffffe20216999a0 amd64_syscall() at 0xffffffff806e4051 = amd64_syscall+0x2c1/frame 0xfffffe2021699ab0 Xfast_syscall() at 0xffffffff806cb2bb = Xfast_syscall+0xfb/frame 0xfffffe2021699ab0 --- syscall (121, FreeBSD ELF64, sys_writev), rip = 0x8019cc6ba, rsp = 0x7fffffffd688, rbp = 0x7fffffffd6c0 --- And I am now collect and save kernel dump. # vmstat -M /var/crash/vmcore.1 -z| grep -i mbuf mbuf_packet: 256, 52276395, 26987, 189713,705271653, 0, 0 mbuf: 256, 52276395, 61994, 782439,46250344524, 0, 0 mbuf_cluster: 2048, 1048576, 49207, 34461, 293307, 0, 0 mbuf_jumbo_page: 4096, 4084093, 61839, 613537,8265099417, 0, 0 mbuf_jumbo_9k: 9216, 3630303, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 2722728, 0, 0, 0, 0, 0 From owner-freebsd-stable@freebsd.org Mon Sep 26 01:12:10 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 656E7BEAFF2 for ; Mon, 26 Sep 2016 01:12:10 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (agora.rdrop.com [IPv6:2607:f678:1010::34]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47F8911F7 for ; Mon, 26 Sep 2016 01:12:10 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (66@localhost [127.0.0.1]) by agora.rdrop.com (8.13.1/8.12.7) with ESMTP id u8Q1C5mk029878 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Sun, 25 Sep 2016 18:12:09 -0700 (PDT) (envelope-from perryh@pluto.rain.com) Received: (from uucp@localhost) by agora.rdrop.com (8.13.1/8.14.2/Submit) with UUCP id u8Q1C53n029877 for freebsd-stable@freebsd.org; Sun, 25 Sep 2016 18:12:05 -0700 (PDT) (envelope-from perryh@pluto.rain.com) Received: from fbsd81 by pluto.rain.com (4.1/SMI-4.1-pluto-M2060407) id AA05088; Sun, 25 Sep 16 17:49:30 PDT Date: Sun, 25 Sep 2016 17:49:29 -0700 From: perryh@pluto.rain.com (Perry Hutchison) To: freebsd-stable@freebsd.org Subject: 10.3 gpart(8) strangeness Message-Id: <57e87099.y0HRwon108cNG6uj%perryh@pluto.rain.com> User-Agent: nail 11.25 7/29/05 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Sep 2016 01:12:10 -0000 I dd'd FreeBSD-10.3-RELEASE-i386-memstick.img to a 4GB flash drive, and booted it into single-user mode where it appeared as da0. Then, to resize the GPT to the media (to make space for another partition): # gpart show da0 # gpart recover da0 # gpart show da0 which appeared to work: the second "gpart show" showed a larger free space following the partitions than the first, and that resizing survived a reboot. However, when I tried to create a 4th partition in that free space: # gpart show da0 # showed 3 partitions and about 3GB of free space # gpart add -t freebsd-ufs da0 # reported "da0p4 added" (or similar) # gpart show da0 # showed 4 partitions including the new one, and # no free space -- as expected # shutdown -r now a "gpart show da0" after the reboot showed 3 partitions and about 3GB of free space, the same as before the "gpart add" operation. In other words, the new partition did not survive the reboot. I tried several variations, e.g. specifying "-f x" on the "gpart add" command (followed by a separate "gpart commit"), and I could never get the new partition to survive a reboot; but labels applied to the 3 pre-existing partitions using "gpart modify -l" did survive. Did I do something wrong, or have I stumbled over some obscure bug in the 10.3 gpart(8)? How do I create a partition, that will survive reboot, in the free space at the end of the i386 10.3 memstick? From owner-freebsd-stable@freebsd.org Mon Sep 26 04:03:32 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5E8D1BE83D4; Mon, 26 Sep 2016 04:03:32 +0000 (UTC) (envelope-from russ.haley@gmail.com) Received: from mail-vk0-x22c.google.com (mail-vk0-x22c.google.com [IPv6:2607:f8b0:400c:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 09688BA; Mon, 26 Sep 2016 04:03:32 +0000 (UTC) (envelope-from russ.haley@gmail.com) Received: by mail-vk0-x22c.google.com with SMTP id z126so37771435vkd.0; Sun, 25 Sep 2016 21:03:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=iADDNrBlTH0JqYCNNhbcwEoGJITLYOtnB0Sll8KcL8U=; b=AENVV+mQDzd/jAzsGq5d+wLxGMQTdYwD/EGWn21H5E97dktyCcWjeok5i1zDI7MZKt FBtzUgbAcpLxcnI6bZeANYzMDbH2er5RgV2hqvsutcenM3cfIwmkvUByu5I3h+TiM6Cp HKJihonCS2E5wYMOBCyWs3Sf9Q8vO8Erux4lr5VLBW9n/BwoZokTSBtIwOzzrX9BZpBH szkQ3cVOYU8owOAfRjLwZYyIivr16EKZtBdCyoO7JM7FQhM62LO5CXFr7y35MlBk0dzj WtG/Ak1lK/H0i0+FsKVIaconn5yAswGiuiWf/pYswzVrwcyoUdJIhzvi42qzbW61OeiM RSEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=iADDNrBlTH0JqYCNNhbcwEoGJITLYOtnB0Sll8KcL8U=; b=UHUcA4/5FHZKfRxjWmtw7/v5WyGYyufEpWlXSs/zQkbaH7SR7euzJIaSw1NdmNH4kr bJirHoW6H0TLdYzsLiFoRgYjKolLcit1aEEFdJqOq1x0yrm6jEH++XUxtZPGMdgn1Oru Rf77E8GpamWIbljuXLkTDIBJXkM6lo6YA/kl7xMAZYrWX7tTNBv/yhb+LGEe3MC6OM9E UfYKeTV7ofv7cnU6HXEP0N2NRGtQlT9ado0qRA0gQJBgvqSBIdFgo8HvwbnMKFya/LxT 8TkDlA8E1npKVI6JaP3R+QE+QL+Uf0kwIQK1F6SVrhMdaMjXcwIzGyGR70YC95ys/84a WfkA== X-Gm-Message-State: AA6/9RmCc8L0zsVMmv1nCO/lWi8gXEQZFg1LHyw8A7z9j6T+CspSrd0v8CRvzCRE1XP6SIC5Y7msKtTAzDNtJQ== X-Received: by 10.31.5.19 with SMTP id 19mr5592437vkf.75.1474862610967; Sun, 25 Sep 2016 21:03:30 -0700 (PDT) MIME-Version: 1.0 Received: by 10.31.124.202 with HTTP; Sun, 25 Sep 2016 21:03:30 -0700 (PDT) In-Reply-To: References: <4076CFFA-7BE2-4E1B-A7E8-08FD8FC27D21@dsl-only.net> <332FA120-31E5-4D31-B63E-A0DFDD7DEFC7@dsl-only.net> <4F04B0C0-DAB1-4EEA-A3A0-4FAA89AD93E2@dsl-only.net> From: Russell Haley Date: Sun, 25 Sep 2016 21:03:30 -0700 Message-ID: Subject: Re: 11.0-RELEASE tier level for arm64/aaarch64 and the officially built arm/armv6 variants? To: Warner Losh Cc: Mark Millard , freebsd-arm , FreeBSD-STABLE Mailing List Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Sep 2016 04:03:32 -0000 On Sun, Sep 25, 2016 at 10:19 AM, Warner Losh wrote: > On Sun, Sep 25, 2016 at 1:13 AM, Russell Haley wro= te: >> On Sat, Sep 24, 2016 at 11:10 PM, Warner Losh wrote: >>> On Sat, Sep 24, 2016 at 10:21 PM, Russell Haley = wrote: >>>> On Sat, Sep 24, 2016 at 5:12 PM, Mark Millard wr= ote: >>>>> On 2016-Sep-24, at 2:11 PM, Warner Losh wrote: >>>>> >>>>>> On Fri, Sep 23, 2016 at 8:29 PM, Mark Millard = wrote: >>>>>>> [A resend since I forget to list free-arm in the To: the first time= .] >>>>>>> >>>>>>> From https://www.freebsd.org/platforms/arm.html : >>>>>>> >>>>>>>> 32-bit ARM is officially a Tier 2 architecture, as the FreeBSD pro= ject does not provide official releases or pre-built packages for this plat= form due to it primarily targeting the embedded arena. However, FreeBSD/ARM= is being actively developed and maintained, is well supported, and provide= s an excellent framework for building ARM-based systems. FreeBSD/arm suppor= ts ARMv4 and ARMv5 processors. FreeBSD/armv6 supports ARMv6 and ARMv7 proce= ssors, including SMP on the latter. >>>>>>> >>>>>>> "does not provide official releases or pre-built packages"? >>>>>>> >>>>>>>> # uname -apKU >>>>>>>> FreeBSD rpi2 11.0-PRERELEASE FreeBSD 11.0-PRERELEASE #5 r304943M: = Sun Aug 28 03:17:54 PDT 2016 markmi@FreeBSDx64:/usr/obj/clang/arm.armv6= /usr/src/sys/RPI2-NODBG arm armv6 1100502 1100502 >>>>>>> >>>>>>>> # pkg search '.*' | wc >>>>>>>> 21349 155540 1596736 >>>>>>> >>>>>>> Will 11.0-RELEASE change the tier level for any of the specific arm= -armv6 variants that have FreeBSD-11.0-*-arm-armv6-*.img* files built, such= as for RPI2? >>>>>>> >>>>>>> Even if all the officially built arm-armv6 variants stay tier 2, th= e wording on the web page likely needs to be changed because so much is bui= lt and available that the above quote claims is not available. >>>>>> >>>>>> armv6 is basically Tier 1 right now, though not as Tier 1 as i386 or >>>>>> amd64 due to the fragmented nature of the arm world. On the platform= s >>>>>> we run on and create releases for, however, it's my opinion that it = is >>>>>> Tier 1: it has been running in production a while, things people >>>>>> expect from a FreeBSD system are present, you can get decent support >>>>>> if you ask questions, there's no known major gotchas in deploying th= is >>>>>> hardware. The only remaining annoying issue is the 'u-boot' problem >>>>>> where we have to have a different u-boot image for every board and n= o >>>>>> standardized way to convert a 'generic' image into one that's specif= ic >>>>>> for specific boards. >>>> >>>> I'll point out again that barebox is an excellent alternative to u-boo= t (IMHO): >>> >>> Doesn't matter, still has the same issues that u-boot has. >> u-boot has a different sources for almost each board we support (due >> to the usual FOSS issues). That is NOT the case in barebox. There is >> one source and it's kept up to date by the team, not the vendors. > > Actually, that's no longer the case. All boards we support have all > their bits in u-boot's main repo. But you can't say no one will fork > barebox to get their support out faster, which is why we have the > current situation in ports... u-boot forked because it was so popular, > and the forks rejoined. Good show, I'm not advocating change for change sake and have had good success with u-boot as a novice. I'll have to have a look in the ports tree. I wasn't aware that so much progress had been made. I wish I had time to skulk on the IRC channel too. :( Russ >>> But does it support the u-boot ABI? >> I'd have to look into this. >>> >>>> - Supports most, if not all of the boards that FreeBSD supports, plus >>>> many that it doesn't >>>> - Single source tree for all boards. Specify build time parameters to >>>> build one or all the images >>>> - Well supported community with central maintainer-ship >>>> - Simple, familiar shell interface (*awesome*) >>>> - Excellent documentation (u-boots is good too though) >>>> - Has support for (U)EFI >>>> - Supports quemu aarch64 (not *quite*sure what the means though) >>> >>> Right, u-boot has all these things, except maybe the shell interface >>> (not sure what you mean by that). >> Instead of stringing together variables and commands, it uses a >> scripting language like a simplified sh. Want to change how something >> boots? Update a script. Save it to disk (it has it's own persistence >> mechanisms) and export it. > > Sounds cool. > >>>> To be fair, I'm not saying the problem is the fault of denx, but >>>> barebox has a lot going for it. The maintainer was very keen to see it >>>> ported top FreeBSD and was willing to support the effort. I ran into >>>> some build time linux api requirements, but he didn't think that would >>>> be much to overcome (and it wasn't I just kept adding the patches he >>>> sent me and the build moved forward. As always, I ran out of time for >>>> the really fun stuff). While I am a hopeless dreamer and I'm sure I've >>>> over simplified the problem, I thought it would be neat to see FreeBSD >>>> upstream support for zfs and ufs to the barebox boot loader and do >>>> away with ubldr. We would then have a modern, easy to use, boot loader >>>> that supports the standard startup toolchain. >>> >>> We can't easily do away with ubldr if we want to support tunables, kern= el >>> modules loaded at boot time and a few other nifty features like nextboo= t. >> >> Are these things not in standard loader? Should they be? > > ubldr is the standard loader, just spelled in a funky way and living > on a place u-boot can see :). We have different loaders for different > environments. even on x86 we have /boot/loader for BIOS boots and > /boot/loader.efi for EFI boots. > >>>> Either way, if installers move to a pkgng based method (so cool) then >>>> installing u-boot and arm binaries from pkg-static will be the same as >>>> x86 (ha ha I said that with a straight face!). >>> >>> Yea, not so much. You have to build the bootable image not on the >>> target system, like you do on x86. >> >> Doesn't the current ports and packages cross build everything that's mar= ked? > > For builds from source, yes. But that's different than installing from > FreeBSD's installer. > >>> We'd have to have something that >>> installs uboot onto a generic image (perhaps with hooks for kernels >>> since those aren't generic on armv6) and then put that into a bootable >>> SD card. >> >> The x86 installer (that I argue is a platform legacy) has to customize >> the bootloader for each installation. If we HAVE to use an installer >> on arm, what wrong prompting the user for some input (i.e. what som >> are you using) while including all the u-boots in the ports tree and >> all the supported kernels, then just installing the correct ones for >> the board (with input from the user)? > > Ummm, you can't boot anything to do that work like you can on x86. > >>>>>>For x86 this is all done with the installer since >>>>>> that boot environment is more standardized. Does this last issue kee= p >>>>>> arm from being Tier 1? That's a judgement call, but I think the >>>>>> project should promote w/o this last issue. >>>> >>>> How does a platform get promoted? Is that something the Core team deci= des? >>> >>> Yes. >>> >>>> I see two facts about current Arm support that show platform maturity: >>>> a) u-boot is in the ports tree and we have Lego-easy build scripts in >>>> crochet that could be called an installation method. Building for arm >>>> is not difficult anymore. >>> >>> Except corchet isn't in the tree, and the solution is horrible. It's >>> a script for each SoC, and those scripts are now scattered about. >>> Plus that's a creation from source model, not a creation from >>> RE produced bits model, which is needed for Tier 1. >> Both correctable sins, especially as it's BSD licensed. My point was >> that even building a SOM specific image is relatively painless with >> the right scripts. Hell, I've even got a custom build script that >> could be modified for generic use. > > Yea, a developer can do it with little pain. However, compared to the > pain to boot Linux on these boards, the pain is quite high and you > need to know way too much to make it work. > > BTW, my 'horrible' comment was given the state of things today: we > have scattered knowledge of how to make images, custom one-off that > work on some boards well, and fail on others. We have crazy, quirky > things that aren't well integrated into the tree. Crochet itself > solves an interesting problem, but does so a bit in isolation. > >>>> b)Arm requires images, not installers. Correct me if I'm wrong but, >>>> installers are a tool primarily invented for x86 PC type computers. >>>> FreeBSD publishes standardized ISOs for all supported Arm platforms >>>> that work by simply "xzcat | dd" onto the sd card (or wherever you >>>> need it). I'm not sure how standardized or manual that build process >>>> is, but I would think that if the Arm platform support is able to keep >>>> up with the standard FreeBSD release cycle (i.e. not break every other >>>> release) then there would be no reason to NOT call it tier 1? >>> >>> Creating the images is currently a pita. That's it. >> I see. So not so mature. > > Yes. It's an area I work on when I have time to make better... > > Warner > >>>> What I don't know about is "official" documentation for building for >>>> arm and supporting cross building to Arm. Will someone need to write >>>> an "Arm Handbook" to be promoted? >>> >>> No. While useful, most of that already exists. >> >> Thanks for the response Warner, I always appreciate the chance to >> learn more about FreeBSD. >> Russ >> >>> Warner >>> >>>>> Interesting and good to know. Thanks. >>>>> >>>>> I might have guessed that going along with the u-boot issue would be = the fanout in: >>>>> >>>>> 11.0/sys/arm/ . . . >>>>> >>>>> allwinner/a10/ >>>>> allwinner/a20/ >>>>> allwinner/a31/ >>>>> allwinner/a83t/ >>>>> allwinner/h3/ >>>>> . . . >>>>> broadcom/bcm2835/ >>>>> . . . >>>>> >>>>> (Full list not shown.) >>>>> >>>>> I was thinking that this might make the tier level specific to the st= atus of each such directory's content so that it was the combination of tha= t and the sysutils/u-boot-*/ status that made the difference for assigning = the level. I'd guess that lack of a usable directory in either place would= not be tier 2 even. Similarly until the required sys/arm/*/* and sysutils/= u-boot-*/ directory-tree content have reached a sufficiently complete statu= s. >>>>> >>>>> I'd expect that there will always be a lag for what exists in the wor= ld vs. what has these materials worked out in FreeBSD. >>>>> >>>>> >>>>>>> Also from https://www.freebsd.org/platforms/arm.html : >>>>>>> >>>>>>>> Initial support for 64-bit ARM is complete. 64-bit ARM platforms f= ollow a set of standard conventions, and a single FreeBSD build will work o= n hardware from multiple vendors. As a result, FreeBSD will provide officia= l releases for FreeBSD/arm64 and packages will be available. FreeBSD/arm64 = is on the path to becoming a Tier 1 architecture. >>>>>>> >>>>>>> Will 11.0-RELEASE make arm64/aarch64 Tier 1? >>>>>>> >>>>>>> [I will note that, while there are no official builds for the Pine6= 4 family (A64 based) that are under the Allwinner arm activity, the SOC's i= nvolved are Cortex-A53 64-bit arm based. They likely do not fit in the "sta= ndard conventions" or arm64/aarch64 would be where they would have been sup= ported. Some rewording might be appropriate for the above quote as well.] >>>>>> >>>>>> No. aarch64 isn't Tier 1 yet. There's many small bits that are >>>>>> missing. It is quite solidly Tier 2, but we don't have a linker, we >>>>>> don't have widespread hardware availability, we don't have productio= n >>>>>> experience with the platform. Most things work, but there's still so= me >>>>>> gotchas. There's still the 'u-boot' problem with many arm64 systems >>>>>> because for systems that use u-boot to bootstrap UEFI, you need a >>>>>> different image for each board (some closely related board families >>>>>> can get by with one to be pedantic). All these issues are still >>>>>> significant barriers to production use. It's not been officially >>>>>> promoted yet and I don't think the time is quite right yet. >>>>> >>>>> Intersting as well. I'd guess that conceptually this probably would a= pply to both: >>>>> >>>>> sys/arm/allwinner/a64/ and sysutils/u-boot/pine64/ >>>>> (presuming, contrary to fact, that 11.0 had sys/arm/allwinner/a64/ ) >>>>> >>>>> and. . . >>>>> >>>>> sys/arm64/cavium/ >>>>> sys/arm64/cloudabi64/ >>>>> >>>>> So just sys/arm/ vs. sys/arm64/ for an aarch64 would not really make = a difference yet for tier level. >>>>> >>>>>> Warner >>>>> >>>>> Thanks again for the notes. >>>>> >>>>> =3D=3D=3D >>>>> Mark Millard >>>>> markmi at dsl-only.net >>>>> >>>>> _______________________________________________ >>>>> freebsd-arm@freebsd.org mailing list >>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-arm >>>>> To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org= " From owner-freebsd-stable@freebsd.org Mon Sep 26 07:12:17 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2B51ABE9A5D for ; Mon, 26 Sep 2016 07:12:17 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-wm0-f47.google.com (mail-wm0-f47.google.com [74.125.82.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AE45D153F for ; Mon, 26 Sep 2016 07:12:16 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-wm0-f47.google.com with SMTP id w84so134289262wmg.1 for ; Mon, 26 Sep 2016 00:12:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=WHf+i3NFcmGpTh8t4O69BhHkXydqqsoYlqJgFCIcm6I=; b=Iyw/WPWsRtctUccp8u7WYdQTMkyU0RGFd3lSr+HBix5uFu69jmXHUlpOGw9iIAqo/e nYCzGaxpgpJi+fJYk09h1tEj7c0DVQRyOpTRxLAC+ST6o6aRDvZh3jaoSGY1/1k9mQ6o ttDOQcRlosgrkcgF8sekqwWMIloQZDpQKMFvdPzWzra1Fq1OCKSAnX7qj/o43whJulrR Zkt+X19LDk6q4yr2itZJQo26WR+1qLXCu+wsE/6bPOK83nwVvvIf/NbT4YiF3G5UjTpy 0CPNFeeyATGwf8wCjv1AbzPlsAjYm2YuVZtXQgCcPjw8WIPKTCY2Xn+ovNdbZryZGQM9 jLVg== X-Gm-Message-State: AE9vXwOvhDnp9jUcrpUziNFiQfLPtcvNLBfGyBFd6wo0vxQUY5bL9zlypX+67zH17vrAMA== X-Received: by 10.194.118.39 with SMTP id kj7mr16077956wjb.172.1474873433317; Mon, 26 Sep 2016 00:03:53 -0700 (PDT) Received: from [172.20.10.4] (176.233.197.178.dynamic.wless.lssmb00p-cgnat.res.cust.swisscom.ch. [178.197.233.176]) by smtp.gmail.com with ESMTPSA id bk7sm20706843wjc.36.2016.09.26.00.03.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Sep 2016 00:03:51 -0700 (PDT) Subject: Re: 11.0 stuck on high network load To: Slawa Olhovchenkov References: <78cbcdc9-f565-1046-c157-2ddd8fcccc62@freebsd.org> <20160919204328.GN2840@zxy.spb.ru> <8ba75d6e-4f01-895e-0aed-53c6c6692cb9@freebsd.org> <20160920202633.GQ2840@zxy.spb.ru> <20160921195155.GW2840@zxy.spb.ru> <20160922095331.GB2840@zxy.spb.ru> <67862b33-63c0-2f23-d254-5ddc55dbb554@freebsd.org> <20160922102045.GC2840@zxy.spb.ru> <20160923191656.GF2840@zxy.spb.ru> Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara From: Julien Charbon Message-ID: <0a1522c0-bf23-8862-7785-94076e2e8d70@freebsd.org> Date: Mon, 26 Sep 2016 09:03:42 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <20160923191656.GF2840@zxy.spb.ru> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="4UdjdloAavhgGhlDNQTC5rJuu8bdjFenc" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Sep 2016 07:12:17 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --4UdjdloAavhgGhlDNQTC5rJuu8bdjFenc Content-Type: multipart/mixed; boundary="GOI7f8WkMNfWtsDtwWXMNSIBf9GVuUV42"; protected-headers="v1" From: Julien Charbon To: Slawa Olhovchenkov Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Message-ID: <0a1522c0-bf23-8862-7785-94076e2e8d70@freebsd.org> Subject: Re: 11.0 stuck on high network load References: <78cbcdc9-f565-1046-c157-2ddd8fcccc62@freebsd.org> <20160919204328.GN2840@zxy.spb.ru> <8ba75d6e-4f01-895e-0aed-53c6c6692cb9@freebsd.org> <20160920202633.GQ2840@zxy.spb.ru> <20160921195155.GW2840@zxy.spb.ru> <20160922095331.GB2840@zxy.spb.ru> <67862b33-63c0-2f23-d254-5ddc55dbb554@freebsd.org> <20160922102045.GC2840@zxy.spb.ru> <20160923191656.GF2840@zxy.spb.ru> In-Reply-To: <20160923191656.GF2840@zxy.spb.ru> --GOI7f8WkMNfWtsDtwWXMNSIBf9GVuUV42 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi Slawa, On 9/23/16 9:16 PM, Slawa Olhovchenkov wrote: > On Thu, Sep 22, 2016 at 01:20:45PM +0300, Slawa Olhovchenkov wrote: >=20 >> On Thu, Sep 22, 2016 at 12:04:40PM +0200, Julien Charbon wrote: >> >>>>> These paths can indeed compete for the same INP lock, as both >>>>> tcp_tw_2msl_scan() calls always start with the first inp found in >>>>> twq_2msl list. But in both cases, this first inp should be quickly= used >>>>> and its lock released anyway, thus that could explain your situatio= n it >>>>> that the TCP stack is doing that all the time, for example: >>>>> >>>>> - Let say that you are running out completely and constantly of tc= ptw, >>>>> and then all connections transitioning to TIME_WAIT state are compe= ting >>>>> with the TIME_WAIT timeout scan that tries to free all the expired >>>>> tcptw. If the stack is doing that all the time, it can appear like= >>>>> "live" locked. >>>>> >>>>> This is just an hypothesis and as usual might be a red herring. >>>>> Anyway, could you run: >>>>> >>>>> $ vmstat -z | head -2; vmstat -z | grep -E 'tcp|sock' >>>> >>>> ITEM SIZE LIMIT USED FREE REQ FAIL S= LEEP >>>> >>>> socket: 864, 4192664, 18604, 25348,49276158, 0= , 0 >>>> tcp_inpcb: 464, 4192664, 34226, 18702,49250593, 0= , 0 >>>> tcpcb: 1040, 4192665, 18424, 18953,49250593, 0= , 0 >>>> tcptw: 88, 16425, 15802, 623,14526919, 8,= 0 >>>> tcpreass: 40, 32800, 15, 2285, 632381, 0,= 0 >>>> >>>> In normal case tcptw is about 16425/600/900 >>>> >>>> And after `sysctl -a | grep tcp` system stuck on serial console and = I am reset it. >>>> >>>>> Ideally, once when everything is ok, and once when you have the is= sue >>>>> to see the differences (if any). >>>>> >>>>> If it appears your are quite low in tcptw, and if you have enough >>>>> memory, could you try increase the tcptw limit using sysctl >>>> >>>> I think this is not eliminate stuck, just may do it less frequency >>> >>> You are right, it would just be a big hint that the tcp_tw_2msl_scan= () >>> contention hypothesis is the right one. As I see you have plenty of >>> memory on your server, thus could you try with: >>> >>> net.inet.tcp.maxtcptw=3D4192665 >>> >>> And see what happen. Just to validate this hypothesis. >> >> This is bad way for validate, with maxtcptw=3D16384 happened is random= >> and can be waited for month. After maxtcptw=3D4192665 I am don't know >> how long need to wait for verification this hypothesis. >> >> More frequency (may be 3-5 times per day) happening less traffic drops= >> (not to zero for minutes). May be this caused also by contention in >> tcp_tw_2msl_scan, but fast resolved (stochastic process). By eating >> CPU power nginx can't service connection and clients closed >> connections and need more TIME_WAIT and can trigered >> tcp_tw_2msl_scan(reuse=3D1). After this we can got live lock. >> >> May be after I learning to catch and dignostic this validation is more= >> accurately. >=20 > Some more bits: >=20 > socket: 864, 4192664, 30806, 790,28524160, 0, = 0 > ipq: 56, 32802, 0, 1278, 1022, 0, = 0 > udp_inpcb: 464, 4192664, 44, 364, 14066, 0, = 0 > udpcb: 32, 4192750, 44, 3081, 14066, 0, = 0 > tcp_inpcb: 464, 4192664, 38558, 378,28476709, 0, = 0 > tcpcb: 1040, 4192665, 30690, 738,28476709, 0, = 0 > tcptw: 88, 32805, 7868, 772, 8412249, 0, = 0 >=20 > last pid: 49575; load averages: 2.00, 2.05, 3.75 up 1+01:12:08 = 22:13:42 > 853 processes: 15 running, 769 sleeping, 35 waiting, 34 lock > CPU 0: 0.0% user, 0.0% nice, 0.0% system, 100% interrupt, 0.0% id= le > CPU 1: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% id= le > CPU 2: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% id= le > CPU 3: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% id= le > CPU 4: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% id= le > CPU 5: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% id= le > CPU 6: 0.0% user, 0.0% nice, 0.4% system, 0.0% interrupt, 99.6% id= le > CPU 7: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% id= le > CPU 8: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% id= le > CPU 9: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% id= le > CPU 10: 0.0% user, 0.0% nice, 0.4% system, 0.0% interrupt, 99.6% id= le > CPU 11: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% id= le > Mem: 8659M Active, 8385M Inact, 107G Wired, 1325M Free > ARC: 99G Total, 88G MFU, 10G MRU, 32K Anon, 167M Header, 529M Other > Swap: 32G Total, 32G Free >=20 > PID UID PRI NICE SIZE RES STATE C TIME WCPU COMMAND > 49566 0 20 0 26264K 7068K CPU10 10 0:00 0.14% top > 5 0 -8 - 0K 192K l2arc_ 6 31:07 0.13% zfskern{= l2arc_fe > 46120 0 20 0 14608K 3868K nanslp 8 0:10 0.10% tcpw > 12 0 -76 - 0K 848K WAIT 1 0:00 0.02% intr{swi= 0: uart=20 >=20 >=20 > total =3D used + free > Max used total used total = used total used total > 2016-09-23 21:46:41 tcptw 32805 389 1665 0 tcpcb 27788 28557 inp= cb 28177 29296 socket 27875 28740 > 2016-09-23 21:46:42 tcptw 32805 409 1665 0 tcpcb 27723 28557 inp= cb 28132 29296 socket 27812 28740 > 2016-09-23 21:46:43 tcptw 32805 405 1665 0 tcpcb 27750 28557 inp= cb 28155 29296 socket 27838 28740 > 2016-09-23 21:46:44 tcptw 32805 409 1665 0 tcpcb 27765 28557 inp= cb 28174 29296 socket 27852 28740 > 2016-09-23 21:46:45 tcptw 32805 422 1665 0 tcpcb 27737 28557 inp= cb 28159 29296 socket 27824 28740 > 2016-09-23 21:46:46 tcptw 32805 444 1665 0 tcpcb 27729 28557 inp= cb 28173 29296 socket 27817 28740 > 2016-09-23 21:46:47 tcptw 32805 444 1665 0 tcpcb 27712 28557 inp= cb 28156 29296 socket 27800 28740 > 2016-09-23 21:46:48 tcptw 32805 428 1665 0 tcpcb 27693 28557 inp= cb 28121 29296 socket 27783 28740 > 2016-09-23 21:46:49 tcptw 32805 431 1665 0 tcpcb 27677 28557 inp= cb 28108 29296 socket 27766 28740 > 2016-09-23 21:46:50 tcptw 32805 455 1665 0 tcpcb 27721 28557 inp= cb 28176 29296 socket 27809 28740 > 2016-09-23 21:46:51 tcptw 32805 432 1665 0 tcpcb 27739 28557 inp= cb 28171 29296 socket 27828 28740 > 2016-09-23 21:46:52 tcptw 32805 434 1665 0 tcpcb 27731 28557 inp= cb 28165 29296 socket 27819 28740 > 2016-09-23 21:46:53 tcptw 32805 431 1665 0 tcpcb 27742 28557 inp= cb 28173 29296 socket 27831 28740 > 2016-09-23 21:46:54 tcptw 32805 424 1665 0 tcpcb 27738 28557 inp= cb 28162 29296 socket 27826 28740 > 2016-09-23 21:46:55 tcptw 32805 397 1665 0 tcpcb 27740 28557 inp= cb 28137 29296 socket 27827 28740 > 2016-09-23 21:46:56 tcptw 32805 412 1665 0 tcpcb 27742 28557 inp= cb 28154 29296 socket 27830 28740 > 2016-09-23 21:46:57 tcptw 32805 418 1665 0 tcpcb 27749 28557 inp= cb 28167 29296 socket 27838 28740 > 2016-09-23 21:46:58 tcptw 32805 426 1665 0 tcpcb 27740 28557 inp= cb 28166 29296 socket 27827 28740 > 2016-09-23 21:46:59 tcptw 32805 423 1665 0 tcpcb 27687 28557 inp= cb 28110 29296 socket 27773 28740 > 2016-09-23 21:47:00 tcptw 32805 426 1665 0 tcpcb 27716 28557 inp= cb 28142 29296 socket 27804 28740 > 2016-09-23 21:47:01 tcptw 32805 437 1665 0 tcpcb 27732 28557 inp= cb 28169 29296 socket 27821 28740 > 2016-09-23 21:47:02 tcptw 32805 471 1665 0 tcpcb 27672 28557 inp= cb 28143 29296 socket 27760 28740 > 2016-09-23 21:47:03 tcptw 32805 426 1665 0 tcpcb 27752 28557 inp= cb 28178 29296 socket 27838 28740 > 2016-09-23 21:47:04 tcptw 32805 402 1665 0 tcpcb 27760 28557 inp= cb 28162 29296 socket 27847 28740 > 2016-09-23 21:47:05 tcptw 32805 406 1665 0 tcpcb 27757 28557 inp= cb 28163 29296 socket 27845 28740 > 2016-09-23 21:47:06 tcptw 32805 443 1665 0 tcpcb 27783 28557 inp= cb 28226 29296 socket 27871 28740 > 2016-09-23 21:47:07 tcptw 32805 484 1665 0 tcpcb 27707 28557 inp= cb 28191 29296 socket 27794 28740 > 2016-09-23 21:47:08 tcptw 32805 473 1665 0 tcpcb 27721 28557 inp= cb 28194 29296 socket 27807 28740 > 2016-09-23 21:47:09 tcptw 32805 432 1665 0 tcpcb 27749 28557 inp= cb 28181 29296 socket 27837 28740 > 2016-09-23 21:47:10 tcptw 32805 421 1665 0 tcpcb 27780 28557 inp= cb 28201 29296 socket 27868 28740 > 2016-09-23 21:47:11 tcptw 32805 530 1665 0 tcpcb 27814 28557 inp= cb 28344 29296 socket 27902 28740 > 2016-09-23 21:47:12 tcptw 32805 680 1665 0 tcpcb 27874 28557 inp= cb 28554 29296 socket 27964 28740 > 2016-09-23 21:47:13 tcptw 32805 832 1665 0 tcpcb 27881 28557 inp= cb 28713 29296 socket 27971 28740 > 2016-09-23 21:47:14 tcptw 32805 997 1665 0 tcpcb 27880 28557 inp= cb 28877 29352 socket 27972 28740 > 2016-09-23 21:47:15 tcptw 32805 1155 1890 0 tcpcb 27931 28557 inp= cb 29086 29560 socket 28023 28740 > 2016-09-23 21:47:16 tcptw 32805 1322 2250 0 tcpcb 27981 28557 inp= cb 29303 29800 socket 28075 28740 > 2016-09-23 21:47:17 tcptw 32805 1496 2385 0 tcpcb 28065 28557 inp= cb 29561 30040 socket 28159 28740 > 2016-09-23 21:47:18 tcptw 32805 1648 2385 0 tcpcb 28151 28557 inp= cb 29799 30280 socket 28245 28740 > 2016-09-23 21:47:19 tcptw 32805 1790 2655 0 tcpcb 28398 28599 inp= cb 30188 30672 socket 28492 28796 > 2016-09-23 21:47:20 tcptw 32805 1954 2655 0 tcpcb 28712 28923 inp= cb 30666 31120 socket 28807 29116 > 2016-09-23 21:47:21 tcptw 32805 2115 3015 0 tcpcb 29061 29244 inp= cb 31176 31576 socket 29156 29468 > 2016-09-23 21:47:22 tcptw 32805 2265 3150 0 tcpcb 29335 29538 inp= cb 31600 32056 socket 29430 29704 > 2016-09-23 21:47:23 tcptw 32805 2424 3150 0 tcpcb 29553 29775 inp= cb 31977 32440 socket 29648 29956 > 2016-09-23 21:47:24 tcptw 32805 2590 3375 0 tcpcb 29711 29901 inp= cb 32301 32744 socket 29807 30112 > 2016-09-23 21:47:25 tcptw 32805 2760 3780 0 tcpcb 29794 30015 inp= cb 32554 33040 socket 29891 30224 > 2016-09-23 21:47:26 tcptw 32805 2935 3915 0 tcpcb 29879 30111 inp= cb 32814 33312 socket 29976 30292 > 2016-09-23 21:47:27 tcptw 32805 3109 3915 0 tcpcb 29953 30195 inp= cb 33062 33584 socket 30050 30392 > 2016-09-23 21:47:28 tcptw 32805 3264 4140 0 tcpcb 30060 30267 inp= cb 33324 33824 socket 30158 30476 > 2016-09-23 21:47:29 tcptw 32805 3435 4275 0 tcpcb 30137 30363 inp= cb 33572 34032 socket 30235 30572 > 2016-09-23 21:47:30 tcptw 32805 3600 4500 0 tcpcb 30221 30489 inp= cb 33821 34304 socket 30320 30644 > 2016-09-23 21:47:31 tcptw 32805 3775 4635 0 tcpcb 30309 30588 inp= cb 34084 34576 socket 30408 30740 > 2016-09-23 21:47:32 tcptw 32805 3936 4770 0 tcpcb 30534 30741 inp= cb 34470 34960 socket 30634 30908 > 2016-09-23 21:47:33 tcptw 32805 4097 4905 0 tcpcb 30744 30951 inp= cb 34841 35352 socket 30844 31160 > 2016-09-23 21:47:34 tcptw 32805 4233 5040 0 tcpcb 31006 31176 inp= cb 35239 35680 socket 31106 31372 > 2016-09-23 21:47:35 tcptw 32805 4366 5265 0 tcpcb 31160 31386 inp= cb 35526 35920 socket 31260 31568 > 2016-09-23 21:47:36 tcptw 32805 4738 5535 0 tcpcb 29529 31428 inp= cb 34267 36016 socket 29629 31596 > 2016-09-23 21:47:37 tcptw 32805 4879 5625 0 tcpcb 29506 31428 inp= cb 34385 36016 socket 29607 31596 > 2016-09-23 21:47:38 tcptw 32805 5011 5895 0 tcpcb 29590 31428 inp= cb 34601 36016 socket 29691 31596 > 2016-09-23 21:47:39 tcptw 32805 5130 5895 0 tcpcb 29713 31428 inp= cb 34843 36016 socket 29815 31596 > 2016-09-23 21:47:40 tcptw 32805 5259 6165 0 tcpcb 29783 31428 inp= cb 35042 36016 socket 29886 31596 > 2016-09-23 21:47:41 tcptw 32805 5378 6255 0 tcpcb 29606 31428 inp= cb 34984 36016 socket 29709 31596 > 2016-09-23 21:47:42 tcptw 32805 5489 6255 0 tcpcb 29638 31428 inp= cb 35127 36016 socket 29741 31596 > 2016-09-23 21:47:43 tcptw 32805 5629 6390 0 tcpcb 29630 31428 inp= cb 35259 36016 socket 29735 31596 > 2016-09-23 21:47:44 tcptw 32805 5754 6660 0 tcpcb 29593 31428 inp= cb 35347 36016 socket 29696 31596 > 2016-09-23 21:47:45 tcptw 32805 5887 6660 0 tcpcb 29606 31428 inp= cb 35493 36016 socket 29709 31596 > 2016-09-23 21:47:46 tcptw 32805 6011 6750 0 tcpcb 29613 31428 inp= cb 35624 36016 socket 29716 31596 > 2016-09-23 21:47:47 tcptw 32805 6128 7020 0 tcpcb 29642 31428 inp= cb 35770 36128 socket 29745 31596 > 2016-09-23 21:47:48 tcptw 32805 6250 7020 0 tcpcb 29742 31428 inp= cb 35992 36416 socket 29845 31596 > 2016-09-23 21:47:49 tcptw 32805 6378 7155 0 tcpcb 29745 31428 inp= cb 36123 36472 socket 29850 31596 > 2016-09-23 21:47:50 tcptw 32805 6486 7290 0 tcpcb 29756 31428 inp= cb 36242 36648 socket 29861 31596 > 2016-09-23 21:47:51 tcptw 32805 6603 7515 0 tcpcb 29807 31428 inp= cb 36410 36792 socket 29912 31596 > 2016-09-23 21:47:52 tcptw 32805 6736 7515 0 tcpcb 29830 31428 inp= cb 36566 36912 socket 29935 31596 > 2016-09-23 21:47:53 tcptw 32805 6852 7785 0 tcpcb 29892 31428 inp= cb 36744 37112 socket 29996 31596 > 2016-09-23 21:47:54 tcptw 32805 6991 7785 0 tcpcb 29876 31428 inp= cb 36867 37288 socket 29981 31596 > 2016-09-23 21:47:55 tcptw 32805 7102 8010 0 tcpcb 29928 31428 inp= cb 37030 37400 socket 30033 31596 > 2016-09-23 21:47:56 tcptw 32805 7227 8010 0 tcpcb 29960 31428 inp= cb 37187 37544 socket 30065 31596 > 2016-09-23 21:47:57 tcptw 32805 7356 8280 0 tcpcb 30031 31428 inp= cb 37387 37752 socket 30136 31596 > 2016-09-23 21:47:58 tcptw 32805 7505 8415 0 tcpcb 30164 31428 inp= cb 37669 38040 socket 30270 31596 > 2016-09-23 21:47:59 tcptw 32805 7618 8415 0 tcpcb 30302 31428 inp= cb 37920 38328 socket 30408 31596 > 2016-09-23 21:48:00 tcptw 32805 7740 8505 0 tcpcb 30478 31428 inp= cb 38218 38560 socket 30584 31596 > 2016-09-23 21:48:01 tcptw 32805 7858 8640 0 tcpcb 30663 31428 inp= cb 38521 38880 socket 30769 31596 So my hypothesis was wrong it is not a contention between pfslowtimo()/tcp_tw_2msl_scan(reuse=3D0) and tcp_input()/tcp_twstart()/tcp_tw_2msl_scan(reuse=3D1) as you still have plenty of unused tcptw. It is an issue between tcp_tw_2msl_scan(reuse=3D0) and something else. -- Julien --GOI7f8WkMNfWtsDtwWXMNSIBf9GVuUV42-- --4UdjdloAavhgGhlDNQTC5rJuu8bdjFenc Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJX6MhXAAoJEKVlQ5Je6dhxduEH/j8Yk508aLefZnH9EpL1TdCw eL1y5ctkXVJRhtAKl8hL7uuWoY0aNLCsJbEK85CcOmzuUm2QWLr1wD22D+A9EFSN LzNt6Q/jvzT9OV1OZYmmrIvDUiaD7Y5Tkli4VlBYem590T65r3zgWx3Llazorwc2 +TMTBGbNByk82zKMZ0DRwykPa5pm3w6oMi5ZMlAUWWMNR0m5AAPfuObFdEawcVXe 3G4HLh76N4hcY1+3DkuXHU1syJFcf6Fpj7xvVltD/mLuLESUSvrPBuQPZna3zX/6 lz0XT4wv6T/kiHnkAoRleZmno70rvFxR6WInHvlUIaMsjIfyKj4O9wpT+YLBg+Y= =WFRA -----END PGP SIGNATURE----- --4UdjdloAavhgGhlDNQTC5rJuu8bdjFenc-- From owner-freebsd-stable@freebsd.org Mon Sep 26 08:51:21 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D8F1ABEA761 for ; Mon, 26 Sep 2016 08:51:21 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-wm0-f48.google.com (mail-wm0-f48.google.com [74.125.82.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 761AE772 for ; Mon, 26 Sep 2016 08:51:21 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-wm0-f48.google.com with SMTP id b130so136490679wmc.0 for ; Mon, 26 Sep 2016 01:51:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=odatWaQZk6NST92pPnF+yH8MUfaCuLH1tH0z6ppfWRo=; b=buqmpIfGfsz+18Qqa7K0Ql1wwX85Ld9cLEXcB6ZA05MACvJ/T854/W1eSCr4Cbc+k2 3ns4FHdF4olOhNWTMN0/HJmLFlwKo9JdGKyDoa8w3Ihzd5dh/TYuRtnCrN1+xMsZWJlb WmfxyogiOrR1G74b1uIUYzi1VroQXxfqQF5QFQhBnu8sGE8h7ExKbpsOmk38mXozPyTM HKku7QYlsUefnlPiTcjOBRAfMTONWb18q8KOSWvJqQU2cDjcqKCaxqyzr+DmZ0O71hly nyf2IuFTuoSBUMAZcLF0p1qW+bO62b2dvd2MPgcTe5cGwIfCHWiFMa9CoWha0F7zGwfd sSYw== X-Gm-Message-State: AE9vXwPfgflWL3c07UY2gvd4OakGH7uZEOPLDyn4IRFYGLlA537XvScuFEITkiydeBxr4A== X-Received: by 10.194.115.38 with SMTP id jl6mr16632342wjb.28.1474879874002; Mon, 26 Sep 2016 01:51:14 -0700 (PDT) Received: from [10.100.64.20] ([217.30.88.44]) by smtp.gmail.com with ESMTPSA id 123sm10061023wmj.5.2016.09.26.01.51.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Sep 2016 01:51:13 -0700 (PDT) Subject: Re: 11.0 stuck on high network load To: Slawa Olhovchenkov References: <20160919204328.GN2840@zxy.spb.ru> <8ba75d6e-4f01-895e-0aed-53c6c6692cb9@freebsd.org> <20160920202633.GQ2840@zxy.spb.ru> <20160921195155.GW2840@zxy.spb.ru> <20160922095331.GB2840@zxy.spb.ru> <67862b33-63c0-2f23-d254-5ddc55dbb554@freebsd.org> <20160922102045.GC2840@zxy.spb.ru> <20160923191656.GF2840@zxy.spb.ru> <20160925185815.GA6177@zxy.spb.ru> Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara From: Julien Charbon Message-ID: Date: Mon, 26 Sep 2016 10:51:07 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <20160925185815.GA6177@zxy.spb.ru> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="TtwNGnolL41w7XTiubjCTpMUQ4E3GcXkQ" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Sep 2016 08:51:21 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --TtwNGnolL41w7XTiubjCTpMUQ4E3GcXkQ Content-Type: multipart/mixed; boundary="LOEuSf0BIKjR7HVfK3G906jP9JLEU0oiC"; protected-headers="v1" From: Julien Charbon To: Slawa Olhovchenkov Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Message-ID: Subject: Re: 11.0 stuck on high network load References: <20160919204328.GN2840@zxy.spb.ru> <8ba75d6e-4f01-895e-0aed-53c6c6692cb9@freebsd.org> <20160920202633.GQ2840@zxy.spb.ru> <20160921195155.GW2840@zxy.spb.ru> <20160922095331.GB2840@zxy.spb.ru> <67862b33-63c0-2f23-d254-5ddc55dbb554@freebsd.org> <20160922102045.GC2840@zxy.spb.ru> <20160923191656.GF2840@zxy.spb.ru> <20160925185815.GA6177@zxy.spb.ru> In-Reply-To: <20160925185815.GA6177@zxy.spb.ru> --LOEuSf0BIKjR7HVfK3G906jP9JLEU0oiC Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 9/25/16 8:58 PM, Slawa Olhovchenkov wrote: > On Fri, Sep 23, 2016 at 10:16:56PM +0300, Slawa Olhovchenkov wrote: >=20 >> On Thu, Sep 22, 2016 at 01:20:45PM +0300, Slawa Olhovchenkov wrote: >> >>> On Thu, Sep 22, 2016 at 12:04:40PM +0200, Julien Charbon wrote: >>> >>>>>> These paths can indeed compete for the same INP lock, as both >>>>>> tcp_tw_2msl_scan() calls always start with the first inp found in >>>>>> twq_2msl list. But in both cases, this first inp should be quickl= y used >>>>>> and its lock released anyway, thus that could explain your situati= on it >>>>>> that the TCP stack is doing that all the time, for example: >>>>>> >>>>>> - Let say that you are running out completely and constantly of t= cptw, >>>>>> and then all connections transitioning to TIME_WAIT state are comp= eting >>>>>> with the TIME_WAIT timeout scan that tries to free all the expired= >>>>>> tcptw. If the stack is doing that all the time, it can appear lik= e >>>>>> "live" locked. >>>>>> >>>>>> This is just an hypothesis and as usual might be a red herring. >>>>>> Anyway, could you run: >>>>>> >>>>>> $ vmstat -z | head -2; vmstat -z | grep -E 'tcp|sock' >>>>> >>>>> ITEM SIZE LIMIT USED FREE REQ FAIL = SLEEP >>>>> >>>>> socket: 864, 4192664, 18604, 25348,49276158, = 0, 0 >>>>> tcp_inpcb: 464, 4192664, 34226, 18702,49250593, = 0, 0 >>>>> tcpcb: 1040, 4192665, 18424, 18953,49250593, = 0, 0 >>>>> tcptw: 88, 16425, 15802, 623,14526919, 8= , 0 >>>>> tcpreass: 40, 32800, 15, 2285, 632381, 0= , 0 >>>>> >>>>> In normal case tcptw is about 16425/600/900 >>>>> >>>>> And after `sysctl -a | grep tcp` system stuck on serial console and= I am reset it. >>>>> >>>>>> Ideally, once when everything is ok, and once when you have the i= ssue >>>>>> to see the differences (if any). >>>>>> >>>>>> If it appears your are quite low in tcptw, and if you have enough= >>>>>> memory, could you try increase the tcptw limit using sysctl >>>>> >>>>> I think this is not eliminate stuck, just may do it less frequency >>>> >>>> You are right, it would just be a big hint that the tcp_tw_2msl_sca= n() >>>> contention hypothesis is the right one. As I see you have plenty of= >>>> memory on your server, thus could you try with: >>>> >>>> net.inet.tcp.maxtcptw=3D4192665 >>>> >>>> And see what happen. Just to validate this hypothesis. >>> >>> This is bad way for validate, with maxtcptw=3D16384 happened is rando= m >>> and can be waited for month. After maxtcptw=3D4192665 I am don't know= >>> how long need to wait for verification this hypothesis. >>> >>> More frequency (may be 3-5 times per day) happening less traffic drop= s >>> (not to zero for minutes). May be this caused also by contention in >>> tcp_tw_2msl_scan, but fast resolved (stochastic process). By eating >>> CPU power nginx can't service connection and clients closed >>> connections and need more TIME_WAIT and can trigered >>> tcp_tw_2msl_scan(reuse=3D1). After this we can got live lock. >>> >>> May be after I learning to catch and dignostic this validation is mor= e >>> accurately. >> >> Some more bits: >> >> socket: 864, 4192664, 30806, 790,28524160, 0, = 0 >> ipq: 56, 32802, 0, 1278, 1022, 0, = 0 >> udp_inpcb: 464, 4192664, 44, 364, 14066, 0, = 0 >> udpcb: 32, 4192750, 44, 3081, 14066, 0, = 0 >> tcp_inpcb: 464, 4192664, 38558, 378,28476709, 0, = 0 >> tcpcb: 1040, 4192665, 30690, 738,28476709, 0, = 0 >> tcptw: 88, 32805, 7868, 772, 8412249, 0, = 0 >> >> last pid: 49575; load averages: 2.00, 2.05, 3.75 up 1+01:12:08 = 22:13:42 >> 853 processes: 15 running, 769 sleeping, 35 waiting, 34 lock >> CPU 0: 0.0% user, 0.0% nice, 0.0% system, 100% interrupt, 0.0% i= dle >> CPU 1: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% i= dle >> CPU 2: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% i= dle >> CPU 3: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% i= dle >> CPU 4: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% i= dle >> CPU 5: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% i= dle >> CPU 6: 0.0% user, 0.0% nice, 0.4% system, 0.0% interrupt, 99.6% i= dle >> CPU 7: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% i= dle >> CPU 8: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% i= dle >> CPU 9: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% i= dle >> CPU 10: 0.0% user, 0.0% nice, 0.4% system, 0.0% interrupt, 99.6% i= dle >> CPU 11: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% i= dle >> Mem: 8659M Active, 8385M Inact, 107G Wired, 1325M Free >> ARC: 99G Total, 88G MFU, 10G MRU, 32K Anon, 167M Header, 529M Other >> Swap: 32G Total, 32G Free >=20 > I can collect some more info. > First: >=20 > 1046 - - R 125:10.79 nginx: worker process (nginx) > 1047 kqread - I 150:11.98 nginx: worker process (nginx) > 1049 kqread - I 145:58.35 nginx: worker process (nginx) > 1050 kqread - I 136:33.36 nginx: worker process (nginx) > 1051 kqread - I 140:59.73 nginx: worker process (nginx) > 1052 kqread - I 137:18.12 nginx: worker process (nginx) >=20 > pid 1046 is nginx running on CPU0 (affinity mask set). >=20 > # procstat -k -k 1046 > PID TID COMM TDNAME KSTACK > 1046 100686 nginx - mi_switch+0xd2 critical_= exit+0x7e lapic_handle_timer+0xb1 Xtimerint+0x8c __mtx_lock_sleep+0x168 z= one_fetch_slab+0x47 zone_import+0x52 zone_alloc_item+0x36 keg_alloc_slab+= 0x63 keg_fetch_slab+0x16e zone_fetch_slab+0x6e zone_import+0x52 uma_zallo= c_arg+0x36e m_getm2+0x14f m_uiotombuf+0x64 sosend_generic+0x356 soo_write= +0x42 dofilewrite+0x87 >=20 > Tracing command nginx pid 1046 tid 100686 td 0xfffff8014485f500 > sched_switch() at 0xffffffff804c956d =3D sched_switch+0x6ad/frame 0xfff= ffe20216992a0 /usr/src/sys/kern/sched_ule.c:1973 > mi_switch() at 0xffffffff804a8d92 =3D mi_switch+0xd2/frame 0xfffffe2021= 6992d0 /usr/src/sys/kern/kern_synch.c:465 > critical_exit() at 0xffffffff804a6bee =3D critical_exit+0x7e/frame 0xff= fffe20216992f0 /usr/src/sys/kern/kern_switch.c:219 > lapic_handle_timer() at 0xffffffff80771701 =3D lapic_handle_timer+0xb1/= frame 0xfffffe2021699330 /usr/src/sys/x86/x86/local_apic.c:1185 > Xtimerint() at 0xffffffff806cbbcc =3D Xtimerint+0x8c/frame 0xfffffe2021= 699330 /usr/src/sys/amd64/amd64/apic_vector.S:135 > --- interrupt, rip =3D 0xffffffff804de424, rsp =3D 0xfffffe2021699400, = rbp =3D 0xfffffe2021699420 --- > lock_delay() at 0xffffffff804de424 =3D lock_delay+0x54/frame 0xfffffe20= 21699420 /usr/src/sys/kern/subr_lock.c:127 > __mtx_lock_sleep() at 0xffffffff80484dc8 =3D __mtx_lock_sleep+0x168/fra= me 0xfffffe20216994a0 /usr/src/sys/kern/kern_mutex.c:512 > zone_fetch_slab() at 0xffffffff806a4257 =3D zone_fetch_slab+0x47/frame = 0xfffffe20216994e0 /usr/src/sys/vm/uma_core.c:2378 > zone_import() at 0xffffffff806a4312 =3D zone_import+0x52/frame 0xfffffe= 2021699530 /usr/src/sys/vm/uma_core.c:2501 > zone_alloc_item() at 0xffffffff806a0986 =3D zone_alloc_item+0x36/frame = 0xfffffe2021699570 /usr/src/sys/vm/uma_core.c:2591 > keg_alloc_slab() at 0xffffffff806a2463 =3D keg_alloc_slab+0x63/frame 0x= fffffe20216995d0 /usr/src/sys/vm/uma_core.c:965 > keg_fetch_slab() at 0xffffffff806a48ce =3D keg_fetch_slab+0x16e/frame 0= xfffffe2021699620 /usr/src/sys/vm/uma_core.c:2349 > zone_fetch_slab() at 0xffffffff806a427e =3D zone_fetch_slab+0x6e/frame = 0xfffffe2021699660 /usr/src/sys/vm/uma_core.c:2375 > zone_import() at 0xffffffff806a4312 =3D zone_import+0x52/frame 0xfffffe= 20216996b0 /usr/src/sys/vm/uma_core.c:2501=20 > uma_zalloc_arg() at 0xffffffff806a147e =3D uma_zalloc_arg+0x36e/frame 0= xfffffe2021699720 /usr/src/sys/vm/uma_core.c:2531 > m_getm2() at 0xffffffff8048231f =3D m_getm2+0x14f/frame 0xfffffe2021699= 790 /usr/src/sys/kern/kern_mbuf.c:830 > m_uiotombuf() at 0xffffffff80516044 =3D m_uiotombuf+0x64/frame 0xfffffe= 20216997e0 /usr/src/sys/kern/uipc_mbuf.c:1535 > sosend_generic() at 0xffffffff8051ce56 =3D sosend_generic+0x356/frame 0= xfffffe20216998a0 > soo_write() at 0xffffffff804fd872 =3D soo_write+0x42/frame 0xfffffe2021= 6998d0 > dofilewrite() at 0xffffffff804f5c97 =3D dofilewrite+0x87/frame 0xfffffe= 2021699920 > kern_writev() at 0xffffffff804f5978 =3D kern_writev+0x68/frame 0xfffffe= 2021699970 > sys_writev() at 0xffffffff804f5be6 =3D sys_writev+0x36/frame 0xfffffe20= 216999a0 > amd64_syscall() at 0xffffffff806e4051 =3D amd64_syscall+0x2c1/frame 0xf= ffffe2021699ab0 > Xfast_syscall() at 0xffffffff806cb2bb =3D Xfast_syscall+0xfb/frame 0xff= fffe2021699ab0 > --- syscall (121, FreeBSD ELF64, sys_writev), rip =3D 0x8019cc6ba, rsp = =3D 0x7fffffffd688, rbp =3D 0x7fffffffd6c0 --- This call stack is quite interesting: 1: A process is calling writev() 2: Kernel calls sosend_generic() that starts allocating memory 3: This allocation is then interrupted by the timer interrupt handler [that could actually trigger tcp_tw_2msl_scan(reuse=3D0)] 4: The timer interrupt handler seems to wait on sched_switch() And fun fact: When sosend_generic() calls m_uiotombuf() it does not hold INP_WLOCK yet... -- Julien --LOEuSf0BIKjR7HVfK3G906jP9JLEU0oiC-- --TtwNGnolL41w7XTiubjCTpMUQ4E3GcXkQ Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJX6OF/AAoJEKVlQ5Je6dhxPtgIAOkdrwJWREyUwwRewrVTz/GR eohxzAM6NBuw7shSEvt/8qgaST8ClzxzuGQgrAf2+hevxNwa4Fo2cGqyfRdBp57v EeDKk2N/zJsiSCRr7HMbj/FWX+Nioa8IvxiABZx9SIhdTPtYD9lIUgSSrNEGWUrQ qw5VA+/i0ufm0pqLr8onZFSauZPRduHb2hpZFnCaxrNK3CeG2V6xYziP8f4anjyy F5P+exeiQ+iPZZ50+E49u2xOnnla3LwnoEO9MTZ/lSjLrVw+cc0L8HHfsFb+euA5 TgamvpQ9eSkx3yQsn1sc2ejc8+mvLQPz0eMLv1CrgCcbz0pHc11WAnVyrhJzc28= =i032 -----END PGP SIGNATURE----- --TtwNGnolL41w7XTiubjCTpMUQ4E3GcXkQ-- From owner-freebsd-stable@freebsd.org Mon Sep 26 09:33:31 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 826E1BE96CE for ; Mon, 26 Sep 2016 09:33:31 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-wm0-x22b.google.com (mail-wm0-x22b.google.com [IPv6:2a00:1450:400c:c09::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0618C197 for ; Mon, 26 Sep 2016 09:33:31 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-wm0-x22b.google.com with SMTP id l132so141227979wmf.0 for ; Mon, 26 Sep 2016 02:33:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=OuMytOTGJl921QYmC0TMbJQfF12OwnDLD5E3lgYgt28=; b=lN4eYKDOWtIs5guIRKonMCe+1IZ2lD2HRCvvt2EPrLDM6uFmRcfZM8HA9wTmYhi/L2 hgzULKAR07Lfs+oIilZYz0h629ci/cIXHAiIR1jtA50aOBULUsmEaZ6Z8J5w5C6QwRpN zy2BqxdwW1qK7A8RaMelpWyJTeyW+Fqa+kd8udu2GolVdPV6Ui9azInsCL9o+tJACsGD Z/Cy5heCPshcodqIU6usuMxflwDQvzzH2EFecxQYkrCL7hqgiYX0wK0Eny4PnN2g5U2Q 9baUQVPc4A3/yoFgsImSbj52k6nl6uCxlWR+fSM3GlXJHmEBzGzX9o2eELOURtcvgA/x nCDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=OuMytOTGJl921QYmC0TMbJQfF12OwnDLD5E3lgYgt28=; b=i5ZR7Fc7cWFzdYjSi1Ut0zTxKQNXj5XKbT9jO98Kefx8Wk/ctx74yCU0Ts5WWQffj9 5YQqY4wR4CBIAb4w2lwPnSxtgAd8QsiWfHRBjaOWaHnT+JhDMNQrBeSey3mgktiklFQv YZktPsKCk60QLLvjHDThuxfHQ/BDmXYWCnGkCBXEQnkzlfxW8WZYBkFi71ObGkPSTTMI GiGePm1sDG3uDUFBPKjvlfHETx/IokpUK41TqjR34uu4oJb7VocyH8hD9A6OM0WztWDW V5rZdjNjXx8vb8Qkf9qFPOJbZmbWjCJrxzL+K5iw8amq5EStyy6yRCGtwdxn4u5lBpe0 TOTg== X-Gm-Message-State: AA6/9RnarlUNQHE8nA7IlOKYspOtdXghO+NDeZksDzheaMkhPiA80LPpxp9NJH6U1TrX/w== X-Received: by 10.28.221.213 with SMTP id u204mr15180385wmg.117.1474882409470; Mon, 26 Sep 2016 02:33:29 -0700 (PDT) Received: from [10.100.64.20] ([217.30.88.44]) by smtp.gmail.com with ESMTPSA id v189sm10229479wmv.12.2016.09.26.02.33.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Sep 2016 02:33:28 -0700 (PDT) Subject: Re: 11.0 stuck on high network load To: Slawa Olhovchenkov References: <20160916183053.GL9397@strugglingcoder.info> <20160916190330.GG2840@zxy.spb.ru> <78cbcdc9-f565-1046-c157-2ddd8fcccc62@freebsd.org> <20160919204328.GN2840@zxy.spb.ru> <8ba75d6e-4f01-895e-0aed-53c6c6692cb9@freebsd.org> <20160920202633.GQ2840@zxy.spb.ru> <20160921195155.GW2840@zxy.spb.ru> <20160923200143.GG2840@zxy.spb.ru> <20160925124626.GI2840@zxy.spb.ru> Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara From: Julien Charbon Message-ID: Date: Mon, 26 Sep 2016 11:33:12 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <20160925124626.GI2840@zxy.spb.ru> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="12H9K1a6tfXsjU4KKUelKedFmxPMXjVBt" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Sep 2016 09:33:31 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --12H9K1a6tfXsjU4KKUelKedFmxPMXjVBt Content-Type: multipart/mixed; boundary="oWtTp0oI5DUmUuqawNkH1QRl3r9fRm6UV"; protected-headers="v1" From: Julien Charbon To: Slawa Olhovchenkov Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Message-ID: Subject: Re: 11.0 stuck on high network load References: <20160916183053.GL9397@strugglingcoder.info> <20160916190330.GG2840@zxy.spb.ru> <78cbcdc9-f565-1046-c157-2ddd8fcccc62@freebsd.org> <20160919204328.GN2840@zxy.spb.ru> <8ba75d6e-4f01-895e-0aed-53c6c6692cb9@freebsd.org> <20160920202633.GQ2840@zxy.spb.ru> <20160921195155.GW2840@zxy.spb.ru> <20160923200143.GG2840@zxy.spb.ru> <20160925124626.GI2840@zxy.spb.ru> In-Reply-To: <20160925124626.GI2840@zxy.spb.ru> --oWtTp0oI5DUmUuqawNkH1QRl3r9fRm6UV Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi Slawa, On 9/25/16 2:46 PM, Slawa Olhovchenkov wrote: > On Fri, Sep 23, 2016 at 11:01:43PM +0300, Slawa Olhovchenkov wrote: >>> On 9/21/16 9:51 PM, Slawa Olhovchenkov wrote: >>>> On Wed, Sep 21, 2016 at 09:11:24AM +0200, Julien Charbon wrote: >>>>> You can also use Dtrace and lockstat (especially with the lockstat= -s >>>>> option): >>>>> >>>>> https://wiki.freebsd.org/DTrace/One-Liners#Kernel_Locks >>>>> https://www.freebsd.org/cgi/man.cgi?query=3Dlockstat&manpath=3DFree= BSD+11.0-RELEASE >>>>> >>>>> But I am less familiar with Dtrace/lockstat tools. >>>> >>>> I am still use old kernel and got lockdown again. >>>> Try using lockstat (I am save more output), interesting may be next:= >>>> >>>> R/W writer spin on writer: 190019 events in 1.070 seconds (177571 ev= ents/sec) >>>> >>>> --------------------------------------------------------------------= ----------- >>>> Count indv cuml rcnt nsec Lock Caller = =20 >>>> 140839 74% 74% 0.00 24659 tcpinp tcp_tw_2msl_sc= an+0xc6 =20 >>>> >>>> nsec ------ Time Distribution ------ count Stack = =20 >>>> 4096 | 913 tcp_twstart+0xa= 3 =20 >>>> 8192 |@@@@@@@@@@@@ 58191 tcp_do_segment+= 0x201f =20 >>>> 16384 |@@@@@@ 29594 tcp_input+0xe1c= =20 >>>> 32768 |@@@@ 23447 ip_input+0x15f = =20 >>>> 65536 |@@@ 16197 =20 >>>> 131072 |@ 8674 =20 >>>> 262144 | 3358 =20 >>>> 524288 | 456 =20 >>>> 1048576 | 9 =20 >>>> --------------------------------------------------------------------= ----------- >>>> Count indv cuml rcnt nsec Lock Caller = =20 >>>> 49180 26% 100% 0.00 15929 tcpinp tcp_tw_2msl_sca= n+0xc6 =20 >>>> >>>> nsec ------ Time Distribution ------ count Stack = =20 >>>> 4096 | 157 pfslowtimo+0x54= =20 >>>> 8192 |@@@@@@@@@@@@@@@ 24796 softclock_call_= cc+0x179=20 >>>> 16384 |@@@@@@ 11223 softclock+0x44 = =20 >>>> 32768 |@@@@ 7426 intr_event_exec= ute_handlers+0x95 >>>> 65536 |@@ 3918 =20 >>>> 131072 | 1363 =20 >>>> 262144 | 278 =20 >>>> 524288 | 19 =20 >>>> --------------------------------------------------------------------= ----------- >>> >>> This is interesting, it seems that you have two call paths competing= >>> for INP locks here: >>> >>> - pfslowtimo()/tcp_tw_2msl_scan(reuse=3D0) and >>> >>> - tcp_input()/tcp_twstart()/tcp_tw_2msl_scan(reuse=3D1) >> >> My current hypothesis: >> >> nginx do write() (or may be close()?) to socket, kernel lock >> first inp in V_twq_2msl, happen callout for pfslowtimo() on the same >> CPU core and tcp_tw_2msl_scan infinity locked on same inp. >> >> In this case you modification can't help, before next try we need some= >> like yeld(). >=20 > Or may be locks leaks. > Or both. You are totally right, pfslowtimo()/tcp_tw_2msl_scan(reuse=3D0) is infinitely blocked on INP_WLOCK() by "something" (that could be related to write()). As I reached my limit of debugging without WITNESS, could you share your /etc/sysctl.conf, /boot/loader.conf files? And any specific configuration you have (like having a Nginx workers affinity, Nginx special options, etc.). Like that I can try to reproduce it on releng/11= =2E0. Thanks. -- Julien --oWtTp0oI5DUmUuqawNkH1QRl3r9fRm6UV-- --12H9K1a6tfXsjU4KKUelKedFmxPMXjVBt Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJX6OtnAAoJEKVlQ5Je6dhxlpsIAJ/qGR/MYHOe1EiiGqpVtL+s Wduh6AxK3/14NBvdjmxEU5RwfKc9tsP+wobaNJ6VZBSr7qqWXZgYM+E76iFiFgKE GuylXa4Rxb55F8W40pQko4YfYRkfrKc5GwthFtamDS3ASduASIUzDMY0Hh7EGoQD QyofDSTn8UEQmw8e4K3ryZ/qEExRh29lCSZBVtQMynoNSPQZf4lRhkM+paZoLiZW cfhw0xSuz9vzP4JbRW2UK0ZfLFzkjCBgUbcslRlc7Rf/969a39e3JxC5NTOnKejx PN7hPQYvLoaVZu6p5JmAuUD9k3y9RvOxxl0xLE/WPmAuhpi8ROKbW7Q+TGVoLrs= =xhuC -----END PGP SIGNATURE----- --12H9K1a6tfXsjU4KKUelKedFmxPMXjVBt-- From owner-freebsd-stable@freebsd.org Mon Sep 26 11:57:13 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 68186BEBB9F for ; Mon, 26 Sep 2016 11:57:13 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-wm0-f46.google.com (mail-wm0-f46.google.com [74.125.82.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 077DB7CE for ; Mon, 26 Sep 2016 11:57:12 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-wm0-f46.google.com with SMTP id 197so23858766wmk.1 for ; Mon, 26 Sep 2016 04:57:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=+2oBuXLjAwTGCS+Y03rMna+RxwWPtmhg3/m6RefGd30=; b=jLHrbRPgcQB+QOZX7FYWIK+y6MU7fxc15yVOCzEEv74pfHerABjmRbsB0+dWACDYtB ZX05qns1TXHEpAgFCyymOK0SxpkfayVDbS1SUHPfFK4gC5hgErB7G/IZugxsu0lhG1LE 9HGhlcmYzX1xMg1S/gAnN1zBoOwI2RRmP8PRGaGD/SKzp/AGfMm5raceOk425esUMaSE +CRN5uPgZ8BIYetnxmmBz9UDDNQTksevsx0w5bq78WtGuyuvEm6SXt+5Rv41jnjZl8Lu ZmHLHe3y8FBCuhSrlJAZxsnyAjQkf0s/E9h08Wpqfz3C1BptmDu3XagGgbbRI82sVfCd Jasw== X-Gm-Message-State: AA6/9RlOmjToZTSwwtEEceBUiLJygS0rtp2Q6I5l7Jfs0J91fabqtfzewxH23mB9g5GynA== X-Received: by 10.28.102.6 with SMTP id a6mr13305987wmc.97.1474891025405; Mon, 26 Sep 2016 04:57:05 -0700 (PDT) Received: from [10.100.64.20] ([217.30.88.44]) by smtp.gmail.com with ESMTPSA id n7sm10864508wmf.18.2016.09.26.04.57.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Sep 2016 04:57:04 -0700 (PDT) Subject: Re: 11.0 stuck on high network load To: Slawa Olhovchenkov References: <20160916183053.GL9397@strugglingcoder.info> <20160916190330.GG2840@zxy.spb.ru> <78cbcdc9-f565-1046-c157-2ddd8fcccc62@freebsd.org> <20160919204328.GN2840@zxy.spb.ru> <8ba75d6e-4f01-895e-0aed-53c6c6692cb9@freebsd.org> <20160920202633.GQ2840@zxy.spb.ru> <20160921195155.GW2840@zxy.spb.ru> <20160923200143.GG2840@zxy.spb.ru> <20160925124626.GI2840@zxy.spb.ru> Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara From: Julien Charbon Message-ID: <064be3ae-8df9-fa22-06ab-346af024afd3@freebsd.org> Date: Mon, 26 Sep 2016 13:57:03 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <20160925124626.GI2840@zxy.spb.ru> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Sep 2016 11:57:13 -0000 Hi Slawa, On 9/25/16 2:46 PM, Slawa Olhovchenkov wrote: > On Fri, Sep 23, 2016 at 11:01:43PM +0300, Slawa Olhovchenkov wrote: >> On Wed, Sep 21, 2016 at 11:25:18PM +0200, Julien Charbon wrote: >>> >>> On 9/21/16 9:51 PM, Slawa Olhovchenkov wrote: >>>> On Wed, Sep 21, 2016 at 09:11:24AM +0200, Julien Charbon wrote: >>>>> You can also use Dtrace and lockstat (especially with the lockstat -s >>>>> option): >>>>> >>>>> https://wiki.freebsd.org/DTrace/One-Liners#Kernel_Locks >>>>> https://www.freebsd.org/cgi/man.cgi?query=lockstat&manpath=FreeBSD+11.0-RELEASE >>>>> >>>>> But I am less familiar with Dtrace/lockstat tools. >>>> >>>> I am still use old kernel and got lockdown again. >>>> Try using lockstat (I am save more output), interesting may be next: >>>> >>>> R/W writer spin on writer: 190019 events in 1.070 seconds (177571 events/sec) >>>> >>>> ------------------------------------------------------------------------------- >>>> Count indv cuml rcnt nsec Lock Caller >>>> 140839 74% 74% 0.00 24659 tcpinp tcp_tw_2msl_scan+0xc6 >>>> >>>> nsec ------ Time Distribution ------ count Stack >>>> 4096 | 913 tcp_twstart+0xa3 >>>> 8192 |@@@@@@@@@@@@ 58191 tcp_do_segment+0x201f >>>> 16384 |@@@@@@ 29594 tcp_input+0xe1c >>>> 32768 |@@@@ 23447 ip_input+0x15f >>>> 65536 |@@@ 16197 >>>> 131072 |@ 8674 >>>> 262144 | 3358 >>>> 524288 | 456 >>>> 1048576 | 9 >>>> ------------------------------------------------------------------------------- >>>> Count indv cuml rcnt nsec Lock Caller >>>> 49180 26% 100% 0.00 15929 tcpinp tcp_tw_2msl_scan+0xc6 >>>> >>>> nsec ------ Time Distribution ------ count Stack >>>> 4096 | 157 pfslowtimo+0x54 >>>> 8192 |@@@@@@@@@@@@@@@ 24796 softclock_call_cc+0x179 >>>> 16384 |@@@@@@ 11223 softclock+0x44 >>>> 32768 |@@@@ 7426 intr_event_execute_handlers+0x95 >>>> 65536 |@@ 3918 >>>> 131072 | 1363 >>>> 262144 | 278 >>>> 524288 | 19 >>>> ------------------------------------------------------------------------------- >>> >>> This is interesting, it seems that you have two call paths competing >>> for INP locks here: >>> >>> - pfslowtimo()/tcp_tw_2msl_scan(reuse=0) and >>> >>> - tcp_input()/tcp_twstart()/tcp_tw_2msl_scan(reuse=1) >> >> My current hypothesis: >> >> nginx do write() (or may be close()?) to socket, kernel lock >> first inp in V_twq_2msl, happen callout for pfslowtimo() on the same >> CPU core and tcp_tw_2msl_scan infinity locked on same inp. >> >> In this case you modification can't help, before next try we need some >> like yeld(). > > Or may be locks leaks. > Or both. Actually one extra debug thing you can do is launching lockstat with below extra options: -H For Hold lock stats -P To get the overall time -s 20 To get the stackstrace To see who is holding the INP lock for so long. Thanks to Hiren for pointing the -H option. -- Julien From owner-freebsd-stable@freebsd.org Mon Sep 26 12:05:01 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DA721BEA866 for ; Mon, 26 Sep 2016 12:05:01 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9C24A19C; Mon, 26 Sep 2016 12:05:01 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1boUeY-0006QB-J2; Mon, 26 Sep 2016 15:04:58 +0300 Date: Mon, 26 Sep 2016 15:04:58 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20160926120458.GK2840@zxy.spb.ru> References: <20160920202633.GQ2840@zxy.spb.ru> <20160921195155.GW2840@zxy.spb.ru> <20160922095331.GB2840@zxy.spb.ru> <67862b33-63c0-2f23-d254-5ddc55dbb554@freebsd.org> <20160922102045.GC2840@zxy.spb.ru> <20160923191656.GF2840@zxy.spb.ru> <20160925185815.GA6177@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Sep 2016 12:05:01 -0000 On Mon, Sep 26, 2016 at 10:51:07AM +0200, Julien Charbon wrote: > > 1049 kqread - I 145:58.35 nginx: worker process (nginx) > > 1050 kqread - I 136:33.36 nginx: worker process (nginx) > > 1051 kqread - I 140:59.73 nginx: worker process (nginx) > > 1052 kqread - I 137:18.12 nginx: worker process (nginx) > > > > pid 1046 is nginx running on CPU0 (affinity mask set). > > > > # procstat -k -k 1046 > > PID TID COMM TDNAME KSTACK > > 1046 100686 nginx - mi_switch+0xd2 critical_exit+0x7e lapic_handle_timer+0xb1 Xtimerint+0x8c __mtx_lock_sleep+0x168 zone_fetch_slab+0x47 zone_import+0x52 zone_alloc_item+0x36 keg_alloc_slab+0x63 keg_fetch_slab+0x16e zone_fetch_slab+0x6e zone_import+0x52 uma_zalloc_arg+0x36e m_getm2+0x14f m_uiotombuf+0x64 sosend_generic+0x356 soo_write+0x42 dofilewrite+0x87 > > > > Tracing command nginx pid 1046 tid 100686 td 0xfffff8014485f500 > > sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe20216992a0 /usr/src/sys/kern/sched_ule.c:1973 > > mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe20216992d0 /usr/src/sys/kern/kern_synch.c:465 > > critical_exit() at 0xffffffff804a6bee = critical_exit+0x7e/frame 0xfffffe20216992f0 /usr/src/sys/kern/kern_switch.c:219 > > lapic_handle_timer() at 0xffffffff80771701 = lapic_handle_timer+0xb1/frame 0xfffffe2021699330 /usr/src/sys/x86/x86/local_apic.c:1185 > > Xtimerint() at 0xffffffff806cbbcc = Xtimerint+0x8c/frame 0xfffffe2021699330 /usr/src/sys/amd64/amd64/apic_vector.S:135 > > --- interrupt, rip = 0xffffffff804de424, rsp = 0xfffffe2021699400, rbp = 0xfffffe2021699420 --- > > lock_delay() at 0xffffffff804de424 = lock_delay+0x54/frame 0xfffffe2021699420 /usr/src/sys/kern/subr_lock.c:127 > > __mtx_lock_sleep() at 0xffffffff80484dc8 = __mtx_lock_sleep+0x168/frame 0xfffffe20216994a0 /usr/src/sys/kern/kern_mutex.c:512 > > zone_fetch_slab() at 0xffffffff806a4257 = zone_fetch_slab+0x47/frame 0xfffffe20216994e0 /usr/src/sys/vm/uma_core.c:2378 > > zone_import() at 0xffffffff806a4312 = zone_import+0x52/frame 0xfffffe2021699530 /usr/src/sys/vm/uma_core.c:2501 > > zone_alloc_item() at 0xffffffff806a0986 = zone_alloc_item+0x36/frame 0xfffffe2021699570 /usr/src/sys/vm/uma_core.c:2591 > > keg_alloc_slab() at 0xffffffff806a2463 = keg_alloc_slab+0x63/frame 0xfffffe20216995d0 /usr/src/sys/vm/uma_core.c:965 > > keg_fetch_slab() at 0xffffffff806a48ce = keg_fetch_slab+0x16e/frame 0xfffffe2021699620 /usr/src/sys/vm/uma_core.c:2349 > > zone_fetch_slab() at 0xffffffff806a427e = zone_fetch_slab+0x6e/frame 0xfffffe2021699660 /usr/src/sys/vm/uma_core.c:2375 > > zone_import() at 0xffffffff806a4312 = zone_import+0x52/frame 0xfffffe20216996b0 /usr/src/sys/vm/uma_core.c:2501 > > uma_zalloc_arg() at 0xffffffff806a147e = uma_zalloc_arg+0x36e/frame 0xfffffe2021699720 /usr/src/sys/vm/uma_core.c:2531 > > m_getm2() at 0xffffffff8048231f = m_getm2+0x14f/frame 0xfffffe2021699790 /usr/src/sys/kern/kern_mbuf.c:830 > > m_uiotombuf() at 0xffffffff80516044 = m_uiotombuf+0x64/frame 0xfffffe20216997e0 /usr/src/sys/kern/uipc_mbuf.c:1535 > > sosend_generic() at 0xffffffff8051ce56 = sosend_generic+0x356/frame 0xfffffe20216998a0 > > soo_write() at 0xffffffff804fd872 = soo_write+0x42/frame 0xfffffe20216998d0 > > dofilewrite() at 0xffffffff804f5c97 = dofilewrite+0x87/frame 0xfffffe2021699920 > > kern_writev() at 0xffffffff804f5978 = kern_writev+0x68/frame 0xfffffe2021699970 > > sys_writev() at 0xffffffff804f5be6 = sys_writev+0x36/frame 0xfffffe20216999a0 > > amd64_syscall() at 0xffffffff806e4051 = amd64_syscall+0x2c1/frame 0xfffffe2021699ab0 > > Xfast_syscall() at 0xffffffff806cb2bb = Xfast_syscall+0xfb/frame 0xfffffe2021699ab0 > > --- syscall (121, FreeBSD ELF64, sys_writev), rip = 0x8019cc6ba, rsp = 0x7fffffffd688, rbp = 0x7fffffffd6c0 --- > > This call stack is quite interesting: > 1: A process is calling writev() > 2: Kernel calls sosend_generic() that starts allocating memory > 3: This allocation is then interrupted by the timer interrupt handler > [that could actually trigger tcp_tw_2msl_scan(reuse=0)] > 4: The timer interrupt handler seems to wait on sched_switch() No, this is more interesting: double call (recuersion) to zone_import()! > And fun fact: When sosend_generic() calls m_uiotombuf() it does not > hold INP_WLOCK yet... Yes, is not INP_WLOCK related, this is like next error. From owner-freebsd-stable@freebsd.org Mon Sep 26 12:12:49 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5DB5DBEAB62 for ; Mon, 26 Sep 2016 12:12:49 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1CD1094E for ; Mon, 26 Sep 2016 12:12:49 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1boUm6-0006dC-Ow; Mon, 26 Sep 2016 15:12:46 +0300 Date: Mon, 26 Sep 2016 15:12:46 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20160926121246.GL2840@zxy.spb.ru> References: <78cbcdc9-f565-1046-c157-2ddd8fcccc62@freebsd.org> <20160919204328.GN2840@zxy.spb.ru> <8ba75d6e-4f01-895e-0aed-53c6c6692cb9@freebsd.org> <20160920202633.GQ2840@zxy.spb.ru> <20160921195155.GW2840@zxy.spb.ru> <20160923200143.GG2840@zxy.spb.ru> <20160925124626.GI2840@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Sep 2016 12:12:49 -0000 On Mon, Sep 26, 2016 at 11:33:12AM +0200, Julien Charbon wrote: > >>> - tcp_input()/tcp_twstart()/tcp_tw_2msl_scan(reuse=1) > >> > >> My current hypothesis: > >> > >> nginx do write() (or may be close()?) to socket, kernel lock > >> first inp in V_twq_2msl, happen callout for pfslowtimo() on the same > >> CPU core and tcp_tw_2msl_scan infinity locked on same inp. > >> > >> In this case you modification can't help, before next try we need some > >> like yeld(). > > > > Or may be locks leaks. > > Or both. > > You are totally right, pfslowtimo()/tcp_tw_2msl_scan(reuse=0) is > infinitely blocked on INP_WLOCK() by "something" (that could be related > to write()). > > As I reached my limit of debugging without WITNESS, could you share > your /etc/sysctl.conf, /boot/loader.conf files? And any specific > configuration you have (like having a Nginx workers affinity, Nginx > special options, etc.). Like that I can try to reproduce it on releng/11.0. I am use double socket server, E5-2620. Double Intel 10G NIC, affinity to CPU 6..11. Nginx affinity to CPU 0..5. I.e. on CPU 0 only nginx worker affinity exist and NIC IRQ handler activity only on CPU 6..11. /boot/loader.conf: kern.geom.label.gptid.enable="0" zfs_load="YES" #### generated by conf.pl ######### hw.memtest.tests=0 machdep.hyperthreading_allowed=0 kern.geom.label.disk_ident.enable=0 if_igb_load=yes if_ix_load=yes hw.ix.num_queues=3 hw.ix.rxd=4096 hw.ix.txd=4096 hw.ix.rx_process_limit=-1 hw.ix.tx_process_limit=-1 if_lagg_load=YES net.link.lagg.default_use_flowid=0 accf_http_load=yes aio_load=yes cc_htcp_load=yes kern.ipc.nmbclusters=1048576 net.inet.tcp.reass.maxsegments=32768 net.inet.tcp.hostcache.cachelimit=0 net.inet.tcp.hostcache.hashsize=32768 net.inet.tcp.syncache.hashsize=32768 #net.inet.tcp.tcbhashsize=262144 net.inet.tcp.tcbhashsize=65536 net.inet.tcp.maxtcptw=16384 kern.pin_default_swi=1 kern.pin_pcpu_swi=1 kern.hwpmc.nbuffers=131072 hw.cxgbe.qsize_rxq=16384 hw.cxgbe.qsize_txq=16384 hw.cxgbe.nrxq10g=3 kernel="kernel.VSTREAM" kernels="kernel" hw.mps.max_chains=3072 ### hw.vga.textmode=1 uhci_load=yes ohci_load=yes ehci_load=yes xhci_load=yes ukbd_load=yes umass_load=yes ### boot_multicons="YES" boot_serial="YES" comconsole_speed="115200" comconsole_port=760 #console="comconsole,vidconsole" console="vidconsole,comconsole" hint.uart.0.flags="0x00" hint.uart.1.flags="0x10" /etc/sysctl.conf: kern.random.sys.harvest.ethernet=0 kern.threads.max_threads_per_proc=20000 net.inet.ip.maxfragpackets=32768 net.inet.ip.fastforwarding=1 kern.ipc.somaxconn=4096 kern.ipc.nmbjumbop=2097152 kern.ipc.maxsockbuf=16777216 net.inet.tcp.sendbuf_max=16777216 net.inet.tcp.recvbuf_max=16777216 net.inet.tcp.sendbuf_inc=16384 net.inet.tcp.sendspace=2097152 #net.inet.tcp.maxtcptw=444800 net.inet.tcp.fast_finwait2_recycle=1 net.inet.tcp.msl=1000 net.inet.tcp.cc.algorithm=htcp net.inet.tcp.per_cpu_timers=1 #net.inet.tcp.syncookies=0 net.inet6.ip6.auto_linklocal=0 kern.maxfiles=300000 kern.maxfilesperproc=80000 #hw.intr_storm_threshold=9000 vfs.zfs.prefetch_disable=1 vfs.zfs.vdev.max_pending=1000 vfs.zfs.l2arc_noprefetch=0 vfs.zfs.l2arc_norw=0 vfs.zfs.l2arc_write_boost=134217728 vfs.zfs.l2arc_write_max=33554432 vfs.aio.max_aio_procs=512 vfs.aio.max_aio_queue_per_proc=8192 vfs.aio.max_aio_per_proc=8192 vfs.aio.max_aio_queue=65536 net.inet.tcp.finwait2_timeout=5000 kern.corefile=/tmp/%N.%P.core kern.sugid_coredump=1 Now (after this nginx lockout) I am use you patch witch modification: act return NULL at write lock and now see only mbuf-related work. From owner-freebsd-stable@freebsd.org Mon Sep 26 12:39:01 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 68D56BE64EF for ; Mon, 26 Sep 2016 12:39:01 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 25F3A7BE; Mon, 26 Sep 2016 12:39:01 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1boVBT-0007Hv-4t; Mon, 26 Sep 2016 15:38:59 +0300 Date: Mon, 26 Sep 2016 15:38:59 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20160926123859.GB6177@zxy.spb.ru> References: <78cbcdc9-f565-1046-c157-2ddd8fcccc62@freebsd.org> <20160919204328.GN2840@zxy.spb.ru> <8ba75d6e-4f01-895e-0aed-53c6c6692cb9@freebsd.org> <20160920202633.GQ2840@zxy.spb.ru> <20160921195155.GW2840@zxy.spb.ru> <20160923200143.GG2840@zxy.spb.ru> <20160925124626.GI2840@zxy.spb.ru> <064be3ae-8df9-fa22-06ab-346af024afd3@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <064be3ae-8df9-fa22-06ab-346af024afd3@freebsd.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Sep 2016 12:39:01 -0000 On Mon, Sep 26, 2016 at 01:57:03PM +0200, Julien Charbon wrote: > > Hi Slawa, > > On 9/25/16 2:46 PM, Slawa Olhovchenkov wrote: > > On Fri, Sep 23, 2016 at 11:01:43PM +0300, Slawa Olhovchenkov wrote: > >> On Wed, Sep 21, 2016 at 11:25:18PM +0200, Julien Charbon wrote: > >>> > >>> On 9/21/16 9:51 PM, Slawa Olhovchenkov wrote: > >>>> On Wed, Sep 21, 2016 at 09:11:24AM +0200, Julien Charbon wrote: > >>>>> You can also use Dtrace and lockstat (especially with the lockstat -s > >>>>> option): > >>>>> > >>>>> https://wiki.freebsd.org/DTrace/One-Liners#Kernel_Locks > >>>>> https://www.freebsd.org/cgi/man.cgi?query=lockstat&manpath=FreeBSD+11.0-RELEASE > >>>>> > >>>>> But I am less familiar with Dtrace/lockstat tools. > >>>> > >>>> I am still use old kernel and got lockdown again. > >>>> Try using lockstat (I am save more output), interesting may be next: > >>>> > >>>> R/W writer spin on writer: 190019 events in 1.070 seconds (177571 events/sec) > >>>> > >>>> ------------------------------------------------------------------------------- > >>>> Count indv cuml rcnt nsec Lock Caller > >>>> 140839 74% 74% 0.00 24659 tcpinp tcp_tw_2msl_scan+0xc6 > >>>> > >>>> nsec ------ Time Distribution ------ count Stack > >>>> 4096 | 913 tcp_twstart+0xa3 > >>>> 8192 |@@@@@@@@@@@@ 58191 tcp_do_segment+0x201f > >>>> 16384 |@@@@@@ 29594 tcp_input+0xe1c > >>>> 32768 |@@@@ 23447 ip_input+0x15f > >>>> 65536 |@@@ 16197 > >>>> 131072 |@ 8674 > >>>> 262144 | 3358 > >>>> 524288 | 456 > >>>> 1048576 | 9 > >>>> ------------------------------------------------------------------------------- > >>>> Count indv cuml rcnt nsec Lock Caller > >>>> 49180 26% 100% 0.00 15929 tcpinp tcp_tw_2msl_scan+0xc6 > >>>> > >>>> nsec ------ Time Distribution ------ count Stack > >>>> 4096 | 157 pfslowtimo+0x54 > >>>> 8192 |@@@@@@@@@@@@@@@ 24796 softclock_call_cc+0x179 > >>>> 16384 |@@@@@@ 11223 softclock+0x44 > >>>> 32768 |@@@@ 7426 intr_event_execute_handlers+0x95 > >>>> 65536 |@@ 3918 > >>>> 131072 | 1363 > >>>> 262144 | 278 > >>>> 524288 | 19 > >>>> ------------------------------------------------------------------------------- > >>> > >>> This is interesting, it seems that you have two call paths competing > >>> for INP locks here: > >>> > >>> - pfslowtimo()/tcp_tw_2msl_scan(reuse=0) and > >>> > >>> - tcp_input()/tcp_twstart()/tcp_tw_2msl_scan(reuse=1) > >> > >> My current hypothesis: > >> > >> nginx do write() (or may be close()?) to socket, kernel lock > >> first inp in V_twq_2msl, happen callout for pfslowtimo() on the same > >> CPU core and tcp_tw_2msl_scan infinity locked on same inp. > >> > >> In this case you modification can't help, before next try we need some > >> like yeld(). > > > > Or may be locks leaks. > > Or both. > > Actually one extra debug thing you can do is launching lockstat with > below extra options: > > -H For Hold lock stats > -P To get the overall time > -s 20 To get the stackstrace > > To see who is holding the INP lock for so long. Thanks to Hiren for > pointing the -H option. At time of this graph I am collect output from `lockstat -PH -s 5 sleep 1` too and don't see any interesting -- I am think lock holded before lockstat run don't detected and don't showed. I still can show collected output, if you need. hundreds of lines. From owner-freebsd-stable@freebsd.org Mon Sep 26 15:20:48 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 60092BE37C6 for ; Mon, 26 Sep 2016 15:20:48 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EBBAD92C; Mon, 26 Sep 2016 15:20:47 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u8QFKgba070433 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 26 Sep 2016 18:20:43 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u8QFKgba070433 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u8QFKgC4070430; Mon, 26 Sep 2016 18:20:42 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 26 Sep 2016 18:20:42 +0300 From: Konstantin Belousov To: Slawa Olhovchenkov Cc: John Baldwin , freebsd-stable@freebsd.org, alc@freebsd.org Subject: Re: nginx and FreeBSD11 Message-ID: <20160926152042.GG38409@kib.kiev.ua> References: <20160920192053.GP2840@zxy.spb.ru> <20160920201925.GI38409@kib.kiev.ua> <20160920203853.GR2840@zxy.spb.ru> <20160920211517.GJ38409@kib.kiev.ua> <20160922075933.GL38409@kib.kiev.ua> <20160922082527.GX2840@zxy.spb.ru> <20160922082740.GN38409@kib.kiev.ua> <20160922083424.GY2840@zxy.spb.ru> <20160922085320.GQ38409@kib.kiev.ua> <20160922093355.GZ2840@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160922093355.GZ2840@zxy.spb.ru> User-Agent: Mutt/1.6.1 (2016-04-27) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Sep 2016 15:20:48 -0000 On Thu, Sep 22, 2016 at 12:33:55PM +0300, Slawa Olhovchenkov wrote: > OK, try this patch. Was the patch tested ? From owner-freebsd-stable@freebsd.org Mon Sep 26 17:01:40 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9B1D1BE9AEA for ; Mon, 26 Sep 2016 17:01:40 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5A684F63; Mon, 26 Sep 2016 17:01:40 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1boZHX-000Dp9-Sg; Mon, 26 Sep 2016 20:01:31 +0300 Date: Mon, 26 Sep 2016 20:01:31 +0300 From: Slawa Olhovchenkov To: Konstantin Belousov Cc: John Baldwin , freebsd-stable@freebsd.org, alc@freebsd.org Subject: Re: nginx and FreeBSD11 Message-ID: <20160926170131.GN2840@zxy.spb.ru> References: <20160920201925.GI38409@kib.kiev.ua> <20160920203853.GR2840@zxy.spb.ru> <20160920211517.GJ38409@kib.kiev.ua> <20160922075933.GL38409@kib.kiev.ua> <20160922082527.GX2840@zxy.spb.ru> <20160922082740.GN38409@kib.kiev.ua> <20160922083424.GY2840@zxy.spb.ru> <20160922085320.GQ38409@kib.kiev.ua> <20160922093355.GZ2840@zxy.spb.ru> <20160926152042.GG38409@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160926152042.GG38409@kib.kiev.ua> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Sep 2016 17:01:40 -0000 On Mon, Sep 26, 2016 at 06:20:42PM +0300, Konstantin Belousov wrote: > On Thu, Sep 22, 2016 at 12:33:55PM +0300, Slawa Olhovchenkov wrote: > > OK, try this patch. > > Was the patch tested ? No more AIO related issused/nginx core dumps. I Can't get long uptime by other issuses (tcp locks and mbuf related) From owner-freebsd-stable@freebsd.org Mon Sep 26 17:22:02 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6DEFBBEA3F9 for ; Mon, 26 Sep 2016 17:22:02 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0BE2EE19 for ; Mon, 26 Sep 2016 17:22:02 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1boZbM-000EIV-1L; Mon, 26 Sep 2016 20:22:00 +0300 Date: Mon, 26 Sep 2016 20:22:00 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20160926172159.GA54003@zxy.spb.ru> References: <78cbcdc9-f565-1046-c157-2ddd8fcccc62@freebsd.org> <20160919204328.GN2840@zxy.spb.ru> <8ba75d6e-4f01-895e-0aed-53c6c6692cb9@freebsd.org> <20160920202633.GQ2840@zxy.spb.ru> <20160921195155.GW2840@zxy.spb.ru> <20160923200143.GG2840@zxy.spb.ru> <20160925124626.GI2840@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Sep 2016 17:22:02 -0000 On Mon, Sep 26, 2016 at 11:33:12AM +0200, Julien Charbon wrote: > > Hi Slawa, > > On 9/25/16 2:46 PM, Slawa Olhovchenkov wrote: > > On Fri, Sep 23, 2016 at 11:01:43PM +0300, Slawa Olhovchenkov wrote: > >>> On 9/21/16 9:51 PM, Slawa Olhovchenkov wrote: > >>>> On Wed, Sep 21, 2016 at 09:11:24AM +0200, Julien Charbon wrote: > >>>>> You can also use Dtrace and lockstat (especially with the lockstat -s > >>>>> option): > >>>>> > >>>>> https://wiki.freebsd.org/DTrace/One-Liners#Kernel_Locks > >>>>> https://www.freebsd.org/cgi/man.cgi?query=lockstat&manpath=FreeBSD+11.0-RELEASE > >>>>> > >>>>> But I am less familiar with Dtrace/lockstat tools. > >>>> > >>>> I am still use old kernel and got lockdown again. > >>>> Try using lockstat (I am save more output), interesting may be next: > >>>> > >>>> R/W writer spin on writer: 190019 events in 1.070 seconds (177571 events/sec) > >>>> > >>>> ------------------------------------------------------------------------------- > >>>> Count indv cuml rcnt nsec Lock Caller > >>>> 140839 74% 74% 0.00 24659 tcpinp tcp_tw_2msl_scan+0xc6 > >>>> > >>>> nsec ------ Time Distribution ------ count Stack > >>>> 4096 | 913 tcp_twstart+0xa3 > >>>> 8192 |@@@@@@@@@@@@ 58191 tcp_do_segment+0x201f > >>>> 16384 |@@@@@@ 29594 tcp_input+0xe1c > >>>> 32768 |@@@@ 23447 ip_input+0x15f > >>>> 65536 |@@@ 16197 > >>>> 131072 |@ 8674 > >>>> 262144 | 3358 > >>>> 524288 | 456 > >>>> 1048576 | 9 > >>>> ------------------------------------------------------------------------------- > >>>> Count indv cuml rcnt nsec Lock Caller > >>>> 49180 26% 100% 0.00 15929 tcpinp tcp_tw_2msl_scan+0xc6 > >>>> > >>>> nsec ------ Time Distribution ------ count Stack > >>>> 4096 | 157 pfslowtimo+0x54 > >>>> 8192 |@@@@@@@@@@@@@@@ 24796 softclock_call_cc+0x179 > >>>> 16384 |@@@@@@ 11223 softclock+0x44 > >>>> 32768 |@@@@ 7426 intr_event_execute_handlers+0x95 > >>>> 65536 |@@ 3918 > >>>> 131072 | 1363 > >>>> 262144 | 278 > >>>> 524288 | 19 > >>>> ------------------------------------------------------------------------------- > >>> > >>> This is interesting, it seems that you have two call paths competing > >>> for INP locks here: > >>> > >>> - pfslowtimo()/tcp_tw_2msl_scan(reuse=0) and > >>> > >>> - tcp_input()/tcp_twstart()/tcp_tw_2msl_scan(reuse=1) > >> > >> My current hypothesis: > >> > >> nginx do write() (or may be close()?) to socket, kernel lock > >> first inp in V_twq_2msl, happen callout for pfslowtimo() on the same > >> CPU core and tcp_tw_2msl_scan infinity locked on same inp. > >> > >> In this case you modification can't help, before next try we need some > >> like yeld(). > > > > Or may be locks leaks. > > Or both. > > You are totally right, pfslowtimo()/tcp_tw_2msl_scan(reuse=0) is > infinitely blocked on INP_WLOCK() by "something" (that could be related > to write()). > > As I reached my limit of debugging without WITNESS, could you share > your /etc/sysctl.conf, /boot/loader.conf files? And any specific > configuration you have (like having a Nginx workers affinity, Nginx > special options, etc.). Like that I can try to reproduce it on releng/11.0. Some more traces from ddb: Tracing command intr pid 12 tid 100103 td 0xfffff8012508ea00 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe2020ea8330 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe2020ea8360 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe2020ea83a0 __mtx_lock_sleep() at 0xffffffff80484d9d = __mtx_lock_sleep+0x13d/frame 0xfffffe2020ea8420 sodealloc() at 0xffffffff8051b992 = sodealloc+0x32/frame 0xfffffe2020ea8460 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe2020ea8490 tcp_do_segment() at 0xffffffff80610226 = tcp_do_segment+0x1666/frame 0xfffffe2020ea8590 tcp_input() at 0xffffffff8060e17c = tcp_input+0xe1c/frame 0xfffffe2020ea86e0 ip_input() at 0xffffffff805a087f = ip_input+0x15f/frame 0xfffffe2020ea8740 netisr_dispatch_src() at 0xffffffff80583db5 = netisr_dispatch_src+0xa5/frame 0xfffffe2020ea87a0 ether_demux() at 0xffffffff80575b3a = ether_demux+0x12a/frame 0xfffffe2020ea87d0 ether_nh_input() at 0xffffffff80576792 = ether_nh_input+0x322/frame 0xfffffe2020ea8830 netisr_dispatch_src() at 0xffffffff80583db5 = netisr_dispatch_src+0xa5/frame 0xfffffe2020ea8890 ether_input() at 0xffffffff80575db6 = ether_input+0x26/frame 0xfffffe2020ea88b0 ixgbe_rxeof() at 0xffffffff813df36b = ixgbe_rxeof+0x7ab/frame 0xfffffe2020ea8990 ixgbe_msix_que() at 0xffffffff813da57c = ixgbe_msix_que+0x8c/frame 0xfffffe2020ea89e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe2020ea8a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe2020ea8a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe2020ea8ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe2020ea8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100105 td 0xfffff8012508e000 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe2020eb2330 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe2020eb2360 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe2020eb23a0 __mtx_lock_sleep() at 0xffffffff80484d9d = __mtx_lock_sleep+0x13d/frame 0xfffffe2020eb2420 sodealloc() at 0xffffffff8051b992 = sodealloc+0x32/frame 0xfffffe2020eb2460 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe2020eb2490 tcp_do_segment() at 0xffffffff80610226 = tcp_do_segment+0x1666/frame 0xfffffe2020eb2590 tcp_input() at 0xffffffff8060e17c = tcp_input+0xe1c/frame 0xfffffe2020eb26e0 ip_input() at 0xffffffff805a087f = ip_input+0x15f/frame 0xfffffe2020eb2740 netisr_dispatch_src() at 0xffffffff80583db5 = netisr_dispatch_src+0xa5/frame 0xfffffe2020eb27a0 ether_demux() at 0xffffffff80575b3a = ether_demux+0x12a/frame 0xfffffe2020eb27d0 ether_nh_input() at 0xffffffff80576792 = ether_nh_input+0x322/frame 0xfffffe2020eb2830 netisr_dispatch_src() at 0xffffffff80583db5 = netisr_dispatch_src+0xa5/frame 0xfffffe2020eb2890 ether_input() at 0xffffffff80575db6 = ether_input+0x26/frame 0xfffffe2020eb28b0 ixgbe_rxeof() at 0xffffffff813df36b = ixgbe_rxeof+0x7ab/frame 0xfffffe2020eb2990 ixgbe_msix_que() at 0xffffffff813da57c = ixgbe_msix_que+0x8c/frame 0xfffffe2020eb29e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe2020eb2a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe2020eb2a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe2020eb2ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe2020eb2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100107 td 0xfffff8012508d500 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe2020ebc2b0 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe2020ebc2e0 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe2020ebc320 __mtx_lock_sleep() at 0xffffffff80484d9d = __mtx_lock_sleep+0x13d/frame 0xfffffe2020ebc3a0 soalloc() at 0xffffffff8051b914 = soalloc+0x1b4/frame 0xfffffe2020ebc3f0 sonewconn() at 0xffffffff8051bb9f = sonewconn+0xbf/frame 0xfffffe2020ebc430 syncache_expand() at 0xffffffff8061b85b = syncache_expand+0x78b/frame 0xfffffe2020ebc590 tcp_input() at 0xffffffff8060e10e = tcp_input+0xdae/frame 0xfffffe2020ebc6e0 ip_input() at 0xffffffff805a087f = ip_input+0x15f/frame 0xfffffe2020ebc740 netisr_dispatch_src() at 0xffffffff80583db5 = netisr_dispatch_src+0xa5/frame 0xfffffe2020ebc7a0 ether_demux() at 0xffffffff80575b3a = ether_demux+0x12a/frame 0xfffffe2020ebc7d0 ether_nh_input() at 0xffffffff80576792 = ether_nh_input+0x322/frame 0xfffffe2020ebc830 netisr_dispatch_src() at 0xffffffff80583db5 = netisr_dispatch_src+0xa5/frame 0xfffffe2020ebc890 ether_input() at 0xffffffff80575db6 = ether_input+0x26/frame 0xfffffe2020ebc8b0 ixgbe_rxeof() at 0xffffffff813df36b = ixgbe_rxeof+0x7ab/frame 0xfffffe2020ebc990 ixgbe_msix_que() at 0xffffffff813da57c = ixgbe_msix_que+0x8c/frame 0xfffffe2020ebc9e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe2020ebca20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe2020ebca70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe2020ebcab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe2020ebcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100111 td 0xfffff801250a2000 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe2020f302f0 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe2020f30320 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe2020f30360 __mtx_lock_sleep() at 0xffffffff80484d9d = __mtx_lock_sleep+0x13d/frame 0xfffffe2020f303e0 sodealloc() at 0xffffffff8051b992 = sodealloc+0x32/frame 0xfffffe2020f30420 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe2020f30450 tcp_twstart() at 0xffffffff8061f0e7 = tcp_twstart+0x2b7/frame 0xfffffe2020f30490 tcp_do_segment() at 0xffffffff80610bdf = tcp_do_segment+0x201f/frame 0xfffffe2020f30590 tcp_input() at 0xffffffff8060e17c = tcp_input+0xe1c/frame 0xfffffe2020f306e0 ip_input() at 0xffffffff805a087f = ip_input+0x15f/frame 0xfffffe2020f30740 netisr_dispatch_src() at 0xffffffff80583db5 = netisr_dispatch_src+0xa5/frame 0xfffffe2020f307a0 ether_demux() at 0xffffffff80575b3a = ether_demux+0x12a/frame 0xfffffe2020f307d0 ether_nh_input() at 0xffffffff80576792 = ether_nh_input+0x322/frame 0xfffffe2020f30830 netisr_dispatch_src() at 0xffffffff80583db5 = netisr_dispatch_src+0xa5/frame 0xfffffe2020f30890 ether_input() at 0xffffffff80575db6 = ether_input+0x26/frame 0xfffffe2020f308b0 ixgbe_rxeof() at 0xffffffff813df36b = ixgbe_rxeof+0x7ab/frame 0xfffffe2020f30990 ixgbe_msix_que() at 0xffffffff813da57c = ixgbe_msix_que+0x8c/frame 0xfffffe2020f309e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe2020f30a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe2020f30a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe2020f30ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe2020f30ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100113 td 0xfffff801250a1500 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe2020f3a2f0 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe2020f3a320 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe2020f3a360 __mtx_lock_sleep() at 0xffffffff80484d9d = __mtx_lock_sleep+0x13d/frame 0xfffffe2020f3a3e0 sodealloc() at 0xffffffff8051b992 = sodealloc+0x32/frame 0xfffffe2020f3a420 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe2020f3a450 tcp_twstart() at 0xffffffff8061f0e7 = tcp_twstart+0x2b7/frame 0xfffffe2020f3a490 tcp_do_segment() at 0xffffffff80610bdf = tcp_do_segment+0x201f/frame 0xfffffe2020f3a590 tcp_input() at 0xffffffff8060e17c = tcp_input+0xe1c/frame 0xfffffe2020f3a6e0 ip_input() at 0xffffffff805a087f = ip_input+0x15f/frame 0xfffffe2020f3a740 netisr_dispatch_src() at 0xffffffff80583db5 = netisr_dispatch_src+0xa5/frame 0xfffffe2020f3a7a0 ether_demux() at 0xffffffff80575b3a = ether_demux+0x12a/frame 0xfffffe2020f3a7d0 ether_nh_input() at 0xffffffff80576792 = ether_nh_input+0x322/frame 0xfffffe2020f3a830 netisr_dispatch_src() at 0xffffffff80583db5 = netisr_dispatch_src+0xa5/frame 0xfffffe2020f3a890 ether_input() at 0xffffffff80575db6 = ether_input+0x26/frame 0xfffffe2020f3a8b0 ixgbe_rxeof() at 0xffffffff813df36b = ixgbe_rxeof+0x7ab/frame 0xfffffe2020f3a990 ixgbe_msix_que() at 0xffffffff813da57c = ixgbe_msix_que+0x8c/frame 0xfffffe2020f3a9e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe2020f3aa20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe2020f3aa70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe2020f3aab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe2020f3aab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100115 td 0xfffff801250a0a00 cpustop_handler() at 0xffffffff80775998 = cpustop_handler+0x28/frame 0xfffffe1f9e182cf0 ipi_nmi_handler() at 0xffffffff8077595a = ipi_nmi_handler+0x4a/frame 0xfffffe1f9e182d10 trap() at 0xffffffff806e2e4a = trap+0x3a/frame 0xfffffe1f9e182f20 nmi_calltrap() at 0xffffffff806cb413 = nmi_calltrap+0x8/frame 0xfffffe1f9e182f20 --- trap 0x13, rip = 0xffffffff8059b9f9, rsp = 0xfffffe2020f44420, rbp = 0xfffffe2020f44420 --- in_pcbref() at 0xffffffff8059b9f9 = in_pcbref+0x9/frame 0xfffffe2020f44420 tcp_tw_2msl_scan() at 0xffffffff8061f1a3 = tcp_tw_2msl_scan+0x73/frame 0xfffffe2020f44450 tcp_twstart() at 0xffffffff8061eed3 = tcp_twstart+0xa3/frame 0xfffffe2020f44490 tcp_do_segment() at 0xffffffff80610bdf = tcp_do_segment+0x201f/frame 0xfffffe2020f44590 tcp_input() at 0xffffffff8060e17c = tcp_input+0xe1c/frame 0xfffffe2020f446e0 ip_input() at 0xffffffff805a087f = ip_input+0x15f/frame 0xfffffe2020f44740 netisr_dispatch_src() at 0xffffffff80583db5 = netisr_dispatch_src+0xa5/frame 0xfffffe2020f447a0 ether_demux() at 0xffffffff80575b3a = ether_demux+0x12a/frame 0xfffffe2020f447d0 ether_nh_input() at 0xffffffff80576792 = ether_nh_input+0x322/frame 0xfffffe2020f44830 netisr_dispatch_src() at 0xffffffff80583db5 = netisr_dispatch_src+0xa5/frame 0xfffffe2020f44890 ether_input() at 0xffffffff80575db6 = ether_input+0x26/frame 0xfffffe2020f448b0 ixgbe_rxeof() at 0xffffffff813df36b = ixgbe_rxeof+0x7ab/frame 0xfffffe2020f44990 ixgbe_msix_que() at 0xffffffff813da57c = ixgbe_msix_que+0x8c/frame 0xfffffe2020f449e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe2020f44a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe2020f44a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe2020f44ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe2020f44ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command ps pid 37011 tid 101992 td 0xfffff80144378000 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe20224367b0 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe20224367e0 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe2022436820 __mtx_lock_sleep() at 0xffffffff80484d9d = __mtx_lock_sleep+0x13d/frame 0xfffffe20224368a0 soalloc() at 0xffffffff8051b914 = soalloc+0x1b4/frame 0xfffffe20224368f0 socreate() at 0xffffffff8051b617 = socreate+0xa7/frame 0xfffffe2022436940 sys_socket() at 0xffffffff8052144d = sys_socket+0xed/frame 0xfffffe20224369a0 amd64_syscall() at 0xffffffff806e4051 = amd64_syscall+0x2c1/frame 0xfffffe2022436ab0 Xfast_syscall() at 0xffffffff806cb2bb = Xfast_syscall+0xfb/frame 0xfffffe2022436ab0 --- syscall (97, FreeBSD ELF64, sys_socket), rip = 0x8011c413a, rsp = 0x7fffffffc748, rbp = 0x7fffffffc770 --- Tracing command cron pid 37008 tid 102228 td 0xfffff801a4090000 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe20228d67b0 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe20228d67e0 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe20228d6820 __mtx_lock_sleep() at 0xffffffff80484d9d = __mtx_lock_sleep+0x13d/frame 0xfffffe20228d68a0 soalloc() at 0xffffffff8051b914 = soalloc+0x1b4/frame 0xfffffe20228d68f0 socreate() at 0xffffffff8051b617 = socreate+0xa7/frame 0xfffffe20228d6940 sys_socket() at 0xffffffff8052144d = sys_socket+0xed/frame 0xfffffe20228d69a0 amd64_syscall() at 0xffffffff806e4051 = amd64_syscall+0x2c1/frame 0xfffffe20228d6ab0 Xfast_syscall() at 0xffffffff806cb2bb = Xfast_syscall+0xfb/frame 0xfffffe20228d6ab0 --- syscall (97, FreeBSD ELF64, sys_socket), rip = 0x800d8c13a, rsp = 0x7fffffffd658, rbp = 0x7fffffffd6f0 --- [many likes] Tracing command intr pid 12 tid 100015 td 0xfffff8011422b000 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe1f9e1cf760 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe1f9e1cf790 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe1f9e1cf7d0 __mtx_lock_sleep() at 0xffffffff80484d9d = __mtx_lock_sleep+0x13d/frame 0xfffffe1f9e1cf850 sodealloc() at 0xffffffff8051b992 = sodealloc+0x32/frame 0xfffffe1f9e1cf890 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe1f9e1cf8c0 tcp_timer_2msl() at 0xffffffff8061dda3 = tcp_timer_2msl+0x1f3/frame 0xfffffe1f9e1cf8f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe1f9e1cf9c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe1f9e1cf9e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe1f9e1cfa20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe1f9e1cfa70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe1f9e1cfab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe1f9e1cfab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100016 td 0xfffff8011422aa00 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe1f9e1d4760 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe1f9e1d4790 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe1f9e1d47d0 __mtx_lock_sleep() at 0xffffffff80484d9d = __mtx_lock_sleep+0x13d/frame 0xfffffe1f9e1d4850 sodealloc() at 0xffffffff8051b992 = sodealloc+0x32/frame 0xfffffe1f9e1d4890 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe1f9e1d48c0 tcp_timer_2msl() at 0xffffffff8061dda3 = tcp_timer_2msl+0x1f3/frame 0xfffffe1f9e1d48f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe1f9e1d49c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe1f9e1d49e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe1f9e1d4a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe1f9e1d4a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe1f9e1d4ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe1f9e1d4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100017 td 0xfffff8011422a500 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe1f9e1d9760 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe1f9e1d9790 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe1f9e1d97d0 __mtx_lock_sleep() at 0xffffffff80484d9d = __mtx_lock_sleep+0x13d/frame 0xfffffe1f9e1d9850 sodealloc() at 0xffffffff8051b992 = sodealloc+0x32/frame 0xfffffe1f9e1d9890 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe1f9e1d98c0 tcp_timer_2msl() at 0xffffffff8061dda3 = tcp_timer_2msl+0x1f3/frame 0xfffffe1f9e1d98f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe1f9e1d99c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe1f9e1d99e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe1f9e1d9a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe1f9e1d9a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe1f9e1d9ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe1f9e1d9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100018 td 0xfffff8011422a000 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe1f9e1de760 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe1f9e1de790 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe1f9e1de7d0 __mtx_lock_sleep() at 0xffffffff80484d9d = __mtx_lock_sleep+0x13d/frame 0xfffffe1f9e1de850 sodealloc() at 0xffffffff8051b992 = sodealloc+0x32/frame 0xfffffe1f9e1de890 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe1f9e1de8c0 tcp_timer_2msl() at 0xffffffff8061dda3 = tcp_timer_2msl+0x1f3/frame 0xfffffe1f9e1de8f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe1f9e1de9c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe1f9e1de9e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe1f9e1dea20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe1f9e1dea70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe1f9e1deab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe1f9e1deab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100019 td 0xfffff8011424da00 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe1f9e1e3760 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe1f9e1e3790 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe1f9e1e37d0 __mtx_lock_sleep() at 0xffffffff80484d9d = __mtx_lock_sleep+0x13d/frame 0xfffffe1f9e1e3850 sodealloc() at 0xffffffff8051b992 = sodealloc+0x32/frame 0xfffffe1f9e1e3890 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe1f9e1e38c0 tcp_timer_2msl() at 0xffffffff8061dda3 = tcp_timer_2msl+0x1f3/frame 0xfffffe1f9e1e38f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe1f9e1e39c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe1f9e1e39e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute-- 7:zsh -- time-stamp -- Sep/26/16 20:00:13 -- -- 7:zsh -- time-stamp -- Sep/26/16 20:00:13 -- _handlers+0x95/frame 0xfffffe1f9e1e3a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe1f9e1e3a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe1f9e1e3ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe1f9e1e3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100020 td 0xfffff8011424d500 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe1f9e1e8760 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe1f9e1e8790 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe1f9e1e87d0 __mtx_lock_sleep() at 0xffffffff80484d9d = __mtx_lock_sleep+0x13d/frame 0xfffffe1f9e1e8850 sodealloc() at 0xffffffff8051b992 = sodealloc+0x32/frame 0xfffffe1f9e1e8890 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe1f9e1e88c0 tcp_timer_2msl() at 0xffffffff8061dda3 = tcp_timer_2msl+0x1f3/frame 0xfffffe1f9e1e88f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe1f9e1e89c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe1f9e1e89e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe1f9e1e8a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe1f9e1e8a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe1f9e1e8ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe1f9e1e8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100021 td 0xfffff8011424d000 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe1f9e1ed760 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe1f9e1ed790 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe1f9e1ed7d0 __mtx_lock_sleep() at 0xffffffff80484d9d = __mtx_lock_sleep+0x13d/frame 0xfffffe1f9e1ed850 sodealloc() at 0xffffffff8051b992 = sodealloc+0x32/frame 0xfffffe1f9e1ed890 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe1f9e1ed8c0 tcp_timer_2msl() at 0xffffffff8061dda3 = tcp_timer_2msl+0x1f3/frame 0xfffffe1f9e1ed8f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe1f9e1ed9c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe1f9e1ed9e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe1f9e1eda20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe1f9e1eda70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe1f9e1edab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe1f9e1edab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100022 td 0xfffff8011424ca00 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe1f9e1f2760 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe1f9e1f2790 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe1f9e1f27d0 __mtx_lock_sleep() at 0xffffffff80484d9d = __mtx_lock_sleep+0x13d/frame 0xfffffe1f9e1f2850 sodealloc() at 0xffffffff8051b992 = sodealloc+0x32/frame 0xfffffe1f9e1f2890 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe1f9e1f28c0 tcp_timer_2msl() at 0xffffffff8061dda3 = tcp_timer_2msl+0x1f3/frame 0xfffffe1f9e1f28f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe1f9e1f29c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe1f9e1f29e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe1f9e1f2a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe1f9e1f2a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe1f9e1f2ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe1f9e1f2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100023 td 0xfffff8011424c500 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe1f9e1f7760 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe1f9e1f7790 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe1f9e1f77d0 __mtx_lock_sleep() at 0xffffffff80484d9d = __mtx_lock_sleep+0x13d/frame 0xfffffe1f9e1f7850 sodealloc() at 0xffffffff8051b992 = sodealloc+0x32/frame 0xfffffe1f9e1f7890 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe1f9e1f78c0 tcp_timer_rexmt() at 0xffffffff8061e454 = tcp_timer_rexmt+0x114/frame 0xfffffe1f9e1f78f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe1f9e1f79c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe1f9e1f79e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe1f9e1f7a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe1f9e1f7a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe1f9e1f7ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe1f9e1f7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100024 td 0xfffff8011424c000 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe1f9e1fc760 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe1f9e1fc790 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe1f9e1fc7d0 __mtx_lock_sleep() at 0xffffffff80484d9d = __mtx_lock_sleep+0x13d/frame 0xfffffe1f9e1fc850 sodealloc() at 0xffffffff8051b992 = sodealloc+0x32/frame 0xfffffe1f9e1fc890 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe1f9e1fc8c0 tcp_timer_2msl() at 0xffffffff8061dda3 = tcp_timer_2msl+0x1f3/frame 0xfffffe1f9e1fc8f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe1f9e1fc9c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe1f9e1fc9e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe1f9e1fca20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe1f9e1fca70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe1f9e1fcab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe1f9e1fcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100025 td 0xfffff8011424ba00 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe0000382760 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe0000382790 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe00003827d0 __mtx_lock_sleep() at 0xffffffff80484d9d = __mtx_lock_sleep+0x13d/frame 0xfffffe0000382850 sodealloc() at 0xffffffff8051b992 = sodealloc+0x32/frame 0xfffffe0000382890 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe00003828c0 tcp_timer_2msl() at 0xffffffff8061dda3 = tcp_timer_2msl+0x1f3/frame 0xfffffe00003828f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe00003829c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe00003829e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe0000382a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe0000382a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe0000382ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe0000382ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100026 td 0xfffff8011424b500 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe00003876f0 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe0000387720 critical_exit() at 0xffffffff804a6bee = critical_exit+0x7e/frame 0xfffffe0000387740 ipi_bitmap_handler() at 0xffffffff80775629 = ipi_bitmap_handler+0x79/frame 0xfffffe0000387780 Xipi_intr_bitmap_handler() at 0xffffffff806cc15e = Xipi_intr_bitmap_handler+0x8e/frame 0xfffffe0000387780 --- interrupt, rip = 0xffffffff80484c1f, rsp = 0xfffffe0000387850, rbp = 0xfffffe0000387850 --- __mtx_lock_flags() at 0xffffffff80484c1f = __mtx_lock_flags+0x2f/frame 0xfffffe0000387850 sodealloc() at 0xffffffff8051b992 = sodealloc+0x32/frame 0xfffffe0000387890 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe00003878c0 tcp_timer_2msl() at 0xffffffff8061dda3 = tcp_timer_2msl+0x1f3/frame 0xfffffe00003878f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe00003879c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe00003879e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe0000387a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe0000387a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe0000387ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe0000387ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- From owner-freebsd-stable@freebsd.org Mon Sep 26 23:55:39 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5DD6DBEB418 for ; Mon, 26 Sep 2016 23:55:39 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-51.reflexion.net [208.70.210.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 23B572F1 for ; Mon, 26 Sep 2016 23:55:39 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 16173 invoked from network); 26 Sep 2016 23:49:39 -0000 Received: from unknown (HELO mail-cs-02.app.dca.reflexion.local) (10.81.19.2) by 0 (rfx-qmail) with SMTP; 26 Sep 2016 23:49:39 -0000 Received: by mail-cs-02.app.dca.reflexion.local (Reflexion email security v8.00.0) with SMTP; Mon, 26 Sep 2016 19:48:52 -0400 (EDT) Received: (qmail 25697 invoked from network); 26 Sep 2016 23:48:51 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 26 Sep 2016 23:48:51 -0000 Received: from [192.168.0.105] (ip70-189-131-151.lv.lv.cox.net [70.189.131.151]) by iron2.pdx.net (Postfix) with ESMTPSA id 0EB91EC8839; Mon, 26 Sep 2016 16:48:51 -0700 (PDT) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: devel/powerpc64-gcc cross-build of libc++ based stable/11 -r306344 buildworld failed: call to . . . lacks nop, can't restore toc; recompile with -fPIC Message-Id: Date: Mon, 26 Sep 2016 16:48:50 -0700 To: FreeBSD Toolchain , FreeBSD PowerPC ML , FreeBSD-STABLE Mailing List , FreeBSD Ports Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Sep 2016 23:55:39 -0000 So far I'm unable to use powerpc64-gcc (as of /usr/ports -r422765) to = build stable/11 -r306344 on amd64 -r306344 : See the > iostream.cpp:(.text.startup+0x1c): call to = `std::__1::ios_base::Init::Init()' lacks nop, can't restore toc; = recompile with -fPIC in the later extraction from the build log. The version of ld (and other = such) in use is: > # /usr/local/bin/powerpc64-freebsd-ld --version > GNU ld (GNU Binutils) 2.25.1 . . . This is because of reported problems with 2.27 for powerpc64. (It will = still be a while before I've access to the powerpc64's again.) As for the log file material. . . > --- all_subdir_lib/libdevdctl --- > Building = /usr/obj/xtoolchain/powerpc.powerpc64/usr/src/lib/libdevdctl/libprivatedev= dctl.so.0.full > --- libprivatedevdctl.so.0.full --- > building shared library libprivatedevdctl.so.0 . . . > --- all_subdir_lib/libdevdctl --- > /usr/local/bin/powerpc64-freebsd-ld: = /usr/obj/xtoolchain/powerpc.powerpc64/usr/src/tmp/../lib/libc++/libc++.a(i= ostream.o): In function `_GLOBAL__sub_I__ZNSt3__13cinE': > iostream.cpp:(.text.startup+0x1c): call to = `std::__1::ios_base::Init::Init()' lacks nop, can't restore toc; = recompile with -fPIC > /usr/local/bin/powerpc64-freebsd-ld: final link failed: Bad value > --- all_subdir_lib/libevent --- > COLLECT_GCC_OPTIONS=3D'-isystem' = '/usr/obj/xtoolchain/powerpc.powerpc64/usr/src/tmp/usr/include' = '-L/usr/obj/xtoolchain/powerpc.powerpc64/usr/src/tmp/usr/lib' '-B' = '/usr/local/powerpc64-freebsd/bin/' '-O2' '-pipe' '-I' = '/usr/src/lib/libevent' '-D' 'HAVE_CLOCK_GETTIME' '-D' 'HAVE_FCNTL_H' = '-D' 'HAVE_POLL' '-D' 'HAVE_SELECT' '-D' 'HAVE_SETFD' '-D' = 'HAVE_STDARG_H' '-D' 'HAVE_SYS_IOCTL_H' '-D' 'HAVE_SYS_TIME_H' '-D' = 'HAVE_UNISTD_H' '-D' 'HAVE_VASPRINTF' '-D' 'HAVE_WORKING_KQUEUE' '-D' = 'VERSION=3D"1.3b"' '-std=3Dgnu99' '-fstack-protector-strong' = '-Wsystem-headers' '-Wall' '-Wno-format-y2k' '-Wno-uninitialized' = '-Wno-pointer-sign' '-Wno-error=3Dunused-function' = '-Wno-error=3Denum-compare' '-Wno-error=3Dlogical-not-parentheses' = '-Wno-error=3Dbool-compare' '-Wno-error=3Duninitialized' = '-Wno-error=3Darray-bounds' '-Wno-error=3Dclobbered' = '-Wno-error=3Dcast-align' '-Wno-error=3Dextra' '-Wno-error=3Dattributes' = '-Wno-error=3Dinline' '-Wno-error=3Dunused-but-set-variable' = '-Wno-error=3Dunused-value' '-Wno-error=3Dstrict-aliasing' = '-Wno-error=3Daddress' '-v' '-c' '-o' 'signal.o' > --- all_subdir_lib/libdevdctl --- > collect2: error: ld returned 1 exit status > *** [libprivatedevdctl.so.0.full] Error code 1 >=20 > make[5]: stopped in /usr/src/lib/libdevdctl > .ERROR_TARGET=3D'libprivatedevdctl.so.0.full' > = .ERROR_META_FILE=3D'/usr/obj/xtoolchain/powerpc.powerpc64/usr/src/lib/libd= evdctl/libprivatedevdctl.so.0.full.meta' > .MAKE.LEVEL=3D'5' > MAKEFILE=3D'' > .MAKE.MODE=3D'meta missing-filemon=3Dyes missing-meta=3Dyes silent=3Dyes= verbose' > .CURDIR=3D'/usr/src/lib/libdevdctl' > .MAKE=3D'make' > .OBJDIR=3D'/usr/obj/xtoolchain/powerpc.powerpc64/usr/src/lib/libdevdctl'= > .TARGETS=3D'all' > DESTDIR=3D'/usr/obj/xtoolchain/powerpc.powerpc64/usr/src/tmp' > LD_LIBRARY_PATH=3D'' > MACHINE=3D'powerpc' > MACHINE_ARCH=3D'powerpc64' > MAKEOBJDIRPREFIX=3D'/usr/obj/xtoolchain/powerpc.powerpc64' > MAKESYSPATH=3D'/usr/src/share/mk' > MAKE_VERSION=3D'20160606' > = PATH=3D'/usr/obj/xtoolchain/powerpc.powerpc64/usr/src/tmp/legacy/usr/sbin:= /usr/obj/xtoolchain/powerpc.powerpc64/usr/src/tmp/legacy/usr/bin:/usr/obj/= xtoolchain/powerpc.powerpc64/usr/src/tmp/legacy/bin:/usr/obj/xtoolchain/po= werpc.powerpc64/usr/src/tmp/usr/sbin:/usr/obj/xtoolchain/powerpc.powerpc64= /usr/src/tmp/usr/bin:/sbin:/bin:/usr/sbin:/usr/bin' > SRCTOP=3D'/usr/src' > OBJTOP=3D'/usr/obj/xtoolchain/powerpc.powerpc64/usr/src' > .MAKE.MAKEFILES=3D'/usr/src/share/mk/sys.mk = /usr/src/share/mk/local.sys.env.mk /usr/src/share/mk/src.sys.env.mk = /root/src.configs/src.conf.powerpc64-xtoolchain.amd64-host = /usr/src/share/mk/bsd.mkopt.mk /root/src.configs/make.conf = /usr/src/share/mk/local.sys.mk /usr/src/share/mk/src.sys.mk = /etc/src.conf /usr/src/lib/libdevdctl/Makefile = /usr/src/share/mk/bsd.lib.mk /usr/src/share/mk/bsd.init.mk = /usr/src/share/mk/bsd.opts.mk /usr/src/share/mk/bsd.cpu.mk = /usr/src/share/mk/local.init.mk /usr/src/share/mk/src.init.mk = /usr/src/lib/libdevdctl/../Makefile.inc /usr/src/share/mk/bsd.own.mk = /usr/src/share/mk/bsd.compiler.mk /usr/src/share/mk/bsd.libnames.mk = /usr/src/share/mk/src.libnames.mk /usr/src/share/mk/src.opts.mk = /usr/src/share/mk/bsd.symver.mk /usr/src/share/mk/bsd.nls.mk = /usr/src/share/mk/bsd.files.mk /usr/src/share/mk/bsd.incs.mk = /usr/src/share/mk/bsd.confs.mk /usr/src/share/mk/bsd.links.mk = /usr/src/share/mk/bsd.dep.mk /usr/src/share/mk/bsd.clang-analyze.mk = /usr/src/share/mk/bsd.obj.mk /usr/src/share/mk/bsd.subdir.mk = /usr/src/share/mk/bsd.sys.mk' > .PATH=3D'. /usr/src/lib/libdevdctl' > 1 error Context details: > # freebsd-version -ku; uname -apKU > 11.0-PRERELEASE > 11.0-PRERELEASE > FreeBSD FreeBSDx64 11.0-PRERELEASE FreeBSD 11.0-PRERELEASE #7 = r306344M: Mon Sep 26 10:29:55 PDT 2016 = markmi@FreeBSDx64:/usr/obj/clang/amd64.amd64/usr/src/sys/GENERIC-NODBG = amd64 amd64 1100503 1100503 > # svnlite info /usr/src | grep "Re[lv][ai:]" > Relative URL: ^/stable/11 > Revision: 306344 > Last Changed Rev: 306344 > # svnlite info /usr/ports | grep "Re[lv][ai:]" > Relative URL: ^/head > Revision: 422765 > Last Changed Rev: 422765 But I have avoided the recent 2.27 of devel/binutils and = devel/powerpc64-binutils : > # svnlite info /usr/ports/devel/powerpc64-binutils/ | grep = "Re[lv][ai:]" > Relative URL: ^/head/devel/powerpc64-binutils > Revision: 407342 > Last Changed Rev: 407342 > # svnlite info /usr/ports/devel/binutils/ | grep "Re[lv][ai:]" > Relative URL: ^/head/devel/binutils > Revision: 416639 > Last Changed Rev: 416639 As for src.conf and related. . . > # more = ~/sys_build_scripts.amd64-host/make_powerpc64vtsc_nodebug_incl_clang_xtool= chain-amd64-host.sh=20 > kldload -n filemon && \ > script = ~/sys_typescripts/typescript_make_powerpc64vtsc_nodebug_incl_clang_xtoolch= ain-amd64-host-$(date +%Y-%m-%d:%H:%M:%S) \ > env __MAKE_CONF=3D"/root/src.configs/make.conf" = SRC_ENV_CONF=3D"/root/src.configs/src.conf.powerpc64-xtoolchain.amd64-host= " \ > WITH_META_MODE=3Dyes \ > MAKEOBJDIRPREFIX=3D"/usr/obj/xtoolchain" \ > make $* > # more ~/src.configs/src.conf.powerpc64-xtoolchain.amd64-host=20 > TO_TYPE=3Dpowerpc64 > TOOLS_TO_TYPE=3D${TO_TYPE} > VERSION_CONTEXT=3D11.0 > # > KERNCONF=3DGENERIC64vtsc-NODEBUG > TARGET=3Dpowerpc > .if ${.MAKE.LEVEL} =3D=3D 0 > TARGET_ARCH=3D${TO_TYPE} > .export TARGET_ARCH > .endif > # > WITHOUT_CROSS_COMPILER=3D > WITHOUT_SYSTEM_COMPILER=3D > # > WITH_LIBCPLUSPLUS=3D > WITHOUT_BINUTILS_BOOTSTRAP=3D > WITHOUT_CLANG_BOOTSTRAP=3D > WITH_CLANG=3D > WITH_CLANG_IS_CC=3D > WITH_CLANG_FULL=3D > WITH_CLANG_EXTRAS=3D > WITH_LLDB=3D > # > WITH_BOOT=3D > # powerpc64 LIB32 builds via gcc 4.9 or later variants that I've tried > # but the LIB32 does not work [crtbeginS code problem(s)] > WITHOUT_LIB32=3D > # > WITHOUT_ELFTOOLCHAIN_BOOTSTRAP=3D > WITHOUT_GCC_BOOTSTRAP=3D > WITHOUT_GCC=3D > WITHOUT_GCC_IS_CC=3D > WITHOUT_GNUCXX=3D > # > NO_WERROR=3D > #WERROR=3D > MALLOC_PRODUCTION=3D > # > WITH_DEBUG_FILES=3D > # > # > # For TO (so-called "cross") stages . . . > # So-called-cross via ${TO_TYPE}-xtoolchain-gcc/${TO_TYPE}-gcc. . . > # TOOLS_TO_TYPE based on ${TO_TYPE}-xtoolchain-gcc related binutils. . = . > # > CROSS_TOOLCHAIN=3D${TO_TYPE}-gcc > X_COMPILER_TYPE=3Dgcc > CROSS_BINUTILS_PREFIX=3D/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/ > .if ${.MAKE.LEVEL} =3D=3D 0 > = XCC=3D/usr/local/bin/${TOOLS_TO_TYPE}-portbld-freebsd${VERSION_CONTEXT}-gc= c > = XCXX=3D/usr/local/bin/${TOOLS_TO_TYPE}-portbld-freebsd${VERSION_CONTEXT}-g= ++ > = XCPP=3D/usr/local/bin/${TOOLS_TO_TYPE}-portbld-freebsd${VERSION_CONTEXT}-c= pp > .export XCC > .export XCXX > .export XCPP > XAS=3D/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/as > XAR=3D/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/ar > XLD=3D/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/ld > XNM=3D/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/nm > XOBJCOPY=3D/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/objcopy > XOBJDUMP=3D/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/objdump > XRANLIB=3D/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/ranlib > XSIZE=3D/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/size > #NO-SUCH: XSTRINGS=3D/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/strings > XSTRINGS=3D/usr/local/bin/${TOOLS_TO_TYPE}-freebsd-strings > .export XAS > .export XAR > .export XLD > .export XNM > .export XOBJCOPY > .export XOBJDUMP > .export XRANLIB > .export XSIZE > .export XSTRINGS > .endif > # > # > # =46rom based on clang (via system). . . > # > .if ${.MAKE.LEVEL} =3D=3D 0 > CC=3D/usr/bin/clang > CXX=3D/usr/bin/clang++ > CPP=3D/usr/bin/clang-cpp > .export CC > .export CXX > .export CPP > .endif > # more /etc/make.conf > WANT_QT_VERBOSE_CONFIGURE=3D1 > # > DEFAULT_VERSIONS+=3Dperl5=3D5.22 > WRKDIRPREFIX=3D/usr/obj/portswork > WITH_DEBUG=3D > WITH_DEBUG_FILES=3D > MALLOC_PRODUCTION=3D > # svnlite status /usr/src > M = /usr/src/contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp > M /usr/src/lib/csu/powerpc64/Makefile > ? /usr/src/sys/amd64/conf/GENERIC-NODBG > ? /usr/src/sys/amd64/include/include > ? /usr/src/sys/arm/conf/RPI2-NODBG > ? /usr/src/sys/arm/include/include > M /usr/src/sys/boot/ofw/Makefile.inc > M /usr/src/sys/boot/powerpc/Makefile.inc > M /usr/src/sys/boot/powerpc/kboot/Makefile > M /usr/src/sys/boot/uboot/Makefile.inc > M /usr/src/sys/conf/Makefile.powerpc > M /usr/src/sys/conf/kern.mk > M /usr/src/sys/conf/kmod.mk > ? /usr/src/sys/powerpc/conf/GENERIC64-NODBG > ? /usr/src/sys/powerpc/conf/GENERIC64vtsc > ? /usr/src/sys/powerpc/conf/GENERIC64vtsc-NODEBUG > ? /usr/src/sys/powerpc/conf/GENERICvtsc > ? /usr/src/sys/powerpc/conf/GENERICvtsc-NODEBUG > ? /usr/src/sys/powerpc/include/include > M /usr/src/sys/powerpc/ofw/ofw_machdep.c > M /usr/src/sys/powerpc/powerpc/exec_machdep.c > ? /usr/src/sys/x86/include/include The include/include's were not added by me. They just point back to the = parent include. The rest of the powerpc64 stuff I've been using for a long time, some of = it is for booting PowerMac G5's reliably. Other powerpc64/powerpc parts = are mostly tied to my clang 3.8.0 investigations for targeting powerpc64 = and powerpc. =3D=3D=3D Mark Millard markmi at dsl-only.net From owner-freebsd-stable@freebsd.org Tue Sep 27 02:47:56 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 01BABBEA01B for ; Tue, 27 Sep 2016 02:47:56 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-68.reflexion.net [208.70.210.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B70B62E5 for ; Tue, 27 Sep 2016 02:47:54 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 16648 invoked from network); 27 Sep 2016 02:48:36 -0000 Received: from unknown (HELO mail-cs-01.app.dca.reflexion.local) (10.81.19.1) by 0 (rfx-qmail) with SMTP; 27 Sep 2016 02:48:36 -0000 Received: by mail-cs-01.app.dca.reflexion.local (Reflexion email security v8.00.0) with SMTP; Mon, 26 Sep 2016 22:47:54 -0400 (EDT) Received: (qmail 9121 invoked from network); 27 Sep 2016 02:47:53 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 27 Sep 2016 02:47:53 -0000 Received: from [192.168.0.105] (ip70-189-131-151.lv.lv.cox.net [70.189.131.151]) by iron2.pdx.net (Postfix) with ESMTPSA id 2107DEC8B8B; Mon, 26 Sep 2016 19:47:47 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: devel/powerpc64-gcc cross-build of libc++ based stable/11 -r306344 buildworld failed: call to . . . lacks nop, can't restore toc; recompile with -fPIC From: Mark Millard In-Reply-To: Date: Mon, 26 Sep 2016 19:47:46 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <3F3E013B-2A68-47F8-A700-4CAA6D56BFB4@dsl-only.net> References: To: FreeBSD Toolchain , FreeBSD PowerPC ML , FreeBSD-STABLE Mailing List , FreeBSD Ports X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Sep 2016 02:47:56 -0000 Quick top post on avoiding the problem: Reverting devel/powerpc64-gcc from -r421598 to -r413189 appears to have = avoided this problem. While buildworld is still building: the build is well past the failure = point reported below. =3D=3D=3D Mark Millard markmi at dsl-only.net On 2016-Sep-26, at 4:48 PM, Mark Millard wrote: So far I'm unable to use powerpc64-gcc (as of /usr/ports -r422765) to = build stable/11 -r306344 on amd64 -r306344 : See the > iostream.cpp:(.text.startup+0x1c): call to = `std::__1::ios_base::Init::Init()' lacks nop, can't restore toc; = recompile with -fPIC in the later extraction from the build log. The version of ld (and other = such) in use is: > # /usr/local/bin/powerpc64-freebsd-ld --version > GNU ld (GNU Binutils) 2.25.1 . . . This is because of reported problems with 2.27 for powerpc64. (It will = still be a while before I've access to the powerpc64's again.) As for the log file material. . . > --- all_subdir_lib/libdevdctl --- > Building = /usr/obj/xtoolchain/powerpc.powerpc64/usr/src/lib/libdevdctl/libprivatedev= dctl.so.0.full > --- libprivatedevdctl.so.0.full --- > building shared library libprivatedevdctl.so.0 . . . > --- all_subdir_lib/libdevdctl --- > /usr/local/bin/powerpc64-freebsd-ld: = /usr/obj/xtoolchain/powerpc.powerpc64/usr/src/tmp/../lib/libc++/libc++.a(i= ostream.o): In function `_GLOBAL__sub_I__ZNSt3__13cinE': > iostream.cpp:(.text.startup+0x1c): call to = `std::__1::ios_base::Init::Init()' lacks nop, can't restore toc; = recompile with -fPIC > /usr/local/bin/powerpc64-freebsd-ld: final link failed: Bad value > --- all_subdir_lib/libevent --- > COLLECT_GCC_OPTIONS=3D'-isystem' = '/usr/obj/xtoolchain/powerpc.powerpc64/usr/src/tmp/usr/include' = '-L/usr/obj/xtoolchain/powerpc.powerpc64/usr/src/tmp/usr/lib' '-B' = '/usr/local/powerpc64-freebsd/bin/' '-O2' '-pipe' '-I' = '/usr/src/lib/libevent' '-D' 'HAVE_CLOCK_GETTIME' '-D' 'HAVE_FCNTL_H' = '-D' 'HAVE_POLL' '-D' 'HAVE_SELECT' '-D' 'HAVE_SETFD' '-D' = 'HAVE_STDARG_H' '-D' 'HAVE_SYS_IOCTL_H' '-D' 'HAVE_SYS_TIME_H' '-D' = 'HAVE_UNISTD_H' '-D' 'HAVE_VASPRINTF' '-D' 'HAVE_WORKING_KQUEUE' '-D' = 'VERSION=3D"1.3b"' '-std=3Dgnu99' '-fstack-protector-strong' = '-Wsystem-headers' '-Wall' '-Wno-format-y2k' '-Wno-uninitialized' = '-Wno-pointer-sign' '-Wno-error=3Dunused-function' = '-Wno-error=3Denum-compare' '-Wno-error=3Dlogical-not-parentheses' = '-Wno-error=3Dbool-compare' '-Wno-error=3Duninitialized' = '-Wno-error=3Darray-bounds' '-Wno-error=3Dclobbered' = '-Wno-error=3Dcast-align' '-Wno-error=3Dextra' '-Wno-error=3Dattributes' = '-Wno-error=3Dinline' '-Wno-error=3Dunused-but-set-variable' = '-Wno-error=3Dunused-value' '-Wno-error=3Dstrict-aliasi ng' '-Wno-error=3Daddress' '-v' '-c' '-o' 'signal.o' > --- all_subdir_lib/libdevdctl --- > collect2: error: ld returned 1 exit status > *** [libprivatedevdctl.so.0.full] Error code 1 >=20 > make[5]: stopped in /usr/src/lib/libdevdctl > .ERROR_TARGET=3D'libprivatedevdctl.so.0.full' > = .ERROR_META_FILE=3D'/usr/obj/xtoolchain/powerpc.powerpc64/usr/src/lib/libd= evdctl/libprivatedevdctl.so.0.full.meta' > .MAKE.LEVEL=3D'5' > MAKEFILE=3D'' > .MAKE.MODE=3D'meta missing-filemon=3Dyes missing-meta=3Dyes silent=3Dyes= verbose' > .CURDIR=3D'/usr/src/lib/libdevdctl' > .MAKE=3D'make' > .OBJDIR=3D'/usr/obj/xtoolchain/powerpc.powerpc64/usr/src/lib/libdevdctl'= > .TARGETS=3D'all' > DESTDIR=3D'/usr/obj/xtoolchain/powerpc.powerpc64/usr/src/tmp' > LD_LIBRARY_PATH=3D'' > MACHINE=3D'powerpc' > MACHINE_ARCH=3D'powerpc64' > MAKEOBJDIRPREFIX=3D'/usr/obj/xtoolchain/powerpc.powerpc64' > MAKESYSPATH=3D'/usr/src/share/mk' > MAKE_VERSION=3D'20160606' > = PATH=3D'/usr/obj/xtoolchain/powerpc.powerpc64/usr/src/tmp/legacy/usr/sbin:= /usr/obj/xtoolchain/powerpc.powerpc64/usr/src/tmp/legacy/usr/bin:/usr/obj/= xtoolchain/powerpc.powerpc64/usr/src/tmp/legacy/bin:/usr/obj/xtoolchain/po= werpc.powerpc64/usr/src/tmp/usr/sbin:/usr/obj/xtoolchain/powerpc.powerpc64= /usr/src/tmp/usr/bin:/sbin:/bin:/usr/sbin:/usr/bin' > SRCTOP=3D'/usr/src' > OBJTOP=3D'/usr/obj/xtoolchain/powerpc.powerpc64/usr/src' > .MAKE.MAKEFILES=3D'/usr/src/share/mk/sys.mk = /usr/src/share/mk/local.sys.env.mk /usr/src/share/mk/src.sys.env.mk = /root/src.configs/src.conf.powerpc64-xtoolchain.amd64-host = /usr/src/share/mk/bsd.mkopt.mk /root/src.configs/make.conf = /usr/src/share/mk/local.sys.mk /usr/src/share/mk/src.sys.mk = /etc/src.conf /usr/src/lib/libdevdctl/Makefile = /usr/src/share/mk/bsd.lib.mk /usr/src/share/mk/bsd.init.mk = /usr/src/share/mk/bsd.opts.mk /usr/src/share/mk/bsd.cpu.mk = /usr/src/share/mk/local.init.mk /usr/src/share/mk/src.init.mk = /usr/src/lib/libdevdctl/../Makefile.inc /usr/src/share/mk/bsd.own.mk = /usr/src/share/mk/bsd.compiler.mk /usr/src/share/mk/bsd.libnames.mk = /usr/src/share/mk/src.libnames.mk /usr/src/share/mk/src.opts.mk = /usr/src/share/mk/bsd.symver.mk /usr/src/share/mk/bsd.nls.mk = /usr/src/share/mk/bsd.files.mk /usr/src/share/mk/bsd.incs.mk = /usr/src/share/mk/bsd.confs.mk /usr/src/share/mk/bsd.links.mk = /usr/src/share/mk/bsd.dep.mk /usr/src/share/mk/bsd.clang-analyze.mk = /usr/src/share/mk/bsd.obj.mk /usr/src/share/mk/bsd.subdir.mk /usr/src/share/mk/bsd.sys.mk' > .PATH=3D'. /usr/src/lib/libdevdctl' > 1 error Context details: > # freebsd-version -ku; uname -apKU > 11.0-PRERELEASE > 11.0-PRERELEASE > FreeBSD FreeBSDx64 11.0-PRERELEASE FreeBSD 11.0-PRERELEASE #7 = r306344M: Mon Sep 26 10:29:55 PDT 2016 = markmi@FreeBSDx64:/usr/obj/clang/amd64.amd64/usr/src/sys/GENERIC-NODBG = amd64 amd64 1100503 1100503 > # svnlite info /usr/src | grep "Re[lv][ai:]" > Relative URL: ^/stable/11 > Revision: 306344 > Last Changed Rev: 306344 > # svnlite info /usr/ports | grep "Re[lv][ai:]" > Relative URL: ^/head > Revision: 422765 > Last Changed Rev: 422765 But I have avoided the recent 2.27 of devel/binutils and = devel/powerpc64-binutils : > # svnlite info /usr/ports/devel/powerpc64-binutils/ | grep = "Re[lv][ai:]" > Relative URL: ^/head/devel/powerpc64-binutils > Revision: 407342 > Last Changed Rev: 407342 > # svnlite info /usr/ports/devel/binutils/ | grep "Re[lv][ai:]" > Relative URL: ^/head/devel/binutils > Revision: 416639 > Last Changed Rev: 416639 As for src.conf and related. . . > # more = ~/sys_build_scripts.amd64-host/make_powerpc64vtsc_nodebug_incl_clang_xtool= chain-amd64-host.sh=20 > kldload -n filemon && \ > script = ~/sys_typescripts/typescript_make_powerpc64vtsc_nodebug_incl_clang_xtoolch= ain-amd64-host-$(date +%Y-%m-%d:%H:%M:%S) \ > env __MAKE_CONF=3D"/root/src.configs/make.conf" = SRC_ENV_CONF=3D"/root/src.configs/src.conf.powerpc64-xtoolchain.amd64-host= " \ > WITH_META_MODE=3Dyes \ > MAKEOBJDIRPREFIX=3D"/usr/obj/xtoolchain" \ > make $* > # more ~/src.configs/src.conf.powerpc64-xtoolchain.amd64-host=20 > TO_TYPE=3Dpowerpc64 > TOOLS_TO_TYPE=3D${TO_TYPE} > VERSION_CONTEXT=3D11.0 > # > KERNCONF=3DGENERIC64vtsc-NODEBUG > TARGET=3Dpowerpc > .if ${.MAKE.LEVEL} =3D=3D 0 > TARGET_ARCH=3D${TO_TYPE} > .export TARGET_ARCH > .endif > # > WITHOUT_CROSS_COMPILER=3D > WITHOUT_SYSTEM_COMPILER=3D > # > WITH_LIBCPLUSPLUS=3D > WITHOUT_BINUTILS_BOOTSTRAP=3D > WITHOUT_CLANG_BOOTSTRAP=3D > WITH_CLANG=3D > WITH_CLANG_IS_CC=3D > WITH_CLANG_FULL=3D > WITH_CLANG_EXTRAS=3D > WITH_LLDB=3D > # > WITH_BOOT=3D > # powerpc64 LIB32 builds via gcc 4.9 or later variants that I've tried > # but the LIB32 does not work [crtbeginS code problem(s)] > WITHOUT_LIB32=3D > # > WITHOUT_ELFTOOLCHAIN_BOOTSTRAP=3D > WITHOUT_GCC_BOOTSTRAP=3D > WITHOUT_GCC=3D > WITHOUT_GCC_IS_CC=3D > WITHOUT_GNUCXX=3D > # > NO_WERROR=3D > #WERROR=3D > MALLOC_PRODUCTION=3D > # > WITH_DEBUG_FILES=3D > # > # > # For TO (so-called "cross") stages . . . > # So-called-cross via ${TO_TYPE}-xtoolchain-gcc/${TO_TYPE}-gcc. . . > # TOOLS_TO_TYPE based on ${TO_TYPE}-xtoolchain-gcc related binutils. . = . > # > CROSS_TOOLCHAIN=3D${TO_TYPE}-gcc > X_COMPILER_TYPE=3Dgcc > CROSS_BINUTILS_PREFIX=3D/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/ > .if ${.MAKE.LEVEL} =3D=3D 0 > = XCC=3D/usr/local/bin/${TOOLS_TO_TYPE}-portbld-freebsd${VERSION_CONTEXT}-gc= c > = XCXX=3D/usr/local/bin/${TOOLS_TO_TYPE}-portbld-freebsd${VERSION_CONTEXT}-g= ++ > = XCPP=3D/usr/local/bin/${TOOLS_TO_TYPE}-portbld-freebsd${VERSION_CONTEXT}-c= pp > .export XCC > .export XCXX > .export XCPP > XAS=3D/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/as > XAR=3D/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/ar > XLD=3D/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/ld > XNM=3D/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/nm > XOBJCOPY=3D/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/objcopy > XOBJDUMP=3D/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/objdump > XRANLIB=3D/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/ranlib > XSIZE=3D/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/size > #NO-SUCH: XSTRINGS=3D/usr/local/${TOOLS_TO_TYPE}-freebsd/bin/strings > XSTRINGS=3D/usr/local/bin/${TOOLS_TO_TYPE}-freebsd-strings > .export XAS > .export XAR > .export XLD > .export XNM > .export XOBJCOPY > .export XOBJDUMP > .export XRANLIB > .export XSIZE > .export XSTRINGS > .endif > # > # > # =46rom based on clang (via system). . . > # > .if ${.MAKE.LEVEL} =3D=3D 0 > CC=3D/usr/bin/clang > CXX=3D/usr/bin/clang++ > CPP=3D/usr/bin/clang-cpp > .export CC > .export CXX > .export CPP > .endif > # more /etc/make.conf > WANT_QT_VERBOSE_CONFIGURE=3D1 > # > DEFAULT_VERSIONS+=3Dperl5=3D5.22 > WRKDIRPREFIX=3D/usr/obj/portswork > WITH_DEBUG=3D > WITH_DEBUG_FILES=3D > MALLOC_PRODUCTION=3D > # svnlite status /usr/src > M = /usr/src/contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp > M /usr/src/lib/csu/powerpc64/Makefile > ? /usr/src/sys/amd64/conf/GENERIC-NODBG > ? /usr/src/sys/amd64/include/include > ? /usr/src/sys/arm/conf/RPI2-NODBG > ? /usr/src/sys/arm/include/include > M /usr/src/sys/boot/ofw/Makefile.inc > M /usr/src/sys/boot/powerpc/Makefile.inc > M /usr/src/sys/boot/powerpc/kboot/Makefile > M /usr/src/sys/boot/uboot/Makefile.inc > M /usr/src/sys/conf/Makefile.powerpc > M /usr/src/sys/conf/kern.mk > M /usr/src/sys/conf/kmod.mk > ? /usr/src/sys/powerpc/conf/GENERIC64-NODBG > ? /usr/src/sys/powerpc/conf/GENERIC64vtsc > ? /usr/src/sys/powerpc/conf/GENERIC64vtsc-NODEBUG > ? /usr/src/sys/powerpc/conf/GENERICvtsc > ? /usr/src/sys/powerpc/conf/GENERICvtsc-NODEBUG > ? /usr/src/sys/powerpc/include/include > M /usr/src/sys/powerpc/ofw/ofw_machdep.c > M /usr/src/sys/powerpc/powerpc/exec_machdep.c > ? /usr/src/sys/x86/include/include The include/include's were not added by me. They just point back to the = parent include. The rest of the powerpc64 stuff I've been using for a long time, some of = it is for booting PowerMac G5's reliably. Other powerpc64/powerpc parts = are mostly tied to my clang 3.8.0 investigations for targeting powerpc64 = and powerpc. =3D=3D=3D Mark Millard markmi at dsl-only.net _______________________________________________ freebsd-ppc@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ppc To unsubscribe, send any mail to "freebsd-ppc-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Tue Sep 27 10:25:43 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44771BEBAFF for ; Tue, 27 Sep 2016 10:25:43 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from cu1176c.smtpx.saremail.com (cu1176c.smtpx.saremail.com [195.16.148.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 08CCA9C4 for ; Tue, 27 Sep 2016 10:25:42 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from [172.16.8.36] (izaro.sarenet.es [192.148.167.11]) by proxypop02.sare.net (Postfix) with ESMTPSA id 1F7D89DD4E7 for ; Tue, 27 Sep 2016 12:16:02 +0200 (CEST) From: Borja Marcos Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.0 \(3226\)) Subject: Status of PCIe Hotplug? Message-Id: Date: Tue, 27 Sep 2016 12:16:01 +0200 To: FreeBSD-STABLE Mailing List X-Mailer: Apple Mail (2.3226) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Sep 2016 10:25:43 -0000 Hi, I have noticed that the GENERIC kernel in 11-STABLE includes the PCI_HP = option, and the hotplug bits seem to be present in the kernel, but I don=E2=80=99t see = any userland support for it. Is it somewhat complete and in that case am I missing something? Thanks! Borja. From owner-freebsd-stable@freebsd.org Tue Sep 27 13:54:01 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7198DBEC197 for ; Tue, 27 Sep 2016 13:54:01 +0000 (UTC) (envelope-from me@janh.de) Received: from mout.kundenserver.de (mout.kundenserver.de [217.72.192.73]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.kundenserver.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C4C67A2C for ; Tue, 27 Sep 2016 13:53:59 +0000 (UTC) (envelope-from me@janh.de) Received: from [192.168.178.49] ([84.180.224.142]) by mrelayeu.kundenserver.de (mreue104) with ESMTPSA (Nemesis) id 0MabU1-1bVHZ63FH7-00K4da; Tue, 27 Sep 2016 15:48:37 +0200 Subject: Re: Status of PCIe Hotplug? References: To: Borja Marcos , FreeBSD-STABLE Mailing List From: Jan Henrik Sylvester Message-ID: <791ee95b-ab4e-f07e-6b8d-0e0b6c49ceb0@janh.de> Date: Tue, 27 Sep 2016 15:48:27 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Provags-ID: V03:K0:/Obabp0KyeAJJdTAy7OvhjQVyP/n+8JvEsz7mOG2uDalJViTq7H +iSIHVEvIF+8aoQUSTlfuKXqzJbs4KtEWMnfeK63zuUFGGz+eMTMooALJbLXDXCzKG/0YEr RBSQb8Zql1DhUjAflGoQgCwN/urbEYcgSNENtPpmij6zGiPdrbaKqLLahk32uoBFjwK3hj7 EttPIdHleGHIaNwk9Zd0g== X-UI-Out-Filterresults: notjunk:1;V01:K0:0UZ5b8bksg4=:d/PINUtGiv+IIWCL+QdPHd j7CqufCtJNleu7Mx4XAkQ1DzJ5mvrBVQGJjBYzM0rP9PpqfzPDOivXTDg/OCzOEQLl8p42edB /qvFfRvL24nGIRrJ0DFDfvuICMnjfntKYRNZmkvS1PebVuXJ0x/Zc9B0Nsdh+HCBIgUiXrV49 9YNs2PEGvQfi19cP8HM+7M3fXKyEvqSzdTAHigtupqhiTmYx4dLKGdiuuqi4qbhIZUd5nNcSG HulJjIZsy0zWkfNrMisrUQ18f+ujim4eCqVz+DaqvKV8+tfCNySjdq3EGYdtFjFwEh4jNLx0G 3e1oAD/ZUPte2S3kstjZifgQSedFZQgFwfkGAJoXs/NVUIxtnCTMigeqST2fiYlP9CJ/JoQBI hehnnd12/IVnMUIrxodrHEacUxryc27Lp2oSFTrPjtW9yGv0Pfk7NgFhsmRaATTMIaDolWZcN ZPXvVeJhoBE4fG9inQO5FrZTMqoDaPrH0Gvou+BVliAskp6fqqrDXk0LHvsMPHXnKuMO86Au/ +BIVFip9oADVnOfh8h/qVVE5jgP0ItmpVW/+ZMa/E2LylTzX8p3qkkxquDxNkt1mNAC8LInK1 T7wbVdhn3Ch3A7HRcRTGq+o9sx/ot9xKZNezSm8QV3hAq8zMH7MiVYjKn7HY8gwlA7dpIUmTz rTGuqdy9/YwVtWPpbaaAJ1OYnTEDwuRgtjcnzAUWkwthFhYGgjvKmXOGF7hBfOatFAkFHZjm/ rwokLkDejFnxmjI/ X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Sep 2016 13:54:01 -0000 On 09/27/2016 12:16, Borja Marcos wrote: > I have noticed that the GENERIC kernel in 11-STABLE includes the PCI_HP option, and the > hotplug bits seem to be present in the kernel, but I don’t see any userland support for it. > > Is it somewhat complete and in that case am I missing something? I do not know kind of userland support you mean. I just tried: Plugging in my USB 3.0 ExpressCard while 11.0 is running, the controller was detected and I was able to use USB devices with it. Great. Unplugging the ExpressCard led to a panic two out of two trials, no matter if I had used it for USB devices before or not. So we have hotplug, but no hotunplug... With 10.3, it was the other way around. Of course, there was no hotplug, but when I accidentally pulled the ExpressCard, the USB controller would just disappear and never reappear without a panic. I do not know what I like better, though. Cheers, Jan Henrik From owner-freebsd-stable@freebsd.org Tue Sep 27 14:06:18 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 92946BEC59B for ; Tue, 27 Sep 2016 14:06:18 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from cu01176a.smtpx.saremail.com (cu01176a.smtpx.saremail.com [195.16.150.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 58CEFE2 for ; Tue, 27 Sep 2016 14:06:17 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from [172.16.8.36] (izaro.sarenet.es [192.148.167.11]) by proxypop03.sare.net (Postfix) with ESMTPSA id 886CF9DC9C5; Tue, 27 Sep 2016 15:57:28 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.0 \(3226\)) Subject: Re: Status of PCIe Hotplug? From: Borja Marcos In-Reply-To: <791ee95b-ab4e-f07e-6b8d-0e0b6c49ceb0@janh.de> Date: Tue, 27 Sep 2016 15:57:27 +0200 Cc: FreeBSD-STABLE Mailing List Content-Transfer-Encoding: quoted-printable Message-Id: References: <791ee95b-ab4e-f07e-6b8d-0e0b6c49ceb0@janh.de> To: Jan Henrik Sylvester X-Mailer: Apple Mail (2.3226) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Sep 2016 14:06:18 -0000 > On 27 Sep 2016, at 15:48, Jan Henrik Sylvester wrote: >=20 > On 09/27/2016 12:16, Borja Marcos wrote: >> I have noticed that the GENERIC kernel in 11-STABLE includes the = PCI_HP option, and the >> hotplug bits seem to be present in the kernel, but I don=E2=80=99t = see any userland support for it. >>=20 >> Is it somewhat complete and in that case am I missing something? >=20 > I do not know kind of userland support you mean. I just tried: >=20 > Plugging in my USB 3.0 ExpressCard while 11.0 is running, the = controller > was detected and I was able to use USB devices with it. Great. Thanks :) I was hoping (and I assume it=E2=80=99s the ultimate goal of the = project) to be able to hot plug PCIe devices such as NVMe drives. On Solaris you can replace them provided you power them off previously = (there=E2=80=99s a command for that, =E2=80=9Chotplug=E2=80=9D). On FreeBSD I=E2=80=99ve tried using devctl but powering off, disabling a = device and enabling it again has led to a panic. Interestingly, I disabled nvme0 using devctl and "nvmecontrol devlist" = didn=E2=80=99t find any nvme controllers despite having 10 controllers and 10 drives. However, = the ZFS pool of 10=20 NVMe drives was working happily. Degraded of course, with one NVMe = missing. Borja. From owner-freebsd-stable@freebsd.org Tue Sep 27 15:51:20 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 908C5BEC31A for ; Tue, 27 Sep 2016 15:51:20 +0000 (UTC) (envelope-from vangyzen@FreeBSD.org) Received: from smtp.vangyzen.net (hotblack.vangyzen.net [IPv6:2607:fc50:1000:7400:216:3eff:fe72:314f]) by mx1.freebsd.org (Postfix) with ESMTP id 5FB95A4B for ; Tue, 27 Sep 2016 15:51:20 +0000 (UTC) (envelope-from vangyzen@FreeBSD.org) Received: from sweettea.beer.town (unknown [76.164.8.130]) by smtp.vangyzen.net (Postfix) with ESMTPSA id 530D556488; Tue, 27 Sep 2016 10:51:19 -0500 (CDT) Subject: Re: Status of PCIe Hotplug? To: Borja Marcos , Jan Henrik Sylvester References: <791ee95b-ab4e-f07e-6b8d-0e0b6c49ceb0@janh.de> Cc: FreeBSD-STABLE Mailing List From: Eric van Gyzen Message-ID: <58f84457-d775-3023-5727-a326bc8df722@FreeBSD.org> Date: Tue, 27 Sep 2016 10:51:15 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Sep 2016 15:51:20 -0000 On 09/27/2016 08:57, Borja Marcos wrote: > >> On 27 Sep 2016, at 15:48, Jan Henrik Sylvester wrote: >> >> On 09/27/2016 12:16, Borja Marcos wrote: >>> I have noticed that the GENERIC kernel in 11-STABLE includes the >>> PCI_HP option, and the hotplug bits seem to be present in the >>> kernel, but I don’t see any userland support for it. >>> >>> Is it somewhat complete and in that case am I missing something? >> >> I do not know kind of userland support you mean. I just tried: >> >> Plugging in my USB 3.0 ExpressCard while 11.0 is running, the >> controller was detected and I was able to use USB devices with it. >> Great. > > Thanks :) > > I was hoping (and I assume it’s the ultimate goal of the project) to > be able to hot plug PCIe devices such as NVMe drives. > > On Solaris you can replace them provided you power them off > previously (there’s a command for that, “hotplugâ€). > > On FreeBSD I’ve tried using devctl but powering off, disabling a > device and enabling it again has led to a panic. > > Interestingly, I disabled nvme0 using devctl and "nvmecontrol > devlist" didn’t find any nvme controllers despite having 10 > controllers and 10 drives. However, the ZFS pool of 10 NVMe drives > was working happily. Degraded of course, with one NVMe missing. To my knowledge, all the necessary PCIe-layer code is present. However, that's just one layer: Many drivers will likely need changes in order to cope with surprise removal of their devices. For that reason, HotPlug needs a lot of testing on a variety of platforms. The FreeBSD developer base is much smaller than its user base, of course, so the variety of our testing is rather limited. You can help immensely by giving us detailed bug reports, either on a mailing list or in Bugzilla. For a panic, the panic messages and stack trace of the current thread will be very helpful. Complete crashinfo(8) output would be great. The most relevant userland tool is devctl, followed closely by devinfo and pciconf. In the case of Jan's USB 3.0 ExpressCard, it's possible that one or all of the USB controller drivers (xhci, ehci, uhci) didn't cope with the surprise removal of the controller. Before removing the card, try detaching the driver(s) with "devctl detach xhciN". There might be more than one device. Use "pciconf -lc" to find the HotPlug-capable pcib devices (bridges). Use devinfo to find which one is your ExpressCard slot and find all the devices attached to it. Then use devctl to detach the devices. There could be a tree of devices; in that case, you can usually start at the level immediately under pcibN; you don't need to detach every device from the bottom up. Once all the devices are detached, you should be able to remove the card safely. Eric From owner-freebsd-stable@freebsd.org Wed Sep 28 07:59:24 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DD0C4C0073F; Wed, 28 Sep 2016 07:59:24 +0000 (UTC) (envelope-from me@janh.de) Received: from mout.kundenserver.de (mout.kundenserver.de [217.72.192.74]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.kundenserver.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F3F6F8; Wed, 28 Sep 2016 07:59:23 +0000 (UTC) (envelope-from me@janh.de) Received: from [192.168.178.49] ([84.175.67.119]) by mrelayeu.kundenserver.de (mreue103) with ESMTPSA (Nemesis) id 0MTNxP-1bOwBk3wlr-00SReP; Wed, 28 Sep 2016 09:59:20 +0200 Subject: Re: Status of PCIe Hotplug? To: Eric van Gyzen References: <791ee95b-ab4e-f07e-6b8d-0e0b6c49ceb0@janh.de> <58f84457-d775-3023-5727-a326bc8df722@FreeBSD.org> Cc: Borja Marcos , FreeBSD-STABLE Mailing List , FreeBSD-USB Mailing List From: Jan Henrik Sylvester Message-ID: Date: Wed, 28 Sep 2016 09:59:10 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <58f84457-d775-3023-5727-a326bc8df722@FreeBSD.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:RcxwPsL8WH6/LQ7NzKz98tDMSZj4DNLKdKFaalReyls7HIJUnCU WnlKqOs7KdS5F1c6Egz7xtuVYR0vhn9t6ThMbdT1BTZS7b6r0QecjsxBd+BzKz2yzq+x8ws MYVKK6+VV+JIRMMQFG6EVimgMXEV8ls1x7gjVR8Cdz8WltThWeWatj3lMyFUXlKwamJUpZY 6yFuX6/9VMNKul22g6kOg== X-UI-Out-Filterresults: notjunk:1;V01:K0:gWDtL+mLyK4=:uTRzq/P5oY1ngo2aeAag95 aa4ElBueqlLbs2+fCZNJS2PWJc3fFb2YCfOs+Vum58MpjaXP/5wk2IxTwHRff1odGzjcsCFuf B7y2BVlEFuJfXY50prDokd0GABcB0BwBbIp+KZBsDDUCWZM8opixWPhL5Ec/8BNJ0+riOimVh lhaMIzBc4NLY7XFbXwBWZSEli//3RBlQAjjUqRYHtlqmeknQXShTTft3o7j8CIsT1eMCoKWnZ iY8n0jIwnUkzPKJNhhyeoKR6wIaV7BVuFY0bPZuD62kcqBrVGbUCcKy/KJAH+Eherm7YiDX2L 2p2vI92qr5Bq4KOqmuIxxT0t8ZHLLkQFPue99uTeLbBxlBPcpoWDFQAnqsWmGSKqUJezL3lH+ QtEV1NSsgBnJ1MTGxWUihyyKx6qfbUfgMU3naIFhNPBe+W7R/K1WLodLiUUeH48GgpcBJYDkQ 0ygvt+DGWcWmLFzuAI16HYut122gEKRKDPKHdEA9Et1MHeT7p222WamtlmxAo1FzxtFf5T7Nk 4yokG3J/X5b5sv6ct8YGuvZwFi6IgLw7v41zx68ZBUdNCJTxGbczkteXtHhhK1oJP7jdaQKUm e3oQXgVELV9+d7tPCmZTIcJJ/Nh7ysGy4oVq9XVSd6eQWqDyyM1NFFFRKN1DSwiNL8+ORc2Ca 7XTMXSH5vne4NUXYsoAFzdPsnTFL2T5oy1OX/j51D+GCuahDXzD6zGFVPVyZ7t2RjMdJDCIqg fWa0F8WdWGrWisSB X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Sep 2016 07:59:25 -0000 On 09/27/2016 17:51, Eric van Gyzen wrote: > In the case of Jan's USB 3.0 ExpressCard, it's possible that one or all > of the USB controller drivers (xhci, ehci, uhci) didn't cope with the > surprise removal of the controller. Before removing the card, try > detaching the driver(s) with "devctl detach xhciN". There might be more > than one device. Use "pciconf -lc" to find the HotPlug-capable pcib > devices (bridges). Use devinfo to find which one is your ExpressCard > slot and find all the devices attached to it. Then use devctl to detach > the devices. There could be a tree of devices; in that case, you can > usually start at the level immediately under pcibN; you don't need to > detach every device from the bottom up. Once all the devices are > detached, you should be able to remove the card safely. Doing "devctl detach xhci0" before the removal of the USB 3.0 ExpressCard, there is no panic, the device gets deattached properly, and I can reconnect it later. Anyhow, because the mechanism holding the ExpressCard is not completely reliable, on the third time inserting the card, it did not hold and I got a panic, because it was immediately ejected without devctl detach. Due to the card not holding firmly, I often pulled it together with the usb device on 10.3-RELEASE and never got a panic. I guess it is a regression in the usb driver dealing with sudden loss of the device. The panic message is below, I guess I should take this discussion to freebsd-usb@, CCed. Thanks, Jan Henrik Fatal trap 9: general protection fault while in kernel mode cpuid = 1; acpic id = 01 instruction pointer = 0x20:0xffffffff80b1549c stack pointer = 0x28:0xfffffe022f62ca00 frame pointer = 0x28:0xfffffe022f62ca70 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 14 (usbus1) trap number = 9 panic: general protection fault cpuid = 1 KDB: stack backtrace: #0 0xffffffff80b24077 at kdb_backtrace+0x67 #1 0xffffffff80ad93e2 at vpanic+0x182 #2 0xffffffff80ad9253 at panic+0x43 #3 0xffffffff80fa0d31 at trap_fatal+0x351 #4 0xffffffff80fa09c8 at trap+0x768 #5 0xffffffff80f84141 at calltrap+0x8 #6 0xffffffff808f2f63 at usb_detach_device+0xf3 #7 0xffffffff808f1d5b at usb_unconfigure+0x2b #8 0xffffffff808f5623 at usb_free_device+0x103 #9 0xffffffff808f58b1 at usb_bus_detach+0x161 #10 0xffffffff80903e95 at usb_process+0x125 #11 0xffffffff80a90055 at fork_exit+0x85 #12 0xffffffff80f8467e at fork_trampoline+0xe Uptime: 18m27s Automatic reboot in 15 seconds - press a key on the console to abort From owner-freebsd-stable@freebsd.org Wed Sep 28 08:27:24 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D2BB4BEC4DC; Wed, 28 Sep 2016 08:27:24 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9B2661BDF; Wed, 28 Sep 2016 08:27:24 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id EA86F1FE023; Wed, 28 Sep 2016 10:27:22 +0200 (CEST) Subject: Re: Status of PCIe Hotplug? To: Jan Henrik Sylvester , Eric van Gyzen References: <791ee95b-ab4e-f07e-6b8d-0e0b6c49ceb0@janh.de> <58f84457-d775-3023-5727-a326bc8df722@FreeBSD.org> Cc: FreeBSD Stable , FreeBSD-USB Mailing List , Borja Marcos From: Hans Petter Selasky Message-ID: <05657098-a8dd-9d0d-ea75-4c3fdc582795@selasky.org> Date: Wed, 28 Sep 2016 10:32:15 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Sep 2016 08:27:24 -0000 Hi, Can you compile your kernel with debugging enabled so that we get the sourcelines in the dump below. Also please include some parts of the preceeding dmesg. Do you have USB compiled into the kernel or loaded as a module? What does kldstat output after you plug the card? What is the dmesg leading up to this? The bug looks like some .ko was unloaded too early, due to refcounts dropping incorrectly. --HPS > > > Fatal trap 9: general protection fault while in kernel mode > cpuid = 1; acpic id = 01 > instruction pointer = 0x20:0xffffffff80b1549c > stack pointer = 0x28:0xfffffe022f62ca00 > frame pointer = 0x28:0xfffffe022f62ca70 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 14 (usbus1) > trap number = 9 > panic: general protection fault > cpuid = 1 > KDB: stack backtrace: > #0 0xffffffff80b24077 at kdb_backtrace+0x67 > #1 0xffffffff80ad93e2 at vpanic+0x182 > #2 0xffffffff80ad9253 at panic+0x43 > #3 0xffffffff80fa0d31 at trap_fatal+0x351 > #4 0xffffffff80fa09c8 at trap+0x768 > #5 0xffffffff80f84141 at calltrap+0x8 > #6 0xffffffff808f2f63 at usb_detach_device+0xf3 > #7 0xffffffff808f1d5b at usb_unconfigure+0x2b > #8 0xffffffff808f5623 at usb_free_device+0x103 > #9 0xffffffff808f58b1 at usb_bus_detach+0x161 > #10 0xffffffff80903e95 at usb_process+0x125 > #11 0xffffffff80a90055 at fork_exit+0x85 > #12 0xffffffff80f8467e at fork_trampoline+0xe > Uptime: 18m27s > Automatic reboot in 15 seconds - press a key on the console to abort From owner-freebsd-stable@freebsd.org Wed Sep 28 09:41:39 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0F1AFC00F2B for ; Wed, 28 Sep 2016 09:41:39 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from cu1176c.smtpx.saremail.com (cu1176c.smtpx.saremail.com [195.16.148.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C241CF83; Wed, 28 Sep 2016 09:41:38 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from [172.16.8.36] (izaro.sarenet.es [192.148.167.11]) by proxypop02.sare.net (Postfix) with ESMTPSA id A17339DC943; Wed, 28 Sep 2016 11:41:35 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.0 \(3226\)) Subject: Re: Status of PCIe Hotplug? From: Borja Marcos In-Reply-To: <58f84457-d775-3023-5727-a326bc8df722@FreeBSD.org> Date: Wed, 28 Sep 2016 11:41:34 +0200 Cc: Jan Henrik Sylvester , FreeBSD-STABLE Mailing List Content-Transfer-Encoding: quoted-printable Message-Id: References: <791ee95b-ab4e-f07e-6b8d-0e0b6c49ceb0@janh.de> <58f84457-d775-3023-5727-a326bc8df722@FreeBSD.org> To: Eric van Gyzen X-Mailer: Apple Mail (2.3226) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Sep 2016 09:41:39 -0000 > On 27 Sep 2016, at 17:51, Eric van Gyzen wrote: >=20 >=20 > To my knowledge, all the necessary PCIe-layer code is present. = However, > that's just one layer: Many drivers will likely need changes in order > to cope with surprise removal of their devices. Thank you very much, that=E2=80=99s what I needed to know :) I saw that = the bits were indeed present, but I was wondering wether I should expect it to work or = not. > For that reason, HotPlug needs a lot of testing on a variety of > platforms. The FreeBSD developer base is much smaller than its user > base, of course, so the variety of our testing is rather limited. You > can help immensely by giving us detailed bug reports, either on a > mailing list or in Bugzilla. For a panic, the panic messages and = stack > trace of the current thread will be very helpful. Complete = crashinfo(8) > output would be great. Of course. Unfortunately, due to poor timing and a DOA server last = month, this server is in a countdown to get into production tomorrow running = Solaris, but I=E2=80=99ll try to get whatever I can today. Thanks! Borja. From owner-freebsd-stable@freebsd.org Wed Sep 28 10:06:56 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 00CC5BE8A16 for ; Wed, 28 Sep 2016 10:06:56 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-wm0-x22c.google.com (mail-wm0-x22c.google.com [IPv6:2a00:1450:400c:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6345A14B for ; Wed, 28 Sep 2016 10:06:55 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-wm0-x22c.google.com with SMTP id b130so62283722wmc.0 for ; Wed, 28 Sep 2016 03:06:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=gDRfkvugcnMbaDg7f8KEuFMyMDy6z8cg/3BszSgIgww=; b=S1IPbpvVcKlxbHwhsVPqfZJL61aq8tjUcCnGf8gEoWTGcmqaPAjJlf0ob8gkA2XTPS p58M6GvCBF1Z/PPIEg7AIsdhqi3Qdt8MsZG2ZE3IreofTFDQZ5/FaJuO7vsRndiRfHpO NGZ04wBMBTOd/3FTmRq6E1CCVfnA/tqSZ59NUzkrQX0IYOaoqq2+jm5s5RUk3anA6u6p r4fbjxOLc7YLDvf9lVI8bjDLov5k31ONQEszYNyas5N2W5+nV8oNuZtjd5g2brQj5agF VM1pNaccqikERoAcZYrekYWyROj0nYCuYDliA9tSjZaZrjfZwa84Y01B3rHG58tV0zZm hOUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=gDRfkvugcnMbaDg7f8KEuFMyMDy6z8cg/3BszSgIgww=; b=mQUFhqfn7Yu1HdfClqRc0DGOaxWsHh7f3AqSXW/Ocr/mv1vj/R+PTQsHQDj0hWotq3 tyxzxffZtjjKV+H+Ev3bXdFZ7HWMlVx4dOjlr+L6/GN5yYp2MHpb7PBzD7ezZm3ieD/D +FArY30fQRDz3k2BVF2g6onweW/rG9vRreMmhhP24g8iOHU235pbEkNbqz16HqQQtl5Z 3eVL0ChvCSyYfZK3Yl1B6v0meYfjLTgvn9tbwqs160MkWYw+W/votRkp1sbuA6NCWrrP VKSHB1Vc7VgRo9suB8asITxMHmKxzeURsql5L8vyRGw5OEP4u5JllYlfDcJuzObZSJrR /IJA== X-Gm-Message-State: AA6/9Rk3X2pooIDbfuQeLZQW+ZdMeu/lh5FX1gbkVtk43BJQl7KiHq4F0wjUuJMEh2a0ow== X-Received: by 10.28.155.203 with SMTP id d194mr7475466wme.72.1475057213643; Wed, 28 Sep 2016 03:06:53 -0700 (PDT) Received: from [10.100.64.19] ([217.30.88.44]) by smtp.gmail.com with ESMTPSA id 142sm7931284wmh.12.2016.09.28.03.06.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 28 Sep 2016 03:06:52 -0700 (PDT) Subject: Re: 11.0 stuck on high network load To: Slawa Olhovchenkov References: <78cbcdc9-f565-1046-c157-2ddd8fcccc62@freebsd.org> <20160919204328.GN2840@zxy.spb.ru> <8ba75d6e-4f01-895e-0aed-53c6c6692cb9@freebsd.org> <20160920202633.GQ2840@zxy.spb.ru> <20160921195155.GW2840@zxy.spb.ru> <20160923200143.GG2840@zxy.spb.ru> <20160925124626.GI2840@zxy.spb.ru> <20160926172159.GA54003@zxy.spb.ru> Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara From: Julien Charbon Message-ID: <62453d9c-b1e4-1129-70ff-654dacea37f9@gmail.com> Date: Wed, 28 Sep 2016 12:06:47 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <20160926172159.GA54003@zxy.spb.ru> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="Grg9p8WN0dsPRVQnwhLgfPmlD08BcOFiP" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Sep 2016 10:06:56 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --Grg9p8WN0dsPRVQnwhLgfPmlD08BcOFiP Content-Type: multipart/mixed; boundary="eJoODbLvCaU8HsWinURaWqn1UNV0S7cOH"; protected-headers="v1" From: Julien Charbon To: Slawa Olhovchenkov Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Message-ID: <62453d9c-b1e4-1129-70ff-654dacea37f9@gmail.com> Subject: Re: 11.0 stuck on high network load References: <78cbcdc9-f565-1046-c157-2ddd8fcccc62@freebsd.org> <20160919204328.GN2840@zxy.spb.ru> <8ba75d6e-4f01-895e-0aed-53c6c6692cb9@freebsd.org> <20160920202633.GQ2840@zxy.spb.ru> <20160921195155.GW2840@zxy.spb.ru> <20160923200143.GG2840@zxy.spb.ru> <20160925124626.GI2840@zxy.spb.ru> <20160926172159.GA54003@zxy.spb.ru> In-Reply-To: <20160926172159.GA54003@zxy.spb.ru> --eJoODbLvCaU8HsWinURaWqn1UNV0S7cOH Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi Slawa, On 9/26/16 7:22 PM, Slawa Olhovchenkov wrote: > On Mon, Sep 26, 2016 at 11:33:12AM +0200, Julien Charbon wrote: >> On 9/25/16 2:46 PM, Slawa Olhovchenkov wrote: >>> On Fri, Sep 23, 2016 at 11:01:43PM +0300, Slawa Olhovchenkov wrote: >>>>> On 9/21/16 9:51 PM, Slawa Olhovchenkov wrote: >>>>>> On Wed, Sep 21, 2016 at 09:11:24AM +0200, Julien Charbon wrote: >>>>>>> You can also use Dtrace and lockstat (especially with the lockst= at -s >>>>>>> option): >>>>>>> >>>>>>> https://wiki.freebsd.org/DTrace/One-Liners#Kernel_Locks >>>>>>> https://www.freebsd.org/cgi/man.cgi?query=3Dlockstat&manpath=3DFr= eeBSD+11.0-RELEASE >>>>>>> >>>>>>> But I am less familiar with Dtrace/lockstat tools. >>>>>> >>>>>> I am still use old kernel and got lockdown again. >>>>>> Try using lockstat (I am save more output), interesting may be nex= t: >>>>>> >>>>>> R/W writer spin on writer: 190019 events in 1.070 seconds (177571 = events/sec) >>>>>> >>>>>> ------------------------------------------------------------------= ------------- >>>>>> Count indv cuml rcnt nsec Lock Caller = =20 >>>>>> 140839 74% 74% 0.00 24659 tcpinp tcp_tw_2msl_= scan+0xc6 =20 >>>>>> >>>>>> nsec ------ Time Distribution ------ count Stack = =20 >>>>>> 4096 | 913 tcp_twstart+0= xa3 =20 >>>>>> 8192 |@@@@@@@@@@@@ 58191 tcp_do_segmen= t+0x201f =20 >>>>>> 16384 |@@@@@@ 29594 tcp_input+0xe= 1c =20 >>>>>> 32768 |@@@@ 23447 ip_input+0x15= f =20 >>>>>> 65536 |@@@ 16197 =20 >>>>>> 131072 |@ 8674 =20 >>>>>> 262144 | 3358 =20 >>>>>> 524288 | 456 =20 >>>>>> 1048576 | 9 =20 >>>>>> ------------------------------------------------------------------= ------------- >>>>>> Count indv cuml rcnt nsec Lock Caller = =20 >>>>>> 49180 26% 100% 0.00 15929 tcpinp tcp_tw_2msl_s= can+0xc6 =20 >>>>>> >>>>>> nsec ------ Time Distribution ------ count Stack = =20 >>>>>> 4096 | 157 pfslowtimo+0x= 54 =20 >>>>>> 8192 |@@@@@@@@@@@@@@@ 24796 softclock_cal= l_cc+0x179=20 >>>>>> 16384 |@@@@@@ 11223 softclock+0x4= 4 =20 >>>>>> 32768 |@@@@ 7426 intr_event_ex= ecute_handlers+0x95 >>>>>> 65536 |@@ 3918 =20 >>>>>> 131072 | 1363 =20 >>>>>> 262144 | 278 =20 >>>>>> 524288 | 19 =20 >>>>>> ------------------------------------------------------------------= ------------- >>>>> >>>>> This is interesting, it seems that you have two call paths competi= ng >>>>> for INP locks here: >>>>> >>>>> - pfslowtimo()/tcp_tw_2msl_scan(reuse=3D0) and >>>>> >>>>> - tcp_input()/tcp_twstart()/tcp_tw_2msl_scan(reuse=3D1) >>>> >>>> My current hypothesis: >>>> >>>> nginx do write() (or may be close()?) to socket, kernel lock >>>> first inp in V_twq_2msl, happen callout for pfslowtimo() on the same= >>>> CPU core and tcp_tw_2msl_scan infinity locked on same inp. >>>> >>>> In this case you modification can't help, before next try we need so= me >>>> like yeld(). >>> >>> Or may be locks leaks. >>> Or both. >> >> You are totally right, pfslowtimo()/tcp_tw_2msl_scan(reuse=3D0) is >> infinitely blocked on INP_WLOCK() by "something" (that could be relate= d >> to write()). >> >> As I reached my limit of debugging without WITNESS, could you share >> your /etc/sysctl.conf, /boot/loader.conf files? And any specific >> configuration you have (like having a Nginx workers affinity, Nginx >> special options, etc.). Like that I can try to reproduce it on releng= /11.0. >=20 > Some more traces from ddb: >=20 > Tracing command intr pid 12 tid 100103 td 0xfffff8012508ea00 > sched_switch() at 0xffffffff804c956d =3D sched_switch+0x6ad/frame 0xfff= ffe2020ea8330 > mi_switch() at 0xffffffff804a8d92 =3D mi_switch+0xd2/frame 0xfffffe2020= ea8360 > turnstile_wait() at 0xffffffff804ef177 =3D turnstile_wait+0x2a7/frame 0= xfffffe2020ea83a0 > __mtx_lock_sleep() at 0xffffffff80484d9d =3D __mtx_lock_sleep+0x13d/fra= me 0xfffffe2020ea8420 > sodealloc() at 0xffffffff8051b992 =3D sodealloc+0x32/frame 0xfffffe2020= ea8460 > tcp_close() at 0xffffffff80618150 =3D tcp_close+0xd0/frame 0xfffffe2020= ea8490 > tcp_do_segment() at 0xffffffff80610226 =3D tcp_do_segment+0x1666/frame = 0xfffffe2020ea8590 > tcp_input() at 0xffffffff8060e17c =3D tcp_input+0xe1c/frame 0xfffffe202= 0ea86e0 > ip_input() at 0xffffffff805a087f =3D ip_input+0x15f/frame 0xfffffe2020e= a8740 > netisr_dispatch_src() at 0xffffffff80583db5 =3D netisr_dispatch_src+0xa= 5/frame 0xfffffe2020ea87a0 > ether_demux() at 0xffffffff80575b3a =3D ether_demux+0x12a/frame 0xfffff= e2020ea87d0 > ether_nh_input() at 0xffffffff80576792 =3D ether_nh_input+0x322/frame 0= xfffffe2020ea8830 > netisr_dispatch_src() at 0xffffffff80583db5 =3D netisr_dispatch_src+0xa= 5/frame 0xfffffe2020ea8890 > ether_input() at 0xffffffff80575db6 =3D ether_input+0x26/frame 0xfffffe= 2020ea88b0 > ixgbe_rxeof() at 0xffffffff813df36b =3D ixgbe_rxeof+0x7ab/frame 0xfffff= e2020ea8990 > ixgbe_msix_que() at 0xffffffff813da57c =3D ixgbe_msix_que+0x8c/frame 0x= fffffe2020ea89e0 > intr_event_execute_handlers() at 0xffffffff8046c605 =3D intr_event_exec= ute_handlers+0x95/frame 0xfffffe2020ea8a20 > ithread_loop() at 0xffffffff8046cc26 =3D ithread_loop+0xa6/frame 0xffff= fe2020ea8a70 > fork_exit() at 0xffffffff8046a211 =3D fork_exit+0x71/frame 0xfffffe2020= ea8ab0 > fork_trampoline() at 0xffffffff806cb50e =3D fork_trampoline+0xe/frame 0= xfffffe2020ea8ab0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >=20 > Tracing command intr pid 12 tid 100105 td 0xfffff8012508e000 > sched_switch() at 0xffffffff804c956d =3D sched_switch+0x6ad/frame 0xfff= ffe2020eb2330 > mi_switch() at 0xffffffff804a8d92 =3D mi_switch+0xd2/frame 0xfffffe2020= eb2360 > turnstile_wait() at 0xffffffff804ef177 =3D turnstile_wait+0x2a7/frame 0= xfffffe2020eb23a0 > __mtx_lock_sleep() at 0xffffffff80484d9d =3D __mtx_lock_sleep+0x13d/fra= me 0xfffffe2020eb2420 > sodealloc() at 0xffffffff8051b992 =3D sodealloc+0x32/frame 0xfffffe2020= eb2460 > tcp_close() at 0xffffffff80618150 =3D tcp_close+0xd0/frame 0xfffffe2020= eb2490 > tcp_do_segment() at 0xffffffff80610226 =3D tcp_do_segment+0x1666/frame = 0xfffffe2020eb2590 > tcp_input() at 0xffffffff8060e17c =3D tcp_input+0xe1c/frame 0xfffffe202= 0eb26e0 > ip_input() at 0xffffffff805a087f =3D ip_input+0x15f/frame 0xfffffe2020e= b2740 > netisr_dispatch_src() at 0xffffffff80583db5 =3D netisr_dispatch_src+0xa= 5/frame 0xfffffe2020eb27a0 > ether_demux() at 0xffffffff80575b3a =3D ether_demux+0x12a/frame 0xfffff= e2020eb27d0 > ether_nh_input() at 0xffffffff80576792 =3D ether_nh_input+0x322/frame 0= xfffffe2020eb2830 > netisr_dispatch_src() at 0xffffffff80583db5 =3D netisr_dispatch_src+0xa= 5/frame 0xfffffe2020eb2890 > ether_input() at 0xffffffff80575db6 =3D ether_input+0x26/frame 0xfffffe= 2020eb28b0 > ixgbe_rxeof() at 0xffffffff813df36b =3D ixgbe_rxeof+0x7ab/frame 0xfffff= e2020eb2990 > ixgbe_msix_que() at 0xffffffff813da57c =3D ixgbe_msix_que+0x8c/frame 0x= fffffe2020eb29e0 > intr_event_execute_handlers() at 0xffffffff8046c605 =3D intr_event_exec= ute_handlers+0x95/frame 0xfffffe2020eb2a20 > ithread_loop() at 0xffffffff8046cc26 =3D ithread_loop+0xa6/frame 0xffff= fe2020eb2a70 > fork_exit() at 0xffffffff8046a211 =3D fork_exit+0x71/frame 0xfffffe2020= eb2ab0 > fork_trampoline() at 0xffffffff806cb50e =3D fork_trampoline+0xe/frame 0= xfffffe2020eb2ab0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >=20 > Tracing command intr pid 12 tid 100107 td 0xfffff8012508d500 > sched_switch() at 0xffffffff804c956d =3D sched_switch+0x6ad/frame 0xfff= ffe2020ebc2b0 > mi_switch() at 0xffffffff804a8d92 =3D mi_switch+0xd2/frame 0xfffffe2020= ebc2e0 > turnstile_wait() at 0xffffffff804ef177 =3D turnstile_wait+0x2a7/frame 0= xfffffe2020ebc320 > __mtx_lock_sleep() at 0xffffffff80484d9d =3D __mtx_lock_sleep+0x13d/fra= me 0xfffffe2020ebc3a0 > soalloc() at 0xffffffff8051b914 =3D soalloc+0x1b4/frame 0xfffffe2020ebc= 3f0 > sonewconn() at 0xffffffff8051bb9f =3D sonewconn+0xbf/frame 0xfffffe2020= ebc430 > syncache_expand() at 0xffffffff8061b85b =3D syncache_expand+0x78b/frame= 0xfffffe2020ebc590 > tcp_input() at 0xffffffff8060e10e =3D tcp_input+0xdae/frame 0xfffffe202= 0ebc6e0 > ip_input() at 0xffffffff805a087f =3D ip_input+0x15f/frame 0xfffffe2020e= bc740 > netisr_dispatch_src() at 0xffffffff80583db5 =3D netisr_dispatch_src+0xa= 5/frame 0xfffffe2020ebc7a0 > ether_demux() at 0xffffffff80575b3a =3D ether_demux+0x12a/frame 0xfffff= e2020ebc7d0 > ether_nh_input() at 0xffffffff80576792 =3D ether_nh_input+0x322/frame 0= xfffffe2020ebc830 > netisr_dispatch_src() at 0xffffffff80583db5 =3D netisr_dispatch_src+0xa= 5/frame 0xfffffe2020ebc890 > ether_input() at 0xffffffff80575db6 =3D ether_input+0x26/frame 0xfffffe= 2020ebc8b0 > ixgbe_rxeof() at 0xffffffff813df36b =3D ixgbe_rxeof+0x7ab/frame 0xfffff= e2020ebc990 > ixgbe_msix_que() at 0xffffffff813da57c =3D ixgbe_msix_que+0x8c/frame 0x= fffffe2020ebc9e0 > intr_event_execute_handlers() at 0xffffffff8046c605 =3D intr_event_exec= ute_handlers+0x95/frame 0xfffffe2020ebca20 > ithread_loop() at 0xffffffff8046cc26 =3D ithread_loop+0xa6/frame 0xffff= fe2020ebca70 > fork_exit() at 0xffffffff8046a211 =3D fork_exit+0x71/frame 0xfffffe2020= ebcab0 > fork_trampoline() at 0xffffffff806cb50e =3D fork_trampoline+0xe/frame 0= xfffffe2020ebcab0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >=20 > Tracing command intr pid 12 tid 100111 td 0xfffff801250a2000 > sched_switch() at 0xffffffff804c956d =3D sched_switch+0x6ad/frame 0xfff= ffe2020f302f0 > mi_switch() at 0xffffffff804a8d92 =3D mi_switch+0xd2/frame 0xfffffe2020= f30320 > turnstile_wait() at 0xffffffff804ef177 =3D turnstile_wait+0x2a7/frame 0= xfffffe2020f30360 > __mtx_lock_sleep() at 0xffffffff80484d9d =3D __mtx_lock_sleep+0x13d/fra= me 0xfffffe2020f303e0 > sodealloc() at 0xffffffff8051b992 =3D sodealloc+0x32/frame 0xfffffe2020= f30420 > tcp_close() at 0xffffffff80618150 =3D tcp_close+0xd0/frame 0xfffffe2020= f30450 > tcp_twstart() at 0xffffffff8061f0e7 =3D tcp_twstart+0x2b7/frame 0xfffff= e2020f30490 > tcp_do_segment() at 0xffffffff80610bdf =3D tcp_do_segment+0x201f/frame = 0xfffffe2020f30590 > tcp_input() at 0xffffffff8060e17c =3D tcp_input+0xe1c/frame 0xfffffe202= 0f306e0 > ip_input() at 0xffffffff805a087f =3D ip_input+0x15f/frame 0xfffffe2020f= 30740 > netisr_dispatch_src() at 0xffffffff80583db5 =3D netisr_dispatch_src+0xa= 5/frame 0xfffffe2020f307a0 > ether_demux() at 0xffffffff80575b3a =3D ether_demux+0x12a/frame 0xfffff= e2020f307d0 > ether_nh_input() at 0xffffffff80576792 =3D ether_nh_input+0x322/frame 0= xfffffe2020f30830 > netisr_dispatch_src() at 0xffffffff80583db5 =3D netisr_dispatch_src+0xa= 5/frame 0xfffffe2020f30890 > ether_input() at 0xffffffff80575db6 =3D ether_input+0x26/frame 0xfffffe= 2020f308b0 > ixgbe_rxeof() at 0xffffffff813df36b =3D ixgbe_rxeof+0x7ab/frame 0xfffff= e2020f30990 > ixgbe_msix_que() at 0xffffffff813da57c =3D ixgbe_msix_que+0x8c/frame 0x= fffffe2020f309e0 > intr_event_execute_handlers() at 0xffffffff8046c605 =3D intr_event_exec= ute_handlers+0x95/frame 0xfffffe2020f30a20 > ithread_loop() at 0xffffffff8046cc26 =3D ithread_loop+0xa6/frame 0xffff= fe2020f30a70 > fork_exit() at 0xffffffff8046a211 =3D fork_exit+0x71/frame 0xfffffe2020= f30ab0 > fork_trampoline() at 0xffffffff806cb50e =3D fork_trampoline+0xe/frame 0= xfffffe2020f30ab0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >=20 > Tracing command intr pid 12 tid 100113 td 0xfffff801250a1500 > sched_switch() at 0xffffffff804c956d =3D sched_switch+0x6ad/frame 0xfff= ffe2020f3a2f0 > mi_switch() at 0xffffffff804a8d92 =3D mi_switch+0xd2/frame 0xfffffe2020= f3a320 > turnstile_wait() at 0xffffffff804ef177 =3D turnstile_wait+0x2a7/frame 0= xfffffe2020f3a360 > __mtx_lock_sleep() at 0xffffffff80484d9d =3D __mtx_lock_sleep+0x13d/fra= me 0xfffffe2020f3a3e0 > sodealloc() at 0xffffffff8051b992 =3D sodealloc+0x32/frame 0xfffffe2020= f3a420 > tcp_close() at 0xffffffff80618150 =3D tcp_close+0xd0/frame 0xfffffe2020= f3a450 > tcp_twstart() at 0xffffffff8061f0e7 =3D tcp_twstart+0x2b7/frame 0xfffff= e2020f3a490 > tcp_do_segment() at 0xffffffff80610bdf =3D tcp_do_segment+0x201f/frame = 0xfffffe2020f3a590 > tcp_input() at 0xffffffff8060e17c =3D tcp_input+0xe1c/frame 0xfffffe202= 0f3a6e0 > ip_input() at 0xffffffff805a087f =3D ip_input+0x15f/frame 0xfffffe2020f= 3a740 > netisr_dispatch_src() at 0xffffffff80583db5 =3D netisr_dispatch_src+0xa= 5/frame 0xfffffe2020f3a7a0 > ether_demux() at 0xffffffff80575b3a =3D ether_demux+0x12a/frame 0xfffff= e2020f3a7d0 > ether_nh_input() at 0xffffffff80576792 =3D ether_nh_input+0x322/frame 0= xfffffe2020f3a830 > netisr_dispatch_src() at 0xffffffff80583db5 =3D netisr_dispatch_src+0xa= 5/frame 0xfffffe2020f3a890 > ether_input() at 0xffffffff80575db6 =3D ether_input+0x26/frame 0xfffffe= 2020f3a8b0 > ixgbe_rxeof() at 0xffffffff813df36b =3D ixgbe_rxeof+0x7ab/frame 0xfffff= e2020f3a990 > ixgbe_msix_que() at 0xffffffff813da57c =3D ixgbe_msix_que+0x8c/frame 0x= fffffe2020f3a9e0 > intr_event_execute_handlers() at 0xffffffff8046c605 =3D intr_event_exec= ute_handlers+0x95/frame 0xfffffe2020f3aa20 > ithread_loop() at 0xffffffff8046cc26 =3D ithread_loop+0xa6/frame 0xffff= fe2020f3aa70 > fork_exit() at 0xffffffff8046a211 =3D fork_exit+0x71/frame 0xfffffe2020= f3aab0 > fork_trampoline() at 0xffffffff806cb50e =3D fork_trampoline+0xe/frame 0= xfffffe2020f3aab0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >=20 > Tracing command intr pid 12 tid 100115 td 0xfffff801250a0a00 > cpustop_handler() at 0xffffffff80775998 =3D cpustop_handler+0x28/frame = 0xfffffe1f9e182cf0 > ipi_nmi_handler() at 0xffffffff8077595a =3D ipi_nmi_handler+0x4a/frame = 0xfffffe1f9e182d10 > trap() at 0xffffffff806e2e4a =3D trap+0x3a/frame 0xfffffe1f9e182f20 > nmi_calltrap() at 0xffffffff806cb413 =3D nmi_calltrap+0x8/frame 0xfffff= e1f9e182f20 > --- trap 0x13, rip =3D 0xffffffff8059b9f9, rsp =3D 0xfffffe2020f44420, = rbp =3D 0xfffffe2020f44420 --- > in_pcbref() at 0xffffffff8059b9f9 =3D in_pcbref+0x9/frame 0xfffffe2020f= 44420 > tcp_tw_2msl_scan() at 0xffffffff8061f1a3 =3D tcp_tw_2msl_scan+0x73/fram= e 0xfffffe2020f44450 > tcp_twstart() at 0xffffffff8061eed3 =3D tcp_twstart+0xa3/frame 0xfffffe= 2020f44490 > tcp_do_segment() at 0xffffffff80610bdf =3D tcp_do_segment+0x201f/frame = 0xfffffe2020f44590 > tcp_input() at 0xffffffff8060e17c =3D tcp_input+0xe1c/frame 0xfffffe202= 0f446e0 > ip_input() at 0xffffffff805a087f =3D ip_input+0x15f/frame 0xfffffe2020f= 44740 > netisr_dispatch_src() at 0xffffffff80583db5 =3D netisr_dispatch_src+0xa= 5/frame 0xfffffe2020f447a0 > ether_demux() at 0xffffffff80575b3a =3D ether_demux+0x12a/frame 0xfffff= e2020f447d0 > ether_nh_input() at 0xffffffff80576792 =3D ether_nh_input+0x322/frame 0= xfffffe2020f44830 > netisr_dispatch_src() at 0xffffffff80583db5 =3D netisr_dispatch_src+0xa= 5/frame 0xfffffe2020f44890 > ether_input() at 0xffffffff80575db6 =3D ether_input+0x26/frame 0xfffffe= 2020f448b0 > ixgbe_rxeof() at 0xffffffff813df36b =3D ixgbe_rxeof+0x7ab/frame 0xfffff= e2020f44990 > ixgbe_msix_que() at 0xffffffff813da57c =3D ixgbe_msix_que+0x8c/frame 0x= fffffe2020f449e0 > intr_event_execute_handlers() at 0xffffffff8046c605 =3D intr_event_exec= ute_handlers+0x95/frame 0xfffffe2020f44a20 > ithread_loop() at 0xffffffff8046cc26 =3D ithread_loop+0xa6/frame 0xffff= fe2020f44a70 > fork_exit() at 0xffffffff8046a211 =3D fork_exit+0x71/frame 0xfffffe2020= f44ab0 > fork_trampoline() at 0xffffffff806cb50e =3D fork_trampoline+0xe/frame 0= xfffffe2020f44ab0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >=20 > Tracing command ps pid 37011 tid 101992 td 0xfffff80144378000 > sched_switch() at 0xffffffff804c956d =3D sched_switch+0x6ad/frame 0xfff= ffe20224367b0 > mi_switch() at 0xffffffff804a8d92 =3D mi_switch+0xd2/frame 0xfffffe2022= 4367e0 > turnstile_wait() at 0xffffffff804ef177 =3D turnstile_wait+0x2a7/frame 0= xfffffe2022436820 > __mtx_lock_sleep() at 0xffffffff80484d9d =3D __mtx_lock_sleep+0x13d/fra= me 0xfffffe20224368a0 > soalloc() at 0xffffffff8051b914 =3D soalloc+0x1b4/frame 0xfffffe2022436= 8f0 > socreate() at 0xffffffff8051b617 =3D socreate+0xa7/frame 0xfffffe202243= 6940 > sys_socket() at 0xffffffff8052144d =3D sys_socket+0xed/frame 0xfffffe20= 224369a0 > amd64_syscall() at 0xffffffff806e4051 =3D amd64_syscall+0x2c1/frame 0xf= ffffe2022436ab0 > Xfast_syscall() at 0xffffffff806cb2bb =3D Xfast_syscall+0xfb/frame 0xff= fffe2022436ab0 > --- syscall (97, FreeBSD ELF64, sys_socket), rip =3D 0x8011c413a, rsp =3D= 0x7fffffffc748, rbp =3D 0x7fffffffc770 --- >=20 > Tracing command cron pid 37008 tid 102228 td 0xfffff801a4090000 > sched_switch() at 0xffffffff804c956d =3D sched_switch+0x6ad/frame 0xfff= ffe20228d67b0 > mi_switch() at 0xffffffff804a8d92 =3D mi_switch+0xd2/frame 0xfffffe2022= 8d67e0 > turnstile_wait() at 0xffffffff804ef177 =3D turnstile_wait+0x2a7/frame 0= xfffffe20228d6820 > __mtx_lock_sleep() at 0xffffffff80484d9d =3D __mtx_lock_sleep+0x13d/fra= me 0xfffffe20228d68a0 > soalloc() at 0xffffffff8051b914 =3D soalloc+0x1b4/frame 0xfffffe20228d6= 8f0 > socreate() at 0xffffffff8051b617 =3D socreate+0xa7/frame 0xfffffe20228d= 6940 > sys_socket() at 0xffffffff8052144d =3D sys_socket+0xed/frame 0xfffffe20= 228d69a0 > amd64_syscall() at 0xffffffff806e4051 =3D amd64_syscall+0x2c1/frame 0xf= ffffe20228d6ab0 > Xfast_syscall() at 0xffffffff806cb2bb =3D Xfast_syscall+0xfb/frame 0xff= fffe20228d6ab0 > --- syscall (97, FreeBSD ELF64, sys_socket), rip =3D 0x800d8c13a, rsp =3D= 0x7fffffffd658, rbp =3D 0x7fffffffd6f0 --- >=20 > [many likes] >=20 > Tracing command intr pid 12 tid 100015 td 0xfffff8011422b000 > sched_switch() at 0xffffffff804c956d =3D sched_switch+0x6ad/frame 0xfff= ffe1f9e1cf760 > mi_switch() at 0xffffffff804a8d92 =3D mi_switch+0xd2/frame 0xfffffe1f9e= 1cf790 > turnstile_wait() at 0xffffffff804ef177 =3D turnstile_wait+0x2a7/frame 0= xfffffe1f9e1cf7d0 > __mtx_lock_sleep() at 0xffffffff80484d9d =3D __mtx_lock_sleep+0x13d/fra= me 0xfffffe1f9e1cf850 > sodealloc() at 0xffffffff8051b992 =3D sodealloc+0x32/frame 0xfffffe1f9e= 1cf890 > tcp_close() at 0xffffffff80618150 =3D tcp_close+0xd0/frame 0xfffffe1f9e= 1cf8c0 > tcp_timer_2msl() at 0xffffffff8061dda3 =3D tcp_timer_2msl+0x1f3/frame 0= xfffffe1f9e1cf8f0 > softclock_call_cc() at 0xffffffff804b4ca9 =3D softclock_call_cc+0x179/f= rame 0xfffffe1f9e1cf9c0 > softclock() at 0xffffffff804b5034 =3D softclock+0x44/frame 0xfffffe1f9e= 1cf9e0 > intr_event_execute_handlers() at 0xffffffff8046c605 =3D intr_event_exec= ute_handlers+0x95/frame 0xfffffe1f9e1cfa20 > ithread_loop() at 0xffffffff8046cc26 =3D ithread_loop+0xa6/frame 0xffff= fe1f9e1cfa70 > fork_exit() at 0xffffffff8046a211 =3D fork_exit+0x71/frame 0xfffffe1f9e= 1cfab0 > fork_trampoline() at 0xffffffff806cb50e =3D fork_trampoline+0xe/frame 0= xfffffe1f9e1cfab0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >=20 > Tracing command intr pid 12 tid 100016 td 0xfffff8011422aa00 > sched_switch() at 0xffffffff804c956d =3D sched_switch+0x6ad/frame 0xfff= ffe1f9e1d4760 > mi_switch() at 0xffffffff804a8d92 =3D mi_switch+0xd2/frame 0xfffffe1f9e= 1d4790 > turnstile_wait() at 0xffffffff804ef177 =3D turnstile_wait+0x2a7/frame 0= xfffffe1f9e1d47d0 > __mtx_lock_sleep() at 0xffffffff80484d9d =3D __mtx_lock_sleep+0x13d/fra= me 0xfffffe1f9e1d4850 > sodealloc() at 0xffffffff8051b992 =3D sodealloc+0x32/frame 0xfffffe1f9e= 1d4890 > tcp_close() at 0xffffffff80618150 =3D tcp_close+0xd0/frame 0xfffffe1f9e= 1d48c0 > tcp_timer_2msl() at 0xffffffff8061dda3 =3D tcp_timer_2msl+0x1f3/frame 0= xfffffe1f9e1d48f0 > softclock_call_cc() at 0xffffffff804b4ca9 =3D softclock_call_cc+0x179/f= rame 0xfffffe1f9e1d49c0 > softclock() at 0xffffffff804b5034 =3D softclock+0x44/frame 0xfffffe1f9e= 1d49e0 > intr_event_execute_handlers() at 0xffffffff8046c605 =3D intr_event_exec= ute_handlers+0x95/frame 0xfffffe1f9e1d4a20 > ithread_loop() at 0xffffffff8046cc26 =3D ithread_loop+0xa6/frame 0xffff= fe1f9e1d4a70 > fork_exit() at 0xffffffff8046a211 =3D fork_exit+0x71/frame 0xfffffe1f9e= 1d4ab0 > fork_trampoline() at 0xffffffff806cb50e =3D fork_trampoline+0xe/frame 0= xfffffe1f9e1d4ab0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >=20 > Tracing command intr pid 12 tid 100017 td 0xfffff8011422a500 > sched_switch() at 0xffffffff804c956d =3D sched_switch+0x6ad/frame 0xfff= ffe1f9e1d9760 > mi_switch() at 0xffffffff804a8d92 =3D mi_switch+0xd2/frame 0xfffffe1f9e= 1d9790 > turnstile_wait() at 0xffffffff804ef177 =3D turnstile_wait+0x2a7/frame 0= xfffffe1f9e1d97d0 > __mtx_lock_sleep() at 0xffffffff80484d9d =3D __mtx_lock_sleep+0x13d/fra= me 0xfffffe1f9e1d9850 > sodealloc() at 0xffffffff8051b992 =3D sodealloc+0x32/frame 0xfffffe1f9e= 1d9890 > tcp_close() at 0xffffffff80618150 =3D tcp_close+0xd0/frame 0xfffffe1f9e= 1d98c0 > tcp_timer_2msl() at 0xffffffff8061dda3 =3D tcp_timer_2msl+0x1f3/frame 0= xfffffe1f9e1d98f0 > softclock_call_cc() at 0xffffffff804b4ca9 =3D softclock_call_cc+0x179/f= rame 0xfffffe1f9e1d99c0 > softclock() at 0xffffffff804b5034 =3D softclock+0x44/frame 0xfffffe1f9e= 1d99e0 > intr_event_execute_handlers() at 0xffffffff8046c605 =3D intr_event_exec= ute_handlers+0x95/frame 0xfffffe1f9e1d9a20 > ithread_loop() at 0xffffffff8046cc26 =3D ithread_loop+0xa6/frame 0xffff= fe1f9e1d9a70 > fork_exit() at 0xffffffff8046a211 =3D fork_exit+0x71/frame 0xfffffe1f9e= 1d9ab0 > fork_trampoline() at 0xffffffff806cb50e =3D fork_trampoline+0xe/frame 0= xfffffe1f9e1d9ab0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >=20 > Tracing command intr pid 12 tid 100018 td 0xfffff8011422a000 > sched_switch() at 0xffffffff804c956d =3D sched_switch+0x6ad/frame 0xfff= ffe1f9e1de760 > mi_switch() at 0xffffffff804a8d92 =3D mi_switch+0xd2/frame 0xfffffe1f9e= 1de790 > turnstile_wait() at 0xffffffff804ef177 =3D turnstile_wait+0x2a7/frame 0= xfffffe1f9e1de7d0 > __mtx_lock_sleep() at 0xffffffff80484d9d =3D __mtx_lock_sleep+0x13d/fra= me 0xfffffe1f9e1de850 > sodealloc() at 0xffffffff8051b992 =3D sodealloc+0x32/frame 0xfffffe1f9e= 1de890 > tcp_close() at 0xffffffff80618150 =3D tcp_close+0xd0/frame 0xfffffe1f9e= 1de8c0 > tcp_timer_2msl() at 0xffffffff8061dda3 =3D tcp_timer_2msl+0x1f3/frame 0= xfffffe1f9e1de8f0 > softclock_call_cc() at 0xffffffff804b4ca9 =3D softclock_call_cc+0x179/f= rame 0xfffffe1f9e1de9c0 > softclock() at 0xffffffff804b5034 =3D softclock+0x44/frame 0xfffffe1f9e= 1de9e0 > intr_event_execute_handlers() at 0xffffffff8046c605 =3D intr_event_exec= ute_handlers+0x95/frame 0xfffffe1f9e1dea20 > ithread_loop() at 0xffffffff8046cc26 =3D ithread_loop+0xa6/frame 0xffff= fe1f9e1dea70 > fork_exit() at 0xffffffff8046a211 =3D fork_exit+0x71/frame 0xfffffe1f9e= 1deab0 > fork_trampoline() at 0xffffffff806cb50e =3D fork_trampoline+0xe/frame 0= xfffffe1f9e1deab0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >=20 > Tracing command intr pid 12 tid 100019 td 0xfffff8011424da00 > sched_switch() at 0xffffffff804c956d =3D sched_switch+0x6ad/frame 0xfff= ffe1f9e1e3760 > mi_switch() at 0xffffffff804a8d92 =3D mi_switch+0xd2/frame 0xfffffe1f9e= 1e3790 > turnstile_wait() at 0xffffffff804ef177 =3D turnstile_wait+0x2a7/frame 0= xfffffe1f9e1e37d0 > __mtx_lock_sleep() at 0xffffffff80484d9d =3D __mtx_lock_sleep+0x13d/fra= me 0xfffffe1f9e1e3850 > sodealloc() at 0xffffffff8051b992 =3D sodealloc+0x32/frame 0xfffffe1f9e= 1e3890 > tcp_close() at 0xffffffff80618150 =3D tcp_close+0xd0/frame 0xfffffe1f9e= 1e38c0 > tcp_timer_2msl() at 0xffffffff8061dda3 =3D tcp_timer_2msl+0x1f3/frame 0= xfffffe1f9e1e38f0 > softclock_call_cc() at 0xffffffff804b4ca9 =3D softclock_call_cc+0x179/f= rame 0xfffffe1f9e1e39c0 > softclock() at 0xffffffff804b5034 =3D softclock+0x44/frame 0xfffffe1f9e= 1e39e0 > intr_event_execute_handlers() at 0xffffffff8046c605 =3D intr_event_exec= ute-- 7:zsh -- time-stamp -- Sep/26/16 20:00:13 -- > -- 7:zsh -- time-stamp -- Sep/26/16 20:00:13 -- > _handlers+0x95/frame 0xfffffe1f9e1e3a20 > ithread_loop() at 0xffffffff8046cc26 =3D ithread_loop+0xa6/frame 0xffff= fe1f9e1e3a70 > fork_exit() at 0xffffffff8046a211 =3D fork_exit+0x71/frame 0xfffffe1f9e= 1e3ab0 > fork_trampoline() at 0xffffffff806cb50e =3D fork_trampoline+0xe/frame 0= xfffffe1f9e1e3ab0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >=20 > Tracing command intr pid 12 tid 100020 td 0xfffff8011424d500 > sched_switch() at 0xffffffff804c956d =3D sched_switch+0x6ad/frame 0xfff= ffe1f9e1e8760 > mi_switch() at 0xffffffff804a8d92 =3D mi_switch+0xd2/frame 0xfffffe1f9e= 1e8790 > turnstile_wait() at 0xffffffff804ef177 =3D turnstile_wait+0x2a7/frame 0= xfffffe1f9e1e87d0 > __mtx_lock_sleep() at 0xffffffff80484d9d =3D __mtx_lock_sleep+0x13d/fra= me 0xfffffe1f9e1e8850 > sodealloc() at 0xffffffff8051b992 =3D sodealloc+0x32/frame 0xfffffe1f9e= 1e8890 > tcp_close() at 0xffffffff80618150 =3D tcp_close+0xd0/frame 0xfffffe1f9e= 1e88c0 > tcp_timer_2msl() at 0xffffffff8061dda3 =3D tcp_timer_2msl+0x1f3/frame 0= xfffffe1f9e1e88f0 > softclock_call_cc() at 0xffffffff804b4ca9 =3D softclock_call_cc+0x179/f= rame 0xfffffe1f9e1e89c0 > softclock() at 0xffffffff804b5034 =3D softclock+0x44/frame 0xfffffe1f9e= 1e89e0 > intr_event_execute_handlers() at 0xffffffff8046c605 =3D intr_event_exec= ute_handlers+0x95/frame 0xfffffe1f9e1e8a20 > ithread_loop() at 0xffffffff8046cc26 =3D ithread_loop+0xa6/frame 0xffff= fe1f9e1e8a70 > fork_exit() at 0xffffffff8046a211 =3D fork_exit+0x71/frame 0xfffffe1f9e= 1e8ab0 > fork_trampoline() at 0xffffffff806cb50e =3D fork_trampoline+0xe/frame 0= xfffffe1f9e1e8ab0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >=20 > Tracing command intr pid 12 tid 100021 td 0xfffff8011424d000 > sched_switch() at 0xffffffff804c956d =3D sched_switch+0x6ad/frame 0xfff= ffe1f9e1ed760 > mi_switch() at 0xffffffff804a8d92 =3D mi_switch+0xd2/frame 0xfffffe1f9e= 1ed790 > turnstile_wait() at 0xffffffff804ef177 =3D turnstile_wait+0x2a7/frame 0= xfffffe1f9e1ed7d0 > __mtx_lock_sleep() at 0xffffffff80484d9d =3D __mtx_lock_sleep+0x13d/fra= me 0xfffffe1f9e1ed850 > sodealloc() at 0xffffffff8051b992 =3D sodealloc+0x32/frame 0xfffffe1f9e= 1ed890 > tcp_close() at 0xffffffff80618150 =3D tcp_close+0xd0/frame 0xfffffe1f9e= 1ed8c0 > tcp_timer_2msl() at 0xffffffff8061dda3 =3D tcp_timer_2msl+0x1f3/frame 0= xfffffe1f9e1ed8f0 > softclock_call_cc() at 0xffffffff804b4ca9 =3D softclock_call_cc+0x179/f= rame 0xfffffe1f9e1ed9c0 > softclock() at 0xffffffff804b5034 =3D softclock+0x44/frame 0xfffffe1f9e= 1ed9e0 > intr_event_execute_handlers() at 0xffffffff8046c605 =3D intr_event_exec= ute_handlers+0x95/frame 0xfffffe1f9e1eda20 > ithread_loop() at 0xffffffff8046cc26 =3D ithread_loop+0xa6/frame 0xffff= fe1f9e1eda70 > fork_exit() at 0xffffffff8046a211 =3D fork_exit+0x71/frame 0xfffffe1f9e= 1edab0 > fork_trampoline() at 0xffffffff806cb50e =3D fork_trampoline+0xe/frame 0= xfffffe1f9e1edab0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >=20 > Tracing command intr pid 12 tid 100022 td 0xfffff8011424ca00 > sched_switch() at 0xffffffff804c956d =3D sched_switch+0x6ad/frame 0xfff= ffe1f9e1f2760 > mi_switch() at 0xffffffff804a8d92 =3D mi_switch+0xd2/frame 0xfffffe1f9e= 1f2790 > turnstile_wait() at 0xffffffff804ef177 =3D turnstile_wait+0x2a7/frame 0= xfffffe1f9e1f27d0 > __mtx_lock_sleep() at 0xffffffff80484d9d =3D __mtx_lock_sleep+0x13d/fra= me 0xfffffe1f9e1f2850 > sodealloc() at 0xffffffff8051b992 =3D sodealloc+0x32/frame 0xfffffe1f9e= 1f2890 > tcp_close() at 0xffffffff80618150 =3D tcp_close+0xd0/frame 0xfffffe1f9e= 1f28c0 > tcp_timer_2msl() at 0xffffffff8061dda3 =3D tcp_timer_2msl+0x1f3/frame 0= xfffffe1f9e1f28f0 > softclock_call_cc() at 0xffffffff804b4ca9 =3D softclock_call_cc+0x179/f= rame 0xfffffe1f9e1f29c0 > softclock() at 0xffffffff804b5034 =3D softclock+0x44/frame 0xfffffe1f9e= 1f29e0 > intr_event_execute_handlers() at 0xffffffff8046c605 =3D intr_event_exec= ute_handlers+0x95/frame 0xfffffe1f9e1f2a20 > ithread_loop() at 0xffffffff8046cc26 =3D ithread_loop+0xa6/frame 0xffff= fe1f9e1f2a70 > fork_exit() at 0xffffffff8046a211 =3D fork_exit+0x71/frame 0xfffffe1f9e= 1f2ab0 > fork_trampoline() at 0xffffffff806cb50e =3D fork_trampoline+0xe/frame 0= xfffffe1f9e1f2ab0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >=20 > Tracing command intr pid 12 tid 100023 td 0xfffff8011424c500 > sched_switch() at 0xffffffff804c956d =3D sched_switch+0x6ad/frame 0xfff= ffe1f9e1f7760 > mi_switch() at 0xffffffff804a8d92 =3D mi_switch+0xd2/frame 0xfffffe1f9e= 1f7790 > turnstile_wait() at 0xffffffff804ef177 =3D turnstile_wait+0x2a7/frame 0= xfffffe1f9e1f77d0 > __mtx_lock_sleep() at 0xffffffff80484d9d =3D __mtx_lock_sleep+0x13d/fra= me 0xfffffe1f9e1f7850 > sodealloc() at 0xffffffff8051b992 =3D sodealloc+0x32/frame 0xfffffe1f9e= 1f7890 > tcp_close() at 0xffffffff80618150 =3D tcp_close+0xd0/frame 0xfffffe1f9e= 1f78c0 > tcp_timer_rexmt() at 0xffffffff8061e454 =3D tcp_timer_rexmt+0x114/frame= 0xfffffe1f9e1f78f0 > softclock_call_cc() at 0xffffffff804b4ca9 =3D softclock_call_cc+0x179/f= rame 0xfffffe1f9e1f79c0 > softclock() at 0xffffffff804b5034 =3D softclock+0x44/frame 0xfffffe1f9e= 1f79e0 > intr_event_execute_handlers() at 0xffffffff8046c605 =3D intr_event_exec= ute_handlers+0x95/frame 0xfffffe1f9e1f7a20 > ithread_loop() at 0xffffffff8046cc26 =3D ithread_loop+0xa6/frame 0xffff= fe1f9e1f7a70 > fork_exit() at 0xffffffff8046a211 =3D fork_exit+0x71/frame 0xfffffe1f9e= 1f7ab0 > fork_trampoline() at 0xffffffff806cb50e =3D fork_trampoline+0xe/frame 0= xfffffe1f9e1f7ab0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >=20 > Tracing command intr pid 12 tid 100024 td 0xfffff8011424c000 > sched_switch() at 0xffffffff804c956d =3D sched_switch+0x6ad/frame 0xfff= ffe1f9e1fc760 > mi_switch() at 0xffffffff804a8d92 =3D mi_switch+0xd2/frame 0xfffffe1f9e= 1fc790 > turnstile_wait() at 0xffffffff804ef177 =3D turnstile_wait+0x2a7/frame 0= xfffffe1f9e1fc7d0 > __mtx_lock_sleep() at 0xffffffff80484d9d =3D __mtx_lock_sleep+0x13d/fra= me 0xfffffe1f9e1fc850 > sodealloc() at 0xffffffff8051b992 =3D sodealloc+0x32/frame 0xfffffe1f9e= 1fc890 > tcp_close() at 0xffffffff80618150 =3D tcp_close+0xd0/frame 0xfffffe1f9e= 1fc8c0 > tcp_timer_2msl() at 0xffffffff8061dda3 =3D tcp_timer_2msl+0x1f3/frame 0= xfffffe1f9e1fc8f0 > softclock_call_cc() at 0xffffffff804b4ca9 =3D softclock_call_cc+0x179/f= rame 0xfffffe1f9e1fc9c0 > softclock() at 0xffffffff804b5034 =3D softclock+0x44/frame 0xfffffe1f9e= 1fc9e0 > intr_event_execute_handlers() at 0xffffffff8046c605 =3D intr_event_exec= ute_handlers+0x95/frame 0xfffffe1f9e1fca20 > ithread_loop() at 0xffffffff8046cc26 =3D ithread_loop+0xa6/frame 0xffff= fe1f9e1fca70 > fork_exit() at 0xffffffff8046a211 =3D fork_exit+0x71/frame 0xfffffe1f9e= 1fcab0 > fork_trampoline() at 0xffffffff806cb50e =3D fork_trampoline+0xe/frame 0= xfffffe1f9e1fcab0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >=20 > Tracing command intr pid 12 tid 100025 td 0xfffff8011424ba00 > sched_switch() at 0xffffffff804c956d =3D sched_switch+0x6ad/frame 0xfff= ffe0000382760 > mi_switch() at 0xffffffff804a8d92 =3D mi_switch+0xd2/frame 0xfffffe0000= 382790 > turnstile_wait() at 0xffffffff804ef177 =3D turnstile_wait+0x2a7/frame 0= xfffffe00003827d0 > __mtx_lock_sleep() at 0xffffffff80484d9d =3D __mtx_lock_sleep+0x13d/fra= me 0xfffffe0000382850 > sodealloc() at 0xffffffff8051b992 =3D sodealloc+0x32/frame 0xfffffe0000= 382890 > tcp_close() at 0xffffffff80618150 =3D tcp_close+0xd0/frame 0xfffffe0000= 3828c0 > tcp_timer_2msl() at 0xffffffff8061dda3 =3D tcp_timer_2msl+0x1f3/frame 0= xfffffe00003828f0 > softclock_call_cc() at 0xffffffff804b4ca9 =3D softclock_call_cc+0x179/f= rame 0xfffffe00003829c0 > softclock() at 0xffffffff804b5034 =3D softclock+0x44/frame 0xfffffe0000= 3829e0 > intr_event_execute_handlers() at 0xffffffff8046c605 =3D intr_event_exec= ute_handlers+0x95/frame 0xfffffe0000382a20 > ithread_loop() at 0xffffffff8046cc26 =3D ithread_loop+0xa6/frame 0xffff= fe0000382a70 > fork_exit() at 0xffffffff8046a211 =3D fork_exit+0x71/frame 0xfffffe0000= 382ab0 > fork_trampoline() at 0xffffffff806cb50e =3D fork_trampoline+0xe/frame 0= xfffffe0000382ab0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >=20 > Tracing command intr pid 12 tid 100026 td 0xfffff8011424b500 > sched_switch() at 0xffffffff804c956d =3D sched_switch+0x6ad/frame 0xfff= ffe00003876f0 > mi_switch() at 0xffffffff804a8d92 =3D mi_switch+0xd2/frame 0xfffffe0000= 387720 > critical_exit() at 0xffffffff804a6bee =3D critical_exit+0x7e/frame 0xff= fffe0000387740 > ipi_bitmap_handler() at 0xffffffff80775629 =3D ipi_bitmap_handler+0x79/= frame 0xfffffe0000387780 > Xipi_intr_bitmap_handler() at 0xffffffff806cc15e =3D Xipi_intr_bitmap_h= andler+0x8e/frame 0xfffffe0000387780 > --- interrupt, rip =3D 0xffffffff80484c1f, rsp =3D 0xfffffe0000387850, = rbp =3D 0xfffffe0000387850 --- > __mtx_lock_flags() at 0xffffffff80484c1f =3D __mtx_lock_flags+0x2f/fram= e 0xfffffe0000387850 > sodealloc() at 0xffffffff8051b992 =3D sodealloc+0x32/frame 0xfffffe0000= 387890 > tcp_close() at 0xffffffff80618150 =3D tcp_close+0xd0/frame 0xfffffe0000= 3878c0 > tcp_timer_2msl() at 0xffffffff8061dda3 =3D tcp_timer_2msl+0x1f3/frame 0= xfffffe00003878f0 > softclock_call_cc() at 0xffffffff804b4ca9 =3D softclock_call_cc+0x179/f= rame 0xfffffe00003879c0 > softclock() at 0xffffffff804b5034 =3D softclock+0x44/frame 0xfffffe0000= 3879e0 > intr_event_execute_handlers() at 0xffffffff8046c605 =3D intr_event_exec= ute_handlers+0x95/frame 0xfffffe0000387a20 > ithread_loop() at 0xffffffff8046cc26 =3D ithread_loop+0xa6/frame 0xffff= fe0000387a70 > fork_exit() at 0xffffffff8046a211 =3D fork_exit+0x71/frame 0xfffffe0000= 387ab0 > fork_trampoline() at 0xffffffff806cb50e =3D fork_trampoline+0xe/frame 0= xfffffe0000387ab0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- Nice stack traces, all threads are blocked in sodealloc() or soalloc() and if you look at how mtx_lock(&so_global_mtx) and mtx_unlock(&so_global_mtx) are used, it is hard to think about a scenario that can lead to this state. I am still trying to reproduce your issue, without success so far. -- Julien --eJoODbLvCaU8HsWinURaWqn1UNV0S7cOH-- --Grg9p8WN0dsPRVQnwhLgfPmlD08BcOFiP Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJX65Y6AAoJEKVlQ5Je6dhxd8cH/3GbB4ohO33iOXTxdggxQk4l HR24PwCIFVvbVoJ+GokjIbaCD33tWHkL3N1BcdNQ2d/l+72HzciDVIsnCr4VSmQ8 n8frasRJrggQaK61DWofBY5e/YIgsKxRh+k3xtiHJpc6us31CS+DR/7w5MgNAVmW 8v59r84iohbUdKceWQ92vWcaIDw7n9dObRptPHWXCuKXDDkLwUC2bWPgv2pljd6+ ODam89eRG3oNffS2xoUgBrlTkkkXFnL2MGsr/57Af9nFcLCtuRVG2n8O9HaRTuzT /mbrZm0zQ8JX7KIoiEQHl3UaXraUqn6TruOiFPzyJI5VXckS7q5cYhwpABRBowA= =q3Qe -----END PGP SIGNATURE----- --Grg9p8WN0dsPRVQnwhLgfPmlD08BcOFiP-- From owner-freebsd-stable@freebsd.org Wed Sep 28 12:41:15 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 528EDBEC8D4 for ; Wed, 28 Sep 2016 12:41:15 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E826C1892 for ; Wed, 28 Sep 2016 12:41:14 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1bpDW1-00052o-P1; Wed, 28 Sep 2016 14:59:09 +0300 Date: Wed, 28 Sep 2016 14:59:09 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20160928115909.GC54003@zxy.spb.ru> References: <8ba75d6e-4f01-895e-0aed-53c6c6692cb9@freebsd.org> <20160920202633.GQ2840@zxy.spb.ru> <20160921195155.GW2840@zxy.spb.ru> <20160923200143.GG2840@zxy.spb.ru> <20160925124626.GI2840@zxy.spb.ru> <20160926172159.GA54003@zxy.spb.ru> <62453d9c-b1e4-1129-70ff-654dacea37f9@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <62453d9c-b1e4-1129-70ff-654dacea37f9@gmail.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Sep 2016 12:41:15 -0000 On Wed, Sep 28, 2016 at 12:06:47PM +0200, Julien Charbon wrote: > > Tracing command intr pid 12 tid 100026 td 0xfffff8011424b500 > > sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe00003876f0 > > mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe0000387720 > > critical_exit() at 0xffffffff804a6bee = critical_exit+0x7e/frame 0xfffffe0000387740 > > ipi_bitmap_handler() at 0xffffffff80775629 = ipi_bitmap_handler+0x79/frame 0xfffffe0000387780 > > Xipi_intr_bitmap_handler() at 0xffffffff806cc15e = Xipi_intr_bitmap_handler+0x8e/frame 0xfffffe0000387780 > > --- interrupt, rip = 0xffffffff80484c1f, rsp = 0xfffffe0000387850, rbp = 0xfffffe0000387850 --- > > __mtx_lock_flags() at 0xffffffff80484c1f = __mtx_lock_flags+0x2f/frame 0xfffffe0000387850 > > sodealloc() at 0xffffffff8051b992 = sodealloc+0x32/frame 0xfffffe0000387890 > > tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe00003878c0 > > tcp_timer_2msl() at 0xffffffff8061dda3 = tcp_timer_2msl+0x1f3/frame 0xfffffe00003878f0 > > softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe00003879c0 > > softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe00003879e0 > > intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe0000387a20 > > ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe0000387a70 > > fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe0000387ab0 > > fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe0000387ab0 > > --- trap 0, rip = 0, rsp = 0, rbp = 0 --- > > Nice stack traces, all threads are blocked in sodealloc() or soalloc() > and if you look at how mtx_lock(&so_global_mtx) and > mtx_unlock(&so_global_mtx) are used, it is hard to think about a > scenario that can lead to this state. > > I am still trying to reproduce your issue, without success so far. May be some hardware-related (low-speed CPU?). Yesternight I am collect new stack traces and kernel dump. May be I can see something? db> ps pid ppid pgrp uid state wmesg wchan cmd 12 0 0 0 RL (threaded) [intr] 100023 RunQ [swi4: clock (8)] 100107 Run CPU 8 [irq291: ix0:q2] 11 0 0 0 RL (threaded) [idle] 100011 CanRun [idle: cpu8] cpuid = 8 dynamic pcpu = 0xfffffe201d69cf00 curthread = 0xfffff8012508d500: pid 12 "irq291: ix0:q2" curpcb = 0xfffffe2020ebcb80 fpcurthread = none idlethread = 0xfffff8011422c500: tid 100011 "idle: cpu8" curpmap = 0xffffffff80d49998 tssp = 0xffffffff80d7fcd0 commontssp = 0xffffffff80d7fcd0 rsp0 = 0xfffffe2020ebcb80 gs32p = 0xffffffff80d86528 ldt = 0xffffffff80d86568 tss = 0xffffffff80d86558 Tracing command nginx pid 1061 tid 101747 td 0xfffff8014b35b500 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe2021b70330 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe2021b70360 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe2021b703a0 __rw_wlock_hard() at 0xffffffff8049c314 = __rw_wlock_hard+0x94/frame 0xfffffe2021b70430 in_lltable_lookup() at 0xffffffff80594823 = in_lltable_lookup+0x83/frame 0xfffffe2021b70450 arpresolve() at 0xffffffff8058d2aa = arpresolve+0x9a/frame 0xfffffe2021b704b0 ether_output() at 0xffffffff805755e2 = ether_output+0x2f2/frame 0xfffffe2021b70550 ip_output() at 0xffffffff805a4200 = ip_output+0x1390/frame 0xfffffe2021b706b0 tcp_output() at 0xffffffff806149d5 = tcp_output+0x17a5/frame 0xfffffe2021b70850 tcp_usr_disconnect() at 0xffffffff80620094 = tcp_usr_disconnect+0x74/frame 0xfffffe2021b70880 soclose() at 0xffffffff8051c238 = soclose+0x38/frame 0xfffffe2021b708b0 _fdrop() at 0xffffffff8045639a = _fdrop+0x1a/frame 0xfffffe2021b708d0 closef() at 0xffffffff80458a53 = closef+0x1e3/frame 0xfffffe2021b70960 closefp() at 0xffffffff804567ad = closefp+0x7d/frame 0xfffffe2021b709a0 amd64_syscall() at 0xffffffff806e4051 = amd64_syscall+0x2c1/frame 0xfffffe2021b70ab0 Xfast_syscall() at 0xffffffff806cb2bb = Xfast_syscall+0xfb/frame 0xfffffe2021b70ab0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8019dbeaa, rsp = 0x7fffffffe6a8, rbp = 0x7fffffffe6c0 --- Tracing command nginx pid 1060 tid 101749 td 0xfffff80126a53a00 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe2021b7a240 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe2021b7a270 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe2021b7a2b0 __rw_wlock_hard() at 0xffffffff8049c314 = __rw_wlock_hard+0x94/frame 0xfffffe2021b7a340 in_lltable_lookup() at 0xffffffff80594823 = in_lltable_lookup+0x83/frame 0xfffffe2021b7a360 arpresolve() at 0xffffffff8058d2aa = arpresolve+0x9a/frame 0xfffffe2021b7a3c0 ether_output() at 0xffffffff805755e2 = ether_output+0x2f2/frame 0xfffffe2021b7a460 ip_output() at 0xffffffff805a4200 = ip_output+0x1390/frame 0xfffffe2021b7a5c0 tcp_output() at 0xffffffff806149d5 = tcp_output+0x17a5/frame 0xfffffe2021b7a760 tcp_usr_send() at 0xffffffff8062078b = tcp_usr_send+0x33b/frame 0xfffffe2021b7a7e0 sosend_generic() at 0xffffffff8051cf36 = sosend_generic+0x436/frame 0xfffffe2021b7a8a0 soo_write() at 0xffffffff804fd872 = soo_write+0x42/frame 0xfffffe2021b7a8d0 dofilewrite() at 0xffffffff804f5c97 = dofilewrite+0x87/frame 0xfffffe2021b7a920 kern_writev() at 0xffffffff804f5978 = kern_writev+0x68/frame 0xfffffe2021b7a970 sys_writev() at 0xffffffff804f5be6 = sys_writev+0x36/frame 0xfffffe2021b7a9a0 amd64_syscall() at 0xffffffff806e4051 = amd64_syscall+0x2c1/frame 0xfffffe2021b7aab0 Xfast_syscall() at 0xffffffff806cb2bb = Xfast_syscall+0xfb/frame 0xfffffe2021b7aab0 --- syscall (121, FreeBSD ELF64, sys_writev), rip = 0x8019cc6ba, rsp = 0x7fffffffd268, rbp = 0x7fffffffd2a0 --- Tracing command nginx pid 1059 tid 101817 td 0xfffff8014de1da00 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe20220cc240 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe20220cc270 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe20220cc2b0 __rw_wlock_hard() at 0xffffffff8049c314 = __rw_wlock_hard+0x94/frame 0xfffffe20220cc340 in_lltable_lookup() at 0xffffffff80594823 = in_lltable_lookup+0x83/frame 0xfffffe20220cc360 arpresolve() at 0xffffffff8058d2aa = arpresolve+0x9a/frame 0xfffffe20220cc3c0 ether_output() at 0xffffffff805755e2 = ether_output+0x2f2/frame 0xfffffe20220cc460 ip_output() at 0xffffffff805a4200 = ip_output+0x1390/frame 0xfffffe20220cc5c0 tcp_output() at 0xffffffff806149d5 = tcp_output+0x17a5/frame 0xfffffe20220cc760 tcp_usr_send() at 0xffffffff8062078b = tcp_usr_send+0x33b/frame 0xfffffe20220cc7e0 sosend_generic() at 0xffffffff8051cf36 = sosend_generic+0x436/frame 0xfffffe20220cc8a0 soo_write() at 0xffffffff804fd872 = soo_write+0x42/frame 0xfffffe20220cc8d0 dofilewrite() at 0xffffffff804f5c97 = dofilewrite+0x87/frame 0xfffffe20220cc920 kern_writev() at 0xffffffff804f5978 = kern_writev+0x68/frame 0xfffffe20220cc970 sys_writev() at 0xffffffff804f5be6 = sys_writev+0x36/frame 0xfffffe20220cc9a0 amd64_syscall() at 0xffffffff806e4051 = amd64_syscall+0x2c1/frame 0xfffffe20220ccab0 Xfast_syscall() at 0xffffffff806cb2bb = Xfast_syscall+0xfb/frame 0xfffffe20220ccab0 --- syscall (121, FreeBSD ELF64, sys_writev), rip = 0x8019cc6ba, rsp = 0x7fffffffcea8, rbp = 0x7fffffffcee0 --- Tracing command nginx pid 1058 tid 101816 td 0xfffff8014ddb2500 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe20220c7330 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe20220c7360 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe20220c73a0 __rw_wlock_hard() at 0xffffffff8049c314 = __rw_wlock_hard+0x94/frame 0xfffffe20220c7430 in_lltable_lookup() at 0xffffffff80594823 = in_lltable_lookup+0x83/frame 0xfffffe20220c7450 arpresolve() at 0xffffffff8058d2aa = arpresolve+0x9a/frame 0xfffffe20220c74b0 ether_output() at 0xffffffff805755e2 = ether_output+0x2f2/frame 0xfffffe20220c7550 ip_output() at 0xffffffff805a4200 = ip_output+0x1390/frame 0xfffffe20220c76b0 tcp_output() at 0xffffffff806149d5 = tcp_output+0x17a5/frame 0xfffffe20220c7850 tcp_usr_disconnect() at 0xffffffff80620094 = tcp_usr_disconnect+0x74/frame 0xfffffe20220c7880 soclose() at 0xffffffff8051c238 = soclose+0x38/frame 0xfffffe20220c78b0 _fdrop() at 0xffffffff8045639a = _fdrop+0x1a/frame 0xfffffe20220c78d0 closef() at 0xffffffff80458a53 = closef+0x1e3/frame 0xfffffe20220c7960 closefp() at 0xffffffff804567ad = closefp+0x7d/frame 0xfffffe20220c79a0 amd64_syscall() at 0xffffffff806e4051 = amd64_syscall+0x2c1/frame 0xfffffe20220c7ab0 Xfast_syscall() at 0xffffffff806cb2bb = Xfast_syscall+0xfb/frame 0xfffffe20220c7ab0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x8019dbeaa, rsp = 0x7fffffffe6a8, rbp = 0x7fffffffe6c0 --- Tracing command nginx pid 1057 tid 101815 td 0xfffff8014dde1500 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe20220c2240 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe20220c2270 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe20220c22b0 __rw_wlock_hard() at 0xffffffff8049c314 = __rw_wlock_hard+0x94/frame 0xfffffe20220c2340 in_lltable_lookup() at 0xffffffff80594823 = in_lltable_lookup+0x83/frame 0xfffffe20220c2360 arpresolve() at 0xffffffff8058d2aa = arpresolve+0x9a/frame 0xfffffe20220c23c0 ether_output() at 0xffffffff805755e2 = ether_output+0x2f2/frame 0xfffffe20220c2460 ip_output() at 0xffffffff805a4200 = ip_output+0x1390/frame 0xfffffe20220c25c0 tcp_output() at 0xffffffff806149d5 = tcp_output+0x17a5/frame 0xfffffe20220c2760 tcp_usr_send() at 0xffffffff8062078b = tcp_usr_send+0x33b/frame 0xfffffe20220c27e0 sosend_generic() at 0xffffffff8051cf36 = sosend_generic+0x436/frame 0xfffffe20220c28a0 soo_write() at 0xffffffff804fd872 = soo_write+0x42/frame 0xfffffe20220c28d0 dofilewrite() at 0xffffffff804f5c97 = dofilewrite+0x87/frame 0xfffffe20220c2920 kern_writev() at 0xffffffff804f5978 = kern_writev+0x68/frame 0xfffffe20220c2970 sys_writev() at 0xffffffff804f5be6 = sys_writev+0x36/frame 0xfffffe20220c29a0 amd64_syscall() at 0xffffffff806e4051 = amd64_syscall+0x2c1/frame 0xfffffe20220c2ab0 Xfast_syscall() at 0xffffffff806cb2bb = Xfast_syscall+0xfb/frame 0xfffffe20220c2ab0 --- syscall (121, FreeBSD ELF64, sys_writev), rip = 0x8019cc6ba, rsp = 0x7fffffffcc98, rbp = 0x7fffffffccd0 --- Tracing command nginx pid 1055 tid 101800 td 0xfffff8014ddfda00 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe2022077240 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe2022077270 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe20220772b0 __rw_wlock_hard() at 0xffffffff8049c314 = __rw_wlock_hard+0x94/frame 0xfffffe2022077340 in_lltable_lookup() at 0xffffffff80594823 = in_lltable_lookup+0x83/frame 0xfffffe2022077360 arpresolve() at 0xffffffff8058d2aa = arpresolve+0x9a/frame 0xfffffe20220773c0 ether_output() at 0xffffffff805755e2 = ether_output+0x2f2/frame 0xfffffe2022077460 ip_output() at 0xffffffff805a4200 = ip_output+0x1390/frame 0xfffffe20220775c0 tcp_output() at 0xffffffff806149d5 = tcp_output+0x17a5/frame 0xfffffe2022077760 tcp_usr_send() at 0xffffffff8062078b = tcp_usr_send+0x33b/frame 0xfffffe20220777e0 sosend_generic() at 0xffffffff8051cf36 = sosend_generic+0x436/frame 0xfffffe20220778a0 soo_write() at 0xffffffff804fd872 = soo_write+0x42/frame 0xfffffe20220778d0 dofilewrite() at 0xffffffff804f5c97 = dofilewrite+0x87/frame 0xfffffe2022077920 kern_writev() at 0xffffffff804f5978 = kern_writev+0x68/frame 0xfffffe2022077970 sys_writev() at 0xffffffff804f5be6 = sys_writev+0x36/frame 0xfffffe20220779a0 amd64_syscall() at 0xffffffff806e4051 = amd64_syscall+0x2c1/frame 0xfffffe2022077ab0 Xfast_syscall() at 0xffffffff806cb2bb = Xfast_syscall+0xfb/frame 0xfffffe2022077ab0 --- syscall (121, FreeBSD ELF64, sys_writev), rip = 0x8019cc6ba, rsp = 0x7fffffffcc58, rbp = 0x7fffffffcc90 --- Tracing command intr pid 12 tid 100015 td 0xfffff8011422b000 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe1f9e1cf7a0 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe1f9e1cf7d0 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe1f9e1cf810 __rw_wlock_hard() at 0xffffffff8049c314 = __rw_wlock_hard+0x94/frame 0xfffffe1f9e1cf8a0 in_losing() at 0xffffffff8059c8e8 = in_losing+0x98/frame 0xfffffe1f9e1cf8c0 tcp_timer_rexmt() at 0xffffffff8061e768 = tcp_timer_rexmt+0x428/frame 0xfffffe1f9e1cf8f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe1f9e1cf9c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe1f9e1cf9e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe1f9e1cfa20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe1f9e1cfa70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe1f9e1cfab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe1f9e1cfab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100016 td 0xfffff8011422aa00 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe1f9e1d47a0 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe1f9e1d47d0 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe1f9e1d4810 __rw_wlock_hard() at 0xffffffff8049c314 = __rw_wlock_hard+0x94/frame 0xfffffe1f9e1d48a0 in_losing() at 0xffffffff8059c8e8 = in_losing+0x98/frame 0xfffffe1f9e1d48c0 tcp_timer_rexmt() at 0xffffffff8061e768 = tcp_timer_rexmt+0x428/frame 0xfffffe1f9e1d48f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe1f9e1d49c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe1f9e1d49e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe1f9e1d4a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe1f9e1d4a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe1f9e1d4ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe1f9e1d4ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100017 td 0xfffff8011422a500 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe1f9e1d9700 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe1f9e1d9730 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe1f9e1d9770 __rw_wlock_hard() at 0xffffffff8049c314 = __rw_wlock_hard+0x94/frame 0xfffffe1f9e1d9800 in_pcbfree() at 0xffffffff8059be10 = in_pcbfree+0x290/frame 0xfffffe1f9e1d9840 tcp_usr_detach() at 0xffffffff8061fff8 = tcp_usr_detach+0x78/frame 0xfffffe1f9e1d9860 sofree() at 0xffffffff8051c0a9 = sofree+0x109/frame 0xfffffe1f9e1d9890 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe1f9e1d98c0 tcp_timer_2msl() at 0xffffffff8061dda3 = tcp_timer_2msl+0x1f3/frame 0xfffffe1f9e1d98f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe1f9e1d99c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe1f9e1d99e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe1f9e1d9a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe1f9e1d9a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe1f9e1d9ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe1f9e1d9ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100018 td 0xfffff8011422a000 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe1f9e1de7c0 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe1f9e1de7f0 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe1f9e1de830 __rw_wlock_hard() at 0xffffffff8049c314 = __rw_wlock_hard+0x94/frame 0xfffffe1f9e1de8c0 tcp_timer_delack() at 0xffffffff8061db1c = tcp_timer_delack+0x2c/frame 0xfffffe1f9e1de8f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe1f9e1de9c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe1f9e1de9e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe1f9e1dea20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe1f9e1dea70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe1f9e1deab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe1f9e1deab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100019 td 0xfffff8011424da00 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe1f9e1e37a0 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe1f9e1e37d0 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe1f9e1e3810 __rw_wlock_hard() at 0xffffffff8049c314 = __rw_wlock_hard+0x94/frame 0xfffffe1f9e1e38a0 in_losing() at 0xffffffff8059c8e8 = in_losing+0x98/frame 0xfffffe1f9e1e38c0 tcp_timer_rexmt() at 0xffffffff8061e768 = tcp_timer_rexmt+0x428/frame 0xfffffe1f9e1e38f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe1f9e1e39c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe1f9e1e39e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe1f9e1e3a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe1f9e1e3a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe1f9e1e3ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe1f9e1e3ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100020 td 0xfffff8011424d500 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe1f9e1e87a0 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe1f9e1e87d0 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe1f9e1e8810 __rw_wlock_hard() at 0xffffffff8049c314 = __rw_wlock_hard+0x94/frame 0xfffffe1f9e1e88a0 in_losing() at 0xffffffff8059c8e8 = in_losing+0x98/frame 0xfffffe1f9e1e88c0 tcp_timer_rexmt() at 0xffffffff8061e768 = tcp_timer_rexmt+0x428/frame 0xfffffe1f9e1e88f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe1f9e1e89c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe1f9e1e89e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe1f9e1e8a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe1f9e1e8a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe1f9e1e8ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe1f9e1e8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100021 td 0xfffff8011424d000 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe1f9e1ed3a0 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe1f9e1ed3d0 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe1f9e1ed410 __rw_wlock_hard() at 0xffffffff8049c314 = __rw_wlock_hard+0x94/frame 0xfffffe1f9e1ed4a0 in_lltable_lookup() at 0xffffffff80594823 = in_lltable_lookup+0x83/frame 0xfffffe1f9e1ed4c0 arpresolve() at 0xffffffff8058d2aa = arpresolve+0x9a/frame 0xfffffe1f9e1ed520 ether_output() at 0xffffffff805755e2 = ether_output+0x2f2/frame 0xfffffe1f9e1ed5c0 ip_output() at 0xffffffff805a4200 = ip_output+0x1390/frame 0xfffffe1f9e1ed720 tcp_output() at 0xffffffff806149d5 = tcp_output+0x17a5/frame 0xfffffe1f9e1ed8c0 tcp_timer_delack() at 0xffffffff8061db8f = tcp_timer_delack+0x9f/frame 0xfffffe1f9e1ed8f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe1f9e1ed9c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe1f9e1ed9e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe1f9e1eda20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe1f9e1eda70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe1f9e1edab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe1f9e1edab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100022 td 0xfffff8011424ca00 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe1f9e1f2700 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe1f9e1f2730 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe1f9e1f2770 __rw_wlock_hard() at 0xffffffff8049c314 = __rw_wlock_hard+0x94/frame 0xfffffe1f9e1f2800 in_pcbfree() at 0xffffffff8059be10 = in_pcbfree+0x290/frame 0xfffffe1f9e1f2840 tcp_usr_detach() at 0xffffffff8061fff8 = tcp_usr_detach+0x78/frame 0xfffffe1f9e1f2860 sofree() at 0xffffffff8051c0a9 = sofree+0x109/frame 0xfffffe1f9e1f2890 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe1f9e1f28c0 tcp_timer_2msl() at 0xffffffff8061dda3 = tcp_timer_2msl+0x1f3/frame 0xfffffe1f9e1f28f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe1f9e1f29c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe1f9e1f29e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe1f9e1f2a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe1f9e1f2a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe1f9e1f2ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe1f9e1f2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100023 td 0xfffff8011424c500 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe1f9e1f7360 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe1f9e1f7390 critical_exit() at 0xffffffff804a6bee = critical_exit+0x7e/frame 0xfffffe1f9e1f73b0 ipi_bitmap_handler() at 0xffffffff80775629 = ipi_bitmap_handler+0x79/frame 0xfffffe1f9e1f73f0 Xipi_intr_bitmap_handler() at 0xffffffff806cc15e = Xipi_intr_bitmap_handler+0x8e/frame 0xfffffe1f9e1f73f0 --- interrupt, rip = 0xffffffff806e16c4, rsp = 0xfffffe1f9e1f74c0, rbp = 0xfffffe1f9e1f74c0 --- bcopy() at 0xffffffff806e16c4 = bcopy+0x24/frame 0xfffffe1f9e1f74c0 arpresolve() at 0xffffffff8058d381 = arpresolve+0x171/frame 0xfffffe1f9e1f7520 ether_output() at 0xffffffff805755e2 = ether_output+0x2f2/frame 0xfffffe1f9e1f75c0 ip_output() at 0xffffffff805a4200 = ip_output+0x1390/frame 0xfffffe1f9e1f7720 tcp_output() at 0xffffffff806149d5 = tcp_output+0x17a5/frame 0xfffffe1f9e1f78c0 tcp_timer_rexmt() at 0xffffffff8061e7de = tcp_timer_rexmt+0x49e/frame 0xfffffe1f9e1f78f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe1f9e1f79c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe1f9e1f79e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe1f9e1f7a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe1f9e1f7a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe1f9e1f7ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe1f9e1f7ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100024 td 0xfffff8011424c000 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe1f9e1fc3a0 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe1f9e1fc3d0 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe1f9e1fc410 __rw_wlock_hard() at 0xffffffff8049c314 = __rw_wlock_hard+0x94/frame 0xfffffe1f9e1fc4a0 in_lltable_lookup() at 0xffffffff80594823 = in_lltable_lookup+0x83/frame 0xfffffe1f9e1fc4c0 arpresolve() at 0xffffffff8058d2aa = arpresolve+0x9a/frame 0xfffffe1f9e1fc520 ether_output() at 0xffffffff805755e2 = ether_output+0x2f2/frame 0xfffffe1f9e1fc5c0 ip_output() at 0xffffffff805a4200 = ip_output+0x1390/frame 0xfffffe1f9e1fc720 tcp_output() at 0xffffffff806149d5 = tcp_output+0x17a5/frame 0xfffffe1f9e1fc8c0 tcp_timer_delack() at 0xffffffff8061db8f = tcp_timer_delack+0x9f/frame 0xfffffe1f9e1fc8f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe1f9e1fc9c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe1f9e1fc9e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe1f9e1fca20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe1f9e1fca70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe1f9e1fcab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe1f9e1fcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100025 td 0xfffff8011424ba00 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe0000382700 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe0000382730 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe0000382770 __rw_wlock_hard() at 0xffffffff8049c314 = __rw_wlock_hard+0x94/frame 0xfffffe0000382800 in_pcbfree() at 0xffffffff8059be10 = in_pcbfree+0x290/frame 0xfffffe0000382840 tcp_usr_detach() at 0xffffffff8061fff8 = tcp_usr_detach+0x78/frame 0xfffffe0000382860 sofree() at 0xffffffff8051c0a9 = sofree+0x109/frame 0xfffffe0000382890 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe00003828c0 tcp_timer_2msl() at 0xffffffff8061dda3 = tcp_timer_2msl+0x1f3/frame 0xfffffe00003828f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe00003829c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe00003829e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe0000382a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe0000382a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe0000382ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe0000382ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100026 td 0xfffff8011424b500 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe0000387700 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe0000387730 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe0000387770 __rw_wlock_hard() at 0xffffffff8049c314 = __rw_wlock_hard+0x94/frame 0xfffffe0000387800 in_pcbfree() at 0xffffffff8059be10 = in_pcbfree+0x290/frame 0xfffffe0000387840 tcp_usr_detach() at 0xffffffff8061fff8 = tcp_usr_detach+0x78/frame 0xfffffe0000387860 sofree() at 0xffffffff8051c0a9 = sofree+0x109/frame 0xfffffe0000387890 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe00003878c0 tcp_timer_2msl() at 0xffffffff8061dda3 = tcp_timer_2msl+0x1f3/frame 0xfffffe00003878f0 softclock_call_cc() at 0xffffffff804b4ca9 = softclock_call_cc+0x179/frame 0xfffffe00003879c0 softclock() at 0xffffffff804b5034 = softclock+0x44/frame 0xfffffe00003879e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe0000387a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe0000387a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe0000387ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe0000387ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100103 td 0xfffff8012508ea00 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe2020ea82d0 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe2020ea8300 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe2020ea8340 __rw_wlock_hard() at 0xffffffff8049c314 = __rw_wlock_hard+0x94/frame 0xfffffe2020ea83d0 in_pcbfree() at 0xffffffff8059be10 = in_pcbfree+0x290/frame 0xfffffe2020ea8410 tcp_usr_detach() at 0xffffffff8061fff8 = tcp_usr_detach+0x78/frame 0xfffffe2020ea8430 sofree() at 0xffffffff8051c0a9 = sofree+0x109/frame 0xfffffe2020ea8460 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe2020ea8490 tcp_do_segment() at 0xffffffff80610226 = tcp_do_segment+0x1666/frame 0xfffffe2020ea8590 tcp_input() at 0xffffffff8060e17c = tcp_input+0xe1c/frame 0xfffffe2020ea86e0 ip_input() at 0xffffffff805a087f = ip_input+0x15f/frame 0xfffffe2020ea8740 netisr_dispatch_src() at 0xffffffff80583db5 = netisr_dispatch_src+0xa5/frame 0xfffffe2020ea87a0 ether_demux() at 0xffffffff80575b3a = ether_demux+0x12a/frame 0xfffffe2020ea87d0 ether_nh_input() at 0xffffffff80576792 = ether_nh_input+0x322/frame 0xfffffe2020ea8830 netisr_dispatch_src() at 0xffffffff80583db5 = netisr_dispatch_src+0xa5/frame 0xfffffe2020ea8890 ether_input() at 0xffffffff80575db6 = ether_input+0x26/frame 0xfffffe2020ea88b0 ixgbe_rxeof() at 0xffffffff813df36b = ixgbe_rxeof+0x7ab/frame 0xfffffe2020ea8990 ixgbe_msix_que() at 0xffffffff813da57c = ixgbe_msix_que+0x8c/frame 0xfffffe2020ea89e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe2020ea8a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe2020ea8a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe2020ea8ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe2020ea8ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100105 td 0xfffff8012508e000 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe2020eb2290 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe2020eb22c0 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe2020eb2300 __rw_wlock_hard() at 0xffffffff8049c314 = __rw_wlock_hard+0x94/frame 0xfffffe2020eb2390 in_pcbfree() at 0xffffffff8059be10 = in_pcbfree+0x290/frame 0xfffffe2020eb23d0 tcp_usr_detach() at 0xffffffff8061fff8 = tcp_usr_detach+0x78/frame 0xfffffe2020eb23f0 sofree() at 0xffffffff8051c0a9 = sofree+0x109/frame 0xfffffe2020eb2420 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe2020eb2450 tcp_twstart() at 0xffffffff8061f0e7 = tcp_twstart+0x2b7/frame 0xfffffe2020eb2490 tcp_do_segment() at 0xffffffff80610bdf = tcp_do_segment+0x201f/frame 0xfffffe2020eb2590 tcp_input() at 0xffffffff8060e17c = tcp_input+0xe1c/frame 0xfffffe2020eb26e0 ip_input() at 0xffffffff805a087f = ip_input+0x15f/frame 0xfffffe2020eb2740 netisr_dispatch_src() at 0xffffffff80583db5 = netisr_dispatch_src+0xa5/frame 0xfffffe2020eb27a0 ether_demux() at 0xffffffff80575b3a = ether_demux+0x12a/frame 0xfffffe2020eb27d0 ether_nh_input() at 0xffffffff80576792 = ether_nh_input+0x322/frame 0xfffffe2020eb2830 netisr_dispatch_src() at 0xffffffff80583db5 = netisr_dispatch_src+0xa5/frame 0xfffffe2020eb2890 ether_input() at 0xffffffff80575db6 = ether_input+0x26/frame 0xfffffe2020eb28b0 ixgbe_rxeof() at 0xffffffff813df36b = ixgbe_rxeof+0x7ab/frame 0xfffffe2020eb2990 ixgbe_msix_que() at 0xffffffff813da57c = ixgbe_msix_que+0x8c/frame 0xfffffe2020eb29e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe2020eb2a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe2020eb2a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe2020eb2ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe2020eb2ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100107 td 0xfffff8012508d500 cpustop_handler() at 0xffffffff80775998 = cpustop_handler+0x28/frame 0xfffffe1f9e16dcf0 ipi_nmi_handler() at 0xffffffff8077595a = ipi_nmi_handler+0x4a/frame 0xfffffe1f9e16dd10 trap() at 0xffffffff806e2e4a = trap+0x3a/frame 0xfffffe1f9e16df20 nmi_calltrap() at 0xffffffff806cb413 = nmi_calltrap+0x8/frame 0xfffffe1f9e16df20 --- trap 0x13, rip = 0xffffffff8049c71d, rsp = 0xfffffe2020ebc410, rbp = 0xfffffe2020ebc420 --- __rw_try_wlock() at 0xffffffff8049c71d = __rw_try_wlock+0x4d/frame 0xfffffe2020ebc420 tcp_tw_2msl_scan() at 0xffffffff8061f1f6 = tcp_tw_2msl_scan+0xc6/frame 0xfffffe2020ebc450 /usr/src/sys/netinet/tcp_timewait.c:707 tcp_twstart() at 0xffffffff8061eed3 = tcp_twstart+0xa3/frame 0xfffffe2020ebc490 tcp_do_segment() at 0xffffffff80610bdf = tcp_do_segment+0x201f/frame 0xfffffe2020ebc590 tcp_input() at 0xffffffff8060e17c = tcp_input+0xe1c/frame 0xfffffe2020ebc6e0 ip_input() at 0xffffffff805a087f = ip_input+0x15f/frame 0xfffffe2020ebc740 netisr_dispatch_src() at 0xffffffff80583db5 = netisr_dispatch_src+0xa5/frame 0xfffffe2020ebc7a0 ether_demux() at 0xffffffff80575b3a = ether_demux+0x12a/frame 0xfffffe2020ebc7d0 ether_nh_input() at 0xffffffff80576792 = ether_nh_input+0x322/frame 0xfffffe2020ebc830 netisr_dispatch_src() at 0xffffffff80583db5 = netisr_dispatch_src+0xa5/frame 0xfffffe2020ebc890 ether_input() at 0xffffffff80575db6 = ether_input+0x26/frame 0xfffffe2020ebc8b0 ixgbe_rxeof() at 0xffffffff813df36b = ixgbe_rxeof+0x7ab/frame 0xfffffe2020ebc990 ixgbe_msix_que() at 0xffffffff813da57c = ixgbe_msix_que+0x8c/frame 0xfffffe2020ebc9e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe2020ebca20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe2020ebca70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe2020ebcab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe2020ebcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command intr pid 12 tid 100111 td 0xfffff801250a2000 sched_switch() at 0xffffffff804c956d = sched_switch+0x6ad/frame 0xfffffe2020f302d0 mi_switch() at 0xffffffff804a8d92 = mi_switch+0xd2/frame 0xfffffe2020f30300 turnstile_wait() at 0xffffffff804ef177 = turnstile_wait+0x2a7/frame 0xfffffe2020f30340 __rw_wlock_hard() at 0xffffffff8049c314 = __rw_wlock_hard+0x94/frame 0xfffffe2020f303d0 in_pcbfree() at 0xffffffff8059be10 = in_pcbfree+0x290/frame 0xfffffe2020f30410 tcp_usr_detach() at 0xffffffff8061fff8 = tcp_usr_detach+0x78/frame 0xfffffe2020f30430 sofree() at 0xffffffff8051c0a9 = sofree+0x109/frame 0xfffffe2020f30460 tcp_close() at 0xffffffff80618150 = tcp_close+0xd0/frame 0xfffffe2020f30490 tcp_do_segment() at 0xffffffff8060f6cb = tcp_do_segment+0xb0b/frame 0xfffffe2020f30590 tcp_input() at 0xffffffff8060e17c = tcp_input+0xe1c/frame 0xfffffe2020f306e0 ip_input() at 0xffffffff805a087f = ip_input+0x15f/frame 0xfffffe2020f30740 netisr_dispatch_src() at 0xffffffff80583db5 = netisr_dispatch_src+0xa5/frame 0xfffffe2020f307a0 ether_demux() at 0xffffffff80575b3a = ether_demux+0x12a/frame 0xfffffe2020f307d0 ether_nh_input() at 0xffffffff80576792 = ether_nh_input+0x322/frame 0xfffffe2020f30830 netisr_dispatch_src() at 0xffffffff80583db5 = netisr_dispatch_src+0xa5/frame 0xfffffe2020f30890 ether_input() at 0xffffffff80575db6 = ether_input+0x26/frame 0xfffffe2020f308b0 ixgbe_rxeof() at 0xffffffff813df36b = ixgbe_rxeof+0x7ab/frame 0xfffffe2020f30990 ixgbe_msix_que() at 0xffffffff813da57c = ixgbe_msix_que+0x8c/frame 0xfffffe2020f309e0 intr_event_execute_handlers() at 0xffffffff8046c605 = intr_event_execute_handlers+0x95/frame 0xfffffe2020f30a20 ithread_loop() at 0xffffffff8046cc26 = ithread_loop+0xa6/frame 0xfffffe2020f30a70 fork_exit() at 0xffffffff8046a211 = fork_exit+0x71/frame 0xfffffe2020f30ab0 fork_trampoline() at 0xffffffff806cb50e = fork_trampoline+0xe/frame 0xfffffe2020f30ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- current version of /usr/src/sys/netinet/tcp_timewait.c 700 inp = tw->tw_inpcb; 701 in_pcbref(inp); 702 TW_RUNLOCK(V_tw_lock); 703 704 //retry: 705 if (INP_INFO_TRY_RLOCK(&V_tcbinfo)) { 706 707 if (INP_TRY_WLOCK(inp)) { 708 tw = intotw(inp); 709 if (in_pcbrele_wlocked(inp)) { 710 KASSERT(tw == NULL, ("%s: held last inp " 711 "reference but tw not NULL", __func__)); 712 INP_INFO_RUNLOCK(&V_tcbinfo); 713 continue; 714 } 715 716 if (tw == NULL) { 717 /* tcp_twclose() has already been called */ 718 INP_WUNLOCK(inp); 719 INP_INFO_RUNLOCK(&V_tcbinfo); 720 continue; 721 } 722 723 tcp_twclose(tw, reuse); 724 INP_INFO_RUNLOCK(&V_tcbinfo); 725 if (reuse) 726 return tw; 727 } else { 728 INP_INFO_RUNLOCK(&V_tcbinfo); 729 goto retry; 730 } 731 } else { 732 /* INP_INFO lock is busy, continue later. */ 733 if (INP_TRY_WLOCK(inp)) { 734 if (!in_pcbrele_wlocked(inp)) 735 INP_WUNLOCK(inp); 736 break; 737 } else { 738 goto retry; 739 } 740 } 741 } 742 retry: 743 return NULL; 744 } From owner-freebsd-stable@freebsd.org Wed Sep 28 19:18:59 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BFAECC00099 for ; Wed, 28 Sep 2016 19:18:59 +0000 (UTC) (envelope-from bounce@ims4.isendservice.com.br) Received: from ims4.isendservice.com.br (ims4.isendservice.com.br [54.232.89.214]) by mx1.freebsd.org (Postfix) with ESMTP id 63C8C1A02 for ; Wed, 28 Sep 2016 19:18:59 +0000 (UTC) (envelope-from bounce@ims4.isendservice.com.br) Received: from localhost (localhost [127.0.0.1]) by ims4.isendservice.com.br (Postfix) with ESMTP id D064A447D0 for ; Wed, 28 Sep 2016 15:51:36 -0300 (BRT) Date: Wed, 28 Sep 2016 15:51:36 -0300 (BRT) From: FIERGS | FATEC - Faculdade SENAI de Tecnologia Reply-To: atendimento@fiergs.org.br To: freebsd-stable@freebsd.org Message-ID: <1113453224.409637.1475088696741.JavaMail.root@ims4> Subject: =?ISO-8859-1?Q?Fa=E7a_sua_P=F3s-Gradua=E7=E3o_na_Fa?= =?ISO-8859-1?Q?culdade_SENAI_=96_=DAltimos_dias?= bounce-key: <2420-22522874-1197302> MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Sep 2016 19:18:59 -0000 From owner-freebsd-stable@freebsd.org Wed Sep 28 22:39:05 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 87935C01BF4; Wed, 28 Sep 2016 22:39:05 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 7BAB5175A; Wed, 28 Sep 2016 22:39:05 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by freefall.freebsd.org (Postfix) with ESMTP id C500719C9; Wed, 28 Sep 2016 22:39:03 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Date: Wed, 28 Sep 2016 22:39:02 +0000 From: Glen Barber To: freebsd-current@FreeBSD.org, freebsd-stable@FreeBSD.org Cc: re@FreeBSD.org, freebsd-announce@FreeBSD.org Subject: [HEADS-UP] 11.0-RELEASE status update Message-ID: <20160928223902.GI88792@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; x-action=pgp-signed X-Operating-System: FreeBSD 11.0-CURRENT amd64 X-SCUD-Definition: Sudden Completely Unexpected Dataloss X-SULE-Definition: Sudden Unexpected Learning Event X-PEKBAC-Definition: Problem Exists, Keyboard Between Admin/Computer User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Sep 2016 22:39:05 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Dear FreeBSD Community: Although the FreeBSD 11.0-RELEASE has not yet been officially announced, many have found images on the Project FTP mirrors. However, please be aware the final 11.0-RELEASE will be rebuilt and republished on the Project mirrors as a result of a few last-minute security fixes we feel are imperative to include in the final release. FreeBSD users already running 11.0-RELEASE will be given instructions on how to safely upgrade systems to the 11.0-RELEASE-p1 in the final announcement email. Those building from source code can obtain the latest security updates from the releng/11.0 branch in Subversion: svn://svn.freebsd.org/base/releng/11.0 As the FreeBSD Project strives to provide the best possible product, the Release Engineering team decided to build an updated release to include the fixes. At present, we expect to have the final release available Wednesday, October 3rd. If you have not yet downloaded 11.0-RELEASE, please wait for the official release announcement. Thank you in advance for your patience waiting for 11.0-RELEASE, and of course for understanding the reasons behind the updated release. Glen On behalf of: re@ -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJX7EaGAAoJEAMUWKVHj+KTtrgP/iaIozjqDQ2poH1i7J+BewmE wov+vRcfmGmBvCFLxGPWDsXsYWYw8HCUNrloBesNlUZNe7BoFKliVrBp7KAN5YRE R+l9AQU8u7UhYoKbM1epB28nDYdLH/veKMpkhyEr2mPglmRDJoJa1JL3xcnRXDj+ yFeeCH5He/jH/ILiO8ChfY8e3aA+K/qMOSicVENW5M2kGs/q0m/i5UZK2LZ+gT7R /eMl0USfW2B5LebHViv3a6GRArfTzBYZKYdoxXH7vUZ1zgb9CcEPfhYBxu41RMe3 I+HquvqzWKPNwG3GhwqPmKfwQt4PHlATkZwddGosIgSmUZRhhD4eR0DWdXD6k/oS iSi7QR8lef6ALcVTjt65JNqzPF/9eUJsZikcI0Ov6I0TkV2yzAGnUNneZQ6+22AS //ZhqWkIu7w1hePJ+Af+SZJDzVdUWzVNiAyMmSFkfW3mFaidyhjR0OULnquG6kSS kdPOdl/RwJzfP3wkFjt56I8YTyk7YQdwNEcEBQUlXlyZOC/NvUH5eebPJ1Va5UDV q0FHFaYiATKvQyZUO3Ne9eLzBdYQhmaPSrvTGXrZw53hgShIBOEnwkJYiEGgySL3 vCDro397boLkRL89HUXwuCFurZp/7g/V+I3w4X45y2/GpC/w7isPX/5YYJloETnR VLGBedKpJbR/5LUJH8Bw =t8IC -----END PGP SIGNATURE----- From owner-freebsd-stable@freebsd.org Wed Sep 28 23:35:45 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F3AE0C01472; Wed, 28 Sep 2016 23:35:44 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id E80CE1F38; Wed, 28 Sep 2016 23:35:44 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by freefall.freebsd.org (Postfix) with ESMTP id 434B71ADE; Wed, 28 Sep 2016 23:35:43 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Date: Wed, 28 Sep 2016 23:35:41 +0000 From: Glen Barber To: freebsd-current@FreeBSD.org, freebsd-stable@FreeBSD.org Cc: re@FreeBSD.org, freebsd-announce@FreeBSD.org Subject: [REVISED] [HEADS-UP] 11.0-RELEASE status update Message-ID: <20160928233541.GJ88792@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; x-action=pgp-signed X-Operating-System: FreeBSD 11.0-CURRENT amd64 X-SCUD-Definition: Sudden Completely Unexpected Dataloss X-SULE-Definition: Sudden Unexpected Learning Event X-PEKBAC-Definition: Problem Exists, Keyboard Between Admin/Computer User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Sep 2016 23:35:45 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Dear FreeBSD Community: [Corrected the date.] Although the FreeBSD 11.0-RELEASE has not yet been officially announced, many have found images on the Project FTP mirrors. However, please be aware the final 11.0-RELEASE will be rebuilt and republished on the Project mirrors as a result of a few last-minute security fixes we feel are imperative to include in the final release. FreeBSD users already running 11.0-RELEASE will be given instructions on how to safely upgrade systems to the 11.0-RELEASE-p1 in the final announcement email. Those building from source code can obtain the latest security updates from the releng/11.0 branch in Subversion: svn://svn.freebsd.org/base/releng/11.0 As the FreeBSD Project strives to provide the best possible product, the Release Engineering team decided to build an updated release to include the fixes. At present, we expect to have the final release available Wednesday, October 5th. If you have not yet downloaded 11.0-RELEASE, please wait for the official release announcement. Thank you in advance for your patience waiting for 11.0-RELEASE, and of course for understanding the reasons behind the updated release. Glen On behalf of: re@ -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJX7FPNAAoJEAMUWKVHj+KT2joP/0/0AOYfTbFUgZeEUXlmdfew 7nS31bQrBCXi7dgPicfSavdvDfqi4sgiw2/+HY3MxpfLWFJ/WNGveiwryGSiapkA V3BJ9MCOZb3ZZTbp0JlwbRk1NyGg4ur0S4L6zD+MXuHE95Kts3m/ON8CiGtNUE+1 rzE7Yr10tsU2Zu1Bvtv8rJa9SfLCln8k2FXtG0pxVWO+cK2xo6v84bjOJdExrB4t eXYoMSoxIyZd1Kv2nLbL1mG7RrLQFVm4TrurMwALI39hVr+IWIvElmo6wndDhTly XE8aMtpgUMp9b4PrQM+BgFVooR4ihFl0cslHfDuBGuiVJMQoa63agUfGAkclc9Na nwiJiwcQStOdHcRAnZNBms9DTeNXDD0whq30JoY45kFRI74wjjqP8oNUCUWEd6e8 n1puD2Zr2fqX0NziwtRg3Hy0EHM+9rQTEDtyHCG05sqTncyU7p6tkd49FfndXqaq h/JkHTP1iyQYsq07GZzyhPA04e/i3N8Djwm+WoRgOlSrItJiPQ/FuqKV0cSERvPR XZm3DPPRt04aOFe7XGrl2IHi+J6LZ5uwYEXiHFb+fPQMuROZ+IJC0Wu56HI2LHGL f5wyPiNE1NJIeYLzIgk3UUrENaylsW4/NsgLFj6TW//24ekF2NR+Nk8u7mvoJuXq vcLDdPW7mReqF13WLzh/ =RcJK -----END PGP SIGNATURE----- From owner-freebsd-stable@freebsd.org Thu Sep 29 05:35:34 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6DCB2C018D0 for ; Thu, 29 Sep 2016 05:35:34 +0000 (UTC) (envelope-from sa.inbox@gmail.com) Received: from mail-oi0-x232.google.com (mail-oi0-x232.google.com [IPv6:2607:f8b0:4003:c06::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 36A921AB7 for ; Thu, 29 Sep 2016 05:35:34 +0000 (UTC) (envelope-from sa.inbox@gmail.com) Received: by mail-oi0-x232.google.com with SMTP id r126so79513999oib.0 for ; Wed, 28 Sep 2016 22:35:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=hZnMkOuXG3LjdD66ha6NoPMpEXM293CqqcoZaiGjkuI=; b=BJaDNqA2DBGUMXfS/QSiV58BIdII5qwYfCliBwDc0GB7aM/LDrPJsgZ6YpxWovg/SA 921gbeg2hB+mWIE8sr/vry9DAeuuU1986kudJa8m1TaZsYz6oVhFagGcKg/FHD93enFP UliHFDIWcHqD34gDBFGpGjSZnfDCcA47ETuzeApYZlD6N0w/rzGLR7oCRsjisIpj1SF/ tIwE9lnTvPCF2V5wtgmmMq94Qwxg+3qTKPL8QcOyuTbwjjnVj0jdH9NrQ4dylGbT2gOk IHxZshx3z99mMt/Wp8ke04TNgIhwb/QiJ5+yOG41a4AMMzBYHNXzlj3XasHCCo7KMFXi +ETw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=hZnMkOuXG3LjdD66ha6NoPMpEXM293CqqcoZaiGjkuI=; b=GTQGf/XR6ykCvdxngwn3IhIKn7S7TvX8AgMhWXH6W1mjMv+hpBB7xbmPvepnOL6TWZ F6Q5KetW/sdVfhUx7TULoTpR1cxtmSfo66onOPt/hcoSNbQGSaVYwzvwyS5kaMyqzgGz UkmFQolpW4Vew42KyXEQandgJ3T8PB2aOGjVaJaHKB1xlTvHf+2ZcOY/McmL18fhkPVp 3PHqJaBScR8Wiln+WCsK5qMbqFUJLXWVwsWhZgqOe/zSlBCfo+kCqA40kC/5LQsYVCpA WbBaUEnLEsMwncKAusyznf2ZSOTvapfdgr6rt6eEzoINpm0FDvz5AR/PE03l/txKRNvT gpuw== X-Gm-Message-State: AE9vXwMF2GwULjn7j522l/tNnLq2EqXdAR6aBPGhiYTPZthUoc7P7hXMklj2RuShYN2yJIl8W+eirjFrJHuI8Q== X-Received: by 10.202.213.9 with SMTP id m9mr40615541oig.63.1475127333279; Wed, 28 Sep 2016 22:35:33 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.37.130 with HTTP; Wed, 28 Sep 2016 22:35:13 -0700 (PDT) From: Alexander Shitov Date: Thu, 29 Sep 2016 08:35:13 +0300 Message-ID: Subject: RE: [REVISED] [HEADS-UP] 11.0-RELEASE status update To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Sep 2016 05:35:34 -0000 Will final 11.0-RELEASE include fixes related to broken Hyper-V compatibility? 11.0-RC2, 11.0-RC3 and non-final 11.0-RELEASE failed to install on Hyper-V 2012 R2 with all current updates. storvsc0: on vmbus0 (probe0:blkvsc0:0:0:0): storvsc scsi_status = 2 (probe0:blkvsc0:0:1:1): invalid LUN 1 ... (probe0:blkvsc0:0:0:1): invalid LUN 1 da0 at blkvsc0 bus 0 scbus1 target 0 lun 0 da0: da0 at blkvsc0 bus 0 scbus1 target 0 lun 0 da0: detached Problem documented in PR https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=212721 --- Regards, Alexander From owner-freebsd-stable@freebsd.org Thu Sep 29 07:43:10 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CC626C01C6F for ; Thu, 29 Sep 2016 07:43:10 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from cu1176c.smtpx.saremail.com (cu1176c.smtpx.saremail.com [195.16.148.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D3C163A0; Thu, 29 Sep 2016 07:43:09 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from [172.16.8.36] (izaro.sarenet.es [192.148.167.11]) by proxypop02.sare.net (Postfix) with ESMTPSA id 342F09DC880; Thu, 29 Sep 2016 09:43:01 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.0 \(3226\)) Subject: Re: Status of PCIe Hotplug? From: Borja Marcos In-Reply-To: Date: Thu, 29 Sep 2016 09:43:00 +0200 Cc: Eric van Gyzen , Jan Henrik Sylvester , FreeBSD-STABLE Mailing List Content-Transfer-Encoding: quoted-printable Message-Id: <94BCCD03-F27E-4038-AE58-5240EE96FE70@sarenet.es> References: <791ee95b-ab4e-f07e-6b8d-0e0b6c49ceb0@janh.de> <58f84457-d775-3023-5727-a326bc8df722@FreeBSD.org> To: Borja Marcos X-Mailer: Apple Mail (2.3226) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Sep 2016 07:43:10 -0000 > On 28 Sep 2016, at 11:41, Borja Marcos wrote: > Of course. Unfortunately, due to poor timing and a DOA server last = month, this > server is in a countdown to get into production tomorrow running = Solaris, but I=E2=80=99ll try > to get whatever I can today. I managed to do some tests. First one, detaching the nvme9 controller. # devctl detach nvme9 Upon doing this, the console displays: Sep 29 08:26:01 pruebanvme2 = kernel: nvme9: detached However, device nvd9 (which is linked to nvme9ns1) does not disappear. I = am not sure that the full hot plug dialogue with the hardware has been performed, either. The = green =E2=80=9Cok to extract=E2=80=9D LED does not turn on unless I disable the associated pcib device. I guess we need a specific =E2=80=9Cdevctl poweroff=E2=80=9D command to = achieve that? /dev/nvd9 is still present, but as a kind of orphan. Whenever I try to = access it I trigger a panic. ---------------------------- pruebanvme2 dumped core - see /var/crash/vmcore.0 Thu Sep 29 08:33:08 CEST 2016 FreeBSD pruebanvme2 11.0-RELEASE FreeBSD 11.0-RELEASE #0: Wed Sep 28 = 12:51:37 CEST 2016 root@pruebanvme2:/usr/obj/usr/src/sys/DEBUG = amd64 panic: vm_fault: fault on nofault entry, addr: fffffe000209b000 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you = are welcome to change it and/or distribute copies of it under certain = conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for = details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: vm_fault: fault on nofault entry, addr: fffffe000209b000 cpuid =3D 37 KDB: stack backtrace: #0 0xffffffff80b24887 at kdb_backtrace+0x67 #1 0xffffffff80ad9bf2 at vpanic+0x182 #2 0xffffffff80ad9a63 at panic+0x43 #3 0xffffffff80e12f01 at vm_fault_hold+0x2721 #4 0xffffffff80e10798 at vm_fault+0x78 #5 0xffffffff80fa1e79 at trap_pfault+0xf9 #6 0xffffffff80fa150c at trap+0x26c #7 0xffffffff80f84a41 at calltrap+0x8 #8 0xffffffff81081e4a at nvd_strategy+0x4a #9 0xffffffff80a11ede at g_disk_start+0x34e #10 0xffffffff80a151d7 at g_io_request+0x4a7 #11 0xffffffff80ab8d45 at physio+0x665 #12 0xffffffff809867b7 at devfs_write_f+0xe7 #13 0xffffffff80b41ae7 at dofilewrite+0x87 #14 0xffffffff80b417c8 at kern_writev+0x68 #15 0xffffffff80b41754 at sys_write+0x84 #16 0xffffffff80fa26ce at amd64_syscall+0x4ce #17 0xffffffff80f84d2b at Xfast_syscall+0xfb Uptime: 19h27m51s Dumping 4772 out of 130928 = MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Reading symbols from /boot/kernel/ums.ko...Reading symbols from = /usr/lib/debug//boot/kernel/ums.ko.debug...done. done. Loaded symbols for /boot/kernel/ums.ko #0 doadump (textdump=3D) at pcpu.h:221 221 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=3D) at pcpu.h:221 #1 0xffffffff80ad9679 in kern_reboot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:366 #2 0xffffffff80ad9c2b in vpanic (fmt=3D,=20 ap=3D) at /usr/src/sys/kern/kern_shutdown.c:759 #3 0xffffffff80ad9a63 in panic (fmt=3D0x0) at /usr/src/sys/kern/kern_shutdown.c:690 #4 0xffffffff80e12f01 in vm_fault_hold (map=3D,=20 vaddr=3D, fault_type=3D,=20= fault_flags=3D, m_hold=3D) at /usr/src/sys/vm/vm_fault.c:330 #5 0xffffffff80e10798 in vm_fault (map=3D0xfffff80100000000,=20 vaddr=3D, fault_type=3D1 '\001',=20 fault_flags=3D) at = /usr/src/sys/vm/vm_fault.c:273 #6 0xffffffff80fa1e79 in trap_pfault (frame=3D0xfffffe201bec2560, = usermode=3D0) at /usr/src/sys/amd64/amd64/trap.c:741 #7 0xffffffff80fa150c in trap (frame=3D0xfffffe201bec2560) at /usr/src/sys/amd64/amd64/trap.c:442 #8 0xffffffff80f84a41 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:236 #9 0xffffffff81085ab8 in nvme_ns_bio_process (ns=3D0xfffffe000209a310,=20= bp=3D0xfffff8019e4cf000, cb_fn=3D0xffffffff81082160 ) at /usr/src/sys/dev/nvme/nvme_ns.c:436 #10 0xffffffff81081e4a in nvd_strategy (bp=3D0xfffff8019e4cf000) at /usr/src/sys/dev/nvd/nvd.c:175 #11 0xffffffff80a11ede in g_disk_start (bp=3D) at /usr/src/sys/geom/geom_disk.c:461 #12 0xffffffff80a151d7 in g_io_request (bp=3D,=20 cp=3D) at /usr/src/sys/geom/geom_io.c:626 #13 0xffffffff80ab8d45 in physio (dev=3D,=20 uio=3D, ioflag=3D) at /usr/src/sys/kern/kern_physio.c:191 #14 0xffffffff809867b7 in devfs_write_f (fp=3D,=20 uio=3D, cred=3D,=20 flags=3D, td=3D0xfffff80125f0f500) at /usr/src/sys/fs/devfs/devfs_vnops.c:1757 #15 0xffffffff80b41ae7 in dofilewrite (td=3D0xfffff80125f0f500, fd=3D4,=20= fp=3D0xfffff80125dd5b40, auio=3D0xfffffe201bec2960,=20 offset=3D, flags=3D0) at file.h:311 #16 0xffffffff80b417c8 in kern_writev (td=3D0xfffff80125f0f500, fd=3D4,=20= auio=3D0xfffffe201bec2960) at /usr/src/sys/kern/sys_generic.c:506 #17 0xffffffff80b41754 in sys_write (td=3D0xfffffe000209a310,=20 uap=3D) at /usr/src/sys/kern/sys_generic.c:419 #18 0xffffffff80fa26ce in amd64_syscall (td=3D, = traced=3D0) at subr_syscall.c:135 #19 0xffffffff80f84d2b in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:396 #20 0x000000080097f71a in ?? () =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94 After rebooting I tried to detach and attach nvme9. When reattached, it = shows up in the console and /var/log/messages, and a new nvd (nvd10) is created # devctl attach pci0:134:0:0 Sep 29 08:57:41 pruebanvme2 kernel: nvme9: mem = 0xfba10000-0xfba13fff irq 59 at device 0.0 numa-domain 1 on pci20 Sep 29 08:57:49 pruebanvme2 kernel: nvd10: NVMe = namespace Sep 29 08:57:49 pruebanvme2 kernel: nvd10: 1907729MB (488378646 4096 = byte sectors) The newly created nvd10 device works, # dd if=3D/dev/zero of=3D/dev/nvd10 bs=3D1m count=3D10 10+0 records in 10+0 records out 10485760 bytes transferred in 0.013463 secs (778874214 bytes/sec) but any attempts to access nvd9 result in a panic, unsurprisingly. So, unless I am wrong, there are two different problems here: 1) The hotplug support is still somewhat incomplete, when doing a detach = it doesn=E2=80=99t complete the handshake with the hardware. Or it shouldn=E2=80=99t when doing a detach = and we need a =E2=80=9Cpoweroff=E2=80=9D command for =E2=80=9Cdevctl=E2=80=9D. 2) The NVMe drivers, in particular, fail to destroy nvd devices = associated to their namespaces. I saw a third problem, specific to NVMe again I guess. Doing this: #devctl detach nvme0 nvmecontrol fails to detect the rest of the NVMe controllers. # nvmecontrol devlist No NVMe controllers found. despite being there none150@pci0:2:0:0: class=3D0x010802 card=3D0x37058086 = chip=3D0x09538086 rev=3D0x01 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 11[50] =3D MSI-X supports 32 messages Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[60] =3D PCI-Express 2 endpoint max data 256(256) FLR NS link x4(x4) speed 8.0(8.0) ASPM disabled(L0s/L1) ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 1 corrected ecap 0002[150] =3D VC 1 max VC0 ecap 0004[180] =3D Power Budgeting 1 ecap 000e[190] =3D ARI 1 ecap 0003[270] =3D Serial 1 55cd2e414d04a4cd ecap 0019[2a0] =3D PCIe Sec 1 lane errors 0 nvme1@pci0:3:0:0: class=3D0x010802 card=3D0x37058086 = chip=3D0x09538086 rev=3D0x01 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 11[50] =3D MSI-X supports 32 messages, enabled Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[60] =3D PCI-Express 2 endpoint max data 256(256) FLR NS link x4(x4) speed 8.0(8.0) ASPM disabled(L0s/L1) ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 1 corrected ecap 0002[150] =3D VC 1 max VC0 ecap 0004[180] =3D Power Budgeting 1 ecap 000e[190] =3D ARI 1 ecap 0003[270] =3D Serial 1 55cd2e414d04ad7b ecap 0019[2a0] =3D PCIe Sec 1 lane errors 0 nvme2@pci0:5:0:0: class=3D0x010802 card=3D0x37058086 = chip=3D0x09538086 rev=3D0x01 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 11[50] =3D MSI-X supports 32 messages, enabled Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[60] =3D PCI-Express 2 endpoint max data 256(256) FLR NS link x4(x4) speed 8.0(8.0) ASPM disabled(L0s/L1) ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 1 corrected ecap 0002[150] =3D VC 1 max VC0 ecap 0004[180] =3D Power Budgeting 1 ecap 000e[190] =3D ARI 1 ecap 0003[270] =3D Serial 1 55cd2e414d04a219 ecap 0019[2a0] =3D PCIe Sec 1 lane errors 0 nvme3@pci0:6:0:0: class=3D0x010802 card=3D0x37058086 = chip=3D0x09538086 rev=3D0x01 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 11[50] =3D MSI-X supports 32 messages, enabled Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[60] =3D PCI-Express 2 endpoint max data 256(256) FLR NS link x4(x4) speed 8.0(8.0) ASPM disabled(L0s/L1) ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 1 corrected ecap 0002[150] =3D VC 1 max VC0 ecap 0004[180] =3D Power Budgeting 1 ecap 000e[190] =3D ARI 1 ecap 0003[270] =3D Serial 1 55cd2e404bdd89db ecap 0019[2a0] =3D PCIe Sec 1 lane errors 0 nvme4@pci0:7:0:0: class=3D0x010802 card=3D0x37058086 = chip=3D0x09538086 rev=3D0x01 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 11[50] =3D MSI-X supports 32 messages, enabled Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[60] =3D PCI-Express 2 endpoint max data 256(256) FLR NS link x4(x4) speed 8.0(8.0) ASPM disabled(L0s/L1) ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 1 corrected ecap 0002[150] =3D VC 1 max VC0 ecap 0004[180] =3D Power Budgeting 1 ecap 000e[190] =3D ARI 1 ecap 0003[270] =3D Serial 1 55cd2e414d04f756 ecap 0019[2a0] =3D PCIe Sec 1 lane errors 0 nvme5@pci0:8:0:0: class=3D0x010802 card=3D0x37058086 = chip=3D0x09538086 rev=3D0x01 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 11[50] =3D MSI-X supports 32 messages, enabled Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[60] =3D PCI-Express 2 endpoint max data 256(256) FLR NS link x4(x4) speed 8.0(8.0) ASPM disabled(L0s/L1) ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 1 corrected ecap 0002[150] =3D VC 1 max VC0 ecap 0004[180] =3D Power Budgeting 1 ecap 000e[190] =3D ARI 1 ecap 0003[270] =3D Serial 1 55cd2e414d04f727 ecap 0019[2a0] =3D PCIe Sec 1 lane errors 0 Looking at the pciconf -lc output I noticed that the hot plug capable = devices are the =E2=80=9Cpcib=E2=80=9D bridges, so I tried disabling a = pcib bridge. nvme9 is attached to pcib20, and detaching pcib20 indeed works. # devctl disable pci0:128:2:3 Sep 29 09:25:33 pruebanvme2 kernel: nvme9: detached Sep 29 09:25:33 pruebanvme2 kernel: pci20: detached Sep 29 09:25:33 pruebanvme2 kernel: pcib20: detached Sep 29 09:27:23 pruebanvme2 kernel: pcib20: irq 56 = at device 2.3 numa-domain 1 on pci14 Sep 29 09:27:23 pruebanvme2 kernel: pcib20: [GIANT-LOCKED] Sep 29 09:27:24 pruebanvme2 kernel: pcib20: Missed HotPlug interrupt = waiting for DLL Active Sep 29 09:27:24 pruebanvme2 kernel: pci20: numa-domain 1 = on pcib20 Sep 29 09:27:24 pruebanvme2 kernel: nvme9: at = device 0.0 numa-domain 1 on pci20 And the green LED is on. Reenabling the device, Sep 29 09:27:24 pruebanvme2 kernel: nvme9: at = device 0.0 numa-domain 1 on pci20 Sep 29 09:27:33 pruebanvme2 kernel: nvd10: NVMe = namespace Sep 29 09:27:33 pruebanvme2 kernel: nvd10: 1907729MB (488378646 4096 = byte sectors) it works, but I failed to write to the device dd if=3D/dev/zero of=3D/dev/nvd10 bs=3D1m count=3D1000 dd: /dev/nvd10: Input/output error 1+0 records in 0+0 records out 0 bytes transferred in 0.064099 secs (0 bytes/sec) Sep 29 09:31:23 pruebanvme2 kernel: nvme9: WRITE sqid:7 cid:127 nsid:1 = lba:0 len:32 Sep 29 09:31:23 pruebanvme2 kernel: nvme9: WRITE FAULTS (02/80) sqid:7 = cid:127 cdw0:0 Sep 29 09:31:23 pruebanvme2 kernel: nvme9: async event occurred (log = page id=3D0x1) Sep 29 09:32:20 pruebanvme2 kernel: nvme9: WRITE sqid:8 cid:127 nsid:1 = lba:0 len:32 Sep 29 09:32:20 pruebanvme2 kernel: nvme9: WRITE FAULTS (02/80) sqid:8 = cid:127 cdw0:0 Sep 29 09:32:20 pruebanvme2 kernel: nvme9: async event occurred (log = page id=3D0x1) This is what I have so far.=20 pciconf -lc and devinfo outputs follow root@pruebanvme2:~ # pciconf -lc none0@pci0:255:8:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f808086 rev=3D0x01 hdr=3D0x00 none1@pci0:255:8:2: class=3D0x110100 card=3D0x084415d9 = chip=3D0x6f328086 rev=3D0x01 hdr=3D0x00 none2@pci0:255:8:3: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f838086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) ecap 000b[100] =3D Vendor 0none3@pci0:255:9:0: class=3D0x088000 = card=3D0x084415d9 chip=3D0x6f908086 rev=3D0x01 hdr=3D0x00 none4@pci0:255:9:2: class=3D0x110100 card=3D0x084415d9 = chip=3D0x6f338086 rev=3D0x01 hdr=3D0x00 none5@pci0:255:9:3: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f938086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) ecap 000b[100] =3D Vendor 0none6@pci0:255:11:0: class=3D0x088000 = card=3D0x084415d9 chip=3D0x6f818086 rev=3D0x01 hdr=3D0x00 none7@pci0:255:11:1: class=3D0x110100 card=3D0x084415d9 = chip=3D0x6f368086 rev=3D0x01 hdr=3D0x00 none8@pci0:255:11:2: class=3D0x110100 card=3D0x084415d9 = chip=3D0x6f378086 rev=3D0x01 hdr=3D0x00 none9@pci0:255:11:3: class=3D0x088000 card=3D0x00000000 = chip=3D0x6f768086 rev=3D0x01 hdr=3D0x00 none10@pci0:255:12:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fe08086 rev=3D0x01 hdr=3D0x00 none11@pci0:255:12:1: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fe18086 rev=3D0x01 hdr=3D0x00 none12@pci0:255:12:2: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fe28086 rev=3D0x01 hdr=3D0x00 none13@pci0:255:12:3: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fe38086 rev=3D0x01 hdr=3D0x00 none14@pci0:255:12:4: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fe48086 rev=3D0x01 hdr=3D0x00 none15@pci0:255:12:5: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fe58086 rev=3D0x01 hdr=3D0x00 none16@pci0:255:12:6: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fe68086 rev=3D0x01 hdr=3D0x00 none17@pci0:255:12:7: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fe78086 rev=3D0x01 hdr=3D0x00 none18@pci0:255:13:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fe88086 rev=3D0x01 hdr=3D0x00 none19@pci0:255:13:1: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fe98086 rev=3D0x01 hdr=3D0x00 none20@pci0:255:15:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6ff88086 rev=3D0x01 hdr=3D0x00 none21@pci0:255:15:1: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6ff98086 rev=3D0x01 hdr=3D0x00 none22@pci0:255:15:4: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6ffc8086 rev=3D0x01 hdr=3D0x00 none23@pci0:255:15:5: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6ffd8086 rev=3D0x01 hdr=3D0x00 none24@pci0:255:15:6: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6ffe8086 rev=3D0x01 hdr=3D0x00 none25@pci0:255:16:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f1d8086 rev=3D0x01 hdr=3D0x00 none26@pci0:255:16:1: class=3D0x110100 card=3D0x084415d9 = chip=3D0x6f348086 rev=3D0x01 hdr=3D0x00 none27@pci0:255:16:5: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f1e8086 rev=3D0x01 hdr=3D0x00 none28@pci0:255:16:6: class=3D0x110100 card=3D0x084415d9 = chip=3D0x6f7d8086 rev=3D0x01 hdr=3D0x00 none29@pci0:255:16:7: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f1f8086 rev=3D0x01 hdr=3D0x00 none30@pci0:255:18:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fa08086 rev=3D0x01 hdr=3D0x00 none31@pci0:255:18:1: class=3D0x110100 card=3D0x084415d9 = chip=3D0x6f308086 rev=3D0x01 hdr=3D0x00 none32@pci0:255:19:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fa88086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none33@pci0:255:19:1: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f718086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none34@pci0:255:19:2: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6faa8086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none35@pci0:255:19:3: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fab8086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none36@pci0:255:19:4: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fac8086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none37@pci0:255:19:5: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fad8086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none38@pci0:255:19:6: class=3D0x088000 card=3D0x00000000 = chip=3D0x6fae8086 rev=3D0x01 hdr=3D0x00 none39@pci0:255:19:7: class=3D0x088000 card=3D0x00000000 = chip=3D0x6faf8086 rev=3D0x01 hdr=3D0x00 none40@pci0:255:20:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fb08086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none41@pci0:255:20:1: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fb18086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none42@pci0:255:20:2: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fb28086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none43@pci0:255:20:3: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fb38086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none44@pci0:255:20:4: class=3D0x088000 card=3D0x00000000 = chip=3D0x6fbc8086 rev=3D0x01 hdr=3D0x00 none45@pci0:255:20:5: class=3D0x088000 card=3D0x00000000 = chip=3D0x6fbd8086 rev=3D0x01 hdr=3D0x00 none46@pci0:255:20:6: class=3D0x088000 card=3D0x00000000 = chip=3D0x6fbe8086 rev=3D0x01 hdr=3D0x00 none47@pci0:255:20:7: class=3D0x088000 card=3D0x00000000 = chip=3D0x6fbf8086 rev=3D0x01 hdr=3D0x00 none48@pci0:255:21:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fb48086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none49@pci0:255:21:1: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fb58086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none50@pci0:255:21:2: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fb68086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none51@pci0:255:21:3: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fb78086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none52@pci0:255:22:0: class=3D0x088000 card=3D0x00000000 = chip=3D0x6f688086 rev=3D0x01 hdr=3D0x00 cap 00[40] =3D unknown none53@pci0:255:22:6: class=3D0x088000 card=3D0x00000000 = chip=3D0x6f6e8086 rev=3D0x01 hdr=3D0x00 none54@pci0:255:22:7: class=3D0x088000 card=3D0x00000000 = chip=3D0x6f6f8086 rev=3D0x01 hdr=3D0x00 none55@pci0:255:23:0: class=3D0x088000 card=3D0x00000000 = chip=3D0x6fd08086 rev=3D0x01 hdr=3D0x00 cap 00[40] =3D unknown none56@pci0:255:23:4: class=3D0x088000 card=3D0x00000000 = chip=3D0x6fb88086 rev=3D0x01 hdr=3D0x00 none57@pci0:255:23:5: class=3D0x088000 card=3D0x00000000 = chip=3D0x6fb98086 rev=3D0x01 hdr=3D0x00 none58@pci0:255:23:6: class=3D0x088000 card=3D0x00000000 = chip=3D0x6fba8086 rev=3D0x01 hdr=3D0x00 none59@pci0:255:23:7: class=3D0x088000 card=3D0x00000000 = chip=3D0x6fbb8086 rev=3D0x01 hdr=3D0x00 none60@pci0:255:30:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f988086 rev=3D0x01 hdr=3D0x00 none61@pci0:255:30:1: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f998086 rev=3D0x01 hdr=3D0x00 none62@pci0:255:30:2: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f9a8086 rev=3D0x01 hdr=3D0x00 none63@pci0:255:30:3: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fc08086 rev=3D0x01 hdr=3D0x00 none64@pci0:255:30:4: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f9c8086 rev=3D0x01 hdr=3D0x00 none65@pci0:255:31:0: class=3D0x088000 card=3D0x00000000 = chip=3D0x6f888086 rev=3D0x01 hdr=3D0x00 none66@pci0:255:31:2: class=3D0x088000 card=3D0x00000000 = chip=3D0x6f8a8086 rev=3D0x01 hdr=3D0x00 none67@pci0:127:8:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f808086 rev=3D0x01 hdr=3D0x00 none68@pci0:127:8:2: class=3D0x110100 card=3D0x084415d9 = chip=3D0x6f328086 rev=3D0x01 hdr=3D0x00 none69@pci0:127:8:3: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f838086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) ecap 000b[100] =3D Vendor 0none70@pci0:127:9:0: class=3D0x088000 = card=3D0x084415d9 chip=3D0x6f908086 rev=3D0x01 hdr=3D0x00 none71@pci0:127:9:2: class=3D0x110100 card=3D0x084415d9 = chip=3D0x6f338086 rev=3D0x01 hdr=3D0x00 none72@pci0:127:9:3: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f938086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) ecap 000b[100] =3D Vendor 0none73@pci0:127:11:0: class=3D0x088000 = card=3D0x084415d9 chip=3D0x6f818086 rev=3D0x01 hdr=3D0x00 none74@pci0:127:11:1: class=3D0x110100 card=3D0x084415d9 = chip=3D0x6f368086 rev=3D0x01 hdr=3D0x00 none75@pci0:127:11:2: class=3D0x110100 card=3D0x084415d9 = chip=3D0x6f378086 rev=3D0x01 hdr=3D0x00 none76@pci0:127:11:3: class=3D0x088000 card=3D0x00000000 = chip=3D0x6f768086 rev=3D0x01 hdr=3D0x00 none77@pci0:127:12:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fe08086 rev=3D0x01 hdr=3D0x00 none78@pci0:127:12:1: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fe18086 rev=3D0x01 hdr=3D0x00 none79@pci0:127:12:2: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fe28086 rev=3D0x01 hdr=3D0x00 none80@pci0:127:12:3: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fe38086 rev=3D0x01 hdr=3D0x00 none81@pci0:127:12:4: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fe48086 rev=3D0x01 hdr=3D0x00 none82@pci0:127:12:5: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fe58086 rev=3D0x01 hdr=3D0x00 none83@pci0:127:12:6: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fe68086 rev=3D0x01 hdr=3D0x00 none84@pci0:127:12:7: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fe78086 rev=3D0x01 hdr=3D0x00 none85@pci0:127:13:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fe88086 rev=3D0x01 hdr=3D0x00 none86@pci0:127:13:1: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fe98086 rev=3D0x01 hdr=3D0x00 none87@pci0:127:15:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6ff88086 rev=3D0x01 hdr=3D0x00 none88@pci0:127:15:1: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6ff98086 rev=3D0x01 hdr=3D0x00 none89@pci0:127:15:4: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6ffc8086 rev=3D0x01 hdr=3D0x00 none90@pci0:127:15:5: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6ffd8086 rev=3D0x01 hdr=3D0x00 none91@pci0:127:15:6: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6ffe8086 rev=3D0x01 hdr=3D0x00 none92@pci0:127:16:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f1d8086 rev=3D0x01 hdr=3D0x00 none93@pci0:127:16:1: class=3D0x110100 card=3D0x084415d9 = chip=3D0x6f348086 rev=3D0x01 hdr=3D0x00 none94@pci0:127:16:5: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f1e8086 rev=3D0x01 hdr=3D0x00 none95@pci0:127:16:6: class=3D0x110100 card=3D0x084415d9 = chip=3D0x6f7d8086 rev=3D0x01 hdr=3D0x00 none96@pci0:127:16:7: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f1f8086 rev=3D0x01 hdr=3D0x00 none97@pci0:127:18:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fa08086 rev=3D0x01 hdr=3D0x00 none98@pci0:127:18:1: class=3D0x110100 card=3D0x084415d9 = chip=3D0x6f308086 rev=3D0x01 hdr=3D0x00 none99@pci0:127:19:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fa88086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none100@pci0:127:19:1: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f718086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none101@pci0:127:19:2: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6faa8086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none102@pci0:127:19:3: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fab8086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none103@pci0:127:19:4: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fac8086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none104@pci0:127:19:5: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fad8086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none105@pci0:127:19:6: class=3D0x088000 card=3D0x00000000 = chip=3D0x6fae8086 rev=3D0x01 hdr=3D0x00 none106@pci0:127:19:7: class=3D0x088000 card=3D0x00000000 = chip=3D0x6faf8086 rev=3D0x01 hdr=3D0x00 none107@pci0:127:20:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fb08086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none108@pci0:127:20:1: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fb18086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none109@pci0:127:20:2: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fb28086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none110@pci0:127:20:3: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fb38086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none111@pci0:127:20:4: class=3D0x088000 card=3D0x00000000 = chip=3D0x6fbc8086 rev=3D0x01 hdr=3D0x00 none112@pci0:127:20:5: class=3D0x088000 card=3D0x00000000 = chip=3D0x6fbd8086 rev=3D0x01 hdr=3D0x00 none113@pci0:127:20:6: class=3D0x088000 card=3D0x00000000 = chip=3D0x6fbe8086 rev=3D0x01 hdr=3D0x00 none114@pci0:127:20:7: class=3D0x088000 card=3D0x00000000 = chip=3D0x6fbf8086 rev=3D0x01 hdr=3D0x00 none115@pci0:127:21:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fb48086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none116@pci0:127:21:1: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fb58086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none117@pci0:127:21:2: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fb68086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none118@pci0:127:21:3: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fb78086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) none119@pci0:127:22:0: class=3D0x088000 card=3D0x00000000 = chip=3D0x6f688086 rev=3D0x01 hdr=3D0x00 cap 00[40] =3D unknown none120@pci0:127:22:6: class=3D0x088000 card=3D0x00000000 = chip=3D0x6f6e8086 rev=3D0x01 hdr=3D0x00 none121@pci0:127:22:7: class=3D0x088000 card=3D0x00000000 = chip=3D0x6f6f8086 rev=3D0x01 hdr=3D0x00 none122@pci0:127:23:0: class=3D0x088000 card=3D0x00000000 = chip=3D0x6fd08086 rev=3D0x01 hdr=3D0x00 cap 00[40] =3D unknown none123@pci0:127:23:4: class=3D0x088000 card=3D0x00000000 = chip=3D0x6fb88086 rev=3D0x01 hdr=3D0x00 none124@pci0:127:23:5: class=3D0x088000 card=3D0x00000000 = chip=3D0x6fb98086 rev=3D0x01 hdr=3D0x00 none125@pci0:127:23:6: class=3D0x088000 card=3D0x00000000 = chip=3D0x6fba8086 rev=3D0x01 hdr=3D0x00 none126@pci0:127:23:7: class=3D0x088000 card=3D0x00000000 = chip=3D0x6fbb8086 rev=3D0x01 hdr=3D0x00 none127@pci0:127:30:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f988086 rev=3D0x01 hdr=3D0x00 none128@pci0:127:30:1: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f998086 rev=3D0x01 hdr=3D0x00 none129@pci0:127:30:2: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f9a8086 rev=3D0x01 hdr=3D0x00 none130@pci0:127:30:3: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6fc08086 rev=3D0x01 hdr=3D0x00 none131@pci0:127:30:4: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f9c8086 rev=3D0x01 hdr=3D0x00 none132@pci0:127:31:0: class=3D0x088000 card=3D0x00000000 = chip=3D0x6f888086 rev=3D0x01 hdr=3D0x00 none133@pci0:127:31:2: class=3D0x088000 card=3D0x00000000 = chip=3D0x6f8a8086 rev=3D0x01 hdr=3D0x00 hostb0@pci0:0:0:0: class=3D0x060000 card=3D0x084415d9 = chip=3D0x6f008086 rev=3D0x01 hdr=3D0x00 cap 10[90] =3D PCI-Express 2 root port max data 128(128) link x0(x4) speed 0.0(5.0) ASPM disabled(L1) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 ecap 000b[100] =3D Vendor 1 ID 2 ecap 000b[144] =3D Vendor 1 ID 4 ecap 000b[1d0] =3D Vendor 1 ID 3 ecap 000b[280] =3D Vendor 1 ID 5 ecap 000b[300] =3D Vendor 1 ID 8 pcib3@pci0:0:1:0: class=3D0x060400 card=3D0x084415d9 = chip=3D0x6f028086 rev=3D0x01 hdr=3D0x01 cap 0d[40] =3D PCI Bridge card=3D0x084415d9 cap 05[60] =3D MSI supports 2 messages, vector masks=20 cap 10[90] =3D PCI-Express 2 root port max data 256(256) ARI enabled link x8(x8) speed 5.0(8.0) slot 0 power limit 25000 mW surprise cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 ecap 000b[100] =3D Vendor 1 ID 2 ecap 000d[110] =3D ACS 1 ecap 0001[148] =3D AER 1 0 fatal 0 non-fatal 0 corrected ecap 000b[1d0] =3D Vendor 1 ID 3 ecap 0019[250] =3D PCIe Sec 1 lane errors 0xff ecap 000b[280] =3D Vendor 1 ID 5 ecap 000b[300] =3D Vendor 1 ID 8 pcib4@pci0:0:2:0: class=3D0x060400 card=3D0x084415d9 = chip=3D0x6f048086 rev=3D0x01 hdr=3D0x01 cap 0d[40] =3D PCI Bridge card=3D0x084415d9 cap 05[60] =3D MSI supports 2 messages, vector masks enabled with 1 = message cap 10[90] =3D PCI-Express 2 root port max data 256(256) ARI enabled link x4(x4) speed 8.0(8.0) slot 3 power limit 0 mW HotPlug(present) surprise Attn = Button PC(on) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 ecap 000b[100] =3D Vendor 1 ID 2 ecap 000d[110] =3D ACS 1 ecap 0001[148] =3D AER 1 0 fatal 0 non-fatal 0 corrected ecap 000b[1d0] =3D Vendor 1 ID 3 ecap 0019[250] =3D PCIe Sec 1 lane errors 0xff ecap 000b[280] =3D Vendor 1 ID 5 ecap 000b[300] =3D Vendor 1 ID 8 pcib5@pci0:0:2:1: class=3D0x060400 card=3D0x084415d9 = chip=3D0x6f058086 rev=3D0x01 hdr=3D0x01 cap 0d[40] =3D PCI Bridge card=3D0x084415d9 cap 05[60] =3D MSI supports 2 messages, vector masks enabled with 1 = message cap 10[90] =3D PCI-Express 2 root port max data 256(256) ARI enabled link x4(x4) speed 8.0(8.0) slot 4 power limit 0 mW HotPlug(present) surprise Attn = Button PC(on) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 ecap 000b[100] =3D Vendor 1 ID 2 ecap 000d[110] =3D ACS 1 ecap 0001[148] =3D AER 1 0 fatal 0 non-fatal 0 corrected ecap 000b[1d0] =3D Vendor 1 ID 3 ecap 0019[250] =3D PCIe Sec 1 lane errors 0xf ecap 000b[280] =3D Vendor 1 ID 5 ecap 000b[300] =3D Vendor 1 ID 8 pcib6@pci0:0:2:2: class=3D0x060400 card=3D0x084415d9 = chip=3D0x6f068086 rev=3D0x01 hdr=3D0x01 cap 0d[40] =3D PCI Bridge card=3D0x084415d9 cap 05[60] =3D MSI supports 2 messages, vector masks=20 cap 10[90] =3D PCI-Express 2 root port max data 128(256) ARI = disabled link x0(x8) speed 0.0(8.0) slot 0 power limit 25000 mW surprise cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 ecap 000b[100] =3D Vendor 1 ID 2 ecap 000d[110] =3D ACS 1 ecap 0001[148] =3D AER 1 0 fatal 0 non-fatal 0 corrected ecap 000b[1d0] =3D Vendor 1 ID 3 ecap 0019[250] =3D PCIe Sec 1 lane errors 0 ecap 000b[280] =3D Vendor 1 ID 5 ecap 000b[300] =3D Vendor 1 ID 8 pcib7@pci0:0:3:0: class=3D0x060400 card=3D0x084415d9 = chip=3D0x6f088086 rev=3D0x01 hdr=3D0x01 cap 0d[40] =3D PCI Bridge card=3D0x084415d9 cap 05[60] =3D MSI supports 2 messages, vector masks enabled with 1 = message cap 10[90] =3D PCI-Express 2 root port max data 256(256) ARI enabled link x4(x4) speed 8.0(8.0) slot 7 power limit 0 mW HotPlug(present) surprise Attn = Button PC(on) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 ecap 000b[100] =3D Vendor 1 ID 2 ecap 000d[110] =3D ACS 1 ecap 0001[148] =3D AER 1 0 fatal 0 non-fatal 0 corrected ecap 000b[1d0] =3D Vendor 1 ID 3 ecap 0019[250] =3D PCIe Sec 1 lane errors 0xffff ecap 000b[280] =3D Vendor 1 ID 5 ecap 000b[300] =3D Vendor 1 ID 8 pcib8@pci0:0:3:1: class=3D0x060400 card=3D0x084415d9 = chip=3D0x6f098086 rev=3D0x01 hdr=3D0x01 cap 0d[40] =3D PCI Bridge card=3D0x084415d9 cap 05[60] =3D MSI supports 2 messages, vector masks enabled with 1 = message cap 10[90] =3D PCI-Express 2 root port max data 256(256) ARI enabled link x4(x4) speed 8.0(8.0) slot 8 power limit 0 mW HotPlug(present) surprise Attn = Button PC(on) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 ecap 000b[100] =3D Vendor 1 ID 2 ecap 000d[110] =3D ACS 1 ecap 0001[148] =3D AER 1 0 fatal 0 non-fatal 0 corrected ecap 000b[1d0] =3D Vendor 1 ID 3 ecap 0019[250] =3D PCIe Sec 1 lane errors 0xf ecap 000b[280] =3D Vendor 1 ID 5 ecap 000b[300] =3D Vendor 1 ID 8 pcib9@pci0:0:3:2: class=3D0x060400 card=3D0x084415d9 = chip=3D0x6f0a8086 rev=3D0x01 hdr=3D0x01 cap 0d[40] =3D PCI Bridge card=3D0x084415d9 cap 05[60] =3D MSI supports 2 messages, vector masks enabled with 1 = message cap 10[90] =3D PCI-Express 2 root port max data 256(256) ARI enabled link x4(x4) speed 8.0(8.0) slot 9 power limit 0 mW HotPlug(present) surprise Attn = Button PC(on) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 ecap 000b[100] =3D Vendor 1 ID 2 ecap 000d[110] =3D ACS 1 ecap 0001[148] =3D AER 1 0 fatal 0 non-fatal 0 corrected ecap 000b[1d0] =3D Vendor 1 ID 3 ecap 0019[250] =3D PCIe Sec 1 lane errors 0xff ecap 000b[280] =3D Vendor 1 ID 5 ecap 000b[300] =3D Vendor 1 ID 8 pcib10@pci0:0:3:3: class=3D0x060400 card=3D0x084415d9 = chip=3D0x6f0b8086 rev=3D0x01 hdr=3D0x01 cap 0d[40] =3D PCI Bridge card=3D0x084415d9 cap 05[60] =3D MSI supports 2 messages, vector masks enabled with 1 = message cap 10[90] =3D PCI-Express 2 root port max data 256(256) ARI enabled link x4(x4) speed 8.0(8.0) slot 10 power limit 0 mW HotPlug(present) surprise Attn = Button PC(on) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 ecap 000b[100] =3D Vendor 1 ID 2 ecap 000d[110] =3D ACS 1 ecap 0001[148] =3D AER 1 0 fatal 0 non-fatal 0 corrected ecap 000b[1d0] =3D Vendor 1 ID 3 ecap 0019[250] =3D PCIe Sec 1 lane errors 0xf ecap 000b[280] =3D Vendor 1 ID 5 ecap 000b[300] =3D Vendor 1 ID 8 none134@pci0:0:4:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f208086 rev=3D0x01 hdr=3D0x00 cap 11[80] =3D MSI-X supports 1 message Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[90] =3D PCI-Express 2 root endpoint max data 128(128) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 none135@pci0:0:4:1: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f218086 rev=3D0x01 hdr=3D0x00 cap 11[80] =3D MSI-X supports 1 message Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[90] =3D PCI-Express 2 root endpoint max data 128(128) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 none136@pci0:0:4:2: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f228086 rev=3D0x01 hdr=3D0x00 cap 11[80] =3D MSI-X supports 1 message Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[90] =3D PCI-Express 2 root endpoint max data 128(128) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 none137@pci0:0:4:3: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f238086 rev=3D0x01 hdr=3D0x00 cap 11[80] =3D MSI-X supports 1 message Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[90] =3D PCI-Express 2 root endpoint max data 128(128) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 none138@pci0:0:4:4: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f248086 rev=3D0x01 hdr=3D0x00 cap 11[80] =3D MSI-X supports 1 message Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[90] =3D PCI-Express 2 root endpoint max data 128(128) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 none139@pci0:0:4:5: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f258086 rev=3D0x01 hdr=3D0x00 cap 11[80] =3D MSI-X supports 1 message Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[90] =3D PCI-Express 2 root endpoint max data 128(128) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 none140@pci0:0:4:6: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f268086 rev=3D0x01 hdr=3D0x00 cap 11[80] =3D MSI-X supports 1 message Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[90] =3D PCI-Express 2 root endpoint max data 128(128) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 none141@pci0:0:4:7: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f278086 rev=3D0x01 hdr=3D0x00 cap 11[80] =3D MSI-X supports 1 message Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[90] =3D PCI-Express 2 root endpoint max data 128(128) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 none142@pci0:0:5:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f288086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 2 root endpoint max data 128(128) none143@pci0:0:5:1: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f298086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) cap 05[80] =3D MSI supports 1 message, 64 bit=20 ecap 000b[100] =3D Vendor 1 ID 6 ecap 000b[110] =3D Vendor 1 ID 6 ecap 000b[120] =3D Vendor 1 ID 6 ecap 000b[130] =3D Vendor 1 ID 6 none144@pci0:0:5:2: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f2a8086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 2 root endpoint max data 128(128) RO NS ioapic0@pci0:0:5:4: class=3D0x080020 card=3D0x084415d9 = chip=3D0x6f2c8086 rev=3D0x01 hdr=3D0x00 cap 10[44] =3D PCI-Express 1 root endpoint max data 128(128) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 none145@pci0:0:17:0: class=3D0xff0000 card=3D0x084415d9 = chip=3D0x8d7c8086 rev=3D0x05 hdr=3D0x00 cap 10[40] =3D PCI-Express 2 root endpoint max data 128(128) link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1) cap 01[80] =3D powerspec 3 supports D0 D3 current D0 ahci0@pci0:0:17:4: class=3D0x010400 card=3D0x084415d9 = chip=3D0x28278086 rev=3D0x05 hdr=3D0x00 cap 05[80] =3D MSI supports 1 message enabled with 1 message cap 01[70] =3D powerspec 3 supports D0 D3 current D0 cap 12[a8] =3D SATA Index-Data Pair none146@pci0:0:22:0: class=3D0x078000 card=3D0x084415d9 = chip=3D0x8d3a8086 rev=3D0x05 hdr=3D0x00 cap 01[50] =3D powerspec 3 supports D0 D3 current D0 cap 05[8c] =3D MSI supports 1 message, 64 bit=20 none147@pci0:0:22:1: class=3D0x078000 card=3D0x084415d9 = chip=3D0x8d3b8086 rev=3D0x05 hdr=3D0x00 cap 01[50] =3D powerspec 3 supports D0 D3 current D0 cap 05[8c] =3D MSI supports 1 message, 64 bit=20 ehci0@pci0:0:26:0: class=3D0x0c0320 card=3D0x084415d9 = chip=3D0x8d2d8086 rev=3D0x05 hdr=3D0x00 cap 01[50] =3D powerspec 2 supports D0 D3 current D0 cap 0a[58] =3D EHCI Debug Port at offset 0xa0 in map 0x14 cap 13[98] =3D PCI Advanced Features: FLR TP pcib11@pci0:0:28:0: class=3D0x060400 card=3D0x084415d9 = chip=3D0x8d108086 rev=3D0xd5 hdr=3D0x01 cap 10[40] =3D PCI-Express 2 root port max data 128(128) link x0(x1) speed 0.0(5.0) ASPM disabled(L0s/L1) cap 05[80] =3D MSI supports 1 message=20 cap 0d[90] =3D PCI Bridge card=3D0x084415d9 cap 01[a0] =3D powerspec 3 supports D0 D3 current D0 pcib12@pci0:0:28:2: class=3D0x060400 card=3D0x084415d9 = chip=3D0x8d148086 rev=3D0xd5 hdr=3D0x01 cap 10[40] =3D PCI-Express 2 root port max data 128(128) link x1(x1) speed 2.5(5.0) ASPM disabled(L0s/L1) slot 0 power limit 0 mW cap 05[80] =3D MSI supports 1 message enabled with 1 message cap 0d[90] =3D PCI Bridge card=3D0x084415d9 cap 01[a0] =3D powerspec 3 supports D0 D3 current D0 ehci1@pci0:0:29:0: class=3D0x0c0320 card=3D0x084415d9 = chip=3D0x8d268086 rev=3D0x05 hdr=3D0x00 cap 01[50] =3D powerspec 2 supports D0 D3 current D0 cap 0a[58] =3D EHCI Debug Port at offset 0xa0 in map 0x14 cap 13[98] =3D PCI Advanced Features: FLR TP isab0@pci0:0:31:0: class=3D0x060100 card=3D0x084415d9 = chip=3D0x8d448086 rev=3D0x05 hdr=3D0x00 cap 09[e0] =3D vendor (length 12) Intel cap 1 version 0 features: AMT, 4 PCI-e x1 slots ahci1@pci0:0:31:2: class=3D0x010601 card=3D0x084415d9 = chip=3D0x8d028086 rev=3D0x05 hdr=3D0x00 cap 05[80] =3D MSI supports 1 message enabled with 1 message cap 01[70] =3D powerspec 3 supports D0 D3 current D0 cap 12[a8] =3D SATA Index-Data Pair none148@pci0:0:31:3: class=3D0x0c0500 card=3D0x084415d9 = chip=3D0x8d228086 rev=3D0x05 hdr=3D0x00 none149@pci0:0:31:6: class=3D0x118000 card=3D0x084415d9 = chip=3D0x8d248086 rev=3D0x05 hdr=3D0x00 cap 01[50] =3D powerspec 3 supports D0 D3 current D0 cap 05[80] =3D MSI supports 1 message=20 ix0@pci0:1:0:0: class=3D0x020000 card=3D0x088a15d9 chip=3D0x15288086 = rev=3D0x01 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 05[50] =3D MSI supports 1 message, 64 bit, vector masks=20 cap 11[70] =3D MSI-X supports 64 messages, enabled Table in map 0x20[0x0], PBA in map 0x20[0x2000] cap 10[a0] =3D PCI-Express 2 endpoint max data 256(512) FLR NS link x8(x8) speed 5.0(5.0) ASPM disabled(L0s/L1) cap 03[e0] =3D VPD ecap 0001[100] =3D AER 2 0 fatal 1 non-fatal 1 corrected ecap 000e[150] =3D ARI 1 ecap 0010[160] =3D SR-IOV 1 IOV disabled, Memory Space disabled, ARI = disabled 0 VFs configured out of 64 supported First VF RID Offset 0x0180, VF RID Stride 0x0002 VF Device ID 0x1515 Page Sizes: 4096 (enabled), 8192, 65536, 262144, = 1048576, 4194304 ecap 000d[1d0] =3D ACS 1 ix1@pci0:1:0:1: class=3D0x020000 card=3D0x088a15d9 chip=3D0x15288086 = rev=3D0x01 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 05[50] =3D MSI supports 1 message, 64 bit, vector masks=20 cap 11[70] =3D MSI-X supports 64 messages, enabled Table in map 0x20[0x0], PBA in map 0x20[0x2000] cap 10[a0] =3D PCI-Express 2 endpoint max data 256(512) FLR NS link x8(x8) speed 5.0(5.0) ASPM disabled(L0s/L1) cap 03[e0] =3D VPD ecap 0001[100] =3D AER 2 0 fatal 1 non-fatal 1 corrected ecap 000e[150] =3D ARI 1 ecap 0010[160] =3D SR-IOV 1 IOV disabled, Memory Space disabled, ARI = disabled 0 VFs configured out of 64 supported First VF RID Offset 0x0180, VF RID Stride 0x0002 VF Device ID 0x1515 Page Sizes: 4096 (enabled), 8192, 65536, 262144, = 1048576, 4194304 ecap 000d[1d0] =3D ACS 1 nvme0@pci0:2:0:0: class=3D0x010802 card=3D0x37058086 = chip=3D0x09538086 rev=3D0x01 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 11[50] =3D MSI-X supports 32 messages, enabled Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[60] =3D PCI-Express 2 endpoint max data 256(256) FLR NS link x4(x4) speed 8.0(8.0) ASPM disabled(L0s/L1) ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 1 corrected ecap 0002[150] =3D VC 1 max VC0 ecap 0004[180] =3D Power Budgeting 1 ecap 000e[190] =3D ARI 1 ecap 0003[270] =3D Serial 1 55cd2e414d04a4cd ecap 0019[2a0] =3D PCIe Sec 1 lane errors 0 nvme1@pci0:3:0:0: class=3D0x010802 card=3D0x37058086 = chip=3D0x09538086 rev=3D0x01 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 11[50] =3D MSI-X supports 32 messages, enabled Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[60] =3D PCI-Express 2 endpoint max data 256(256) FLR NS link x4(x4) speed 8.0(8.0) ASPM disabled(L0s/L1) ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 1 corrected ecap 0002[150] =3D VC 1 max VC0 ecap 0004[180] =3D Power Budgeting 1 ecap 000e[190] =3D ARI 1 ecap 0003[270] =3D Serial 1 55cd2e414d04ad7b ecap 0019[2a0] =3D PCIe Sec 1 lane errors 0 nvme2@pci0:5:0:0: class=3D0x010802 card=3D0x37058086 = chip=3D0x09538086 rev=3D0x01 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 11[50] =3D MSI-X supports 32 messages, enabled Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[60] =3D PCI-Express 2 endpoint max data 256(256) FLR NS link x4(x4) speed 8.0(8.0) ASPM disabled(L0s/L1) ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 1 corrected ecap 0002[150] =3D VC 1 max VC0 ecap 0004[180] =3D Power Budgeting 1 ecap 000e[190] =3D ARI 1 ecap 0003[270] =3D Serial 1 55cd2e414d04a219 ecap 0019[2a0] =3D PCIe Sec 1 lane errors 0 nvme3@pci0:6:0:0: class=3D0x010802 card=3D0x37058086 = chip=3D0x09538086 rev=3D0x01 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 11[50] =3D MSI-X supports 32 messages, enabled Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[60] =3D PCI-Express 2 endpoint max data 256(256) FLR NS link x4(x4) speed 8.0(8.0) ASPM disabled(L0s/L1) ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 1 corrected ecap 0002[150] =3D VC 1 max VC0 ecap 0004[180] =3D Power Budgeting 1 ecap 000e[190] =3D ARI 1 ecap 0003[270] =3D Serial 1 55cd2e404bdd89db ecap 0019[2a0] =3D PCIe Sec 1 lane errors 0 nvme4@pci0:7:0:0: class=3D0x010802 card=3D0x37058086 = chip=3D0x09538086 rev=3D0x01 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 11[50] =3D MSI-X supports 32 messages, enabled Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[60] =3D PCI-Express 2 endpoint max data 256(256) FLR NS link x4(x4) speed 8.0(8.0) ASPM disabled(L0s/L1) ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 1 corrected ecap 0002[150] =3D VC 1 max VC0 ecap 0004[180] =3D Power Budgeting 1 ecap 000e[190] =3D ARI 1 ecap 0003[270] =3D Serial 1 55cd2e414d04f756 ecap 0019[2a0] =3D PCIe Sec 1 lane errors 0 nvme5@pci0:8:0:0: class=3D0x010802 card=3D0x37058086 = chip=3D0x09538086 rev=3D0x01 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 11[50] =3D MSI-X supports 32 messages, enabled Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[60] =3D PCI-Express 2 endpoint max data 256(256) FLR NS link x4(x4) speed 8.0(8.0) ASPM disabled(L0s/L1) ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 1 corrected ecap 0002[150] =3D VC 1 max VC0 ecap 0004[180] =3D Power Budgeting 1 ecap 000e[190] =3D ARI 1 ecap 0003[270] =3D Serial 1 55cd2e414d04f727 ecap 0019[2a0] =3D PCIe Sec 1 lane errors 0 pcib13@pci0:10:0:0: class=3D0x060400 card=3D0x084415d9 = chip=3D0x11501a03 rev=3D0x03 hdr=3D0x01 cap 05[50] =3D MSI supports 1 message, 64 bit=20 cap 01[78] =3D powerspec 3 supports D0 D1 D2 D3 current D0 cap 10[80] =3D PCI-Express 1 PCI bridge max data 128(128) NS link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1) cap 0d[a4] =3D PCI Bridge card=3D0x084415d9 ecap 0002[100] =3D VC 1 max VC0 vgapci0@pci0:11:0:0: class=3D0x030000 card=3D0x084415d9 = chip=3D0x20001a03 rev=3D0x30 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D1 D2 D3 current D0 cap 05[50] =3D MSI supports 4 messages, 64 bit=20 pcib15@pci0:128:0:0: class=3D0x060400 card=3D0x00008086 = chip=3D0x6f018086 rev=3D0x01 hdr=3D0x01 cap 0d[40] =3D PCI Bridge card=3D0x00008086 cap 05[60] =3D MSI supports 2 messages, vector masks=20 cap 10[90] =3D PCI-Express 2 root port max data 128(128) ARI = disabled link x0(x4) speed 0.0(5.0) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 ecap 000b[100] =3D Vendor 1 ID 2 ecap 000d[110] =3D ACS 1 ecap 0001[148] =3D AER 1 0 fatal 0 non-fatal 0 corrected ecap 000b[1d0] =3D Vendor 1 ID 3 ecap 000b[280] =3D Vendor 1 ID 5 ecap 000b[300] =3D Vendor 1 ID 8 pcib16@pci0:128:1:0: class=3D0x060400 card=3D0x084415d9 = chip=3D0x6f028086 rev=3D0x01 hdr=3D0x01 cap 0d[40] =3D PCI Bridge card=3D0x084415d9 cap 05[60] =3D MSI supports 2 messages, vector masks=20 cap 10[90] =3D PCI-Express 2 root port max data 256(256) ARI enabled link x8(x8) speed 5.0(8.0) slot 1 power limit 25000 mW surprise cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 ecap 000b[100] =3D Vendor 1 ID 2 ecap 000d[110] =3D ACS 1 ecap 0001[148] =3D AER 1 0 fatal 0 non-fatal 0 corrected ecap 000b[1d0] =3D Vendor 1 ID 3 ecap 0019[250] =3D PCIe Sec 1 lane errors 0xff ecap 000b[280] =3D Vendor 1 ID 5 ecap 000b[300] =3D Vendor 1 ID 8 pcib17@pci0:128:2:0: class=3D0x060400 card=3D0x084415d9 = chip=3D0x6f048086 rev=3D0x01 hdr=3D0x01 cap 0d[40] =3D PCI Bridge card=3D0x084415d9 cap 05[60] =3D MSI supports 2 messages, vector masks enabled with 1 = message cap 10[90] =3D PCI-Express 2 root port max data 256(256) ARI enabled link x4(x4) speed 8.0(8.0) slot 14 power limit 0 mW HotPlug(present) surprise Attn = Button PC(on) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 ecap 000b[100] =3D Vendor 1 ID 2 ecap 000d[110] =3D ACS 1 ecap 0001[148] =3D AER 1 0 fatal 0 non-fatal 0 corrected ecap 000b[1d0] =3D Vendor 1 ID 3 ecap 0019[250] =3D PCIe Sec 1 lane errors 0xffff ecap 000b[280] =3D Vendor 1 ID 5 ecap 000b[300] =3D Vendor 1 ID 8 pcib18@pci0:128:2:1: class=3D0x060400 card=3D0x084415d9 = chip=3D0x6f058086 rev=3D0x01 hdr=3D0x01 cap 0d[40] =3D PCI Bridge card=3D0x084415d9 cap 05[60] =3D MSI supports 2 messages, vector masks enabled with 1 = message cap 10[90] =3D PCI-Express 2 root port max data 256(256) ARI enabled link x4(x4) speed 8.0(8.0) slot 15 power limit 0 mW HotPlug(present) surprise Attn = Button PC(on) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 ecap 000b[100] =3D Vendor 1 ID 2 ecap 000d[110] =3D ACS 1 ecap 0001[148] =3D AER 1 0 fatal 0 non-fatal 0 corrected ecap 000b[1d0] =3D Vendor 1 ID 3 ecap 0019[250] =3D PCIe Sec 1 lane errors 0xf ecap 000b[280] =3D Vendor 1 ID 5 ecap 000b[300] =3D Vendor 1 ID 8 pcib19@pci0:128:2:2: class=3D0x060400 card=3D0x084415d9 = chip=3D0x6f068086 rev=3D0x01 hdr=3D0x01 cap 0d[40] =3D PCI Bridge card=3D0x084415d9 cap 05[60] =3D MSI supports 2 messages, vector masks enabled with 1 = message cap 10[90] =3D PCI-Express 2 root port max data 256(256) ARI enabled link x4(x4) speed 8.0(8.0) slot 16 power limit 0 mW HotPlug(present) surprise Attn = Button PC(on) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 ecap 000b[100] =3D Vendor 1 ID 2 ecap 000d[110] =3D ACS 1 ecap 0001[148] =3D AER 1 0 fatal 0 non-fatal 0 corrected ecap 000b[1d0] =3D Vendor 1 ID 3 ecap 0019[250] =3D PCIe Sec 1 lane errors 0xff ecap 000b[280] =3D Vendor 1 ID 5 ecap 000b[300] =3D Vendor 1 ID 8 pcib20@pci0:128:2:3: class=3D0x060400 card=3D0x084415d9 = chip=3D0x6f078086 rev=3D0x01 hdr=3D0x01 cap 0d[40] =3D PCI Bridge card=3D0x084415d9 cap 05[60] =3D MSI supports 2 messages, vector masks enabled with 1 = message cap 10[90] =3D PCI-Express 2 root port max data 256(256) ARI enabled link x4(x4) speed 8.0(8.0) slot 17 power limit 0 mW HotPlug(present) surprise Attn = Button PC(on) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 ecap 000b[100] =3D Vendor 1 ID 2 ecap 000d[110] =3D ACS 1 ecap 0001[148] =3D AER 1 0 fatal 0 non-fatal 0 corrected ecap 000b[1d0] =3D Vendor 1 ID 3 ecap 0019[250] =3D PCIe Sec 1 lane errors 0xf ecap 000b[280] =3D Vendor 1 ID 5 ecap 000b[300] =3D Vendor 1 ID 8 pcib21@pci0:128:3:0: class=3D0x060400 card=3D0x084415d9 = chip=3D0x6f088086 rev=3D0x01 hdr=3D0x01 cap 0d[40] =3D PCI Bridge card=3D0x084415d9 cap 05[60] =3D MSI supports 2 messages, vector masks=20 cap 10[90] =3D PCI-Express 2 root port max data 128(256) ARI = disabled link x0(x8) speed 0.0(8.0) slot 2 power limit 25000 mW surprise cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 ecap 000b[100] =3D Vendor 1 ID 2 ecap 000d[110] =3D ACS 1 ecap 0001[148] =3D AER 1 0 fatal 0 non-fatal 0 corrected ecap 000b[1d0] =3D Vendor 1 ID 3 ecap 0019[250] =3D PCIe Sec 1 lane errors 0 ecap 000b[280] =3D Vendor 1 ID 5 ecap 000b[300] =3D Vendor 1 ID 8 pcib22@pci0:128:3:2: class=3D0x060400 card=3D0x084415d9 = chip=3D0x6f0a8086 rev=3D0x01 hdr=3D0x01 cap 0d[40] =3D PCI Bridge card=3D0x084415d9 cap 05[60] =3D MSI supports 2 messages, vector masks=20 cap 10[90] =3D PCI-Express 2 root port max data 128(256) ARI = disabled link x0(x8) speed 0.0(8.0) slot 1 power limit 25000 mW surprise cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 ecap 000b[100] =3D Vendor 1 ID 2 ecap 000d[110] =3D ACS 1 ecap 0001[148] =3D AER 1 0 fatal 0 non-fatal 0 corrected ecap 000b[1d0] =3D Vendor 1 ID 3 ecap 0019[250] =3D PCIe Sec 1 lane errors 0 ecap 000b[280] =3D Vendor 1 ID 5 ecap 000b[300] =3D Vendor 1 ID 8 none150@pci0:128:4:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f208086 rev=3D0x01 hdr=3D0x00 cap 11[80] =3D MSI-X supports 1 message Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[90] =3D PCI-Express 2 root endpoint max data 128(128) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 none151@pci0:128:4:1: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f218086 rev=3D0x01 hdr=3D0x00 cap 11[80] =3D MSI-X supports 1 message Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[90] =3D PCI-Express 2 root endpoint max data 128(128) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 none152@pci0:128:4:2: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f228086 rev=3D0x01 hdr=3D0x00 cap 11[80] =3D MSI-X supports 1 message Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[90] =3D PCI-Express 2 root endpoint max data 128(128) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 none153@pci0:128:4:3: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f238086 rev=3D0x01 hdr=3D0x00 cap 11[80] =3D MSI-X supports 1 message Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[90] =3D PCI-Express 2 root endpoint max data 128(128) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 none154@pci0:128:4:4: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f248086 rev=3D0x01 hdr=3D0x00 cap 11[80] =3D MSI-X supports 1 message Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[90] =3D PCI-Express 2 root endpoint max data 128(128) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 none155@pci0:128:4:5: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f258086 rev=3D0x01 hdr=3D0x00 cap 11[80] =3D MSI-X supports 1 message Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[90] =3D PCI-Express 2 root endpoint max data 128(128) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 none156@pci0:128:4:6: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f268086 rev=3D0x01 hdr=3D0x00 cap 11[80] =3D MSI-X supports 1 message Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[90] =3D PCI-Express 2 root endpoint max data 128(128) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 none157@pci0:128:4:7: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f278086 rev=3D0x01 hdr=3D0x00 cap 11[80] =3D MSI-X supports 1 message Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[90] =3D PCI-Express 2 root endpoint max data 128(128) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 none158@pci0:128:5:0: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f288086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 2 root endpoint max data 128(128) none159@pci0:128:5:1: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f298086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 1 root endpoint max data 128(128) cap 05[80] =3D MSI supports 1 message, 64 bit=20 ecap 000b[100] =3D Vendor 1 ID 6 ecap 000b[110] =3D Vendor 1 ID 6 ecap 000b[120] =3D Vendor 1 ID 6 ecap 000b[130] =3D Vendor 1 ID 6 none160@pci0:128:5:2: class=3D0x088000 card=3D0x084415d9 = chip=3D0x6f2a8086 rev=3D0x01 hdr=3D0x00 cap 10[40] =3D PCI-Express 2 root endpoint max data 128(128) RO NS ioapic1@pci0:128:5:4: class=3D0x080020 card=3D0x084415d9 = chip=3D0x6f2c8086 rev=3D0x01 hdr=3D0x00 cap 10[44] =3D PCI-Express 1 root endpoint max data 128(128) cap 01[e0] =3D powerspec 3 supports D0 D3 current D0 ix2@pci0:130:0:0: class=3D0x020000 card=3D0x061115d9 = chip=3D0x10fb8086 rev=3D0x01 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 05[50] =3D MSI supports 1 message, 64 bit, vector masks=20 cap 11[70] =3D MSI-X supports 64 messages, enabled Table in map 0x20[0x0], PBA in map 0x20[0x2000] cap 10[a0] =3D PCI-Express 2 endpoint max data 256(512) FLR NS link x8(x8) speed 5.0(5.0) ASPM disabled(L0s) cap 03[e0] =3D VPD ecap 0001[100] =3D AER 1 0 fatal 1 non-fatal 1 corrected ecap 0003[140] =3D Serial 1 0cc47affffbb4bc6 ecap 000e[150] =3D ARI 1 ecap 0010[160] =3D SR-IOV 1 IOV disabled, Memory Space disabled, ARI = disabled 0 VFs configured out of 64 supported First VF RID Offset 0x0180, VF RID Stride 0x0002 VF Device ID 0x10ed Page Sizes: 4096 (enabled), 8192, 65536, 262144, = 1048576, 4194304 ix3@pci0:130:0:1: class=3D0x020000 card=3D0x061115d9 = chip=3D0x10fb8086 rev=3D0x01 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 05[50] =3D MSI supports 1 message, 64 bit, vector masks=20 cap 11[70] =3D MSI-X supports 64 messages, enabled Table in map 0x20[0x0], PBA in map 0x20[0x2000] cap 10[a0] =3D PCI-Express 2 endpoint max data 256(512) FLR NS link x8(x8) speed 5.0(5.0) ASPM disabled(L0s) cap 03[e0] =3D VPD ecap 0001[100] =3D AER 1 0 fatal 1 non-fatal 1 corrected ecap 0003[140] =3D Serial 1 0cc47affffbb4bc6 ecap 000e[150] =3D ARI 1 ecap 0010[160] =3D SR-IOV 1 IOV disabled, Memory Space disabled, ARI = disabled 0 VFs configured out of 64 supported First VF RID Offset 0x0180, VF RID Stride 0x0002 VF Device ID 0x10ed Page Sizes: 4096 (enabled), 8192, 65536, 262144, = 1048576, 4194304 nvme6@pci0:131:0:0: class=3D0x010802 card=3D0x37058086 = chip=3D0x09538086 rev=3D0x01 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 11[50] =3D MSI-X supports 32 messages, enabled Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[60] =3D PCI-Express 2 endpoint max data 256(256) FLR NS link x4(x4) speed 8.0(8.0) ASPM disabled(L0s/L1) ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 1 corrected ecap 0002[150] =3D VC 1 max VC0 ecap 0004[180] =3D Power Budgeting 1 ecap 000e[190] =3D ARI 1 ecap 0003[270] =3D Serial 1 55cd2e414ca74979 ecap 0019[2a0] =3D PCIe Sec 1 lane errors 0 nvme7@pci0:132:0:0: class=3D0x010802 card=3D0x37058086 = chip=3D0x09538086 rev=3D0x01 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 11[50] =3D MSI-X supports 32 messages, enabled Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[60] =3D PCI-Express 2 endpoint max data 256(256) FLR NS link x4(x4) speed 8.0(8.0) ASPM disabled(L0s/L1) ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 1 corrected ecap 0002[150] =3D VC 1 max VC0 ecap 0004[180] =3D Power Budgeting 1 ecap 000e[190] =3D ARI 1 ecap 0003[270] =3D Serial 1 55cd2e414d04f5e0 ecap 0019[2a0] =3D PCIe Sec 1 lane errors 0 nvme8@pci0:133:0:0: class=3D0x010802 card=3D0x37058086 = chip=3D0x09538086 rev=3D0x01 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 11[50] =3D MSI-X supports 32 messages, enabled Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[60] =3D PCI-Express 2 endpoint max data 256(256) FLR NS link x4(x4) speed 8.0(8.0) ASPM disabled(L0s/L1) ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 1 corrected ecap 0002[150] =3D VC 1 max VC0 ecap 0004[180] =3D Power Budgeting 1 ecap 000e[190] =3D ARI 1 ecap 0003[270] =3D Serial 1 55cd2e414d04f739 ecap 0019[2a0] =3D PCIe Sec 1 lane errors 0 nvme9@pci0:134:0:0: class=3D0x010802 card=3D0x37058086 = chip=3D0x09538086 rev=3D0x01 hdr=3D0x00 cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 11[50] =3D MSI-X supports 32 messages, enabled Table in map 0x10[0x2000], PBA in map 0x10[0x3000] cap 10[60] =3D PCI-Express 2 endpoint max data 256(256) FLR NS link x4(x4) speed 8.0(8.0) ASPM disabled(L0s/L1) ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 1 corrected ecap 0002[150] =3D VC 1 max VC0 ecap 0004[180] =3D Power Budgeting 1 ecap 000e[190] =3D ARI 1 ecap 0003[270] =3D Serial 1 55cd2e414d04a4cc ecap 0019[2a0] =3D PCIe Sec 1 lane errors 0 root@pruebanvme2:~ #=20 root@pruebanvme2:~ # devinfo -v nexus0 vtvga0 cryptosoft0 apic0 ram0 acpi0 unknown pnpinfo _HID=3DPNP0A03 _UID=3D255 at handle=3D\_SB_.UNC3 = (disabled) unknown pnpinfo _HID=3DPNP0A03 _UID=3D191 at handle=3D\_SB_.UNC2 = (disabled) pcib0 pnpinfo _HID=3DPNP0A03 _UID=3D127 at handle=3D\_SB_.UNC1 pci0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f80 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D8 function=3D0 = dbsf=3Dpci0:255:8:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f32 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x110100 at slot=3D8 function=3D2 = dbsf=3Dpci0:255:8:2 unknown pnpinfo vendor=3D0x8086 device=3D0x6f83 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D8 function=3D3 = dbsf=3Dpci0:255:8:3 unknown pnpinfo vendor=3D0x8086 device=3D0x6f90 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D9 function=3D0 = dbsf=3Dpci0:255:9:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f33 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x110100 at slot=3D9 function=3D2 = dbsf=3Dpci0:255:9:2 unknown pnpinfo vendor=3D0x8086 device=3D0x6f93 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D9 function=3D3 = dbsf=3Dpci0:255:9:3 unknown pnpinfo vendor=3D0x8086 device=3D0x6f81 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D11 function=3D0 = dbsf=3Dpci0:255:11:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f36 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x110100 at slot=3D11 function=3D1 = dbsf=3Dpci0:255:11:1 unknown pnpinfo vendor=3D0x8086 device=3D0x6f37 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x110100 at slot=3D11 function=3D2 = dbsf=3Dpci0:255:11:2 unknown pnpinfo vendor=3D0x8086 device=3D0x6f76 subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D11 function=3D3 = dbsf=3Dpci0:255:11:3 unknown pnpinfo vendor=3D0x8086 device=3D0x6fe0 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D12 function=3D0 = dbsf=3Dpci0:255:12:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6fe1 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D12 function=3D1 = dbsf=3Dpci0:255:12:1 unknown pnpinfo vendor=3D0x8086 device=3D0x6fe2 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D12 function=3D2 = dbsf=3Dpci0:255:12:2 unknown pnpinfo vendor=3D0x8086 device=3D0x6fe3 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D12 function=3D3 = dbsf=3Dpci0:255:12:3 unknown pnpinfo vendor=3D0x8086 device=3D0x6fe4 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D12 function=3D4 = dbsf=3Dpci0:255:12:4 unknown pnpinfo vendor=3D0x8086 device=3D0x6fe5 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D12 function=3D5 = dbsf=3Dpci0:255:12:5 unknown pnpinfo vendor=3D0x8086 device=3D0x6fe6 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D12 function=3D6 = dbsf=3Dpci0:255:12:6 unknown pnpinfo vendor=3D0x8086 device=3D0x6fe7 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D12 function=3D7 = dbsf=3Dpci0:255:12:7 unknown pnpinfo vendor=3D0x8086 device=3D0x6fe8 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D13 function=3D0 = dbsf=3Dpci0:255:13:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6fe9 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D13 function=3D1 = dbsf=3Dpci0:255:13:1 unknown pnpinfo vendor=3D0x8086 device=3D0x6ff8 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D15 function=3D0 = dbsf=3Dpci0:255:15:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6ff9 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D15 function=3D1 = dbsf=3Dpci0:255:15:1 unknown pnpinfo vendor=3D0x8086 device=3D0x6ffc subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D15 function=3D4 = dbsf=3Dpci0:255:15:4 unknown pnpinfo vendor=3D0x8086 device=3D0x6ffd subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D15 function=3D5 = dbsf=3Dpci0:255:15:5 unknown pnpinfo vendor=3D0x8086 device=3D0x6ffe subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D15 function=3D6 = dbsf=3Dpci0:255:15:6 unknown pnpinfo vendor=3D0x8086 device=3D0x6f1d subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D16 function=3D0 = dbsf=3Dpci0:255:16:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f34 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x110100 at slot=3D16 function=3D1 = dbsf=3Dpci0:255:16:1 unknown pnpinfo vendor=3D0x8086 device=3D0x6f1e subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D16 function=3D5 = dbsf=3Dpci0:255:16:5 unknown pnpinfo vendor=3D0x8086 device=3D0x6f7d subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x110100 at slot=3D16 function=3D6 = dbsf=3Dpci0:255:16:6 unknown pnpinfo vendor=3D0x8086 device=3D0x6f1f subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D16 function=3D7 = dbsf=3Dpci0:255:16:7 unknown pnpinfo vendor=3D0x8086 device=3D0x6fa0 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D18 function=3D0 = dbsf=3Dpci0:255:18:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f30 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x110100 at slot=3D18 function=3D1 = dbsf=3Dpci0:255:18:1 unknown pnpinfo vendor=3D0x8086 device=3D0x6fa8 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D19 function=3D0 = dbsf=3Dpci0:255:19:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f71 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D19 function=3D1 = dbsf=3Dpci0:255:19:1 unknown pnpinfo vendor=3D0x8086 device=3D0x6faa subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D19 function=3D2 = dbsf=3Dpci0:255:19:2 unknown pnpinfo vendor=3D0x8086 device=3D0x6fab subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D19 function=3D3 = dbsf=3Dpci0:255:19:3 unknown pnpinfo vendor=3D0x8086 device=3D0x6fac subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D19 function=3D4 = dbsf=3Dpci0:255:19:4 unknown pnpinfo vendor=3D0x8086 device=3D0x6fad subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D19 function=3D5 = dbsf=3Dpci0:255:19:5 unknown pnpinfo vendor=3D0x8086 device=3D0x6fae subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D19 function=3D6 = dbsf=3Dpci0:255:19:6 unknown pnpinfo vendor=3D0x8086 device=3D0x6faf subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D19 function=3D7 = dbsf=3Dpci0:255:19:7 unknown pnpinfo vendor=3D0x8086 device=3D0x6fb0 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D20 function=3D0 = dbsf=3Dpci0:255:20:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6fb1 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D20 function=3D1 = dbsf=3Dpci0:255:20:1 unknown pnpinfo vendor=3D0x8086 device=3D0x6fb2 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D20 function=3D2 = dbsf=3Dpci0:255:20:2 unknown pnpinfo vendor=3D0x8086 device=3D0x6fb3 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D20 function=3D3 = dbsf=3Dpci0:255:20:3 unknown pnpinfo vendor=3D0x8086 device=3D0x6fbc subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D20 function=3D4 = dbsf=3Dpci0:255:20:4 unknown pnpinfo vendor=3D0x8086 device=3D0x6fbd subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D20 function=3D5 = dbsf=3Dpci0:255:20:5 unknown pnpinfo vendor=3D0x8086 device=3D0x6fbe subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D20 function=3D6 = dbsf=3Dpci0:255:20:6 unknown pnpinfo vendor=3D0x8086 device=3D0x6fbf subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D20 function=3D7 = dbsf=3Dpci0:255:20:7 unknown pnpinfo vendor=3D0x8086 device=3D0x6fb4 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D21 function=3D0 = dbsf=3Dpci0:255:21:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6fb5 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D21 function=3D1 = dbsf=3Dpci0:255:21:1 unknown pnpinfo vendor=3D0x8086 device=3D0x6fb6 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D21 function=3D2 = dbsf=3Dpci0:255:21:2 unknown pnpinfo vendor=3D0x8086 device=3D0x6fb7 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D21 function=3D3 = dbsf=3Dpci0:255:21:3 unknown pnpinfo vendor=3D0x8086 device=3D0x6f68 subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D22 function=3D0 = dbsf=3Dpci0:255:22:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f6e subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D22 function=3D6 = dbsf=3Dpci0:255:22:6 unknown pnpinfo vendor=3D0x8086 device=3D0x6f6f subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D22 function=3D7 = dbsf=3Dpci0:255:22:7 unknown pnpinfo vendor=3D0x8086 device=3D0x6fd0 subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D23 function=3D0 = dbsf=3Dpci0:255:23:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6fb8 subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D23 function=3D4 = dbsf=3Dpci0:255:23:4 unknown pnpinfo vendor=3D0x8086 device=3D0x6fb9 subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D23 function=3D5 = dbsf=3Dpci0:255:23:5 unknown pnpinfo vendor=3D0x8086 device=3D0x6fba subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D23 function=3D6 = dbsf=3Dpci0:255:23:6 unknown pnpinfo vendor=3D0x8086 device=3D0x6fbb subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D23 function=3D7 = dbsf=3Dpci0:255:23:7 unknown pnpinfo vendor=3D0x8086 device=3D0x6f98 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D30 function=3D0 = dbsf=3Dpci0:255:30:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f99 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D30 function=3D1 = dbsf=3Dpci0:255:30:1 unknown pnpinfo vendor=3D0x8086 device=3D0x6f9a subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D30 function=3D2 = dbsf=3Dpci0:255:30:2 unknown pnpinfo vendor=3D0x8086 device=3D0x6fc0 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D30 function=3D3 = dbsf=3Dpci0:255:30:3 unknown pnpinfo vendor=3D0x8086 device=3D0x6f9c subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D30 function=3D4 = dbsf=3Dpci0:255:30:4 unknown pnpinfo vendor=3D0x8086 device=3D0x6f88 subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D31 function=3D0 = dbsf=3Dpci0:255:31:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f8a subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D31 function=3D2 = dbsf=3Dpci0:255:31:2 pcib1 pnpinfo _HID=3DPNP0A03 _UID=3D63 at handle=3D\_SB_.UNC0 pci1 unknown pnpinfo vendor=3D0x8086 device=3D0x6f80 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D8 function=3D0 = dbsf=3Dpci0:127:8:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f32 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x110100 at slot=3D8 function=3D2 = dbsf=3Dpci0:127:8:2 unknown pnpinfo vendor=3D0x8086 device=3D0x6f83 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D8 function=3D3 = dbsf=3Dpci0:127:8:3 unknown pnpinfo vendor=3D0x8086 device=3D0x6f90 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D9 function=3D0 = dbsf=3Dpci0:127:9:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f33 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x110100 at slot=3D9 function=3D2 = dbsf=3Dpci0:127:9:2 unknown pnpinfo vendor=3D0x8086 device=3D0x6f93 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D9 function=3D3 = dbsf=3Dpci0:127:9:3 unknown pnpinfo vendor=3D0x8086 device=3D0x6f81 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D11 function=3D0 = dbsf=3Dpci0:127:11:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f36 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x110100 at slot=3D11 function=3D1 = dbsf=3Dpci0:127:11:1 unknown pnpinfo vendor=3D0x8086 device=3D0x6f37 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x110100 at slot=3D11 function=3D2 = dbsf=3Dpci0:127:11:2 unknown pnpinfo vendor=3D0x8086 device=3D0x6f76 subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D11 function=3D3 = dbsf=3Dpci0:127:11:3 unknown pnpinfo vendor=3D0x8086 device=3D0x6fe0 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D12 function=3D0 = dbsf=3Dpci0:127:12:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6fe1 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D12 function=3D1 = dbsf=3Dpci0:127:12:1 unknown pnpinfo vendor=3D0x8086 device=3D0x6fe2 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D12 function=3D2 = dbsf=3Dpci0:127:12:2 unknown pnpinfo vendor=3D0x8086 device=3D0x6fe3 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D12 function=3D3 = dbsf=3Dpci0:127:12:3 unknown pnpinfo vendor=3D0x8086 device=3D0x6fe4 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D12 function=3D4 = dbsf=3Dpci0:127:12:4 unknown pnpinfo vendor=3D0x8086 device=3D0x6fe5 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D12 function=3D5 = dbsf=3Dpci0:127:12:5 unknown pnpinfo vendor=3D0x8086 device=3D0x6fe6 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D12 function=3D6 = dbsf=3Dpci0:127:12:6 unknown pnpinfo vendor=3D0x8086 device=3D0x6fe7 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D12 function=3D7 = dbsf=3Dpci0:127:12:7 unknown pnpinfo vendor=3D0x8086 device=3D0x6fe8 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D13 function=3D0 = dbsf=3Dpci0:127:13:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6fe9 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D13 function=3D1 = dbsf=3Dpci0:127:13:1 unknown pnpinfo vendor=3D0x8086 device=3D0x6ff8 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D15 function=3D0 = dbsf=3Dpci0:127:15:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6ff9 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D15 function=3D1 = dbsf=3Dpci0:127:15:1 unknown pnpinfo vendor=3D0x8086 device=3D0x6ffc subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D15 function=3D4 = dbsf=3Dpci0:127:15:4 unknown pnpinfo vendor=3D0x8086 device=3D0x6ffd subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D15 function=3D5 = dbsf=3Dpci0:127:15:5 unknown pnpinfo vendor=3D0x8086 device=3D0x6ffe subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D15 function=3D6 = dbsf=3Dpci0:127:15:6 unknown pnpinfo vendor=3D0x8086 device=3D0x6f1d subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D16 function=3D0 = dbsf=3Dpci0:127:16:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f34 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x110100 at slot=3D16 function=3D1 = dbsf=3Dpci0:127:16:1 unknown pnpinfo vendor=3D0x8086 device=3D0x6f1e subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D16 function=3D5 = dbsf=3Dpci0:127:16:5 unknown pnpinfo vendor=3D0x8086 device=3D0x6f7d subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x110100 at slot=3D16 function=3D6 = dbsf=3Dpci0:127:16:6 unknown pnpinfo vendor=3D0x8086 device=3D0x6f1f subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D16 function=3D7 = dbsf=3Dpci0:127:16:7 unknown pnpinfo vendor=3D0x8086 device=3D0x6fa0 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D18 function=3D0 = dbsf=3Dpci0:127:18:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f30 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x110100 at slot=3D18 function=3D1 = dbsf=3Dpci0:127:18:1 unknown pnpinfo vendor=3D0x8086 device=3D0x6fa8 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D19 function=3D0 = dbsf=3Dpci0:127:19:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f71 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D19 function=3D1 = dbsf=3Dpci0:127:19:1 unknown pnpinfo vendor=3D0x8086 device=3D0x6faa subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D19 function=3D2 = dbsf=3Dpci0:127:19:2 unknown pnpinfo vendor=3D0x8086 device=3D0x6fab subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D19 function=3D3 = dbsf=3Dpci0:127:19:3 unknown pnpinfo vendor=3D0x8086 device=3D0x6fac subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D19 function=3D4 = dbsf=3Dpci0:127:19:4 unknown pnpinfo vendor=3D0x8086 device=3D0x6fad subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D19 function=3D5 = dbsf=3Dpci0:127:19:5 unknown pnpinfo vendor=3D0x8086 device=3D0x6fae subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D19 function=3D6 = dbsf=3Dpci0:127:19:6 unknown pnpinfo vendor=3D0x8086 device=3D0x6faf subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D19 function=3D7 = dbsf=3Dpci0:127:19:7 unknown pnpinfo vendor=3D0x8086 device=3D0x6fb0 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D20 function=3D0 = dbsf=3Dpci0:127:20:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6fb1 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D20 function=3D1 = dbsf=3Dpci0:127:20:1 unknown pnpinfo vendor=3D0x8086 device=3D0x6fb2 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D20 function=3D2 = dbsf=3Dpci0:127:20:2 unknown pnpinfo vendor=3D0x8086 device=3D0x6fb3 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D20 function=3D3 = dbsf=3Dpci0:127:20:3 unknown pnpinfo vendor=3D0x8086 device=3D0x6fbc subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D20 function=3D4 = dbsf=3Dpci0:127:20:4 unknown pnpinfo vendor=3D0x8086 device=3D0x6fbd subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D20 function=3D5 = dbsf=3Dpci0:127:20:5 unknown pnpinfo vendor=3D0x8086 device=3D0x6fbe subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D20 function=3D6 = dbsf=3Dpci0:127:20:6 unknown pnpinfo vendor=3D0x8086 device=3D0x6fbf subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D20 function=3D7 = dbsf=3Dpci0:127:20:7 unknown pnpinfo vendor=3D0x8086 device=3D0x6fb4 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D21 function=3D0 = dbsf=3Dpci0:127:21:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6fb5 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D21 function=3D1 = dbsf=3Dpci0:127:21:1 unknown pnpinfo vendor=3D0x8086 device=3D0x6fb6 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D21 function=3D2 = dbsf=3Dpci0:127:21:2 unknown pnpinfo vendor=3D0x8086 device=3D0x6fb7 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D21 function=3D3 = dbsf=3Dpci0:127:21:3 unknown pnpinfo vendor=3D0x8086 device=3D0x6f68 subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D22 function=3D0 = dbsf=3Dpci0:127:22:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f6e subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D22 function=3D6 = dbsf=3Dpci0:127:22:6 unknown pnpinfo vendor=3D0x8086 device=3D0x6f6f subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D22 function=3D7 = dbsf=3Dpci0:127:22:7 unknown pnpinfo vendor=3D0x8086 device=3D0x6fd0 subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D23 function=3D0 = dbsf=3Dpci0:127:23:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6fb8 subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D23 function=3D4 = dbsf=3Dpci0:127:23:4 unknown pnpinfo vendor=3D0x8086 device=3D0x6fb9 subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D23 function=3D5 = dbsf=3Dpci0:127:23:5 unknown pnpinfo vendor=3D0x8086 device=3D0x6fba subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D23 function=3D6 = dbsf=3Dpci0:127:23:6 unknown pnpinfo vendor=3D0x8086 device=3D0x6fbb subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D23 function=3D7 = dbsf=3Dpci0:127:23:7 unknown pnpinfo vendor=3D0x8086 device=3D0x6f98 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D30 function=3D0 = dbsf=3Dpci0:127:30:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f99 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D30 function=3D1 = dbsf=3Dpci0:127:30:1 unknown pnpinfo vendor=3D0x8086 device=3D0x6f9a subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D30 function=3D2 = dbsf=3Dpci0:127:30:2 unknown pnpinfo vendor=3D0x8086 device=3D0x6fc0 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D30 function=3D3 = dbsf=3Dpci0:127:30:3 unknown pnpinfo vendor=3D0x8086 device=3D0x6f9c subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D30 function=3D4 = dbsf=3Dpci0:127:30:4 unknown pnpinfo vendor=3D0x8086 device=3D0x6f88 subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D31 function=3D0 = dbsf=3Dpci0:127:31:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f8a subvendor=3D0x0000= subdevice=3D0x0000 class=3D0x088000 at slot=3D31 function=3D2 = dbsf=3Dpci0:127:31:2 unknown pnpinfo _HID=3DACPI0004 _UID=3D0 at handle=3D\_SB_.SCK0 cpu0 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP00 = _PXM=3D0 acpi_perf0 est0 p4tcc0 acpi_throttle0 cpufreq0 cpu1 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP01 = _PXM=3D0 acpi_perf1 est1 p4tcc1 acpi_throttle1 cpufreq1 cpu2 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP02 = _PXM=3D0 acpi_perf2 est2 p4tcc2 acpi_throttle2 cpufreq2 cpu3 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP03 = _PXM=3D0 acpi_perf3 est3 p4tcc3 acpi_throttle3 cpufreq3 cpu4 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP04 = _PXM=3D0 acpi_perf4 est4 p4tcc4 acpi_throttle4 cpufreq4 cpu5 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP05 = _PXM=3D0 acpi_perf5 est5 p4tcc5 acpi_throttle5 cpufreq5 cpu6 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP06 = _PXM=3D0 acpi_perf6 est6 p4tcc6 acpi_throttle6 cpufreq6 cpu7 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP07 = _PXM=3D0 acpi_perf7 est7 p4tcc7 acpi_throttle7 cpufreq7 cpu8 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP08 = _PXM=3D0 acpi_perf8 est8 p4tcc8 acpi_throttle8 cpufreq8 cpu9 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP09 = _PXM=3D0 acpi_perf9 est9 p4tcc9 acpi_throttle9 cpufreq9 cpu10 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP0A = _PXM=3D0 acpi_perf10 est10 p4tcc10 acpi_throttle10 cpufreq10 cpu11 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP0B = _PXM=3D0 acpi_perf11 est11 p4tcc11 acpi_throttle11 cpufreq11 cpu12 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP0C = _PXM=3D0 acpi_perf12 est12 p4tcc12 acpi_throttle12 cpufreq12 cpu13 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP0D = _PXM=3D0 acpi_perf13 est13 p4tcc13 acpi_throttle13 cpufreq13 cpu14 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP0E = _PXM=3D0 acpi_perf14 est14 p4tcc14 acpi_throttle14 cpufreq14 cpu15 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP0F = _PXM=3D0 acpi_perf15 est15 p4tcc15 acpi_throttle15 cpufreq15 cpu16 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP10 = _PXM=3D0 acpi_perf16 est16 p4tcc16 acpi_throttle16 cpufreq16 cpu17 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP11 = _PXM=3D0 acpi_perf17 est17 p4tcc17 acpi_throttle17 cpufreq17 cpu18 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP12 = _PXM=3D0 acpi_perf18 est18 p4tcc18 acpi_throttle18 cpufreq18 cpu19 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP13 = _PXM=3D0 acpi_perf19 est19 p4tcc19 acpi_throttle19 cpufreq19 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP14 = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP15 = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP16 = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP17 = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP18 = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP19 = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP1A = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP1B = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP1C = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP1D = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP1E = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP1F = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP20 = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP21 = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP22 = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP23 = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP24 = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP25 = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP26 = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP27 = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP28 = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP29 = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP2A = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP2B = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP2C = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP2D = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP2E = _PXM=3D0 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK0.CP2F = _PXM=3D0 unknown pnpinfo _HID=3DACPI0004 _UID=3D0 at handle=3D\_SB_.SCK1 cpu20 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP00 = _PXM=3D1 acpi_perf20 est20 p4tcc20 acpi_throttle20 cpufreq20 cpu21 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP01 = _PXM=3D1 acpi_perf21 est21 p4tcc21 acpi_throttle21 cpufreq21 cpu22 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP02 = _PXM=3D1 acpi_perf22 est22 p4tcc22 acpi_throttle22 cpufreq22 cpu23 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP03 = _PXM=3D1 acpi_perf23 est23 p4tcc23 acpi_throttle23 cpufreq23 cpu24 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP04 = _PXM=3D1 acpi_perf24 est24 p4tcc24 acpi_throttle24 cpufreq24 cpu25 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP05 = _PXM=3D1 acpi_perf25 est25 p4tcc25 acpi_throttle25 cpufreq25 cpu26 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP06 = _PXM=3D1 acpi_perf26 est26 p4tcc26 acpi_throttle26 cpufreq26 cpu27 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP07 = _PXM=3D1 acpi_perf27 est27 p4tcc27 acpi_throttle27 cpufreq27 cpu28 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP08 = _PXM=3D1 acpi_perf28 est28 p4tcc28 acpi_throttle28 cpufreq28 cpu29 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP09 = _PXM=3D1 acpi_perf29 est29 p4tcc29 acpi_throttle29 cpufreq29 cpu30 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP0A = _PXM=3D1 acpi_perf30 est30 p4tcc30 acpi_throttle30 cpufreq30 cpu31 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP0B = _PXM=3D1 acpi_perf31 est31 p4tcc31 acpi_throttle31 cpufreq31 cpu32 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP0C = _PXM=3D1 acpi_perf32 est32 p4tcc32 acpi_throttle32 cpufreq32 cpu33 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP0D = _PXM=3D1 acpi_perf33 est33 p4tcc33 acpi_throttle33 cpufreq33 cpu34 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP0E = _PXM=3D1 acpi_perf34 est34 p4tcc34 acpi_throttle34 cpufreq34 cpu35 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP0F = _PXM=3D1 acpi_perf35 est35 p4tcc35 acpi_throttle35 cpufreq35 cpu36 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP10 = _PXM=3D1 acpi_perf36 est36 p4tcc36 acpi_throttle36 cpufreq36 cpu37 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP11 = _PXM=3D1 acpi_perf37 est37 p4tcc37 acpi_throttle37 cpufreq37 cpu38 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP12 = _PXM=3D1 acpi_perf38 est38 p4tcc38 acpi_throttle38 cpufreq38 cpu39 pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP13 = _PXM=3D1 acpi_perf39 est39 p4tcc39 acpi_throttle39 cpufreq39 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP14 = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP15 = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP16 = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP17 = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP18 = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP19 = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP1A = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP1B = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP1C = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP1D = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP1E = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP1F = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP20 = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP21 = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP22 = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP23 = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP24 = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP25 = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP26 = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP27 = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP28 = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP29 = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP2A = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP2B = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP2C = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP2D = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP2E = _PXM=3D1 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK1.CP2F = _PXM=3D1 unknown pnpinfo _HID=3DACPI0004 _UID=3D0 at handle=3D\_SB_.SCK2 = (disabled) unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP00 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP01 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP02 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP03 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP04 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP05 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP06 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP07 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP08 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP09 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP0A = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP0B = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP0C = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP0D = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP0E = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP0F = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP10 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP11 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP12 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP13 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP14 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP15 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP16 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP17 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP18 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP19 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP1A = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP1B = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP1C = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP1D = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP1E = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP1F = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP20 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP21 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP22 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP23 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP24 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP25 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP26 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP27 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP28 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP29 = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP2A = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP2B = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP2C = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP2D = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP2E = _PXM=3D2 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK2.CP2F = _PXM=3D2 unknown pnpinfo _HID=3DACPI0004 _UID=3D0 at handle=3D\_SB_.SCK3 = (disabled) unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP00 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP01 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP02 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP03 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP04 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP05 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP06 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP07 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP08 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP09 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP0A = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP0B = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP0C = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP0D = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP0E = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP0F = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP10 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP11 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP12 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP13 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP14 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP15 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP16 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP17 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP18 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP19 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP1A = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP1B = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP1C = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP1D = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP1E = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP1F = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP20 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP21 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP22 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP23 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP24 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP25 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP26 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP27 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP28 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP29 = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP2A = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP2B = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP2C = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP2D = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP2E = _PXM=3D3 unknown pnpinfo _HID=3DACPI0007 _UID=3D0 at handle=3D\_SB_.SCK3.CP2F = _PXM=3D3 pcib2 pnpinfo _HID=3DPNP0A08 _UID=3D0 at handle=3D\_SB_.PCI0 _PXM=3D0 pci2 hostb0 pnpinfo vendor=3D0x8086 device=3D0x6f00 subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x060000 at slot=3D0 function=3D0 = dbsf=3Dpci0:0:0:0 handle=3D\_SB_.PCI0.IIOP pcib3 pnpinfo vendor=3D0x8086 device=3D0x6f02 subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x060400 at slot=3D1 function=3D0 = dbsf=3Dpci0:0:1:0 handle=3D\_SB_.PCI0.BR1A pci3 ix0 pnpinfo vendor=3D0x8086 device=3D0x1528 subvendor=3D0x15d9= subdevice=3D0x088a class=3D0x020000 at slot=3D0 function=3D0 = dbsf=3Dpci0:1:0:0 handle=3D\_SB_.PCI0.BR1A.H000 ix1 pnpinfo vendor=3D0x8086 device=3D0x1528 subvendor=3D0x15d9= subdevice=3D0x088a class=3D0x020000 at slot=3D0 function=3D1 = dbsf=3Dpci0:1:0:1 handle=3D\_SB_.PCI0.BR1A.H001 pcib4 pnpinfo vendor=3D0x8086 device=3D0x6f04 subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x060400 at slot=3D2 function=3D0 = dbsf=3Dpci0:0:2:0 handle=3D\_SB_.PCI0.BR2A pci4 nvme0 pnpinfo vendor=3D0x8086 device=3D0x0953 = subvendor=3D0x8086 subdevice=3D0x3705 class=3D0x010802 at slot=3D0 = function=3D0 dbsf=3Dpci0:2:0:0 handle=3D\_SB_.PCI0.BR2A.H000 pcib5 pnpinfo vendor=3D0x8086 device=3D0x6f05 subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x060400 at slot=3D2 function=3D1 = dbsf=3Dpci0:0:2:1 handle=3D\_SB_.PCI0.BR2B pci5 nvme1 pnpinfo vendor=3D0x8086 device=3D0x0953 = subvendor=3D0x8086 subdevice=3D0x3705 class=3D0x010802 at slot=3D0 = function=3D0 dbsf=3Dpci0:3:0:0 handle=3D\_SB_.PCI0.BR2B.H000 pcib6 pnpinfo vendor=3D0x8086 device=3D0x6f06 subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x060400 at slot=3D2 function=3D2 = dbsf=3Dpci0:0:2:2 handle=3D\_SB_.PCI0.BR2C pci6 pcib7 pnpinfo vendor=3D0x8086 device=3D0x6f08 subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x060400 at slot=3D3 function=3D0 = dbsf=3Dpci0:0:3:0 handle=3D\_SB_.PCI0.BR3A pci7 nvme2 pnpinfo vendor=3D0x8086 device=3D0x0953 = subvendor=3D0x8086 subdevice=3D0x3705 class=3D0x010802 at slot=3D0 = function=3D0 dbsf=3Dpci0:5:0:0 handle=3D\_SB_.PCI0.BR3A.H000 pcib8 pnpinfo vendor=3D0x8086 device=3D0x6f09 subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x060400 at slot=3D3 function=3D1 = dbsf=3Dpci0:0:3:1 handle=3D\_SB_.PCI0.BR3B pci8 nvme3 pnpinfo vendor=3D0x8086 device=3D0x0953 = subvendor=3D0x8086 subdevice=3D0x3705 class=3D0x010802 at slot=3D0 = function=3D0 dbsf=3Dpci0:6:0:0 handle=3D\_SB_.PCI0.BR3B.H000 pcib9 pnpinfo vendor=3D0x8086 device=3D0x6f0a subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x060400 at slot=3D3 function=3D2 = dbsf=3Dpci0:0:3:2 handle=3D\_SB_.PCI0.BR3C pci9 nvme4 pnpinfo vendor=3D0x8086 device=3D0x0953 = subvendor=3D0x8086 subdevice=3D0x3705 class=3D0x010802 at slot=3D0 = function=3D0 dbsf=3Dpci0:7:0:0 handle=3D\_SB_.PCI0.BR3C.H000 pcib10 pnpinfo vendor=3D0x8086 device=3D0x6f0b subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x060400 at slot=3D3 function=3D3 = dbsf=3Dpci0:0:3:3 handle=3D\_SB_.PCI0.BR3D pci10 nvme5 pnpinfo vendor=3D0x8086 device=3D0x0953 = subvendor=3D0x8086 subdevice=3D0x3705 class=3D0x010802 at slot=3D0 = function=3D0 dbsf=3Dpci0:8:0:0 handle=3D\_SB_.PCI0.BR3D.H000 unknown pnpinfo vendor=3D0x8086 device=3D0x6f20 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D4 function=3D0 = dbsf=3Dpci0:0:4:0 handle=3D\_SB_.PCI0.CB0A unknown pnpinfo vendor=3D0x8086 device=3D0x6f21 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D4 function=3D1 = dbsf=3Dpci0:0:4:1 handle=3D\_SB_.PCI0.CB0B unknown pnpinfo vendor=3D0x8086 device=3D0x6f22 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D4 function=3D2 = dbsf=3Dpci0:0:4:2 handle=3D\_SB_.PCI0.CB0C unknown pnpinfo vendor=3D0x8086 device=3D0x6f23 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D4 function=3D3 = dbsf=3Dpci0:0:4:3 handle=3D\_SB_.PCI0.CB0D unknown pnpinfo vendor=3D0x8086 device=3D0x6f24 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D4 function=3D4 = dbsf=3Dpci0:0:4:4 handle=3D\_SB_.PCI0.CB0E unknown pnpinfo vendor=3D0x8086 device=3D0x6f25 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D4 function=3D5 = dbsf=3Dpci0:0:4:5 handle=3D\_SB_.PCI0.CB0F unknown pnpinfo vendor=3D0x8086 device=3D0x6f26 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D4 function=3D6 = dbsf=3Dpci0:0:4:6 handle=3D\_SB_.PCI0.CB0G unknown pnpinfo vendor=3D0x8086 device=3D0x6f27 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D4 function=3D7 = dbsf=3Dpci0:0:4:7 handle=3D\_SB_.PCI0.CB0H unknown pnpinfo vendor=3D0x8086 device=3D0x6f28 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D5 function=3D0 = dbsf=3Dpci0:0:5:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f29 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D5 function=3D1 = dbsf=3Dpci0:0:5:1 handle=3D\_SB_.PCI0.MHP0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f2a subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D5 function=3D2 = dbsf=3Dpci0:0:5:2 ioapic0 pnpinfo vendor=3D0x8086 device=3D0x6f2c subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x080020 at slot=3D5 function=3D4 = dbsf=3Dpci0:0:5:4 unknown pnpinfo vendor=3D0x8086 device=3D0x8d7c subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0xff0000 at slot=3D17 function=3D0 = dbsf=3Dpci0:0:17:0 handle=3D\_SB_.PCI0.EVMR ahci0 pnpinfo vendor=3D0x8086 device=3D0x2827 subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x010400 at slot=3D17 function=3D4 = dbsf=3Dpci0:0:17:4 handle=3D\_SB_.PCI0.EVSS ahcich0 at channel=3D0 ahcich1 at channel=3D1 ahcich2 at channel=3D2 ahcich3 at channel=3D3 ahciem0 unknown pnpinfo vendor=3D0x8086 device=3D0x8d3a subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x078000 at slot=3D22 function=3D0 = dbsf=3Dpci0:0:22:0 handle=3D\_SB_.PCI0.HECI unknown pnpinfo vendor=3D0x8086 device=3D0x8d3b subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x078000 at slot=3D22 function=3D1 = dbsf=3Dpci0:0:22:1 handle=3D\_SB_.PCI0.HEC2 ehci0 pnpinfo vendor=3D0x8086 device=3D0x8d2d subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x0c0320 at slot=3D26 function=3D0 = dbsf=3Dpci0:0:26:0 handle=3D\_SB_.PCI0.EHC2 usbus0 uhub0 uhub3 pnpinfo vendor=3D0x8087 product=3D0x800a = devclass=3D0x09 devsubclass=3D0x00 devproto=3D0x01 sernum=3D"" = release=3D0x0005 mode=3Dhost intclass=3D0x09 ints at bus=3D0 hubaddr=3D1 = port=3D1 devaddr=3D2 interface=3D0 ugen=3Dugen0.2 umass0 pnpinfo vendor=3D0x048d product=3D0x1177 = devclass=3D0x00 devsubclass=3D0x00 devproto=3D0x00 sernum=3D"" = release=3D0x0100 mode=3Dhost intclass=3D0x08 ints at bus=3D0 hubaddr=3D2 = port=3D1 devaddr=3D3 interface=3D0 ugen=3Dugen0.3 pcib11 pnpinfo vendor=3D0x8086 device=3D0x8d10 subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x060400 at slot=3D28 function=3D0 = dbsf=3Dpci0:0:28:0 handle=3D\_SB_.PCI0.RP01 pci11 pcib12 pnpinfo vendor=3D0x8086 device=3D0x8d14 subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x060400 at slot=3D28 function=3D2 = dbsf=3Dpci0:0:28:2 handle=3D\_SB_.PCI0.RP03 pci12 pcib13 pnpinfo vendor=3D0x1a03 device=3D0x1150 = subvendor=3D0x15d9 subdevice=3D0x0844 class=3D0x060400 at slot=3D0 = function=3D0 dbsf=3Dpci0:10:0:0 handle=3D\_SB_.PCI0.RP03.BR88 pci13 vgapci0 pnpinfo vendor=3D0x1a03 device=3D0x2000 = subvendor=3D0x15d9 subdevice=3D0x0844 class=3D0x030000 at slot=3D0 = function=3D0 dbsf=3Dpci0:11:0:0 handle=3D\_SB_.PCI0.RP03.BR88.D089 drm0 drmn0 ehci1 pnpinfo vendor=3D0x8086 device=3D0x8d26 subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x0c0320 at slot=3D29 function=3D0 = dbsf=3Dpci0:0:29:0 handle=3D\_SB_.PCI0.EHC1 usbus1 uhub1 uhub2 pnpinfo vendor=3D0x8087 product=3D0x8002 = devclass=3D0x09 devsubclass=3D0x00 devproto=3D0x01 sernum=3D"" = release=3D0x0005 mode=3Dhost intclass=3D0x09 ints at bus=3D1 hubaddr=3D1 = port=3D1 devaddr=3D2 interface=3D0 ugen=3Dugen1.2 uhub4 pnpinfo vendor=3D0x0557 product=3D0x7000 = devclass=3D0x09 devsubclass=3D0x00 devproto=3D0x01 sernum=3D"" = release=3D0x0000 mode=3Dhost intclass=3D0x09 ints at bus=3D1 hubaddr=3D2 = port=3D8 devaddr=3D3 interface=3D0 ugen=3Dugen1.3 ukbd0 pnpinfo vendor=3D0x0557 product=3D0x2419 = devclass=3D0x00 devsubclass=3D0x00 devproto=3D0x00 sernum=3D"" = release=3D0x0100 mode=3Dhost intclass=3D0x03 ints at bus=3D1 hubaddr=3D3 = port=3D1 devaddr=3D4 interface=3D0 ugen=3Dugen1.4 ums0 pnpinfo vendor=3D0x0557 product=3D0x2419 = devclass=3D0x00 devsubclass=3D0x00 devproto=3D0x00 sernum=3D"" = release=3D0x0100 mode=3Dhost intclass=3D0x03 ints at bus=3D1 hubaddr=3D3 = port=3D1 devaddr=3D4 interface=3D1 ugen=3Dugen1.4 isab0 pnpinfo vendor=3D0x8086 device=3D0x8d44 subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x060100 at slot=3D31 function=3D0 = dbsf=3Dpci0:0:31:0 handle=3D\_SB_.PCI0.LPC0 isa0 sc0 vga0 orm0 atkbdc0 fdc0 ppc0 ahci1 pnpinfo vendor=3D0x8086 device=3D0x8d02 subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x010601 at slot=3D31 function=3D2 = dbsf=3Dpci0:0:31:2 handle=3D\_SB_.PCI0.SAT1 ahcich4 at channel=3D0 ahcich5 at channel=3D1 ahcich6 at channel=3D2 ahcich7 at channel=3D3 ahcich8 at channel=3D4 ahcich9 at channel=3D5 ahciem1 unknown pnpinfo vendor=3D0x8086 device=3D0x8d22 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x0c0500 at slot=3D31 function=3D3 = dbsf=3Dpci0:0:31:3 handle=3D\_SB_.PCI0.SMBS unknown pnpinfo vendor=3D0x8086 device=3D0x8d24 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x118000 at slot=3D31 function=3D6 = dbsf=3Dpci0:0:31:6 handle=3D\_SB_.PCI0.TERM unknown pnpinfo _HID=3DPNP0003 _UID=3D0 at handle=3D\_SB_.PCI0.APIC atdma0 pnpinfo _HID=3DPNP0200 _UID=3D0 at = handle=3D\_SB_.PCI0.LPC0.DMAC atrtc0 pnpinfo _HID=3DPNP0B00 _UID=3D0 at = handle=3D\_SB_.PCI0.LPC0.RTC_ unknown pnpinfo _HID=3DPNP0000 _UID=3D0 at = handle=3D\_SB_.PCI0.LPC0.PIC_ fpupnp0 pnpinfo _HID=3DPNP0C04 _UID=3D0 at = handle=3D\_SB_.PCI0.LPC0.FPU_ attimer0 pnpinfo _HID=3DPNP0100 _UID=3D0 at = handle=3D\_SB_.PCI0.LPC0.TMR_ unknown pnpinfo _HID=3DPNP0800 _UID=3D0 at = handle=3D\_SB_.PCI0.LPC0.SPKR hpet0 pnpinfo _HID=3DPNP0103 _UID=3D0 at handle=3D\_SB_.PCI0.LPC0.HPET= acpi_sysresource0 pnpinfo _HID=3DPNP0C02 _UID=3D16 at = handle=3D\_SB_.PCI0.LPC0.XTRA acpi_sysresource1 pnpinfo _HID=3DPNP0C02 _UID=3D0 at = handle=3D\_SB_.PCI0.LPC0.SIO1 uart0 pnpinfo _HID=3DPNP0501 _UID=3D0 at handle=3D\_SB_.PCI0.LPC0.UAR1= uart1 pnpinfo _HID=3DPNP0501 _UID=3D1 at handle=3D\_SB_.PCI0.LPC0.UAR2= unknown pnpinfo _HID=3DIPI0001 _UID=3D0 at = handle=3D\_SB_.PCI0.LPC0.SPMI pcib14 pnpinfo _HID=3DPNP0A08 _UID=3D64 at handle=3D\_SB_.PCI1 = _PXM=3D1 pci14 pcib15 pnpinfo vendor=3D0x8086 device=3D0x6f01 subvendor=3D0x8086 = subdevice=3D0x0000 class=3D0x060400 at slot=3D0 function=3D0 = dbsf=3Dpci0:128:0:0 handle=3D\_SB_.PCI1.IIOP pci15 pcib16 pnpinfo vendor=3D0x8086 device=3D0x6f02 subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x060400 at slot=3D1 function=3D0 = dbsf=3Dpci0:128:1:0 handle=3D\_SB_.PCI1.QR1A pci16 ix2 pnpinfo vendor=3D0x8086 device=3D0x10fb subvendor=3D0x15d9= subdevice=3D0x0611 class=3D0x020000 at slot=3D0 function=3D0 = dbsf=3Dpci0:130:0:0 handle=3D\_SB_.PCI1.QR1A.H000 ix3 pnpinfo vendor=3D0x8086 device=3D0x10fb subvendor=3D0x15d9= subdevice=3D0x0611 class=3D0x020000 at slot=3D0 function=3D1 = dbsf=3Dpci0:130:0:1 handle=3D\_SB_.PCI1.QR1A.H001 pcib17 pnpinfo vendor=3D0x8086 device=3D0x6f04 subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x060400 at slot=3D2 function=3D0 = dbsf=3Dpci0:128:2:0 handle=3D\_SB_.PCI1.QR2A pci17 nvme6 pnpinfo vendor=3D0x8086 device=3D0x0953 = subvendor=3D0x8086 subdevice=3D0x3705 class=3D0x010802 at slot=3D0 = function=3D0 dbsf=3Dpci0:131:0:0 handle=3D\_SB_.PCI1.QR2A.H000 pcib18 pnpinfo vendor=3D0x8086 device=3D0x6f05 subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x060400 at slot=3D2 function=3D1 = dbsf=3Dpci0:128:2:1 handle=3D\_SB_.PCI1.QR2B pci18 nvme7 pnpinfo vendor=3D0x8086 device=3D0x0953 = subvendor=3D0x8086 subdevice=3D0x3705 class=3D0x010802 at slot=3D0 = function=3D0 dbsf=3Dpci0:132:0:0 handle=3D\_SB_.PCI1.QR2B.H000 pcib19 pnpinfo vendor=3D0x8086 device=3D0x6f06 subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x060400 at slot=3D2 function=3D2 = dbsf=3Dpci0:128:2:2 handle=3D\_SB_.PCI1.QR2C pci19 nvme8 pnpinfo vendor=3D0x8086 device=3D0x0953 = subvendor=3D0x8086 subdevice=3D0x3705 class=3D0x010802 at slot=3D0 = function=3D0 dbsf=3Dpci0:133:0:0 handle=3D\_SB_.PCI1.QR2C.H000 pcib20 pnpinfo vendor=3D0x8086 device=3D0x6f07 subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x060400 at slot=3D2 function=3D3 = dbsf=3Dpci0:128:2:3 handle=3D\_SB_.PCI1.QR2D pci20 nvme9 pnpinfo vendor=3D0x8086 device=3D0x0953 = subvendor=3D0x8086 subdevice=3D0x3705 class=3D0x010802 at slot=3D0 = function=3D0 dbsf=3Dpci0:134:0:0 handle=3D\_SB_.PCI1.QR2D.H000 pcib21 pnpinfo vendor=3D0x8086 device=3D0x6f08 subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x060400 at slot=3D3 function=3D0 = dbsf=3Dpci0:128:3:0 handle=3D\_SB_.PCI1.QR3A pci21 pcib22 pnpinfo vendor=3D0x8086 device=3D0x6f0a subvendor=3D0x15d9 = subdevice=3D0x0844 class=3D0x060400 at slot=3D3 function=3D2 = dbsf=3Dpci0:128:3:2 handle=3D\_SB_.PCI1.QR3C pci22 unknown pnpinfo vendor=3D0x8086 device=3D0x6f20 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D4 function=3D0 = dbsf=3Dpci0:128:4:0 handle=3D\_SB_.PCI1.CB1A unknown pnpinfo vendor=3D0x8086 device=3D0x6f21 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D4 function=3D1 = dbsf=3Dpci0:128:4:1 handle=3D\_SB_.PCI1.CB1B unknown pnpinfo vendor=3D0x8086 device=3D0x6f22 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D4 function=3D2 = dbsf=3Dpci0:128:4:2 handle=3D\_SB_.PCI1.CB1C unknown pnpinfo vendor=3D0x8086 device=3D0x6f23 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D4 function=3D3 = dbsf=3Dpci0:128:4:3 handle=3D\_SB_.PCI1.CB1D unknown pnpinfo vendor=3D0x8086 device=3D0x6f24 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D4 function=3D4 = dbsf=3Dpci0:128:4:4 handle=3D\_SB_.PCI1.CB1E unknown pnpinfo vendor=3D0x8086 device=3D0x6f25 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D4 function=3D5 = dbsf=3Dpci0:128:4:5 handle=3D\_SB_.PCI1.CB1F unknown pnpinfo vendor=3D0x8086 device=3D0x6f26 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D4 function=3D6 = dbsf=3Dpci0:128:4:6 handle=3D\_SB_.PCI1.CB1G unknown pnpinfo vendor=3D0x8086 device=3D0x6f27 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D4 function=3D7 = dbsf=3Dpci0:128:4:7 handle=3D\_SB_.PCI1.CB1H unknown pnpinfo vendor=3D0x8086 device=3D0x6f28 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D5 function=3D0 = dbsf=3Dpci0:128:5:0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f29 subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D5 function=3D1 = dbsf=3Dpci0:128:5:1 handle=3D\_SB_.PCI1.MHP0 unknown pnpinfo vendor=3D0x8086 device=3D0x6f2a subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x088000 at slot=3D5 function=3D2 = dbsf=3Dpci0:128:5:2 ioapic1 pnpinfo vendor=3D0x8086 device=3D0x6f2c subvendor=3D0x15d9= subdevice=3D0x0844 class=3D0x080020 at slot=3D5 function=3D4 = dbsf=3Dpci0:128:5:4 unknown pnpinfo _HID=3DPNP0A08 _UID=3D128 at handle=3D\_SB_.PCI2 = _PXM=3D2 (disabled) unknown pnpinfo _HID=3DPNP0A08 _UID=3D192 at handle=3D\_SB_.PCI3 = _PXM=3D3 (disabled) acpi_button0 pnpinfo _HID=3DPNP0C0C _UID=3D170 at handle=3D\_SB_.PWRB unknown pnpinfo _HID=3DPNP0C33 _UID=3D0 at handle=3D\_SB_.WERR unknown pnpinfo _HID=3DPNP0C14 _UID=3D0 at handle=3D\_SB_.SRIO unknown pnpinfo _HID=3DPNP0C31 _UID=3D1 at handle=3D\_SB_.TPM_ = (disabled) pci_link0 pnpinfo _HID=3DPNP0C0F _UID=3D1 at handle=3D\_SB_.LNKA pci_link1 pnpinfo _HID=3DPNP0C0F _UID=3D2 at handle=3D\_SB_.LNKB pci_link2 pnpinfo _HID=3DPNP0C0F _UID=3D3 at handle=3D\_SB_.LNKC pci_link3 pnpinfo _HID=3DPNP0C0F _UID=3D4 at handle=3D\_SB_.LNKD pci_link4 pnpinfo _HID=3DPNP0C0F _UID=3D5 at handle=3D\_SB_.LNKE pci_link5 pnpinfo _HID=3DPNP0C0F _UID=3D6 at handle=3D\_SB_.LNKF pci_link6 pnpinfo _HID=3DPNP0C0F _UID=3D7 at handle=3D\_SB_.LNKG pci_link7 pnpinfo _HID=3DPNP0C0F _UID=3D8 at handle=3D\_SB_.LNKH unknown pnpinfo _HID=3DACPI000D _UID=3D0 at handle=3D\_SB_.PMT1 acpi_timer0 pnpinfo unknown at unknown root@pruebanvme2:~ #=20 From owner-freebsd-stable@freebsd.org Thu Sep 29 14:32:21 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8C9AFC01135 for ; Thu, 29 Sep 2016 14:32:21 +0000 (UTC) (envelope-from dweimer@dweimer.net) Received: from webmail.dweimer.net (24-240-198-188.static.stls.mo.charter.com [24.240.198.188]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 53EFEDF3 for ; Thu, 29 Sep 2016 14:32:20 +0000 (UTC) (envelope-from dweimer@dweimer.net) Received: from webmail.dweimer.local (localhost [10.9.5.2]) by webmail.dweimer.net (8.15.2/8.15.2) with ESMTPS id u8TEWCF7097115 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Thu, 29 Sep 2016 09:32:12 -0500 (CDT) (envelope-from dweimer@dweimer.net) Received: (from www@localhost) by webmail.dweimer.local (8.15.2/8.15.2/Submit) id u8TEWBjs097114; Thu, 29 Sep 2016 09:32:11 -0500 (CDT) (envelope-from dweimer@dweimer.net) X-Authentication-Warning: webmail.dweimer.local: www set sender to dweimer@dweimer.net using -f To: FreeBSD Stable Subject: File Name Too Long? MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 29 Sep 2016 09:32:11 -0500 From: "Dean E. Weimer" Organization: dweimer.net Reply-To: dweimer@dweimer.net Mail-Reply-To: dweimer@dweimer.net Message-ID: <974bc572bedda786fdc18a41085952c1@dweimer.net> X-Sender: dweimer@dweimer.net User-Agent: Roundcube Webmail/1.2.1 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Sep 2016 14:32:21 -0000 I discovered, unfortunately by deleting a jail by accident, that my backup process isn't working. At least it was only the operating system part of the jail, I still have all the data so I just need to reinstall the operating system. While the ports are in the process of building I started to investigate the cause, because the backup logs report everything was fine. I have a custom pre-backup script I wrote that takes snapshots of my ZFS datasets, and then mounts those under /mnt/backup with nullfs mount points to the .zfs/snapshot/.. directories then I back them up rather than the live file system, allowing me to stop some services that don't restore from a running state correctly and then restart after the snapshot so downtime is only a couple of minutes instead of full length of backups. It appeared to be running perfectly, without errors, but apparently the script isn't reporting some nullfs mount failures, so why are the failing, turns out it thinks the file name is too long? but looking at the mount(2) man page it states this: [ENAMETOOLONG] A component of a pathname exceeded 255 characters, or the entire length of a path name exceeded 1023 characters. I can see that at some point under this, I may reach that 1023 limit, but what of the total 71 characters in this path has a problem? /jails/unifi/ROOT/.zfs/snapshot/11.0-RELEASE-r306379-2016.09.28--bsnap root@freebsd:/jails/unifi/ROOT/.zfs/snapshot # ls ls: 11.0-RELEASE-r306379-2016.09.28--bsnap: File name too long I thought, maybe its a ZFS specific error, and ran across this: http://lists.freebsd.org/pipermail/freebsd-fs/2010-March/007964.html [..snip..] From looking at the code, I think you hitting this limit: /* * Be ultra-paranoid about making sure the type and fspath * variables will fit in our mp buffers, including the * terminating NUL. */ if (strlen(fstype) >= MFSNAMELEN || strlen(fspath) >= MNAMELEN) return (ENAMETOOLONG); in vfs_domount() or vfs_donmount(). This is FreeBSD limit caused by statfs structure: /* * filesystem statistics */ [...] #define MNAMELEN 88 /* size of on/from name bufs */ [...] struct statfs { [...] char f_mntfromname[MNAMELEN];/* mounted filesystem */ char f_mntonname[MNAMELEN]; /* directory on which mounted */ }; When you list .zfs/snapshot/ directory (especially with -l option) ZFS mounts snapshots on lookup and this is this mount that fails. [..snip..] I can seemingly due anything else with the snapshot, clone, send, receive its just that I am unable to access the files on it through .zfs/snapshot/.. I am trying to find what the limit is here from this, because this one here works. /jails/webmail/usr-local-subversion/.zfs/snapshot/usr-local-subversion--bsnap its longer in total length than most of the ones that are failing. /jails/unifi/ROOT/.zfs/snapshot/11.0-RELEASE-r306379-2016.09.28--bsnap So it appears that its in the name, and not the mount point. this one works as well, which is my ZFS boot environment on the main system zraid/ROOT/11.0-RELEASE-r306379-2016.09.28 snapshot is /.zfs/snapshot/11.0-RELEASE-r306379-2016.09.28--bsnap So its not just the last component of the zfs dataset name, which is in this case the same. I am trying to wrap my head around this and find where the limit is so I can adjust my naming conventions used and actually get backups of all of my data. Turns out all of my jail operating system paths aren't being backed up, fortunately at least all of the data file systems for the jails are. -- Thanks, Dean E. Weimer http://www.dweimer.net/ From owner-freebsd-stable@freebsd.org Thu Sep 29 15:40:44 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 844B2C025C2 for ; Thu, 29 Sep 2016 15:40:44 +0000 (UTC) (envelope-from shashaness@hotmail.com) Received: from COL004-OMC3S8.hotmail.com (col004-omc3s8.hotmail.com [65.55.34.146]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "*.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 43F781630 for ; Thu, 29 Sep 2016 15:40:43 +0000 (UTC) (envelope-from shashaness@hotmail.com) Received: from NAM02-CY1-obe.outbound.protection.outlook.com ([65.55.34.136]) by COL004-OMC3S8.hotmail.com over TLS secured channel with Microsoft SMTPSVC(7.5.7601.23008); Thu, 29 Sep 2016 08:39:38 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=7QB/ZhW0wxwhJ3C6QVSK7FZuOuulzi9UA69OC/YSVlI=; b=a+QrRj7COxVMJFeqr7x0ASoWyqiTw661YDPDhTcVgp8yJ0qJ91xF8vax0ZXMzbEBnEBTuKV327RuM6r1dQg4grM0wGZd4lOmmseDJrD+OBzM4nBmIiaa3RVhYsxNCt4q2Pziwd37qU9C1fc/FjSK4O3rHGM17SdN8io3C4DrnvzgQK0QapQSuE91PY7RRpdCgcHlc+vUMZkXZPrwiu1qNdNPAs10Yzut3Tfe4CouJYtQIHnIsUmTxFkpfOpw+erMyjxDtXelh5URdeVS3mzog6gSda9V/LtHXhVpTLeaKje/BZ27x2RzR3AeVVJnNnFQ168SUfwBB3gcFDqJGBMoLg== Received: from SN1NAM02FT020.eop-nam02.prod.protection.outlook.com (10.152.72.58) by SN1NAM02HT207.eop-nam02.prod.protection.outlook.com (10.152.73.27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.629.5; Thu, 29 Sep 2016 15:39:38 +0000 Received: from CY1PR14MB0520.namprd14.prod.outlook.com (10.152.72.51) by SN1NAM02FT020.mail.protection.outlook.com (10.152.72.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.629.5 via Frontend Transport; Thu, 29 Sep 2016 15:39:37 +0000 Received: from CY1PR14MB0520.namprd14.prod.outlook.com ([10.164.71.150]) by CY1PR14MB0520.namprd14.prod.outlook.com ([10.164.71.150]) with mapi id 15.01.0639.013; Thu, 29 Sep 2016 15:39:37 +0000 From: Shawn Bakhtiar To: "freebsd-stable@freebsd.org" Subject: Error with auto mount Thread-Topic: Error with auto mount Thread-Index: AQHSGmevOcBzgb4JbUGJe0DkdJdGIA== Date: Thu, 29 Sep 2016 15:39:37 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=softfail (sender IP is 10.152.72.51) smtp.mailfrom=hotmail.com; freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=fail action=none header.from=hotmail.com; received-spf: SoftFail (protection.outlook.com: domain of transitioning hotmail.com discourages use of 10.152.72.51 as permitted sender) x-ms-exchange-messagesentrepresentingtype: 1 x-eopattributedmessage: 0 x-microsoft-exchange-diagnostics: 1; SN1NAM02HT207; 6:y/TLfZv0mv98r3fHiYKexv+S3DkUGuUavOcqPPaJbG8EueBkPEDv6dxiGImRwHpZyg5T61N2xDSqGdbXU9GuoLA1+MCQ7RJy46WnQA/qHXobvlspoBXm8ToAeAsJ1yvFUW7W7cCUYGgaU5NvhqMiu3o3cbuvmZSE6qF+CR1IiB50YG+cdtoX1ek6O58tIby7pHZR8m1/NZutOkKfS62O1x/N+M/MJe9swk/qGgOwvMkW5pDh0eAr1hR0J7EINBpugt0yQSAQuMGRSKuW6kAWOYO3fKsah2AOC2WQiIXrmIs=; 5:0pjPpplVcpdHqogQOeZeXbtTUM1+YLuyJadWLwh+Xrg9XtJBJVfgbYNl8/BPuCCGalzFfZJhC3unMX7hu0vd8SZxRhQpqhiYKx23ybDybfbG8+JwAlLE1zR8s251mv0v+lKHwovJcfgv+QRl6orhOg==; 24:YK29Wd6bZiIyKVcU/ztqSJZKoCMO5MMI5cutp4fO4difFR118hoQAj4zwyGv6TTLjezesCQszesZ2NRv5TW+O4hFZtfDcx/xuLgGvxzlspw=; 7:lIc33ABwB7J8QprbdULWBXNdH1K60ElYJYDvJSUxdGjpu13haE/pmHpDabA9cxZPionqSJGbJtrvl75MfTjawLvfV97vIcWaCvt/i2WDrhXGakztyH9ph7UwgYF6bjkrGxiQPVX5yXuiYLHTcT6LD1FVYEl+Vc3cf8L9tmZ4C+9F0KL9spyNfZ2wG4OT8WG9IS65puxQremV1YEulr0sQ/NgIBAaCH1rtumDxK/lfKf11cPs4iY5VnEYG0SYKjAzBjmMAxjHUCvL4z8VnHpO9VXneJ4VSmeQwA3uMzMnGrrqRgzSxP/u98lu63a3ObRE x-forefront-antispam-report: EFV:NLI; SFV:NSPM; SFS:(10019020)(98900003); DIR:OUT; SFP:1102; SCL:1; SRVR:SN1NAM02HT207; H:CY1PR14MB0520.namprd14.prod.outlook.com; FPR:; SPF:None; LANG:en; x-ms-office365-filtering-correlation-id: 5fa8a0b0-58cd-4516-9c86-08d3e87ed0c8 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(1601124038)(1603103081)(1601125047); SRVR:SN1NAM02HT207; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(432015012)(82015046); SRVR:SN1NAM02HT207; BCL:0; PCL:0; RULEID:; SRVR:SN1NAM02HT207; x-forefront-prvs: 00808B16F3 spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: hotmail.com X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Sep 2016 15:39:37.0525 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1NAM02HT207 X-OriginalArrivalTime: 29 Sep 2016 15:39:38.0792 (UTC) FILETIME=[B03D3280:01D21A67] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Sep 2016 15:40:44 -0000 Good morning, I followed the instructions here https://www.freebsd.org/doc/handbook/usb-d= isks.html to mount a Western Digital 8TB USB drive, which (using auto mount= ) created the mount point /media/da1p1 but when cd into the directory and d= o an 'ls' it takes a long time returning total 0, and when I try something = like 'touch t' I get and an 'Input/output error'. dmesg output below. I act= ually don't need to auto mount I need to mount the drive at boot so I simpl= y want to add it to my fstab, I was just testing the auto mount features fo= r fun. When I try to do it manually i get: root@correo:/ # mount -t msdosfs /dev/da1p1 /media/USBBackupDrive mount_msdosfs: /dev/da1p1: Invalid argument same with: root@correo:/ # mount /dev/da1p1 /media/USBBackupDrive mount: /dev/da1p1: Invalid argument Any help would be greatly appreciated, Shawn dmesg output: ciss0: *** Surface Analysis Pass Information, LDrv=3D0 Number of passes=3D3= 139 ciss0: *** Surface Analysis Pass Information, LDrv=3D0 Number of passes=3D3= 140 ciss0: *** Surface Analysis Pass Information, LDrv=3D0 Number of passes=3D3= 141 ugen0.6: at usbus0 umass0: o= n usbus0 umass0: SCSI over Bulk-Only; quirks =3D 0x8000 umass0:2:0:-1: Attached to scbus2 da1 at umass-sim0 bus 0 scbus2 target 0 lun 0 da1: Fixed Direct Access SPC-4 SCSI device da1: Serial Number 564B4A4D44554758 da1: 400.000MB/s transfers da1: 7630885MB (15628052480 512 byte sectors: 255H 63S/T 972801C) da1: quirks=3D0x2 ses0 at umass-sim0 bus 0 scbus2 target 0 lun 1 ses0: Fixed Enclosure Services SPC-4 SCSI device ses0: Serial Number 564B4A4D44554758 ses0: 400.000MB/s transfers ses0: SCSI-3 ENC Device WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 From owner-freebsd-stable@freebsd.org Thu Sep 29 16:07:28 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D1BFCC011CB for ; Thu, 29 Sep 2016 16:07:28 +0000 (UTC) (envelope-from shashaness@hotmail.com) Received: from COL004-OMC1S5.hotmail.com (col004-omc1s5.hotmail.com [65.55.34.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "*.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 89AA7C21 for ; Thu, 29 Sep 2016 16:07:28 +0000 (UTC) (envelope-from shashaness@hotmail.com) Received: from NAM02-CY1-obe.outbound.protection.outlook.com ([65.55.34.8]) by COL004-OMC1S5.hotmail.com over TLS secured channel with Microsoft SMTPSVC(7.5.7601.23008); Thu, 29 Sep 2016 09:07:21 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=W6CluQs5Et6EIpU9bFJBP47l7JalMKrGd2ht15FaKBA=; b=pR5K4HRNFoMNM8olqimaqGBMQP3U1o+ka/jsgm/Bjq2PfhfW+DcSfbw1IO8s1ZMJ7bnRuBA/mPhM1BmeFTzpwxTeecXMxPjY4JHAH50Fy6eCwCnAm1lIZxegxxMQSIS5nDcq875xdnJg3IVRQKiWYnLo7LDciuL5KFmgowZRDs/uERT0PAqPX5lT+YqAURUx2fJppssXh3pplWfx+9r6/0r8QW71UvSUzrQpZOFg2i6oBKSpfbOVUtGXLsDyUIto3rYGEkmgDXrCmcQh9sZXGD4kotpHgET2bMcHNenNAqdLtjjDAFQZjAVnTpI369ccI+lqBMJ6h05ha7zoJ7QVAw== Received: from CY1NAM02FT034.eop-nam02.prod.protection.outlook.com (10.152.74.60) by CY1NAM02HT058.eop-nam02.prod.protection.outlook.com (10.152.75.1) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.629.5; Thu, 29 Sep 2016 16:07:20 +0000 Received: from CY1PR14MB0520.namprd14.prod.outlook.com (10.152.74.53) by CY1NAM02FT034.mail.protection.outlook.com (10.152.75.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.629.5 via Frontend Transport; Thu, 29 Sep 2016 16:07:20 +0000 Received: from CY1PR14MB0520.namprd14.prod.outlook.com ([10.164.71.150]) by CY1PR14MB0520.namprd14.prod.outlook.com ([10.164.71.150]) with mapi id 15.01.0639.013; Thu, 29 Sep 2016 16:07:20 +0000 From: Shawn Bakhtiar To: "freebsd-stable@freebsd.org" Subject: Re: Error with auto mount Thread-Topic: Error with auto mount Thread-Index: AQHSGmevOcBzgb4JbUGJe0DkdJdGIKCQokgA Date: Thu, 29 Sep 2016 16:07:20 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=softfail (sender IP is 10.152.74.53) smtp.mailfrom=hotmail.com; freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=fail action=none header.from=hotmail.com; received-spf: SoftFail (protection.outlook.com: domain of transitioning hotmail.com discourages use of 10.152.74.53 as permitted sender) x-ms-exchange-messagesentrepresentingtype: 1 x-eopattributedmessage: 0 x-microsoft-exchange-diagnostics: 1; CY1NAM02HT058; 6:Pru1Vxmz+t8S/ubXqYqASg1/aoEgHWaBPZ3G0EmbbaTorRANBsyEuRzFGMWOJUkbnCYZe0rIBNU4bTqgLDhjuEaxQi6Bdzl88PhC383pvqYto5lRWh4m4Q6kmXqUUd2xKQmZm+OJ0RvC88xG/ljI+xgQNsu9yUWK63eURuuyafzG9n4BPK4kAwbLHWlhM3dGuuF7ZIOI4OATcMCPlL0DSMz1/zNXE9qRwSmslP+YpnVogUctJKq/p7KHX6AgxmIBf1T1hM85yMc/4n7ro6KcvP1yY/tGFEnYu0rRrDGc3g0=; 5:dTmvyFWmDisfHFGSm+QhMuSY9tVPz1EiM6chas6lnn6IyZUE1LF1Ea9uNmi+sEGF/6c8+TB+yyy5CKN2sLuQw/7hW3FtvyTP864Ll8t7zs2lydMABOIh2PakO9Ne5lsN/kJ22rr2F9Xr+cRFshDpCA==; 24:m2kA7TOOoL+M4wqz/tF8j25x4j+qX+ke91dXIZZevXpeNMX7zd1xU/S5tDOtmO02nD+vnDGzzrAg4mFXNjN32xozp4DSi14yrsUOhPaB+kQ=; 7:snt8oWWrK4t49dtHDkl+pgcknH85FhzgaqiLbWciHRCHaUjtskGxyBIoT9wv5IBRBontrZOWvklQtlwQd4niaQe/wzRphC1tdI39i1E1qvcUu4W4fZj6G34f3QeKq0ylzsCsCCGhzKJT/Z7vxyAMc6HyNi2Up2ZCVg68FK9gJt8ykv4Pr4UvYiFEabuJEuhjgkRTuMZDfBGjtr2OryAnB9wFzK0unUotz313Fe+fp3TuYialoECKhgSAR6JmEc1Z5w1B0wBJSqnXAy0G0dl+6fTwAHx7x87KXWzjwD3vKU5H6Saj7cT7iyGhwZ0t9PFB x-forefront-antispam-report: EFV:NLI; SFV:NSPM; SFS:(10019020)(98900003); DIR:OUT; SFP:1102; SCL:1; SRVR:CY1NAM02HT058; H:CY1PR14MB0520.namprd14.prod.outlook.com; FPR:; SPF:None; LANG:en; x-ms-office365-filtering-correlation-id: da8e66d5-d85c-43de-9556-08d3e882b010 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(1601124038)(1603103081)(1601125047); SRVR:CY1NAM02HT058; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(432015012)(82015046); SRVR:CY1NAM02HT058; BCL:0; PCL:0; RULEID:; SRVR:CY1NAM02HT058; x-forefront-prvs: 00808B16F3 spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: hotmail.com X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Sep 2016 16:07:20.2153 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1NAM02HT058 X-OriginalArrivalTime: 29 Sep 2016 16:07:21.0947 (UTC) FILETIME=[8F8E82B0:01D21A6B] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Sep 2016 16:07:28 -0000 Never mind... Answered my own question Needed to follow the instructions here to delete existing partition off the= drive and re-creat a new partition https://www.freebsd.org/doc/en/books/handbook/disks-adding.html It looks like FreeBSD either does not support or does not understand the de= fault partition on the Western Digital It was: root@correo:/ # gpart show /dev/da1 =3D> 34 15628052413 da1 GPT (7.3T) 34 2014 - free - (1.0M) 2048 15628048384 1 ms-basic-data (7.3T) 15628050432 2015 - free - (1.0M) Changed it to: root@correo:/mnt/USBBD # gpart show /dev/da1 =3D> 34 15628052413 da1 GPT (7.3T) 34 2014 - free - (1.0M) 2048 15628048384 1 freebsd-ufs (7.3T) 15628050432 2015 - free - (1.0M) added it to my fstab: # Device Mountpoint FStype Options Dump Pass# /dev/da0p2 / ufs rw 1 1 /dev/da0p3 none swap sw 0 0 /dev/da1p1 /mnt/USBBD ufs rw 2 2 And was able to mount by issuing command: mount /mnt/USBBD On Sep 29, 2016, at 8:39 AM, Shawn Bakhtiar > wrote: Good morning, I followed the instructions here https://www.freebsd.org/doc/handbook/usb-d= isks.html to mount a Western Digital 8TB USB drive, which (using auto mount= ) created the mount point /media/da1p1 but when cd into the directory and d= o an 'ls' it takes a long time returning total 0, and when I try something = like 'touch t' I get and an 'Input/output error'. dmesg output below. I act= ually don't need to auto mount I need to mount the drive at boot so I simpl= y want to add it to my fstab, I was just testing the auto mount features fo= r fun. When I try to do it manually i get: root@correo:/ # mount -t msdosfs /dev/da1p1 /media/USBBackupDrive mount_msdosfs: /dev/da1p1: Invalid argument same with: root@correo:/ # mount /dev/da1p1 /media/USBBackupDrive mount: /dev/da1p1: Invalid argument Any help would be greatly appreciated, Shawn dmesg output: ciss0: *** Surface Analysis Pass Information, LDrv=3D0 Number of passes=3D3= 139 ciss0: *** Surface Analysis Pass Information, LDrv=3D0 Number of passes=3D3= 140 ciss0: *** Surface Analysis Pass Information, LDrv=3D0 Number of passes=3D3= 141 ugen0.6: at usbus0 umass0: o= n usbus0 umass0: SCSI over Bulk-Only; quirks =3D 0x8000 umass0:2:0:-1: Attached to scbus2 da1 at umass-sim0 bus 0 scbus2 target 0 lun 0 da1: Fixed Direct Access SPC-4 SCSI device da1: Serial Number 564B4A4D44554758 da1: 400.000MB/s transfers da1: 7630885MB (15628052480 512 byte sectors: 255H 63S/T 972801C) da1: quirks=3D0x2 ses0 at umass-sim0 bus 0 scbus2 target 0 lun 1 ses0: Fixed Enclosure Services SPC-4 SCSI device ses0: Serial Number 564B4A4D44554758 ses0: 400.000MB/s transfers ses0: SCSI-3 ENC Device WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 WARNING: autofs_trigger_one: request for /media/da1p1/ completed with error= 5 _______________________________________________ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Fri Sep 30 00:40:04 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3CF2CC0061E for ; Fri, 30 Sep 2016 00:40:04 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id 2F73A3B7; Fri, 30 Sep 2016 00:40:04 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id 11A8B5A; Fri, 30 Sep 2016 00:40:04 +0000 (UTC) Date: Fri, 30 Sep 2016 00:39:54 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: jenkins-admin@FreeBSD.org, freebsd-stable@FreeBSD.org Message-ID: <851098527.1.1475196004016.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Build failed in Jenkins: FreeBSD_stable_10 #415 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkKKb2VAfYQKfu1t7qk4nR5qzUBEI+UqT4BPec4qHVhqUy0FFdq50sMH+3y9bCDNOufctov6VqTNffZ3YXArnZK95YF0OX97fh+E9txYOUX1adc+TikcKjuYpHmL5dE62eaZTI+4A5jnRonskQ1PaoIFz0Kbu4mWzkFsmdiXTraGzomXq4cHUCATA2+K4eDYgjXEQI30z3GOMmmZ4t/+6QGk1cMb/BqMWHbn80AsRCb4tU7Hpd72XLDpsuO7YRP1Q0CjmNAuBOTj+sFiiOe6U9HpqOlQN+iFUvBdZo/ybuy5Kh71cAaYQNL68cYdZJ6binH/DkG3KY/fS7DFYAeuwjwIDAQAB X-Jenkins-Job: FreeBSD_stable_10 X-Jenkins-Result: FAILURE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Sep 2016 00:40:04 -0000 See ------------------------------------------ [...truncated 324902 lines...] [192.168.10.2] out: lib/libc/gen/basedirname_test:basename_posix -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/basedirname_test:dirname_posix -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/dir_test:seekdir_basic -> passed [0.008s] [192.168.10.2] out: lib/libc/gen/dir_test:telldir_leak -> passed [0.009s] [192.168.10.2] out: lib/libc/gen/floatunditf_test:floatunditf -> skipped: Requires long double support [0.004s] [192.168.10.2] out: lib/libc/gen/fnmatch_test:fnmatch_backslashes -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/fnmatch_test:fnmatch_casefold -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/fnmatch_test:fnmatch_leadingdir -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/fnmatch_test:fnmatch_noescape -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/fnmatch_test:fnmatch_pathname -> passed [0.005s] [192.168.10.2] out: lib/libc/gen/fnmatch_test:fnmatch_period -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/fpclassify_test:no_test -> skipped: Test not available on this architecture [0.004s] [192.168.10.2] out: lib/libc/gen/fpsetmask_test:no_test -> skipped: Test not available on this architecture. [0.004s] [192.168.10.2] out: lib/libc/gen/fpsetround_test:fpsetround_basic -> skipped: Test not applicable on this architecture. [0.004s] [192.168.10.2] out: lib/libc/gen/ftok_test:ftok_err -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/ftok_test:ftok_link -> passed [0.008s] [192.168.10.2] out: lib/libc/gen/getcwd_test:getcwd_err -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/getcwd_test:getcwd_fts -> passed [0.106s] [192.168.10.2] out: lib/libc/gen/getgrent_test:getgrent_loop -> passed [0.005s] [192.168.10.2] out: lib/libc/gen/getgrent_test:getgrent_setgid -> passed [0.024s] [192.168.10.2] out: lib/libc/gen/glob_test:glob_star_not -> passed [0.005s] [192.168.10.2] out: lib/libc/gen/humanize_number_test:humanize_number_basic -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/humanize_number_test:humanize_number_big -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/isnan_test:isinf_basic -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/isnan_test:isnan_basic -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/nice_test:nice_err -> expected_failure: nice(incr) with incr < 0 fails with unprivileged users and sets errno == EPERM; see PR # 189821 for more details: /builds/workspace/FreeBSD_stable_10/src/contrib/netbsd-tests/lib/libc/gen/t_nice.c:89: Expected errno 1, got 13, in nice(i) == -1 [0.005s] [192.168.10.2] out: lib/libc/gen/nice_test:nice_priority -> passed [0.011s] [192.168.10.2] out: lib/libc/gen/nice_test:nice_root -> passed [0.005s] [192.168.10.2] out: lib/libc/gen/nice_test:nice_thread -> passed [0.006s] [192.168.10.2] out: lib/libc/gen/pause_test:pause_basic -> passed [1.006s] [192.168.10.2] out: lib/libc/gen/pause_test:pause_kill -> passed [2.082s] [192.168.10.2] out: lib/libc/gen/raise_test:raise_err -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/raise_test:raise_ret -> passed [1.032s] [192.168.10.2] out: lib/libc/gen/raise_test:raise_sig -> passed [0.005s] [192.168.10.2] out: lib/libc/gen/raise_test:raise_stress -> passed [0.025s] [192.168.10.2] out: lib/libc/gen/realpath_test:realpath_basic -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/realpath_test:realpath_huge -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/realpath_test:realpath_symlink -> passed [0.007s] [192.168.10.2] out: lib/libc/gen/setdomainname_test:setdomainname_basic -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/setdomainname_test:setdomainname_limit -> passed [0.005s] [192.168.10.2] out: lib/libc/gen/setdomainname_test:setdomainname_perm -> passed [0.005s] [192.168.10.2] out: lib/libc/gen/sethostname_test:sethostname_basic -> skipped: screws up the test host's hostname on FreeBSD [0.004s] [192.168.10.2] out: lib/libc/gen/sethostname_test:sethostname_limit -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/sethostname_test:sethostname_perm -> passed [0.005s] [192.168.10.2] out: lib/libc/gen/sleep_test:kevent -> passed [26.691s] [192.168.10.2] out: lib/libc/gen/sleep_test:nanosleep -> expected_failure: Long reschedule latency due to PR kern/43997: Reschedule latency -7277947111325552072 exceeds allowable fuzz 40000000 [0.052s] [192.168.10.2] out: lib/libc/gen/sleep_test:select -> passed [36.902s] [192.168.10.2] out: lib/libc/gen/sleep_test:sleep -> passed [29.275s] [192.168.10.2] out: lib/libc/gen/syslog_test:syslog_pthread -> passed [0.005s] [192.168.10.2] out: lib/libc/gen/time_test:time_copy -> passed [0.005s] [192.168.10.2] out: lib/libc/gen/time_test:time_mono -> passed [10.452s] [192.168.10.2] out: lib/libc/gen/time_test:time_timeofday -> passed [0.005s] [192.168.10.2] out: lib/libc/gen/ttyname_test:ttyname_err -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/ttyname_test:ttyname_r_err -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/ttyname_test:ttyname_r_stdin -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/ttyname_test:ttyname_stdin -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/vis_test:strunvis_hex -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/vis_test:strvis_basic -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/vis_test:strvis_empty -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/vis_test:strvis_null -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/execve/execve_test:t_execve_null -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/posix_spawn/fileactions_test:t_spawn_empty_fileactions -> passed [0.006s] [192.168.10.2] out: lib/libc/gen/posix_spawn/fileactions_test:t_spawn_fileactions -> passed [0.005s] [192.168.10.2] out: lib/libc/gen/posix_spawn/fileactions_test:t_spawn_open_nonexistent -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/posix_spawn/fileactions_test:t_spawn_openmode -> passed [0.011s] [192.168.10.2] out: lib/libc/gen/posix_spawn/fileactions_test:t_spawn_reopen -> passed [0.005s] [192.168.10.2] out: lib/libc/gen/posix_spawn/spawn_test:t_spawn_child -> passed [0.007s] [192.168.10.2] out: lib/libc/gen/posix_spawn/spawn_test:t_spawn_ls -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/posix_spawn/spawn_test:t_spawn_missing -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/posix_spawn/spawn_test:t_spawn_nonexec -> passed [0.005s] [192.168.10.2] out: lib/libc/gen/posix_spawn/spawn_test:t_spawn_zero -> passed [0.005s] [192.168.10.2] out: lib/libc/gen/posix_spawn/spawn_test:t_spawnp_ls -> passed [0.004s] [192.168.10.2] out: lib/libc/gen/posix_spawn/spawnattr_test:t_spawnattr -> passed [0.006s] [192.168.10.2] out: lib/libc/hash/sha2_test:t_sha256 -> passed [0.005s] [192.168.10.2] out: lib/libc/hash/sha2_test:t_sha384 -> passed [0.005s] [192.168.10.2] out: lib/libc/hash/sha2_test:t_sha512 -> passed [0.005s] [192.168.10.2] out: lib/libc/hash/hash_test:md5 -> passed [0.065s] [192.168.10.2] out: lib/libc/hash/hash_test:sha1 -> passed [0.187s] [192.168.10.2] out: lib/libc/inet/inet_network_test:inet_addr_basic -> passed [0.004s] [192.168.10.2] out: lib/libc/inet/inet_network_test:inet_addr_err -> passed [0.004s] [192.168.10.2] out: lib/libc/inet/inet_network_test:inet_network_basic -> passed [0.004s] [192.168.10.2] out: lib/libc/inet/inet_network_test:inet_network_err -> passed [0.004s] [192.168.10.2] out: lib/libc/net/ether_test:ether_aton -> passed [0.004s] [192.168.10.2] out: lib/libc/net/ether_test:ether_aton_bad -> passed [0.005s] [192.168.10.2] out: lib/libc/net/ether_test:ether_aton_r -> passed [0.004s] [192.168.10.2] out: lib/libc/net/ether_test:ether_aton_r_bad -> passed [0.004s] [192.168.10.2] out: lib/libc/net/ether_test:ether_line -> passed [0.004s] [192.168.10.2] out: lib/libc/net/ether_test:ether_line_bad_1 -> passed [0.004s] [192.168.10.2] out: lib/libc/net/ether_test:ether_line_bad_2 -> passed [0.004s] [192.168.10.2] out: lib/libc/net/ether_test:ether_ntoa -> passed [0.004s] [192.168.10.2] out: lib/libc/net/ether_test:ether_ntoa_r -> passed [0.004s] [192.168.10.2] out: lib/libc/net/eui64_aton_test:hex_ascii -> passed [0.004s] [192.168.10.2] out: lib/libc/net/eui64_aton_test:id_ascii -> passed [0.004s] [192.168.10.2] out: lib/libc/net/eui64_aton_test:id_colon_ascii -> passed [0.005s] [192.168.10.2] out: lib/libc/net/eui64_aton_test:mac_ascii -> passed [0.004s] [192.168.10.2] out: lib/libc/net/eui64_aton_test:mac_colon_ascii -> passed [0.004s] [192.168.10.2] out: lib/libc/net/eui64_ntoa_test:id_ascii -> passed [0.004s] [192.168.10.2] out: lib/libc/net/getprotoent_test:endprotoent_rewind -> passed [0.005s] [192.168.10.2] out: lib/libc/net/getprotoent_test:getprotobyname_basic -> passed [0.005s] [192.168.10.2] out: lib/libc/net/getprotoent_test:getprotobyname_err -> passed [0.005s] [192.168.10.2] out: lib/libc/net/getprotoent_test:getprotobynumber_basic -> passed [0.004s] [192.168.10.2] out: lib/libc/net/getprotoent_test:getprotobynumber_err -> passed [0.005s] [192.168.10.2] out: lib/libc/net/getprotoent_test:getprotoent_next -> passed [0.005s] [192.168.10.2] out: lib/libc/net/getprotoent_test:setprotoent_rewind -> passed [0.004s] [192.168.10.2] out: lib/libc/net/ether_aton_test:tc_ether_aton -> passed [0.004s] [192.168.10.2] out: lib/libc/net/nsdispatch_test:recurse -> passed [0.069s] [192.168.10.2] out: lib/libc/net/protoent_test:protoent -> passed [0.028s] [192.168.10.2] out: lib/libc/net/servent_test:servent -> passed [0.335s] [192.168.10.2] out: lib/libc/nss/getaddrinfo_test:pf_inet -> passed [0.011s] [192.168.10.2] out: lib/libc/nss/getaddrinfo_test:pf_inet6 -> passed [0.008s] [192.168.10.2] out: lib/libc/nss/getaddrinfo_test:pf_inet6_with_snapshot -> passed [0.010s] [192.168.10.2] out: lib/libc/nss/getaddrinfo_test:pf_inet_with_snapshot -> passed [0.010s] [192.168.10.2] out: lib/libc/nss/getaddrinfo_test:pf_unspec -> passed [0.010s] [192.168.10.2] out: lib/libc/nss/getaddrinfo_test:pf_unspec_with_snapshot -> passed [0.010s] [192.168.10.2] out: lib/libc/nss/getgr_test:getgrent -> passed [0.006s] [192.168.10.2] out: lib/libc/nss/getgr_test:getgrent_with_snapshot -> passed [0.006s] [192.168.10.2] out: lib/libc/nss/getgr_test:getgrent_with_two_pass -> passed [0.005s] [192.168.10.2] out: lib/libc/nss/getgr_test:getgrgid -> passed [0.005s] [192.168.10.2] out: lib/libc/nss/getgr_test:getgrgid_with_snapshot -> passed [0.006s] [192.168.10.2] out: lib/libc/nss/getgr_test:getgrnam -> passed [0.005s] [192.168.10.2] out: lib/libc/nss/getgr_test:getgrnam_with_snapshot -> passed [0.007s] [192.168.10.2] out: lib/libc/nss/gethostby_test:gethostbyaddr_getnameinfo_ipv4 -> passed [0.008s] [192.168.10.2] out: lib/libc/nss/gethostby_test:gethostbyaddr_getnameinfo_ipv6 -> passed [0.008s] [192.168.10.2] out: lib/libc/nss/gethostby_test:gethostbyaddr_ipv4 -> passed [0.008s] [192.168.10.2] out: lib/libc/nss/gethostby_test:gethostbyaddr_ipv4_with_snapshot -> passed [0.010s] [192.168.10.2] out: lib/libc/nss/gethostby_test:gethostbyaddr_ipv6 -> passed [0.009s] [192.168.10.2] out: lib/libc/nss/gethostby_test:gethostbyaddr_ipv6_AI_V4MAPPED -> passed [0.009s] [192.168.10.2] out: lib/libc/nss/gethostby_test:gethostbyaddr_ipv6_with_snapshot -> passed [0.010s] [192.168.10.2] out: lib/libc/nss/gethostby_test:gethostbyaddr_ipv6_with_snapshot_AI_V4MAPPED -> passed [0.009s] [192.168.10.2] out: lib/libc/nss/gethostby_test:gethostbyname2_getaddrinfo_ipv4 -> passed [0.012s] [192.168.10.2] out: lib/libc/nss/gethostby_test:gethostbyname2_getaddrinfo_ipv6 -> passed [0.012s] [192.168.10.2] out: lib/libc/nss/gethostby_test:gethostbyname2_ipv4 -> passed [0.008s] [192.168.10.2] out: lib/libc/nss/gethostby_test:gethostbyname2_ipv4_with_snapshot -> passed [0.009s] [192.168.10.2] out: lib/libc/nss/gethostby_test:gethostbyname2_ipv6 -> passed [0.008s] [192.168.10.2] out: lib/libc/nss/gethostby_test:gethostbyname2_ipv6_AI_V4MAPPED -> passed [0.008s] [192.168.10.2] out: lib/libc/nss/gethostby_test:gethostbyname2_ipv6_with_snapshot -> passed [0.010s] [192.168.10.2] out: lib/libc/nss/gethostby_test:gethostbyname2_ipv6_with_snapshot_AI_V4MAPPED -> passed [0.009s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyaddr_getnameinfo_ipv4 -> passed [0.009s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyaddr_getnameinfo_ipv6 -> passed [0.008s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyaddr_ipv4 -> passed [0.009s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyaddr_ipv4_with_snapshot -> passed [0.010s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyaddr_ipv6 -> passed [0.008s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyaddr_ipv6_AI_V4MAPPED -> passed [0.008s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyaddr_ipv6_AI_V4MAPPED_CFG -> passed [0.009s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyaddr_ipv6_AI_V4MAPPED_CFG_AI_ALL -> passed [0.009s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyaddr_ipv6_with_snapshot -> passed [0.009s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyaddr_ipv6_with_snapshot_AI_V4MAPPED -> passed [0.010s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyaddr_ipv6_with_snapshot_AI_V4MAPPED_CFG -> passed [0.010s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyaddr_ipv6_with_snapshot_AI_V4MAPPED_CFG_AI_ALL -> passed [0.009s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyname_getaddrinfo_ipv4 -> passed [0.012s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyname_getaddrinfo_ipv6 -> passed [0.012s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyname_ipv4 -> passed [0.008s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyname_ipv4_AI_ADDRCONFIG -> passed [0.009s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyname_ipv4_with_snapshot -> passed [0.009s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyname_ipv4_with_snapshot_AI_ADDRCONFIG -> passed [0.010s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyname_ipv6 -> passed [0.008s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyname_ipv6_AI_ADDRCONFIG -> passed [0.009s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyname_ipv6_AI_V4MAPPED -> passed [0.008s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyname_ipv6_AI_V4MAPPED_CFG -> passed [0.008s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyname_ipv6_AI_V4MAPPED_CFG_AI_ADDRCONFIG -> passed [0.008s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyname_ipv6_AI_V4MAPPED_CFG_AI_ALL -> passed [0.010s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyname_ipv6_with_snapshot -> passed [0.010s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyname_ipv6_with_snapshot_AI_ADDRCONFIG -> passed [0.009s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyname_ipv6_with_snapshot_AI_V4MAPPED -> passed [0.010s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyname_ipv6_with_snapshot_AI_V4MAPPED_CFG -> passed [0.009s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyname_ipv6_with_snapshot_AI_V4MAPPED_CFG_AI_ADDRCONFIG -> passed [0.009s] [192.168.10.2] out: lib/libc/nss/gethostby_test:getipnodebyname_ipv6_with_snapshot_AI_V4MAPPED_CFG_AI_ALL -> passed [0.009s] [192.168.10.2] out: lib/libc/nss/getpw_test:build_snapshot -> passed [0.006s] [192.168.10.2] out: lib/libc/nss/getpw_test:getpwent -> passed [0.006s] [192.168.10.2] out: lib/libc/nss/getpw_test:getpwent_with_snapshot -> passed [0.006s] [192.168.10.2] out: lib/libc/nss/getpw_test:getpwent_with_two_pass -> passed [0.006s] [192.168.10.2] out: lib/libc/nss/getpw_test:getpwnam -> passed [0.006s] [192.168.10.2] out: lib/libc/nss/getpw_test:getpwnam_with_snapshot -> passed [0.006s] [192.168.10.2] out: lib/libc/nss/getpw_test:getpwuid -> passed [0.006s] [192.168.10.2] out: lib/libc/nss/getpw_test:getpwuid_with_snapshot -> passed [0.006s] [192.168.10.2] out: lib/libc/nss/getproto_test:build_snapshot -> passed [0.006s] [192.168.10.2] out: lib/libc/nss/getproto_test:getprotobyname -> passed [0.019s] [192.168.10.2] out: lib/libc/nss/getproto_test:getprotobyname_with_snapshot -> passed [0.021s] [192.168.10.2] out: lib/libc/nss/getproto_test:getprotobynumber -> passed [0.012s] [192.168.10.2] out: lib/libc/nss/getproto_test:getprotobynumber_with_snapshot -> passed [0.014s] [192.168.10.2] out: lib/libc/nss/getproto_test:getprotoent -> passed [0.006s] [192.168.10.2] out: lib/libc/nss/getproto_test:getprotoent_with_snapshot -> passed [0.007s] [192.168.10.2] out: lib/libc/nss/getproto_test:getprotoent_with_two_pass -> passed [0.006s] [192.168.10.2] out: lib/libc/nss/getrpc_test:build_snapshot -> passed [0.010s] [192.168.10.2] out: lib/libc/nss/getrpc_test:getrpcbyent -> passed [0.005s] [192.168.10.2] out: lib/libc/nss/getrpc_test:getrpcbyent_with_snapshot -> passed [0.007s] [192.168.10.2] out: lib/libc/nss/getrpc_test:getrpcbyent_with_two_pass -> passed [0.006s] [192.168.10.2] out: lib/libc/nss/getrpc_test:getrpcbyname -> passed [0.010s] [192.168.10.2] out: lib/libc/nss/getrpc_test:getrpcbyname_with_snapshot -> passed [0.011s] [192.168.10.2] out: lib/libc/nss/getrpc_test:getrpcbynumber -> passed [0.008s] [192.168.10.2] out: lib/libc/nss/getrpc_test:getrpcbynumber_with_snapshot -> passed [0.009s] [192.168.10.2] out: lib/libc/nss/getserv_test:build_snapshot -> passed [0.023s] [192.168.10.2] out: lib/libc/nss/getserv_test:getservbyent -> passed [0.021s] [192.168.10.2] out: lib/libc/nss/getserv_test:getservbyent_with_snapshot -> passed [0.039s] [192.168.10.2] out: lib/libc/nss/getserv_test:getservbyent_with_two_pass -> passed [0.031s] [192.168.10.2] out: lib/libc/nss/getserv_test:getservbyname -> passed [1.034s] [192.168.10.2] out: lib/libc/nss/getserv_test:getservbyname_with_snapshot -> passed [1.053s] [192.168.10.2] out: lib/libc/nss/getserv_test:getservbyport -> passed [1.028s] [192.168.10.2] out: lib/libc/nss/getserv_test:getservbyport_with_snapshot -> passed [1.052s] [192.168.10.2] out: lib/libc/nss/getusershell_test:getusershell_with_snapshot -> passed [0.008s] [192.168.10.2] out: lib/libc/nss/getusershell_test:getusershell_with_two_pass -> passed [0.005s] [192.168.10.2] out: lib/libc/regex/exhaust_test:regcomp_too_big -> passed [0.389s] [192.168.10.2] out: lib/libc/regex/regex_att_test:basic -> passed [0.012s] [192.168.10.2] out: lib/libc/regex/regex_att_test:categorization -> passed [0.007s] [192.168.10.2] out: lib/libc/regex/regex_att_test:forcedassoc -> passed [0.006s] [192.168.10.2] out: lib/libc/regex/regex_att_test:leftassoc -> expected_failure: The expected and matched groups are mismatched on FreeBSD: 12 checks failed as expected; see output for more details [0.007s] [192.168.10.2] out: lib/libc/regex/regex_att_test:nullsubexpr -> passed [0.006s] [192.168.10.2] out: lib/libc/regex/regex_att_test:repetition -> passed [0.007s] [192.168.10.2] out: lib/libc/regex/regex_att_test:rightassoc -> passed [0.006s] [192.168.10.2] out: lib/libc/regex/regex_test:anchor -> passed [0.106s] [192.168.10.2] out: lib/libc/regex/regex_test:backref -> passed [0.058s] [192.168.10.2] out: lib/libc/regex/regex_test:basic -> passed [0.059s] [192.168.10.2] out: lib/libc/regex/regex_test:bracket -> passed [0.063s] [192.168.10.2] out: lib/libc/regex/regex_test:c_comments -> passed [0.121s] [192.168.10.2] out: lib/libc/regex/regex_test:complex -> passed [0.071s] [192.168.10.2] out: lib/libc/regex/regex_test:error -> passed [0.069s] [192.168.10.2] out: lib/libc/regex/regex_test:meta -> passed [0.060s] [192.168.10.2] out: lib/libc/regex/regex_test:nospec -> passed [0.056s] [192.168.10.2] out: lib/libc/regex/regex_test:paren -> passed [0.060s] [192.168.10.2] out: lib/libc/regex/regex_test:regress -> passed [0.061s] [192.168.10.2] out: lib/libc/regex/regex_test:repet_bounded -> passed [0.076s] [192.168.10.2] out: lib/libc/regex/regex_test:repet_multi -> passed [0.059s] [192.168.10.2] out: lib/libc/regex/regex_test:repet_ordinary -> passed [0.059s] [192.168.10.2] out: lib/libc/regex/regex_test:startend -> passed [0.060s] [192.168.10.2] out: lib/libc/regex/regex_test:subexp -> passed [0.059s] [192.168.10.2] out: lib/libc/regex/regex_test:subtle -> passed [0.055s] [192.168.10.2] out: lib/libc/regex/regex_test:word_bound -> passed [0.056s] [192.168.10.2] out: lib/libc/regex/regex_test:zero -> passed [0.061s] [192.168.10.2] out: lib/libc/resolv/resolv_test:getaddrinfo_test -> passed [2.133s] [192.168.10.2] out: lib/libc/resolv/resolv_test:gethostby_test -> passed [2.058s] [192.168.10.2] out: lib/libc/resolv/resolv_test:getipnodeby_test -> passed [2.046s] Resuming build at Fri Sep 30 00:38:07 GMT 2016 after Jenkins restart Waiting to resume part of FreeBSD_stable_10 #415: chaos.ysv.freebsd.org is offline Waiting to resume part of FreeBSD_stable_10 #415: chaos.ysv.freebsd.org is offline Waiting to resume part of FreeBSD_stable_10 #415: chaos.ysv.freebsd.org is offline Waiting to resume part of FreeBSD_stable_10 #415: chaos.ysv.freebsd.org is offline Waiting to resume part of FreeBSD_stable_10 #415: chaos.ysv.freebsd.org is offline Waiting to resume part of FreeBSD_stable_10 #415: chaos.ysv.freebsd.org is offline Waiting to resume part of FreeBSD_stable_10 #415: chaos.ysv.freebsd.org is offline Waiting to resume part of FreeBSD_stable_10 #415: chaos.ysv.freebsd.org is offline Waiting to resume part of FreeBSD_stable_10 #415: chaos.ysv.freebsd.org is offline Ready to run at Fri Sep 30 00:39:53 GMT 2016 [192.168.10.2] out: lib/libc/rpc/rpc_test:get_svc_addr_tcp -> Killed [Pipeline] } [Pipeline] // node [Pipeline] node Running on master in /usr/local/jenkins/workspace/FreeBSD_stable_10 [Pipeline] { [Pipeline] step From owner-freebsd-stable@freebsd.org Fri Sep 30 04:53:47 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 140ABC01A53 for ; Fri, 30 Sep 2016 04:53:47 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id 063F5CC5; Fri, 30 Sep 2016 04:53:47 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id 51D0168; Fri, 30 Sep 2016 04:53:47 +0000 (UTC) Date: Fri, 30 Sep 2016 04:53:47 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: jenkins-admin@FreeBSD.org, freebsd-stable@FreeBSD.org Message-ID: <1736192730.2.1475211227217.JavaMail.jenkins@jenkins-9.freebsd.org> In-Reply-To: <851098527.1.1475196004016.JavaMail.jenkins@jenkins-9.freebsd.org> References: <851098527.1.1475196004016.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Jenkins build is back to normal : FreeBSD_stable_10 #416 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkKKb2VAfYQKfu1t7qk4nR5qzUBEI+UqT4BPec4qHVhqUy0FFdq50sMH+3y9bCDNOufctov6VqTNffZ3YXArnZK95YF0OX97fh+E9txYOUX1adc+TikcKjuYpHmL5dE62eaZTI+4A5jnRonskQ1PaoIFz0Kbu4mWzkFsmdiXTraGzomXq4cHUCATA2+K4eDYgjXEQI30z3GOMmmZ4t/+6QGk1cMb/BqMWHbn80AsRCb4tU7Hpd72XLDpsuO7YRP1Q0CjmNAuBOTj+sFiiOe6U9HpqOlQN+iFUvBdZo/ybuy5Kh71cAaYQNL68cYdZJ6binH/DkG3KY/fS7DFYAeuwjwIDAQAB X-Jenkins-Job: FreeBSD_stable_10 X-Jenkins-Result: SUCCESS X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Sep 2016 04:53:47 -0000 https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/416/ From owner-freebsd-stable@freebsd.org Fri Sep 30 11:31:01 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CD603C030FD; Fri, 30 Sep 2016 11:31:01 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id C249CF6F; Fri, 30 Sep 2016 11:31:00 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA19772; Fri, 30 Sep 2016 14:30:58 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1bpw1q-0001Ej-BV; Fri, 30 Sep 2016 14:30:58 +0300 To: freebsd-current@FreeBSD.org, freebsd-stable@FreeBSD.org, freebsd-acpi@FreeBSD.org From: Andriy Gapon Subject: aibs(4) / atk0110 support for newer systems Message-ID: <86cf8380-ac6f-55f0-f0f8-16000d7f04b2@FreeBSD.org> Date: Fri, 30 Sep 2016 14:30:02 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Sep 2016 11:31:01 -0000 I've written a patch for aibs(4) (also known as ASUS AI Booster and ATK0110) that adds support for discovering and querying sensors using newer GGRP ("get group"?) and GITM ("get item"?) methods: https://people.freebsd.org/~avg/aibs-ggrp-gitm.diff If you are using the driver could you please test that the patch does not break it for you? If you have an ASUS motherboard where aibs does not find sensors could you please check if there is any improvement with the patch? Testing the patch should be as easy as building, installing and loading aibs module (found in sys/modules/acpi/aibs). Well, also don't forget to apply the patch with patch -p1 :-) -- Andriy Gapon From owner-freebsd-stable@freebsd.org Fri Sep 30 12:42:00 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B3B78C00E14 for ; Fri, 30 Sep 2016 12:42:00 +0000 (UTC) (envelope-from torfinn.ingolfsen@getmail.no) Received: from bouvier.getmail.no (bouvier.getmail.no [84.210.184.8]) by mx1.freebsd.org (Postfix) with ESMTP id 6E2C0AFA for ; Fri, 30 Sep 2016 12:41:59 +0000 (UTC) (envelope-from torfinn.ingolfsen@getmail.no) Received: from localhost (localhost [127.0.0.1]) by bouvier.getmail.no (Postfix) with ESMTP id 46AC2531BE for ; Fri, 30 Sep 2016 14:41:51 +0200 (CEST) Received: from bouvier.getmail.no ([127.0.0.1]) by localhost (bouvier.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 6mFf2qDz7KQY for ; Fri, 30 Sep 2016 14:41:51 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by bouvier.getmail.no (Postfix) with ESMTP id D9E36531C1 for ; Fri, 30 Sep 2016 14:41:50 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.8.4 bouvier.getmail.no D9E36531C1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=getmail.no; s=8A9C8B4C-D727-11E2-8095-B6466E6B3FA2; t=1475239310; bh=yELMfuWq46TU5OH0XxgS75XBerN1hDNHummhMfL/HjM=; h=Date:From:To:Subject:Message-Id:Mime-Version:Content-Type: Content-Transfer-Encoding; b=47dIWRugVqXCSaDN6cIAUFfkT9dY0AZdzeYJtDWXPiW1Xw1tCoelihNJURxRGgPC2 L9bQcNtyXHpMDPqJjhS8EZiWIXMBdMILOVL25AhdgI67hXN3XhJIfm0PU9v+DHJ69H hKXRIZgY9TU4+Y2YvSQjnUW7y9LNRGhiglZ+nkwQ= X-Virus-Scanned: amavisd-new at bouvier.get.c.bitbit.net Received: from bouvier.getmail.no ([127.0.0.1]) by localhost (bouvier.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 37U_pvGRmmNe for ; Fri, 30 Sep 2016 14:41:50 +0200 (CEST) Received: from kg-core1.kg4.no (cm-84.209.39.108.getinternet.no [84.209.39.108]) by bouvier.getmail.no (Postfix) with ESMTPSA id A441D531BE for ; Fri, 30 Sep 2016 14:41:50 +0200 (CEST) Date: Fri, 30 Sep 2016 14:41:50 +0200 From: Torfinn Ingolfsen To: freebsd-stable@freebsd.org Subject: Re: 11.0 stuck on high network load Message-Id: <20160930144150.9822e7a27f5b51cf9c9db427@getmail.no> In-Reply-To: <62453d9c-b1e4-1129-70ff-654dacea37f9@gmail.com> References: <78cbcdc9-f565-1046-c157-2ddd8fcccc62@freebsd.org> <20160919204328.GN2840@zxy.spb.ru> <8ba75d6e-4f01-895e-0aed-53c6c6692cb9@freebsd.org> <20160920202633.GQ2840@zxy.spb.ru> <20160921195155.GW2840@zxy.spb.ru> <20160923200143.GG2840@zxy.spb.ru> <20160925124626.GI2840@zxy.spb.ru> <20160926172159.GA54003@zxy.spb.ru> <62453d9c-b1e4-1129-70ff-654dacea37f9@gmail.com> X-Mailer: Sylpheed 3.5.1 (GTK+ 2.24.29; amd64-portbld-freebsd9.3) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Sep 2016 12:42:00 -0000 On Wed, 28 Sep 2016 12:06:47 +0200 Julien Charbon wrote: > > I am still trying to reproduce your issue, without success so far. All: please remember to trim your quotes. Thank you. Carry on. -- Regards, Torfinn Ingolfsen From owner-freebsd-stable@freebsd.org Fri Sep 30 12:57:08 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 627D8C02401 for ; Fri, 30 Sep 2016 12:57:08 +0000 (UTC) (envelope-from torfinn.ingolfsen@getmail.no) Received: from bouvier.getmail.no (bouvier.getmail.no [84.210.184.8]) by mx1.freebsd.org (Postfix) with ESMTP id 20286135E for ; Fri, 30 Sep 2016 12:57:07 +0000 (UTC) (envelope-from torfinn.ingolfsen@getmail.no) Received: from localhost (localhost [127.0.0.1]) by bouvier.getmail.no (Postfix) with ESMTP id 4370553258 for ; Fri, 30 Sep 2016 14:57:05 +0200 (CEST) Received: from bouvier.getmail.no ([127.0.0.1]) by localhost (bouvier.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id ab41w6O9Slq8 for ; Fri, 30 Sep 2016 14:57:04 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by bouvier.getmail.no (Postfix) with ESMTP id C958B5325F for ; Fri, 30 Sep 2016 14:57:04 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.8.4 bouvier.getmail.no C958B5325F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=getmail.no; s=8A9C8B4C-D727-11E2-8095-B6466E6B3FA2; t=1475240224; bh=yrAaaKJYZTZb8nbDRhyIEulj5XqOThig+HqbWU2VVMU=; h=Date:From:To:Subject:Message-Id:Mime-Version:Content-Type: Content-Transfer-Encoding; b=VQavPbIVQxBIPq0D9MwkZoAr+IhPH/H5dbOsW0cbOY1g0+Jk29VxMiCheKGrkdyeh zCROCNjuNkYEamiKtpSAbqiz38Q7Jd7xypLdvKV8GGV5ti9blrKkKLFjN9WOoMkNbW cg8R8uakeFHouWeF3ioFa2ODEcsduYN7u1vMXN3c= X-Virus-Scanned: amavisd-new at bouvier.get.c.bitbit.net Received: from bouvier.getmail.no ([127.0.0.1]) by localhost (bouvier.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 5Rj-EJNr3tFz for ; Fri, 30 Sep 2016 14:57:04 +0200 (CEST) Received: from kg-core1.kg4.no (cm-84.209.39.108.getinternet.no [84.209.39.108]) by bouvier.getmail.no (Postfix) with ESMTPSA id A3E1953258 for ; Fri, 30 Sep 2016 14:57:04 +0200 (CEST) Date: Fri, 30 Sep 2016 14:57:04 +0200 From: Torfinn Ingolfsen To: freebsd-stable@freebsd.org Subject: Re: aibs(4) / atk0110 support for newer systems Message-Id: <20160930145704.4dbc9d90011154b38493964e@getmail.no> In-Reply-To: <86cf8380-ac6f-55f0-f0f8-16000d7f04b2@FreeBSD.org> References: <86cf8380-ac6f-55f0-f0f8-16000d7f04b2@FreeBSD.org> X-Mailer: Sylpheed 3.5.1 (GTK+ 2.24.29; amd64-portbld-freebsd9.3) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Sep 2016 12:57:08 -0000 On Fri, 30 Sep 2016 14:30:02 +0300 Andriy Gapon wrote: > Testing the patch should be as easy as building, installing and loading aibs > module (found in sys/modules/acpi/aibs). Well, also don't forget to apply the > patch with patch -p1 :-) The patch doesn't apply cleanly on FreeBSD 9.3-stable: root@kg-core1# patch -C -p1 < /home/tingo/dl/aibs-ggrp-gitm.diff Hmm... Looks like a unified diff to me... The text leading up to this was: -------------------------- |diff --git a/sys/dev/acpi_support/atk0110.c b/sys/dev/acpi_support/atk0110.c |index e79a0e00e3577..743684100f737 100644 |--- a/sys/dev/acpi_support/atk0110.c |+++ b/sys/dev/acpi_support/atk0110.c -------------------------- Patching file sys/dev/acpi_support/atk0110.c using Plan A... Hunk #1 succeeded at 28. Hunk #2 succeeded at 52. Hunk #3 succeeded at 78. Hunk #4 succeeded at 124. Hunk #5 succeeded at 362. Hunk #6 succeeded at 370. Hunk #7 failed at 391. Hunk #8 succeeded at 435. Hunk #9 succeeded at 450. Hunk #10 failed at 488. 2 out of 10 hunks failed--saving rejects to sys/dev/acpi_support/atk0110.c.rej done I'm running root@kg-core1# uname -a FreeBSD kg-core1.kg4.no 9.3-STABLE FreeBSD 9.3-STABLE #3 r304838: Fri Aug 26 12:11:25 CEST 2016 root@kg-core1.kg4.no:/usr/obj/usr/src/sys/GENERIC amd64 -- Torfinn Ingolfsen From owner-freebsd-stable@freebsd.org Fri Sep 30 14:34:54 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D0D24C0374F; Fri, 30 Sep 2016 14:34:54 +0000 (UTC) (envelope-from dweimer@dweimer.net) Received: from webmail.dweimer.net (24-240-198-188.static.stls.mo.charter.com [24.240.198.188]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AE56CB6; Fri, 30 Sep 2016 14:34:54 +0000 (UTC) (envelope-from dweimer@dweimer.net) Received: from webmail.dweimer.local (localhost [10.9.5.2]) by webmail.dweimer.net (8.15.2/8.15.2) with ESMTPS id u8UEYltD079759 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 30 Sep 2016 09:34:47 -0500 (CDT) (envelope-from dweimer@dweimer.net) Received: (from www@localhost) by webmail.dweimer.local (8.15.2/8.15.2/Submit) id u8UEYl2T079758; Fri, 30 Sep 2016 09:34:47 -0500 (CDT) (envelope-from dweimer@dweimer.net) X-Authentication-Warning: webmail.dweimer.local: www set sender to dweimer@dweimer.net using -f To: FreeBSD Stable Subject: Re: File Name Too Long? MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 30 Sep 2016 09:34:47 -0500 From: "Dean E. Weimer" Cc: owner-freebsd-stable@freebsd.org Organization: dweimer.net Reply-To: dweimer@dweimer.net Mail-Reply-To: dweimer@dweimer.net In-Reply-To: <974bc572bedda786fdc18a41085952c1@dweimer.net> References: <974bc572bedda786fdc18a41085952c1@dweimer.net> Message-ID: X-Sender: dweimer@dweimer.net User-Agent: Roundcube Webmail/1.2.1 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Sep 2016 14:34:54 -0000 On 2016-09-29 9:32 am, Dean E. Weimer wrote: > I discovered, unfortunately by deleting a jail by accident, that my > backup process isn't working. At least it was only the operating > system part of the jail, I still have all the data so I just need to > reinstall the operating system. While the ports are in the process of > building I started to investigate the cause, because the backup logs > report everything was fine. > > I have a custom pre-backup script I wrote that takes snapshots of my > ZFS datasets, and then mounts those under /mnt/backup with nullfs > mount points to the .zfs/snapshot/.. directories then I back them up > rather than the live file system, allowing me to stop some services > that don't restore from a running state correctly and then restart > after the snapshot so downtime is only a couple of minutes instead of > full length of backups. > > It appeared to be running perfectly, without errors, but apparently > the script isn't reporting some nullfs mount failures, so why are the > failing, turns out it thinks the file name is too long? but looking at > the mount(2) man page it states this: > > [ENAMETOOLONG] A component of a pathname exceeded 255 characters, > or > the entire length of a path name exceeded 1023 > characters. > > I can see that at some point under this, I may reach that 1023 limit, > but what of the total 71 characters in this path has a problem? > > /jails/unifi/ROOT/.zfs/snapshot/11.0-RELEASE-r306379-2016.09.28--bsnap > > root@freebsd:/jails/unifi/ROOT/.zfs/snapshot # ls > ls: 11.0-RELEASE-r306379-2016.09.28--bsnap: File name too long > > I thought, maybe its a ZFS specific error, and ran across this: > http://lists.freebsd.org/pipermail/freebsd-fs/2010-March/007964.html > > [..snip..] > From looking at the code, I think you hitting this limit: > > /* > * Be ultra-paranoid about making sure the type and fspath > * variables will fit in our mp buffers, including the > * terminating NUL. > */ > if (strlen(fstype) >= MFSNAMELEN || strlen(fspath) >= MNAMELEN) > return (ENAMETOOLONG); > > in vfs_domount() or vfs_donmount(). > > This is FreeBSD limit caused by statfs structure: > > /* > * filesystem statistics > */ > [...] > #define MNAMELEN 88 /* size of on/from name bufs */ > [...] > struct statfs { > [...] > char f_mntfromname[MNAMELEN];/* mounted filesystem */ > char f_mntonname[MNAMELEN]; /* directory on which mounted */ > }; > > When you list .zfs/snapshot/ directory (especially with -l option) ZFS > mounts snapshots on lookup and this is this mount that fails. > [..snip..] > > I can seemingly due anything else with the snapshot, clone, send, > receive its just that I am unable to access the files on it through > .zfs/snapshot/.. > > I am trying to find what the limit is here from this, because this one > here works. > > /jails/webmail/usr-local-subversion/.zfs/snapshot/usr-local-subversion--bsnap > > its longer in total length than most of the ones that are failing. > > /jails/unifi/ROOT/.zfs/snapshot/11.0-RELEASE-r306379-2016.09.28--bsnap > > So it appears that its in the name, and not the mount point. > > this one works as well, which is my ZFS boot environment on the main > system > zraid/ROOT/11.0-RELEASE-r306379-2016.09.28 > snapshot is /.zfs/snapshot/11.0-RELEASE-r306379-2016.09.28--bsnap > > So its not just the last component of the zfs dataset name, which is > in this case the same. > > I am trying to wrap my head around this and find where the limit is so > I can adjust my naming conventions used and actually get backups of > all of my data. Turns out all of my jail operating system paths aren't > being backed up, fortunately at least all of the data file systems for > the jails are. I found a solution, I was naming the snapshots with the dataset name, which I think was causing the issue. The following didn't seem to long to be an issue /jails/unifi/ROOT/.zfs/snapshot/11.0-RELEASE-r306379-2016.09.28--bsnap But apparently the snapshot name was zraid/jails/unifi/11.0-RELEASE-r306379-2016.09.28@11.0-RELEASE-r306379-2016.09.28--bsnap Still not sure how it adds up to too long, both full paths together aren't over 255, at 160, but apparently something else is added in there. I was able to easily modify my backup script to not include the last part of the dataset in the snapshot name and simply use -bsnap-, as the name. it appears to avoid all the issues, and my backups from last night include all the files. /jails/unifi/ROOT/.zfs/snapshot/-bsnap- zraid/jails/unifi/11.0-RELEASE-r306379-2016.09.28@-bsnap- The total path now only adds up to 98, I haven't done any testing yet to find out where the limit is hit, The longest combination of these I had last night would have added up to 135, and that worked -- Thanks, Dean E. Weimer http://www.dweimer.net/ From owner-freebsd-stable@freebsd.org Fri Sep 30 14:39:07 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DF506C038FD; Fri, 30 Sep 2016 14:39:07 +0000 (UTC) (envelope-from bsd@bontempi.net) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B6BC73F0; Fri, 30 Sep 2016 14:39:07 +0000 (UTC) (envelope-from bsd@bontempi.net) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 5289F210D8; Fri, 30 Sep 2016 10:39:06 -0400 (EDT) Received: from web3 ([10.202.2.213]) by compute2.internal (MEProxy); Fri, 30 Sep 2016 10:39:06 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=bontempi.net; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-sasl-enc :x-sasl-enc; s=mesmtp; bh=Y1R6oCteFhQiUg+IVsVXB14TDdo=; b=ArZ317 QzEhYayGCyBLxqTlpVMvMooJCWVnuXQ5GeEpumv/ckpRrmu+pCOfyV5op6m9jwTz nqBCg6/47XCJCqhvj2OU4q+2fRWDj/F/JZFEz26ITRD+Z9SapA6HEpArDcOSj4eJ 1D+WFdJHtRL7DEJJeOaPEZBo0/HBb3mFqGJ/0= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-sasl-enc:x-sasl-enc; s=smtpout; bh=Y1R6oCteFhQiUg+ IVsVXB14TDdo=; b=mR8yeFShjUhVUmF6sCm3LIIYFOboNN3T9LZqAbk093hI++b hLqHajzYrUJQbsToT6SdvpgwOBEv+44ZPlPv+JVXtmRjZGUp1+YmxDn3nOq/nNKP NF2w0ZpvG/IJaXRGaAWBrHAvH7zVg/eVYRbpo2r2uEG5Bq+2yAFPO2ZnG3kA= Received: by mailuser.nyi.internal (Postfix, from userid 99) id 21E7B2EC53; Fri, 30 Sep 2016 10:39:06 -0400 (EDT) Message-Id: <1475246346.1079471.741974313.29D6C2CB@webmail.messagingengine.com> X-Sasl-Enc: +9MQUzkEvJeM5jHA5ss6rc6OGYiV5TUuo2kkg+lyaIKr 1475246346 From: Priyadarshan To: "Dean E. Weimer" , FreeBSD Stable Cc: owner-freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Mailer: MessagingEngine.com Webmail Interface - ajax-334cb6bb Subject: Re: File Name Too Long? Date: Fri, 30 Sep 2016 14:39:06 +0000 In-Reply-To: References: <974bc572bedda786fdc18a41085952c1@dweimer.net> X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Sep 2016 14:39:08 -0000 On Fri, 30 Sep 2016, at 14:34, Dean E. Weimer wrote: > On 2016-09-29 9:32 am, Dean E. Weimer wrote: > > I discovered, unfortunately by deleting a jail by accident, that my > > backup process isn't working. At least it was only the operating > > system part of the jail, I still have all the data so I just need to > > reinstall the operating system. While the ports are in the process of > > building I started to investigate the cause, because the backup logs > > report everything was fine. > > > > I have a custom pre-backup script I wrote that takes snapshots of my > > ZFS datasets, and then mounts those under /mnt/backup with nullfs > > mount points to the .zfs/snapshot/.. directories then I back them up > > rather than the live file system, allowing me to stop some services > > that don't restore from a running state correctly and then restart > > after the snapshot so downtime is only a couple of minutes instead of > > full length of backups. > > > > It appeared to be running perfectly, without errors, but apparently > > the script isn't reporting some nullfs mount failures, so why are the > > failing, turns out it thinks the file name is too long? but looking at > > the mount(2) man page it states this: > > > > [ENAMETOOLONG] A component of a pathname exceeded 255 characters, > > or > > the entire length of a path name exceeded 1023 > > characters. > > > > I can see that at some point under this, I may reach that 1023 limit, > > but what of the total 71 characters in this path has a problem? > > > > /jails/unifi/ROOT/.zfs/snapshot/11.0-RELEASE-r306379-2016.09.28--bsnap > > > > root@freebsd:/jails/unifi/ROOT/.zfs/snapshot # ls > > ls: 11.0-RELEASE-r306379-2016.09.28--bsnap: File name too long > > > > I thought, maybe its a ZFS specific error, and ran across this: > > http://lists.freebsd.org/pipermail/freebsd-fs/2010-March/007964.html > > > > [..snip..] > > From looking at the code, I think you hitting this limit: > > > > /* > > * Be ultra-paranoid about making sure the type and fspath > > * variables will fit in our mp buffers, including the > > * terminating NUL. > > */ > > if (strlen(fstype) >= MFSNAMELEN || strlen(fspath) >= MNAMELEN) > > return (ENAMETOOLONG); > > > > in vfs_domount() or vfs_donmount(). > > > > This is FreeBSD limit caused by statfs structure: > > > > /* > > * filesystem statistics > > */ > > [...] > > #define MNAMELEN 88 /* size of on/from name bufs */ > > [...] > > struct statfs { > > [...] > > char f_mntfromname[MNAMELEN];/* mounted filesystem */ > > char f_mntonname[MNAMELEN]; /* directory on which mounted */ > > }; > > > > When you list .zfs/snapshot/ directory (especially with -l option) ZFS > > mounts snapshots on lookup and this is this mount that fails. > > [..snip..] > > > > I can seemingly due anything else with the snapshot, clone, send, > > receive its just that I am unable to access the files on it through > > .zfs/snapshot/.. > > > > I am trying to find what the limit is here from this, because this one > > here works. > > > > /jails/webmail/usr-local-subversion/.zfs/snapshot/usr-local-subversion--bsnap > > > > its longer in total length than most of the ones that are failing. > > > > /jails/unifi/ROOT/.zfs/snapshot/11.0-RELEASE-r306379-2016.09.28--bsnap > > > > So it appears that its in the name, and not the mount point. > > > > this one works as well, which is my ZFS boot environment on the main > > system > > zraid/ROOT/11.0-RELEASE-r306379-2016.09.28 > > snapshot is /.zfs/snapshot/11.0-RELEASE-r306379-2016.09.28--bsnap > > > > So its not just the last component of the zfs dataset name, which is > > in this case the same. > > > > I am trying to wrap my head around this and find where the limit is so > > I can adjust my naming conventions used and actually get backups of > > all of my data. Turns out all of my jail operating system paths aren't > > being backed up, fortunately at least all of the data file systems for > > the jails are. > > I found a solution, I was naming the snapshots with the dataset name, > which I think was causing the issue. > > The following didn't seem to long to be an issue > /jails/unifi/ROOT/.zfs/snapshot/11.0-RELEASE-r306379-2016.09.28--bsnap > > But apparently the snapshot name was > zraid/jails/unifi/11.0-RELEASE-r306379-2016.09.28@11.0-RELEASE-r306379-2016.09.28--bsnap > > Still not sure how it adds up to too long, both full paths together > aren't over 255, at 160, but apparently something else is added in > there. I was able to easily modify my backup script to not include the > last part of the dataset in the snapshot name and simply use -bsnap-, as > the name. it appears to avoid all the issues, and my backups from last > night include all the files. > > /jails/unifi/ROOT/.zfs/snapshot/-bsnap- > zraid/jails/unifi/11.0-RELEASE-r306379-2016.09.28@-bsnap- > > The total path now only adds up to 98, I haven't done any testing yet to > find out where the limit is hit, The longest combination of these I had > last night would have added up to 135, and that worked > > -- > Thanks, > Dean E. Weimer > http://www.dweimer.net/ This may be related: http://iocage.readthedocs.io/en/latest/known-issues.html#character-mount-path-limitation Priyadarshan From owner-freebsd-stable@freebsd.org Fri Sep 30 14:52:52 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 263BFC03F18; Fri, 30 Sep 2016 14:52:52 +0000 (UTC) (envelope-from dweimer@dweimer.net) Received: from webmail.dweimer.net (24-240-198-188.static.stls.mo.charter.com [24.240.198.188]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 02C47FBB; Fri, 30 Sep 2016 14:52:51 +0000 (UTC) (envelope-from dweimer@dweimer.net) Received: from webmail.dweimer.local (localhost [10.9.5.2]) by webmail.dweimer.net (8.15.2/8.15.2) with ESMTPS id u8UEqlE2080354 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 30 Sep 2016 09:52:48 -0500 (CDT) (envelope-from dweimer@dweimer.net) Received: (from www@localhost) by webmail.dweimer.local (8.15.2/8.15.2/Submit) id u8UEql6h080353; Fri, 30 Sep 2016 09:52:47 -0500 (CDT) (envelope-from dweimer@dweimer.net) X-Authentication-Warning: webmail.dweimer.local: www set sender to dweimer@dweimer.net using -f To: Priyadarshan Subject: Re: File Name Too Long? MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 30 Sep 2016 09:52:47 -0500 From: "Dean E. Weimer" Cc: FreeBSD Stable , owner-freebsd-stable@freebsd.org Organization: dweimer.net Reply-To: dweimer@dweimer.net Mail-Reply-To: dweimer@dweimer.net In-Reply-To: <1475246346.1079471.741974313.29D6C2CB@webmail.messagingengine.com> References: <974bc572bedda786fdc18a41085952c1@dweimer.net> <1475246346.1079471.741974313.29D6C2CB@webmail.messagingengine.com> Message-ID: <7cb903d1b035e6bc1d311c1c37f57fd7@dweimer.net> X-Sender: dweimer@dweimer.net User-Agent: Roundcube Webmail/1.2.1 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Sep 2016 14:52:52 -0000 On 2016-09-30 9:39 am, Priyadarshan wrote: > On Fri, 30 Sep 2016, at 14:34, Dean E. Weimer wrote: >> On 2016-09-29 9:32 am, Dean E. Weimer wrote: >> > I discovered, unfortunately by deleting a jail by accident, that my >> > backup process isn't working. At least it was only the operating >> > system part of the jail, I still have all the data so I just need to >> > reinstall the operating system. While the ports are in the process of >> > building I started to investigate the cause, because the backup logs >> > report everything was fine. >> > >> > I have a custom pre-backup script I wrote that takes snapshots of my >> > ZFS datasets, and then mounts those under /mnt/backup with nullfs >> > mount points to the .zfs/snapshot/.. directories then I back them up >> > rather than the live file system, allowing me to stop some services >> > that don't restore from a running state correctly and then restart >> > after the snapshot so downtime is only a couple of minutes instead of >> > full length of backups. >> > >> > It appeared to be running perfectly, without errors, but apparently >> > the script isn't reporting some nullfs mount failures, so why are the >> > failing, turns out it thinks the file name is too long? but looking at >> > the mount(2) man page it states this: >> > >> > [ENAMETOOLONG] A component of a pathname exceeded 255 characters, >> > or >> > the entire length of a path name exceeded 1023 >> > characters. >> > >> > I can see that at some point under this, I may reach that 1023 limit, >> > but what of the total 71 characters in this path has a problem? >> > >> > /jails/unifi/ROOT/.zfs/snapshot/11.0-RELEASE-r306379-2016.09.28--bsnap >> > >> > root@freebsd:/jails/unifi/ROOT/.zfs/snapshot # ls >> > ls: 11.0-RELEASE-r306379-2016.09.28--bsnap: File name too long >> > >> > I thought, maybe its a ZFS specific error, and ran across this: >> > http://lists.freebsd.org/pipermail/freebsd-fs/2010-March/007964.html >> > >> > [..snip..] >> > From looking at the code, I think you hitting this limit: >> > >> > /* >> > * Be ultra-paranoid about making sure the type and fspath >> > * variables will fit in our mp buffers, including the >> > * terminating NUL. >> > */ >> > if (strlen(fstype) >= MFSNAMELEN || strlen(fspath) >= MNAMELEN) >> > return (ENAMETOOLONG); >> > >> > in vfs_domount() or vfs_donmount(). >> > >> > This is FreeBSD limit caused by statfs structure: >> > >> > /* >> > * filesystem statistics >> > */ >> > [...] >> > #define MNAMELEN 88 /* size of on/from name bufs */ >> > [...] >> > struct statfs { >> > [...] >> > char f_mntfromname[MNAMELEN];/* mounted filesystem */ >> > char f_mntonname[MNAMELEN]; /* directory on which mounted */ >> > }; >> > >> > When you list .zfs/snapshot/ directory (especially with -l option) ZFS >> > mounts snapshots on lookup and this is this mount that fails. >> > [..snip..] >> > >> > I can seemingly due anything else with the snapshot, clone, send, >> > receive its just that I am unable to access the files on it through >> > .zfs/snapshot/.. >> > >> > I am trying to find what the limit is here from this, because this one >> > here works. >> > >> > /jails/webmail/usr-local-subversion/.zfs/snapshot/usr-local-subversion--bsnap >> > >> > its longer in total length than most of the ones that are failing. >> > >> > /jails/unifi/ROOT/.zfs/snapshot/11.0-RELEASE-r306379-2016.09.28--bsnap >> > >> > So it appears that its in the name, and not the mount point. >> > >> > this one works as well, which is my ZFS boot environment on the main >> > system >> > zraid/ROOT/11.0-RELEASE-r306379-2016.09.28 >> > snapshot is /.zfs/snapshot/11.0-RELEASE-r306379-2016.09.28--bsnap >> > >> > So its not just the last component of the zfs dataset name, which is >> > in this case the same. >> > >> > I am trying to wrap my head around this and find where the limit is so >> > I can adjust my naming conventions used and actually get backups of >> > all of my data. Turns out all of my jail operating system paths aren't >> > being backed up, fortunately at least all of the data file systems for >> > the jails are. >> >> I found a solution, I was naming the snapshots with the dataset name, >> which I think was causing the issue. >> >> The following didn't seem to long to be an issue >> /jails/unifi/ROOT/.zfs/snapshot/11.0-RELEASE-r306379-2016.09.28--bsnap >> >> But apparently the snapshot name was >> zraid/jails/unifi/11.0-RELEASE-r306379-2016.09.28@11.0-RELEASE-r306379-2016.09.28--bsnap >> >> Still not sure how it adds up to too long, both full paths together >> aren't over 255, at 160, but apparently something else is added in >> there. I was able to easily modify my backup script to not include the >> last part of the dataset in the snapshot name and simply use -bsnap-, >> as >> the name. it appears to avoid all the issues, and my backups from last >> night include all the files. >> >> /jails/unifi/ROOT/.zfs/snapshot/-bsnap- >> zraid/jails/unifi/11.0-RELEASE-r306379-2016.09.28@-bsnap- >> >> The total path now only adds up to 98, I haven't done any testing yet >> to >> find out where the limit is hit, The longest combination of these I >> had >> last night would have added up to 135, and that worked >> >> -- >> Thanks, >> Dean E. Weimer >> http://www.dweimer.net/ > > > This may be related: > > http://iocage.readthedocs.io/en/latest/known-issues.html#character-mount-path-limitation > > Priyadarshan Thanks, that's probably it, the original snapshot name with its full data set path added up to 89, with that in mind I can edit my script to throw a warning if this limit is hit so that my backup logs will let me know if a data set gets missed. I need to edit it anyways so that a warning gets logged on the mount failure which was already occurring. It looks like I escaped the errors, so that the script returned successful and didn't make the Bacula backup job fail so that the data that did get mounted would be backed up, but forgot to write the error to the log. -- Thanks, Dean E. Weimer http://www.dweimer.net/ From owner-freebsd-stable@freebsd.org Fri Sep 30 16:01:16 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 17D28C03787 for ; Fri, 30 Sep 2016 16:01:16 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 49F271F44 for ; Fri, 30 Sep 2016 16:01:14 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA20211; Fri, 30 Sep 2016 19:01:06 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1bq0FF-0001TH-Sn; Fri, 30 Sep 2016 19:01:05 +0300 Subject: Re: aibs(4) / atk0110 support for newer systems To: Torfinn Ingolfsen , freebsd-stable@FreeBSD.org References: <86cf8380-ac6f-55f0-f0f8-16000d7f04b2@FreeBSD.org> <20160930145704.4dbc9d90011154b38493964e@getmail.no> From: Andriy Gapon Message-ID: <7d498084-ec05-d4c9-5f49-6aef32495caf@FreeBSD.org> Date: Fri, 30 Sep 2016 19:00:09 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <20160930145704.4dbc9d90011154b38493964e@getmail.no> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Sep 2016 16:01:16 -0000 On 30/09/2016 15:57, Torfinn Ingolfsen wrote: > On Fri, 30 Sep 2016 14:30:02 +0300 > Andriy Gapon wrote: > >> Testing the patch should be as easy as building, installing and loading aibs >> module (found in sys/modules/acpi/aibs). Well, also don't forget to apply the >> patch with patch -p1 :-) > > The patch doesn't apply cleanly on FreeBSD 9.3-stable: > root@kg-core1# patch -C -p1 < /home/tingo/dl/aibs-ggrp-gitm.diff > Hmm... Looks like a unified diff to me... > The text leading up to this was: > -------------------------- > |diff --git a/sys/dev/acpi_support/atk0110.c b/sys/dev/acpi_support/atk0110.c > |index e79a0e00e3577..743684100f737 100644 > |--- a/sys/dev/acpi_support/atk0110.c > |+++ b/sys/dev/acpi_support/atk0110.c > -------------------------- > Patching file sys/dev/acpi_support/atk0110.c using Plan A... > Hunk #1 succeeded at 28. > Hunk #2 succeeded at 52. > Hunk #3 succeeded at 78. > Hunk #4 succeeded at 124. > Hunk #5 succeeded at 362. > Hunk #6 succeeded at 370. > Hunk #7 failed at 391. > Hunk #8 succeeded at 435. > Hunk #9 succeeded at 450. > Hunk #10 failed at 488. > 2 out of 10 hunks failed--saving rejects to sys/dev/acpi_support/atk0110.c.rej > done > > I'm running > root@kg-core1# uname -a > FreeBSD kg-core1.kg4.no 9.3-STABLE FreeBSD 9.3-STABLE #3 r304838: Fri Aug 26 12:11:25 CEST 2016 root@kg-core1.kg4.no:/usr/obj/usr/src/sys/GENERIC amd64 Please show the .rej file. Perhaps gpatch --merge=diff3 would work better than the base system patch if the conflicts are trivial enough. -- Andriy Gapon From owner-freebsd-stable@freebsd.org Fri Sep 30 16:03:18 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9C13C0398F for ; Fri, 30 Sep 2016 16:03:18 +0000 (UTC) (envelope-from mnd999@gmail.com) Received: from mail-wm0-x230.google.com (mail-wm0-x230.google.com [IPv6:2a00:1450:400c:c09::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3D9DB3F4; Fri, 30 Sep 2016 16:03:18 +0000 (UTC) (envelope-from mnd999@gmail.com) Received: by mail-wm0-x230.google.com with SMTP id b4so55497467wmb.0; Fri, 30 Sep 2016 09:03:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:subject:from:to:cc:date:mime-version; bh=06gvERHVvkQXd8DqAlPlzL+8dPjHVMIVjDj4+QPPUsk=; b=xxyziCnlQdyYqUkbc+ElLl/CgWnfyHhpzHUD/HzVJW0LYlfh4ajQi4FjmdHsKytWCj wufE3/e2jrO1VxmuXFqmIhpdh49nwAj/1TqN42o7nrU2tW4jCwd9D4r7CZ8567ezU/TE V+buFIYhhdMu7pJfmXzaOMDup4IbmpgcNMZShSDWeWPkE4MHk1LLxsWjGRJ/MoHhpLgs VempMWtcRXAyNCRA6VawgjSdMmM8b23s9/Wy0m50RhGFykqK++rMcpASqNA7W9QFkxno 6mb9HFTLa6K/k9sT3jwBw+jdaASNY5dTBSG/ywXOT6FJUp49gBHgPzU7ZJv+ol6t8p/8 QkNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:subject:from:to:cc:date:mime-version; bh=06gvERHVvkQXd8DqAlPlzL+8dPjHVMIVjDj4+QPPUsk=; b=DT5Hl8F9gRS/Zcxt9pDBGUDK/UBy6ciKudO/TLaDC3Gs9OVgYngMt5pQMe+maaDBrq ZJfL28nEBnh+jANvRQD19RyvSsliknE4ljErSkYcUHBSuD83uaaQPIO8+68lLxnpX3nr 8Wqd1e1UK/xTfl/3+bfZBd9LOA4A8zw/JzT95bTAa9XWzqGhe9M0qIMSf9dnizDTny8R uI5OCUwtGO6I4IGR6ug3DKBcSDbbh8jI4hYoeHXcdFqAJH86H5UY7LACWhEVhkZYCBF3 hLaho60LFPP07b8ZlDmkHAISvPiqAyq9TwXR8pAzDGLyBPLi4E101Cb//jAQOh9MnyeA O11A== X-Gm-Message-State: AA6/9Rm7NWMCJQifJy4gYyysb453ca8kvMAKQDztxe8avAZvSK+EWKJ1bivmO0KajeflOg== X-Received: by 10.194.82.72 with SMTP id g8mr7236497wjy.214.1475251396461; Fri, 30 Sep 2016 09:03:16 -0700 (PDT) Received: from 5.d.2.2.2.4.3.7.8.5.a.2.f.2.9.e.2.a.6.7.3.a.d.0.0.b.8.0.1.0.0.2.ip6.arpa (5.d.2.2.2.4.3.7.8.5.a.2.f.2.9.e.2.a.6.7.3.a.d.0.0.b.8.0.1.0.0.2.ip6.arpa. [2001:8b0:da3:76a2:e92f:2a58:7342:22d5]) by smtp.googlemail.com with ESMTPSA id a84sm4867252wme.6.2016.09.30.09.03.15 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Sep 2016 09:03:15 -0700 (PDT) Message-ID: <1475251389.41013.1.camel@gmail.com> Subject: Re: aibs(4) / atk0110 support for newer systems From: Mark Dixon To: Andriy Gapon Cc: freebsd-stable@FreeBSD.org Date: Fri, 30 Sep 2016 17:03:09 +0100 Content-Type: multipart/signed; micalg="pgp-sha256"; protocol="application/pgp-signature"; boundary="=-xbhNo+h+86fY4nB2zOpI" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Sep 2016 16:03:18 -0000 --=-xbhNo+h+86fY4nB2zOpI Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Andriy, I tried your patch on an Asus Z170-Deluxe on 11-RELEASEt and aibs did not seem to pick up anything before or after. Not 100% what sensors this board has. Regards, Mark --=-xbhNo+h+86fY4nB2zOpI Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCAAGBQJX7oy+AAoJEJlj26yR63HULygQALZ9oZuFows0EyEX28J2C35z 3Y6s1D8BZKAwnvCToT0mxXRNNKllWfG+X0WDliUcTI2oTlTw/ZEkLjMWQX8hws1v OcoT/2VdXDndFKMU8zhxzsi0MhTsrYS5RlsjCyq9r2ZA9nzhHwyvSY0FPL2cxL7K DEG7wUxJX3Xa7NkQoDZhTI4lAB2dwaj2eDSkJGH/5V+STxnmn+YATIatBpttngua hazbebqobFaO0XvyxjkWCqtgNejwXIvhx6H6l02uahxsLv9m8UXSa4KWvzSsWV02 rlYEKKzaX9eVpGj/Rf/Y4qZQhOZI1BrzsfVjFcxBhZerxTiFTHUb9hq69Ubvw1Fy hSKeTwF2ajRhC8w+I/1WiT7Gi76FM6MzupoKmFVIKBdxPpRnvmP/G3bp0/VQEpLu BKvAa49VSeuxGLN5R81cP9PCa/kfmGmpXMoVLi3Qz0hxK+4RJf5u7wCVTULiM0kK nybKTYMWinyI2fbuZSOigkWzHIgCrhVvpJVaYZXE3mX1S89Q9xpkXxlP4tnaHeI9 ybp4z5o2E+5KfcYIS7bqNUBPF6HkL53vIPMtqhemC24P0FrBeO9vIA2gYY3+0wXD YFVv2l8OuYXz73raUrZ3tdDTBOl7/emrvla3eXTnqRl91Juo+dhxsT6LEJ1rXGln O3udDP+mKnpwcgw4b4Xu =g/5b -----END PGP SIGNATURE----- --=-xbhNo+h+86fY4nB2zOpI-- From owner-freebsd-stable@freebsd.org Fri Sep 30 16:37:39 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 20A5DC03775 for ; Fri, 30 Sep 2016 16:37:39 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 31BAA1BA for ; Fri, 30 Sep 2016 16:37:37 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA20284; Fri, 30 Sep 2016 19:37:34 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1bq0oY-0001Vd-Gh; Fri, 30 Sep 2016 19:37:34 +0300 Subject: Re: aibs(4) / atk0110 support for newer systems To: Mark Dixon References: <1475251389.41013.1.camel@gmail.com> Cc: freebsd-stable@FreeBSD.org From: Andriy Gapon Message-ID: <762b6d74-2239-7119-669c-f5c47d77f640@FreeBSD.org> Date: Fri, 30 Sep 2016 19:36:38 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <1475251389.41013.1.camel@gmail.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Sep 2016 16:37:39 -0000 On 30/09/2016 19:03, Mark Dixon wrote: > Hi Andriy, > > I tried your patch on an Asus Z170-Deluxe on 11-RELEASEt and aibs did > not seem to pick up anything before or after. Not 100% what sensors > this board has. If acpidump -d | fgrep ATK0110 doesn't produce any result, then there is no chance of the driver working. Otherwise I would be interested in a full acpidump -dt output. -- Andriy Gapon From owner-freebsd-stable@freebsd.org Fri Sep 30 19:07:39 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B4F35C04C2C for ; Fri, 30 Sep 2016 19:07:39 +0000 (UTC) (envelope-from torfinn.ingolfsen@getmail.no) Received: from lamora.getmail.no (lamora.getmail.no [84.210.184.7]) by mx1.freebsd.org (Postfix) with ESMTP id 464E812B for ; Fri, 30 Sep 2016 19:07:38 +0000 (UTC) (envelope-from torfinn.ingolfsen@getmail.no) Received: from localhost (localhost [127.0.0.1]) by lamora.getmail.no (Postfix) with ESMTP id 3510A108DBF for ; Fri, 30 Sep 2016 20:59:29 +0200 (CEST) Received: from lamora.getmail.no ([127.0.0.1]) by localhost (lamora.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id V-1cTQH9v1dm for ; Fri, 30 Sep 2016 20:59:28 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by lamora.getmail.no (Postfix) with ESMTP id B7B76107A80 for ; Fri, 30 Sep 2016 20:59:28 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.8.4 lamora.getmail.no B7B76107A80 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=getmail.no; s=8A9C8B4C-D727-11E2-8095-B6466E6B3FA2; t=1475261968; bh=9oo+dzmUjSaSU9R0/MOJn7wjiZUAlHgoEfTxz74FRQw=; h=Date:From:To:Subject:Message-Id:Mime-Version:Content-Type: Content-Transfer-Encoding; b=CUoAR2H+YDFg8wKBJ/ftYFjeOsLdj2fkUy/uuOSKPzQP+za4LYS/xHeZhiEtUoky+ zl49ux5fSnAMiz1zsNqaCwd/WNymjbM9PWf2PCvB+pQLsBKa9LL1L9/cSSma7FfVEt Tx/0tNVlO37A35dqfxUSIP06eTXEu62n8Feq/N/Y= X-Virus-Scanned: amavisd-new at lamora.get.c.bitbit.net Received: from lamora.getmail.no ([127.0.0.1]) by localhost (lamora.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 0sFlZRSGPs40 for ; Fri, 30 Sep 2016 20:59:28 +0200 (CEST) Received: from kg-core1.kg4.no (cm-84.209.39.108.getinternet.no [84.209.39.108]) by lamora.getmail.no (Postfix) with ESMTPSA id 90334108DBF for ; Fri, 30 Sep 2016 20:59:28 +0200 (CEST) Date: Fri, 30 Sep 2016 20:59:28 +0200 From: Torfinn Ingolfsen To: freebsd-stable@freebsd.org Subject: Re: aibs(4) / atk0110 support for newer systems Message-Id: <20160930205928.77d7e74f7bd1a35fcf1aa50a@getmail.no> In-Reply-To: <7d498084-ec05-d4c9-5f49-6aef32495caf@FreeBSD.org> References: <86cf8380-ac6f-55f0-f0f8-16000d7f04b2@FreeBSD.org> <20160930145704.4dbc9d90011154b38493964e@getmail.no> <7d498084-ec05-d4c9-5f49-6aef32495caf@FreeBSD.org> X-Mailer: Sylpheed 3.5.1 (GTK+ 2.24.29; amd64-portbld-freebsd9.3) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Sep 2016 19:07:39 -0000 On Fri, 30 Sep 2016 19:00:09 +0300 Andriy Gapon wrote: > On 30/09/2016 15:57, Torfinn Ingolfsen wrote: > > On Fri, 30 Sep 2016 14:30:02 +0300 > > Andriy Gapon wrote: > > > >> Testing the patch should be as easy as building, installing and loading aibs > >> module (found in sys/modules/acpi/aibs). Well, also don't forget to apply the > >> patch with patch -p1 :-) > > > > The patch doesn't apply cleanly on FreeBSD 9.3-stable: > > root@kg-core1# patch -C -p1 < /home/tingo/dl/aibs-ggrp-gitm.diff > > Hmm... Looks like a unified diff to me... > > The text leading up to this was: > > -------------------------- > > |diff --git a/sys/dev/acpi_support/atk0110.c b/sys/dev/acpi_support/atk0110.c > > |index e79a0e00e3577..743684100f737 100644 > > |--- a/sys/dev/acpi_support/atk0110.c > > |+++ b/sys/dev/acpi_support/atk0110.c > > -------------------------- > > Patching file sys/dev/acpi_support/atk0110.c using Plan A... > > Hunk #1 succeeded at 28. > > Hunk #2 succeeded at 52. > > Hunk #3 succeeded at 78. > > Hunk #4 succeeded at 124. > > Hunk #5 succeeded at 362. > > Hunk #6 succeeded at 370. > > Hunk #7 failed at 391. > > Hunk #8 succeeded at 435. > > Hunk #9 succeeded at 450. > > Hunk #10 failed at 488. > > 2 out of 10 hunks failed--saving rejects to sys/dev/acpi_support/atk0110.c.rej > > done > > > > I'm running > > root@kg-core1# uname -a > > FreeBSD kg-core1.kg4.no 9.3-STABLE FreeBSD 9.3-STABLE #3 r304838: Fri Aug 26 12:11:25 CEST 2016 root@kg-core1.kg4.no:/usr/obj/usr/src/sys/GENERIC amd64 > > Please show the .rej file. As I used '-C' as an argument to patch, there isn't one: root@kg-core1# l /sys/dev/acpi_support/atk0110.c.rej ls: /sys/dev/acpi_support/atk0110.c.rej: No such file or directory -- Torfinn Ingolfsen From owner-freebsd-stable@freebsd.org Fri Sep 30 19:36:08 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B0F87C0368A for ; Fri, 30 Sep 2016 19:36:08 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-yw0-x22a.google.com (mail-yw0-x22a.google.com [IPv6:2607:f8b0:4002:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6F738A05 for ; Fri, 30 Sep 2016 19:36:08 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: by mail-yw0-x22a.google.com with SMTP id i129so75147385ywb.0 for ; Fri, 30 Sep 2016 12:36:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=longcount-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=N6W5kyNvnKyEVgGPVb1yUTpbd50o/DxlubiGvVMsEAE=; b=Hyzbv6WQGFkK2UDamC9trA/Ijew20zysGV49vuVSVoNHs2RtgFe7FeeaJFeLOy/vgf F+eIW0/nR1IRFgkw1GFh9/HEOew+ezffuhI9TFQeGv5++YicR3Y6kts156bn0LH9zaMC Q/Y3yTOQyvXS6GlT8e6i/asNISF3cD9Q+UTjfU+OV1D5Jrc4ReG0DcJq77bMtZgCN0un V20EArm7lR6DfYJ1GUrX80kFaNxIadfKCOvKMLBZMFznL1UhJYqZ+PTElMCZT8w90NXR OR1AIYZ7De7YrYdNPYd7WsHrnzMHPy241HIiquBCSo2HY7HFL3j2vas+a1Y9c7X2P7vc cjGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=N6W5kyNvnKyEVgGPVb1yUTpbd50o/DxlubiGvVMsEAE=; b=VAB6uQ7Fzwb6J7IZ+eronfOui56X/pLLXVWqxMipTfThEqxoUMJP6OAP6C2Nrhnksn BxFRL5ULHfi02Po45TWrbx318Tz3zGM4iAK9U6FJSUMDbipEIkDotc0eyReIGx+Fc0pb jw5xmDrihCtfh3cvXCAbPZoHBz7MgwJ7IwpbPTtbsFcaNjgJZgdNTeovikl8ZbNpPWeR cO2N3qLslV4MhGZz4Ltye0OcO9xOztDIt6gJxlqg0ZE/hKPehrO/Nd62Jg5GIN3T1MyB 5otAL9OpTyJC0hAERS85aKkPzPgO8+7psyo5XnCQqYxi+V8FZqafY+/w9lVkFEwiPLuu VJCw== X-Gm-Message-State: AA6/9RmE6rHUaVuJUUxb+KvU/Gv+wfTCVKLssvLJ3qVw2cmGuyaHrwo+NyV3VQ8gbntzxp+YPu/xeknEGzT2nQ== X-Received: by 10.13.198.194 with SMTP id i185mr5592640ywd.132.1475264167371; Fri, 30 Sep 2016 12:36:07 -0700 (PDT) MIME-Version: 1.0 Received: by 10.13.230.68 with HTTP; Fri, 30 Sep 2016 12:36:06 -0700 (PDT) X-Originating-IP: [24.44.110.108] In-Reply-To: <7cb903d1b035e6bc1d311c1c37f57fd7@dweimer.net> References: <974bc572bedda786fdc18a41085952c1@dweimer.net> <1475246346.1079471.741974313.29D6C2CB@webmail.messagingengine.com> <7cb903d1b035e6bc1d311c1c37f57fd7@dweimer.net> From: Mark Saad Date: Fri, 30 Sep 2016 15:36:06 -0400 Message-ID: Subject: Re: File Name Too Long? To: dweimer@dweimer.net Cc: Priyadarshan , owner-freebsd-stable@freebsd.org, FreeBSD Stable Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Sep 2016 19:36:08 -0000 On Fri, Sep 30, 2016 at 10:52 AM, Dean E. Weimer wrote: > On 2016-09-30 9:39 am, Priyadarshan wrote: > >> On Fri, 30 Sep 2016, at 14:34, Dean E. Weimer wrote: >> >>> On 2016-09-29 9:32 am, Dean E. Weimer wrote: >>> > I discovered, unfortunately by deleting a jail by accident, that my >>> > backup process isn't working. At least it was only the operating >>> > system part of the jail, I still have all the data so I just need to >>> > reinstall the operating system. While the ports are in the process of >>> > building I started to investigate the cause, because the backup logs >>> > report everything was fine. >>> > >>> > I have a custom pre-backup script I wrote that takes snapshots of my >>> > ZFS datasets, and then mounts those under /mnt/backup with nullfs >>> > mount points to the .zfs/snapshot/.. directories then I back them up >>> > rather than the live file system, allowing me to stop some services >>> > that don't restore from a running state correctly and then restart >>> > after the snapshot so downtime is only a couple of minutes instead of >>> > full length of backups. >>> > >>> > It appeared to be running perfectly, without errors, but apparently >>> > the script isn't reporting some nullfs mount failures, so why are the >>> > failing, turns out it thinks the file name is too long? but looking at >>> > the mount(2) man page it states this: >>> > >>> > [ENAMETOOLONG] A component of a pathname exceeded 255 characters, >>> > or >>> > the entire length of a path name exceeded 1023 >>> > characters. >>> > >>> > I can see that at some point under this, I may reach that 1023 limit, >>> > but what of the total 71 characters in this path has a problem? >>> > >>> > /jails/unifi/ROOT/.zfs/snapshot/11.0-RELEASE-r306379-2016.09.28--bsnap >>> > >>> > root@freebsd:/jails/unifi/ROOT/.zfs/snapshot # ls >>> > ls: 11.0-RELEASE-r306379-2016.09.28--bsnap: File name too long >>> > >>> > I thought, maybe its a ZFS specific error, and ran across this: >>> > http://lists.freebsd.org/pipermail/freebsd-fs/2010-March/007964.html >>> > >>> > [..snip..] >>> > From looking at the code, I think you hitting this limit: >>> > >>> > /* >>> > * Be ultra-paranoid about making sure the type and fspath >>> > * variables will fit in our mp buffers, including the >>> > * terminating NUL. >>> > */ >>> > if (strlen(fstype) >= MFSNAMELEN || strlen(fspath) >= MNAMELEN) >>> > return (ENAMETOOLONG); >>> > >>> > in vfs_domount() or vfs_donmount(). >>> > >>> > This is FreeBSD limit caused by statfs structure: >>> > >>> > /* >>> > * filesystem statistics >>> > */ >>> > [...] >>> > #define MNAMELEN 88 /* size of on/from name bufs */ >>> > [...] >>> > struct statfs { >>> > [...] >>> > char f_mntfromname[MNAMELEN];/* mounted filesystem */ >>> > char f_mntonname[MNAMELEN]; /* directory on which mounted */ >>> > }; >>> > >>> > When you list .zfs/snapshot/ directory (especially with -l option) ZFS >>> > mounts snapshots on lookup and this is this mount that fails. >>> > [..snip..] >>> > >>> > I can seemingly due anything else with the snapshot, clone, send, >>> > receive its just that I am unable to access the files on it through >>> > .zfs/snapshot/.. >>> > >>> > I am trying to find what the limit is here from this, because this one >>> > here works. >>> > >>> > /jails/webmail/usr-local-subversion/.zfs/snapshot/usr-local- >>> subversion--bsnap >>> > >>> > its longer in total length than most of the ones that are failing. >>> > >>> > /jails/unifi/ROOT/.zfs/snapshot/11.0-RELEASE-r306379-2016.09.28--bsnap >>> > >>> > So it appears that its in the name, and not the mount point. >>> > >>> > this one works as well, which is my ZFS boot environment on the main >>> > system >>> > zraid/ROOT/11.0-RELEASE-r306379-2016.09.28 >>> > snapshot is /.zfs/snapshot/11.0-RELEASE-r306379-2016.09.28--bsnap >>> > >>> > So its not just the last component of the zfs dataset name, which is >>> > in this case the same. >>> > >>> > I am trying to wrap my head around this and find where the limit is so >>> > I can adjust my naming conventions used and actually get backups of >>> > all of my data. Turns out all of my jail operating system paths aren't >>> > being backed up, fortunately at least all of the data file systems for >>> > the jails are. >>> >>> I found a solution, I was naming the snapshots with the dataset name, >>> which I think was causing the issue. >>> >>> The following didn't seem to long to be an issue >>> /jails/unifi/ROOT/.zfs/snapshot/11.0-RELEASE-r306379-2016.09.28--bsnap >>> >>> But apparently the snapshot name was >>> zraid/jails/unifi/11.0-RELEASE-r306379-2016.09.28@11.0- >>> RELEASE-r306379-2016.09.28--bsnap >>> >>> Still not sure how it adds up to too long, both full paths together >>> aren't over 255, at 160, but apparently something else is added in >>> there. I was able to easily modify my backup script to not include the >>> last part of the dataset in the snapshot name and simply use -bsnap-, as >>> the name. it appears to avoid all the issues, and my backups from last >>> night include all the files. >>> >>> /jails/unifi/ROOT/.zfs/snapshot/-bsnap- >>> zraid/jails/unifi/11.0-RELEASE-r306379-2016.09.28@-bsnap- >>> >>> The total path now only adds up to 98, I haven't done any testing yet to >>> find out where the limit is hit, The longest combination of these I had >>> last night would have added up to 135, and that worked >>> >>> -- >>> Thanks, >>> Dean E. Weimer >>> http://www.dweimer.net/ >>> >> >> >> This may be related: >> >> http://iocage.readthedocs.io/en/latest/known-issues.html#cha >> racter-mount-path-limitation >> >> Priyadarshan >> > > Having run into this before in nfs exports , and pulling my hair out; I ran into this fix . I know its from a long time ago; but I am willing to put up a bounty to get this bumped to 512 http://www.secnetix.de/olli/FreeBSD/mnamelen.hawk > Thanks, that's probably it, the original snapshot name with its full data > set path added up to 89, with that in mind I can edit my script to throw a > warning if this limit is hit so that my backup logs will let me know if a > data set gets missed. I need to edit it anyways so that a warning gets > logged on the mount failure which was already occurring. It looks like I > escaped the errors, so that the script returned successful and didn't make > the Bacula backup job fail so that the data that did get mounted would be > backed up, but forgot to write the error to the log. > > -- > Thanks, > Dean E. Weimer > http://www.dweimer.net/ > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > -- mark saad | nonesuch@longcount.org From owner-freebsd-stable@freebsd.org Fri Sep 30 21:09:19 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 73B38C04B8A for ; Fri, 30 Sep 2016 21:09:19 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id CB9673A5 for ; Fri, 30 Sep 2016 21:09:18 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id AAA20875; Sat, 01 Oct 2016 00:09:16 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1bq53U-0001li-C8; Sat, 01 Oct 2016 00:09:16 +0300 Subject: Re: aibs(4) / atk0110 support for newer systems To: Torfinn Ingolfsen , freebsd-stable@FreeBSD.org References: <86cf8380-ac6f-55f0-f0f8-16000d7f04b2@FreeBSD.org> <20160930145704.4dbc9d90011154b38493964e@getmail.no> <7d498084-ec05-d4c9-5f49-6aef32495caf@FreeBSD.org> <20160930205928.77d7e74f7bd1a35fcf1aa50a@getmail.no> From: Andriy Gapon Message-ID: <7a868c22-e0bd-f677-e4ad-2bdf6f3605d0@FreeBSD.org> Date: Sat, 1 Oct 2016 00:08:15 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <20160930205928.77d7e74f7bd1a35fcf1aa50a@getmail.no> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Sep 2016 21:09:19 -0000 On 30/09/2016 21:59, Torfinn Ingolfsen wrote: > As I used '-C' as an argument to patch, there isn't one: > root@kg-core1# l /sys/dev/acpi_support/atk0110.c.rej > ls: /sys/dev/acpi_support/atk0110.c.rej: No such file or directory Right. So, I guess I have to specifically ask to not use -C option? :-) -- Andriy Gapon From owner-freebsd-stable@freebsd.org Fri Sep 30 21:45:11 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 11133C033F9 for ; Fri, 30 Sep 2016 21:45:11 +0000 (UTC) (envelope-from dave@mischler.com) Received: from teaspoon.mischlersflorist.com (rrcs-72-45-221-198.nys.biz.rr.com [72.45.221.198]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A73DBD40 for ; Fri, 30 Sep 2016 21:45:10 +0000 (UTC) (envelope-from dave@mischler.com) Received: from firkin.mischler.com ([192.168.163.29]) by teaspoon.mischlersflorist.com (8.15.2/8.14.7) with ESMTP id u8ULJtYT022545; Fri, 30 Sep 2016 17:19:55 -0400 (EDT) (envelope-from dave@mischler.com) Message-ID: <1475270395.24760.4.camel@mischler.com> Subject: Can't build 11-stable without BPF From: Dave Mischler Reply-To: dave@mischler.com To: freebsd-stable@freebsd.org Date: Fri, 30 Sep 2016 17:19:55 -0400 Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Sep 2016 21:45:11 -0000 When I remove the "device bpf" from the kernel configuration the resulting "make buildkernel KERNCONF=XXX" fails to compile in module lmc, source file if_lmc.c The problem seems to be that DEV_BPF is not defined, but this is not tested for. From owner-freebsd-stable@freebsd.org Fri Sep 30 22:11:08 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D227FC03E89 for ; Fri, 30 Sep 2016 22:11:08 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-61.reflexion.net [208.70.210.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 85EDBE8E for ; Fri, 30 Sep 2016 22:11:07 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 11970 invoked from network); 30 Sep 2016 22:04:12 -0000 Received: from unknown (HELO rtc-sm-01.app.dca.reflexion.local) (10.81.150.1) by 0 (rfx-qmail) with SMTP; 30 Sep 2016 22:04:12 -0000 Received: by rtc-sm-01.app.dca.reflexion.local (Reflexion email security v8.00.0) with SMTP; Fri, 30 Sep 2016 18:04:27 -0400 (EDT) Received: (qmail 20775 invoked from network); 30 Sep 2016 22:04:26 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 30 Sep 2016 22:04:26 -0000 Received: from [192.168.0.105] (ip70-189-131-151.lv.lv.cox.net [70.189.131.151]) by iron2.pdx.net (Postfix) with ESMTPSA id 6C011EC901D; Fri, 30 Sep 2016 15:04:21 -0700 (PDT) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Re: make universe fails with MAKEOBJDIRPREFIX [cross builds part: sh: ./make_keys: Exec format error for init_keytry.h] Date: Fri, 30 Sep 2016 15:04:18 -0700 Message-Id: <4464F4B4-6ABB-4DD8-8FC4-8E8F123F54A3@dsl-only.net> Cc: FreeBSD-STABLE Mailing List To: FreeBSD Current , Bryan Drewery , FreeBSD Toolchain , vangyzen@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Sep 2016 22:11:08 -0000 Quoting: > --- lib/ncurses/ncurses__L --- > ./make_keys keys.list > init_keytry.h . . . > --- lib/ncurses/ncurses__L --- > sh: ./make_keys: Exec format error > *** [init_keytry.h] Error code 126 Since the reworking of the build handling in recent (many) months "sh: = ./make_keys: Exec format error" is a long standing problem for = buildworld for cross builds. The native part of the build execution is = trying to execute the cross-build target's make_keys (that was likely = left from a prior cross-build and now doing incremental-build activity). = lib/ncurses/ncurses__L is not the only place that this can happen. There = is past list exchanges for this issue with other examples. I rebuild after cleanworld when this happens in order to force a full = rebuild instead of an incremental one --but I only build one cross build = target at a time, not universe. I've never had such an after-cleanworld = rebuild fail: only some later incremental builds. Luckily in my experience so far most incremental builds do not go back = through those areas (such as lib/ncurses/ncurses__L ) and so do not try = to execute make_keys (stated in terms of this example). So the = incremental builds for me are usually effective --but sometimes = lib/ncurses/ncurses__L is rebuilt. It has been a fairly long time since my 11.0 related activities = (tracking stable/11 now) have run into this but likely = lib/ncurses/ncurses__L (and the like) simply have not rebuilt in my = cross builds. I've no head (CURRENT) activities going on at this point. =3D=3D=3D Mark Millard markmi at dsl-only.net From owner-freebsd-stable@freebsd.org Fri Sep 30 23:00:47 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D278C046EE for ; Fri, 30 Sep 2016 23:00:47 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id 710C11D58; Fri, 30 Sep 2016 23:00:47 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id 98BE7CF; Fri, 30 Sep 2016 23:00:47 +0000 (UTC) Date: Fri, 30 Sep 2016 23:00:47 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: jenkins-admin@FreeBSD.org, freebsd-stable@FreeBSD.org Message-ID: <1617557779.3.1475276447373.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Jenkins build became unstable: FreeBSD_stable_10 #418 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkKKb2VAfYQKfu1t7qk4nR5qzUBEI+UqT4BPec4qHVhqUy0FFdq50sMH+3y9bCDNOufctov6VqTNffZ3YXArnZK95YF0OX97fh+E9txYOUX1adc+TikcKjuYpHmL5dE62eaZTI+4A5jnRonskQ1PaoIFz0Kbu4mWzkFsmdiXTraGzomXq4cHUCATA2+K4eDYgjXEQI30z3GOMmmZ4t/+6QGk1cMb/BqMWHbn80AsRCb4tU7Hpd72XLDpsuO7YRP1Q0CjmNAuBOTj+sFiiOe6U9HpqOlQN+iFUvBdZo/ybuy5Kh71cAaYQNL68cYdZJ6binH/DkG3KY/fS7DFYAeuwjwIDAQAB X-Jenkins-Job: FreeBSD_stable_10 X-Jenkins-Result: UNSTABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Sep 2016 23:00:47 -0000 https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/418/ From owner-freebsd-stable@freebsd.org Sat Oct 1 01:56:05 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6951BC0342F for ; Sat, 1 Oct 2016 01:56:05 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id 5B8F11BD2; Sat, 1 Oct 2016 01:56:05 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id 725E2D3; Sat, 1 Oct 2016 01:56:04 +0000 (UTC) Date: Sat, 1 Oct 2016 01:56:00 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: jenkins-admin@FreeBSD.org, freebsd-stable@FreeBSD.org Message-ID: <2100233956.4.1475286960914.JavaMail.jenkins@jenkins-9.freebsd.org> In-Reply-To: <1617557779.3.1475276447373.JavaMail.jenkins@jenkins-9.freebsd.org> References: <1617557779.3.1475276447373.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Jenkins build is still unstable: FreeBSD_stable_10 #419 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkKKb2VAfYQKfu1t7qk4nR5qzUBEI+UqT4BPec4qHVhqUy0FFdq50sMH+3y9bCDNOufctov6VqTNffZ3YXArnZK95YF0OX97fh+E9txYOUX1adc+TikcKjuYpHmL5dE62eaZTI+4A5jnRonskQ1PaoIFz0Kbu4mWzkFsmdiXTraGzomXq4cHUCATA2+K4eDYgjXEQI30z3GOMmmZ4t/+6QGk1cMb/BqMWHbn80AsRCb4tU7Hpd72XLDpsuO7YRP1Q0CjmNAuBOTj+sFiiOe6U9HpqOlQN+iFUvBdZo/ybuy5Kh71cAaYQNL68cYdZJ6binH/DkG3KY/fS7DFYAeuwjwIDAQAB X-Jenkins-Job: FreeBSD_stable_10 X-Jenkins-Result: UNSTABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Oct 2016 01:56:05 -0000 https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/419/ From owner-freebsd-stable@freebsd.org Sat Oct 1 16:04:06 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0B97FC0538B; Sat, 1 Oct 2016 16:04:06 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id E4F26126D; Sat, 1 Oct 2016 16:04:04 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Sat, 01 Oct 2016 18:02:59 +0200 Authentication-Results: connect.ultra-secure.de; iprev=pass; auth=pass (plain); spf=none smtp.mailfrom=ultra-secure.de Received-SPF: None (connect.ultra-secure.de: domain of ultra-secure.de does not designate 217.71.83.52 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=217.71.83.52; helo=[192.168.1.200]; envelope-from= Received: from [192.168.1.200] (217-071-083-052.ip-tech.ch [217.71.83.52]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id 8162B673-F8D6-4535-AEF2-116319825659.1 envelope-from (authenticated bits=0); Sat, 01 Oct 2016 18:02:47 +0200 From: Rainer Duffner Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: zfs receive leads to a stuck server Date: Sat, 1 Oct 2016 18:02:40 +0200 Message-Id: <4707908B-4868-4AA6-ADD6-D24121EFAE38@ultra-secure.de> Cc: freebsd-stable To: FreeBSD Filesystems Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Mailer: Apple Mail (2.3124) X-Haraka-GeoIP: EU, CH, 451km X-Haraka-ASN: 24951 X-Haraka-GeoIP-Received: X-Haraka-ASN: 24951 217.71.80.0/20 X-Haraka-ASN-CYMRU: asn=24951 net=217.71.80.0/20 country=CH assignor=ripencc date=2003-08-07 X-Haraka-FCrDNS: 217-071-083-052.ip-tech.ch X-Haraka-p0f: os="Mac OS X " link_type="DSL" distance=12 total_conn=1 shared_ip=N X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 1257, bad: 1, connections: 1567, history: 1256, asn_score: 736, asn_connections: 826, asn_good: 738, asn_bad: 2, pass:asn, relaying X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Oct 2016 16:04:06 -0000 Hi, I posted this before, but I didn=E2=80=99t really get an answer and = I=E2=80=99m still looking for ways to debug this. I have to servers (HP DL 380 Gen8, 192 GB RAM, 6C CPU). I=E2=80=99ve outfitted them with HP=E2=80=99s H22x cards (really OEMed = 9207-8x, three altogether) that I recently cross-flashed to LSI=E2=80=99s = latest firmware. They have: - 14 600GB SAS disks in their internal cages (2x ZFs mirror to boot = from and 2x RAIDZ2) - another 12 1200 GB SAS disks in an external HP 2700 enclosure (2x = RAID-Z2) - all the RAID Z2s together form a single zpool These are NFS-servers (master + warm standby) and MySQL slaves. The client uses the NFS-server and previously used both MySQL-Slaves for = SELECT queries only (via a middleware that seems to be able to sort this = out). When this system was built, we had twelve single RAID0 logical volumes = on HP P4x0 RAID-controllers (with 2 GB cache), merged into two RAID-Z2 = vdevs for storage (and a RAID1 boot-drive). However, the customer outgrew that solution and because there are no = native HP utilities for managing the RAID-cards on FreeBSD, each time a = drive dies on such a setup, you have to reboot the server to make sure = ZFS =E2=80=9Esees=E2=80=9C the new drive (go into adapter BIOS, destroy = RAID, create RAID=E2=80=A6). So, I switched to LSI-type JBOD HBAs. I had to re-install both servers from scratch, obviously, and also = upgraded to 10.3 (from 10.1). I have zfsnap (sysutils/zfsnap) create hourly, daily and weekly = snapshots that I transfer via zxfer from server1 to server2. Previously, this only took a couple of seconds. But since I switched to = LSI HBAs, it=E2=80=99s taking longer - even though the deltas are really = small in most cases. After 03:00 AM, when zfsnap deletes some snapshots on the source system, = it even takes several minutes to zxfer the snapshots. At that point, it=E2=80=99s really noticeable that the 2nd system = complete freezes. The first system (that zfs sends) is absolutely = unaffected and continues like normal. I=E2=80=99ve setup a cron-job that outputs iostat and vmstat into = second-timestamped files and a few seconds after the zxfer starts, it = doesn=E2=80=99t generate any new files until shortly before it finishes. With these hangs, the customer can=E2=80=99t use the 2nd MySQL-server = and is thus unhappy. I=E2=80=99ve upgraded to 11.0-RELEAE on the 2nd system - but it hasn=E2=80= =99t helped (it might even be slower actually). How can I debug this? What=E2=80=99s the best way forward? I don=E2=80=99t have any ZFS errors (checksum etc.). So, I doubt it=E2=80=99= s a cable. Here=E2=80=99s various system information ------------------------------------------------------------------------ ZFS Subsystem Report Sat Oct 1 17:50:48 2016 ------------------------------------------------------------------------ System Information: Kernel Version: 1100122 (osreldate) Hardware Platform: amd64 Processor Architecture: amd64 ZFS Storage pool Version: 5000 ZFS Filesystem Version: 5 FreeBSD 11.0-RELEASE #0 r306211: Thu Sep 22 21:43:30 UTC 2016 root 5:50PM up 2 days, 19:50, 1 users, load averages: 0.11, 0.09, 0.08 ------------------------------------------------------------------------ System Memory: 0.26% 506.77 MiB Active, 11.52% 21.54 GiB Inact 62.29% 116.49 GiB Wired, 0.00% 0 Cache 25.93% 48.49 GiB Free, 0.00% 0 Gap Real Installed: 192.00 GiB Real Available: 99.96% 191.93 GiB Real Managed: 97.44% 187.01 GiB Logical Total: 192.00 GiB Logical Used: 63.53% 121.97 GiB Logical Free: 36.47% 70.03 GiB Kernel Memory: 2.15 GiB Data: 98.41% 2.12 GiB Text: 1.59% 35.02 MiB Kernel Memory Map: 187.01 GiB Size: 40.16% 75.10 GiB Free: 59.84% 111.92 GiB ------------------------------------------------------------------------ ARC Summary: (HEALTHY) Memory Throttle Count: 0 ARC Misc: Deleted: 2.16m Recycle Misses: 0 Mutex Misses: 1.92k Evict Skips: 54.73k ARC Size: 64.39% 41.21 GiB Target Size: (Adaptive) 100.00% 64.00 GiB Min Size (Hard Limit): 36.33% 23.25 GiB Max Size (High Water): 2:1 64.00 GiB ARC Size Breakdown: Recently Used Cache Size: 89.29% 57.14 GiB Frequently Used Cache Size: 10.71% 6.86 GiB ARC Hash Breakdown: Elements Max: 5.37m Elements Current: 92.43% 4.97m Collisions: 2.58m Chain Max: 5 Chains: 333.76k ------------------------------------------------------------------------ ARC Efficiency: 265.65m Cache Hit Ratio: 61.83% 164.25m Cache Miss Ratio: 38.17% 101.40m Actual Hit Ratio: 55.18% 146.57m Data Demand Efficiency: 10.17% 7.66m Data Prefetch Efficiency: 0.00% 32 CACHE HITS BY CACHE LIST: Anonymously Used: 8.77% 14.41m Most Recently Used: 29.22% 48.00m Most Frequently Used: 60.02% 98.58m Most Recently Used Ghost: 1.67% 2.74m Most Frequently Used Ghost: 0.33% 536.90k CACHE HITS BY DATA TYPE: Demand Data: 0.47% 778.95k Prefetch Data: 0.00% 0 Demand Metadata: 88.30% 145.04m Prefetch Metadata: 11.22% 18.43m CACHE MISSES BY DATA TYPE: Demand Data: 6.79% 6.88m Prefetch Data: 0.00% 32 Demand Metadata: 84.42% 85.60m Prefetch Metadata: 8.79% 8.91m ------------------------------------------------------------------------ L2ARC is disabled ------------------------------------------------------------------------ File-Level Prefetch: (HEALTHY) DMU Efficiency: 970.94m Hit Ratio: 0.22% 2.09m Miss Ratio: 99.78% 968.85m Colinear: 0 Hit Ratio: 100.00% 0 Miss Ratio: 100.00% 0 Stride: 0 Hit Ratio: 100.00% 0 Miss Ratio: 100.00% 0 DMU Misc: Reclaim: 0 Successes: 100.00% 0 Failures: 100.00% 0 Streams: 0 +Resets: 100.00% 0 -Resets: 100.00% 0 Bogus: 0 ------------------------------------------------------------------------ VDEV cache is disabled ------------------------------------------------------------------------ ZFS Tunables (sysctl): kern.maxusers 12619 vm.kmem_size 200803987456 vm.kmem_size_scale 1 vm.kmem_size_min 0 vm.kmem_size_max 1319413950874 vfs.zfs.trim.max_interval 1 vfs.zfs.trim.timeout 30 vfs.zfs.trim.txg_delay 32 vfs.zfs.trim.enabled 1 vfs.zfs.vol.unmap_enabled 1 vfs.zfs.vol.recursive 0 vfs.zfs.vol.mode 1 vfs.zfs.version.zpl 5 vfs.zfs.version.spa 5000 vfs.zfs.version.acl 1 vfs.zfs.version.ioctl 6 vfs.zfs.debug 1 vfs.zfs.super_owner 0 vfs.zfs.sync_pass_rewrite 2 vfs.zfs.sync_pass_dont_compress 5 vfs.zfs.sync_pass_deferred_free 2 vfs.zfs.zio.exclude_metadata 0 vfs.zfs.zio.use_uma 1 vfs.zfs.cache_flush_disable 0 vfs.zfs.zil_replay_disable 0 vfs.zfs.min_auto_ashift 12 vfs.zfs.max_auto_ashift 13 vfs.zfs.vdev.trim_max_pending 10000 vfs.zfs.vdev.bio_delete_disable 0 vfs.zfs.vdev.bio_flush_disable 0 vfs.zfs.vdev.write_gap_limit 4096 vfs.zfs.vdev.read_gap_limit 32768 vfs.zfs.vdev.aggregation_limit 131072 vfs.zfs.vdev.trim_max_active 64 vfs.zfs.vdev.trim_min_active 1 vfs.zfs.vdev.scrub_max_active 2 vfs.zfs.vdev.scrub_min_active 1 vfs.zfs.vdev.async_write_max_active 10 vfs.zfs.vdev.async_write_min_active 1 vfs.zfs.vdev.async_read_max_active 3 vfs.zfs.vdev.async_read_min_active 1 vfs.zfs.vdev.sync_write_max_active 10 vfs.zfs.vdev.sync_write_min_active 10 vfs.zfs.vdev.sync_read_max_active 10 vfs.zfs.vdev.sync_read_min_active 10 vfs.zfs.vdev.max_active 1000 vfs.zfs.vdev.async_write_active_max_dirty_percent60 vfs.zfs.vdev.async_write_active_min_dirty_percent30 vfs.zfs.vdev.mirror.non_rotating_seek_inc1 vfs.zfs.vdev.mirror.non_rotating_inc 0 vfs.zfs.vdev.mirror.rotating_seek_offset1048576 vfs.zfs.vdev.mirror.rotating_seek_inc 5 vfs.zfs.vdev.mirror.rotating_inc 0 vfs.zfs.vdev.trim_on_init 1 vfs.zfs.vdev.cache.bshift 16 vfs.zfs.vdev.cache.size 0 vfs.zfs.vdev.cache.max 16384 vfs.zfs.vdev.metaslabs_per_vdev 200 vfs.zfs.txg.timeout 5 vfs.zfs.space_map_blksz 4096 vfs.zfs.spa_slop_shift 5 vfs.zfs.spa_asize_inflation 24 vfs.zfs.deadman_enabled 1 vfs.zfs.deadman_checktime_ms 5000 vfs.zfs.deadman_synctime_ms 1000000 vfs.zfs.debug_flags 0 vfs.zfs.recover 0 vfs.zfs.spa_load_verify_data 1 vfs.zfs.spa_load_verify_metadata 1 vfs.zfs.spa_load_verify_maxinflight 10000 vfs.zfs.ccw_retry_interval 300 vfs.zfs.check_hostid 1 vfs.zfs.mg_fragmentation_threshold 85 vfs.zfs.mg_noalloc_threshold 0 vfs.zfs.condense_pct 200 vfs.zfs.metaslab.bias_enabled 1 vfs.zfs.metaslab.lba_weighting_enabled 1 vfs.zfs.metaslab.fragmentation_factor_enabled1 vfs.zfs.metaslab.preload_enabled 1 vfs.zfs.metaslab.preload_limit 3 vfs.zfs.metaslab.unload_delay 8 vfs.zfs.metaslab.load_pct 50 vfs.zfs.metaslab.min_alloc_size 33554432 vfs.zfs.metaslab.df_free_pct 4 vfs.zfs.metaslab.df_alloc_threshold 131072 vfs.zfs.metaslab.debug_unload 0 vfs.zfs.metaslab.debug_load 0 vfs.zfs.metaslab.fragmentation_threshold70 vfs.zfs.metaslab.gang_bang 16777217 vfs.zfs.free_bpobj_enabled 1 vfs.zfs.free_max_blocks -1 vfs.zfs.no_scrub_prefetch 0 vfs.zfs.no_scrub_io 0 vfs.zfs.resilver_min_time_ms 3000 vfs.zfs.free_min_time_ms 1000 vfs.zfs.scan_min_time_ms 1000 vfs.zfs.scan_idle 50 vfs.zfs.scrub_delay 4 vfs.zfs.resilver_delay 2 vfs.zfs.top_maxinflight 32 vfs.zfs.zfetch.array_rd_sz 1048576 vfs.zfs.zfetch.max_distance 8388608 vfs.zfs.zfetch.min_sec_reap 2 vfs.zfs.zfetch.max_streams 8 vfs.zfs.prefetch_disable 0 vfs.zfs.delay_scale 500000 vfs.zfs.delay_min_dirty_percent 60 vfs.zfs.dirty_data_sync 67108864 vfs.zfs.dirty_data_max_percent 10 vfs.zfs.dirty_data_max_max 4294967296 vfs.zfs.dirty_data_max 4294967296 vfs.zfs.max_recordsize 1048576 vfs.zfs.mdcomp_disable 0 vfs.zfs.nopwrite_enabled 1 vfs.zfs.dedup.prefetch 1 vfs.zfs.l2c_only_size 0 vfs.zfs.mfu_ghost_data_lsize 57031680 vfs.zfs.mfu_ghost_metadata_lsize 24999013888 vfs.zfs.mfu_ghost_size 25056045568 vfs.zfs.mfu_data_lsize 7619584 vfs.zfs.mfu_metadata_lsize 2427032064 vfs.zfs.mfu_size 7355872768 vfs.zfs.mru_ghost_data_lsize 0 vfs.zfs.mru_ghost_metadata_lsize 27116232704 vfs.zfs.mru_ghost_size 27116232704 vfs.zfs.mru_data_lsize 12935902208 vfs.zfs.mru_metadata_lsize 18539549696 vfs.zfs.mru_size 31494193152 vfs.zfs.anon_data_lsize 0 vfs.zfs.anon_metadata_lsize 0 vfs.zfs.anon_size 233472 vfs.zfs.l2arc_norw 1 vfs.zfs.l2arc_feed_again 1 vfs.zfs.l2arc_noprefetch 1 vfs.zfs.l2arc_feed_min_ms 200 vfs.zfs.l2arc_feed_secs 1 vfs.zfs.l2arc_headroom 2 vfs.zfs.l2arc_write_boost 8388608 vfs.zfs.l2arc_write_max 8388608 vfs.zfs.arc_meta_limit 17179869184 vfs.zfs.arc_free_target 339900 vfs.zfs.arc_shrink_shift 7 vfs.zfs.arc_average_blocksize 8192 vfs.zfs.arc_min 24966280704 vfs.zfs.arc_max 68719476736 = =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94 Copyright (c) 1992-2016 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights = reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-RELEASE #0 r306211: Thu Sep 22 21:43:30 UTC 2016 root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on = LLVM 3.8.0) VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz (2294.52-MHz K8-class CPU) Origin=3D"GenuineIntel" Id=3D0x206d7 Family=3D0x6 Model=3D0x2d = Stepping=3D7 = Features=3D0xbfebfbff = Features2=3D0x1fbee3ff AMD Features=3D0x2c100800 AMD Features2=3D0x1 XSAVE Features=3D0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID TSC: P-state invariant, performance statistics real memory =3D 206158430208 (196608 MB) avail memory =3D 200330240000 (191049 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 hardware threads random: unblocking device. ACPI BIOS Warning (bug): Invalid length for FADT/Pm1aControlBlock: 32, = using default 16 (20160527/tbfadt-733) ACPI BIOS Warning (bug): Invalid length for FADT/Pm2ControlBlock: 32, = using default 8 (20160527/tbfadt-733) ioapic1 irqs 24-47 on motherboard ioapic0 irqs 0-23 on motherboard random: entropy device external interface kbd1 at kbdmux0 netmap: loaded module module_register_init: MOD_LOAD (vesa, 0xffffffff8101c950, 0) error 19 vtvga0: on motherboard cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) cpu0: numa-domain 0 on acpi0 cpu1: numa-domain 0 on acpi0 cpu2: numa-domain 0 on acpi0 cpu3: numa-domain 0 on acpi0 cpu4: numa-domain 0 on acpi0 cpu5: numa-domain 0 on acpi0 cpu6: numa-domain 0 on acpi0 cpu7: numa-domain 0 on acpi0 cpu8: numa-domain 0 on acpi0 cpu9: numa-domain 0 on acpi0 cpu10: numa-domain 0 on acpi0 cpu11: numa-domain 0 on acpi0 attimer0: port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 atrtc0: port 0x70-0x71 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x908-0x90b on acpi0 pcib0: numa-domain 0 on acpi0 pcib0: _OSC returned error 0x10 pci0: numa-domain 0 on pcib0 pcib1: at device 1.0 numa-domain 0 on pci0 pci1: numa-domain 0 on pcib1 mps0: port 0x6000-0x60ff mem = 0xfbef0000-0xfbefffff,0xfbe80000-0xfbebffff irq 26 at device 0.0 = numa-domain 0 on pci1 mps0: Firmware: 20.00.07.00, Driver: 21.01.00.00-fbsd mps0: IOCCapabilities: = 5285c= pcib2: at device 1.1 numa-domain 0 on pci0 pci2: numa-domain 0 on pcib2 pcib3: at device 2.0 numa-domain 0 on pci0 pci3: numa-domain 0 on pcib3 bge0: mem = 0xfabf0000-0xfabfffff,0xfabe0000-0xfabeffff,0xfabd0000-0xfabdffff irq 32 = at device 0.0 numa-domain 0 on pci3 bge0: APE FW version: NCSI v1.3.12.0 bge0: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E miibus0: numa-domain 0 on bge0 brgphy0: PHY 1 on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, = 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge0: Using defaults for TSO: 65518/35/2048 bge0: Ethernet address: 2c:44:fd:91:c2:bc bge1: mem = 0xfabc0000-0xfabcffff,0xfabb0000-0xfabbffff,0xfaba0000-0xfabaffff irq 36 = at device 0.1 numa-domain 0 on pci3 bge1: APE FW version: NCSI v1.3.12.0 bge1: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E miibus1: numa-domain 0 on bge1 brgphy1: PHY 2 on miibus1 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, = 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge1: Using defaults for TSO: 65518/35/2048 bge1: Ethernet address: 2c:44:fd:91:c2:bd bge2: mem = 0xfab90000-0xfab9ffff,0xfab80000-0xfab8ffff,0xfab70000-0xfab7ffff irq 32 = at device 0.2 numa-domain 0 on pci3 bge2: APE FW version: NCSI v1.3.12.0 bge2: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E miibus2: numa-domain 0 on bge2 brgphy2: PHY 3 on miibus2 brgphy2: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, = 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge2: Using defaults for TSO: 65518/35/2048 bge2: Ethernet address: 2c:44:fd:91:c2:be bge3: mem = 0xfab60000-0xfab6ffff,0xfab50000-0xfab5ffff,0xfab40000-0xfab4ffff irq 36 = at device 0.3 numa-domain 0 on pci3 bge3: APE FW version: NCSI v1.3.12.0 bge3: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E miibus3: numa-domain 0 on bge3 brgphy3: PHY 4 on miibus3 brgphy3: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, = 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge3: Using defaults for TSO: 65518/35/2048 bge3: Ethernet address: 2c:44:fd:91:c2:bf pcib4: at device 2.1 numa-domain 0 on pci0 pci4: numa-domain 0 on pcib4 pcib5: at device 2.2 numa-domain 0 on pci0 pci5: numa-domain 0 on pcib5 pcib6: at device 2.3 numa-domain 0 on pci0 pci6: numa-domain 0 on pcib6 pcib7: at device 3.0 numa-domain 0 on pci0 pci7: numa-domain 0 on pcib7 mps1: port 0x5000-0x50ff mem = 0xfbdf0000-0xfbdfffff,0xfbd80000-0xfbdbffff irq 40 at device 0.0 = numa-domain 0 on pci7 mps1: Firmware: 20.00.07.00, Driver: 21.01.00.00-fbsd mps1: IOCCapabilities: = 5285c= pcib8: at device 3.1 numa-domain 0 on pci0 pci8: numa-domain 0 on pcib8 pcib9: at device 3.2 numa-domain 0 on pci0 pci9: numa-domain 0 on pcib9 pcib10: at device 3.3 numa-domain 0 on pci0 pci10: numa-domain 0 on pcib10 pcib11: at device 17.0 numa-domain 0 on pci0 pci11: numa-domain 0 on pcib11 ehci0: mem 0xfac60000-0xfac603ff irq = 21 at device 26.0 numa-domain 0 on pci0 usbus0: EHCI version 1.0 usbus0 numa-domain 0 on ehci0 pcib12: at device 28.0 numa-domain 0 on pci0 pci12: numa-domain 0 on pcib12 mps2: port 0x7000-0x70ff mem = 0xfbff0000-0xfbffffff,0xfbf80000-0xfbfbffff irq 16 at device 0.0 = numa-domain 0 on pci12 mps2: Firmware: 20.00.07.00, Driver: 21.01.00.00-fbsd mps2: IOCCapabilities: = 5285c= pcib13: at device 28.7 numa-domain 0 on pci0 pci13: numa-domain 0 on pcib13 vgapci0: mem = 0xf9000000-0xf9ffffff,0xfbce0000-0xfbce3fff,0xfb000000-0xfb7fffff irq 16 = at device 0.1 numa-domain 0 on pci13 vgapci0: Boot video device uhci0: port 0x3c00-0x3c1f irq = 16 at device 0.4 numa-domain 0 on pci13 usbus1 numa-domain 0 on uhci0 ehci1: mem 0xfac50000-0xfac503ff irq = 20 at device 29.0 numa-domain 0 on pci0 usbus2: EHCI version 1.0 usbus2 numa-domain 0 on ehci1 pcib14: at device 30.0 numa-domain 0 on pci0 pci14: numa-domain 0 on pcib14 isab0: at device 31.0 numa-domain 0 on pci0 isa0: numa-domain 0 on isab0 atapci0: port = 0x4000-0x4007,0x4008-0x400b,0x4010-0x4017,0x4018-0x401b,0x4020-0x402f,0x40= 30-0x403f irq 17 at device 31.2 numa-domain 0 on pci0 ata2: at channel 0 on atapci0 ata3: at channel 1 on atapci0 acpi_tz0: on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] uart1: port 0x2f8-0x2ff irq = 3 on acpi0 orm0: at iomem 0xc0000-0xc7fff on isa0 ppc0: cannot reserve I/O port range uart0: at port 0x3f8 irq 4 = flags 0x10 on isa0 uart0: console (115200,n,8,1) est0: numa-domain 0 on cpu0 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 221b00001a00 device_attach: est0 attach returned 6 est1: numa-domain 0 on cpu1 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 221b00001a00 device_attach: est1 attach returned 6 est2: numa-domain 0 on cpu2 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 221b00001a00 device_attach: est2 attach returned 6 est3: numa-domain 0 on cpu3 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 221b00001a00 device_attach: est3 attach returned 6 est4: numa-domain 0 on cpu4 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 221b00001a00 device_attach: est4 attach returned 6 est5: numa-domain 0 on cpu5 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 221b00001a00 device_attach: est5 attach returned 6 est6: numa-domain 0 on cpu6 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 221b00001a00 device_attach: est6 attach returned 6 est7: numa-domain 0 on cpu7 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 221b00001a00 device_attach: est7 attach returned 6 est8: numa-domain 0 on cpu8 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 221b00001a00 device_attach: est8 attach returned 6 est9: numa-domain 0 on cpu9 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 221b00001a00 device_attach: est9 attach returned 6 est10: numa-domain 0 on cpu10 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 221b00001a00 device_attach: est10 attach returned 6 est11: numa-domain 0 on cpu11 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 221b00001a00 device_attach: est11 attach returned 6 usbus0: 480Mbps High Speed USB v2.0 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec nvme cam probe device init usbus1: 12Mbps Full Speed USB v1.0 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 usbus2: 480Mbps High Speed USB v2.0 ugen2.1: at usbus2 uhub2: on usbus2 uhub1: 2 ports with 2 removable, self powered uhub0: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered ugen1.2: at usbus1 ukbd0: on usbus1 kbd2 at ukbd0 ugen0.2: at usbus0 uhub3: = on usbus0 ugen2.2: at usbus2 uhub4: = on usbus2 uhub3: 6 ports with 6 removable, self powered uhub4: 8 ports with 8 removable, self powered ugen2.3: at usbus2 uhub5: = on usbus2 uhub5: 2 ports with 1 removable, self powered ses0 at mps2 bus 0 scbus2 target 10 lun 0 ses0: Fixed Enclosure Services SPC-3 SCSI = device ses0: Serial Number 7CE539P12M =20 ses0: 600.000MB/s transfers ses0: Command Queueing enabled ses0: SCSI-3 ENC Device (da0:mps0:0:8:0): UNMAPPED (da1:da0 at mps0 bus 0 scbus0 target 8 lun 0 da0: Fixed Direct Access SPC-4 SCSI device da0: Serial Number S0M1F2B10000B418HVFX da0: 600.000MB/s transfers da0: Command Queueing enabled da0: 572325MB (1172123568 512 byte sectors) mps0:0:9:0): UNMAPPED (da2:da1 at mps0 bus 0 scbus0 target 9 lun 0 da1: Fixed Direct Access SPC-4 SCSI device da1: Serial Number S0M1F7CG0000M419BS87 da1: 600.000MB/s transfers da1: Command Queueing enabled da1: 572325MB (1172123568 512 byte sectors) ses0: (none): SAS Device Slot Element: 1 Phys at Slot 1, Not All Phys ses0: phy 0: SAS device type 1 id 0 ses0: phy 0: protocols: Initiator( None ) Target( SSP ) ses0: phy 0: parent 5001438035a2617f addr 5000c5008ec197c9 ses0: (none): SAS Device Slot Element: 1 Phys at Slot 2, Not All Phys ses0: phy 0: SAS device type 1 id 0 ses0: phy 0: protocols: Initiator( None ) Target( SSP ) ses0: phy 0: parent 5001438035a2617f addr 5000c5008ebde785 ses0: (none): SAS Device Slot Element: 1 Phys at Slot 3, Not All Phys ses0: phy 0: SAS device type 1 id 0 ses0: phy 0: protocols: Initiator( None ) Target( SSP ) ses0: phy 0: parent 5001438035a2617f addr 5000c5008ec1ae59 ses0: (none): SAS Device Slot Element: 1 Phys at Slot 4, Not All Phys ses0: phy 0: SAS device type 1 id 0 ses0: phy 0: protocols: Initiator( None ) Target( SSP ) ses0: phy 0: parent 5001438035a2617f addr 5000c5008eb2e04d ses0: (none): SAS Device Slot Element: 1 Phys at Slot 5, Not All Phys ses0: phy 0: SAS device type 1 id 0 ses0: phy 0: protocols: Initiator( None ) Target( SSP ) ses0: phy 0: parent 5001438035a2617f addr 5000c5008ebdfe25 ses0: (none): SAS Device Slot Element: 1 Phys at Slot 6, Not All Phys ses0: phy 0: SAS device type 1 id 0 ses0: phy 0: protocols: Initiator( None ) Target( SSP ) ses0: phy 0: parent 5001438035a2617f addr 5000c5008ebfea71 ses0: (none): SAS Device Slot Element: 1 Phys at Slot 7, Not All Phys ses0: phy 0: SAS device type 1 id 0 ses0: phy 0: protocols: Initiator( None ) Target( SSP ) ses0: phy 0: parent 5001438035a2617f addr 5000c5008ec3f06d ses0: (none): SAS Device Slot Element: 1 Phys at Slot 8, Not All Phys ses0: phy 0: SAS device type 1 id 0 ses0: phy 0: protocols: Initiator( None ) Target( SSP ) ses0: phy 0: parent 5001438035a2617f addr 5000c5008ebf2ab5 ses0: (none): SAS Device Slot Element: 1 Phys at Slot 9, Not All Phys ses0: phy 0: SAS device type 1 id 0 ses0: phy 0: protocols: Initiator( None ) Target( SSP ) ses0: phy 0: parent 5001438035a2617f addr 5000c5008ec3c2f9 ses0: (none): SAS Device Slot Element: 1 Phys at Slot 10, Not All Phys ses0: phy 0: SAS device type 1 id 0 ses0: phy 0: protocols: Initiator( None ) Target( SSP ) ses0: phy 0: parent 5001438035a2617f addr 5000c5008ebedb35 ses0: (none): SAS Device Slot Element: 1 Phys at Slot 11, Not All Phys ses0: phy 0: SAS device type 1 id 0 ses0: phy 0: protocols: Initiator( None ) Target( SSP ) ses0: phy 0: parent 5001438035a2617f addr 5000c5008ec012b9 ses0: (none): SAS Device Slot Element: 1 Phys at Slot 12, Not All Phys ses0: phy 0: SAS device type 1 id 0 ses0: phy 0: protocols: Initiator( None ) Target( SSP ) ses0: phy 0: parent 5001438035a2617f addr 5000c5008ec58d09 mps0:0:10:0): UNMAPPED (da3:da2 at mps0 bus 0 scbus0 target 10 lun 0 da2: Fixed Direct Access SPC-4 SCSI device da2: Serial Number S0M1EQSL0000B419CE37 da2: 600.000MB/s transfers da2: Command Queueing enabled da2: 572325MB (1172123568 512 byte sectors) mps0:0:11:0): UNMAPPED (da4:da3 at mps0 bus 0 scbus0 target 11 lun 0 da3: Fixed Direct Access SPC-4 SCSI device da3: Serial Number S0M1EQHL0000B419CG8Y da3: 600.000MB/s transfers da3: Command Queueing enabled da3: 572325MB (1172123568 512 byte sectors) mps0:0:12:0): UNMAPPED (da5:da4 at mps0 bus 0 scbus0 target 12 lun 0 da4: Fixed Direct Access SPC-4 SCSI device da4: Serial Number S0M1ERQR0000M41919NZ da4: 600.000MB/s transfers da4: Command Queueing enabled da4: 572325MB (1172123568 512 byte sectors) mps0:0:13:0): UNMAPPED (da6:da5 at mps0 bus 0 scbus0 target 13 lun 0 da5: Fixed Direct Access SPC-4 SCSI device da5: Serial Number S0M1EP8H0000M419667Y da5: 600.000MB/s transfers da5: Command Queueing enabled da5: 572325MB (1172123568 512 byte sectors) mps0:0:14:0): UNMAPPED (da7:da6 at mps0 bus 0 scbus0 target 14 lun 0 da6: Fixed Direct Access SPC-4 SCSI device da6: Serial Number S0M1F9B00000B419CQF1 da6: 600.000MB/s transfers da6: Command Queueing enabled da6: 572325MB (1172123568 512 byte sectors) mps0:0:15:0): UNMAPPED (da8:da7 at mps0 bus 0 scbus0 target 15 lun 0 da7: Fixed Direct Access SPC-4 SCSI device da7: Serial Number S0M1ER7Y0000B418J2VA da7: 600.000MB/s transfers da7: Command Queueing enabled da7: 572325MB (1172123568 512 byte sectors) mps1:0:8:0): UNMAPPED (da9:da8 at mps1 bus 0 scbus1 target 8 lun 0 da8: Fixed Direct Access SPC-4 SCSI device da8: Serial Number S0M1E2VZ0000B419CQAF da8: 600.000MB/s transfers da8: Command Queueing enabled da8: 572325MB (1172123568 512 byte sectors) mps1:0:9:0): UNMAPPED (da10:da9 at mps1 bus 0 scbus1 target 9 lun 0 da9: Fixed Direct Access SPC-4 SCSI device da9: Serial Number S0M1ESN20000M418DKFL da9: 600.000MB/s transfers da9: Command Queueing enabled da9: 572325MB (1172123568 512 byte sectors) mps1:0:10:0): UNMAPPED (da11:da10 at mps1 bus 0 scbus1 target 10 lun 0 da10: Fixed Direct Access SPC-4 SCSI device da10: Serial Number S0M1F8V00000B419CPWR da10: 600.000MB/s transfers da10: Command Queueing enabled da10: 572325MB (1172123568 512 byte sectors) mps1:0:11:0): UNMAPPED (da12:da11 at mps1 bus 0 scbus1 target 11 lun 0 da11: Fixed Direct Access SPC-4 SCSI device da11: Serial Number S0M1ESLL0000B419DALG da11: 600.000MB/s transfers da11: Command Queueing enabled da11: 572325MB (1172123568 512 byte sectors) mps1:0:12:0): UNMAPPED (da13:da12 at mps1 bus 0 scbus1 target 12 lun 0 da12: Fixed Direct Access SPC-4 SCSI device da12: Serial Number S0M1DXJR0000B419CJKG da12: 600.000MB/s transfers da12: Command Queueing enabled da12: 572325MB (1172123568 512 byte sectors) mps1:0:13:0): UNMAPPED (da14:da13 at mps1 bus 0 scbus1 target 13 lun 0 da13: Fixed Direct Access SPC-4 SCSI device da13: Serial Number S0M1ES7R0000B419CJZX da13: 600.000MB/s transfers da13: Command Queueing enabled da13: 572325MB (1172123568 512 byte sectors) mps1:0:14:0): UNMAPPED (da15:da14 at mps1 bus 0 scbus1 target 14 lun 0 da14: Fixed Direct Access SPC-4 SCSI device da14: Serial Number S0M19J9D0000B419CEEC da14: 600.000MB/s transfers da14: Command Queueing enabled da14: 572325MB (1172123568 512 byte sectors) mps1:0:15:0): UNMAPPED (da16:da15 at mps1 bus 0 scbus1 target 15 lun 0 da15: Fixed Direct Access SPC-4 SCSI device da15: Serial Number S0M1EQPR0000B419CGAC da15: 600.000MB/s transfers da15: Command Queueing enabled da15: 572325MB (1172123568 512 byte sectors) mps2:0:11:0): UNMAPPED (da17:da16 at mps2 bus 0 scbus2 target 11 lun 0 da16: Fixed Direct Access SPC-4 SCSI device da16: Serial Number S3L29R3L0000M608CP3Y da16: 600.000MB/s transfers da16: Command Queueing enabled da16: 1144641MB (2344225968 512 byte sectors) mps2:0:13:0): UNMAPPED (da18:da17 at mps2 bus 0 scbus2 target 13 lun 0 da17: Fixed Direct Access SPC-4 SCSI device da17: Serial Number S3L29XFQ0000M608CQD6 da17: 600.000MB/s transfers da17: Command Queueing enabled da17: 1144641MB (2344225968 512 byte sectors) mps2:0:14:0): UNMAPPED (da19:da18 at mps2 bus 0 scbus2 target 14 lun 0 da18: Fixed Direct Access SPC-4 SCSI device da18: Serial Number S3L29QTK0000M608CN9K da18: 600.000MB/s transfers da18: Command Queueing enabled da18: 1144641MB (2344225968 512 byte sectors) mps2:0:15:0): UNMAPPED (da20:da19 at mps2 bus 0 scbus2 target 15 lun 0 da19: Fixed Direct Access SPC-4 SCSI device da19: Serial Number S3L29TAA0000M608CS65 da19: 600.000MB/s transfers da19: Command Queueing enabled da19: 1144641MB (2344225968 512 byte sectors) mps2:0:16:0): UNMAPPED (da21:da20 at mps2 bus 0 scbus2 target 16 lun 0 da20: Fixed Direct Access SPC-4 SCSI device da20: Serial Number S3L28ZFA0000M607T1JN da20: 600.000MB/s transfers da20: Command Queueing enabled da20: 1144641MB (2344225968 512 byte sectors) mps2:0:17:0): UNMAPPED (da22:da21 at mps2 bus 0 scbus2 target 17 lun 0 da21: Fixed Direct Access SPC-4 SCSI device da21: Serial Number S3L29PG90000M6089MMG da21: 600.000MB/s transfers da21: Command Queueing enabled da21: 1144641MB (2344225968 512 byte sectors) mps2:0:18:0): UNMAPPED (da23:da22 at mps2 bus 0 scbus2 target 18 lun 0 da22: Fixed Direct Access SPC-4 SCSI device da22: Serial Number S3L29RHR0000M608STP7 da22: 600.000MB/s transfers da22: Command Queueing enabled da22: 1144641MB (2344225968 512 byte sectors) mps2:0:19:0): UNMAPPED (da24:da23 at mps2 bus 0 scbus2 target 19 lun 0 da23: Fixed Direct Access SPC-4 SCSI device da23: Serial Number S3L29VQT0000M6069ZU8 da23: 600.000MB/s transfers da23: Command Queueing enabled da23: 1144641MB (2344225968 512 byte sectors) mps2:0:20:0): UNMAPPED (da25:da24 at mps2 bus 0 scbus2 target 20 lun 0 da24: Fixed Direct Access SPC-4 SCSI device da24: Serial Number S3L2A7WM0000M608SSU7 da24: 600.000MB/s transfers da24: Command Queueing enabled da24: 1144641MB (2344225968 512 byte sectors) mps2:0:21:0): UNMAPPED (da26:da25 at mps2 bus 0 scbus2 target 21 lun 0 da25: Fixed Direct Access SPC-4 SCSI device da25: Serial Number S3L29GXN0000M605A4C6 da25: 600.000MB/s transfers da25: Command Queueing enabled da25: 1144641MB (2344225968 512 byte sectors) mps2:0:22:0): UNMAPPED (da27:da26 at mps2 bus 0 scbus2 target 22 lun 0 da26: Fixed Direct Access SPC-4 SCSI device da26: Serial Number S3L29TPT0000M608CSAA da26: 600.000MB/s transfers da26: Command Queueing enabled da26: 1144641MB (2344225968 512 byte sectors) mps2:0:23:0): UNMAPPED da27 at mps2 bus 0 scbus2 target 23 lun 0 da27: Fixed Direct Access SPC-4 SCSI device da27: Serial Number S3L2A4EJ0000M607J095 da27: 600.000MB/s transfers da27: Command Queueing enabled da27: 1144641MB (2344225968 512 byte sectors) SMP: AP CPU #1 Launched! SMP: AP CPU #10 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #8 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #11 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #9 Launched! SMP: AP CPU #5 Launched! Timecounter "TSC-low" frequency 1147258217 Hz quality 1000 Trying to mount root from zfs:zroot/ROOT/default []... GEOM_MIRROR: Device mirror/swap launched (2/2). GEOM_ELI: Device mirror/swap.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: software bge0: link state changed to DOWN bge1: link state changed to DOWN bge2: link state changed to DOWN ums0: on usbus1 bge0: link state changed to UP bge1: link state changed to UP bge2: link state changed to UP ugen1.2: at usbus1 (disconnected) ukbd0: at uhub1, port 1, addr 2 (disconnected) ums0: at uhub1, port 1, addr 2 (disconnected) vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1ESLL_C1S03. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1ESLL_C1S03. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1ESLL_C1S03. vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1F8V0_C1S04. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1F8V0_C1S04. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F8V0_C1S04. vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1EQPR_C1S05. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1EQPR_C1S05. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1EQPR_C1S05. vdev_geom_close_locked:336[1]: Closing access to gpt/S0M19J9D_C1S06. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M19J9D_C1S06. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M19J9D_C1S06. vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1ES7R_C1S07. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1ES7R_C1S07. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1ES7R_C1S07. vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1DXJR_C1S08. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1DXJR_C1S08. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1DXJR_C1S08. vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1EQHL_C2S01. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1EQHL_C2S01. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1EQHL_C2S01. vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1EQSL_C2S02. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1EQSL_C2S02. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1EQSL_C2S02. vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1F7CG_C2S03. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1F7CG_C2S03. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F7CG_C2S03. vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1F2B1_C2S04. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1F2B1_C2S04. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F2B1_C2S04. vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1ER7Y_C2S05. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1ER7Y_C2S05. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1ER7Y_C2S05. vdev_geom_close_locked:336[1]: Closing access to gpt/S0M1F9B0_C2S06. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1F9B0_C2S06. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F9B0_C2S06. vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29R3L_EC1_S01. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L29R3L_EC1_S01. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29R3L_EC1_S01. vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29XFQ_EC1_S02. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L29XFQ_EC1_S02. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29XFQ_EC1_S02. vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29QTK_EC1_S03. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L29QTK_EC1_S03. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29QTK_EC1_S03. vdev_geom_close_locked:336[1]: Closing access to gpt/S3L28ZFA_EC1_S04. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L28ZFA_EC1_S04. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L28ZFA_EC1_S04. vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29PG9_EC1_S05. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L29PG9_EC1_S05. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29PG9_EC1_S05. vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29TAA_EC1_S06. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L29TAA_EC1_S06. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29TAA_EC1_S06. vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29RHR_EC1_S07. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L29RHR_EC1_S07. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29RHR_EC1_S07. vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29VQT_EC1_S08. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L29VQT_EC1_S08. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29VQT_EC1_S08. vdev_geom_close_locked:336[1]: Closing access to gpt/S3L2A7WM_EC1_S09. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L2A7WM_EC1_S09. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L2A7WM_EC1_S09. vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29GXN_EC1_S10. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L29GXN_EC1_S10. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29GXN_EC1_S10. vdev_geom_close_locked:336[1]: Closing access to gpt/S3L29TPT_EC1_S11. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L29TPT_EC1_S11. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29TPT_EC1_S11. vdev_geom_close_locked:336[1]: Closing access to gpt/S3L2A4EJ_EC1_S12. vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L2A4EJ_EC1_S12. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L2A4EJ_EC1_S12. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S0M1ESLL_C1S03. vdev_geom_attach:192[1]: Attaching to gpt/S0M1ESLL_C1S03. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1ESLL_C1S03. vdev_geom_read_config:429[1]: Reading config from gpt/S0M1ESLL_C1S03... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1ESLL_C1S03. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1ESLL_C1S03. vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1ESLL_C1S03. vdev_geom_attach:192[1]: Attaching to gpt/S0M1ESLL_C1S03. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1ESLL_C1S03. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S0M1F8V0_C1S04. vdev_geom_attach:192[1]: Attaching to gpt/S0M1F8V0_C1S04. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F8V0_C1S04. vdev_geom_read_config:429[1]: Reading config from gpt/S0M1F8V0_C1S04... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1F8V0_C1S04. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F8V0_C1S04. vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1F8V0_C1S04. vdev_geom_attach:192[1]: Attaching to gpt/S0M1F8V0_C1S04. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F8V0_C1S04. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S0M1EQPR_C1S05. vdev_geom_attach:192[1]: Attaching to gpt/S0M1EQPR_C1S05. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1EQPR_C1S05. vdev_geom_read_config:429[1]: Reading config from gpt/S0M1EQPR_C1S05... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1EQPR_C1S05. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1EQPR_C1S05. vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1EQPR_C1S05. vdev_geom_attach:192[1]: Attaching to gpt/S0M1EQPR_C1S05. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1EQPR_C1S05. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S0M19J9D_C1S06. vdev_geom_attach:192[1]: Attaching to gpt/S0M19J9D_C1S06. vdev_geom_attach:256[1]: Created consumer for gpt/S0M19J9D_C1S06. vdev_geom_read_config:429[1]: Reading config from gpt/S0M19J9D_C1S06... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M19J9D_C1S06. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M19J9D_C1S06. vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M19J9D_C1S06. vdev_geom_attach:192[1]: Attaching to gpt/S0M19J9D_C1S06. vdev_geom_attach:256[1]: Created consumer for gpt/S0M19J9D_C1S06. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S0M1ES7R_C1S07. vdev_geom_attach:192[1]: Attaching to gpt/S0M1ES7R_C1S07. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1ES7R_C1S07. vdev_geom_read_config:429[1]: Reading config from gpt/S0M1ES7R_C1S07... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1ES7R_C1S07. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1ES7R_C1S07. vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1ES7R_C1S07. vdev_geom_attach:192[1]: Attaching to gpt/S0M1ES7R_C1S07. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1ES7R_C1S07. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S0M1DXJR_C1S08. vdev_geom_attach:192[1]: Attaching to gpt/S0M1DXJR_C1S08. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1DXJR_C1S08. vdev_geom_read_config:429[1]: Reading config from gpt/S0M1DXJR_C1S08... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1DXJR_C1S08. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1DXJR_C1S08. vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1DXJR_C1S08. vdev_geom_attach:192[1]: Attaching to gpt/S0M1DXJR_C1S08. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1DXJR_C1S08. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S0M1EQHL_C2S01. vdev_geom_attach:192[1]: Attaching to gpt/S0M1EQHL_C2S01. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1EQHL_C2S01. vdev_geom_read_config:429[1]: Reading config from gpt/S0M1EQHL_C2S01... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1EQHL_C2S01. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1EQHL_C2S01. vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1EQHL_C2S01. vdev_geom_attach:192[1]: Attaching to gpt/S0M1EQHL_C2S01. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1EQHL_C2S01. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S0M1EQSL_C2S02. vdev_geom_attach:192[1]: Attaching to gpt/S0M1EQSL_C2S02. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1EQSL_C2S02. vdev_geom_read_config:429[1]: Reading config from gpt/S0M1EQSL_C2S02... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1EQSL_C2S02. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1EQSL_C2S02. vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1EQSL_C2S02. vdev_geom_attach:192[1]: Attaching to gpt/S0M1EQSL_C2S02. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1EQSL_C2S02. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S0M1F7CG_C2S03. vdev_geom_attach:192[1]: Attaching to gpt/S0M1F7CG_C2S03. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F7CG_C2S03. vdev_geom_read_config:429[1]: Reading config from gpt/S0M1F7CG_C2S03... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1F7CG_C2S03. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F7CG_C2S03. vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1F7CG_C2S03. vdev_geom_attach:192[1]: Attaching to gpt/S0M1F7CG_C2S03. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F7CG_C2S03. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S0M1F2B1_C2S04. vdev_geom_attach:192[1]: Attaching to gpt/S0M1F2B1_C2S04. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F2B1_C2S04. vdev_geom_read_config:429[1]: Reading config from gpt/S0M1F2B1_C2S04... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1F2B1_C2S04. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F2B1_C2S04. vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1F2B1_C2S04. vdev_geom_attach:192[1]: Attaching to gpt/S0M1F2B1_C2S04. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F2B1_C2S04. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S0M1ER7Y_C2S05. vdev_geom_attach:192[1]: Attaching to gpt/S0M1ER7Y_C2S05. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1ER7Y_C2S05. vdev_geom_read_config:429[1]: Reading config from gpt/S0M1ER7Y_C2S05... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1ER7Y_C2S05. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1ER7Y_C2S05. vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1ER7Y_C2S05. vdev_geom_attach:192[1]: Attaching to gpt/S0M1ER7Y_C2S05. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1ER7Y_C2S05. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S0M1F9B0_C2S06. vdev_geom_attach:192[1]: Attaching to gpt/S0M1F9B0_C2S06. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F9B0_C2S06. vdev_geom_read_config:429[1]: Reading config from gpt/S0M1F9B0_C2S06... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S0M1F9B0_C2S06. vdev_geom_detach:311[1]: Destroying consumer to gpt/S0M1F9B0_C2S06. vdev_attach_ok:651[1]: guids match for provider /dev/gpt/S0M1F9B0_C2S06. vdev_geom_attach:192[1]: Attaching to gpt/S0M1F9B0_C2S06. vdev_geom_attach:256[1]: Created consumer for gpt/S0M1F9B0_C2S06. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S3L29R3L_EC1_S01. vdev_geom_attach:192[1]: Attaching to gpt/S3L29R3L_EC1_S01. vdev_geom_attach:256[1]: Created consumer for gpt/S3L29R3L_EC1_S01. vdev_geom_read_config:429[1]: Reading config from = gpt/S3L29R3L_EC1_S01... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L29R3L_EC1_S01. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29R3L_EC1_S01. vdev_attach_ok:651[1]: guids match for provider = /dev/gpt/S3L29R3L_EC1_S01. vdev_geom_attach:192[1]: Attaching to gpt/S3L29R3L_EC1_S01. vdev_geom_attach:256[1]: Created consumer for gpt/S3L29R3L_EC1_S01. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S3L29XFQ_EC1_S02. vdev_geom_attach:192[1]: Attaching to gpt/S3L29XFQ_EC1_S02. vdev_geom_attach:256[1]: Created consumer for gpt/S3L29XFQ_EC1_S02. vdev_geom_read_config:429[1]: Reading config from = gpt/S3L29XFQ_EC1_S02... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L29XFQ_EC1_S02. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29XFQ_EC1_S02. vdev_attach_ok:651[1]: guids match for provider = /dev/gpt/S3L29XFQ_EC1_S02. vdev_geom_attach:192[1]: Attaching to gpt/S3L29XFQ_EC1_S02. vdev_geom_attach:256[1]: Created consumer for gpt/S3L29XFQ_EC1_S02. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S3L29QTK_EC1_S03. vdev_geom_attach:192[1]: Attaching to gpt/S3L29QTK_EC1_S03. vdev_geom_attach:256[1]: Created consumer for gpt/S3L29QTK_EC1_S03. vdev_geom_read_config:429[1]: Reading config from = gpt/S3L29QTK_EC1_S03... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L29QTK_EC1_S03. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29QTK_EC1_S03. vdev_attach_ok:651[1]: guids match for provider = /dev/gpt/S3L29QTK_EC1_S03. vdev_geom_attach:192[1]: Attaching to gpt/S3L29QTK_EC1_S03. vdev_geom_attach:256[1]: Created consumer for gpt/S3L29QTK_EC1_S03. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S3L28ZFA_EC1_S04. vdev_geom_attach:192[1]: Attaching to gpt/S3L28ZFA_EC1_S04. vdev_geom_attach:256[1]: Created consumer for gpt/S3L28ZFA_EC1_S04. vdev_geom_read_config:429[1]: Reading config from = gpt/S3L28ZFA_EC1_S04... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L28ZFA_EC1_S04. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L28ZFA_EC1_S04. vdev_attach_ok:651[1]: guids match for provider = /dev/gpt/S3L28ZFA_EC1_S04. vdev_geom_attach:192[1]: Attaching to gpt/S3L28ZFA_EC1_S04. vdev_geom_attach:256[1]: Created consumer for gpt/S3L28ZFA_EC1_S04. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S3L29PG9_EC1_S05. vdev_geom_attach:192[1]: Attaching to gpt/S3L29PG9_EC1_S05. vdev_geom_attach:256[1]: Created consumer for gpt/S3L29PG9_EC1_S05. vdev_geom_read_config:429[1]: Reading config from = gpt/S3L29PG9_EC1_S05... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L29PG9_EC1_S05. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29PG9_EC1_S05. vdev_attach_ok:651[1]: guids match for provider = /dev/gpt/S3L29PG9_EC1_S05. vdev_geom_attach:192[1]: Attaching to gpt/S3L29PG9_EC1_S05. vdev_geom_attach:256[1]: Created consumer for gpt/S3L29PG9_EC1_S05. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S3L29TAA_EC1_S06. vdev_geom_attach:192[1]: Attaching to gpt/S3L29TAA_EC1_S06. vdev_geom_attach:256[1]: Created consumer for gpt/S3L29TAA_EC1_S06. vdev_geom_read_config:429[1]: Reading config from = gpt/S3L29TAA_EC1_S06... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L29TAA_EC1_S06. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29TAA_EC1_S06. vdev_attach_ok:651[1]: guids match for provider = /dev/gpt/S3L29TAA_EC1_S06. vdev_geom_attach:192[1]: Attaching to gpt/S3L29TAA_EC1_S06. vdev_geom_attach:256[1]: Created consumer for gpt/S3L29TAA_EC1_S06. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S3L29RHR_EC1_S07. vdev_geom_attach:192[1]: Attaching to gpt/S3L29RHR_EC1_S07. vdev_geom_attach:256[1]: Created consumer for gpt/S3L29RHR_EC1_S07. vdev_geom_read_config:429[1]: Reading config from = gpt/S3L29RHR_EC1_S07... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L29RHR_EC1_S07. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29RHR_EC1_S07. vdev_attach_ok:651[1]: guids match for provider = /dev/gpt/S3L29RHR_EC1_S07. vdev_geom_attach:192[1]: Attaching to gpt/S3L29RHR_EC1_S07. vdev_geom_attach:256[1]: Created consumer for gpt/S3L29RHR_EC1_S07. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S3L29VQT_EC1_S08. vdev_geom_attach:192[1]: Attaching to gpt/S3L29VQT_EC1_S08. vdev_geom_attach:256[1]: Created consumer for gpt/S3L29VQT_EC1_S08. vdev_geom_read_config:429[1]: Reading config from = gpt/S3L29VQT_EC1_S08... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L29VQT_EC1_S08. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29VQT_EC1_S08. vdev_attach_ok:651[1]: guids match for provider = /dev/gpt/S3L29VQT_EC1_S08. vdev_geom_attach:192[1]: Attaching to gpt/S3L29VQT_EC1_S08. vdev_geom_attach:256[1]: Created consumer for gpt/S3L29VQT_EC1_S08. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S3L2A7WM_EC1_S09. vdev_geom_attach:192[1]: Attaching to gpt/S3L2A7WM_EC1_S09. vdev_geom_attach:256[1]: Created consumer for gpt/S3L2A7WM_EC1_S09. vdev_geom_read_config:429[1]: Reading config from = gpt/S3L2A7WM_EC1_S09... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L2A7WM_EC1_S09. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L2A7WM_EC1_S09. vdev_attach_ok:651[1]: guids match for provider = /dev/gpt/S3L2A7WM_EC1_S09. vdev_geom_attach:192[1]: Attaching to gpt/S3L2A7WM_EC1_S09. vdev_geom_attach:256[1]: Created consumer for gpt/S3L2A7WM_EC1_S09. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S3L29GXN_EC1_S10. vdev_geom_attach:192[1]: Attaching to gpt/S3L29GXN_EC1_S10. vdev_geom_attach:256[1]: Created consumer for gpt/S3L29GXN_EC1_S10. vdev_geom_read_config:429[1]: Reading config from = gpt/S3L29GXN_EC1_S10... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L29GXN_EC1_S10. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29GXN_EC1_S10. vdev_attach_ok:651[1]: guids match for provider = /dev/gpt/S3L29GXN_EC1_S10. vdev_geom_attach:192[1]: Attaching to gpt/S3L29GXN_EC1_S10. vdev_geom_attach:256[1]: Created consumer for gpt/S3L29GXN_EC1_S10. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S3L29TPT_EC1_S11. vdev_geom_attach:192[1]: Attaching to gpt/S3L29TPT_EC1_S11. vdev_geom_attach:256[1]: Created consumer for gpt/S3L29TPT_EC1_S11. vdev_geom_read_config:429[1]: Reading config from = gpt/S3L29TPT_EC1_S11... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L29TPT_EC1_S11. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L29TPT_EC1_S11. vdev_attach_ok:651[1]: guids match for provider = /dev/gpt/S3L29TPT_EC1_S11. vdev_geom_attach:192[1]: Attaching to gpt/S3L29TPT_EC1_S11. vdev_geom_attach:256[1]: Created consumer for gpt/S3L29TPT_EC1_S11. vdev_geom_open_by_path:744[1]: Found provider by name = /dev/gpt/S3L2A4EJ_EC1_S12. vdev_geom_attach:192[1]: Attaching to gpt/S3L2A4EJ_EC1_S12. vdev_geom_attach:256[1]: Created consumer for gpt/S3L2A4EJ_EC1_S12. vdev_geom_read_config:429[1]: Reading config from = gpt/S3L2A4EJ_EC1_S12... vdev_geom_detach:297[1]: Detaching consumer. Provider = gpt/S3L2A4EJ_EC1_S12. vdev_geom_detach:311[1]: Destroying consumer to gpt/S3L2A4EJ_EC1_S12. vdev_attach_ok:651[1]: guids match for provider = /dev/gpt/S3L2A4EJ_EC1_S12. vdev_geom_attach:192[1]: Attaching to gpt/S3L2A4EJ_EC1_S12. vdev_geom_attach:256[1]: Created consumer for gpt/S3L2A4EJ_EC1_S12. console=3D"comconsole,vidconsole" comconsole_speed=3D"115200" geom_mirror_load=3D"YES" kern.geom.label.gptid.enable=3D"0" kern.geom.label.disk_ident.enable=3D"0" zfs_load=3D"YES" kern.cam.scsi_delay=3D15000 # $FreeBSD: releng/11.0/etc/sysctl.conf 112200 2003-03-13 18:43:50Z mux = $ # # This file is read when going to multi-user and its contents piped = thru # ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for = details. # # Uncomment this to prevent users from seeing information about = processes that # are being run under another UID. #security.bsd.see_other_uids=3D0 kern.ipc.shm_use_phys=3D1 kern.ipc.somaxconn=3D16384 # seems to be higher anyway #kern.ipc.maxsockets=3D131072 #kern.maxfiles=3D131072 #kern.maxfilesperproc=3D104856 #kern.threads.max_threads_per_proc=3D4096 net.inet.tcp.fast_finwait2_recycle=3D1 net.inet.tcp.finwait2_timeout=3D15000 net.inet.tcp.msl=3D5000 machdep.panic_on_nmi=3D0 net.inet6.ip6.auto_flowlabel=3D0 security.bsd.see_other_gids=3D0 security.bsd.see_other_uids=3D0 # was 49152 net.inet.ip.portrange.hifirst=3D10000 security.bsd.unprivileged_proc_debug=3D0 net.inet.ip.redirect=3D0 net.inet6.ip6.redirect=3D0 net.inet.icmp.drop_redirect=3D1 net.inet6.icmp6.rediraccept=3D0 security.bsd.hardlink_check_uid=3D1 security.bsd.hardlink_check_gid=3D1 kern.coredump=3D0 kern.nodump_coredump=3D1 net.inet.ip.random_id=3D1 net.inet.ip.check_interface=3D1 net.inet.tcp.blackhole=3D1 net.inet.udp.blackhole=3D1 security.bsd.unprivileged_read_msgbuf=3D0 vfs.zfs.min_auto_ashift=3D12 vfs.zfs.arc_max=3D68719476736 I=E2=80=99ve set vfs.zfs.debug to 1. Does anybody know what the attach/detach messages from GEOM mean? From owner-freebsd-stable@freebsd.org Sat Oct 1 22:24:18 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 20B0CA9421F for ; Sat, 1 Oct 2016 22:24:18 +0000 (UTC) (envelope-from vangyzen@FreeBSD.org) Received: from smtp.vangyzen.net (hotblack.vangyzen.net [IPv6:2607:fc50:1000:7400:216:3eff:fe72:314f]) by mx1.freebsd.org (Postfix) with ESMTP id E748AC16 for ; Sat, 1 Oct 2016 22:24:17 +0000 (UTC) (envelope-from vangyzen@FreeBSD.org) Received: from ford.home.vangyzen.net (unknown [76.164.15.242]) by smtp.vangyzen.net (Postfix) with ESMTPSA id C6BBD56488; Sat, 1 Oct 2016 17:24:16 -0500 (CDT) Subject: Re: Can't build 11-stable without BPF To: dave@mischler.com References: <1475270395.24760.4.camel@mischler.com> Cc: freebsd-stable@freebsd.org From: Eric van Gyzen Message-ID: <8dafc1ed-a8a3-697c-7149-3cef9754164c@FreeBSD.org> Date: Sat, 1 Oct 2016 17:24:13 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <1475270395.24760.4.camel@mischler.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Oct 2016 22:24:18 -0000 On 09/30/2016 16:19, Dave Mischler wrote: > When I remove the "device bpf" from the kernel configuration the > resulting "make buildkernel KERNCONF=XXX" fails to compile in module > lmc, source file if_lmc.c > > The problem seems to be that DEV_BPF is not defined, but this is not > tested for. Thanks for the report. I just fixed it in head. I'll merge it to stable/11 in a few days. https://svnweb.freebsd.org/changeset/base/306567 Cheers, Eric