From owner-svn-src-head@freebsd.org Fri Jul 6 11:29:25 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0CC0C1040567; Fri, 6 Jul 2018 11:29:25 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AFCBD7B051; Fri, 6 Jul 2018 11:29:24 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from mail-lj1-f178.google.com (mail-lj1-f178.google.com [209.85.208.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) (Authenticated sender: kevans) by smtp.freebsd.org (Postfix) with ESMTPSA id 4DCFFEDE7; Fri, 6 Jul 2018 11:29:24 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: by mail-lj1-f178.google.com with SMTP id p6-v6so8916452ljc.5; Fri, 06 Jul 2018 04:29:24 -0700 (PDT) X-Gm-Message-State: APt69E1X1oZ18v2HrIaFPmTG7O1LobbC9Waha9BSszfsp+yRNWKHbG09 Wb1A4oZyLj/vSi3s2nRHUjTKJodcoQUS9Z+cgNU= X-Google-Smtp-Source: AAOMgpd4qDEVXVXRiEFIXzbwniJz6Xq1jWPtK39taTRJ8z1RVCFOpcOQkm4uaz53XqTOjfKwGobPX2obPH8JY5G4UUU= X-Received: by 2002:a2e:80d3:: with SMTP id r19-v6mr4155965ljg.85.1530876562667; Fri, 06 Jul 2018 04:29:22 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a2e:5742:0:0:0:0:0 with HTTP; Fri, 6 Jul 2018 04:29:02 -0700 (PDT) In-Reply-To: References: <201807060111.w661B76W005640@repo.freebsd.org> From: Kyle Evans Date: Fri, 6 Jul 2018 06:29:02 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: svn commit: r336019 - in head: . usr.sbin/config To: Bryan Drewery Cc: src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org, Li-Wen Hsu Content-Type: text/plain; charset="UTF-8" X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2018 11:29:25 -0000 On Fri, Jul 6, 2018 at 1:08 AM, Bryan Drewery wrote: > On 7/5/18 6:53 PM, Kyle Evans wrote: >> On Thu, Jul 5, 2018 at 8:11 PM, Kyle Evans wrote: >>> Author: kevans >>> Date: Fri Jul 6 01:11:06 2018 >>> New Revision: 336019 >>> URL: https://svnweb.freebsd.org/changeset/base/336019 >>> >>> Log: >>> config(8): De-dupe hint/env vars within a single file >>> >>> r335653 flipped the order in which hints/env files are concatenated to match >>> the order in which vars are processed by the kernel. This is the other >>> hammer to drop. >>> >>> Use nv(9) to de-dupe entries within a single `hint` or `env` file, using the >>> latest value specified for a key. This leaves some duplicates if a variable >>> is specified in multiple hint/env files or via `envvar` in a kernel config, >>> but the reversed order of concatenation (from r335653) makes this a >>> non-issue as the latest-specified version will be seen first. >>> >>> This change also silently rewrote hint bits to use the same sanitization >>> process that ian@ wrote for r335642. To the kernel, hints and env vars are >>> basically the same thing through early boot, then get merged into the >>> dynamic environment once kmem becomes available and the dynamic environment >>> is created. They should be subjected to the same restrictions. >>> >>> libnv has been added to -legacy for the time being to support the build of >>> config(8) with the new cnvlist API. >>> >>> Tested with: universe (11 host & 12 host) >>> MFC after: 1 month >>> >> >> This seems to be causing pretty consistent config(8) failures on CI, >> kernels with neither hints nor environment variables, that neither of >> my universe builds nor any of my post-commit builds can reproduce. Can >> anyone else reproduce these failures that might be willing to give me >> a hint as to what kind of setup causes this? >> >> 01:47:58 cd /usr/src/sys/arm/conf; >> PATH=/usr/obj/usr/src/arm.armv7/tmp/legacy/usr/sbin:/usr/obj/usr/src/arm.armv7/tmp/legacy/usr/bin:/usr/obj/usr/src/arm.armv7/tmp/legacy/bin:/usr/obj/usr/src/arm.armv7/tmp/usr/sbin:/usr/obj/usr/src/arm.armv7/tmp/usr/bin:/sbin:/bin:/usr/sbin:/usr/bin >> config -d /usr/obj/usr/src/arm.armv7/sys/GENERIC -I >> '/usr/src/sys/arm/conf' '/usr/src/sys/arm/conf/GENERIC' >> 01:47:58 config: /usr/src/sys/arm/conf/GENERIC: No error: 0 >> > > I've seen this error with broken ABI. The -I${SRCTOP}/sys on the config > build may be contributing to the problem since sys/stat.h is ino64 and > the builder may not be. Really should not be bringing in the source > sys/ directory for any of the early host tool phases. There's some hack > patterns to bring in a limited amount of headers but in this case the > headers should be getting staged somewhere already. > Note legacy: > ${MAKE} DIRPRFX=${_tool}/ DESTDIR=${WORLDTMP}/legacy includes; \ > > So -IWORLDTMP/legacy/sys should find the nv headers. This is already > included via: > tools/build/mk/Makefile.boot:CFLAGS+= -I${WORLDTMP}/legacy/usr/include > Which is brought in for bootstrap-tools though via: > MAKEFLAGS="-m ${.CURDIR}/tools/build/mk ${.MAKEFLAGS}" > Ahh, that explains it. I did a little bit of a dirty hack in r336026 to install the needed libnv headers with libnv- otherwise, we'd still be relying on the host libnv headers since sys/ bits don't normally get pulled in with `make includes` (from the look of it). It feels wrong to have a userland lib installing headers into ${INCLUDEDIR}/sys, but I think that's a libnv problem. This should fix the observed problem with CI for the time being.