From owner-freebsd-reproducibility@freebsd.org Tue Mar 31 14:08:27 2020 Return-Path: Delivered-To: freebsd-reproducibility@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F0BF62A71E5 for ; Tue, 31 Mar 2020 14:08:27 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-io1-f49.google.com (mail-io1-f49.google.com [209.85.166.49]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48sB4g6g1Fz4Qwb for ; Tue, 31 Mar 2020 14:08:19 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-io1-f49.google.com with SMTP id k9so21861465iov.7 for ; Tue, 31 Mar 2020 07:08:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to :content-transfer-encoding; bh=8QxeER9IG+bxmklTlUA2xRRNGpkstMLkEQk6GnxEDTk=; b=YRyNwu8ntlXLpR9LkClhG6Y/mZ+kxWbfkNaH2Y5f7heTnY29cc1ITwslPuUxrDa+xO u691EpfdwhgFTfjAHzjOMYgpkr35E3PXFP47KXzUikKQV0PADU+cJiISdf80KjOq1E2X GPpUp1uh4yfDpIRpUqAsIy1RKzBV3h2gXghsQVQ5vl8qL/+wc8Cwk4Te6UcPYcV3o9Nf rWef6UNrnD/gQPXtSdNUFXu9D77pv+/p1CqGw91tojKoVU6Rmvc4LZVauJ1OL0CJ8adM jVJZxE/bNoiMntSzzTwW50BK1qHH3qTlWOns5jRdhGfOqIZLbOviCdIk+5QAuWsaOhO+ MNDA== X-Gm-Message-State: ANhLgQ0JL6VRexfz6gssa9uIdKfKMIu9FQjPa2w/Sv9UrlYUj1BVhEuo no8kFj1C+EuXiScgnWyvmTvp1IgdzgfdSW0ycPf3f8+v X-Google-Smtp-Source: ADFU+vs+qut5C62+4FQmk+z2U+hk9FvVQV3XRgtHyoNP32m/uWDVZj9VdvL8lXF3LAIFcCBE/T4jvZGd/fCU2H211ug= X-Received: by 2002:a6b:fd08:: with SMTP id c8mr16043195ioi.102.1585663689496; Tue, 31 Mar 2020 07:08:09 -0700 (PDT) MIME-Version: 1.0 From: Ed Maste Date: Tue, 31 Mar 2020 10:07:56 -0400 Message-ID: Subject: Nonreproducible kernel modules on stable/12 To: freebsd-reproducibility@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 48sB4g6g1Fz4Qwb X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.49 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com X-Spamd-Result: default: False [-3.45 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-reproducibility@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_TRACE(0.00)[0:+]; RCVD_IN_DNSWL_NONE(0.00)[49.166.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-1.45)[ip: (-6.09), ipnet: 209.85.128.0/17(-0.65), asn: 15169(-0.45), country: US(-0.05)]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; DMARC_NA(0.00)[freebsd.org]; RWL_MAILSPIKE_POSSIBLE(0.00)[49.166.85.209.rep.mailspike.net : 127.0.0.17]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; RCVD_TLS_ALL(0.00)[]; TO_DOM_EQ_FROM_DOM(0.00)[] X-BeenThere: freebsd-reproducibility@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions regarding FreeBSD reproducible builds List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2020 14:08:28 -0000 Originally reported off-list, during a stable/12 freebsd-update build kernel modules are not reproducible and thus get included in the update. diffoscope reports that the .SUNW_ctf section is responsible: --- cam.0.ko +++ cam.1.ko =E2=94=9C=E2=94=80=E2=94=80 readelf --wide --sections {} ... =E2=94=82 [30] .shstrtab STRTAB 0000000000000000 0ea828 0001f8 00 0 0 1 =E2=94=82 - [31] .SUNW_ctf PROGBITS 0000000000000000 0f29d8 00e2ad 00 32 0 4 =E2=94=82 + [31] .SUNW_ctf PROGBITS 0000000000000000 0f29d8 00e2b1 00 32 0 4 =E2=94=82 [32] .symtab SYMTAB 0000000000000000 0e12b8 009570 18 33 1001 8 ... =E2=94=9C=E2=94=80=E2=94=80 readelf --wide --decompress --hex-dump=3D.SUNW_= ctf {} =E2=94=82 @@ -1,3630 +1,3631 @@ =E2=94=82 =E2=94=82 Hex dump of section '.SUNW_ctf': =E2=94=82 0x00000000 f1cf0201 00000000 00000000 00000000 ...............= . =E2=94=82 0x00000010 08000000 96040000 fc250000 f4310100 .........%...1.= . =E2=94=82 - 0x00000020 6aeb0000 78dabc7d 077c54c5 16f7ccbd j...x..}.|T....= . =E2=94=82 - 0x00000030 774b42ca 26b48524 b0402001 02440d10 wKB.&..$.@ ..D.= . ... =E2=94=82 + 0x00000020 6feb0000 78dabc7d 077c54c5 16f7ccdc o...x..}.|T....= . =E2=94=82 + 0x00000030 bb252165 13da4242 586a0204 081820d4 .%!e..BBXj.... = . (And build-ID and GNU debuglink differ as a result.) Running ctfdump and comparing the result shows the issue: --- a 2020-03-31 10:04:39.766567000 -0400 +++ b 2020-03-31 10:04:44.663292000 -0400 @@ -11,11 +11,11 @@ cth_funcoff =3D 1174 cth_typeoff =3D 9724 cth_stroff =3D 78324 - cth_strlen =3D 60266 + cth_strlen =3D 60271 - Label Table ------------------------------------------------------------= ---- - 2407 FreeBSD 12.1-STABLE amd64 1201512 + 2407 FreeBSD 12.1-FUTUREBUILD amd64 1201512 - Data Objects -----------------------------------------------------------= ---- Aside, we should add ctfdump to diffoscope. From owner-freebsd-reproducibility@freebsd.org Tue Mar 31 14:23:48 2020 Return-Path: Delivered-To: freebsd-reproducibility@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CC69E2A7739 for ; Tue, 31 Mar 2020 14:23:48 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-x231.google.com (mail-oi1-x231.google.com [IPv6:2607:f8b0:4864:20::231]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48sBQV6zyZz4Wrl; Tue, 31 Mar 2020 14:23:46 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi1-x231.google.com with SMTP id m14so19096531oic.0; Tue, 31 Mar 2020 07:23:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=yBK4gsldWJlI0vU1AMaVbYjq1U+RiLPhcOLcTGZxgB8=; b=W1vH+Yq/4h+Rg2juX5rFivW8Q21k2qO8OfyXBW6vzpy059la1JDBVQ4iB5Tx7dcJH1 jOKvh8jLi+/nH+gXd6aldadRsiHuejxmQwzbQMkTWZFmX+WCNhDKGPkyZNbr1WoGHScU 65iZzo1DZOLQ478CE3EFdE8NLyG14Gw0ZRocrj4jNT6pfa6ctUyky7VEBgro1qznOgWE huZcxuVDbZuYVX6Qq/lhWnWk715xuNBSFikF9PVYdZrU0NHKjTcA+z4EUzfYZJOu10Je gL818JzbtksKCc2ZELMwHJ2dqKV7aGZiGVpl9A/upUhD35PChDRlZMnu0lCCWWZMNnCj NdUg== X-Gm-Message-State: ANhLgQ0h8JNMrZU+UmxmODBgjeu01i8dVnDQ7I+s2M8djRLIQWgjPcw0 jeo7zmDeBc3nfRIJQaLWaTHwXhnftZ4bOHTWQMyR+M9x X-Google-Smtp-Source: ADFU+vtvXIdq4KKhRt8PZVyECIq+LuvUVXyGwoegWD9eGNbW/xT6NtTr0mXtBoZvc36q4+7CczaJgkQ7YKIC6Who0nA= X-Received: by 2002:aca:b955:: with SMTP id j82mr2384109oif.57.1585664613690; Tue, 31 Mar 2020 07:23:33 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Tue, 31 Mar 2020 08:23:22 -0600 Message-ID: Subject: Re: Nonreproducible kernel modules on stable/12 To: Ed Maste Cc: freebsd-reproducibility@freebsd.org X-Rspamd-Queue-Id: 48sBQV6zyZz4Wrl X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-reproducibility@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions regarding FreeBSD reproducible builds List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2020 14:23:48 -0000 On Tue, Mar 31, 2020 at 8:08 AM Ed Maste wrote: > Originally reported off-list, during a stable/12 freebsd-update build > kernel modules are not reproducible and thus get included in the > update. > > diffoscope reports that the .SUNW_ctf section is responsible: > > --- cam.0.ko > +++ cam.1.ko > =E2=94=9C=E2=94=80=E2=94=80 readelf --wide --sections {} > ... > =E2=94=82 [30] .shstrtab STRTAB 0000000000000000 0ea8= 28 > 0001f8 00 0 0 1 > =E2=94=82 - [31] .SUNW_ctf PROGBITS 0000000000000000 0f29= d8 > 00e2ad 00 32 0 4 > =E2=94=82 + [31] .SUNW_ctf PROGBITS 0000000000000000 0f29= d8 > 00e2b1 00 32 0 4 > =E2=94=82 [32] .symtab SYMTAB 0000000000000000 0e12= b8 > 009570 18 33 1001 8 > ... > =E2=94=9C=E2=94=80=E2=94=80 readelf --wide --decompress --hex-dump=3D.SUN= W_ctf {} > =E2=94=82 @@ -1,3630 +1,3631 @@ > =E2=94=82 > =E2=94=82 Hex dump of section '.SUNW_ctf': > =E2=94=82 0x00000000 f1cf0201 00000000 00000000 00000000 .............= ... > =E2=94=82 0x00000010 08000000 96040000 fc250000 f4310100 .........%...= 1.. > =E2=94=82 - 0x00000020 6aeb0000 78dabc7d 077c54c5 16f7ccbd j...x..}.|T..= ... > =E2=94=82 - 0x00000030 774b42ca 26b48524 b0402001 02440d10 wKB.&..$.@ ..= D.. > ... > =E2=94=82 + 0x00000020 6feb0000 78dabc7d 077c54c5 16f7ccdc o...x..}.|T..= ... > =E2=94=82 + 0x00000030 bb252165 13da4242 586a0204 081820d4 .%!e..BBXj...= . . > > (And build-ID and GNU debuglink differ as a result.) > > Running ctfdump and comparing the result shows the issue: > > --- a 2020-03-31 10:04:39.766567000 -0400 > +++ b 2020-03-31 10:04:44.663292000 -0400 > @@ -11,11 +11,11 @@ > cth_funcoff =3D 1174 > cth_typeoff =3D 9724 > cth_stroff =3D 78324 > - cth_strlen =3D 60266 > + cth_strlen =3D 60271 > > - Label Table > ---------------------------------------------------------------- > > - 2407 FreeBSD 12.1-STABLE amd64 1201512 > + 2407 FreeBSD 12.1-FUTUREBUILD amd64 1201512 > > - Data Objects > --------------------------------------------------------------- > > Aside, we should add ctfdump to diffoscope. > Good sleuthing Ed! That "FUTUREBUILD" string points to the problem. Warner's commit "caused" the problem only in the sense that BRANCH_OVERRIDE didn't actually work beforehand. So freebsd-update-build was setting BRANCH_OVERRIDE to no effect. Now that Warner has fixed BRANCH_OVERRIDE, we'll have to fix freebsd-update-build, which may be as simple as not setting BRANCH_OVERRIDE during that phase of the build. I'll test that now, but it will take a few hours. -Alan From owner-freebsd-reproducibility@freebsd.org Tue Mar 31 21:56:06 2020 Return-Path: Delivered-To: freebsd-reproducibility@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A764226DA6A for ; Tue, 31 Mar 2020 21:56:06 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x730.google.com (mail-qk1-x730.google.com [IPv6:2607:f8b0:4864:20::730]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48sNSB4q7wz4VTm for ; Tue, 31 Mar 2020 21:55:54 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x730.google.com with SMTP id 139so14999776qkd.9 for ; Tue, 31 Mar 2020 14:55:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=QY023jzOSUTfHskZkHm+ZmWe5AQv18ZOHDDE8BuSg84=; b=uoe/DosFiqRR2nqnKnIA3riVX2FrNHEcLyrCHXPunMWqTlw60S8sAfbjYkuQta9OFa X3t/rz8Euas4IzoQ0g4R5XJXX08VksHGkz5lke4S1iu5bjLpzp6Vo/dvPe0hSUGt/7N/ 5ijTN0IUL4kQcLPk6wvvRCSmw585lvDfrDOuO6Xx8kz8gJKDv7tSWudCbjeD5QKuZRkG TQnTUfEERe+tCMtfU75Z2nPmhf7sG68UYbSPETiR1AyobmwyMA1Gqn6wSWlRBOZq+Y2r 3lFOnzwbBBfi9H2c5pHoApl9Am5iXnsIA5NpdLtIz0N5RHcwUuCpRNKIQbsfn5iqydIo XeMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=QY023jzOSUTfHskZkHm+ZmWe5AQv18ZOHDDE8BuSg84=; b=afCagkO2vi0CpKtm2ZzrWh3BecYXtRBBqXtg1Wt0cXDryj0QzkeC9IJyE8obEEXlai pNFUdlLxyVXuJmyNJ2i68s+rF03yUHVL+xEIkPp2/gaK6AJPJGrbWirou9t9AKtLFmgC GGTp3fU5gFNpXVrM0PscZFnX6qH/PWc+wTZsTu9W6DBAuC7/+MiBfnPS6Bg+Q9+Jiu+l VmRiZ1gdvaNheVpLotWd+W75GU0iaPcjYLEW+PJMKMumkxkW09/mcvbj4TxaBCiy2rDb 2jn2Tul1yBhnJJULuo1y6zjqEX9ty7FPKHUh5sPo2/lN/9iFwHyNuk8oFTvmPNT3/uHy vzrg== X-Gm-Message-State: ANhLgQ0RNAz7AFRD+nrz7PdqZXh/DjfUQjkhHgjGltWPGj0lh+90i1/Y ycypwhzq94Y/3r17hmVZAH+5ite4gIE8mxgSYI2QTw== X-Google-Smtp-Source: ADFU+vvVS8PVmqKFCTBtMlJz7wghfpAQ3eYzUsOa+wrZSu7iCFINoeTDtvqTifssxdmvEKYRxporlXHOVxiYjZjDBh8= X-Received: by 2002:a05:620a:200b:: with SMTP id c11mr7028847qka.380.1585691745807; Tue, 31 Mar 2020 14:55:45 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Tue, 31 Mar 2020 15:55:34 -0600 Message-ID: Subject: Re: Nonreproducible kernel modules on stable/12 To: Ed Maste Cc: freebsd-reproducibility@freebsd.org X-Rspamd-Queue-Id: 48sNSB4q7wz4VTm X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=uoe/DosF; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::730) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-3.01 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-reproducibility@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; URI_COUNT_ODD(1.00)[1]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[0.3.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.01)[ip: (-9.19), ipnet: 2607:f8b0::/32(-0.35), asn: 15169(-0.45), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Mailman-Approved-At: Wed, 01 Apr 2020 09:48:46 +0000 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-reproducibility@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions regarding FreeBSD reproducible builds List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2020 21:56:06 -0000 On Tue, Mar 31, 2020 at 8:08 AM Ed Maste wrote: > Originally reported off-list, during a stable/12 freebsd-update build > kernel modules are not reproducible and thus get included in the > update. > > diffoscope reports that the .SUNW_ctf section is responsible: > > --- cam.0.ko > +++ cam.1.ko > =E2=94=9C=E2=94=80=E2=94=80 readelf --wide --sections {} > ... > =E2=94=82 [30] .shstrtab STRTAB 0000000000000000 0ea8= 28 > 0001f8 00 0 0 1 > =E2=94=82 - [31] .SUNW_ctf PROGBITS 0000000000000000 0f29= d8 > 00e2ad 00 32 0 4 > =E2=94=82 + [31] .SUNW_ctf PROGBITS 0000000000000000 0f29= d8 > 00e2b1 00 32 0 4 > =E2=94=82 [32] .symtab SYMTAB 0000000000000000 0e12= b8 > 009570 18 33 1001 8 > ... > =E2=94=9C=E2=94=80=E2=94=80 readelf --wide --decompress --hex-dump=3D.SUN= W_ctf {} > =E2=94=82 @@ -1,3630 +1,3631 @@ > =E2=94=82 > =E2=94=82 Hex dump of section '.SUNW_ctf': > =E2=94=82 0x00000000 f1cf0201 00000000 00000000 00000000 .............= ... > =E2=94=82 0x00000010 08000000 96040000 fc250000 f4310100 .........%...= 1.. > =E2=94=82 - 0x00000020 6aeb0000 78dabc7d 077c54c5 16f7ccbd j...x..}.|T..= ... > =E2=94=82 - 0x00000030 774b42ca 26b48524 b0402001 02440d10 wKB.&..$.@ ..= D.. > ... > =E2=94=82 + 0x00000020 6feb0000 78dabc7d 077c54c5 16f7ccdc o...x..}.|T..= ... > =E2=94=82 + 0x00000030 bb252165 13da4242 586a0204 081820d4 .%!e..BBXj...= . . > > (And build-ID and GNU debuglink differ as a result.) > > Running ctfdump and comparing the result shows the issue: > > --- a 2020-03-31 10:04:39.766567000 -0400 > +++ b 2020-03-31 10:04:44.663292000 -0400 > @@ -11,11 +11,11 @@ > cth_funcoff =3D 1174 > cth_typeoff =3D 9724 > cth_stroff =3D 78324 > - cth_strlen =3D 60266 > + cth_strlen =3D 60271 > > - Label Table > ---------------------------------------------------------------- > > - 2407 FreeBSD 12.1-STABLE amd64 1201512 > + 2407 FreeBSD 12.1-FUTUREBUILD amd64 1201512 > So is this before/after the change? When I built stable/12 twice in a row, I get identical binaries. I get the same md5's for all the .ko's build as part of buildkernel/installkernel. Why is this string even in the binary? The string FUTUREBUILD doesn't appear at all in newvers.sh. Nor anywhere in the FreeBSD tree. This has got to be a FreeBSD update builder bug. Where's the source to it? The bug is in there. Warner From owner-freebsd-reproducibility@freebsd.org Wed Apr 1 19:21:08 2020 Return-Path: Delivered-To: freebsd-reproducibility@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 102D327E7B8 for ; Wed, 1 Apr 2020 19:21:08 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io1-xd2e.google.com (mail-io1-xd2e.google.com [IPv6:2607:f8b0:4864:20::d2e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48swys5D55z3HX7 for ; Wed, 1 Apr 2020 19:20:53 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io1-xd2e.google.com with SMTP id h131so984511iof.1 for ; Wed, 01 Apr 2020 12:20:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=cJR/XvGYTmUoXWj9yrZzVRhalHgH4uTxaiJRbgDgQUU=; b=i6vHmnGeXdiZeGO53+Q0YK9USzRNhJomUiMV7EdUpYIJGCN31wePwifH3oFGW2ilQF uZ6Ai3qUfYWjgOCFWeMnt4DHqTf4qyQ/zlzkksvVIG/p/0b28o4S6Urk/hgM/3+kRZzd X08CicmziJbEw7bx5mJcijrlwXXJHdh5LZ0PMvk0YuWyoNGrflzJntE9k1TvHFQCe9EG 1kX+gWCUPJhAMVV2bJ39PKYQvClS05yYpdFNj9LyfXSfYSjyqnH3voxhXJAMpPMc1m2p /wqb9tZcICm5TgM421DN7sdAxwTcY/Q4V7Y/vxEJW1rbWibaK3aNV71HFrqGE/cBmzUC tihA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=cJR/XvGYTmUoXWj9yrZzVRhalHgH4uTxaiJRbgDgQUU=; b=W86ZoE9WLz8A95EoU3AoncWy9QDHx7Pm4YlrA2ZQdQKAnpfqgXV26fhZiGScwRpGPQ ti/SJ6e6MgoCcoX9t97ypAP/ijGp28frSQpw8vQgj9ayN9FMhbqahWQ0+JsmdTsdFgIH BFH3iz1b+jiovQqYqmLRfpNmoPkotURfynAv2itvz5tx4P6+vY/21683OvVJfJXaRvSD Aj/qtDXYrcro3EVXYSrEz8P53KjJvUCvtC94327T1fem4v/L7aVc3lgRB07UlbUUZzIF /qJGIk7huf7wSpO9sd6/QBZQIEmDG9OeIm9yc80OOLCU+Qc6/GjmoOJ0L1TxcSoHwZR+ QWGg== X-Gm-Message-State: ANhLgQ1va0KKEnGqYOZ2gmpOsMdy8LYjz0/PrTq2MX+p1nqTVpII8/qU OyBmuoRppGzDlOln4sY+SGmMqiSK/W1f5MehVc5tbVa2 X-Google-Smtp-Source: ADFU+vvVnt4luisEKXITjnARwa4Yg+Pgqqvebhq5YtstXD1zKPcqlbiflrF9KmtKgYEWut+5pBevcy+CBbsNttrXvXg= X-Received: by 2002:a05:620a:200b:: with SMTP id c11mr11532104qka.380.1585768540882; Wed, 01 Apr 2020 12:15:40 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Wed, 1 Apr 2020 13:15:30 -0600 Message-ID: Subject: Re: Nonreproducible kernel modules on stable/12 To: Ed Maste Cc: freebsd-reproducibility@freebsd.org X-Rspamd-Queue-Id: 48swys5D55z3HX7 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=i6vHmnGe; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::d2e) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-2.39 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-reproducibility@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; URI_COUNT_ODD(1.00)[1]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[e.2.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-1.39)[ip: (-6.12), ipnet: 2607:f8b0::/32(-0.35), asn: 15169(-0.45), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Mailman-Approved-At: Thu, 02 Apr 2020 07:57:22 +0000 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-reproducibility@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions regarding FreeBSD reproducible builds List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2020 19:21:08 -0000 On Wed, Apr 1, 2020 at 1:10 PM Warner Losh wrote: > Sigh > > freebsd-update-build is being naughty: > > sed -i "" -E "s,^(BRANCH=3D.*)\",\1$2\"," > > and editing newvers.sh directly. > > This doesn't work. > > However, the change was > > BRANCH=3D"STABLE" > to > BRANCH=3D${BRANCH_OVERRIDE:-STABLE} > > maybe a quick hack is to add back the quotes or to change the above sed t= o > > sed -i "" -E "s,^(BRANCH=3D.*),\1\"$2\"," > > which would work on all branches because the expression > > BRANCH=3D"STABLE""-p2" > > is well formed as is > > BRANCH=3D${BRANCH_OVERRIDE:-STABLE}"-p2" > Or it could just set BRANCH_OVERRIDE to what it wants and not the crazy FUTUREBRANCH thing... Warner > Warner > > On Tue, Mar 31, 2020 at 8:08 AM Ed Maste wrote: > >> Originally reported off-list, during a stable/12 freebsd-update build >> kernel modules are not reproducible and thus get included in the >> update. >> >> diffoscope reports that the .SUNW_ctf section is responsible: >> >> --- cam.0.ko >> +++ cam.1.ko >> =E2=94=9C=E2=94=80=E2=94=80 readelf --wide --sections {} >> ... >> =E2=94=82 [30] .shstrtab STRTAB 0000000000000000 0ea= 828 >> 0001f8 00 0 0 1 >> =E2=94=82 - [31] .SUNW_ctf PROGBITS 0000000000000000 0f2= 9d8 >> 00e2ad 00 32 0 4 >> =E2=94=82 + [31] .SUNW_ctf PROGBITS 0000000000000000 0f2= 9d8 >> 00e2b1 00 32 0 4 >> =E2=94=82 [32] .symtab SYMTAB 0000000000000000 0e1= 2b8 >> 009570 18 33 1001 8 >> ... >> =E2=94=9C=E2=94=80=E2=94=80 readelf --wide --decompress --hex-dump=3D.SU= NW_ctf {} >> =E2=94=82 @@ -1,3630 +1,3631 @@ >> =E2=94=82 >> =E2=94=82 Hex dump of section '.SUNW_ctf': >> =E2=94=82 0x00000000 f1cf0201 00000000 00000000 00000000 ............= .... >> =E2=94=82 0x00000010 08000000 96040000 fc250000 f4310100 .........%..= .1.. >> =E2=94=82 - 0x00000020 6aeb0000 78dabc7d 077c54c5 16f7ccbd j...x..}.|T.= .... >> =E2=94=82 - 0x00000030 774b42ca 26b48524 b0402001 02440d10 wKB.&..$.@ .= .D.. >> ... >> =E2=94=82 + 0x00000020 6feb0000 78dabc7d 077c54c5 16f7ccdc o...x..}.|T.= .... >> =E2=94=82 + 0x00000030 bb252165 13da4242 586a0204 081820d4 .%!e..BBXj..= .. . >> >> (And build-ID and GNU debuglink differ as a result.) >> >> Running ctfdump and comparing the result shows the issue: >> >> --- a 2020-03-31 10:04:39.766567000 -0400 >> +++ b 2020-03-31 10:04:44.663292000 -0400 >> @@ -11,11 +11,11 @@ >> cth_funcoff =3D 1174 >> cth_typeoff =3D 9724 >> cth_stroff =3D 78324 >> - cth_strlen =3D 60266 >> + cth_strlen =3D 60271 >> >> - Label Table >> ---------------------------------------------------------------- >> >> - 2407 FreeBSD 12.1-STABLE amd64 1201512 >> + 2407 FreeBSD 12.1-FUTUREBUILD amd64 1201512 >> >> - Data Objects >> --------------------------------------------------------------- >> >> Aside, we should add ctfdump to diffoscope. >> > From owner-freebsd-reproducibility@freebsd.org Wed Apr 1 20:13:25 2020 Return-Path: Delivered-To: freebsd-reproducibility@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 006052A17AB for ; Wed, 1 Apr 2020 20:13:24 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48sy7G30zJz47dX for ; Wed, 1 Apr 2020 20:13:13 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-pg1-x544.google.com with SMTP id l14so691343pgb.1 for ; Wed, 01 Apr 2020 13:13:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=YFZMiOKkSY+E0NxAt6YCV5RHiZ+TUZGHoDyjssIpKTk=; b=AWJ8Zm3PYSG2c6QfF9Z09JV3pYqESoLGMmSHtGQ5OVqORqBOpSu1jphw2OWx/WRitq Bgugmdb4t8tEeqNZQphPJnc4SfLKnvzarYbVdrFcCZrlQhq61M8FOQkUyf8A40KZNRhe 3PaifMX+rH9WWWlSb0jxE/OFvV3H539eV5geONpC2FAzwbWm9wIeMsZNITMt510sbQ3H x1eDikqw+C1XyPt3glpAKyLu9hx151ep1XPMFyr9dM+syHinJovRvBfSoxXseJDJZr2X q8v+lUXBfDeCcqTEB/67qn34DzvSsgAhkc0KaZA5gRtolEF7JFU7O8zozvxtoCRMCZdO VNzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=YFZMiOKkSY+E0NxAt6YCV5RHiZ+TUZGHoDyjssIpKTk=; b=mSKZx0lC0locJdabKAZQ7DumwbP+JetggEiXuM4O6Vog9Ikuq4TQmz4xoIjEHQgcCY h2qzeg43zEAKiF9RzDtJ6pnQzvI/T4xRHGNFPI7Ddk00DOUMwNCym47ZHFTfeee7A6YK GYnx4q1aofO4kyzX03FVjJjV6CSa1Nh4bNoEZM/rrRTjGZvoKGxlQrK1bcj6Yh07e6dI GDDJWEL5t1z2Uq6DU88EcOh04sUubnee4pQSKiJQ5VllG5gj8QCifZ5CVooZVezJKVFA +wPdBuRGIaEw/ZI9TZgdxdqCJ/QvMa9h6puQuSLa2kNTjMNZMG5/tTHm+KGE+aB3h/Y0 1zDQ== X-Gm-Message-State: AGi0PuZeLvJHBop/R+w4QGroqx9qG7LnS8osT66eptagdmyOIF4eYIE6 Yqo+J633zUZzvlA9naMaHVApTtGGcfCOF7WZMMJAuZkyilI= X-Google-Smtp-Source: ADFU+vs11HlU9jEHSr0y/Fnbt1GnspHXye+UooRnOJK2r35IL0hStmoCbFzTL638o6U7GwKqgbqFafHKx4SKO5n56WU= X-Received: by 2002:a05:6214:611:: with SMTP id z17mr23418960qvw.125.1585768224184; Wed, 01 Apr 2020 12:10:24 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Wed, 1 Apr 2020 13:10:13 -0600 Message-ID: Subject: Re: Nonreproducible kernel modules on stable/12 To: Ed Maste Cc: freebsd-reproducibility@freebsd.org X-Rspamd-Queue-Id: 48sy7G30zJz47dX X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=AWJ8Zm3P; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::544) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-1.10 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.96)[-0.962,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-0.96)[-0.963,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-reproducibility@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; URI_COUNT_ODD(1.00)[1]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[4.4.5.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-0.17)[ip: (-0.02), ipnet: 2607:f8b0::/32(-0.35), asn: 15169(-0.45), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Mailman-Approved-At: Thu, 02 Apr 2020 08:54:30 +0000 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-reproducibility@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions regarding FreeBSD reproducible builds List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2020 20:13:25 -0000 Sigh freebsd-update-build is being naughty: sed -i "" -E "s,^(BRANCH=3D.*)\",\1$2\"," and editing newvers.sh directly. This doesn't work. However, the change was BRANCH=3D"STABLE" to BRANCH=3D${BRANCH_OVERRIDE:-STABLE} maybe a quick hack is to add back the quotes or to change the above sed to sed -i "" -E "s,^(BRANCH=3D.*),\1\"$2\"," which would work on all branches because the expression BRANCH=3D"STABLE""-p2" is well formed as is BRANCH=3D${BRANCH_OVERRIDE:-STABLE}"-p2" Warner On Tue, Mar 31, 2020 at 8:08 AM Ed Maste wrote: > Originally reported off-list, during a stable/12 freebsd-update build > kernel modules are not reproducible and thus get included in the > update. > > diffoscope reports that the .SUNW_ctf section is responsible: > > --- cam.0.ko > +++ cam.1.ko > =E2=94=9C=E2=94=80=E2=94=80 readelf --wide --sections {} > ... > =E2=94=82 [30] .shstrtab STRTAB 0000000000000000 0ea8= 28 > 0001f8 00 0 0 1 > =E2=94=82 - [31] .SUNW_ctf PROGBITS 0000000000000000 0f29= d8 > 00e2ad 00 32 0 4 > =E2=94=82 + [31] .SUNW_ctf PROGBITS 0000000000000000 0f29= d8 > 00e2b1 00 32 0 4 > =E2=94=82 [32] .symtab SYMTAB 0000000000000000 0e12= b8 > 009570 18 33 1001 8 > ... > =E2=94=9C=E2=94=80=E2=94=80 readelf --wide --decompress --hex-dump=3D.SUN= W_ctf {} > =E2=94=82 @@ -1,3630 +1,3631 @@ > =E2=94=82 > =E2=94=82 Hex dump of section '.SUNW_ctf': > =E2=94=82 0x00000000 f1cf0201 00000000 00000000 00000000 .............= ... > =E2=94=82 0x00000010 08000000 96040000 fc250000 f4310100 .........%...= 1.. > =E2=94=82 - 0x00000020 6aeb0000 78dabc7d 077c54c5 16f7ccbd j...x..}.|T..= ... > =E2=94=82 - 0x00000030 774b42ca 26b48524 b0402001 02440d10 wKB.&..$.@ ..= D.. > ... > =E2=94=82 + 0x00000020 6feb0000 78dabc7d 077c54c5 16f7ccdc o...x..}.|T..= ... > =E2=94=82 + 0x00000030 bb252165 13da4242 586a0204 081820d4 .%!e..BBXj...= . . > > (And build-ID and GNU debuglink differ as a result.) > > Running ctfdump and comparing the result shows the issue: > > --- a 2020-03-31 10:04:39.766567000 -0400 > +++ b 2020-03-31 10:04:44.663292000 -0400 > @@ -11,11 +11,11 @@ > cth_funcoff =3D 1174 > cth_typeoff =3D 9724 > cth_stroff =3D 78324 > - cth_strlen =3D 60266 > + cth_strlen =3D 60271 > > - Label Table > ---------------------------------------------------------------- > > - 2407 FreeBSD 12.1-STABLE amd64 1201512 > + 2407 FreeBSD 12.1-FUTUREBUILD amd64 1201512 > > - Data Objects > --------------------------------------------------------------- > > Aside, we should add ctfdump to diffoscope. >