From owner-freebsd-stable@freebsd.org Fri Feb 26 00:23:15 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AD1E954ED8C for ; Fri, 26 Feb 2021 00:23:15 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x72b.google.com (mail-qk1-x72b.google.com [IPv6:2607:f8b0:4864:20::72b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Dmr3Q6x8Mz4SyQ for ; Fri, 26 Feb 2021 00:23:14 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x72b.google.com with SMTP id h8so7571559qkk.6 for ; Thu, 25 Feb 2021 16:23:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=wzcv+DQSih7RM2T55yDVrXlFV6yZqIT3UY9ecl5ezA0=; b=GRTPrHlxptMsZ7JLfKIPmm7GGuR+djJJ4WNgOT/gLE+VJto48IMe45lAgtvrtoiVIb HsRqzrtKYYt0JMpOcisidyhnNG7Y0QOevSwbUEz3ZuLDFaKWjuJZynNjzqpHSE4fDx5m xfNxeLRwSrPNHW0qbC/piVkfuH6wn3fARgrYI5+/p+EQvkJEDbDV4Lhww8AkyZyAy5l8 Q3ad7n9Mqvp/7xGw3tyxNa6okUFqmVV3MvFUW9AtqRly2AJIhae6EceYEgBmvQpo8v/F MjV0K30YrAbIZJgR9vODnYs4TAWF9UFqmbrWLchfDmUZaX8vngM653FV3kDRJ4ZwARHa Qz2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=wzcv+DQSih7RM2T55yDVrXlFV6yZqIT3UY9ecl5ezA0=; b=hKDK6UYapcLfHRluAZX0QIUcnzwYlSL+YRHf60vRACluqdsUkaAmBvAJGMj7p2X1ko gqfbTWTZXtT3z+He7WW3Bo32gnlgZjMlLi/r008rJfscMU+8/J0EELA5PAqWeGQPOS+d uKuukgvBuegLjbexVmUWr8eNTL2HoD8SdEb/ChjvrkiDZO8RL9Nl852bG3puINiBBYZn b+UMii3YtrpVKEBKC1CQOx9NLgA4YH5qsraW72Uq6mu3ZfnDZqGwZpuUAODUYVp6cfdW hl1KCgf+8MlNE1NVkiXheVZ2dS7RYnoxgcbONNjGXBOT5GSoM/5kASUVMloaEsvMaKE4 ISyA== X-Gm-Message-State: AOAM5335HNLJI0TdpXeVepskYEstLiBWE28y4UdcoWt0sMzx8gtTCET4 G1PDuFP8qfgYq9rAcWFDUcN3U9eBDVaGaMTNwkPHLg== X-Google-Smtp-Source: ABdhPJyyG3jlnHX0Qz2hizoK+q7d+3jNf64UZMpk4f1k5tJtiBdBELy6YDK/pA+yoiZEhHKhTow5N5HMc43ykGX/K48= X-Received: by 2002:a37:a151:: with SMTP id k78mr304963qke.359.1614298993787; Thu, 25 Feb 2021 16:23:13 -0800 (PST) MIME-Version: 1.0 References: <1748076.jFELhIj8lM@ravel> <3308997.ajJYar8FF2@ravel> <001a5401-c334-5937-4ce3-315ff89e34be@denninger.net> <7d4e7a1f-da3e-2860-62b1-7be88123bee9@denninger.net> In-Reply-To: <7d4e7a1f-da3e-2860-62b1-7be88123bee9@denninger.net> From: Warner Losh Date: Thu, 25 Feb 2021 17:22:59 -0700 Message-ID: Subject: Re: How do I know if my 13-stable has security patches? To: Karl Denninger Cc: FreeBSD-STABLE Mailing List X-Rspamd-Queue-Id: 4Dmr3Q6x8Mz4SyQ X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=GRTPrHlx; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::72b) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-2.00 / 15.00]; RCVD_TLS_ALL(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::72b:from]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::72b:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::72b:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; SUBJECT_ENDS_QUESTION(1.00)[]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; MAILMAN_DEST(0.00)[freebsd-stable] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2021 00:23:15 -0000 Before I get into the blow by blow (which can sound nit-picky, despite my best efforts), I would like to apologize. It wasn't completely appreciated how clearly the dependencies that the nXXXXX number being generated needed to be communicated. And for that I apologize. When they are met, we have all the pieces we need to at build time to generate the nXXXXX number and none of the fallback methods are needed. I'll work to update the docs to clearly communicate this since it is completely absent from my current docs. On Thu, Feb 25, 2021 at 2:56 PM Karl Denninger wrote: > On 2/25/2021 15:56, Warner Losh wrote: > > > On Thu, Feb 25, 2021 at 6:37 AM Karl Denninger wrote: > >> On 2/25/2021 04:30, Olivier Certner wrote: >> >> Neither command is what I'd call 'intuitive', so it would have taken >> me a >> >> long time to find either of them. I cut and pasted the 'git branch' >> command >> >> and it took me a moment to realize what that meant. Never ran "grep >> -l" on >> >> a pipe, I guess. >> > You made me laugh! Apart from relatively simple commands, git's >> interface is >> > far from intuitive. That's the reason why I regret that it became the >> hugely >> > dominant DVCS. >> >> Regression doesn't have to come to a project, but if the tools you >> choose do things like this then you have to work around them as a >> project to avoid the issue, and that might wind up being somewhat of a >> PITA. >> >> This specific issue is IMHO quite severe in terms of operational >> impact. I track -STABLE but don't load "new things" all the time. For >> security-related things it's more important to know if I've got >> something out there in a specific instance where it may apply (and not >> care in others where it doesn't; aka the recent Xen thing if you're not >> using Xen.) Otherwise if everything is running as it should do I wish >> to risk introducing bugs along with improvements? If not in a >> security-related context, frequently not. >> >> Well, this used to be easy. Is your "uname" r-number HIGHER than the >> "when fixed" revision? You're good. Now, nope. Now I have to go dig >> source to know because there is no longer a "revision number" that >> monotonically increments with each commit so there is no longer a way to >> have a "point in time" view of the source, as-committed, for a given >> checked-out version. >> >> IMHO that's a fairly serious regression for the person responsible for >> keeping security-related things up to date and something the project >> should find a way to fix before rolling the next -RELEASE. (Yeah, I know >> that's almost-certain to not happen but it's not like this issue wasn't >> known since moving things over to git.) >> > > We should likely just publish the 'v' number in the advisories. It's > basically a count back to the start of the project. We put that number in > uname already. > > You can also find out the 'v' number in the latest advisories by cloning > the repo and doing the same thing we do in newvers.sh: > % git rev-list --first-parent --count $HASH > and that will tell you. This needn't be on the target machine since the > hashes are stable across the world. > > (list of further "stuff") > > But that's my entire point Warner. > > The time (and present items) on a given machine to know whether it is > covered by a given advisory under the "svn view of the world" is one > command, and no sources. That is, if the advisory says "r123456" has the > fix, then if I do a "uname -v" and get something larger, it's safe. > > If I get something smaller it's not. > If you built from a full clone, then it's the same today. You can look at the nXXXXX that's in the kernel string along with the branch and know if you should upgrade or not. However, if you built from a shallow clone, that's no longer possible. If you need this functionality, you must build from sources that are from a tree that has access to the full git repo. This detail was poorly communicated, I'll be the first to admit. And to be fair, to get the rXXXXXX number, however, you also needed to have a subversion metadata around as well (it is much the same as having the full clone now). And there's also the date which is added to the uname, if you didn't do a reproducible build. If it is older than the security advisory, then you'll need to update. And most ways that update the kernel preserve the build time in the metadata for /boot/kernel/kernel if you did a reproducible build (though this may not be true if you use makefs) . If the dates are newer, though, you'll need to do the hash dance. > I don't need the source on the machine, I don't need svn on the target or, > for that matter, do I need to know if the source tree I have on a build > machine is coherent with whatever is on the running machine. > Still don't need these things, completely. You just need the hash that's reported. If you build from a pull of the full tree, you'll have the nXXXX number and you have what you want. If you didn't, you'll at least have the hash and can find out, via methods I've described. If you build before the git cutover w/o access to the subversion metadata, you won't have these numbers either. Building from a subsetted git tree is a convenience, and you are giving up functionality if you choose to do that (though the information is still recoverable should you need to know). It's unfortunate (and likely damn frustrating to people with currently deployed stable/13 systems) I didn't clearly document this dependency when I wrote up the user guide. The only time you'll need to change is if you build from a shallow cloned tree. > I simply need to know if the source that built the code that is running > was updated *after* the commit that fixes the problem. What if the source *isn't > on that machine *because you build on some system and then distribute? > Does every machine now have to be coherent with your source repository in > order to be able to figure out where you are or worse, it must keep the > source from which that specific installation, individually, was built? *What > if the source isn't there at all *because you run binary code and update > with freebsd-update? > Sources not there or not doesn't matter. It's all about where the binaries were built. I believe freebsd-update builds such that the nXXXX number will be in the kernel. And there's also always the ls -l /bin/ls method which can give you a date userland was built, and the 'file /bin/ls' which will give the revision level that it was built at (though freebsd-update spikes things a bit to defeat this method if you updated that way. file /bin/ls will tell you the _FreeBSD_version it was built with, which gives a good clue if you need to update. 'freebsd-update fetch' will download any new fixes without applying them. You can use 'freebsd-update updatessready' to see if that results in there being updates. freebsd-update generally pegs to a release, so some of the finer-points that would affect self-built systems on stable don't apply. > Unless I've missed something that's what was lost and IMHO needs to be > restored; a way to know that in seconds with nothing other than the > operating OS on the box (e.g. via uname) and the advisory with its "greater > than X is safe" from the mailing list. Am I misunderstanding the current > state of things in this regard? > Only in a subset of configurations. If you build from one of those, then you accept you'll need to do more than just mental math to know if you need to upgrade the system. However, you can know from just uname -a output: you have a hash. Somewhere there's a tree with all the latest revs. Between the two, you can determine if the system needs to be updated. If you really need to be able to do the mental math, you should build from a full clone. Finally, I'd like to reiterate that, until now, these requirements haven't been properly appreciated or documented. I also get that change is hard. Between the two of these, this issue is generating a lot of angst and concern.... Warner