From owner-freebsd-arch@FreeBSD.ORG Tue Nov 25 17:11:13 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F2B3AE42 for ; Tue, 25 Nov 2014 17:11:13 +0000 (UTC) Received: from mail-pa0-f47.google.com (mail-pa0-f47.google.com [209.85.220.47]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B40DB971 for ; Tue, 25 Nov 2014 17:11:13 +0000 (UTC) Received: by mail-pa0-f47.google.com with SMTP id kq14so960485pab.6 for ; Tue, 25 Nov 2014 09:11:07 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:message-id:references:to; bh=2F6+vf/f441ZCxYMKT7yD5k2JyBehNPNbJcjoh+2y0Q=; b=OBSa1mcIYLzc7s+ODKpVMcqFV/JMtSArTBUlmjzamhw2GInHLduq1yyah67ylmcHyx 26p0cRy+MaMbrHqwDtCWG1bqtG23uyrCJixy6TUP+YR+YaaaHS4PYJdqXYWERXDMdV1w 1d3F4ROPeYBPQzew1sXyL+hYpbFvlHuwhwW9ki79Y6Hb3k9pbQOR//z09sWKQ18yL0VZ GZEYL3BeGKJq6bVyeaZmIffgo+Rzuh0UY19Q85Qp7UUGcHCny3AyDN/YtKbeaXM0XEHj BaMCMwocqFG24sdwIUYBqkEkQ6YRqWOMM/k3xELLexzh5dWHQ6lTJOnEjQg1dI80Y3hn /SDg== X-Gm-Message-State: ALoCoQmQoGkuNo5UR+M9qH4is/trJJ5kcnQ7QDncZMT01GSXboXJBrtDIxCddiKLQOWHzx5s0YFC X-Received: by 10.66.159.201 with SMTP id xe9mr45689398pab.13.1416935466817; Tue, 25 Nov 2014 09:11:06 -0800 (PST) Received: from [10.64.26.139] ([69.53.236.236]) by mx.google.com with ESMTPSA id uy8sm2019654pab.44.2014.11.25.09.11.04 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 25 Nov 2014 09:11:05 -0800 (PST) Sender: Warner Losh Subject: Re: svn commit: r274739 - head/sys/mips/conf Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) Content-Type: multipart/signed; boundary="Apple-Mail=_3E3AC760-EC54-425C-AEB4-F42CF2CF15DB"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.5b2 From: Warner Losh In-Reply-To: <1416925387.1147.437.camel@revolution.hippie.lan> Date: Tue, 25 Nov 2014 10:11:03 -0700 Message-Id: <18F34536-CA8B-4365-BDD9-C2D23E6067AA@bsdimp.com> References: <201411200552.sAK5qnXP063073@svn.freebsd.org> <20141120084832.GE24601@funkthat.com> <20141121092245.GI99957@funkthat.com> <1416582989.1147.250.camel@revolution.hippie.lan> <026FEB8A-CA8C-472F-A8E4-DA3D0AC44B34@grondar.org> <1416596266.1147.290.camel@revolution.hippie.lan> <1416598889.1147.297.camel@revolution.hippie.lan> <86egsvueqk.fsf@nine.des.no> <1416691274.1147.339.camel@revolution.hippie.lan> <398A380D-49AF-480C-8842-8835F81EF641@grondar.org> <1416806894.1147.362.camel@revolution.hippie.lan> <18B8A926-59C0-49B4-ADA3-A11688609852@grondar.org> <1416841268.1147.386.camel@revolution.hippie.lan> <86wq6k9okk.fsf@nine.des.no> <8661e3wtk6.fsf@nine.des.no> <86oarvvaet.fsf@nine.des.no> <86egsrxypx.fsf@nine.des.no> <1416925387.1147.437.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1993) Cc: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , Mark R V Murray , arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Nov 2014 17:11:14 -0000 --Apple-Mail=_3E3AC760-EC54-425C-AEB4-F42CF2CF15DB Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 > On Nov 25, 2014, at 7:23 AM, Ian Lepore wrote: >=20 > On Tue, 2014-11-25 at 13:15 +0100, Dag-Erling Sm=F8rgrav wrote: >> Ian, please try the attached patch. >>=20 >> The way this is intended to be used is that you set up a system with = an >> /etc/rc that does nothing else than mount the root file system and >> append the output from "sysctl -b hw.attachtimes" to a log file, then >> reboot. You let it run for as long as you wish, then copy the log = file >> to another machine and run the attachtimes utility (source code = below) >> on the log file to extract the data and display either the raw = numbers >> or a histogram showing the distribution. >>=20 >> See freefall:~des/software/attachtimes.tgz for an example of how to = set >> this up. All you need is the loader and kernel, a minimal /etc, and = the >> tools required by /etc/rc (sh, fsck, mount, df, sysctl, halt, = reboot). >> The example contains quite a bit more than that because I wanted to = be >> able to boot it in single-user mode for debugging. >>=20 >> The patch can easily be modified to record the actual timestamps = instead >> of just the delta, but remember to modify the utility as well, or the >> output will be complete nonsense. >>=20 >> DES >=20 > Getting the results for this is going to take a while... I can't get = the > system past mountroot right now. I have no idea why. It's been = months > since I last successfully booted -current on this old hardware. I'm > also very busy with $work. I have similar hardware on my desk, and I might be able to find a little time to boot it. > For the 3 numbers that are identical at the start, I think we're = reading > garbage in binuptime() because the clock driver isn't working yet. = The > numbers first change with at91_st0, that's the clock driver. = atmelarm0 > is the parent of everything listed between nexus and it, so the large > number there is probably the subtraction of the post-init clock value > from the pre-init clock garbage. Yea, at best this bug is solved by initializing something to zero :) > In a more general sense, I'm going to repeat a couple things... >=20 > The clock all this is being measured with runs at 32 KHz. That's 'K=92.= Since these systems run at 200MHz or 400MHz, it makes a lot of sense that we=92re seeing the same number. That=92s like ~10k instructions = between ticks. And since that is really 32,768Hz, it makes sense that the bottom 15 bits are zeros given how bintime encodes things. > This output is evidence that the system behaves exactly as I've been > saying it behaves for a long long while. You seem convinced, I don't > understand why, that there must be some error here. I don't = understand > why you think a system like this would behave any differently each = time > it is powered on. The only actual entropy involved is whatever minor > thermal transients may exist in the crystal oscillator. With 32KHz > resolution (or even a few MHz) that amounts to not a lot of measurable > variation at all. Yes, we=92re 4 orders of magnitude above the =91changes every = instruction or so=92 notion that Mark had sent out. > The repeatability of the boot sequence of hardware like this is nearly > perfect at the resolution we're measuring. While that may be bad for > gathering entropy, it's a wonderful thing when you're debugging, = because > problems that would be almost impossible to nail down on modern = complex > hardware are 100% reproducible by just hitting the reset button. That > reproducibility extends all the way into multiuser mode unless there = is > a network connection where packet arrival times start adding > interrupt-based entropy. Yea, every boot it is the same registers that are being read, in the same sequence, resulting in very similar cache patterns and performance profiles. I=92d be surprised if anything apart from the ethernet chip=92s state was different between boots. And even the ethernet=92s stuff has a low variance until interrupts are turned on=85 Warner --Apple-Mail=_3E3AC760-EC54-425C-AEB4-F42CF2CF15DB Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJUdLgnAAoJEGwc0Sh9sBEAb7oP/Al2E4SXLa702GmiwHCllEQr PCl1vTy9XuBCQfgKNiT9M4MDFADg818/ZAL+kFxXVxCUvW1Lr4bUoy51bGTrdnpR +6/IGcSZ3UXm606wwuqj1FVorlgrVTxnXybCZNRQbiHZxsDUPnEIMWEyjUMAReBu RHkUUyYWSLRe2P/Mx30fMOygJUx1Q3NRAvhWn/gQk/viq6p9gIuR8dbwAL1tmAQX Ve1JBa6rkhp+d1/kWbSuImE3wSCIjYv8RF+/unxCZxJ1ocr2RQwXd/LavHOG7GAa e9VJb5jMRvTx/Gr5pby37UAeQGVQ4gIbiU75b9TNJXNMvi9z6f8GWbP+bn15U9tK 0cFe8ww6QZaP7L/XuWUtfOM4IjUMK4+2UVHf1XiePVACJ5WHt8sMlzMJWDVocenC VaN+QazmVJqpQxPEMQQyukDHJ9QwKbpHt/qMy8BK75d65s7rBXuwYiPBg0oYvIAk 2HBio3IkWgji0fsZzZMdIfMndie+Rd1YzUwv0ZAK9La0iIMgbrpw+8iDdvMQHtP+ y5AC+8UfFGz6VaZ+ZpYbqbx5GDtz8kwyI6cfjOFlcgf5OUgqZXsixbJJUI5Yz0d6 ODIwoK+sjPkqDAeODsbFXPzzdts2J+8STD7UlUasudcTHf5we9ekuMTJJuNED/BI xEoMYEJJONikyZjuXhkl =Ugyv -----END PGP SIGNATURE----- --Apple-Mail=_3E3AC760-EC54-425C-AEB4-F42CF2CF15DB--