From owner-freebsd-stable@freebsd.org Mon Sep 11 23:34:30 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9860DE07481 for ; Mon, 11 Sep 2017 23:34:30 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 768607C1FD for ; Mon, 11 Sep 2017 23:34:30 +0000 (UTC) (envelope-from ian@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 72AC4E0747F; Mon, 11 Sep 2017 23:34:30 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 72291E0747E for ; Mon, 11 Sep 2017 23:34:30 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from nov-007-i455.relay.mailchannels.net (nov-007-i455.relay.mailchannels.net [46.232.183.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8B09E7C1F7 for ; Mon, 11 Sep 2017 23:34:23 +0000 (UTC) (envelope-from ian@freebsd.org) X-Sender-Id: _forwarded-from|73.78.92.27 Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 9483F5C1422 for ; Mon, 11 Sep 2017 23:34:13 +0000 (UTC) Received: from outbound1a.eu.mailhop.org (unknown [100.96.146.203]) (Authenticated sender: duocircle) by relay.mailchannels.net (Postfix) with ESMTPA id EDD6F5C10D3 for ; Mon, 11 Sep 2017 23:34:12 +0000 (UTC) X-Sender-Id: _forwarded-from|73.78.92.27 Received: from outbound1a.eu.mailhop.org (outbound1a.eu.mailhop.org [172.20.107.195]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.9.14); Mon, 11 Sep 2017 23:34:13 +0000 X-MC-Relay: Junk X-MailChannels-SenderId: _forwarded-from|73.78.92.27 X-MailChannels-Auth-Id: duocircle X-Interest-Left: 0e0a14e077d5eb95_1505172853400_212957668 X-MC-Loop-Signature: 1505172853400:647474909 X-MC-Ingress-Time: 1505172853400 X-MHO-User: b3487f62-9749-11e7-a893-25625093991c X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.eu.mailhop.org (Halon) with ESMTPSA id b3487f62-9749-11e7-a893-25625093991c; Mon, 11 Sep 2017 23:34:06 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id v8BNY5Nm013618; Mon, 11 Sep 2017 17:34:05 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <1505172845.32063.90.camel@freebsd.org> Subject: Re: entropy lockup From: Ian Lepore To: Morgan Reed , Vick Khera Cc: FreeBSD Stable List Date: Mon, 11 Sep 2017 17:34:05 -0600 In-Reply-To: References: Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Sep 2017 23:34:30 -0000 On Tue, 2017-09-12 at 09:20 +1000, Morgan Reed wrote: > In all likelihood the process wasn't "hung" per-se, more likely > random > hadn't been seeded yet and as such you *can't* get 4096b of entropy > out of > it (so the process was attempting to do its job, just nothing on the > device). >=20 > The issue is basically that there are very few "good" entropy sources > in a > VM environment, and as such (particularly on a machine which is only > running SSH) there's not enough data available to seed random. >=20 > Check 'man 4 random' for some discussion. >=20 The save-entropy files get written from a crontab entry. =A0That means the system is already up and running, which means it's well past the point of hanging because it needs to be seeded. =A0Once it is seeded, it can never "run out of entropy". So all in all, this is a genuine problem of some sort. =A0The fact that the save-entropy process has accumulated 48+ minutes of cpu time is another problem indicator. =A0The process state is RL, runnable and waiting to acquire a lock, which seems contradictory (unless that means it's waiting for a spinlock maybe). -- Ian > On Tue, Sep 12, 2017 at 12:02 AM, Vick Khera wrote: >=20 > >=20 > > I just had a VM running in Google's cloud become totally useless, > > and I > > tracked it down to the save-entropy operation. > >=20 > > Basically this process was sucking up all CPU, even when nothing > > else > > running other than my ssh shell: > >=20 > > % ps axuw803 > > USER=A0=A0=A0=A0=A0PID=A0=A0%CPU %MEM=A0=A0VSZ=A0=A0RSS TT=A0=A0STAT = STARTED=A0=A0=A0=A0=A0TIME COMMAND > > operator 803 100.0=A0=A00.1 8336 2096=A0=A0-=A0=A0RL=A0=A0=A008:55=A0= =A0=A048:20.14 dd > > if=3D/dev/random of=3Dsaved-entropy.1 bs=3D4096 count=3D1 > >=20 > >=20 > > The process is unkillable, and I cannot even get the system to shut > > down. > > That has been hanging for about 10 minutes so far, with the last > > output > > being > >=20 > > System shutdown time has arrive90 second watchdSep 11 09:50:02 > > yertle init: > > /bin/sh on /etc/rc.shutdown terminated abnormally, going to single > > user > > mode > > Sep 11 09:50:47 init: some processes would not die; ps axl advised > > Waiting (max 60 seconds) for system process `vnlru' to stop... done > > Waiting (max 60 seconds) for system process `bufdaemon' to stop... > > done > > Waiting (max 60 seconds) for system process `syncer' to stop... > > Syncing disks, vnodes remaining... 4 4 4 4 4 4 4 timed out > > 2 2 2 2 2 2 2 > >=20 > >=20 > > Running FreeBSD 11.1-p1 on a 1CPU standard machine in GCE. > >=20 > > What's the proper recovery from this kind of lockup, or how to > > prevent it? > > I've never encountered this on a bare metal system, or other KVM > > based > > machines. > > _______________________________________________ > > freebsd-stable@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebs > > d.org" > >=20 >=20 >=20