From owner-freebsd-current@freebsd.org Fri Nov 13 11:06:15 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A2B682D72F5 for ; Fri, 13 Nov 2020 11:06:15 +0000 (UTC) (envelope-from yasu@utahime.org) Received: from maybe.home.utahime.org (gate.home.utahime.org [183.180.29.210]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CXbHp3jfxz3mPZ for ; Fri, 13 Nov 2020 11:06:14 +0000 (UTC) (envelope-from yasu@utahime.org) Received: from eastasia.home.utahime.org (eastasia.home.utahime.org [192.168.174.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by maybe.home.utahime.org (Postfix) with ESMTPS id 883132B29A for ; Fri, 13 Nov 2020 20:06:04 +0900 (JST) Received: from localhost (rolling.home.utahime.org [192.168.174.11]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by eastasia.home.utahime.org (Postfix) with ESMTPSA id 6AA2F41F9C; Fri, 13 Nov 2020 20:06:03 +0900 (JST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.0 at eastasia.home.utahime.org Date: Fri, 13 Nov 2020 20:04:59 +0900 (JST) Message-Id: <20201113.200459.520180046556100070.yasu@utahime.org> To: freebsd-current@freebsd.org Subject: Re: Shutdown errors and timeout From: Yasuhiro KIMURA In-Reply-To: <65b1ff51-a946-61d0-79d9-104c1e053554@gmail.com> References: <65b1ff51-a946-61d0-79d9-104c1e053554@gmail.com> X-Mailer: Mew version 6.8 on Emacs 27.1 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4CXbHp3jfxz3mPZ X-Spamd-Bar: / X-Spamd-Result: default: False [-0.70 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+a:spf-authorized.utahime.org]; TO_DN_NONE(0.00)[]; HFILTER_HELO_IP_A(1.00)[maybe.home.utahime.org]; HFILTER_HELO_NORES_A_OR_MX(0.30)[maybe.home.utahime.org]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[utahime.org:+]; NEURAL_HAM_SHORT(-1.00)[-0.997]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[183.180.29.210:from]; ASN(0.00)[asn:2519, ipnet:183.180.0.0/16, country:JP]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[utahime.org:s=maybe2019112701]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[utahime.org]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[183.180.29.210:from:127.0.2.255]; MID_CONTAINS_FROM(1.00)[]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2020 11:06:15 -0000 From: Johan Hendriks Subject: Shutdown errors and timeout Date: Fri, 13 Nov 2020 11:35:53 +0100 > Hello all, i have two FreeBSD 13 machines, one is a bare metal and one > is virtualbox machine which i both update about once a week. > > The vritual machine seems to fail stopping something and gives a > timeout after 90 sec. > > The console ends with > > Writing entropy file: . > Writing early boot entropy file: . > > 90 second watchdog timeout expired. Shutdown terminated. > Fri Nov13 11:20:40 CEST 2020 > Nov 13 11:20:40 test-head init[1]: /etc/rc.shutdown terminated > abnormally, going to single user mode > ... > > On the bare metal machine i see the following. > Writing entropy file: . > Writing early boot entropy file: . > cannot unmount '/var/run': umount failed > cannot unmount '/var/log': umount failed > cannot unmount '/var': umount failed > cannot unmount '/usr/home': umount failed > cannot unmount '/usr': umount failed > cannot unmount '/': umount failed > (snip) > > The pools have not been upgraded after the latest openzfs import, > maybe that is related? > > FreeBSD test-freebsd-head 13.0-CURRENT FreeBSD 13.0-CURRENT #2 > r367585: > > First thing i noticed is about a week ago. I'm facing same problem with 13.0-CURRENT amd64 r367487 and virtualbox. In my case I use autofs to mount remote file system of 12.2-RELEASE amd64 server with NFSv4. When there is still filesystem mounted by autofs, then watchdog timeout happens while shutdown. The watchdog timeout can be worked around by executing `automount -fu` before shutting down. But 'cannot unmount ...' error messages are still displayed. I added 'rc_debug="YES"' to /etc/rc.conf and checked which rc script causes this message. Then it is displayed when following `zfs_stop` function of /etc/rc.d/zfs is executed. ---------------------------------------------------------------------- zfs_stop_main() { zfs unshare -a zfs unmount -a } ---------------------------------------------------------------------- At this point syslog process still running and it opens some files under /var/log. So it make sence that `zfs unmount -a` results in the message. Probably order of executing each rc script in shutdown time should be changed so `/etc/rc.d/zfs faststop` is executed after all processes other than `init' are exited. --- Yasuhiro KIMURA