From owner-freebsd-hackers@freebsd.org Sun Oct 4 21:38:01 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6AB3A43433B for ; Sun, 4 Oct 2020 21:38:01 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C4HCC5NLTz4K0j for ; Sun, 4 Oct 2020 21:37:59 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x841.google.com with SMTP id j22so7858865qtj.8 for ; Sun, 04 Oct 2020 14:37:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=M1WdzsNlmmChkEbJ54jlYFOVt+gtDzgMOm+n7d6sSMI=; b=TQomG56X3rcWRkSFybvEWHS8veg9JWBGWUyMJ/lA/VlNmrSKOWxCZAIgjDQkzysu+k WpVqZldkpbuBsfcVk+9dyTGA3IvOK0nMp1JqFg8leyJ/yZGiT508wE1T8VkMbTxeK5R8 sydp4GkWNLS6j140X0zI9fxiCBse7X48DoWIKq15zk4w1XzkWupUWzru/YS0wt6KyTjA a+X45I1rPiVoE0+/8Y+UWXH4N1VI/io9xzDBU2TuKqa8E307tVwD6tjux0dqEHh8T4RL Orv2RlKe5GGiQjqRyvZkmkP865NgrKerUGeSYyuehg5U7YLtBmxVzgOJY4A6hm47Nz1S Zt3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=M1WdzsNlmmChkEbJ54jlYFOVt+gtDzgMOm+n7d6sSMI=; b=Yn+upr4RuMg4J7wEMA3zOJtNsdOxDwxN2xTw34tFfjhGHTJfnRnHSPNKups3RClPfs f/sPTuKLsapkL3/2OZtfSyTSRXBOad2BwICdo9PVNoKjX63nTkPpA8oJHwXXpNoXcYxo 5n5LwByf0XxlL2SXiTNSMShGTTXu4Ow3/CJRxdfNhyDFVmJMqHYfEOlf3NJwdY4x5Z3d KhucMKx9lWyYQsDP5ZwWModXNbwifuOF4aZfAM/++cr4bc6dF+xg4cT2dNWKv2P8yanh yGcgv8PC4jAk0RQe2FDGy+ovkQE3momIjN9ns9d7V9EL30W+oG2KQyO+kofTV5VAk63e YW3g== X-Gm-Message-State: AOAM530SL1RYkljJ7mhVFc933aHbzS5OsHCXPLXDQOe/Ay2a8+GqGLB/ ed5RLKUDtVBxOMmcbTjv31+sXKjl+1OTe3QGRrSu9Q== X-Google-Smtp-Source: ABdhPJxiUK10DP7igU0qd3SbTCYSsP/P77nmcDDE6KAkq4DR3Nxp6wBhPxT2i4/+vj+6IKp573GXHq6k+H1NK9N6dgA= X-Received: by 2002:ac8:4cd0:: with SMTP id l16mr11356581qtv.175.1601847478464; Sun, 04 Oct 2020 14:37:58 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Sun, 4 Oct 2020 15:37:46 -0600 Message-ID: Subject: Re: How this wired boot timing bug comes, init rc scripts or zfs? To: Rick Macklem Cc: Meowthink , FreeBSD Hackers X-Rspamd-Queue-Id: 4C4HCC5NLTz4K0j X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=TQomG56X; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::841) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [1.35 / 15.00]; URI_COUNT_ODD(1.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.88)[-0.876]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(0.16)[0.162]; NEURAL_HAM_LONG(-0.93)[-0.934]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::841:from]; HTTP_TO_IP(1.00)[]; R_SPF_NA(0.00)[no SPF record]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Oct 2020 21:38:01 -0000 On Sun, Oct 4, 2020, 3:28 PM Rick Macklem wrote: > Meowthink wrote: > >Hello hackers, > >Recently I installworld and rebooted a server, seems working, but my > >kerberized nfsd, precisely gssd, is not functional. > >At first I thought it may be a bug from stable, so I did some trivial > >tests, replacing the kernel with releng one, then the whole world, but > >found this is nothing related to the kernel, and triggers randomly > >when rebooting some recent stable/11 world (releng/11.4 seems fine). > >To dig it deeper, here is what the console showed when failing: > >``` > >Oct 3 20:23:59 r kernel: Starting file system checks: > >Oct 3 20:23:59 r kernel: /etc/rc: WARNING: run_rc_command: cannot run > >/usr/sbin/gssd > >Oct 3 20:23:59 r kernel: Mounting local filesystems:. > >Oct 3 20:23:59 r kernel: Updating CPU Microcode... > >Oct 3 20:23:59 r kernel: Done. > >Oct 3 20:23:59 r kernel: Starting ctld. > >Oct 3 20:23:59 r kernel: ctld: bind(2) failed for [::]: Can't assign > >requested address > >Oct 3 20:23:59 r kernel: ctld: bind(2) failed for 0.0.0.0: Can't > >assign requested address > >Oct 3 20:23:59 r kernel: ctld: failed to apply configuration; exiting > >Oct 3 20:23:59 r kernel: /etc/rc: WARNING: failed to start ctld > >Oct 3 20:23:59 r kernel: ELF ldconfig path: /lib /usr/lib > >/usr/lib/compat /usr/local/lib /usr/dt/lib /usr/local/lib/compat > >/usr/local/lib/gcc9 /usr/local/lib/graphviz /usr/local/lib/nss > >/usr/local/lib/perl5/5.28/mach/CORE /usr/local/lib/pth > >/usr/local/lib/qt4 /usr/local/lib/qt5 /usr/local/lib/samba4 > >/usr/local/llvm10/lib /usr/local/share/chromium > >Oct 3 20:23:59 r kernel: 32-bit compatibility ldconfig path: > >/usr/lib32 /usr/local/lib32/compat > >Oct 3 20:23:59 r kernel: Setting hostname: r.domain.net. > >Oct 3 20:23:59 r kernel: Setting up harvesting: > > >[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,NET_ETHER,NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED > >Oct 3 20:23:59 r kernel: Feeding entropy: . > >Oct 3 20:23:59 r kernel: Starting Network: lo0 bge0 bge1. > >``` > >Then I realized I have / and /usr in separated zfs(same zpool). It may > >be that / mounted but /usr not. Thus I changed my /etc/rc.d/gssd line > >7 to # REQUIRE: mountcritlocal. > I don't know what has changed post-11.4 to cause this. > However, the problem with adding "mountcritlocal" is that it assumes > /usr is a locally mounted file system and not NFS mounted nor a subtree > of "/". > --> I think a better solution might be to move gssd to /sbin, which should > always be a part of the root fs. (/etc/rc.d/gssd already has "root" > as > REQUIRED.) > Make the move. It is an early utility in reality. Warner > > I have a similar problem with the rpc.tlsclntd daemon I have developed > for NFS-over-TLS. > - Neither sec=krb5[ip] nor tls can be used for an NFS mounted root, > but I think we just have to live with that? > > Do others have any suggestions? rick > > By the way, /etc/rc.d/ctld to # > REQUIRE: netif. Everything works fine, even rebooting several times. > What I am confused is how this happens. It seems that /etc/rc.d/gssd > (in addition, /etc/rc.d/ctld) hasn't been changed since 2016. Both > gssd and init in stable/11 have no functional changes since > releng/11.4. Maybe zfs? but it's in the kernel, and kernel r366306 > with releng/11.4 world works(though I only tested few times since > rebooting is too boring). > Any ideas? > > Cheers, > meowthink > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" >