Date: Mon, 5 Oct 2020 00:41:39 +0000 From: Rick Macklem <rmacklem@uoguelph.ca> To: Meowthink <meowthink@gmail.com>, FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: How this wired boot timing bug comes, init rc scripts or zfs? Message-ID: <YTBPR01MB396691C2D793FE99A6E944DFDD0C0@YTBPR01MB3966.CANPRD01.PROD.OUTLOOK.COM> In-Reply-To: <CABnABobzpHyCFp-yd8EsGHCaOqkrMSvutzYSnju-rLg=DCZD4A@mail.gmail.com> References: <CABnABobDWk7DSO8vfhqBYqXp9qxiQgeLWFU8-dJsvuyoPHzj_g@mail.gmail.com> <YTBPR01MB39665C1FA31BE3E275E2C6C4DD0F0@YTBPR01MB3966.CANPRD01.PROD.OUTLOOK.COM> <CANCZdfpwV%2BocH7-JuG==JGUF67-1xOuz2GPZNqDkhAGWCJLLiQ@mail.gmail.com>, <CABnABobzpHyCFp-yd8EsGHCaOqkrMSvutzYSnju-rLg=DCZD4A@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Meowthink wrote:=0A= >On Mon, Oct 5, 2020 at 5:37 AM Warner Losh <imp@bsdimp.com> wrote:=0A= >>=0A= >>=0A= >>=0A= >> On Sun, Oct 4, 2020, 3:28 PM Rick Macklem <rmacklem@uoguelph.ca> wrote:= =0A= >>>=0A= >>> Meowthink wrote:=0A= >>> >Hello hackers,=0A= >>> >Recently I installworld and rebooted a server, seems working, but my= =0A= >>> >kerberized nfsd, precisely gssd, is not functional.=0A= >>> >At first I thought it may be a bug from stable, so I did some trivial= =0A= >>> >tests, replacing the kernel with releng one, then the whole world, but= =0A= >>> >found this is nothing related to the kernel, and triggers randomly=0A= >>> >when rebooting some recent stable/11 world (releng/11.4 seems fine).= =0A= >>> >To dig it deeper, here is what the console showed when failing:=0A= >>> >```=0A= >>> >Oct 3 20:23:59 r kernel: Starting file system checks:=0A= >>> >Oct 3 20:23:59 r kernel: /etc/rc: WARNING: run_rc_command: cannot run= =0A= >>> >/usr/sbin/gssd=0A= >>> >Oct 3 20:23:59 r kernel: Mounting local filesystems:.=0A= >>> >Oct 3 20:23:59 r kernel: Updating CPU Microcode...=0A= >>> >Oct 3 20:23:59 r kernel: Done.=0A= >>> >Oct 3 20:23:59 r kernel: Starting ctld.=0A= >>> >Oct 3 20:23:59 r kernel: ctld: bind(2) failed for [::]: Can't assign= =0A= >>> >requested address=0A= >>> >Oct 3 20:23:59 r kernel: ctld: bind(2) failed for 0.0.0.0: Can't=0A= >>> >assign requested address=0A= >>> >Oct 3 20:23:59 r kernel: ctld: failed to apply configuration; exiting= =0A= >>> >Oct 3 20:23:59 r kernel: /etc/rc: WARNING: failed to start ctld=0A= >>> >Oct 3 20:23:59 r kernel: ELF ldconfig path: /lib /usr/lib=0A= >>> >/usr/lib/compat /usr/local/lib /usr/dt/lib /usr/local/lib/compat=0A= >>> >/usr/local/lib/gcc9 /usr/local/lib/graphviz /usr/local/lib/nss=0A= >>> >/usr/local/lib/perl5/5.28/mach/CORE /usr/local/lib/pth=0A= >>> >/usr/local/lib/qt4 /usr/local/lib/qt5 /usr/local/lib/samba4=0A= >>> >/usr/local/llvm10/lib /usr/local/share/chromium=0A= >>> >Oct 3 20:23:59 r kernel: 32-bit compatibility ldconfig path:=0A= >>> >/usr/lib32 /usr/local/lib32/compat=0A= >>> >Oct 3 20:23:59 r kernel: Setting hostname: r.domain.net.=0A= >>> >Oct 3 20:23:59 r kernel: Setting up harvesting:=0A= >>> >[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,NET_ETHER,NET_TUN,MOUSE,KEYBOARD= ,ATTACH,CACHED=0A= >>> >Oct 3 20:23:59 r kernel: Feeding entropy: .=0A= >>> >Oct 3 20:23:59 r kernel: Starting Network: lo0 bge0 bge1.=0A= >>> >```=0A= >>> >Then I realized I have / and /usr in separated zfs(same zpool). It may= =0A= >>> >be that / mounted but /usr not. Thus I changed my /etc/rc.d/gssd line= =0A= >>> >7 to # REQUIRE: mountcritlocal.=0A= >>> I don't know what has changed post-11.4 to cause this.=0A= >>> However, the problem with adding "mountcritlocal" is that it assumes=0A= >>> /usr is a locally mounted file system and not NFS mounted nor a subtree= =0A= >>> of "/".=0A= >>> --> I think a better solution might be to move gssd to /sbin, which sho= uld=0A= >>> always be a part of the root fs. (/etc/rc.d/gssd already has "roo= t" as=0A= >>> REQUIRED.)=0A= >>=0A= >>=0A= >> Make the move. It is an early utility in reality.=0A= >>=0A= >> Warner=0A= >=0A= >But take care :)=0A= >$ ldd /usr/sbin/gssd=0A= >/usr/sbin/gssd:=0A= > libgssapi.so.10 =3D> /usr/lib/libgssapi.so.10 (0x80082b000)=0A= > libkrb5.so.11 =3D> /usr/lib/libkrb5.so.11 (0x800a35000)=0A= > libroken.so.11 =3D> /usr/lib/libroken.so.11 (0x800cb4000)=0A= > libc.so.7 =3D> /lib/libc.so.7 (0x800ec7000)=0A= > libasn1.so.11 =3D> /usr/lib/libasn1.so.11 (0x80127e000)=0A= > libcom_err.so.5 =3D> /usr/lib/libcom_err.so.5 (0x801521000)=0A= > libcrypt.so.5 =3D> /lib/libcrypt.so.5 (0x801723000)=0A= > libcrypto.so.8 =3D> /lib/libcrypto.so.8 (0x801a00000)=0A= > libhx509.so.11 =3D> /usr/lib/libhx509.so.11 (0x801e75000)=0A= > libwind.so.11 =3D> /usr/lib/libwind.so.11 (0x8020c2000)=0A= > libheimbase.so.11 =3D> /usr/lib/libheimbase.so.11 (0x8022ea000)=0A= > libprivateheimipcc.so.11 =3D> /usr/lib/libprivateheimipcc.so.11=0A= >(0x8024ee000)=0A= > libthr.so.3 =3D> /lib/libthr.so.3 (0x8026f1000)=0A= Good point. Moving all those libraries isn't worth the effort, imho.=0A= =0A= I suppose "mountcritlocal" is harmless and fixes the case where /usr=0A= is a separately mounted local FS.=0A= (I can't resist pointing out that we no longer need to worry about the=0A= size limitation of a 2.5Mbyte RK05 disk, so having /usr on a separate=0A= file system may no longer be a critical requirement, but...;-)=0A= =0A= rick=0A= =0A= >>=0A= >>=0A= >> I have a similar problem with the rpc.tlsclntd daemon I have developed= =0A= >> for NFS-over-TLS.=0A= >> - Neither sec=3Dkrb5[ip] nor tls can be used for an NFS mounted root,=0A= >> but I think we just have to live with that?=0A= >>=0A= >> Do others have any suggestions? rick=0A= >>=0A= >> By the way, /etc/rc.d/ctld to #=0A= >> REQUIRE: netif. Everything works fine, even rebooting several times.=0A= >> What I am confused is how this happens. It seems that /etc/rc.d/gssd=0A= >> (in addition, /etc/rc.d/ctld) hasn't been changed since 2016. Both=0A= >> gssd and init in stable/11 have no functional changes since=0A= >> releng/11.4. Maybe zfs? but it's in the kernel, and kernel r366306=0A= >> with releng/11.4 world works(though I only tested few times since=0A= >> rebooting is too boring).=0A= >> Any ideas?=0A= >>=0A= >> Cheers,=0A= >> meowthink=0A= >> _______________________________________________=0A= >> freebsd-hackers@freebsd.org mailing list=0A= >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers=0A= >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.or= g"=0A= >>=0A= >> _______________________________________________=0A= >> freebsd-hackers@freebsd.org mailing list=0A= >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers=0A= >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.or= g"=0A= _______________________________________________=0A= freebsd-hackers@freebsd.org mailing list=0A= https://lists.freebsd.org/mailman/listinfo/freebsd-hackers=0A= To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"= =0A= =0A=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?YTBPR01MB396691C2D793FE99A6E944DFDD0C0>