From owner-freebsd-hackers@freebsd.org Sun Oct 4 02:50:34 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E05163F75C3 for ; Sun, 4 Oct 2020 02:50:34 +0000 (UTC) (envelope-from meowthink@gmail.com) Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [IPv6:2a00:1450:4864:20::62d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C3pBK5RmGz45tn for ; Sun, 4 Oct 2020 02:50:33 +0000 (UTC) (envelope-from meowthink@gmail.com) Received: by mail-ej1-x62d.google.com with SMTP id ly6so5729096ejb.8 for ; Sat, 03 Oct 2020 19:50:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=GMlKA2EBEm++4GrxGaPBu7Cg90Pt+rZCspf6D89X5Yk=; b=SgcVlDkmyrjZBWafoU5J4Vs2F/G9KWsFb+BQnPD+L8h2No2tCiQK61Gpov60gtq4QC GEgnlmEQXmylTLDMz4WtjPj/cUoLErJSqCo7CNRO80xPuskZRtzjEpgT+s2IK5Djdox6 CFhUB5RLIL4lU8yq/M92z/5vbjTHuZeKenAAvdSL1btGhwjDT2QFro3cSQbSWCH6K20n Iy+j+Tn++vBoQWp6fJI9Ub04MvApMLh9t58IzoYUkOLGtJrn1JpOHRpCJx7kqiBk6PVR 6nawqbc/6BPNBuEkqV1xvcTVYi2e2YyKi7OhD3G9evMdBraYTgw7tiQOUnuJHrjdCWus PtEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=GMlKA2EBEm++4GrxGaPBu7Cg90Pt+rZCspf6D89X5Yk=; b=CpHong4SFhN605S5RtgCQfLI6w1T9dEHyv2fA8ipTq8GC3MQpEHD78xYyFqRoqrNJa 2+dU9I3oQFlpcGucRqZZUPgyYxq493n6Lze3mlVXGsW+uetOeWYwuYny6D+cA0kzDb7L HbRAIYlvlQjIYXPuw2TeGNknMQZVbxdE56YKF7odhp10kki4IIIPv81kdWmWQPwuwaaZ RgWBTQDJPOZUEOFAu38imi1a3uRrnWdIOwPg8w4WL8pwiV9ApgJSzeELhjxXQbZqhY/S 7Cqwus7N7eE+huw2H73OK4m3olNpfF038ACDZTQ2ynQ8HR2InOEN1GR5DUMT3q3u6t4e AMPA== X-Gm-Message-State: AOAM5311gciHoPuNcR9NM2yVPA36fXJvUbnUYZFWmXp9cu6FDZcJIMxJ oSJAdiVfzC5l7/HCJlvWbAN+VqM87thUV1rpgOyOxLTml9M= X-Google-Smtp-Source: ABdhPJxVRBq4xTCHAr2GrTvqcmcnXLwegM167dak3YuqSFxbqG25GrWLG+iUwDXdqit2aFnO2U1AcMsG1CFf5wDuNI4= X-Received: by 2002:a17:906:6a07:: with SMTP id o7mr8787866ejr.454.1601779831788; Sat, 03 Oct 2020 19:50:31 -0700 (PDT) MIME-Version: 1.0 From: Meowthink Date: Sun, 4 Oct 2020 10:50:20 +0800 Message-ID: Subject: How this wired boot timing bug comes, init rc scripts or zfs? To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4C3pBK5RmGz45tn X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=SgcVlDkm; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of meowthink@gmail.com designates 2a00:1450:4864:20::62d as permitted sender) smtp.mailfrom=meowthink@gmail.com X-Spamd-Result: default: False [-2.20 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.93)[-0.928]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.02)[-1.021]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62d:from]; NEURAL_HAM_SHORT(-0.25)[-0.246]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; SUBJECT_ENDS_QUESTION(1.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Oct 2020 02:50:34 -0000 Hello hackers, Recently I installworld and rebooted a server, seems working, but my kerberized nfsd, precisely gssd, is not functional. At first I thought it may be a bug from stable, so I did some trivial tests, replacing the kernel with releng one, then the whole world, but found this is nothing related to the kernel, and triggers randomly when rebooting some recent stable/11 world (releng/11.4 seems fine). To dig it deeper, here is what the console showed when failing: ``` Oct 3 20:23:59 r kernel: Starting file system checks: Oct 3 20:23:59 r kernel: /etc/rc: WARNING: run_rc_command: cannot run /usr/sbin/gssd Oct 3 20:23:59 r kernel: Mounting local filesystems:. Oct 3 20:23:59 r kernel: Updating CPU Microcode... Oct 3 20:23:59 r kernel: Done. Oct 3 20:23:59 r kernel: Starting ctld. Oct 3 20:23:59 r kernel: ctld: bind(2) failed for [::]: Can't assign requested address Oct 3 20:23:59 r kernel: ctld: bind(2) failed for 0.0.0.0: Can't assign requested address Oct 3 20:23:59 r kernel: ctld: failed to apply configuration; exiting Oct 3 20:23:59 r kernel: /etc/rc: WARNING: failed to start ctld Oct 3 20:23:59 r kernel: ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/dt/lib /usr/local/lib/compat /usr/local/lib/gcc9 /usr/local/lib/graphviz /usr/local/lib/nss /usr/local/lib/perl5/5.28/mach/CORE /usr/local/lib/pth /usr/local/lib/qt4 /usr/local/lib/qt5 /usr/local/lib/samba4 /usr/local/llvm10/lib /usr/local/share/chromium Oct 3 20:23:59 r kernel: 32-bit compatibility ldconfig path: /usr/lib32 /usr/local/lib32/compat Oct 3 20:23:59 r kernel: Setting hostname: r.domain.net. Oct 3 20:23:59 r kernel: Setting up harvesting: [UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,NET_ETHER,NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED Oct 3 20:23:59 r kernel: Feeding entropy: . Oct 3 20:23:59 r kernel: Starting Network: lo0 bge0 bge1. ``` Then I realized I have / and /usr in separated zfs(same zpool). It may be that / mounted but /usr not. Thus I changed my /etc/rc.d/gssd line 7 to # REQUIRE: mountcritlocal. By the way, /etc/rc.d/ctld to # REQUIRE: netif. Everything works fine, even rebooting several times. What I am confused is how this happens. It seems that /etc/rc.d/gssd (in addition, /etc/rc.d/ctld) hasn't been changed since 2016. Both gssd and init in stable/11 have no functional changes since releng/11.4. Maybe zfs? but it's in the kernel, and kernel r366306 with releng/11.4 world works(though I only tested few times since rebooting is too boring). Any ideas? Cheers, meowthink