From owner-freebsd-hackers@freebsd.org Sun Oct 4 21:28:11 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3DE01434357 for ; Sun, 4 Oct 2020 21:28:11 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-eopbgr670062.outbound.protection.outlook.com [40.107.67.62]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C4Gzt1BhYz4J3l for ; Sun, 4 Oct 2020 21:28:09 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ev9RhiX8dbFF/xzuFWpWSXKkWAN+SDU7cCzYmMcSXqXJt91HceLUc/ti6kA9ItgiQ6FirZnfhSBh+q5pNmHrhdl083oh4RUkd/lQvnX0FbjijmYaX7UshyFFHA/4MELHvGjt9NlKwBRov7IlTqq1AL+GzJa4x8xUQSw2vds6ZUS917tUt6JPuFlyttSMohmirEpIOFZdXhSSb8nOAbI6EKcI0tSQ/dfhhEi5QYfabELxyoX5jzGbOMrmRTlS5LTF5Y2xScy5IT+gxyMb6J2BNSkx54mSBMdHBIY2k0LiTF8UxZLD0rdA5Z4THFiugX863wszof2Lw/m9c/CgMCf+Bg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2CIL/mm0zNrScwmNNT9f3a1oWPmeqnCHDpXQBVaWPLM=; b=Abpgoo+2cFFSJrzV7wdCq8mdBA/thD3G0ZB9pOKNXwf2jxFcTXBEyFKsmnr7ADfaO7bzoOo5hWFfnXYpb0BPDaChSz1Mt1Un0AqxVI1jKvhPCWow1EpQSr6Ck7kw6wgUNW1wB8BfhWeZDtoAZtCnloqrQdzg5I0mfoSo3D0uKOQc1JAD8hyD0JcKwUqd+DxRyy/LsYDmicpkh3Af8ek6heFlP3CE9F/bHF4uOVwijcSlVqEam1NwgsPmHoYfiy8KpZV4dLUFMAPzfrKSrHnSgFih1eRUffUOwydaBpDZmeWOaW3K+CgJO+CyU0VKiwOWsntHHqRy7YDBTwUXCFfbBQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2CIL/mm0zNrScwmNNT9f3a1oWPmeqnCHDpXQBVaWPLM=; b=aHr8CAaOdrHTrQRwRIYm5tN+A2xbLhwh9T9r9+eS+6Naa81XJNOhOdlkWnv/lvOEsnpywkS3eySTHfEk3N49WTvmhkRh2L3b2jM6lSpGxnGgJ7rr9ORhQ0/QNwZiAWniAT1vlB1Pvm2cLRiG89wAhoB58ExmeB1JAWw8u02YU62Pwdu8mIf0wrF7mC59khTxlsKutFtt4tAGW3bwSb+JlUpJDHH199Gh8QwCk6tP2I6hKpWw/8aI86xqRjn6CvmddNrLKlhYI0YDqH9sj1luT0OPMQB12X8J8KmUrFWJDegeuwn0c4DwSGm73B5p8d9aNqFzBxn3BCgccDTVEZmEoQ== Received: from YTBPR01MB3966.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:24::27) by YTOPR0101MB2314.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:17::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3412.22; Sun, 4 Oct 2020 21:28:08 +0000 Received: from YTBPR01MB3966.CANPRD01.PROD.OUTLOOK.COM ([fe80::687f:d85a:a0a3:bd20]) by YTBPR01MB3966.CANPRD01.PROD.OUTLOOK.COM ([fe80::687f:d85a:a0a3:bd20%6]) with mapi id 15.20.3433.044; Sun, 4 Oct 2020 21:28:08 +0000 From: Rick Macklem To: Meowthink , "freebsd-hackers@freebsd.org" Subject: Re: How this wired boot timing bug comes, init rc scripts or zfs? Thread-Topic: How this wired boot timing bug comes, init rc scripts or zfs? Thread-Index: AQHWmfkl7m9uvwZgSESfOu7kxuLuh6mH8/LQ Date: Sun, 4 Oct 2020 21:28:08 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6f75eea7-86e9-4114-ba89-08d868ac6375 x-ms-traffictypediagnostic: YTOPR0101MB2314: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: A58hJ1r4reTaF3xcddNsad1tAL5z8hw3VpXHvio/a7ZUChgTRJi2qqTehSc70fl2/24ZEt1EpyFtdpBdy5+pkShOy1oh8KiY7bJ2No5wiHYQTc8+67/c2Q2quQdl2ukN+9yNlLoveYyevyDp0OgeQ6oQGXH/HQaN+twusPS47/bZ6P/TDKsKUhZ8an2gsF99QdhK1SaVtQFCIDMBTl4DALiSjUOItRw/kjbXbHDZGVZzB8nnDBhdseogWDO6Gd3heJW2mQMQfnalGlX9LWh7yVtwyyj5+ukqfW2GVzWJFBW3/ad2I29HbWMWc1CFnhuzh9jcbapnN4OEokj8kuoVOxP7nDmYOpLptmEKVAP+4+OqzZ9eeO1/BSOsyJYZBEL7gFTcps+ka6pf1IB1B9OKwJXroVLfEtVAgLmoHSErDEoz0Kg0PgZfOu1MdiEGzRs0x1IKIbiwJtC7VMpzaVdLYQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:YTBPR01MB3966.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(346002)(366004)(136003)(376002)(396003)(39850400004)(86362001)(9686003)(8936002)(2906002)(5660300002)(83380400001)(52536014)(66556008)(66476007)(66946007)(76116006)(110136005)(786003)(33656002)(316002)(478600001)(6506007)(7696005)(66446008)(966005)(186003)(71200400001)(8676002)(55016002)(64756008); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: 95rKlunjFEGmnmlIDSYvQboNhADSwhwk+tUZAvOHs9oGKToMZlDOLKK31rDcclW9FIVyQyOKQgp70Ly2iolXQFc7GBdbCSh3HwTCikAFwOGPaXs7fDg225x5o/gNv3F9r60tKpsZ+9t7lsj3QVcLmDIAtf76rPE198GqG4I0WlBl+P4CxsBqMcB1g5J18NG+gq3SwiXfNCy/NN/5Bge0yfGn7xr/BTZuEjEeH4l64BNQcnBKdV7rJC3WWRL7sbgRr/CWHiZbslPu3KzfdkK/X5jE7we2Ms72UcqP43D5dJsgRCRQ9a3DUqRBOxkxVoiC+1Z9ql5hAgsrc0z4ZYEeT8mzGT16mKpo1MquvzA5uujOP24Q00CZIpVg8rV1eMQWmpPd12T838GzXuUOzQniVzPSYptWcrizOTkYAsJEj4+k+YdRbvkzTSuiSXkjFQvcraOUGo8ybPjQH501oOgJg+5fHqIzq6xMXWlLx+HYMvJ+wu5hXJrapny8Uv8zW6TCptn57XXArhV+QPczEJhijJwUetN1qoV6xhsxk+gaU0tJ+rhPpNFodcVHf1gZA1KaRJWAZVSd9aLDz0FO7Dy3iJKEZ9qBuN51FF/bvvfj54qCT8spW8KcYwlWiwgvXUcIwR2LhQrSTwIDnST/CtF/UGqNTtk2KV+Lf1zDkian9NNMChS5MN2qtIk/l3YxScLU0RQn+xwxbv/I03/28QSpwA== x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: YTBPR01MB3966.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 6f75eea7-86e9-4114-ba89-08d868ac6375 X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Oct 2020 21:28:08.4549 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Y2PGLC1X5fsp0LI07BaukKPNH6CqnPi5kSJmLCUQi2HffydkhxyhvVU0/U5CeMqsd6h1JlN8OwMLZBlDcMficA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YTOPR0101MB2314 X-Rspamd-Queue-Id: 4C4Gzt1BhYz4J3l X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector1 header.b=aHr8CAaO; dmarc=pass (policy=none) header.from=uoguelph.ca; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.67.62 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-5.41 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[uoguelph.ca:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[uoguelph.ca,none]; NEURAL_HAM_SHORT(-1.33)[-1.328]; FREEMAIL_TO(0.00)[gmail.com,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US]; RCVD_IN_DNSWL_LOW(-0.10)[40.107.67.62:from]; NEURAL_HAM_MEDIUM(-0.98)[-0.983]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector1]; FREEFALL_USER(0.00)[rmacklem]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SUBJECT_ENDS_QUESTION(1.00)[]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.107.67.62:from]; MAILMAN_DEST(0.00)[freebsd-hackers] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Oct 2020 21:28:11 -0000 Meowthink wrote:=0A= >Hello hackers,=0A= >Recently I installworld and rebooted a server, seems working, but my=0A= >kerberized nfsd, precisely gssd, is not functional.=0A= >At first I thought it may be a bug from stable, so I did some trivial=0A= >tests, replacing the kernel with releng one, then the whole world, but=0A= >found this is nothing related to the kernel, and triggers randomly=0A= >when rebooting some recent stable/11 world (releng/11.4 seems fine).=0A= >To dig it deeper, here is what the console showed when failing:=0A= >```=0A= >Oct 3 20:23:59 r kernel: Starting file system checks:=0A= >Oct 3 20:23:59 r kernel: /etc/rc: WARNING: run_rc_command: cannot run=0A= >/usr/sbin/gssd=0A= >Oct 3 20:23:59 r kernel: Mounting local filesystems:.=0A= >Oct 3 20:23:59 r kernel: Updating CPU Microcode...=0A= >Oct 3 20:23:59 r kernel: Done.=0A= >Oct 3 20:23:59 r kernel: Starting ctld.=0A= >Oct 3 20:23:59 r kernel: ctld: bind(2) failed for [::]: Can't assign=0A= >requested address=0A= >Oct 3 20:23:59 r kernel: ctld: bind(2) failed for 0.0.0.0: Can't=0A= >assign requested address=0A= >Oct 3 20:23:59 r kernel: ctld: failed to apply configuration; exiting=0A= >Oct 3 20:23:59 r kernel: /etc/rc: WARNING: failed to start ctld=0A= >Oct 3 20:23:59 r kernel: ELF ldconfig path: /lib /usr/lib=0A= >/usr/lib/compat /usr/local/lib /usr/dt/lib /usr/local/lib/compat=0A= >/usr/local/lib/gcc9 /usr/local/lib/graphviz /usr/local/lib/nss=0A= >/usr/local/lib/perl5/5.28/mach/CORE /usr/local/lib/pth=0A= >/usr/local/lib/qt4 /usr/local/lib/qt5 /usr/local/lib/samba4=0A= >/usr/local/llvm10/lib /usr/local/share/chromium=0A= >Oct 3 20:23:59 r kernel: 32-bit compatibility ldconfig path:=0A= >/usr/lib32 /usr/local/lib32/compat=0A= >Oct 3 20:23:59 r kernel: Setting hostname: r.domain.net.=0A= >Oct 3 20:23:59 r kernel: Setting up harvesting:=0A= >[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,NET_ETHER,NET_TUN,MOUSE,KEYBOARD,ATT= ACH,CACHED=0A= >Oct 3 20:23:59 r kernel: Feeding entropy: .=0A= >Oct 3 20:23:59 r kernel: Starting Network: lo0 bge0 bge1.=0A= >```=0A= >Then I realized I have / and /usr in separated zfs(same zpool). It may=0A= >be that / mounted but /usr not. Thus I changed my /etc/rc.d/gssd line=0A= >7 to # REQUIRE: mountcritlocal. =0A= I don't know what has changed post-11.4 to cause this.=0A= However, the problem with adding "mountcritlocal" is that it assumes=0A= /usr is a locally mounted file system and not NFS mounted nor a subtree=0A= of "/".=0A= --> I think a better solution might be to move gssd to /sbin, which should= =0A= always be a part of the root fs. (/etc/rc.d/gssd already has "root" a= s=0A= REQUIRED.)=0A= =0A= I have a similar problem with the rpc.tlsclntd daemon I have developed=0A= for NFS-over-TLS.=0A= - Neither sec=3Dkrb5[ip] nor tls can be used for an NFS mounted root,=0A= but I think we just have to live with that?=0A= =0A= Do others have any suggestions? rick=0A= =0A= By the way, /etc/rc.d/ctld to #=0A= REQUIRE: netif. Everything works fine, even rebooting several times.=0A= What I am confused is how this happens. It seems that /etc/rc.d/gssd=0A= (in addition, /etc/rc.d/ctld) hasn't been changed since 2016. Both=0A= gssd and init in stable/11 have no functional changes since=0A= releng/11.4. Maybe zfs? but it's in the kernel, and kernel r366306=0A= with releng/11.4 world works(though I only tested few times since=0A= rebooting is too boring).=0A= Any ideas?=0A= =0A= Cheers,=0A= meowthink=0A= _______________________________________________=0A= freebsd-hackers@freebsd.org mailing list=0A= https://lists.freebsd.org/mailman/listinfo/freebsd-hackers=0A= To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"= =0A= =0A=