Date: Wed, 30 Mar 2022 20:59:24 -0700 From: Kevin Oberman <rkoberman@gmail.com> To: Peter <pmc@citylink.dinoex.sub.org> Cc: "Bjoern A. Zeeb" <bzeeb-lists@lists.zabbadoz.net>, FreeBSD-STABLE Mailing List <freebsd-stable@freebsd.org> Subject: Re: Slow startup from D19488 (rtsol: sendmsg: Permission denied) Message-ID: <CAN6yY1uhryDh7b-TRLNCXeSmPMCMBva_NXaDackfpSgbgP9FOA@mail.gmail.com> In-Reply-To: <YkOcv0SQ5Wlr/6Qt@gate.intra.daemon.contact> References: <YkN2acB17mOkMlF5@gate.intra.daemon.contact> <alpine.BSF.2.00.2203292154240.68830@ai.fobar.qr> <YkOcv0SQ5Wlr/6Qt@gate.intra.daemon.contact>
next in thread | previous in thread | raw e-mail | index | archive | help
--00000000000046dd3e05db7bb37a Content-Type: text/plain; charset="UTF-8" On Tue, Mar 29, 2022 at 5:10 PM Peter <pmc@citylink.dinoex.sub.org> wrote: > > Hello Bjoern, > > thanks much for the quick reply! > > On Tue, Mar 29, 2022 at 10:04:11PM +0000, Bjoern A. Zeeb wrote: > ! On Tue, 29 Mar 2022, Peter wrote: > ! > ! Hi, > ! > ! I am a bit puzzled as after two years you are the first one to report > ! that problem to my knowledge for either base system or jails. > > This is what greatly wonders me, too. So I was stronly thinking > that I am doing something wrong or unusual. But I cannot figure > it out, it just seems that the detrimental effect of the change > cannot be avoided (e.g. "service jail start" takes quite long now - > there's a lot of them). > > ! > after upgrading 12.3 to stable/13, I am seeing these > ! > errors in all my jails: > ! > > ! > > Additional TCP/IP options: log_in_vain=1. > ! > > ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib > ! > /usr/local/lib/c cmpat/pkg /usr/local/lib/compat/pkg > ! > > 32-bit compatibility ldconfig path: > ! > > rtsol: sendmsg on nrail1l: Permission denied > ! > > rtsol: sendmsg on nrail1l: Permission denied > ! > > rtsol: sendmsg on nrail1l: Permission denied > ! > > Starting Network: lo0 nrail1l. > ! > ! Can you give us a full startup log? > > It's the above, right from the beginning, and then follows: > > > lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 > > options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> > > inet 127.0.0.1 netmask 0xff000000 > > inet6 ::1 prefixlen 128 > > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 > > groups: lo > > nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> > > nrail1l: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu > 1500 > > options=28<VLAN_MTU,JUMBO_MTU> > > ether 06:1d:92:01:01:0a > > hwaddr 58:9c:fc:10:28:71 > > inet ************* netmask ********** broadcast ************* > > inet6 fe80::41d:92ff:fe01:10a%nrail1l prefixlen 64 scopeid 0x2 > > inet6 fd00:************ prefixlen 120 > > media: Ethernet autoselect (1000baseT <full-duplex>) > > status: active > > nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL> > > Starting rtsold. > > add host 127.0.0.1: gateway lo0 fib 0: route already in table > > add net default: gateway ************* > > Additional inet routing options: log ICMP redirect=YES. > > add host ::1: gateway lo0 fib 0: route already in table > > add net fe80::: gateway ::1 > > add net ff02::: gateway ::1 > > add net ::ffff:0.0.0.0: gateway ::1 > > add net ::0.0.0.0: gateway ::1 > > add net default: gateway fd00:************* > > Flushed all rules. > > Firewall rules loaded. > > Firewall logging pseudo-interface (ipfw0) created. > > Creating and/or trimming log files. > > Updating /var/run/os-release done. > > Clearing /tmp (X related). > > Updating motd:. > > Starting syslogd. > > Starting rapp. > > Starting cron. > > Starting sendmail. > > Starting sendmail_msp_queue. > > Performing sanity check on sshd configuration. > > Starting sshd. > > > > Wed Mar 30 00:52:15 CEST 2022 > > ! > Searching the cause I find change 1b5be7204eaeeaf aka D19488 > ! > > ! > This doesn't work, because the firewall is not yet present. This is > ! > ! Given you are talking firewall, I assume you are using vnet jails? > > Yes. > > ! And given you are talking ipfw I assume your default policy is deny > ! and not accept? > > Yes. > > ! And given rtsol runs I assume you have IPv6 configured and in use? > > Yes. Here is how I do it: > https://daemon.contact/ankh/articles/X3OyjgTpuv > > ! The same issue then should also happen in your base system on boot? > > No. The base system does (second level) prefix delegation and has > ipv6_gateway_enable="YES" and rtsold_enable="NO" and is not affected. > > There is one vnet jail intended as VPN server, which also has these > parameters in rc.conf and is also not affected. > > (I did not yet bother to figure out why, The shell code run from > rc.d/netif is a bit lenghty...) > > ! > happening in rc.d/netif, and that must run before rc.d/ipfw in any > ! > case, because the firewall needs to see the netifs. > ! > ! I thought ipfw could log deal with interfaces coming and going? > > Maybe it can, but then modifying the rc.d logic so to get "ipfw" run > before "netif" - that does likely open a box of worms. > > Furthermore, I do use ipfw as a genuine rerouting+filtering > framework, and that logic is entirely based on the interfaces; all > rules belong to exactly two interfaces. Here is a short abstract > of the idea: > https://forums.freebsd.org/threads/ipfw-or-pf.46706/post-561760 > > > cheerio, > PMc > > This may be irrelevant, but updating to the stable branch is not recommended as it is not regularly tested. Updating to 13.0-Release and then to stable is less likely to be problematic. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 --00000000000046dd3e05db7bb37a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><div class=3D"gmail_default" style=3D"fon= t-family:tahoma,sans-serif;font-size:small">On Tue, Mar 29, 2022 at 5:10 PM= Peter <<a href=3D"mailto:pmc@citylink.dinoex.sub.org">pmc@citylink.dino= ex.sub.org</a>> wrote:<br></div></div><div class=3D"gmail_quote"><blockq= uote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1p= x solid rgb(204,204,204);padding-left:1ex"><br> Hello Bjoern,<br> <br> =C2=A0 thanks much for the quick reply!<br> <br> On Tue, Mar 29, 2022 at 10:04:11PM +0000, Bjoern A. Zeeb wrote:<br> ! On Tue, 29 Mar 2022, Peter wrote:<br> ! <br> ! Hi,<br> ! <br> ! I am a bit puzzled as after two years you are the first one to report<br> ! that problem to my knowledge for either base system or jails.<br> <br> This is what greatly wonders me, too. So I was stronly thinking<br> that I am doing something wrong or unusual. But I cannot figure<br> it out, it just seems that the detrimental effect of the change<br> cannot be avoided (e.g. "service jail start" takes quite long now= -<br> there's a lot of them).<br> <br> ! >=C2=A0 after upgrading 12.3 to stable/13, I am seeing these<br> ! > errors in all my jails:<br> ! > <br> ! > > Additional TCP/IP options: log_in_vain=3D1.<br> ! > > ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib= <br> ! >=C2=A0 =C2=A0 =C2=A0/usr/local/lib/c cmpat/pkg /usr/local/lib/compat/= pkg<br> ! > > 32-bit compatibility ldconfig path:<br> ! > > rtsol: sendmsg on nrail1l: Permission denied<br> ! > > rtsol: sendmsg on nrail1l: Permission denied<br> ! > > rtsol: sendmsg on nrail1l: Permission denied<br> ! > > Starting Network: lo0 nrail1l.<br> ! <br> ! Can you give us a full startup log?<br> <br> It's the above, right from the beginning, and then follows:<br> <br> > lo0: flags=3D8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16= 384<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0options=3D680003<RXCSUM,TXCSUM,LIN= KSTATE,RXCSUM_IPV6,TXCSUM_IPV6><br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0inet 127.0.0.1 netmask 0xff000000<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0inet6 ::1 prefixlen 128<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0inet6 fe80::1%lo0 prefixlen 64 scopei= d 0x1<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0groups: lo<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0nd6 options=3D21<PERFORMNUD,AUTO_L= INKLOCAL><br> > nrail1l: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> me= tric 0 mtu 1500<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0options=3D28<VLAN_MTU,JUMBO_MTU>= ;<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ether 06:1d:92:01:01:0a<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0hwaddr 58:9c:fc:10:28:71<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0inet ************* netmask **********= broadcast *************<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0inet6 fe80::41d:92ff:fe01:10a%nrail1l= prefixlen 64 scopeid 0x2<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0inet6 fd00:************ prefixlen 120= <br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0media: Ethernet autoselect (1000baseT= <full-duplex>)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0status: active<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0nd6 options=3D23<PERFORMNUD,ACCEPT= _RTADV,AUTO_LINKLOCAL><br> > Starting rtsold.<br> > add host <a href=3D"http://127.0.0.1" rel=3D"noreferrer" target=3D"_bl= ank">127.0.0.1</a>: gateway lo0 fib 0: route already in table<br> > add net default: gateway *************<br> > Additional inet routing options: log ICMP redirect=3DYES.<br> > add host ::1: gateway lo0 fib 0: route already in table<br> > add net fe80::: gateway ::1<br> > add net ff02::: gateway ::1<br> > add net ::ffff:<a href=3D"http://0.0.0.0" rel=3D"noreferrer" target=3D= "_blank">0.0.0.0</a>: gateway ::1<br> > add net ::<a href=3D"http://0.0.0.0" rel=3D"noreferrer" target=3D"_bla= nk">0.0.0.0</a>: gateway ::1<br> > add net default: gateway fd00:*************<br> > Flushed all rules.<br> > Firewall rules loaded.<br> > Firewall logging pseudo-interface (ipfw0) created.<br> > Creating and/or trimming log files.<br> > Updating /var/run/os-release done.<br> > Clearing /tmp (X related).<br> > Updating motd:.<br> > Starting syslogd.<br> > Starting rapp.<br> > Starting cron.<br> > Starting sendmail.<br> > Starting sendmail_msp_queue.<br> > Performing sanity check on sshd configuration.<br> > Starting sshd.<br> > <br> > Wed Mar 30 00:52:15 CEST 2022<br> <br> ! > Searching the cause I find change=C2=A0 1b5be7204eaeeaf=C2=A0 aka=C2= =A0 D19488<br> ! > <br> ! > This doesn't work, because the firewall is not yet present. This= is<br> ! <br> ! Given you are talking firewall, I assume you are using vnet jails?<br> <br> Yes.<br> <br> ! And given you are talking ipfw I assume your default policy is deny<br> ! and not accept?<br> <br> Yes.<br> <br> ! And given rtsol runs I assume you have IPv6 configured and in use?<br> <br> Yes. Here is how I do it:<br> <a href=3D"https://daemon.contact/ankh/articles/X3OyjgTpuv" rel=3D"noreferr= er" target=3D"_blank">https://daemon.contact/ankh/articles/X3OyjgTpuv</a><b= r> <br> ! The same issue then should also happen in your base system on boot?<br> <br> No. The base system does (second level) prefix delegation and has<br> ipv6_gateway_enable=3D"YES" and rtsold_enable=3D"NO" an= d is not affected.<br> <br> There is one vnet jail intended as VPN server, which also has these<br> parameters in rc.conf and is also not affected.<br> <br> (I did not yet bother to figure out why, The shell code run from<br> rc.d/netif is a bit lenghty...)<br> <br> ! > happening in rc.d/netif, and that must run before rc.d/ipfw in any<b= r> ! > case, because the firewall needs to see the netifs.<br> ! <br> ! I thought ipfw could log deal with interfaces coming and going?<br> <br> Maybe it can, but then modifying the rc.d logic so to get "ipfw" = run<br> before "netif" - that does likely open a box of worms.<br> <br> Furthermore, I do use ipfw as a genuine rerouting+filtering<br> framework, and that logic is entirely based on the interfaces; all<br> rules belong to exactly two interfaces. Here is a short abstract<br> of the idea:<br> <a href=3D"https://forums.freebsd.org/threads/ipfw-or-pf.46706/post-561760"= rel=3D"noreferrer" target=3D"_blank">https://forums.freebsd.org/threads/ip= fw-or-pf.46706/post-561760</a><br> <br> <br> cheerio,<br> PMc<br> <br> </blockquote></div><div style=3D"font-family:tahoma,sans-serif;font-size:sm= all" class=3D"gmail_default">This may be irrelevant, but updating to the st= able branch is not recommended as it is not regularly tested. Updating to 1= 3.0-Release and then to stable is less likely to be problematic. <br></div>= -- <br><div dir=3D"ltr" class=3D"gmail_signature"><div dir=3D"ltr"><div><di= v dir=3D"ltr"><div><div dir=3D"ltr"><div><div dir=3D"ltr">Kevin Oberman, Pa= rt time kid herder and retired Network Engineer<br>E-mail: <a href=3D"mailt= o:rkoberman@gmail.com" target=3D"_blank">rkoberman@gmail.com</a><br></div><= div>PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683</div></div></= div></div></div></div></div></div></div> --00000000000046dd3e05db7bb37a--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAN6yY1uhryDh7b-TRLNCXeSmPMCMBva_NXaDackfpSgbgP9FOA>