Date: Sat, 29 Oct 2022 07:16:46 +0200 (CEST) From: Ronald Klop <ronald-lists@klop.ws> To: mike@karels.net, freebsd-arm@freebsd.org Subject: Re: wake-on-lan lost from rpi4 (works on rpi3) Message-ID: <2086140007.15494.1667020606677@localhost> In-Reply-To: <1539824734.231347.1666722708425@localhost>
next in thread | previous in thread | raw e-mail | index | archive | help
------=_Part_15493_225212243.1667020606673
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Van: Ronald Klop <ronald-lists@klop.ws>
Datum: 25 oktober 2022 20:32
Aan: mike@karels.net
CC: freebsd-arm@freebsd.org
Onderwerp: Re: wake-on-lan lost from rpi4 (works on rpi3)
>
>
>
>
> Van: Mike Karels
> Datum: dinsdag, 25 oktober 2022 18:11
> Aan: Ronald Klop
> CC: freebsd-arm@freebsd.org
> Onderwerp: Re: wake-on-lan lost from rpi4 (works on rpi3)
>>
>> I wrote:
>> > On 25 Oct 2022, at 9:13, Ronald Klop wrote:
>>
>> > > Van: Mike Karels
>> > > Datum: dinsdag, 25 oktober 2022 14:55
>> > > Aan: Ronald Klop
>> > > CC: freebsd-arm@freebsd.org
>> > > Onderwerp: Re: wake-on-lan lost from rpi4 (works on rpi3)
>> > >>
>> > >> On 25 Oct 2022, at 6:57, Ronald Klop wrote:
>> > >>
>> > >>> Hi,
>> > >>>
>> > >>> When I do "wake ue0 `cat /home/ronald/freenas.ethernet `" on my RPI3B+ my NAS boots and tcpdump shows the following line on my rpi3 as well as on my router.
>> > >>> 13:39:44.344111 IPX 00217046.6c:da:00:21:70:46.6cda > 00217046.6c:da:00:21:70:46.6cda: ipx-#6cda 65505
>> > >>>
>> > >>> When I do "wake genet0 `cat /home/ronald/freenas.ethernet `" on my RPI4 my NAS does not boot and tcpdump only show this on my rpi4 and *not* on my router.
>> > >>> 13:37:26.448251 IPX 00217046.6c:da:00:21:70:46.6cda > 00217046.6c:da:00:21:70:46.6cda: ipx-#6cda 65505
>> > >>>
>> > >>> Firewall ipfw does not indicate that it blocks anything.
>> > >>>
>> > >>> RPI4 runs:
>> > >>> FreeBSD rpi4 14.0-CURRENT FreeBSD 14.0-CURRENT #11 main-4f0c9b76cf: Sat Aug 13 23:59:19 CEST 2022 ronald@rpi4:/home/ronald/dev/obj/home/ronald/dev/freebsd/arm64.aarch64/sys/GENERIC-NODEBUG arm64
>> > >>>
>> > >>> genet0: flags=8943 metric 0 mtu 1500
>> > >>> options=68000b
>> > >>>
>> > >>>
>> > >>> RPI3 runs:
>> > >>> FreeBSD rpi3 13.1-RELEASE-p2 FreeBSD 13.1-RELEASE-p2 GENERIC arm64
>> > >>>
>> > >>> ue0: flags=8943 metric 0 mtu 1500
>> > >>> options=80009
>> > >>>
>> > >>>
>> > >>> Does genet0 not support these packages?
>> > >>> What can prevent this packet to go on the ethernet while tcpdump still shows it is outgoing on the interface?
>> > >>> genet0 does have a vlan configured connected to a bridge0 for some jails
>> > >>> vlan3: flags=8943 metric 0 mtu 1500
>> > >>> options=80000
>> > >>> groups: vlan
>> > >>> vlan: 3 vlanproto: 802.1q vlanpcp: 0 parent interface: genet0
>> > >>>
>> > >>> ue0 is itself connected to a bridge0
>> > >>
>> > >> Is the RPI3 also on a vlan and bridge?
>> > >>
>> > >> Is the router (or switch) expecting the packet on the vlan?
>> > >> If so, maybe you need to send on vlan3 rather than genet0.
>> > >>
>> > >> The genet interface has some oddities about packet layouts;
>> > >> that could be hitting here. You could try disabling TCP TX
>> > >> checksum offload: ifconfig genet0 -txcsum -txcsum6; that
>> > >> simplifies some of the issues.
>> > >>
>> > >> Mike
>> > >>
>> > >>> Regards,
>> > >>> Ronald.
>> > >
>> > >
>> > > Hi,
>> > >
>> > > Thanks for the hint. I experimented some further.
>> > >
>> > > Disabling -txcsum -rxcsum didn't matter.
>>
>> > -rxcsum doesn't matter, but you need to do -txcsum6 as well as
>> > -txcsum. (See the code that calls gen_parse_tx.)
>>
>> > > But when I use bridge0 or vlan3 as device then it goes into the wire but on the VLAN which is not where my NAS is which I want to boot.
>> > >
>> > > Looking at the driver I found this interesting peace of code.
>> > >
>> > > static int
>> > > gen_parse_tx(struct mbuf *m, int csum_flags) {
>> > > ...
>> > > if (ether_type == ETHERTYPE_IP) {
>> > > COPY(((struct ip *)p)->ip_hl << 2);
>> > > offset += ((struct ip *)p)->ip_hl << 2;
>> > > } else if (ether_type == ETHERTYPE_IPV6) {
>> > > COPY(sizeof(struct ip6_hdr));
>> > > offset += sizeof(struct ip6_hdr);
>> > > } else {
>> > > /*
>> > > * Unknown whether other cases require moving a header;
>> > > * ARP works without.
>> > > */
>> > > }
>> > > ...
>> > > }
>> > >
>> > > There is also some code which handles EHTERTYPE_VLAN.
>> > >
>> > > I don't have time to start debugging this today. But would this ring a bell to anybody in connection to a wake-on-lan packet?
>>
>> > I ran some experiments with tx checksum disabled, and it seems to
>> > matter. I have a change for the last block you listed that might
>> > help, but I haven't tested it yet.
>>
>> This patch seems to work:
>>
>> diff --git a/sys/arm64/broadcom/genet/if_genet.c b/sys/arm64/broadcom/genet/if_genet.c
>> index 327af8acbcdd..b7ab86e7931d 100644
>> --- a/sys/arm64/broadcom/genet/if_genet.c
>> +++ b/sys/arm64/broadcom/genet/if_genet.c
>> @@ -1305,6 +1305,8 @@ gen_parse_tx(struct mbuf *m, int csum_flags)
>> * Unknown whether other cases require moving a header;
>> * ARP works without.
>> */
>> + COPY(min(gen_tx_hdr_min, m->m_len));
>> + offset += min(gen_tx_hdr_min, m->m_len);
>> }
>> return (offset);
>> #undef COPY
>>
>> It needs an updated comment, but should be testable.
>>
>> Mike
>>
>> > > Regards,
>> > > Ronald.
>>
>>
>
> Hi,
>
> Thanks. Disabling both -txcsum and -txcsum6 works for me also.
> I can test your patch tommorow or on Thursday. I'm first testing a big mongodb 6.0 patch I hope to commit soon and this build occupies my rpi4 for some more hours.
>
> Regards,
> Ronald.
>
Hi,
Your patch works for me. I can send wake-on-lan packets now. Should I test other specific things also?
Regards,
Ronald.
------=_Part_15493_225212243.1667020606673
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: 7bit
<html><head></head><body><br><p><small><strong>Van:</strong> Ronald Klop <ronald-lists@klop.ws><br><strong>Datum:</strong> 25 oktober 2022 20:32<br><strong>Aan:</strong> mike@karels.net<br><strong>CC:</strong> freebsd-arm@freebsd.org<br><strong>Onderwerp:</strong> Re: wake-on-lan lost from rpi4 (works on rpi3)<br></small></p><blockquote style="margin-left: 5px; border-left: 3px solid #ccc; margin-right: 0px; padding-left: 5px;"><div class="MessageRFC822Viewer do_not_remove" id="P"><!-- P -->
<!-- processMimeMessage --><div class="MultipartAlternativeViewer do_not_remove"><!-- P.P -->
<div class="TextHTMLViewer do_not_remove" id="P.P.P"><!-- P.P.P -->
<p><strong>Van:</strong> Mike Karels <mike@karels.net><br>
<strong>Datum:</strong> dinsdag, 25 oktober 2022 18:11<br>
<strong>Aan:</strong> Ronald Klop <ronald-lists@klop.ws><br>
<strong>CC:</strong> freebsd-arm@freebsd.org<br>
<strong>Onderwerp:</strong> Re: wake-on-lan lost from rpi4 (works on rpi3)</ronald-lists@klop.ws></mike@karels.net></p>
<blockquote style="padding-right: 0px; padding-left: 5px; margin-left: 5px; border-left: #000000 2px solid; margin-right: 0px">
<div class="MessageRFC822Viewer do_not_remove" id="P">
<div class="TextPlainViewer do_not_remove" id="P.P">I wrote:<br>
> On 25 Oct 2022, at 9:13, Ronald Klop wrote:<br>
<br>
> > Van: Mike Karels <mike@karels.net><br>
> > Datum: dinsdag, 25 oktober 2022 14:55<br>
> > Aan: Ronald Klop <ronald-lists@klop.ws><br>
> > CC: freebsd-arm@freebsd.org<br>
> > Onderwerp: Re: wake-on-lan lost from rpi4 (works on rpi3)<br>
> >><br>
> >> On 25 Oct 2022, at 6:57, Ronald Klop wrote:<br>
> >><br>
> >>> Hi,<br>
> >>><br>
> >>> When I do "wake ue0 `cat /home/ronald/freenas.ethernet `" on my RPI3B+ my NAS boots and tcpdump shows the following line on my rpi3 as well as on my router.<br>
> >>> 13:39:44.344111 IPX 00217046.6c:da:00:21:70:46.6cda > 00217046.6c:da:00:21:70:46.6cda: ipx-#6cda 65505<br>
> >>><br>
> >>> When I do "wake genet0 `cat /home/ronald/freenas.ethernet `" on my RPI4 my NAS does not boot and tcpdump only show this on my rpi4 and *not* on my router.<br>
> >>> 13:37:26.448251 IPX 00217046.6c:da:00:21:70:46.6cda > 00217046.6c:da:00:21:70:46.6cda: ipx-#6cda 65505<br>
> >>><br>
> >>> Firewall ipfw does not indicate that it blocks anything.<br>
> >>><br>
> >>> RPI4 runs:<br>
> >>> FreeBSD rpi4 14.0-CURRENT FreeBSD 14.0-CURRENT #11 main-4f0c9b76cf: Sat Aug 13 23:59:19 CEST 2022 ronald@rpi4:/home/ronald/dev/obj/home/ronald/dev/freebsd/arm64.aarch64/sys/GENERIC-NODEBUG arm64<br>
> >>><br>
> >>> genet0: flags=8943<up,broadcast,running,promisc,simplex,multicast> metric 0 mtu 1500<br>
> >>> options=68000b<rxcsum,txcsum,vlan_mtu,linkstate,rxcsum_ipv6,txcsum_ipv6><br>
> >>><br>
> >>><br>
> >>> RPI3 runs:<br>
> >>> FreeBSD rpi3 13.1-RELEASE-p2 FreeBSD 13.1-RELEASE-p2 GENERIC arm64<br>
> >>><br>
> >>> ue0: flags=8943<up,broadcast,running,promisc,simplex,multicast> metric 0 mtu 1500<br>
> >>> options=80009<rxcsum,vlan_mtu,linkstate><br>
> >>><br>
> >>><br>
> >>> Does genet0 not support these packages?<br>
> >>> What can prevent this packet to go on the ethernet while tcpdump still shows it is outgoing on the interface?<br>
> >>> genet0 does have a vlan configured connected to a bridge0 for some jails<br>
> >>> vlan3: flags=8943<up,broadcast,running,promisc,simplex,multicast> metric 0 mtu 1500<br>
> >>> options=80000<linkstate><br>
> >>> groups: vlan<br>
> >>> vlan: 3 vlanproto: 802.1q vlanpcp: 0 parent interface: genet0<br>
> >>><br>
> >>> ue0 is itself connected to a bridge0<br>
> >><br>
> >> Is the RPI3 also on a vlan and bridge?<br>
> >><br>
> >> Is the router (or switch) expecting the packet on the vlan?<br>
> >> If so, maybe you need to send on vlan3 rather than genet0.<br>
> >><br>
> >> The genet interface has some oddities about packet layouts;<br>
> >> that could be hitting here. You could try disabling TCP TX<br>
> >> checksum offload: ifconfig genet0 -txcsum -txcsum6; that<br>
> >> simplifies some of the issues.<br>
> >><br>
> >> Mike<br>
> >><br>
> >>> Regards,<br>
> >>> Ronald.<br>
> ><br>
> ><br>
> > Hi,<br>
> ><br>
> > Thanks for the hint. I experimented some further.<br>
> ><br>
> > Disabling -txcsum -rxcsum didn't matter.<br>
<br>
> -rxcsum doesn't matter, but you need to do -txcsum6 as well as<br>
> -txcsum. (See the code that calls gen_parse_tx.)<br>
<br>
> > But when I use bridge0 or vlan3 as device then it goes into the wire but on the VLAN which is not where my NAS is which I want to boot.<br>
> ><br>
> > Looking at the driver I found this interesting peace of code.<br>
> ><br>
> > static int<br>
> > gen_parse_tx(struct mbuf *m, int csum_flags) {<br>
> > ...<br>
> > if (ether_type == ETHERTYPE_IP) {<br>
> > COPY(((struct ip *)p)->ip_hl << 2);<br>
> > offset += ((struct ip *)p)->ip_hl << 2;<br>
> > } else if (ether_type == ETHERTYPE_IPV6) {<br>
> > COPY(sizeof(struct ip6_hdr));<br>
> > offset += sizeof(struct ip6_hdr);<br>
> > } else {<br>
> > /*<br>
> > * Unknown whether other cases require moving a header;<br>
> > * ARP works without.<br>
> > */<br>
> > }<br>
> > ...<br>
> > }<br>
> ><br>
> > There is also some code which handles EHTERTYPE_VLAN.<br>
> ><br>
> > I don't have time to start debugging this today. But would this ring a bell to anybody in connection to a wake-on-lan packet?<br>
<br>
> I ran some experiments with tx checksum disabled, and it seems to<br>
> matter. I have a change for the last block you listed that might<br>
> help, but I haven't tested it yet.<br>
<br>
This patch seems to work:<br>
<br>
diff --git a/sys/arm64/broadcom/genet/if_genet.c b/sys/arm64/broadcom/genet/if_genet.c<br>
index 327af8acbcdd..b7ab86e7931d 100644<br>
--- a/sys/arm64/broadcom/genet/if_genet.c<br>
+++ b/sys/arm64/broadcom/genet/if_genet.c<br>
@@ -1305,6 +1305,8 @@ gen_parse_tx(struct mbuf *m, int csum_flags)<br>
* Unknown whether other cases require moving a header;<br>
* ARP works without.<br>
*/<br>
+ COPY(min(gen_tx_hdr_min, m->m_len));<br>
+ offset += min(gen_tx_hdr_min, m->m_len);<br>
}<br>
return (offset);<br>
#undef COPY<br>
<br>
It needs an updated comment, but should be testable.<br>
<br>
Mike<br>
<br>
> > Regards,<br>
> > Ronald.<br>
<br>
</linkstate></up,broadcast,running,promisc,simplex,multicast></rxcsum,vlan_mtu,linkstate></up,broadcast,running,promisc,simplex,multicast></rxcsum,txcsum,vlan_mtu,linkstate,rxcsum_ipv6,txcsum_ipv6></up,broadcast,running,promisc,simplex,multicast></ronald-lists@klop.ws></mike@karels.net></div>
</div>
</blockquote>
<br>
Hi,<br>
<br>
Thanks. Disabling both -txcsum and -txcsum6 works for me also.<br>
I can test your patch tommorow or on Thursday. I'm first testing a big mongodb 6.0 patch I hope to commit soon and this build occupies my rpi4 for some more hours.<br>
<br>
Regards,<br>
Ronald.<br>
</div></div></div><!-- TextHTMLViewer -->
<!-- MultipartAlternativeViewer -->
<!-- MessageRFC822Viewer -->
</blockquote><br><br>Hi,<div><br></div><div>Your patch works for me. I can send wake-on-lan packets now. Should I test other specific things also?</div><div><br></div><div>Regards,</div><div><br></div><div>Ronald. </div></body></html>
------=_Part_15493_225212243.1667020606673--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2086140007.15494.1667020606677>