From nobody Tue Oct 25 18:31:48 2022 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MxgWq3BQPz4gZX3 for ; Tue, 25 Oct 2022 18:31:51 +0000 (UTC) (envelope-from SRS0=dp/f=22=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4MxgWp2vq1z3M0r for ; Tue, 25 Oct 2022 18:31:50 +0000 (UTC) (envelope-from SRS0=dp/f=22=klop.ws=ronald-lists@realworks.nl) Date: Tue, 25 Oct 2022 20:31:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1666722708; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=yMJ27P2+TFMb7qz0UvgK3fS2M4hL3s+g2XJeOVPImoA=; b=o7M+7rA4WrG1VqQRcoYdV7CFwvOc3YcVqt/30zR4AjXzRRSfq34UFts3kPCNxt+GerNAjp ZCvmWdbEVXrkAoIcoN+PmPrcQSDkcltFYdHsJjruTNwwxiObwX9suEmyD8kjXDcQEBw46A ogqGMCSca+GeTF+y1M6wV5PNmCLXd9UlplHpgtTu15lYSKVg7zYBn0EVrYwronaC7N6QPx 1EIKlzx3YCSy8n2cCEEIj6o0IApYb5xXoE+IUlR2kgf6bc1eZbsX4C4Dp1Ve8v/iJyvCvl dZCXAvfLQlI5WVKL5jIF8r6YZQX8ylZ9G6cJmQhLKGQDYHW7xuucTzX/ARmkdQ== From: Ronald Klop To: mike@karels.net Cc: freebsd-arm@freebsd.org Message-ID: <1539824734.231347.1666722708425@localhost> In-Reply-To: <202210251611.29PGBsbq016681@mail.karels.net> References: Your message of Tue, 25 Oct 2022 10:22:38 -0500. <676FFB6B-0D38-47D4-AED8-EA2D5F685F43@karels.net> <202210251611.29PGBsbq016681@mail.karels.net> Subject: Re: wake-on-lan lost from rpi4 (works on rpi3) List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_231346_1677104857.1666722708420" X-Mailer: Realworks (628.35) Importance: Normal X-Priority: 3 (Normal) X-Rspamd-Queue-Id: 4MxgWp2vq1z3M0r X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=klop.ws header.s=rw2 header.b=o7M+7rA4; dmarc=pass (policy=quarantine) header.from=klop.ws; spf=pass (mx1.freebsd.org: domain of "SRS0=dp/f=22=klop.ws=ronald-lists@realworks.nl" designates 194.109.157.24 as permitted sender) smtp.mailfrom="SRS0=dp/f=22=klop.ws=ronald-lists@realworks.nl" X-Spamd-Result: default: False [-3.18 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-0.99)[-0.994]; NEURAL_HAM_SHORT(-0.99)[-0.990]; DMARC_POLICY_ALLOW(-0.50)[klop.ws,quarantine]; MID_RHS_NOT_FQDN(0.50)[]; FORGED_SENDER(0.30)[ronald-lists@klop.ws,SRS0=dp/f=22=klop.ws=ronald-lists@realworks.nl]; R_SPF_ALLOW(-0.20)[+ip4:194.109.157.0/24]; R_DKIM_ALLOW(-0.20)[klop.ws:s=rw2]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_ZERO(0.00)[0]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; ARC_NA(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL]; FROM_HAS_DN(0.00)[]; HAS_X_PRIO_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[klop.ws:+]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_NEQ_ENVFROM(0.00)[ronald-lists@klop.ws,SRS0=dp/f=22=klop.ws=ronald-lists@realworks.nl] X-ThisMailContainsUnwantedMimeParts: N ------=_Part_231346_1677104857.1666722708420 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Van: Mike Karels Datum: dinsdag, 25 oktober 2022 18:11 Aan: Ronald Klop CC: freebsd-arm@freebsd.org Onderwerp: Re: wake-on-lan lost from rpi4 (works on rpi3) > > I wrote: > > On 25 Oct 2022, at 9:13, Ronald Klop wrote: > > > > Van: Mike Karels > > > Datum: dinsdag, 25 oktober 2022 14:55 > > > Aan: Ronald Klop > > > CC: freebsd-arm@freebsd.org > > > Onderwerp: Re: wake-on-lan lost from rpi4 (works on rpi3) > > >> > > >> On 25 Oct 2022, at 6:57, Ronald Klop wrote: > > >> > > >>> Hi, > > >>> > > >>> When I do "wake ue0 `cat /home/ronald/freenas.ethernet `" on my RPI3B+ my NAS boots and tcpdump shows the following line on my rpi3 as well as on my router. > > >>> 13:39:44.344111 IPX 00217046.6c:da:00:21:70:46.6cda > 00217046.6c:da:00:21:70:46.6cda: ipx-#6cda 65505 > > >>> > > >>> When I do "wake genet0 `cat /home/ronald/freenas.ethernet `" on my RPI4 my NAS does not boot and tcpdump only show this on my rpi4 and *not* on my router. > > >>> 13:37:26.448251 IPX 00217046.6c:da:00:21:70:46.6cda > 00217046.6c:da:00:21:70:46.6cda: ipx-#6cda 65505 > > >>> > > >>> Firewall ipfw does not indicate that it blocks anything. > > >>> > > >>> RPI4 runs: > > >>> FreeBSD rpi4 14.0-CURRENT FreeBSD 14.0-CURRENT #11 main-4f0c9b76cf: Sat Aug 13 23:59:19 CEST 2022 ronald@rpi4:/home/ronald/dev/obj/home/ronald/dev/freebsd/arm64.aarch64/sys/GENERIC-NODEBUG arm64 > > >>> > > >>> genet0: flags=8943 metric 0 mtu 1500 > > >>> options=68000b > > >>> > > >>> > > >>> RPI3 runs: > > >>> FreeBSD rpi3 13.1-RELEASE-p2 FreeBSD 13.1-RELEASE-p2 GENERIC arm64 > > >>> > > >>> ue0: flags=8943 metric 0 mtu 1500 > > >>> options=80009 > > >>> > > >>> > > >>> Does genet0 not support these packages? > > >>> What can prevent this packet to go on the ethernet while tcpdump still shows it is outgoing on the interface? > > >>> genet0 does have a vlan configured connected to a bridge0 for some jails > > >>> vlan3: flags=8943 metric 0 mtu 1500 > > >>> options=80000 > > >>> groups: vlan > > >>> vlan: 3 vlanproto: 802.1q vlanpcp: 0 parent interface: genet0 > > >>> > > >>> ue0 is itself connected to a bridge0 > > >> > > >> Is the RPI3 also on a vlan and bridge? > > >> > > >> Is the router (or switch) expecting the packet on the vlan? > > >> If so, maybe you need to send on vlan3 rather than genet0. > > >> > > >> The genet interface has some oddities about packet layouts; > > >> that could be hitting here. You could try disabling TCP TX > > >> checksum offload: ifconfig genet0 -txcsum -txcsum6; that > > >> simplifies some of the issues. > > >> > > >> Mike > > >> > > >>> Regards, > > >>> Ronald. > > > > > > > > > Hi, > > > > > > Thanks for the hint. I experimented some further. > > > > > > Disabling -txcsum -rxcsum didn't matter. > > > -rxcsum doesn't matter, but you need to do -txcsum6 as well as > > -txcsum. (See the code that calls gen_parse_tx.) > > > > But when I use bridge0 or vlan3 as device then it goes into the wire but on the VLAN which is not where my NAS is which I want to boot. > > > > > > Looking at the driver I found this interesting peace of code. > > > > > > static int > > > gen_parse_tx(struct mbuf *m, int csum_flags) { > > > ... > > > if (ether_type == ETHERTYPE_IP) { > > > COPY(((struct ip *)p)->ip_hl << 2); > > > offset += ((struct ip *)p)->ip_hl << 2; > > > } else if (ether_type == ETHERTYPE_IPV6) { > > > COPY(sizeof(struct ip6_hdr)); > > > offset += sizeof(struct ip6_hdr); > > > } else { > > > /* > > > * Unknown whether other cases require moving a header; > > > * ARP works without. > > > */ > > > } > > > ... > > > } > > > > > > There is also some code which handles EHTERTYPE_VLAN. > > > > > > I don't have time to start debugging this today. But would this ring a bell to anybody in connection to a wake-on-lan packet? > > > I ran some experiments with tx checksum disabled, and it seems to > > matter. I have a change for the last block you listed that might > > help, but I haven't tested it yet. > > This patch seems to work: > > diff --git a/sys/arm64/broadcom/genet/if_genet.c b/sys/arm64/broadcom/genet/if_genet.c > index 327af8acbcdd..b7ab86e7931d 100644 > --- a/sys/arm64/broadcom/genet/if_genet.c > +++ b/sys/arm64/broadcom/genet/if_genet.c > @@ -1305,6 +1305,8 @@ gen_parse_tx(struct mbuf *m, int csum_flags) > * Unknown whether other cases require moving a header; > * ARP works without. > */ > + COPY(min(gen_tx_hdr_min, m->m_len)); > + offset += min(gen_tx_hdr_min, m->m_len); > } > return (offset); > #undef COPY > > It needs an updated comment, but should be testable. > > Mike > > > > Regards, > > > Ronald. > > Hi, Thanks. Disabling both -txcsum and -txcsum6 works for me also. I can test your patch tommorow or on Thursday. I'm first testing a big mongodb 6.0 patch I hope to commit soon and this build occupies my rpi4 for some more hours. Regards, Ronald. ------=_Part_231346_1677104857.1666722708420 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit  

Van: Mike Karels <mike@karels.net>
Datum: dinsdag, 25 oktober 2022 18:11
Aan: Ronald Klop <ronald-lists@klop.ws>
CC: freebsd-arm@freebsd.org
Onderwerp: Re: wake-on-lan lost from rpi4 (works on rpi3)

I wrote:
> On 25 Oct 2022, at 9:13, Ronald Klop wrote:

> > Van: Mike Karels <mike@karels.net>
> > Datum: dinsdag, 25 oktober 2022 14:55
> > Aan: Ronald Klop <ronald-lists@klop.ws>
> > CC: freebsd-arm@freebsd.org
> > Onderwerp: Re: wake-on-lan lost from rpi4 (works on rpi3)
> >>
> >> On 25 Oct 2022, at 6:57, Ronald Klop wrote:
> >>
> >>> Hi,
> >>>
> >>> When I do "wake ue0 `cat /home/ronald/freenas.ethernet `" on my RPI3B+ my NAS boots and tcpdump shows the following line on my rpi3 as well as on my router.
> >>> 13:39:44.344111 IPX 00217046.6c:da:00:21:70:46.6cda > 00217046.6c:da:00:21:70:46.6cda: ipx-#6cda 65505
> >>>
> >>> When I do "wake genet0 `cat /home/ronald/freenas.ethernet `" on my RPI4 my NAS does not boot and tcpdump only show this on my rpi4 and *not* on my router.
> >>> 13:37:26.448251 IPX 00217046.6c:da:00:21:70:46.6cda > 00217046.6c:da:00:21:70:46.6cda: ipx-#6cda 65505
> >>>
> >>> Firewall ipfw does not indicate that it blocks anything.
> >>>
> >>> RPI4 runs:
> >>> FreeBSD rpi4 14.0-CURRENT FreeBSD 14.0-CURRENT #11 main-4f0c9b76cf: Sat Aug 13 23:59:19 CEST 2022     ronald@rpi4:/home/ronald/dev/obj/home/ronald/dev/freebsd/arm64.aarch64/sys/GENERIC-NODEBUG arm64
> >>>
> >>> genet0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
> >>>    options=68000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
> >>>
> >>>
> >>> RPI3 runs:
> >>> FreeBSD rpi3 13.1-RELEASE-p2 FreeBSD 13.1-RELEASE-p2 GENERIC arm64
> >>>
> >>> ue0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
> >>>    options=80009<RXCSUM,VLAN_MTU,LINKSTATE>
> >>>
> >>>
> >>> Does genet0 not support these packages?
> >>> What can prevent this packet to go on the ethernet while tcpdump still shows it is outgoing on the interface?
> >>> genet0 does have a vlan configured connected to a bridge0 for some jails
> >>> vlan3: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
> >>>    options=80000<LINKSTATE>
> >>>    groups: vlan
> >>>    vlan: 3 vlanproto: 802.1q vlanpcp: 0 parent interface: genet0
> >>>
> >>> ue0 is itself connected to a bridge0
> >>
> >> Is the RPI3 also on a vlan and bridge?
> >>
> >> Is the router (or switch) expecting the packet on the vlan?
> >> If so, maybe you need to send on vlan3 rather than genet0.
> >>
> >> The genet interface has some oddities about packet layouts;
> >> that could be hitting here.  You could try disabling TCP TX
> >> checksum offload: ifconfig genet0 -txcsum -txcsum6; that
> >> simplifies some of the issues.
> >>
> >>         Mike
> >>
> >>> Regards,
> >>> Ronald.
> >
> >
> > Hi,
> >
> > Thanks for the hint. I experimented some further.
> >
> > Disabling -txcsum -rxcsum didn't matter.

> -rxcsum doesn't matter, but you need to do -txcsum6 as well as
> -txcsum.  (See the code that calls gen_parse_tx.)

> > But when I use bridge0 or vlan3 as device then it goes into the wire but on the VLAN which is not where my NAS is which I want to boot.
> >
> > Looking at the driver I found this interesting peace of code.
> >
> > static int
> > gen_parse_tx(struct mbuf *m, int csum_flags) {
> > ...
> >        if (ether_type == ETHERTYPE_IP) {
> >                COPY(((struct ip *)p)->ip_hl << 2);
> >                offset += ((struct ip *)p)->ip_hl << 2;
> >        } else if (ether_type == ETHERTYPE_IPV6) {
> >                COPY(sizeof(struct ip6_hdr));
> >                offset += sizeof(struct ip6_hdr);
> >        } else {
> >                /*
> >                 * Unknown whether other cases require moving a header;
> >                 * ARP works without.
> >                 */
> >        }
> > ...
> > }
> >
> > There is also some code which handles EHTERTYPE_VLAN.
> >
> > I don't have time to start debugging this today. But would this ring a bell to anybody in connection to a wake-on-lan packet?

> I ran some experiments with tx checksum disabled, and it seems to
> matter.  I have a change for the last block you listed that might
> help, but I haven't tested it yet.

This patch seems to work:

diff --git a/sys/arm64/broadcom/genet/if_genet.c b/sys/arm64/broadcom/genet/if_genet.c
index 327af8acbcdd..b7ab86e7931d 100644
--- a/sys/arm64/broadcom/genet/if_genet.c
+++ b/sys/arm64/broadcom/genet/if_genet.c
@@ -1305,6 +1305,8 @@ gen_parse_tx(struct mbuf *m, int csum_flags)
         * Unknown whether other cases require moving a header;
         * ARP works without.
         */
+       COPY(min(gen_tx_hdr_min, m->m_len));
+       offset += min(gen_tx_hdr_min, m->m_len);
    }
    return (offset);
 #undef COPY

It needs an updated comment, but should be testable.

        Mike

> > Regards,
> > Ronald.

 

Hi,

Thanks. Disabling both -txcsum and -txcsum6 works for me also.
I can test your patch tommorow or on Thursday. I'm first testing a big mongodb 6.0 patch I hope to commit soon and this build occupies my rpi4 for some more hours.

Regards,
Ronald.
  ------=_Part_231346_1677104857.1666722708420--