Date: Wed, 11 Sep 2019 20:56:28 -0700 From: Anish <akgupt3@gmail.com> To: Andriy Gapon <avg@freebsd.org> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: ixv + PCBGROUP + RSS: problem establishing an outgoing TCP connection Message-ID: <CALnRwMT0tsH-surx0P7s4i=MEoO%2B3R-VrydGKQ97xtbt=Vzo4A@mail.gmail.com> In-Reply-To: <29ed38c7-5835-b354-368a-f9ebaddc9a7d@FreeBSD.org> References: <db5f87ba-990c-cf22-97e5-10ada01df529@FreeBSD.org> <29ed38c7-5835-b354-368a-f9ebaddc9a7d@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
>if_ixv.c does not include opt_rss.h. Because of this IXGBE_FEATURE_RSS
gets
defined to zero (in ixgbe_features.h). So, instead of of using
rss_getkey() to
get the RSS key, the driver just generates a random one.
I have not looked at ixv but as far as I understand our build process,
opt_rss.h need to be included.
-Anish
On Tue, Sep 10, 2019 at 11:16 PM Andriy Gapon <avg@freebsd.org> wrote:
> On 10/09/2019 12:14, Andriy Gapon wrote:
> >
> > This happens on an EC2 instance with ixv driver.
>
> I wonder if anyone ever tested ixv with PCBGROUP...
> I see a trivial but severe bug.
> if_ixv.c does not include opt_rss.h. Because of this IXGBE_FEATURE_RSS
> gets
> defined to zero (in ixgbe_features.h). So, instead of of using
> rss_getkey() to
> get the RSS key, the driver just generates a random one.
> No surprise then that the hardware (VF) produces totally different hashes=
.
> But maybe that's not all.
>
> On top of that, I wonder why the driver enables RSS in the hardware when
> feat_en
> does not have IXGBE_FEATURE_RSS.
> Could anyone please explain the logic behind that?
> Please see ixv_initialize_rss_mapping.
> For example:
> if (adapter->feat_en & IXGBE_FEATURE_RSS) {
> /* Fetch the configured RSS key */
> rss_getkey((uint8_t *)&rss_key);
> } else {
> /* set up random bits */
> arc4rand(&rss_key, sizeof(rss_key), 0);
> }
> And so on.
>
> Additionally, I found this bit of information:
> The limitation for VF RSS on Intel=C2=AE 82599 10 Gigabit Ethernet Contro=
ller
> is: The
> hash and key are shared among PF and all VF, the RETA table with 128
> entries is
> also shared among PF and all VF; So it could not to provide a method to
> query
> the hash and reta content per VF on guest, while, if possible, please
> query them
> on host for the shared RETA information.
>
> And my "hardware" is exactly 82599 VF.
> I hacked the driver to not call ixv_initialize_rss_mapping() at all, but
> even
> with that change the packet descriptors had IXGBE_RXDADV_RSSTYPE_IPV4_TCP
> in
> pkt_info. Maybe it's because of how PF was configured.
> So, I wonder if ixgbe_isc_rxd_pkt_get() should be modified to not set
> iri_flowid
> and iri_rsstype under some conditions.
>
> > When I try to establish an outgoing TCP connection I see the following
> exchange.
> > Local side sends SYN, it receives SYN+ACK and immediately sends RST.
> > I tracked this down to in_pcblookup_mbuf() failing to find the
> corresponding inpcb.
> >
> > I dug a bit deeper and this is my understanding of the issue.
> >
> > When tcp_connect() calls in_pcbrehash() the inpcb gets placed into a
> group
> > determined by in_pcbgroup_bytuple() [see in_pcbgroup_update and
> > in_pcbgroup_byinpcb]. The inpcb does not have INP_RSS_BUCKET_SET. Bot=
h
> > addresses and ports are populated at that time.
> >
> > When the reply packet is received, in_pcblookup_mbuf() uses
> in_pcbgroup_byhash()
> > to look up the group because the packet has M_HASHTYPE_RSS_TCP_IPV4.
> > The problem is that in_pcbgroup_byhash() returns a different group and
> the inpcb
> > cannot be found.
> >
> > I am very new to this code, so I would appreciate any help with further
> > debugging and root causing the problem.
>
>
>
>
> --
> Andriy Gapon
> _______________________________________________
> freebsd-net@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CALnRwMT0tsH-surx0P7s4i=MEoO%2B3R-VrydGKQ97xtbt=Vzo4A>