Skip site navigation (1)Skip section navigation (2) 
Date:      Fri, 11 Jan 2013 09:18:07 +0200
From:      Jacques Fourie <jacques.fourie@gmail.com>
To:        Hackers freeBSD <freebsd-hackers@freebsd.org>
Subject:   Inconsistent TCP state in tcp_do_segment()
Message-ID:  <CALX0vxAKNMdP6ZvXf0mvk5MKdRL45D82AYBy8Ras-zRDmixa3Q@mail.gmail.com>
next in thread | raw e-mail | index | archive | help 
Hi,

I've been using the kernel socket API and noticed that every once in a
while the data received at the remote end of a TCP connection doesn't match
the data sent locally using sosend(). I tracked it down to the piece of
code in tcp_do_segment() starting at line 2665: (svn rev 245286)

               if (acked > so->so_snd.sb_cc) {
                        tp->snd_wnd -= so->so_snd.sb_cc;
                        sbdrop_locked(&so->so_snd, (int)so->so_snd.sb_cc);
                        ourfinisacked = 1;
                } else {
                        sbdrop_locked(&so->so_snd, acked);
                        tp->snd_wnd -= acked;
                        ourfinisacked = 0;
                }
                sowwakeup_locked(so);
                /* Detect una wraparound. */
                if (!IN_RECOVERY(tp->t_flags) &&
                    SEQ_GT(tp->snd_una, tp->snd_recover) &&
                    SEQ_LEQ(th->th_ack, tp->snd_recover))
                        tp->snd_recover = th->th_ack - 1;
                /* XXXLAS: Can this be moved up into cc_post_recovery? */
                if (IN_RECOVERY(tp->t_flags) &&
                    SEQ_GEQ(th->th_ack, tp->snd_recover)) {
                        EXIT_RECOVERY(tp->t_flags);
                }
                tp->snd_una = th->th_ack;
                if (tp->t_flags & TF_SACK_PERMIT) {
                        if (SEQ_GT(tp->snd_una, tp->snd_recover))
                                tp->snd_recover = tp->snd_una;
                }
                if (SEQ_LT(tp->snd_nxt, tp->snd_una))
                        tp->snd_nxt = tp->snd_una;

The issue is that sowwakeup_locked() is called before all the tcpcb fields
are updated to account for the current ACK processing as can be seen from
the tp->snd_una = th->th_ack in line 2686. My code that uses the kernel
socket API calls sosend() in the upcall path resulting from
sowwakeup_locked() which in turn can lead to tcp_output() being called with
inconsistent TCP state. If I move the call to sowwakeup_locked() to after
the 'if (SEQ_LT(tp->snd_nxt, tp->snd_una))' line in the code snippet above
the data corruption issue seems to be fixed.

Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CALX0vxAKNMdP6ZvXf0mvk5MKdRL45D82AYBy8Ras-zRDmixa3Q>