Date: Sun, 1 Jun 2014 08:21:55 -0700 From: Adrian Chadd <adrian@freebsd.org> To: FreeBSD Net <freebsd-net@freebsd.org>, "freebsd-arch@freebsd.org" <freebsd-arch@freebsd.org> Subject: [rfc] TCP timewait and credential handling - why would we get a TW with no credential? Message-ID: <CAJ-Vmo=JwL7gFE2xfXa9cx4rYdkDf87qoy32H%2BSguRPogT2p=w@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Hi, I've been seeing this panic under high very short-term connection TCP throughput tests (30,000 connections a second) with SO_REUSEPORT: Current language: auto; currently minimal (kgdb) frame 11 #11 0xffffffff80a6bdf1 in tcp_twclose (tw=0xfffff801348b5780, reuse=0) at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/tcp_timewait.c:644 644 crfree(tw->tw_cred); (kgdb) frame 11 #11 0xffffffff80a6bdf1 in tcp_twclose (tw=0xfffff801348b5780, reuse=0) at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/tcp_timewait.c:644 644 crfree(tw->tw_cred); (kgdb) print tw $1 = (struct tcptw *) 0xfffff801348b5780 (kgdb) print tw->tw_cred $2 = (struct ucred *) 0x0 (kgdb) and: #10 0xffffffff808d017e in crfree (cr=0x0) at /usr/home/adrian/git/github/erikarn/freebsd/sys/kern/kern_prot.c:1837 #11 0xffffffff80a6bdf1 in tcp_twclose (tw=0xfffff801348b5780, reuse=0) at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/tcp_timewait.c:644 #12 0xffffffff80a6bc53 in tcp_twcheck (inp=<value optimized out>, to=<value optimized out>, th=<value optimized out>, m=<value optimized out>, tlen=<value optimized out>) at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/tcp_timewait.c:425 #13 0xffffffff80a5cf3f in tcp_input (m=<value optimized out>, off0=<value optimized out>) at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/tcp_input.c:949 #14 0xffffffff809fa4e7 in ip_input (m=0xfffff8004831e000) at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/ip_input.c:729 #15 0xffffffff80998441 in netisr_dispatch_src (proto=<value optimized out>, source=<value optimized out>, m=0xffffffff) at /usr/home/adrian/git/github/erikarn/freebsd/sys/net/netisr.c:972 #16 0xffffffff80990173 in ether_demux (ifp=<value optimized out>, m=0xfffff8004831e000) at /usr/home/adrian/git/github/erikarn/freebsd/sys/net/if_ethersubr.c:775 #17 0xffffffff80990dce in ether_nh_input (m=<value optimized out>) at /usr/home/adrian/git/github/erikarn/freebsd/sys/net/if_ethersubr.c:582 #18 0xffffffff80998441 in netisr_dispatch_src (proto=<value optimized out>, source=<value optimized out>, m=0xffffffff) at /usr/home/adrian/git/github/erikarn/freebsd/sys/net/netisr.c:972 #19 0xffffffff809903c6 in ether_input (ifp=<value optimized out>, m=0x0) at /usr/home/adrian/git/github/erikarn/freebsd/sys/net/if_ethersubr.c:683 #20 0xffffffff804e9d67 in igb_rxeof (count=99) at /usr/home/adrian/git/github/erikarn/freebsd/sys/dev/e1000/if_igb.c:4882 #21 0xffffffff804ea47f in igb_msix_que (arg=0xfffff8000d47e670) at /usr/home/adrian/git/github/erikarn/freebsd/sys/dev/e1000/if_igb.c:1626 #22 0xffffffff808aef03 in intr_event_execute_handlers (p=<value optimized out>, ie=0xfffff8000f76cb00) at /usr/home/adrian/git/github/erikarn/freebsd/sys/kern/kern_intr.c:1263 #23 0xffffffff808af866 in ithread_loop (arg=0xfffff8000f908d20) at /usr/home/adrian/git/github/erikarn/freebsd/sys/kern/kern_intr.c:1276 #24 0xffffffff808acc41 in fork_exit (callout=0xffffffff808af7d0 <ithread_loop>, arg=0xfffff8000f908d20, frame=0xfffffe1046c2cac0) at /usr/home/adrian/git/github/erikarn/freebsd/sys/kern/kern_fork.c:977 #25 0xffffffff80c939ae in fork_trampoline () at /usr/home/adrian/git/github/erikarn/freebsd/sys/amd64/amd64/exception.S:605 #26 0x0000000000000000 in ?? () ... there's only one path to deleting the credentials and that's via tcp_twclose() -> tcp_tw_2msl_stop(). Has anyone seen this kind of problem before? -a
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-Vmo=JwL7gFE2xfXa9cx4rYdkDf87qoy32H%2BSguRPogT2p=w>