Date: Tue, 9 Sep 2014 13:41:25 -0700 From: Adrian Chadd <adrian@freebsd.org> To: FreeBSD Net <freebsd-net@freebsd.org> Subject: inpcb lookups in udp_send -> inpcb laddr and flowtable Message-ID: <CAJ-VmomvWV0uxrxh%2BP3UbGOpc1rvY89UXnFfwWJ-j%2B%2B-n1m=BA@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Hi!
I found this whilst doing UDP RSS testing.
root@adrian-hackbox:/home/adrian/work/freebsd/head/src/sys/net #
sysctl debug.lock.prof.stats | head -2 ; sysctl debug.lock.prof.stats
| sort -nk4 | tail -5
debug.lock.prof.stats:
max wait_max total wait_total count avg wait_avg
cnt_hold cnt_lock name
40 11 114900 55276 359106 0 0
0 20962 /usr/home/adrian/work/freebsd/head/src/sys/dev/ixgbe/ixgbe.c:1550
(sleep mutex:ix0:tx(6))
21 11 144253 71959 454169 0 0
0 27506 /usr/home/adrian/work/freebsd/head/src/sys/dev/ixgbe/ixgbe.c:1550
(sleep mutex:ix0:tx(1))
9 10 3132066 85999 42942422 0 0
0 35943 /usr/home/adrian/work/freebsd/head/src/sys/netinet/udp_usrreq.c:369
(sleep mutex:so_rcv)
676 1713 805307 169654 7076283 0 0
0 69741 /usr/home/adrian/work/freebsd/head/src/sys/netinet/in_pcb.c:963
(sleep mutex:rtentry)
674 4426 2111324 6415012 7076284 0 0
0 489745 /usr/home/adrian/work/freebsd/head/src/sys/net/route.c:420
(sleep mutex:rtentry)
then...
root@adrian-hackbox:/home/adrian/git/github/erikarn/freebsd-rss/rss-udp-srv
# dtrace -n 'fbt::rtalloc1_fib:entry { @[stack()] = count(); }'
dtrace: description 'fbt::rtalloc1_fib:entry ' matched 1 probe
^[[A^C
kernel`rtalloc_ign_fib+0x98
kernel`in_pcbladdr+0x7d
kernel`in_pcbconnect_setup+0x1af
kernel`udp_send+0x4af
kernel`sosend_dgram+0x30b
kernel`kern_sendit+0x20b
kernel`sendit+0x129
kernel`sys_sendto+0x4d
kernel`amd64_syscall+0x25a
kernel`0xffffffff80cd178b
1848628
It turns out that if a local address isn't bound (ie, it's INADDR_ANY)
then in_pcbladdr() will do a route table lookup to find the interface
to transmit from.
The v4/v6 laddr paths don't know about flowtable; they just do a
routing table lookup.
I'm currently experimenting with a kernel fix to use flowtable here
for IPv4 UDP sockets but I'm not sure about the IPv6 path (the source
interface/address selection is much crazier with all the various scope
bits going on.)
I'd like some help with the IPv6 source selection please!
Thanks!
-a
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-VmomvWV0uxrxh%2BP3UbGOpc1rvY89UXnFfwWJ-j%2B%2B-n1m=BA>