Date: Fri, 24 Jan 2014 00:26:34 -0500 From: J David <j.david.lists@gmail.com> To: Rick Macklem <rmacklem@uoguelph.ca> Cc: freebsd-net@freebsd.org Subject: Re: Terrible NFS performance under 9.2-RELEASE? Message-ID: <CABXB=RSp2qkaNubuqmLFRiB2gaZK6oizmJy%2BmdCSh5xJ25UhTQ@mail.gmail.com> In-Reply-To: <CABXB=RRP=N5_sOQrhov_3WVHd3awRjT4-oqd-BUrKgmPPk6fpg@mail.gmail.com> References: <CABXB=RR5NgRpqXA8XQDks3HhxC6QQW3aG%2BsH4BUxFL7kbFuJ_g@mail.gmail.com> <58591523.15519962.1390533482068.JavaMail.root@uoguelph.ca> <CABXB=RRP=N5_sOQrhov_3WVHd3awRjT4-oqd-BUrKgmPPk6fpg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Here's a pair of quick tcpdumps with TSO/LRO on and off showing that tcpdump does not reassemble small packets into larger ones: TSO/LRO on: 05:21:54.956061 IP 172.20.20.169.24265 > 172.20.20.162.5001: Flags [S], seq 2538331145, win 65535, options [mss 1460,nop,wscale 9,sackOK,TS val 2932122 ecr 0], length 0 05:21:54.956239 IP 172.20.20.162.5001 > 172.20.20.169.24265: Flags [S.], seq 3371756775, ack 2538331146, win 65535, options [mss 1460,nop,wscale 9,sackOK,TS val 2833562423 ecr 2932122], length 0 05:21:54.956292 IP 172.20.20.169.24265 > 172.20.20.162.5001: Flags [.], ack 1, win 2050, options [nop,nop,TS val 2932132 ecr 2833562423], length 0 05:21:54.956372 IP 172.20.20.169.24265 > 172.20.20.162.5001: Flags [P.], seq 1:25, ack 1, win 2050, options [nop,nop,TS val 2932132 ecr 2833562423], length 24 05:21:54.956432 IP 172.20.20.162.5001 > 172.20.20.169.24265: Flags [.], ack 1, win 2050, options [nop,nop,TS val 2833562423 ecr 2932132], length 0 05:21:54.956495 IP 172.20.20.169.24265 > 172.20.20.162.5001: Flags [.], seq 25:4381, ack 1, win 2050, options [nop,nop,TS val 2932132 ecr 2833562423], length 4356 05:21:54.956604 IP 172.20.20.162.5001 > 172.20.20.169.24265: Flags [.], ack 4381, win 2041, options [nop,nop,TS val 2833562423 ecr 2932132], length 0 05:21:54.956620 IP 172.20.20.169.24265 > 172.20.20.162.5001: Flags [.], seq 4381:11657, ack 1, win 2050, options [nop,nop,TS val 2932132 ecr 2833562423], length 7276 05:21:55.050656 IP 172.20.20.162.5001 > 172.20.20.169.24265: Flags [.], ack 11657, win 2050, options [nop,nop,TS val 2833562523 ecr 2932132], length 0 05:21:55.050686 IP 172.20.20.169.24265 > 172.20.20.162.5001: Flags [.], seq 11657:21829, ack 1, win 2050, options [nop,nop,TS val 2932222 ecr 2833562523], length 10172 05:21:55.150644 IP 172.20.20.162.5001 > 172.20.20.169.24265: Flags [.], ack 21829, win 2050, options [nop,nop,TS val 2833562623 ecr 2932222], length 0 05:21:55.150674 IP 172.20.20.169.24265 > 172.20.20.162.5001: Flags [.], seq 21829:34897, ack 1, win 2050, options [nop,nop,TS val 2932322 ecr 2833562623], length 13068 05:21:55.250629 IP 172.20.20.162.5001 > 172.20.20.169.24265: Flags [.], ack 34897, win 2050, options [nop,nop,TS val 2833562723 ecr 2932322], length 0 05:21:55.250658 IP 172.20.20.169.24265 > 172.20.20.162.5001: Flags [.], seq 34897:50861, ack 1, win 2050, options [nop,nop,TS val 2932422 ecr 2833562723], length 15964 05:21:55.350647 IP 172.20.20.162.5001 > 172.20.20.169.24265: Flags [.], ack 50861, win 2050, options [nop,nop,TS val 2833562823 ecr 2932422], length 0 05:21:55.350677 IP 172.20.20.169.24265 > 172.20.20.162.5001: Flags [.], seq 50861:69721, ack 1, win 2050, options [nop,nop,TS val 2932522 ecr 2833562823], length 18860 05:21:55.450656 IP 172.20.20.162.5001 > 172.20.20.169.24265: Flags [.], ack 69721, win 2050, options [nop,nop,TS val 2833562923 ecr 2932522], length 0 05:21:55.450686 IP 172.20.20.169.24265 > 172.20.20.162.5001: Flags [.], seq 69721:91477, ack 1, win 2050, options [nop,nop,TS val 2932622 ecr 2833562923], length 21756 05:21:55.550577 IP 172.20.20.162.5001 > 172.20.20.169.24265: Flags [.], ack 91477, win 2050, options [nop,nop,TS val 2833563023 ecr 2932622], length 0 05:21:55.550608 IP 172.20.20.169.24265 > 172.20.20.162.5001: Flags [.], seq 91477:116129, ack 1, win 2050, options [nop,nop,TS val 2932722 ecr 2833563023], length 24652 05:21:55.650645 IP 172.20.20.162.5001 > 172.20.20.169.24265: Flags [.], ack 116129, win 2050, options [nop,nop,TS val 2833563123 ecr 2932722], length 0 05:21:55.650676 IP 172.20.20.169.24265 > 172.20.20.162.5001: Flags [.], seq 116129:143677, ack 1, win 2050, options [nop,nop,TS val 2932822 ecr 2833563123], length 27548 05:21:55.750643 IP 172.20.20.162.5001 > 172.20.20.169.24265: Flags [.], ack 143677, win 2050, options [nop,nop,TS val 2833563223 ecr 2932822], length 0 05:21:55.750675 IP 172.20.20.169.24265 > 172.20.20.162.5001: Flags [.], seq 143677:174121, ack 1, win 2050, options [nop,nop,TS val 2932922 ecr 2833563223], length 30444 05:21:55.850636 IP 172.20.20.162.5001 > 172.20.20.169.24265: Flags [.], ack 174121, win 2050, options [nop,nop,TS val 2833563323 ecr 2932922], length 0 05:21:55.850667 IP 172.20.20.169.24265 > 172.20.20.162.5001: Flags [.], seq 174121:207461, ack 1, win 2050, options [nop,nop,TS val 2933022 ecr 2833563323], length 33340 TSO/LRO off: 05:19:34.556302 IP 172.20.20.169.53485 > 172.20.20.162.5001: Flags [S], seq 529322163, win 65535, options [mss 1460,nop,wscale 9,sackOK,TS val 2791722 ecr 0], length 0 05:19:34.556414 IP 172.20.20.162.5001 > 172.20.20.169.53485: Flags [S.], seq 3835815533, ack 529322164, win 65535, options [mss 1460,nop,wscale 9,sackOK,TS val 1931664416 ecr 2791722], length 0 05:19:34.556443 IP 172.20.20.169.53485 > 172.20.20.162.5001: Flags [.], ack 1, win 2050, options [nop,nop,TS val 2791732 ecr 1931664416], length 0 05:19:34.556505 IP 172.20.20.169.53485 > 172.20.20.162.5001: Flags [P.], seq 1:25, ack 1, win 2050, options [nop,nop,TS val 2791732 ecr 1931664416], length 24 05:19:34.556604 IP 172.20.20.169.53485 > 172.20.20.162.5001: Flags [.], seq 25:1473, ack 1, win 2050, options [nop,nop,TS val 2791732 ecr 1931664416], length 1448 05:19:34.556621 IP 172.20.20.162.5001 > 172.20.20.169.53485: Flags [.], ack 1, win 2050, options [nop,nop,TS val 1931664416 ecr 2791732], length 0 05:19:34.556648 IP 172.20.20.169.53485 > 172.20.20.162.5001: Flags [.], seq 1473:2921, ack 1, win 2050, options [nop,nop,TS val 2791732 ecr 1931664416], length 1448 05:19:34.556672 IP 172.20.20.169.53485 > 172.20.20.162.5001: Flags [.], seq 2921:4369, ack 1, win 2050, options [nop,nop,TS val 2791732 ecr 1931664416], length 1448 05:19:34.556711 IP 172.20.20.162.5001 > 172.20.20.169.53485: Flags [.], ack 1473, win 2047, options [nop,nop,TS val 1931664416 ecr 2791732], length 0 05:19:34.556730 IP 172.20.20.169.53485 > 172.20.20.162.5001: Flags [.], seq 4369:5817, ack 1, win 2050, options [nop,nop,TS val 2791732 ecr 1931664416], length 1448 05:19:34.556743 IP 172.20.20.169.53485 > 172.20.20.162.5001: Flags [.], seq 5817:7265, ack 1, win 2050, options [nop,nop,TS val 2791732 ecr 1931664416], length 1448 05:19:34.556755 IP 172.20.20.162.5001 > 172.20.20.169.53485: Flags [.], ack 4369, win 2041, options [nop,nop,TS val 1931664416 ecr 2791732], length 0 05:19:34.556774 IP 172.20.20.169.53485 > 172.20.20.162.5001: Flags [.], seq 7265:8713, ack 1, win 2050, options [nop,nop,TS val 2791732 ecr 1931664416], length 1448 05:19:34.556793 IP 172.20.20.169.53485 > 172.20.20.162.5001: Flags [.], seq 8713:10161, ack 1, win 2050, options [nop,nop,TS val 2791732 ecr 1931664416], length 1448 05:19:34.556813 IP 172.20.20.169.53485 > 172.20.20.162.5001: Flags [.], seq 10161:11609, ack 1, win 2050, options [nop,nop,TS val 2791732 ecr 1931664416], length 1448 05:19:34.556830 IP 172.20.20.169.53485 > 172.20.20.162.5001: Flags [.], seq 11609:13057, ack 1, win 2050, options [nop,nop,TS val 2791732 ecr 1931664416], length 1448 05:19:34.556848 IP 172.20.20.162.5001 > 172.20.20.169.53485: Flags [.], ack 7265, win 2036, options [nop,nop,TS val 1931664416 ecr 2791732], length 0 05:19:34.556865 IP 172.20.20.169.53485 > 172.20.20.162.5001: Flags [.], seq 13057:14505, ack 1, win 2050, options [nop,nop,TS val 2791732 ecr 1931664416], length 1448 05:19:34.556881 IP 172.20.20.169.53485 > 172.20.20.162.5001: Flags [.], seq 14505:15953, ack 1, win 2050, options [nop,nop,TS val 2791732 ecr 1931664416], length 1448 05:19:34.556893 IP 172.20.20.169.53485 > 172.20.20.162.5001: Flags [.], seq 15953:17401, ack 1, win 2050, options [nop,nop,TS val 2791732 ecr 1931664416], length 1448 05:19:34.556912 IP 172.20.20.169.53485 > 172.20.20.162.5001: Flags [.], seq 17401:18849, ack 1, win 2050, options [nop,nop,TS val 2791732 ecr 1931664416], length 1448 05:19:34.556929 IP 172.20.20.162.5001 > 172.20.20.169.53485: Flags [.], ack 10161, win 2030, options [nop,nop,TS val 1931664416 ecr 2791732], length 0 The stated lengths represent the length of each IP packet as it appears on the "wire." The result is the same whether it is obtained from the client, the server, or the KVM host snooping directly on the bridge. Thanks!
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CABXB=RSp2qkaNubuqmLFRiB2gaZK6oizmJy%2BmdCSh5xJ25UhTQ>