From owner-freebsd-net@freebsd.org Tue May 29 11:33:07 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 838FFF746AC for ; Tue, 29 May 2018 11:33:07 +0000 (UTC) (envelope-from satan@ukr.net) Received: from hell.ukr.net (hell.ukr.net [212.42.67.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0E07A84D76 for ; Tue, 29 May 2018 11:33:06 +0000 (UTC) (envelope-from satan@ukr.net) Received: from satan by hell.ukr.net with local ID 1fNcsC-000FYI-As ; Tue, 29 May 2018 14:33:04 +0300 Date: Tue, 29 May 2018 14:33:04 +0300 From: Vitalij Satanivskij To: "Andrey V. Elsukov" Cc: Vitalij Satanivskij , freebsd-net@freebsd.org Subject: Re: GRE/gif/netgraph tunnel speed on 10Gbit channel Message-ID: <20180529113304.GA59501@hell.ukr.net> References: <20180529071733.GA18182@hell.ukr.net> <58deda3e-6eda-bade-0461-97014a64182e@yandex.ru> <20180529100544.GA58705@hell.ukr.net> <347581ff-1f32-c8c4-dce8-9c4632f4cd84@yandex.ru> <20180529105853.GA59230@hell.ukr.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.0 (2018-05-17) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 May 2018 11:33:07 -0000 AVE> AVE> So, is there no difference at all? The same bit rate with and without AVE> loaded module? Can you share your configs and parameters used for testing? KERNEL - GENERIC No additional config in src.conf make.conf, no sysct.conf or anything else in configs 1) no module mtu on ixl - 1500 mtu on gre0 - 1476 iperf -s on one side iperf -c xx.xx.xx.xx on not tunneled ip (just external ip of machine) [ 5] 1.70-1.80 sec 112 MBytes 9.42 Gbits/sec 0 1001 KBytes [ 5] 1.80-1.90 sec 112 MBytes 9.42 Gbits/sec 0 1.03 MBytes [ 5] 1.90-2.00 sec 112 MBytes 9.42 Gbits/sec 0 1.08 MBytes [ 5] 2.00-2.10 sec 111 MBytes 9.27 Gbits/sec 0 1.15 MBytes [ 5] 2.10-2.20 sec 112 MBytes 9.42 Gbits/sec 0 1.19 MBytes [ 5] 2.20-2.30 sec 112 MBytes 9.42 Gbits/sec 0 1.26 MBytes [ 5] 2.30-2.40 sec 112 MBytes 9.42 Gbits/sec 0 1.30 MBytes [ 5] 2.40-2.50 sec 112 MBytes 9.42 Gbits/sec 0 1.36 MBytes [ 5] 2.50-2.60 sec 112 MBytes 9.42 Gbits/sec 0 1.43 MBytes [ 5] 2.60-2.70 sec 112 MBytes 9.41 Gbits/sec 0 1.48 MBytes [ 5] 2.70-2.80 sec 112 MBytes 9.42 Gbits/sec 0 1.54 MBytes [ 5] 2.80-2.90 sec 112 MBytes 9.41 Gbits/sec 0 1.59 MBytes [ 5] 2.90-3.00 sec 112 MBytes 9.42 Gbits/sec 0 1.65 MBytes [ 5] 3.00-3.10 sec 112 MBytes 9.41 Gbits/sec 0 1.70 MBytes [ 5] 3.10-3.20 sec 112 MBytes 9.42 Gbits/sec 0 1.75 MBytes [ 5] 3.20-3.30 sec 112 MBytes 9.41 Gbits/sec 0 1.75 MBytes [ 5] 3.30-3.40 sec 112 MBytes 9.41 Gbits/sec 0 1.75 MBytes [ 5] 3.40-3.50 sec 112 MBytes 9.42 Gbits/sec 0 1.75 MBytes [ 5] 3.50-3.60 sec 112 MBytes 9.42 Gbits/sec 0 1.75 MBytes vmstat 1 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad0 ad1 in sy cs us sy id 1 0 0 257M 116G 869 0 5 0 4157 9 0 0 7147 112204 21811 0 1 99 0 0 0 257M 116G 11 0 0 0 0 12 2 2 101 292 296 0 0 100 1 0 0 264M 116G 407 0 2 0 58 12 41 40 31104 278655 102434 1 5 94 1 0 0 264M 116G 2 0 0 0 0 12 0 0 56021 197963 193978 0 7 93 1 0 0 264M 116G 1 0 0 0 0 12 0 0 57059 137992 188298 0 7 93 1 0 0 272M 116G 770 0 5 0 375 12 1 3 71077 254412 243456 0 5 95 1 0 0 272M 116G 1 0 0 0 0 13 0 0 75368 206223 247025 0 6 94 1 0 0 272M 116G 0 0 0 0 0 13 40 41 69876 247532 240423 0 5 95 And same via GRE tunnel [ 5] 6.70-6.80 sec 20.9 MBytes 1.75 Gbits/sec 0 898 KBytes [ 5] 6.80-6.90 sec 21.0 MBytes 1.76 Gbits/sec 0 898 KBytes [ 5] 6.90-7.00 sec 21.2 MBytes 1.77 Gbits/sec 0 898 KBytes [ 5] 7.00-7.10 sec 21.1 MBytes 1.75 Gbits/sec 0 898 KBytes [ 5] 7.10-7.20 sec 20.5 MBytes 1.74 Gbits/sec 0 898 KBytes [ 5] 7.20-7.30 sec 21.0 MBytes 1.77 Gbits/sec 0 898 KBytes [ 5] 7.30-7.40 sec 20.7 MBytes 1.74 Gbits/sec 0 898 KBytes [ 5] 7.40-7.50 sec 21.6 MBytes 1.76 Gbits/sec 0 898 KBytes [ 5] 7.50-7.60 sec 20.9 MBytes 1.78 Gbits/sec 0 898 KBytes [ 5] 7.60-7.70 sec 20.7 MBytes 1.75 Gbits/sec 0 898 KBytes [ 5] 7.70-7.80 sec 21.1 MBytes 1.77 Gbits/sec 0 898 KBytes [ 5] 7.80-7.90 sec 21.2 MBytes 1.77 Gbits/sec 0 898 KBytes [ 5] 7.90-8.00 sec 21.0 MBytes 1.76 Gbits/sec 0 898 KBytes [ 5] 8.00-8.10 sec 20.9 MBytes 1.76 Gbits/sec 0 898 KBytes [ 5] 8.10-8.20 sec 21.2 MBytes 1.78 Gbits/sec 0 898 KBytes [ 5] 8.20-8.30 sec 21.5 MBytes 1.76 Gbits/sec 0 898 KBytes [ 5] 8.30-8.40 sec 20.4 MBytes 1.75 Gbits/sec 0 898 KBytes [ 5] 8.40-8.50 sec 20.6 MBytes 1.75 Gbits/sec 0 898 KBytes vmstat 1 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad0 ad1 in sy cs us sy id 1 0 0 255M 116G 617 0 3 0 2909 10 0 0 19471 121742 63686 0 2 98 0 0 0 261M 116G 343 0 0 0 58 12 0 0 111 520 389 0 0 100 1 0 0 262M 116G 65 0 2 0 0 11 0 0 21690 219265 74525 0 8 91 1 0 0 262M 116G 2 0 0 0 0 12 0 0 26270 49481 92287 0 11 89 1 0 0 262M 116G 1 0 0 0 0 11 0 0 25317 43089 90289 0 10 90 1 0 0 262M 116G 1 0 0 0 0 12 41 40 24708 38927 90367 0 7 93 1 0 0 262M 116G 0 0 0 0 0 12 0 0 23264 31238 85293 0 8 92 1 0 0 262M 116G 0 0 0 0 0 12 0 0 23148 30335 85191 0 8 92 1 0 0 262M 116G 1 0 0 0 0 12 0 0 23217 31722 85472 0 7 93 1 0 0 262M 116G 0 0 0 0 0 12 0 0 23302 31449 85535 0 7 93 2) module loaded via GRE [ 5] 5.00-5.10 sec 22.2 MBytes 1.86 Gbits/sec 0 1.16 MBytes [ 5] 5.10-5.20 sec 21.8 MBytes 1.83 Gbits/sec 0 1.18 MBytes [ 5] 5.20-5.30 sec 22.3 MBytes 1.87 Gbits/sec 0 1.21 MBytes [ 5] 5.30-5.40 sec 22.3 MBytes 1.88 Gbits/sec 0 1.23 MBytes [ 5] 5.40-5.50 sec 22.1 MBytes 1.86 Gbits/sec 0 1.26 MBytes [ 5] 5.50-5.60 sec 22.0 MBytes 1.84 Gbits/sec 0 1.28 MBytes [ 5] 5.60-5.70 sec 21.9 MBytes 1.84 Gbits/sec 0 1.30 MBytes [ 5] 5.70-5.80 sec 21.9 MBytes 1.84 Gbits/sec 0 1.33 MBytes [ 5] 5.80-5.90 sec 22.9 MBytes 1.91 Gbits/sec 0 1.35 MBytes [ 5] 5.90-6.00 sec 24.8 MBytes 2.07 Gbits/sec 0 1.38 MBytes [ 5] 6.00-6.10 sec 24.5 MBytes 2.06 Gbits/sec 0 1.40 MBytes [ 5] 6.10-6.20 sec 24.5 MBytes 2.07 Gbits/sec 0 1.42 MBytes [ 5] 6.20-6.30 sec 24.6 MBytes 2.06 Gbits/sec 0 1.45 MBytes [ 5] 6.30-6.40 sec 24.7 MBytes 2.07 Gbits/sec 0 1.47 MBytes [ 5] 6.40-6.50 sec 24.7 MBytes 2.06 Gbits/sec 0 1.49 MBytes [ 5] 6.50-6.60 sec 24.4 MBytes 2.06 Gbits/sec 0 1.52 MBytes [ 5] 6.60-6.70 sec 24.8 MBytes 2.06 Gbits/sec 0 1.54 MBytes [ 5] 6.70-6.80 sec 24.4 MBytes 2.06 Gbits/sec 0 1.56 MBytes [ 5] 6.80-6.90 sec 24.2 MBytes 2.04 Gbits/sec 0 1.58 MBytes [ 5] 6.90-7.00 sec 24.4 MBytes 2.04 Gbits/sec 0 1.60 MBytes [ 5] 7.00-7.10 sec 24.3 MBytes 2.04 Gbits/sec 0 1.62 MBytes [ 5] 7.10-7.20 sec 24.6 MBytes 2.03 Gbits/sec 0 1.64 MBytes [ 5] 7.20-7.30 sec 23.8 MBytes 2.02 Gbits/sec 0 1.66 MBytes [ 5] 7.30-7.40 sec 23.5 MBytes 1.96 Gbits/sec 0 1.68 MBytes [ 5] 7.40-7.50 sec 23.0 MBytes 1.94 Gbits/sec 0 1.70 MBytes [ 5] 7.50-7.60 sec 21.3 MBytes 1.80 Gbits/sec 0 1.72 MBytes ^C[ 5] 7.60-7.70 sec 20.6 MBytes 1.73 Gbits/sec 0 1.73 MBytes vmstat 1 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad0 ad1 in sy cs us sy id 1 0 0 255M 116G 468 0 2 0 2186 10 0 0 15520 93549 51073 0 2 98 0 0 0 261M 116G 344 0 0 0 58 11 0 0 114 517 321 0 0 100 1 0 0 262M 116G 64 0 2 0 0 11 0 0 17815 194736 63712 0 8 92 1 0 0 262M 116G 1 0 0 0 0 12 0 0 27141 49249 98206 0 11 89 1 0 0 262M 116G 2 0 0 0 0 12 42 40 22715 41965 91683 0 10 90 1 0 0 262M 116G 0 0 0 0 0 11 0 0 25471 68910 100217 0 8 92 1 0 0 262M 116G 1 0 0 0 0 12 0 0 34890 102570 127725 0 7 93 1 0 0 262M 116G 0 0 0 0 0 12 0 0 33420 93203 124609 0 7 93 1 0 0 262M 116G 1 0 0 0 0 12 0 0 29117 66641 115689 0 7 93 So yes small improvement exist.