From owner-freebsd-stable@freebsd.org Thu Nov 1 06:53:55 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0B1E710DD5BB for ; Thu, 1 Nov 2018 06:53:55 +0000 (UTC) (envelope-from gcr+freebsd-stable@tharned.org) Received: from roadkill.tharned.org (tunnel294749-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:107f::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "", Issuer "Tharned" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8F1376BFA7 for ; Thu, 1 Nov 2018 06:53:54 +0000 (UTC) (envelope-from gcr+freebsd-stable@tharned.org) Received: from flake.tharned.org ([IPv6:2001:470:1f11:107f:f471:2353:b30c:aa7a]) (authenticated bits=0) by roadkill.tharned.org (8.15.2/8.15.2) with ESMTPSA id wA16rFIP033191 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 1 Nov 2018 01:53:52 -0500 (CDT) (envelope-from gcr+freebsd-stable@tharned.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tharned.org; s=2017; t=1541055232; bh=I42lA0B8j4kPx8uC13GR19mLRjvK7G82htx3uVPs6Ms=; h=From:To:Subject:Date; b=wJO4GVX9b1hXc7hpi0czygV3hnh+xQcqOZmO2q5VUtVxzvcxbdE/9abq5OKH7IsjK pMZBtrDmnkwSIcoqGErumH7ZNDP/GMTcCS6ZmtfzWESBdwmZjZHsqV8g76mWGAyPWG /qxatjsHCnctjYAhBVdyqEK/6jcfGgM6fSOctUPgW0Q1RNY6sPIixuiUs+G/i/p2LZ 4Gj3+EAlf0hgfLryDdl1ppyrXbGRVqVXCRjnvlXX2g7GJJHATMn9TS3PYnoeZQYJMj UMOPCrdUgvcxtxwyPJ/3DjD1WmwNv71ogLpG4nQqrYGzhdHMEp3fTdzKwcyUI8b2fq BhzLPxPubPOTQ== X-Authentication-Warning: roadkill.tharned.org: Host [IPv6:2001:470:1f11:107f:f471:2353:b30c:aa7a] claimed to be flake.tharned.org From: Greg Rivers To: freebsd-stable@freebsd.org Subject: High UDP loss on 11.2-RELEASE-p4 Date: Thu, 01 Nov 2018 01:53:15 -0500 Message-ID: <5249318.XtdPABXNba@no.place.like.home> User-Agent: KMail/4.14.10 (FreeBSD/11.2-RELEASE-p4; KDE/4.14.38; amd64; ; ) MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.2 (roadkill.tharned.org [IPv6:2001:470:1f10:107f:0:0:0:2]); Thu, 01 Nov 2018 01:53:52 -0500 (CDT) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Nov 2018 06:53:55 -0000 I'm seeing considerable packet loss running recursive DNS (BIND 9.12.2) on 11.2. This is on server grade hardware (HP DL380G8) with an Intel 10Gb NIC (ix). I spun up iperf3; the results were disappointing: $ iperf3 --get-server-output -c r5 -u -b 1g -R Connecting to host r5, port 5201 Reverse mode, remote host r5 is sending [ 5] local xxx.xxx.242.11 port 56921 connected to xxx.xxx.235.238 port 5201 [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams [ 5] 0.00-1.00 sec 77.0 MBytes 645 Mbits/sec 0.003 ms 32835/88105 (37%) [ 5] 1.00-2.00 sec 72.2 MBytes 605 Mbits/sec 0.002 ms 33813/85638 (39%) [ 5] 2.00-3.00 sec 71.3 MBytes 599 Mbits/sec 0.003 ms 34291/85534 (40%) [ 5] 3.00-4.00 sec 70.0 MBytes 587 Mbits/sec 0.012 ms 35459/85757 (41%) [ 5] 4.00-5.00 sec 71.9 MBytes 603 Mbits/sec 0.002 ms 34053/85688 (40%) [ 5] 5.00-6.00 sec 72.7 MBytes 609 Mbits/sec 0.004 ms 33354/85555 (39%) [ 5] 6.00-7.00 sec 71.7 MBytes 602 Mbits/sec 0.002 ms 33984/85472 (40%) [ 5] 7.00-8.00 sec 71.0 MBytes 596 Mbits/sec 0.002 ms 34618/85623 (40%) [ 5] 8.00-9.00 sec 70.8 MBytes 594 Mbits/sec 0.002 ms 34773/85630 (41%) [ 5] 9.00-10.00 sec 71.3 MBytes 598 Mbits/sec 0.002 ms 34593/85784 (40%) - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams [ 5] 0.00-10.00 sec 1.17 GBytes 1.00 Gbits/sec 0.000 ms 0/858786 (0%) sender [ 5] 0.00-10.00 sec 720 MBytes 604 Mbits/sec 0.002 ms 341773/858786 (40%) receiver Server output: ----------------------------------------------------------- Accepted connection from xxx.xxx.242.11, port 54163 [ 5] local xxx.xxx.235.238 port 5201 connected to xxx.xxx.242.11 port 56921 [ ID] Interval Transfer Bandwidth Total Datagrams [ 5] 0.00-1.00 sec 111 MBytes 930 Mbits/sec 79657 [ 5] 1.00-2.00 sec 119 MBytes 999 Mbits/sec 85536 [ 5] 2.00-3.00 sec 119 MBytes 1.00 Gbits/sec 85625 [ 5] 3.00-4.00 sec 119 MBytes 999 Mbits/sec 85541 [ 5] 4.00-5.00 sec 119 MBytes 1.00 Gbits/sec 85667 [ 5] 5.00-6.00 sec 119 MBytes 1000 Mbits/sec 85592 [ 5] 6.00-7.00 sec 119 MBytes 1.00 Gbits/sec 85720 [ 5] 7.00-8.00 sec 119 MBytes 998 Mbits/sec 85457 [ 5] 8.00-9.00 sec 119 MBytes 1.00 Gbits/sec 85622 [ 5] 9.00-10.00 sec 119 MBytes 1000 Mbits/sec 85603 [ 5] 10.00-10.10 sec 12.2 MBytes 1.02 Gbits/sec 8767 - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams [ 5] 0.00-10.10 sec 1.17 GBytes 993 Mbits/sec 0.000 ms 0/858787 (0%) 40% loss at 1Gb. I'm running with the following tunes, which did improve DNS service when I applied them: dev.ix.0.tx_processing_limit: -1 hw.ix.flow_control: 0 hw.ix.num_queues: 8 hw.ix.rx_process_limit: -1 hw.ix.rxd: 2048 hw.ix.tx_process_limit: -1 hw.ix.txd: 2048 kern.ipc.maxsockbuf: 16777216 kern.random.harvest.mask: 351 machdep.hyperthreading_allowed: 0 net.inet.ip.intr_queue_maxlen: 2048 net.inet.raw.maxdgram: 16384 net.inet.raw.recvspace: 16384 net.isr.bindthreads: 1 net.isr.defaultqlimit: 2048 net.isr.maxthreads: 16 net.link.ifqmaxlen: 4096 net.local.stream.recvspace: 16384 net.local.stream.sendspace: 16384 net.route.netisr_maxqlen: 2048 Are there any known problems with 11.2 and UDP? Can anyone else reproduce this? Any insight or suggestions? -- Greg