From owner-freebsd-net@FreeBSD.ORG Mon Dec 29 22:20:41 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 417711065672 for ; Mon, 29 Dec 2008 22:20:41 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 055ED8FC1F for ; Mon, 29 Dec 2008 22:20:41 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTP id 8912846B17; Mon, 29 Dec 2008 17:20:40 -0500 (EST) Date: Mon, 29 Dec 2008 22:20:40 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Yony Yossef In-Reply-To: <20def4870812240600n6edbcad7k2100a0ccbe49f0dd@mail.gmail.com> Message-ID: References: <20def4870812240600n6edbcad7k2100a0ccbe49f0dd@mail.gmail.com> User-Agent: Alpine 1.10 (BSF 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: eitans@mellanox.co.il, freebsd-net@freebsd.org, Yehonatan Yossef , liranl@mellanox.co.il, yevgenyp@mellanox.co.il Subject: Re: net.inet.udp.blackhole issue X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Dec 2008 22:20:42 -0000 On Wed, 24 Dec 2008, Yony Yossef wrote: > I'm facing lots of UDP "Connection refused" errors while running multistream > iperf test. Analyzing it with wireshark showed several "ICMP Port > Unreachable" problems. > > I've overriden it with "sysctl net.inet.udp.blackhole=1", but I'm not sure > this is the correct thing to do, I feel like I've sweeped the problem under > the carpet. > > PS - I see similar failures with TCP bidirectional iperf test, it can also > be overriden by "sysctl net.inet.tcp.blackhole=1". > > My question is - can it be a NIC problem? If so, how can a driver problem > cause an iperf UDP socket to be in a "non listening state"? Hi Yony: This is fairly unlikely to be a NIC problem, although anything is possible in software. I'm not familiar with iperf, but generally speaking ICMP port unreachable is a result of packets arriving at a closed socket; net.inet.udp.blackhole suppresses that ICMP: if (udp_blackhole) goto badheadlocked; if (badport_bandlim(BANDLIM_ICMP_UNREACH) < 0) goto badheadlocked; *ip = save_ip; ip->ip_len += iphlen; icmp_error(m, ICMP_UNREACH, ICMP_UNREACH_PORT, 0, 0); I think I'd suspect an application bug/feature, in which socket gets closed and opened during execution and once in a while a datagram is delivered in that window. Perhaps packets are being delivered with a non-trivial delay causing them to arrive after the application has timed out waiting for it? Robert N M Watson Computer Laboratory University of Cambridge