From owner-freebsd-net@FreeBSD.ORG Tue Sep 25 08:20:11 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4A053106566B for ; Tue, 25 Sep 2012 08:20:03 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id BC09D8FC08 for ; Tue, 25 Sep 2012 08:20:03 +0000 (UTC) Received: by pbbrp8 with SMTP id rp8so4822456pbb.13 for ; Tue, 25 Sep 2012 01:20:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:in-reply-to:mime-version:content-transfer-encoding :content-type:message-id:cc:x-mailer:from:subject:date:to; bh=OJHGPHGys/ummV8hFd59LBf2uevBPeBdXeIWAui1ovo=; b=Yce20qFnLaRkjamnjjm8876nfbg1bRixnQeu4UWE8dW51Rha1V1AUKgeGXnarp8L4r irqBbAqFo5iPQIdLUxCuI3tsy3Bh2EJBVaJZ+5OazlQ2lvJfRwC6fONPY0DG/6Ma03lv 34Gxor8Yy+yy/SMj3PR63HjuaMHoV4tbZDkHluqofsQM1T0jWlvOb0gmIcABs3BWI52F jN5wmYm3Hfqu+cRLp4Vnp9L9IIEcrQyqL30J+iaZX3+sZkQkB/2jN9D4y2KTTTgtNqWB Yu1TZ7KTyDuGbKs/X/BMLCkfs1ZbTQ+7kMio+pEVIjZz64v+MDIuwzZ63BJNSWdOCNAe DvUA== Received: by 10.68.237.38 with SMTP id uz6mr44414249pbc.23.1348561202891; Tue, 25 Sep 2012 01:20:02 -0700 (PDT) Received: from [192.168.20.12] (c-24-19-191-56.hsd1.wa.comcast.net. [24.19.191.56]) by mx.google.com with ESMTPS id oe5sm2215707pbb.8.2012.09.25.01.20.01 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 25 Sep 2012 01:20:02 -0700 (PDT) References: <5060884C.3050709@monkeybrains.net> <506154C7.3040209@sepehrs.com> <50615F6F.1070105@monkeybrains.net> In-Reply-To: <50615F6F.1070105@monkeybrains.net> Mime-Version: 1.0 (1.0) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Message-Id: <834A688D-CDFC-44A1-91BE-0A3E1CFE9207@gmail.com> X-Mailer: iPhone Mail (9B206) From: Garrett Cooper Date: Tue, 25 Sep 2012 01:20:00 -0700 To: "Rudy (bulk)" Cc: freebsd-net , Hooman Fazaeli Subject: Re: ping: sendto: No buffer space available X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Sep 2012 08:20:11 -0000 On Sep 25, 2012, at 12:38 AM, "Rudy (bulk)" wrote:= > On 9/24/12 11:52 PM, Hooman Fazaeli wrote: >> sysctl dev.em.1=20 >=20 > =46rom the side having the 'No buffer space available' (FreeBSD 8.3 Sep 1= 3 2012) >=20 > # sysctl dev.em.1 > dev.em.1.%desc: Intel(R) PRO/1000 Network Connection 7.3.2 > dev.em.1.%driver: em > dev.em.1.%location: slot=3D0 function=3D0 > dev.em.1.%pnpinfo: vendor=3D0x8086 device=3D0x10d3 subvendor=3D0x15d9 subd= evice=3D0x0000 class=3D0x020000 > dev.em.1.%parent: pci5 > dev.em.1.nvm: -1 > dev.em.1.debug: -1 > dev.em.1.fc: 3 > dev.em.1.rx_int_delay: 0 > dev.em.1.tx_int_delay: 66 > dev.em.1.rx_abs_int_delay: 66 > dev.em.1.tx_abs_int_delay: 66 > dev.em.1.rx_processing_limit: 200 > dev.em.1.eee_control: 0 > dev.em.1.link_irq: 6379725883 > dev.em.1.mbuf_alloc_fail: 0 > dev.em.1.cluster_alloc_fail: 0 > dev.em.1.dropped: 0 > dev.em.1.tx_dma_fail: 0 > dev.em.1.rx_overruns: 0 > dev.em.1.watchdog_timeouts: 0 > dev.em.1.device_control: 1477444168 > dev.em.1.rx_control: 67141634 > dev.em.1.fc_high_water: 18432 > dev.em.1.fc_low_water: 16932 > dev.em.1.queue0.txd_head: 188 > dev.em.1.queue0.txd_tail: 188 > dev.em.1.queue0.tx_irq: 760427663 > dev.em.1.queue0.no_desc_avail: 0 > dev.em.1.queue0.rxd_head: 300 > dev.em.1.queue0.rxd_tail: 297 > dev.em.1.queue0.rx_irq: 838300057 > dev.em.1.mac_stats.excess_coll: 0 > dev.em.1.mac_stats.single_coll: 0 > dev.em.1.mac_stats.multiple_coll: 0 > dev.em.1.mac_stats.late_coll: 0 > dev.em.1.mac_stats.collision_count: 0 > dev.em.1.mac_stats.symbol_errors: 0 > dev.em.1.mac_stats.sequence_errors: 0 > dev.em.1.mac_stats.defer_count: 0 > dev.em.1.mac_stats.missed_packets: 580251107926 > dev.em.1.mac_stats.recv_no_buff: 895 > dev.em.1.mac_stats.recv_undersize: 0 > dev.em.1.mac_stats.recv_fragmented: 0 > dev.em.1.mac_stats.recv_oversize: 0 > dev.em.1.mac_stats.recv_jabber: 0 > dev.em.1.mac_stats.recv_errs: 0 > dev.em.1.mac_stats.crc_errs: 0 > dev.em.1.mac_stats.alignment_errs: 0 > dev.em.1.mac_stats.coll_ext_errs: 0 > dev.em.1.mac_stats.xon_recvd: 809 > dev.em.1.mac_stats.xon_txd: 684 > dev.em.1.mac_stats.xoff_recvd: 580251112172 > dev.em.1.mac_stats.xoff_txd: 580251108668 > dev.em.1.mac_stats.total_pkts_recvd: 582154845658 > dev.em.1.mac_stats.good_pkts_recvd: 1903732156 > dev.em.1.mac_stats.bcast_pkts_recvd: 923 > dev.em.1.mac_stats.mcast_pkts_recvd: 0 > dev.em.1.mac_stats.rx_frames_64: 257128416 > dev.em.1.mac_stats.rx_frames_65_127: 702676478 > dev.em.1.mac_stats.rx_frames_128_255: 225331435 > dev.em.1.mac_stats.rx_frames_256_511: 59888288 > dev.em.1.mac_stats.rx_frames_512_1023: 47777176 > dev.em.1.mac_stats.rx_frames_1024_1522: 610930363 > dev.em.1.mac_stats.good_octets_recvd: 1057190106675 > dev.em.1.mac_stats.good_octets_txd: 1502996801989 > dev.em.1.mac_stats.total_pkts_txd: 582709483882 > dev.em.1.mac_stats.good_pkts_txd: 2458374408 > dev.em.1.mac_stats.bcast_pkts_txd: 73 > dev.em.1.mac_stats.mcast_pkts_txd: 0 > dev.em.1.mac_stats.tx_frames_64: 314613253 > dev.em.1.mac_stats.tx_frames_65_127: 841961719 > dev.em.1.mac_stats.tx_frames_128_255: 268669868 > dev.em.1.mac_stats.tx_frames_256_511: 73341358 > dev.em.1.mac_stats.tx_frames_512_1023: 62765737 > dev.em.1.mac_stats.tx_frames_1024_1522: 897022473 > dev.em.1.mac_stats.tso_txd: 1880 > dev.em.1.mac_stats.tso_ctx_fail: 0 > dev.em.1.interrupts.asserts: 6331439142 > dev.em.1.interrupts.rx_pkt_timer: 0 > dev.em.1.interrupts.rx_abs_timer: 0 > dev.em.1.interrupts.tx_pkt_timer: 0 > dev.em.1.interrupts.tx_abs_timer: 0 > dev.em.1.interrupts.tx_queue_empty: 0 > dev.em.1.interrupts.tx_queue_min_thresh: 0 > dev.em.1.interrupts.rx_desc_min_thresh: 0 > dev.em.1.interrupts.rx_overrun: 74346455 >=20 >=20 > And the the other end of the link (FreeBSD 9.0-STABLE Feb 1 2012) >=20 > # sysctl dev.em.2 > dev.em.2.%desc: Intel(R) PRO/1000 Network Connection 7.2.3 > dev.em.2.%driver: em > dev.em.2.%location: slot=3D0 function=3D0 > dev.em.2.%pnpinfo: vendor=3D0x8086 device=3D0x10d3 subvendor=3D0x15d9 subd= evice=3D0x10d3 class=3D0x020000 > dev.em.2.%parent: pci7 > dev.em.2.nvm: -1 > dev.em.2.debug: -1 > dev.em.2.rx_int_delay: 0 > dev.em.2.tx_int_delay: 66 > dev.em.2.rx_abs_int_delay: 66 > dev.em.2.tx_abs_int_delay: 66 > dev.em.2.rx_processing_limit: 100 > dev.em.2.flow_control: 3 > dev.em.2.eee_control: 0 > dev.em.2.link_irq: 6379294926 > dev.em.2.mbuf_alloc_fail: 0 > dev.em.2.cluster_alloc_fail: 0 > dev.em.2.dropped: 0 > dev.em.2.tx_dma_fail: 0 > dev.em.2.rx_overruns: 0 > dev.em.2.watchdog_timeouts: 0 > dev.em.2.device_control: 1477444168 > dev.em.2.rx_control: 67141634 > dev.em.2.fc_high_water: 18432 > dev.em.2.fc_low_water: 16932 > dev.em.2.queue0.txd_head: 735 > dev.em.2.queue0.txd_tail: 735 > dev.em.2.queue0.tx_irq: 839960061 > dev.em.2.queue0.no_desc_avail: 0 > dev.em.2.queue0.rxd_head: 237 > dev.em.2.queue0.rxd_tail: 236 > dev.em.2.queue0.rx_irq: 762108556 > dev.em.2.mac_stats.excess_coll: 0 > dev.em.2.mac_stats.single_coll: 0 > dev.em.2.mac_stats.multiple_coll: 0 > dev.em.2.mac_stats.late_coll: 0 > dev.em.2.mac_stats.collision_count: 0 > dev.em.2.mac_stats.symbol_errors: 0 > dev.em.2.mac_stats.sequence_errors: 0 > dev.em.2.mac_stats.defer_count: 0 > dev.em.2.mac_stats.missed_packets: 580252415422 > dev.em.2.mac_stats.recv_no_buff: 3211 > dev.em.2.mac_stats.recv_undersize: 0 > dev.em.2.mac_stats.recv_fragmented: 0 > dev.em.2.mac_stats.recv_oversize: 0 > dev.em.2.mac_stats.recv_jabber: 0 > dev.em.2.mac_stats.recv_errs: 0 > dev.em.2.mac_stats.crc_errs: 0 > dev.em.2.mac_stats.alignment_errs: 0 > dev.em.2.mac_stats.coll_ext_errs: 0 > dev.em.2.mac_stats.xon_recvd: 684 > dev.em.2.mac_stats.xon_txd: 811 > dev.em.2.mac_stats.xoff_recvd: 580252412652 > dev.em.2.mac_stats.xoff_txd: 580252416163 > dev.em.2.mac_stats.total_pkts_recvd: 582710830267 > dev.em.2.mac_stats.good_pkts_recvd: 2458413107 > dev.em.2.mac_stats.bcast_pkts_recvd: 162 > dev.em.2.mac_stats.mcast_pkts_recvd: 0 > dev.em.2.mac_stats.rx_frames_64: 314616449 > dev.em.2.mac_stats.rx_frames_65_127: 841966916 > dev.em.2.mac_stats.rx_frames_128_255: 268689241 > dev.em.2.mac_stats.rx_frames_256_511: 73346849 > dev.em.2.mac_stats.rx_frames_512_1023: 62768450 > dev.em.2.mac_stats.rx_frames_1024_1522: 897025202 > dev.em.2.mac_stats.good_octets_recvd: 1503009815756 > dev.em.2.mac_stats.good_octets_txd: 1056695683673 > dev.em.2.mac_stats.total_pkts_txd: 582155360950 > dev.em.2.mac_stats.good_pkts_txd: 1902943976 > dev.em.2.mac_stats.bcast_pkts_txd: 8862 > dev.em.2.mac_stats.mcast_pkts_txd: 5 > dev.em.2.mac_stats.tx_frames_64: 257064868 > dev.em.2.mac_stats.tx_frames_65_127: 702386644 > dev.em.2.mac_stats.tx_frames_128_255: 225240646 > dev.em.2.mac_stats.tx_frames_256_511: 59862797 > dev.em.2.mac_stats.tx_frames_512_1023: 47743269 > dev.em.2.mac_stats.tx_frames_1024_1522: 610645752 > dev.em.2.mac_stats.tso_txd: 12910 > dev.em.2.mac_stats.tso_ctx_fail: 0 > dev.em.2.interrupts.asserts: 6331624353 > dev.em.2.interrupts.rx_pkt_timer: 1 > dev.em.2.interrupts.rx_abs_timer: 0 > dev.em.2.interrupts.tx_pkt_timer: 0 > dev.em.2.interrupts.tx_abs_timer: 0 > dev.em.2.interrupts.tx_queue_empty: 0 > dev.em.2.interrupts.tx_queue_min_thresh: 0 > dev.em.2.interrupts.rx_desc_min_thresh: 0 > dev.em.2.interrupts.rx_overrun: 74250608 >=20 >=20 > I should note that I had the two devices set to mtu of 9000 at first, ... Good data point! I was using jumbo frames at the time (9k buffers). Differen= ce being that I raised my buffer sizes to ridiculous limits (250k iirc). Thanks! -Garrett=