From owner-freebsd-net@FreeBSD.ORG Sat Jul 19 03:34:49 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 70E9496E for ; Sat, 19 Jul 2014 03:34:49 +0000 (UTC) Received: from mail-qg0-x22f.google.com (mail-qg0-x22f.google.com [IPv6:2607:f8b0:400d:c04::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 30AF92F68 for ; Sat, 19 Jul 2014 03:34:49 +0000 (UTC) Received: by mail-qg0-f47.google.com with SMTP id i50so3769848qgf.20 for ; Fri, 18 Jul 2014 20:34:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=IsKxbeb78G3FjT0R+NRwPfnryv4r/GuQWF4t4XzaMoI=; b=qbsOEVTEIcEbonn7Ww6lxxsD6kINRCuLbwRfz15tlD+sWGkSK7KIWR5ybvEvMkzA9H VLvz63CSScSUyejxl/fUv2iLkKw3RkldWGtZfrsERoDJh8jdMzbq76bIrIDZngOoirHN JvFi46QuLgcGy7702iIBGOZ+/GepClIPAxNw29Pkgah5c5+1c6Fe/rpB41JZey261NdT OFg65SLaN1beq/l3CS3vGHtVV/TF+EujS0r9KrBhp4urQRU1gaw+bDjKtpCNokl/xang 7op2mc9cVbs7A1HBf0WBeQpOXXw4SeGCemMH8KsvkvSzaqodZG+h8Yr/rr4PGaFNSA/7 c9TA== MIME-Version: 1.0 X-Received: by 10.229.171.196 with SMTP id i4mr14934441qcz.15.1405740888293; Fri, 18 Jul 2014 20:34:48 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.1.6 with HTTP; Fri, 18 Jul 2014 20:34:48 -0700 (PDT) In-Reply-To: <20140719053318.I15959@besplex.bde.org> References: <20140719053318.I15959@besplex.bde.org> Date: Fri, 18 Jul 2014 20:34:48 -0700 X-Google-Sender-Auth: 2QgPtedPlrYmwj0jBHxikPv50sI Message-ID: Subject: Re: UDP sendto() returning ENOBUFS - "No buffer space available" From: Adrian Chadd To: Bruce Evans Content-Type: text/plain; charset=UTF-8 Cc: hiren panchasara , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Jul 2014 03:34:49 -0000 Hi, On 18 July 2014 13:40, Bruce Evans wrote: > On Fri, 18 Jul 2014, hiren panchasara wrote: > >> On Wed, Jul 16, 2014 at 11:00 AM, Adrian Chadd wrote: >>> >>> Hi! >>> >>> So the UDP transmit path is udp_usrreqs->pru_send() == udp_send() -> >>> udp_output() -> ip_output() >>> >>> udp_output() does do a M_PREPEND() which can return ENOBUFS. ip_output >>> can also return ENOBUFS. >>> >>> it doesn't look like the socket code (eg sosend_dgram()) is doing any >>> buffering - it's just copying the frame and stuffing it up to the >>> driver. No queuing involved before the NIC. >> >> >> Right. Thanks for confirming. > > > Most buffering should be in ifq above the NIC. For UDP, I think > udp_output() puts buffers on the ifq and calls the driver for every > one, but the driver shouldn't do anything for most calls. The > driver can't possibly do anything if its ring buffer is full, and > shouldn't do anything if it is nearly full. Buffers accumulate in > the ifq until the driver gets around to them or the queue fills up. > Most ENOBUFS errors are for when it fills up. It can very easily > fill up, especially since it is too small in most configurations. > Just loop calling sendto(). This will fill the ifq almost > instantly unless the hardware is faster than the software. For if_transmit() drivers, there's no ifp queue. The queuing is being done in the driver. For drivers with if_transmit(), they may end up doing direct DMA ring dispatch or they may have a buf_ring in front of it.There's no ifq anymore. It upsets the ALTQ people too. -a