Date: Sun, 06 Aug 2000 20:52:41 -0700 From: Peter Wemm <peter@netplex.com.au> To: Archie Cobbs <archie@whistle.com> Cc: jayanth <jayanth@yahoo-inc.com>, freebsd-net@FreeBSD.ORG Subject: Re: Max data queued for UDP Message-ID: <200008070352.UAA65444@netplex.com.au> In-Reply-To: <200008030001.RAA28226@bubba.whistle.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Archie Cobbs wrote: > jayanth writes: > > while running some tests for udp with very small packets(25 bytes) > > I found that the total amount of data that can be queued > > at the socket buffer(just queued, not read) is very limited, > > close to around 4k - 5k bytes. > > One of the issues is that the driver uses a cluster for > > every data packet, however small the data packet is. > > There is a per socket limit on the total amount of network > > memory that can be used for a socket (sb->sb_mbmax), which > > is typically 256k. There is also a variable that keeps > > count of the total n/w network memory actually used > > by the socket (sb->mb_cnt). The mb_cnt variable is > > incremented by a cluster + sizeof(mbuf) for every > > incoming packet. So for very small udp packets the total > > no. of packets that can be queued is > > sb_mbmax / (cluster + mbuf) = 256K / (2048 + 256) = 113 packets. > > At this point, the sbspace() macro declares that there is > > no space in the socket buffer, in terms of n/w memory. > > (sbmax - sb_mbcnt becomes -ve) > > > > TCP already has an sbcompress() and I have code that does > > the same for UDP. The code basically compresses the > > data from the cluster into the mbuf and frees the > > cluster, if the data fits in one mbuf. > > > > I am a little hesitant to do it in the driver code or > > the ether_input() code as it might do a copy for > > every protocol including tcp. > > > > feedback, please... > > IMHO, it makes sense to do it for UDP and raw sockets, but not > generically. TCP sockets generally copy the data in the recieve path to pack it into the socket buffers. UDP is the odd one out that enqueues the entire data mbuf (usually a cluster these days on a busmaster ethernet controller) and another mbuf for the address, and possibly a third mbuf for control data. For small UDP packets, this is hideously inefficient. Wasting a whole mbuf to hold a sockaddr_in (or sockaddr_in6) is terrible. Unconditionally wasting a whole mbuf+cluster (2.25K of KVM) to hold something like 50 bytes of data is pretty bad. Considering the advent of voice over IP (often UDP), we need a way to decide if or when to copy to a smaller object. Jayanth's patch pretty much does this. It moves the data from the cluster to the already-existing mbuf. A possible improvement might be to create a socket option or sysctl to specify a threshhold (or group of threshholds). Some sort of indicator about when to copy and when it is not worth it. > When you say "the driver uses a cluster for every data packet, > however small the data packet is" are you referring to a specific > Ethernet driver? If so then that driver is lame. It should at > least have two possibilities: (a) normal mbuf, and (b) cluster. > It's very easy and fast to make that minimal decision. Not when it is preallocated and the driver busmaster DMA's into a space provided by the driver. You cannot know in advance what the size of a packet will be. Some hardware has dual queues where you can use small and large inbound buffers and have the hardware choose an appropriate sized one. But this isn't available on most current 100baseT hardware. > -Archie Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200008070352.UAA65444>