Date: Fri, 21 Jun 1996 15:37:08 +0200 (MET DST) From: grog@lemis.de (Greg Lehey) To: dennis@etinc.com (Dennis) Cc: hackers@freebsd.org (FreeBSD Hackers) Subject: Re: IP question: who should return ENOBUFS? Message-ID: <199606211337.PAA11280@allegro.lemis.de> In-Reply-To: <199606202107.RAA08181@etinc.com> from "Dennis" at Jun 20, 96 05:07:08 pm
next in thread | previous in thread | raw e-mail | index | archive | help
Dennis writes:
>
>> I Cc'd David on this since I'm not sure he reads hackers, and he's
>> responsible for the piece of code in question.
>>
>>> I've just spent a fair amount of time trying to figure out why our
>>> ISDN software jams up if it can't establish a connection. The
>>> symptoms are that if you can't establish a connection before a certain
>>> number of packets have been sent, the whole interface just returns:
>>>
>>> === root@freebie (/dev/ttyp0) /sys/netinet 11 -> ping 192.109.197.38
>>> PING 192.109.197.38 (192.109.197.38): 56 data bytes
>>> ping: sendto: No buffer space available
>>> ping: wrote 192.109.197.38 64 chars, ret=-1
>>> ping: sendto: No buffer space available
>> ....
>>
>> . After a bit more investigation, I found this code in
>>> ip_output.c:
>>>
>>> /*
>>> * Verify that we have any chance at all of being able to queue
>>> * the packet or packet fragments
>>> */
>>> if ((ifp->if_snd.ifq_len + ip->ip_len / ifp->if_mtu + 1) >=
>>> ifp->if_snd.ifq_maxlen) {
>>> error = ENOBUFS;
>>> goto bad;
>>> }
>>>
>>> I think this is bogus. It's not present in the BSD/OS version, so I
>>> assume it was added in FreeBSD. The problem is, it gives you no
>>> possibility of recovery: the queue is full and stays that way. Can
>>> anybody give me an idea of why it's there, when the interface is
>>> perfectly capable of looking after itself?
>>
>> Here's the log messages when David made the change.
>>
>> ----------------------------
>> revision 1.3
>> date: 1994/08/01 12:01:45; author: davidg; state: Exp; lines: +10 -0
>> fixed bug where large amounts of unidirectional UDP traffic would fill
>> the interface output queue and further udp packets would be fragmented
>> and only partially sent - keeping the output queue full and jamming the
>> network, but not actually getting any real work done (because you can't
>> send just 'part' of a udp packet - if you fragment it, you must send
>> the whole thing). The fix involves adding a check to make sure that the
>> output queue has sufficient space for all of the fragments.
>
> ENOBUFS causes a QUENCH to be sent, which is arguably correct
> if the problem is truly a queue overload.
Not here it doesn't. This code just drops the packet and tells the
application what it did. In particular, in my scenario (inability to
establish an ISDN connection), the queue stays full. The only
solution is to take the interface down and bring it back up again.
> Another problem is that you need queue management based on the available
> bandwidth of the interface. You can't have the same management
> scheme or thresholds for a 10Mbs interface as you do for a 56kbs one.
Correct. I think that ISDN is another case where the queue management
needs to be rethought, since we're betting that we can establish a
connection, and if we lose our bet, we're in bad shape. I'd say that
if we can't establish a connection by the timeout, we should flush the
whole queue, but maybe somebody has a better idea.
> This is precisely why we have the interface manage the queue depth.
> Additionally, queue managment should be based on queue size in bits
> (or bytes), not number of packets. 100 60 byte packets is not backup...
> 100 1000 byte packets is. Using packet counts is simply wrong.
Seems reasonable.
Greg
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199606211337.PAA11280>