Date: Wed, 30 Jul 2003 12:32:02 -0400 From: Haesu <haesu@towardex.com> To: freebsd-isp@freebsd.org, freebsd-questions@freebsd.org Subject: sendto: No buffer space available Message-ID: <20030730163202.GA8118@scylla.towardex.com>
next in thread | raw e-mail | index | archive | help
Hello,
We have a FreeBSD box here that we use to route some GRE tunnels and ipv6 gif tunnels. We use zebra for dynamic routing running zebra, bgpd, ospfd, and ospf6d.
We have about 12 FreeBSD boxes with exact same configuration, the only
difference is just the IP address of each interface.
None of them fail but this one box...
Everyday, this box stops all networking. I can still console in and stuff.. When
I typed 'ping 127.0.0.1' at the console after networking locked up, it says:
ping: sendto: No buffer space available
The only solution seems to be rebooting it everyday... It happens every 12 hours
or so...
This is not related with mbuf, etc either, as netstat -m doesn't show any
issues.
The box has one IP address and IPv6 address in addition to 127.0.0.1 on lo0
interface. It also has a ds0 interface with 10.5.5.5/30 assigned to ds0.
This is exact same configuration on all other boxes, and none of them fail but
this one.
I've swapped out NICs with different vendors 3 times (tried, xl, dc, and now rl)
I've also swapped out the whole box, and also swapped out the whole hard drive
and did full reinstall. And problem still persists and it's definately not
hardware as I swapped everything out... (unless the 3 NIC vendors above are all
exhibiting same issue)
I tried to look on Google but nothing useful that corelates to this particular
issue..
Any help would be very appreciated :)
Thanks,
-hc
The box is running FreeBSD 4.8-STABLE
FreeBSD necsis 4.8-STABLE FreeBSD 4.8-STABLE #0: Tue Jul 29 13:10:11 GMT 2003 root@necsis:/usr/src/sys/compile/router i386
Following is output of netstat -s AFTER the networking locks up with no buffer
space available error:
tcp:
30115 packets sent
17167 data packets (1232152 bytes)
301 data packets (54320 bytes) retransmitted
0 resends initiated by MTU discovery
12416 ack-only packets (10931 delayed)
0 URG only packets
0 window probe packets
41 window update packets
280 control packets
28010 packets received
16762 acks (for 1236693 bytes)
140 duplicate acks
0 acks for unsent data
13205 packets (567038 bytes) received in-sequence
43 completely duplicate packets (818 bytes)
0 old duplicate packets
2 packets with some dup. data (38 bytes duped)
9 out-of-order packets (240 bytes)
0 packets (0 bytes) of data after window
0 window probes
31 window update packets
0 packets received after close
0 discarded for bad checksums
0 discarded for bad header offset fields
0 discarded because packet too short
252 connection requests
18 connection accepts
6 bad connection attempts
0 listen queue overflows
30 connections established (including accepts)
288 connections closed (including 10 drops)
23 connections updated cached RTT on close
23 connections updated cached RTT variance on close
11 connections updated cached ssthresh on close
164 embryonic connections dropped
16643 segments updated rtt (of 16929 attempts)
1566 retransmit timeouts
10 connections dropped by rexmit timeout
0 persist timeouts
0 connections dropped by persist timeout
161 keepalive timeouts
0 keepalive probes sent
161 connections dropped by keepalive
96 correct ACK header predictions
10392 correct data packet header predictions
19 syncache entries added
6 retransmitted
2 dupsyn
0 dropped
18 completed
0 bucket overflow
0 cache overflow
0 reset
0 stale
0 aborted
0 badack
1 unreach
0 zone failures
0 cookies sent
0 cookies received
udp:
196 datagrams received
0 with incomplete header
0 with bad data length field
0 with bad checksum
1 with no checksum
61 dropped due to no socket
3 broadcast/multicast datagrams dropped due to no socket
0 dropped due to full socket buffers
0 not for hashed pcb
132 delivered
132 datagrams output
ip:
2154646 total packets received
0 bad header checksums
0 with size smaller than minimum
0 with data size < data length
0 with ip length > max ip packet size
0 with header length < data size
0 with data length < header length
0 with bad options
0 with incorrect version number
366 fragments received
0 fragments dropped (dup or out of space)
0 fragments dropped after timeout
183 packets reassembled ok
1118217 packets for this host
11 packets for unknown/unsupported protocol
1031566 packets forwarded (0 packets fast forwarded)
265 packets not forwardable
3573 packets received for unknown multicast group
0 redirects sent
772758 packets sent from this host
13697 packets sent with fabricated ip header
0 output packets dropped due to no bufs, etc.
0 output packets discarded due to no route
0 output datagrams fragmented
0 fragments created
366 datagrams that can't be fragmented
0 tunneling packets that can't find gif
0 datagrams with bad address in header
icmp:
1075 calls to icmp_error
0 errors not generated 'cuz old message was icmp
Output histogram:
echo reply: 28
destination unreachable: 244
time exceeded: 831
0 messages with bad code fields
0 messages < minimum length
0 bad checksums
0 messages with bad length
0 multicast echo requests ignored
0 multicast timestamp requests ignored
Input histogram:
destination unreachable: 21
echo: 28
time exceeded: 111
28 message responses generated
1 invalid return address
1 no return route
ICMP address mask responses are disabled
igmp:
0 messages received
0 messages received with too few bytes
0 messages received with bad checksum
0 membership queries received
0 membership queries received with invalid field(s)
0 membership reports received
0 membership reports received with invalid field(s)
0 membership reports received for groups to which we belong
9 membership reports sent
ip6:
26952 total packets received
0 with size smaller than minimum
0 with data size < data length
0 with bad options
0 with incorrect version number
0 fragments received
0 fragments dropped (dup or out of space)
0 fragments dropped after timeout
0 fragments that exceeded limit
0 packets reassembled ok
20061 packets for this host
35 packets forwarded
108 packets not forwardable
0 redirects sent
29834 packets sent from this host
0 packets sent with fabricated ip header
0 output packets dropped due to no bufs, etc.
82 output packets discarded due to no route
0 output datagrams fragmented
0 fragments created
0 datagrams that can't be fragmented
0 packets that violated scope rules
108 multicast packets which we don't join
Input histogram:
TCP: 12122
UDP: 12
ICMP6: 6761
OSPF: 8057
Mbuf statistics:
26308 one mbuf
644 one ext mbuf
0 two or more ext mbuf
0 packets whose headers are not continuous
0 tunneling packets that can't find gif
0 packets discarded due to too may headers
0 failures of source address selection
source addresses on an outgoing I/F
4 link-locals
44 globals
source addresses of same scope
4 link-locals
44 globals
9346 forward cache hit
10034 forward cache miss
icmp6:
0 calls to icmp_error
0 errors not generated because old message was icmp error or so
0 errors not generated because rate limitation
Output histogram:
multicast listener report: 49
multicast listener done: 4
neighbor solicitation: 4265
neighbor advertisement: 3239
0 messages with bad code fields
0 messages < minimum length
0 bad checksums
0 messages with bad length
Input histogram:
unreach: 3
neighbor solicitation: 3239
neighbor advertisement: 3506
Histogram of error messages to be generated:
0 no route
0 administratively prohibited
0 beyond scope
0 address unreachable
0 port unreachable
0 packet too big
0 time exceed transit
0 time exceed reassembly
0 erroneous header field
0 unrecognized next header
0 unrecognized option
0 redirect
0 unknown
0 message responses generated
0 messages with too many ND options
0 messages with bad ND options
0 bad neighbor solicitation messages
0 bad neighbor advertisement messages
0 bad router solicitation messages
0 bad router advertisement messages
0 bad redirect messages
0 path MTU changes
--
Sincerely,
Haesu C.
TowardEX Technologies, Inc.
WWW: http://www.towardex.com
E-mail: haesu@towardex.com
Cell: (978) 394-2867
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20030730163202.GA8118>