Date: Fri, 22 Oct 2004 13:45:59 -0600 From: "Stephane Raimbault" <segr@hotmail.com> To: gnn@FreeBSD.org Cc: net@FreeBSD.org Subject: Re: Error 49, socket problem? Message-ID: <BAY24-F25XGA7zyWR4A0000aae2@hotmail.com>
next in thread | raw e-mail | index | archive | help
The servers are busier today and error 49 is comming up frequently now.
Here are some results of what you asked for.
# netstat -m
305/5152/26624 mbufs in use (current/peak/max):
305 mbufs allocated to data
292/3592/6656 mbuf clusters in use (current/peak/max)
8472 Kbytes allocated to network (42% of mb_map in use)
0 requests for memory denied
0 requests for memory delayed
0 calls to protocol drain routines
# ifconfig -a
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
options=3<rxcsum,txcsum>
inet 10.0.12.15 netmask 0xffffffe0 broadcast 10.0.12.31
inet6 fe80::230:48ff:fe29:7f48%em0 prefixlen 64 scopeid 0x1
ether 00:30:48:29:7f:48
media: Ethernet 100baseTX <full-duplex>
status: active
em1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
options=3<rxcsum,txcsum>
ether 00:30:48:29:7f:49
media: Ethernet autoselect
status: no carrier
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet 127.0.0.1 netmask 0xff000000
ppp0: flags=8010<POINTOPOINT,MULTICAST> mtu 1500
sl0: flags=c010<POINTOPOINT,LINK2,MULTICAST> mtu 552
faith0: flags=8002<BROADCAST,MULTICAST> mtu 1500
# netstat -rn
Routing tables
Internet:
Destination Gateway Flags Refs Use Netif Expire
default 10.0.12.1 UGSc 588 1292 em0
10.0.12/27 link#1 UC 13 0 em0
10.0.12.1 00:00:5e:00:01:05 UHLW 588 0 em0 1177
10.0.12.8 00:0c:f1:77:a5:d8 UHLW 97 7723205 em0 229
10.0.12.10 00:30:48:29:7f:34 UHLW 0 22 em0 552
10.0.12.11 00:30:48:29:80:38 UHLW 0 1165712 em0 987
10.0.12.12 00:30:48:29:80:d4 UHLW 0 172 em0 1018
10.0.12.13 00:30:48:27:75:61 UHLW 215 35228169 em0 1193
10.0.12.14 00:30:48:29:80:84 UHLW 1 27992157 em0 959
10.0.12.15 00:30:48:29:7f:48 UHLW 1 12597865 lo0
10.0.12.22 00:30:48:23:bb:f0 UHLW 2125 303510829 em0 1034
10.0.12.23 00:e0:81:21:1b:78 UHLW 0 242 em0 697
10.0.12.26 00:0d:61:03:80:98 UHLW 0 8736 em0 975
10.0.12.30 00:c0:4f:9e:b9:c7 UHLW 0 848 em0 1003
10.0.12.31 ff:ff:ff:ff:ff:ff UHLWb 0 2347026 em0
127.0.0.1 127.0.0.1 UH 1022 153926288 lo0
Internet6:
Destination Gateway Flags
Netif Expire
::1 ::1 UH
lo0
fe80::%em0/64 link#1 UC
em0
fe80::230:48ff:fe29:7f48%em0 00:30:48:29:7f:48 UHL
lo0
fe80::%lo0/64 fe80::1%lo0 Uc
lo0
fe80::1%lo0 link#3 UHL
lo0
ff01::/32 ::1 U
lo0
ff02::%em0/32 link#1 UC
em0
ff02::%lo0/32 ::1 UC
lo0
# uptime
1:39PM up 31 days, 3:02, 3 users, load averages: 1.34, 1.27, 1.07
# sysctl -a | grep socket
kern.ipc.maxsockets: 12328
socket: 192, 12328, 5320, 1080, 124222928
jail.socket_unixiproute_only: 1
# tail -f /wwwlogs/httpd-error.log
[Fri Oct 22 13:39:18 2004] [error] (49)Can't assign requested address:
proxy: HTTP: attempt to connect to 127.0.0.1:81 (127.0.0.1) failed
[Fri Oct 22 13:39:20 2004] [error] (49)Can't assign requested address:
proxy: HTTP: attempt to connect to 127.0.0.1:81 (127.0.0.1) failed
[Fri Oct 22 13:39:20 2004] [error] (49)Can't assign requested address:
proxy: HTTP: attempt to connect to 127.0.0.1:81 (127.0.0.1) failed
[Fri Oct 22 13:39:21 2004] [error] (49)Can't assign requested address:
proxy: HTTP: attempt to connect to 127.0.0.1:81 (127.0.0.1) failed
[Fri Oct 22 13:39:24 2004] [error] (49)Can't assign requested address:
proxy: HTTP: attempt to connect to 127.0.0.1:81 (127.0.0.1) failed
[Fri Oct 22 13:39:39 2004] [error] (49)Can't assign requested address:
proxy: HTTP: attempt to connect to 127.0.0.1:81 (127.0.0.1) failed
[Fri Oct 22 13:39:39 2004] [error] (49)Can't assign requested address:
proxy: HTTP: attempt to connect to 127.0.0.1:81 (127.0.0.1) failed
[Fri Oct 22 13:39:40 2004] [error] (49)Can't assign requested address:
proxy: HTTP: attempt to connect to 127.0.0.1:81 (127.0.0.1) failed
[Fri Oct 22 13:39:43 2004] [error] (49)Can't assign requested address:
proxy: HTTP: attempt to connect to 127.0.0.1:81 (127.0.0.1) failed
[Fri Oct 22 13:39:43 2004] [error] (49)Can't assign requested address:
proxy: HTTP: attempt to connect to 127.0.0.1:81 (127.0.0.1) failed
what else can I check for here... it's quite mysterious as several network
based apps on the server fail, yet nothing in /var/log/messages indicating a
problem.
>From: gnn@FreeBSD.org
>To: "Stephane Raimbault" <segr@hotmail.com>
>CC: net@FreeBSD.org
>Subject: Re: Error 49, socket problem?
>Date: Thu, 14 Oct 2004 10:46:58 +0900
>
>At Wed, 13 Oct 2004 09:36:03 -0600,
>Stephane Raimbault wrote:
> >
> > I doubt it's a DoS attack, however it could very well be.
> >
>
>Well, probably not, given what you told us below.
>
> > In this particular setup, apache runs on port 80 and 81. A slimed down
> > version of apache handles basic http requests on port 80 and on port 81,
>we
> > have a beefier version of apache with php running applications. We
>proxy
> > the request from port 80 to port 81 so the client only ever sees
>connections
> > to port 80.
> >
> > What I find strange is other applications on the server freak out when
>this
> > is happening in the same manner. such as php not able to make mysql
> > connections with a similar error. The errors aren't limited to
>127.0.0.1
> > either, it seems to be an overall problem with the box, not specifically
>the
> > loopback or apache.
> >
> > PHP Warning: mysql_connect(): Can't connect to MySQL server on
> > 'dbm.xxx.xxx.com' (49) in /www/index.php on line 4060
> >
> > In this case the above dbm.xxx.xxx.com resolves to 10.0.12.22 which is
>one
> > of the MySQL server's in the cluster.
> >
> > I've ruled out that it's a problem with the MySQL server in this case,
> > because I have 4 other web servers (running apache) able to connect to
>that
> > MySQL server during the same time. All web servers in this cluster seem
>to
> > exhibit the error randomly at diffrent times. Not only during high peak
> > traffic times as previously thought.
> >
> > Are there buffers I could be running out of? What should I be checking?
> > Perhaps something in sysctl... If I know what I should probably be
>looking
> > at, I can probably monitor and have certain variables logged / graphed
>for a
> > better idea of what is going on.
> >
>
>What version of FreeBSD are you running?
>
>I would check the interfaces (ifconfig -a), routing table (netstat
>-rn), and then the rest of the network statistics (man netstat) when
>this happens. Does it happen reliably or intermittently?
>
>Later,
>George
_________________________________________________________________
MSNŽ Calendar keeps you organized and takes the effort out of scheduling
get-togethers.
http://join.msn.com/?pgmarket=en-ca&page=byoa/prem&xAPID=1994&DI=1034&SU=http://hotmail.com/enca&HL=Market_MSNIS_Taglines
Start enjoying all the benefits of MSNŽ Premium right now and get the
first two months FREE*.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?BAY24-F25XGA7zyWR4A0000aae2>