From owner-freebsd-questions@FreeBSD.ORG Wed May 5 14:39:38 2004 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 50C5C16A4CF; Wed, 5 May 2004 14:39:38 -0700 (PDT) Received: from watcher.puryear-it.com (ip-66-186-248-99.eatel.net [66.186.248.99]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1D63B43DB9; Wed, 5 May 2004 14:38:22 -0700 (PDT) (envelope-from dap99@i-55.com) Received: from localhost (unknown [127.0.0.1]) by watcher.puryear-it.com (Postfix) with ESMTP id 0A9F134D1F; Wed, 5 May 2004 16:36:04 -0500 (CDT) Received: from watcher.puryear-it.com ([127.0.0.1]) by localhost (watcher.puryear-it.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 61052-01; Wed, 5 May 2004 16:36:02 -0500 (CDT) Received: from yourqqh4336axf (localhost [127.0.0.1]) by watcher.puryear-it.com (Postfix) with SMTP id 3078234D1E; Wed, 5 May 2004 16:36:02 -0500 (CDT) Message-ID: <015e01c432e9$1d77b650$6501a8c0@yourqqh4336axf> From: "adp" To: "Charles Swiger" References: <01a501c432ce$adf20e30$4b0a000a@yourqqh4336axf> <207CE1A8-9ECB-11D8-8DD7-003065ABFD92@mac.com> Date: Wed, 5 May 2004 16:34:28 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2720.3000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2739.300 X-Virus-Scanned: by amavisd-new cc: questions@freebsd.org cc: performance@freebsd.org Subject: Re: Abnormal network errors? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 May 2004 21:39:38 -0000 ----- Original Message ----- From: "Charles Swiger" > On May 5, 2004, at 2:27 PM, adp wrote: > > On this server I'm thinking I need two things: > > > > 1. More sockets available. > > 2. Larger sockbufs for send and recv. > > > > Is this an accurate assessment? > > Given the application of this system, you might want to up the value of > kern.ipc.nmbclusters by a factor of four or so (it's NBMCLUSTERS in the > kernel config file). However, it's not essential-- your "netstat -m" > is OK, and your TCP send and receive windows are reasonably sized as-is > by default. Several problems. First, we are hosting a DNS server on this box. The DNS resolves domains we are auth. for very fast, or anything in its cache very fast, but anything else is SLOW or times-out. Also, our www server (another box) is responding slowly in general (4-6 seconds). > > What is "2432320 packets for unknown/unsupported protocol"? What > > specifically does this mean? (In other words, what should I do to > > resolve > > this?) > > It means machines are sending non-IP traffic on your network, which is > normal if you have Windows protocols (NetBEUI, SPX/IPX) or Macs > (AppleTalk) around. Or chatty network devices like some printers.... What is 802.1d? I am getting a lot of this: 16:21:16.788617 802.1d config 8000.00:04:27:d1:cb:d3.8019 root 8000.00:03:6c:51:a2:a7 pathcost 8 age 2 max 20 hello 2 fdelay 15 And this: 16:21:15.424508 CDP v2, ttl=180s DevID 'Six2' Addr (1): IPv4 10.2.254.62 PortID 'FastEthernet0/12' CAP 0x0a[|cdp] I'm at a colo. > See /usr/include/net/ethernet.h for an idea, or maybe "tcpdump not ip" > might give some idea of what's going by. > > > What about "921363 calls to icmp_error"? > > ICMP messages like responding to a ping, or people sending traffic with > RFC-1918 unroutable addresses (gives "dest unreachable")... That's weird. I tried 'tcpdump icmp' and see a few errors right off the bat: # tcpdump -n icmp tcpdump: listening on rl0 16:27:46.633262 192.168.42.71 > 192.168.42.76: icmp: 192.168.42.71 udp port 1249 unreachable 16:27:53.639237 192.168.42.71 > 192.168.42.76: icmp: 192.168.42.71 udp port 1280 unreachable 16:28:02.579417 192.168.42.71 > 192.168.42.76: icmp: 192.168.42.71 udp port 1204 unreachable 16:28:07.716527 192.168.42.71 > 192.168.42.76: icmp: 192.168.42.71 udp port 1510 unreachable 16:28:08.589910 192.168.42.71 > 192.168.42.76: icmp: 192.168.42.71 udp port 1218 unreachable 16:28:15.668697 192.168.42.71 > 192.168.42.76: icmp: 192.168.42.71 udp port 1327 unreachable 16:28:33.581427 192.168.42.71 > 192.168.42.76: icmp: 192.168.42.71 udp port 1355 unreachable Hmm. I am running a DNS server in a jail on the NFS server. We have been getting very slow responses times from it. Seems related. > > Under tcp I have "481930 embryonic connections dropped". I assume that > > means > > I don't have enough sockets available for when this server gets loaded. > > Correct? > > More likely, these are someone doing a port scan and leaving half-open > connections lying around to get cleaned up. We are behind a managed NetScreen firewall. I can't see how anyone is port-scanning us, unless they are just scanning the few ports we have open to the world. > It might be helpful if you gave us some idea as to what the performance > problem you were seeing was? Is NFS access slow, or some such? Are > you seeing errors or collisions in netstat -i or in whatever statistics > your switch keeps per port? > > The following areas struck me as being relevant: > > > # ifconfig rl0 > > First, consider upgrading to a fxp or dc-based NIC. Noted. > > udp: > > 272987897 datagrams received > > [ ... ] > > 19976574 dropped due to full socket buffers > > This is high enough to represent a concern, agreed. How do I fix this then? I assume I don't have enough sockets available. Is there a way to see where I am peaking? I'm thinking adding more memory will increase the system-set defaults (also read 'man tuning'). > > ip: > > 578001924 total packets received > [ ... ] > > 4899083 fragments received > > 4 fragments dropped (dup or out of space) > > 750 fragments dropped after timeout > > 842689 packets reassembled ok > [ ... ] > > 609745425 packets sent from this host > > 1914687 output datagrams fragmented > > 10496350 fragments created > > Second, you're fragmenting a relatively large number of packets going > by, you ought to see what's going on with your MTU and pMTU discovery. > I suppose if you're using large UDP datagrams with NFS, that might be > it. > > [ The machines I've got around with comparible traffic volume might > have 400 frags received, and 10 transmitted, or some such. ] Could this be related to my switch or anything else?