From owner-freebsd-questions Tue Aug 28 14:18:50 2001 Delivered-To: freebsd-questions@freebsd.org Received: from aries.ai.net (aries.ai.net [205.134.163.4]) by hub.freebsd.org (Postfix) with ESMTP id 4D55237B403; Tue, 28 Aug 2001 14:18:34 -0700 (PDT) (envelope-from deepak@ai.net) Received: from blood (pool-138-88-45-161.res.east.verizon.net [138.88.45.161]) by aries.ai.net (8.9.3/8.9.3) with SMTP id RAA15488; Tue, 28 Aug 2001 17:21:42 -0400 (EDT) (envelope-from deepak@ai.net) Reply-To: From: "Deepak Jain" To: "Tom Samplonius" Cc: "FreeBSD-Questions" , "freebsd-isp@FreeBSD. ORG" Subject: RE: Interesting Router Question Date: Tue, 28 Aug 2001 17:20:41 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) In-Reply-To: Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Thanks for the response - The MBUFs, during the attack, were never maxed. If they were, it would have been a simple matter to address the problem. Watching the netstat -i on the customer router showed an INSTANT return to normal behavior within seconds of applying the list on the Cisco. The cisco was trapping 4000 icmps per second. This router does not run Apache and has normally nothing to pay attention to other than routing. All packets through the box were seeing send-buffer exhaustion. Even when the threshold for responses went up, the problem did not leave and the MBUF situation did not report a difference. Thanks for the input, Deepak Jain AiNET -----Original Message----- From: owner-freebsd-questions@FreeBSD.ORG [mailto:owner-freebsd-questions@FreeBSD.ORG]On Behalf Of Tom Samplonius Sent: Monday, August 27, 2001 9:17 PM To: Deepak Jain Cc: FreeBSD-Questions; freebsd-isp@FreeBSD. ORG Subject: Re: Interesting Router Question On Mon, 27 Aug 2001, Deepak Jain wrote: > The router was under some kind of ICMP attack: > > For about 30 minutes: > icmp-response bandwidth limit 96304/200 pps ... Looks like multiple attempts to open ports that have nothing listening. An aggressive port scan will do that. So will shutting Apache down while lots of people are hitting your web site. > The router is a dual 600mhz PIII and had a load average of about 0.2 peak > during the entire event, but was running out of buffer space. A ping would > return "No buffer space available". Performance became atrocious with high > packet loss and latency, but completely buffer related. You need more buffer space then. > The mbuf settings are as follows: > > 1235/2640/67584 mbufs in use (current/peak/max): > 1195 mbufs allocated to data > 40 mbufs allocated to packet headers > 592/1054/16896 mbuf clusters in use (current/peak/max) > 2768 Kbytes allocated to network (5% of mb_map in use) > 0 requests for memory denied > 0 requests for memory delayed > 0 calls to protocol drain routines I assume this after a reboot, not directly after or during the attack, as none of the maximums have been hit. You should double your mbuf clusters. ... > What settings need to be tweaked to allow more ICMP-related buffers to allow > the system's CPU to discard packets normally. ipfw didn't help or hurt this > performance [i.e., blocking ICMPs or not] same result. > > The solution was to install an ICMP filter on the Cisco feeding this > customer. You need to find out what ports the traffic was directed at, and stop that. You can also have FreeBSD ignore open packets to unused ports. You could use ipfw to block traffic directed at the system's own interfaces. That will probably fix the problem entirely. Since it is only routing, it should not receive traffic destined to its own IPs from outside the network. I'm not convinced that the ICMP filter on the Cisco actually helped. The site might be under several kinds of attacks, but what you've shown is that the server is sending ICMP, not receiving. > > Under normal circumstances, this is what a netstat -i 1 returns: > > input (Total) output > packets errs bytes packets errs bytes colls > 43001 0 12845737 42965 0 12715776 0 > 42589 0 12426503 42624 0 12299112 0 > 42485 0 12804047 42409 0 12675087 0 > 42059 0 12324347 42060 0 12197342 0 > 42989 0 13004977 42985 0 12875017 0 > 42331 0 12608670 42353 0 12481620 0 > 42327 0 12941571 42252 0 12815136 0 > 42435 0 12414956 42451 0 12288774 0 > 43408 0 13065007 43369 0 12932819 0 > 42849 0 12649420 42853 0 12521309 0 > 42328 0 12918886 42349 0 12788549 0 > 44085 0 13469072 44009 0 13337215 0 > 47849 0 14434350 47686 0 14272423 0 > > Thanks for any assistance, > > Deepak Jain > AiNET Tom To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message