From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 21 16:51:15 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2CB5E106566B for ; Tue, 21 Sep 2010 16:51:15 +0000 (UTC) (envelope-from ozkan.kirik@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id DBB498FC25 for ; Tue, 21 Sep 2010 16:51:14 +0000 (UTC) Received: by qwg5 with SMTP id 5so5016819qwg.13 for ; Tue, 21 Sep 2010 09:51:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=5BCveOt79RqRp3ivO/xbk1bkfxos3Of73yZbYWFDhtM=; b=AxDjGzgVpeVMthJSXbHafE8544PdSMeZ/cX0LYAgn4nU2XH1qxbocEw8oIa6j1LE99 uu5VY42WsOgWNsVJhZhW4bKhkFG3IpkbEIZnrnXHGaOpcMVTLMG0BNUF8zRzTrdfNakI 8Exhj70qm8mtxvEjN9gtXqDUu1vJWthiUwcSg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=co8cthDVkT1rHRQqFMTJ+4BeLvkv8hWAxbQgdGUmtyNt87HPoThS2hiT67PDXLyHAU BVpf0iVlM3l+gwQS2hkNPyEE0n7K+iNhZNr8uYE/F3mAdWhSB2jQkF5TpD7TTIgw3EzU qDcIy302ODcP0n9fMJQOngxehPOluIM8R66ug= MIME-Version: 1.0 Received: by 10.229.251.79 with SMTP id mr15mr7530655qcb.37.1285086198944; Tue, 21 Sep 2010 09:23:18 -0700 (PDT) Received: by 10.229.192.204 with HTTP; Tue, 21 Sep 2010 09:23:18 -0700 (PDT) Date: Tue, 21 Sep 2010 19:23:18 +0300 Message-ID: From: =?ISO-8859-1?Q?=D6zkan_KIRIK?= To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Kernel side buffer overflow issue X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Sep 2010 16:51:15 -0000 Hi, I am using FreeBSD 8.1-STABLE-201008 snapshot. System behaves strangely. Unexpected and meaningless messages seen at consoles. You can download the screen shot from : http://193.255.128.30/~ryland/syslogd.jpg Additionally default router changes unexpectedly. I tried all FreeBSD 7.1, 7.2, 7.3, 8.1-STABLE-201008 releases ( both i386 and amd64 ). All this versions are affected. I inspected logs if someone logged in or changed route (with route -n monitor command). When the default route changed, there isn't any messages at the "route -n monitor" command output. I think there can be a buffer overflow in kernel code. When dummynet enabled, this problem could be seen more frequently. This problem repeats once per 10 minute. I wrote a shell script which monitors the default router. I saw that sometimes netstat -rn shows that default router is changed as 10.0.16.251 or 10.6.10.240 etc. which are client IP addresses but routing still routes to right router 193.X.Y.Z . After a while, routing really fails. You can download the tcpdump capture file from http://193.255.128.30/~ryland/flowdata_10_0_16_251 . This file captured while the default router changes. Tcpdump capture, belongs to the IP Address which shown in default router (10.0.16.251) the tcpdump command: tcpdump -w /home/flowdata_10_0_16_251 -ni bce0.116 host 10.0.16.251 ---------------------------------------------------------------------- dummynet rules are: 30000 pipe 3 tcp from 10.0.0.0/8,192.168.0.0/16,172.16.0.0/12 to any dst-port 8000,80,22,25,88,110,443,1720,1863,1521,3389,4489 via em0 // Upload 30000 pipe 3 udp from 10.0.0.0/8,192.168.0.0/16,172.16.0.0/12 to any dst-port 53 via em0 // Upload 30000 pipe 4 tcp from 10.0.0.0/8,192.168.0.0/16,172.16.0.0/12 to any via em0 // Upload 30000 pipe 4 udp from 10.0.0.0/8,192.168.0.0/16,172.16.0.0/12 to any via em0 // Upload .... LOTS OF NAT RULES HERE (in kernel nat) 60000 pipe 1 tcp from any 8000,80,22,25,88,110,443,1720,1863,1521,3389,4489 to 10.0.0.0/8,192.168.0.0/16,172.16.0.0/12 via bce0* // Download 60000 pipe 1 udp from any 53 to 10.0.0.0/8,192.168.0.0/16,172.16.0.0/12 via bce0* // Download 60000 pipe 2 tcp from any to 10.0.0.0/8,192.168.0.0/16,172.16.0.0/12 via bce0* // Download 60000 pipe 2 udp from any to 10.0.0.0/8,192.168.0.0/16,172.16.0.0/12 via bce0* // Download /sbin/ipfw pipe 1 config bw 8192Kbit/s mask dst-ip 0xffffffff /sbin/ipfw pipe 3 config bw 1024Kbit/s mask src-ip 0xffffffff /sbin/ipfw pipe 2 config bw 4096Kbit/s mask dst-ip 0xffffffff /sbin/ipfw pipe 4 config bw 1024Kbit/s mask src-ip 0xffffffff ---------------------------------------------------------------------- sysctl vars: net.inet.ip.fw.dyn_max=65535 net.inet.ip.fw.dyn_ack_lifetime=100 net.inet.ip.fw.dyn_short_lifetime=10 net.inet.ip.fw.one_pass=0 kern.maxfiles=65000 kern.ipc.somaxconn=1024 net.inet.ip.process_options=0 net.inet.ip.fastforwarding=1 net.link.ether.ipfw=1 net.inet.ip.fw.dyn_buckets=65536 kern.maxvnodes=400000 net.inet.ip.dummynet.hash_size=256 ( also tried with 8192 ) net.inet.ip.dummynet.pipe_slot_limit=500 net.inet.ip.dummynet.io_fast=1 ---------------------------------------------------------------------- /boot/loader.conf: autoboot_delay="1" beastie_disable="YES" kern.ipc.nmbclusters=98304 vm.kmem_size="2048M" vm.kmem_size_max="2048M" splash_bmp_load="YES" vesa_load="YES" bitmap_load="YES" bitmap_name="/boot/splash.bmp" hw.ata.ata_dma=0 kern.hz="10000" ---------------------------------------------------------------------- kernel config ( additionally to GENERIC ): device tap device if_bridge device vlan device carp options GEOM_BDE options IPFIREWALL options IPFIREWALL_VERBOSE options HZ=4000 options IPFIREWALL_VERBOSE_LIMIT=4000 options IPFIREWALL_FORWARD options IPFIREWALL_DEFAULT_TO_ACCEPT options IPFIREWALL_NAT options DUMMYNET options IPDIVERT options IPSTEALTH options NETGRAPH options NETGRAPH_IPFW options LIBALIAS options NETGRAPH_NAT options NETGRAPH_PPPOE options NETGRAPH_SOCKET options NETGRAPH_ETHER options DEVICE_POLLING device crypto options IPSEC ---------------------------------------------------------------------- Some Information about network: System has 3 NICS as WAN, LAN, DMZ. There are VLANs on WAN and LAN interfaces Throuput between 20Mbps and 100Mbps. Any ideas? Regards, Ozkan KIRIK Mersin University @ Turkey