From owner-freebsd-net@FreeBSD.ORG Tue May 6 19:43:44 2008 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CAF15106566C for ; Tue, 6 May 2008 19:43:44 +0000 (UTC) (envelope-from oleksandr@samoylyk.sumy.ua) Received: from mail.telesweet.net (news.telesweet.net [194.110.252.16]) by mx1.freebsd.org (Postfix) with ESMTP id 4390C8FC1C for ; Tue, 6 May 2008 19:43:43 +0000 (UTC) (envelope-from oleksandr@samoylyk.sumy.ua) Received: from localhost (localhost [127.0.0.1]) by mail.telesweet.net (Postfix) with ESMTP id 4AD501006F; Tue, 6 May 2008 22:43:41 +0300 (EEST) X-Virus-Scanned: by Telesweet Mail Virus Scanner X-Spam-Flag: NO X-Spam-Score: -1.44 X-Spam-Level: X-Spam-Status: No, score=-1.44 tagged_above=-999 required=5 tests=[ALL_TRUSTED=-1.44] Received: from [10.0.0.109] (pigeon-work.telesweet [10.0.0.109]) by mail.telesweet.net (Postfix) with ESMTP id D0835B826; Tue, 6 May 2008 22:43:23 +0300 (EEST) Message-ID: <4820B4DD.80105@samoylyk.sumy.ua> Date: Tue, 06 May 2008 22:43:25 +0300 From: Oleksandr Samoylyk User-Agent: Thunderbird 2.0.0.12 (X11/20080227) MIME-Version: 1.0 To: Julian Elischer References: <48207C8B.4020509@samoylyk.sumy.ua> <48209BC4.5080602@elischer.org> In-Reply-To: <48209BC4.5080602@elischer.org> Content-Type: text/plain; charset=KOI8-U; format=flowed Content-Transfer-Encoding: 7bit Cc: net@freebsd.org Subject: Re: Problems with netgraph X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 May 2008 19:43:44 -0000 Julian Elischer wrote: > Oleksandr Samoylyk wrote: >> Dear developers, >> >> Please read this thread: >> http://lists.freebsd.org/pipermail/freebsd-net/2008-May/017891.html >> >> I'm using no encryption and no compression in mpd, so netgraph should >> fly. >> It seems to get bad after 500 sessions... >> >> In FreeBSD 6 - it's swi1: net - 100% CPU >> In FreeBSD 7 - it's em0 taskq - 100% CPU > > unfortunatly I've been totally ignoring this thread because it said > "trouble with em" in the topic.. > If you'd said > trouble with mpd then maybe I'd have looked earlier.. > >> >> After playing with it I can make guess that's a netgraph problem. >> >> Maybe it's a poor design in netgraph the way it handle its tables and >> it's is probably not designed to add so many interfaces. > > how many? > # vmstat -m | grep netgraph netgraph_msg 0 0K - 1355522 64,128,256,512,1024 netgraph_node 12110 3028K - 28829 256 netgraph_hook 29648 3706K - 66104 128 netgraph 8076 12663K - 15209 64,256,1024,4096 netgraph_sock 4 1K - 5489 128 netgraph_path 1 1K - 728112 16,32 netgraph_mppc 0 0K - 1 1024 netgraph_ksock 1349 169K - 3414 128 netgraph_iface 1346 169K - 2367 128 netgraph_ppp 1346 16152K - 2367 netgraph_bpf 16020 2003K - 33296 128 # ifconfig -a | grep ng | wc -l 1341 I saw a peak with about 1900 ng interfaces. That was tooo sloowwww.... last pid: 39304; load averages: 1.12, 1.23, 1.20 up 0+01:15:38 22:39:04 93 processes: 10 running, 70 sleeping, 13 waiting CPU states: 0.3% user, 0.0% nice, 15.3% system, 3.1% interrupt, 81.3% idle Mem: 46M Active, 7772K Inact, 109M Wired, 128K Cache, 16M Buf, 7756M Free Swap: 4096M Total, 4096M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 819 root 23 50 0 92764K 36424K select 1 0:00 207.18% mpd5 17 root 1 171 ki31 0K 16K RUN 1 71:41 100.00% idle: cpu1 14 root 1 171 ki31 0K 16K CPU4 4 69:16 100.00% idle: cpu4 15 root 1 171 ki31 0K 16K CPU3 3 68:11 100.00% idle: cpu3 29 root 1 -68 - 0K 16K CPU6 6 60:50 100.00% em0 taskq 18 root 1 171 ki31 0K 16K CPU0 0 71:58 97.17% idle: cpu0 16 root 1 171 ki31 0K 16K CPU2 2 68:13 95.65% idle: cpu2 11 root 1 171 ki31 0K 16K CPU7 7 66:23 86.38% idle: cpu7 13 root 1 171 ki31 0K 16K CPU5 5 65:28 71.97% idle: cpu5 19 root 1 -44 - 0K 16K WAIT 5 19:53 31.59% swi1: net 30 root 1 -68 - 0K 16K - 7 8:41 12.89% em1 taskq 12 root 1 171 ki31 0K 16K RUN 6 12:41 0.00% idle: cpu6 20 root 1 -32 - 0K 16K WAIT 3 1:12 0.00% swi4: clock # uname -v FreeBSD 7.0-STABLE #0: Mon May 5 01:11:23 EEST 2008 CPU: Intel(R) Xeon(R) CPU E5335 @ 2.00GHz (2000.01-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x6f7 Stepping = 7 Features=0xbfebfbff Features2=0x4e33d AMD Features=0x20100800 AMD Features2=0x1 Cores per package: 4 FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs usable memory = 8580038656 (8182 MB) avail memory = 8292810752 (7908 MB) # ngctl list ngctl: send msg: No buffer space available # cat /boot/loader.conf loader_logo="beastie" autoboot_delay="3" hw.em.rxd="1024" hw.em.txd="1024" net.inet.tcp.tcbhashsize="4096" kern.ipc.maxpipekva="32000000" net.graph.recvspace="128000" net.graph.maxdgram="128000" Linux with poptop was running quit smoothly. I thought that netgraph would be fast as hell. -- Oleksandr Samoylyk OVS-RIPE