From owner-freebsd-net@FreeBSD.ORG Sun May 27 07:20:08 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 468A1106566B for ; Sun, 27 May 2012 07:20:08 +0000 (UTC) (envelope-from crapsh@monkeybrains.net) Received: from lavash.monkeybrains.net (mail.monkeybrains.net [208.69.40.9]) by mx1.freebsd.org (Postfix) with ESMTP id 25D848FC0A for ; Sun, 27 May 2012 07:20:07 +0000 (UTC) Received: from Computer-of-Penelope.local (199-83-222-51.PUBLIC.monkeybrains.net [199.83.222.51]) (authenticated bits=0) by lavash.monkeybrains.net (8.14.4/8.14.4) with ESMTP id q4R7K0Z1063836 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 27 May 2012 00:20:00 -0700 (PDT) (envelope-from crapsh@monkeybrains.net) Message-ID: <4FC1D535.50304@monkeybrains.net> Date: Sun, 27 May 2012 00:18:13 -0700 From: "Rudy (bulk)" User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: Nikolay Denev References: <4FBDF520.1060607@monkeybrains.net> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.97.3 at lavash.monkeybrains.net X-Virus-Status: Clean Cc: freebsd-net Subject: Re: Bug? adding a vlan on ix is not elegant X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 May 2012 07:20:08 -0000 8.3-STABLE Rudy On 5/24/12 2:42 AM, Nikolay Denev wrote: > On May 24, 2012, at 11:45 AM, Rudy (bulk) wrote: > >> If I have my ix0 up and add another vlan... eg >> ifconfig vlan777 vlandev ix0 vlan777 10.77.7.1/24 >> the act of creating a vlan causes all the other vlans to go offline for 15 seconds. >> >> Rudy > What version of FreeBSD are you running? > The little interruption might be due to the vlan hwfilter reprogramming on the adapter, > but 15 secs sound too much. > > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sun May 27 16:27:04 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 840E8106564A for ; Sun, 27 May 2012 16:27:04 +0000 (UTC) (envelope-from darrenr@freebsd.org) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) by mx1.freebsd.org (Postfix) with ESMTP id 37ACF8FC0A for ; Sun, 27 May 2012 16:27:04 +0000 (UTC) Received: from compute5.internal (compute5.nyi.mail.srv.osa [10.202.2.45]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 7698A21165 for ; Sun, 27 May 2012 12:20:49 -0400 (EDT) Received: from frontend1.nyi.mail.srv.osa ([10.202.2.160]) by compute5.internal (MEProxy); Sun, 27 May 2012 12:20:49 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:date:from:reply-to :mime-version:to:subject:content-type:content-transfer-encoding; s=smtpout; bh=PCBkJ29XSxrF4ejSxcdtQrsmb2E=; b=f2idciQf4+qvJXoib SGKd8hOwZnyVdjA/8Wn3GAfwy5sDI8Z69uwgDvQ1yYwpPwdBVYw0bpcafUjSrP6U FJKMQy3MkYkzksd2yeYm2t2qcnGrI6n6MQDFMpdoEeYv8BX4j6CeJaGZze8fVpBg o6BSLNn0j4IdFLkPSU6oQFs+0E= X-Sasl-enc: MfKTju1NQ3I/I5N4ywK0bQ/BHlYs6DToZ+QXDME+sdYH 1338135648 Received: from [192.168.1.124] (unknown [202.45.110.141]) by mail.messagingengine.com (Postfix) with ESMTPA id 303D58E01F1 for ; Sun, 27 May 2012 12:20:47 -0400 (EDT) Message-ID: <4FC254C9.1050408@freebsd.org> Date: Mon, 28 May 2012 02:22:33 +1000 From: Darren Reed Organization: FreeBSD User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: rsh forking to infinity and beyond X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: darrenr@freebsd.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 May 2012 16:27:04 -0000 I've run into a rather strange problem with rsh on FreeBSD 9.0 where rsh seems to become a fork bomb. How or why this happens, I do not know. If I run it from the command line with a shell, it behaves perfectly normal. Even inside $() with pdksh. But when I put it in my script... What does the problem look like? See below. The script in question does this: x=$(truss -o rsh.$host.tr -f ${RSH} -l root ${host} echo hello_world) ... Normally there would be no truss, but I'm trying to work out why this is happening. Unfortunately, truss does not reveal any amount of nested forking. I'm at a loss to explain what's going on. Anyone? Darren -+= 00001 root /sbin/init -- |--= 00127 root adjkerntz -i |--= 00743 root /sbin/devd |--= 00909 root /usr/sbin/syslogd -s |-+= 01149 root /usr/sbin/sshd | \-+= 01255 root sshd: darrenr [priv] (sshd) | \-+- 01258 darrenr sshd: darrenr@pts/0 (sshd) | \-+= 01259 darrenr -tcsh (tcsh) | \-+= 01328 root su root -c exec tcsh | \-+= 01329 root tcsh | |-+= 01392 root /bin/ksh ./setup.sh (ksh93) | | \-+- 01394 root truss -o rsh.freebsd.tr -f rsh -l root freebsd ec ho hello_world | | |-+- 01395 root rsh -l root freebsd echo hello_world | | | \--- 01454 root | | |--- 01400 root | | \--- 01455 root | \-+= 02045 root pstree | \--- 02046 root ps -axwwo user,pid,ppid,pgid,command |--= 01156 root sendmail: accepting connections (sendmail) |--= 01160 smmsp sendmail: Queue runner@00:30:00 for /var/spool/clientmqueue (s endmail) |--= 01166 root /usr/sbin/cron -s |--= 01191 root /usr/sbin/inetd -wW -C 60 |--= 01226 root /usr/libexec/getty Pc ttyv0 |--= 01227 root /usr/libexec/getty Pc ttyv1 |--= 01228 root /usr/libexec/getty Pc ttyv2 |--= 01229 root /usr/libexec/getty Pc ttyv3 |--= 01230 root /usr/libexec/getty Pc ttyv4 |--= 01231 root /usr/libexec/getty Pc ttyv5 |--= 01232 root /usr/libexec/getty Pc ttyv6 |--= 01233 root /usr/libexec/getty Pc ttyv7 |-+- 01397 root rsh freebsd -l root echo hello_world | \--- 01401 root |-+- 01399 root rsh freebsd -l root echo hello_world | \--- 01404 root |-+- 01403 root rsh freebsd -l root echo hello_world | \--- 01407 root |-+- 01406 root rsh freebsd -l root echo hello_world | \--- 01410 root |-+- 01409 root rsh freebsd -l root echo hello_world | \--- 01413 root |-+- 01412 root rsh freebsd -l root echo hello_world | \--- 01416 root |-+- 01415 root rsh freebsd -l root echo hello_world ... |-+- 02026 root rsh freebsd -l root echo hello_world | \--- 02030 root |-+- 02029 root rsh freebsd -l root echo hello_world | \--- 02033 root |-+- 02032 root rsh freebsd -l root echo hello_world | \--- 02036 root |-+- 02035 root rsh freebsd -l root echo hello_world | \--- 02039 root \-+- 02038 root rsh freebsd -l root echo hello_world \-+- 02040 root rsh freebsd -l root echo hello_world \--- 02041 root rsh freebsd -l root echo hello_world From owner-freebsd-net@FreeBSD.ORG Sun May 27 17:12:06 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E102C1065677; Sun, 27 May 2012 17:12:06 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) by mx1.freebsd.org (Postfix) with ESMTP id 2FAEA8FC21; Sun, 27 May 2012 17:12:06 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 302A425D3A00; Sun, 27 May 2012 17:12:05 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 64ACCBE7DAD; Sun, 27 May 2012 17:12:04 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id 5jXtrwDdPaTv; Sun, 27 May 2012 17:12:03 +0000 (UTC) Received: from orange-en1.sbone.de (orange-en1.sbone.de [IPv6:fde9:577b:c1a9:31:cabc:c8ff:fecf:e8e3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 3444DBE7DAF; Sun, 27 May 2012 17:12:03 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: "Bjoern A. Zeeb" In-Reply-To: <9CBC841B-6339-41BC-A91B-83D47B84B9B4@FreeBSD.org> Date: Sun, 27 May 2012 17:12:02 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <6C49CEBF-37E0-468D-A07D-616C2E1B588D@FreeBSD.org> References: <9CBC841B-6339-41BC-A91B-83D47B84B9B4@FreeBSD.org> To: net@freebsd.org X-Mailer: Apple Mail (2.1084) Cc: current@FreeBSD.org Subject: Re: WARNING - DO NOT test: IPv6 offload support in HEAD + patch for stable/9 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 May 2012 17:12:07 -0000 On 26. May 2012, at 14:01 , Bjoern A. Zeeb wrote: Hey. > WARNING - please refrain from testing IPv6 or updating your HEAD if = you do > not have any of the above two NICs and rely on IPv6, or if you have = updated and > are experiencing problems. Disabling -txcsum -tso for the moment = should be an > often unhelpful workaround. It seems I was just lucky with my choice = of other It was not, as there was a further bug, which was fixed last night with http://svn.freebsd.org/changeset/base/236130 To fix the full problem, here's a proposed patch for testing on the = latest HEAD or review: http://people.freebsd.org/~bz/20120527-02-fix-v6-csum-offload.diff I'd be happy to hear back from anyone on short notice that it works for = him, and I'll get it in for (possible) further refinements. /bz --=20 Bjoern A. Zeeb You have to have visions! It does not matter how good you are. It matters what good you do! From owner-freebsd-net@FreeBSD.ORG Sun May 27 19:12:51 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 749A31065674 for ; Sun, 27 May 2012 19:12:51 +0000 (UTC) (envelope-from darrenr@freebsd.org) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) by mx1.freebsd.org (Postfix) with ESMTP id 3DE178FC16 for ; Sun, 27 May 2012 19:12:51 +0000 (UTC) Received: from compute1.internal (compute1.nyi.mail.srv.osa [10.202.2.41]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 1784F20D9B for ; Sun, 27 May 2012 15:12:50 -0400 (EDT) Received: from frontend1.nyi.mail.srv.osa ([10.202.2.160]) by compute1.internal (MEProxy); Sun, 27 May 2012 15:12:50 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:date:from:reply-to :mime-version:to:subject:content-type:content-transfer-encoding; s=smtpout; bh=zlxKpBJLyjE4lVE8vhty1gBFbHY=; b=L/5Fx07JSoum+RQRp eGBmxe4gs9EMoHwpRnawIJVaknCfFaIwlTjU3s8sTShJYi85+7atuWLVgRBIsVPK x71s3v9E9Px26Hca4hFAv2cg1pGRfMn9tDcBW9JhDA4Lt45VYB9NlF9NLIS3M2WH TtDOIXjse1q5gqE9dFFJXrRvyc= X-Sasl-enc: agKh1xfnT0gK3/86YLEM6SR6HtQ9MmT0DiWObOjDQDs4 1338145968 Received: from [192.168.1.124] (unknown [202.45.110.141]) by mail.messagingengine.com (Postfix) with ESMTPA id C62D78E01EF for ; Sun, 27 May 2012 15:12:46 -0400 (EDT) Message-ID: <4FC27D17.5000706@freebsd.org> Date: Mon, 28 May 2012 05:14:31 +1000 From: Darren Reed Organization: FreeBSD User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Environment variable "RSH"... X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: darrenr@freebsd.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 May 2012 19:12:51 -0000 It would seem that setting $RSH to "rsh" is the cause of my problem and that rcmd(3) cannot deal with RSH pointing to itself. I've filed a PR on this (possibly duplicate) but the email with the PR# is taking its time. Darren From owner-freebsd-net@FreeBSD.ORG Mon May 28 06:14:59 2012 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E5C9F106564A for ; Mon, 28 May 2012 06:14:59 +0000 (UTC) (envelope-from randy@psg.com) Received: from ran.psg.com (ran.psg.com [IPv6:2001:418:1::36]) by mx1.freebsd.org (Postfix) with ESMTP id C56D88FC08 for ; Mon, 28 May 2012 06:14:59 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=rair.psg.com.psg.com) by ran.psg.com with esmtp (Exim 4.77 (FreeBSD)) (envelope-from ) id 1SYtEI-0006pu-Ma for freebsd-net@FreeBSD.org; Mon, 28 May 2012 06:14:58 +0000 Date: Mon, 28 May 2012 15:14:57 +0900 Message-ID: From: Randy Bush To: freebsd-net User-Agent: Wanderlust/2.15.9 (Almost Unreal) Emacs/22.3 Mule/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII Cc: Subject: complex v6 gateway - live by tunnels die by tunnels X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 May 2012 06:15:00 -0000 problem: some devices are getting a v6 address and find the gateway, i.e. a lionized macbook air. but a ripe atlas probe is getting an address but not gateway. environment and config: the router is a soekris 5501 gateway running FreeBSD soek0.psg.com 9.0-STABLE FreeBSD 9.0-STABLE #0: Fri Jan 27 03:05:06 GMT 2012 root@soek0.psg.com:/usr/obj/usr/src/sys/SOEK0 i386 .------------------------------. | | | b --wlan0| | r | 192.168.0.0/24 WAN IIJ | i --- vr1| LAN hosts, PPP/NAT ---|vr0[PPPoE][ppp]tun0--d | DHCP Clients 210.138.216.50 | g --- vr2| ... | e | | 0 --- vr3| | | `------------------------------' and tunneled v6 over that ### IPv6 options: ### ipv6_activate_all_interfaces=YES ipv6_gateway_enable=YES #ipv6_cpe_wanif=????? maybe this would help, but what interface? route6d_enable=YES route6d_flags="-A 2001:240:6a8::/48,gif0 -O 2001:240:6a8::/48,gif0" rtsold_enable=YES rtadvd_enable=YES rtadvd_interfaces="vr0 bridge0" gif_interfaces=gif0 gifconfig_gif0="210.138.216.50 210.138.77.245" ipv6_static_routes=gif ipv6_route_gif="default -interface gif0" soek0.psg.com:/root# ifconfig vr0: flags=8843 metric 0 mtu 1500 options=8280b ether 00:00:24:c8:b3:28 inet6 fe80::200:24ff:fec8:b328%vr0 prefixlen 64 scopeid 0x1 nd6 options=21 media: Ethernet autoselect (100baseTX ) status: active vr1: flags=8943 metric 0 mtu 1500 options=82809 ether 00:00:24:c8:b3:29 inet6 fe80::200:24ff:fec8:b329%vr1 prefixlen 64 scopeid 0x2 nd6 options=21 media: Ethernet autoselect (100baseTX ) status: active vr2: flags=8943 metric 0 mtu 1500 options=82809 ether 00:00:24:c8:b3:2a inet6 fe80::200:24ff:fec8:b32a%vr2 prefixlen 64 scopeid 0x3 nd6 options=21 media: Ethernet autoselect (none) status: no carrier vr3: flags=8943 metric 0 mtu 1500 options=82809 ether 00:00:24:c8:b3:2b inet6 fe80::200:24ff:fec8:b32b%vr3 prefixlen 64 scopeid 0x4 nd6 options=21 media: Ethernet autoselect (none) status: no carrier ath0: flags=8843 metric 0 mtu 2290 ether 00:0b:6b:83:59:25 nd6 options=21 media: IEEE 802.11 Wireless Ethernet autoselect mode 11g status: running ipfw0: flags=8801 metric 0 mtu 65536 nd6 options=21 lo0: flags=8049 metric 0 mtu 16384 options=3 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x9 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 bridge0: flags=8843 metric 0 mtu 1500 ether 02:52:fd:5d:b5:00 inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255 inet6 fe80::452:fdff:fe5d:b500%bridge0 prefixlen 64 scopeid 0xa inet6 2001:240:6a8::1 prefixlen 64 nd6 options=21 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: wlan0 flags=143 ifmaxaddr 0 port 12 priority 128 path cost 370370 member: vr3 flags=143 ifmaxaddr 0 port 4 priority 128 path cost 55 member: vr2 flags=143 ifmaxaddr 0 port 3 priority 128 path cost 55 member: vr1 flags=143 ifmaxaddr 0 port 2 priority 128 path cost 55 gif0: flags=8051 metric 0 mtu 1280 tunnel inet 210.138.216.50 --> 210.138.77.245 inet6 fe80::200:24ff:fec8:b328%gif0 prefixlen 64 scopeid 0xb nd6 options=21 options=1 wlan0: flags=8943 metric 0 mtu 1500 ether 00:0b:6b:83:59:25 inet6 fe80::20b:6bff:fe83:5925%wlan0 prefixlen 64 scopeid 0xc nd6 options=21 media: IEEE 802.11 Wireless Ethernet autoselect mode 11g status: running ssid rgnet-crypt channel 11 (2462 MHz 11g) bssid 00:0b:6b:83:59:25 country US ecm authmode WPA2/802.11i privacy MIXED deftxkey 2 TKIP 2:128-bit TKIP 3:128-bit txpower 21 scanvalid 60 protmode CTS wme burst dtimperiod 1 -dfs tun0: flags=8051 metric 0 mtu 1454 options=80000 inet 210.138.216.50 --> 210.149.34.72 netmask 0xffffffff nd6 options=21 Opened by PID 1398 tun1: flags=8051 metric 0 mtu 1500 options=80000 inet6 fe80::200:24ff:fec8:b328%tun1 prefixlen 64 scopeid 0xe inet 192.168.0.65 --> 192.168.0.66 netmask 0xffffffff nd6 options=21 Opened by PID 2098 randy From owner-freebsd-net@FreeBSD.ORG Mon May 28 06:20:41 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A7E96106566B for ; Mon, 28 May 2012 06:20:41 +0000 (UTC) (envelope-from mitya@yandex-team.ru) Received: from dinosaur.yandex.ru (dinosaur.yandex.ru [77.88.34.8]) by mx1.freebsd.org (Postfix) with ESMTP id 38B4F8FC0A for ; Mon, 28 May 2012 06:20:40 +0000 (UTC) Received: from Dmitrys-MacBook-Pro.local (v3-150-205.yandex.net [84.201.150.205]) by dinosaur.yandex.ru (Postfix) with ESMTP id 3W17Gn6gWmz2prv for ; Mon, 28 May 2012 10:20:33 +0400 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1338186033; bh=Xm/2KVunbv37MA4TrQQDjKmYp8CUwJ7s6L9fANlDUl4=; h=Date:From:To:Subject:References:In-Reply-To; b=ICtQTDFV4yRAlolynrbKomJ86VfpYw4DisV7eN8la/A7aLtDZjCZD4l7QQlaGn9s0 ZYtnCZZY6E187KhV40z15tdEe/xChQKFqST2K1M9ViA+bU0DCegPbKs8SDsEcGdYfy xbVF4WNWxSuXR2phEP+V53wcszwY5Gr93eju0XMQ= Message-ID: <4FC31931.6050502@yandex-team.ru> Date: Mon, 28 May 2012 10:20:33 +0400 From: Dmitry Sivachenko User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: freebsd-net@freebsd.org References: <4FB2BFD6.8040607@yandex-team.ru> In-Reply-To: <4FB2BFD6.8040607@yandex-team.ru> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: network stops working X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 May 2012 06:20:41 -0000 Just for reference: I am almost sure that recent MFC of Intel drivers to stable/9 fixed that issue. I observed no network-related issues on my few test servers since that upgrade. On 5/16/12 12:43 AM, Dmitry Sivachenko wrote: > Hello! > > I am using FreeBSD-9-stable. > I use Intel network cards (em, igb) with mtu=9000 (it's important, the > problem disappears if I switch to mtu=1500). > > I have a number of servers running a few web-services written in our > company. > > After a few days of working network suddenly stops functioning. > There is nothing in log/messages. It just stops working. > > If I execute ifconfig down/up, I get the following error message: > > # ifconfig net0 down > # ifconfig net0 up > em1: Could not setup receive structures > # > > and network still does not work. > > If I stop mentioned programs serving web requests (even not all of them, > just random one), ifconfig net0 up command succeeds and network resumes > its operations... for more few days and then the same problems comes back. > > Consider netstat -m output: > > netstat -m > 1025/4945/5970 mbufs in use (current/cache/total) > 0/3446/3446/262144 mbuf clusters in use (current/cache/total/max) > 0/2090 mbuf+clusters out of packet secondary zone in use (current/cache) > 0/1230/1230/65536 4k (page size) jumbo clusters in use > (current/cache/total/max) > 1023/4907/5930/65536 9k jumbo clusters in use (current/cache/total/max) > ^^^^^^^^^^^^^^^^^^^^^^^ > Note that max is much higher that total. > > > 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) > 9463K/57211K/66674K bytes allocated to network (current/cache/total) > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/2542746/0 requests for jumbo clusters denied (4k/9k/16k) > ^^^^^^^^^^^^ > There are 9k jumbo clusters allocations denied. > > 0/0/0 sfbufs in use (current/peak/max) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 641 requests for I/O initiated by sendfile > 0 calls to protocol drain routines > > I am attaching vmstat -z output below for reference. > > What can be the cause of 9k jumbo closters allocation denies? > What additional information cat I provide to help trach this down? > > Thanks in advance! > > > vmstat -z > ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP > > UMA Kegs: 208, 0, 92, 10, 92, 0, 0 > UMA Zones: 3456, 0, 92, 0, 92, 0, 0 > UMA Slabs: 568, 0, 6270, 702, 2222844, 0, 0 > UMA RCntSlabs: 568, 0, 8883, 581, 7364017, 0, 0 > UMA Hash: 256, 0, 1, 14, 4, 0, 0 > 16 Bucket: 152, 0, 39, 136, 185, 0, 0 > 32 Bucket: 280, 0, 130, 206, 401, 0, 0 > 64 Bucket: 536, 0, 138, 149, 470, 84, 0 > 128 Bucket: 1048, 0, 1812, 639, 1736739,16793, 0 > VM OBJECT: 232, 0, 52210, 7454,20006697, 0, 0 > MAP: 232, 0, 7, 25, 7, 0, 0 > KMAP ENTRY: 120, 1549690, 3102, 2602, 6160517, 0, 0 > MAP ENTRY: 120, 0, 7131, 3409,54473178, 0, 0 > fakepg: 120, 0, 0, 589, 53928, 0, 0 > mt_zone: 4112, 0, 276, 3, 276, 0, 0 > 16: 16, 0, 4047, 4353,3946273892, 0, 0 > 32: 32, 0, 3955, 2408,13600755, 0, 0 > 64: 64, 0, 11979, 7509,116304518, 0, 0 > 128: 128, 0, 20979, 36673,90194687, 0, 0 > 256: 256, 0, 5569, 16886,137956814, 0, 0 > 512: 512, 0, 4368, 5194, 8638401, 0, 0 > 1024: 1024, 0, 84, 1196, 4033660, 0, 0 > 2048: 2048, 0, 199, 1751, 1203566, 0, 0 > 4096: 4096, 0, 318, 2704, 1142726, 0, 0 > Files: 80, 0, 5613, 5097,2006566966, 0, 0 > TURNSTILE: 136, 0, 5605, 855, 6379, 0, 0 > umtx pi: 96, 0, 0, 0, 0, 0, 0 > PROC: 1160, 0, 63, 3657, 709134, 0, 0 > THREAD: 1112, 0, 3969, 1635, 524885, 0, 0 > SLEEPQUEUE: 80, 0, 5605, 1094, 6379, 0, 0 > VMSPACE: 392, 0, 43, 2387, 709117, 0, 0 > cpuset: 72, 0, 2, 98, 2, 0, 0 > mbuf_packet: 256, 0, 0, 1765,4356065566, 0, 0 > mbuf: 256, 0, 1024, 3181,26760389876, 0, 0 > mbuf_cluster: 2048, 262144, 1765, 1681,1318169467, 0, 0 > mbuf_jumbo_page: 4096, 65536, 0, 1230,61648445, 0, 0 > mbuf_jumbo_9k: 9216, 65536, 1023, 4907,8926430021,2542746, 0 > mbuf_jumbo_16k: 16384, 3200, 0, 0, 0, 0, 0 > mbuf_ext_refcnt: 4, 0, 0, 2352, 5422000, 0, 0 > NetGraph items: 72, 4118, 0, 58, 5, 0, 0 > NetGraph data items: 72, 522, 0, 58, 2, 0, 0 > g_bio: 232, 0, 0, 4544,300858503, 0, 0 > ttyinq: 160, 0, 180, 252, 615, 0, 0 > ttyoutq: 256, 0, 95, 175, 327, 0, 0 > ata_request: 328, 0, 0, 0, 0, 0, 0 > ata_composite: 336, 0, 0, 0, 0, 0, 0 > VNODE: 480, 0, 78944, 3840, 952605, 0, 0 > VNODEPOLL: 112, 0, 0, 0, 1, 0, 0 > NAMEI: 1024, 0, 0, 1632,139785175, 0, 0 > S VFS Cache: 108, 0, 87356, 1579, 556273, 0, 0 > L VFS Cache: 328, 0, 2924, 1768, 15274, 0, 0 > DIRHASH: 1024, 0, 9, 1259, 20893, 0, 0 > NCLNODE: 560, 0, 0, 0, 0, 0, 0 > Mountpoints: 768, 0, 5, 10, 5, 0, 0 > AIO: 208, 0, 0, 0, 0, 0, 0 > AIOP: 32, 0, 0, 0, 0, 0, 0 > AIOCB: 480, 0, 0, 0, 0, 0, 0 > AIOL: 128, 0, 0, 0, 0, 0, 0 > AIOLIO: 272, 0, 0, 0, 0, 0, 0 > pipe: 728, 0, 7, 2928, 515298, 0, 0 > ksiginfo: 112, 0, 3680, 3250, 192170, 0, 0 > itimer: 344, 0, 1, 21, 6, 0, 0 > KNOTE: 128, 0, 31, 4406,11850035546, 0, 0 > socket: 680, 262140, 66, 4986,1949420563, 0, 0 > ipq: 56, 8253, 0, 0, 0, 0, 0 > udp_inpcb: 392, 262140, 20, 2170, 1133408, 0, 0 > udpcb: 16, 262248, 20, 3676, 1133408, 0, 0 > tcp_inpcb: 392, 262140, 3373, 24417,1947917187, 0, 0 > tcpcb: 976, 262140, 26, 5010,1947917187, 0, 0 > tcptw: 72, 41000, 3347, 24153,1352095629, 0, 0 > syncache: 152, 15375, 0, 2375,2029926070, 0, 0 > hostcache: 136, 153608, 5, 667, 4953, 0, 0 > tcpreass: 40, 16464, 0, 2436,27465901, 0, 0 > sackhole: 32, 0, 0, 909, 41329, 0, 0 > sctp_ep: 1368, 25600, 0, 0, 0, 0, 0 > sctp_asoc: 2288, 40000, 0, 0, 0, 0, 0 > sctp_laddr: 48, 80064, 0, 216, 13, 0, 0 > sctp_raddr: 704, 80000, 0, 0, 0, 0, 0 > sctp_chunk: 136, 400008, 0, 0, 0, 0, 0 > sctp_readq: 104, 400032, 0, 0, 0, 0, 0 > sctp_stream_msg_out: 112, 400026, 0, 0, 0, 0, 0 > sctp_asconf: 40, 400008, 0, 0, 0, 0, 0 > sctp_asconf_ack: 48, 400032, 0, 0, 0, 0, 0 > ripcb: 392, 262140, 0, 70, 138, 0, 0 > unpcb: 240, 262144, 14, 1986, 369808, 0, 0 > rtentry: 200, 0, 19, 38, 37, 0, 0 > IPFW dynamic rule: 120, 0, 0, 0, 0, 0, 0 > divcb: 392, 262140, 0, 0, 0, 0, 0 > g_stripe_zone: 131072, 100, 0, 0, 0, 0, 0 > selfd: 56, 0, 6286, 2534,62657688, 0, 0 > SWAPMETA: 288, 116519, 1109, 2817,14987719, 0, 0 > FFS inode: 168, 0, 78896, 5232, 952545, 0, 0 > FFS1 dinode: 128, 0, 0, 0, 0, 0, 0 > FFS2 dinode: 256, 0, 78896, 5014, 952545, 0, 0 > > > > cat /boot/loader.conf > userconfig_script_load="YES" > vm.exec_map_entries="48" > hw.igb.rxd="256" > hw.igb.txd="256" > hw.usb.no_pf="1" > From owner-freebsd-net@FreeBSD.ORG Mon May 28 08:55:46 2012 Return-Path: Delivered-To: net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8B90B1065670 for ; Mon, 28 May 2012 08:55:46 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.64.117]) by mx1.freebsd.org (Postfix) with ESMTP id 1E39D8FC15 for ; Mon, 28 May 2012 08:55:42 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.5/8.14.5) with ESMTP id q4S8taBf021138 for ; Mon, 28 May 2012 12:55:36 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.5/8.14.5/Submit) id q4S8tabK021137 for net@FreeBSD.org; Mon, 28 May 2012 12:55:36 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Mon, 28 May 2012 12:55:36 +0400 From: Gleb Smirnoff To: net@FreeBSD.org Message-ID: <20120528085536.GE92865@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Subject: CFR: if_group events Was: [glebius@FreeBSD.org: svn commit: r236168 - projects/pf/head/sys/net] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 May 2012 08:55:46 -0000 Hello, networkers! Any opinions on the below change, that I made in the projects/pf branch? Before the change we've got all ifnet arrival departure events called w/o locks held, as well as group change event. But the group arrival departure event handlers did hold locks. IMHO, after my change things get more consistent. Eventually I'd like to merge this to head. Any objections? Also, what I don't like, is that "group change" event is called always right after addition or deletion of a group, that is right after sending "group addition" or "group removal" event. Looks like an unneeded explicitness and tautology to me. Your opinions? ----- Forwarded message from Gleb Smirnoff ----- Date: Mon, 28 May 2012 08:50:01 +0000 (UTC) From: Gleb Smirnoff To: src-committers@FreeBSD.org, svn-src-projects@FreeBSD.org Subject: svn commit: r236168 - projects/pf/head/sys/net Author: glebius Date: Mon May 28 08:50:00 2012 New Revision: 236168 URL: http://svn.freebsd.org/changeset/base/236168 Log: Invoke group attach/detach handlers after releasing locks, so that event subscribers could use M_WAITOK. This should reduce lock contention as well. Modified: projects/pf/head/sys/net/if.c Modified: projects/pf/head/sys/net/if.c ============================================================================== --- projects/pf/head/sys/net/if.c Mon May 28 07:34:52 2012 (r236167) +++ projects/pf/head/sys/net/if.c Mon May 28 08:50:00 2012 (r236168) @@ -1084,6 +1084,7 @@ if_addgroup(struct ifnet *ifp, const cha struct ifg_list *ifgl; struct ifg_group *ifg = NULL; struct ifg_member *ifgm; + int new = 0; if (groupname[0] && groupname[strlen(groupname) - 1] >= '0' && groupname[strlen(groupname) - 1] <= '9') @@ -1124,8 +1125,8 @@ if_addgroup(struct ifnet *ifp, const cha strlcpy(ifg->ifg_group, groupname, sizeof(ifg->ifg_group)); ifg->ifg_refcnt = 0; TAILQ_INIT(&ifg->ifg_members); - EVENTHANDLER_INVOKE(group_attach_event, ifg); TAILQ_INSERT_TAIL(&V_ifg_head, ifg, ifg_next); + new = 1; } ifg->ifg_refcnt++; @@ -1139,6 +1140,8 @@ if_addgroup(struct ifnet *ifp, const cha IFNET_WUNLOCK(); + if (new) + EVENTHANDLER_INVOKE(group_attach_event, ifg); EVENTHANDLER_INVOKE(group_change_event, groupname); return (0); @@ -1177,10 +1180,11 @@ if_delgroup(struct ifnet *ifp, const cha if (--ifgl->ifgl_group->ifg_refcnt == 0) { TAILQ_REMOVE(&V_ifg_head, ifgl->ifgl_group, ifg_next); + IFNET_WUNLOCK(); EVENTHANDLER_INVOKE(group_detach_event, ifgl->ifgl_group); free(ifgl->ifgl_group, M_TEMP); - } - IFNET_WUNLOCK(); + } else + IFNET_WUNLOCK(); free(ifgl, M_TEMP); @@ -1221,11 +1225,12 @@ if_delgroups(struct ifnet *ifp) if (--ifgl->ifgl_group->ifg_refcnt == 0) { TAILQ_REMOVE(&V_ifg_head, ifgl->ifgl_group, ifg_next); + IFNET_WUNLOCK(); EVENTHANDLER_INVOKE(group_detach_event, ifgl->ifgl_group); free(ifgl->ifgl_group, M_TEMP); - } - IFNET_WUNLOCK(); + } else + IFNET_WUNLOCK(); free(ifgl, M_TEMP); ----- End forwarded message ----- -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Mon May 28 11:07:33 2012 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B9A841065673 for ; Mon, 28 May 2012 11:07:33 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id A2CF28FC1C for ; Mon, 28 May 2012 11:07:33 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q4SB7XH1063422 for ; Mon, 28 May 2012 11:07:33 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q4SB7WY1063420 for freebsd-net@FreeBSD.org; Mon, 28 May 2012 11:07:32 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 28 May 2012 11:07:32 GMT Message-Id: <201205281107.q4SB7WY1063420@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 May 2012 11:07:33 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/168294 net [ixgbe] [patch] ixgbe driver compiled in kernel has no o kern/168183 net [bce] bce driver hang system o kern/168152 net [xl] Periodically, the network card xl0 stops working o kern/167947 net [setfib] [patch] arpresolve checks only the default FI o kern/167768 net [ipfilter] Fatal trap in ipfilter/ipnat o kern/167603 net [ip] IP fragment reassembly's broken: file transfer ov o kern/167500 net [em] [panic] Kernel panics in em driver o kern/167325 net [netinet] [patch] sosend sometimes return EINVAL with o kern/167202 net [igmp]: Sending multiple IGMP packets crashes kernel o kern/167059 net [tcp] [panic] System does panic in in_pcbbind() and ha o kern/166940 net [ipfilter] [panic] Double fault in kern 8.2 o kern/166550 net [netinet] [patch] Some log lines about arp do not incl o kern/166462 net [gre] gre(4) when using a tunnel source address from c o kern/166372 net [patch] ipfilter drops UDP packets with zero checksum o kern/166285 net [arp] FreeBSD v8.1 REL p8 arp: unknown hardware addres o kern/166255 net [net] [patch] It should be possible to disable "promis o kern/165963 net [panic] [ipf] ipfilter/nat NULL pointer deference o kern/165903 net mbuf leak o kern/165863 net [panic] [netinet] [patch] in_lltable_prefix_free() rac o kern/165643 net [net] [patch] Missing vnet restores in net/if_ethersub o kern/165622 net [ndis][panic][patch] Unregistered use of FPU in kernel s kern/165562 net [request] add support for Intel i350 in FreeBSD 7.4 o kern/165526 net [bxe] UDP packets checksum calculation whithin if_bxe o kern/165488 net [ppp] [panic] Fatal trap 12 jails and ppp , kernel wit o kern/165305 net [ip6] [request] Feature parity between IP_TOS and IPV6 o kern/165296 net [vlan] [patch] Fix EVL_APPLY_VLID, update EVL_APPLY_PR o kern/165181 net [igb] igb freezes after about 2 weeks of uptime o kern/165174 net [patch] [tap] allow tap(4) to keep its address on clos o kern/165152 net [ip6] Does not work through the issue of ipv6 addresse o kern/164495 net [igb] connect double head igb to switch cause system t o kern/164490 net [pfil] Incorrect IP checksum on pfil pass from ip_outp o kern/164475 net [gre] gre misses RUNNING flag after a reboot o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162926 net [ipfilter] Infinite loop in ipfilter with fragmented I o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161381 net [re] RTL8169SC - re0: PHY write failed o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160293 net [ieee80211] ppanic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159603 net [netinet] [patch] in_ifscrubprefix() - network route c o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157209 net [ip6] [patch] locking error in rip6_input() (sys/netin o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156283 net [ip6] [patch] nd6_ns_input - rtalloc_mpath does not re o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155772 net ifconfig(8): ioctl (SIOCAIFADDR): File exists on direc o kern/155680 net [multicast] problems with multicast s kern/155642 net [request] Add driver for Realtek RTL8191SE/RTL8192SE W o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel o kern/155030 net [igb] igb(4) DEVICE_POLLING does not work with carp(4) o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [request]: Port brcm80211 driver from Linux to FreeBSD o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl o kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153772 net [ixgbe] [patch] sysctls reference wrong XON/XOFF varia o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149937 net [ipfilter] [patch] kernel panic in ipfilter IP fragmen o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149117 net [inet] [patch] in_pcbbind: redundant test o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147894 net [ipsec] IPv6-in-IPv4 does not work inside an ESP-only o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146082 net [ng_l2tp] a false invaliant check was performed in ng_ o kern/146037 net [panic] mpd + CoA = kernel panic o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi s kern/143666 net [ip6] [request] PMTU black hole detection not implemen o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143593 net [ipsec] When using IPSec, tcpdump doesn't show outgoin o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143208 net [ipsec] [gif] IPSec over gif interface not working o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed f kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph o kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139565 net [ipfilter] ipfilter ioctl SIOCDELST broken o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/139058 net [ipfilter] mbuf cluster leak on FreeBSD 7.2 o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r o kern/138177 net [ipfilter] FreeBSD crashing repeatedly in ip_nat.c:257 f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o bin/136994 net [patch] ifconfig(8) print carp mac address o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o conf/132851 net [patch] rc.conf(5): allow to setfib(1) for service run o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132554 net [ipl] There is no ippool start script/ipfilter magic t o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/132277 net [crypto] [ipsec] poor performance using cryptodevice f o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o kern/131601 net [ipfilter] [panic] 7-STABLE panic in nat_finalise (tcp o bin/131567 net [socket] [patch] Update for regression/sockets/unix_cm o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o conf/130555 net [rc.d] [patch] No good way to set ipfilter variables a o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123796 net [ipf] FreeBSD 6.1+VPN+ipnat+ipf: port mapping does not o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o bin/121359 net [patch] [security] ppp(8): fix local stack overflow in o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o kern/117271 net [tap] OpenVPN TAP uses 99% CPU on releng_6 when if_tap o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106438 net [ipf] ipfilter: keep state does not seem to allow repl o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/101948 net [ipf] [panic] Kernel Panic Trap No 12 Page Fault - cau o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98978 net [ipf] [patch] ipfilter drops OOW packets under 6.1-Rel o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o bin/98218 net wpa_supplicant(8) blacklist not working o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 s kern/91777 net [ipf] [patch] wrong behaviour with skip rule inside an o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/87521 net [ipf] [panic] using ipfilter "auth" keyword leads to k o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/86103 net [ipf] Illegal NAT Traversal in IPFilter o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ p kern/85320 net [gre] [patch] possible depletion of kernel stack in ip o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation s kern/77195 net [ipf] [patch] ipfilter ioctl SIOCGNATL does not match o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/70904 net [ipf] ipfilter ipnat problem with h323 proxy support o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message o kern/65616 net IPSEC can't detunnel GRE packets after real ESP encryp s kern/60293 net [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/34665 net [ipf] [hang] ipfilter rcmd proxy "hangs". o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/27474 net [ipf] [ppp] Interactive use of user PPP and ipfilter c f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 402 problems total. From owner-freebsd-net@FreeBSD.ORG Mon May 28 11:35:49 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B6ABF106564A for ; Mon, 28 May 2012 11:35:49 +0000 (UTC) (envelope-from darrenr@freebsd.org) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) by mx1.freebsd.org (Postfix) with ESMTP id 81E3A8FC12 for ; Mon, 28 May 2012 11:35:49 +0000 (UTC) Received: from compute3.internal (compute3.nyi.mail.srv.osa [10.202.2.43]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 0448021393 for ; Mon, 28 May 2012 07:35:49 -0400 (EDT) Received: from frontend1.nyi.mail.srv.osa ([10.202.2.160]) by compute3.internal (MEProxy); Mon, 28 May 2012 07:35:49 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:date:from:reply-to :mime-version:to:subject:content-type:content-transfer-encoding; s=smtpout; bh=H8uPxkZtATLjrgzpTM06ZjsERJ8=; b=j3hgeqj5TL7rchvlh R1UB/luPewLTM452sDv4BsE0KqbvBUBoK0S5QYH0ywOtH06ZQgfCrS5+k+8rxt5Q GuGTbBzAb2JBSv4ap1icODB9JS4g2Zkgnhp2XhUgOkuoOzvV50ieiVttpw5qMI7c CE163uwOhgQzLwI8JvuVT12y0k= X-Sasl-enc: QFINAyn/VA3QHPnYSdokRcBoMxcZpqinGp8QdFUiVfZD 1338204948 Received: from [192.168.1.124] (unknown [202.45.110.141]) by mail.messagingengine.com (Postfix) with ESMTPA id 5613C8E01CE for ; Mon, 28 May 2012 07:35:48 -0400 (EDT) Message-ID: <4FC36377.1080306@freebsd.org> Date: Mon, 28 May 2012 21:37:27 +1000 From: Darren Reed Organization: FreeBSD User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Point-to-point connection between jails? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: darrenr@freebsd.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 May 2012 11:35:49 -0000 I've looked through the list of network interfaces that are supported with FreeBSD and none seem to meet my needs. What I want is a network interface that I can configure up in jail A with address 10.1.1.1 and for which I can configure a mate in jail B with the address 10.2.2.2 and use the network interface as the means by which two jails can directly communicate with each other without the need to send any packets out of the machine. Or another way to do this would be to have a virtual network (something like the "internal network" that VirtualBox has or the host only network supported by VMWware Workstation) defined somewhere and for there to be a specific driver that could be configured and attached to a jail and that virtual network so that you could have N:M communication between jails. Is what I'm looking for already present and google is failing me or is the above functionality the basis for future work, be it planned or otherwise? Darren From owner-freebsd-net@FreeBSD.ORG Mon May 28 13:28:26 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E208106564A; Mon, 28 May 2012 13:28:26 +0000 (UTC) (envelope-from "."@babolo.ru) Received: from smtp1.babolo.ru (smtp1.babolo.ru [195.9.14.139]) by mx1.freebsd.org (Postfix) with ESMTP id BF1C58FC0A; Mon, 28 May 2012 13:28:25 +0000 (UTC) Received: from cicuta.babolo.ru (cicuta.babolo.ru [194.58.246.5]) by smtp1.babolo.ru (8.14.2/8.14.2) with SMTP id q4SDPR9D088821; Mon, 28 May 2012 17:25:27 +0400 (MSK) (envelope-from .@babolo.ru) Received: (nullmailer pid 30217 invoked by uid 136); Mon, 28 May 2012 13:23:00 -0000 Date: Mon, 28 May 2012 17:23:00 +0400 From: Aleksandr A Babaylov <.@babolo.ru> To: Darren Reed Message-ID: <20120528132300.GA30188@babolo.ru> References: <4FC36377.1080306@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4FC36377.1080306@freebsd.org> Cc: freebsd-net@freebsd.org Subject: Re: Point-to-point connection between jails? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 May 2012 13:28:26 -0000 On Mon, May 28, 2012 at 09:37:27PM +1000, Darren Reed wrote: > I've looked through the list of network interfaces that are > supported with FreeBSD and none seem to meet my needs. What > I want is a network interface that I can configure up in > jail A with address 10.1.1.1 and for which I can configure > a mate in jail B with the address 10.2.2.2 and use the > network interface as the means by which two jails can > directly communicate with each other without the need to > send any packets out of the machine. Or another way to do > this would be to have a virtual network (something like the > "internal network" that VirtualBox has or the host only > network supported by VMWware Workstation) defined somewhere > and for there to be a specific driver that could be > configured and attached to a jail and that virtual network > so that you could have N:M communication between jails. > > Is what I'm looking for already present and google is failing > me or is the above functionality the basis for future work, > be it planned or otherwise? ifconfig lo1 create ifconfig lo1 inet 127.1.2.3/24 ifconfig lo1 inet 127.1.2.4/32 alias launch jail A with IP 127.1.2.3 and jail B with IP 127.1.2.4 No any packet leaves host. 127.1.2.0/24 will be something like the "internal network" From owner-freebsd-net@FreeBSD.ORG Mon May 28 13:46:00 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0C1BC1065670 for ; Mon, 28 May 2012 13:46:00 +0000 (UTC) (envelope-from nvass@gmx.com) Received: from mailout-us.gmx.com (mailout-us.gmx.com [74.208.5.67]) by mx1.freebsd.org (Postfix) with SMTP id B837C8FC16 for ; Mon, 28 May 2012 13:45:59 +0000 (UTC) Received: (qmail invoked by alias); 28 May 2012 13:45:52 -0000 Received: from g224205173.adsl.alicedsl.de (EHLO [192.168.178.28]) [92.224.205.173] by mail.gmx.com (mp-us002) with SMTP; 28 May 2012 09:45:52 -0400 X-Authenticated: #46156728 X-Provags-ID: V01U2FsdGVkX1+0coBRr4K04oBVF5Vzrwq3xvJd7d5ny2mmUo1PAK NG4cdksLarg0xW Message-ID: <4FC3818A.8080801@gmx.com> Date: Mon, 28 May 2012 15:45:46 +0200 From: Nikos Vassiliadis User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.17) Gecko/20110414 Thunderbird/3.1.10 MIME-Version: 1.0 To: darrenr@freebsd.org References: <4FC36377.1080306@freebsd.org> In-Reply-To: <4FC36377.1080306@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 Cc: freebsd-net@freebsd.org Subject: Re: Point-to-point connection between jails? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 May 2012 13:46:00 -0000 On 5/28/2012 1:37 PM, Darren Reed wrote: > I've looked through the list of network interfaces that are > supported with FreeBSD and none seem to meet my needs. What > I want is a network interface that I can configure up in > jail A with address 10.1.1.1 and for which I can configure > a mate in jail B with the address 10.2.2.2 and use the > network interface as the means by which two jails can > directly communicate with each other without the need to > send any packets out of the machine. Or another way to do > this would be to have a virtual network (something like the > "internal network" that VirtualBox has or the host only > network supported by VMWware Workstation) defined somewhere > and for there to be a specific driver that could be > configured and attached to a jail and that virtual network > so that you could have N:M communication between jails. > > Is what I'm looking for already present and google is failing > me or is the above functionality the basis for future work, > be it planned or otherwise? It seems like a loopback interface does this. root@raidmadi:/home/nik # jls JID IP Address Hostname Path 3 10.2.3.4 / 4 10.7.3.4 / root@raidmadi:/home/nik # ifconfig lo1 lo1: flags=8049 metric 0 mtu 16384 options=3 inet 10.2.3.4 netmask 0xff000000 inet 10.7.3.4 netmask 0xff000000 root@raidmadi:/home/nik # Maybe you want 'real' isolation? like with epair and VIMAGE? Did I misunderstand your question? Nikos From owner-freebsd-net@FreeBSD.ORG Mon May 28 14:12:29 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AB62F1065672 for ; Mon, 28 May 2012 14:12:29 +0000 (UTC) (envelope-from darrenr@freebsd.org) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) by mx1.freebsd.org (Postfix) with ESMTP id 6E41C8FC16 for ; Mon, 28 May 2012 14:12:29 +0000 (UTC) Received: from compute3.internal (compute3.nyi.mail.srv.osa [10.202.2.43]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id B6FB620697; Mon, 28 May 2012 10:12:28 -0400 (EDT) Received: from frontend1.nyi.mail.srv.osa ([10.202.2.160]) by compute3.internal (MEProxy); Mon, 28 May 2012 10:12:28 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:date:from:reply-to :mime-version:to:cc:subject:references:in-reply-to:content-type :content-transfer-encoding; s=smtpout; bh=72CB0A2c6LZ+0F3kSToatc P1CIA=; b=bSYzJDmENwznxTgE5jHUTtiKdjpvjnCS/kscc2xptLTkbpVjylCkW9 z0ezwpw5ykVUrYHoKAHbx7wyUM1GyXpbLyNE6bjbtkq0oUuPQNXVljTWRO5OM4Av SF70vyzMwh3iaaK8Lkpca0IYQRr3XSZZnIX5n6AhWKd+N5E0pYmhE= X-Sasl-enc: +rQCyGBxPlI3wDM7w3YFVqnn4omjq2/cU98rcSdexO5t 1338214348 Received: from [192.168.1.124] (unknown [202.45.110.141]) by mail.messagingengine.com (Postfix) with ESMTPA id C2D0A8E0209; Mon, 28 May 2012 10:12:27 -0400 (EDT) Message-ID: <4FC3882C.5030105@freebsd.org> Date: Tue, 29 May 2012 00:14:04 +1000 From: Darren Reed Organization: FreeBSD User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: Nikos Vassiliadis References: <4FC36377.1080306@freebsd.org> <4FC3818A.8080801@gmx.com> In-Reply-To: <4FC3818A.8080801@gmx.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: Point-to-point connection between jails? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: darrenr@freebsd.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 May 2012 14:12:29 -0000 On 28/05/2012 11:45 PM, Nikos Vassiliadis wrote: > On 5/28/2012 1:37 PM, Darren Reed wrote: >> I've looked through the list of network interfaces that are >> supported with FreeBSD and none seem to meet my needs. What >> I want is a network interface that I can configure up in >> jail A with address 10.1.1.1 and for which I can configure >> a mate in jail B with the address 10.2.2.2 and use the >> network interface as the means by which two jails can >> directly communicate with each other without the need to >> send any packets out of the machine. Or another way to do >> this would be to have a virtual network (something like the >> "internal network" that VirtualBox has or the host only >> network supported by VMWware Workstation) defined somewhere >> and for there to be a specific driver that could be >> configured and attached to a jail and that virtual network >> so that you could have N:M communication between jails. >> >> Is what I'm looking for already present and google is failing >> me or is the above functionality the basis for future work, >> be it planned or otherwise? > > It seems like a loopback interface does this. > > root@raidmadi:/home/nik # jls > JID IP Address Hostname Path > 3 10.2.3.4 / > 4 10.7.3.4 / > root@raidmadi:/home/nik # ifconfig lo1 > lo1: flags=8049 metric 0 mtu 16384 > options=3 > inet 10.2.3.4 netmask 0xff000000 > inet 10.7.3.4 netmask 0xff000000 > root@raidmadi:/home/nik # > > Maybe you want 'real' isolation? like with epair and VIMAGE? Yes, I was after real isolation but this might work. Darren From owner-freebsd-net@FreeBSD.ORG Mon May 28 16:00:16 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A24F81065672 for ; Mon, 28 May 2012 16:00:16 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 81B7A8FC12 for ; Mon, 28 May 2012 16:00:13 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id ECABA7300A; Mon, 28 May 2012 18:12:40 +0200 (CEST) Date: Mon, 28 May 2012 18:12:40 +0200 From: Luigi Rizzo To: net@freebsd.org Message-ID: <20120528161240.GA38291@onelab2.iet.unipi.it> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Cc: Subject: some questions on virtual machine bridging. X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 May 2012 16:00:16 -0000 I am doing some experiments with implementing a software bridge between virtual machines, using netmap as the communication API. I have a first prototype up and running and it is quite fast (10 Mpps with 60-byte frames, 4 Mpps with 1500 byte frames, compared to the ~500-800Kpps @60 bytes that you get with the tap interface used by openvswitch or the native linux bridging). I was wondering if anyone has comments/suggestions on the following: * what kind of API is used by the various virtualization solution to do virtual machine switching ? - On linux, kvm seems to rely on "tap" interfaces and native linux bridging, which i believe is more or less the same solution used by FreeBSD. - Slightly less efficient is perhaps the use of a socket and multicast packets, or bpf. - and of course, using PCI passthrough you get more or less hw speed (constrained by the OS), but need support from an external switch or the NIC itself to do forwarding between different ports. anything else ? * any high-performance virtual switching solution around ? As mentioned, i have measured native linux bridging and in-kernel ovs and the numbers are above (not surprising; the tap involves a syscall on each packet if i am not mistaken, and internally you need a data copy) * how many ports should i support ? * the hash function normally used for bridging (both in Linux and in FreeBSD -- see the latter below) is one of the Jenkins functions. It seems to take about 20ns to compute on my machine, which is a non-negligible amount of time (haven't tried to optimize it). Any reference on why this is so popular ? cheers luigi --- below, the hash function used by FreeBSD bridging --- /* * The following hash function is adapted from "Hash Functions" by Bob Jenkins * ("Algorithm Alley", Dr. Dobbs Journal, September 1997). * * http://www.burtleburtle.net/bob/hash/spooky.html */ #define mix(a, b, c) \ do { \ a -= b; a -= c; a ^= (c >> 13); \ b -= c; b -= a; b ^= (a << 8); \ c -= a; c -= b; c ^= (b >> 13); \ a -= b; a -= c; a ^= (c >> 12); \ b -= c; b -= a; b ^= (a << 16); \ c -= a; c -= b; c ^= (b >> 5); \ a -= b; a -= c; a ^= (c >> 3); \ b -= c; b -= a; b ^= (a << 10); \ c -= a; c -= b; c ^= (b >> 15); \ } while (/*CONSTCOND*/0) static __inline uint32_t nm_bridge_rthash(const uint8_t *addr) { uint32_t a = 0x9e3779b9, b = 0x9e3779b9, c = 0; // hask key b += addr[5] << 8; b += addr[4]; a += addr[3] << 24; a += addr[2] << 16; a += addr[1] << 8; a += addr[0]; mix(a, b, c); #define BRIDGE_RTHASH_MASK (NM_BDG_HASH-1) return (c & BRIDGE_RTHASH_MASK); } ----------------------------------------------------------------- From owner-freebsd-net@FreeBSD.ORG Mon May 28 19:08:18 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C5DA4106566B for ; Mon, 28 May 2012 19:08:18 +0000 (UTC) (envelope-from krzysiek@airnet.opole.pl) Received: from base.airnet.opole.pl (ns2.airmax.pl [176.111.128.3]) by mx1.freebsd.org (Postfix) with ESMTP id 7DA438FC15 for ; Mon, 28 May 2012 19:08:17 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by base.airnet.opole.pl (Postfix) with ESMTP id DF08A7FF049 for ; Mon, 28 May 2012 21:00:20 +0200 (CEST) Received: from base.airnet.opole.pl ([127.0.0.1]) by localhost (mail.airnet.opole.pl [127.0.0.1]) (maiad, port 10024) with ESMTP id 10913-10 for ; Mon, 28 May 2012 21:00:20 +0200 (CEST) Received: from [10.10.11.223] (unknown [176.111.138.12]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: krzysiek@airnet.opole.pl) by base.airnet.opole.pl (Postfix) with ESMTPSA id B651F7FF040 for ; Mon, 28 May 2012 21:00:20 +0200 (CEST) Message-ID: <4FC3CB40.6060704@airnet.opole.pl> Date: Mon, 28 May 2012 21:00:16 +0200 From: Krzysztof Barcikowski User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit Subject: ipfw: invalid oid len 0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 May 2012 19:08:18 -0000 Hi! Sometimes when I try to list IPFW pipes, I get the following message: # ipfw pipe show ipfw: invalid oid len 0 Why? Is there something wrong with my configuration? Best regards! Krzysztof Barcikowski From owner-freebsd-net@FreeBSD.ORG Mon May 28 22:12:58 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D230B1065670 for ; Mon, 28 May 2012 22:12:58 +0000 (UTC) (envelope-from camzal@trash-mail.com) Received: from sam.nabble.com (sam.nabble.com [216.139.236.26]) by mx1.freebsd.org (Postfix) with ESMTP id AABF58FC0C for ; Mon, 28 May 2012 22:12:58 +0000 (UTC) Received: from [192.168.236.26] (helo=sam.nabble.com) by sam.nabble.com with esmtp (Exim 4.72) (envelope-from ) id 1SZ8BN-0005Xg-Om for freebsd-net@freebsd.org; Mon, 28 May 2012 15:12:57 -0700 Date: Mon, 28 May 2012 15:12:57 -0700 (PDT) From: camzal To: freebsd-net@freebsd.org Message-ID: <1338243177761-5713012.post@n5.nabble.com> In-Reply-To: <20120425210856.GB8498@michelle.cdnetworks.com> References: <20120425210856.GB8498@michelle.cdnetworks.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Re: Realtek 8111F X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 May 2012 22:12:58 -0000 I have teh same problem and I wasn't successful to compile a driver. Can somebody upload a compiled amd64 driver? -- View this message in context: http://freebsd.1045724.n5.nabble.com/Realtek-8111F-tp5661828p5713012.html Sent from the freebsd-net mailing list archive at Nabble.com. From owner-freebsd-net@FreeBSD.ORG Tue May 29 00:21:11 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5E980106566B; Tue, 29 May 2012 00:21:11 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id E23B08FC08; Tue, 29 May 2012 00:21:09 +0000 (UTC) Received: from julian-mac.elischer.org (c-67-180-24-15.hsd1.ca.comcast.net [67.180.24.15]) (authenticated bits=0) by vps1.elischer.org (8.14.5/8.14.5) with ESMTP id q4SNuBTT091477 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 28 May 2012 16:56:13 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <4FC410A9.4000502@freebsd.org> Date: Mon, 28 May 2012 16:56:25 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.28) Gecko/20120306 Thunderbird/3.1.20 MIME-Version: 1.0 To: darrenr@freebsd.org References: <4FC36377.1080306@freebsd.org> <4FC3818A.8080801@gmx.com> <4FC3882C.5030105@freebsd.org> In-Reply-To: <4FC3882C.5030105@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Nikos Vassiliadis Subject: Re: Point-to-point connection between jails? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 May 2012 00:21:11 -0000 On 5/28/12 7:14 AM, Darren Reed wrote: > On 28/05/2012 11:45 PM, Nikos Vassiliadis wrote: >> On 5/28/2012 1:37 PM, Darren Reed wrote: >>> I've looked through the list of network interfaces that are >>> supported with FreeBSD and none seem to meet my needs. What >>> I want is a network interface that I can configure up in >>> jail A with address 10.1.1.1 and for which I can configure >>> a mate in jail B with the address 10.2.2.2 and use the >>> network interface as the means by which two jails can >>> directly communicate with each other without the need to >>> send any packets out of the machine. Or another way to do >>> this would be to have a virtual network (something like the >>> "internal network" that VirtualBox has or the host only >>> network supported by VMWware Workstation) defined somewhere >>> and for there to be a specific driver that could be >>> configured and attached to a jail and that virtual network >>> so that you could have N:M communication between jails. >>> >>> Is what I'm looking for already present and google is failing >>> me or is the above functionality the basis for future work, >>> be it planned or otherwise? >> It seems like a loopback interface does this. >> >> root@raidmadi:/home/nik # jls >> JID IP Address Hostname Path >> 3 10.2.3.4 / >> 4 10.7.3.4 / >> root@raidmadi:/home/nik # ifconfig lo1 >> lo1: flags=8049 metric 0 mtu 16384 >> options=3 >> inet 10.2.3.4 netmask 0xff000000 >> inet 10.7.3.4 netmask 0xff000000 >> root@raidmadi:/home/nik # >> >> Maybe you want 'real' isolation? like with epair and VIMAGE? > Yes, I was after real isolation but this might work. what you want is epair which is a pseudo driver pair, specifically designed to connect two vimage jails to each other. > Darren > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to"freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Tue May 29 07:50:55 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 99E021065675; Tue, 29 May 2012 07:50:55 +0000 (UTC) (envelope-from darrenr@freebsd.org) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) by mx1.freebsd.org (Postfix) with ESMTP id 5DDA28FC14; Tue, 29 May 2012 07:50:55 +0000 (UTC) Received: from compute2.internal (compute2.nyi.mail.srv.osa [10.202.2.42]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id C628C20E27; Tue, 29 May 2012 03:50:54 -0400 (EDT) Received: from frontend2.nyi.mail.srv.osa ([10.202.2.161]) by compute2.internal (MEProxy); Tue, 29 May 2012 03:50:54 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:date:from:reply-to :mime-version:to:cc:subject:references:in-reply-to:content-type :content-transfer-encoding; s=smtpout; bh=elcfcOAxs+J43d8zpe+EKY aJsQ8=; b=bQrm7F8atRZXYYFEufhvLC7Ty+SFZQSGNk+skBRXpR/GCiDXvy0gIl t2Pw27LVNQ5evdcecZygrVjk1xtCCG2esZK5b5hDqrbH6Z+OXhmTzxz9bjjGqsFm fPA3uz8t9fn0UwBaTFATJnOz86oV9X0P4UjkM2oZ3rBjwL8ZJ/SfQ= X-Sasl-enc: qUCScGKp5ES9o1bEqC8MmcLsYPhoqby142p8pdDg01dR 1338277854 Received: from [192.168.1.124] (unknown [202.45.110.141]) by mail.messagingengine.com (Postfix) with ESMTPA id 99BD44825F7; Tue, 29 May 2012 03:50:53 -0400 (EDT) Message-ID: <4FC4802E.4070105@freebsd.org> Date: Tue, 29 May 2012 17:52:14 +1000 From: Darren Reed Organization: FreeBSD User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: Julian Elischer References: <4FC36377.1080306@freebsd.org> <4FC3818A.8080801@gmx.com> <4FC3882C.5030105@freebsd.org> <4FC410A9.4000502@freebsd.org> In-Reply-To: <4FC410A9.4000502@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Nikos Vassiliadis Subject: Re: Point-to-point connection between jails? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: darrenr@freebsd.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 May 2012 07:50:55 -0000 On 29/05/2012 9:56 AM, Julian Elischer wrote: > On 5/28/12 7:14 AM, Darren Reed wrote: >> On 28/05/2012 11:45 PM, Nikos Vassiliadis wrote: >>> On 5/28/2012 1:37 PM, Darren Reed wrote: >>>> I've looked through the list of network interfaces that are >>>> supported with FreeBSD and none seem to meet my needs. What >>>> I want is a network interface that I can configure up in >>>> jail A with address 10.1.1.1 and for which I can configure >>>> a mate in jail B with the address 10.2.2.2 and use the >>>> network interface as the means by which two jails can >>>> directly communicate with each other without the need to >>>> send any packets out of the machine. Or another way to do >>>> this would be to have a virtual network (something like the >>>> "internal network" that VirtualBox has or the host only >>>> network supported by VMWware Workstation) defined somewhere >>>> and for there to be a specific driver that could be >>>> configured and attached to a jail and that virtual network >>>> so that you could have N:M communication between jails. >>>> >>>> Is what I'm looking for already present and google is failing >>>> me or is the above functionality the basis for future work, >>>> be it planned or otherwise? >>> It seems like a loopback interface does this. >>> >>> root@raidmadi:/home/nik # jls >>> JID IP Address Hostname Path >>> 3 10.2.3.4 / >>> 4 10.7.3.4 / >>> root@raidmadi:/home/nik # ifconfig lo1 >>> lo1: flags=8049 metric 0 mtu 16384 >>> options=3 >>> inet 10.2.3.4 netmask 0xff000000 >>> inet 10.7.3.4 netmask 0xff000000 >>> root@raidmadi:/home/nik # >>> >>> Maybe you want 'real' isolation? like with epair and VIMAGE? >> Yes, I was after real isolation but this might work. > > what you want is epair which is a pseudo driver pair, > specifically designed to connect two vimage jails to each other. Yes, that's it. A good example of using epairs can be found here: http://zewaren.net/site/?q=node/71 Something like this should be documented better on freebsd.org. Darren From owner-freebsd-net@FreeBSD.ORG Tue May 29 10:20:13 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A3598106566C for ; Tue, 29 May 2012 10:20:13 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 681AA8FC0A for ; Tue, 29 May 2012 10:20:13 +0000 (UTC) Received: from ds4.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id 66FC46368 for ; Tue, 29 May 2012 10:20:12 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id D612F8EC4; Tue, 29 May 2012 12:20:11 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: net@freebsd.org Date: Tue, 29 May 2012 12:20:11 +0200 Message-ID: <864nqz70h0.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Subject: 10 Gbps NIC selection X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 May 2012 10:20:13 -0000 Given the choice between the following adapters: Broadcom 5720 Broadcom 5719 Broadcom 57800 Broadcom 57810 Intel i350 Intel x520 Intel X540 which one would be the best choice for FreeBSD 9? Am I right to surmise that we don't support Broadcom chips at all? (please don't answer "Chelsio", these are the only choices our supplier offers) DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-net@FreeBSD.ORG Tue May 29 16:20:18 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9A3A7106564A; Tue, 29 May 2012 16:20:18 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id 4FABB8FC0A; Tue, 29 May 2012 16:20:18 +0000 (UTC) Received: from julian-mac.elischer.org (c-67-180-24-15.hsd1.ca.comcast.net [67.180.24.15]) (authenticated bits=0) by vps1.elischer.org (8.14.5/8.14.5) with ESMTP id q4TGKFnK095312 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 29 May 2012 09:20:16 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <4FC4F74C.4080104@freebsd.org> Date: Tue, 29 May 2012 09:20:28 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.28) Gecko/20120306 Thunderbird/3.1.20 MIME-Version: 1.0 To: darrenr@freebsd.org References: <4FC36377.1080306@freebsd.org> <4FC3818A.8080801@gmx.com> <4FC3882C.5030105@freebsd.org> <4FC410A9.4000502@freebsd.org> <4FC4802E.4070105@freebsd.org> In-Reply-To: <4FC4802E.4070105@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Nikos Vassiliadis Subject: Re: Point-to-point connection between jails? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 May 2012 16:20:18 -0000 On 5/29/12 12:52 AM, Darren Reed wrote: > On 29/05/2012 9:56 AM, Julian Elischer wrote: >> On 5/28/12 7:14 AM, Darren Reed wrote: >>> On 28/05/2012 11:45 PM, Nikos Vassiliadis wrote: >>>> On 5/28/2012 1:37 PM, Darren Reed wrote: >>>>> I've looked through the list of network interfaces that are >>>>> supported with FreeBSD and none seem to meet my needs. What >>>>> I want is a network interface that I can configure up in >>>>> jail A with address 10.1.1.1 and for which I can configure >>>>> a mate in jail B with the address 10.2.2.2 and use the >>>>> network interface as the means by which two jails can >>>>> directly communicate with each other without the need to >>>>> send any packets out of the machine. Or another way to do >>>>> this would be to have a virtual network (something like the >>>>> "internal network" that VirtualBox has or the host only >>>>> network supported by VMWware Workstation) defined somewhere >>>>> and for there to be a specific driver that could be >>>>> configured and attached to a jail and that virtual network >>>>> so that you could have N:M communication between jails. >>>>> >>>>> Is what I'm looking for already present and google is failing >>>>> me or is the above functionality the basis for future work, >>>>> be it planned or otherwise? >>>> It seems like a loopback interface does this. >>>> >>>> root@raidmadi:/home/nik # jls >>>> JID IP Address Hostname Path >>>> 3 10.2.3.4 / >>>> 4 10.7.3.4 / >>>> root@raidmadi:/home/nik # ifconfig lo1 >>>> lo1: flags=8049 metric 0 mtu 16384 >>>> options=3 >>>> inet 10.2.3.4 netmask 0xff000000 >>>> inet 10.7.3.4 netmask 0xff000000 >>>> root@raidmadi:/home/nik # >>>> >>>> Maybe you want 'real' isolation? like with epair and VIMAGE? >>> Yes, I was after real isolation but this might work. >> what you want is epair which is a pseudo driver pair, >> specifically designed to connect two vimage jails to each other. > Yes, that's it. A good example of using epairs can be found here: > http://zewaren.net/site/?q=node/71 though you don't need the bridge part if you don't want your jail bridged through to the internet. You can also achieve the same thing using netgraph. > Something like this should be documented better on freebsd.org. > > Darren > > From owner-freebsd-net@FreeBSD.ORG Tue May 29 21:36:44 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EC03A106566C for ; Tue, 29 May 2012 21:36:44 +0000 (UTC) (envelope-from seanbru@yahoo-inc.com) Received: from mrout1-b.corp.bf1.yahoo.com (mrout1-b.corp.bf1.yahoo.com [98.139.253.104]) by mx1.freebsd.org (Postfix) with ESMTP id 9C0728FC16 for ; Tue, 29 May 2012 21:36:44 +0000 (UTC) Received: from [IPv6:::1] (rideseveral.corp.yahoo.com [10.73.160.231]) by mrout1-b.corp.bf1.yahoo.com (8.14.4/8.14.4/y.out) with ESMTP id q4TLaLpd037364; Tue, 29 May 2012 14:36:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yahoo-inc.com; s=cobra; t=1338327382; bh=4903j5hxdC/Nr2nByiQ5HysefXcxWPj32LqHzMQVJs0=; h=Subject:From:To:Cc:In-Reply-To:References:Content-Type:Date: Message-ID:Mime-Version:Content-Transfer-Encoding; b=ASa9qvOmyBTpDq3dFzZU3WiZ8pSRf3rQVNI6Fuh+/Ys6qlXXdSHuoWibJkJ+kkjce Zrq1c1/0srth0Yxri1g/XUJZmVBa0N/mWQK2ezKwELSUEXSVChskWntxr+JeHa3QSP Qr12cEKZDwytZqap7j/JDnhVtP4c5hdg4u9aFHXE= From: Sean Bruno To: Dag-Erling =?ISO-8859-1?Q?Sm=F8rgrav?= In-Reply-To: <864nqz70h0.fsf@ds4.des.no> References: <864nqz70h0.fsf@ds4.des.no> Content-Type: text/plain; charset="UTF-8" Date: Tue, 29 May 2012 14:36:21 -0700 Message-ID: <1338327381.2850.11.camel@powernoodle-l7.corp.yahoo.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 8bit X-Milter-Version: master.31+4-gbc07cd5+ X-CLX-ID: 327381006 Cc: "net@freebsd.org" Subject: Re: 10 Gbps NIC selection X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 May 2012 21:36:45 -0000 On Tue, 2012-05-29 at 03:20 -0700, Dag-Erling Smørgrav wrote: > Given the choice between the following adapters: > Here's my list of "stuff and things", I hope this is helpful > Broadcom 5720 Haven't gotten this one working on the Dell R series I'm testing (thought this was a 1G chipset) > Broadcom 5719 Thought this was a 1G? > Broadcom 57800 > Broadcom 57810 If these are 10G E cards, the would need to be added to bxe(4) which it doesn't appear they are? I know I haven't used them. > Intel i350 > Intel x520 > Intel X540 The X540T appears to be supported via ixgbe(4) ... no testing from my side on this one unfortuneately, but Jack is pretty active and probably knows the current level of support. Mailing list activity seems to indicate that Intel 10GE is more likely to "just work" if I'm not mistaken. with my "purple" hat on, we pretty much swear by Intel 10G AFAIK. Sean From owner-freebsd-net@FreeBSD.ORG Tue May 29 21:45:11 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 62E001065672 for ; Tue, 29 May 2012 21:45:11 +0000 (UTC) (envelope-from jeffrey.e.pieper@intel.com) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by mx1.freebsd.org (Postfix) with ESMTP id 33A568FC16 for ; Tue, 29 May 2012 21:45:11 +0000 (UTC) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga101.jf.intel.com with ESMTP; 29 May 2012 14:45:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.67,351,1309762800"; d="scan'208";a="149966456" Received: from orsmsx606.amr.corp.intel.com ([10.22.226.128]) by orsmga002.jf.intel.com with ESMTP; 29 May 2012 14:45:05 -0700 Received: from orsmsx105.amr.corp.intel.com (10.22.225.132) by orsmsx606.amr.corp.intel.com (10.22.226.128) with Microsoft SMTP Server (TLS) id 8.2.255.0; Tue, 29 May 2012 14:45:04 -0700 Received: from orsmsx101.amr.corp.intel.com ([169.254.8.121]) by ORSMSX105.amr.corp.intel.com ([169.254.4.232]) with mapi id 14.01.0355.002; Tue, 29 May 2012 14:45:04 -0700 From: "Pieper, Jeffrey E" To: Sean Bruno , =?utf-8?B?RGFnLUVybGluZyBTbcO4cmdyYXY=?= Thread-Topic: 10 Gbps NIC selection Thread-Index: AQHNPeM+FMqizXZ/EkKxXodMWe5mGZbhTBOw Date: Tue, 29 May 2012 21:45:04 +0000 Message-ID: <2A35EA60C3C77D438915767F458D6568481A7E72@ORSMSX101.amr.corp.intel.com> References: <864nqz70h0.fsf@ds4.des.no> <1338327381.2850.11.camel@powernoodle-l7.corp.yahoo.com> In-Reply-To: <1338327381.2850.11.camel@powernoodle-l7.corp.yahoo.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.22.254.140] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Cc: "net@freebsd.org" Subject: RE: 10 Gbps NIC selection X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 May 2012 21:45:11 -0000 DQoNCi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpGcm9tOiBvd25lci1mcmVlYnNkLW5ldEBm cmVlYnNkLm9yZyBbbWFpbHRvOm93bmVyLWZyZWVic2QtbmV0QGZyZWVic2Qub3JnXSBPbiBCZWhh bGYgT2YgU2VhbiBCcnVubw0KU2VudDogVHVlc2RheSwgTWF5IDI5LCAyMDEyIDI6MzYgUE0NClRv OiBEYWctRXJsaW5nIFNtw7hyZ3Jhdg0KQ2M6IG5ldEBmcmVlYnNkLm9yZw0KU3ViamVjdDogUmU6 IDEwIEdicHMgTklDIHNlbGVjdGlvbg0KDQoNCj4gICBJbnRlbCBpMzUwDQo+ICAgSW50ZWwgeDUy MA0KPiAgIEludGVsIFg1NDANCg0KDQpPbmUgc21hbGwgIG5pdCAtIGkzNTAgaXMgYSAgR2IgcGFy dC4NCg0KSmVmZg0K From owner-freebsd-net@FreeBSD.ORG Tue May 29 21:48:17 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6C0EC1065675 for ; Tue, 29 May 2012 21:48:17 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 2D8C68FC17 for ; Tue, 29 May 2012 21:48:16 +0000 (UTC) Received: from ds4.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id 24BA86586; Tue, 29 May 2012 21:48:16 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id E8EB08F66; Tue, 29 May 2012 23:48:15 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Sean Bruno References: <864nqz70h0.fsf@ds4.des.no> <1338327381.2850.11.camel@powernoodle-l7.corp.yahoo.com> Date: Tue, 29 May 2012 23:48:15 +0200 In-Reply-To: <1338327381.2850.11.camel@powernoodle-l7.corp.yahoo.com> (Sean Bruno's message of "Tue, 29 May 2012 14:36:21 -0700") Message-ID: <861um24q1s.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: "net@freebsd.org" Subject: Re: 10 Gbps NIC selection X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 May 2012 21:48:17 -0000 Sean Bruno writes: > Dag-Erling Sm=C3=B8rgrav writes: > > Given the choice between the following adapters: > > Broadcom 5720 > Haven't gotten this one working on the Dell R series I'm testing > (thought this was a 1G chipset) > > Broadcom 5719 > Thought this was a 1G? > > Broadcom 57800 > > Broadcom 57810 > If these are 10G E cards, the would need to be added to bxe(4) which it > doesn't appear they are? I know I haven't used them. It is quite possible that our supplier either misunderstood the question or misformatted his answer. I can't find a bxe(4) man page in yesterday's head, so I'll pretend it doesn't exist. > > Intel i350 > > Intel x520 > > Intel X540 > The X540T appears to be supported via ixgbe(4) [...] I believe they're all supported by either ixgb(4) or ixgbe(4). However, given that a) the i350 is a quad-port card, which disqualifies it for our purposes b) our supplier's x540s are RJ45 only c) someone told me off-list that they used x520s and were happy with them I decided to get two dual-SFP x520s. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-net@FreeBSD.ORG Wed May 30 10:44:31 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2E5A4106566C for ; Wed, 30 May 2012 10:44:31 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (muon.cran.org.uk [IPv6:2a01:348:0:15:5d59:5c40:0:1]) by mx1.freebsd.org (Postfix) with ESMTP id B28E58FC08 for ; Wed, 30 May 2012 10:44:30 +0000 (UTC) Received: from muon.cran.org.uk (localhost [127.0.0.1]) by muon.cran.org.uk (Postfix) with ESMTP id F2B61E6546 for ; Wed, 30 May 2012 11:45:02 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=cran.org.uk; h=message-id :date:from:mime-version:to:subject:content-type :content-transfer-encoding; s=mail; bh=GJV5E3f+aVWFtpxwNMKmaRbZN eU=; b=jcfxNdZ8hRKNK2RfgbfL8ZXnKPlS1LV93xvDo0jFX+fpsKdS7rsd279wt ccOszql5x/1e3s80vz4LW+Sycd+FzJHbuEmyfeEm+iUDcuIYA/nL1kXb3xGoRpbo kQqgjgFQHQogE6kZh8+3Bpqs4dVMMh9UJmaLVKG6GIp1+lub58= DomainKey-Signature: a=rsa-sha1; c=nofws; d=cran.org.uk; h=message-id :date:from:mime-version:to:subject:content-type :content-transfer-encoding; q=dns; s=mail; b=olWiaOgcPHYznw6Rx1I 5VOpWMVKhn03SK8pvzDShRhPwxaVWAls+Gqd/dhPp0nDqZNe1ZjtJwb/K/zt3lcW iQgntsuamEdtCAEYvWxJDW+RtV37ikUfglaSd5yRa9ktfJB0Hdg/oj0XBQ6/xKVd qeI9uRWuf45ILAPvTLEOEjUY= Received: from [192.168.2.115] (unknown [93.89.81.205]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA id AF66AE652F for ; Wed, 30 May 2012 11:45:02 +0100 (BST) Message-ID: <4FC5FA0B.5010609@cran.org.uk> Date: Wed, 30 May 2012 11:44:27 +0100 From: Bruce Cran User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: if_em and large mtu bug? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 May 2012 10:44:31 -0000 Are there any known problems with if_em and jumbo packets? I've found that my ssh connection breaks (when running dmesg etc.) if I configure the mtu to 9216 via rc.conf - but if I initially use a 1500 byte mtu and then manually configure it to 9216 then no problems occur. I'm running the latest -current: the hardware is: em0: port 0xdc00-0xdc1f mem 0xfbde0000-0xfbdfffff,0xfbd00000-0xfbd7ffff,0xfbddc000-0xfbddffff irq 16 at device 0.0 on pci1 em0: Using MSIX interrupts with 3 vectors -- Bruce Cran From owner-freebsd-net@FreeBSD.ORG Wed May 30 11:36:19 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2BE70106566B for ; Wed, 30 May 2012 11:36:19 +0000 (UTC) (envelope-from egrosbein@rdtc.ru) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13::5]) by mx1.freebsd.org (Postfix) with ESMTP id 86E308FC15 for ; Wed, 30 May 2012 11:36:17 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.5/8.14.5) with ESMTP id q4UBaAge083492; Wed, 30 May 2012 18:36:11 +0700 (NOVT) (envelope-from egrosbein@rdtc.ru) Message-ID: <4FC6062A.3010002@rdtc.ru> Date: Wed, 30 May 2012 18:36:10 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7 MIME-Version: 1.0 To: Bruce Cran References: <4FC5FA0B.5010609@cran.org.uk> In-Reply-To: <4FC5FA0B.5010609@cran.org.uk> Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 8bit Cc: freebsd-net@freebsd.org Subject: Re: if_em and large mtu bug? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 May 2012 11:36:19 -0000 30.05.2012 17:44, Bruce Cran ÐÉÛÅÔ: > Are there any known problems with if_em and jumbo packets? I've found > that my ssh connection breaks (when running dmesg etc.) if I configure > the mtu to 9216 via rc.conf - but if I initially use a 1500 byte mtu and > then manually configure it to 9216 then no problems occur. > > I'm running the latest -current: the hardware is: > > em0: port 0xdc00-0xdc1f mem > 0xfbde0000-0xfbdfffff,0xfbd00000-0xfbd7ffff,0xfbddc000-0xfbddffff irq 16 > at device 0.0 on pci1 > em0: Using MSIX interrupts with 3 vectors I confirm this with FreeBSD 8.2-STABLE/amd64, em(4) 7.3.2, Intel PRO/1000 EB. There is a workaround, though: ifconfig em0 -rxcsum -txcsum mtu 9000 work from the beginning. Eugene Grosbein From owner-freebsd-net@FreeBSD.ORG Wed May 30 12:30:31 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 34515106564A for ; Wed, 30 May 2012 12:30:31 +0000 (UTC) (envelope-from charybdis@telenet.be) Received: from sam.nabble.com (sam.nabble.com [216.139.236.26]) by mx1.freebsd.org (Postfix) with ESMTP id 0C0CC8FC0C for ; Wed, 30 May 2012 12:30:31 +0000 (UTC) Received: from [192.168.236.26] (helo=sam.nabble.com) by sam.nabble.com with esmtp (Exim 4.72) (envelope-from ) id 1SZi2o-0003Ve-NN for freebsd-net@freebsd.org; Wed, 30 May 2012 05:30:30 -0700 Date: Wed, 30 May 2012 05:30:30 -0700 (PDT) From: Kherim To: freebsd-net@freebsd.org Message-ID: <1338381030717-5713365.post@n5.nabble.com> In-Reply-To: <1338243177761-5713012.post@n5.nabble.com> References: <20120425210856.GB8498@michelle.cdnetworks.com> <1338243177761-5713012.post@n5.nabble.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Re: Realtek 8111F X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 May 2012 12:30:31 -0000 You can install this very easily (P8H77-I, fresh freebsd 9.0 install): 1) download the three driver files: http://www.freebsd.org/cgi/cvsweb.cgi/~checkout~/src/sys/dev/re/if_re.c?rev=1.222;content-type=text%2Fplain if_re.c http://www.freebsd.org/cgi/cvsweb.cgi/~checkout~/src/sys/pci/if_rl.c?rev=1.201;content-type=text%2Fplain if_rl.c http://www.freebsd.org/cgi/cvsweb.cgi/~checkout~/src/sys/pci/if_rlreg.h?rev=1.125;content-type=text%2Fplain if_rlreg.h 2) navigate to your download folder (or put it on a usb pen and navigate to that one) 3) copy files to the rght folder: # cp if_re.c /usr/src/sys/dev/re/if_re.c # cp if_rl.c /usr/src/sys/pci/if_rl.c # cp if_rlreg.h /usr/src/sys/pci/if_rlreg.h 4) recompile your kernel: # cd /usr/src/sys/i386/conf # cp GENERIC GENERIC.org # cd /usr/src # make buildkernel KERNCONF=GENERIC # make installkernel KERNCONF=GENERIC # reboot 5) check if adaptor is present # ifconfig -> should show something like: re0: .... ether .... inet .... 5) configurate your adaptor (easy dns config) # ee /etc/rc.conf 6) add the following line: # ifconfig_re0="DHCP" 7) save the file # reboot -- View this message in context: http://freebsd.1045724.n5.nabble.com/Realtek-8111F-tp5661828p5713365.html Sent from the freebsd-net mailing list archive at Nabble.com. From owner-freebsd-net@FreeBSD.ORG Wed May 30 15:00:33 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 861FA106564A for ; Wed, 30 May 2012 15:00:33 +0000 (UTC) (envelope-from bounces+73574-866e-freebsd-net=freebsd.org@sendgrid.me) Received: from o3.shared.sendgrid.net (o3.shared.sendgrid.net [208.117.48.85]) by mx1.freebsd.org (Postfix) with SMTP id 413928FC19 for ; Wed, 30 May 2012 15:00:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.info; h= message-id:date:from:mime-version:to:subject:content-type; s= smtpapi; bh=5Le9fEzoa1HSOl8LvmmipEwr77E=; b=ayAqSZZY+oPni8GiuOWw UidGDN/6ybEdGXXbjsl+pY9dANvjKgyPV8y3PmS8ofON9hKkPSg9wKyGAWgjDU5h rs4+3Xy1uyhespC5pVRTHQ7FdHy/AtoBRTLxMLHCbvl9yi0Z4ttWYiIxeVJo+AIM Wb3+vFZYKAn0T7T/yHII0kY= Received: by 10.41.149.112 with SMTP id f04-09.31929.4FC636105 Wed, 30 May 2012 15:00:32 +0000 (UTC) Received: from mail.tarsnap.com (unknown [10.9.180.5]) by mi4 (SG) with ESMTP id 4fc63610.462b.267c915 for ; Wed, 30 May 2012 10:00:32 -0500 (CST) Received: (qmail 73152 invoked from network); 30 May 2012 14:53:01 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by mail.tarsnap.com with ESMTP; 30 May 2012 14:53:01 -0000 Received: (qmail 31530 invoked from network); 30 May 2012 14:59:24 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by clamshell.daemonology.net with SMTP; 30 May 2012 14:59:24 -0000 Message-ID: <4FC635CC.5030608@freebsd.org> Date: Wed, 30 May 2012 07:59:24 -0700 From: Colin Percival User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:12.0) Gecko/20120509 Thunderbird/12.0.1 MIME-Version: 1.0 To: freebsd-net@freebsd.org X-Enigmail-Version: 1.5pre Content-Type: multipart/mixed; boundary="------------080401060809040608040406" X-Sendgrid-EID: ISGKkmHlRE12gnWy0TjFyGQSFR1WnpjQdICm3qu6YxE/M8yWr2tPxOGiwm4he0mAEepFs0TbBdEmUPMRyIxrR8WoxRmMW1VG+yyAPwl6IHFpeSuY4vX6lciQwghM98FFHfbqGhea1ieN2qzy3vk64vFt/BMKgB5pTgM3ezkwFt8= Subject: [please review] TSO mbuf chain length limiting patch X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 May 2012 15:00:33 -0000 This is a multi-part message in MIME format. --------------080401060809040608040406 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Hi all, The Xen virtual network interface has an issue (ok, really the issue is with the linux back-end, but that's what most people are using) where it can't handle scatter-gather writes with lots of pieces, aka. long mbuf chains. This currently bites us hard with TSO enabled, since it produces said long mbuf chains. I've attached a patch which: (1) adds a IFCAP_MB_CHAIN_LIMIT "capability" and a if_mbuf_chain_limit field to struct ifnet, (2) sets these in netfront when the IFCAP_TSO4 flag is set, (3) extends tcp_maxmtu to read the if_mbuf_chain_limit value (it doesn't exactly fit here, but this is where we look up interface properties...), (4) adds a tx_chain_limit field to struct tcpcb, (5) makes tcp_mss_update set tx_chain_limit using tcp_maxmtu, (6) adds a new m_copy_nbufs function which truncates the copied mbuf chain after a specified number of mbufs, and returns the copied data length, (7) uses these in tcp_output (rearranging some code so that if the output length is modified, statistics are updated with the correct value). I'm hoping to commit this and MFC it before 9.1-RELEASE; I believe I've avoided breaking any ABIs. In my testing on EC2 with 10 GbE and TSO turned on, I get ~250-300 Mbps without this patch and 3-4 Gbps with this patch; this replaces a patch I was using in my EC2 builds which did (6)&(7) above with a hard-coded maximum mbuf chain length. Please tell me all the things I did wrong. :-) -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid --------------080401060809040608040406 Content-Type: text/plain; charset=us-ascii; name="tx_chain_limit.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="tx_chain_limit.patch" Index: netinet/tcp_input.c =================================================================== --- netinet/tcp_input.c (revision 235780) +++ netinet/tcp_input.c (working copy) @@ -3301,6 +3301,7 @@ struct inpcb *inp = tp->t_inpcb; struct hc_metrics_lite metrics; int origoffer; + int tx_chain_limit = 0; #ifdef INET6 int isipv6 = ((inp->inp_vflag & INP_IPV6) != 0) ? 1 : 0; size_t min_protoh = isipv6 ? @@ -3321,8 +3322,10 @@ /* Initialize. */ #ifdef INET6 if (isipv6) { - maxmtu = tcp_maxmtu6(&inp->inp_inc, mtuflags); + maxmtu = tcp_maxmtu6_ext(&inp->inp_inc, mtuflags, + &tx_chain_limit); tp->t_maxopd = tp->t_maxseg = V_tcp_v6mssdflt; + tp->t_tx_chain_limit = tx_chain_limit; } #endif #if defined(INET) && defined(INET6) @@ -3330,8 +3333,10 @@ #endif #ifdef INET { - maxmtu = tcp_maxmtu(&inp->inp_inc, mtuflags); + maxmtu = tcp_maxmtu_ext(&inp->inp_inc, mtuflags, + &tx_chain_limit); tp->t_maxopd = tp->t_maxseg = V_tcp_mssdflt; + tp->t_tx_chain_limit = tx_chain_limit; } #endif Index: netinet/tcp_subr.c =================================================================== --- netinet/tcp_subr.c (revision 235780) +++ netinet/tcp_subr.c (working copy) @@ -1708,7 +1708,7 @@ * tcp_mss_update to get the peer/interface MTU. */ u_long -tcp_maxmtu(struct in_conninfo *inc, int *flags) +tcp_maxmtu_ext(struct in_conninfo *inc, int *flags, int * tx_chain_limit) { struct route sro; struct sockaddr_in *dst; @@ -1738,15 +1738,26 @@ ifp->if_hwassist & CSUM_TSO) *flags |= CSUM_TSO; } + if (tx_chain_limit != NULL) { + if (ifp->if_capabilities & IFCAP_MB_CHAIN_LIMIT) + *tx_chain_limit = ifp->if_mbuf_chain_limit; + } RTFREE(sro.ro_rt); } return (maxmtu); } + +u_long +tcp_maxmtu(struct in_conninfo *inc, int *flags) +{ + + return (tcp_maxmtu_ext(inc, flags, NULL)); +} #endif /* INET */ #ifdef INET6 u_long -tcp_maxmtu6(struct in_conninfo *inc, int *flags) +tcp_maxmtu6_ext(struct in_conninfo *inc, int *flags, int * tx_chain_limit) { struct route_in6 sro6; struct ifnet *ifp; @@ -1775,11 +1786,22 @@ ifp->if_hwassist & CSUM_TSO) *flags |= CSUM_TSO; } + if (tx_chain_limit != NULL) { + if (ifp->if_capabilities & IFCAP_MB_CHAIN_LIMIT) + *tx_chain_limit = ifp->if_mbuf_chain_limit; + } RTFREE(sro6.ro_rt); } return (maxmtu); } + +u_long +tcp_maxmtu6(struct in_conninfo *inc, int *flags) +{ + + return (tcp_maxmtu6_ext(inc, flags, NULL)); +} #endif /* INET6 */ #ifdef IPSEC Index: netinet/tcp_var.h =================================================================== --- netinet/tcp_var.h (revision 235780) +++ netinet/tcp_var.h (working copy) @@ -208,7 +208,8 @@ u_int t_keepintvl; /* interval between keepalives */ u_int t_keepcnt; /* number of keepalives before close */ - uint32_t t_ispare[8]; /* 5 UTO, 3 TBD */ + uint32_t t_tx_chain_limit; /* max # chained mbufs to tx for TSO */ + uint32_t t_ispare[7]; /* 5 UTO, 2 TBD */ void *t_pspare2[4]; /* 4 TBD */ uint64_t _pad[6]; /* 6 TBD (1-2 CC/RTT?) */ }; @@ -674,7 +675,9 @@ #endif void tcp_input(struct mbuf *, int); u_long tcp_maxmtu(struct in_conninfo *, int *); +u_long tcp_maxmtu_ext(struct in_conninfo *, int *, int *); u_long tcp_maxmtu6(struct in_conninfo *, int *); +u_long tcp_maxmtu6_ext(struct in_conninfo *, int *, int *); void tcp_mss_update(struct tcpcb *, int, int, struct hc_metrics_lite *, int *); void tcp_mss(struct tcpcb *, int); Index: netinet/tcp_output.c =================================================================== --- netinet/tcp_output.c (revision 235780) +++ netinet/tcp_output.c (working copy) @@ -801,16 +801,6 @@ struct mbuf *mb; u_int moff; - if ((tp->t_flags & TF_FORCEDATA) && len == 1) - TCPSTAT_INC(tcps_sndprobe); - else if (SEQ_LT(tp->snd_nxt, tp->snd_max) || sack_rxmit) { - tp->t_sndrexmitpack++; - TCPSTAT_INC(tcps_sndrexmitpack); - TCPSTAT_ADD(tcps_sndrexmitbyte, len); - } else { - TCPSTAT_INC(tcps_sndpack); - TCPSTAT_ADD(tcps_sndbyte, len); - } MGETHDR(m, M_DONTWAIT, MT_DATA); if (m == NULL) { SOCKBUF_UNLOCK(&so->so_snd); @@ -842,7 +832,8 @@ mtod(m, caddr_t) + hdrlen); m->m_len += len; } else { - m->m_next = m_copy(mb, moff, (int)len); + m->m_next = m_copy_nbufs(mb, moff, (int)len, + M_DONTWAIT, &len, tp->t_tx_chain_limit); if (m->m_next == NULL) { SOCKBUF_UNLOCK(&so->so_snd); (void) m_free(m); @@ -851,6 +842,18 @@ } } + /* Update stats here as m_copy_nbufs may have adjusted len. */ + if ((tp->t_flags & TF_FORCEDATA) && len == 1) + TCPSTAT_INC(tcps_sndprobe); + else if (SEQ_LT(tp->snd_nxt, tp->snd_max) || sack_rxmit) { + tp->t_sndrexmitpack++; + TCPSTAT_INC(tcps_sndrexmitpack); + TCPSTAT_ADD(tcps_sndrexmitbyte, len); + } else { + TCPSTAT_INC(tcps_sndpack); + TCPSTAT_ADD(tcps_sndbyte, len); + } + /* * If we're sending everything we've got, set PUSH. * (This will keep happy those implementations which only Index: sys/mbuf.h =================================================================== --- sys/mbuf.h (revision 235780) +++ sys/mbuf.h (working copy) @@ -894,6 +894,7 @@ int, int, int, int); struct mbuf *m_copypacket(struct mbuf *, int); void m_copy_pkthdr(struct mbuf *, struct mbuf *); +struct mbuf *m_copy_nbufs(struct mbuf *, int, int, int, long *, uint32_t); struct mbuf *m_copyup(struct mbuf *n, int len, int dstoff); struct mbuf *m_defrag(struct mbuf *, int); void m_demote(struct mbuf *, int); Index: kern/uipc_mbuf.c =================================================================== --- kern/uipc_mbuf.c (revision 235780) +++ kern/uipc_mbuf.c (working copy) @@ -525,12 +525,14 @@ * only their reference counts are incremented. */ struct mbuf * -m_copym(struct mbuf *m, int off0, int len, int wait) +m_copy_nbufs(struct mbuf *m, int off0, int len, int wait, long * outlen, + uint32_t nbufmax) { struct mbuf *n, **np; int off = off0; struct mbuf *top; int copyhdr = 0; + int len_orig = len; KASSERT(off >= 0, ("m_copym, negative off %d", off)); KASSERT(len >= 0, ("m_copym, negative len %d", len)); @@ -546,7 +548,7 @@ } np = ⊤ top = 0; - while (len > 0) { + while (len > 0 && --nbufmax != 0) { if (m == NULL) { KASSERT(len == M_COPYALL, ("m_copym, length > size of mbuf chain")); @@ -584,6 +586,9 @@ if (top == NULL) mbstat.m_mcfail++; /* XXX: No consistency. */ + if (outlen) + *outlen = len_orig - len; + return (top); nospace: m_freem(top); @@ -591,6 +596,13 @@ return (NULL); } +struct mbuf * +m_copym(struct mbuf *m, int off0, int len, int wait) +{ + + return (m_copy_nbufs(m, off0, len, wait, NULL, 0)); +} + /* * Returns mbuf chain with new head for the prepending case. * Copies from mbuf (chain) n from off for len to mbuf (chain) m Index: dev/xen/netfront/netfront.c =================================================================== --- dev/xen/netfront/netfront.c (revision 235780) +++ dev/xen/netfront/netfront.c (working copy) @@ -2040,6 +2040,9 @@ if (val) { np->xn_ifp->if_capabilities |= IFCAP_TSO4|IFCAP_LRO; printf(" feature-gso-tcp4"); + + np->xn_ifp->if_capabilities |= IFCAP_MB_CHAIN_LIMIT; + np->xn_ifp->if_mbuf_chain_limit = np->maxfrags; } printf("\n"); Index: net/if.h =================================================================== --- net/if.h (revision 235780) +++ net/if.h (working copy) @@ -230,6 +230,7 @@ #define IFCAP_VLAN_HWTSO 0x40000 /* can do IFCAP_TSO on VLANs */ #define IFCAP_LINKSTATE 0x80000 /* the runtime link state is dynamic */ #define IFCAP_NETMAP 0x100000 /* netmap mode supported/enabled */ +#define IFCAP_MB_CHAIN_LIMIT 0x200000 /* TX mbuf chain length limited */ #define IFCAP_HWCSUM (IFCAP_RXCSUM | IFCAP_TXCSUM) #define IFCAP_TSO (IFCAP_TSO4 | IFCAP_TSO6) Index: net/if_var.h =================================================================== --- net/if_var.h (revision 235780) +++ net/if_var.h (working copy) @@ -205,7 +205,8 @@ * be used with care where binary compatibility is required. */ char if_cspare[3]; - int if_ispare[4]; + int if_mbuf_chain_limit; /* if IFCAP_MB_CHAIN_LIMIT */ + int if_ispare[3]; void *if_pspare[8]; /* 1 netmap, 7 TDB */ }; --------------080401060809040608040406-- From owner-freebsd-net@FreeBSD.ORG Wed May 30 15:30:54 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DA658106566B; Wed, 30 May 2012 15:30:54 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.freebsd.org (Postfix) with ESMTP id 634158FC0A; Wed, 30 May 2012 15:30:54 +0000 (UTC) Received: from [192.168.200.2] (c-24-125-204-77.hsd1.va.comcast.net [24.125.204.77]) (authenticated bits=0) by duke.cs.duke.edu (8.14.5/8.14.5) with ESMTP id q4UFUlO8017589 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 30 May 2012 11:30:47 -0400 (EDT) X-DKIM: Sendmail DKIM Filter v2.8.3 duke.cs.duke.edu q4UFUlO8017589 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=cs.duke.edu; s=mail; t=1338391847; bh=eY5/99f72/hyIsYnn4tGISkXykojrcJ5uz6e5MpfhH0=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=EP+i3NMB/jQEhh8qV4IjDPrZu7wRU/uLbN1AxldHX2YXJ7V0il7a/OvMgNayu3AT4 xD+mlIQkkGO+SIaWUpJL0r/plzrB7ya5Uq5EJhgvJUe/7amqIp1oWP5morn+y+dzFj NCMV7aqcguvBRvN9OCKiz2qU2Qj6kmj11Iup6BiPY2hGFvToJhXJGVOkzQStRoVVVo FHVV7baXC8YQxIGEpRarbx681xirnHzZ/7wvrej55/J3Ms74s7+G2qlXHxEEXuDWhO uHG5NQnsgGpB7NTwRqCWwDz7ADC3wsrpSGvcVHAgisAIisgbE/WBbcamfa2K+27cWi 76kCpxvNVbWug== Message-ID: <4FC63D27.70807@cs.duke.edu> Date: Wed, 30 May 2012 11:30:47 -0400 From: Andrew Gallatin User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:11.0) Gecko/20120329 Thunderbird/11.0.1 MIME-Version: 1.0 To: freebsd-net@freebsd.org References: <4FC635CC.5030608@freebsd.org> In-Reply-To: <4FC635CC.5030608@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Colin Percival Subject: Re: [please review] TSO mbuf chain length limiting patch X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 May 2012 15:30:54 -0000 On 05/30/12 10:59, Colin Percival wrote: > Hi all, > > The Xen virtual network interface has an issue (ok, really the issue is with > the linux back-end, but that's what most people are using) where it can't > handle scatter-gather writes with lots of pieces, aka. long mbuf chains. > This currently bites us hard with TSO enabled, since it produces said long > mbuf chains. Colin, Thanks for pointing me at this. I've been talking about this with bz@ a little. I've never been clear about what the max TSO size supported by FreeBSD is. The NIC I maintain (mxge) is limited to 64K - epsilon for both IPv4 *AND* IPv6. Up until now, this has been enforced by the 16-bit ip length limit of IPv4 and we have not had IPv6 TSO until this week. With IPv6, I'm worried that FreeBSD may now send packets down larger than I could handle. In my case, however, the problem is not s/g list length, but rather it is internal limits in the NIC which limit us to 64K - epsilon for IPv6 as well. I think there may be other NICs in the same boat for IPv6 (and maybe even some which cannot handle the full 64K for IPv4). Your approach would not work well for my size limit. For example, I'd have to set the limit to 4 mbufs to stay under 64KB. This would be assuming the worst case of 16KB jumbo mbufs, so that would limit me to ~8KB per TSO if 2KB mbufs were used. I think a better approach would be to have a limit on the size of the pre-segmented TCP payload size sent to the driver. I tend to think that this would be more generically useful, and it is a better match for the NDIS APIs, where a driver must specify the max TSO size. I think the changes to the TCP stack might be simpler (eg, they would seem to jive better with the existing "maxmtu" approach). I think this could work for you as well. You could set the Xen max tso size to be 32K (derived from 18 pages/skb, multiplied by a typical 2KB mbuf size, with some slack built in). If the chain was too large, you could m_defrag it down to size. Drew From owner-freebsd-net@FreeBSD.ORG Wed May 30 15:58:18 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1268B1065670 for ; Wed, 30 May 2012 15:58:18 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 6501F8FC1E for ; Wed, 30 May 2012 15:58:17 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id BC42AB9BF; Wed, 30 May 2012 11:58:16 -0400 (EDT) From: John Baldwin To: freebsd-net@freebsd.org Date: Wed, 30 May 2012 11:29:51 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p13; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201205301129.51295.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 30 May 2012 11:58:16 -0400 (EDT) Cc: Vijay Singh Subject: Re: Intel 1G Tx hangs X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 May 2012 15:58:18 -0000 On Thursday, May 24, 2012 1:19:31 am Vijay Singh wrote: > Hi, I have been using 8.2 based e1000 drivers and I'm seeing watchdog > timeouts. I am in the process of updating the drivers to 8-stable, but > wanted to check here if others have seen anything like this in the > 8.1/8.2 drivers. Just fyi, I am seeing the Tx hang in if_lem.c. Hmm, if_lem.c hasn't really changed in a long while. There are fixes to if_em.c and if_igb.c, but not any to if_lem.c that I am aware of. -- John Baldwin From owner-freebsd-net@FreeBSD.ORG Wed May 30 16:48:24 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B50F71065672; Wed, 30 May 2012 16:48:24 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 1ACF48FC0C; Wed, 30 May 2012 16:48:23 +0000 (UTC) Received: by werg1 with SMTP id g1so16611wer.13 for ; Wed, 30 May 2012 09:48:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=XJWLw269s2kEd882OdEJ5GXXnCb6sGJK96PuH+si0tg=; b=i8o8+PfOGm337EGluMl8b3iwAZIRrbMo6xxi9yyN6JZx4NEEfjCunrvDfKtj6yuSkO xtQHZAR/UbZ+fbFsJ6hUzHbvwmrgQqvUN3doINsY4bQ4q1lPp8RRXowpUlcHXE+QbSX8 dpikz63aX2xbQCyLPKl3DgugPDInp5hVT5AF+g4+v8yK5v69OgabFgR1vg5bdarhYGla Jm/EC+ru8DL13qOtfsjtuBR9435Kivx+vreoQi74mVtxyXl7OvjCXri8XYejJyDHk80E tYWyUSTYFCD8AoIy5lEJNsi23N6ersSiRt/U3cTicS28PLG75z3xuLmjT2gDOVjjXXoN BDqg== MIME-Version: 1.0 Received: by 10.216.228.88 with SMTP id e66mr10950078weq.208.1338396491877; Wed, 30 May 2012 09:48:11 -0700 (PDT) Received: by 10.180.105.232 with HTTP; Wed, 30 May 2012 09:48:11 -0700 (PDT) In-Reply-To: <201205301129.51295.jhb@freebsd.org> References: <201205301129.51295.jhb@freebsd.org> Date: Wed, 30 May 2012 09:48:11 -0700 Message-ID: From: Jack Vogel To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org, Vijay Singh Subject: Re: Intel 1G Tx hangs X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 May 2012 16:48:24 -0000 LOL, the whole reason for making lem was to have it not change, but someone that intent was lost on people, my failing i guess :) Of course, that didn't mean if its broken... Jack On Wed, May 30, 2012 at 8:29 AM, John Baldwin wrote: > On Thursday, May 24, 2012 1:19:31 am Vijay Singh wrote: > > Hi, I have been using 8.2 based e1000 drivers and I'm seeing watchdog > > timeouts. I am in the process of updating the drivers to 8-stable, but > > wanted to check here if others have seen anything like this in the > > 8.1/8.2 drivers. Just fyi, I am seeing the Tx hang in if_lem.c. > > Hmm, if_lem.c hasn't really changed in a long while. There are fixes > to if_em.c and if_igb.c, but not any to if_lem.c that I am aware of. > > -- > John Baldwin > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Wed May 30 16:50:47 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 338AA106566B; Wed, 30 May 2012 16:50:47 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 8FAD68FC18; Wed, 30 May 2012 16:50:46 +0000 (UTC) Received: by werg1 with SMTP id g1so18742wer.13 for ; Wed, 30 May 2012 09:50:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=YNsp0X/iQVgiIDAqVQiSO2ipJl0dgq5n71XUZBDEtyc=; b=E85Co+mrFqNyM0zmSxOej0PMGevzwsbI/KQe2z5eHpfpOEQ1jlRG5JOWDkbepnOrBZ 1m1WnkGmCLeKqbULE5Tg1yQHq9NnyrVEbaOKZev8LjeNEOhO7Adfy1b5kmUuG9iFKRsB jpf/a1RB1c+WL288QAWrvKtkQxInyROo6iLbevMQGJNiZP/otOARGgjgcPAXRdn3nizq eIjFS2/ugMx490bgtWu8ETDY/WpgN7LjDcDaUvJTceEQxA+IcCAejsDHp/9S3AsRO/aS pnmwX1IXeBoJAIfLNnpbT/7hrlQLdmKTDU54Jta1jDuCgHJ37nmPKpBIitR9pQVKiwZe vudw== MIME-Version: 1.0 Received: by 10.216.140.33 with SMTP id d33mr11047879wej.113.1338396645481; Wed, 30 May 2012 09:50:45 -0700 (PDT) Received: by 10.180.105.232 with HTTP; Wed, 30 May 2012 09:50:45 -0700 (PDT) In-Reply-To: <4FC63D27.70807@cs.duke.edu> References: <4FC635CC.5030608@freebsd.org> <4FC63D27.70807@cs.duke.edu> Date: Wed, 30 May 2012 09:50:45 -0700 Message-ID: From: Jack Vogel To: Andrew Gallatin Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org, Colin Percival Subject: Re: [please review] TSO mbuf chain length limiting patch X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 May 2012 16:50:47 -0000 On Wed, May 30, 2012 at 8:30 AM, Andrew Gallatin wrote: > On 05/30/12 10:59, Colin Percival wrote: > >> Hi all, >> >> The Xen virtual network interface has an issue (ok, really the issue is >> with >> the linux back-end, but that's what most people are using) where it can't >> handle scatter-gather writes with lots of pieces, aka. long mbuf chains. >> This currently bites us hard with TSO enabled, since it produces said long >> mbuf chains. >> > > Colin, > > Thanks for pointing me at this. I've been talking about this > with bz@ a little. > > I've never been clear about what the max TSO size supported by FreeBSD > is. The NIC I maintain (mxge) is limited to 64K - epsilon for both > IPv4 *AND* IPv6. Up until now, this has been enforced by the 16-bit > ip length limit of IPv4 and we have not had IPv6 TSO until this week. > With IPv6, I'm worried that FreeBSD may now send packets down larger > than I could handle. In my case, however, the problem is not s/g list > length, but rather it is internal limits in the NIC which limit us to > 64K - epsilon for IPv6 as well. I think there may be other NICs in > the same boat for IPv6 (and maybe even some which cannot handle the > full 64K for IPv4). > > Your approach would not work well for my size limit. For > example, I'd have to set the limit to 4 mbufs to stay under 64KB. > This would be assuming the worst case of 16KB jumbo mbufs, so > that would limit me to ~8KB per TSO if 2KB mbufs were used. > > I think a better approach would be to have a limit on the size of the > pre-segmented TCP payload size sent to the driver. I tend to think > that this would be more generically useful, and it is a better match > for the NDIS APIs, where a driver must specify the max TSO size. I > think the changes to the TCP stack might be simpler (eg, they > would seem to jive better with the existing "maxmtu" approach). > > I think this could work for you as well. You could set the Xen max > tso size to be 32K (derived from 18 pages/skb, multiplied by a typical > 2KB mbuf size, with some slack built in). If the chain was too large, > you could m_defrag it down to size. > Think I favor Drew's idea as well for what that's worth. Jack From owner-freebsd-net@FreeBSD.ORG Wed May 30 17:10:23 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 24D8F1065670 for ; Wed, 30 May 2012 17:10:23 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-ee0-f54.google.com (mail-ee0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id A84338FC14 for ; Wed, 30 May 2012 17:10:22 +0000 (UTC) Received: by eeke49 with SMTP id e49so20952eek.13 for ; Wed, 30 May 2012 10:10:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding :x-gm-message-state; bh=a7OIR4SSzXmcvI4/KcNSrvLSboRKMcpaWZ+EFAPWPQw=; b=PUODhkkP3txMeIGvcxkTTJ0NuKQkH/hmapuVXoRWbMx1W5cmm/vVrhjqh1TwAxLx6k 9aWDAUh5cWSf6/nGjRsr5TOrdZC/S8plOgnP2Oi7igH3yP/0b+bbS5jWv2E274xaMVy3 dFZlreNerfJNYZHGGQ7BT1bZ1TTXRrQ6egagKI9fePdoCwlPQ4wXFPYDyz5uZs3nEb4A JSK6CXM0H3eKnCM8M8ZdSQvhdYzfBXJKBb2R67jF0Nomesety2vk2GBHU2QF9e4+89is mTMjHJQjeJnBFZdzyC2FYHIDUJWqCYGhVpnmpVp36xjRjovNhKAgzsdPWuMC+nGsSaSF Y7PQ== Received: by 10.14.45.68 with SMTP id o44mr6594684eeb.66.1338397820786; Wed, 30 May 2012 10:10:20 -0700 (PDT) Received: from dfleuriot-at-hi-media.com ([83.167.62.196]) by mx.google.com with ESMTPS id q53sm1000863eef.8.2012.05.30.10.10.19 (version=SSLv3 cipher=OTHER); Wed, 30 May 2012 10:10:20 -0700 (PDT) Message-ID: <4FC6547A.2040308@my.gd> Date: Wed, 30 May 2012 19:10:18 +0200 From: Damien Fleuriot User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: freebsd-net@freebsd.org References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Gm-Message-State: ALoCoQlw3GnDahoibJZybn+Vh0dq3++snFz5vz41CnSFbsakLmjqr5dXDuLdj7AXDcznRZvXL7t/ Subject: Re: Intel 1G Tx hangs X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 May 2012 17:10:23 -0000 On 5/24/12 7:19 AM, Vijay Singh wrote: > Hi, I have been using 8.2 based e1000 drivers and I'm seeing watchdog > timeouts. I am in the process of updating the drivers to 8-stable, but > wanted to check here if others have seen anything like this in the > 8.1/8.2 drivers. Just fyi, I am seeing the Tx hang in if_lem.c. > > -vijay > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" Running 20ish guests in KVM with emulated e1000 cards and doing fine. Got a few physical ones in production as well, no hangs here. Mix of 8.1-RELEASE , 8.2-RELEASE , 8-STABLE From owner-freebsd-net@FreeBSD.ORG Wed May 30 17:30:08 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 576E8106566C for ; Wed, 30 May 2012 17:30:08 +0000 (UTC) (envelope-from lacombar@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id D88248FC0C for ; Wed, 30 May 2012 17:30:07 +0000 (UTC) Received: by werg1 with SMTP id g1so51433wer.13 for ; Wed, 30 May 2012 10:30:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=SvKfDvsiCr6f++2RnkEwx16GO5qf9Phb6yFavHIzMu8=; b=s6noVO/x2ZKYhg0Ee3+QQEQZ8wNxMqg/2TQX5uNCzYpV+ZmUNYAM527sxJ7h0rFmA3 9r+Agd3WTIsgmdqRPZjHiL9YA8iR5MZMhSaZVGdNEKCMT17g63W9LLiv41KS+D/0E6sK pyhkCNHjNutVagSef4bpWUTTbMr5xU3RurLJDwreQPiSoCgY1mVcGjAVTBFYy+KGYO/h 07POzMR/U1MfoK+HQfHNKaL9K64KuXL2330aLE8WHSzP4nMNkNnuuTbN4C/5q31SaWLb tv/1KvvNngOltwJoEalKXDl5+6N4m8O+syCCUaSjTm4KJnK2MCdSSoXfxRblYv0ojrLb m26w== MIME-Version: 1.0 Received: by 10.216.195.98 with SMTP id o76mr10841553wen.202.1338399004536; Wed, 30 May 2012 10:30:04 -0700 (PDT) Received: by 10.216.163.136 with HTTP; Wed, 30 May 2012 10:30:04 -0700 (PDT) In-Reply-To: <4FC6547A.2040308@my.gd> References: <4FC6547A.2040308@my.gd> Date: Wed, 30 May 2012 13:30:04 -0400 Message-ID: From: Arnaud Lacombe To: Damien Fleuriot Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net@freebsd.org Subject: Re: Intel 1G Tx hangs X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 May 2012 17:30:08 -0000 Hi, On Wed, May 30, 2012 at 1:10 PM, Damien Fleuriot wrote: > On 5/24/12 7:19 AM, Vijay Singh wrote: >> Hi, I have been using 8.2 based e1000 drivers and I'm seeing watchdog >> timeouts. I am in the process of updating the drivers to 8-stable, but >> wanted to check here if others have seen anything like this in the >> 8.1/8.2 drivers. Just fyi, I am seeing the Tx hang in if_lem.c. >> >> -vijay >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > Running 20ish guests in KVM with emulated e1000 cards and doing fine. > lucky you! I've been seeing complete RX failure with multiple emulated e1000 in qemu guest, cf kern/168246. It did not seem to have yet picked much interest from maintainer... - Arnaud > Got a few physical ones in production as well, no hangs here. > > > Mix of 8.1-RELEASE , 8.2-RELEASE , 8-STABLE > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Wed May 30 17:38:36 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1C3CD106564A for ; Wed, 30 May 2012 17:38:36 +0000 (UTC) (envelope-from vijju.singh@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id E2ADC8FC0C for ; Wed, 30 May 2012 17:38:35 +0000 (UTC) Received: by pbbro2 with SMTP id ro2so286707pbb.13 for ; Wed, 30 May 2012 10:38:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=4hnBUi/ruzp95p8fYIDmiCgwFcjq9RtC1Br4ydWLMIs=; b=WIkF4XlG3zTk6zdbDoRcTDaI69V8JTErROAmhazqBfE5i4VY2FQAL8V7jkjiIgWMQz KbLP1gk4taHX7Ww2GVTY7Pta5KxwY20aqPeDcJ8lfFzYmKAccy1qkIlSVsQ9HJb0JGdD TaveqczoaIlDq0BHiz7ZQ4zLpC0lm5i4P1emOQYwqIUiVY+ot+b8UA2vcmTT0ZhKIJfa IbVvyO9DlJDXlr4zAUJQ6VPe6OY/oZ+ECoHxqiHy+1W+oWMLfyVPWrGEyvRIHvMH3kbL ma0zA/83fbySp1bRtzDoBJ6g+IngFahByB0rve/Gzi9iKTgbDGRhhMnNFJPUuF2X3Ucu YItA== MIME-Version: 1.0 Received: by 10.68.193.233 with SMTP id hr9mr51293672pbc.99.1338399515498; Wed, 30 May 2012 10:38:35 -0700 (PDT) Received: by 10.142.165.21 with HTTP; Wed, 30 May 2012 10:38:35 -0700 (PDT) In-Reply-To: References: <4FC6547A.2040308@my.gd> Date: Wed, 30 May 2012 10:38:35 -0700 Message-ID: From: Vijay Singh To: Arnaud Lacombe Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org, Damien Fleuriot Subject: Re: Intel 1G Tx hangs X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 May 2012 17:38:36 -0000 >> Running 20ish guests in KVM with emulated e1000 cards and doing fine. >> > lucky you! I've been seeing complete RX failure with multiple emulated > e1000 in qemu guest, cf kern/168246. It did not seem to have yet > picked much interest from maintainer... > > =A0- Arnaud > >> Got a few physical ones in production as well, no hangs here. FYI, I updated our 8.2 code base to 8-stable e1k drivers and things appear to have stabilized. @jfv, the if_lem.c version did change from 1.0.1 to 1.0.4. -vijay From owner-freebsd-net@FreeBSD.ORG Wed May 30 22:26:09 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 321F1106567C for ; Wed, 30 May 2012 22:26:09 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) by mx1.freebsd.org (Postfix) with ESMTP id D84CC8FC08 for ; Wed, 30 May 2012 22:26:08 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id A437725D385D; Wed, 30 May 2012 22:26:07 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id BB88ABE8144; Wed, 30 May 2012 22:26:06 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id dkb5VMNaPyjJ; Wed, 30 May 2012 22:26:05 +0000 (UTC) Received: from orange-en1.sbone.de (orange-en1.sbone.de [IPv6:fde9:577b:c1a9:31:cabc:c8ff:fecf:e8e3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 81C29BE8145; Wed, 30 May 2012 22:26:05 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=utf-8 From: "Bjoern A. Zeeb" In-Reply-To: <4FC6062A.3010002@rdtc.ru> Date: Wed, 30 May 2012 22:26:04 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4FC5FA0B.5010609@cran.org.uk> <4FC6062A.3010002@rdtc.ru> To: Eugene Grosbein X-Mailer: Apple Mail (2.1084) Cc: Bruce Cran , freebsd-net@freebsd.org Subject: Re: if_em and large mtu bug? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 May 2012 22:26:09 -0000 On 30. May 2012, at 11:36 , Eugene Grosbein wrote: > 30.05.2012 17:44, Bruce Cran =D0=BF=D0=B8=D1=88=D0=B5=D1=82: >> Are there any known problems with if_em and jumbo packets? I've found=20= >> that my ssh connection breaks (when running dmesg etc.) if I = configure=20 >> the mtu to 9216 via rc.conf - but if I initially use a 1500 byte mtu = and=20 >> then manually configure it to 9216 then no problems occur. >>=20 That is due to another known bug; if you check netstat -rn and the mtu there on your routes you'll figure. if you remove your addresses/routes and re-add them after changing the = mtu, you'll see it as well. >> I'm running the latest -current: the hardware is: >>=20 >> em0: port 0xdc00-0xdc1f = mem=20 >> 0xfbde0000-0xfbdfffff,0xfbd00000-0xfbd7ffff,0xfbddc000-0xfbddffff irq = 16=20 >> at device 0.0 on pci1 >> em0: Using MSIX interrupts with 3 vectors >=20 > I confirm this with FreeBSD 8.2-STABLE/amd64, > em(4) 7.3.2, Intel PRO/1000 EB. >=20 > There is a workaround, though: ifconfig em0 -rxcsum -txcsum mtu 9000 > work from the beginning. >=20 > Eugene Grosbein >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" --=20 Bjoern A. Zeeb You have to have visions! It does not matter how good you are. It matters what good you do! From owner-freebsd-net@FreeBSD.ORG Wed May 30 22:31:31 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B780F106566B for ; Wed, 30 May 2012 22:31:31 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (bird.sbone.de [46.4.1.90]) by mx1.freebsd.org (Postfix) with ESMTP id 63B5A8FC0A for ; Wed, 30 May 2012 22:31:31 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 592D125D3A90; Wed, 30 May 2012 22:31:30 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 84344BE8145; Wed, 30 May 2012 22:31:29 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id ZFAfMmLFt0p8; Wed, 30 May 2012 22:31:28 +0000 (UTC) Received: from orange-en1.sbone.de (orange-en1.sbone.de [IPv6:fde9:577b:c1a9:31:cabc:c8ff:fecf:e8e3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id A6D14BE8144; Wed, 30 May 2012 22:31:28 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: "Bjoern A. Zeeb" In-Reply-To: Date: Wed, 30 May 2012 22:31:26 +0000 Content-Transfer-Encoding: 7bit Message-Id: <5A66084E-051B-4ECA-B39B-96DD3D97A073@lists.zabbadoz.net> References: <4FC6547A.2040308@my.gd> To: Vijay Singh X-Mailer: Apple Mail (2.1084) Cc: FreeBSD Networking Mailing List Subject: Re: Intel 1G Tx hangs X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 May 2012 22:31:31 -0000 On 30. May 2012, at 17:38 , Vijay Singh wrote: >>> Running 20ish guests in KVM with emulated e1000 cards and doing fine. >>> >> lucky you! I've been seeing complete RX failure with multiple emulated >> e1000 in qemu guest, cf kern/168246. It did not seem to have yet >> picked much interest from maintainer... >> >> - Arnaud >> >>> Got a few physical ones in production as well, no hangs here. > > FYI, I updated our 8.2 code base to 8-stable e1k drivers and things > appear to have stabilized. > > @jfv, the if_lem.c version did change from 1.0.1 to 1.0.4. for what it's worth I can freeze all my network traffic inside VMware when running lem on HEAD doing a few special things; I had not yet had the time to debug it and ifconfig down/up on console can clear it. I'll see if I find the time to look into it to conclude whether it is a problem of lem or vmware. /bz -- Bjoern A. Zeeb You have to have visions! It does not matter how good you are. It matters what good you do! From owner-freebsd-net@FreeBSD.ORG Wed May 30 22:36:58 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3BB961065670 for ; Wed, 30 May 2012 22:36:58 +0000 (UTC) (envelope-from bounces+73574-866e-freebsd-net=freebsd.org@sendgrid.me) Received: from o3.shared.sendgrid.net (o3.shared.sendgrid.net [208.117.48.85]) by mx1.freebsd.org (Postfix) with SMTP id DBE738FC14 for ; Wed, 30 May 2012 22:36:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.info; h= message-id:date:from:mime-version:to:cc:subject:references :in-reply-to:content-type:content-transfer-encoding; s=smtpapi; bh=Ki1s/HupIZswUwhPcC/s395l3OE=; b=VY59otgXrc5adc5VbjjxT+RQRX6B iVysOPtqPxrTqvbKOdlFKMStRv05tXLV53DnXAoYP3NPgNvNl6/EVj07TJghBgkc 871hGXs7870PkFcMyJdZ37jmHw0yXwC+j30cLAAR9xRG8NC58M3gkqpvKeCZ9EVx olTo83D+S4uaOeM= Received: by 10.4.35.224 with SMTP id mf3.10363.4FC6A107A Wed, 30 May 2012 17:36:55 -0500 (CDT) Received: from mail.tarsnap.com (unknown [10.8.49.124]) by mi2 (SG) with ESMTP id 4fc6a107.5003.61c724 for ; Wed, 30 May 2012 17:36:55 -0500 (CST) Received: (qmail 77098 invoked from network); 30 May 2012 22:29:23 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by mail.tarsnap.com with ESMTP; 30 May 2012 22:29:23 -0000 Received: (qmail 34495 invoked from network); 30 May 2012 22:35:46 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by clamshell.daemonology.net with SMTP; 30 May 2012 22:35:46 -0000 Message-ID: <4FC6A0C1.3010307@freebsd.org> Date: Wed, 30 May 2012 15:35:45 -0700 From: Colin Percival User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:12.0) Gecko/20120509 Thunderbird/12.0.1 MIME-Version: 1.0 To: Andrew Gallatin References: <4FC635CC.5030608@freebsd.org> <4FC63D27.70807@cs.duke.edu> In-Reply-To: <4FC63D27.70807@cs.duke.edu> X-Enigmail-Version: 1.5pre Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Sendgrid-EID: ISGKkmHlRE12gnWy0TjFyGQSFR1WnpjQdICm3qu6YxGdYKAvy/gc4T2jUQa000uADlkkFAtN0740Q3+Xcn+vpnC6RFIdeP7ZMIhNA/0EifUH+J2P8T/3Ra8e1qTlj4/E8LGiirHKpD8Hr4xJamr0JA== X-SendGrid-Contentd-ID: {"test_id":1338417415} Cc: freebsd-net@freebsd.org Subject: Re: [please review] TSO mbuf chain length limiting patch X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 May 2012 22:36:58 -0000 On 05/30/12 08:30, Andrew Gallatin wrote: > On 05/30/12 10:59, Colin Percival wrote: >> The Xen virtual network interface has an issue (ok, really the issue is with >> the linux back-end, but that's what most people are using) where it can't >> handle scatter-gather writes with lots of pieces, aka. long mbuf chains. >> This currently bites us hard with TSO enabled, since it produces said long >> mbuf chains. > > I've never been clear about what the max TSO size supported by FreeBSD > is. The NIC I maintain (mxge) is limited to 64K - epsilon for both > IPv4 *AND* IPv6. Up until now, this has been enforced by the 16-bit > ip length limit of IPv4 and we have not had IPv6 TSO until this week. > With IPv6, I'm worried that FreeBSD may now send packets down larger > than I could handle. In my case, however, the problem is not s/g list > length, but rather it is internal limits in the NIC which limit us to > 64K - epsilon for IPv6 as well. I think there may be other NICs in > the same boat for IPv6 (and maybe even some which cannot handle the > full 64K for IPv4). > > Your approach would not work well for my size limit. For > example, I'd have to set the limit to 4 mbufs to stay under 64KB. > This would be assuming the worst case of 16KB jumbo mbufs, so > that would limit me to ~8KB per TSO if 2KB mbufs were used. Right, the problem you describe isn't the one I was trying to solve. :-) > I think a better approach would be to have a limit on the size of the > pre-segmented TCP payload size sent to the driver. I tend to think > that this would be more generically useful, and it is a better match > for the NDIS APIs, where a driver must specify the max TSO size. I > think the changes to the TCP stack might be simpler (eg, they > would seem to jive better with the existing "maxmtu" approach). > > I think this could work for you as well. You could set the Xen max > tso size to be 32K (derived from 18 pages/skb, multiplied by a typical > 2KB mbuf size, with some slack built in). If the chain was too large, > you could m_defrag it down to size. Sounds good -- I don't want to m_defrag too often, but I imagine in most cases when TSO is being invoked most of the mbufs will have 2 kB each. This should also make the patch simpler by avoiding the need to modify uipc_mbuf.c; if we just limit the TSO payload size then the TCP stack can figure things out by itself. Are you working on a patch, or should I put one together? -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-net@FreeBSD.ORG Thu May 31 00:18:07 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 41E661065670; Thu, 31 May 2012 00:18:07 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.freebsd.org (Postfix) with ESMTP id 082C38FC1B; Thu, 31 May 2012 00:18:06 +0000 (UTC) Received: from [192.168.200.2] (c-24-125-204-77.hsd1.va.comcast.net [24.125.204.77]) (authenticated bits=0) by duke.cs.duke.edu (8.14.5/8.14.5) with ESMTP id q4V0I5X8015236 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 30 May 2012 20:18:06 -0400 (EDT) X-DKIM: Sendmail DKIM Filter v2.8.3 duke.cs.duke.edu q4V0I5X8015236 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=cs.duke.edu; s=mail; t=1338423486; bh=L0mRfPw+ai4cQPGkTQ7zeQXF+Sn0tDkjFsp8as3xt/w=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=cZH9LFHkn0/nc9kXgJjTXgCbABk6P8QUOb3uDXVspF0qW1ZBFEytkCjoy4oTSZphz 1pETyNXl7j4i7cyaNalRmKoZLrf2qRwVgMIZ+FnfTc+huYWyr/9widcHud49uKpzFq m1hoDydoX37CgRzJF/FgZbpZ4P1NtbrBsxQahIkep55RMtGPfplaHSyVa/Yj989dfN OzGFmjeavxRQsImHycdMawsaCDcLd44FWZuBgFJQw3DvUFK2i2VceX8otXuzxYduV7 rKXr+wjxAjb7QLd+RPMGMXtjyhK/lzRAyDiWjv8jj9+sVaf5XUyRMbHl0kYjeY1v/K wBQDB3scrGbHQ== Message-ID: <4FC6B8BD.3060108@cs.duke.edu> Date: Wed, 30 May 2012 20:18:05 -0400 From: Andrew Gallatin User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:11.0) Gecko/20120329 Thunderbird/11.0.1 MIME-Version: 1.0 To: Colin Percival References: <4FC635CC.5030608@freebsd.org> <4FC63D27.70807@cs.duke.edu> <4FC6A0C1.3010307@freebsd.org> In-Reply-To: <4FC6A0C1.3010307@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: [please review] TSO mbuf chain length limiting patch X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 00:18:07 -0000 On 05/30/12 18:35, Colin Percival wrote: > On 05/30/12 08:30, Andrew Gallatin wrote: >> On 05/30/12 10:59, Colin Percival wrote: >>> The Xen virtual network interface has an issue (ok, really the issue is with >>> the linux back-end, but that's what most people are using) where it can't >>> handle scatter-gather writes with lots of pieces, aka. long mbuf chains. >>> This currently bites us hard with TSO enabled, since it produces said long >>> mbuf chains. >> >> I've never been clear about what the max TSO size supported by FreeBSD >> is. The NIC I maintain (mxge) is limited to 64K - epsilon for both >> IPv4 *AND* IPv6. Up until now, this has been enforced by the 16-bit >> ip length limit of IPv4 and we have not had IPv6 TSO until this week. >> With IPv6, I'm worried that FreeBSD may now send packets down larger >> than I could handle. In my case, however, the problem is not s/g list >> length, but rather it is internal limits in the NIC which limit us to >> 64K - epsilon for IPv6 as well. I think there may be other NICs in >> the same boat for IPv6 (and maybe even some which cannot handle the >> full 64K for IPv4). >> >> Your approach would not work well for my size limit. For >> example, I'd have to set the limit to 4 mbufs to stay under 64KB. >> This would be assuming the worst case of 16KB jumbo mbufs, so >> that would limit me to ~8KB per TSO if 2KB mbufs were used. > > Right, the problem you describe isn't the one I was trying to solve. :-) > >> I think a better approach would be to have a limit on the size of the >> pre-segmented TCP payload size sent to the driver. I tend to think >> that this would be more generically useful, and it is a better match >> for the NDIS APIs, where a driver must specify the max TSO size. I >> think the changes to the TCP stack might be simpler (eg, they >> would seem to jive better with the existing "maxmtu" approach). >> >> I think this could work for you as well. You could set the Xen max >> tso size to be 32K (derived from 18 pages/skb, multiplied by a typical >> 2KB mbuf size, with some slack built in). If the chain was too large, >> you could m_defrag it down to size. > > Sounds good -- I don't want to m_defrag too often, but I imagine in most > cases when TSO is being invoked most of the mbufs will have 2 kB each. > This should also make the patch simpler by avoiding the need to modify > uipc_mbuf.c; if we just limit the TSO payload size then the TCP stack can > figure things out by itself. > > Are you working on a patch, or should I put one together? > No, I'd like to, but I'm afraid that I just don't have the time right now. I would very much appreciate it if you could put it together. I'd be happy to review it. Thanks, Drew From owner-freebsd-net@FreeBSD.ORG Thu May 31 00:41:58 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 05A8F106566B for ; Thu, 31 May 2012 00:41:58 +0000 (UTC) (envelope-from seanbru@yahoo-inc.com) Received: from mrout1-b.corp.bf1.yahoo.com (mrout1-b.corp.bf1.yahoo.com [98.139.253.104]) by mx1.freebsd.org (Postfix) with ESMTP id 88BAE8FC22 for ; Thu, 31 May 2012 00:41:57 +0000 (UTC) Received: from [IPv6:::1] (rideseveral.corp.yahoo.com [10.73.160.231]) by mrout1-b.corp.bf1.yahoo.com (8.14.4/8.14.4/y.out) with ESMTP id q4V0fniq082078; Wed, 30 May 2012 17:41:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yahoo-inc.com; s=cobra; t=1338424910; bh=77txSRXthzfzzbp97B8UTlhdGzUX9IdovxcG55L4xmA=; h=Subject:From:Reply-To:To:Content-Type:Date:Message-ID: Mime-Version:Content-Transfer-Encoding; b=XsjFZjDzFqQp+eaKdU3YLDSLNskZteMOMBP7VFtjSIAKAWKPqhsz346mhvto45IUA qrdljtl2znBsTTEPgmPBHmxwbByk6mMpjMXrr8fzzml26vbRmOdlN3PelhGQxlpvMP Ta3FnSUk1+a2gfjSQDbQ5X8xSKjeyXHyoJ6PQbt4= From: Sean Bruno To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset="UTF-8" Date: Wed, 30 May 2012 17:41:49 -0700 Message-ID: <1338424909.9051.12.camel@powernoodle-l7.corp.yahoo.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-Milter-Version: master.31+4-gbc07cd5+ X-CLX-ID: 424910000 Subject: bce(4) man page updates X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: sbruno@freebsd.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 00:41:58 -0000 Was trolling around inside of bce(4) and the Broadcom docs today and made the following update to the man page. Thoughts? Sean http://people.freebsd.org/~sbruno/bce_man.txt From owner-freebsd-net@FreeBSD.ORG Thu May 31 03:33:27 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 728BA106564A for ; Thu, 31 May 2012 03:33:27 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id EFF7C8FC0A for ; Thu, 31 May 2012 03:33:26 +0000 (UTC) Received: by werg1 with SMTP id g1so401587wer.13 for ; Wed, 30 May 2012 20:33:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=66k6swB/vEaCV4NOel1QksW3UTPtL6BBCO8j9x19Tww=; b=uzNs4GOyfHAR6efvoE5lXtnDDuezPxzNcl8La0yu7cxS6+KT5nqnAvC6KhPLghJE1s iYcU5P2XVpPwFXDVYIcUJkXsMkkxdxYqJVryUBjgnOxC9wA8A3L2ouJW59bDsc8plimz pzTSmFDO0LK8jZ0GrPLUTzgdSDk7+haCFAYzIqPlSExMivJa5C2vIkk9k+Nw/6bz5Xri pzfIUxpmyBC96bpCN/5wfEXCiCtcsw+mtGhDr52gJubQArrQU5b0KEYhPyVPe4uB/wLL 1y9CVmkol/9gWQDXYYyfxYojPY/e1hLH9pZYnqQNLXw9f0vrNB5G1Q/zBvWgDbSFwKxz 4UXQ== MIME-Version: 1.0 Received: by 10.216.206.164 with SMTP id l36mr12280106weo.154.1338435205740; Wed, 30 May 2012 20:33:25 -0700 (PDT) Received: by 10.223.155.4 with HTTP; Wed, 30 May 2012 20:33:25 -0700 (PDT) In-Reply-To: <4FBF88CE.20209@cs.duke.edu> References: <4FBF88CE.20209@cs.duke.edu> Date: Wed, 30 May 2012 20:33:25 -0700 Message-ID: From: Kevin Oberman To: Andrew Gallatin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org, Andrew Gallatin Subject: Re: Major performance hit with ToS setting X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 03:33:27 -0000 On Fri, May 25, 2012 at 6:27 AM, Andrew Gallatin wro= te: > On 05/24/12 18:55, Kevin Oberman wrote: > >> >> This is,of course, on a 10G interface. On 7.3 there is little > > > Hi Kevin, > > > What you're seeing looks almost like a checksum is bad, or > there is some other packet damage. =A0Do you see any > error counters increasing if you run netstat -s before > and after the test & compare the results? > > Thinking that, perhaps, this was a bug in my mxge(4), I attempted > to reproduce it this morning between =A08.3 and 9.0 boxes and > failed to see the bad behavior.. > > % nuttcp-6.1.2 -c32t -t diablo1-m < /dev/zero > =A09161.7500 MB / =A010.21 sec =3D 7526.5792 Mbps 53 %TX 97 %RX 0 host-re= trans > 0.11 msRTT > % nuttcp-6.1.2 =A0-t diablo1-m < /dev/zero > =A09140.6180 MB / =A010.21 sec =3D 7509.8270 Mbps 53 %TX 97 %RX 0 host-re= trans > 0.11 msRTT > > > However, I don't have any 8.2-r box handy, so I cannot > exactly repro your experiment... Drew and Bjorn, At this point the flying fickle finger of fate (oops, just dated myself) is pointing to a bug in the CUBIC congestion control, which we run. But its really weird in several ways. I built another system from the same source files and it works fine, unlike all of the existing systems. I need to confirm that all systems have identical hardware including the Myricom firmware. I suspect some edge case is biting only in unusual cases. I used SIFTR at the suggestion of Lawrence Stewart who headed the project to bring plugable congestion algorithms to FreeBSD and found really odd congestion behavior. First, I do see a triple ACK, but the congestion window suddenly drops from 73K to 8K. If I understand CUBIC, it should half the congestion window, not what is happening.. It then increases slowly (in slow start) to 82K. while the slow-start bytes are INCREASING, the congestion window again goes to 8K while the SS size moves from 36K up to 52K. It just continues to bound wildly between 8K (always the low point) and between 64k and 82K. The swings start at 83K and, over the first few seconds the peaks drop to about 64K. I am trying to think of any way that anything other then the CC algorithm could do this, but have not to this point. I will try installing Hamilton and see how it works. On the other hand, how could changing the ToS bits trigger this behavior? I have sent all of my data to Lawrence Stewart and I expect to here from him soon, but I'd appreciate it if you can provide any other idea on what could cause this. I might also mention that about 4 years ago when I was testing 10G cards I saw something similar (using tcptrace) when testing between a Myricom card and a Chelsio, but that is a pretty vague daqta point and I no longer have the traces to examine. Again, if you want to look at network performance issues, SIFTR is an awesome tool. --=20 R. Kevin Oberman, Network Engineer E-mail: kob6558@gmail.com From owner-freebsd-net@FreeBSD.ORG Thu May 31 04:01:16 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C6558106564A for ; Thu, 31 May 2012 04:01:16 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.freebsd.org (Postfix) with ESMTP id 7F33B8FC0A for ; Thu, 31 May 2012 04:01:16 +0000 (UTC) Received: from [192.168.200.2] (c-24-125-204-77.hsd1.va.comcast.net [24.125.204.77]) (authenticated bits=0) by duke.cs.duke.edu (8.14.5/8.14.5) with ESMTP id q4V41FZp021735 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 31 May 2012 00:01:15 -0400 (EDT) X-DKIM: Sendmail DKIM Filter v2.8.3 duke.cs.duke.edu q4V41FZp021735 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=cs.duke.edu; s=mail; t=1338436875; bh=B1OOx69Vr1TMS6rUWg+IgqpgSwJ07/xFuHorvou2ZMw=; h=Message-ID:Date:From:MIME-Version:To:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=a0fk4GVeONb7/yFx0Y8+hZVZotsrVoTPPJhz4pAcvuYq1ZR5/rSpDnGeeNbJ/eJo0 Qaz5hFTN5Rb34S7BMrCqXU6pbKPFq+E9PlIasaVGk7qYZTYTZvxWPLKVrJ/Z9Epgf1 +evzIj8krT6Jix5v5SXO5U9ZC97S2NShmiJGn1PH/Rz81DMpGvNHvpLwCdo/ZI+hed 0Csk8PLlVJAhdEhArlfC8cPpnaroc+F0AFsg/iC/3o8PloM3zOoq0jX7ASc40lt9/t xKHpMdNzLflfrBYGRgu+LAAka82UBzoZvVKYvxyJyYdas1R7W5lEE+dPfcZIK0qzkA gD3pQqtU20Ekg== Message-ID: <4FC6ED0B.5080000@cs.duke.edu> Date: Thu, 31 May 2012 00:01:15 -0400 From: Andrew Gallatin User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:11.0) Gecko/20120329 Thunderbird/11.0.1 MIME-Version: 1.0 To: freebsd-net@freebsd.org References: <20120528161240.GA38291@onelab2.iet.unipi.it> In-Reply-To: <20120528161240.GA38291@onelab2.iet.unipi.it> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: some questions on virtual machine bridging. X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 04:01:16 -0000 On 05/28/12 12:12, Luigi Rizzo wrote: > I am doing some experiments with implementing a software bridge > between virtual machines, using netmap as the communication API. > > I have a first prototype up and running and it is quite fast (10 Mpps > with 60-byte frames, 4 Mpps with 1500 byte frames, compared to the > ~500-800Kpps @60 bytes that you get with the tap interface used by > openvswitch or the native linux bridging). That is awesome! > - and of course, using PCI passthrough you get more or less hw speed > (constrained by the OS), but need support from an external switch > or the NIC itself to do forwarding between different ports. > anything else ? In terms of PCI passthrough / SR-IOV there are the emerging/competing EVB and VEPA standards to allow VM<->VM communication to go on the wire to a "real" switch, then back to the correct VM. > * any high-performance virtual switching solution around ? > As mentioned, i have measured native linux bridging and in-kernel ovs > and the numbers are above (not surprising; the tap involves a syscall > on each packet if i am not mistaken, and internally you need a > data copy) You should probably compare to ESXi. I've seen ~1Mpps going to or from from 1..N VMs and in or out a port on a 10GbE interface with ESX4 and newer. Drew From owner-freebsd-net@FreeBSD.ORG Thu May 31 12:46:21 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4E0311065677; Thu, 31 May 2012 12:46:21 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (wonkity.com [67.158.26.137]) by mx1.freebsd.org (Postfix) with ESMTP id 0AE0B8FC0C; Thu, 31 May 2012 12:46:20 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.5/8.14.5) with ESMTP id q4VCkKAQ082141; Thu, 31 May 2012 06:46:20 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.5/8.14.5/Submit) with ESMTP id q4VCkKcM082138; Thu, 31 May 2012 06:46:20 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Thu, 31 May 2012 06:46:20 -0600 (MDT) From: Warren Block To: sbruno@freebsd.org In-Reply-To: <1338424909.9051.12.camel@powernoodle-l7.corp.yahoo.com> Message-ID: References: <1338424909.9051.12.camel@powernoodle-l7.corp.yahoo.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (wonkity.com [127.0.0.1]); Thu, 31 May 2012 06:46:20 -0600 (MDT) Cc: "freebsd-net@freebsd.org" Subject: Re: bce(4) man page updates X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 12:46:21 -0000 On Wed, 30 May 2012, Sean Bruno wrote: > Was trolling around inside of bce(4) and the Broadcom docs today and > made the following update to the man page. Thoughts? > > Sean > > http://people.freebsd.org/~sbruno/bce_man.txt Nice! Just minor suggestions. The two "whether or not" sentences ought to be just "Enable/Disable ..." like the rest. A couple of "Cannot be assigned the value 0" sentences can be simpler as "Cannot be set to 0" if that doesn't change the meaning. "Valid values are in the range from 0-256." can be simpler as "Values from 0-256 are valid." Many "The default value is" which could be simplified to "The default is", or added to the end of a previous sentence as an aside: "Values from 0-100 are valid (default 18)." From owner-freebsd-net@FreeBSD.ORG Thu May 31 17:16:02 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2CE9C1065670; Thu, 31 May 2012 17:16:02 +0000 (UTC) (envelope-from seanbru@yahoo-inc.com) Received: from mrout1-b.corp.bf1.yahoo.com (mrout1-b.corp.bf1.yahoo.com [98.139.253.104]) by mx1.freebsd.org (Postfix) with ESMTP id CD9818FC0C; Thu, 31 May 2012 17:16:01 +0000 (UTC) Received: from [IPv6:::1] (rideseveral.corp.yahoo.com [10.73.160.231]) by mrout1-b.corp.bf1.yahoo.com (8.14.4/8.14.4/y.out) with ESMTP id q4VHFbLG006574; Thu, 31 May 2012 10:15:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yahoo-inc.com; s=cobra; t=1338484539; bh=bsD8JsT/BP/+fD2W7Ajn2j1DJxxFAv73mXmyfmD5MZc=; h=Subject:From:To:Cc:In-Reply-To:References:Content-Type:Date: Message-ID:Mime-Version:Content-Transfer-Encoding; b=o2X/Pv6QaZrX1LrUjDeqZXVfwuLQK+dsLeDSKX6jypi4x9z5V2ggtvPyagQYxtVM0 BrZblcj+tQ+sKTUn756qFO6DKL+6WOu7bYZwogkpjsXQvsM3kbV9nv0qvK206SLphn Wozqt+Ep16NW9NTiARwFaLHkEE+pxpu1mT87tNV8= From: Sean Bruno To: Warren Block In-Reply-To: References: <1338424909.9051.12.camel@powernoodle-l7.corp.yahoo.com> Content-Type: text/plain; charset="UTF-8" Date: Thu, 31 May 2012 10:15:37 -0700 Message-ID: <1338484537.2985.8.camel@powernoodle-l7.corp.yahoo.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-Milter-Version: master.31+4-gbc07cd5+ X-CLX-ID: 484538000 Cc: "freebsd-net@freebsd.org" Subject: Re: bce(4) man page updates X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 17:16:02 -0000 On Thu, 2012-05-31 at 05:46 -0700, Warren Block wrote: > On Wed, 30 May 2012, Sean Bruno wrote: > > > Was trolling around inside of bce(4) and the Broadcom docs today and > > made the following update to the man page. Thoughts? > > > > Sean > > > > http://people.freebsd.org/~sbruno/bce_man.txt > > Nice! > > Just minor suggestions. > > The two "whether or not" sentences ought to be just "Enable/Disable ..." > like the rest. > > A couple of "Cannot be assigned the value 0" sentences can be simpler as > "Cannot be set to 0" if that doesn't change the meaning. > > "Valid values are in the range from 0-256." can be simpler as > "Values from 0-256 are valid." > > Many "The default value is" which could be simplified to "The default > is", or added to the end of a previous sentence as an aside: > "Values from 0-100 are valid (default 18)." Indeed. Much more gooder wordingness. Updated! http://people.freebsd.org/~sbruno/bce_man.txt Sean From owner-freebsd-net@FreeBSD.ORG Thu May 31 17:53:46 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7B4C6106564A; Thu, 31 May 2012 17:53:46 +0000 (UTC) (envelope-from seanbru@yahoo-inc.com) Received: from mrout1-b.corp.bf1.yahoo.com (mrout1-b.corp.bf1.yahoo.com [98.139.253.104]) by mx1.freebsd.org (Postfix) with ESMTP id 24BB98FC0C; Thu, 31 May 2012 17:53:46 +0000 (UTC) Received: from [IPv6:::1] (rideseveral.corp.yahoo.com [10.73.160.231]) by mrout1-b.corp.bf1.yahoo.com (8.14.4/8.14.4/y.out) with ESMTP id q4VHrT36021735; Thu, 31 May 2012 10:53:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yahoo-inc.com; s=cobra; t=1338486810; bh=2f13zPp2/n3iaIwic8QPpb2UTwEijraii4bH5oNi2v8=; h=Subject:From:To:Cc:In-Reply-To:References:Content-Type:Date: Message-ID:Mime-Version:Content-Transfer-Encoding; b=gdlm2J3KW6g8QFvH5S10zkLj+QBdAJDVfWWOdR1+yTSjvfU3g03OYb+/Gl5RpszpB PYo0xu4g7LNvlMvaC6uxmg7xN1wByqEQWFnSnKzsbEqT4dkzHYlrnJ/uxeaNDI0sKt nLoUHaGiwrotCHOVXb7EXj4PcChVUmIY34F8otBI= From: Sean Bruno To: Warren Block In-Reply-To: <1338484537.2985.8.camel@powernoodle-l7.corp.yahoo.com> References: <1338424909.9051.12.camel@powernoodle-l7.corp.yahoo.com> <1338484537.2985.8.camel@powernoodle-l7.corp.yahoo.com> Content-Type: text/plain; charset="UTF-8" Date: Thu, 31 May 2012 10:53:28 -0700 Message-ID: <1338486809.2985.10.camel@powernoodle-l7.corp.yahoo.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-Milter-Version: master.31+4-gbc07cd5+ X-CLX-ID: 486809003 Cc: "freebsd-net@freebsd.org" Subject: Re: bce(4) man page updates X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 17:53:46 -0000 On Thu, 2012-05-31 at 10:15 -0700, Sean Bruno wrote: > On Thu, 2012-05-31 at 05:46 -0700, Warren Block wrote: > > On Wed, 30 May 2012, Sean Bruno wrote: > > > > > Was trolling around inside of bce(4) and the Broadcom docs today and > > > made the following update to the man page. Thoughts? > > > > > > Sean > > > > > > http://people.freebsd.org/~sbruno/bce_man.txt > > > > Nice! > > > > Just minor suggestions. > > > > The two "whether or not" sentences ought to be just "Enable/Disable ..." > > like the rest. > > > > A couple of "Cannot be assigned the value 0" sentences can be simpler as > > "Cannot be set to 0" if that doesn't change the meaning. > > > > "Valid values are in the range from 0-256." can be simpler as > > "Values from 0-256 are valid." > > > > Many "The default value is" which could be simplified to "The default > > is", or added to the end of a previous sentence as an aside: > > "Values from 0-100 are valid (default 18)." > > > Indeed. Much more gooder wordingness. Updated! > > http://people.freebsd.org/~sbruno/bce_man.txt > > Sean > Updated again, looks like one of the tuneables has changed in -head: hw.bce.loose_rx_mtu --> hw.bce.strict_rx_mtu Sean From owner-freebsd-net@FreeBSD.ORG Thu May 31 19:49:30 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2FE601065672; Thu, 31 May 2012 19:49:30 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (wonkity.com [67.158.26.137]) by mx1.freebsd.org (Postfix) with ESMTP id DADFE8FC12; Thu, 31 May 2012 19:49:29 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.5/8.14.5) with ESMTP id q4VJnSND084220; Thu, 31 May 2012 13:49:28 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.5/8.14.5/Submit) with ESMTP id q4VJnS6u084217; Thu, 31 May 2012 13:49:28 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Thu, 31 May 2012 13:49:28 -0600 (MDT) From: Warren Block To: Sean Bruno In-Reply-To: <1338486809.2985.10.camel@powernoodle-l7.corp.yahoo.com> Message-ID: References: <1338424909.9051.12.camel@powernoodle-l7.corp.yahoo.com> <1338484537.2985.8.camel@powernoodle-l7.corp.yahoo.com> <1338486809.2985.10.camel@powernoodle-l7.corp.yahoo.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (wonkity.com [127.0.0.1]); Thu, 31 May 2012 13:49:29 -0600 (MDT) Cc: "freebsd-net@freebsd.org" Subject: Re: bce(4) man page updates X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 19:49:30 -0000 On Thu, 31 May 2012, Sean Bruno wrote: >> >> Indeed. Much more gooder wordingness. Updated! >> >> http://people.freebsd.org/~sbruno/bce_man.txt >> > > Updated again, looks like one of the tuneables has changed in -head: > > hw.bce.loose_rx_mtu --> hw.bce.strict_rx_mtu One other idea that's a little shorter on multiple values: "this value can only be of the set 1, 2, 4 or 8 (default 2)." --> "this value can only be of the set 1, 2 (default), 4 or 8." These are subjective. For example, I prefer "goodinger wordiness." From owner-freebsd-net@FreeBSD.ORG Thu May 31 23:48:04 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1B48B106566B; Thu, 31 May 2012 23:48:04 +0000 (UTC) (envelope-from seanbru@yahoo-inc.com) Received: from mrout1-b.corp.bf1.yahoo.com (mrout1-b.corp.bf1.yahoo.com [98.139.253.104]) by mx1.freebsd.org (Postfix) with ESMTP id C3B588FC08; Thu, 31 May 2012 23:48:03 +0000 (UTC) Received: from [IPv6:::1] (rideseveral.corp.yahoo.com [10.73.160.231]) by mrout1-b.corp.bf1.yahoo.com (8.14.4/8.14.4/y.out) with ESMTP id q4VNlpcT065096; Thu, 31 May 2012 16:47:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yahoo-inc.com; s=cobra; t=1338508073; bh=Ron0aMOX8ycI6A54Qi9qc4gLoULBcH7dI34Ifn4oVsw=; h=Subject:From:To:Cc:In-Reply-To:References:Content-Type:Date: Message-ID:Mime-Version:Content-Transfer-Encoding; b=V+yeq9Sj1xFNqFS4fIlgLmO+uVF7BL894lOsR5Z5gwT9sWPDAfHy9b1Az3RYB9X0y XGrceoF1GmA8nAvbUjRlUVe6liL6bkN6EKlNCujlkxjUHkS5jlNgJGJnaKabtfV1gv NWxuAyF8Zs2OHvwq/Ul6+S7t6a5JLinmvh312VrE= From: Sean Bruno To: Warren Block In-Reply-To: References: <1338424909.9051.12.camel@powernoodle-l7.corp.yahoo.com> <1338484537.2985.8.camel@powernoodle-l7.corp.yahoo.com> <1338486809.2985.10.camel@powernoodle-l7.corp.yahoo.com> Content-Type: text/plain; charset="UTF-8" Date: Thu, 31 May 2012 16:47:51 -0700 Message-ID: <1338508071.2985.22.camel@powernoodle-l7.corp.yahoo.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-Milter-Version: master.31+4-gbc07cd5+ X-CLX-ID: 508072001 Cc: "freebsd-net@freebsd.org" Subject: Re: bce(4) man page updates X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 23:48:04 -0000 On Thu, 2012-05-31 at 12:49 -0700, Warren Block wrote: > One other idea that's a little shorter on multiple values: > > "this value can only be of the set 1, 2, 4 or 8 (default 2)." > > --> "this value can only be of the set 1, 2 (default), 4 or 8." Hrm, I don't really like the look of that in this context because all of the other default values are at the end of the description. If there isn't a strong objection, I think I'll leave it as is and commit it. http://people.freebsd.org/~sbruno/bce_man.txt sean From owner-freebsd-net@FreeBSD.ORG Fri Jun 1 02:16:12 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CFB6D1065670 for ; Fri, 1 Jun 2012 02:16:12 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id 530D38FC1E for ; Fri, 1 Jun 2012 02:16:12 +0000 (UTC) Received: from lstewart.caia.swin.edu.au (lstewart.caia.swin.edu.au [136.186.229.95]) by lauren.room52.net (Postfix) with ESMTPSA id 56B017E84A; Fri, 1 Jun 2012 12:16:10 +1000 (EST) Message-ID: <4FC825E9.20401@freebsd.org> Date: Fri, 01 Jun 2012 12:16:09 +1000 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:10.0.2) Gecko/20120311 Thunderbird/10.0.2 MIME-Version: 1.0 To: Kevin Oberman References: <4FBF88CE.20209@cs.duke.edu> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lauren.room52.net Cc: freebsd-net@freebsd.org, Andrew Gallatin , Andrew Gallatin Subject: Re: Major performance hit with ToS setting X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jun 2012 02:16:12 -0000 On 05/31/12 13:33, Kevin Oberman wrote: > On Fri, May 25, 2012 at 6:27 AM, Andrew Gallatin wrote: >> On 05/24/12 18:55, Kevin Oberman wrote: >> >>> >>> This is,of course, on a 10G interface. On 7.3 there is little >> >> >> Hi Kevin, >> >> >> What you're seeing looks almost like a checksum is bad, or >> there is some other packet damage. Do you see any >> error counters increasing if you run netstat -s before >> and after the test& compare the results? >> >> Thinking that, perhaps, this was a bug in my mxge(4), I attempted >> to reproduce it this morning between 8.3 and 9.0 boxes and >> failed to see the bad behavior.. >> >> % nuttcp-6.1.2 -c32t -t diablo1-m< /dev/zero >> 9161.7500 MB / 10.21 sec = 7526.5792 Mbps 53 %TX 97 %RX 0 host-retrans >> 0.11 msRTT >> % nuttcp-6.1.2 -t diablo1-m< /dev/zero >> 9140.6180 MB / 10.21 sec = 7509.8270 Mbps 53 %TX 97 %RX 0 host-retrans >> 0.11 msRTT >> >> >> However, I don't have any 8.2-r box handy, so I cannot >> exactly repro your experiment... > > Drew and Bjorn, > > At this point the flying fickle finger of fate (oops, just dated > myself) is pointing to a bug in the CUBIC congestion control, which we > run. But its really weird in several ways. I can't find any evidence (yet) of cubic being responsible for the problems you're seeing. > I built another system from the same source files and it works fine, > unlike all of the existing systems. I need to confirm that all systems > have identical hardware including the Myricom firmware. I suspect some > edge case is biting only in unusual cases. > > I used SIFTR at the suggestion of Lawrence Stewart who headed the > project to bring plugable congestion algorithms to FreeBSD and found > really odd congestion behavior. First, I do see a triple ACK, but the > congestion window suddenly drops from 73K to 8K. If I understand > CUBIC, it should half the congestion window, not what is happening.. > It then increases slowly (in slow start) to 82K. while the slow-start > bytes are INCREASING, the congestion window again goes to 8K while the > SS size moves from 36K up to 52K. It just continues to bound wildly > between 8K (always the low point) and between 64k and 82K. The swings > start at 83K and, over the first few seconds the peaks drop to about > 64K. > > I am trying to think of any way that anything other then the CC > algorithm could do this, but have not to this point. I will try > installing Hamilton and see how it works. On the other hand, how could > changing the ToS bits trigger this behavior? As mentioned to you privately but I'll repeat here for the record, HD and CHD are not good algorithms to compare cubic's behaviour against, because they're delay based and will reduce cwnd in response to observing changes in RTT i.e. their cwnd plots will look jumpy even though no losses are occurring. Comparing cubic against newreno or htcp would be more appropriate. > I have sent all of my data to Lawrence Stewart and I expect to here > from him soon, but I'd appreciate it if you can provide any other idea > on what could cause this. Ok, so here are some thoughts from looking at the siftr data you forwarded me privately: - Column 11 shows the snd_wnd for the flow, which mirrors the rcv_wnd advertised by the receiver. It starts at 65k (default for net.inet.tcp.recvspace) and fairly early on jumps to above 1Mb. This implies socket buffer autotuning on the receiver is working correctly and that your connection is not rcv_wnd limited - good on this front. - Column 16 shows the MSS. It's 8k, so you're using jumbo frames. Jumbo frames are notorious for trigger interesting bugs at lower layers in the stack. Are you keeping track of your jumbo cluster usage via "netstat -m"? Can you trigger the bug if you switch to 1500MTU? - Column 17 shows the smoothed RTT estimate for the flow, in weird units. You chopped off the siftr enable log line which reports the data necessary to correctly decode the SRTT value, but assuming you've left HZ at 1000 and TCP_RTT_SCALE is the default of 32, the kernel thinks the receiver is ~56ms away, and the delay fluctuates up to ~70ms, most likely attributable to build up of queuing delays along the path. - Column 19 shows the TCP control block flags for the connection. If you AND the number in this field with flags from , you can figure out if a particular flag is set. Also easy to paste the number into a calculator and switch to hex or binary mode to figure out which bits are set. 'cat bwctl.siftr | cut -f19 -d "," | less' will let you easily see the range of flag combinations. Most of the interesting flags are in the higher order bits. The value 16778208 early on indicates TF_TSO is set. The value 554697696 which appears periodically in the trace indicates TF_CONGRECOVERY|TF_FASTRECOVERY i.e. 3 dupacks triggered a fast recovery episode. 1092625377 indicates TF_WASCRECOVERY|TF_WASFRECOVERY i.e. an RTO happened. Value 1630544864 is interesting, as it indicates TF_WASCRECOVERY|TF_CONGRECOVERY|TF_WASFRECOVERY|TF_FASTRECOVERY, which I think is a minor bug (the TF_WAS flags should have been cleared), though shouldn't affect your connection. - Columns 21 and 22 show the size and occupancy of the socket send buffer respectively. The size grows showing autotuning is kicking in, and the occupancy is kept high which means the app is ensuring the send pipeline doesn't stall - good on this front. Without pcap trace data it's hard to obtain further insight into what exactly is going on, but it would appear you are suffering from frequent packet loss, quite possibly internal to the stack on the sender. More digging required, and I would suggest you first look at trying to reproduce with a 1500 byte MTU. Cheers, Lawrence From owner-freebsd-net@FreeBSD.ORG Fri Jun 1 02:48:14 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9C230106564A for ; Fri, 1 Jun 2012 02:48:14 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id 58AE88FC12 for ; Fri, 1 Jun 2012 02:48:14 +0000 (UTC) Received: from lstewart.caia.swin.edu.au (lstewart.caia.swin.edu.au [136.186.229.95]) by lauren.room52.net (Postfix) with ESMTPSA id 095C77E891; Fri, 1 Jun 2012 12:48:13 +1000 (EST) Message-ID: <4FC82D6C.4050309@freebsd.org> Date: Fri, 01 Jun 2012 12:48:12 +1000 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:10.0.2) Gecko/20120311 Thunderbird/10.0.2 MIME-Version: 1.0 To: Kevin Oberman References: <4FBF88CE.20209@cs.duke.edu> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lauren.room52.net Cc: freebsd-net@freebsd.org, Andrew Gallatin , Andrew Gallatin Subject: Re: Major performance hit with ToS setting X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jun 2012 02:48:14 -0000 On 05/31/12 13:33, Kevin Oberman wrote: [snip] > I used SIFTR at the suggestion of Lawrence Stewart who headed the > project to bring plugable congestion algorithms to FreeBSD and found > really odd congestion behavior. First, I do see a triple ACK, but the > congestion window suddenly drops from 73K to 8K. If I understand > CUBIC, it should half the congestion window, not what is happening.. > It then increases slowly (in slow start) to 82K. while the slow-start > bytes are INCREASING, the congestion window again goes to 8K while the > SS size moves from 36K up to 52K. It just continues to bound wildly > between 8K (always the low point) and between 64k and 82K. The swings > start at 83K and, over the first few seconds the peaks drop to about > 64K. Oh, and a comment about this behaviour. Dropping back to 8k (1MSS) is only nasty if the TF_{CONG|FAST}RECOVERY flags are *not* set i.e. if you see cwnd grow, drop to 8k with those flags set, and then when the flags are unset, cwnd starts at the value of ssthresh, then that is perfectly normal recovery behaviour. What *is* nasty is if an RTO fires, which will reset cwnd to 8k, ssthresh to 2*MSS and make the connection effectively start from scratch again. There is evidence of RTOs in your siftr output, which is bad news e.g here's one example of 2 side-by-side log lines from your trace: # Direction,time,ssthresh,cwnd,flags i,1338319593.574706,27044,27044,1630544864 o,1338319593.831482,16384,8192,1092625377 Note the 300ms gap, and how cwnd resets to 1MSS and flags go from 1630544864 (TF_WASCRECOVERY|TF_CONGRECOVERY|TF_WASFRECOVERY|TF_FASTRECOVERY) to 1092625377 (TF_WASCRECOVERY|TF_WASFRECOVERY). Cheers, Lawrence From owner-freebsd-net@FreeBSD.ORG Fri Jun 1 07:12:49 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 59CAC10656F7 for ; Fri, 1 Jun 2012 07:12:49 +0000 (UTC) (envelope-from jmg@h2.funkthat.com) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) by mx1.freebsd.org (Postfix) with ESMTP id 32D1E8FC0C for ; Fri, 1 Jun 2012 07:12:49 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id q516qLri053013 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 31 May 2012 23:52:21 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id q516qJtl053012; Thu, 31 May 2012 23:52:19 -0700 (PDT) (envelope-from jmg) Date: Thu, 31 May 2012 23:52:19 -0700 From: John-Mark Gurney To: "Bjoern A. Zeeb" Message-ID: <20120601065219.GF91292@funkthat.com> Mail-Followup-To: "Bjoern A. Zeeb" , freebsd-net@freebsd.org References: <4FC5FA0B.5010609@cran.org.uk> <4FC6062A.3010002@rdtc.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Thu, 31 May 2012 23:52:21 -0700 (PDT) Cc: freebsd-net@freebsd.org Subject: Re: if_em and large mtu bug? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jun 2012 07:12:49 -0000 Bjoern A. Zeeb wrote this message on Wed, May 30, 2012 at 22:26 +0000: > On 30. May 2012, at 11:36 , Eugene Grosbein wrote: > > > 30.05.2012 17:44, Bruce Cran ??????????: > >> Are there any known problems with if_em and jumbo packets? I've found > >> that my ssh connection breaks (when running dmesg etc.) if I configure > >> the mtu to 9216 via rc.conf - but if I initially use a 1500 byte mtu and > >> then manually configure it to 9216 then no problems occur. > >> > > That is due to another known bug; > if you check netstat -rn and the mtu there on your routes you'll figure. > if you remove your addresses/routes and re-add them after changing the mtu, > you'll see it as well. Just to comment, this isn't a bug.. it's that FreeBSD allows you to control the mtu to a host via the routing table... Previously it would always max out the routes MTU meaning that you could never reduce the MTU to a problematic host, while keeping the mtu larger for others i.e. have a mixed network w/ jumbo and normal frames... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Fri Jun 1 07:46:02 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D9F98106564A; Fri, 1 Jun 2012 07:46:02 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9A3BC8FC12; Fri, 1 Jun 2012 07:46:02 +0000 (UTC) Received: by pbbro2 with SMTP id ro2so2783188pbb.13 for ; Fri, 01 Jun 2012 00:46:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=fUrxBai8IanPq9d7CZuGpDA9MEUWiH3PmS6IE84TuMs=; b=Rr69UdW822DLCZccTQiel7YzAL231FB6eE4OPfQQML3U97zuN8Tc9TVgtXjUdtTHPE I8K2ijkltlfWmYf+YVj5++Zr00Ni3ozjEu59wSLMbhdCFKeBhJrjyzDxHkqsfcO5jYFC p3d5i+HdAhMx9IGgUlBeLBPQivA48uK/b4xbXMsLcnpSpYUj2IYVPD23pi4AfZvtMSJb JqfiXB3npI/Ak6hZBdmOoatIi5zvpE0po1A2X0KnkzCwRtdEaMOhJ+zFDzGbVGcgGJkZ WWcnL3jAG/mf2F/izgAVc+yhIFoGtgQUjpwr3DvXOOElMKd3tMHzy+jjJtxdh+CGeQ3C l6dQ== MIME-Version: 1.0 Received: by 10.68.135.201 with SMTP id pu9mr7516597pbb.146.1338536762123; Fri, 01 Jun 2012 00:46:02 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.142.203.2 with HTTP; Fri, 1 Jun 2012 00:46:02 -0700 (PDT) Date: Fri, 1 Jun 2012 00:46:02 -0700 X-Google-Sender-Auth: Fch6zSN3RyKHCEY0JIZEdj8yVJE Message-ID: From: Adrian Chadd To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-wireless@freebsd.org Subject: if_start / if_transmit handling and packet ordering - how can I guarantee it? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jun 2012 07:46:03 -0000 Hi all, I've been pushing my ath 11n driver hard (250Mbit UDP) and I've found a rather interesting behaviour, at least on my SMP machine. I'm running single direction UDP iperf tests. This is all on stable/9, with -head ath/net80211 stacks. The receiver logs about 40/20000 frames a second as being "out of order". The net80211 stack and ath software TX aggregation path isn't reordering frames. So I reviewed the code again and I wondered - is what I'm doing with ath_start() and the TX side locking allowing for multiple ath_start() invocations to occur, where they interleave processing frames? The short answer is yes. Then, something else hit me - I wondered whether ath_start() itself is being preempted by the ath taskqueue (say, doing RX, or TX completion) - where ath_start() is then called. Ie; * iperf -> sendto() -> socket layer -> net80211 ieee80211_start() -> if_transmit -> if_start() -> ath_start() And then an interrupt coming in, during ath_start() having dequeued a frame, but before it had placed it on any hardware/software queue: * ath0 taskqueue -> ath_tx_proc() (tx completion) -> ath_start() Or, similarly, CPU #0 running iperf, CPU #1 running the ath taskqueue, and one or the other being preempted by something higher priority (eg an interrupt coming in) between having removed a frame from the ifnet queue and it being thrown into the software/hardware queue. The ath driver locking only holds the TX locks as long as is needed to do individual TX queue operations (ie, queue a frame to the software / hardware queue) rather than holding it for the entirety of the TX path. Because of this, I wonder if preemption is possibly causing issues. So I then went grovelling through ixgbe to see what it does with if_transmit. It holds the TX lock for that particular NIC hardware queue for the entirety of: * a TX to hardware operation (which can be an individual frame queue, or servicing the whole bufring contents); * completing the frames via _txeof(), until nothing more needs to be done. So by holding the TX lock for the entirety of the queue operation, it means that any other entries into that particular TX path for that NIC queue will stall, waiting for the lock, and thus effectively be serialised, avoiding my initial issue. Any TX completion which leads to further transmissions from the bufring (and simultaneous incoming TX frames) will block each other. Ok, so now that I've mostly tried to lucidly dump what's going on- what do people think about holding the locks for (potentially) so long? I know iwn(4) holds the driver lock for as long as it can for _everything_, so it avoids this issue. But again, I don't really like the idea of holding a lock for this long. Does anyone else have any other ideas? FWIW - I temporarily converted the ath driver to make ath_start() enqueue a taskqueue task, which then did all of the TX inside the taskqueue. This serialised with TX completion and RX, which can also start TX; and all of my out of order issues went away. I unfortunately then become very, very susceptible to scheduling latency (hence my initial post about KTR, SMP, preemption and what looks to be like Cx/idle/powerd issues. If I run my laptop (Lenovo T60) with all the power/Cx stuff disabled, I can quite happily sustain 240-250MBit of UDP without any reordering or packet latency. Thanks in advance (and phew!) Adrian From owner-freebsd-net@FreeBSD.ORG Fri Jun 1 16:19:49 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B66F0106566C for ; Fri, 1 Jun 2012 16:19:49 +0000 (UTC) (envelope-from jeffrey.e.pieper@intel.com) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by mx1.freebsd.org (Postfix) with ESMTP id 874378FC17 for ; Fri, 1 Jun 2012 16:19:49 +0000 (UTC) Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga101.jf.intel.com with ESMTP; 01 Jun 2012 09:19:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.67,352,1309762800"; d="scan'208";a="147077103" Received: from orsmsx606.amr.corp.intel.com ([10.22.226.128]) by orsmga001.jf.intel.com with ESMTP; 01 Jun 2012 09:19:49 -0700 Received: from orsmsx105.amr.corp.intel.com (10.22.225.132) by orsmsx606.amr.corp.intel.com (10.22.226.128) with Microsoft SMTP Server (TLS) id 8.2.255.0; Fri, 1 Jun 2012 09:19:48 -0700 Received: from orsmsx101.amr.corp.intel.com ([169.254.8.121]) by ORSMSX105.amr.corp.intel.com ([169.254.4.232]) with mapi id 14.01.0355.002; Fri, 1 Jun 2012 09:19:48 -0700 From: "Pieper, Jeffrey E" To: John-Mark Gurney , "Bjoern A. Zeeb" Thread-Topic: if_em and large mtu bug? Thread-Index: AQHNPlFCv5wZkRdS2EOB3+G/DquDa5biqYcAgAC1lQCAAh/GgIAAKA2g Date: Fri, 1 Jun 2012 16:19:47 +0000 Message-ID: <2A35EA60C3C77D438915767F458D6568481AA470@ORSMSX101.amr.corp.intel.com> References: <4FC5FA0B.5010609@cran.org.uk> <4FC6062A.3010002@rdtc.ru> <20120601065219.GF91292@funkthat.com> In-Reply-To: <20120601065219.GF91292@funkthat.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.22.254.139] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-net@freebsd.org" Subject: RE: if_em and large mtu bug? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jun 2012 16:19:49 -0000 -----Original Message----- From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd-net@freebsd.org] = On Behalf Of John-Mark Gurney Sent: Thursday, May 31, 2012 11:52 PM To: Bjoern A. Zeeb Cc: freebsd-net@freebsd.org Subject: Re: if_em and large mtu bug? Bjoern A. Zeeb wrote this message on Wed, May 30, 2012 at 22:26 +0000: > On 30. May 2012, at 11:36 , Eugene Grosbein wrote: >=20 > > 30.05.2012 17:44, Bruce Cran ??????????: > >> Are there any known problems with if_em and jumbo packets? I've found= =20 > >> that my ssh connection breaks (when running dmesg etc.) if I configure= =20 > >> the mtu to 9216 via rc.conf - but if I initially use a 1500 byte mtu a= nd=20 > >> then manually configure it to 9216 then no problems occur. > >>=20 >=20 > That is due to another known bug; > if you check netstat -rn and the mtu there on your routes you'll figure. > if you remove your addresses/routes and re-add them after changing the mt= u, > you'll see it as well. Just to comment, this isn't a bug.. it's that FreeBSD allows you to control the mtu to a host via the routing table... Previously it would always max out the routes MTU meaning that you could never reduce the MTU to a problematic host, while keeping the mtu larger for others i.e. have a mixed network w/ jumbo and normal frames... --=20 John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." This is what I use (I'm JFV's validation engineer): Ifconfig emX mtu =20 Doing this adds it to the routing table and seems to avoid any issues.=20 I hope this helps, Jeff _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Fri Jun 1 23:04:36 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 43774106564A for ; Fri, 1 Jun 2012 23:04:36 +0000 (UTC) (envelope-from hbcheng@berkeley.edu) Received: from cm03fe.IST.Berkeley.EDU (cm03fe.IST.Berkeley.EDU [169.229.218.144]) by mx1.freebsd.org (Postfix) with ESMTP id 2D5AC8FC0A for ; Fri, 1 Jun 2012 23:04:36 +0000 (UTC) Received: from 99-28-68-143.lightspeed.sndgca.sbcglobal.net ([99.28.68.143] helo=[192.168.1.85]) by cm03fe.ist.berkeley.edu with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.76) (auth plain:hbcheng@berkeley.edu) (envelope-from ) id 1SaatQ-0000iH-Ct for freebsd-net@freebsd.org; Fri, 01 Jun 2012 16:04:30 -0700 Message-ID: <1338591864.3484.12.camel@Zanzibar-II.gateway.2wire.net> From: Hao Bryan Cheng To: freebsd-net@freebsd.org Date: Fri, 01 Jun 2012 16:04:24 -0700 Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.3 (3.2.3-1.fc16) Content-Transfer-Encoding: 7bit Mime-Version: 1.0 Subject: NAT with Port-block Allocation in FreeBSD? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jun 2012 23:04:36 -0000 Hello, I apologize in advance if this is the wrong place for this posting. I am a developer on the circe captive portal system (net-mgmt/circe). Our system currently uses either netgraph or FreeBSD's in-kernel NAT (configurable) as a one-to-one NAT facility to provide access control for wireless clients. IP address pressure has pushed us towards implementing many-to-one NAT. However, the primary deployment of our software here at UC Berkeley requires us to be able to track bandwidth usage, security notices, and copyright takedown requests on a per-client basis. Traditional many-to-one NAT generates an unreasonable amount of logging data for our clients, which we expect to number in the low thousands. To mitigate the logging/accounting burden, we're investigating port block allocation, described in http://tools.ietf.org/html/draft-tsou-behave-natx4-log-reduction-02. By allocating a block of ports for each client, we can drastically reduce the amount of logging that we have to do to be able to uniquely trace a copyright infringement notice back to the individual user. Preliminary investigation of both IPFW's NAT facility and netgraph's ng_nat node did not uncover any trivial method of performing port-block allocation in many-to-one NAT. Has anybody here had any experience implementing a many-to-one NAT box with FreeBSD that made use of port-block allocation? Alternatively, is there any documentation or resources that somebody could point me towards to get started? Thanks in advance for your help. -- Hao "Bryan" Cheng Lead Unix Systems Administrator for Network Access Control Student Affairs- IT UC Berkeley From owner-freebsd-net@FreeBSD.ORG Sat Jun 2 13:21:29 2012 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 195C9106564A; Sat, 2 Jun 2012 13:21:29 +0000 (UTC) (envelope-from maxim@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id E056A8FC08; Sat, 2 Jun 2012 13:21:28 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q52DLS0p028841; Sat, 2 Jun 2012 13:21:28 GMT (envelope-from maxim@freefall.freebsd.org) Received: (from maxim@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q52DLRLi028660; Sat, 2 Jun 2012 13:21:27 GMT (envelope-from maxim) Date: Sat, 2 Jun 2012 13:21:27 GMT Message-Id: <201206021321.q52DLRLi028660@freefall.freebsd.org> To: sb@waeme.net, maxim@FreeBSD.org, freebsd-net@FreeBSD.org From: maxim@FreeBSD.org Cc: Subject: Re: kern/168294: [ixgbe] [patch] ixgbe driver compiled in kernel has no Flow Director support although loaded as a module has one X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jun 2012 13:21:29 -0000 Synopsis: [ixgbe] [patch] ixgbe driver compiled in kernel has no Flow Director support although loaded as a module has one State-Changed-From-To: open->patched State-Changed-By: maxim State-Changed-When: Sat Jun 2 13:19:59 UTC 2012 State-Changed-Why: jfv@ has committed the patch, see r235920. http://www.freebsd.org/cgi/query-pr.cgi?pr=168294 From owner-freebsd-net@FreeBSD.ORG Sat Jun 2 15:51:34 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A967C106564A for ; Sat, 2 Jun 2012 15:51:34 +0000 (UTC) (envelope-from darrenr@freebsd.org) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by mx1.freebsd.org (Postfix) with ESMTP id 72C0F8FC0C for ; Sat, 2 Jun 2012 15:51:34 +0000 (UTC) Received: from compute5.internal (compute5.nyi.mail.srv.osa [10.202.2.45]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id C910D2084A for ; Sat, 2 Jun 2012 11:51:33 -0400 (EDT) Received: from frontend1.nyi.mail.srv.osa ([10.202.2.160]) by compute5.internal (MEProxy); Sat, 02 Jun 2012 11:51:33 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:date:from:reply-to :mime-version:to:subject:content-type:content-transfer-encoding; s=smtpout; bh=1R5xz4Jiytz1woxiZNfTQWBX3CU=; b=ToZMl4dP6yNTo5TbX bIEkPU9+4iActnpgwp4+hK0AwmPZ0COwnlw0Wu6U9IKz2r+7t5F6LS61utdHUw5D jPyyd5qXE1yCnuvIN/ODLTPGz3htQQYD7HNDfoReFfi2iTZ99zNWkv7TtFN72Qgd bujwMzakLJKQ6RUJQDTuPPdrm4= X-Sasl-enc: WtNxThVyTwk0wtxtc2xuZSiTK11rZkywN5hB8UFNsvLC 1338652293 Received: from [192.168.1.124] (unknown [202.45.110.141]) by mail.messagingengine.com (Postfix) with ESMTPA id 0BD0B8E01F5 for ; Sat, 2 Jun 2012 11:51:32 -0400 (EDT) Message-ID: <4FCA36F6.8040308@freebsd.org> Date: Sun, 03 Jun 2012 01:53:26 +1000 From: Darren Reed Organization: FreeBSD User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: bin/102701 - ifconfig xx0 inet6 delete always fails X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: darrenr@freebsd.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jun 2012 15:51:34 -0000 Is there any reason that this patch hasn't been applied to -current? I've just run into this and I can't believe that it still exists, given that it falls into the "low hanging fruit" category. I'll note that if it wasn't for subversion, I'd be doing a commit rather than an email. Darren From owner-freebsd-net@FreeBSD.ORG Sat Jun 2 17:40:16 2012 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 48F8B1065670 for ; Sat, 2 Jun 2012 17:40:16 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 1B1CF8FC19 for ; Sat, 2 Jun 2012 17:40:16 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q52HeFVG016064 for ; Sat, 2 Jun 2012 17:40:15 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q52HeFRT016063; Sat, 2 Jun 2012 17:40:15 GMT (envelope-from gnats) Date: Sat, 2 Jun 2012 17:40:15 GMT Message-Id: <201206021740.q52HeFRT016063@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: Marc Albers Cc: Subject: Re: kern/167768: [ipfilter] Fatal trap in ipfilter/ipnat X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Marc Albers List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jun 2012 17:40:16 -0000 The following reply was made to PR kern/167768; it has been noted by GNATS. From: Marc Albers To: bug-followup@FreeBSD.org, bsdbug@bospaling.nl Cc: Subject: Re: kern/167768: [ipfilter] Fatal trap in ipfilter/ipnat Date: Sat, 2 Jun 2012 19:30:48 +0200 switching the external (re0) and internal (em0) interfaces does not = help. Still the same kernel panic.= From owner-freebsd-net@FreeBSD.ORG Sat Jun 2 19:11:43 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2A5BA106566B; Sat, 2 Jun 2012 19:11:43 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) by mx1.freebsd.org (Postfix) with ESMTP id A9E4C8FC1B; Sat, 2 Jun 2012 19:11:42 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 7A07F25D3A9D; Sat, 2 Jun 2012 19:11:41 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 9B6C6BE8427; Sat, 2 Jun 2012 19:11:40 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id VdMZKNuh1BTq; Sat, 2 Jun 2012 19:11:39 +0000 (UTC) Received: from orange-en1.sbone.de (orange-en1.sbone.de [IPv6:fde9:577b:c1a9:31:cabc:c8ff:fecf:e8e3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 23C40BE840E; Sat, 2 Jun 2012 19:11:38 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: "Bjoern A. Zeeb" In-Reply-To: <4FCA36F6.8040308@freebsd.org> Date: Sat, 2 Jun 2012 19:11:37 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <57EDF758-FF3F-4BA5-B44E-398846B92925@lists.zabbadoz.net> References: <4FCA36F6.8040308@freebsd.org> To: darrenr@freebsd.org X-Mailer: Apple Mail (2.1084) Cc: freebsd-net@freebsd.org Subject: Re: bin/102701 - ifconfig xx0 inet6 delete always fails X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jun 2012 19:11:43 -0000 On 2. Jun 2012, at 15:53 , Darren Reed wrote: > Is there any reason that this patch hasn't been applied > to -current? I've just run into this and I can't believe > that it still exists, given that it falls into the "low > hanging fruit" category. I'll note that if it wasn't for > subversion, I'd be doing a commit rather than an email. ifconfig [-L] [-k] [-m] [-n] interface [create] address_family = [address [dest_address]] [parameters] ... The following parameters may be set with ifconfig: ... -alias Remove the network address specified. This would be used = if you incorrectly specified an alias, or it was no longer needed. = If you have incorrectly set an NS address having the side = effect of specifying the host portion, removing all NS addresses will = allow you to respecify the host portion. ... delete Another name for the -alias parameter. Parameters go last as clearly stated in the beginning of the man page = and that works well: root@lion3:/home/test # ifconfig lo0 inet6 2001:db8:ffff::ffff/128 alias root@lion3:/home/test # ifconfig lo0 inet6 2001:db8:ffff::ffff/128 = delete root@lion3:/home/test #=20 The fact that for inet you could give it in the beginning and other = minor things leniently allowed have caused a lot of trouble the last years. We try to stay more strict even if there's no real grammar, even if it wasn't for SVN. /bz --=20 Bjoern A. Zeeb You have to have visions! It does not matter how good you are. It matters what good you do!