From owner-freebsd-net@FreeBSD.ORG Sun Feb 14 10:39:17 2010 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 031AF1065693 for ; Sun, 14 Feb 2010 10:39:17 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id CDDA08FC16 for ; Sun, 14 Feb 2010 10:39:16 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1EAdGpe027104 for ; Sun, 14 Feb 2010 10:39:16 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o1EAdGTM027103; Sun, 14 Feb 2010 10:39:16 GMT (envelope-from gnats) Date: Sun, 14 Feb 2010 10:39:16 GMT Message-Id: <201002141039.o1EAdGTM027103@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: dfilter@FreeBSD.ORG (dfilter service) Cc: Subject: Re: kern/139079: commit references a PR X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: dfilter service List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Feb 2010 10:39:17 -0000 The following reply was made to PR kern/139079; it has been noted by GNATS. From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/139079: commit references a PR Date: Sun, 14 Feb 2010 09:35:24 +0000 (UTC) Author: gavin Date: Sun Feb 14 09:34:27 2010 New Revision: 203855 URL: http://svn.freebsd.org/changeset/base/203855 Log: Merge r200530 from head: Don't panic on failure to attach if we fail before or during the if_alloc() of ifp. This fixes the panic reported in the PRs, but not the attach failure. PR: kern/139079, kern/143874 Tested by: Steven Noonan Reviewed by: thompsa Modified: stable/8/sys/dev/wpi/if_wpi.c Directory Properties: stable/8/sys/ (props changed) stable/8/sys/amd64/include/xen/ (props changed) stable/8/sys/cddl/contrib/opensolaris/ (props changed) stable/8/sys/contrib/dev/acpica/ (props changed) stable/8/sys/contrib/pf/ (props changed) stable/8/sys/dev/xen/xenpci/ (props changed) stable/8/sys/netinet/ (props changed) Modified: stable/8/sys/dev/wpi/if_wpi.c ============================================================================== --- stable/8/sys/dev/wpi/if_wpi.c Sun Feb 14 07:20:58 2010 (r203854) +++ stable/8/sys/dev/wpi/if_wpi.c Sun Feb 14 09:34:27 2010 (r203855) @@ -713,13 +713,14 @@ wpi_detach(device_t dev) { struct wpi_softc *sc = device_get_softc(dev); struct ifnet *ifp = sc->sc_ifp; - struct ieee80211com *ic = ifp->if_l2com; + struct ieee80211com *ic; int ac; - ieee80211_draintask(ic, &sc->sc_restarttask); - ieee80211_draintask(ic, &sc->sc_radiotask); - if (ifp != NULL) { + ic = ifp->if_l2com; + + ieee80211_draintask(ic, &sc->sc_restarttask); + ieee80211_draintask(ic, &sc->sc_radiotask); wpi_stop(sc); callout_drain(&sc->watchdog_to); callout_drain(&sc->calib_to); _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Feb 14 10:39:18 2010 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A2E2C106568D for ; Sun, 14 Feb 2010 10:39:18 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 793748FC12 for ; Sun, 14 Feb 2010 10:39:18 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1EAdIwb027109 for ; Sun, 14 Feb 2010 10:39:18 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o1EAdIpp027108; Sun, 14 Feb 2010 10:39:18 GMT (envelope-from gnats) Date: Sun, 14 Feb 2010 10:39:18 GMT Message-Id: <201002141039.o1EAdIpp027108@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: dfilter@FreeBSD.ORG (dfilter service) Cc: Subject: Re: kern/143874: commit references a PR X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: dfilter service List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Feb 2010 10:39:18 -0000 The following reply was made to PR kern/143874; it has been noted by GNATS. From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/143874: commit references a PR Date: Sun, 14 Feb 2010 09:35:25 +0000 (UTC) Author: gavin Date: Sun Feb 14 09:34:27 2010 New Revision: 203855 URL: http://svn.freebsd.org/changeset/base/203855 Log: Merge r200530 from head: Don't panic on failure to attach if we fail before or during the if_alloc() of ifp. This fixes the panic reported in the PRs, but not the attach failure. PR: kern/139079, kern/143874 Tested by: Steven Noonan Reviewed by: thompsa Modified: stable/8/sys/dev/wpi/if_wpi.c Directory Properties: stable/8/sys/ (props changed) stable/8/sys/amd64/include/xen/ (props changed) stable/8/sys/cddl/contrib/opensolaris/ (props changed) stable/8/sys/contrib/dev/acpica/ (props changed) stable/8/sys/contrib/pf/ (props changed) stable/8/sys/dev/xen/xenpci/ (props changed) stable/8/sys/netinet/ (props changed) Modified: stable/8/sys/dev/wpi/if_wpi.c ============================================================================== --- stable/8/sys/dev/wpi/if_wpi.c Sun Feb 14 07:20:58 2010 (r203854) +++ stable/8/sys/dev/wpi/if_wpi.c Sun Feb 14 09:34:27 2010 (r203855) @@ -713,13 +713,14 @@ wpi_detach(device_t dev) { struct wpi_softc *sc = device_get_softc(dev); struct ifnet *ifp = sc->sc_ifp; - struct ieee80211com *ic = ifp->if_l2com; + struct ieee80211com *ic; int ac; - ieee80211_draintask(ic, &sc->sc_restarttask); - ieee80211_draintask(ic, &sc->sc_radiotask); - if (ifp != NULL) { + ic = ifp->if_l2com; + + ieee80211_draintask(ic, &sc->sc_restarttask); + ieee80211_draintask(ic, &sc->sc_radiotask); wpi_stop(sc); callout_drain(&sc->watchdog_to); callout_drain(&sc->calib_to); _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Feb 14 13:30:07 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 97E451065672 for ; Sun, 14 Feb 2010 13:30:07 +0000 (UTC) (envelope-from egorenar@googlemail.com) Received: from mail-ew0-f211.google.com (mail-ew0-f211.google.com [209.85.219.211]) by mx1.freebsd.org (Postfix) with ESMTP id DF19D8FC12 for ; Sun, 14 Feb 2010 13:30:06 +0000 (UTC) Received: by ewy3 with SMTP id 3so4212338ewy.13 for ; Sun, 14 Feb 2010 05:30:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=DGFsmDNb3Zk+QskkPb5Ug/8VKG7NGc+qRwpqvmoUW9I=; b=pjO6ZGXQNBrnjpEwojiaYAUPYVQqGyRVPLmqrj9S3b+qFNoa9ruFWMdB55n3V2OkzR i1OdAWTBXLsDGAkTJVmADYtG9tyImhaz/W600M0YDmT4osOmmpAK7QZlj/zXefjS2d2G sYL3Ltn/PhBzHKNQ/8ofa2b1we5zfjxArG3R0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=IZ2vhRGy33o3dPGX4LiYHYUO+1+blf0oIRyreo31fNp0hcXABoVwkcvEhMVZzsT87a 3DchHyOmULMmmD9ECbBWqnwqQuBW7w/1xsWZo2F4BgsT3WkuA3NR8ooHid9uE1+oYFg/ t0sZeApNyt10c/eSi7jliDI/GaUh604LXhQ7o= MIME-Version: 1.0 Received: by 10.213.109.214 with SMTP id k22mr1735524ebp.83.1266154205760; Sun, 14 Feb 2010 05:30:05 -0800 (PST) In-Reply-To: <4B65D816.5010709@errno.com> References: <2d3b7e441001271104j14836df4j14428d34561dead1@mail.gmail.com> <2d3b7e441001271108p2a40dbbbwaa6af9679d61ab@mail.gmail.com> <4B64FC76.5060506@errno.com> <2d3b7e441001302334t5baa1102pdd1ee0309d32c2fd@mail.gmail.com> <4B65D816.5010709@errno.com> Date: Sun, 14 Feb 2010 14:30:05 +0100 Message-ID: <2d3b7e441002140530m176ccf5csd120f1029263413b@mail.gmail.com> From: Alexander Egorenkov To: Sam Leffler Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org Subject: Re: A-MPDU transmission in net80211 on FreeBSD 8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Feb 2010 13:30:07 -0000 Hi, thanks for your help, I implemented A-MPDU Tx in my Ralink driver now and it works very good. I have average Tx rate about 4.5~5 MBytes/s now :-) The Ralink chip that i have implements the frame queue in hardware so i had only to assign sequence numbers, set BA window size and MPDU density and it worked, lucky me :-) Alex. On Sun, Jan 31, 2010 at 8:20 PM, Sam Leffler wrote: > Alexander Egorenkov wrote: > >> Why doesn't 802.11 stack assign sequence numbers to A-MPDU frames ? >> > > Because if net80211 does the assignment it may be wrong. As the comment > says, if tx aggregation causes frames to be q'd above the h/w then by the > time they are sent OTA the pre-assigned seq# may be invalidated by other > frames going out ahead of it. > > > When sequence numbers are not assigned to A-MPDU frames, then BA doesn't >> work with my AP. >> I tried to assign sequence numbers to A-MPDU frames in my device driver >> and then BAs worked >> with my AP. >> > > That is what the comment says to do. > > > And what is meant by aggregation queue ? Where is that queue anf how do i >> use it ? >> > > The aggregation q is the mechanism used to hold frames waiting for > additional frames to aggregated into an A-MSDU/A-MPDU. The queue is > typically wherever the aggregation work is done. Some devices do this in > h/w, others require the host handle this. When done in the host it can be > done in the driver or above. The intent has always been to have net80211 > implement tx aggregation that a driver can fallback on but I never did the > work. All the 11n drivers I've done have either handled tx aggregation in > h/w or in the driver. > > >> Thanks. >> >> >> On Sun, Jan 31, 2010 at 4:43 AM, Sam Leffler > sam@errno.com>> wrote: >> >> Alexander Egorenkov wrote: >> >> Sorry, i posted the wrong comment. >> Here is the comment which i don't understand: >> >> /* >> * NB: don't assign a sequence # to potential >> * aggregates; we expect this happens at the >> * point the frame comes off any aggregation q >> * as otherwise we may introduce holes in the >> * BA sequence space and/or make window accouting >> * more difficult. >> * >> * XXX may want to control this with a driver >> * capability; this may also change when we pull >> * aggregation up into net80211 >> */ >> >> Thanks. >> >> >> What is unclear? >> >> Sam >> >> >> >> On Wed, Jan 27, 2010 at 8:04 PM, Alexander Egorenkov < >> egorenar@googlemail.com > wrote: >> >> Hi, >> >> i'm implementing a device driver for a 802.11n NIC under >> FreeBSD 8 >> und experimented with A-MPDU transmission. I looked into >> net80211 code >> and there is some code which implements this feature but it >> worked not very >> well for me. >> I noticed e.g. that sequence numbers are not assigned to >> A-MPDU frames >> and found this comment in file ieee80211_output.c : >> >> >> /* >> * Check if A-MPDU tx aggregation is setup or if we >> * should try to enable it. The sta must be associated >> * with HT and A-MPDU enabled for use. When the policy >> * routine decides we should enable A-MPDU we issue an >> * ADDBA request and wait for a reply. The frame being >> * encapsulated will go out w/o using A-MPDU, or >> possibly >> * it might be collected by the driver and >> held/retransmit. >> * The default ic_ampdu_enable routine handles >> staggering >> * ADDBA requests in case the receiver NAK's us or we >> are >> * otherwise unable to establish a BA stream. >> */ >> >> Can somebody elaborate this description to me please. >> >> Thanks. >> >> ALex. >> >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing >> >> list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to >> "freebsd-net-unsubscribe@freebsd.org >> " >> >> >> >> >> > From owner-freebsd-net@FreeBSD.ORG Sun Feb 14 19:57:54 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EB1351065676 for ; Sun, 14 Feb 2010 19:57:54 +0000 (UTC) (envelope-from sam@errno.com) Received: from ebb.errno.com (ebb.errno.com [69.12.149.25]) by mx1.freebsd.org (Postfix) with ESMTP id B46548FC19 for ; Sun, 14 Feb 2010 19:57:54 +0000 (UTC) Received: from ice.local ([10.0.0.115]) (authenticated bits=0) by ebb.errno.com (8.13.6/8.12.6) with ESMTP id o1EJvrUu078436 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 14 Feb 2010 11:57:53 -0800 (PST) (envelope-from sam@errno.com) Message-ID: <4B7855C1.3090101@errno.com> Date: Sun, 14 Feb 2010 11:57:53 -0800 From: Sam Leffler User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Alexander Egorenkov References: <2d3b7e441001271104j14836df4j14428d34561dead1@mail.gmail.com> <2d3b7e441001271108p2a40dbbbwaa6af9679d61ab@mail.gmail.com> <4B64FC76.5060506@errno.com> <2d3b7e441001302334t5baa1102pdd1ee0309d32c2fd@mail.gmail.com> <4B65D816.5010709@errno.com> <2d3b7e441002140530m176ccf5csd120f1029263413b@mail.gmail.com> In-Reply-To: <2d3b7e441002140530m176ccf5csd120f1029263413b@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-DCC-x.dcc-servers-Metrics: ebb.errno.com; whitelist Cc: freebsd-net@freebsd.org Subject: Re: A-MPDU transmission in net80211 on FreeBSD 8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Feb 2010 19:57:55 -0000 That's great to hear. I take it this is HT20? I believe you can get 40-60 Mb/s @ MCS15 w/o AMPDU and 80-100 Mb/s w/ AMPDU. In HT40 I've gotten 160-180 Mb/s for TCP netperf (5GHz) and 220+ Mb/s for unidirectional streams (netperf -t UDP_STREAM). Note that once you get to these higher rates things like TCP window sizes become important and doing things like TSO in s/w can give noticeable speed boosts. Sam Alexander Egorenkov wrote: > Hi, > > thanks for your help, I implemented A-MPDU Tx in my Ralink driver now > and it works very good. > I have average Tx rate about 4.5~5 MBytes/s now :-) The Ralink chip that > i have implements the frame queue in hardware so i had only to assign > sequence numbers, set BA window size and MPDU density and it worked, > lucky me :-) > > Alex. > > On Sun, Jan 31, 2010 at 8:20 PM, Sam Leffler > wrote: > > Alexander Egorenkov wrote: > > Why doesn't 802.11 stack assign sequence numbers to A-MPDU frames ? > > > Because if net80211 does the assignment it may be wrong. As the > comment says, if tx aggregation causes frames to be q'd above the > h/w then by the time they are sent OTA the pre-assigned seq# may be > invalidated by other frames going out ahead of it. > > > When sequence numbers are not assigned to A-MPDU frames, then BA > doesn't work with my AP. > I tried to assign sequence numbers to A-MPDU frames in my device > driver and then BAs worked > with my AP. > > > That is what the comment says to do. > > > And what is meant by aggregation queue ? Where is that queue anf > how do i use it ? > > > The aggregation q is the mechanism used to hold frames waiting for > additional frames to aggregated into an A-MSDU/A-MPDU. The queue is > typically wherever the aggregation work is done. Some devices do > this in h/w, others require the host handle this. When done in the > host it can be done in the driver or above. The intent has always > been to have net80211 implement tx aggregation that a driver can > fallback on but I never did the work. All the 11n drivers I've done > have either handled tx aggregation in h/w or in the driver. > > > Thanks. > > > On Sun, Jan 31, 2010 at 4:43 AM, Sam Leffler >> wrote: > > Alexander Egorenkov wrote: > > Sorry, i posted the wrong comment. > Here is the comment which i don't understand: > > /* > * NB: don't assign a sequence # to potential > * aggregates; we expect this happens at the > * point the frame comes off any aggregation q > * as otherwise we may introduce holes in the > * BA sequence space and/or make window accouting > * more difficult. > * > * XXX may want to control this with a driver > * capability; this may also change when we pull > * aggregation up into net80211 > */ > > Thanks. > > > What is unclear? > > Sam > > > > On Wed, Jan 27, 2010 at 8:04 PM, Alexander Egorenkov < > egorenar@googlemail.com > >> wrote: > > Hi, > > i'm implementing a device driver for a 802.11n NIC under > FreeBSD 8 > und experimented with A-MPDU transmission. I looked into > net80211 code > and there is some code which implements this feature > but it > worked not very > well for me. > I noticed e.g. that sequence numbers are not assigned to > A-MPDU frames > and found this comment in file ieee80211_output.c : > > > /* > * Check if A-MPDU tx aggregation is setup or > if we > * should try to enable it. The sta must be > associated > * with HT and A-MPDU enabled for use. When > the policy > * routine decides we should enable A-MPDU we > issue an > * ADDBA request and wait for a reply. The > frame being > * encapsulated will go out w/o using A-MPDU, > or possibly > * it might be collected by the driver and > held/retransmit. > * The default ic_ampdu_enable routine handles > staggering > * ADDBA requests in case the receiver NAK's > us or we are > * otherwise unable to establish a BA stream. > */ > > Can somebody elaborate this description to me please. > > Thanks. > > ALex. > > > _______________________________________________ > freebsd-net@freebsd.org > > mailing > > list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org > > >" > > > > > > From owner-freebsd-net@FreeBSD.ORG Sun Feb 14 20:30:28 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 04726106568B for ; Sun, 14 Feb 2010 20:30:28 +0000 (UTC) (envelope-from egorenar@googlemail.com) Received: from mail-ew0-f211.google.com (mail-ew0-f211.google.com [209.85.219.211]) by mx1.freebsd.org (Postfix) with ESMTP id 614FE8FC17 for ; Sun, 14 Feb 2010 20:30:27 +0000 (UTC) Received: by ewy3 with SMTP id 3so4480011ewy.13 for ; Sun, 14 Feb 2010 12:30:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=GGD6iR7ejfFDkBLiIZxebSg3SdETqL5koWzHs7x4dgw=; b=aILK8VTIasL2WrcflTqX5Xb2rtizgWa/hp3GOQe3MGGnYr28V2GFVCL0pQHQ9Twn25 pKYY8fPaZT4V/l18Bh+Bqtk8Rk/CPktxj8E33l5ch72Re/JY2KYgwhtubTCqNUylK+8L de9Ubjj4533d6DbbrzGDc/DtJ7jPc0WTRglGE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=g3H6xKhNaJrpQyVr4p+Vt/wNX76MlBM+YZUIXNhtVjDHqRKLvgca0Or3Lc3zTjLilR j2LiOkGxVcbjS0BnhTw5lfoKe4NkrdKh37lt/RiUiqKTROndEm6cJGZ7QFYmytt+wE/N J0Ux3Krry9tPSqChhKSaXuyEA0f3FWXcS0iro= MIME-Version: 1.0 Received: by 10.213.38.3 with SMTP id z3mr2971946ebd.69.1266179426163; Sun, 14 Feb 2010 12:30:26 -0800 (PST) In-Reply-To: <4B7855C1.3090101@errno.com> References: <2d3b7e441001271104j14836df4j14428d34561dead1@mail.gmail.com> <2d3b7e441001271108p2a40dbbbwaa6af9679d61ab@mail.gmail.com> <4B64FC76.5060506@errno.com> <2d3b7e441001302334t5baa1102pdd1ee0309d32c2fd@mail.gmail.com> <4B65D816.5010709@errno.com> <2d3b7e441002140530m176ccf5csd120f1029263413b@mail.gmail.com> <4B7855C1.3090101@errno.com> Date: Sun, 14 Feb 2010 21:30:26 +0100 Message-ID: <2d3b7e441002141230u320b5b78y995fc91c82820b0d@mail.gmail.com> From: Alexander Egorenkov To: Sam Leffler Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org Subject: Re: A-MPDU transmission in net80211 on FreeBSD 8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Feb 2010 20:30:28 -0000 Wow, 160-180 Mb/s. That's great :-) No, it's HT40 but my AP is not very good, it sends to me only A-MSDUs but no A-MPDUs (don't know why, i guess because it's cheap), and there are many WLANs in my neighbourhood in 2.4GHZ and no WLANs in 5GHZ but unfortunately i have an 2GHZ only AP. Furthermore, i don't have any documentation about the Ralink chip i have, only Linux source code from Ralink which i study in order to learn how the device works :-) But it's OK, i'm making progress. For which devices did you develop 802.11n drivers ? And what AP did you use in your tests ? Alex. On Sun, Feb 14, 2010 at 8:57 PM, Sam Leffler wrote: > That's great to hear. I take it this is HT20? I believe you can get 40-60 > Mb/s @ MCS15 w/o AMPDU and 80-100 Mb/s w/ AMPDU. In HT40 I've gotten > 160-180 Mb/s for TCP netperf (5GHz) and 220+ Mb/s for unidirectional streams > (netperf -t UDP_STREAM). Note that once you get to these higher rates > things like TCP window sizes become important and doing things like TSO in > s/w can give noticeable speed boosts. > > Sam > > Alexander Egorenkov wrote: > >> Hi, >> >> thanks for your help, I implemented A-MPDU Tx in my Ralink driver now and >> it works very good. >> I have average Tx rate about 4.5~5 MBytes/s now :-) The Ralink chip that i >> have implements the frame queue in hardware so i had only to assign sequence >> numbers, set BA window size and MPDU density and it worked, lucky me :-) >> >> Alex. >> >> On Sun, Jan 31, 2010 at 8:20 PM, Sam Leffler > sam@errno.com>> wrote: >> >> Alexander Egorenkov wrote: >> >> Why doesn't 802.11 stack assign sequence numbers to A-MPDU frames ? >> >> >> Because if net80211 does the assignment it may be wrong. As the >> comment says, if tx aggregation causes frames to be q'd above the >> h/w then by the time they are sent OTA the pre-assigned seq# may be >> invalidated by other frames going out ahead of it. >> >> >> When sequence numbers are not assigned to A-MPDU frames, then BA >> doesn't work with my AP. >> I tried to assign sequence numbers to A-MPDU frames in my device >> driver and then BAs worked >> with my AP. >> >> >> That is what the comment says to do. >> >> >> And what is meant by aggregation queue ? Where is that queue anf >> how do i use it ? >> >> >> The aggregation q is the mechanism used to hold frames waiting for >> additional frames to aggregated into an A-MSDU/A-MPDU. The queue is >> typically wherever the aggregation work is done. Some devices do >> this in h/w, others require the host handle this. When done in the >> host it can be done in the driver or above. The intent has always >> been to have net80211 implement tx aggregation that a driver can >> fallback on but I never did the work. All the 11n drivers I've done >> have either handled tx aggregation in h/w or in the driver. >> >> >> Thanks. >> >> >> On Sun, Jan 31, 2010 at 4:43 AM, Sam Leffler > > >> >> wrote: >> >> Alexander Egorenkov wrote: >> >> Sorry, i posted the wrong comment. >> Here is the comment which i don't understand: >> >> /* >> * NB: don't assign a sequence # to potential >> * aggregates; we expect this happens at the >> * point the frame comes off any aggregation q >> * as otherwise we may introduce holes in the >> * BA sequence space and/or make window accouting >> * more difficult. >> * >> * XXX may want to control this with a driver >> * capability; this may also change when we pull >> * aggregation up into net80211 >> */ >> >> Thanks. >> >> >> What is unclear? >> >> Sam >> >> >> >> On Wed, Jan 27, 2010 at 8:04 PM, Alexander Egorenkov < >> egorenar@googlemail.com >> > >> >> wrote: >> >> Hi, >> >> i'm implementing a device driver for a 802.11n NIC under >> FreeBSD 8 >> und experimented with A-MPDU transmission. I looked into >> net80211 code >> and there is some code which implements this feature >> but it >> worked not very >> well for me. >> I noticed e.g. that sequence numbers are not assigned to >> A-MPDU frames >> and found this comment in file ieee80211_output.c : >> >> >> /* >> * Check if A-MPDU tx aggregation is setup or >> if we >> * should try to enable it. The sta must be >> associated >> * with HT and A-MPDU enabled for use. When >> the policy >> * routine decides we should enable A-MPDU we >> issue an >> * ADDBA request and wait for a reply. The >> frame being >> * encapsulated will go out w/o using A-MPDU, >> or possibly >> * it might be collected by the driver and >> held/retransmit. >> * The default ic_ampdu_enable routine handles >> staggering >> * ADDBA requests in case the receiver NAK's >> us or we are >> * otherwise unable to establish a BA stream. >> */ >> >> Can somebody elaborate this description to me please. >> >> Thanks. >> >> ALex. >> >> >> _______________________________________________ >> freebsd-net@freebsd.org >> > > mailing >> >> list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to >> "freebsd-net-unsubscribe@freebsd.org >> >> > >" >> >> >> >> >> >> >> > From owner-freebsd-net@FreeBSD.ORG Sun Feb 14 22:28:17 2010 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 223F71065670; Sun, 14 Feb 2010 22:28:17 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id EF5A58FC2A; Sun, 14 Feb 2010 22:28:16 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1EMSFaD074472; Sun, 14 Feb 2010 22:28:15 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o1EMSFru074468; Sun, 14 Feb 2010 22:28:15 GMT (envelope-from linimon) Date: Sun, 14 Feb 2010 22:28:15 GMT Message-Id: <201002142228.o1EMSFru074468@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/143939: [ipfw] [em] ipfw nat and em interface rxcsum problem X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Feb 2010 22:28:17 -0000 Old Synopsis: ipfw nat and em interface rxcsum problem New Synopsis: [ipfw] [em] ipfw nat and em interface rxcsum problem Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Feb 14 22:27:58 UTC 2010 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=143939 From owner-freebsd-net@FreeBSD.ORG Mon Feb 15 11:01:04 2010 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7D8D11065695 for ; Mon, 15 Feb 2010 11:01:04 +0000 (UTC) (envelope-from sobomax@FreeBSD.org) Received: from sippysoft.com (gk1.360sip.com [72.236.70.240]) by mx1.freebsd.org (Postfix) with ESMTP id 39C728FC15 for ; Mon, 15 Feb 2010 11:01:03 +0000 (UTC) Received: from [192.168.1.38] ([70.71.167.197]) (authenticated bits=0) by sippysoft.com (8.14.3/8.14.3) with ESMTP id o1FB11lS007253 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 15 Feb 2010 03:01:02 -0800 (PST) (envelope-from sobomax@FreeBSD.org) Message-ID: <4B79297D.9080403@FreeBSD.org> Date: Mon, 15 Feb 2010 03:01:17 -0800 From: Maxim Sobolev Organization: Sippy Software, Inc. User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: freebsd-net@FreeBSD.org, FreeBSD Hackers Content-Type: text/plain; charset=KOI8-U; format=flowed Content-Transfer-Encoding: 7bit Cc: Jack Vogel Subject: Sudden mbuf demand increase and shortage under the load X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Feb 2010 11:01:04 -0000 Hi, Our company have a FreeBSD based product that consists of the numerous interconnected processes and it does some high-PPS UDP processing (30-50K PPS is not uncommon). We are seeing some strange periodic failures under the load in several such systems, which usually evidences itself in IPC (even through unix domain sockets) suddenly either breaking down or pausing and restoring only some time later (like 5-10 minutes). The only sign of failure I managed to find was the increase of the "requests for mbufs denied" in the netstat -m and number of total mbuf clusters (nmbclusters) raising up to the limit. I have tried to raise some network-related limits (most notably maxusers and nmbclusters), but it has not helped with the issue - it's still happening from time to time to us. Below you can find output from the netstat -m few minutes right after that shortage period - you see that somehow the system has allocated huge amount of memory for the network (700MB), with only tiny amount of that being actually in use. This is for the kern.ipc.nmbclusters: 302400. Eventually the system reclaims all that memory and goes back to its normal use of 30-70MB. This problem is killing us, so any suggestions are greatly appreciated. My current hypothesis is that due to some issues either with the network driver or network subsystem itself, the system goes insane and "eats" up all mbufs up to nmbclusters limit. But since mbufs are shared between network and local IPC, IPC goes down as well. We observe this issue with systems using both em(4) driver and igb(4) driver. I believe both drivers share the same design, however I am not sure if this is some kind of design flaw in the driver or part of a larger problem with the network subsystem. This happens on amd64 7.2-RELEASE and 7.3-PRERELEASE alike, with 8GB of memory. I have not tried upgrading to 8.0, this is production system so upgrading will not be easy. I don't believe there are some differences that let us hope that this problem will go away after upgrade, but I can try it as the last resort. As I said, this is very critical issue, so I can provide any additional debug information upon request. We are ready to go as far as paying somebody reasonable amount of money for tracking down and resolving the issue. Regards, -- Maksym Sobolyev Sippy Software, Inc. Internet Telephony (VoIP) Experts T/F: +1-646-651-1110 Web: http://www.sippysoft.com MSN: sales@sippysoft.com Skype: SippySoft [ssp-root@ds-467 /usr/src]$ netstat -m 17061/417669/434730 mbufs in use (current/cache/total) 10420/291980/302400/302400 mbuf clusters in use (current/cache/total/max) 10420/0 mbuf+clusters out of packet secondary zone in use (current/cache) 19/1262/1281/51200 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/25600 9k jumbo clusters in use (current/cache/total/max) 0/0/0/12800 16k jumbo clusters in use (current/cache/total/max) 25181K/693425K/718606K bytes allocated to network (current/cache/total) 1246681/129567494/67681640 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines [FEW MINUTES LATER] [ssp-root@ds-467 /usr/src]$ netstat -m 10001/84574/94575 mbufs in use (current/cache/total) 6899/6931/13830/302400 mbuf clusters in use (current/cache/total/max) 6899/6267 mbuf+clusters out of packet secondary zone in use (current/cache) 2/1151/1153/51200 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/25600 9k jumbo clusters in use (current/cache/total/max) 0/0/0/12800 16k jumbo clusters in use (current/cache/total/max) 16306K/39609K/55915K bytes allocated to network (current/cache/total) 1246681/129567494/67681640 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines From owner-freebsd-net@FreeBSD.ORG Mon Feb 15 11:07:06 2010 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 440D5106566B for ; Mon, 15 Feb 2010 11:07:06 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 312D88FC17 for ; Mon, 15 Feb 2010 11:07:06 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1FB76I2070394 for ; Mon, 15 Feb 2010 11:07:06 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o1FB752h070390 for freebsd-net@FreeBSD.org; Mon, 15 Feb 2010 11:07:05 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 15 Feb 2010 11:07:05 GMT Message-Id: <201002151107.o1FB752h070390@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Feb 2010 11:07:06 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/143939 net [ipfw] [em] ipfw nat and em interface rxcsum problem o kern/143874 net [wpi] Wireless 3945ABG error. wpi0 could not allocate o kern/143868 net [ath] [patch] allow Atheros watchdog timeout to be tun o kern/143855 net [bpf] [patch] non-blocking BPF reads return -1/EWOULDB o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to o kern/143788 net [iwi] wpa_supplicant(8) can't set privacy on iwi inter s kern/143673 net [stf] [request] there should be a way to support multi s kern/143666 net [ip6] [request] PMTU black hole detection not implemen o kern/143627 net [ieee80211] [panic] A bug in ht_send_action_ba_addba c o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143595 net [wpi] [panic] Creating virtual interface over wpi0 in o kern/143593 net [ipsec] When using IPSec, tcpdump doesn't show outgoin o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143573 net [em] em(4) NIC crashes intermittently o kern/143285 net [em] [regression] jumbo frames broken in 8.0 o kern/143208 net [ipsec] [gif] IPSec over gif interface not working o conf/143079 net hostapd(8) startup missing multi wlan functionality o kern/143074 net [wi]: wi driver triggers panic o kern/143046 net [mxge] [panic] panics since mxge(4) update o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142907 net [wpi] if_wpi unstable on ibm/lenovo x60 -- suspect fir o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142766 net [ipw] [regression] ipw(4) with Intel PRO/wireless 2100 o bin/142547 net wpa_supplicant(8) drops connection on key renegotiatio o kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142019 net [em] em needs "ifconfig em0 down up" when link was gon o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 o kern/141843 net [em] [vlan] Intel txcsum and assigned vlan invoke wron o kern/141777 net [rum] [patch] Support usbdevs / rum(4) for Buffalo WLI f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/141720 net [sctp] [lor] [hang] sctp-create vs. sctp-it causes sys o kern/141698 net [sctp] [panic] Own lock on stcb at return from input o kern/141697 net [sctp] [panic] lock (sleep mutex) sctp-tcb not locked o kern/141696 net [rum] [panic] rum(4)+ vimage = kernel panic o kern/141695 net [sctp] [panic] kernel page fault with non-sleepable lo o kern/141314 net Network Performance has decreased by 30% [regression] o kern/141285 net [em] hangs down/up intel nic during creating vlan o kern/141023 net [carp] CARP arp replays with wrong src mac o kern/140970 net [bce] The two NetXtreme II BCM5709S NICs on our HP Bl4 o kern/140796 net [ath] [panic] privileged instruction fault o kern/140778 net [em] randomly panic in vlan/em o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140728 net [em] [patch] Fast irq registration in em driver o kern/140684 net [bce] Broadcom NetXtreme II BCM5709 1000Base-T - fail o kern/140647 net [em] [patch] e1000 driver does not correctly handle mu o kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc s kern/140597 net [request] implement Lost Retransmission Detection o kern/140567 net [ath] [patch] ath is not worked on my notebook PC o kern/140564 net [wpi] Problem with Intel(R) PRO/Wireless 3945ABG o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140326 net [em] em0: watchdog timeout when communicating to windo o kern/140245 net [ath] [panic] Kernel panic during network activity on o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/140051 net [bce] [arp] ARP not sent through Bridge Firewall with o kern/139761 net [bce] bce driver on IBM HS22 [No PHY found on Child MI o kern/139565 net [ipfilter] ipfilter ioctl SIOCDELST broken o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL o kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139162 net [fwip] [panic] 8.0-RC1 panics if using IP over firewir o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/139079 net [wpi] Failure to attach wpi(4) o kern/139058 net [ipfilter] mbuf cluster leak on FreeBSD 7.2 o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138739 net [wpi] wpi(4) does not work very well under 8.0-BETA4 o kern/138694 net [bge] FreeBSD 6.3 release does not recognize Broadcom o amd64/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138660 net [igb] igb driver troubles in 8.0-BETA4 o kern/138652 net [tcp] TCP window scaling value calculated incorrectly? o kern/138620 net [lagg] [patch] lagg port bpf-writes blocked o kern/138427 net [wpi] [panic] Kernel panic after trying set monitor wl o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r o kern/138177 net [ipfilter] FreeBSD crashing repeatedly in ip_nat.c:257 o kern/138046 net [tcp] tcp sockets stay in SYN_SENT even after receivin o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 o bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137795 net [sctp] [panic] mtx_lock() of destroyed mutex o kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o kern/137775 net [netgraph] [patch] Add XMIT_FAILOVER to ng_one2many o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137592 net [ath] panic - 7-STABLE (Aug 7, 2009 UTC) crashes on ne o bin/137484 net [patch] Integer overflow in wpa_supplicant(8) base64 e o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137317 net [tcp] logs full of syncache problems o kern/137279 net [bge] [panic] Page fault (fatal trap 12) NFS server w/ o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o bin/136994 net [patch] ifconfig(8) print carp mac address o kern/136943 net [wpi] [lor] wpi0_com_lock / wpi0 o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136876 net [bge] bge will not resume properly after suspend o kern/136836 net [ath] atheros card stops functioning after about 12 ho o bin/136661 net [patch] ndp(8) ignores -f option o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/136482 net [age] Attansic L1 Gigabit Ethernet recieves multicasts o kern/136426 net [panic] spawning several dhclients in parallel panics o kern/136168 net [em] em driver initialization fails on Intel 5000PSL m o kern/135836 net [bce] bce BCM5709 Watchdog after warm boot - ok after o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/135222 net [igb] low speed routing between two igb interfaces o kern/134956 net [em] FreeBSD 7.1 & 7.2, Intel PRO/1000 PT Quad Port Se o kern/134931 net [route] Route messages sent to all socket listeners re o kern/134658 net [bce] bce driver fails on PowerEdge m610 blade. o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134401 net [msk] [panic] Kernel Fatal trap 12: page fault while i o kern/134168 net [ral] ral driver problem on RT2525 2.4GHz transceiver o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/134079 net [em] "em0: Invalid MAC address" in FreeBSD-Current ( 8 o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133902 net [tun] Killing tun0 iface ssh tunnel causes Panic Strin o kern/133786 net [netinet] [patch] ip_input might cause kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133613 net [wpi] [panic] kernel panic in wpi(4) o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133328 net [bge] [panic] Kernel panics with Windows7 client o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre o kern/133218 net [carp] [hang] use of carp(4) causes system to freeze f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133204 net [msk] msk driver timeouts o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132991 net [bge] if_bge low performance problem f bin/132911 net ip6fw(8): argument type of fill_icmptypes is wrong and o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o kern/132885 net [wlan] 802.1x broken after SVN rev 189592 o conf/132851 net [patch] rc.conf(5): allow to setfib(1) for service run o kern/132832 net [netinet] [patch] tcp_output() might generate invalid o bin/132798 net [patch] ggatec(8): ggated/ggatec connection slowdown p o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132722 net [ath] Wifi ath0 associates fine with AP, but DHCP or I o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132669 net [xl] 3c905-TX send DUP! in reply on ping (sometime) o kern/132554 net [ipl] There is no ippool start script/ipfilter magic t o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/132285 net [carp] alias gives incorrect hash in dmesg o kern/132277 net [crypto] [ipsec] poor performance using cryptodevice f o kern/132107 net [carp] carp(4) advskew setting ignored when carp IP us o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o bin/131567 net [socket] [patch] Update for regression/sockets/unix_cm o kern/131549 net ifconfig(8) can't clear 'monitor' mode on the wireless o kern/131536 net [netinet] [patch] kernel does allow manipulation of su o bin/131365 net route(8): route add changes interpretation of network o kern/131162 net [ath] Atheros driver bugginess and kernel crashes o kern/131153 net [iwi] iwi doesn't see a wireless network f kern/131087 net [ipw] [panic] ipw / iwi - no sent/received packets; iw f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o conf/130555 net [rc.d] [patch] No good way to set ipfilter variables a o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour o kern/129750 net [ath] Atheros AR5006 exits on "cannot map register spa f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow o kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129352 net [xl] [patch] xl0 watchdog timeout o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o bin/128954 net ifconfig(8) deletes valid routes o kern/128917 net [wpi] [panic] if_wpi and wpa+tkip causing kernel panic o kern/128884 net [msk] if_msk page fault while in kernel mode o kern/128840 net [igb] page fault under load with igb/LRO o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o conf/128334 net [request] use wpa_cli in the "WPA DHCP" situation o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127928 net [tcp] [patch] TCP bandwidth gets squeezed every time t o kern/127834 net [ixgbe] [patch] wrong error counting o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) s kern/127587 net [bge] [request] if_bge(4) doesn't support BCM576X fami f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/127102 net [wpi] Intel 3945ABG low throughput o kern/127057 net [udp] Unable to send UDP packet via IPv6 socket to IPv o kern/127050 net [carp] ipv6 does not work on carp interfaces [regressi o kern/126945 net [carp] CARP interface destruction with ifconfig destro o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o bin/126822 net wpa_supplicant(8): WPA PSK does not work in adhoc mode o kern/126714 net [carp] CARP interface renaming makes system no longer o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126688 net [ixgbe] [patch] 1.4.7 ixgbe driver panic with 4GB and o kern/126475 net [ath] [panic] ath pcmcia card inevitably panics under o kern/126339 net [ipw] ipw driver drops the connection o kern/126214 net [ath] txpower problem with Atheros wifi card o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125816 net [carp] [if_bridge] carp stuck in init when using bridg f kern/125502 net [ral] ifconfig ral0 scan produces no output unless in o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124767 net [iwi] Wireless connection using iwi0 driver (Intel 220 o kern/124753 net [ieee80211] net80211 discards power-save queue packets o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124127 net [msk] watchdog timeout (missed Tx interrupts) -- recov o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. p kern/123961 net [vr] [patch] Allow vr interface to handle vlans o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123796 net [ipf] FreeBSD 6.1+VPN+ipnat+ipf: port mapping does not o bin/123633 net ifconfig(8) doesn't set inet and ether address in one f kern/123617 net [tcp] breaking connection when client downloading file o kern/123603 net [tcp] tcp_do_segment and Received duplicate SYN o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o kern/123429 net [nfe] [hang] "ifconfig nfe up" causes a hard system lo o kern/123347 net [bge] bge1: watchdog timeout -- linkstate changed to D o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123256 net [wpi] panic: blockable sleep lock with wpi(4) f kern/123172 net [bce] Watchdog timeout problems with if_bce o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices o kern/122928 net [em] interface watchdog timeouts and stops receiving p f kern/122839 net [em] FreeBSD 7 multicast routing problem f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122772 net [em] em0 taskq panic, tcp reassembly bug causes radix o kern/122743 net [mbuf] [panic] vm_page_unwire: invalid wire count: 0 o kern/122697 net [ath] Atheros card is not well supported o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122195 net [ed] Alignment problems in if_ed o kern/122058 net [em] [panic] Panic on em1: taskq o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw o kern/121872 net [wpi] driver fails to attach on a fujitsu-siemens s711 s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121706 net [netinet] [patch] "rtfree: 0xc4383870 has 1 refs" emit o kern/121624 net [em] [regression] Intel em WOL fails after upgrade to o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o bin/121359 net [patch] [security] ppp(8): fix local stack overflow in o kern/121298 net [em] [panic] Fatal trap 12: page fault while in kernel o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/121080 net [bge] IPv6 NUD problem on multi address config on bge0 o kern/120966 net [rum] kernel panic with if_rum and WPA encryption p docs/120945 net [patch] ip6(4) man page lacks documentation for TCLASS o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o kern/120232 net [nfe] [patch] Bring in nfe(4) to RELENG_6 o kern/120130 net [carp] [panic] carp causes kernel panics in any conste o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o sparc/118932 net [panic] 7.0-BETA4/sparc-64 kernel panic in rip_output a kern/118879 net [bge] [patch] bge has checksum problems on the 5703 ch o kern/118727 net [netgraph] [patch] [request] add new ng_pf module a kern/118238 net [bce] [patch] bce driver shows "no carrier" on Intel S s kern/117717 net [panic] Kernel panic with Bittorrent client. o kern/117448 net [carp] 6.2 kernel crash [regression] o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o kern/117271 net [tap] OpenVPN TAP uses 99% CPU on releng_6 when if_tap o kern/117043 net [em] Intel PWLA8492MT Dual-Port Network adapter EEPROM o kern/116837 net [tun] [panic] [patch] ifconfig tunX destroy: panic o kern/116747 net [ndis] FreeBSD 7.0-CURRENT crash with Dell TrueMobile o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113895 net [xl] xl0 fails on 6.2-RELEASE but worked fine on 5.5-R o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o kern/112570 net [bge] packet loss with bge driver on BCM5704 chipset o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111457 net [ral] ral(4) freeze o kern/110140 net [ipw] ipw fails under load o kern/109733 net [bge] bge link state issues [regression] o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o kern/109308 net [pppd] [panic] Multiple panics kernel ppp suspected [r o kern/109251 net [re] [patch] if_re cardbus card won't attach o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] o kern/108542 net [bce] Huge network latencies with 6.2-RELEASE / STABLE o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o kern/107850 net [bce] bce driver link negotiation is faulty o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106438 net [ipf] ipfilter: keep state does not seem to allow repl o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/106243 net [nve] double fault panic in if_nve.c on high loads o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/105348 net [ath] ath device stopps TX o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/104485 net [bge] Broadcom BCM5704C: Intermittent on newer chip ve o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/101948 net [ipf] [panic] Kernel Panic Trap No 12 Page Fault - cau o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98978 net [ipf] [patch] ipfilter drops OOW packets under 6.1-Rel o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o bin/98218 net wpa_supplicant(8) blacklist not working f bin/97392 net ppp(8) hangs instead terminating o kern/97306 net [netgraph] NG_L2TP locks after connection with failed f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/96030 net [bfe] [patch] Install hangs with Broadcomm 440x NIC in o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear s kern/94863 net [bge] [patch] hack to get bge(4) working on IBM e326m o kern/94162 net [bge] 6.x kenel stale with bge(4) o kern/93886 net [ath] Atheros/D-Link DWL-G650 long delay to associate f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi f kern/92552 net A serious bug in most network drivers from 5.X to 6.X s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/92090 net [bge] bge0: watchdog timeout -- resetting o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 s kern/91777 net [ipf] [patch] wrong behaviour with skip rule inside an o kern/91594 net [em] FreeBSD > 5.4 w/ACPI fails to detect Intel Pro/10 o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/90890 net [vr] Problems with network: vr0: tx shutdown timeout s kern/90086 net [hang] 5.4p8 on supermicro P8SCT hangs during boot if f kern/88082 net [ath] [panic] cts protection for ath0 causes panic o kern/87521 net [ipf] [panic] using ipfilter "auth" keyword leads to k o kern/87506 net [vr] [patch] Fix alias support on vr interfaces s kern/86920 net [ndis] ifconfig: SIOCS80211: Invalid argument [regress o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86103 net [ipf] Illegal NAT Traversal in IPFilter o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o kern/85266 net [xe] [patch] xe(4) driver does not recognise Xircom XE o kern/84202 net [ed] [patch] Holtek HT80232 PCI NIC recognition on Fre o bin/82975 net route change does not parse classfull network as given o bin/82185 net [patch] ndp(8) can delete the incorrect entry s kern/81147 net [net] [patch] em0 reinitialization while adding aliase o kern/80853 net [ed] [patch] add support for Compex RL2000/ISA in PnP o kern/79895 net [ipf] 5.4-RC2 breaks ipfilter NAT when using netgraph f kern/79262 net [dc] Adaptec ANA-6922 not fully supported o bin/79228 net [patch] extend arp(8) to be able to create blackhole r o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if p kern/77913 net [wi] [patch] Add the APDL-325 WLAN pccard to wi(4) o kern/77341 net [ip6] problems with IPV6 implementation o kern/77273 net [ipf] ipfilter breaks ipv6 statefull filtering on 5.3 s kern/77195 net [ipf] [patch] ipfilter ioctl SIOCGNATL does not match o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time f kern/73538 net [bge] problem with the Broadcom BCM5788 Gigabit Ethern o kern/71469 net default route to internet magically disappears with mu o kern/70904 net [ipf] ipfilter ipnat problem with h323 proxy support o kern/64556 net [sis] [patch] if_sis short cable fix problems with Net s kern/60293 net [patch] FreeBSD arp poison patch o kern/54383 net [nfs] [patch] NFS root configurations without dynamic f i386/45773 net [bge] Softboot causes autoconf failure on Broadcom 570 s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr s kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/35442 net [sis] [patch] Problem transmitting runts in if_sis dri o kern/34665 net [ipf] [hang] ipfilter rcmd proxy "hangs". o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna o kern/27474 net [ipf] [ppp] Interactive use of user PPP and ipfilter c o conf/23063 net [arp] [patch] for static ARP tables in rc.network 399 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Feb 15 14:50:28 2010 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C406710656A4; Mon, 15 Feb 2010 14:50:28 +0000 (UTC) (envelope-from babkin@verizon.net) Received: from vms173005pub.verizon.net (vms173005pub.verizon.net [206.46.173.5]) by mx1.freebsd.org (Postfix) with ESMTP id A31D18FC22; Mon, 15 Feb 2010 14:50:28 +0000 (UTC) Received: from verizon.net ([unknown] [173.54.27.21]) by vms173005.mailsrvcs.net (Sun Java(tm) System Messaging Server 7u2-7.02 32bit (built Apr 16 2009)) with ESMTPA id <0KXV00L0DYFDI671@vms173005.mailsrvcs.net>; Mon, 15 Feb 2010 07:50:02 -0600 (CST) Sender: root Message-id: <4B79205B.619A0A1A@verizon.net> Date: Mon, 15 Feb 2010 05:22:19 -0500 From: Sergey Babkin X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 4.7-RELEASE i386) X-Accept-Language: en, ru MIME-version: 1.0 To: Maxim Sobolev References: <4B79297D.9080403@FreeBSD.org> Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7bit Cc: freebsd-net@FreeBSD.org, Jack Vogel , FreeBSD Hackers Subject: Re: Sudden mbuf demand increase and shortage under the load X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Feb 2010 14:50:28 -0000 Maxim Sobolev wrote: > > Hi, > > Our company have a FreeBSD based product that consists of the numerous > interconnected processes and it does some high-PPS UDP processing > (30-50K PPS is not uncommon). We are seeing some strange periodic > failures under the load in several such systems, which usually evidences > itself in IPC (even through unix domain sockets) suddenly either > breaking down or pausing and restoring only some time later (like 5-10 > minutes). The only sign of failure I managed to find was the increase of > the "requests for mbufs denied" in the netstat -m and number of total > mbuf clusters (nmbclusters) raising up to the limit. As a simple idea: UDP is not flow-controlled. So potentially nothing stops an application from sending the packets as fast as it can. If it's faster than the network card can process, they would start collecting. So this might be worth a try as a way to reproduce the problem and see if the system has a safeguard against it or not. Another possibility: what happens if a process is bound to an UDP socket but doesn't actually read the data from it? FreeBSD used to be pretty good at it, just throwing away the data beyond a certain limit, SVR4 was running out of network memory. But it might have changed, so might be worth a look too. -SB From owner-freebsd-net@FreeBSD.ORG Mon Feb 15 19:32:30 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 56650106566B; Mon, 15 Feb 2010 19:32:30 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from ey-out-2122.google.com (ey-out-2122.google.com [74.125.78.24]) by mx1.freebsd.org (Postfix) with ESMTP id B00008FC1C; Mon, 15 Feb 2010 19:32:29 +0000 (UTC) Received: by ey-out-2122.google.com with SMTP id 9so595356eyd.9 for ; Mon, 15 Feb 2010 11:32:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=CTd6o3RQPxGpvVMfATWSBaTca+9Zv0Z8mH4owPlV2h0=; b=kmSp2yg2gAVUfXFbQFkxMRyd8ffwPwzxzctoesfluQLcyecnMrryLb/30agU1pz72r VO3OVMIBXCH6ItldTrz8Js9uzXZAcwbJoFiu2YuQfsuKjuL0+jogkW3NlSC2T+oB/Nyu 4mjKiBobFLBHVr/VTRoaRtBJsK4jQeKb2qfAQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=qU6c6VlPFnfT0Xeg1qtWZIuSevoYitPVergJHMwaVT5JVoDk1UnHgBsed84HHTAGEA hu73eT5LeYqRXYSq/Dt/xLxGWtRyv8wn/vAUkf0JxzYNkLqt/MuHw+vIo0r1oTQEGSJC dBTEz2CuSMX7d6lvQYA12M4vUG0lz4A6KY65s= MIME-Version: 1.0 Received: by 10.216.157.1 with SMTP id n1mr3450290wek.188.1266262348281; Mon, 15 Feb 2010 11:32:28 -0800 (PST) In-Reply-To: <4B79297D.9080403@FreeBSD.org> References: <4B79297D.9080403@FreeBSD.org> Date: Mon, 15 Feb 2010 11:32:28 -0800 Message-ID: <2a41acea1002151132p3e58d4bu7adbbed527d5a81f@mail.gmail.com> From: Jack Vogel To: Maxim Sobolev Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org, FreeBSD Hackers Subject: Re: Sudden mbuf demand increase and shortage under the load X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Feb 2010 19:32:30 -0000 Can you tell me more about the system with the problem, does it have both em and igb driven interfaces? Jack 2010/2/15 Maxim Sobolev > Hi, > > Our company have a FreeBSD based product that consists of the numerous > interconnected processes and it does some high-PPS UDP processing (30-50K > PPS is not uncommon). We are seeing some strange periodic failures under the > load in several such systems, which usually evidences itself in IPC (even > through unix domain sockets) suddenly either breaking down or pausing and > restoring only some time later (like 5-10 minutes). The only sign of failure > I managed to find was the increase of the "requests for mbufs denied" in the > netstat -m and number of total mbuf clusters (nmbclusters) raising up to the > limit. > > I have tried to raise some network-related limits (most notably maxusers > and nmbclusters), but it has not helped with the issue - it's still > happening from time to time to us. Below you can find output from the > netstat -m few minutes right after that shortage period - you see that > somehow the system has allocated huge amount of memory for the network > (700MB), with only tiny amount of that being actually in use. This is for > the kern.ipc.nmbclusters: 302400. Eventually the system reclaims all that > memory and goes back to its normal use of 30-70MB. > > This problem is killing us, so any suggestions are greatly appreciated. My > current hypothesis is that due to some issues either with the network driver > or network subsystem itself, the system goes insane and "eats" up all mbufs > up to nmbclusters limit. But since mbufs are shared between network and > local IPC, IPC goes down as well. > > We observe this issue with systems using both em(4) driver and igb(4) > driver. I believe both drivers share the same design, however I am not sure > if this is some kind of design flaw in the driver or part of a larger > problem with the network subsystem. > > This happens on amd64 7.2-RELEASE and 7.3-PRERELEASE alike, with 8GB of > memory. I have not tried upgrading to 8.0, this is production system so > upgrading will not be easy. I don't believe there are some differences that > let us hope that this problem will go away after upgrade, but I can try it > as the last resort. > > As I said, this is very critical issue, so I can provide any additional > debug information upon request. We are ready to go as far as paying somebody > reasonable amount of money for tracking down and resolving the issue. > > Regards, > -- > Maksym Sobolyev > Sippy Software, Inc. > Internet Telephony (VoIP) Experts > T/F: +1-646-651-1110 > Web: http://www.sippysoft.com > MSN: sales@sippysoft.com > Skype: SippySoft > > > [ssp-root@ds-467 /usr/src]$ netstat -m > 17061/417669/434730 mbufs in use (current/cache/total) > 10420/291980/302400/302400 mbuf clusters in use (current/cache/total/max) > 10420/0 mbuf+clusters out of packet secondary zone in use (current/cache) > 19/1262/1281/51200 4k (page size) jumbo clusters in use > (current/cache/total/max) > 0/0/0/25600 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/12800 16k jumbo clusters in use (current/cache/total/max) > 25181K/693425K/718606K bytes allocated to network (current/cache/total) > 1246681/129567494/67681640 requests for mbufs denied > (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0/0/0 sfbufs in use (current/peak/max) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > 0 calls to protocol drain routines > > [FEW MINUTES LATER] > > [ssp-root@ds-467 /usr/src]$ netstat -m > 10001/84574/94575 mbufs in use (current/cache/total) > 6899/6931/13830/302400 mbuf clusters in use (current/cache/total/max) > 6899/6267 mbuf+clusters out of packet secondary zone in use (current/cache) > 2/1151/1153/51200 4k (page size) jumbo clusters in use > (current/cache/total/max) > 0/0/0/25600 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/12800 16k jumbo clusters in use (current/cache/total/max) > 16306K/39609K/55915K bytes allocated to network (current/cache/total) > 1246681/129567494/67681640 requests for mbufs denied > (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0/0/0 sfbufs in use (current/peak/max) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > 0 calls to protocol drain routines > From owner-freebsd-net@FreeBSD.ORG Mon Feb 15 21:11:45 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7309F106566B; Mon, 15 Feb 2010 21:11:45 +0000 (UTC) (envelope-from Albert.Shih@obspm.fr) Received: from smtp-int-m.obspm.fr (smtp-int-m.obspm.fr [145.238.187.15]) by mx1.freebsd.org (Postfix) with ESMTP id 0EDB88FC08; Mon, 15 Feb 2010 21:11:44 +0000 (UTC) Received: from obspm.fr (pcjas.obspm.fr [145.238.184.233]) by smtp-int-m.obspm.fr (8.14.3/8.14.3/SIO Observatoire de Paris - 07/2009) with ESMTP id o1FLBfMo009454 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 15 Feb 2010 22:11:43 +0100 Date: Mon, 15 Feb 2010 22:11:41 +0100 From: Albert Shih To: freebsd-net@freebsd.org, freebsd-pf@freebsd.org Message-ID: <20100215211141.GK96648@obspm.fr> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit User-Agent: Mutt/1.5.20 (2009-06-14) X-Miltered: at smtp-int-m.obspm.fr with ID 4B79B88D.000 by Joe's j-chkmail (http : // j-chkmail dot ensmp dot fr)! X-j-chkmail-Enveloppe: 4B79B88D.000/145.238.184.233/pcjas.obspm.fr/obspm.fr/ X-j-chkmail-Score: MSGID : 4B79B88D.000 on smtp-int-m.obspm.fr : j-chkmail score : . : R=. U=. O=. B=0.010 -> S=0.010 X-j-chkmail-Status: Ham Cc: Subject: Possible bug in TSO or in pf on bce X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Feb 2010 21:11:45 -0000 Hi all, I'm not a tcp/ip guru, so I don't known if it's a bug or not. The situation is little complexe, so I'm going to explain that. I've one server with tree interfaces two bce and one bge. All test is on two bce. This server running FreeBSD-7.2-p6 and have lot of jail (but the problem is the same for one jail, so I assume I've just one jail). The bce0 and bce1 are in different vlan. The jail is on bce1 (meaning the jail IP is on the bce1 subnet). The default gateway is on bce0 So to make all traffic of the jail pass only throught bce1 and not using bce0 I'm using pf with something like pass out route-to (bce1 bce1_subnet_gw) from jail_IP to ! bce1_subnet keep state pass in on bce1 reply-to (bce1 bce1_subnet_gw) from ! bce1_subnet to jail_IP keep state if I do that all traffic pass through the right interface (bce1), but...the bandwith drop to ~60kb/s (on gigabit interface). So I find the problem is with TSO, if I deactivated the TSO the bandwith is return to normal. I don't knwon if it's a bug in PF (the problem is same if I use scrub or not) or in the TSO support of bce. I can run some few tests if someone like to debug, but because the server is in production I cannot make lot of test. Regards. JAS -- Albert SHIH SIO batiment 15 Observatoire de Paris Meudon 5 Place Jules Janssen 92195 Meudon Cedex Téléphone : 01 45 07 76 26/06 86 69 95 71 Heure local/Local time: Lun 15 fév 2010 22:10:52 CET From owner-freebsd-net@FreeBSD.ORG Tue Feb 16 06:19:22 2010 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D18E21065676; Tue, 16 Feb 2010 06:19:22 +0000 (UTC) (envelope-from sobomax@FreeBSD.org) Received: from sippysoft.com (gk1.360sip.com [72.236.70.240]) by mx1.freebsd.org (Postfix) with ESMTP id 23B1E8FC1C; Tue, 16 Feb 2010 06:19:22 +0000 (UTC) Received: from [192.168.1.38] ([70.71.167.197]) (authenticated bits=0) by sippysoft.com (8.14.3/8.14.3) with ESMTP id o1G6JHTL017344 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 15 Feb 2010 22:19:18 -0800 (PST) (envelope-from sobomax@FreeBSD.org) Message-ID: <4B7A38F5.3090404@FreeBSD.org> Date: Mon, 15 Feb 2010 22:19:33 -0800 From: Maxim Sobolev Organization: Sippy Software, Inc. User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: Sergey Babkin References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> In-Reply-To: <4B79205B.619A0A1A@verizon.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Alfred Perlstein , freebsd-net@FreeBSD.org, "David G. Lawrence" , Jack Vogel , FreeBSD Hackers Subject: Re: Sudden mbuf demand increase and shortage under the load X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 06:19:23 -0000 Sergey Babkin wrote: > Maxim Sobolev wrote: >> Hi, >> >> Our company have a FreeBSD based product that consists of the numerous >> interconnected processes and it does some high-PPS UDP processing >> (30-50K PPS is not uncommon). We are seeing some strange periodic >> failures under the load in several such systems, which usually evidences >> itself in IPC (even through unix domain sockets) suddenly either >> breaking down or pausing and restoring only some time later (like 5-10 >> minutes). The only sign of failure I managed to find was the increase of >> the "requests for mbufs denied" in the netstat -m and number of total >> mbuf clusters (nmbclusters) raising up to the limit. > > As a simple idea: UDP is not flow-controlled. So potentially > nothing stops an application from sending the packets as fast > as it can. If it's faster than the network card can process, > they would start collecting. So this might be worth a try > as a way to reproduce the problem and see if the system has > a safeguard against it or not. > > Another possibility: what happens if a process is bound to > an UDP socket but doesn't actually read the data from it? > FreeBSD used to be pretty good at it, just throwing away > the data beyond a certain limit, SVR4 was running out of > network memory. But it might have changed, so might be > worth a look too. Thanks. Yes, the latter could be actually the case. The former is less likely since the system doesn't generate so much traffic by itself, but rather relays what it receives from the network pretty much in 1:1 ratio. It could happen though, if somehow the output path has been stalled. However, netstat -I igb0 shows zero Oerrs, which I guess means that we can rule that out too, unless there is some bug in the driver. So we are looking for potential issues that can cause UDP forwarding application to stall and not dequeue packets on time. So far we have identified some culprits in application logic that can cause such stalls in the unlikely event of gettimeofday() time going backwards. I've seen some messages from ntpd around the time of the problem, although it's unclear whether those are result of the that mbuf shortage or could indicate the root issue. We've also added some debug output to catch any abnormalities in the processing times. In any case I am a little bit surprised on how easy the FreeBSD can let mbuf storage to overflow. I'd expect it to be more aggressive in dropping things received from network once one application stalls. Combined with the fact that we apparently use shared storage for different kinds of network activity and perhaps IPC too, this gives an easy opportunity for DOS attacks. To me, separate limits for separate protocols or even classes of traffic (i.e. local/remote) would make much sense. Thanks to everybody for useful tips and suggestions, I will do more research along the lines and let you know once we either resolve the case or when I have more diagnostic output. Regards, -- Maksym Sobolyev Sippy Software, Inc. Internet Telephony (VoIP) Experts T/F: +1-646-651-1110 Web: http://www.sippysoft.com MSN: sales@sippysoft.com Skype: SippySoft From owner-freebsd-net@FreeBSD.ORG Tue Feb 16 11:29:56 2010 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8A904106566C for ; Tue, 16 Feb 2010 11:29:56 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) by mx1.freebsd.org (Postfix) with ESMTP id 1A9C58FC14 for ; Tue, 16 Feb 2010 11:29:56 +0000 (UTC) Received: from elsa.codelab.cz (localhost.codelab.cz [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 4A60A19E023; Tue, 16 Feb 2010 12:10:39 +0100 (CET) Received: from [192.168.1.2] (r5bb235.net.upc.cz [86.49.61.235]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id BCE5719E019; Tue, 16 Feb 2010 12:10:36 +0100 (CET) Message-ID: <4B7A7D2C.9040200@quip.cz> Date: Tue, 16 Feb 2010 12:10:36 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.7) Gecko/20100104 SeaMonkey/2.0.2 MIME-Version: 1.0 To: Maxim Sobolev References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> <4B7A38F5.3090404@FreeBSD.org> In-Reply-To: <4B7A38F5.3090404@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Alfred Perlstein , freebsd-net@FreeBSD.org, Sergey Babkin , "David G. Lawrence" , Jack Vogel Subject: Re: Sudden mbuf demand increase and shortage under the load X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 11:29:56 -0000 Maxim Sobolev wrote: > Sergey Babkin wrote: >> Maxim Sobolev wrote: >>> Hi, >>> >>> Our company have a FreeBSD based product that consists of the numerous >>> interconnected processes and it does some high-PPS UDP processing >>> (30-50K PPS is not uncommon). We are seeing some strange periodic >>> failures under the load in several such systems, which usually evidences >>> itself in IPC (even through unix domain sockets) suddenly either >>> breaking down or pausing and restoring only some time later (like 5-10 >>> minutes). The only sign of failure I managed to find was the increase of >>> the "requests for mbufs denied" in the netstat -m and number of total >>> mbuf clusters (nmbclusters) raising up to the limit. >> >> As a simple idea: UDP is not flow-controlled. So potentially >> nothing stops an application from sending the packets as fast as it >> can. If it's faster than the network card can process, >> they would start collecting. So this might be worth a try >> as a way to reproduce the problem and see if the system has >> a safeguard against it or not. >> >> Another possibility: what happens if a process is bound to >> an UDP socket but doesn't actually read the data from it? >> FreeBSD used to be pretty good at it, just throwing away >> the data beyond a certain limit, SVR4 was running out of >> network memory. But it might have changed, so might be >> worth a look too. > > Thanks. Yes, the latter could be actually the case. The former is less > likely since the system doesn't generate so much traffic by itself, but > rather relays what it receives from the network pretty much in 1:1 > ratio. It could happen though, if somehow the output path has been > stalled. However, netstat -I igb0 shows zero Oerrs, which I guess means > that we can rule that out too, unless there is some bug in the driver. > > So we are looking for potential issues that can cause UDP forwarding > application to stall and not dequeue packets on time. So far we have > identified some culprits in application logic that can cause such stalls > in the unlikely event of gettimeofday() time going backwards. I've seen > some messages from ntpd around the time of the problem, although it's > unclear whether those are result of the that mbuf shortage or could > indicate the root issue. We've also added some debug output to catch any > abnormalities in the processing times. > > In any case I am a little bit surprised on how easy the FreeBSD can let > mbuf storage to overflow. I'd expect it to be more aggressive in > dropping things received from network once one application stalls. > Combined with the fact that we apparently use shared storage for > different kinds of network activity and perhaps IPC too, this gives an > easy opportunity for DOS attacks. To me, separate limits for separate > protocols or even classes of traffic (i.e. local/remote) would make much > sense. Can it be related to this issue somehow? http://lists.freebsd.org/pipermail/freebsd-current/2009-August/011013.html http://lists.freebsd.org/pipermail/freebsd-current/2009-August/010740.html It was tested on FreeBSD 8 and high UDP traffic on igb interfaces emits messages "GET BUF: dmamap load failure - 12" and later results in kernel panic. We have not received any response to this report. Miroslav Lachman From owner-freebsd-net@FreeBSD.ORG Tue Feb 16 12:44:53 2010 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3DE601065672; Tue, 16 Feb 2010 12:44:53 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 15E588FC13; Tue, 16 Feb 2010 12:44:53 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1GCiqOg046649; Tue, 16 Feb 2010 12:44:52 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o1GCiqRH046645; Tue, 16 Feb 2010 12:44:52 GMT (envelope-from linimon) Date: Tue, 16 Feb 2010 12:44:52 GMT Message-Id: <201002161244.o1GCiqRH046645@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/144000: [tcp] setting TCP_MAXSEG by setsockopt() does not seem to have any effect X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 12:44:53 -0000 Old Synopsis: [tcp] ignore set TCP_MAXSEG by setsockopt() New Synopsis: [tcp] setting TCP_MAXSEG by setsockopt() does not seem to have any effect Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Tue Feb 16 12:43:48 UTC 2010 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=144000 From owner-freebsd-net@FreeBSD.ORG Tue Feb 16 12:53:34 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 61412106566B for ; Tue, 16 Feb 2010 12:53:34 +0000 (UTC) (envelope-from patrick.ale@gmail.com) Received: from mail-bw0-f223.google.com (mail-bw0-f223.google.com [209.85.218.223]) by mx1.freebsd.org (Postfix) with ESMTP id E53338FC1B for ; Tue, 16 Feb 2010 12:53:33 +0000 (UTC) Received: by bwz23 with SMTP id 23so1001023bwz.13 for ; Tue, 16 Feb 2010 04:53:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=cthkJhZ7v0VFKddQIHlBNRRYcXk4H0bIquepJnFs3zo=; b=hEaW8fcN+af5tjl9EHWshDIWsvoJa8DmQZqdcD7hsNOkFuX3nrejYQRnrijqlRQZwP 6zUYjknydJLNA5mngqVKu4HXnuY3KV9xpaw1D/e0MdFj+DSyKQkJINWbLNQ8TRk5nbvw IQjTU9Rtw8C1J9s8pe/P0Loy0sxvgm4MTZiio= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=QgMvXwKZRQJKWWU7B2yFxyP168mDCk3m6Yb5xiXbQ3zNWxwAM62Vnw90S/h698Jm4Z jlVTd5VYQgGH8OFXDgF6/SBllQ+qk3KsP4HVUhNjmCz421f090tlTz8MnpsXePsr9KoV UkSn6+tUSrrLzpcbWk+te2YCFm6taaGgdui7U= MIME-Version: 1.0 Received: by 10.204.49.73 with SMTP id u9mr104330bkf.105.1266323375534; Tue, 16 Feb 2010 04:29:35 -0800 (PST) Date: Tue, 16 Feb 2010 13:29:35 +0100 Message-ID: <8d158e1f1002160429m747efaf1h478b6b9cf00b96c@mail.gmail.com> From: Patrick Ale To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Attansic L1 Gigabit discovered on install but no link X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 12:53:34 -0000 Good afternoon, I tried to install FreeBSD 8.0 about 30 minutes ago and during the installation I encountered a problem. My network card is detected but on DHCP Server discoverage I see that no link is detected and that the driver/dhcp discovery process is giving up on trying. I understoodthat the Attansic L1 Gigabit card's support is sort of problematic due to the 'openness' of the documentation but I'd really like to know how I can help or what I can do to get this driver working on my Acer Aspire 8920 laptop. Buying another NIC for this particular thing is not really an option. Any suggestion is welcome, my C programming skills are not really to write home about so I am afraid I can't contribute much on that point but if you need test dummies or feel like I can help out or if you need more information, please let me know. Thank you for your time in advance, Patrick From owner-freebsd-net@FreeBSD.ORG Tue Feb 16 17:04:09 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C7DFF106566B for ; Tue, 16 Feb 2010 17:04:09 +0000 (UTC) (envelope-from serena.zanetta@guest.telecomitalia.it) Received: from GRFEDG702BA020.telecomitalia.it (grfedg702ba020.telecomitalia.it [156.54.233.201]) by mx1.freebsd.org (Postfix) with ESMTP id 516BB8FC12 for ; Tue, 16 Feb 2010 17:04:09 +0000 (UTC) Received: from GRFHUB701BA020.griffon.local (10.188.101.111) by GRFEDG702BA020.telecomitalia.it (10.188.45.101) with Microsoft SMTP Server (TLS) id 8.2.176.0; Tue, 16 Feb 2010 17:53:17 +0100 Received: from GRFMBX706BA020.griffon.local ([10.188.101.19]) by grfhub701ba020.griffon.local ([10.188.101.111]) with mapi; Tue, 16 Feb 2010 17:53:17 +0100 From: Zanetta Serena To: "'freebsd-net@freebsd.org'" Date: Tue, 16 Feb 2010 17:53:16 +0100 Thread-Topic: RAW socket under netgraph Thread-Index: AcqvKJRGrMcJvnydRamEUvChCuSkDw== Message-ID: <67B1924593CA5A4AB2C1BEE195D24F97AB1F4AD5@GRFMBX706BA020.griffon.local> Accept-Language: it-IT, en-US Content-Language: it-IT X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: it-IT, en-US MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: RAW socket under netgraph X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 17:04:09 -0000 Hi, I am trying to implement the raw socket mechanism under Netgraph by setting= the ng_ksocket node type as a raw socket. My purpose is to fill "by hand" = the IP and UDP headers and then forward the so created packet to the raw so= cket to be delivered to the appropriate next hop. I suppose I need to set the socket options as: setsockopt(sd,IPPROTO_IP,IP_HDRINCL,...). I found that the equivalent in the Netgraph is the control message "setopt"= . I've tried to set the options as: ngctl msg node_name: setopt {level=3DIPPROTO_IP name=3DIP_HDRINCL value=3D= } but it doesn't work. Also I don't know exactly which is the value of the "v= alue" parameter. I would ask you how to set the options for the raw socket = and if it is correct to type that command line. Thank you, Serena From owner-freebsd-net@FreeBSD.ORG Tue Feb 16 17:23:25 2010 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8CAD6106566B; Tue, 16 Feb 2010 17:23:25 +0000 (UTC) (envelope-from sobomax@FreeBSD.org) Received: from sippysoft.com (gk1.360sip.com [72.236.70.240]) by mx1.freebsd.org (Postfix) with ESMTP id 52D5D8FC1F; Tue, 16 Feb 2010 17:23:24 +0000 (UTC) Received: from [192.168.1.38] ([70.71.167.197]) (authenticated bits=0) by sippysoft.com (8.14.3/8.14.3) with ESMTP id o1GHNHcl022112 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 16 Feb 2010 09:23:21 -0800 (PST) (envelope-from sobomax@FreeBSD.org) Message-ID: <4B7AD495.30900@FreeBSD.org> Date: Tue, 16 Feb 2010 09:23:33 -0800 From: Maxim Sobolev Organization: Sippy Software, Inc. User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: Miroslav Lachman <000.fbsd@quip.cz> References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> <4B7A38F5.3090404@FreeBSD.org> <4B7A7D2C.9040200@quip.cz> In-Reply-To: <4B7A7D2C.9040200@quip.cz> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Alfred Perlstein , freebsd-net@FreeBSD.org, Sergey Babkin , "David G. Lawrence" , Jack Vogel Subject: Re: Sudden mbuf demand increase and shortage under the load X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 17:23:25 -0000 Miroslav Lachman wrote: > Can it be related to this issue somehow? > > http://lists.freebsd.org/pipermail/freebsd-current/2009-August/011013.html > http://lists.freebsd.org/pipermail/freebsd-current/2009-August/010740.html > > It was tested on FreeBSD 8 and high UDP traffic on igb interfaces emits > messages "GET BUF: dmamap load failure - 12" and later results in kernel > panic. > We have not received any response to this report. Could be the issue, however in our case there is no panic, just that all userland activity in the system ceases for 2 minutes after it reaches certain network load level. -Maxim From owner-freebsd-net@FreeBSD.ORG Tue Feb 16 17:45:27 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3464B106566B for ; Tue, 16 Feb 2010 17:45:27 +0000 (UTC) (envelope-from freebsd@insightbb.com) Received: from mxsf13.insightbb.com (mxsf13.insightbb.com [74.128.0.95]) by mx1.freebsd.org (Postfix) with ESMTP id EE2278FC16 for ; Tue, 16 Feb 2010 17:45:26 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.49,485,1262581200"; d="scan'208";a="44057007" Received: from unknown (HELO asav02.insightbb.com) ([172.31.249.123]) by mxsf13.insightbb.com with ESMTP; 16 Feb 2010 12:45:25 -0500 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApsEAKZoekvQLicL/2dsb2JhbACcDb4FhFsEgxQagl4 X-IronPort-AV: E=Sophos;i="4.49,485,1262581200"; d="scan'208";a="347826753" Received: from 208-46-39-11.dia.static.qwest.net (HELO laptop2.stevenfriedrich.org) ([208.46.39.11]) by asavout02.manage.insightbb.com with ESMTP; 16 Feb 2010 12:45:24 -0500 From: Steven Friedrich To: freebsd-net@freebsd.org Date: Tue, 16 Feb 2010 12:45:16 -0500 User-Agent: KMail/1.12.4 (FreeBSD/8.0-STABLE; KDE/4.3.5; i386; ; ) References: <8d158e1f1002160429m747efaf1h478b6b9cf00b96c@mail.gmail.com> In-Reply-To: <8d158e1f1002160429m747efaf1h478b6b9cf00b96c@mail.gmail.com> X-Face: i~b2iK'Z*tJ)pO9@6lJG=k7>N, V~YMq":Iwdl!m|A"g, N@)'|zb[{ Cc: Patrick Ale Subject: Re: Attansic L1 Gigabit discovered on install but no link X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 17:45:27 -0000 On Tuesday 16 February 2010 07:29:35 am Patrick Ale wrote: > Good afternoon, > > I tried to install FreeBSD 8.0 about 30 minutes ago and during the > installation I encountered a problem. > > My network card is detected but on DHCP Server discoverage I see that > no link is detected and that the driver/dhcp discovery process is > giving up on trying. > > I understoodthat the Attansic L1 Gigabit card's support is sort of > problematic due to the 'openness' of the documentation but I'd really > like to know how I can help or what I can do to get this driver > working on my Acer Aspire 8920 laptop. Buying another NIC for this > particular thing is not really an option. I don't want to stop you from pursuing getting this NIC working, but why don't you have an option here? I would suspect that your laptop has USB ports and Netgear and others make USB Ethernet in wired and wireless versions. I haven't checked to see if any are gigabit speed, but connecting at a slower speed beats no connection. 8o) You do have to select one carefully though. I bought a Netgear WG111T (wireless) several years ago, but it's only recently that a driver is available in FreeBSD, and mine doesn't work yet. A developer did buy one and made some changes, but I'm not sure all the issues are resolved just yet. But I have a Netgear FA-120 (I think)(wired) and it works just fine. I also have a couple LinkSys (wired), but I've lost confidence in LinkSys. I won't buy any of their products again. > Any suggestion is welcome, my C programming skills are not really to > write home about so I am afraid I can't contribute much on that point > but if you need test dummies or feel like I can help out or if you > need more information, please let me know. > > > Thank you for your time in advance, > > > Patrick > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Tue Feb 16 18:11:06 2010 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3D53A1065698; Tue, 16 Feb 2010 18:11:06 +0000 (UTC) (envelope-from sobomax@FreeBSD.org) Received: from sippysoft.com (gk1.360sip.com [72.236.70.240]) by mx1.freebsd.org (Postfix) with ESMTP id B1C918FC13; Tue, 16 Feb 2010 18:11:05 +0000 (UTC) Received: from [192.168.1.38] ([70.71.167.197]) (authenticated bits=0) by sippysoft.com (8.14.3/8.14.3) with ESMTP id o1GIB2Zj022403 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 16 Feb 2010 10:11:02 -0800 (PST) (envelope-from sobomax@FreeBSD.org) Message-ID: <4B7ADFC6.7020202@FreeBSD.org> Date: Tue, 16 Feb 2010 10:11:18 -0800 From: Maxim Sobolev Organization: Sippy Software, Inc. User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: Sergey Babkin References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> In-Reply-To: <4B79205B.619A0A1A@verizon.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Alfred Perlstein , freebsd-net@FreeBSD.org, "David G. Lawrence" , Jack Vogel , FreeBSD Hackers Subject: Re: Sudden mbuf demand increase and shortage under the load (igb issue?) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 18:11:06 -0000 OK, here is some new data that I think rules out any issues with the applications. Following Alfred's suggestion I have made a script to run every second and output some system statistics: date netstat -m vmstat -i ps -axl pstat -T vmstat -z sysctl -a The problem had hit us again today several times and upon investigating the log I found that increase in the mbuf usage happened in one step - going from normal 10% to 100% between two script runs. What is more interesting, is that time from two such subsequent runs were about 2 minutes apart (instead of 1 second as it should be) and when inspecting cron logs I noticed the same time gap in there. I ruled out any VM starvation as a cause of the delay because system has plenty of free memory. The incoming network traffic was not sufficient to starve VM so quickly either - it was about 7MB/sec at that time, so even if all receivers stopped draining their buffers it should have taken at least 1-2 seconds to fill up mbuf cache and create demand for an additional kernel memory. The failure would likely to be more gradual and I should have seen how it builds up in the debug log. So it looks like kernel issue of a sort, which causes all userland activity to cease for 2 minutes when the system reaches certain load. Mbuf build-up is only the by-product of this, not really a cause. igb(4) is being the primary suspect now, since we have other machines with more load not having this problem and we don't have anybody else using this driver. The chip is the following: igb0@pci0:5:0:0: class=0x020000 card=0x323f103c chip=0x10c98086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' class = network subclass = ethernet igb1@pci0:5:0:1: class=0x020000 card=0x323f103c chip=0x10c98086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' class = network subclass = ethernet Hardware in question is a new HP DL160G6. I have also checked IPMI logs and sensors and have not found any issue in there as well. No sensors reported off-range values and chassis temperature is within normal limits. I am not sure how to debug this problem further. We are now investigating opportunity to install external non-igb card to the server and see if it solves the issue. I have the whole log if anyone wants to take a closer peek. Regards, -- Maksym Sobolyev Sippy Software, Inc. Internet Telephony (VoIP) Experts T/F: +1-646-651-1110 Web: http://www.sippysoft.com MSN: sales@sippysoft.com Skype: SippySoft From owner-freebsd-net@FreeBSD.ORG Tue Feb 16 18:13:57 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 65A291065672 for ; Tue, 16 Feb 2010 18:13:57 +0000 (UTC) (envelope-from patrick.ale@gmail.com) Received: from mail-bw0-f223.google.com (mail-bw0-f223.google.com [209.85.218.223]) by mx1.freebsd.org (Postfix) with ESMTP id E8D128FC17 for ; Tue, 16 Feb 2010 18:13:56 +0000 (UTC) Received: by bwz23 with SMTP id 23so1322557bwz.13 for ; Tue, 16 Feb 2010 10:13:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=Cvj8FSYgdyOxtIs1+csIbcCUr4yxL0pbSuaMKzfGoRM=; b=tkchWruP6K8UMSQJCqYH/hVcDGYmZJnrzi9yCmHydcGeQJ7tzcEyVC7ud830zFr0K1 aPLV4+SO5XGXhOfXEpRs4x9EhvU4ha6LevtnkNrUBFfQYmlTw6b3hk7YWeHFmciDFWGH iCuuhAm+cnDYc7fouGgYzRVsfUsSMKfZgwyIo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=L2nIsdOJVwIy7VP/yyVsSfjqP5rWvfEMc7pZdWDSOVTjP2fZ3L2NY1Re1NJmbHQp+P hd3jh7+EMvPZY46aR14h1z7cebmfZtI1g5dVI5VEkR8uAx2XFdfiaNz3H2BnF+NUwJLm QV6W5KqXJB50ImVTVPNQMG3o14a/whyCKSA74= MIME-Version: 1.0 Received: by 10.204.36.71 with SMTP id s7mr4465497bkd.171.1266344035767; Tue, 16 Feb 2010 10:13:55 -0800 (PST) In-Reply-To: <201002161245.17047.freebsd@insightbb.com> References: <8d158e1f1002160429m747efaf1h478b6b9cf00b96c@mail.gmail.com> <201002161245.17047.freebsd@insightbb.com> Date: Tue, 16 Feb 2010 19:13:55 +0100 Message-ID: <8d158e1f1002161013m5eaa6e84n24e5e646053aca2b@mail.gmail.com> From: Patrick Ale To: Steven Friedrich Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org Subject: Re: Attansic L1 Gigabit discovered on install but no link X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 18:13:57 -0000 On Tue, Feb 16, 2010 at 6:45 PM, Steven Friedrich w= rote: > On Tuesday 16 February 2010 07:29:35 am Patrick Ale wrote: > I don't want to stop you from pursuing getting this NIC working, but why = don't > you have an option here? =A0I would suspect that your laptop has USB port= s and > Netgear and others make USB Ethernet in wired and wireless versions. I ha= ven't > checked to see if any are gigabit speed, but connecting at a slower speed > beats no connection. 8o) Hi, You have a good point there actually, I could buy a new card for USB or PCI-E if the need was really high. Then again, I run two other OS-es on this laptop without network problems so it would be more economical to just boot to Solaris 10 or Linux when I need network. It's not a biggie for me if the network card doesn't work under FreeBSD, I want to mess with it localy mainly. Then again, if there is a driver shipped with an operating system and it doesn't seem to work, I'd like to report it and see how I can help out. So yea, just to make it very clear, I am not in a rush or will go completely beserk like I see some people do by reading back the mailing archives cause "their newest 10gigabit Wantastic Ethernet card with coffee making capabilities from late 2009 doesn't work under the latest FreeBSD release and how come it works under Windows, FreeBSD is imature" ;-) On a serious note though, there is a driver shipped with FreeBSD 8.0 and I assume it is there cause it's believed to work. Any pointer to getting this card would be highly appreciated, kernel reconfiguration hints, patches, external code, I am all in for it. Thanks all! Patrick From owner-freebsd-net@FreeBSD.ORG Tue Feb 16 18:20:36 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1CEA0106566B for ; Tue, 16 Feb 2010 18:20:36 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by mx1.freebsd.org (Postfix) with ESMTP id DD3AA8FC1D for ; Tue, 16 Feb 2010 18:20:35 +0000 (UTC) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.14.3/8.14.3) with ESMTP id o1GIKYlX011517; Tue, 16 Feb 2010 13:20:34 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <201002161820.o1GIKYlX011517@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Tue, 16 Feb 2010 13:20:52 -0500 To: Patrick Ale , freebsd-net@freebsd.org From: Mike Tancsa In-Reply-To: <8d158e1f1002160429m747efaf1h478b6b9cf00b96c@mail.gmail.com > References: <8d158e1f1002160429m747efaf1h478b6b9cf00b96c@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Cc: Subject: Re: Attansic L1 Gigabit discovered on install but no link X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 18:20:36 -0000 At 07:29 AM 2/16/2010, Patrick Ale wrote: >Good afternoon, > >I tried to install FreeBSD 8.0 about 30 minutes ago and during the >installation I encountered a problem. > >My network card is detected but on DHCP Server discoverage I see that >no link is detected and that the driver/dhcp discovery process is >giving up on trying. Hi, Try finishing off the install and then see if you can manually get it working. I have see in the past some drivers bugs causing difficulty with DHCP. Perhaps try and manually set the PHY speed and IP address. I dont see any obvious fixes that went into the tree Post 8.0R, but if you can get things partially working, or enough to file a PR, someone might be able to fix things up from there. What does ifconfig -a show ? also pciconf -lvc ...for the part that shows the nic driver. If its the ae driver, try ifconfig ae0 media 10baseT/UTP up and then dhclient ae0 or manually set the IP address and see if that works. ---Mike -------------------------------------------------------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet since 1994 www.sentex.net Cambridge, Ontario Canada www.sentex.net/mike From owner-freebsd-net@FreeBSD.ORG Tue Feb 16 18:23:49 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D206A106566C for ; Tue, 16 Feb 2010 18:23:49 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 63AAC8FC13 for ; Tue, 16 Feb 2010 18:23:49 +0000 (UTC) Received: by vws20 with SMTP id 20so426388vws.13 for ; Tue, 16 Feb 2010 10:23:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=T7lzqGxlcA07Upfmcp5lJ+DReoD3esjudY94WvNZzeE=; b=T163rQByPfN9blW72VAl04TbpdMY+2yeLqk2rDCjVTo1H9rURrR3OYk3Da+gm+OcxU n5PMZyW+8UVWNh0DGuE/XwzgrhbMjlsaJFCzMYQgf+6brOth37I5XDyb3UJe4pvp+RdD /iFxuiHzdMy3XRktKVLeI17W/uL0HgXt9N9Q8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=GQ7bNfdW6PtFsWykW7/XwwK/+dMVtSK+B0QiNJfehn592a+GUp9X4nhjt5y6p6w32L aTSnJHs77odcCze2YQwNc/DKuwCYjneH7pcjcmNSKdvPz62gTKLgh3bDh1OqhA9xfj2S abhuQEM2SAyg6KWTrEsS1ZYFX3Qwlrvi2MX5w= Received: by 10.220.124.135 with SMTP id u7mr1095185vcr.231.1266344628532; Tue, 16 Feb 2010 10:23:48 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 28sm9456522vws.11.2010.02.16.10.23.46 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 16 Feb 2010 10:23:47 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Tue, 16 Feb 2010 10:23:06 -0800 From: Pyun YongHyeon Date: Tue, 16 Feb 2010 10:23:06 -0800 To: Albert Shih Message-ID: <20100216182306.GC1394@michelle.cdnetworks.com> References: <20100215211141.GK96648@obspm.fr> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="9jxsPFA5p3P2qPhR" Content-Disposition: inline In-Reply-To: <20100215211141.GK96648@obspm.fr> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org, freebsd-pf@freebsd.org Subject: Re: Possible bug in TSO or in pf on bce X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 18:23:50 -0000 --9jxsPFA5p3P2qPhR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Mon, Feb 15, 2010 at 10:11:41PM +0100, Albert Shih wrote: > Hi all, > > I'm not a tcp/ip guru, so I don't known if it's a bug or not. > > The situation is little complexe, so I'm going to explain that. > > I've one server with tree interfaces two bce and one bge. All test is on > two bce. > > This server running FreeBSD-7.2-p6 and have lot of jail (but the problem is > the same for one jail, so I assume I've just one jail). The bce0 and bce1 > are in different vlan. > > The jail is on bce1 (meaning the jail IP is on the bce1 subnet). > > The default gateway is on bce0 > > So to make all traffic of the jail pass only throught bce1 and not using > bce0 I'm using pf with something like > > pass out route-to (bce1 bce1_subnet_gw) from jail_IP to ! bce1_subnet keep state > pass in on bce1 reply-to (bce1 bce1_subnet_gw) from ! bce1_subnet to jail_IP keep state > > if I do that all traffic pass through the right interface (bce1), but...the > bandwith drop to ~60kb/s (on gigabit interface). > > So I find the problem is with TSO, if I deactivated the TSO the bandwith is > return to normal. > > I don't knwon if it's a bug in PF (the problem is same if I use scrub or > not) or in the TSO support of bce. > At first I thought you hit one of edge case of TSO on bce(4). But it seems the issue comes from pf's route handling. When I ported pf from OpenBSD, there was no TSO capability in FreeBSD at that time so the pf_route() had no special handling code for TSO. Since it was long time ago I'm not sure whether it's correct or not but try attached patch. Apart from TSO FreeBSD got several new features like fib, flow-table and vnet. We may need to check whether these new features are still working with pf(4). --9jxsPFA5p3P2qPhR Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="pf.routeto.patch" Index: sys/contrib/pf/net/pf.c =================================================================== --- sys/contrib/pf/net/pf.c (revision 203960) +++ sys/contrib/pf/net/pf.c (working copy) @@ -6375,6 +6375,7 @@ m0->m_pkthdr.csum_flags &= ifp->if_hwassist; if (ntohs(ip->ip_len) <= ifp->if_mtu || + (m0->m_pkthdr.csum_flags & ifp->if_hwassist & CSUM_TSO) != 0 || (ifp->if_hwassist & CSUM_FRAGMENT && ((ip->ip_off & htons(IP_DF)) == 0))) { /* @@ -6449,7 +6450,7 @@ * Too large for interface; fragment if possible. * Must be able to put at least 8 bytes per fragment. */ - if (ip->ip_off & htons(IP_DF)) { + if (ip->ip_off & htons(IP_DF) || (m0->m_pkthdr.csum_flags & CSUM_TSO)) { KMOD_IPSTAT_INC(ips_cantfrag); if (r->rt != PF_DUPTO) { #ifdef __FreeBSD__ --9jxsPFA5p3P2qPhR-- From owner-freebsd-net@FreeBSD.ORG Tue Feb 16 18:36:16 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A31B81065670 for ; Tue, 16 Feb 2010 18:36:16 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-qy0-f189.google.com (mail-qy0-f189.google.com [209.85.221.189]) by mx1.freebsd.org (Postfix) with ESMTP id 11D538FC14 for ; Tue, 16 Feb 2010 18:36:14 +0000 (UTC) Received: by qyk27 with SMTP id 27so4105084qyk.3 for ; Tue, 16 Feb 2010 10:36:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=dpb+I3v9VN6IKRzKyn9pT9+9hQklNLV1Vfoohui8p+Y=; b=BRBTHpLNbEKH+/N0qlTSksu4vBRtS6HtM3YqEqwnFGljqbNUr5GRmZvvoNJfw2PCGm sOrtgJRkr++DMpe+bPOReFhAs4ZDVzLAggkovgFtybb78+ljyjTPlx0nXXmwdBFjxckw vmQZtU7R6xLU3q3drfqtLyn47jrmfH+VuG2GU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=D3l25FT+zAyZZ3rd37BR6OrwkFUnFnr77ZcAB+kl5PutI6qUISPUK3jbCSbubcDrBI tubF2ypSg4ZuROx2XB1FbeG4Gbs1AJaIUj+Z9olWROYljIAIvd4bzrxXdFMFcVjs3uN+ xG+znjV9CTli92BudICPLjFR9JB3HUhW54a+w= Received: by 10.224.140.144 with SMTP id i16mr1069537qau.149.1266345374112; Tue, 16 Feb 2010 10:36:14 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 5sm3514924qwg.58.2010.02.16.10.36.11 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 16 Feb 2010 10:36:12 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Tue, 16 Feb 2010 10:35:31 -0800 From: Pyun YongHyeon Date: Tue, 16 Feb 2010 10:35:31 -0800 To: Patrick Ale Message-ID: <20100216183531.GD1394@michelle.cdnetworks.com> References: <8d158e1f1002160429m747efaf1h478b6b9cf00b96c@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8d158e1f1002160429m747efaf1h478b6b9cf00b96c@mail.gmail.com> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org Subject: Re: Attansic L1 Gigabit discovered on install but no link X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 18:36:16 -0000 On Tue, Feb 16, 2010 at 01:29:35PM +0100, Patrick Ale wrote: > Good afternoon, > > I tried to install FreeBSD 8.0 about 30 minutes ago and during the > installation I encountered a problem. > > My network card is detected but on DHCP Server discoverage I see that > no link is detected and that the driver/dhcp discovery process is > giving up on trying. > I assume you're using age(4). age(4) is one of driver that still does full re-initialization while dhclient(8) is running, how about unplugging the UTP cable and replug it while DHCP is in progress? > I understoodthat the Attansic L1 Gigabit card's support is sort of > problematic due to the 'openness' of the documentation but I'd really > like to know how I can help or what I can do to get this driver > working on my Acer Aspire 8920 laptop. Buying another NIC for this > particular thing is not really an option. > There are several variants of L1 so please show me the dmesg output. > Any suggestion is welcome, my C programming skills are not really to > write home about so I am afraid I can't contribute much on that point > but if you need test dummies or feel like I can help out or if you > need more information, please let me know. > > > Thank you for your time in advance, > > > Patrick > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Feb 16 18:46:46 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A3C1D106568B for ; Tue, 16 Feb 2010 18:46:46 +0000 (UTC) (envelope-from julian@elischer.org) Received: from out-0.mx.aerioconnect.net (out-0-3.mx.aerioconnect.net [216.240.47.63]) by mx1.freebsd.org (Postfix) with ESMTP id 840CE8FC22 for ; Tue, 16 Feb 2010 18:46:46 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o1GIkjYS017278; Tue, 16 Feb 2010 10:46:45 -0800 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id AC2CB2D6013; Tue, 16 Feb 2010 10:46:44 -0800 (PST) Message-ID: <4B7AE844.3090007@elischer.org> Date: Tue, 16 Feb 2010 10:47:32 -0800 From: Julian Elischer User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: pyunyh@gmail.com References: <20100215211141.GK96648@obspm.fr> <20100216182306.GC1394@michelle.cdnetworks.com> In-Reply-To: <20100216182306.GC1394@michelle.cdnetworks.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: Albert Shih , freebsd-pf@freebsd.org, freebsd-net@freebsd.org Subject: Re: Possible bug in TSO or in pf on bce X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 18:46:46 -0000 Pyun YongHyeon wrote: > On Mon, Feb 15, 2010 at 10:11:41PM +0100, Albert Shih wrote: >> Hi all, >> >> I'm not a tcp/ip guru, so I don't known if it's a bug or not. >> >> The situation is little complexe, so I'm going to explain that. >> >> I've one server with tree interfaces two bce and one bge. All test is on >> two bce. >> >> This server running FreeBSD-7.2-p6 and have lot of jail (but the problem is >> the same for one jail, so I assume I've just one jail). The bce0 and bce1 >> are in different vlan. >> >> The jail is on bce1 (meaning the jail IP is on the bce1 subnet). >> >> The default gateway is on bce0 >> >> So to make all traffic of the jail pass only throught bce1 and not using >> bce0 I'm using pf with something like >> >> pass out route-to (bce1 bce1_subnet_gw) from jail_IP to ! bce1_subnet keep state >> pass in on bce1 reply-to (bce1 bce1_subnet_gw) from ! bce1_subnet to jail_IP keep state >> >> if I do that all traffic pass through the right interface (bce1), but...the >> bandwith drop to ~60kb/s (on gigabit interface). >> >> So I find the problem is with TSO, if I deactivated the TSO the bandwith is >> return to normal. >> >> I don't knwon if it's a bug in PF (the problem is same if I use scrub or >> not) or in the TSO support of bce. >> > > At first I thought you hit one of edge case of TSO on bce(4). But > it seems the issue comes from pf's route handling. When I ported pf > from OpenBSD, there was no TSO capability in FreeBSD at that time > so the pf_route() had no special handling code for TSO. Since it > was long time ago I'm not sure whether it's correct or not but try > attached patch. > > Apart from TSO FreeBSD got several new features like fib, > flow-table and vnet. We may need to check whether these new > features are still working with pf(4). yes, in 8.0 you have options ot do what you want to do in several ways. From a quick look, both multi-FIBs and vnet may be directly applicable to you. As for pf, it works with multi fibs but the patch for vnet depends upon teh installation of a newer revision of pf and that may not be possible in 8.x. (ceri may be able so shed more light on that, I'm assuming he's seeing this.) > > > ------------------------------------------------------------------------ > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Feb 16 18:54:25 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 73C65106566B; Tue, 16 Feb 2010 18:54:25 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-qy0-f173.google.com (mail-qy0-f173.google.com [209.85.221.173]) by mx1.freebsd.org (Postfix) with ESMTP id D46EB8FC17; Tue, 16 Feb 2010 18:54:24 +0000 (UTC) Received: by qyk4 with SMTP id 4so3869244qyk.8 for ; Tue, 16 Feb 2010 10:54:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=eGqe6sls0r5N+1WduLvK0Ddg5D6lF51v1ED33UHP8rw=; b=eKKYoVFlAZvXJjbUxtcMMO1HBthocSEcQSCvHmIV9bX/wgrc6YxxNfK77OdeQFF/6R bgRvHfTDSGYIEtnP5rowhhb7pEBR8RKPcGeTfEbPd9/dChmShBbOQdjqArmuV/Ww2YHd N4K4ygKTjiPf41hO0X6aCPGBK3pDYJlOL356k= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=bfl1XIQjlmjHZ3RzoN5QWBsfgx2G+1+hvm3Ai3jmXviD9pongetVJnWBl3U/wEUIl1 8Bd066MdhDuH/OvaNdtElml9rNO+xwNDBCftZ/heob7oMRjJtxF0TK2BWO9yQVJF7Nh4 /N5FdpbNlTeDaWQY6JVvGgPN+UHM8IkEPe1gs= Received: by 10.224.107.77 with SMTP id a13mr955235qap.312.1266346463767; Tue, 16 Feb 2010 10:54:23 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 4sm19427854qwe.33.2010.02.16.10.54.21 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 16 Feb 2010 10:54:22 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Tue, 16 Feb 2010 10:53:41 -0800 From: Pyun YongHyeon Date: Tue, 16 Feb 2010 10:53:41 -0800 To: Maxim Sobolev Message-ID: <20100216185341.GE1394@michelle.cdnetworks.com> References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> <4B7ADFC6.7020202@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B7ADFC6.7020202@FreeBSD.org> User-Agent: Mutt/1.4.2.3i Cc: Sergey Babkin , freebsd-net@freebsd.org, Alfred Perlstein , Jack Vogel , FreeBSD Hackers , "David G. Lawrence" Subject: Re: Sudden mbuf demand increase and shortage under the load (igb issue?) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 18:54:25 -0000 On Tue, Feb 16, 2010 at 10:11:18AM -0800, Maxim Sobolev wrote: > OK, here is some new data that I think rules out any issues with the > applications. Following Alfred's suggestion I have made a script to run > every second and output some system statistics: > > date > netstat -m > vmstat -i > ps -axl > pstat -T > vmstat -z > sysctl -a > > The problem had hit us again today several times and upon investigating > the log I found that increase in the mbuf usage happened in one step - > going from normal 10% to 100% between two script runs. What is more > interesting, is that time from two such subsequent runs were about 2 > minutes apart (instead of 1 second as it should be) and when inspecting > cron logs I noticed the same time gap in there. I ruled out any VM > starvation as a cause of the delay because system has plenty of free > memory. The incoming network traffic was not sufficient to starve VM so > quickly either - it was about 7MB/sec at that time, so even if all > receivers stopped draining their buffers it should have taken at least > 1-2 seconds to fill up mbuf cache and create demand for an additional > kernel memory. The failure would likely to be more gradual and I should > have seen how it builds up in the debug log. > > So it looks like kernel issue of a sort, which causes all userland > activity to cease for 2 minutes when the system reaches certain load. > Mbuf build-up is only the by-product of this, not really a cause. igb(4) > is being the primary suspect now, since we have other machines with more > load not having this problem and we don't have anybody else using this > driver. The chip is the following: > > igb0@pci0:5:0:0: class=0x020000 card=0x323f103c chip=0x10c98086 > rev=0x01 hdr=0x00 > vendor = 'Intel Corporation' > class = network > subclass = ethernet > igb1@pci0:5:0:1: class=0x020000 card=0x323f103c chip=0x10c98086 > rev=0x01 hdr=0x00 > vendor = 'Intel Corporation' > class = network > subclass = ethernet > > Hardware in question is a new HP DL160G6. I have also checked IPMI logs > and sensors and have not found any issue in there as well. No sensors > reported off-range values and chassis temperature is within normal limits. > > I am not sure how to debug this problem further. We are now > investigating opportunity to install external non-igb card to the server > and see if it solves the issue. > I know there were a couple of igb(4) issues in 7.x but it seems recent fixes didn't make it into stable/7 and 7.3-RELEASE. But the issues I know does not explain the symptom of your issue. One of issues that could be related with the issue was igb(4) took a lot of CPU cycles as it incorrectly rescheduled to get more frames instead of dropping some frames when heavy UDP traffic hits the controller(e.g. 64bytes UDP torture test). > I have the whole log if anyone wants to take a closer peek. > From owner-freebsd-net@FreeBSD.ORG Tue Feb 16 18:56:43 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8795D10656A4 for ; Tue, 16 Feb 2010 18:56:43 +0000 (UTC) (envelope-from jamesbrandongooch@gmail.com) Received: from mail-ew0-f214.google.com (mail-ew0-f214.google.com [209.85.219.214]) by mx1.freebsd.org (Postfix) with ESMTP id 20B658FC1F for ; Tue, 16 Feb 2010 18:56:42 +0000 (UTC) Received: by ewy6 with SMTP id 6so1307665ewy.14 for ; Tue, 16 Feb 2010 10:56:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:cc:content-type; bh=3Yz2Fsz5OEW6pAUdVG0eUtY4VTn0O6Nmvd4QW1rUfyo=; b=s3/4gOHlX6OVodnEYKwIp2PBiuYcZd9Y6ZIMvJX3o78CmOrc1MQ/dTe0HGRtl4yqYf YFrv91/IO4WkkFBNo62S27nExNiUgcxbb15XvJLipTvEuX/w3sOA/UDBuX/dhg3hTm1/ 3YX4FDyLfjaBndKlcNoYN+wgQNRc3JrysFMkY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:cc:content-type; b=jrGw8zkdFeTJpL77uKEjPc9hZ7kMcr07j+HdrGGeBZw4ZhYJvuiqUqyXntLOh4uLIH qJkkAWB+a7dEYO/XvbGc0Rn3Lp5dYiHh+21hKwdcyriTd/BrdLydJFUPbEa4gw5FJvBU wAtrAH7lTJMOCpWulZTy+Ih8zX7qa/2/AmdsQ= MIME-Version: 1.0 Received: by 10.213.46.67 with SMTP id i3mr1809686ebf.87.1266346601640; Tue, 16 Feb 2010 10:56:41 -0800 (PST) Date: Tue, 16 Feb 2010 12:56:41 -0600 Message-ID: <179b97fb1002161056r33938b49y731eb58689311599@mail.gmail.com> From: Brandon Gooch To: freebsd-docs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net@freebsd.org Subject: Multiple Routing Tables, FIBs, setfib(1) usage X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 18:56:43 -0000 Are there plans to update the documentation regarding multiple routing tables? http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/network-routing.html From owner-freebsd-net@FreeBSD.ORG Tue Feb 16 19:13:46 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E37D41065672 for ; Tue, 16 Feb 2010 19:13:46 +0000 (UTC) (envelope-from julian@elischer.org) Received: from out-0.mx.aerioconnect.net (out-0-3.mx.aerioconnect.net [216.240.47.63]) by mx1.freebsd.org (Postfix) with ESMTP id C36B98FC14 for ; Tue, 16 Feb 2010 19:13:46 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o1GJDko4025366; Tue, 16 Feb 2010 11:13:46 -0800 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id D883C2D6011; Tue, 16 Feb 2010 11:13:45 -0800 (PST) Message-ID: <4B7AEE99.8020006@elischer.org> Date: Tue, 16 Feb 2010 11:14:33 -0800 From: Julian Elischer User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Brandon Gooch References: <179b97fb1002161056r33938b49y731eb58689311599@mail.gmail.com> In-Reply-To: <179b97fb1002161056r33938b49y731eb58689311599@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: freebsd-net@freebsd.org, freebsd-docs@freebsd.org Subject: Re: Multiple Routing Tables, FIBs, setfib(1) usage X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 19:13:47 -0000 Brandon Gooch wrote: > Are there plans to update the documentation regarding multiple routing tables? > > http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/network-routing.html > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" good idea not sure how one does that. but it does need looking at for sure! From owner-freebsd-net@FreeBSD.ORG Tue Feb 16 20:13:00 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C2A371065672; Tue, 16 Feb 2010 20:13:00 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 518DE8FC15; Tue, 16 Feb 2010 20:12:59 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o1GJwt3I003727 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 16 Feb 2010 21:58:55 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3) with ESMTP id o1GJwtKV027562; Tue, 16 Feb 2010 21:58:55 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3/Submit) id o1GJwtoa027561; Tue, 16 Feb 2010 21:58:55 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 16 Feb 2010 21:58:55 +0200 From: Kostik Belousov To: Maxim Sobolev Message-ID: <20100216195855.GG50403@deviant.kiev.zoral.com.ua> References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> <4B7ADFC6.7020202@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Bqc0IY4JZZt50bUr" Content-Disposition: inline In-Reply-To: <4B7ADFC6.7020202@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-net@freebsd.org, FreeBSD Hackers Subject: Re: Sudden mbuf demand increase and shortage under the load (igb issue?) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 20:13:00 -0000 --Bqc0IY4JZZt50bUr Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable [Trimmed Cc: list] On Tue, Feb 16, 2010 at 10:11:18AM -0800, Maxim Sobolev wrote: > OK, here is some new data that I think rules out any issues with the=20 > applications. Following Alfred's suggestion I have made a script to run= =20 > every second and output some system statistics: >=20 > date > netstat -m > vmstat -i > ps -axl > pstat -T > vmstat -z > sysctl -a >=20 > The problem had hit us again today several times and upon investigating= =20 > the log I found that increase in the mbuf usage happened in one step -=20 > going from normal 10% to 100% between two script runs. What is more=20 > interesting, is that time from two such subsequent runs were about 2=20 > minutes apart (instead of 1 second as it should be) and when inspecting= =20 > cron logs I noticed the same time gap in there. I ruled out any VM=20 > starvation as a cause of the delay because system has plenty of free=20 > memory. The incoming network traffic was not sufficient to starve VM so= =20 > quickly either - it was about 7MB/sec at that time, so even if all=20 > receivers stopped draining their buffers it should have taken at least=20 > 1-2 seconds to fill up mbuf cache and create demand for an additional=20 > kernel memory. The failure would likely to be more gradual and I should= =20 > have seen how it builds up in the debug log. >=20 > So it looks like kernel issue of a sort, which causes all userland=20 > activity to cease for 2 minutes when the system reaches certain load.=20 > Mbuf build-up is only the by-product of this, not really a cause. igb(4)= =20 > is being the primary suspect now, since we have other machines with more= =20 > load not having this problem and we don't have anybody else using this=20 > driver. The chip is the following: >=20 > igb0@pci0:5:0:0: class=3D0x020000 card=3D0x323f103c chip=3D0x10c98= 086=20 > rev=3D0x01 hdr=3D0x00 > vendor =3D 'Intel Corporation' > class =3D network > subclass =3D ethernet > igb1@pci0:5:0:1: class=3D0x020000 card=3D0x323f103c chip=3D0x10c98= 086=20 > rev=3D0x01 hdr=3D0x00 > vendor =3D 'Intel Corporation' > class =3D network > subclass =3D ethernet >=20 > Hardware in question is a new HP DL160G6. I have also checked IPMI logs= =20 > and sensors and have not found any issue in there as well. No sensors=20 > reported off-range values and chassis temperature is within normal limits. >=20 > I am not sure how to debug this problem further. We are now=20 > investigating opportunity to install external non-igb card to the server= =20 > and see if it solves the issue. >=20 > I have the whole log if anyone wants to take a closer peek. How much physical memory do you have installed in the machine ? If it is > 16Gb, try to remove some. --Bqc0IY4JZZt50bUr Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkt6+P4ACgkQC3+MBN1Mb4gn+QCgvaSwNrcvigYcLCXLwV81i8j/ mzYAoNghlDps8yyiQieR1r9ejiPpnkKx =9c1c -----END PGP SIGNATURE----- --Bqc0IY4JZZt50bUr-- From owner-freebsd-net@FreeBSD.ORG Tue Feb 16 22:32:33 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1E898106566B for ; Tue, 16 Feb 2010 22:32:33 +0000 (UTC) (envelope-from martin.lopreiato@gmail.com) Received: from mail-iw0-f175.google.com (mail-iw0-f175.google.com [209.85.223.175]) by mx1.freebsd.org (Postfix) with ESMTP id DF1B78FC17 for ; Tue, 16 Feb 2010 22:32:32 +0000 (UTC) Received: by iwn5 with SMTP id 5so2753261iwn.9 for ; Tue, 16 Feb 2010 14:32:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=R7NHNbvrL4rRKnYiNiqn/x7NLrlIaM+gh4V5BEesB58=; b=psGSSuY25xJ6cX54DvLjSZ8PyHCHFSq0zHSjq9rhWnRGxul3JrEQGXm9f0vT3G92mi Rdx5I2DcUqQrs5FGn1qqrPih+KhPT2NmiZA3lAZOQS5nb+5eb6GoYSbnNDNs/H9aNR2l JdnnskEHynzwIm5ZdCUFy/GJWRkqq9kmvdjU0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=myR+19uGyewjrshBnU41JSRDng7ymv5DEjFDu3Lxgq6/X6lGjjOHJ7QdT/xMV2wIed VBrYzKDTZUmHeBmEzej/GJbHvC2ouBXLXB/jekET0m3ItQDmT3eEFsrSHUrHE2oY4Ak3 YHUZxi25RtLRWKBr6bfyZWGkXzlHRiWQTVlKc= MIME-Version: 1.0 Received: by 10.231.144.15 with SMTP id x15mr11046391ibu.99.1266358169919; Tue, 16 Feb 2010 14:09:29 -0800 (PST) Date: Tue, 16 Feb 2010 19:09:29 -0300 Message-ID: From: Martin Lopreiato To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Forging ipv6 addresses? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 22:32:33 -0000 hello, i want to send a packet with a forged ipv6 address. i tried using sendmsg() with ancilliary data, but it seems I can only select an address that has been configured on my system. if I select an arbitrary ipv6 address that i want to forge, i get "can't assign requested address" (in freebsd). isn't it possible to forge an ipv6 address with sendmsg()? note: if i use a configured address, my code works perfectly. so the error message i'm getting when trying to forge an ipv6 address does not seem to be related to a bug in my code. thnx, /m From owner-freebsd-net@FreeBSD.ORG Tue Feb 16 22:40:57 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D77A21065692 for ; Tue, 16 Feb 2010 22:40:57 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from asmtpout024.mac.com (asmtpout024.mac.com [17.148.16.99]) by mx1.freebsd.org (Postfix) with ESMTP id C3AD28FC22 for ; Tue, 16 Feb 2010 22:40:57 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; charset=us-ascii Received: from cswiger1.apple.com ([17.209.4.71]) by asmtp024.mac.com (Sun Java(tm) System Messaging Server 6.3-8.01 (built Dec 16 2008; 32bit)) with ESMTPSA id <0KXY00APAHNW8590@asmtp024.mac.com> for freebsd-net@freebsd.org; Tue, 16 Feb 2010 14:40:46 -0800 (PST) X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx engine=5.0.0-0908210000 definitions=main-1002160193 From: Chuck Swiger In-reply-to: Date: Tue, 16 Feb 2010 14:40:44 -0800 Message-id: References: To: Martin Lopreiato X-Mailer: Apple Mail (2.1077) Cc: freebsd-net@freebsd.org Subject: Re: Forging ipv6 addresses? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 22:40:57 -0000 Hi-- On Feb 16, 2010, at 2:09 PM, Martin Lopreiato wrote: > note: if i use a configured address, my code works perfectly. so the > error message i'm getting when trying to forge an ipv6 address does > not seem to be related to a bug in my code. You're not trying to send this traffic from a jail, by any chance? Consider using PCAP/BPF interface or a wrapper like libnet.... Regards, -- -Chuck From owner-freebsd-net@FreeBSD.ORG Wed Feb 17 00:21:25 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2CD921065672 for ; Wed, 17 Feb 2010 00:21:25 +0000 (UTC) (envelope-from martin.lopreiato@gmail.com) Received: from mail-iw0-f175.google.com (mail-iw0-f175.google.com [209.85.223.175]) by mx1.freebsd.org (Postfix) with ESMTP id E89C58FC15 for ; Wed, 17 Feb 2010 00:21:24 +0000 (UTC) Received: by iwn5 with SMTP id 5so2834482iwn.9 for ; Tue, 16 Feb 2010 16:21:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=C5ApO+1Xeq4+mluiMTEPnuw7AE1BsX2tdEKiTjQrlKU=; b=pr8nzIVQ7CxnO3y3+YakUVR/KGOgZkha6DsZLCt7svgSOiOCZzrpvpfa+lzUqvygJn XuHVY1+IAIiQtQkBLgJF3bKFbw96lPaEElIixi835hPDm1cxqUytoViNj1tbCBj5a0q+ MQvI6ei4BqMtKZ85D/30AOuJXgrAqcwl/33r4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=mrOm78Lcf4R3d+Px3xMa5VrwhOWM+YqJGNTzmu45eqkUkI+WpbN7lhUePKXcL+QEZM 0PmK8R7VfuIaWJ1sC5iud2HVdSf0M8fEHrxwiQUp4ty+E5kBacHG2y/n7h7fHFEeYVlO +TMWqEHVVTF2prwG69bNfqOds2LePSgFbpwnw= MIME-Version: 1.0 Received: by 10.231.160.149 with SMTP id n21mr2497230ibx.93.1266366084012; Tue, 16 Feb 2010 16:21:24 -0800 (PST) In-Reply-To: References: Date: Tue, 16 Feb 2010 21:21:23 -0300 Message-ID: From: Martin Lopreiato To: Chuck Swiger Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org Subject: Re: Forging ipv6 addresses? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Feb 2010 00:21:25 -0000 hi, On Feb 16, 2010, at 2:09 PM, Martin Lopreiato wrote: > > note: if i use a configured address, my code works perfectly. so the > > error message i'm getting when trying to forge an ipv6 address does > > not seem to be related to a bug in my code. > > You're not trying to send this traffic from a jail, by any chance? > no. Consider using PCAP/BPF interface or a wrapper like libnet.... > pcap/bpf wouldn't do: i'd have to take care of neighbor-discovery and routing myself. libnet would probably do, but.. hey! it's a simple ipv6 packet. and at this point, i'm curious about whether there's a limitation on the addresses you can "forge" with sendmsg() fwiw, linux seems to have an ipv6 version of ipv5}4 hdrinclude socket option. that's nice, too. thanx, /m From owner-freebsd-net@FreeBSD.ORG Wed Feb 17 03:16:27 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B83111065670 for ; Wed, 17 Feb 2010 03:16:27 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from mail.meer.net (mail.meer.net [64.13.141.3]) by mx1.freebsd.org (Postfix) with ESMTP id 9EDA68FC14 for ; Wed, 17 Feb 2010 03:16:27 +0000 (UTC) Received: from mail2.meer.net (mail2.meer.net [64.13.141.16]) by mail.meer.net (8.13.3/8.13.3/meer) with ESMTP id o1H3F5Bx099107; Tue, 16 Feb 2010 19:15:05 -0800 (PST) (envelope-from gnn@neville-neil.com) Received: from [192.168.0.12] (cpe-68-175-72-246.nyc.res.rr.com [68.175.72.246]) (authenticated bits=0) by mail2.meer.net (8.14.1/8.14.3) with ESMTP id o1H3F4dg077445 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 16 Feb 2010 19:15:05 -0800 (PST) (envelope-from gnn@neville-neil.com) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: George Neville-Neil In-Reply-To: Date: Tue, 16 Feb 2010 22:15:04 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <977CA846-505E-4D96-8555-6D1A21F2BC18@neville-neil.com> References: To: Martin Lopreiato X-Mailer: Apple Mail (2.1077) Cc: freebsd-net@freebsd.org Subject: Re: Forging ipv6 addresses? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Feb 2010 03:16:27 -0000 On Feb 16, 2010, at 19:21 , Martin Lopreiato wrote: > hi, >=20 > On Feb 16, 2010, at 2:09 PM, Martin Lopreiato wrote: >>> note: if i use a configured address, my code works perfectly. so the >>> error message i'm getting when trying to forge an ipv6 address does >>> not seem to be related to a bug in my code. >>=20 >> You're not trying to send this traffic from a jail, by any chance? >>=20 >=20 > no. >=20 >=20 > Consider using PCAP/BPF interface or a wrapper like libnet.... >>=20 >=20 > pcap/bpf wouldn't do: i'd have to take care of neighbor-discovery and > routing myself. libnet would probably do, but.. hey! it's a simple = ipv6 > packet. and at this point, i'm curious about whether there's a = limitation on > the addresses you can "forge" with sendmsg() >=20 > fwiw, linux seems to have an ipv6 version of ipv5}4 hdrinclude socket > option. that's nice, too. >=20 I'd recommend pcs (http://pcs.sf.net) which has a lot of IPv6 support. Best, George= From owner-freebsd-net@FreeBSD.ORG Wed Feb 17 14:10:01 2010 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 94D22106566B for ; Wed, 17 Feb 2010 14:10:01 +0000 (UTC) (envelope-from minotaur@crete.org.ua) Received: from relay.padonki.org.ua (relay.padonki.org.ua [193.0.227.26]) by mx1.freebsd.org (Postfix) with ESMTP id 5337C8FC13 for ; Wed, 17 Feb 2010 14:10:00 +0000 (UTC) Received: from minotaur by relay.padonki.org.ua with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1NhjvE-0000Eu-OM; Wed, 17 Feb 2010 15:26:32 +0200 Date: Wed, 17 Feb 2010 15:26:32 +0200 From: Alexander Shikoff To: freebsd-net@FreeBSD.org Message-ID: <20100217132632.GA756@crete.org.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-u Content-Disposition: inline User-Agent: Mutt/1.5.18 (2008-05-17) Sender: Alexander Shikoff Cc: Subject: mpd has hung X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Feb 2010 14:10:01 -0000 Hello All, I have mpd 5.3 running on 8.0-RC1 as PPPoE server (now only 5 clients). Today mpd process hung and I cannot kill it with -9 signal, and I cannot access it's console via telnet. State of process in `top` output is STOP: 73551 root 2 44 0 29588K 5692K STOP 6 0:32 0.00% mpd5 # procstat -kk 73551 PID TID COMM TDNAME KSTACK 73551 100233 mpd5 - mi_switch+0x16f sleepq_wait+0x42 _cv_wait+0x111 flowtable_flush+0x51 if_detach+0x2f2 ng_iface_shutdown+0x1e ng_rmnode+0x167 ng_apply_item+0xef7 ng_snd_item+0x2ce ngc_send+0x1d2 sosend_generic+0x3f6 kern_sendit+0x13d sendit+0xdc sendto+0x4d syscall+0x1da Xfast_syscall+0xe1 73551 100502 mpd5 - mi_switch+0x16f thread_suspend_switch+0xc6 thread_single+0x1b6 exit1+0x72 sigexit+0x7c postsig+0x306 ast+0x279 doreti_ast+0x1f Is there a way to stop a process without rebooting a whole system? Thanks in advance! P.S. I'm ready for experiments with it before tonight, but I cannot force system to crash in order to get crash dump right now. -- MINO-RIPE From owner-freebsd-net@FreeBSD.ORG Wed Feb 17 14:39:32 2010 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 13464106566B; Wed, 17 Feb 2010 14:39:32 +0000 (UTC) (envelope-from brucec@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id DF36C8FC19; Wed, 17 Feb 2010 14:39:31 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1HEdVg7018889; Wed, 17 Feb 2010 14:39:31 GMT (envelope-from brucec@freefall.freebsd.org) Received: (from brucec@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o1HEdVw7018885; Wed, 17 Feb 2010 14:39:31 GMT (envelope-from brucec) Date: Wed, 17 Feb 2010 14:39:31 GMT Message-Id: <201002171439.o1HEdVw7018885@freefall.freebsd.org> To: brucec@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: brucec@FreeBSD.org Cc: Subject: Re: kern/142772: [libc] lla_lookup: new lle malloc failed X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Feb 2010 14:39:32 -0000 Synopsis: [libc] lla_lookup: new lle malloc failed Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: brucec Responsible-Changed-When: Wed Feb 17 14:38:34 UTC 2010 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=142772 From owner-freebsd-net@FreeBSD.ORG Wed Feb 17 20:30:05 2010 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4A153106566C for ; Wed, 17 Feb 2010 20:30:05 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 3A0888FC22 for ; Wed, 17 Feb 2010 20:30:05 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1HKU596020458 for ; Wed, 17 Feb 2010 20:30:05 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o1HKU54f020455; Wed, 17 Feb 2010 20:30:05 GMT (envelope-from gnats) Date: Wed, 17 Feb 2010 20:30:05 GMT Message-Id: <201002172030.o1HKU54f020455@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: Jeff Mo Cc: Subject: Re: kern/125239: [gre] kernel crash when using gre X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Jeff Mo List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Feb 2010 20:30:05 -0000 The following reply was made to PR kern/125239; it has been noted by GNATS. From: Jeff Mo To: bug-followup@freebsd.org, damien.saucez@uclouvain.be Cc: Subject: Re: kern/125239: [gre] kernel crash when using gre Date: Wed, 17 Feb 2010 10:32:00 -0800 --00504502c5f9e57206047fd010ce Content-Type: text/plain; charset=ISO-8859-1 Hello all, I found the solution about why this bug occurs : I would like to contribute my knowledge to FreeBSD website and you can find my solution in the attached. Please let me know your comment and look forward to your reply. Thanks, -- Jeff Mo Santa Clara University Linux+, SCJP, SCWCD, MCSD --00504502c5f9e57206047fd010ce Content-Type: application/octet-stream; name=kern_125239 Content-Disposition: attachment; filename=kern_125239 Content-Transfer-Encoding: base64 X-Attachment-Id: f_g5sgk88g0 VGhlIHByb2JsZW0gaW4gMTI1MjM5IG9jY3VycyBiZWNhdXNlIHRoZSBpbXBsZW1lbnRhdGlvbiBv ZiBpbl9zZXRfdHVubmVsIGZ1bmN0aW9uIGluCgphZl9pbmV0LmMgd3JvbmdseSBjcmVhdGVzIGEg bG9jYWwgc3RydWN0IHZhcmlhYmxlIChhZGRyZXEpLCByYXRoZXIgdGhhbiB1c2luZyBleGlzdGlu ZwoKZ2xvYmFsIHN0cnVjdCB2YXJpYWJsZShpbl9hZGRyZXEpLCB0byBzdG9yZSByZWxhdGVkIGFk ZHJlc3MgaW5mb3JtYXRpb24gb2YgdHVubmVsLgoKVGhlcmVmb3JlLCB3aGVuIHdlIHVzZSBnZGIg dG8gdHJhY2UgdGhlIGNvZGUgYmVmb3JlIGVudGVyaW5nIGlvY3RsLCB3ZSBmb3VuZCB0aGF0IGFm cAoKLT5hZl9hZGRyZXEgZG9lcyBub3QgY2FycnkgYW55IHNvdXJjZSBhbmQgZGVzdGluYXRpb24g YWRkcmVzcyBpbmZvcm1hdGlvbiBpbnRvIGtlcm5lbC4KVGhlIGZvbGxvd2luZyBzaG93cyB0aGUg dHJhY2luZyByZXN1bHQgYmVmb3JlIG1vZGlmaWNhdGlvbjoKCihnZGIpIHAgKihzdHJ1Y3QgaWZh bGlhc3JlcSAqKWFmcC0+YWZfYWRkcmVxCiQxID0gewogIGlmcmFfbmFtZSA9ICdcMCcgPHJlcGVh dHMgMTUgdGltZXM+LAogIGlmcmFfYWRkciA9IHsKICAgIHNhX2xlbiA9IDAgJ1wwJywKICAgIHNh X2ZhbWlseSA9IDAgJ1wwJywKICAgIHNhX2RhdGEgPSAnXDAnIDxyZXBlYXRzIDEzIHRpbWVzPgog IH0sCiAgaWZyYV9icm9hZGFkZHIgPSB7CiAgICBzYV9sZW4gPSAwICdcMCcsCiAgICBzYV9mYW1p bHkgPSAwICdcMCcsCiAgICBzYV9kYXRhID0gJ1wwJyA8cmVwZWF0cyAxMyB0aW1lcz4KICB9LAog IGlmcmFfbWFzayA9IHsKICAgIHNhX2xlbiA9IDE2ICdcMDIwJywKICAgIHNhX2ZhbWlseSA9IDAg J1wwJywKICAgIHNhX2RhdGEgPSAiXDAwMFwwMDDDg8K/w4PCv8ODwr/Dg8K/XDAwMFwwMDBcMDAw XDAwMFwwMDBcMDAwXDAwMCIKICB9Cn0KCk91ciBwcm9wb3NlZCBzb2x1dGlvbiBpcyB2ZXJ5IHN0 cmFpZ2h0Zm9yd2FyZC4gSW5zdGVhZCBvZiBkZWNsYXJpbmcgYSBuZXcgbG9jYWwKCnZhcmlhYmxl LCB3ZSBzaG91bGQgc3RvcmUgYWRkcmVzcyBkYXRhIGluIHRoZSBnbG9iYWwgdmFyaWFibGUgdGhh dCB3aWxsIGJlIHVzZWQgYXMgYQoKcGFyYW1ldGVyIG9mIGlvY3RsIGZ1Y3Rpb24uIE9uY2Ugb3Vy IGNoYW5nZXMgYXJlIGFwcGxpZWQsIFRoZSBzeXN0ZW0gcnVucyBhcyBub3JtYWwgYXMgaXQgc2hv dWxkIGJlIGFuZCBub3QgY3Jhc2ggYWdhaW4uCgpUaGUgZm9sbG93aW5nIGZpeCBpcyBhcHBsaWVk IHRvIHNvdXJjZSBmcm9tIHRoZSBGcmVlQlNEIDcuMCByZWxlYXNlCgotLS0gYWZfaW5ldC5jLm9s ZCAgICAgICAyMDA4LTA5LTI5IDExOjI1OjMyLjAwMDAwMDAwMCArMDAwMAorKysgYWZfaW5ldC5j ICAgMjAwOC0xMC0wMSAwMjowMzoxMS4wMDAwMDAwMDAgKzAwMDAKQEAgLTE2NywxMyArMTY3LDEy IEBACiBzdGF0aWMgdm9pZAogaW5fc2V0X3R1bm5lbChpbnQgcywgc3RydWN0IGFkZHJpbmZvICpz cmNyZXMsIHN0cnVjdCBhZGRyaW5mbyAqZHN0cmVzKQogewotICAgICAgIHN0cnVjdCBpZmFsaWFz cmVxIGFkZHJlcTsKLSAgICAgICBtZW1zZXQoJmFkZHJlcSwgMCwgc2l6ZW9mKGFkZHJlcSkpOwot ICAgICAgIHN0cm5jcHkoYWRkcmVxLmlmcmFfbmFtZSwgbmFtZSwgSUZOQU1TSVopOwotICAgICAg IG1lbWNweSgmYWRkcmVxLmlmcmFfYWRkciwgc3JjcmVzLT5haV9hZGRyLCBzcmNyZXMtPmFpX2Fk ZHItPnNhX2xlbik7Ci0gICAgICAgbWVtY3B5KCZhZGRyZXEuaWZyYV9kc3RhZGRyLCBkc3RyZXMt PmFpX2FkZHIsIGRzdHJlcy0+YWlfYWRkci0+c2FfbGVuKTsKKyAgICAgICBtZW1zZXQoJmluX2Fk ZHJlcSwgMCwgc2l6ZW9mKGluX2FkZHJlcSkpOworICAgICAgIHN0cm5jcHkoJmluX2FkZHJlcS5p ZnJhX25hbWUsIG5hbWUsIElGTkFNU0laKTsKKyAgICAgICBtZW1jcHkoJmluX2FkZHJlcS5pZnJh X2FkZHIsIHNyY3Jlcy0+YWlfYWRkciwgc3JjcmVzLT5haV9hZGRyLT5zYV9sZW4pOworICAgICAg IG1lbWNweSgmaW5fYWRkcmVxLmlmcmFfZHN0YWRkciwgZHN0cmVzLT5haV9hZGRyLCBkc3RyZXMt PmFpX2FkZHItPnNhX2xlbik7CgotICAgICAgIGlmIChpb2N0bChzLCBTSU9DU0lGUEhZQUREUiwg JmFkZHJlcSkgPCAwKQorICAgICAgIGlmIChpb2N0bChzLCBTSU9DU0lGUEhZQUREUiwgJmluX2Fk ZHJlcSkgPCAwKQogICAgICAgICAgICAgICAgd2FybigiU0lPQ1NJRlBIWUFERFIiKTsKIH0KClRo ZSBmb2xsb3dpbmcgc2hvd3MgdGhlIHRyYWNpbmcgcmVzdWx0IGFmdGVyIG1vZGlmaWNhdGlvbjoK CihnZGIpIHAgKihzdHJ1Y3QgaWZhbGlhc3JlcSAqKWFmcC0+YWZfYWRkcmVxCiQxID0gewogIGlm cmFfbmFtZSA9ICJncmUwIiwgJ1wwJyA8cmVwZWF0cyAxMSB0aW1lcz4sCiAgaWZyYV9hZGRyID0g ewogICAgc2FfbGVuID0gMTYgJ1wwMjAnLAogICAgc2FfZmFtaWx5ID0gMiAnXDAwMicsCiAgICBz YV9kYXRhID0gIlwwMDBcMDAwXG5cblxuXDAwMVwwMDBcMDAwXDAwMFwwMDBcMDAwXDAwMFwwMDAi CiAgfSwKICBpZnJhX2Jyb2FkYWRkciA9IHsKICAgIHNhX2xlbiA9IDE2ICdcMDIwJywKICAgIHNh X2ZhbWlseSA9IDIgJ1wwMDInLAogICAgc2FfZGF0YSA9ICJcMDAwXDAwMFxuXG5cblwwMDJcMDAw XDAwMFwwMDBcMDAwXDAwMFwwMDBcMDAwIgogIH0sCiAgaWZyYV9tYXNrID0gewogICAgc2FfbGVu ID0gMTYgJ1wwMjAnLAogICAgc2FfZmFtaWx5ID0gMCAnXDAnLAogICAgc2FfZGF0YSA9ICJcMDAw XDAwMMODwr/Dg8K/w4PCv8ODwr9cMDAwXDAwMFwwMDBcMDAwXDAwMFwwMDBcMDAwIgogIH0KfQoK CkNoaWVoLUZ1IE1vLCBTdHVkZW50CkNPRU4gMjg0LCBPcGVyYXRpbmcgU3lzdGVtcyBDYXNlIFN0 dWR5ClNhbnRhIENsYXJhIFVuaXZlcnNpdHkKCg== --00504502c5f9e57206047fd010ce-- From owner-freebsd-net@FreeBSD.ORG Wed Feb 17 22:10:17 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B5F181065679 for ; Wed, 17 Feb 2010 22:10:17 +0000 (UTC) (envelope-from chris@chrullrich.net) Received: from chrullrich.net (v29774.1blu.de [88.84.143.174]) by mx1.freebsd.org (Postfix) with ESMTP id 7D8EC8FC0A for ; Wed, 17 Feb 2010 22:10:17 +0000 (UTC) Received: from [84.166.42.170] (helo=[192.168.2.4]) by chrullrich.net with esmtpa (Exim 4.69) (envelope-from ) id 1Nhrem-0006OD-A9 for freebsd-net@freebsd.org; Wed, 17 Feb 2010 22:42:05 +0100 Message-ID: <4B7C62AF.6000904@chrullrich.net> Date: Wed, 17 Feb 2010 22:42:07 +0100 From: Christian Ullrich User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; de; rv:1.9.1.5) Gecko/20091204 Thunderbird/3.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit Subject: Routing into overlapping subnets X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Feb 2010 22:10:17 -0000 Hello all, I'm having a routing problem I can't wrap my head around. Consider this situation: I want to provide some common resources (printers etc.) to a number of clients in separate networks. They each have their own physical network, and these networks are supposed to stay isolated, with the exception of what I'm trying to set up. I have no control over the IP addressing used by the clients, except that it's all RFC1918, but as far as I know, they are all using the same addresses. What I thought would work was this: On a central switch, set up one VLAN per client network and ensure that the switch only forwards between the port assigned to the client and one (trunk) port where I put my router. On the router, create the required VLAN interfaces and put the "public" addresses for my printers on them (one in each RFC1918 net, but the same on each VLAN), then use pf to NAT the destination for packets coming in on each VLAN to the actual printers, and use "reply-to" to make sure the responses go out through the same interface. Now I'm stuck. With one client, that works fine, but when I add another one (different interface, same client address), the packets are dropped and dmesg shows lots of "arpresolve: can't allocate llinfo for (client address)". I suppose that's because there is already an ARP entry for that IP address, and the kernel cannot add another one. If that's true, here's what I don't understand: When the time comes to resolve IP addresses to hardware addresses, the kernel has already routed the packet to an outgoing interface, so it should be enough to consider the ARP entries associated with that interface to know where to send the packet. Why doesn't the kernel look up an ARP table entry by both IP address and interface? Is there anything I can do to make this work? It's not an everyday setup, I know, but I'd really like to make it work. Thanks for any help. -- Christian From owner-freebsd-net@FreeBSD.ORG Wed Feb 17 23:50:05 2010 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 77402106566B for ; Wed, 17 Feb 2010 23:50:05 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 4CD368FC13 for ; Wed, 17 Feb 2010 23:50:05 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1HNo5aZ008703 for ; Wed, 17 Feb 2010 23:50:05 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o1HNo5Vt008702; Wed, 17 Feb 2010 23:50:05 GMT (envelope-from gnats) Date: Wed, 17 Feb 2010 23:50:05 GMT Message-Id: <201002172350.o1HNo5Vt008702@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: Matt Reimer Cc: Subject: Re: kern/143939: [ipfw] [em] ipfw nat and em interface rxcsum problem X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Matt Reimer List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Feb 2010 23:50:05 -0000 The following reply was made to PR kern/143939; it has been noted by GNATS. From: Matt Reimer To: bug-followup@FreeBSD.org, dima_bsd@inbox.lv Cc: Subject: Re: kern/143939: [ipfw] [em] ipfw nat and em interface rxcsum problem Date: Wed, 17 Feb 2010 15:30:47 -0800 Same thing happens for me on 8-STABLE every time. Matt From owner-freebsd-net@FreeBSD.ORG Thu Feb 18 02:34:22 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 564C91065672 for ; Thu, 18 Feb 2010 02:34:22 +0000 (UTC) (envelope-from steve@ibctech.ca) Received: from smtp.ibctech.ca (v6.ibctech.ca [IPv6:2607:f118::b6]) by mx1.freebsd.org (Postfix) with SMTP id B88978FC13 for ; Thu, 18 Feb 2010 02:34:21 +0000 (UTC) Received: (qmail 50377 invoked by uid 89); 18 Feb 2010 02:38:10 -0000 Received: from unknown (HELO ?192.168.1.114?) (steve@ibctech.ca@::ffff:208.70.104.100) by ::ffff:208.70.104.210 with ESMTPA; 18 Feb 2010 02:38:10 -0000 Message-ID: <4B7CA72A.4050202@ibctech.ca> Date: Wed, 17 Feb 2010 21:34:18 -0500 From: Steve Bertrand User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100111 Lightning/1.0b1 Thunderbird/3.0.1 MIME-Version: 1.0 To: Christian Ullrich References: <4B7C62AF.6000904@chrullrich.net> In-Reply-To: <4B7C62AF.6000904@chrullrich.net> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: Routing into overlapping subnets X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 02:34:22 -0000 On 2010.02.17 16:42, Christian Ullrich wrote: > Hello all, > > I'm having a routing problem I can't wrap my head around. Consider this > situation: I want to provide some common resources (printers etc.) to a > number of clients in separate networks. They each have their own > physical network, and these networks are supposed to stay isolated, with > the exception of what I'm trying to set up. I have no control over the > IP addressing used by the clients, except that it's all RFC1918, but as > far as I know, they are all using the same addresses. > > What I thought would work was this: On a central switch, set up one VLAN > per client network and ensure that the switch only forwards between the > port assigned to the client and one (trunk) port where I put my router. > On the router, create the required VLAN interfaces and put the "public" > addresses for my printers on them (one in each RFC1918 net, but the same > on each VLAN), then use pf to NAT the destination for packets coming in > on each VLAN to the actual printers, and use "reply-to" to make sure the > responses go out through the same interface. > > Now I'm stuck. With one client, that works fine, but when I add another > one (different interface, same client address), the packets are dropped > and dmesg shows lots of "arpresolve: can't allocate llinfo for (client > address)". I suppose that's because there is already an ARP entry for > that IP address, and the kernel cannot add another one. > > If that's true, here's what I don't understand: When the time comes to > resolve IP addresses to hardware addresses, the kernel has already > routed the packet to an outgoing interface, so it should be enough to > consider the ARP entries associated with that interface to know where to > send the packet. Why doesn't the kernel look up an ARP table entry by > both IP address and interface? That's not how the protocols were designed, and thankfully so. Imagine the potential for spoofing if this were allowed by default ;) > Is there anything I can do to make this work? It's not an everyday > setup, I know, but I'd really like to make it work. I have a couple of ideas, but need to understand better of your setup. Advise if this seems semi-accurate: - you house global resources for a bunch of clients at a central location - you have limited public IP addresses to do this with, or your central location is located within the same 'building' as all of the clients - you have several clients with overlapping 1918 space - you need a method to have two instances of eg 192.168.1.110 accessing a single central resource, but which will be coming in on two separate interfaces (physical or virtual) - the central services (ie printer) doesn't have the capability to house more than a single IPv4 address - you do not want to be open to the potential for one client accessing the others networks - you have absolute control over the pf box is this right? Steve From owner-freebsd-net@FreeBSD.ORG Thu Feb 18 05:31:56 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 10A1B1065672 for ; Thu, 18 Feb 2010 05:31:56 +0000 (UTC) (envelope-from chris@chrullrich.net) Received: from chrullrich.net (v29774.1blu.de [88.84.143.174]) by mx1.freebsd.org (Postfix) with ESMTP id CA85E8FC0C for ; Thu, 18 Feb 2010 05:31:55 +0000 (UTC) Received: from [84.166.111.197] (helo=[192.168.2.4]) by chrullrich.net with esmtpa (Exim 4.69) (envelope-from ) id 1NhyzP-0004Cw-HI for freebsd-net@freebsd.org; Thu, 18 Feb 2010 06:31:51 +0100 Message-ID: <4B7CD0CB.4080105@chrullrich.net> Date: Thu, 18 Feb 2010 06:31:55 +0100 From: Christian Ullrich User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; de; rv:1.9.1.5) Gecko/20091204 Thunderbird/3.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org References: <4B7C62AF.6000904@chrullrich.net> <4B7CA72A.4050202@ibctech.ca> In-Reply-To: <4B7CA72A.4050202@ibctech.ca> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: Routing into overlapping subnets X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 05:31:56 -0000 * Steve Bertrand wrote: > On 2010.02.17 16:42, Christian Ullrich wrote: >> send the packet. Why doesn't the kernel look up an ARP table entry by >> both IP address and interface? > > That's not how the protocols were designed, and thankfully so. Imagine > the potential for spoofing if this were allowed by default ;) You're right, of course. I had not considered that. > I have a couple of ideas, but need to understand better of your setup. > Advise if this seems semi-accurate: > > - you house global resources for a bunch of clients at a central location > - you have limited public IP addresses to do this with, or your central > location is located within the same 'building' as all of the clients The latter. > - you have several clients with overlapping 1918 space > - you need a method to have two instances of eg 192.168.1.110 accessing > a single central resource, but which will be coming in on two separate > interfaces (physical or virtual) > - the central services (ie printer) doesn't have the capability to house > more than a single IPv4 address > - you do not want to be open to the potential for one client accessing > the others networks > - you have absolute control over the pf box > > is this right? Exactly right. -- Christian From owner-freebsd-net@FreeBSD.ORG Thu Feb 18 09:18:10 2010 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5637D106566B for ; Thu, 18 Feb 2010 09:18:10 +0000 (UTC) (envelope-from minotaur@crete.org.ua) Received: from relay.padonki.org.ua (relay.padonki.org.ua [193.0.227.26]) by mx1.freebsd.org (Postfix) with ESMTP id EC3758FC0A for ; Thu, 18 Feb 2010 09:18:09 +0000 (UTC) Received: from minotaur by relay.padonki.org.ua with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1Ni2WO-000BAZ-Ed; Thu, 18 Feb 2010 11:18:08 +0200 Date: Thu, 18 Feb 2010 11:18:08 +0200 From: Alexander Shikoff To: Dennis Yusupoff Message-ID: <20100218091808.GA42801@crete.org.ua> References: <200912291120.nBTBKD75055992@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-u Content-Disposition: inline In-Reply-To: <200912291120.nBTBKD75055992@freefall.freebsd.org> User-Agent: Mutt/1.5.18 (2008-05-17) Sender: Alexander Shikoff Cc: freebsd-net@FreeBSD.org Subject: Re: kern/141285: [em] hangs down/up intel nic during creating vlan X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Alexander Shikoff List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 09:18:10 -0000 On Tue, Dec 29, 2009 at 11:20:13AM +0000, Dennis Yusupoff wrote: > > This bug looks exactly as mine: > http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dkern/141843 > > And IMHO, it's a really critical severity. I confirm the issue. Exactly the same up-downs on 8.0-RELEASE-p2 (amd64) 7.2-RELEASE (amd64) does not have this problem. -- MINO-RIPE From owner-freebsd-net@FreeBSD.ORG Thu Feb 18 14:35:14 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A90E7106566B for ; Thu, 18 Feb 2010 14:35:14 +0000 (UTC) (envelope-from frank.schuster01@web.de) Received: from fmmailgate09.web.de (fmmailgate09.web.de [217.72.192.184]) by mx1.freebsd.org (Postfix) with ESMTP id 6E2248FC17 for ; Thu, 18 Feb 2010 14:35:14 +0000 (UTC) Received: from web.de by fmmailgate09.web.de (Postfix) with SMTP id EFACA2B51823 for ; Thu, 18 Feb 2010 15:35:12 +0100 (CET) Received: from [109.40.144.236] by freemailng1204.web.de with HTTP; Thu, 18 Feb 2010 15:35:12 +0100 Date: Thu, 18 Feb 2010 15:35:12 +0100 Message-Id: <2001237861@web.de> MIME-Version: 1.0 From: frank.schuster01@web.de To: freebsd-net@freebsd.org Precedence: fm-user Organization: http://freemail.web.de/ X-Provags-Id: V01U2FsdGVkX18jLgzpn0dywPDzgSy7T29WL2QJCQ9qHuxTOjnZ1PQeGZXp6 guuMbzWChDmZWS78h5d0t40a3aHPpCg1DBW11tjGgN/UoSo1DDFFfwD2BUBZ g== Content-Type: text/plain; charset=iso-8859-15 Content-Transfer-Encoding: quoted-printable Subject: Wireshark, netperf and sctp problem X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 14:35:14 -0000 Hello, I have a problem with my little network test. I want to test sctp between ubuntu and freebsd. On ubuntu I wrote a iptables rule which delete every tenth (10) data packa= ge. Freebsd has no firewall rules. But only ubuntu sends data and freebsd only acknowledge this data (i did t= he test with netperf). On the sending side (ubuntu) I see in wireshark 24 data packages, and on t= he receiving side (freebsd) I saw 82 packages. But I can't explain, why freebsd receives more packages as delivered from = ubuntu=3F (I have currently only one freebsd system). Any ideas or something additional notes to help me=3F Is the problem by wireshark or the kernel implementation=3F Regards Frank =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F GRATIS f=FCr alle WEB.DE-Nutzer: Die maxdome Movie-FLAT! Jetzt freischalten unter http://movieflat.web.de From owner-freebsd-net@FreeBSD.ORG Thu Feb 18 15:04:33 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 56919106566C for ; Thu, 18 Feb 2010 15:04:33 +0000 (UTC) (envelope-from Michael.Tuexen@lurchi.franken.de) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) by mx1.freebsd.org (Postfix) with ESMTP id E6D348FC22 for ; Thu, 18 Feb 2010 15:04:32 +0000 (UTC) Received: from [192.168.1.121] (p508FF7C1.dip.t-dialin.net [80.143.247.193]) by mail-n.franken.de (Postfix) with ESMTP id A9C571C0B4606; Thu, 18 Feb 2010 16:04:30 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=iso-8859-1 From: =?iso-8859-1?Q?Michael_T=FCxen?= In-Reply-To: <2001237861@web.de> Date: Thu, 18 Feb 2010 16:04:29 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <06BD9632-59EE-4B7F-A586-99413D9DDB70@lurchi.franken.de> References: <2001237861@web.de> To: frank.schuster01@web.de X-Mailer: Apple Mail (2.1077) Cc: freebsd-net@freebsd.org Subject: Re: Wireshark, netperf and sctp problem X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 15:04:33 -0000 Hi Frank, you might want to send both wireshark trace to me and I would have a = look. If it is a bulk transfer, maybe the Ubuntu machine did not capture all = packets. The number of dropped packets is displayed when you stop capturing. Best regards Michael On Feb 18, 2010, at 3:35 PM, frank.schuster01@web.de wrote: > Hello, >=20 > I have a problem with my little network test. > I want to test sctp between ubuntu and freebsd. > On ubuntu I wrote a iptables rule which delete every tenth (10) data = package. Freebsd has no firewall rules. > But only ubuntu sends data and freebsd only acknowledge this data (i = did the test with netperf). >=20 > On the sending side (ubuntu) I see in wireshark 24 data packages, and = on the receiving side (freebsd) I saw 82 packages. > But I can't explain, why freebsd receives more packages as delivered = from ubuntu? (I have currently only one freebsd system). >=20 > Any ideas or something additional notes to help me? >=20 > Is the problem by wireshark or the kernel implementation? >=20 > Regards > Frank > ______________________________________________________ > GRATIS f=FCr alle WEB.DE-Nutzer: Die maxdome Movie-FLAT! > Jetzt freischalten unter http://movieflat.web.de >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Thu Feb 18 15:32:53 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 58127106568D for ; Thu, 18 Feb 2010 15:32:53 +0000 (UTC) (envelope-from nvass9573@gmx.com) Received: from mailout-eu.gmx.com (mailout-eu.gmx.com [213.165.64.42]) by mx1.freebsd.org (Postfix) with SMTP id BC2BD8FC32 for ; Thu, 18 Feb 2010 15:32:52 +0000 (UTC) Received: (qmail invoked by alias); 18 Feb 2010 15:32:51 -0000 Received: from adsl-40.91.140.30.tellas.gr (EHLO [192.168.73.193]) [91.140.30.40] by mail.gmx.com (mp-eu002) with SMTP; 18 Feb 2010 16:32:51 +0100 X-Authenticated: #46156728 X-Provags-ID: V01U2FsdGVkX18CBK9fHhACUCFSHS0mdOBANSj/XkgBz05a7P2PtH KRHPNl24KHU7nO Message-ID: <4B7D5D95.20007@gmx.com> Date: Thu, 18 Feb 2010 17:32:37 +0200 From: Nikos Vassiliadis User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1 MIME-Version: 1.0 To: Alexander Shikoff References: <20100217132632.GA756@crete.org.ua> In-Reply-To: <20100217132632.GA756@crete.org.ua> Content-Type: text/plain; charset=KOI8-U; format=flowed Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 X-FuHaFi: 0.51000000000000001 Cc: freebsd-net@freebsd.org Subject: Re: mpd has hung X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 15:32:53 -0000 On 2/17/2010 3:26 PM, Alexander Shikoff wrote: > Hello All, > > I have mpd 5.3 running on 8.0-RC1 as PPPoE server (now only 5 clients). > Today mpd process hung and I cannot kill it with -9 signal, and I cannot > access it's console via telnet. > > State of process in `top` output is STOP: > 73551 root 2 44 0 29588K 5692K STOP 6 0:32 0.00% mpd5 > > # procstat -kk 73551 > PID TID COMM TDNAME KSTACK > 73551 100233 mpd5 - mi_switch+0x16f sleepq_wait+0x42 _cv_wait+0x111 flowtable_flush+0x51 if_detach+0x2f2 ng_iface_shutdown+0x1e ng_rmnode+0x167 ng_apply_item+0xef7 ng_snd_item+0x2ce ngc_send+0x1d2 sosend_generic+0x3f6 kern_sendit+0x13d sendit+0xdc sendto+0x4d syscall+0x1da Xfast_syscall+0xe1 > 73551 100502 mpd5 - mi_switch+0x16f thread_suspend_switch+0xc6 thread_single+0x1b6 exit1+0x72 sigexit+0x7c postsig+0x306 ast+0x279 doreti_ast+0x1f > > Is there a way to stop a process without rebooting a whole system? > Thanks in advance! > > P.S. I'm ready for experiments with it before tonight, but I cannot > force system to crash in order to get crash dump right now. > It's probably too late now, but are you sure that nobody pressed CTLR-Z while in the mpd console??? CTLR-Z will send SIGSTOP to the process and the process will stop. While stopped, all processing stops(including receiving SIGKILL, you cannot kill it, and the signals are queued). You have to send SIGCONT for the process to continue. Just a thought, Nikos From owner-freebsd-net@FreeBSD.ORG Thu Feb 18 15:32:54 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A47E010656AC for ; Thu, 18 Feb 2010 15:32:54 +0000 (UTC) (envelope-from mahan@mahan.org) Received: from ns.mahan.org (ns.mahan.org [67.116.10.138]) by mx1.freebsd.org (Postfix) with ESMTP id 368BD8FC31 for ; Thu, 18 Feb 2010 15:32:53 +0000 (UTC) Received: from Gypsy.mahan.org (crowTrobot [67.116.10.140]) by ns.mahan.org (8.13.6/8.13.6) with ESMTP id o1IFaEAA073419 for ; Thu, 18 Feb 2010 07:36:14 -0800 (PST) (envelope-from mahan@mahan.org) Message-ID: <4B7D5DA0.1020500@mahan.org> Date: Thu, 18 Feb 2010 07:32:48 -0800 From: Patrick Mahan User-Agent: Thunderbird 2.0.0.22 (X11/20090605) MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Issues with em(4) device under FreeBSD 8.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 15:32:54 -0000 All, I have seen a few mentions on the mailing lists in regard to issues with em(4) and FreeBSD 8.0 with regard to throughput. We are also seeing similar issues on HP Proliant systems with this HP GE interfaces. Previously we were running FreeBSD 6.2 and iperf was showing ~900 Mbits/sec between two directly connected systems. After the upgrade, iperf only shows around ~350 Mbits/sec. This seems only to be happening on the HP's. When we upgrade another x86 box (privately built) we are seeing ~900 Mbits/sec even to one of the HP systems. I haven't seen anything yet to account for this behavior. Has anyone else seen similar issues? Thanks, Patrick From owner-freebsd-net@FreeBSD.ORG Thu Feb 18 17:04:53 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1D1AD106566C for ; Thu, 18 Feb 2010 17:04:53 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [128.238.9.200]) by mx1.freebsd.org (Postfix) with ESMTP id B422A8FC16 for ; Thu, 18 Feb 2010 17:04:52 +0000 (UTC) Received: (qmail 16517 invoked from network); 18 Feb 2010 17:04:51 -0000 Received: from unknown (HELO ?10.0.0.158?) (spawk@128.238.64.31) by acm.poly.edu with AES256-SHA encrypted SMTP; 18 Feb 2010 17:04:51 -0000 Message-ID: <4B7D72BF.1040104@acm.poly.edu> Date: Thu, 18 Feb 2010 12:02:55 -0500 From: Boris Kochergin User-Agent: Thunderbird 2.0.0.23 (X11/20091021) MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: CARP vs. if_bridge X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 17:04:53 -0000 Ahoy. I'm seeing what appears to be erroneous interaction between CARP and if_bridge on multiple machines with a variety of Ethernet controllers and architectures. I've observed it on 7.2-R and 8.0-R. The test setup is simple enough: CARP master: FreeBSD t30 8.0-RELEASE-p1 FreeBSD 8.0-RELEASE-p1 #5: Sun Feb 14 20:22:41 EST 2010 root@t30:/usr/obj/usr/src/sys/T30 i386 lo0: flags=8049 metric 0 mtu 16384 options=3 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet6 ::1 prefixlen 128 inet 127.0.0.1 netmask 0xff000000 dc0: flags=8943 metric 0 mtu 1500 options=8 ether 00:04:5a:a8:e0:bf inet 192.168.0.2 netmask 0xffffff00 broadcast 192.168.0.255 media: Ethernet autoselect (100baseTX ) status: active carp0: flags=49 metric 0 mtu 1500 inet 192.168.0.1 netmask 0xffffff00 carp: MASTER vhid 1 advbase 1 advskew 0 CARP backup: FreeBSD ultra5 8.0-RELEASE FreeBSD 8.0-RELEASE #0: Thu Feb 18 15:19:39 UTC 2010 boris@ultra5:/usr/obj/usr/src/sys/GENERIC.carp sparc64 hme0: flags=8802 metric 0 mtu 1500 options=b ether 08:00:20:f5:65:d4 media: Ethernet autoselect xl0: flags=8943 metric 0 mtu 1500 options=9 ether 00:01:03:2c:06:6d inet 192.168.0.3 netmask 0xffffff00 broadcast 192.168.0.255 media: Ethernet autoselect (100baseTX ) status: active lo0: flags=8049 metric 0 mtu 16384 options=3 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 inet6 ::1 prefixlen 128 inet 127.0.0.1 netmask 0xff000000 carp0: flags=49 metric 0 mtu 1500 inet 192.168.0.1 netmask 0xffffff00 carp: MASTER vhid 1 advbase 1 advskew 100 bridge0: flags=8843 metric 0 mtu 1500 ether 3a:e6:09:2d:da:bc id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: xl0 flags=143 ifmaxaddr 0 port 2 priority 128 path cost 200000 member: hme0 flags=8 ifmaxaddr 0 port 1 priority 128 path cost 200000 In summary, I have a basic CARP configuration and, on the backup CARP machine, a bridge with the CARP device's physical interface in it. The purpose of this setup is the ability to monitor traffic passing through that interface using another machine. If the master CARP machine is disconnected from the network, the CARP interface on the backup machine correctly changes to the MASTER state, but does not act on traffic bound for the shared IP address--192.168.0.1. tcpdump shows the traffic coming in on the correct physical interface, but it is never replied to, or, in the case of routing, forwarded. Removing xl0 from the bridge on the backup machine instantly fixes this, and the shared IP address behaves as expected. Adding xl0 back to the bridge while the backup CARP interface is in the MASTER state keeps things running correctly, so the problem is only observed when xl0 is part of the bridge during the CARP transition from BACKUP to MASTER. Thoughts? -Boris From owner-freebsd-net@FreeBSD.ORG Thu Feb 18 18:58:17 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1376B106566B for ; Thu, 18 Feb 2010 18:58:17 +0000 (UTC) (envelope-from egorenar@googlemail.com) Received: from ey-out-2122.google.com (ey-out-2122.google.com [74.125.78.27]) by mx1.freebsd.org (Postfix) with ESMTP id A16598FC08 for ; Thu, 18 Feb 2010 18:58:16 +0000 (UTC) Received: by ey-out-2122.google.com with SMTP id 9so1385081eyd.9 for ; Thu, 18 Feb 2010 10:58:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=p2s64Tf+FxfYFHxGyRRTt5chtOiAwX2/EFeMfBCUsXg=; b=GsXqvD6gLAaBMFBVqBYLuygIrXxXB5QtGCN6D9YAIl4/KxKVyIu/ld2sj/8G9wQS6S yQPRHyW68q3QgP5fetsxqWs9S0+GqjdQJyo3kF5wBdkKOmqfNFxCDzekdAd5m6wSEmt5 gD7eAjgP+CYewMtXFRB/h2O3B87Mw5ytqerUE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=iKuqTZWOPCsbI5U4GIYbmZPibo5mbJ49UJUSw41AxlTqe/Roe/7GWQ3vEUluGc+pwB u2DqZa14YcuKXdIHIJD0xVi/DI5axlZCHCjnzYX0DjIxUteGAfqfvEIFWNntxsrdl5Dq 32eGFwsxDDcXAQ+9kvIcqnoKH0McSFiumadRQ= MIME-Version: 1.0 Received: by 10.213.0.212 with SMTP id 20mr7153446ebc.41.1266519495475; Thu, 18 Feb 2010 10:58:15 -0800 (PST) Date: Thu, 18 Feb 2010 19:58:15 +0100 Message-ID: <2d3b7e441002181058u5006595an5c636843945bcca7@mail.gmail.com> From: Alexander Egorenkov To: freebsd-net@freebsd.org X-Mailman-Approved-At: Thu, 18 Feb 2010 19:09:21 +0000 Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Missing WME information element causes problems with 802.11n X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 18:58:17 -0000 I have encountered a problem with a 802.11n router Belkin F5D8631au. The beacon and association response frames sent by this router do not contain WME information element although 802.11n mode is enabled. These frames contain HT capability IE and HT info. Because WME IE is missing in association responses, the net80211 stack does not set IEEE80211_NODE_QOS flag (See ieee80211_sta.c:sta_recv_mgmt:IEEE80211_FC0_SUBTYPE_ASSOC_RESP). But the flag IEEE80211_NODE_HT is set because the frame contains HT capability and HT info. So, because IEEE80211_NODE_QOS is not set, all outgoing DATA frames sent to the Belkin AP do not contain QoS field in the 802.11 frame header. And it causes problems with the Belkin AP. Is the QoS not mandatory for 802.11n mode ? Why is QoS enabled only if an WME IE is found in association response ? Would it be not right to enable QoS also if HT mode is enabled but no WME IE was found ? Thanks. From owner-freebsd-net@FreeBSD.ORG Thu Feb 18 19:08:18 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8F8C6106568F for ; Thu, 18 Feb 2010 19:08:18 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3E7D58FC12 for ; Thu, 18 Feb 2010 19:08:17 +0000 (UTC) Received: by vws14 with SMTP id 14so861589vws.13 for ; Thu, 18 Feb 2010 11:08:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=lS5MQijfVUfBFSPQZXezzG32M7KRauwCbX/JWRGzAMM=; b=etyC51W5AdzGac3KAD3DCp+cv6elihPCVXBQLMkoo20WhvVohHk/LfpXm7A+MRe1N+ zb5trGCEvDhYA27oXCfp4cz4A8rTwTAv8XSOf0felxX8Xe/y0wvRWh83LBw4YGoB2s/O /ICQSNWRAmDR+2vW/BPazMRrenEjdmN0znWqo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=bLFqTaD8fQ6qCTJnqSiAjd2fgUxgQJekNTLCsZMPmW51Vb3KreLuCsU70FD1tLKK61 W/bvuPr65inVKdCY4tyafDZXXpTTQZUdXmhUFZZv1Th/yEsnvqFPtG82sMzMbCZQi/qX XNcdeFXHuATolyw/lREDP4m339VMM/lkqbaH8= Received: by 10.220.126.167 with SMTP id c39mr3352717vcs.131.1266520097294; Thu, 18 Feb 2010 11:08:17 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 23sm29659360vws.6.2010.02.18.11.08.15 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 18 Feb 2010 11:08:16 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Thu, 18 Feb 2010 11:07:36 -0800 From: Pyun YongHyeon Date: Thu, 18 Feb 2010 11:07:36 -0800 To: Patrick Mahan Message-ID: <20100218190736.GA11675@michelle.cdnetworks.com> References: <4B7D5DA0.1020500@mahan.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B7D5DA0.1020500@mahan.org> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org Subject: Re: Issues with em(4) device under FreeBSD 8.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 19:08:18 -0000 On Thu, Feb 18, 2010 at 07:32:48AM -0800, Patrick Mahan wrote: > All, > > I have seen a few mentions on the mailing lists in regard to issues > with em(4) and FreeBSD 8.0 with regard to throughput. > > We are also seeing similar issues on HP Proliant systems with > this HP GE interfaces. Previously we were running FreeBSD 6.2 and > iperf was showing ~900 Mbits/sec between two directly connected > systems. After the upgrade, iperf only shows around ~350 Mbits/sec. > > This seems only to be happening on the HP's. When we upgrade another > x86 box (privately built) we are seeing ~900 Mbits/sec even to > one of the HP systems. > > I haven't seen anything yet to account for this behavior. Has anyone > else seen similar issues? > I know there is a possible Tx checksum offloading issue but testing with iperf may not hit the case. One of the big change made in 8.x is switching to buf_ring which will take advantage of multi TX queues. So the only guess I have is TCP segment reordering caused by em(4) and buf_ring interface. Can you capture the traffic on receiver side and check whether you see out-of-ordered TCP segment delivery? If the theory is right you may see the lots of out-of-ordered segments(this could be easily checked on receiver side with netstat -s after clearing the stats) and it will be more frequently seen when TCP window scaling is used. Personally I had trouble to reproduce it on my environments. It may depend on specific workloads. > Thanks, > > Patrick From owner-freebsd-net@FreeBSD.ORG Thu Feb 18 23:18:53 2010 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 47858106566B for ; Thu, 18 Feb 2010 23:18:53 +0000 (UTC) (envelope-from wxs@atarininja.org) Received: from syn.atarininja.org (syn.csh.rit.edu [129.21.50.215]) by mx1.freebsd.org (Postfix) with ESMTP id 26F708FC0A for ; Thu, 18 Feb 2010 23:18:52 +0000 (UTC) Received: by syn.atarininja.org (Postfix, from userid 1001) id 603A75C18; Thu, 18 Feb 2010 18:02:31 -0500 (EST) Date: Thu, 18 Feb 2010 18:02:31 -0500 From: Wesley Shields To: freebsd-net@FreeBSD.org Message-ID: <20100218230231.GB53467@atarininja.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Subject: Please review: kern/143855 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 23:18:53 -0000 I'm looking for someone with knowledge of BPF to review this patch: http://people.freebsd.org/~wxs/bpf.diff It's basically a fixed version of the one in kern/143855. Using the test program provided I was able to get the callback to be called with this patch. Without it the callback was never called. If I can get someone to review it I would like to commit it (I'm not a src committer) and eventually MFC it. Anyone want to work with me on fixing this easy problem? -- WXS From owner-freebsd-net@FreeBSD.ORG Fri Feb 19 01:05:06 2010 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 028331065676; Fri, 19 Feb 2010 01:05:06 +0000 (UTC) (envelope-from sobomax@FreeBSD.org) Received: from sippysoft.com (gk1.360sip.com [72.236.70.240]) by mx1.freebsd.org (Postfix) with ESMTP id 5ABCA8FC0C; Fri, 19 Feb 2010 01:05:05 +0000 (UTC) Received: from [192.168.1.38] ([70.71.167.197]) (authenticated bits=0) by sippysoft.com (8.14.3/8.14.3) with ESMTP id o1J151pb046553 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 18 Feb 2010 17:05:02 -0800 (PST) (envelope-from sobomax@FreeBSD.org) Message-ID: <4B7DE3CC.7040704@FreeBSD.org> Date: Thu, 18 Feb 2010 17:05:16 -0800 From: Maxim Sobolev Organization: Sippy Software, Inc. User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: Sergey Babkin References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> <4B7ADFC6.7020202@FreeBSD.org> In-Reply-To: <4B7ADFC6.7020202@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Alfred Perlstein , freebsd-net@FreeBSD.org, "David G. Lawrence" , Jack Vogel , FreeBSD Hackers Subject: Re: Sudden mbuf demand increase and shortage under the load (igb issue?) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 01:05:06 -0000 Folks, Indeed, it looks like igb(4) issue. Replacing the card with the desktop-grade em(4)-supported card has fixed the problem for us. The system has been happily pushing 110mbps worth of RTP traffic and 2000 concurrent calls without any problems for two days now. em0@pci0:7:0:0: class=0x020000 card=0xa01f8086 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' class = network subclass = ethernet em0: port 0xec00-0xec1f mem 0xfbee0000-0xfbefffff,0xfbe00000-0xfbe7ffff,0xfbedc000-0xfbedffff irq 24 at device 0.0 on pci7 em0: Using MSIX interrupts em0: [ITHREAD] em0: [ITHREAD] em0: [ITHREAD] em0: Ethernet address: 00:1b:21:50:02:49 I really think that this has to be addressed before 7.3 release is out. FreeBSD used to be famous for its excellent network performance and it's shame to see that deteriorating due to sub-standard quality drivers. Especially when there is a multi-billion vendor supporting the driver in question. No finger pointing, but it really looks like either somebody is not doing his job or the said vendor doesn't care so much about supporting FreeBSD. I am pretty sure the vendor in question has access to numerous load-testing tools, that should have catched this issue. This is the second time during the past 6 months I have issue with the quality of the Intel-based drivers - the first one is filed as kern/140326, which has stalled apparently despite me providing all necessary debug information. Regards, -- Maksym Sobolyev Sippy Software, Inc. Internet Telephony (VoIP) Experts T/F: +1-646-651-1110 Web: http://www.sippysoft.com MSN: sales@sippysoft.com Skype: SippySoft From owner-freebsd-net@FreeBSD.ORG Fri Feb 19 03:43:37 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 886FA106566C; Fri, 19 Feb 2010 03:43:37 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id E0E8F8FC0A; Fri, 19 Feb 2010 03:43:36 +0000 (UTC) Received: by vws14 with SMTP id 14so1034266vws.13 for ; Thu, 18 Feb 2010 19:43:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=nhp52rbB1zv6YcnoYDDpyul2eQw7N/Erc47mTnTIPTc=; b=uo9cJIR6hwUMRnC5KpCas3m0Cuvl2JKLuREF87gN2Rw+iuXkM7efx1jtvczTOAIzTX 0oZH0VSoS9BHvLhdq5jlFBBIR8fjskxAsvksQ8LT+NXxYOMlVsbBujYauAu56Mc3QC+g qvSgctcT/v7FvkRpgr/zjaie420KL4Lc3klTE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=geHJnWcQGOk3hPXobx7Sj0l4l6C/k1+ZO7HbsgCgkCR2Hy0vVAYfqNy6Hk9vdd6tSm yB+7axgQJ8eVnEGYD6u6bIj1d5hM/uRu4xT62ykLD8obR2zBQL9LEoc2CHAcfNlfpqaV BdlqseZ878HS7XTc5hIi1IssathshIsbsfbqo= Received: by 10.220.127.4 with SMTP id e4mr3740751vcs.79.1266551016030; Thu, 18 Feb 2010 19:43:36 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 30sm33174334vws.1.2010.02.18.19.43.33 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 18 Feb 2010 19:43:34 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Thu, 18 Feb 2010 19:42:55 -0800 From: Pyun YongHyeon Date: Thu, 18 Feb 2010 19:42:55 -0800 To: Maxim Sobolev Message-ID: <20100219034255.GG11675@michelle.cdnetworks.com> References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> <4B7ADFC6.7020202@FreeBSD.org> <4B7DE3CC.7040704@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="Md/poaVZ8hnGTzuv" Content-Disposition: inline In-Reply-To: <4B7DE3CC.7040704@FreeBSD.org> User-Agent: Mutt/1.4.2.3i Cc: Sergey Babkin , freebsd-net@freebsd.org, Alfred Perlstein , Jack Vogel , FreeBSD Hackers , "David G. Lawrence" Subject: Re: Sudden mbuf demand increase and shortage under the load (igb issue?) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 03:43:37 -0000 --Md/poaVZ8hnGTzuv Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Thu, Feb 18, 2010 at 05:05:16PM -0800, Maxim Sobolev wrote: > Folks, > > Indeed, it looks like igb(4) issue. Replacing the card with the > desktop-grade em(4)-supported card has fixed the problem for us. The > system has been happily pushing 110mbps worth of RTP traffic and 2000 > concurrent calls without any problems for two days now. > > em0@pci0:7:0:0: class=0x020000 card=0xa01f8086 chip=0x10d38086 rev=0x00 > hdr=0x00 > vendor = 'Intel Corporation' > class = network > subclass = ethernet > > em0: port 0xec00-0xec1f mem > 0xfbee0000-0xfbefffff,0xfbe00000-0xfbe7ffff,0xfbedc000-0xfbedffff irq 24 > at device 0.0 on pci7 > em0: Using MSIX interrupts > em0: [ITHREAD] > em0: [ITHREAD] > em0: [ITHREAD] > em0: Ethernet address: 00:1b:21:50:02:49 > > I really think that this has to be addressed before 7.3 release is out. > FreeBSD used to be famous for its excellent network performance and it's > shame to see that deteriorating due to sub-standard quality drivers. > Especially when there is a multi-billion vendor supporting the driver in > question. No finger pointing, but it really looks like either somebody > is not doing his job or the said vendor doesn't care so much about > supporting FreeBSD. I am pretty sure the vendor in question has access > to numerous load-testing tools, that should have catched this issue. > > This is the second time during the past 6 months I have issue with the > quality of the Intel-based drivers - the first one is filed as > kern/140326, which has stalled apparently despite me providing all > necessary debug information. > I can reproduce this bug on my box and I guess the root cause comes from PBA(Packet Buffer Allocation) configuration. Some controllers seems to require more TX buffer size to use 9000 MTU. The datasheet is not clear which controller has how much amount of Packet Buffer storage. This parameter seems to affect performance a lot because increasing TX buffer size results in decreasing RX buffer size. The attached patch seems to fix the issue for me but Jack may know better the hardware details as publicly available datasheet seems to be useless here. --Md/poaVZ8hnGTzuv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="em.ich10.jumbo.diff" Index: sys/dev/e1000/if_em.c =================================================================== --- sys/dev/e1000/if_em.c (revision 204011) +++ sys/dev/e1000/if_em.c (working copy) @@ -1384,7 +1384,12 @@ case e1000_ich9lan: case e1000_ich10lan: case e1000_pchlan: +#if 1 + /* Make 9000 MTU work. */ + pba = E1000_PBA_18K; +#else pba = E1000_PBA_10K; +#endif break; case e1000_ich8lan: pba = E1000_PBA_8K; --Md/poaVZ8hnGTzuv-- From owner-freebsd-net@FreeBSD.ORG Fri Feb 19 06:36:42 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BB500106566B for ; Fri, 19 Feb 2010 06:36:42 +0000 (UTC) (envelope-from dahayes@dhayes.caia.swin.edu.au) Received: from dhayes.caia.swin.edu.au (dhayes.caia.swin.edu.au [136.186.229.107]) by mx1.freebsd.org (Postfix) with ESMTP id EB7728FC0C for ; Fri, 19 Feb 2010 06:36:41 +0000 (UTC) Received: from dhayes.caia.swin.edu.au (localhost [127.0.0.1]) by dhayes.caia.swin.edu.au (8.14.3/8.14.3) with ESMTP id o1J6Hrl7017178; Fri, 19 Feb 2010 17:17:53 +1100 (EST) (envelope-from dahayes@dhayes.caia.swin.edu.au) Received: (from dahayes@localhost) by dhayes.caia.swin.edu.au (8.14.3/8.14.3/Submit) id o1J6HPiX017116; Fri, 19 Feb 2010 17:17:25 +1100 (EST) (envelope-from dahayes) Date: Fri, 19 Feb 2010 17:17:25 +1100 From: David Hayes To: end2end-interest@postel.org, iccrg@cs.ucl.ac.uk, freebsd-net@freebsd.org Message-ID: <20100219061725.GA91121@dhayes.caia.swin.edu.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Cc: Lawrence Stewart , grenville armitage Subject: Software for FreeBSD TCP R&D X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 06:36:42 -0000 Hi All, We're pleased to announce the release of a substantial set of new and updated software for TCP research using FreeBSD: - Hamilton Delay Congestion Control Algorithm v0.1 (new) - Vegas Congestion Control Algorithm v0.1 (new) - ERTT Khelp Module v0.1 (new) - Khelp Framework for FreeBSD v0.1.0 (new) - Modular TCP Congestion Control for FreeBSD v0.9.4 (update) Everything is available from [1] as BSD licenced source, along with other papers, patches and software relevant/useful for protocol analysis, debugging and experimental research. Hamilton Delay Congestion Control Algorithm v0.1 [2,3] ------------------------------------------------ A FreeBSD loadable kernel module that implements an experimental delay based congestion control algorithm proposed by the Hamilton Institute [4]. It provides a first step toward the ability of delay based algorithms to fairly coexist with loss based algorithms. Vegas Congestion Control Algorithm v0.1 [5,6] -------------------------------------- A FreeBSD loadable kernel module that implements the VEGAS congestion control algorithm [7]. ERTT Khelp Module v0.1 [8,9] ---------------------- A FreeBSD loadable kernel module that provides Enhanced Round Trip Time (ERTT) measurements for use by delay and rate based TCP congestion control algorithms. Robust estimates of RTT are provided even when the receiver uses delayed acknowledgements, TCP segmentation offload (TSO) or Selective Acknowledgements (SACK). This module is required by the delay based congestion control algorithms. Khelp Framework for FreeBSD v0.1.0 [10,11] ---------------------------------- A FreeBSD kernel patch that provides support for generic kernel modules known as "helpers" to hook into arbitrary points within the kernel and provide service(s) to the running system. This forms the foundation for the ERTT Khelp module. Modular TCP Congestion Control for FreeBSD v0.9.4 [12,13] ------------------------------------------------- This revision of the Modular TCP Congestion control framework patch provides a current snapshot of the project's subversion branch, with numerous tweaks, bug fixes and enhancements to support the new delay based congestion control algorithms we have been working on. This software was developed as part of the NewTCP research project at Swinburne University's Centre for Advanced Internet Architectures, made possible in part by a grant from the Cisco University Research Program Fund at Community Foundation Silicon Valley. Testing and development was further assisted by a grant from the FreeBSD Foundation. We welcome any feedback and hope you enjoy playing with the code and tools. Cheers, David Hayes, Lawrence Stewart and Grenville Armitage http://caia.swin.edu.au [1] http://caia.swin.edu.au/urp/newtcp/tools.html [2] http://caia.swin.edu.au/newtcp/tools/cc_hd-0.1.tar.gz [3] http://caia.swin.edu.au/newtcp/tools/cc_hd-readme-0.1.txt [4] L. Budzisz, R. Stanojevic, R. Shorten, and F. Baker, "A strategy for fair coexistence of loss and delay-based congestion control algorithms", IEEE Commun. Lett., vol. 13, no. 7, pp. 555-557, Jul. 2009. [5] http://caia.swin.edu.au/newtcp/tools/cc_vegas-0.1.tar.gz [6] http://caia.swin.edu.au/newtcp/tools/cc_vegas-readme-0.1.txt [7] L. S. Brakmo and L. L. Peterson, "TCP Vegas: end to end congestion avoidance on a global internet", IEEE J. Sel. Areas Commun., vol. 13, no. 8, pp. 1465-1480, Oct. 1995. [8] http://caia.swin.edu.au/newtcp/tools/h_ertt-0.1.tar.gz [9] http://caia.swin.edu.au/newtcp/tools/h_ertt-readme-0.1.txt [10] http://caia.swin.edu.au/newtcp/tools/caia_khelp_framework_v0.1.0_9.x.r203910.patch [11] http://caia.swin.edu.au/newtcp/tools/khelp-readme-0.1.0.txt [12] http://caia.swin.edu.au/newtcp/tools/caia_modularcc_v0.9.4_9.x.r203910.patch [13] http://caia.swin.edu.au/newtcp/tools/modularcc-readme-0.9.4.txt -- David Hayes Research Fellow Centre for Advanced Internet Architectures Faculty of Information and Communication Technologies Swinburne University of Technology, Australia http://caia.swin.edu.au/cv/dahayes/ From owner-freebsd-net@FreeBSD.ORG Fri Feb 19 07:27:53 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A3BA91065679 for ; Fri, 19 Feb 2010 07:27:53 +0000 (UTC) (envelope-from frank.schuster01@web.de) Received: from fmmailgate04.web.de (fmmailgate04.web.de [217.72.192.242]) by mx1.freebsd.org (Postfix) with ESMTP id 689168FC0A for ; Fri, 19 Feb 2010 07:27:53 +0000 (UTC) Received: from web.de by fmmailgate04.web.de (Postfix) with SMTP id 244AC64F938E; Fri, 19 Feb 2010 08:27:52 +0100 (CET) Received: from [109.40.144.236] by freemailng1206.web.de with HTTP; Fri, 19 Feb 2010 08:27:50 +0100 Date: Fri, 19 Feb 2010 08:27:50 +0100 Message-Id: <2002812000@web.de> MIME-Version: 1.0 From: frank.schuster01@web.de To: =?iso-8859-15?Q?Michael_T=FCxen?= Precedence: fm-user Organization: http://freemail.web.de/ X-Provags-Id: V01U2FsdGVkX1/xSaZe5R/AmfHkEIrqsJsR4l0DS7dTC6c6cXTJ4hkNCbKwm 2Avm8bvsEmG8gC7aOMKOQwS2i8l0aUs5XjY/CB5qUxoMej7zzwZ99DyVPdwp w== Content-Type: text/plain; charset=iso-8859-15 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org Subject: Re: Wireshark, netperf and sctp problem X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 07:27:53 -0000 Hi Michael, yeah, I see dropped packages. But did you mean only that wireshark didn't capture all packages on ubuntu= =3F Or did you mean that the packages will be really dropped by the kernel=3F Because I think it's the first one, can I display the dropped packages - i= s there any configuration for wireshark=3F Regards Frank > Hi Frank, >=20 > you might want to send both wireshark trace to me and I would have a loo= k. >=20 > If it is a bulk transfer, maybe the Ubuntu machine did not capture all p= ackets. > The number of dropped packets is displayed when you stop capturing. >=20 > Best regards > Michael >=20 > On Feb 18, 2010, at 3:35 PM, frank.schuster01@web.de wrote: >=20 > > Hello, > >=20 > > I have a problem with my little network test. > > I want to test sctp between ubuntu and freebsd. > > On ubuntu I wrote a iptables rule which delete every tenth (10) data p= ackage. Freebsd has no firewall rules. > > But only ubuntu sends data and freebsd only acknowledge this data (i d= id the test with netperf). > >=20 > > On the sending side (ubuntu) I see in wireshark 24 data packages, and = on the receiving side (freebsd) I saw 82 packages. > > But I can't explain, why freebsd receives more packages as delivered f= rom ubuntu=3F (I have currently only one freebsd system). > >=20 > > Any ideas or something additional notes to help me=3F > >=20 > > Is the problem by wireshark or the kernel implementation=3F > >=20 > > Regards > > Frank =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F NEU: Mit WEB.DE DSL =FCber 1000,- =BF sparen! http://produkte.web.de/go/02/ From owner-freebsd-net@FreeBSD.ORG Fri Feb 19 07:43:22 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BAFFB1065670; Fri, 19 Feb 2010 07:43:22 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-ew0-f225.google.com (mail-ew0-f225.google.com [209.85.219.225]) by mx1.freebsd.org (Postfix) with ESMTP id C64038FC08; Fri, 19 Feb 2010 07:43:21 +0000 (UTC) Received: by ewy25 with SMTP id 25so799632ewy.3 for ; Thu, 18 Feb 2010 23:43:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=gS2pZogCVsn7ZQAc+VM2BGGhSh+VhuZyej80Um4K6uc=; b=X2tOIdCfIMHw4bLGCa+sYOlxw/sTi7dQH0dQCxDVWBHOeDiaX83jxvublnkZ7x33uK QUD0x5DUBnugM0c9hUh3j9Wy576yJK1uxbi3TJ+BIjsuVffvWsovI6W1F2srQYCRoixv Kiig0AQDoZtNePO/N5Y3SxcZxX4afomRFGvQA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=SC6f/hSW8+2nJohdIu5Jz4EBRtL04TUmBo9+wMsLN5Kho7q3zx5ZaY56JIwzp4MQwC yLeN61gScXCKLmh+dwnS2bsjYWkO50BA3kqP0IcDe5336+KTB9/D82sduw8vE/IDBhhM eA9I/9Uv2ZbgaBkcPu/vS/n5R84WFdt856cR0= MIME-Version: 1.0 Received: by 10.216.90.137 with SMTP id e9mr1044955wef.141.1266565400684; Thu, 18 Feb 2010 23:43:20 -0800 (PST) In-Reply-To: <20100219034255.GG11675@michelle.cdnetworks.com> References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> <4B7ADFC6.7020202@FreeBSD.org> <4B7DE3CC.7040704@FreeBSD.org> <20100219034255.GG11675@michelle.cdnetworks.com> Date: Thu, 18 Feb 2010 23:43:20 -0800 Message-ID: <2a41acea1002182343i1ef0124fm8e2f6cc56846454c@mail.gmail.com> From: Jack Vogel To: pyunyh@gmail.com Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Maxim Sobolev , Sergey Babkin , FreeBSD Hackers , Alfred Perlstein , freebsd-net@freebsd.org, "David G. Lawrence" Subject: Re: Sudden mbuf demand increase and shortage under the load (igb issue?) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 07:43:22 -0000 This thread is confusing, first he says its an igb problem, then you offer an em patch :) I have an important rev of igb that I am about ready to release, anyone that wishes to test against a problem they have would be welcome to have early access, just let me know. I am not sure about this ich10 change, there are client NICs that specifically do NOT support jumbo frames, I'll need to look into it tomorrow at work. Jack On Thu, Feb 18, 2010 at 7:42 PM, Pyun YongHyeon wrote: > On Thu, Feb 18, 2010 at 05:05:16PM -0800, Maxim Sobolev wrote: > > Folks, > > > > Indeed, it looks like igb(4) issue. Replacing the card with the > > desktop-grade em(4)-supported card has fixed the problem for us. The > > system has been happily pushing 110mbps worth of RTP traffic and 2000 > > concurrent calls without any problems for two days now. > > > > em0@pci0:7:0:0: class=0x020000 card=0xa01f8086 chip=0x10d38086 rev=0x00 > > hdr=0x00 > > vendor = 'Intel Corporation' > > class = network > > subclass = ethernet > > > > em0: port 0xec00-0xec1f mem > > 0xfbee0000-0xfbefffff,0xfbe00000-0xfbe7ffff,0xfbedc000-0xfbedffff irq 24 > > at device 0.0 on pci7 > > em0: Using MSIX interrupts > > em0: [ITHREAD] > > em0: [ITHREAD] > > em0: [ITHREAD] > > em0: Ethernet address: 00:1b:21:50:02:49 > > > > I really think that this has to be addressed before 7.3 release is out. > > FreeBSD used to be famous for its excellent network performance and it's > > shame to see that deteriorating due to sub-standard quality drivers. > > Especially when there is a multi-billion vendor supporting the driver in > > question. No finger pointing, but it really looks like either somebody > > is not doing his job or the said vendor doesn't care so much about > > supporting FreeBSD. I am pretty sure the vendor in question has access > > to numerous load-testing tools, that should have catched this issue. > > > > This is the second time during the past 6 months I have issue with the > > quality of the Intel-based drivers - the first one is filed as > > kern/140326, which has stalled apparently despite me providing all > > necessary debug information. > > > > I can reproduce this bug on my box and I guess the root cause comes > from PBA(Packet Buffer Allocation) configuration. Some controllers > seems to require more TX buffer size to use 9000 MTU. The datasheet > is not clear which controller has how much amount of Packet Buffer > storage. This parameter seems to affect performance a lot because > increasing TX buffer size results in decreasing RX buffer size. The > attached patch seems to fix the issue for me but Jack may know > better the hardware details as publicly available datasheet seems > to be useless here. > From owner-freebsd-net@FreeBSD.ORG Fri Feb 19 07:59:20 2010 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1D937106566C; Fri, 19 Feb 2010 07:59:20 +0000 (UTC) (envelope-from sobomax@FreeBSD.org) Received: from sippysoft.com (gk1.360sip.com [72.236.70.240]) by mx1.freebsd.org (Postfix) with ESMTP id D1AD98FC19; Fri, 19 Feb 2010 07:59:19 +0000 (UTC) Received: from [192.168.1.38] ([70.71.167.197]) (authenticated bits=0) by sippysoft.com (8.14.3/8.14.3) with ESMTP id o1J7xGZJ048745 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 18 Feb 2010 23:59:17 -0800 (PST) (envelope-from sobomax@FreeBSD.org) Message-ID: <4B7E44E4.4090608@FreeBSD.org> Date: Thu, 18 Feb 2010 23:59:32 -0800 From: Maxim Sobolev Organization: Sippy Software, Inc. User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: Jack Vogel References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> <4B7ADFC6.7020202@FreeBSD.org> <4B7DE3CC.7040704@FreeBSD.org> <20100219034255.GG11675@michelle.cdnetworks.com> <2a41acea1002182343i1ef0124fm8e2f6cc56846454c@mail.gmail.com> In-Reply-To: <2a41acea1002182343i1ef0124fm8e2f6cc56846454c@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: pyunyh@gmail.com, Sergey Babkin , freebsd-net@FreeBSD.org, Alfred Perlstein , FreeBSD Hackers , "David G. Lawrence" Subject: Re: Sudden mbuf demand increase and shortage under the load (igb issue?) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 07:59:20 -0000 Jack Vogel wrote: > This thread is confusing, first he says its an igb problem, then you > offer an em patch :) I suspect it could be patch for the kern/140326. -Maxim From owner-freebsd-net@FreeBSD.ORG Fri Feb 19 13:21:31 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A2C01106566B for ; Fri, 19 Feb 2010 13:21:31 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.17.8]) by mx1.freebsd.org (Postfix) with ESMTP id 34B478FC12 for ; Fri, 19 Feb 2010 13:21:30 +0000 (UTC) Received: from vampire.homelinux.org (dslb-088-066-054-245.pools.arcor-ip.net [88.66.54.245]) by mrelayeu.kundenserver.de (node=mrbap1) with ESMTP (Nemesis) id 0M8K0I-1Nv1rf3fU2-00vl03; Fri, 19 Feb 2010 14:21:30 +0100 Received: (qmail 58640 invoked from network); 19 Feb 2010 13:21:29 -0000 Received: from f8x64.laiers.local (192.168.4.188) by ns1.laiers.local with SMTP; 19 Feb 2010 13:21:29 -0000 From: Max Laier Organization: FreeBSD To: freebsd-net@freebsd.org Date: Fri, 19 Feb 2010 14:21:28 +0100 User-Agent: KMail/1.12.4 (FreeBSD/8.0-RELEASE-p2; KDE/4.3.5; amd64; ; ) References: <4B7D72BF.1040104@acm.poly.edu> In-Reply-To: <4B7D72BF.1040104@acm.poly.edu> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201002191421.28699.max@love2party.net> X-Provags-ID: V01U2FsdGVkX18Cgm/lMqLdhhrHBnRZWxifm+AdoXto3zyB0tc 8BsVHYr+Xegzck4upVcwlber7q5lARFZyPMjyNtUCIXUZQcF+p G71f9r//uusZckwltLoPg== Cc: Boris Kochergin Subject: Re: CARP vs. if_bridge X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 13:21:31 -0000 On Thursday 18 February 2010 18:02:55 Boris Kochergin wrote: > Ahoy. I'm seeing what appears to be erroneous interaction between CARP > and if_bridge on multiple machines with a variety of Ethernet > controllers and architectures. I've observed it on 7.2-R and 8.0-R. The > test setup is simple enough: > > CARP master: > > FreeBSD t30 8.0-RELEASE-p1 FreeBSD 8.0-RELEASE-p1 #5: Sun Feb 14 > 20:22:41 EST 2010 root@t30:/usr/obj/usr/src/sys/T30 i386 > > lo0: flags=8049 metric 0 mtu 16384 > options=3 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 > inet6 ::1 prefixlen 128 > inet 127.0.0.1 netmask 0xff000000 > dc0: flags=8943 metric 0 > mtu 1500 > options=8 > ether 00:04:5a:a8:e0:bf > inet 192.168.0.2 netmask 0xffffff00 broadcast 192.168.0.255 > media: Ethernet autoselect (100baseTX ) > status: active > carp0: flags=49 metric 0 mtu 1500 > inet 192.168.0.1 netmask 0xffffff00 > carp: MASTER vhid 1 advbase 1 advskew 0 > > CARP backup: > > FreeBSD ultra5 8.0-RELEASE FreeBSD 8.0-RELEASE #0: Thu Feb 18 15:19:39 > UTC 2010 boris@ultra5:/usr/obj/usr/src/sys/GENERIC.carp sparc64 > > hme0: flags=8802 metric 0 mtu 1500 > options=b > ether 08:00:20:f5:65:d4 > media: Ethernet autoselect > xl0: flags=8943 metric 0 > mtu 1500 > options=9 > ether 00:01:03:2c:06:6d > inet 192.168.0.3 netmask 0xffffff00 broadcast 192.168.0.255 > media: Ethernet autoselect (100baseTX ) > status: active > lo0: flags=8049 metric 0 mtu 16384 > options=3 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 > inet6 ::1 prefixlen 128 > inet 127.0.0.1 netmask 0xff000000 > carp0: flags=49 metric 0 mtu 1500 > inet 192.168.0.1 netmask 0xffffff00 > carp: MASTER vhid 1 advbase 1 advskew 100 > bridge0: flags=8843 metric 0 mtu > 1500 > ether 3a:e6:09:2d:da:bc > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > member: xl0 flags=143 > ifmaxaddr 0 port 2 priority 128 path cost 200000 > member: hme0 flags=8 > ifmaxaddr 0 port 1 priority 128 path cost 200000 > > In summary, I have a basic CARP configuration and, on the backup CARP > machine, a bridge with the CARP device's physical interface in it. The > purpose of this setup is the ability to monitor traffic passing through > that interface using another machine. If the master CARP machine is > disconnected from the network, the CARP interface on the backup machine > correctly changes to the MASTER state, but does not act on traffic bound > for the shared IP address--192.168.0.1. tcpdump shows the traffic coming > in on the correct physical interface, but it is never replied to, or, in > the case of routing, forwarded. Removing xl0 from the bridge on the > backup machine instantly fixes this, and the shared IP address behaves > as expected. Adding xl0 back to the bridge while the backup CARP > interface is in the MASTER state keeps things running correctly, so the > problem is only observed when xl0 is part of the bridge during the CARP > transition from BACKUP to MASTER. Thoughts? I assume the bridge filters out the traffic as it thinks the destination is elsewhere (it has previously seen ARPs from the other MASTER entering via xl0). It shouldn't do that, but that's a different story. You can try to force edge or ptp status on xl0, not sure if this does the trick, but it's worth a try. Regards, Max From owner-freebsd-net@FreeBSD.ORG Fri Feb 19 13:41:41 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D612C1065672 for ; Fri, 19 Feb 2010 13:41:41 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-bw0-f216.google.com (mail-bw0-f216.google.com [209.85.218.216]) by mx1.freebsd.org (Postfix) with ESMTP id 5B62C8FC0A for ; Fri, 19 Feb 2010 13:41:40 +0000 (UTC) Received: by bwz8 with SMTP id 8so55068bwz.3 for ; Fri, 19 Feb 2010 05:41:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:to:cc:subject:references :organization:from:date:in-reply-to:message-id:user-agent :mime-version:content-type:content-transfer-encoding; bh=ybzX2xBzIWDWXxZza91utIzDB9rpimA6EDczgm6MhFY=; b=CK0L4bPlAg2iK6OHJWLgP3RQ3cCSWtpzLh7YXz7cuQaoCBdx0bwg7025b67/V84pz2 A6uMJ+vSKaKmaE+fvyC1QExxYWbunND1VX7GKmKDEUcV/zayrbsybQvE1eG6Z9wDr62B OY7kFHFvyIyVzrB+kILVDNi5Ag/cbLg5j/EKE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=to:cc:subject:references:organization:from:date:in-reply-to :message-id:user-agent:mime-version:content-type :content-transfer-encoding; b=w8BvmHkTLB5NtosbU8oYb++CtU070QqoPZuMSFaMxsKowjd53YZ/FIGea1m8nm9ylB 5445CAbZuJ3ch4agP5st5mzNLoNJAk+5iD95mRbhWUJx6WlMGcmk5ZMnrxA6kqw6zKH1 IM1Vf30qXccQA4pSlLi4/cBiT/jfKVyDY9/ZM= Received: by 10.204.32.198 with SMTP id e6mr3552229bkd.169.1266586899946; Fri, 19 Feb 2010 05:41:39 -0800 (PST) Received: from localhost (ms.singlescrowd.net [80.85.90.67]) by mx.google.com with ESMTPS id 13sm59183bwz.3.2010.02.19.05.41.37 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 19 Feb 2010 05:41:38 -0800 (PST) To: Nikos Vassiliadis References: <20100217132632.GA756@crete.org.ua> <4B7D5D95.20007@gmx.com> Organization: TOA Ukraine From: Mikolaj Golub Date: Fri, 19 Feb 2010 15:41:36 +0200 In-Reply-To: <4B7D5D95.20007@gmx.com> (Nikos Vassiliadis's message of "Thu\, 18 Feb 2010 17\:32\:37 +0200") Message-ID: <86bpflqr5b.fsf@zhuzha.ua1> User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 8bit Cc: freebsd-net@freebsd.org, Alexander Shikoff Subject: Re: mpd has hung X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 13:41:41 -0000 On Thu, 18 Feb 2010 17:32:37 +0200 Nikos Vassiliadis wrote: > On 2/17/2010 3:26 PM, Alexander Shikoff wrote: >> Hello All, >> >> I have mpd 5.3 running on 8.0-RC1 as PPPoE server (now only 5 clients). >> Today mpd process hung and I cannot kill it with -9 signal, and I cannot >> access it's console via telnet. >> >> State of process in `top` output is STOP: >> 73551 root 2 44 0 29588K 5692K STOP 6 0:32 0.00% mpd5 >> >> # procstat -kk 73551 >> PID TID COMM TDNAME KSTACK >> 73551 100233 mpd5 - mi_switch+0x16f sleepq_wait+0x42 _cv_wait+0x111 flowtable_flush+0x51 if_detach+0x2f2 ng_iface_shutdown+0x1e ng_rmnode+0x167 ng_apply_item+0xef7 ng_snd_item+0x2ce ngc_send+0x1d2 sosend_generic+0x3f6 kern_sendit+0x13d sendit+0xdc sendto+0x4d syscall+0x1da Xfast_syscall+0xe1 >> 73551 100502 mpd5 - mi_switch+0x16f thread_suspend_switch+0xc6 thread_single+0x1b6 exit1+0x72 sigexit+0x7c postsig+0x306 ast+0x279 doreti_ast+0x1f >> >> Is there a way to stop a process without rebooting a whole system? >> Thanks in advance! >> >> P.S. I'm ready for experiments with it before tonight, but I cannot >> force system to crash in order to get crash dump right now. >> > > It's probably too late now, but are you sure that nobody pressed > CTLR-Z while in the mpd console??? > > CTLR-Z will send SIGSTOP to the process and the process will > stop. While stopped, all processing stops(including receiving > SIGKILL, you cannot kill it, and the signals are queued). You > have to send SIGCONT for the process to continue. We were discussing this problem with Alexander in another (Russian/Ukrainian speaking) maillist. And it looks like the problem is the following. mpd5 thread was detaching ng interface and when doing flowtable_flush() it slept in cv_wait waiting for flowclean_cycles variable to be updated. It should have been awaken by flowcleaner thread but this thread got stuck in endless loop, supposedly in flowtable_clean_vnet()/flowtable_free_stale(), I think because of inconsistent state of some lists (iface?) due to if_detach being in progress. BTW the problem repeated after reboot. Below is some debugging info we have managed to get so far. So, stack traces of the threads of gotten stuck mpd5 process: PID TID COMM TDNAME KSTACK 73551 100233 mpd5 - mi_switch+0x16f sleepq_wait+0x42 _cv_wait+0x111 flowtable_flush+0x51 if_detach+0x2f2 ng_iface_shutdown+0x1e ng_rmnode+0x167 ng_apply_item+0xef7 ng_snd_item+0x2ce ngc_send+0x1d2 sosend_generic+0x3f6 kern_sendit+0x13d sendit+0xdc sendto+0x4d syscall+0x1da Xfast_syscall+0xe1 73551 100502 mpd5 - mi_switch+0x16f thread_suspend_switch+0xc6 thread_single+0x1b6 exit1+0x72 sigexit+0x7c postsig+0x306 ast+0x279 doreti_ast+0x1f proctat -k for flowcleaner: PID TID COMM TDNAME KSTACK 19 100088 flowcleaner - # top -S -H 10000 |grep 'flowcleaner\|73551' 19 root 44 - 0K 16K CPU2 2 23.8H 0.00% flowcleaner 73551 root 44 0 29588K 5692K STOP 6 0:32 0.00% {mpd5} 73551 root 44 0 29588K 5692K STOP 6 0:01 0.00% {mpd5} (kgdb) print flowclean_cycles $4 = 563 And several hours later: # top -S -H 10000 | grep 'flowcleaner\|73551' 19 root 44 - 0K 16K CPU2 2 28.6H 0.00% flowcleaner 73551 root 44 0 29588K 5692K STOP 6 0:32 0.00% {mpd5} 73551 root 44 0 29588K 5692K STOP 6 0:01 0.00% {mpd5} (kgdb) print flowclean_cycles $1 = 563 So, flowcleaner was in endless loop and as flowclean_cycles did not change it was somewhere in flowtable_clean_vnet()/flowtable_free_stale(). mdp5 thread was in cv_wait() (according to proctat -k) holding flowclean_lock: (kgdb) print flowclean_lock $1 = {lock_object = {lo_name = 0xffffffff804d4f3f "flowclean lock", lo_flags = 16973824, lo_data = 0, lo_witness = 0x0}, mtx_lock = 18446742977425993728} (kgdb) print *(struct thread *) (flowclean_lock->mtx_lock & ~0x3) $5 = {td_lock = 0xffffffff8068bec0, td_proc = 0xffffff0012c07000, td_plist = {tqe_next = 0x0, tqe_prev = 0xffffff00c06483a0}, td_runq = {tqe_next = 0x0, tqe_prev = 0xffffffff8068c128}, td_slpq = {tqe_next = 0x0, tqe_prev = 0xffffff00c0681240}, td_lockq = {tqe_next = 0x0, tqe_prev = 0xffffff80eba7f820}, td_cpuset = 0xffffff0001830dc8, td_sel = 0xffffff0031639400, td_sleepqueue = 0xffffff00c0681240, td_turnstile = 0xffffff00c0677900, td_umtxq = 0xffffff0031aea300, td_tid = 100408, td_sigqueue = {sq_signals = {__bits = {0, 0, 0, 0}}, sq_kill = {__bits = {0, 0, 0, 0}}, sq_list = {tqh_first = 0x0, tqh_last = 0xffffff00c06870a0}, sq_proc = 0xffffff0012c07000, sq_flags = 1}, td_flags = 34820, td_inhibitors = 0, td_pflags = 0, td_dupfd = 0, td_sqqueue = 0, td_wchan = 0x0, td_wmesg = 0x0, td_lastcpu = 0 '\0', td_oncpu = 0 '\0', td_owepreempt = 0 '\0', td_tsqueue = 0 '\0', td_locks = 30, td_rw_rlocks = 0, td_lk_slocks = 0, td_blocked = 0x0, td_lockname = 0x0, td_contested = {lh_first = 0x0}, td_sleeplocks = 0x0, td_intr_nesting_level = 0, td_pinned = 0, td_ucred = 0xffffff01d5ea7500, td_estcpu = 0, td_slptick = 0, td_ru = {ru_utime = {tv_sec = 0, tv_usec = 0}, ru_stime = {tv_sec = 0, tv_usec = 0}, ru_maxrss = 5484, ru_ixrss = 22943676, ru_idrss = 75964776, ru_isrss = 12339456, ru_minflt = 300, ru_majflt = 0, ru_nswap = 0, ru_inblock = 3, ru_oublock = 0, ru_msgsnd = 140, ru_msgrcv = 19, ru_nsignals = 0, ru_nvcsw = 87104012, ru_nivcsw = 5062}, td_incruntime = 273488034786, td_runtime = 850522426220, td_pticks = 48190, td_sticks = 15544, td_iticks = 0, td_uticks = 0, td_intrval = 0, td_oldsigmask = {__bits = {0, 0, 0, 0}}, td_sigmask = {__bits = {0, 0, 0, 0}}, td_generation = 87109074, td_sigstk = {ss_sp = 0x0, ss_size = 0, ss_flags = 0}, td_xsig = 0, td_profil_addr = 0, td_profil_ticks = 0, td_name = "mpd5", '\0' , td_fpop = 0x0, td_dbgflags = 0, td_ng_outbound = 0, td_osd = {osd_nslots = 0, osd_slots = 0x0, osd_next = {le_next = 0x0, le_prev = 0x0}}, td_rqindex = 34 '"', td_base_pri = 136 '\210', td_priority = 136 '\210', td_pri_class = 3 '\003', td_user_pri = 181 'µ', td_base_user_pri = 181 'µ', td_pcb = 0xffffff80ebb97d40, td_state = TDS_RUNNING, td_retval = {0, 56}, td_slpcallout = {c_links = {sle = {sle_next = 0xffffffff806ac720}, tqe = {tqe_next = 0xffffffff806ac720, tqe_prev = 0xffffff80a5610f90}}, c_time = 805103, c_arg = 0xffffff00c0687000, c_func = 0xffffffff80311050 , c_lock = 0x0, c_flags = 18, c_cpu = 0}, td_frame = 0xffffff80ebb97c80, td_kstack_obj = 0xffffff00c04fbca8, td_kstack = 18446743527908524032, td_kstack_pages = 4, td_unused1 = 0x0, td_unused2 = 0, td_unused3 = 0, td_critnest = 1, td_md = {md_spinlock_count = 1, md_saved_flags = 582}, td_sched = 0xffffff00c0687368, td_ar = 0x0, td_syscalls = 566, td_lprof = {{lh_first = 0x0}, { lh_first = 0x0}}, td_dtrace = 0x0, td_errno = 0, td_vnet = 0x0, td_vnet_lpush = 0x0} (kgdb) print *((struct thread *) (flowclean_lock->mtx_lock & ~0x3))->td_proc $6 = {p_list = {le_next = 0xffffff006eb2d000, le_prev = 0xffffff0012c06000}, p_threads = {tqh_first = 0xffffff00c0648390, tqh_last = 0xffffff00c0687010}, p_slock = { lock_object = {lo_name = 0xffffffff804c8a20 "process slock", lo_flags = 720896, lo_data = 0, lo_witness = 0x0}, mtx_lock = 4}, p_ucred = 0xffffff01d5ea7500, p_fd = 0xffffff0012f96a00, p_fdtol = 0x0, p_stats = 0xffffff00128ab200, p_limit = 0xffffff0012ab6200, p_limco = {c_links = {sle = {sle_next = 0x0}, tqe = { tqe_next = 0x0, tqe_prev = 0x0}}, c_time = 0, c_arg = 0x0, c_func = 0, c_lock = 0xffffff0012c070f8, c_flags = 0, c_cpu = 0}, p_sigacts = 0xffffff0004c09000, p_flag = 268960896, p_state = PRS_NORMAL, p_pid = 67727, p_hash = {le_next = 0x0, le_prev = 0xffffff800021e478}, p_pglist = {le_next = 0x0, le_prev = 0xffffff003163a110}, p_pptr = 0xffffff000183a000, p_sibling = {le_next = 0xffffff0004c54000, le_prev = 0xffffff000183a0f0}, p_children = { lh_first = 0x0}, p_mtx = {lock_object = {lo_name = 0xffffffff804c8a13 "process lock", lo_flags = 21168128, lo_data = 0, lo_witness = 0x0}, mtx_lock = 4}, p_ksi = 0xffffff0004af5380, p_sigqueue = {sq_signals = {__bits = {0, 0, 0, 0}}, sq_kill = {__bits = {0, 0, 0, 0}}, sq_list = {tqh_first = 0x0, tqh_last = 0xffffff0012c07140}, sq_proc = 0xffffff0012c07000, sq_flags = 1}, p_oppid = 0, p_vmspace = 0xffffff0012960dc8, p_swtick = 797875, p_realtimer = { it_interval = {tv_sec = 0, tv_usec = 0}, it_value = {tv_sec = 0, tv_usec = 0}}, p_ru = {ru_utime = {tv_sec = 0, tv_usec = 0}, ru_stime = {tv_sec = 0, tv_usec = 0}, ru_maxrss = 0, ru_ixrss = 0, ru_idrss = 0, ru_isrss = 0, ru_minflt = 168, ru_majflt = 0, ru_nswap = 0, ru_inblock = 4, ru_oublock = 0, ru_msgsnd = 47, ru_msgrcv = 6, ru_nsignals = 0, ru_nvcsw = 19, ru_nivcsw = 4}, p_rux = {rux_runtime = 577829035842, rux_uticks = 1, rux_sticks = 32657, rux_iticks = 0, rux_uu = 23270, rux_su = 231217022, rux_tu = 231224103}, p_crux = {rux_runtime = 3633037, rux_uticks = 0, rux_sticks = 1, rux_iticks = 0, rux_uu = 0, rux_su = 1453, rux_tu = 1453}, p_profthreads = 0, p_exitthreads = 0, p_traceflag = 0, p_tracevp = 0x0, p_tracecred = 0x0, p_textvp = 0xffffff01ff63c938, p_lock = 0, p_sigiolst = {slh_first = 0x0}, p_sigparent = 20, p_sig = 0, p_code = 0, p_stops = 0, p_stype = 0, p_step = 0 '\0', p_pfsflags = 0 '\0', p_nlminfo = 0x0, p_aioinfo = 0x0, p_singlethread = 0xffffff00c0648390, p_suspcount = 1, p_xthread = 0x0, p_boundary_count = 0, p_pendingcnt = 0, p_itimers = 0x0, p_magic = 3203398350, p_osrel = 800107, p_comm = "mpd5", '\0' , p_pgrp = 0xffffff003163a100, p_sysent = 0xffffffff8064bfc0, p_args = 0xffffff0004b629c0, p_cpulimit = 9223372036854775807, p_nice = 0 '\0', p_fibnum = 0, p_xstat = 0, p_klist = {kl_list = { slh_first = 0x0}, kl_lock = 0xffffffff802aaac0 , kl_unlock = 0xffffffff802aaa90 , kl_assert_locked = 0xffffffff802a7de0 , kl_assert_unlocked = 0xffffffff802a7df0 , kl_lockarg = 0xffffff0012c070f8}, p_numthreads = 2, p_md = {md_ldt = 0x0, md_ldt_sd = {sd_lolimit = 0, sd_lobase = 0, sd_type = 0, sd_dpl = 0, sd_p = 0, sd_hilimit = 0, sd_xx0 = 0, sd_gran = 0, sd_hibase = 0, sd_xx1 = 0, sd_mbz = 0, sd_xx2 = 0}}, p_itcallout = {c_links = {sle = {sle_next = 0x0}, tqe = { tqe_next = 0x0, tqe_prev = 0x0}}, c_time = 0, c_arg = 0x0, c_func = 0, c_lock = 0x0, c_flags = 16, c_cpu = 0}, p_acflag = 17, p_peers = 0x0, p_leader = 0xffffff0012c07000, p_emuldata = 0x0, p_label = 0x0, p_sched = 0xffffff0012c07460, p_ktr = {stqh_first = 0x0, stqh_last = 0xffffff0012c07430}, p_mqnotifier = {lh_first = 0x0}, p_dtrace = 0x0, p_pwait = {cv_description = 0xffffffff804c919f "ppwait", cv_waiters = 0}} Unfortunately there is no stack trace for flowcleaner. I have asked Alexander to make the kernel panic on the next reboot and provide backtrace of flowcleaner thread from the crush dump but I don't know if he has managed to do this (this is a production host, which complicates things). -- Mikolaj Golub From owner-freebsd-net@FreeBSD.ORG Fri Feb 19 14:43:28 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 47588106566B for ; Fri, 19 Feb 2010 14:43:28 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [128.238.9.200]) by mx1.freebsd.org (Postfix) with ESMTP id B52C28FC13 for ; Fri, 19 Feb 2010 14:43:27 +0000 (UTC) Received: (qmail 46155 invoked from network); 19 Feb 2010 14:43:26 -0000 Received: from unknown (HELO ?10.0.0.158?) (spawk@128.238.64.31) by acm.poly.edu with AES256-SHA encrypted SMTP; 19 Feb 2010 14:43:26 -0000 Message-ID: <4B7EA31A.3080204@acm.poly.edu> Date: Fri, 19 Feb 2010 09:41:30 -0500 From: Boris Kochergin User-Agent: Thunderbird 2.0.0.23 (X11/20091021) MIME-Version: 1.0 To: Max Laier References: <4B7D72BF.1040104@acm.poly.edu> <201002191421.28699.max@love2party.net> In-Reply-To: <201002191421.28699.max@love2party.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: CARP vs. if_bridge X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 14:43:28 -0000 Max Laier wrote: > On Thursday 18 February 2010 18:02:55 Boris Kochergin wrote: > >> Ahoy. I'm seeing what appears to be erroneous interaction between CARP >> and if_bridge on multiple machines with a variety of Ethernet >> controllers and architectures. I've observed it on 7.2-R and 8.0-R. The >> test setup is simple enough: >> >> CARP master: >> >> FreeBSD t30 8.0-RELEASE-p1 FreeBSD 8.0-RELEASE-p1 #5: Sun Feb 14 >> 20:22:41 EST 2010 root@t30:/usr/obj/usr/src/sys/T30 i386 >> >> lo0: flags=8049 metric 0 mtu 16384 >> options=3 >> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 >> inet6 ::1 prefixlen 128 >> inet 127.0.0.1 netmask 0xff000000 >> dc0: flags=8943 metric 0 >> mtu 1500 >> options=8 >> ether 00:04:5a:a8:e0:bf >> inet 192.168.0.2 netmask 0xffffff00 broadcast 192.168.0.255 >> media: Ethernet autoselect (100baseTX ) >> status: active >> carp0: flags=49 metric 0 mtu 1500 >> inet 192.168.0.1 netmask 0xffffff00 >> carp: MASTER vhid 1 advbase 1 advskew 0 >> >> CARP backup: >> >> FreeBSD ultra5 8.0-RELEASE FreeBSD 8.0-RELEASE #0: Thu Feb 18 15:19:39 >> UTC 2010 boris@ultra5:/usr/obj/usr/src/sys/GENERIC.carp sparc64 >> >> hme0: flags=8802 metric 0 mtu 1500 >> options=b >> ether 08:00:20:f5:65:d4 >> media: Ethernet autoselect >> xl0: flags=8943 metric 0 >> mtu 1500 >> options=9 >> ether 00:01:03:2c:06:6d >> inet 192.168.0.3 netmask 0xffffff00 broadcast 192.168.0.255 >> media: Ethernet autoselect (100baseTX ) >> status: active >> lo0: flags=8049 metric 0 mtu 16384 >> options=3 >> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 >> inet6 ::1 prefixlen 128 >> inet 127.0.0.1 netmask 0xff000000 >> carp0: flags=49 metric 0 mtu 1500 >> inet 192.168.0.1 netmask 0xffffff00 >> carp: MASTER vhid 1 advbase 1 advskew 100 >> bridge0: flags=8843 metric 0 mtu >> 1500 >> ether 3a:e6:09:2d:da:bc >> id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 >> maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200 >> root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 >> member: xl0 flags=143 >> ifmaxaddr 0 port 2 priority 128 path cost 200000 >> member: hme0 flags=8 >> ifmaxaddr 0 port 1 priority 128 path cost 200000 >> >> In summary, I have a basic CARP configuration and, on the backup CARP >> machine, a bridge with the CARP device's physical interface in it. The >> purpose of this setup is the ability to monitor traffic passing through >> that interface using another machine. If the master CARP machine is >> disconnected from the network, the CARP interface on the backup machine >> correctly changes to the MASTER state, but does not act on traffic bound >> for the shared IP address--192.168.0.1. tcpdump shows the traffic coming >> in on the correct physical interface, but it is never replied to, or, in >> the case of routing, forwarded. Removing xl0 from the bridge on the >> backup machine instantly fixes this, and the shared IP address behaves >> as expected. Adding xl0 back to the bridge while the backup CARP >> interface is in the MASTER state keeps things running correctly, so the >> problem is only observed when xl0 is part of the bridge during the CARP >> transition from BACKUP to MASTER. Thoughts? >> > > I assume the bridge filters out the traffic as it thinks the destination is > elsewhere (it has previously seen ARPs from the other MASTER entering via > xl0). It shouldn't do that, but that's a different story. You can try to > force edge or ptp status on xl0, not sure if this does the trick, but it's > worth a try. > > Regards, > Max > Sure. No go, though, I'm afraid. It's not an operational show-stopper for me, at least. In the worst case, I can always hack up a PCAP program to copy the frames around in user space. -Boris From owner-freebsd-net@FreeBSD.ORG Fri Feb 19 17:52:38 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A3BEF106566B; Fri, 19 Feb 2010 17:52:38 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-qy0-f183.google.com (mail-qy0-f183.google.com [209.85.221.183]) by mx1.freebsd.org (Postfix) with ESMTP id 0DCF58FC1D; Fri, 19 Feb 2010 17:52:37 +0000 (UTC) Received: by qyk13 with SMTP id 13so195235qyk.13 for ; Fri, 19 Feb 2010 09:52:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=W8Zj/EEqIbdkgFJwYHKKxZFwq/gDt86PoOFCFUuFgEQ=; b=HKcGOGrvUor6k49wbis9rM2NF+z1vyZJigFxJ5cPPbhyvpa+/MSaqXN0jPST2gFSjr sXY16uM5wpii2r9HzWi6vNC+WbtXiBRgAkTtZi+EsJsO5Mmw27O39k3ut4lKq6WWYD2X 4sfTAYpLWeNF8XuY77Ss1GvDsMvZURT9HEzfs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=G+bAMMPYMd2QyLOQK5lbb1mzdi0ln1V8vwCgw4XIkP5XmiuG3ulfs7onpBfcyDxtP7 4tvffD23is3VsBreuxFjdYvPaHMhblJUH3OHzsIhg1QPr8LWPFvlPu6hLkp6Bhh/JVFt 4zmzZYEvU9+9KjGa2wcI0Fp5cBH8jP/xP9Otc= Received: by 10.224.73.76 with SMTP id p12mr3612694qaj.187.1266601957068; Fri, 19 Feb 2010 09:52:37 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 6sm1221684qwk.10.2010.02.19.09.52.33 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 19 Feb 2010 09:52:35 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Fri, 19 Feb 2010 09:51:55 -0800 From: Pyun YongHyeon Date: Fri, 19 Feb 2010 09:51:55 -0800 To: Jack Vogel Message-ID: <20100219175155.GH11675@michelle.cdnetworks.com> References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> <4B7ADFC6.7020202@FreeBSD.org> <4B7DE3CC.7040704@FreeBSD.org> <20100219034255.GG11675@michelle.cdnetworks.com> <2a41acea1002182343i1ef0124fm8e2f6cc56846454c@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2a41acea1002182343i1ef0124fm8e2f6cc56846454c@mail.gmail.com> User-Agent: Mutt/1.4.2.3i Cc: Maxim Sobolev , Sergey Babkin , FreeBSD Hackers , Alfred Perlstein , freebsd-net@freebsd.org, "David G. Lawrence" Subject: Re: Sudden mbuf demand increase and shortage under the load (igb issue?) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 17:52:38 -0000 On Thu, Feb 18, 2010 at 11:43:20PM -0800, Jack Vogel wrote: > This thread is confusing, first he says its an igb problem, then you offer > an em patch :) > > I have an important rev of igb that I am about ready to release, anyone that > wishes to > test against a problem they have would be welcome to have early access, just > let me > know. > > I am not sure about this ich10 change, there are client NICs that > specifically do NOT > support jumbo frames, I'll need to look into it tomorrow at work. Hmm, then how could I see advertised MSS 8960 on this controller? I guess em(4) already checks MAC type to limit jumbo frame support on these controllers. 09:44:18.701271 IP (tos 0x0, ttl 64, id 32888, offset 0, flags [DF], proto TCP (6), length 60) 192.168.30.2.55754 > 192.168.30.1.22: S, cksum 0xbd82 (incorrect (-> 0x3f62), 529388419:529388419(0) win 65535 09:44:18.701538 IP (tos 0x0, ttl 64, id 40268, offset 0, flags [DF], proto TCP (6), length 60) 192.168.30.1.22 > 192.168.30.2.55754: S, cksum 0xa6ce (correct), 3828990973:3828990973(0) ack 529388420 win 65535 09:44:18.701549 IP (tos 0x0, ttl 64, id 32889, offset 0, flags [DF], proto TCP (6), length 52) 192.168.30.2.55754 > 192.168.30.1.22: ., cksum 0xbd7a (incorrect (-> 0xd2e1), 1:1(0) ack 1 win 8192 em0@pci0:0:25:0: class=0x020000 card=0x027f1028 chip=0x10de8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel Gigabit network connection (83567LM-3 )' class = network subclass = ethernet bar [10] = type Memory, range 32, base 0xfdae0000, size 131072, enabled bar [14] = type Memory, range 32, base 0xfdad9000, size 4096, enabled bar [18] = type I/O Port, range 32, base 0xecc0, size 32, enabled cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message cap 13[e0] = PCI Advanced Features: FLR TP > > Jack > > > On Thu, Feb 18, 2010 at 7:42 PM, Pyun YongHyeon wrote: > > > On Thu, Feb 18, 2010 at 05:05:16PM -0800, Maxim Sobolev wrote: > > > Folks, > > > > > > Indeed, it looks like igb(4) issue. Replacing the card with the > > > desktop-grade em(4)-supported card has fixed the problem for us. The > > > system has been happily pushing 110mbps worth of RTP traffic and 2000 > > > concurrent calls without any problems for two days now. > > > > > > em0@pci0:7:0:0: class=0x020000 card=0xa01f8086 chip=0x10d38086 rev=0x00 > > > hdr=0x00 > > > vendor = 'Intel Corporation' > > > class = network > > > subclass = ethernet > > > > > > em0: port 0xec00-0xec1f mem > > > 0xfbee0000-0xfbefffff,0xfbe00000-0xfbe7ffff,0xfbedc000-0xfbedffff irq 24 > > > at device 0.0 on pci7 > > > em0: Using MSIX interrupts > > > em0: [ITHREAD] > > > em0: [ITHREAD] > > > em0: [ITHREAD] > > > em0: Ethernet address: 00:1b:21:50:02:49 > > > > > > I really think that this has to be addressed before 7.3 release is out. > > > FreeBSD used to be famous for its excellent network performance and it's > > > shame to see that deteriorating due to sub-standard quality drivers. > > > Especially when there is a multi-billion vendor supporting the driver in > > > question. No finger pointing, but it really looks like either somebody > > > is not doing his job or the said vendor doesn't care so much about > > > supporting FreeBSD. I am pretty sure the vendor in question has access > > > to numerous load-testing tools, that should have catched this issue. > > > > > > This is the second time during the past 6 months I have issue with the > > > quality of the Intel-based drivers - the first one is filed as > > > kern/140326, which has stalled apparently despite me providing all > > > necessary debug information. > > > > > > > I can reproduce this bug on my box and I guess the root cause comes > > from PBA(Packet Buffer Allocation) configuration. Some controllers > > seems to require more TX buffer size to use 9000 MTU. The datasheet > > is not clear which controller has how much amount of Packet Buffer > > storage. This parameter seems to affect performance a lot because > > increasing TX buffer size results in decreasing RX buffer size. The > > attached patch seems to fix the issue for me but Jack may know > > better the hardware details as publicly available datasheet seems > > to be useless here. > > From owner-freebsd-net@FreeBSD.ORG Fri Feb 19 18:48:28 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7135E106566C for ; Fri, 19 Feb 2010 18:48:28 +0000 (UTC) (envelope-from rihad@mail.ru) Received: from mx39.mail.ru (mx39.mail.ru [94.100.176.53]) by mx1.freebsd.org (Postfix) with ESMTP id 2BC6B8FC15 for ; Fri, 19 Feb 2010 18:48:28 +0000 (UTC) Received: from [217.25.27.27] (port=60340 helo=[217.25.27.27]) by mx39.mail.ru with asmtp id 1NiXtq-000KeX-00 for freebsd-net@freebsd.org; Fri, 19 Feb 2010 21:48:26 +0300 Message-ID: <4B7EDD00.5080308@mail.ru> Date: Fri, 19 Feb 2010 22:48:32 +0400 From: rihad User-Agent: Thunderbird 2.0.0.23 (X11/20090817) MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam: Not detected X-Mras: Ok Subject: Slow speeds experienced with Dummynet X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rihad List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 18:48:28 -0000 Hi, all, Recalling my old posting "dummynet dropping too many packets" dated October 4, 2009, the problem isn't over just yet. This time, there are no interface i/o drops (just a reminder: we have 2 bce(4) GigE cards connected to a Cisco router, one for input, and one for output. The box itself does some traffic accounting and enforces speed limits w/ ipfw/dummynet. There are normally around 5-6k users online). Traffic shaping is accomplished by this ipfw rule: pipe tablearg ip from any to table(0) out where table(0) contains those 5-6k IP addresses. The pipes themselves are GRED (or taildrop, it doesn't matter): ipfw pipe 512 config bw 512kbit/s mask dst-ip 0xffffffff gred 0.002/900/1000/0.1 queue 1000 Taking this template the speeds range from 512 to tens of mbps. With the setup as above very many users complain about very slow downloads, slow browsing. systat -ifstat, refreshed every 5 seconds, does reveal large differences between subsequent displays: if around 800-850 mbps is what's to be expected, it doesn't stay within those limits, also jumping to as low as 620+, and to somewhere in the 700's, Now imagine this: once I turn dummynet off (by short-circuiting a "allow ip from any to any" before the "pipe tablearg" rule) all user complaints stop, with traffic load staying stably at around 930-950 mbps. Does this have anything to do with "dummynet bursts"? How can I beat that? If I keep the pipe queue size at 2000 slots, the net.inet.ip.dummynet.io_pkt_drops sysctl stops increasing, once I start tweaking the value to as low as 100 slots, the value starts raising constantly at about 300-500 pps. I had hoped that smaller queue sizes would battle the negative effects of dummynet burstiness, it did so, I guess, but not in a very decisive manner. FreeBSD 7.1-RELEASE-p10 kern.hz=4000 kern.ipc.nmbclusters=111111 net.inet.ip.fastforwarding=1 net.inet.ip.dummynet.io_fast=1 net.isr.direct=0 net.inet.ip.intr_queue_maxlen=5000 net.inet.ip.dummynet.hash_size=512 net.inet.ip.dummynet.max_chain_len=16 net.inet.ip.intr_queue_drops: 0 systat -ip shows zero output drops at times of trouble. netstat -s's "output packets dropped due to no bufs, etc." is also fine. netstat -m shows nothing suspicious. p.s: Two "bloody expensive" Intel 10 GigE's are on their way to us to replace the Broadcom cards, meanwhile what should I try doing? Thanks for reading. From owner-freebsd-net@FreeBSD.ORG Fri Feb 19 22:29:51 2010 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 06FA3106566B; Fri, 19 Feb 2010 22:29:51 +0000 (UTC) (envelope-from jkim@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D31908FC12; Fri, 19 Feb 2010 22:29:50 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1JMTo7K080341; Fri, 19 Feb 2010 22:29:50 GMT (envelope-from jkim@freefall.freebsd.org) Received: (from jkim@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o1JMTopd080337; Fri, 19 Feb 2010 22:29:50 GMT (envelope-from jkim) Date: Fri, 19 Feb 2010 22:29:50 GMT Message-Id: <201002192229.o1JMTopd080337@freefall.freebsd.org> To: jkim@FreeBSD.org, freebsd-net@FreeBSD.org, jkim@FreeBSD.org From: jkim@FreeBSD.org Cc: Subject: Re: kern/143855: [bpf] [patch] non-blocking BPF reads return -1/EWOULDBLOCK until the store buffer fills up X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 22:29:51 -0000 Synopsis: [bpf] [patch] non-blocking BPF reads return -1/EWOULDBLOCK until the store buffer fills up Responsible-Changed-From-To: freebsd-net->jkim Responsible-Changed-By: jkim Responsible-Changed-When: Fri Feb 19 22:29:28 UTC 2010 Responsible-Changed-Why: I'll take it. http://www.freebsd.org/cgi/query-pr.cgi?pr=143855 From owner-freebsd-net@FreeBSD.ORG Sat Feb 20 00:46:49 2010 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from [127.0.0.1] (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by hub.freebsd.org (Postfix) with ESMTP id 1EFBA1065670; Sat, 20 Feb 2010 00:46:49 +0000 (UTC) (envelope-from jkim@FreeBSD.org) From: Jung-uk Kim To: freebsd-net@FreeBSD.org Date: Fri, 19 Feb 2010 19:46:39 -0500 User-Agent: KMail/1.6.2 References: <20100218230231.GB53467@atarininja.org> In-Reply-To: <20100218230231.GB53467@atarininja.org> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201002191946.41126.jkim@FreeBSD.org> Cc: Wesley Shields , Guy Harris Subject: Re: Please review: kern/143855 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 00:46:49 -0000 On Thursday 18 February 2010 06:02 pm, Wesley Shields wrote: > I'm looking for someone with knowledge of BPF to review this patch: > > http://people.freebsd.org/~wxs/bpf.diff > > It's basically a fixed version of the one in kern/143855. Using the > test program provided I was able to get the callback to be called > with this patch. Without it the callback was never called. If I can > get someone to review it I would like to commit it (I'm not a src > committer) and eventually MFC it. > > Anyone want to work with me on fixing this easy problem? I committed a slightly different version. Thanks! Jung-uk Kim From owner-freebsd-net@FreeBSD.ORG Sat Feb 20 09:50:37 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1018C106566B for ; Sat, 20 Feb 2010 09:50:37 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id AC58D8FC0C for ; Sat, 20 Feb 2010 09:50:36 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id B522D730A1; Sat, 20 Feb 2010 10:59:41 +0100 (CET) Date: Sat, 20 Feb 2010 10:59:41 +0100 From: Luigi Rizzo To: rihad Message-ID: <20100220095941.GA82976@onelab2.iet.unipi.it> References: <4B7EDD00.5080308@mail.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B7EDD00.5080308@mail.ru> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org Subject: Re: Slow speeds experienced with Dummynet X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 09:50:37 -0000 On Fri, Feb 19, 2010 at 10:48:32PM +0400, rihad wrote: > Hi, all, > > Recalling my old posting "dummynet dropping too many packets" dated > October 4, 2009, the problem isn't over just yet. This time, there are > no interface i/o drops (just a reminder: we have 2 bce(4) GigE cards > connected to a Cisco router, one for input, and one for output. The box > itself does some traffic accounting and enforces speed limits w/ > ipfw/dummynet. There are normally around 5-6k users online). If i remember well, the previous discussion ended when you raised the intr_queue_maxlen (and perhaps increased HZ) to avoid that the bursts produced by periodic invocation of dummynet_io() could overflow that queue. >From the rest of your post it is not completely clear if you have not found any working configuration, or there is some setting (e.g. with "queue 1000" or larger) which do produce a smooth experience for your customers. Another thing i'd like to understand is whether all of your pipes have a /32 mask, or there are some which cover multiple hosts. Typical TCP connections have around 50 packets in flight when the connection is fully open (which in turn is hard to happen on a 512k pipe) so a queue of 100-200 is unlikely to overflow. In fact, long queues are very detrimental for customers because they increase the delay of the congestion control loop -- as a rule of thumb, you should try to limit the queue size to at most 1-2s of data. cheers luigi > Traffic shaping is accomplished by this ipfw rule: > pipe tablearg ip from any to table(0) out > where table(0) contains those 5-6k IP addresses. The pipes themselves > are GRED (or taildrop, it doesn't matter): > ipfw pipe 512 config bw 512kbit/s mask dst-ip 0xffffffff gred > 0.002/900/1000/0.1 queue 1000 > Taking this template the speeds range from 512 to tens of mbps. With the > setup as above very many users complain about very slow downloads, slow > browsing. systat -ifstat, refreshed every 5 seconds, does reveal large > differences between subsequent displays: if around 800-850 mbps is > what's to be expected, it doesn't stay within those limits, also jumping > to as low as 620+, and to somewhere in the 700's, Now imagine this: once > I turn dummynet off (by short-circuiting a "allow ip from any to any" > before the "pipe tablearg" rule) all user complaints stop, with traffic > load staying stably at around 930-950 mbps. > > Does this have anything to do with "dummynet bursts"? How can I beat > that? If I keep the pipe queue size at 2000 slots, the > net.inet.ip.dummynet.io_pkt_drops sysctl stops increasing, once I start > tweaking the value to as low as 100 slots, the value starts raising > constantly at about 300-500 pps. I had hoped that smaller queue sizes > would battle the negative effects of dummynet burstiness, it did so, I > guess, but not in a very decisive manner. > > > FreeBSD 7.1-RELEASE-p10 > kern.hz=4000 > kern.ipc.nmbclusters=111111 > net.inet.ip.fastforwarding=1 > net.inet.ip.dummynet.io_fast=1 > net.isr.direct=0 > net.inet.ip.intr_queue_maxlen=5000 > net.inet.ip.dummynet.hash_size=512 > net.inet.ip.dummynet.max_chain_len=16 > > net.inet.ip.intr_queue_drops: 0 > systat -ip shows zero output drops at times of trouble. netstat -s's > "output packets dropped due to no bufs, etc." is also fine. netstat -m > shows nothing suspicious. > > p.s: Two "bloody expensive" Intel 10 GigE's are on their way to us to > replace the Broadcom cards, meanwhile what should I try doing? Thanks > for reading. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sat Feb 20 11:10:41 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 39D9E106568D for ; Sat, 20 Feb 2010 11:10:41 +0000 (UTC) (envelope-from rihad@mail.ru) Received: from mx76.mail.ru (mx76.mail.ru [94.100.176.91]) by mx1.freebsd.org (Postfix) with ESMTP id EA3418FC0A for ; Sat, 20 Feb 2010 11:10:40 +0000 (UTC) Received: from [217.25.27.27] (port=16521 helo=[217.25.27.27]) by mx76.mail.ru with asmtp id 1NinEN-0001np-00; Sat, 20 Feb 2010 14:10:39 +0300 Message-ID: <4B7FC334.7000108@mail.ru> Date: Sat, 20 Feb 2010 15:10:44 +0400 From: rihad User-Agent: Thunderbird 2.0.0.23 (X11/20090817) MIME-Version: 1.0 To: Luigi Rizzo References: <4B7EDD00.5080308@mail.ru> <20100220095941.GA82976@onelab2.iet.unipi.it> In-Reply-To: <20100220095941.GA82976@onelab2.iet.unipi.it> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam: Not detected X-Mras: Ok Cc: freebsd-net@freebsd.org Subject: Re: Slow speeds experienced with Dummynet X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 11:10:41 -0000 Luigi Rizzo wrote: > On Fri, Feb 19, 2010 at 10:48:32PM +0400, rihad wrote: >> Hi, all, >> >> Recalling my old posting "dummynet dropping too many packets" dated >> October 4, 2009, the problem isn't over just yet. This time, there are >> no interface i/o drops (just a reminder: we have 2 bce(4) GigE cards >> connected to a Cisco router, one for input, and one for output. The box >> itself does some traffic accounting and enforces speed limits w/ >> ipfw/dummynet. There are normally around 5-6k users online). > > If i remember well, the previous discussion ended when you > raised the intr_queue_maxlen (and perhaps increased HZ) to > avoid that the bursts produced by periodic invocation of > dummynet_io() could overflow that queue. > I've never seen intr_queue_drops rise. It was HZ=2000 and changing if_bce.c to force ifp->if_snd.ifq_drv_maxlen = 4096 (8192 now) stopped the output drops back then. Lately we've been experiencing interface-level input drops as well (as showed by netstat -i Ierrs column), setting HZ=4000 solved the issue. >>From the rest of your post it is not completely clear if you have > not found any working configuration, or there is some setting (e.g. > with "queue 1000" or larger) which do produce a smooth experience > for your customers. It's pretty hard to set slots in terms of seconds, as slots may vary in size. I'll try setting queue sizes to Kbytes for the duration of 1-2 seconds as you suggested, and use the taildrop queuing algorithm with it. > > Another thing i'd like to understand is whether all of your pipes > have a /32 mask, or there are some which cover multiple hosts. > Typical TCP connections have around 50 packets in flight when the > connection is fully open (which in turn is hard to happen on a 512k > pipe) so a queue of 100-200 is unlikely to overflow. > Yes, all masks are currently /32. 512 isn't used that much, speeds of several mbps are. > In fact, long queues are very detrimental for customers because > they increase the delay of the congestion control loop -- as a rule > of thumb, you should try to limit the queue size to at most 1-2s > of data. > From owner-freebsd-net@FreeBSD.ORG Sat Feb 20 11:21:51 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 54503106568F for ; Sat, 20 Feb 2010 11:21:51 +0000 (UTC) (envelope-from nvass9573@gmx.com) Received: from mailout-eu.gmx.com (mailout-eu.gmx.com [213.165.64.42]) by mx1.freebsd.org (Postfix) with SMTP id 99F9D8FC13 for ; Sat, 20 Feb 2010 11:21:50 +0000 (UTC) Received: (qmail invoked by alias); 20 Feb 2010 11:21:48 -0000 Received: from adsl-40.91.140.30.tellas.gr (EHLO [192.168.73.198]) [91.140.30.40] by mail.gmx.com (mp-eu001) with SMTP; 20 Feb 2010 12:21:48 +0100 X-Authenticated: #46156728 X-Provags-ID: V01U2FsdGVkX184Yj03G9E4cqTN+EAWUK3KdMKHCBf6WG4b5yY53t /U5eGmvmOdxVh7 Message-ID: <4B7FC5AD.3020405@gmx.com> Date: Sat, 20 Feb 2010 13:21:17 +0200 From: Nikos Vassiliadis User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1 MIME-Version: 1.0 To: Mikolaj Golub References: <20100217132632.GA756@crete.org.ua> <4B7D5D95.20007@gmx.com> <86bpflqr5b.fsf@zhuzha.ua1> In-Reply-To: <86bpflqr5b.fsf@zhuzha.ua1> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 X-FuHaFi: 0.69999999999999996 Cc: freebsd-net@freebsd.org, Alexander Shikoff Subject: Re: mpd has hung X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 11:21:51 -0000 On 2/19/2010 3:41 PM, Mikolaj Golub wrote: > We were discussing this problem with Alexander in another (Russian/Ukrainian > speaking) maillist. And it looks like the problem is the following. I see. Could you keep posting your findings to net@ as well? Thanks, Nikos From owner-freebsd-net@FreeBSD.ORG Sat Feb 20 11:30:08 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4FD5D1065670 for ; Sat, 20 Feb 2010 11:30:08 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.cksoft.de (mail.cksoft.de [IPv6:2001:4068:10::3]) by mx1.freebsd.org (Postfix) with ESMTP id DA4358FC08 for ; Sat, 20 Feb 2010 11:30:07 +0000 (UTC) Received: from localhost (amavis.fra.cksoft.de [192.168.74.71]) by mail.cksoft.de (Postfix) with ESMTP id E287141C75B; Sat, 20 Feb 2010 12:30:06 +0100 (CET) X-Virus-Scanned: amavisd-new at cksoft.de Received: from mail.cksoft.de ([192.168.74.103]) by localhost (amavis.fra.cksoft.de [192.168.74.71]) (amavisd-new, port 10024) with ESMTP id WL9vcMXADnz0; Sat, 20 Feb 2010 12:30:06 +0100 (CET) Received: by mail.cksoft.de (Postfix, from userid 66) id 1605541C712; Sat, 20 Feb 2010 12:30:06 +0100 (CET) Received: from maildrop.int.zabbadoz.net (maildrop.int.zabbadoz.net [10.111.66.10]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.int.zabbadoz.net (Postfix) with ESMTP id 18E904448EC; Sat, 20 Feb 2010 11:27:22 +0000 (UTC) Date: Sat, 20 Feb 2010 11:27:21 +0000 (UTC) From: "Bjoern A. Zeeb" X-X-Sender: bz@maildrop.int.zabbadoz.net To: Mikolaj Golub In-Reply-To: <86bpflqr5b.fsf@zhuzha.ua1> Message-ID: <20100220112639.L27327@maildrop.int.zabbadoz.net> References: <20100217132632.GA756@crete.org.ua> <4B7D5D95.20007@gmx.com> <86bpflqr5b.fsf@zhuzha.ua1> X-OpenPGP-Key: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-net@freebsd.org Subject: Re: mpd has hung X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 11:30:08 -0000 On Fri, 19 Feb 2010, Mikolaj Golub wrote: > On Thu, 18 Feb 2010 17:32:37 +0200 Nikos Vassiliadis wrote: > >> On 2/17/2010 3:26 PM, Alexander Shikoff wrote: >>> Hello All, >>> >>> I have mpd 5.3 running on 8.0-RC1 as PPPoE server (now only 5 clients). >>> Today mpd process hung and I cannot kill it with -9 signal, and I cannot >>> access it's console via telnet. >>> >>> State of process in `top` output is STOP: >>> 73551 root 2 44 0 29588K 5692K STOP 6 0:32 0.00% mpd5 >>> >>> # procstat -kk 73551 >>> PID TID COMM TDNAME KSTACK >>> 73551 100233 mpd5 - mi_switch+0x16f sleepq_wait+0x42 _cv_wait+0x111 flowtable_flush+0x51 if_detach+0x2f2 ng_iface_shutdown+0x1e ng_rmnode+0x167 ng_apply_item+0xef7 ng_snd_item+0x2ce ngc_send+0x1d2 sosend_generic+0x3f6 kern_sendit+0x13d sendit+0xdc sendto+0x4d syscall+0x1da Xfast_syscall+0xe1 >>> 73551 100502 mpd5 - mi_switch+0x16f thread_suspend_switch+0xc6 thread_single+0x1b6 exit1+0x72 sigexit+0x7c postsig+0x306 ast+0x279 doreti_ast+0x1f >>> >>> Is there a way to stop a process without rebooting a whole system? >>> Thanks in advance! >>> >>> P.S. I'm ready for experiments with it before tonight, but I cannot >>> force system to crash in order to get crash dump right now. >>> >> >> It's probably too late now, but are you sure that nobody pressed >> CTLR-Z while in the mpd console??? >> >> CTLR-Z will send SIGSTOP to the process and the process will >> stop. While stopped, all processing stops(including receiving >> SIGKILL, you cannot kill it, and the signals are queued). You >> have to send SIGCONT for the process to continue. > > We were discussing this problem with Alexander in another (Russian/Ukrainian > speaking) maillist. And it looks like the problem is the following. > > mpd5 thread was detaching ng interface and when doing flowtable_flush() it > slept in cv_wait waiting for flowclean_cycles variable to be updated. It > should have been awaken by flowcleaner thread but this thread got stuck in > endless loop, supposedly in flowtable_clean_vnet()/flowtable_free_stale(), I > think because of inconsistent state of some lists (iface?) due to if_detach > being in progress. I have patches that are out for review. -- Bjoern A. Zeeb It will not break if you know what you are doing. From owner-freebsd-net@FreeBSD.ORG Sat Feb 20 12:05:07 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD711106566B for ; Sat, 20 Feb 2010 12:05:07 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.cksoft.de (mail.cksoft.de [IPv6:2001:4068:10::3]) by mx1.freebsd.org (Postfix) with ESMTP id 6D4D58FC14 for ; Sat, 20 Feb 2010 12:05:07 +0000 (UTC) Received: from localhost (amavis.fra.cksoft.de [192.168.74.71]) by mail.cksoft.de (Postfix) with ESMTP id CCF0A41C6DB; Sat, 20 Feb 2010 13:05:06 +0100 (CET) X-Virus-Scanned: amavisd-new at cksoft.de Received: from mail.cksoft.de ([192.168.74.103]) by localhost (amavis.fra.cksoft.de [192.168.74.71]) (amavisd-new, port 10024) with ESMTP id hikAxLY4Iuws; Sat, 20 Feb 2010 13:05:06 +0100 (CET) Received: by mail.cksoft.de (Postfix, from userid 66) id 21A9541C6B4; Sat, 20 Feb 2010 13:05:06 +0100 (CET) Received: from maildrop.int.zabbadoz.net (maildrop.int.zabbadoz.net [10.111.66.10]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.int.zabbadoz.net (Postfix) with ESMTP id 1C1464448EC; Sat, 20 Feb 2010 12:04:36 +0000 (UTC) Date: Sat, 20 Feb 2010 12:04:35 +0000 (UTC) From: "Bjoern A. Zeeb" X-X-Sender: bz@maildrop.int.zabbadoz.net To: Mikolaj Golub In-Reply-To: <20100220112639.L27327@maildrop.int.zabbadoz.net> Message-ID: <20100220115850.T27327@maildrop.int.zabbadoz.net> References: <20100217132632.GA756@crete.org.ua> <4B7D5D95.20007@gmx.com> <86bpflqr5b.fsf@zhuzha.ua1> <20100220112639.L27327@maildrop.int.zabbadoz.net> X-OpenPGP-Key: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-net@freebsd.org Subject: Re: mpd has hung X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 12:05:07 -0000 On Sat, 20 Feb 2010, Bjoern A. Zeeb wrote: > On Fri, 19 Feb 2010, Mikolaj Golub wrote: > >> On Thu, 18 Feb 2010 17:32:37 +0200 Nikos Vassiliadis wrote: >> >>> On 2/17/2010 3:26 PM, Alexander Shikoff wrote: >>>> Hello All, >>>> >>>> I have mpd 5.3 running on 8.0-RC1 as PPPoE server (now only 5 clients). >>>> Today mpd process hung and I cannot kill it with -9 signal, and I cannot >>>> access it's console via telnet. >>>> >>>> State of process in `top` output is STOP: >>>> 73551 root 2 44 0 29588K 5692K STOP 6 0:32 0.00% mpd5 >>>> >>>> # procstat -kk 73551 >>>> PID TID COMM TDNAME KSTACK >>>> 73551 100233 mpd5 - mi_switch+0x16f >>>> sleepq_wait+0x42 _cv_wait+0x111 flowtable_flush+0x51 if_detach+0x2f2 >>>> ng_iface_shutdown+0x1e ng_rmnode+0x167 ng_apply_item+0xef7 >>>> ng_snd_item+0x2ce ngc_send+0x1d2 sosend_generic+0x3f6 kern_sendit+0x13d >>>> sendit+0xdc sendto+0x4d syscall+0x1da Xfast_syscall+0xe1 >>>> 73551 100502 mpd5 - mi_switch+0x16f >>>> thread_suspend_switch+0xc6 thread_single+0x1b6 exit1+0x72 sigexit+0x7c >>>> postsig+0x306 ast+0x279 doreti_ast+0x1f >>>> >>>> Is there a way to stop a process without rebooting a whole system? >>>> Thanks in advance! >>>> >>>> P.S. I'm ready for experiments with it before tonight, but I cannot >>>> force system to crash in order to get crash dump right now. >>>> >>> >>> It's probably too late now, but are you sure that nobody pressed >>> CTLR-Z while in the mpd console??? >>> >>> CTLR-Z will send SIGSTOP to the process and the process will >>> stop. While stopped, all processing stops(including receiving >>> SIGKILL, you cannot kill it, and the signals are queued). You >>> have to send SIGCONT for the process to continue. >> >> We were discussing this problem with Alexander in another >> (Russian/Ukrainian >> speaking) maillist. And it looks like the problem is the following. >> >> mpd5 thread was detaching ng interface and when doing flowtable_flush() it >> slept in cv_wait waiting for flowclean_cycles variable to be updated. It >> should have been awaken by flowcleaner thread but this thread got stuck in >> endless loop, supposedly in flowtable_clean_vnet()/flowtable_free_stale(), >> I >> think because of inconsistent state of some lists (iface?) due to if_detach >> being in progress. > > I have patches that are out for review. I am not sure if they apply cleanly as they are broken out of the tail side of a larger patchset. If you are not using VIMAGEs you could ignore the ones I marked with (*). http://people.freebsd.org/~bz/20100216-10-ft-cv.diff http://people.freebsd.org/~bz/20100216-11-ft-debugging.diff http://people.freebsd.org/~bz/20100216-12-ft-cleanup.diff (*) http://people.freebsd.org/~bz/20100216-13-ft-ll-cleanup.diff http://people.freebsd.org/~bz/20100216-18-ft-free.diff (*) If you are still seeing the hang and have DDB support in your kernel, then break into the debugger and save the complete output of ddb> ps before rebooting. Regards, Bjoern -- Bjoern A. Zeeb It will not break if you know what you are doing. From owner-freebsd-net@FreeBSD.ORG Sat Feb 20 21:51:28 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BAFA0106568B; Sat, 20 Feb 2010 21:51:28 +0000 (UTC) (envelope-from sam@errno.com) Received: from ebb.errno.com (ebb.errno.com [69.12.149.25]) by mx1.freebsd.org (Postfix) with ESMTP id 8DF4C8FC1E; Sat, 20 Feb 2010 21:51:28 +0000 (UTC) Received: from ice.local ([10.0.0.115]) (authenticated bits=0) by ebb.errno.com (8.13.6/8.12.6) with ESMTP id o1KLpRcc021461 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 20 Feb 2010 13:51:28 -0800 (PST) (envelope-from sam@errno.com) Message-ID: <4B80595F.2050406@errno.com> Date: Sat, 20 Feb 2010 13:51:27 -0800 From: Sam Leffler User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Alexander Egorenkov References: <2d3b7e441002100446k36493e24ldef6d4e335092b7f@mail.gmail.com> <4B73274E.1050902@freebsd.org> <2d3b7e441002101401x2f6f5f28i5ee3b63c18e6b1ed@mail.gmail.com> In-Reply-To: <2d3b7e441002101401x2f6f5f28i5ee3b63c18e6b1ed@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-DCC-x.dcc-servers-Metrics: ebb.errno.com; whitelist Cc: freebsd-net@freebsd.org, Rui Paulo Subject: Re: net80211 and PPI header X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 21:51:28 -0000 Alexander Egorenkov wrote: >> What exactly do you need? We should be able to extend radiotap. >> > > 1. Not only one RSSI but for every antenna (also in dBm) > 2. HT greenfield/HT mixed indicator > 4. Number of spatial streams (STBC) > 3. A-MPDU support (MPDU density, A-MPDU reassembly) > > The most important one is A-MPDU support, i think. > > Wireshark supports PPI header and can handle A-MPDUs very nicely. > > That's it for now :-) I discussed integrating PPI w/ radiotap back when I did the existing 11n support but never got anywhere (>3 years ago?). The PPI stuff was done under contract to Intel and the folks involved never contacted anyone about doing it within radiotap instead. It looked entirely possible to leverage the PPI decoder in wireshark to handle AMPDU reassembly from the radiotap decoder but I never got to it. As to the other state Greenfield was nonexistent (and unclear if it'd make it out of draft status) when I did stuff or I'd have reserved a bit for it. The # of streams can be implied from the MCS unless I misunderstand. I do want the per-antenna/chain state (rssi, nf, evm) but was looking for things to stabilize before adding to radiotap--each device/driver exports different data and I wanted something that was enough of a superset to handle the most devices. Adopting PPI data structures would be reasonable. I would prefer to not emit PPI but instead augment radiotap. We've standardized on radiotap for 802.11 and all the drivers now use it (or should use it). I'll leave it to others to deal w/ the politics of the radiotap noobs; the technical details of doing this are straightforward. Sam From owner-freebsd-net@FreeBSD.ORG Sat Feb 20 21:55:47 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EF4BF106566B for ; Sat, 20 Feb 2010 21:55:46 +0000 (UTC) (envelope-from rpaulo@gmail.com) Received: from mail-ew0-f211.google.com (mail-ew0-f211.google.com [209.85.219.211]) by mx1.freebsd.org (Postfix) with ESMTP id 7E7CD8FC0C for ; Sat, 20 Feb 2010 21:55:46 +0000 (UTC) Received: by ewy3 with SMTP id 3so1213415ewy.13 for ; Sat, 20 Feb 2010 13:55:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:subject:mime-version :content-type:from:in-reply-to:date:cc:content-transfer-encoding :message-id:references:to:x-mailer; bh=tZUVYmOISkW+fR0RIn18Wq0/JPU548wsXhbJMy/5YNo=; b=dl4jaqb+7jOrABJSVadXp/807B/g9nQohcGzCAksDN+jD9gFQlC+2mh9TWaCgEjwh/ vj2P/RRkc3qh605klpRxaeYbduL1gRcilcn386HC0wwqI2X7S62ubbg62M7EXxZFOC4r CHyhfxsVH4K5JWu6/Z0hv8GxAveJytnlhnSys= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; b=ZxkP4gJ2LHV0ZFmRECuKpbEcueTQmXZW1mIS5ZS7cajcsR6DGdJXjue0RVkg/2lejL lKnu8xA2Ix/JfS1QPrCWkkOCMmR7qfCBeHU1uaJFZ/qo6WVuqusGppuJnZJTVsAkCwQo S0t9Fa2MSexm1pN/rrKGHkd+mlvRir/v53i4g= Received: by 10.213.100.160 with SMTP id y32mr2419971ebn.49.1266702945404; Sat, 20 Feb 2010 13:55:45 -0800 (PST) Received: from ?10.0.10.4? (54.81.54.77.rev.vodafone.pt [77.54.81.54]) by mx.google.com with ESMTPS id 28sm5127548eye.31.2010.02.20.13.55.42 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sat, 20 Feb 2010 13:55:43 -0800 (PST) Sender: Rui Paulo Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Rui Paulo In-Reply-To: <4B80595F.2050406@errno.com> Date: Sat, 20 Feb 2010 21:55:41 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <2d3b7e441002100446k36493e24ldef6d4e335092b7f@mail.gmail.com> <4B73274E.1050902@freebsd.org> <2d3b7e441002101401x2f6f5f28i5ee3b63c18e6b1ed@mail.gmail.com> <4B80595F.2050406@errno.com> To: Sam Leffler X-Mailer: Apple Mail (2.1077) Cc: freebsd-net@freebsd.org, Alexander Egorenkov Subject: Re: net80211 and PPI header X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 21:55:47 -0000 On 20 Feb 2010, at 21:51, Sam Leffler wrote: > Alexander Egorenkov wrote: >>> What exactly do you need? We should be able to extend radiotap. >>>=20 >> 1. Not only one RSSI but for every antenna (also in dBm) >> 2. HT greenfield/HT mixed indicator >> 4. Number of spatial streams (STBC) >> 3. A-MPDU support (MPDU density, A-MPDU reassembly) >> The most important one is A-MPDU support, i think. >> Wireshark supports PPI header and can handle A-MPDUs very nicely. >> That's it for now :-) >=20 > I discussed integrating PPI w/ radiotap back when I did the existing = 11n support but never got anywhere (>3 years ago?). The PPI stuff was = done under contract to Intel and the folks involved never contacted = anyone about doing it within radiotap instead. It looked entirely = possible to leverage the PPI decoder in wireshark to handle AMPDU = reassembly from the radiotap decoder but I never got to it. >=20 > As to the other state Greenfield was nonexistent (and unclear if it'd = make it out of draft status) when I did stuff or I'd have reserved a bit = for it. The # of streams can be implied from the MCS unless I = misunderstand. I do want the per-antenna/chain state (rssi, nf, evm) = but was looking for things to stabilize before adding to radiotap--each = device/driver exports different data and I wanted something that was = enough of a superset to handle the most devices. Adopting PPI data = structures would be reasonable. >=20 > I would prefer to not emit PPI but instead augment radiotap. We've = standardized on radiotap for 802.11 and all the drivers now use it (or = should use it). I'll leave it to others to deal w/ the politics of the = radiotap noobs; the technical details of doing this are straightforward. FWIW, I have very basic support of PPI headers on my 11n branch. -- Rui Paulo From owner-freebsd-net@FreeBSD.ORG Sat Feb 20 21:55:54 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C5B7F10656C3 for ; Sat, 20 Feb 2010 21:55:54 +0000 (UTC) (envelope-from sam@errno.com) Received: from ebb.errno.com (ebb.errno.com [69.12.149.25]) by mx1.freebsd.org (Postfix) with ESMTP id 83DE98FC13 for ; Sat, 20 Feb 2010 21:55:54 +0000 (UTC) Received: from ice.local ([10.0.0.115]) (authenticated bits=0) by ebb.errno.com (8.13.6/8.12.6) with ESMTP id o1KLtrri021494 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 20 Feb 2010 13:55:54 -0800 (PST) (envelope-from sam@errno.com) Message-ID: <4B805A69.7030004@errno.com> Date: Sat, 20 Feb 2010 13:55:53 -0800 From: Sam Leffler User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Alexander Egorenkov References: <2d3b7e441002181058u5006595an5c636843945bcca7@mail.gmail.com> In-Reply-To: <2d3b7e441002181058u5006595an5c636843945bcca7@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-DCC-x.dcc-servers-Metrics: ebb.errno.com; whitelist Cc: freebsd-net@freebsd.org Subject: Re: Missing WME information element causes problems with 802.11n X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 21:55:54 -0000 Alexander Egorenkov wrote: > I have encountered a problem with a 802.11n router Belkin F5D8631au. > The beacon and association response frames sent by this router do not > contain > WME information element although 802.11n mode is enabled. These frames > contain > HT capability IE and HT info. Because WME IE is missing in association > responses, > the net80211 stack does not set IEEE80211_NODE_QOS flag > (See ieee80211_sta.c:sta_recv_mgmt:IEEE80211_FC0_SUBTYPE_ASSOC_RESP). > But the flag IEEE80211_NODE_HT is set because the frame contains HT > capability and HT info. > > So, because IEEE80211_NODE_QOS is not set, all outgoing DATA frames sent to > the Belkin AP > do not contain QoS field in the 802.11 frame header. And it causes problems > with the Belkin AP. > > Is the QoS not mandatory for 802.11n mode ? > Why is QoS enabled only if an WME IE is found in association response ? > Would it be not right to enable QoS also if HT mode is enabled but no WME IE > was found ? The WME ie is mandatory; these routers are non compliant. We can probably hack net80211 to auto-enable WME if an HTCAP ie is present but that is a total hack. Were the HT ie's using the IEEE codes or the Vendor OUI codes? It could be this is old Broadcom code--at one point Broadcom intentionally didn't advertise WME. If this is the legacy HT stuff then perhaps we can add the auto-enable conditional on the legacy HT support. Sam From owner-freebsd-net@FreeBSD.ORG Sat Feb 20 22:55:51 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9EA89106566B for ; Sat, 20 Feb 2010 22:55:51 +0000 (UTC) (envelope-from jake@stnlabs.com) Received: from server514.appriver.com (server514a.exghost.com [72.32.253.68]) by mx1.freebsd.org (Postfix) with ESMTP id 693608FC0C for ; Sat, 20 Feb 2010 22:55:50 +0000 (UTC) X-Policy: GLOBAL - stnlabs.com X-Primary: jake@stnlabs.com X-Note: This Email was scanned by AppRiver SecureTide X-ALLOW: jake@stnlabs.com ALLOWED X-Virus-Scan: V- X-Note: Spam Tests Failed: X-Country-Path: UNITED STATES->UNITED STATES->UNITED STATES X-Note-Sending-IP: 72.32.253.140 X-Note-Reverse-DNS: fe01.exg4.exghost.com X-Note-WHTLIST: jake@stnlabs.com X-Note: User Rule Hits: X-Note: Global Rule Hits: G173 G174 G175 G176 G180 G181 G192 G279 X-Note: Encrypt Rule Hits: X-Note: Mail Class: ALLOWEDSENDER X-Note: Headers Injected Received: from [72.32.253.140] (HELO FE01.exg4.exghost.com) by server514.appriver.com (CommuniGate Pro SMTP 5.3.2) with ESMTP id 101544368 for freebsd-net@freebsd.org; Sat, 20 Feb 2010 15:55:56 -0600 Received: from be18.exg4.exghost.com ([72.32.253.175]) by FE01.exg4.exghost.com with Microsoft SMTPSVC(6.0.3790.3959); Sat, 20 Feb 2010 15:55:50 -0600 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Sat, 20 Feb 2010 15:55:46 -0600 Message-ID: <0D51275610C0C04588FA911C811B687E02480269@be18.exg4.exghost.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: xbox 360 + atheros 5212 connection issues Thread-Index: Acqyd3SeXImpGPA/QKuPSSiBR1Xneg== From: "Jake Baillie" To: X-OriginalArrivalTime: 20 Feb 2010 21:55:50.0463 (UTC) FILETIME=[773A1CF0:01CAB277] Subject: xbox 360 + atheros 5212 connection issues X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 22:55:51 -0000 Hello, I'm having trouble getting an Xbox 360 to connect to my newly upgraded FreeBSD based wireless access point. The Xbox 360 can successfully locate the AP in a scan, but when trying to connect it simply fails. If you've never used an Xbox 360 before it is quite useless in reporting debugging information, so all debugging will either have to come from the FreeBSD box or perhaps another computer set to monitor mode. I'd like to request some help in debugging this as I have no idea where to start. The FreeBSD box is running 8-RELEASE. Datapoints: 1. The Xbox connected successfully to my FreeBSD 7-RELEASE based AP with no issues. 2. The Xbox connected successfully to multiple off the shelf consumer grade APs I have sitting around. 3. The Xbox cannot connect to a Linux based AP using madwifi-0.9.4 or later trunk versions. It also cannot connect using the (more recent) ath5k driver on Linux. I only mention this as I tried setting up a Slackware box out of frustration to no avail. (FYI - there is a very old ticket on the madwifi site that was never closed with the exact problem that I am having: http://madwifi-project.org/ticket/1509) 4. Many other devices are connecting successfully to the 8-RELEASE AP including a couple of netbooks, a laptop, a Canon wireless printer, an iPod touch, a BlackBerry, and an Apple Wireless bridge. 5. The network is using WPA2-PSK through hostapd. Turning off authentication and opening the network completely does not help. -- The hardware is a Cisco Aironet PCI card with an Atheros 5212 chip: # dmesg | grep ath ath0: mem 0xe1300000-0xe130ffff irq 16 at device 4.0 on pci5 ath0: [ITHREAD] ath0: AR5212 mac 5.9 RF5112 phy 4.3 # ifconfig ath0 ath0: flags=3D8843 metric 0 mtu 2290 ether 00:40:96:b8:13:53 media: IEEE 802.11 Wireless Ethernet autoselect mode 11g status: running # ifconfig wlan0 wlan0: flags=3D8943 = metric 0 mtu 1500 ether 00:40:96:b8:13:53 media: IEEE 802.11 Wireless Ethernet autoselect mode 11g status: running ssid freebsd channel 6 (2437 Mhz 11g) bssid 00:40:96:b8:13:53 regdomain FCC3 indoor ecm authmode WPA2/802.11i privacy MIXED deftxkey 3 AES-CCM 2:128-bit AES-CCM 3:128-bit txpower 21 scanvalid 60 protmode CTS wme burst dtimperiod 1 -dfs Any help is appreciated. cheers, -- jake