From owner-freebsd-net@FreeBSD.ORG Mon Jan 28 06:09:02 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9572B16A417 for ; Mon, 28 Jan 2008 06:09:02 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.179]) by mx1.freebsd.org (Postfix) with ESMTP id 5FF4B13C507 for ; Mon, 28 Jan 2008 06:09:02 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: by wa-out-1112.google.com with SMTP id k17so2835579waf.3 for ; Sun, 27 Jan 2008 22:09:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:received:date:from:to:cc:subject:message-id:reply-to:references:mime-version:content-type:content-disposition:in-reply-to:user-agent; bh=8YC7yjU3+ZI5VoYNwvDa4jnfUEtMZFw4eyIqOsn7/F8=; b=tMs/EST17Wvs78m00LQaKbTk5xBBbwpUBlhRAbI4AQaLrOMK0e82vaNNoOPj2djpKGGRXc5/9AINEGS3rVhb4dB2TfJ68ayXc4u+PdvC/nsAK8VId9Jc8hBvfCJTLz6sUhF9qvk3xOwcs/M6FYGbqr7RO0CwF+wtwMN/LhxWCH0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:reply-to:references:mime-version:content-type:content-disposition:in-reply-to:user-agent; b=hRig5N9HWZhQznI14GYOHA25OqYvZT4HiWLXQaNtwDzykAlvamQosfsiVZQ0nhkFQNqjOGhQlNADXneMKS7fRxS6hf8fzEIIuERyNbDncaXVxubxzG03kKmM2qnDQ9tD02K9P2oEVTxcqSd+wRXFsXpotJ0VrxOX4GfMa6NSNhU= Received: by 10.114.78.1 with SMTP id a1mr983988wab.14.1201500541746; Sun, 27 Jan 2008 22:09:01 -0800 (PST) Received: from michelle.cdnetworks.co.kr ( [211.53.35.84]) by mx.google.com with ESMTPS id j29sm10783780waf.18.2008.01.27.22.08.58 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 27 Jan 2008 22:09:00 -0800 (PST) Received: from michelle.cdnetworks.co.kr (localhost.cdnetworks.co.kr [127.0.0.1]) by michelle.cdnetworks.co.kr (8.13.5/8.13.5) with ESMTP id m0S68sh7002085 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 28 Jan 2008 15:08:54 +0900 (KST) (envelope-from pyunyh@gmail.com) Received: (from yongari@localhost) by michelle.cdnetworks.co.kr (8.13.5/8.13.5/Submit) id m0S68s7n002084; Mon, 28 Jan 2008 15:08:54 +0900 (KST) (envelope-from pyunyh@gmail.com) Date: Mon, 28 Jan 2008 15:08:54 +0900 From: Pyun YongHyeon To: Yousif Hassan Message-ID: <20080128060854.GA1240@cdnetworks.co.kr> References: <1201125022.2106.67.camel@localhost> <20080123222047.GA14264@lor.one-eyed-alien.net> <1201190313.2591.7.camel@localhost> <20080124163634.GA25331@lor.one-eyed-alien.net> <1201198042.19736.5.camel@localhost> <20080125051059.GA22410@cdnetworks.co.kr> <1201275694.1537.13.camel@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1201275694.1537.13.camel@localhost> User-Agent: Mutt/1.4.2.1i Cc: freebsd-net@freebsd.org, Brooks Davis , "Bruce M. Simpson" Subject: Re: network interface monitoring? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jan 2008 06:09:02 -0000 On Fri, Jan 25, 2008 at 10:41:34AM -0500, Yousif Hassan wrote: > Hi Pyun YongHyeon, > > First, I'd like to say thank you for sending this and trying to resolve > my (and others') problems with bfe driver. > > First the good news - your patch appears to solve nearly all of the > issues I've discovered and/or reported. After installing the kernel > with your patch, under normal circumstances, link up and down events are > detected automatically by the kernel now, and passed to userland. I > tested this with some customd devd scripts to make sure the devd > notification from if_net.c was ok, and it was. This is greatly improved > behavior! > > A couple minor nits - > > First: The first hunk out of the first file in the patch didn't apply > cleanly for me but it could be because I'm on a different file revision? > I'll attach the .rej file. It's no big deal, because it was trivial to > adjust it by hand, then I was able to compile. > > Second: There's one last remaining issue. If I set the bfe0_enable > parameter in rc.conf to "DHCP NOAUTO" (so that it doesn't try to do > anything on boot, because it can slow the process down while it's > negotiating an address), then the link state changes get queued up (as > before) until I manually run ifconfig once. After that one time, > everything from that point on is fine (meaning your changes are > working). For all I know, this might be unrelated to your driver patch, > but it was interesting. Setting the bfe0_enable to "DHCP" meant > everything worked fine from the start. Setting it to "UP" also was fine > in terms of your link state changes working (although in this > case /etc/rc.d/dhclient script won't work because the interface needs to > be configured in rc.conf to use DHCP; I must run "/sbin/dhclient bfe0" > manually in that case). > I guess that is not related with link state handling of bfe(4). I'll commit the change to HEAD. > In all cases, THANK YOU - this is much much much better than before and > I can live with using bfe0_enable="DHCP" instead of "DHCP NOAUTO". Feel > free to ask me for more testing if you want to try and investigate the > one remaining queue issue. > Thanks for testing and feedback! > --Yousif > > On Fri, 2008-01-25 at 14:10 +0900, Pyun YongHyeon wrote: > > On Thu, Jan 24, 2008 at 01:07:22PM -0500, Yousif Hassan wrote: > > > On Thu, 2008-01-24 at 10:36 -0600, Brooks Davis wrote: > > > > On Thu, Jan 24, 2008 at 10:58:33AM -0500, Yousif Hassan wrote: > > > > > Thank you to all who responded. > > > > > > > > > > The suggestion was made to use devd or ifstated. Both sound like > > > > > excellent tools, but I'm currently being tripped up by a core problem - > > > > > both tools rely on the kernel to notify userland of link state changes > > > > > (which makes complete sense!). This is all well and good - but the > > > > > current issue I'm seeing is that the link state doesn't get updated > > > > > without running "ifconfig" again - is this by design? A known "issue?" > > > > > > > > > > An example: > > > > > 1. Unplug network cable from bfe0 > > > > > 2. I run ifconfig > > > > > 3. I see that interface bfe0's status is "no carrier". Good. > > > > > 4. I plug the cable into bfe0 > > > > > 5. Wait... wait... look in /var/log/messages... wait more.. NO STATE > > > > > CHANGE - the longest I've waited was 2 minutes, which is already too > > > > > long > > > > > 6. run "ifconfig" again > > > > > 7. Link state immediately changes, logs to /var/log/messages, devd > > > > > scripts run > > > > > > > > > > Is this a known behavior? It seems like the link state changes should > > > > > happen automatically, without something to "trigger" them. Isn't there > > > > > some kind of poll or interrupt sequence? I'm on 6.3 RC2 (will upgrade > > > > > to 6.3-RELEASE imminently) but can't imagine this code changed? Does > > > > > this work differently/better in 7.0? > > > > > > > > It's known but not well understood and is a driver bug. > > > > > > I was afraid you'd say that. Thanks for the info. > > > > > > I found PR kern/109733: [bge] bge link state issues (regression) > > > which, while referring to a different driver, has some of the same > > > symptoms. > > > > > > In the meantime, I scripted something that calls ifconfig every 30 > > > seconds or so, redirected its output to null, and put it in the > > > background and nice'd it; this seems to do the trick, albeit via a > > > horrid hack. Due to the bug you mentioned, sometimes link state events > > > queue up, too, and get passed to userland at once, which is no kind of > > > fun, but the script still works. > > > > > > > Try attached patch. I don't know whether it works or not but it > > seems that link state was not correctly tracked down by stock bfe(4). > > The attached patch does the following. > > - conversion to callout(9) API. > > - added missing lock in bfe_ifmedia_sts(). > > - implemented miibus_statchg method to track link state. > > - use our callout to drive watchdog timer. > > - restart Tx routine if pending queued packets are present in > > watchdog handler. > > - fixed blindly resetting watchdog timer in bfe_txeof(). > > > > I guess the above is minimal patch to get correct link state. > > If I had the hardware I would have rewritten bfe_encap/bfe_newbuf > > to use bus_dmamap_load_mbuf_sg(9). :( > > -- Regards, Pyun YongHyeon