From owner-freebsd-current@FreeBSD.ORG Fri Mar 13 00:18:38 2009 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3F861065673 for ; Fri, 13 Mar 2009 00:18:38 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from web63904.mail.re1.yahoo.com (web63904.mail.re1.yahoo.com [69.147.97.119]) by mx1.freebsd.org (Postfix) with SMTP id 84C368FC15 for ; Fri, 13 Mar 2009 00:18:38 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: (qmail 81769 invoked by uid 60001); 13 Mar 2009 00:18:37 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1236903517; bh=WCzhEy2t7c39ZSJhq42WgeA4TCIeu+BUno1dMuXNllQ=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=xTzp+0mEexhgAUUSqITWGUsoGzCJZf5vlYm8y70c32o8PyinCOm07rHhZY6EOPWyN0ImOzwWf5JTT88fDMnsgSMDvC6e7XTplUJYKM/f6fyW4/pDwpQv6xuau8qeWyg5yo1Ot+uDOMhx4pSPdkMGG51q2iatSof4R+ZFxITeDV0= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=E8x7MRmdEeWRneIk4blGk8p4t0EAEsNjxuMt0zARJ1oBWJX3R78w5QTDih+NJCPS87L9m23+sgINiK38usKEZ3FLfO0czggV6x6MmPfg5ULSvElu61wGkC4xnR6mx8uHueXc3hjDepMnZFJo2wrjZNwcCt8EiaPcwKJtOKafuNg=; Message-ID: <857006.73926.qm@web63904.mail.re1.yahoo.com> X-YMail-OSG: EYfx58QVM1lgVG7KvtE1r9CKb1aGrNbNLXQH39KnRCq7ipUezz9Gtk_C42PtCwARg7mwz_19_oRkx9UEVIGFIFcCTRLSjSuWUlYYwcn02.TWzfs3H8JZZGVtPYPB26ufn_5YJMtHCL4wTZ8uGnyFaHqoSkHNyXAbW3EBzeEIfHw27R4f3DfTIOTyksIK6kVEdvWM8YDptwrzHV73USFGS6GDHsTvQhVbyWhxw2NviAJ8KL4KSAA- Received: from [98.242.222.229] by web63904.mail.re1.yahoo.com via HTTP; Thu, 12 Mar 2009 17:18:37 PDT X-Mailer: YahooMailWebService/0.7.289.1 Date: Thu, 12 Mar 2009 17:18:37 -0700 (PDT) From: Barney Cordoba To: Scott Long In-Reply-To: <49B99DE3.6050401@samsco.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: current@freebsd.org Subject: Re: Interrupt routine usage not shown by top in 8.0 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: barney_cordoba@yahoo.com List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Mar 2009 00:18:39 -0000 --- On Thu, 3/12/09, Scott Long wrote: > From: Scott Long > Subject: Re: Interrupt routine usage not shown by top in 8.0 > To: barney_cordoba@yahoo.com > Cc: current@freebsd.org > Date: Thursday, March 12, 2009, 7:42 PM > Barney Cordoba wrote: > > I'm fireing 400Kpps at a udp blackhole port. > I'm getting 6000 interrupts > > per second on em3: > > > > testbox# vmstat -i; sleep 1; vmstat -i > > interrupt total rate > > irq1: atkbd0 1 0 > > irq6: fdc0 1 0 > > irq17: uhci1+ 2226 9 > > irq18: uhci2 ehci+ 9 0 > > cpu0: timer 470507 1993 > > irq256: em0 665 2 > > irq259: em3 1027684 4354 > > cpu1: timer 470272 1992 > > cpu3: timer 470273 1992 > > cpu2: timer 470273 1992 > > Total 2911911 12338 > > > > interrupt total rate > > irq1: atkbd0 1 0 > > irq6: fdc0 1 0 > > irq17: uhci1+ 2226 9 > > irq18: uhci2 ehci+ 9 0 > > cpu0: timer 472513 1993 > > irq256: em0 668 2 > > irq259: em3 1033703 4361 > > cpu1: timer 472278 1992 > > cpu3: timer 472279 1992 > > cpu2: timer 472279 1992 > > Total 2925957 12345 > > > > > > top -SH shows: > > > > PID STATE C TIME CPU COMMAND > > 10 CPU3 3 7:32 100.00% idle > > 10 CPU2 2 7:32 100.00% idle > > 10 RUN 0 7:31 100.00% idle > > 10 CPU1 1 7:31 100.00% idle > > > > This implies that CPU usage is substantially > under-reported in general > > by the system. Note that I've modified > em_irq_fast() to call em_handle_rxtx() directly rather than > scheduling a task to illustrate > > the problem > > > > With unmodified code, what do you see? Are you sending > valid UDP frames with valid checksums and a valid port, or > is everything that you're blasting at the interface > getting dropped right away? Calling em_handle_rxtx() > directly will cause a very quick panic once you start > handling real traffic and you encounter a lock. > > Scott I think you're mistaken. I'm also accessing the system via an em port (and running top) and em_handle_rxtx() is self contained lock-wise. The taskqueue doesn't obtain a lock before calling the routine. As I mentioned, they're being dumped into a udp blackhole, which implies that I have udp.blackhole set and the port is unused. I can see the packets hit the udp socket so its working as expected: 853967872 dropped due to no socket With unmodified code, the tasq shows 25% usage or so. I'm not sure what the point of your criticism for what clearly is a test. Are you implying that the system can receive 400K pps with 6000 ints/sec and record 0% usage because of a coding imperfection? Or are you implying that the 25% usage is all due to launching tasks unnecessarily and process switching? Barney