From owner-freebsd-net@FreeBSD.ORG Thu Apr 9 08:46:36 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F212C1065670 for ; Thu, 9 Apr 2009 08:46:35 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from web63901.mail.re1.yahoo.com (web63901.mail.re1.yahoo.com [69.147.97.116]) by mx1.freebsd.org (Postfix) with SMTP id 44EE38FC22 for ; Thu, 9 Apr 2009 08:46:34 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: (qmail 50282 invoked by uid 60001); 9 Apr 2009 08:46:33 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1239266793; bh=c+mMKOb/sPD5NT2W3z9f21Q8DEL30pjUU1TYlcoW2vc=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=el1TQaNFskYy6fDzBbdutaxTjufCADCStmNQkaMnRysxfSzgA+yDZ57gQaMv74Gus7fb77obyX/jgnjGZMg8gtcN9emUyHzsHjXCXnrhDhHCuC7eI4OeYyx0AzU26dq5+uCM1cQEe02DJQf7Q51afczNPa0GSjWr5o45ZkyWHTo= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=o68sFePnJq/rPh3ceG6jfBW3nls1D4bQzK4zzW68k4Rj2gIBvzIbUiBzJOqsu5RPsorFNlCQK5PySPQpyyfHPGA2Kf0eytogkdo1Q4S/xssUhEJfp1UxjesXug623hQGdvNOL35N9KbeeS7GMqZK8HwT7yZ9Tw830rGN8Sd/EgM=; Message-ID: <792562.49628.qm@web63901.mail.re1.yahoo.com> X-YMail-OSG: kM08mBsVM1mDDC2sKkY.BnuTQYSJtMaELpo5zvSVnQ6uvGgvpwsIukBWFI58anExShEW6lLo.muDZOFMlyRe7qcZPMD_QMm2.PaZTv9LlOaDMZZD8pWwz1h0P5SnYM5MSKSZdJR0S9_dpQZmV.6I3exGxitUa8dfO_VNh0SS9E33msolhD9eYhQa4hNje06CAgOPJRlNYX7tXXxGlr81Ub6GXk.bnnUliGsIqGzwWvYuUGab8FSXpoXXUcYelZyVgsl9E.jQD13m3wCbs1SCFDKjLuWluTiAPNjIFf9s6enhX42c6o7Kj7DdN07V Received: from [98.242.222.229] by web63901.mail.re1.yahoo.com via HTTP; Thu, 09 Apr 2009 01:46:33 PDT X-Mailer: YahooMailWebService/0.7.289.1 Date: Thu, 9 Apr 2009 01:46:33 -0700 (PDT) From: Barney Cordoba To: Robert Watson In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-net@freebsd.org, Ivan Voras Subject: Re: Advice on a multithreaded netisr patch? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: barney_cordoba@yahoo.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Apr 2009 08:46:36 -0000 --- On Wed, 4/8/09, Robert Watson wrote: > From: Robert Watson > Subject: Re: Advice on a multithreaded netisr patch? > To: "Barney Cordoba" > Cc: "Ivan Voras" , freebsd-net@freebsd.org > Date: Wednesday, April 8, 2009, 9:16 AM > On Wed, 8 Apr 2009, Barney Cordoba wrote: > > > Is there any work being done on lighter weight locks > for queues? It seems ridiculous to avoid using queues > because of lock contention when the locks are only > protecting a couple lines of code. > > My reading is that there are two, closely related, things > going on: the first is lock contention, and the second is > cache line contention. We have a primitive in 8.x > (don't think it's been MFC'd yet) for a lockless > atomic buffer primitive for use in drivers and other parts > of the stack. However, that addresses only lock contention, > not line contention, which at a high PPS will be an issue as > well. Only by moving to independent data structures (i.e., > on independent cache lines) can we reduce line contention. > > Robert N M Watson > Computer Laboratory > University of Cambridge Are mutexes smart enough to know to yield to higher priority threads that are waiting immediately? Such as mtx_unlock() { do_unlock_stuff(); if (higher_pri_waiting) sched_yield() } Also is there a way from the structure or flags to determing is some other thread is waiting on the lock, such as? mtx_unlock(&mtx); if (mtx.someone_is_waiting) sched_yield(); or better yet if (higher_priority_is_waiting) sched_yield() I don't quite have a handle on how the turnstile works, but it seems that there is a lot of time waiting for very short-lived locks. If the tasks are on different cpus, what is the granularity of the wait time for a lock that is cleared almost immediately after trying it? Also, is the waiting only extended when the threads are running on the same cpu? Barney