From owner-freebsd-ipfw@FreeBSD.ORG Tue Apr 1 04:55:15 2008 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2C58A1065670; Tue, 1 Apr 2008 04:55:15 +0000 (UTC) (envelope-from root@mmu.edu.my) Received: from staff.cyber.mmu.edu.my (staff.cyber.mmu.edu.my [203.106.62.12]) by mx1.freebsd.org (Postfix) with ESMTP id 6B7808FC14; Tue, 1 Apr 2008 04:55:14 +0000 (UTC) (envelope-from root@mmu.edu.my) Received: by staff.cyber.mmu.edu.my (Postfix, from userid 0) id C5DAE4D64FF; Tue, 1 Apr 2008 12:35:27 +0800 (MYT) Received: from mx2.freebsd.org (mx2.freebsd.org [69.147.83.53]) by mmu.edu.my (Postfix) with ESMTP id 97D8A55E498 for ; Thu, 27 Mar 2008 17:16:05 +0800 (MYT) Received: from hub.freebsd.org (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id BC7FF1A7363; Thu, 27 Mar 2008 09:15:35 +0000 (UTC) (envelope-from owner-freebsd-hackers@freebsd.org) Received: from hub.freebsd.org (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 42D6C1065746; Thu, 27 Mar 2008 09:15:33 +0000 (UTC) (envelope-from owner-freebsd-hackers@freebsd.org) Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D147A1065673; Thu, 27 Mar 2008 09:15:22 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id A58998FC13; Thu, 27 Mar 2008 09:15:22 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id EBB7346B8F; Thu, 27 Mar 2008 05:15:21 -0400 (EDT) Date: Thu, 27 Mar 2008 09:15:21 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Julian Elischer In-Reply-To: <47EA8860.3060709@elischer.org> Message-ID: <20080327090909.T34007@fledge.watson.org> References: <47E79636.1000909@FreeBSD.org> <47E7EAA8.7020101@elischer.org> <47EA8860.3060709@elischer.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Sender: owner-freebsd-hackers@freebsd.org Errors-To: owner-freebsd-hackers@freebsd.org Cc: vadim_nuclight@mail.ru, freebsd-hackers@freebsd.org, freebsd-ipfw@freebsd.org Subject: Re: [HEADS UP!] IPFW Ideas: possible SoC 2008 candidate X-BeenThere: freebsd-ipfw@freebsd.org List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2008 04:55:15 -0000 On Wed, 26 Mar 2008, Julian Elischer wrote: > it wouldn't.. you'd add them together before presenting them. but every time > a packet changes a counter that is shared, there is a chance that it is > being altered by another processor, so if you have fine grained locking in > ipfw, you really should use atomic adds, which are slow, or accept possibl > collisions (which might be ok) but still cause a lot of cross cpu TLB > flushing. In malloc(9) and uma(9), we maintain per-CPU stats, coalescing only for presentation, relying on soft critical sections rather than locks to pretect consistency. What's worth remembering, however, is that recent multicore machines have significantly optimized the cost of atomic operations on cache lines held for write by the current CPU, and so the cost of locking has dramatically fallen in the last few years. This re-emphasizes the importance of careful cacheline management for per-CPU data structures (particularly, don't put data written by multiple CPUs in the same cacheline if you want the benefits of per-CPU access). Where read-write locking is the best model, Stephan's recent work on rmlocks looks quite promising. In my micro-benchmarks, on recent hardware it performs extremely well on SMP for read locks, but still requires optimization for UP-compiled kernels. For stats and writable structures, such as per-CPU caches, rmlocks aren't very helpful, but when compared with replicating infrequently written data structures across many CPUs, rwlocks/rmlocks offer a much simpler and less error-prone programming model. We need to see more optimization and measurement done on rmlocks for 8.x, and the lack of full priority propagation for rwlocks has to be kept in mind. Robert N M Watson Computer Laboratory University of Cambridge _______________________________________________ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"