From owner-cvs-all@FreeBSD.ORG Thu Jul 21 12:09:02 2005 Return-Path: X-Original-To: cvs-all@FreeBSD.org Delivered-To: cvs-all@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 26E2A16A424; Thu, 21 Jul 2005 12:09:02 +0000 (GMT) (envelope-from glebius@FreeBSD.org) Received: from repoman.freebsd.org (repoman.freebsd.org [216.136.204.115]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0D6FC43D73; Thu, 21 Jul 2005 12:08:38 +0000 (GMT) (envelope-from glebius@FreeBSD.org) Received: from repoman.freebsd.org (localhost [127.0.0.1]) by repoman.freebsd.org (8.13.1/8.13.1) with ESMTP id j6LC8bb4078743; Thu, 21 Jul 2005 12:08:37 GMT (envelope-from glebius@repoman.freebsd.org) Received: (from glebius@localhost) by repoman.freebsd.org (8.13.1/8.13.1/Submit) id j6LC8bUK078742; Thu, 21 Jul 2005 12:08:37 GMT (envelope-from glebius) Message-Id: <200507211208.j6LC8bUK078742@repoman.freebsd.org> From: Gleb Smirnoff Date: Thu, 21 Jul 2005 12:08:37 +0000 (UTC) To: src-committers@FreeBSD.org, cvs-src@FreeBSD.org, cvs-all@FreeBSD.org X-FreeBSD-CVS-Branch: HEAD Cc: Subject: cvs commit: src/sys/netgraph ng_base.c X-BeenThere: cvs-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: CVS commit messages for the entire tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Jul 2005 12:09:02 -0000 glebius 2005-07-21 12:08:37 UTC FreeBSD src repository Modified files: sys/netgraph ng_base.c Log: Problem description: At the end of ng_snd_item(), node queue is processed. In certain netgraph setups deep recursive calls can occur. For example this happens, when two nodes are connected and can send items to each other in both directions. If, for some reason, both nodes have a lot of items in their queues, then the processing thread will recurse between these two nodes, delivering items left and right, going deeper in the stack. Other setups can suffer from deep recursion, too. The following factors can influence risk of deep netgraph call: - periodical write-access events on node - combination of slow link and fast one in one graph - net.inet.ip.fastforwarding Changes made: - In ng_acquire_{read,write}() do not dequeue another item. Instead, call ng_setisr() for this node. - At the end of ng_snd_item(), do not process queue. Call ng_setisr(), if there are any dequeueable items on node queue. - In ng_setisr() narrow worklist mutex holding. - In ng_setisr() assert queue mutex. Theoretically, the first two changes should negatively affect performance. To check this, some profiling was made: 1) In general real tasks, no noticable performance difference was found. 2) The following test was made: two multithreaded nodes and one single-threaded were connected into a ring. A large queues of packets were sent around this ring. Time to pass the ring N times was measured. This is a very vacuous test: no items/mbufs are allocated, no upcalls or downcalls outside of netgraph. It doesn't represent a real load, it is a stress test for ng_acquire_{read,write}() and item queueing functions. Surprisingly, the performance impact was positive! New code is 13% faster on UP and 17% faster on SMP, in this particular test. The problem was originally found, described, analyzed and original patch was written by Roselyn Lee from Vernier Networks. Thanks! Submitted by: Roselyn Lee Revision Changes Path 1.103 +17 -67 src/sys/netgraph/ng_base.c