From owner-svn-src-projects@FreeBSD.ORG Mon Aug 26 12:55:54 2013 Return-Path: Delivered-To: svn-src-projects@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 8D1873EB; Mon, 26 Aug 2013 12:55:54 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6BB95219F; Mon, 26 Aug 2013 12:55:54 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7QCts55057865; Mon, 26 Aug 2013 12:55:54 GMT (envelope-from mav@svn.freebsd.org) Received: (from mav@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id r7QCts7t057864; Mon, 26 Aug 2013 12:55:54 GMT (envelope-from mav@svn.freebsd.org) Message-Id: <201308261255.r7QCts7t057864@svn.freebsd.org> From: Alexander Motin Date: Mon, 26 Aug 2013 12:55:54 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r254908 - projects/camlock/sys/geom X-SVN-Group: projects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Aug 2013 12:55:54 -0000 Author: mav Date: Mon Aug 26 12:55:53 2013 New Revision: 254908 URL: http://svnweb.freebsd.org/changeset/base/254908 Log: Use per-consumer/provider locks from mtx_pool() to serialize the statistics update instead of global ones. Additional lock acquisition for queued dispatch may add some overhead, but instead it removes any collisions for direct dispatch. Modified: projects/camlock/sys/geom/geom_io.c Modified: projects/camlock/sys/geom/geom_io.c ============================================================================== --- projects/camlock/sys/geom/geom_io.c Mon Aug 26 12:18:51 2013 (r254907) +++ projects/camlock/sys/geom/geom_io.c Mon Aug 26 12:55:53 2013 (r254908) @@ -465,6 +465,7 @@ g_io_request(struct bio *bp, struct g_co { struct g_provider *pp; struct thread *td; + struct mtx *mtxp; int direct, error, first; KASSERT(cp != NULL, ("NULL cp in g_io_request")); @@ -538,27 +539,28 @@ g_io_request(struct bio *bp, struct g_co direct = 0; #endif + if (!TAILQ_EMPTY(&g_classifier_tailq) && !bp->bio_classifier1) { + g_bioq_lock(&g_bio_run_down); + g_run_classifiers(bp); + g_bioq_unlock(&g_bio_run_down); + } + /* * The statistics collection is lockless, as such, but we * can not update one instance of the statistics from more * than one thread at a time, so grab the lock first. - * - * We also use the lock to protect the list of classifiers. */ - g_bioq_lock(&g_bio_run_down); - - if (!TAILQ_EMPTY(&g_classifier_tailq) && !bp->bio_classifier1) - g_run_classifiers(bp); - + mtxp = mtx_pool_find(mtxpool_sleep, pp); + mtx_lock(mtxp); if (g_collectstats & 1) devstat_start_transaction(pp->stat, &bp->bio_t0); if (g_collectstats & 2) devstat_start_transaction(cp->stat, &bp->bio_t0); - pp->nstart++; cp->nstart++; + mtx_unlock(mtxp); + if (direct) { - g_bioq_unlock(&g_bio_run_down); error = g_io_check(bp); if (error >= 0) { CTR3(KTR_GEOM, "g_io_request g_io_check on bp %p " @@ -569,6 +571,7 @@ g_io_request(struct bio *bp, struct g_co } bp->bio_to->geom->start(bp); } else { + g_bioq_lock(&g_bio_run_down); first = TAILQ_EMPTY(&g_bio_run_down.bio_queue); TAILQ_INSERT_TAIL(&g_bio_run_down.bio_queue, bp, bio_queue); bp->bio_flags |= BIO_ONQUEUE; @@ -587,6 +590,7 @@ g_io_deliver(struct bio *bp, int error) struct g_consumer *cp; struct g_provider *pp; struct thread *td; + struct mtx *mtxp; int direct, first; KASSERT(bp != NULL, ("NULL bp in g_io_deliver")); @@ -656,20 +660,22 @@ g_io_deliver(struct bio *bp, int error) */ if (g_collectstats) binuptime(&now); - g_bioq_lock(&g_bio_run_up); + mtxp = mtx_pool_find(mtxpool_sleep, cp); + mtx_lock(mtxp); if (g_collectstats & 1) devstat_end_transaction_bio_bt(pp->stat, bp, &now); if (g_collectstats & 2) devstat_end_transaction_bio_bt(cp->stat, bp, &now); - cp->nend++; pp->nend++; + mtx_unlock(mtxp); + if (error != ENOMEM) { bp->bio_error = error; if (direct) { - g_bioq_unlock(&g_bio_run_up); biodone(bp); } else { + g_bioq_lock(&g_bio_run_up); first = TAILQ_EMPTY(&g_bio_run_up.bio_queue); TAILQ_INSERT_TAIL(&g_bio_run_up.bio_queue, bp, bio_queue); bp->bio_flags |= BIO_ONQUEUE; @@ -680,7 +686,6 @@ g_io_deliver(struct bio *bp, int error) } return; } - g_bioq_unlock(&g_bio_run_up); if (bootverbose) printf("ENOMEM %p on %p(%s)\n", bp, pp, pp->name);