From owner-svn-src-head@FreeBSD.ORG Thu Oct 27 16:12:26 2011 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F536106566B; Thu, 27 Oct 2011 16:12:26 +0000 (UTC) (envelope-from pjd@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 449038FC0A; Thu, 27 Oct 2011 16:12:26 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id p9RGCQ9M014226; Thu, 27 Oct 2011 16:12:26 GMT (envelope-from pjd@svn.freebsd.org) Received: (from pjd@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id p9RGCQab014223; Thu, 27 Oct 2011 16:12:26 GMT (envelope-from pjd@svn.freebsd.org) Message-Id: <201110271612.p9RGCQab014223@svn.freebsd.org> From: Pawel Jakub Dawidek Date: Thu, 27 Oct 2011 16:12:26 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r226840 - head/sys/geom/eli X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Oct 2011 16:12:26 -0000 Author: pjd Date: Thu Oct 27 16:12:25 2011 New Revision: 226840 URL: http://svn.freebsd.org/changeset/base/226840 Log: Before this change when GELI detected hardware crypto acceleration it will start only one worker thread. For software crypto it will start by default N worker threads where N is the number of available CPUs. This is not optimal if hardware crypto is AES-NI, which uses CPU for AES calculations. Change that to always start one worker thread for every available CPU. Number of worker threads per GELI provider can be easly reduced with kern.geom.eli.threads sysctl/tunable and even for software crypto it should be reduced when using more providers. While here, when number of threads exceeds number of CPUs avilable don't reduce this number, assume the user knows what he is doing. Reported by: Yuri Karaban MFC after: 3 days Modified: head/sys/geom/eli/g_eli.c head/sys/geom/eli/g_eli.h Modified: head/sys/geom/eli/g_eli.c ============================================================================== --- head/sys/geom/eli/g_eli.c Thu Oct 27 14:15:26 2011 (r226839) +++ head/sys/geom/eli/g_eli.c Thu Oct 27 16:12:25 2011 (r226840) @@ -443,16 +443,15 @@ g_eli_worker(void *arg) sc = wr->w_softc; #ifdef SMP /* Before sched_bind() to a CPU, wait for all CPUs to go on-line. */ - if (mp_ncpus > 1 && sc->sc_crypto == G_ELI_CRYPTO_SW && - g_eli_threads == 0) { + if (sc->sc_cpubind) { while (!smp_started) tsleep(wr, 0, "geli:smp", hz / 4); } #endif thread_lock(curthread); sched_prio(curthread, PUSER); - if (sc->sc_crypto == G_ELI_CRYPTO_SW && g_eli_threads == 0) - sched_bind(curthread, wr->w_number); + if (sc->sc_cpubind) + sched_bind(curthread, wr->w_number % mp_ncpus); thread_unlock(curthread); G_ELI_DEBUG(1, "Thread %s started.", curthread->td_proc->p_comm); @@ -813,11 +812,7 @@ g_eli_create(struct gctl_req *req, struc threads = g_eli_threads; if (threads == 0) threads = mp_ncpus; - else if (threads > mp_ncpus) { - /* There is really no need for too many worker threads. */ - threads = mp_ncpus; - G_ELI_DEBUG(0, "Reducing number of threads to %u.", threads); - } + sc->sc_cpubind = (mp_ncpus > 1 && threads == mp_ncpus); for (i = 0; i < threads; i++) { if (g_eli_cpu_is_disabled(i)) { G_ELI_DEBUG(1, "%s: CPU %u disabled, skipping.", @@ -857,9 +852,6 @@ g_eli_create(struct gctl_req *req, struc goto failed; } LIST_INSERT_HEAD(&sc->sc_workers, wr, w_next); - /* If we have hardware support, one thread is enough. */ - if (sc->sc_crypto == G_ELI_CRYPTO_HW) - break; } /* Modified: head/sys/geom/eli/g_eli.h ============================================================================== --- head/sys/geom/eli/g_eli.h Thu Oct 27 14:15:26 2011 (r226839) +++ head/sys/geom/eli/g_eli.h Thu Oct 27 16:12:25 2011 (r226840) @@ -192,6 +192,7 @@ struct g_eli_softc { size_t sc_sectorsize; u_int sc_bytes_per_sector; u_int sc_data_per_sector; + boolean_t sc_cpubind; /* Only for software cryptography. */ struct bio_queue_head sc_queue;