From owner-freebsd-current@freebsd.org Thu Oct 24 19:20:37 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3BEA3177F27 for ; Thu, 24 Oct 2019 19:20:37 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f194.google.com (mail-oi1-f194.google.com [209.85.167.194]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46zcXP0wMCz4Vlh; Thu, 24 Oct 2019 19:20:36 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi1-f194.google.com with SMTP id g81so21637438oib.8; Thu, 24 Oct 2019 12:20:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=0cTwGaz1PPY9j8Ne5ycrQNwDtgcvr9swPifGiJFEUMU=; b=XshsD2p+muTKkPNznxb3E/cf2mPxvxZGPvGOMJRCqM0azP4fGKJkqrCkfhPWJkhFen EGNMtyr2Jmrb8AKtdaJHl7Xh4a70fqpYjAg74DwLeA+lpURjpjf37X36OX+E4YjqRkD3 rFcC4qdjeSAGv58JwfKk8p1xQQQjn1RMl7RYaU5xxSnfZHk+mvsca4nnEPhw5EBO0J0f tiir0m2LJP4S65EoVnoWps1hgI24FVfCJaprX5L85IItce1rbHd4ahnfh7ca51oz5CeW nOtNCm7u4oMTW5tY5wBBJCs3SP71GrTzm3uuiEo5GlAJFXrmDcO1ce75LdV9z0zhKrFt q2tg== X-Gm-Message-State: APjAAAXKtxNI7qwAmD8BkPHJBzcpPhO0s+5cN8YZpRZfsvZbwhujafz1 y97NDy8G4U1/CDiaGYzG1fiZGWonbjOesKPx6ODadDjD X-Google-Smtp-Source: APXvYqwf5e6lfq8+VbPBnYUxPfprVGxkrZrOeDUPTvxxCsCj7c4n1nF5nmVczBP6uQSeWDOD/YjVmq97T18gZGinn4Y= X-Received: by 2002:aca:cf92:: with SMTP id f140mr6279222oig.57.1571944835032; Thu, 24 Oct 2019 12:20:35 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Thu, 24 Oct 2019 13:20:23 -0600 Message-ID: Subject: Re: Best thread pool implementation for the base system? To: Ian Lepore Cc: FreeBSD CURRENT X-Rspamd-Queue-Id: 46zcXP0wMCz4Vlh X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2019 19:20:37 -0000 On Thu, Oct 24, 2019 at 12:57 PM Ian Lepore wrote: > On Thu, 2019-10-24 at 12:46 -0600, Alan Somers wrote: > > I count 5 thread pool implementations in contrib: > > * cddl/compat/opensolaris/misc/thread_pool.c > > * contrib/apr-util/misc/apr_thread_pool.c > > * contrib/llvm/lib/Support/ThreadPool.cpp > > * contrib/openmp/runtime/src/kmp_tasking.cpp > > * contrib/ofed/opensm/complib/cl_threadpool.c > > > > However, I can't find any examples outside of contrib. I'd like to > > use one > > in /sbin/geli. Shall I roll my own (as everybody else apparently > > does), or > > is there something I'm not aware of? > > > > > > Whenever the subject of thread pools comes up at $work I pose this > question: What task is it you need to accomplish where the cost of > pthread_create/delete is significant in relation to the actual work > that will be done during the lifetime of the thread? > > Over the years, the answers have been such that we never created a > thread pool class or helpers. What we did eventually come up with was > basically an async taskqueue that had a single thread, because almost > always the answer to the question was something like "the work to be > done is really tiny and not-time-critical, it just needs to be done on > a different thread to avoid [recursion|deadlock|whatever]." > > So I'd say the first thing to do is be sure that the best solution > isn't just to pthread_create() as needed. If it turns out the cost of > pthread_create() is like 1-2% of the total work to be done, you may not > need a pool of pre-created threads. > > -- Ian > Well, the time needed by pthread_create/delete is probably not significant. But the memory consumption is. I could have up to 400 tasks, and I don't want to run all of them in parallel because all of those stacks add up. But I definitely need some amount of parallelism. FYI my interest is in speeding up "geli attach" by parallelizing the I/O to multiple disks. Currently it attaches to the disks one at a time. -Alan