From owner-freebsd-current@freebsd.org Mon Oct 9 20:25:54 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5547DE3AF18 for ; Mon, 9 Oct 2017 20:25:54 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-cys01nam02on0065.outbound.protection.outlook.com [104.47.37.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F050B7E1FD; Mon, 9 Oct 2017 20:25:52 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YQXPR0101MB0997.CANPRD01.PROD.OUTLOOK.COM (52.132.78.18) by YQXPR0101MB1367.CANPRD01.PROD.OUTLOOK.COM (52.132.81.19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7; Mon, 9 Oct 2017 20:25:51 +0000 Received: from YQXPR0101MB0997.CANPRD01.PROD.OUTLOOK.COM ([fe80::7548:a72a:e054:70d5]) by YQXPR0101MB0997.CANPRD01.PROD.OUTLOOK.COM ([fe80::7548:a72a:e054:70d5%13]) with mapi id 15.20.0077.011; Mon, 9 Oct 2017 20:25:51 +0000 From: Rick Macklem To: Ian Lepore , "freebsd-current@freebsd.org" Subject: Re: RFC how to use kernel procs/threads efficiently Thread-Topic: RFC how to use kernel procs/threads efficiently Thread-Index: AQHTPtSLQYujf8n5KEGHc2bjAF54kqLXMAcAgATKEsQ= Date: Mon, 9 Oct 2017 20:25:51 +0000 Message-ID: References: , <1507317060.86205.268.camel@freebsd.org> In-Reply-To: <1507317060.86205.268.camel@freebsd.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; YQXPR0101MB1367; 6:2DIHaTH3H81tKCTGthZMgpK5NbUC8fTY4at5VxYvAZEGzLhR6ZwLuPDvK1nJlcv9p8DYWZAGO54rdDxe9VpZgErzLnvsZd7EPMlZQw90hrRrZxwOjNsUDgeQkBtXYMOpNLa3UNo3j+X/bMSTK0g3krOX2BzbHLo0IiJ9b+EXKzGHfexwcxXlzv8sKb5ShXbqwWb0dTcP9EfJnjtOpr7zOvdqzYNVZ6IalRO+lB5rJXNix4vW+X+6yVlKX1LkL1LL0B15bHbeIFieRc+gIfjJdcgKKJWD4V+S+YeKb0IsMlEPBiK8ghzEHXKfXNi/4B4/V+wcvZazVjVjw7fh7SUfeg==; 5:AGbJXSOTbKiNMreFoz+/UrWajFkcyanq9hLJh5x4UIyVF0kQ263VHtlcyBlAV1ziJZPY5tppJ0UKRj39pU9WyC54Ypd3jr1YUvJ3sRoKjCbwgG7XvK5kq9UE4ECGNaVT7EcMo47/lV/busyoTSK5sQ==; 24:ngrQMc7ntq2NRADsuLDSWjORjbVMCQLQa0x70hre0Wn4KSkwHgSsuOXVlmIIi3n75SBWrxAW8SmnonIw40VG5H/rM/DE+eeRmFe2Kvq6WtA=; 7:es6tzQl2UlO80RSjvOwZKqyYa1N5MC7cmp/aimwPywtFTmYx/WO+rk02hpaNPV6b7VzfRaAGw8ay+irmVCn0NcO4E5AMlIwzXL7ktBfUGq7azyWlRdyihajIlHqi3DEWBM6dSsM3Iicsu61Vjc3C8z+jUG/cFUO0I84MLr+99+kzeE/32QglsLU+bEvGOBPu2TBYxQLSoy/OUCS7M9FYPaSjZl5WAL+GILAaOSLFoUM= x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: ed656760-b369-46e1-94d0-08d50f53ef64 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254152)(2017052603199)(201703131423075)(201703031133081)(201702281549075); SRVR:YQXPR0101MB1367; x-ms-traffictypediagnostic: YQXPR0101MB1367: x-exchange-antispam-report-test: UriScan:; x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(3002001)(93006095)(93001095)(10201501046)(100000703101)(100105400095)(6041248)(20161123564025)(20161123555025)(20161123562025)(201703131423075)(201702281529075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:YQXPR0101MB1367; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:YQXPR0101MB1367; x-forefront-prvs: 045584D28C x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(346002)(376002)(189002)(24454002)(199003)(450100002)(74316002)(5660300001)(55016002)(14454004)(6306002)(7696004)(74482002)(53936002)(97736004)(68736007)(966005)(81166006)(2501003)(6246003)(478600001)(102836003)(2906002)(305945005)(8676002)(86362001)(81156014)(110136005)(3660700001)(8936002)(76176999)(786003)(33656002)(6436002)(3280700002)(54356999)(229853002)(101416001)(189998001)(2950100002)(105586002)(5250100002)(316002)(9686003)(50986999)(6506006)(25786009)(106356001)(2900100001); DIR:OUT; SFP:1101; SCL:1; SRVR:YQXPR0101MB1367; H:YQXPR0101MB0997.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Oct 2017 20:25:51.4435 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQXPR0101MB1367 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Oct 2017 20:25:54 -0000 ________________________________________ Ian Lepore wrote: [stuff snipped] >taskqueue(9) is an existing mechanism to enqueue functions to execute >asynch using a pool of threads, but it doesn't answer the scalability >questions. In fact it may make them harder, inasmuch as I don't think >there's a mechanism to dynamically adjust the number of threads after >first calling taskqueue_start_threads(). I've coded it using taskqueue and it seems to work ok. The patch is here, in case anyone would like to review it: https://www.reviews.freebsd.org/D12632 I don't know what the overheads are (or even how to measure/compare them), but I suspect it is less than a kproc_create()/kproc_exit() for ever= y RPC. I also don't think having a fixed # of threads is a problem. Since NFS I/O is so bursty, recent I/O activity doesn't give a good indication of how man= y threads will be needed soon. In other words, it can go from no I/O to heavy I/O and back to no I/O rapidly. --> As such, having a fixed reasonable # of threads is probably the best that can be done. - The current patch has the # of threads as a sysctl with a default o= f 32. Thanks for your comments and feel free to review it, if you'd like, rick