From owner-freebsd-stable@freebsd.org Sun Aug 13 18:25:45 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1DF56DD0A34; Sun, 13 Aug 2017 18:25:45 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 89BA27D9E3; Sun, 13 Aug 2017 18:25:43 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from [IPv6:2a02:2698:26:ba6d:b8c2:5ea7:b69b:1849] (dynamic-2a02-2698-26-0-0.perm.ertelecom.ru [IPv6:2a02:2698:26:ba6d:b8c2:5ea7:b69b:1849] (may be forged)) (authenticated bits=0) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPSA id v7DIPcfZ058994 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 13 Aug 2017 23:25:38 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1502648738; bh=EzrOpIoJuONDvA70FFhY/Lidt4qvvxquAR8qPyJRz8I=; h=Subject:To:Cc:References:From:Date:In-Reply-To; b=tZr4MHnr7MVvBeCars1wCjh9enj1pEaXhEpN0t3yNkrs+tugfPYAVw1HWnU8fEjks jywUijPPD23lpjEYrN4KHR7RNKySriprN3J/RZ1BMXpURJ5IOvahVBHzOiyW+93ggs VqkQzEnxHGztvCSYPcWsNtAM9f7lO6q3js9/HkCg= Subject: Re: zfs listing and CPU To: "freebsd-fs@FreeBSD.org" Cc: freebsd-stable References: <399c4309-c7a6-a7e9-299f-9675224c090d@norma.perm.ru> From: "Eugene M. Zheganin" Message-ID: Date: Sun, 13 Aug 2017 23:25:36 +0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-GB X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Aug 2017 18:25:45 -0000 On 13.08.2017 16:13, Tenzin Lhakhang wrote: > You may want to have an async zfs-get program/script that regularly > does a zfs get -Ho and stores then in a local cache (redis or your own > program) at a set interval and then the api can hit the cache instead > of directly running get or list. I cannot because the cache will become stale on first new entity creation, which happens all the time. > - Some silly person will try to benchmark your zfs web-API and > overload your server with zfs processes. > - Example: let me run [ ab -c 10 -n 10000 http://yourserver/zfs-api/list ] > -- Let me run 10 concurrent connection with a total of 10k requests to > your api (it's a simple one liner -- people will be tempted to > benchmark like this). > > Example: > https://github.com/tlhakhan/ideal-potato/blob/master/zdux/routers/zfs/service.js#L9 > - This is a JS example, but you can easily script it or another > language (golang) for cache separation and another program for the API. > > Also, zfs does have a -c property to get cached values -- these values > are stored in an internal zfs process cache. The -c doesn't help if > you have 1000(0)s of filesystems, a single list can still take > minutes. Sending the list is also several megabytes. Doesn't have on FreeBSD. Thanks. Eugene.