Date: Mon, 14 Aug 2017 07:15:07 -0400 From: Tenzin Lhakhang <tenzin.lhakhang@gmail.com> To: "Eugene M. Zheganin" <emz@norma.perm.ru> Cc: "freebsd-fs@FreeBSD.org" <freebsd-fs@freebsd.org>, freebsd-stable <freebsd-stable@freebsd.org> Subject: Re: zfs listing and CPU Message-ID: <CALcn87z8=FVey=KNARtG5PcbE6dz-Z%2BPj%2BtHuDoVx7MsztKYEA@mail.gmail.com> In-Reply-To: <c07474d7-b47d-2795-405c-fe8d275e6145@norma.perm.ru> References: <aa26e888-05ef-c876-abf3-778ff08f4857@norma.perm.ru> <E3238E24-3AFA-4F2C-A299-D52E4D152097@kraus-haus.org> <399c4309-c7a6-a7e9-299f-9675224c090d@norma.perm.ru> <CALcn87zig7b37Q_JByVMrWyPPF=qr%2BWO%2BcB4aP4-wFp-_B4FOw@mail.gmail.com> <c07474d7-b47d-2795-405c-fe8d275e6145@norma.perm.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
Interesting, even the illumos man page doesn't have the -c defined. I remember reading about it at one point. I can only get `zfs get -Hrpc all` working, even though -c is not documented. So, I retract my statement on the zfs list with cached. # zfs get -*Hpc* -o property,value name,used,avail zones name zones used 1219313598464 available 761203195904 # I use the above way to create easily parseable zfs_dataset objects. As for needing immediate/latest version of zfs list on entity creation -- you ideally want a way to do cache invalidation or use some sort of message/event driven system between the client that creates the entity and the server. You can perform the cache invalidation by monitoring the 'creation' events that come to your API, or by tailing zpool history [pool] for changes. - I recently read this article and they show a nice way to decouple the storage events and the api, architecture is not entirely new but can be adapted to your case. - https://medium.freecodecamp.org/million-websockets-and-go-cc58418460bb Thanks, Tenzin On Sun, Aug 13, 2017 at 2:25 PM, Eugene M. Zheganin <emz@norma.perm.ru> wrote: > On 13.08.2017 16:13, Tenzin Lhakhang wrote: > >> You may want to have an async zfs-get program/script that regularly does >> a zfs get -Ho and stores then in a local cache (redis or your own program) >> at a set interval and then the api can hit the cache instead of directly >> running get or list. >> > I cannot because the cache will become stale on first new entity creation, > which happens all the time. > > - Some silly person will try to benchmark your zfs web-API and overload >> your server with zfs processes. >> - Example: let me run [ ab -c 10 -n 10000 http://yourserver/zfs-api/list >> ] >> -- Let me run 10 concurrent connection with a total of 10k requests to >> your api (it's a simple one liner -- people will be tempted to benchmark >> like this). >> >> Example: >> https://github.com/tlhakhan/ideal-potato/blob/master/zdux/ro >> uters/zfs/service.js#L9 >> - This is a JS example, but you can easily script it or another language >> (golang) for cache separation and another program for the API. >> >> Also, zfs does have a -c property to get cached values -- these values >> are stored in an internal zfs process cache. The -c doesn't help if you >> have 1000(0)s of filesystems, a single list can still take minutes. >> Sending the list is also several megabytes. >> > Doesn't have on FreeBSD. > > > > Thanks. > Eugene. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CALcn87z8=FVey=KNARtG5PcbE6dz-Z%2BPj%2BtHuDoVx7MsztKYEA>