Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 27 May 2014 22:03:43 -0700
From:      Eitan Adler <eadler@freebsd.org>
To:        David Chisnall <theraven@theravensnest.org>
Cc:        vsevolod@freebsd.org, soc-status@freebsd.org, Pawel Jakub Dawidek <pjd@freebsd.org>, jonathan@freebsd.org, Zaro Korchev <zkorchev@mail.bg>
Subject:   Re: [Machine readable output from userland utilities] report
Message-ID:  <CAF6rxgn8%2Bk27ebWhZOH7TBBY7KX7MuNhsw%2BXm3SJT9sRRLuLrg@mail.gmail.com>
In-Reply-To: <899129C5-977C-4CE7-A873-460D69D6EA85@theravensnest.org>
References:  <8D1B686D-1AAA-4E07-9270-E42699110561@mail.bg> <4890861C-FC91-445D-AE9B-31CD5FDFD0A9@theravensnest.org> <15BC1D7C-B909-48DB-AB6D-FF0F0F9C2B0A@mail.bg> <899129C5-977C-4CE7-A873-460D69D6EA85@theravensnest.org>

next in thread | previous in thread | raw e-mail | index | archive | help
On 26 May 2014 10:30, David Chisnall <theraven@theravensnest.org> wrote:
> On 26 May 2014, at 15:34, Zaro Korchev <zkorchev@mail.bg> wrote:

[ Zaro and I were convering about this on a different thread ]
>
>> I considered using libucl and libnv. The problem with libucl is that it =
does not support streamed output so, after a discussion with my mentors, I =
decided to implement this prototype version using YAJL. This is not a final=
 decision but that's what I'm using at the moment.

The design will look like:
- applications write to libnv for serialization
- pass libnv pairs to wrapper library
- which in turn unwraps them and passes them off to formatting library
such as libucl or yajl or whatnot

>
> That's fine, as long as you're prototyping an interface for an abstractio=
n layer it doesn't matter much what is on the back end, I just wanted to ma=
ke sure that you were thinking about libucl as an eventual back end.

This should be on the roadmap.  By the end the wrapper should work
with many applications and multiple formats.  That said, as I said to
Zaro earlier, I would rather he focus on getting more utility support
than a a lot format support.  1 or 2 for the latter is fine.

> I've added Vsevolod to the cc list, as he's the author of libucl - perhap=
s he can add the missing functionality that you require.

Cool.

> I definitely agree that streaming is important - we want to be able to co=
nstruct pipes of these, although hopefully the total amount of data won't b=
e huge.

>> I may use libnv soon - I just haven't had need for it yet. It has one li=
mitation that I'm concerned about - it does not support arrays (the only su=
pported composite data type is key, value pairs).

> Arrays in libnv came up at BSDCan.  Apparently someone (Pawel?) has patch=
es for arrays, but didn't commit them because there were no consumers in
the base system that needed them.  It sounds like you've just
volunteered as a beta tester ;-)

pjd, any comments?

> It would also be good to consider prepending a header to each stream so t=
hat tools can consume them without having to be aware of the format.  JSON =
has the nice property that it can be spotted quite easily be examining the =
first 4 bytes (in any unicode encoding).  I'm not sure if UCL and NV have t=
he same property - if they do, then we don't need to worry.

Good point. I did not think of this.  I think it makes sense to always
prepend a 'format id' or 'module name' since not all formats are
guessable.


--=20
Eitan Adler
Source, Ports, Doc committer
Bugmeister, Ports Security teams



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAF6rxgn8%2Bk27ebWhZOH7TBBY7KX7MuNhsw%2BXm3SJT9sRRLuLrg>