Date: Mon, 2 Jun 2014 16:04:30 +0300 From: Zaro Korchev <zkorchev@mail.bg> To: Jonathan Anderson <jonathan.robert.anderson@gmail.com> Cc: Vsevolod Stakhov <vsevolod@FreeBSD.org>, Eitan Adler <eadler@FreeBSD.org>, David Chisnall <theraven@theravensnest.org>, Pawel Jakub Dawidek <pjd@freebsd.org>, soc-status@freebsd.org, Jonathan Anderson <jonathan@FreeBSD.org> Subject: Re: [Machine readable output from userland utilities] report Message-ID: <82CCE1ED-525A-4F24-9911-BE3A26D2FC4F@mail.bg> In-Reply-To: <1ED224CA-9E91-459F-9ADE-E614755D1AA2@gmail.com> References: <8D1B686D-1AAA-4E07-9270-E42699110561@mail.bg> <4890861C-FC91-445D-AE9B-31CD5FDFD0A9@theravensnest.org> <15BC1D7C-B909-48DB-AB6D-FF0F0F9C2B0A@mail.bg> <899129C5-977C-4CE7-A873-460D69D6EA85@theravensnest.org> <5385BD49.5020602@FreeBSD.org> <105B3699-2A41-4384-8D02-523D8445436B@mail.bg> <1ED224CA-9E91-459F-9ADE-E614755D1AA2@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Jonathan
I just saw your email. I must have missed it earlier. Sorry about that.
The functions were just some example names. I forgot to put the sol_ =
prefix in all of them.
I understand your concerns about multi-threading. The idea is to have =
functions that serialize the object in an allocated buffer as it is =
constructed. Here is a more detailed example of what I mean:
char *ucldump_object_start(const struct ucldump *restrict ctx, char =
*buf, size_t buf_len)
{
if (ctx->format =3D=3D UCL_EMIT_JSON_COMPACT)
{
if (buf_len < 1) return 0;
buf[0] =3D '{';
return buf + 1;
}
// etc.
}
char *ucldump_object_key(const struct ucldump *restrict ctx, char =
*restrict buf, size_t buf_len, const char *restrict key, size_t key_len)
{
if (ctx->format =3D=3D UCL_EMIT_JSON_COMPACT)
{
// Make sure there is enough space for
// "key":
if (buf_len < (1 + key_len + 1 + 1)) return 0;
*buf++ =3D '"';
*buf++ =3D '"';
memcpy(buf, key, key_len);
buf +=3D key_len;
*buf++ =3D ':';
return buf;
}
// etc.
}
This is still just an example. I don't know what will be the most =
appropriate way to implement this.
The idea is that for JSON and YAML sol_object_start() can call =
ucldump_object_start(), sol_object_key can call ucldump_object_key() and =
so on. This way UCL can take care of the exact output format while SOL =
provides a uniform API for all formats (including other formats like XML =
that can use a different backend).
Since the serialized data is written to an allocated buffer, such =
functions will not be an issue for multi-threading. Multi-threaded =
applications just need to use separate buffers and separate =
files/sockets/pipes/whatever (which they must do anyway).
Zaro
Le 28 May 2014 =E0 15:16, Jonathan Anderson a =E9crit :
> On 28 May 2014, at 9:36, Zaro Korchev <zkorchev@mail.bg> wrote:
>> The idea is to serialize the object as it is constructed. That way it =
is not necessary to keep all the data in memory at any given moment. =
This also allows to create pipeline when the second application starts =
consuming output before the first one finished producing it.
>>=20
>> for example:
>> object_start(buf, ...) writes to buf:
>> {
>> object_key(buf, ..., "foo") writes to buf:
>> "foo"
>> sol_object_integer(buf, ..., 42) writes to buf:
>> : 42
>> sol_object_end(buf, ...) writes to buf:
>> }
>=20
> I notice that you have a mix of "object_" and "sol_object_" calls: are =
these all purely serialization functions (and should they all be =
prefixed with "sol_")? It would be unfortunate to commit to a stateful =
object *creation* API, as that would make things difficult for =
multithreaded work (e.g. each thread creates an object, each of which is =
added to the top-level object as it is completed rather than in lexical =
order).
>=20
>=20
> Jon
> --
> Jonathan Anderson
>=20
> jonathan@FreeBSD.org
> http://freebsd.org/~jonathan/
>=20
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?82CCE1ED-525A-4F24-9911-BE3A26D2FC4F>