From owner-soc-status@FreeBSD.ORG Wed May 28 05:04:14 2014 Return-Path: Delivered-To: soc-status@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D46C211F for ; Wed, 28 May 2014 05:04:14 +0000 (UTC) Received: from mail-qg0-x22e.google.com (mail-qg0-x22e.google.com [IPv6:2607:f8b0:400d:c04::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8B5602AD9 for ; Wed, 28 May 2014 05:04:14 +0000 (UTC) Received: by mail-qg0-f46.google.com with SMTP id q108so15831989qgd.19 for ; Tue, 27 May 2014 22:04:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type:content-transfer-encoding; bh=t9sCwYpcL2I0fpYatcVzCIPm1wvM5+jA4bWDkdV8QsE=; b=ndXUamVg+TV7zZ8KtNQeUe8j2mgGFgRoInWrDECAjyNXrRYTPNWC/Q+dctZ40Gc8Iz S36kYp7d/3oqxy/qzVxEBEiPE+oUoC8xDiBJkaUtgmop8xnY08ull3s+TNHlBZnxu17/ x0VF5aqRNv7YgJhP5pVk4PmFkA2HYtI2K37eM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type :content-transfer-encoding; bh=t9sCwYpcL2I0fpYatcVzCIPm1wvM5+jA4bWDkdV8QsE=; b=UDsdn+fmKB1UO1Y1ExP2jXaMidmMq3gFxLzc9/G15Y7Te4YhuiG79auhA87m9OnvzQ D+RGpwbtgg20OajqzrKU7mB1YEx9PDAn41A9Y/2I70gKLtFF79wW6ZyncoQzpql8Nba/ XZBnm3DPSjgrq4zCLfEm0fGKdXpP/rcSVGBAy04RnguOXU2s1nyTux6y48ikYg9hnG8K rq8LNrA74R0sa/bLDI+/bLAw/royeqUGDvctASdut76VmpDpdTl4j1lEJd8BRCrdONdq hU6ZGebI+n+5JFdFkeACWHh5OSJS3PVmuGTwkDCqpDvzzqs/d3FxpzVhmANLqmxFqWJw U3Wg== X-Gm-Message-State: ALoCoQknNF+PFrGZfSvAVZa3N/j27clGa5vhnFqSCJ+giTL2x6u+H5gwwHaau+GSYhUyYjjEg29B X-Received: by 10.224.135.132 with SMTP id n4mr3930314qat.23.1401253453650; Tue, 27 May 2014 22:04:13 -0700 (PDT) MIME-Version: 1.0 Sender: lists@eitanadler.com Received: by 10.96.147.135 with HTTP; Tue, 27 May 2014 22:03:43 -0700 (PDT) In-Reply-To: <899129C5-977C-4CE7-A873-460D69D6EA85@theravensnest.org> References: <8D1B686D-1AAA-4E07-9270-E42699110561@mail.bg> <4890861C-FC91-445D-AE9B-31CD5FDFD0A9@theravensnest.org> <15BC1D7C-B909-48DB-AB6D-FF0F0F9C2B0A@mail.bg> <899129C5-977C-4CE7-A873-460D69D6EA85@theravensnest.org> From: Eitan Adler Date: Tue, 27 May 2014 22:03:43 -0700 X-Google-Sender-Auth: PWS3fZcxRRcbbtkSWPUw4Z-HR2c Message-ID: Subject: Re: [Machine readable output from userland utilities] report To: David Chisnall Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: vsevolod@freebsd.org, soc-status@freebsd.org, Pawel Jakub Dawidek , jonathan@freebsd.org, Zaro Korchev X-BeenThere: soc-status@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Summer of Code Status Reports and Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 May 2014 05:04:14 -0000 On 26 May 2014 10:30, David Chisnall wrote: > On 26 May 2014, at 15:34, Zaro Korchev wrote: [ Zaro and I were convering about this on a different thread ] > >> I considered using libucl and libnv. The problem with libucl is that it = does not support streamed output so, after a discussion with my mentors, I = decided to implement this prototype version using YAJL. This is not a final= decision but that's what I'm using at the moment. The design will look like: - applications write to libnv for serialization - pass libnv pairs to wrapper library - which in turn unwraps them and passes them off to formatting library such as libucl or yajl or whatnot > > That's fine, as long as you're prototyping an interface for an abstractio= n layer it doesn't matter much what is on the back end, I just wanted to ma= ke sure that you were thinking about libucl as an eventual back end. This should be on the roadmap. By the end the wrapper should work with many applications and multiple formats. That said, as I said to Zaro earlier, I would rather he focus on getting more utility support than a a lot format support. 1 or 2 for the latter is fine. > I've added Vsevolod to the cc list, as he's the author of libucl - perhap= s he can add the missing functionality that you require. Cool. > I definitely agree that streaming is important - we want to be able to co= nstruct pipes of these, although hopefully the total amount of data won't b= e huge. >> I may use libnv soon - I just haven't had need for it yet. It has one li= mitation that I'm concerned about - it does not support arrays (the only su= pported composite data type is key, value pairs). > Arrays in libnv came up at BSDCan. Apparently someone (Pawel?) has patch= es for arrays, but didn't commit them because there were no consumers in the base system that needed them. It sounds like you've just volunteered as a beta tester ;-) pjd, any comments? > It would also be good to consider prepending a header to each stream so t= hat tools can consume them without having to be aware of the format. JSON = has the nice property that it can be spotted quite easily be examining the = first 4 bytes (in any unicode encoding). I'm not sure if UCL and NV have t= he same property - if they do, then we don't need to worry. Good point. I did not think of this. I think it makes sense to always prepend a 'format id' or 'module name' since not all formats are guessable. --=20 Eitan Adler Source, Ports, Doc committer Bugmeister, Ports Security teams