From owner-freebsd-hackers@freebsd.org Tue Sep 11 10:44:12 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AC899108CC14 for ; Tue, 11 Sep 2018 10:44:12 +0000 (UTC) (envelope-from churchers@gmail.com) Received: from mail-ua1-x929.google.com (mail-ua1-x929.google.com [IPv6:2607:f8b0:4864:20::929]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 426F871C55 for ; Tue, 11 Sep 2018 10:44:12 +0000 (UTC) (envelope-from churchers@gmail.com) Received: by mail-ua1-x929.google.com with SMTP id r15-v6so20075817uao.1 for ; Tue, 11 Sep 2018 03:44:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=g4W6FxsFR8/7/r7rOLw0uu54NgQdRFQzb8r1QkauqR0=; b=e0djroLiup/h6LVzCbhjKGW0h0ziEf5mrRGiTJU2BQIqMf2yveXwnfiWEYn0MiDRO1 E2rcK1QBCNcrPmbtjjZlvtcGIRR9FkTXvaWHt/flzvP4F0GJ/rRXhOGSetlWF0Pjtchf u27pkexVO1IA2dIJp2Y6TZi3krllN+DEpuC5qFls6LPuUWryKuzW8Z2Rt3b7qhuFiiOU EuuloHmT4jZIEv3S/spmKV24TAQotuBbyi3Jhs+Ly0D2/qS2CFeLprNnx9ZrErNq7UPQ DH+QztacajDQsbjdayz4B3OVlDwF0uM7utBum54x9M139/m8rEPtxfwkaqD7lBPJzaXE /AuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=g4W6FxsFR8/7/r7rOLw0uu54NgQdRFQzb8r1QkauqR0=; b=tsDm4Pm5VQMpcbGjcoD31Wv/sfpL56EV+Y/m9Tfj1pCVVs1FvLXzoNr4Uw0dQR/4Wn au5fXNwqLJgdpqImPIGlVOx2muoouckS/qqrJFwfaWv2M0KazXu/DJeWXWmVyeYZt8OG jVO6x06QucZrEUJCTZL3I9pI42x9ODHFf7qxOlvM3mYrXX+/Xrue3kLeiqfCGCEoWspc I300NWK+3mZ+CanR4Lp4Spr1D8Pl18/u/z8mBdCLXnNxF/2RHHI9+D1xZPLL1PkY+CCs NmauhUlDnjGbXVbwfuIPfx+FmgID4MHeO9sNAFtZ6NypBBabMGilACT78hQcbc0b4h1H dh9g== X-Gm-Message-State: APzg51CyVUGllo1fIqlstwtPZmYCJtWPOcMzgn1a8sGQOtMEwTpgr5d/ ZDod5rIgbiqU6awB9cvU069Pegpl5ME7lcgCXrp0oGNyQ64= X-Google-Smtp-Source: ANB0VdZrsYMAI0eS9zvKCF0pno1Es8kjrSW2LcuN8uIbAY71wQ63MGOqeL001TUqH2Q3M1hzNupeaA/fuU5/mR8OMHc= X-Received: by 2002:ab0:144a:: with SMTP id c10-v6mr8459200uae.65.1536662651526; Tue, 11 Sep 2018 03:44:11 -0700 (PDT) MIME-Version: 1.0 References: <201809110129.w8B1TG1Z022255@idle.juniper.net> In-Reply-To: <201809110129.w8B1TG1Z022255@idle.juniper.net> From: Matt Churchyard Date: Tue, 11 Sep 2018 11:44:15 +0100 Message-ID: Subject: Re: Getting valid JSON output with xo(1) To: phil@juniper.net Cc: freebsd-hackers@freebsd.org, sjg@juniper.net X-Mailman-Approved-At: Tue, 11 Sep 2018 11:00:37 +0000 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2018 10:44:12 -0000 Hi Phil, Thanks for the detailed response. The part about HTML makes perfect sense. Each outer div is a "line", so without the "\n", you may well want to create further output on the same line by making additional calls. (Assuming of course that you can use the --not-first option to stop it creating another "line" div) As you mention, the fact that xo(1) is designed to be called multiple times in order to build up the output from fragments makes some of this stuff awkward. Thanks for pointing out the --not-first option. I think the sample from #61 looks fairly reasonable. Not that I want to create you guys extra work..., I've no doubt your primary focus is libxo itself which already works fine. Just seems that the xo command line utility is quite hampered if it can't create lists (in JSON at least). At the moment I'm getting around it fairly simply by using something like the following, (the tr calls fortunately only have an effect on JSON output.), which works, but is a lot of jumping through hoops just to get valid JSON. xo -pJ --open machines | tr '{' '[' (if JSON & not first) echo "," (if JSON) echo "{" (if JSON) xo -pJ "{:key1}{:key2}{:key3}" data1 data2 data3 (if not JSON) xo -pJ --wrap machine "{:key1}{:key2}{:key3}" data1 data2 data3 (if JSON) echo "}" xo -pJ --close machines | tr '}' ']' Regards, Matt On Tue, Sep 11, 2018 at 2:28 AM Phil Shafer wrote: > Matt Churchyard writes: > >Sorry if this is the wrong list, I wasn't sure which was the most > >appropriate. > > Best path for xo-related bugs is https://github.com/Juniper/libxo/issues > Email directly to phil@ is also reasonable. > > >If I choose HTML output I get the following invalid output which is > missing > >closing tags. > > > >
> >
test1
> >
,
> >
2GB
> >
> >
test2
> >
,
> >
4GB
> > > >This can be fixed by adding "\n" to the end of the format lines, which may > >be required, but it seems strange that I can so easily generate broken > >output. > > Yes, without a trailing newline, libxo can't declare the output > line "over", so it doesn't make the
. The problem is that > there's no mechanism to "xo" that your output line is continued > from the previous one, and since "xo" is stateless, it will happily > start the next chunk of output with a new open div. I'll add a > "--continuation" option. (#58) > > >"bhyve": { > > "machines": { > > "machine": { > > "name": "test1", > > "memory": "2GB" > > } "machine": { > > "name": "test2", > > "memory": "4GB" > > } > > } > >} > > There's an "xo" option called "--not-first" for exactly these > situations; place that option on the second invocation so it will > know to emit the leading comma. This should be documented (#59). > > >1) The entire document needs to be inside braces "{}" so that "bhyve" is a > >member of an object. > > "xo" doesn't currently do this, since it assume you're generating > JSON piecemeal and have a better idea that it of what you want > to do with the output, but I've opened issue #60 for adding > a "--object-wrapper" option that will perform this. > > >2) There's a missing comma between the two machine entries > > The "--not-first" option will repair this. > > >3) "machines" is an object, which erroneously contains 2 "machine" keys > > This is a bug. "xo" lacks a means of knowing that an object is a > list. One might expect that labeling "name as a key would perform > this, but it doesn't, and it's not clear that this is enough > information. Since "xo" is stateless, it cannot really handle > that "--wrapper machine" applies only to the first item. It > would have to know not to generate the closing "}" for machine, > but it's stateless, so I'd need a means of telling it that. > > In C code, we have "xo_open_list" and "xo_open_instance", like: > > > https://libxo.readthedocs.io/en/latest/api.html?highlight=xo_open_instance#lists-and-instances > > But with "xo" I don't have lists and instances. I'll need to > find a means of adding these, though that would make "xo" more > cumbersome and it would lack the FSM (and the stack) that catches > missing calls in libxo. > > xo $opts --open-list machine > for name in red green blue; do > xo $opts --open-instance machine > xo $opts "Machine {k:name} has {:memory}\n" $name `get-mem ~name` > xo $opts --close-instance machine > done > xo $opts --close-list machine > > Which is very exact but less that beautiful. This is #61. > > Thanks, > Phil >