From owner-freebsd-hackers@freebsd.org Tue Sep 11 01:28:42 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 679EC10A2D37 for ; Tue, 11 Sep 2018 01:28:42 +0000 (UTC) (envelope-from phil@juniper.net) Received: from mx0a-00273201.pphosted.com (mx0a-00273201.pphosted.com [208.84.65.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.pphosted.com", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BF7507E48D for ; Tue, 11 Sep 2018 01:28:41 +0000 (UTC) (envelope-from phil@juniper.net) Received: from pps.filterd (m0108156.ppops.net [127.0.0.1]) by mx0a-00273201.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w8B1Nd1M004840; Mon, 10 Sep 2018 18:28:40 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juniper.net; h=message-id : from : to : cc : subject : in-reply-to : mime-version : content-type : content-id : content-transfer-encoding : date; s=PPS1017; bh=NmS+fYD93RhHwo9U8fGEQBVz8Efi/8ysjfXCx11brdg=; b=gkUsFUK4bIdcgWLlmfaZ7BtwWfUB4LAIAuGTUT+M+xT07t/UxxuPGPbo9UHuNGw8Us0f Ai+6QR+t+S/X1Tt4qP/kROo33R7IU8cw9rJszArdqLDgnX57WUCeFzOVEIa+EuQQasDo 4y9k/oPJQTKuuh6DfRIv5DGxSuXJtDJMr16sqkxL7TJ4+ltLwHD7vUkRqYuzhNSRrlm8 GDx4go+bDG703BNjECeN8Gd/Jibyp5Qb6qk6yQ78XTMAOrXO01nn0GJRp3PjC0Yo7Ls3 +UpWuXO0uhut45uPBRu+gCwDR25zE7HhY9jcVGAwrSG+bCD1Zq9s7DaGYQ57dPBG1jRY Ew== Received: from nam01-by2-obe.outbound.protection.outlook.com (mail-by2nam01lp0180.outbound.protection.outlook.com [216.32.181.180]) by mx0a-00273201.pphosted.com with ESMTP id 2mdssms3ud-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Mon, 10 Sep 2018 18:28:40 -0700 Received: from BN3PR05CA0010.namprd05.prod.outlook.com (2603:10b6:400::20) by MWHPR05MB3119.namprd05.prod.outlook.com (2603:10b6:300:b2::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1143.9; Tue, 11 Sep 2018 01:28:38 +0000 Received: from DM3NAM05FT013.eop-nam05.prod.protection.outlook.com (2a01:111:f400:7e51::204) by BN3PR05CA0010.outlook.office365.com (2603:10b6:400::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1143.6 via Frontend Transport; Tue, 11 Sep 2018 01:28:37 +0000 Received-SPF: SoftFail (protection.outlook.com: domain of transitioning juniper.net discourages use of 66.129.239.13 as permitted sender) Received: from P-EXFEND-EQX-02.jnpr.net (66.129.239.13) by DM3NAM05FT013.mail.protection.outlook.com (10.152.98.122) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.20.1164.5 via Frontend Transport; Tue, 11 Sep 2018 01:28:37 +0000 Received: from P-EXFEND-EQX-02.jnpr.net (10.104.8.55) by P-EXFEND-EQX-02.jnpr.net (10.104.8.55) with Microsoft SMTP Server (TLS) id 15.0.847.32; Mon, 10 Sep 2018 18:28:24 -0700 Received: from P-EMFE01C-SAC.jnpr.net (172.24.192.43) by P-EXFEND-EQX-02.jnpr.net (10.104.8.55) with Microsoft SMTP Server (TLS) id 15.0.847.32 via Frontend Transport; Mon, 10 Sep 2018 18:28:24 -0700 Received: from p-mailhub01.juniper.net (10.47.226.20) by P-EMFE01C-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.123.3; Mon, 10 Sep 2018 18:28:24 -0700 Received: from idle.juniper.net (idleski.juniper.net [172.25.4.26]) by p-mailhub01.juniper.net (8.14.4/8.11.3) with ESMTP id w8B1SMSq027487; Mon, 10 Sep 2018 18:28:23 -0700 (envelope-from phil@juniper.net) Received: from idle.juniper.net (localhost [127.0.0.1]) by idle.juniper.net (8.15.2/8.15.2) with ESMTP id w8B1TG1Z022255; Mon, 10 Sep 2018 21:29:16 -0400 (EDT) (envelope-from phil@juniper.net) Message-ID: <201809110129.w8B1TG1Z022255@idle.juniper.net> From: Phil Shafer To: Matt Churchyard CC: , Simon Gerraty Subject: Re: Getting valid JSON output with xo(1) In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <22253.1536629356.1@idle.juniper.net> Content-Transfer-Encoding: quoted-printable Date: Mon, 10 Sep 2018 21:29:16 -0400 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:66.129.239.13; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10019020)(136003)(396003)(376002)(346002)(39860400002)(2980300002)(199004)(189003)(69596002)(8746002)(2810700001)(1411001)(336012)(4326008)(26005)(5660300001)(76506005)(77096007)(81156014)(53416004)(126002)(316002)(186003)(107886003)(54906003)(14444005)(6246003)(8936002)(6346003)(229853002)(6306002)(46406003)(39060400002)(68736007)(476003)(81166006)(7126003)(11346002)(2906002)(7696005)(486006)(106466001)(426003)(105596002)(53936002)(86362001)(50466002)(8276002)(23726003)(478600001)(1076002)(47776003)(97736004)(305945005)(97756001)(356003)(966005)(8676002)(6916009); DIR:OUT; SFP:1102; SCL:1; SRVR:MWHPR05MB3119; H:P-EXFEND-EQX-02.jnpr.net; FPR:; SPF:SoftFail; LANG:en; PTR:InfoDomainNonexistent; A:1; MX:1; X-Microsoft-Exchange-Diagnostics: 1; DM3NAM05FT013; 1:yQFDFoigjQEeh4MT8T1rTXy9Uhnyh6zB7kndgMJJaOX6eLRcNV8MmlhLEPc/N03njuDYW0SjZX7PaYL4qk8cLvkDt3LAp1n1pvvzo8ix2myQOHB7m1cWRPrf5dykpdKY X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2f7c3db4-e962-4ad0-6668-08d61785e60e X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(4534165)(4627221)(201703031133081)(201702281549075)(5600074)(711020)(2017052603328)(7153060); SRVR:MWHPR05MB3119; X-Microsoft-Exchange-Diagnostics: 1; MWHPR05MB3119; 3:0pk6rS0Td4HCl8C6W90TnAyNrXUzWO9vLlXUDwMvJ/Pz3LU5zpMmAXDbwu/9F1wmpy8Hrxx+KbPBu1W1ETPkPgYVU51gfc7SRlWaLz0HiDicfb+yHGnHgC0ujrlYcfhxy8/w2MMBd7tA3tPOUPVmA0S2vFFaSS2qKXeoHyhtbL3u5GLXDYLb+flN7ZvHTF/n/EJUl59K7pcxOu/CmrC7iQgK3VDNUFz8sussHLdwcJz6W9P1g5e3+OkKlAr82tEkEcwuqhCK2sNdwKd+/bYjo80bZi6g5fCpw4yPghNI3Q0yv9rGAt+ckMgK7bgRoGRQ4iA6s9/be2BQSGEBQZ7KwkCfHBd5UTuAgeouP4Rj/RQ=; 25:G39y5JNIuINDrksc+64pQKXlTLNFpFIKAbZp1MpooFClVVMoSsZFuvZZXR9ZVhjUekhk94lpVAsalTA5qjdVeKO4IduMp2MqEO+irtp9CCpY1/YcrTYwnPsO4tla/YzoHCGZDqwxoagXppFw2RLBx2XleA0fT3LcIRF1PaDG1+PZiJflCG5RMb60k8ArFSdoQ1iEUhuvM3n6Z9VwQ3xM0bggEuBWNJO+rZcOGfZEAV0AhgHwD5H2muv01fhUGD/Ms+QEm8xeRDERAvvwQB63Ia1WXlZ0+hDc72KnwJeGhtMR5kmi2JFfxDWubC3Q7n7f9mqfnxryhWEvfwtJwCzZeg== X-MS-TrafficTypeDiagnostic: MWHPR05MB3119: X-Microsoft-Exchange-Diagnostics: 1; MWHPR05MB3119; 31:C/baea9U6Wba0XukUkAKLA6YkGS5Aga2qqadEwsgFosmBg+fLcUsUfw4YNxihZPT3KDAQSzHuMuOcSRUm8Euzr1D/ymc3cV86RBv6ojnLjoQgfy4u1dWvBN1N9h79NHvjQ/2dYjUmj36UTet6yT3yaOlGapaSfWxFHgxzEcFz6v44Iaw6ePbovLEO6XDlCC1ovXImMZbR9Wpo5w2VLgwdqy/EmW66uiPaPbb7HcW5aM=; 20:Af+8mzs40HtJSqvsMgOAiourwq2/pzmMzZPekeJxPNbA6wKOaMRz08br6nzBqb80qeI0dX8qvTkZYD15d9Utq7+drnCJggbTNjG31Ley+Ehf4ZJmnvr4M28XiegF44BY5/UGNPzBilyD1FpTecYh322C4e7Ad2HMcidVnYRo7tYX6udqnqdYslL4DQXX9JvwqckRnp47VGudjhoMkeJmV61GU8hN8/yI3yJs16hxQwKzLhCtt3ffFi4wJeMuHH0Yrxu9vZBG7Lc5oNvbujExqsZysShDeqDsMuYt3y/8ZEij07hfMhftCmJ5bELEDfOMzBVLouh5znqIfRMGpMysxlJ8+EWGE8O14jDJuQu2/v8nMPbcoFHmSb6WRFeWgc0iUrFV5pbnuiQETa59H7s/68p59CkUfHeIWxrSL/06eFLM2+ZI1GYayd0MGB3xEWpMC28W5GNDCZguJShJ1v49G748T7r9aozptwgTke0cj90rzF81uIsPlH95/9SqCacS X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(166708455590820)(17755550239193); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3002001)(93006095)(93003095)(10201501046)(3231344)(944501410)(52105095)(6055026)(149027)(150027)(6041310)(20161123560045)(20161123558120)(20161123562045)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(201708071742011)(7699050)(76991033); SRVR:MWHPR05MB3119; BCL:0; PCL:0; RULEID:; SRVR:MWHPR05MB3119; X-Microsoft-Exchange-Diagnostics: 1; MWHPR05MB3119; 4:rQI1x2uPJor17m0ARI8h99tmsG8OSjO2yKuuAQINXwHrI1XbeutyV4Sa10ocCDIvMxiMF3btEq56yEqxCPSB7zK0Nx0ERXsUlOEikJg9r7iSE8A9iQsbsCb7z3Na4j1K8LNnPPQ2pr2UpPSYoTCdjboA0kVmEK7fHplHbXCWqyJkB7vXIc3vu9r8CtPbDAFw81AT6xb2Hdk8BWDWZykj+nmZ19tVlin0RWAy9tIl2GrXJD8Yv0rfcDyCDsXktsewSHzteJzekQ/xsLhqI1h/+ROYhyZaT//H9uD8uzJ7+dPFWI36U724ojd/RGD/t8C24NYelBr9fa8BmvdhqhbYQXsuWgcA1D0N8H0JvcDhDQw= X-Forefront-PRVS: 0792DBEAD0 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; MWHPR05MB3119; 23:DUbi5+uo6HLpMgZtbtVRm/aqV6cYjIogUpUov1pL6?= =?us-ascii?Q?8gTPYz4qJGZRID7LSGmc5O7Vd3ybS0Ok3AWHPSEa2RONwXSdKpUjqEIaHykt?= =?us-ascii?Q?Ql3MJ+eJCtQFSM+JBU69Ap0Etov7VWWIpvTSTN3nVcrH/H6ghI3LCTawSRqn?= =?us-ascii?Q?7P7LuHMaqWmyNCGt1cxwe7mgBAo3vH12o3mc7PWaL8+KG7qhI/isYuXfQw40?= =?us-ascii?Q?fzLC+mMAd0J4Gu64QwyNLaCb6GcokK/d6ONIYBMYQgwiPokvb8nrbkwe3Mww?= =?us-ascii?Q?/uCUf8BcICYa1Jr+uEG5gv7jBu0mLqEo8xQAsBfND76T+Jim+cGU6yaT5iqp?= =?us-ascii?Q?99WYXWgoACfbRCW+PS4MPg7B5WlivbSoe84nknRLJZsQbhK6ywC37M8YDQaQ?= =?us-ascii?Q?64nDZWVV2cP0hgwX7E6EkL6GH/RfAifJDhc3MMYZwra09MMA/9iyWXeue2MA?= =?us-ascii?Q?d7qwqTug4ursvBoWehPK31ZNmR9BkT7sx40n3nR1C0dKmIdqXvR25dgbupRx?= =?us-ascii?Q?c02xQyslWg03MTliM7DOJJgvSG1qfaHHINKYLFXFcZUtmay3xMvryGttwJ4F?= =?us-ascii?Q?ZT9FK6awlJyv0V+/tWEmhLVYtlRGbYalZY3cRkr9lT2QkTUhr5ki4rtm55m8?= =?us-ascii?Q?VPB23KP4GfL8S0r7P/dQl/B0ljPbTxO17CYPA7wPmxxIsCd+geM4G5H5zdPE?= =?us-ascii?Q?gnZazHpHssyIBcXr0soR//MGNYaEyGm43LGAlEsD791DXu8wsvmcAn0e60aT?= =?us-ascii?Q?RfQUR5EFs+Ttn+r8387l8FqeXn5ZtrSJJVeomTsvdRiNY1D5ObfG0wLDu6ty?= =?us-ascii?Q?KoAA66VcRLaKp9/UCzXoZd4891Lz1KARsH8YUp/zFncJHKWOboxINOMDkUG5?= =?us-ascii?Q?BDE5hx8eexhZIr57lZxg+RiEic7/e0+AEOJIuqI/I2BSxYusx8EgzULROYit?= =?us-ascii?Q?8/EqWXy9J1Uue+5j4YiVVya/6lWCAtVx4SbOMcc+Yp3x5ewWBgprTyn0pWfQ?= =?us-ascii?Q?yIYK0kKMZwDq8xK8FFcKjuJDeY4I+EypoyOx+CZaJ1W1UUlzQ3ER/Nz8fi5U?= =?us-ascii?Q?PXadK5s2RdTtFXfu8/dTHod9R2W08yBf52sRdj4tFZ2+6lQledMFqqnUjZaK?= =?us-ascii?Q?03TqGNQnnlCgMM1S5OOMHo20EvGcRg6iM3arZguiu8J1rbzENR4WjnEfPfAv?= =?us-ascii?Q?EQMbG82V7KP+TpaQEwhUNZrV5h1t1WrgWUmBfYcPezFzoLng4oHXbJvseLNH?= =?us-ascii?Q?iGtQdTmf+Mj3ywr8uIvS9vyMTwpwOoUuKHZ6Y90qw59hRe8014q9kd0nBG2Q?= =?us-ascii?B?UT09?= X-Microsoft-Antispam-Message-Info: Bf0+57SmLX4PGlBZNuZQWaTWblYpjjhrShsJ6QKIt7sApxdZUdZ77KcUxW2wCUmK1NTFxn2QR0sWTsZNPi6KoaCtxGKTPc/+Rt1Z1UUrp2zsk9VB5fwdShHwMVVEf94wLMQZ+5OMGFf50V3Hz8B2Q/Jg1PKq/y2KNWuHVy4/yArgPAAhg5OghDD1+3E1imaZceVykd1FXplb+oFD9AMsVchsyuxEwvW5VzJKaMqBDtacdnvX88s4qEja5qG4S4PZPh9w+wnJbfgE3g9o2e88m61d2wosIhUHlbRlKTFg89GYLZqIxGkCBnw+Rd11Y1UO50QH3yNSYf0IdbY8MdDeyYo2jKJwVR0roTSk6lxLhYo= X-Microsoft-Exchange-Diagnostics: 1; MWHPR05MB3119; 6:rvv1ScwQ3V7JzDtToAaV/uSk6JRSSRDe6GV+IfpiU4xclIRTYclG6ZMP8yl99ArSmycbgHO8WUjMbQGwcRmIP1S8ujA4tr00AQVrEbKjGAyCgpkwJVqM/flOYYzaZdmdluh0oyT+T/QD0zhtY1ZKlETo5PoO1bP3niVagw7JZvCar1JXWR4xdH3bib4QYMyPBRBSzNFYRYP6pumzNzrG/PKyyIq61uTzXntrqejdcQ4mGdONZbUy7t2ogy4T9Ml4PWj371/BdMOgLi/dS5OoIxwoHtT7HPKRp6QERVh4Z2hktq1b+Bnokj13a3/oA/WZxcWcGoSeMLrUaCxLnhx5cwHlHRhSW8jWQ23ioPvWie6IB1IAs7baeujH8ALgBES3X1tSG8oQM8zRDn+L7bqI3OFBPkAj7uP8W6BfiCtvzedKzTlvQOarXndiy6kwVa+SFVBbmzJg3L2M/JgXluMKNA==; 5:/OfXhyN/BX4Jzg4avLnRCSn8pPPEMoG4gBG3vLiZI1GnWwO82QW1UylDp6YxR8qzzmC9U4McFGLHECnwKWt7Hrwm8fEH8lni6QQV+1+gOPJjrnkyPOyzslQos5GN6yAfmIf9z5p4LqQjC3PrCV/btNUMHSrK1maZbRFCuT1+GS0=; 7:/4yhtg0qiYhmGq4Aa+rjdAQj+jW2H/8G+RV80OKZf41M6OrjbjMhg8a3kPrFvmCzyv2Ltiy5lyZEheRVBF/hZuPUQcROQ6IQFUZMeJgH/D3jOZEv3NwYQPQvg3JSkEUTBNr0LiV6yzf02IB9Cvb7u1KnWlBWTjedbErpyse6+55+S+PwfrUwddHUJ8VsxEKcs7bqABF76TcP1JT0xBSZJa7DmeBQp/85nC8HiTWhWrwRwe/O9D2SfSaVnoIQnv3G SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: juniper.net X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Sep 2018 01:28:37.6046 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2f7c3db4-e962-4ad0-6668-08d61785e60e X-MS-Exchange-CrossTenant-Id: bea78b3c-4cdb-4130-854a-1d193232e5f4 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=bea78b3c-4cdb-4130-854a-1d193232e5f4; Ip=[66.129.239.13]; Helo=[P-EXFEND-EQX-02.jnpr.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR05MB3119 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-09-10_12:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_spam_notspam policy=outbound_spam score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1809110014 X-Mailman-Approved-At: Tue, 11 Sep 2018 01:45:13 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2018 01:28:42 -0000 Matt Churchyard writes: >Sorry if this is the wrong list, I wasn't sure which was the most >appropriate. Best path for xo-related bugs is https://github.com/Juniper/libxo/issues Email directly to phil@ is also reasonable. >If I choose HTML output I get the following invalid output which is missi= ng >closing tags. > >
>
test1
>
,
>
2GB
>
>
test2
>
,
>
4GB
> >This can be fixed by adding "\n" to the end of the format lines, which ma= y >be required, but it seems strange that I can so easily generate broken >output. Yes, without a trailing newline, libxo can't declare the output line "over", so it doesn't make the
. The problem is that there's no mechanism to "xo" that your output line is continued from the previous one, and since "xo" is stateless, it will happily start the next chunk of output with a new open div. I'll add a "--continuation" option. (#58) >"bhyve": { > "machines": { > "machine": { > "name": "test1", > "memory": "2GB" > } "machine": { > "name": "test2", > "memory": "4GB" > } > } >} There's an "xo" option called "--not-first" for exactly these situations; place that option on the second invocation so it will know to emit the leading comma. This should be documented (#59). >1) The entire document needs to be inside braces "{}" so that "bhyve" is = a >member of an object. "xo" doesn't currently do this, since it assume you're generating JSON piecemeal and have a better idea that it of what you want to do with the output, but I've opened issue #60 for adding a "--object-wrapper" option that will perform this. >2) There's a missing comma between the two machine entries The "--not-first" option will repair this. >3) "machines" is an object, which erroneously contains 2 "machine" keys This is a bug. "xo" lacks a means of knowing that an object is a list. One might expect that labeling "name as a key would perform this, but it doesn't, and it's not clear that this is enough information. Since "xo" is stateless, it cannot really handle that "--wrapper machine" applies only to the first item. It would have to know not to generate the closing "}" for machine, but it's stateless, so I'd need a means of telling it that. In C code, we have "xo_open_list" and "xo_open_instance", like: https://libxo.readthedocs.io/en/latest/api.html?highlight=3Dxo_open_instan= ce#lists-and-instances But with "xo" I don't have lists and instances. I'll need to find a means of adding these, though that would make "xo" more cumbersome and it would lack the FSM (and the stack) that catches missing calls in libxo. xo $opts --open-list machine for name in red green blue; do xo $opts --open-instance machine xo $opts "Machine {k:name} has {:memory}\n" $name `get-mem ~name` xo $opts --close-instance machine done xo $opts --close-list machine Which is very exact but less that beautiful. This is #61. Thanks, Phil