From owner-svn-src-head@freebsd.org Thu Nov 7 03:57:07 2019 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 78E781A62D7; Thu, 7 Nov 2019 03:57:07 +0000 (UTC) (envelope-from phil@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 477qNM29gDz43vY; Thu, 7 Nov 2019 03:57:07 +0000 (UTC) (envelope-from phil@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2EA6A7DA5; Thu, 7 Nov 2019 03:57:07 +0000 (UTC) (envelope-from phil@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id xA73v7Pw048847; Thu, 7 Nov 2019 03:57:07 GMT (envelope-from phil@FreeBSD.org) Received: (from phil@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id xA73v41Y048836; Thu, 7 Nov 2019 03:57:04 GMT (envelope-from phil@FreeBSD.org) Message-Id: <201911070357.xA73v41Y048836@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: phil set sender to phil@FreeBSD.org using -f From: Phil Shafer Date: Thu, 7 Nov 2019 03:57:04 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r354427 - in head: contrib/libxo contrib/libxo/doc contrib/libxo/encoder contrib/libxo/encoder/cbor contrib/libxo/encoder/csv contrib/libxo/encoder/test contrib/libxo/libxo contrib/libx... X-SVN-Group: head X-SVN-Commit-Author: phil X-SVN-Commit-Paths: in head: contrib/libxo contrib/libxo/doc contrib/libxo/encoder contrib/libxo/encoder/cbor contrib/libxo/encoder/csv contrib/libxo/encoder/test contrib/libxo/libxo contrib/libxo/tests/core contrib/libx... X-SVN-Commit-Revision: 354427 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Nov 2019 03:57:07 -0000 Author: phil Date: Thu Nov 7 03:57:04 2019 New Revision: 354427 URL: https://svnweb.freebsd.org/changeset/base/354427 Log: Import libxo-1.3.0: - move from "oxtradoc" to RST/Sphinx documentation - new "csv" encoder, which allows path and leaf lists - address warnings from PVS-Stdio tool - add "xolint" detected errors to the documentation Added: head/contrib/libxo/doc/encoders.rst - copied unchanged from r354426, vendor/Juniper/libxo/dist/doc/encoders.rst head/contrib/libxo/doc/top-link.html.in - copied unchanged from r354426, vendor/Juniper/libxo/dist/doc/top-link.html.in head/contrib/libxo/encoder/csv/ - copied from r354426, vendor/Juniper/libxo/dist/encoder/csv/ head/contrib/libxo/tests/core/saved/test_01.Ecsv1.err - copied unchanged from r354426, vendor/Juniper/libxo/dist/tests/core/saved/test_01.Ecsv1.err head/contrib/libxo/tests/core/saved/test_01.Ecsv1.out - copied unchanged from r354426, vendor/Juniper/libxo/dist/tests/core/saved/test_01.Ecsv1.out head/contrib/libxo/tests/core/saved/test_01.Ecsv2.err - copied unchanged from r354426, vendor/Juniper/libxo/dist/tests/core/saved/test_01.Ecsv2.err head/contrib/libxo/tests/core/saved/test_01.Ecsv2.out - copied unchanged from r354426, vendor/Juniper/libxo/dist/tests/core/saved/test_01.Ecsv2.out head/contrib/libxo/tests/core/saved/test_01.Ecsv3.err - copied unchanged from r354426, vendor/Juniper/libxo/dist/tests/core/saved/test_01.Ecsv3.err head/contrib/libxo/tests/core/saved/test_01.Ecsv3.out - copied unchanged from r354426, vendor/Juniper/libxo/dist/tests/core/saved/test_01.Ecsv3.out head/lib/libxo/encoder/ head/lib/libxo/encoder/Makefile (contents, props changed) head/lib/libxo/encoder/csv/ head/lib/libxo/encoder/csv/Makefile (contents, props changed) Deleted: head/contrib/libxo/doc/libxo-manual.html head/contrib/libxo/doc/libxo.txt Modified: head/contrib/libxo/Makefile.am head/contrib/libxo/README.md head/contrib/libxo/configure.ac head/contrib/libxo/doc/Makefile.am head/contrib/libxo/doc/api.rst head/contrib/libxo/doc/conf.py head/contrib/libxo/doc/faq.rst head/contrib/libxo/doc/field-formatting.rst head/contrib/libxo/doc/field-modifiers.rst head/contrib/libxo/doc/field-roles.rst head/contrib/libxo/doc/index.rst head/contrib/libxo/doc/options.rst head/contrib/libxo/doc/xo.rst head/contrib/libxo/doc/xohtml.rst head/contrib/libxo/doc/xolint.rst head/contrib/libxo/doc/xopo.rst head/contrib/libxo/encoder/Makefile.am head/contrib/libxo/encoder/cbor/enc_cbor.c head/contrib/libxo/encoder/test/enc_test.c head/contrib/libxo/libxo/add.man.in head/contrib/libxo/libxo/libxo.c head/contrib/libxo/libxo/xo_buf.h head/contrib/libxo/libxo/xo_encoder.c head/contrib/libxo/libxo/xo_encoder.h head/contrib/libxo/tests/core/Makefile.am head/contrib/libxo/tests/core/saved/test_01.E.out head/contrib/libxo/tests/core/saved/test_01.H.out head/contrib/libxo/tests/core/saved/test_01.HIPx.out head/contrib/libxo/tests/core/saved/test_01.HP.out head/contrib/libxo/tests/core/saved/test_01.J.out head/contrib/libxo/tests/core/saved/test_01.JP.out head/contrib/libxo/tests/core/saved/test_01.T.out head/contrib/libxo/tests/core/saved/test_01.X.out head/contrib/libxo/tests/core/saved/test_01.XP.out head/contrib/libxo/tests/core/saved/test_02.E.out head/contrib/libxo/tests/core/saved/test_03.E.out head/contrib/libxo/tests/core/saved/test_04.E.out head/contrib/libxo/tests/core/saved/test_05.E.out head/contrib/libxo/tests/core/saved/test_06.E.out head/contrib/libxo/tests/core/saved/test_07.E.out head/contrib/libxo/tests/core/saved/test_08.E.out head/contrib/libxo/tests/core/saved/test_09.E.out head/contrib/libxo/tests/core/saved/test_10.E.out head/contrib/libxo/tests/core/saved/test_11.E.out head/contrib/libxo/tests/core/saved/test_12.E.err head/contrib/libxo/tests/core/saved/test_12.E.out head/contrib/libxo/tests/core/saved/test_12.H.err head/contrib/libxo/tests/core/saved/test_12.HIPx.err head/contrib/libxo/tests/core/saved/test_12.HP.err head/contrib/libxo/tests/core/saved/test_12.J.err head/contrib/libxo/tests/core/saved/test_12.JP.err head/contrib/libxo/tests/core/saved/test_12.T.err head/contrib/libxo/tests/core/saved/test_12.X.err head/contrib/libxo/tests/core/saved/test_12.XP.err head/contrib/libxo/tests/core/test_01.c head/contrib/libxo/tests/core/test_02.c head/contrib/libxo/tests/core/test_03.c head/contrib/libxo/tests/core/test_08.c head/contrib/libxo/tests/core/test_09.c head/contrib/libxo/tests/core/test_10.c head/contrib/libxo/tests/core/test_11.c head/contrib/libxo/tests/core/test_12.c head/contrib/libxo/tests/gettext/gt_01.c head/contrib/libxo/tests/gettext/saved/gt_01.H.out head/contrib/libxo/tests/gettext/saved/gt_01.HIPx.out head/contrib/libxo/tests/gettext/saved/gt_01.HP.out head/contrib/libxo/tests/gettext/saved/gt_01.J.out head/contrib/libxo/tests/gettext/saved/gt_01.JP.out head/contrib/libxo/tests/gettext/saved/gt_01.T.out head/contrib/libxo/tests/gettext/saved/gt_01.X.out head/contrib/libxo/tests/gettext/saved/gt_01.XP.out head/contrib/libxo/tests/xo/saved/xo_02.H.err head/contrib/libxo/tests/xo/saved/xo_02.HIPx.err head/contrib/libxo/tests/xo/saved/xo_02.HP.err head/contrib/libxo/tests/xo/saved/xo_02.J.err head/contrib/libxo/tests/xo/saved/xo_02.JP.err head/contrib/libxo/tests/xo/saved/xo_02.T.err head/contrib/libxo/tests/xo/saved/xo_02.X.err head/contrib/libxo/tests/xo/saved/xo_02.XP.err head/contrib/libxo/xo/xo.c head/contrib/libxo/xolint/xolint.pl head/lib/libxo/Makefile head/lib/libxo/add.man head/lib/libxo/xo_config.h head/usr.bin/xohtml/xohtml.sh Directory Properties: head/contrib/libxo/ (props changed) Modified: head/contrib/libxo/Makefile.am ============================================================================== --- head/contrib/libxo/Makefile.am Thu Nov 7 03:47:13 2019 (r354426) +++ head/contrib/libxo/Makefile.am Thu Nov 7 03:57:04 2019 (r354427) @@ -32,7 +32,6 @@ errors: docs: @(cd doc ; ${MAKE} docs) - DIST_FILES_DIR = ~/Dropbox/dist-files/ GH_PAGES_DIR = gh-pages/ GH_PAGES_DIR_VER = gh-pages/${PACKAGE_VERSION} @@ -49,18 +48,20 @@ upload: dist upload-docs upload-xohtml-files @echo "Remember to run:" @echo " gt tag ${PACKAGE_VERSION}" -upload-docs: docs - @echo "Uploading libxo-manual.html ... " - @-[ -d ${GH_PAGES_DIR} ] \ - && echo "Updating manual on gh-pages ..." \ - && mkdir -p ${GH_PAGES_DIR_VER} \ - && cp doc/libxo-manual.html ${GH_PAGES_DIR} \ - && cp doc/libxo-manual.html ${GH_PAGES_DIR_VER} \ +upload-docs: docs upload-html + +upload-html: + @echo "Uploading html ... " + @-[ -d ${GH_PAGES_DIR} -a -d doc/html ] \ + && echo "Updating html on gh-pages ..." \ + && mkdir -p ${GH_PAGES_DIR_VER}/html \ + && cp doc/top-link.html ${GH_PAGES_DIR}/libxo.html \ + && cp -r doc/html/* ${GH_PAGES_DIR_VER}/html/ \ && (cd ${GH_PAGES_DIR} \ - && git add ${PACKAGE_VERSION} \ - && git add libxo-manual.html \ + && git add libxo.html \ + && git add ${PACKAGE_VERSION}/html \ && git commit -m 'new docs' \ - libxo-manual.html ${PACKAGE_VERSION} \ + libxo.html ${PACKAGE_VERSION}/html \ && git push origin gh-pages ) ; true upload-xohtml-files: Modified: head/contrib/libxo/README.md ============================================================================== --- head/contrib/libxo/README.md Thu Nov 7 03:47:13 2019 (r354426) +++ head/contrib/libxo/README.md Thu Nov 7 03:57:04 2019 (r354427) @@ -10,6 +10,9 @@ application calls a function "xo_emit" to product outp described in a format string. A "field descriptor" tells libxo what the field is and what it means. +Imagine a simplified ``wc`` that emits its output fields in a single +xo_emit call: + ``` xo_emit(" {:lines/%7ju/%ju} {:words/%7ju/%ju} " "{:characters/%7ju/%ju}{d:filename/%s}\n", Modified: head/contrib/libxo/configure.ac ============================================================================== --- head/contrib/libxo/configure.ac Thu Nov 7 03:47:13 2019 (r354426) +++ head/contrib/libxo/configure.ac Thu Nov 7 03:57:04 2019 (r354427) @@ -12,7 +12,7 @@ # AC_PREREQ(2.2) -AC_INIT([libxo], [1.0.4], [phil@juniper.net]) +AC_INIT([libxo], [1.3.0], [phil@juniper.net]) AM_INIT_AUTOMAKE([-Wall -Werror foreign -Wno-portability]) # Support silent build rules. Requires at least automake-1.11. @@ -452,6 +452,7 @@ AC_CONFIG_FILES([ libxo/add.man encoder/Makefile encoder/cbor/Makefile + encoder/csv/Makefile encoder/test/Makefile xo/Makefile xolint/Makefile @@ -459,6 +460,7 @@ AC_CONFIG_FILES([ xopo/Makefile packaging/libxo.pc doc/Makefile + doc/top-link.html tests/Makefile tests/core/Makefile tests/gettext/Makefile Modified: head/contrib/libxo/doc/Makefile.am ============================================================================== --- head/contrib/libxo/doc/Makefile.am Thu Nov 7 03:47:13 2019 (r354426) +++ head/contrib/libxo/doc/Makefile.am Thu Nov 7 03:57:04 2019 (r354427) @@ -8,68 +8,22 @@ # using the SOFTWARE, you agree to be bound by the terms of that # LICENSE. -if HAVE_OXTRADOC -OXTRADOC_DIR = ${SLAX_OXTRADOCDIR} -OXTRADOC_PREFIX = ${OXTRADOC_DIR} -OXTRADOC = ${OXTRADOC_DIR}/oxtradoc -SLAXPROC_BINDIR = ${SLAX_BINDIR} +doc docs: xolint.rst html -XML2RFC = ${OXTRADOC_DIR}/xml2rfc.tcl -XML2HTMLDIR = ${OXTRADOC_DIR} -XML2HTMLBIN = ${XML2HTMLDIR}/rfc2629-to-html.slax -SLAXPROC = ${SLAX_BINDIR}/slaxproc +# +# The contents of xolint.rst is generated based on xolint.pl, since we +# really want this to be self-documenting. But readthedocs.org needs this +# data to be _in_ repo. So we generate this file on command only, and +# the developer needs to commit any changes. +# -SLAXPROC_ARGS = \ - -a oxtradoc-dir ${OXTRADOC_DIR} \ - -a oxtradoc-install-dir ${OXTRADOC_DIR} \ - -a anchor-prefix docs +xolint.rst: ${top_srcdir}/xolint/xolint.pl + perl ${top_srcdir}/xolint/xolint.pl -D > ${top_srcdir}/doc/xolint.rst -SLAXPROC_ARGS_INLINE = \ - -a oxtradoc-inline yes +SPHINX = python3 -msphinx -SLAXPROC_ARGS += ${SLAXPROC_ARGS_INLINE} - -XML2HTML = \ - ${SLAXPROC} -g -e -I ${OXTRADOC_DIR} -I . \ - ${SLAXPROC_ARGS} \ - ${XML2HTMLBIN} - -OX_ARGS = -P ${OXTRADOC_PREFIX} -L ${OXTRADOC_PREFIX} -OX_ARGS += -S ${SLAXPROC} -p doc -OX_CMD = ${PERL} ${PERLOPTS} ${OXTRADOC} ${OX_ARGS} -OXTRADOC_CMD = ${OX_CMD} - -OUTPUT = libxo-manual -INPUT = libxo - -EXTRA_DIST = \ - ${INPUT}.txt \ - ${OUTPUT}.html \ - ${OUTPUT}.txt - -doc docs: ${OUTPUT}.txt ${OUTPUT}.html - -${OUTPUT}.txt: ${INPUT}.txt ${OXTRADOC} xolint.txt - ${OXTRADOC_CMD} -m text -o $@ $< - -${OUTPUT}.html: ${INPUT}.txt ${OXTRADOC} ${XML2HTMLBIN} xolint.txt - ${OXTRADOC_CMD} -m html -o $@ $< - -xolint.txt: ${top_srcdir}/xolint/xolint.pl - perl ${top_srcdir}/xolint/xolint.pl -D > xolint.txt - -CLEANFILES = \ -xolint.txt \ -${INPUT}.xml \ -${INPUT}.fxml \ -${OUTPUT}.txt \ -${OUTPUT}.html -else -doc docs: - @${ECHO} "The 'oxtradoc' tool is not installed; see libslax.org" -endif - -SPHINX = python3.4 -msphinx - html sphinx sphinx-html: - ${SPHINX} -M html ${srcdir} . + ${SPHINX} -M html ${srcdir} . -N -E + +singlehtml: + ${SPHINX} -M singlehtml ${srcdir} . -N -E Modified: head/contrib/libxo/doc/api.rst ============================================================================== --- head/contrib/libxo/doc/api.rst Thu Nov 7 03:47:13 2019 (r354426) +++ head/contrib/libxo/doc/api.rst Thu Nov 7 03:57:04 2019 (r354427) @@ -1,4 +1,4 @@ -.. index: API +.. index:: API The libxo API ============= @@ -155,14 +155,14 @@ Output Styles (XO_STYLE\_\*) The libxo functions accept a set of output styles: -=============== ========================= - Flag Description -=============== ========================= - XO_STYLE_TEXT Traditional text output - XO_STYLE_XML XML encoded data - XO_STYLE_JSON JSON encoded data - XO_STYLE_HTML HTML encoded data -=============== ========================= + =============== ========================= + Flag Description + =============== ========================= + XO_STYLE_TEXT Traditional text output + XO_STYLE_XML XML encoded data + XO_STYLE_JSON JSON encoded data + XO_STYLE_HTML HTML encoded data + =============== ========================= The "XML", "JSON", and "HTML" output styles all use the UTF-8 character encoding. "TEXT" using locale-based encoding. @@ -256,26 +256,26 @@ Flags (XOF\_\*) The set of valid flags include: -=================== ========================================= - Flag Description -=================== ========================================= - XOF_CLOSE_FP Close file pointer on `xo_destroy` - XOF_COLOR Enable color and effects in output - XOF_COLOR_ALLOWED Allow color/effect for terminal output - XOF_DTRT Enable "do the right thing" mode - XOF_INFO Display info data attributes (HTML) - XOF_KEYS Emit the key attribute (XML) - XOF_NO_ENV Do not use the :ref:`libxo-options` env var - XOF_NO_HUMANIZE Display humanization (TEXT, HTML) - XOF_PRETTY Make "pretty printed" output - XOF_UNDERSCORES Replaces hyphens with underscores - XOF_UNITS Display units (XML, HMTL) - XOF_WARN Generate warnings for broken calls - XOF_WARN_XML Generate warnings in XML on stdout - XOF_XPATH Emit XPath expressions (HTML) - XOF_COLUMNS Force xo_emit to return columns used - XOF_FLUSH Flush output after each `xo_emit` call -=================== ========================================= + =================== ========================================= + Flag Description + =================== ========================================= + XOF_CLOSE_FP Close file pointer on `xo_destroy` + XOF_COLOR Enable color and effects in output + XOF_COLOR_ALLOWED Allow color/effect for terminal output + XOF_DTRT Enable "do the right thing" mode + XOF_INFO Display info data attributes (HTML) + XOF_KEYS Emit the key attribute (XML) + XOF_NO_ENV Do not use the :ref:`libxo-options` env var + XOF_NO_HUMANIZE Display humanization (TEXT, HTML) + XOF_PRETTY Make "pretty printed" output + XOF_UNDERSCORES Replaces hyphens with underscores + XOF_UNITS Display units (XML, HMTL) + XOF_WARN Generate warnings for broken calls + XOF_WARN_XML Generate warnings in XML on stdout + XOF_XPATH Emit XPath expressions (HTML) + XOF_COLUMNS Force xo_emit to return columns used + XOF_FLUSH Flush output after each `xo_emit` call + =================== ========================================= The `XOF_CLOSE_FP` flag will trigger the call of the *close_func* (provided via `xo_set_writer`) when the handle is destroyed. @@ -300,12 +300,12 @@ regardless of whether warnings are enabled. If the style is `XO_STYLE_HTML`, the following additional flags can be used: -=============== ========================================= - Flag Description -=============== ========================================= - XOF_XPATH Emit "data-xpath" attributes - XOF_INFO Emit additional info fields -=============== ========================================= + =============== ========================================= + Flag Description + =============== ========================================= + XOF_XPATH Emit "data-xpath" attributes + XOF_INFO Emit additional info fields + =============== ========================================= The `XOF_XPATH` flag enables the emission of XPath expressions detailing the hierarchy of XML elements used to encode the data field, if the @@ -317,11 +317,11 @@ output. See :ref:`field-information` for details. If the style is `XO_STYLE_XML`, the following additional flags can be used: -=============== ========================================= - Flag Description -=============== ========================================= - XOF_KEYS Flag "key" fields for XML -=============== ========================================= + =============== ========================================= + Flag Description + =============== ========================================= + XOF_KEYS Flag "key" fields for XML + =============== ========================================= The `XOF_KEYS` flag adds "key" attribute to the XML encoding for field definitions that use the "k" modifier. The key attribute has @@ -1308,52 +1308,52 @@ These values are defined in . The priority value indicates the importance and potential impact of each message: -============= ======================================================= - Priority Description -============= ======================================================= - LOG_EMERG A panic condition, normally broadcast to all users - LOG_ALERT A condition that should be corrected immediately - LOG_CRIT Critical conditions - LOG_ERR Generic errors - LOG_WARNING Warning messages - LOG_NOTICE Non-error conditions that might need special handling - LOG_INFO Informational messages - LOG_DEBUG Developer-oriented messages -============= ======================================================= + ============= ======================================================= + Priority Description + ============= ======================================================= + LOG_EMERG A panic condition, normally broadcast to all users + LOG_ALERT A condition that should be corrected immediately + LOG_CRIT Critical conditions + LOG_ERR Generic errors + LOG_WARNING Warning messages + LOG_NOTICE Non-error conditions that might need special handling + LOG_INFO Informational messages + LOG_DEBUG Developer-oriented messages + ============= ======================================================= The facility value indicates the source of message, in fairly generic terms: -=============== ======================================================= - Facility Description -=============== ======================================================= - LOG_AUTH The authorization system (e.g. :manpage:`login(1)`) - LOG_AUTHPRIV As LOG_AUTH, but logged to a privileged file - LOG_CRON The cron daemon: :manpage:`cron(8)` - LOG_DAEMON System daemons, not otherwise explicitly listed - LOG_FTP The file transfer protocol daemons - LOG_KERN Messages generated by the kernel - LOG_LPR The line printer spooling system - LOG_MAIL The mail system - LOG_NEWS The network news system - LOG_SECURITY Security subsystems, such as :manpage:`ipfw(4)` - LOG_SYSLOG Messages generated internally by :manpage:`syslogd(8)` - LOG_USER Messages generated by user processes (default) - LOG_UUCP The uucp system - LOG_LOCAL0..7 Reserved for local use -=============== ======================================================= + =============== ======================================================= + Facility Description + =============== ======================================================= + LOG_AUTH The authorization system (e.g. :manpage:`login(1)`) + LOG_AUTHPRIV As LOG_AUTH, but logged to a privileged file + LOG_CRON The cron daemon: :manpage:`cron(8)` + LOG_DAEMON System daemons, not otherwise explicitly listed + LOG_FTP The file transfer protocol daemons + LOG_KERN Messages generated by the kernel + LOG_LPR The line printer spooling system + LOG_MAIL The mail system + LOG_NEWS The network news system + LOG_SECURITY Security subsystems, such as :manpage:`ipfw(4)` + LOG_SYSLOG Messages generated internally by :manpage:`syslogd(8)` + LOG_USER Messages generated by user processes (default) + LOG_UUCP The uucp system + LOG_LOCAL0..7 Reserved for local use + =============== ======================================================= In addition to the values listed above, xo_open_log accepts a set of addition flags requesting specific logging behaviors: -============ ==================================================== - Flag Description -============ ==================================================== - LOG_CONS If syslogd fails, attempt to write to /dev/console - LOG_NDELAY Open the connection to :manpage:`syslogd(8)` immediately - LOG_PERROR Write the message also to standard error output - LOG_PID Log the process id with each message -============ ==================================================== + ============ ==================================================== + Flag Description + ============ ==================================================== + LOG_CONS If syslogd fails, attempt to write to /dev/console + LOG_NDELAY Open the connection to :manpage:`syslogd(8)` immediately + LOG_PERROR Write the message also to standard error output + LOG_PID Log the process id with each message + ============ ==================================================== .. index:: xo_syslog @@ -1588,26 +1588,26 @@ processing model of libxo. Content is formatted withi callbacks are made to the encoder's handler function when data is ready to be processed: -======================= ======================================= - Operation Meaning (Base function) -======================= ======================================= - XO_OP_CREATE Called when the handle is created - XO_OP_OPEN_CONTAINER Container opened (xo_open_container) - XO_OP_CLOSE_CONTAINER Container closed (xo_close_container) - XO_OP_OPEN_LIST List opened (xo_open_list) - XO_OP_CLOSE_LIST List closed (xo_close_list) - XO_OP_OPEN_LEAF_LIST Leaf list opened (xo_open_leaf_list) - XO_OP_CLOSE_LEAF_LIST Leaf list closed (xo_close_leaf_list) - XO_OP_OPEN_INSTANCE Instance opened (xo_open_instance) - XO_OP_CLOSE_INSTANCE Instance closed (xo_close_instance) - XO_OP_STRING Field with Quoted UTF-8 string - XO_OP_CONTENT Field with content - XO_OP_FINISH Finish any pending output - XO_OP_FLUSH Flush any buffered output - XO_OP_DESTROY Clean up resources - XO_OP_ATTRIBUTE An attribute name/value pair - XO_OP_VERSION A version string -======================= ======================================= + ======================= ======================================= + Operation Meaning (Base function) + ======================= ======================================= + XO_OP_CREATE Called when the handle is created + XO_OP_OPEN_CONTAINER Container opened (xo_open_container) + XO_OP_CLOSE_CONTAINER Container closed (xo_close_container) + XO_OP_OPEN_LIST List opened (xo_open_list) + XO_OP_CLOSE_LIST List closed (xo_close_list) + XO_OP_OPEN_LEAF_LIST Leaf list opened (xo_open_leaf_list) + XO_OP_CLOSE_LEAF_LIST Leaf list closed (xo_close_leaf_list) + XO_OP_OPEN_INSTANCE Instance opened (xo_open_instance) + XO_OP_CLOSE_INSTANCE Instance closed (xo_close_instance) + XO_OP_STRING Field with Quoted UTF-8 string + XO_OP_CONTENT Field with content + XO_OP_FINISH Finish any pending output + XO_OP_FLUSH Flush any buffered output + XO_OP_DESTROY Clean up resources + XO_OP_ATTRIBUTE An attribute name/value pair + XO_OP_VERSION A version string + ======================= ======================================= For all the open and close operations, the name parameter holds the name of the construct. For string, content, and attribute operations, Modified: head/contrib/libxo/doc/conf.py ============================================================================== --- head/contrib/libxo/doc/conf.py Thu Nov 7 03:47:13 2019 (r354426) +++ head/contrib/libxo/doc/conf.py Thu Nov 7 03:57:04 2019 (r354427) @@ -21,7 +21,15 @@ # import sys # sys.path.insert(0, os.path.abspath('.')) +import subprocess +# +# Instead of hardcoding the version number here, we read it from the +# project's configure script +# +vers_cmd = "grep AC_INIT ../configure.ac | awk '{ print substr($2, 2, length($2) - 3);}'" +version = subprocess.check_output(vers_cmd, shell=True).decode("utf-8") + # -- General configuration ------------------------------------------------ # If your documentation needs a minimal Sphinx version, state it here. @@ -47,7 +55,7 @@ master_doc = 'index' # General information about the project. project = 'libxo' -copyright = '2017, Juniper Networks' +copyright = '2017-2019, Juniper Networks Inc' author = 'Phil Shafer' default_role = 'code' primary_domain = 'c' @@ -58,9 +66,9 @@ smart_quotes = False # built documents. # # The short X.Y version. -version = '0.8.4' +#version = 'develop' # The full version, including alpha/beta/rc tags. -release = '0.8.4' +release = version # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. Copied: head/contrib/libxo/doc/encoders.rst (from r354426, vendor/Juniper/libxo/dist/doc/encoders.rst) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/contrib/libxo/doc/encoders.rst Thu Nov 7 03:57:04 2019 (r354427, copy of r354426, vendor/Juniper/libxo/dist/doc/encoders.rst) @@ -0,0 +1,269 @@ +.. index:: encoder + +Encoders +======== + +This section gives an overview of encoders, details on the encoders +that ship with libxo, and documentation for developers of future +encoders. + +Overview +-------- + +The libxo library contains software to generate four "built-in" +formats: text, XML, JSON, and HTML. These formats are common and +useful, but there are other common and useful formats that users will +want, and including them all in the libxo software would be difficult +and cumbersome. + +To allow support for additional encodings, libxo includes a +"pluggable" extension mechanism for dynamically loading new encoders. +libxo-based applications can automatically use any installed encoder. + +Use the "encoder=XXX" option to access encoders. The following +example uses the "cbor" encoder, saving the output into a file:: + + df --libxo encoder=cbor > df-output.cbor + +Encoders can support specific options that can be accessed by +following the encoder name with a colon (':') and one of more options, +separated by a plus sign "+":: + + df --libxo encoder=csv:path=filesystem+leaf=name+no-header + +This example instructs libxo to load the "csv" encoder and pass the +following options:: + + path=filesystem + leaf=name + no-header + +Each of these option is interpreted by the encoder, and all such +options names and semantics are specific to the particular encoder. +Refer to the intended encoder for documentation on its options. + +.. _csv_encoder: + +CSV - Comma Separated Values +---------------------------- + +libxo ships with a custom encoder for "CSV" files, a common format for +comma separated values. The output of the CSV encoder can be loaded +directly into spreadsheets or similar applications. + +A standard for CSV files is provided in :RFC:`4180`, but since the +format predates that standard by decades, there are many minor +differences in CSV file consumers and their expectations. The CSV +encoder has a number of options to tailor output to those +expectations. + +Consider the following XML:: + + % list-items --libxo xml,pretty + + + + GRO-000-415 + gum + 1412 + 54 + 10 + + + HRD-000-212 + rope + 85 + 4 + 2 + + + HRD-000-517 + ladder + 0 + 2 + 1 + + + + +This output is a list of `instances` (named "item"), each containing a +set of `leafs` ("sku", "name", etc). + +The CSV encoder will emit the leaf values in this output as `fields` +inside a CSV `record`, which is a line containing a set of +comma-separated values:: + + % list-items --libxo encoder=csv + sku,name,sold,in-stock,on-order + GRO-000-415,gum,1412,54,10 + HRD-000-212,rope,85,4,2 + HRD-000-517,ladder,0,2,1 + +Be aware that since the CSV encoder looks for data instances, when +used with :ref:`xo`, the `--instance` option will be needed:: + + % xo --libxo encoder=csv --instance foo 'The {:product} is {:status}\n' stereo "in route" + product,status + stereo,in route + +.. _csv_path: + +The `path` Option +~~~~~~~~~~~~~~~~~ + +By default, the CSV encoder will attempt to emit any list instance +generated by the application. In some cases, this may be +unacceptable, and a specific list may be desired. + +Use the "path" option to limit the processing of output to a specific +hierarchy. The path should be one or more names of containers or +lists. + +For example, if the "list-items" application generates other lists, +the user can give "path=top/data/item" as a path:: + + % list-items --libxo encoder=csv:path=top/data/item + sku,name,sold,in-stock,on-order + GRO-000-415,gum,1412,54,10 + HRD-000-212,rope,85,4,2 + HRD-000-517,ladder,0,2,1 + +Paths are "relative", meaning they need not be a complete set +of names to the list. This means that "path=item" may be sufficient +for the above example. + +.. _csv_leafs: + +The `leafs` Option +~~~~~~~~~~~~~~~~~~ + +The CSV encoding requires that all lines of output have the same +number of fields with the same order. In contrast, XML and JSON allow +any order (though libxo forces key leafs to appear before other +leafs). + +To maintain a consistent set of fields inside the CSV file, the same +set of leafs must be selected from each list item. By default, the +CSV encoder records the set of leafs that appear in the first list +instance it processes, and extract only those leafs from future +instances. If the first instance is missing a leaf that is desired by +the consumer, the "leaf" option can be used to ensure that an empty +value is recorded for instances that lack a particular leaf. + +The "leafs" option can also be used to exclude leafs, limiting the +output to only those leafs provided. + +In addition, the order of the output fields follows the order in which +the leafs are listed. "leafs=one.two" and "leafs=two.one" give +distinct output. + +So the "leafs" option can be used to expand, limit, and order the set +of leafs. + +The value of the leafs option should be one or more leaf names, +separated by a period ("."):: + + % list-items --libxo encoder=csv:leafs=sku.on-order + sku,on-order + GRO-000-415,10 + HRD-000-212,2 + HRD-000-517,1 + % list-items -libxo encoder=csv:leafs=on-order.sku + on-order,sku + 10,GRO-000-415 + 2,HRD-000-212 + 1,HRD-000-517 + +Note that since libxo uses terminology from YANG (:RFC:`7950`), the +data modeling language for NETCONF (:RFC:`6241`), which uses "leafs" +as the plural form of "leaf". libxo follows that convention. + +.. _csv_no_header: + +The `no-header` Option +~~~~~~~~~~~~~~~~~~~~~~ + +CSV files typical begin with a line that defines the fields included +in that file, in an attempt to make the contents self-defining:: + + sku,name,sold,in-stock,on-order + GRO-000-415,gum,1412,54,10 + HRD-000-212,rope,85,4,2 + HRD-000-517,ladder,0,2,1 + +There is no reliable mechanism for determining whether this header +line is included, so the consumer must make an assumption. + +The csv encoder defaults to producing the header line, but the +"no-header" option can be included to avoid the header line. + +.. _csv_no_quotes: + +The `no-quotes` Option +~~~~~~~~~~~~~~~~~~~~~~ + +:RFC:`4180` specifies that fields containing spaces should be quoted, but +many CSV consumers do not handle quotes. The "no-quotes" option +instruct the CSV encoder to avoid the use of quotes. + +.. _csv_dos: + +The `dos` Option +~~~~~~~~~~~~~~~~ + +:RFC:`4180` defines the end-of-line marker as a carriage return +followed by a newline. This `CRLF` convention dates from the distant +past, but its use was anchored in the 1980s by the `DOS` operating +system. + +The CSV encoder defaults to using the standard Unix end-of-line +marker, a simple newline. Use the "dos" option to use the `CRLF` +convention. + +The Encoder API +--------------- + +The encoder API consists of three distinct phases: + +- loading the encoder +- initializing the encoder +- feeding operations to the encoder + +To load the encoder, libxo will open a shared library named: + + ${prefix}/lib/libxo/encoder/${name}.enc + +This file is typically a symbolic link to a dynamic library, suitable +for `dlopen`(). libxo looks for a symbol called +`xo_encoder_library_init` inside that library and calls it with the +arguments defined in the header file "xo_encoder.h". This function +should look as follows:: + + int + xo_encoder_library_init (XO_ENCODER_INIT_ARGS) + { + arg->xei_version = XO_ENCODER_VERSION; + arg->xei_handler = test_handler; + + return 0; + } + +Several features here allow for future compatibility: the macro +XO_ENCODER_INIT_ARGS allows the arguments to this function change over +time, and the XO_ENCODER_VERSION allows the library to tell libxo +which version of the API it was compiled with. + +The function places in xei_handler should be have the signature:: + + static int + test_handler (XO_ENCODER_HANDLER_ARGS) + { + ... + +This function will be called with the "op" codes defined in +"xo_encoder.h". Each op code represents a distinct event in the libxo +processing model. For example OP_OPEN_CONTAINER tells the encoder +that a new container has been opened, and the encoder can behave in an +appropriate manner. + + Modified: head/contrib/libxo/doc/faq.rst ============================================================================== --- head/contrib/libxo/doc/faq.rst Thu Nov 7 03:47:13 2019 (r354426) +++ head/contrib/libxo/doc/faq.rst Thu Nov 7 03:57:04 2019 (r354427) @@ -202,8 +202,7 @@ will lead users to ask the difference between the two there is no difference, use only one of the field names. If there is a difference, change the names to make that difference more obvious. -.. ignore for now, since we want can't have generated content - What does this message mean? - ---------------------------- +What does this message mean? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - !!include-file xolint.txt +.. include:: xolint.rst Modified: head/contrib/libxo/doc/field-formatting.rst ============================================================================== --- head/contrib/libxo/doc/field-formatting.rst Thu Nov 7 03:47:13 2019 (r354426) +++ head/contrib/libxo/doc/field-formatting.rst Thu Nov 7 03:57:04 2019 (r354427) @@ -53,48 +53,48 @@ removed eventually. The format character is described in the following table: -===== ================= ====================== - Ltr Argument Type Format -===== ================= ====================== - d int base 10 (decimal) - i int base 10 (decimal) - o int base 8 (octal) - u unsigned base 10 (decimal) - x unsigned base 16 (hex) - X unsigned long base 16 (hex) - D long base 10 (decimal) - O unsigned long base 8 (octal) - U unsigned long base 10 (decimal) - e double [-]d.ddde+-dd - E double [-]d.dddE+-dd - f double [-]ddd.ddd - F double [-]ddd.ddd - g double as 'e' or 'f' - G double as 'E' or 'F' - a double [-]0xh.hhhp[+-]d - A double [-]0Xh.hhhp[+-]d - c unsigned char a character - C wint_t a character - s char \* a UTF-8 string - S wchar_t \* a unicode/WCS string - p void \* '%#lx' -===== ================= ====================== + ===== ================= ====================== + Ltr Argument Type Format + ===== ================= ====================== + d int base 10 (decimal) + i int base 10 (decimal) + o int base 8 (octal) + u unsigned base 10 (decimal) + x unsigned base 16 (hex) + X unsigned long base 16 (hex) + D long base 10 (decimal) + O unsigned long base 8 (octal) + U unsigned long base 10 (decimal) + e double [-]d.ddde+-dd + E double [-]d.dddE+-dd + f double [-]ddd.ddd + F double [-]ddd.ddd + g double as 'e' or 'f' + G double as 'E' or 'F' + a double [-]0xh.hhhp[+-]d + A double [-]0Xh.hhhp[+-]d + c unsigned char a character + C wint_t a character + s char \* a UTF-8 string + S wchar_t \* a unicode/WCS string + p void \* '%#lx' + ===== ================= ====================== The 'h' and 'l' modifiers affect the size and treatment of the argument: -===== ============= ==================== - Mod d, i o, u, x, X -===== ============= ==================== - hh signed char unsigned char - h short unsigned short - l long unsigned long - ll long long unsigned long long - j intmax_t uintmax_t - t ptrdiff_t ptrdiff_t - z size_t size_t - q quad_t u_quad_t -===== ============= ==================== + ===== ============= ==================== + Mod d, i o, u, x, X + ===== ============= ==================== + hh signed char unsigned char + h short unsigned short + l long unsigned long + ll long long unsigned long long + j intmax_t uintmax_t + t ptrdiff_t ptrdiff_t + z size_t size_t + q quad_t u_quad_t + ===== ============= ==================== .. index:: UTF-8 .. index:: Locale @@ -122,14 +122,14 @@ style:: xo_emit("All strings are utf-8 content {:tag/%ls}", L"except for wide strings"); -======== ================== =============================== - Format Argument Type Argument Contents -======== ================== =============================== - %s const char \* UTF-8 string - %S const char \* UTF-8 string (alias for '%ls') - %ls const wchar_t \* Wide character UNICODE string - %hs const char * locale-based string -======== ================== =============================== + ======== ================== =============================== + Format Argument Type Argument Contents + ======== ================== =============================== + %s const char \* UTF-8 string + %S const char \* UTF-8 string (alias for '%ls') + %ls const wchar_t \* Wide character UNICODE string + %hs const char * locale-based string + ======== ================== =============================== .. admonition:: "Long", not "locale" @@ -266,21 +266,21 @@ incompatible with printf-like testing: If none of these features are in use by your code, then using the "_p" variants might be wise: -================== ======================== - Function printf-like Equivalent -================== ======================== - xo_emit_hv xo_emit_hvp - xo_emit_h xo_emit_hp - xo_emit xo_emit_p - xo_emit_warn_hcv xo_emit_warn_hcvp - xo_emit_warn_hc xo_emit_warn_hcp - xo_emit_warn_c xo_emit_warn_cp - xo_emit_warn xo_emit_warn_p - xo_emit_warnx xo_emit_warnx_p - xo_emit_err xo_emit_err_p - xo_emit_errx xo_emit_errx_p - xo_emit_errc xo_emit_errc_p -================== ======================== + ================== ======================== + Function printf-like Equivalent + ================== ======================== + xo_emit_hv xo_emit_hvp + xo_emit_h xo_emit_hp + xo_emit xo_emit_p + xo_emit_warn_hcv xo_emit_warn_hcvp + xo_emit_warn_hc xo_emit_warn_hcp + xo_emit_warn_c xo_emit_warn_cp + xo_emit_warn xo_emit_warn_p + xo_emit_warnx xo_emit_warnx_p + xo_emit_err xo_emit_err_p + xo_emit_errx xo_emit_errx_p + xo_emit_errc xo_emit_errc_p + ================== ======================== .. index:: performance .. index:: XOEF_RETAIN @@ -305,16 +305,16 @@ xo_emit_f() function. A complete set of xo_emit_f fun match all the xo_emit function signatures (with handles, varadic argument, and printf-like flags): -================== ======================== - Function Flags Equivalent -================== ======================== - xo_emit_hv xo_emit_hvf - xo_emit_h xo_emit_hf - xo_emit xo_emit_f - xo_emit_hvp xo_emit_hvfp - xo_emit_hp xo_emit_hfp - xo_emit_p xo_emit_fp -================== ======================== + ================== ======================== + Function Flags Equivalent + ================== ======================== + xo_emit_hv xo_emit_hvf + xo_emit_h xo_emit_hf + xo_emit xo_emit_f + xo_emit_hvp xo_emit_hvfp + xo_emit_hp xo_emit_hfp + xo_emit_p xo_emit_fp + ================== ======================== The format string must be immutable across multiple calls to xo_emit_f(), since the library retains the string. Typically this is done by using Modified: head/contrib/libxo/doc/field-modifiers.rst ============================================================================== --- head/contrib/libxo/doc/field-modifiers.rst Thu Nov 7 03:47:13 2019 (r354426) +++ head/contrib/libxo/doc/field-modifiers.rst Thu Nov 7 03:57:04 2019 (r354427) @@ -8,26 +8,26 @@ Field Modifiers Field modifiers are flags which modify the way content emitted for particular output styles: -=== =============== =================================================== - M Name Description -=== =============== =================================================== - a argument The content appears as a 'const char \*' argument - c colon A colon (":") is appended after the label - d display Only emit field for display styles (text/HTML) - e encoding Only emit for encoding styles (XML/JSON) - g gettext Call gettext on field's render content - h humanize (hn) Format large numbers in human-readable style -\ hn-space Humanize: Place space between numeric and unit -\ hn-decimal Humanize: Add a decimal digit, if number < 10 -\ hn-1000 Humanize: Use 1000 as divisor instead of 1024 - k key Field is a key, suitable for XPath predicates - l leaf-list Field is a leaf-list - n no-quotes Do not quote the field when using JSON style - p plural Gettext: Use comma-separated plural form - q quotes Quote the field when using JSON style - t trim Trim leading and trailing whitespace - w white A blank (" ") is appended after the label -=== =============== =================================================== + === =============== =================================================== + M Name Description + === =============== =================================================== + a argument The content appears as a 'const char \*' argument + c colon A colon (":") is appended after the label + d display Only emit field for display styles (text/HTML) + e encoding Only emit for encoding styles (XML/JSON) + g gettext Call gettext on field's render content + h humanize (hn) Format large numbers in human-readable style + \ hn-space Humanize: Place space between numeric and unit + \ hn-decimal Humanize: Add a decimal digit, if number < 10 + \ hn-1000 Humanize: Use 1000 as divisor instead of 1024 + k key Field is a key, suitable for XPath predicates + l leaf-list Field is a leaf-list + n no-quotes Do not quote the field when using JSON style + p plural Gettext: Use comma-separated plural form + q quotes Quote the field when using JSON style + t trim Trim leading and trailing whitespace + w white A blank (" ") is appended after the label + === =============== =================================================== Roles and modifiers can also use more verbose names, when preceded by a comma. For example, the modifier string "Lwc" (or "L,white,colon") Modified: head/contrib/libxo/doc/field-roles.rst ============================================================================== --- head/contrib/libxo/doc/field-roles.rst Thu Nov 7 03:47:13 2019 (r354426) +++ head/contrib/libxo/doc/field-roles.rst Thu Nov 7 03:57:04 2019 (r354427) @@ -8,24 +8,26 @@ Field Roles Field roles are optional, and indicate the role and formatting of the content. The roles are listed below; only one role is permitted: -=== ============== ================================================= -R Name Description -=== ============== ================================================= -C color Field has color and effect controls -D decoration Field is non-text (e.g., colon, comma) -E error Field is an error message -G gettext Call gettext(3) on the format string -L label Field is text that prefixes a value -N note Field is text that follows a value -P padding Field is spaces needed for vertical alignment -T title Field is a title value for headings -U units Field is the units for the previous value field -V value Field is the name of field (the default) -W warning Field is a warning message -[ start-anchor Begin a section of anchored variable-width text -] stop-anchor End a section of anchored variable-width text -=== ============== ================================================= + === ============== ================================================= + R Name Description + === ============== ================================================= + C color Field has color and effect controls + D decoration Field is non-text (e.g., colon, comma) + E error Field is an error message + G gettext Call gettext(3) on the format string + L label Field is text that prefixes a value + N note Field is text that follows a value + P padding Field is spaces needed for vertical alignment *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***