From owner-freebsd-arch@FreeBSD.ORG Tue Dec 23 22:23:08 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8AF80195 for ; Tue, 23 Dec 2014 22:23:08 +0000 (UTC) Received: from mail-wg0-x22e.google.com (mail-wg0-x22e.google.com [IPv6:2a00:1450:400c:c00::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1EE731B1A for ; Tue, 23 Dec 2014 22:23:08 +0000 (UTC) Received: by mail-wg0-f46.google.com with SMTP id x13so10162493wgg.33 for ; Tue, 23 Dec 2014 14:23:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=WdPoatZwbC3erG4MRdM6cx4dU65rznXDuJ6vayorg6I=; b=ncfRhqoDLOfBfJFunhC3WGF0WyxRD6e6dQS1ho7TYzrhITyNwIeSICHHhtV+OMO3IG mL6zZC69BO6svVJ4lp+g0fW9ipJwIfcOniBOSYqhUdvqHaUDSZt4zqEQ8w6Xxe5fM81v 6UATHrxfj/S3qF96431TZNXCZhUxDbgFOzdP/iuj5mOgkBzTPnrh0zcmIVKUvVEUXyW/ 50PYmqbonMLT1j+GU6xIjA6I9RpcGl5fQB6kipcok1OxRLOGe7gFwMX6noZUyWKllw0x rORPc72N5XCWuaKbXS5LMOZGWp/toPqoy1FrfuDq/L/1mVYjmS0TTUdIjVaDukvYFR0t 8EBg== MIME-Version: 1.0 X-Received: by 10.180.20.6 with SMTP id j6mr44495925wie.59.1419373386428; Tue, 23 Dec 2014 14:23:06 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.216.106.195 with HTTP; Tue, 23 Dec 2014 14:23:06 -0800 (PST) Date: Tue, 23 Dec 2014 14:23:06 -0800 X-Google-Sender-Auth: ESlh3BDZeqGKmV0fN4Xqri8BCIY Message-ID: Subject: libxo in /usr/lib, but binaries in /bin //sbin need it? From: Adrian Chadd To: "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Dec 2014 22:23:08 -0000 Hi all, I just noticed that some stuff in /bin (eg /bin/df) requires stuff now in /usr/lib (ie, libxo). Is this intentional? Should we move libxo to /lib? -adrian From owner-freebsd-arch@FreeBSD.ORG Wed Dec 24 06:28:45 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8328B4C2; Wed, 24 Dec 2014 06:28:45 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id 6C84F1D41; Wed, 24 Dec 2014 06:28:45 +0000 (UTC) Received: from AlfredMacbookAir.local (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id 7B3F1341F859; Tue, 23 Dec 2014 22:28:39 -0800 (PST) Message-ID: <549A5D1E.7020700@freebsd.org> Date: Tue, 23 Dec 2014 22:28:46 -0800 From: Alfred Perlstein Organization: FreeBSD User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Phil Shafer Subject: Re: XML Output: libxo - provide single API to output TXT, XML, JSON and HTML References: <201410210217.s9L2HECn097421@idle.juniper.net> In-Reply-To: <201410210217.s9L2HECn097421@idle.juniper.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Marcel Moolenaar , John-Mark Gurney , "Simon J. Gerraty" , arch@freebsd.org, Poul-Henning Kamp , freebsd-arch , Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Dec 2014 06:28:45 -0000 Folks, I see libxo has landed in FreeBSD, but it appears that the userland mods to the utilities never made it in? Is my assessment correct? /usr/src/usr.bin/netstat % find /usr/src/ | xargs grep LIBXO | grep -v /usr/src/contrib/libxo | grep -v /usr/src/lib/libxo /usr/src/usr.bin/xo/Makefile:LIBXO= ${.CURDIR:H:H}/contrib/libxo /usr/src/usr.bin/xo/Makefile:.PATH: ${LIBXO}/xo /usr/src/usr.bin/xo/Makefile:CFLAGS+=-I${LIBXO}/libxo The patches against netstat are stale and were never applied. I have not tried other utilities. /usr/src/usr.bin/netstat % wc -l *.rej 547 if.c.rej 549 inet.c.rej 236 inet6.c.rej 96 main.c.rej 68 mroute.c.rej 39 netgraph.c.rej 415 route.c.rej 24 unix.c.rej 1974 total /usr/src/usr.bin/netstat % ls -l *.rej -rw-r--r-- 1 alfred wheel 17486 Dec 23 22:15 if.c.rej -rw-r--r-- 1 alfred wheel 22687 Dec 23 22:15 inet.c.rej -rw-r--r-- 1 alfred wheel 11111 Dec 23 22:15 inet6.c.rej -rw-r--r-- 1 alfred wheel 2325 Dec 23 22:15 main.c.rej -rw-r--r-- 1 alfred wheel 2989 Dec 23 22:15 mroute.c.rej -rw-r--r-- 1 alfred wheel 945 Dec 23 22:15 netgraph.c.rej -rw-r--r-- 1 alfred wheel 11785 Dec 23 22:15 route.c.rej -rw-r--r-- 1 alfred wheel 821 Dec 23 22:15 unix.c.rej Ouch... Is this slated for being freshened up? As I recall the student who did the GSoC had most of the code ready to commit, utilities included, however we as a group decided not to take the student's code because Juniper was going to provide libxo AND port a few basic utilities to use it. -Alfred On 10/20/14 7:17 PM, Phil Shafer wrote: > Alfred Perlstein writes: >> It's now been 2 months since the last discussion about this. >> Where is the code from Juniper? > Code's still in github, if you want to take a look. It's been > pretty much done since the end of August: > > https://github.com/Juniper/libxo/graphs/contributors > > I've got patches to seven bsd utilities here: > > https://github.com/Juniper/libxo/tree/master/patches > > Docs are still here: > > http://juniper.github.io/libxo/libxo-manual.html > > but I've also written man pages as well (back in Sept). At this > point, I think it's completed. I was tempted on the plane ride > home last Thursday to add a flag for calling humanize_number(), but > I resisted. > > Marcel's working to get it into freebsd. > > Thanks, > Phil > From owner-freebsd-arch@FreeBSD.ORG Wed Dec 24 06:56:29 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BC89988F for ; Wed, 24 Dec 2014 06:56:29 +0000 (UTC) Received: from mail-wi0-x229.google.com (mail-wi0-x229.google.com [IPv6:2a00:1450:400c:c05::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4E26C2D65 for ; Wed, 24 Dec 2014 06:56:29 +0000 (UTC) Received: by mail-wi0-f169.google.com with SMTP id r20so15000603wiv.2 for ; Tue, 23 Dec 2014 22:56:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:reply-to:from:to:subject:date:mime-version:content-type :content-transfer-encoding; bh=oKgORboWUAthuOowLXDtAoJsZZ1iauWTx2SPgkw7nUQ=; b=b3Jykh+52rjmGOSAyBQCZ0r0yWSe8YIsXzMqhUUP1Fm2eC67WBPnlbcJHqt6yLTT6M vuUcRPeMBNSIC02f1yQJ6FyxqkSIwh9hAXEhrKb+lHGBm0+DtchiXjUuy2XEjZIr2cvi 4Osn1IUiTMJVGWdaUlRXTSN0ZXK0HAE1Pz9i9k0NuRY8zE8PY3SfENhbHGA8BoPpJjW7 6UCZbBz5ZPGpSgUn1BwQjwul5mrpc1MnJUfupBPdt7GakwxMtsUePFjSFaLZwwuTs2gO EMKPnQKcEpQ739IxF6Nxk1zoXbxQz3UHzDfqNBz2geCha/Bbe8zx4ybwTxBaqn4yHk0P xKMQ== X-Received: by 10.180.211.34 with SMTP id mz2mr48133302wic.56.1419404187779; Tue, 23 Dec 2014 22:56:27 -0800 (PST) Received: from alipour-PC ([5.234.133.240]) by mx.google.com with ESMTPSA id qg11sm20029921wic.17.2014.12.23.22.56.25 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 23 Dec 2014 22:56:27 -0800 (PST) Message-ID: <03836cdd-41997-036d435013588@alipour-pc> Reply-To: "Global Researchers Journals" From: "Global Researchers Journals" To: freebsd-arch@freebsd.org Subject: Call for Paper December 2014 { Vol 4 | Issue 12} Date: Wed, 24 Dec 2014 10:26:27 +0330 X-Priority: 3 MIME-Version: 1.0 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Dec 2014 06:56:29 -0000 Call for Paper Dear Colleagues You are cordially invited to submit or recommend papers to: [1]http://www.grjournals.com December2014 (Volume 4 | Issue 12) ·Journal of Physiology and Pharmacology Advances (JPPA) [2]http://grjournals.com/Default.aspx?tabid=6537 · Journal of Animal Production Advances (JAPA) [3]http://grjournals.com/Default.aspx?tabid=6538 Journal of Animal Science Advances (JASA) [4]http://grjournals.com/Default.aspx?tabid=6401 Journal of Veterinary Advances (JVA) [5]http://grjournals.com/Default.aspx?tabid=6536 Journal of Recent Advances in Agriculture (JRAA) http://grjournals.com/Default.aspx?tabid=6878 Global Researchers Journals, a fast track peer-reviewed and open access academic journal published by Grjournals Publishing, which is one of the largest open access journal publishers around the world. Grjournals is using online article submission, review and tracking system for quality and quick review processing. Journal provides rapid publication of research article. After 30 days Rapid Review Process by the editorial/review board members or outside experts, an accepted paper will be placed under In Press within 24 hours and will be published in the next issue. Instructions for authors are available on our website: [6]http://www.grjournals.com Submitted papers must follow the Instructions to authors to be considered for review and publication. Refereeing of manuscripts is conducted anonymously and the identity of the referees is not disclosed. The manuscripts which get an acceptance will publish with DOI number. Your Manuscript(s) can be one of these kinds: Review, Original Article, Case Report, Short Communications, Technical Notes, Mini Review Article and Hypothesis. Some of Abstracted/Index in: CAB reviews, Chemical Abstract Service (CAS), Genamics JournalSeek, Index Directory of Open Access Journals (DOAJ), Index Electronic Journals Library and SCIRUS, ISC and the World most Popular University Electronic Library. [7]http://grjournals.com/Defaul t.aspx?tabid=7329 Now you can clear the clutter by accessing your favorite journals online: · Full text, full archive that's always there when you need it · Easy access anywhere, anytime and anyhow · Impact your practice, not the environment NOTICE: Authors that cite [8]www.grjournals.com manuscripts as reference in their ISI articles, they can send their manuscripts to one of above journals as FREE of charge. After evaluation and get an acceptance it will publish without any Article Processing Fee with DOI. We apologize if you have received this email twice, or our journal is not your field. With Warm Regards Sincerely, Grjournals team Site: [9]www.grjournals.com E_Mail: [10]grjournals@gmail.com References 1. http://www.grjournals.com/ 2. http://grjournals.com/Default.aspx?tabid=6537 3. http://grjournals.com/Default.aspx?tabid=6538 4. http://grjournals.com/Default.aspx?tabid=6401 5. http://grjournals.com/Default.aspx?tabid=6536 6. http://www.grjournals.com/ 7. http://grjournals.com/Default.aspx?tabid=7329 8. http://www.grjournals.com/ 9. http://www.grjournals.com/ 10. mailto:grjournals@gmail.com From owner-freebsd-arch@FreeBSD.ORG Wed Dec 24 07:23:10 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3C494DFA; Wed, 24 Dec 2014 07:23:10 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id 25248319F; Wed, 24 Dec 2014 07:23:10 +0000 (UTC) Received: from AlfredMacbookAir.local (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id E1362341F859; Tue, 23 Dec 2014 23:23:09 -0800 (PST) Message-ID: <549A69E4.7020508@freebsd.org> Date: Tue, 23 Dec 2014 23:23:16 -0800 From: Alfred Perlstein Organization: FreeBSD User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Phil Shafer Subject: Re: XML Output: libxo - provide single API to output TXT, XML, JSON and HTML References: <201410210217.s9L2HECn097421@idle.juniper.net> <549A5D1E.7020700@freebsd.org> In-Reply-To: <549A5D1E.7020700@freebsd.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Marcel Moolenaar , John-Mark Gurney , "Simon J. Gerraty" , arch@freebsd.org, Poul-Henning Kamp , freebsd-arch , Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Dec 2014 07:23:10 -0000 I have a github branch where I've started to merge the netstat patch in. I'm not sure I'm going to finish this one, if someone wants to pick it up, send me a pull request. The patch's .rej files are checked in so that it's easy to merge via a side-by-side editor of the file and .rej file. https://github.com/splbio/freebsd/tree/ap_libxo_netstat https://github.com/splbio/freebsd/compare/ap_libxo_netstat?expand=1 -Alfred On 12/23/14 10:28 PM, Alfred Perlstein wrote: > Folks, > > I see libxo has landed in FreeBSD, but it appears that the userland > mods to the utilities never made it in? Is my assessment correct? > > /usr/src/usr.bin/netstat % find /usr/src/ | xargs grep LIBXO | grep -v > /usr/src/contrib/libxo | grep -v /usr/src/lib/libxo > /usr/src/usr.bin/xo/Makefile:LIBXO= ${.CURDIR:H:H}/contrib/libxo > /usr/src/usr.bin/xo/Makefile:.PATH: ${LIBXO}/xo > /usr/src/usr.bin/xo/Makefile:CFLAGS+=-I${LIBXO}/libxo > > The patches against netstat are stale and were never applied. I have > not tried other utilities. > > /usr/src/usr.bin/netstat % wc -l *.rej > 547 if.c.rej > 549 inet.c.rej > 236 inet6.c.rej > 96 main.c.rej > 68 mroute.c.rej > 39 netgraph.c.rej > 415 route.c.rej > 24 unix.c.rej > 1974 total > > /usr/src/usr.bin/netstat % ls -l *.rej > -rw-r--r-- 1 alfred wheel 17486 Dec 23 22:15 if.c.rej > -rw-r--r-- 1 alfred wheel 22687 Dec 23 22:15 inet.c.rej > -rw-r--r-- 1 alfred wheel 11111 Dec 23 22:15 inet6.c.rej > -rw-r--r-- 1 alfred wheel 2325 Dec 23 22:15 main.c.rej > -rw-r--r-- 1 alfred wheel 2989 Dec 23 22:15 mroute.c.rej > -rw-r--r-- 1 alfred wheel 945 Dec 23 22:15 netgraph.c.rej > -rw-r--r-- 1 alfred wheel 11785 Dec 23 22:15 route.c.rej > -rw-r--r-- 1 alfred wheel 821 Dec 23 22:15 unix.c.rej > > Ouch... > > Is this slated for being freshened up? > > As I recall the student who did the GSoC had most of the code ready to > commit, utilities included, however we as a group decided not to take > the student's code because Juniper was going to provide libxo AND port > a few basic utilities to use it. > > -Alfred > > On 10/20/14 7:17 PM, Phil Shafer wrote: >> Alfred Perlstein writes: >>> It's now been 2 months since the last discussion about this. >>> Where is the code from Juniper? >> Code's still in github, if you want to take a look. It's been >> pretty much done since the end of August: >> >> https://github.com/Juniper/libxo/graphs/contributors >> >> I've got patches to seven bsd utilities here: >> >> https://github.com/Juniper/libxo/tree/master/patches >> >> Docs are still here: >> >> http://juniper.github.io/libxo/libxo-manual.html >> >> but I've also written man pages as well (back in Sept). At this >> point, I think it's completed. I was tempted on the plane ride >> home last Thursday to add a flag for calling humanize_number(), but >> I resisted. >> >> Marcel's working to get it into freebsd. >> >> Thanks, >> Phil >> > From owner-freebsd-arch@FreeBSD.ORG Wed Dec 24 19:50:00 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 62514FB9; Wed, 24 Dec 2014 19:50:00 +0000 (UTC) Received: from mail.westryn.net (mail.westryn.net [199.48.135.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 091272D35; Wed, 24 Dec 2014 19:50:00 +0000 (UTC) Received: from sneffels.westryn.net (225x169.ouraynet.com [204.16.225.169]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.westryn.net (Postfix) with ESMTPSA id B50C094323F; Wed, 24 Dec 2014 12:44:06 -0700 (MST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) Subject: Re: XML Output: libxo - provide single API to output TXT, XML, JSON and HTML From: Kim Shrier In-Reply-To: <549A69E4.7020508@freebsd.org> Date: Wed, 24 Dec 2014 12:44:05 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <201410210217.s9L2HECn097421@idle.juniper.net> <549A5D1E.7020700@freebsd.org> <549A69E4.7020508@freebsd.org> To: Alfred Perlstein X-Mailer: Apple Mail (2.1993) Cc: Marcel Moolenaar , Phil Shafer , John-Mark Gurney , "Simon J. Gerraty" , arch@freebsd.org, Poul-Henning Kamp , freebsd-arch , Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Dec 2014 19:50:00 -0000 > On Dec 24, 2014, at 12:23 AM, Alfred Perlstein = wrote: >=20 > I have a github branch where I've started to merge the netstat patch = in. >=20 > I'm not sure I'm going to finish this one, if someone wants to pick it = up, send me a pull request. >=20 > The patch's .rej files are checked in so that it's easy to merge via a = side-by-side editor of the file and .rej file. >=20 > https://github.com/splbio/freebsd/tree/ap_libxo_netstat >=20 > https://github.com/splbio/freebsd/compare/ap_libxo_netstat?expand=3D1 >=20 > -Alfred >=20 I have some time to look at this. Just to make sure I understand what is needed. I have cloned your = repository and checked out the ap_libxo_netstat branch. I see the = following files in usr.bin/netstat: inet.c.rej inet6.c.rej main.c.rej mroute.c.rej netgraph.c.rej route.c.rej unix.c.rej Is all you need is to have these rejected patches merged in or is there = more? Kim From owner-freebsd-arch@FreeBSD.ORG Wed Dec 24 19:54:06 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6479020C; Wed, 24 Dec 2014 19:54:06 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 4CAE62E2D; Wed, 24 Dec 2014 19:54:06 +0000 (UTC) Received: from AlfredMacbookAir.local (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id 39F04341F85F; Wed, 24 Dec 2014 11:54:05 -0800 (PST) Message-ID: <549B19E2.7020300@freebsd.org> Date: Wed, 24 Dec 2014 11:54:10 -0800 From: Alfred Perlstein Organization: FreeBSD User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Kim Shrier Subject: Re: XML Output: libxo - provide single API to output TXT, XML, JSON and HTML References: <201410210217.s9L2HECn097421@idle.juniper.net> <549A5D1E.7020700@freebsd.org> <549A69E4.7020508@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Marcel Moolenaar , Phil Shafer , John-Mark Gurney , "Simon J. Gerraty" , arch@freebsd.org, Poul-Henning Kamp , freebsd-arch , Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Dec 2014 19:54:06 -0000 On 12/24/14 11:44 AM, Kim Shrier wrote: >> On Dec 24, 2014, at 12:23 AM, Alfred Perlstein wrote: >> >> I have a github branch where I've started to merge the netstat patch in. >> >> I'm not sure I'm going to finish this one, if someone wants to pick it up, send me a pull request. >> >> The patch's .rej files are checked in so that it's easy to merge via a side-by-side editor of the file and .rej file. >> >> https://github.com/splbio/freebsd/tree/ap_libxo_netstat >> >> https://github.com/splbio/freebsd/compare/ap_libxo_netstat?expand=1 >> >> -Alfred >> > I have some time to look at this. > > Just to make sure I understand what is needed. I have cloned your repository and checked out the ap_libxo_netstat branch. I see the following files in usr.bin/netstat: > > inet.c.rej > inet6.c.rej > main.c.rej > mroute.c.rej > netgraph.c.rej > route.c.rej > unix.c.rej > > Is all you need is to have these rejected patches merged in or is there more? > > Kim > Kim, Yes. I am actually blowing through this today, I have inet.c.rej merged and inet6.c.rej almost merged. If you want to start at unix.c.rej and work your way towards me that would be cool. I'll email to this list as I make progress and cc you. Other option would be to grab a different utility and start merging that? You can grab other utilities here: https://github.com/Juniper/libxo/tree/master/patches I have commit access, so if you want to send pull requests to my branch I can incorporate your changes into the upstream later. Let me know which direction you want to take. thank you! -Alfred From owner-freebsd-arch@FreeBSD.ORG Wed Dec 24 20:20:28 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B7BEF59F; Wed, 24 Dec 2014 20:20:28 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 69A7C31DF; Wed, 24 Dec 2014 20:20:28 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id sBOKKLMw086381 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 24 Dec 2014 12:20:22 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id sBOKKLk6086380; Wed, 24 Dec 2014 12:20:21 -0800 (PST) (envelope-from jmg) Date: Wed, 24 Dec 2014 12:20:21 -0800 From: John-Mark Gurney To: Alfred Perlstein Subject: Re: XML Output: libxo - provide single API to output TXT, XML, JSON and HTML Message-ID: <20141224202021.GQ1396@funkthat.com> Mail-Followup-To: Alfred Perlstein , Phil Shafer , Marcel Moolenaar , "Simon J. Gerraty" , Poul-Henning Kamp , freebsd-arch , Konstantin Belousov , adrian@FreeBSD.org References: <201410210217.s9L2HECn097421@idle.juniper.net> <549A5D1E.7020700@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <549A5D1E.7020700@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Wed, 24 Dec 2014 12:20:22 -0800 (PST) Cc: adrian@freebsd.org, Marcel Moolenaar , Phil Shafer , "Simon J. Gerraty" , Poul-Henning Kamp , freebsd-arch , Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Dec 2014 20:20:28 -0000 Alfred Perlstein wrote this message on Tue, Dec 23, 2014 at 22:28 -0800: > I see libxo has landed in FreeBSD, but it appears that the userland mods > to the utilities never made it in? Is my assessment correct? Some has made it in... and that introduced a new issue... If you have a system w/ split root and /usr, then you won't be able to use all the utilities in / because of this: # ldd `which df` /bin/df: libxo.so.0 => /usr/lib/libxo.so.0 (0x800820000) libutil.so.9 => /lib/libutil.so.9 (0x800a2f000) libc.so.7 => /lib/libc.so.7 (0x800c41000) Notice how df now depends upon a library in /usr which is a big no-no... libxo needs to be moved to /lib or we need to rip out the support from all utilities that live in /... Adrian pointed this out already, but hasn't gotten any traction... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Wed Dec 24 20:22:09 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CF05D698; Wed, 24 Dec 2014 20:22:09 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id B9D6932A0; Wed, 24 Dec 2014 20:22:09 +0000 (UTC) Received: from AlfredMacbookAir.local (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id 704CF341F853; Wed, 24 Dec 2014 12:22:09 -0800 (PST) Message-ID: <549B2076.9050009@freebsd.org> Date: Wed, 24 Dec 2014 12:22:14 -0800 From: Alfred Perlstein Organization: FreeBSD User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Phil Shafer , Marcel Moolenaar , "Simon J. Gerraty" , Poul-Henning Kamp , freebsd-arch , Konstantin Belousov , adrian@FreeBSD.org Subject: Re: XML Output: libxo - provide single API to output TXT, XML, JSON and HTML References: <201410210217.s9L2HECn097421@idle.juniper.net> <549A5D1E.7020700@freebsd.org> <20141224202021.GQ1396@funkthat.com> In-Reply-To: <20141224202021.GQ1396@funkthat.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Dec 2014 20:22:09 -0000 On 12/24/14 12:20 PM, John-Mark Gurney wrote: > Alfred Perlstein wrote this message on Tue, Dec 23, 2014 at 22:28 -0800: >> I see libxo has landed in FreeBSD, but it appears that the userland mods >> to the utilities never made it in? Is my assessment correct? > Some has made it in... and that introduced a new issue... If you have > a system w/ split root and /usr, then you won't be able to use all > the utilities in / because of this: > # ldd `which df` > /bin/df: > libxo.so.0 => /usr/lib/libxo.so.0 (0x800820000) > libutil.so.9 => /lib/libutil.so.9 (0x800a2f000) > libc.so.7 => /lib/libc.so.7 (0x800c41000) > > Notice how df now depends upon a library in /usr which is a big no-no... > > libxo needs to be moved to /lib or we need to rip out the support from > all utilities that live in /... > > Adrian pointed this out already, but hasn't gotten any traction... > I'll move the lib to /lib. What other things are using it? I can't seem to sort it out via grep unfortunately... -Alfred From owner-freebsd-arch@FreeBSD.ORG Wed Dec 24 20:30:01 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1405E7DC; Wed, 24 Dec 2014 20:30:01 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id 862903419; Wed, 24 Dec 2014 20:29:38 +0000 (UTC) Received: from AlfredMacbookAir.local (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id 11A83341F87B; Wed, 24 Dec 2014 12:29:38 -0800 (PST) Message-ID: <549B2237.5040701@freebsd.org> Date: Wed, 24 Dec 2014 12:29:43 -0800 From: Alfred Perlstein Organization: FreeBSD User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Kim Shrier Subject: Re: XML Output: libxo - provide single API to output TXT, XML, JSON and HTML References: <201410210217.s9L2HECn097421@idle.juniper.net> <549A5D1E.7020700@freebsd.org> <549A69E4.7020508@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Marcel Moolenaar , Phil Shafer , John-Mark Gurney , "Simon J. Gerraty" , arch@freebsd.org, Poul-Henning Kamp , freebsd-arch , Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Dec 2014 20:30:01 -0000 Kim, all I have left for netstat at this point is: netgraph.c.rej route.c.rej unix.c.rej I believe this will just be a few more hours of work, I'm off to get final gifts for girlfriend, I'll be back and let you know how I'm doing. did you decide to work on netstat or one of the other programs? On 12/24/14 11:44 AM, Kim Shrier wrote: >> On Dec 24, 2014, at 12:23 AM, Alfred Perlstein wrote: >> >> I have a github branch where I've started to merge the netstat patch in. >> >> I'm not sure I'm going to finish this one, if someone wants to pick it up, send me a pull request. >> >> The patch's .rej files are checked in so that it's easy to merge via a side-by-side editor of the file and .rej file. >> >> https://github.com/splbio/freebsd/tree/ap_libxo_netstat >> >> https://github.com/splbio/freebsd/compare/ap_libxo_netstat?expand=1 >> >> -Alfred >> > I have some time to look at this. > > Just to make sure I understand what is needed. I have cloned your repository and checked out the ap_libxo_netstat branch. I see the following files in usr.bin/netstat: > > inet.c.rej > inet6.c.rej > main.c.rej > mroute.c.rej > netgraph.c.rej > route.c.rej > unix.c.rej > > Is all you need is to have these rejected patches merged in or is there more? > > Kim > From owner-freebsd-arch@FreeBSD.ORG Wed Dec 24 21:15:02 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B4B272F1; Wed, 24 Dec 2014 21:15:02 +0000 (UTC) Received: from mail.westryn.net (mail.westryn.net [199.48.135.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 138FB1884; Wed, 24 Dec 2014 21:15:02 +0000 (UTC) Received: from sneffels.westryn.net (225x169.ouraynet.com [204.16.225.169]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.westryn.net (Postfix) with ESMTPSA id 9F16994323F; Wed, 24 Dec 2014 14:14:59 -0700 (MST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) Subject: Re: XML Output: libxo - provide single API to output TXT, XML, JSON and HTML From: Kim Shrier In-Reply-To: <549B2237.5040701@freebsd.org> Date: Wed, 24 Dec 2014 14:14:58 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <70E1870D-39D0-4578-8FB4-D24CB41562A5@westryn.net> References: <201410210217.s9L2HECn097421@idle.juniper.net> <549A5D1E.7020700@freebsd.org> <549A69E4.7020508@freebsd.org> <549B2237.5040701@freebsd.org> To: Alfred Perlstein X-Mailer: Apple Mail (2.1993) Cc: Marcel Moolenaar , Phil Shafer , John-Mark Gurney , "Simon J. Gerraty" , arch@freebsd.org, Poul-Henning Kamp , freebsd-arch , Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Dec 2014 21:15:02 -0000 I just did unix.c.rej and am starting on route.c.rej. Kim > On Dec 24, 2014, at 1:29 PM, Alfred Perlstein = wrote: >=20 > Kim, all I have left for netstat at this point is: >=20 > netgraph.c.rej > route.c.rej > unix.c.rej >=20 > I believe this will just be a few more hours of work, I'm off to get = final gifts for girlfriend, I'll be back and let you know how I'm doing. >=20 > did you decide to work on netstat or one of the other programs? >=20 > On 12/24/14 11:44 AM, Kim Shrier wrote: >>> On Dec 24, 2014, at 12:23 AM, Alfred Perlstein = wrote: >>>=20 >>> I have a github branch where I've started to merge the netstat patch = in. >>>=20 >>> I'm not sure I'm going to finish this one, if someone wants to pick = it up, send me a pull request. >>>=20 >>> The patch's .rej files are checked in so that it's easy to merge via = a side-by-side editor of the file and .rej file. >>>=20 >>> https://github.com/splbio/freebsd/tree/ap_libxo_netstat >>>=20 >>> https://github.com/splbio/freebsd/compare/ap_libxo_netstat?expand=3D1 >>>=20 >>> -Alfred >>>=20 >> I have some time to look at this. >>=20 >> Just to make sure I understand what is needed. I have cloned your = repository and checked out the ap_libxo_netstat branch. I see the = following files in usr.bin/netstat: >>=20 >> inet.c.rej >> inet6.c.rej >> main.c.rej >> mroute.c.rej >> netgraph.c.rej >> route.c.rej >> unix.c.rej >>=20 >> Is all you need is to have these rejected patches merged in or is = there more? >>=20 >> Kim >>=20 >=20 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 24 21:15:41 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 04BAF3A0 for ; Wed, 24 Dec 2014 21:15:41 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id E3BD21896 for ; Wed, 24 Dec 2014 21:15:40 +0000 (UTC) Received: from AlfredMacbookAir.local (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id BF871341F853 for ; Wed, 24 Dec 2014 13:15:40 -0800 (PST) Message-ID: <549B2D01.4060004@mu.org> Date: Wed, 24 Dec 2014 13:15:45 -0800 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: XML Output: libxo - provide single API to output TXT, XML, JSON and HTML References: <201410210217.s9L2HECn097421@idle.juniper.net> <549A5D1E.7020700@freebsd.org> <549A69E4.7020508@freebsd.org> <549B2237.5040701@freebsd.org> <70E1870D-39D0-4578-8FB4-D24CB41562A5@westryn.net> In-Reply-To: <70E1870D-39D0-4578-8FB4-D24CB41562A5@westryn.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Dec 2014 21:15:41 -0000 Wait, I just finished up route.c. I think we're nearly done! On 12/24/14 1:14 PM, Kim Shrier wrote: > I just did unix.c.rej and am starting on route.c.rej. > > Kim > >> On Dec 24, 2014, at 1:29 PM, Alfred Perlstein wrote: >> >> Kim, all I have left for netstat at this point is: >> >> netgraph.c.rej >> route.c.rej >> unix.c.rej >> >> I believe this will just be a few more hours of work, I'm off to get final gifts for girlfriend, I'll be back and let you know how I'm doing. >> >> did you decide to work on netstat or one of the other programs? >> >> On 12/24/14 11:44 AM, Kim Shrier wrote: >>>> On Dec 24, 2014, at 12:23 AM, Alfred Perlstein wrote: >>>> >>>> I have a github branch where I've started to merge the netstat patch in. >>>> >>>> I'm not sure I'm going to finish this one, if someone wants to pick it up, send me a pull request. >>>> >>>> The patch's .rej files are checked in so that it's easy to merge via a side-by-side editor of the file and .rej file. >>>> >>>> https://github.com/splbio/freebsd/tree/ap_libxo_netstat >>>> >>>> https://github.com/splbio/freebsd/compare/ap_libxo_netstat?expand=1 >>>> >>>> -Alfred >>>> >>> I have some time to look at this. >>> >>> Just to make sure I understand what is needed. I have cloned your repository and checked out the ap_libxo_netstat branch. I see the following files in usr.bin/netstat: >>> >>> inet.c.rej >>> inet6.c.rej >>> main.c.rej >>> mroute.c.rej >>> netgraph.c.rej >>> route.c.rej >>> unix.c.rej >>> >>> Is all you need is to have these rejected patches merged in or is there more? >>> >>> Kim >>> > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Wed Dec 24 21:21:05 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 05956493 for ; Wed, 24 Dec 2014 21:21:05 +0000 (UTC) Received: from mail.westryn.net (mail.westryn.net [199.48.135.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D66E918F3 for ; Wed, 24 Dec 2014 21:21:04 +0000 (UTC) Received: from sneffels.westryn.net (225x169.ouraynet.com [204.16.225.169]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.westryn.net (Postfix) with ESMTPSA id 298AF94323F; Wed, 24 Dec 2014 14:21:03 -0700 (MST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) Subject: Re: XML Output: libxo - provide single API to output TXT, XML, JSON and HTML From: Kim Shrier In-Reply-To: <549B2D01.4060004@mu.org> Date: Wed, 24 Dec 2014 14:21:02 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <641115DC-284A-437D-A53E-1E5E1DD86DD1@westryn.net> References: <201410210217.s9L2HECn097421@idle.juniper.net> <549A5D1E.7020700@freebsd.org> <549A69E4.7020508@freebsd.org> <549B2237.5040701@freebsd.org> <70E1870D-39D0-4578-8FB4-D24CB41562A5@westryn.net> <549B2D01.4060004@mu.org> To: Alfred Perlstein X-Mailer: Apple Mail (2.1993) Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Dec 2014 21:21:05 -0000 Should I just push up my changes to unix.c and submit a pull request for = that? Kim > On Dec 24, 2014, at 2:15 PM, Alfred Perlstein wrote: >=20 > Wait, I just finished up route.c. >=20 > I think we're nearly done! >=20 >=20 > On 12/24/14 1:14 PM, Kim Shrier wrote: >> I just did unix.c.rej and am starting on route.c.rej. >>=20 >> Kim >>=20 >>> On Dec 24, 2014, at 1:29 PM, Alfred Perlstein = wrote: >>>=20 >>> Kim, all I have left for netstat at this point is: >>>=20 >>> netgraph.c.rej >>> route.c.rej >>> unix.c.rej >>>=20 >>> I believe this will just be a few more hours of work, I'm off to get = final gifts for girlfriend, I'll be back and let you know how I'm doing. >>>=20 >>> did you decide to work on netstat or one of the other programs? >>>=20 >>> On 12/24/14 11:44 AM, Kim Shrier wrote: >>>>> On Dec 24, 2014, at 12:23 AM, Alfred Perlstein = wrote: >>>>>=20 >>>>> I have a github branch where I've started to merge the netstat = patch in. >>>>>=20 >>>>> I'm not sure I'm going to finish this one, if someone wants to = pick it up, send me a pull request. >>>>>=20 >>>>> The patch's .rej files are checked in so that it's easy to merge = via a side-by-side editor of the file and .rej file. >>>>>=20 >>>>> https://github.com/splbio/freebsd/tree/ap_libxo_netstat >>>>>=20 >>>>> = https://github.com/splbio/freebsd/compare/ap_libxo_netstat?expand=3D1 >>>>>=20 >>>>> -Alfred >>>>>=20 >>>> I have some time to look at this. >>>>=20 >>>> Just to make sure I understand what is needed. I have cloned your = repository and checked out the ap_libxo_netstat branch. I see the = following files in usr.bin/netstat: >>>>=20 >>>> inet.c.rej >>>> inet6.c.rej >>>> main.c.rej >>>> mroute.c.rej >>>> netgraph.c.rej >>>> route.c.rej >>>> unix.c.rej >>>>=20 >>>> Is all you need is to have these rejected patches merged in or is = there more? >>>>=20 >>>> Kim >>>>=20 >> _______________________________________________ >> freebsd-arch@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-arch >> To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org" >>=20 >=20 > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Wed Dec 24 21:49:57 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 930EC8FF; Wed, 24 Dec 2014 21:49:57 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id 78F4F6487F; Wed, 24 Dec 2014 21:49:57 +0000 (UTC) Received: from AlfredMacbookAir.local (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id E38E6341F853; Wed, 24 Dec 2014 13:49:56 -0800 (PST) Message-ID: <549B3509.9090502@freebsd.org> Date: Wed, 24 Dec 2014 13:50:01 -0800 From: Alfred Perlstein Organization: FreeBSD User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Kim Shrier Subject: Re: XML Output: libxo - provide single API to output TXT, XML, JSON and HTML References: <201410210217.s9L2HECn097421@idle.juniper.net> <549A5D1E.7020700@freebsd.org> <549A69E4.7020508@freebsd.org> <549B2237.5040701@freebsd.org> <70E1870D-39D0-4578-8FB4-D24CB41562A5@westryn.net> In-Reply-To: <70E1870D-39D0-4578-8FB4-D24CB41562A5@westryn.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Marcel Moolenaar , Phil Shafer , John-Mark Gurney , "Simon J. Gerraty" , arch@freebsd.org, Poul-Henning Kamp , freebsd-arch , Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Dec 2014 21:49:57 -0000 Kim, nice!! I'm doing some testing, I need to do some cleanup. Already took your changes on github. I'll push the combined changes shortly. On 12/24/14 1:14 PM, Kim Shrier wrote: > I just did unix.c.rej and am starting on route.c.rej. > > Kim > >> On Dec 24, 2014, at 1:29 PM, Alfred Perlstein wrote: >> >> Kim, all I have left for netstat at this point is: >> >> netgraph.c.rej >> route.c.rej >> unix.c.rej >> >> I believe this will just be a few more hours of work, I'm off to get final gifts for girlfriend, I'll be back and let you know how I'm doing. >> >> did you decide to work on netstat or one of the other programs? >> >> On 12/24/14 11:44 AM, Kim Shrier wrote: >>>> On Dec 24, 2014, at 12:23 AM, Alfred Perlstein wrote: >>>> >>>> I have a github branch where I've started to merge the netstat patch in. >>>> >>>> I'm not sure I'm going to finish this one, if someone wants to pick it up, send me a pull request. >>>> >>>> The patch's .rej files are checked in so that it's easy to merge via a side-by-side editor of the file and .rej file. >>>> >>>> https://github.com/splbio/freebsd/tree/ap_libxo_netstat >>>> >>>> https://github.com/splbio/freebsd/compare/ap_libxo_netstat?expand=1 >>>> >>>> -Alfred >>>> >>> I have some time to look at this. >>> >>> Just to make sure I understand what is needed. I have cloned your repository and checked out the ap_libxo_netstat branch. I see the following files in usr.bin/netstat: >>> >>> inet.c.rej >>> inet6.c.rej >>> main.c.rej >>> mroute.c.rej >>> netgraph.c.rej >>> route.c.rej >>> unix.c.rej >>> >>> Is all you need is to have these rejected patches merged in or is there more? >>> >>> Kim >>> > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Wed Dec 24 21:56:35 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 25C93B42; Wed, 24 Dec 2014 21:56:35 +0000 (UTC) Received: from mail.westryn.net (mail.westryn.net [199.48.135.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5DD2064992; Wed, 24 Dec 2014 21:56:34 +0000 (UTC) Received: from sneffels.westryn.net (225x169.ouraynet.com [204.16.225.169]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.westryn.net (Postfix) with ESMTPSA id 25DC794323F; Wed, 24 Dec 2014 14:56:32 -0700 (MST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) Subject: Re: XML Output: libxo - provide single API to output TXT, XML, JSON and HTML From: Kim Shrier In-Reply-To: <549B3509.9090502@freebsd.org> Date: Wed, 24 Dec 2014 14:56:31 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <201410210217.s9L2HECn097421@idle.juniper.net> <549A5D1E.7020700@freebsd.org> <549A69E4.7020508@freebsd.org> <549B2237.5040701@freebsd.org> <70E1870D-39D0-4578-8FB4-D24CB41562A5@westryn.net> <549B3509.9090502@freebsd.org> To: Alfred Perlstein X-Mailer: Apple Mail (2.1993) Cc: Marcel Moolenaar , Phil Shafer , John-Mark Gurney , "Simon J. Gerraty" , arch@freebsd.org, Poul-Henning Kamp , freebsd-arch , Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Dec 2014 21:56:35 -0000 If I have time, are there other utilities that need merging? Kim > On Dec 24, 2014, at 2:50 PM, Alfred Perlstein = wrote: >=20 > Kim, nice!! >=20 > I'm doing some testing, I need to do some cleanup. >=20 > Already took your changes on github. I'll push the combined changes = shortly. >=20 > On 12/24/14 1:14 PM, Kim Shrier wrote: >> I just did unix.c.rej and am starting on route.c.rej. >>=20 >> Kim >>=20 >>> On Dec 24, 2014, at 1:29 PM, Alfred Perlstein = wrote: >>>=20 >>> Kim, all I have left for netstat at this point is: >>>=20 >>> netgraph.c.rej >>> route.c.rej >>> unix.c.rej >>>=20 >>> I believe this will just be a few more hours of work, I'm off to get = final gifts for girlfriend, I'll be back and let you know how I'm doing. >>>=20 >>> did you decide to work on netstat or one of the other programs? >>>=20 >>> On 12/24/14 11:44 AM, Kim Shrier wrote: >>>>> On Dec 24, 2014, at 12:23 AM, Alfred Perlstein = wrote: >>>>>=20 >>>>> I have a github branch where I've started to merge the netstat = patch in. >>>>>=20 >>>>> I'm not sure I'm going to finish this one, if someone wants to = pick it up, send me a pull request. >>>>>=20 >>>>> The patch's .rej files are checked in so that it's easy to merge = via a side-by-side editor of the file and .rej file. >>>>>=20 >>>>> https://github.com/splbio/freebsd/tree/ap_libxo_netstat >>>>>=20 >>>>> = https://github.com/splbio/freebsd/compare/ap_libxo_netstat?expand=3D1 >>>>>=20 >>>>> -Alfred >>>>>=20 >>>> I have some time to look at this. >>>>=20 >>>> Just to make sure I understand what is needed. I have cloned your = repository and checked out the ap_libxo_netstat branch. I see the = following files in usr.bin/netstat: >>>>=20 >>>> inet.c.rej >>>> inet6.c.rej >>>> main.c.rej >>>> mroute.c.rej >>>> netgraph.c.rej >>>> route.c.rej >>>> unix.c.rej >>>>=20 >>>> Is all you need is to have these rejected patches merged in or is = there more? >>>>=20 >>>> Kim >>>>=20 >> _______________________________________________ >> freebsd-arch@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-arch >> To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org" >>=20 >=20 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 24 22:04:47 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F2B0AD5C; Wed, 24 Dec 2014 22:04:46 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id D290D64AEC; Wed, 24 Dec 2014 22:04:46 +0000 (UTC) Received: from AlfredMacbookAir.local (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id 5748C341F853; Wed, 24 Dec 2014 14:04:45 -0800 (PST) Message-ID: <549B3882.20407@freebsd.org> Date: Wed, 24 Dec 2014 14:04:50 -0800 From: Alfred Perlstein Organization: FreeBSD User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Kim Shrier Subject: Re: XML Output: libxo - provide single API to output TXT, XML, JSON and HTML References: <201410210217.s9L2HECn097421@idle.juniper.net> <549A5D1E.7020700@freebsd.org> <549A69E4.7020508@freebsd.org> <549B2237.5040701@freebsd.org> <70E1870D-39D0-4578-8FB4-D24CB41562A5@westryn.net> <549B3509.9090502@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Marcel Moolenaar , Phil Shafer , John-Mark Gurney , "Simon J. Gerraty" , arch@freebsd.org, Poul-Henning Kamp , freebsd-arch , Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Dec 2014 22:04:47 -0000 Kim, Can you please look at: https://github.com/Juniper/libxo/tree/master/patches See if we can merge more of those in. I'm not sure which are already done. Or if you would like, looking to convert other utilities would be great. -Alfred On 12/24/14 1:56 PM, Kim Shrier wrote: > If I have time, are there other utilities that need merging? > > Kim > >> On Dec 24, 2014, at 2:50 PM, Alfred Perlstein wrote: >> >> Kim, nice!! >> >> I'm doing some testing, I need to do some cleanup. >> >> Already took your changes on github. I'll push the combined changes shortly. >> >> On 12/24/14 1:14 PM, Kim Shrier wrote: >>> I just did unix.c.rej and am starting on route.c.rej. >>> >>> Kim >>> >>>> On Dec 24, 2014, at 1:29 PM, Alfred Perlstein wrote: >>>> >>>> Kim, all I have left for netstat at this point is: >>>> >>>> netgraph.c.rej >>>> route.c.rej >>>> unix.c.rej >>>> >>>> I believe this will just be a few more hours of work, I'm off to get final gifts for girlfriend, I'll be back and let you know how I'm doing. >>>> >>>> did you decide to work on netstat or one of the other programs? >>>> >>>> On 12/24/14 11:44 AM, Kim Shrier wrote: >>>>>> On Dec 24, 2014, at 12:23 AM, Alfred Perlstein wrote: >>>>>> >>>>>> I have a github branch where I've started to merge the netstat patch in. >>>>>> >>>>>> I'm not sure I'm going to finish this one, if someone wants to pick it up, send me a pull request. >>>>>> >>>>>> The patch's .rej files are checked in so that it's easy to merge via a side-by-side editor of the file and .rej file. >>>>>> >>>>>> https://github.com/splbio/freebsd/tree/ap_libxo_netstat >>>>>> >>>>>> https://github.com/splbio/freebsd/compare/ap_libxo_netstat?expand=1 >>>>>> >>>>>> -Alfred >>>>>> >>>>> I have some time to look at this. >>>>> >>>>> Just to make sure I understand what is needed. I have cloned your repository and checked out the ap_libxo_netstat branch. I see the following files in usr.bin/netstat: >>>>> >>>>> inet.c.rej >>>>> inet6.c.rej >>>>> main.c.rej >>>>> mroute.c.rej >>>>> netgraph.c.rej >>>>> route.c.rej >>>>> unix.c.rej >>>>> >>>>> Is all you need is to have these rejected patches merged in or is there more? >>>>> >>>>> Kim >>>>> >>> _______________________________________________ >>> freebsd-arch@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-arch >>> To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" >>> > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Thu Dec 25 02:19:59 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0D442D9; Thu, 25 Dec 2014 02:19:59 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id E4AB63BD2; Thu, 25 Dec 2014 02:19:49 +0000 (UTC) Received: from AlfredMacbookAir.local (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id 51B1B341F85A; Wed, 24 Dec 2014 18:19:49 -0800 (PST) Message-ID: <549B7449.3080507@freebsd.org> Date: Wed, 24 Dec 2014 18:19:53 -0800 From: Alfred Perlstein Organization: FreeBSD User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Phil Shafer , Marcel Moolenaar , "Simon J. Gerraty" , Poul-Henning Kamp , freebsd-arch , Konstantin Belousov , adrian@FreeBSD.org Subject: Re: XML Output: libxo - provide single API to output TXT, XML, JSON and HTML References: <201410210217.s9L2HECn097421@idle.juniper.net> <549A5D1E.7020700@freebsd.org> <20141224202021.GQ1396@funkthat.com> In-Reply-To: <20141224202021.GQ1396@funkthat.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Dec 2014 02:19:59 -0000 A revision has been created to track this: https://reviews.freebsd.org/D1370 On 12/24/14 12:20 PM, John-Mark Gurney wrote: > Alfred Perlstein wrote this message on Tue, Dec 23, 2014 at 22:28 -0800: >> I see libxo has landed in FreeBSD, but it appears that the userland mods >> to the utilities never made it in? Is my assessment correct? > Some has made it in... and that introduced a new issue... If you have > a system w/ split root and /usr, then you won't be able to use all > the utilities in / because of this: > # ldd `which df` > /bin/df: > libxo.so.0 => /usr/lib/libxo.so.0 (0x800820000) > libutil.so.9 => /lib/libutil.so.9 (0x800a2f000) > libc.so.7 => /lib/libc.so.7 (0x800c41000) > > Notice how df now depends upon a library in /usr which is a big no-no... > > libxo needs to be moved to /lib or we need to rip out the support from > all utilities that live in /... > > Adrian pointed this out already, but hasn't gotten any traction... > From owner-freebsd-arch@FreeBSD.ORG Thu Dec 25 02:51:40 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E400D6EB; Thu, 25 Dec 2014 02:51:40 +0000 (UTC) Received: from mail-pa0-x22f.google.com (mail-pa0-x22f.google.com [IPv6:2607:f8b0:400e:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AD00F1B02; Thu, 25 Dec 2014 02:51:40 +0000 (UTC) Received: by mail-pa0-f47.google.com with SMTP id kq14so11002161pab.6; Wed, 24 Dec 2014 18:51:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=l+UZVFG7LBmkSirgspxNy+aJjfPlk/qNxzC12w29BkE=; b=TK+BURL/p0s5DH97HxJPy9nl7JaJmPTuznR8eeR9zhFff4QJZKfATIK7pLJUwXmTmP nUt6JOjOSTFLxR7bGInxbZrVokoTtkHM1Gthb8bw9jZDfS6EOBFfd8d6S8xVMREgTwjK 2GdRHCAOpsCygtKk+zld/cuwQLH0O+XrngwFqfI7hY7vj9lH04ZqgzvJ3FyLsz45zgeZ Ahs8d3jLk117Sk2VIYjM5T3AowgB3SMgYZyUEHR67dB8ZP36vzPVBxl5Ie75WaYeaZtm Tnp7t+IrxrLUpe3Ycd3o1j0eLpWNHzu3I7+ocWscRON2xpfBr8R+wVMJ1/DVpjZwxkrY bP/g== X-Received: by 10.70.37.4 with SMTP id u4mr19898343pdj.40.1419475900250; Wed, 24 Dec 2014 18:51:40 -0800 (PST) Received: from [10.157.91.197] (mobile-166-171-251-172.mycingular.net. [166.171.251.172]) by mx.google.com with ESMTPSA id x8sm24140686pdi.7.2014.12.24.18.51.02 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 24 Dec 2014 18:51:39 -0800 (PST) References: <201410210217.s9L2HECn097421@idle.juniper.net> <549A5D1E.7020700@freebsd.org> <20141224202021.GQ1396@funkthat.com> <549B7449.3080507@freebsd.org> Mime-Version: 1.0 (1.0) In-Reply-To: <549B7449.3080507@freebsd.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: X-Mailer: iPhone Mail (12B440) From: Garrett Cooper Subject: Re: XML Output: libxo - provide single API to output TXT, XML, JSON and HTML Date: Wed, 24 Dec 2014 18:50:56 -0800 To: Alfred Perlstein Cc: "adrian@FreeBSD.org" , Marcel Moolenaar , Phil Shafer , "Simon J. Gerraty" , Poul-Henning Kamp , freebsd-arch , Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Dec 2014 02:51:41 -0000 > On Dec 24, 2014, at 18:19, Alfred Perlstein wrote: >=20 > A revision has been created to track this: https://reviews.freebsd.org/D13= 70 Hi Alfred, Please add the entry to ObsoleteFiles.inc. once that's done, please comm= it the change. Thanks :)!!= From owner-freebsd-arch@FreeBSD.ORG Thu Dec 25 03:18:57 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A9ABACCD; Thu, 25 Dec 2014 03:18:57 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 91B6964C58; Thu, 25 Dec 2014 03:18:57 +0000 (UTC) Received: from AlfredMacbookAir.local (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id 34E74341F85A; Wed, 24 Dec 2014 19:18:55 -0800 (PST) Message-ID: <549B8223.4010602@freebsd.org> Date: Wed, 24 Dec 2014 19:18:59 -0800 From: Alfred Perlstein Organization: FreeBSD User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Garrett Cooper Subject: Re: XML Output: libxo - provide single API to output TXT, XML, JSON and HTML References: <201410210217.s9L2HECn097421@idle.juniper.net> <549A5D1E.7020700@freebsd.org> <20141224202021.GQ1396@funkthat.com> <549B7449.3080507@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: "adrian@FreeBSD.org" , Marcel Moolenaar , Phil Shafer , "Simon J. Gerraty" , Poul-Henning Kamp , freebsd-arch , Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Dec 2014 03:18:57 -0000 There is a fix for the wc(1) output problem here: https://reviews.freebsd.org/D1378 On 12/24/14 6:50 PM, Garrett Cooper wrote: >> On Dec 24, 2014, at 18:19, Alfred Perlstein wrote: >> >> A revision has been created to track this: https://reviews.freebsd.org/D1370 > Hi Alfred, > Please add the entry to ObsoleteFiles.inc. once that's done, please commit the change. > Thanks :)!! > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Thu Dec 25 05:53:56 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0D7DCB38; Thu, 25 Dec 2014 05:53:56 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id E9EE62690; Thu, 25 Dec 2014 05:53:55 +0000 (UTC) Received: from AlfredMacbookAir.local (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id B093C341F85A; Wed, 24 Dec 2014 21:53:54 -0800 (PST) Message-ID: <549BA675.9070107@freebsd.org> Date: Wed, 24 Dec 2014 21:53:57 -0800 From: Alfred Perlstein Organization: FreeBSD User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: "Simon J. Gerraty" Subject: Libxo bugs and fixes. References: <201408141640.s7EGe422096656@idle.juniper.net> <53ED57F2.5020808@mu.org> <20140815053604.9E40B580A2@chaos.jnpr.net> <53EDB0EF.6090902@mu.org> <20140815173830.93832580A2@chaos.jnpr.net> <53EEA74B.9070107@mu.org> <20140816045254.5F47E580A2@chaos.jnpr.net> In-Reply-To: <20140816045254.5F47E580A2@chaos.jnpr.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Marcel Moolenaar , Phil Shafer , John-Mark Gurney , arch@freebsd.org, Poul-Henning Kamp , freebsd-arch , Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Dec 2014 05:53:56 -0000 On 8/15/14 9:52 PM, Simon J. Gerraty wrote: > On Fri, 15 Aug 2014 17:35:23 -0700, Alfred Perlstein writes: >> should get towards converting more utils to using it. However if we are >> going to perpetually add frameworky things, but not convert over >> userland tools to the actual framework, then that is a potential problem >> worth calling out. > Indeed. Again that's why I prefer to see this (the library at least) > done by someone who's been doing this sort of thing successfuly for > ages. OK, So I've been hacking on this over the last day or so and have the following deltas up for review: 1) We need libxo to support explicit fflush: https://reviews.freebsd.org/D1379 2) We need libxo to support time series data. I don't have a patch for that, but I would like to ask for ideas on this. This would be extremely helpful for instance "netstat 1". would like to get a very accurate timestamp with each row. Suggestions on how to do this correctly AND *consistently* via libxo would make sense. 3) I have netstat mostly converted over, however not all the json is valid, I will be doing another pass ASAP on this. https://reviews.freebsd.org/D1380 4) Can I as politely as I know how, ask that Juniper keep the patches against the userland utils up to date either in a project branch in SVN, or simply in a fork of the github repo? This would make life so much easer to deal with. As an example you can look at my github repo where I have the netstat code forward ported on a branch here: https://github.com/splbio/freebsd/tree/ap_libxo_netstat and here: https://github.com/splbio/freebsd/compare/ap_libxo_netstat?expand=1 -Alfred From owner-freebsd-arch@FreeBSD.ORG Thu Dec 25 07:22:36 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0D7DFFFC; Thu, 25 Dec 2014 07:22:36 +0000 (UTC) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2on0132.outbound.protection.outlook.com [65.55.169.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4A4403FDD; Thu, 25 Dec 2014 07:22:33 +0000 (UTC) Received: from BLUPR05MB435.namprd05.prod.outlook.com (10.141.27.150) by BLUPR05MB338.namprd05.prod.outlook.com (10.141.24.140) with Microsoft SMTP Server (TLS) id 15.1.49.12; Thu, 25 Dec 2014 07:22:27 +0000 Received: from BL2PR05CA0038.namprd05.prod.outlook.com (10.255.226.38) by BLUPR05MB435.namprd05.prod.outlook.com (10.141.27.150) with Microsoft SMTP Server (TLS) id 15.1.49.12; Thu, 25 Dec 2014 07:22:26 +0000 Received: from BL2FFO11FD051.protection.gbl (2a01:111:f400:7c09::162) by BL2PR05CA0038.outlook.office365.com (2a01:111:e400:c04::38) with Microsoft SMTP Server (TLS) id 15.1.49.12 via Frontend Transport; Thu, 25 Dec 2014 07:22:26 +0000 Received: from P-EMF02-SAC.jnpr.net (66.129.239.16) by BL2FFO11FD051.mail.protection.outlook.com (10.173.161.213) with Microsoft SMTP Server (TLS) id 15.1.49.13 via Frontend Transport; Thu, 25 Dec 2014 07:22:25 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF02-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Wed, 24 Dec 2014 23:22:24 -0800 Received: from chaos.jnpr.net (chaos.jnpr.net [172.21.16.28]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id sBP7MNW11858; Wed, 24 Dec 2014 23:22:24 -0800 (PST) (envelope-from sjg@juniper.net) Received: from chaos (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id AFBC4580A3; Wed, 24 Dec 2014 23:22:23 -0800 (PST) To: Alfred Perlstein Subject: Re: XML Output: libxo - provide single API to output TXT, XML, JSON and HTML In-Reply-To: <549B7449.3080507@freebsd.org> References: <201410210217.s9L2HECn097421@idle.juniper.net> <549A5D1E.7020700@freebsd.org> <20141224202021.GQ1396@funkthat.com> <549B7449.3080507@freebsd.org> Comments: In-reply-to: Alfred Perlstein message dated "Wed, 24 Dec 2014 18:19:53 -0800." From: "Simon J. Gerraty" X-Mailer: MH-E 8.0.3; nmh 1.3; GNU Emacs 22.3.1 Date: Wed, 24 Dec 2014 23:22:23 -0800 Message-ID: <14549.1419492143@chaos> MIME-Version: 1.0 Content-Type: text/plain X-EOPAttributedMessage: 0 Received-SPF: SoftFail (protection.outlook.com: domain of transitioning juniper.net discourages use of 66.129.239.16 as permitted sender) Authentication-Results: spf=softfail (sender IP is 66.129.239.16) smtp.mailfrom=sjg@juniper.net; X-Forefront-Antispam-Report: CIP:66.129.239.16; CTRY:US; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10019020)(6009001)(189002)(51704005)(24454002)(199003)(77096005)(64706001)(15975445007)(107046002)(50226001)(4396001)(6806004)(19580395003)(19580405001)(86362001)(69596002)(89996001)(68736005)(105596002)(76176999)(2950100001)(99396003)(106466001)(81156004)(50986999)(87936001)(20776003)(62966003)(47776003)(77156002)(97736003)(110136001)(120916001)(93886004)(50466002)(117636001)(76506005)(84676001)(33716001)(21056001)(46102003)(57986006)(48376002)(42262002)(62816006); DIR:OUT; SFP:1102; SCL:1; SRVR:BLUPR05MB435; H:P-EMF02-SAC.jnpr.net; FPR:; SPF:SoftFail; MLV:sfv; PTR:InfoDomainNonexistent; MX:1; A:1; LANG:en; X-Microsoft-Antispam: UriScan:;UriScan:; X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:;SRVR:BLUPR05MB435; X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004); SRVR:BLUPR05MB435; X-Forefront-PRVS: 04362AC73B X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:;SRVR:BLUPR05MB435; X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Dec 2014 07:22:25.8270 (UTC) X-MS-Exchange-CrossTenant-Id: bea78b3c-4cdb-4130-854a-1d193232e5f4 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=bea78b3c-4cdb-4130-854a-1d193232e5f4; Ip=[66.129.239.16] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR05MB435 X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:;SRVR:BLUPR05MB338; X-OriginatorOrg: juniper.net Cc: adrian@freebsd.org, Marcel Moolenaar , Phil Shafer , Poul-Henning Kamp , freebsd-arch , Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Dec 2014 07:22:36 -0000 Alfred Perlstein wrote: > A revision has been created to track this: https://reviews.freebsd.org/D1370 Thanks - looks fine. I just looked and didn't see any conclusion to the discussuion about phabricator logins, so don't yet (know if I) have one. > > /bin/df: > > libxo.so.0 => /usr/lib/libxo.so.0 (0x800820000) > > libutil.so.9 => /lib/libutil.so.9 (0x800a2f000) > > libc.so.7 => /lib/libc.so.7 (0x800c41000) > > > > Notice how df now depends upon a library in /usr which is a big no-no... > > > > libxo needs to be moved to /lib or we need to rip out the support from > > all utilities that live in /... > > > > Adrian pointed this out already, but hasn't gotten any traction... > > From owner-freebsd-arch@FreeBSD.ORG Thu Dec 25 07:32:29 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 72F9A1A4; Thu, 25 Dec 2014 07:32:29 +0000 (UTC) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1bon0110.outbound.protection.outlook.com [157.56.111.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AC392640F7; Thu, 25 Dec 2014 07:32:27 +0000 (UTC) Received: from BL2PR05CA0014.namprd05.prod.outlook.com (10.255.226.14) by CO1PR05MB443.namprd05.prod.outlook.com (10.141.73.152) with Microsoft SMTP Server (TLS) id 15.1.49.12; Thu, 25 Dec 2014 07:32:18 +0000 Received: from BN1BFFO11FD030.protection.gbl (2a01:111:f400:7c10::1:193) by BL2PR05CA0014.outlook.office365.com (2a01:111:e400:c04::14) with Microsoft SMTP Server (TLS) id 15.1.49.12 via Frontend Transport; Thu, 25 Dec 2014 07:32:17 +0000 Received: from P-EMF02-SAC.jnpr.net (66.129.239.16) by BN1BFFO11FD030.mail.protection.outlook.com (10.58.144.93) with Microsoft SMTP Server (TLS) id 15.1.49.13 via Frontend Transport; Thu, 25 Dec 2014 07:32:17 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF02-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Wed, 24 Dec 2014 23:32:07 -0800 Received: from chaos.jnpr.net (chaos.jnpr.net [172.21.16.28]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id sBP7W6W15995; Wed, 24 Dec 2014 23:32:06 -0800 (PST) (envelope-from sjg@juniper.net) Received: from chaos (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id 3C176580A3; Wed, 24 Dec 2014 23:32:06 -0800 (PST) To: Alfred Perlstein Subject: Re: Libxo bugs and fixes. In-Reply-To: <549BA675.9070107@freebsd.org> References: <201408141640.s7EGe422096656@idle.juniper.net> <53ED57F2.5020808@mu.org> <20140815053604.9E40B580A2@chaos.jnpr.net> <53EDB0EF.6090902@mu.org> <20140815173830.93832580A2@chaos.jnpr.net> <53EEA74B.9070107@mu.org> <20140816045254.5F47E580A2@chaos.jnpr.net> <549BA675.9070107@freebsd.org> Comments: In-reply-to: Alfred Perlstein message dated "Wed, 24 Dec 2014 21:53:57 -0800." From: "Simon J. Gerraty" X-Mailer: MH-E 8.0.3; nmh 1.3; GNU Emacs 22.3.1 Date: Wed, 24 Dec 2014 23:32:06 -0800 Message-ID: <29784.1419492726@chaos> MIME-Version: 1.0 Content-Type: text/plain X-EOPAttributedMessage: 0 Received-SPF: SoftFail (protection.outlook.com: domain of transitioning juniper.net discourages use of 66.129.239.16 as permitted sender) Authentication-Results: spf=softfail (sender IP is 66.129.239.16) smtp.mailfrom=sjg@juniper.net; X-Forefront-Antispam-Report: CIP:66.129.239.16; CTRY:US; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10019020)(6009001)(199003)(189002)(24454002)(51704005)(86362001)(84676001)(50226001)(117636001)(76506005)(47776003)(15975445007)(64706001)(20776003)(77096005)(76176999)(93886004)(50986999)(4396001)(105596002)(107046002)(57986006)(110136001)(2950100001)(33716001)(62966003)(120916001)(97736003)(89996001)(69596002)(6806004)(19580395003)(106466001)(50466002)(68736005)(46102003)(99396003)(77156002)(21056001)(87936001)(81156004)(48376002)(19580405001)(42262002)(62816006); DIR:OUT; SFP:1102; SCL:1; SRVR:CO1PR05MB443; H:P-EMF02-SAC.jnpr.net; FPR:; SPF:SoftFail; MLV:sfv; PTR:InfoDomainNonexistent; A:1; MX:1; LANG:en; X-Microsoft-Antispam: UriScan:; X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:;SRVR:CO1PR05MB443; X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004); SRVR:CO1PR05MB443; X-Forefront-PRVS: 04362AC73B X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:;SRVR:CO1PR05MB443; X-OriginatorOrg: juniper.net X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Dec 2014 07:32:17.5080 (UTC) X-MS-Exchange-CrossTenant-Id: bea78b3c-4cdb-4130-854a-1d193232e5f4 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=bea78b3c-4cdb-4130-854a-1d193232e5f4; Ip=[66.129.239.16] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR05MB443 Cc: Marcel Moolenaar , Phil Shafer , John-Mark Gurney , arch@freebsd.org, Poul-Henning Kamp , freebsd-arch , Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Dec 2014 07:32:29 -0000 Alfred Perlstein wrote: > 1) We need libxo to support explicit fflush: > https://reviews.freebsd.org/D1379 This is probably sub-optimal. Supporting flushing isn't the same thing as blindly doing it. Probably better if caller can indicate a need for flushing regularly. Will try and catch up with Phil to discuss. > 2) We need libxo to support time series data. I don't have a patch > for that, but I would like to ask for ideas on this. This would be > extremely helpful for instance "netstat 1". would like to get a very > accurate timestamp with each row. Suggestions on how to do this > correctly AND *consistently* via libxo would make sense. > > 3) I have netstat mostly converted over, however not all the json is > valid, I will be doing another pass ASAP on this. > https://reviews.freebsd.org/D1380 > > 4) Can I as politely as I know how, ask that Juniper keep the patches > against the userland utils up to date either in a project branch in > SVN, or simply in a fork of the github repo? This would make life so > much easer to deal with. I assume that commits would be more use? From owner-freebsd-arch@FreeBSD.ORG Thu Dec 25 14:38:14 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4C2607E3; Thu, 25 Dec 2014 14:38:14 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id 3171764C3E; Thu, 25 Dec 2014 14:38:14 +0000 (UTC) Received: from [10.0.1.106] (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id C2578341F85F; Thu, 25 Dec 2014 06:38:13 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: Libxo bugs and fixes. From: Alfred Perlstein X-Mailer: iPhone Mail (12B440) In-Reply-To: <29784.1419492726@chaos> Date: Thu, 25 Dec 2014 06:38:12 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <26E73A9E-6EA4-4698-885D-BD91906D32D2@mu.org> References: <201408141640.s7EGe422096656@idle.juniper.net> <53ED57F2.5020808@mu.org> <20140815053604.9E40B580A2@chaos.jnpr.net> <53EDB0EF.6090902@mu.org> <20140815173830.93832580A2@chaos.jnpr.net> <53EEA74B.9070107@mu.org> <20140816045254.5F47E580A2@chaos.jnpr.net> <549BA675.9070107@freebsd.org> <29784.1419492726@chaos> To: "Simon J. Gerraty" Cc: Marcel Moolenaar , Phil Shafer , John-Mark Gurney , Alfred Perlstein , "arch@freebsd.org" , Poul-Henning Kamp , freebsd-arch , Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Dec 2014 14:38:14 -0000 > On Dec 24, 2014, at 11:32 PM, Simon J. Gerraty wrote: >=20 > Alfred Perlstein wrote: >> 1) We need libxo to support explicit fflush: >> https://reviews.freebsd.org/D1379 >=20 > This is probably sub-optimal. =20 > Supporting flushing isn't the same thing as blindly doing it. > Probably better if caller can indicate a need for flushing regularly. > Will try and catch up with Phil to discuss. Yes, the patch leaves it to the caller. The otherwise when you run "netstat -= -libxo json 1" you only see output every BUFSIZE bytes from netstat.=20 With the fix I added the xo_flush calls actually work. Which I think is what= you want as well.=20 >=20 >> 2) We need libxo to support time series data. I don't have a patch >> for that, but I would like to ask for ideas on this. This would be >> extremely helpful for instance "netstat 1". would like to get a very >> accurate timestamp with each row. Suggestions on how to do this >> correctly AND *consistently* via libxo would make sense. >>=20 >> 3) I have netstat mostly converted over, however not all the json is >> valid, I will be doing another pass ASAP on this. >> https://reviews.freebsd.org/D1380 >>=20 >> 4) Can I as politely as I know how, ask that Juniper keep the patches >> against the userland utils up to date either in a project branch in >> SVN, or simply in a fork of the github repo? This would make life so >> much easer to deal with. >=20 > I assume that commits would be more use? Yes! Also we should really have unit tests made to make sure the json is valid. I= 've found some bugs and I am fixing them, but it will take some time and I w= ant to avoid regressions.=20 > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" >=20 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 25 17:46:10 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CED19BE3; Thu, 25 Dec 2014 17:46:10 +0000 (UTC) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1on0148.outbound.protection.outlook.com [157.56.110.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EAF3B2AB6; Thu, 25 Dec 2014 17:46:09 +0000 (UTC) Received: from CO2PR05CA047.namprd05.prod.outlook.com (10.141.241.175) by BN1PR05MB439.namprd05.prod.outlook.com (10.141.58.22) with Microsoft SMTP Server (TLS) id 15.1.49.12; Thu, 25 Dec 2014 17:31:23 +0000 Received: from BN1BFFO11FD050.protection.gbl (2a01:111:f400:7c10::1:199) by CO2PR05CA047.outlook.office365.com (2a01:111:e400:1429::47) with Microsoft SMTP Server (TLS) id 15.1.49.12 via Frontend Transport; Thu, 25 Dec 2014 17:31:22 +0000 Received: from P-EMF02-SAC.jnpr.net (66.129.239.16) by BN1BFFO11FD050.mail.protection.outlook.com (10.58.145.5) with Microsoft SMTP Server (TLS) id 15.1.49.13 via Frontend Transport; Thu, 25 Dec 2014 17:31:22 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF02-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Thu, 25 Dec 2014 09:31:20 -0800 Received: from chaos.jnpr.net (chaos.jnpr.net [172.21.16.28]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id sBPHVJW89220; Thu, 25 Dec 2014 09:31:19 -0800 (PST) (envelope-from sjg@juniper.net) Received: from chaos (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id 692D8580A3; Thu, 25 Dec 2014 09:31:19 -0800 (PST) To: Alfred Perlstein Subject: Re: Libxo bugs and fixes. In-Reply-To: <26E73A9E-6EA4-4698-885D-BD91906D32D2@mu.org> References: <201408141640.s7EGe422096656@idle.juniper.net> <53ED57F2.5020808@mu.org> <20140815053604.9E40B580A2@chaos.jnpr.net> <53EDB0EF.6090902@mu.org> <20140815173830.93832580A2@chaos.jnpr.net> <53EEA74B.9070107@mu.org> <20140816045254.5F47E580A2@chaos.jnpr.net> <549BA675.9070107@freebsd.org> <29784.1419492726@chaos> <26E73A9E-6EA4-4698-885D-BD91906D32D2@mu.org> Comments: In-reply-to: Alfred Perlstein message dated "Thu, 25 Dec 2014 06:38:12 -0800." From: "Simon J. Gerraty" X-Mailer: MH-E 8.0.3; nmh 1.3; GNU Emacs 22.3.1 Date: Thu, 25 Dec 2014 09:31:19 -0800 Message-ID: <18276.1419528679@chaos> MIME-Version: 1.0 Content-Type: text/plain X-EOPAttributedMessage: 0 Received-SPF: SoftFail (protection.outlook.com: domain of transitioning juniper.net discourages use of 66.129.239.16 as permitted sender) Authentication-Results: spf=softfail (sender IP is 66.129.239.16) smtp.mailfrom=sjg@juniper.net; X-Forefront-Antispam-Report: CIP:66.129.239.16; CTRY:US; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10019020)(6009001)(189002)(199003)(4396001)(50226001)(97736003)(33716001)(57986006)(76176999)(48376002)(76506005)(64706001)(20776003)(84676001)(46102003)(21056001)(93886004)(47776003)(86362001)(107046002)(2950100001)(89996001)(87936001)(92566001)(50466002)(50986999)(31966008)(68736005)(62966003)(77156002)(19580395003)(19580405001)(105596002)(6806004)(117636001)(120916001)(77096005)(69596002)(106466001)(99396003)(110136001)(81156004)(42262002)(62816006); DIR:OUT; SFP:1102; SCL:1; SRVR:BN1PR05MB439; H:P-EMF02-SAC.jnpr.net; FPR:; SPF:SoftFail; MLV:sfv; PTR:InfoDomainNonexistent; MX:1; A:1; LANG:en; X-Microsoft-Antispam: UriScan:; X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:;SRVR:BN1PR05MB439; X-Forefront-PRVS: 04362AC73B X-OriginatorOrg: juniper.net X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Dec 2014 17:31:22.0827 (UTC) X-MS-Exchange-CrossTenant-Id: bea78b3c-4cdb-4130-854a-1d193232e5f4 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=bea78b3c-4cdb-4130-854a-1d193232e5f4; Ip=[66.129.239.16] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN1PR05MB439 Cc: Marcel Moolenaar , Phil Shafer , John-Mark Gurney , Alfred Perlstein , "arch@freebsd.org" , Poul-Henning Kamp , freebsd-arch , Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Dec 2014 17:46:11 -0000 Alfred Perlstein wrote: > Yes, the patch leaves it to the caller. The otherwise when you run "netstat --libxo json 1" you only see output every BUFSIZE bytes from netstat. I was refering to the call to xo_flush in xo_emit. The application best knows what are suitable points at which to flush its output. If you eliminate that call, it might be ok From owner-freebsd-arch@FreeBSD.ORG Thu Dec 25 18:01:29 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9AFEA2AA; Thu, 25 Dec 2014 18:01:29 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id 809922D9A; Thu, 25 Dec 2014 18:01:29 +0000 (UTC) Received: from AlfredMacbookAir.local (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id EA1BC341F85F; Thu, 25 Dec 2014 10:01:28 -0800 (PST) Message-ID: <549C50FB.6050908@mu.org> Date: Thu, 25 Dec 2014 10:01:31 -0800 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: "Simon J. Gerraty" Subject: Re: Libxo bugs and fixes. References: <201408141640.s7EGe422096656@idle.juniper.net> <53ED57F2.5020808@mu.org> <20140815053604.9E40B580A2@chaos.jnpr.net> <53EDB0EF.6090902@mu.org> <20140815173830.93832580A2@chaos.jnpr.net> <53EEA74B.9070107@mu.org> <20140816045254.5F47E580A2@chaos.jnpr.net> <549BA675.9070107@freebsd.org> <29784.1419492726@chaos> <26E73A9E-6EA4-4698-885D-BD91906D32D2@mu.org> <18276.1419528679@chaos> In-Reply-To: <18276.1419528679@chaos> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Marcel Moolenaar , Phil Shafer , John-Mark Gurney , Alfred Perlstein , "arch@freebsd.org" , Poul-Henning Kamp , freebsd-arch , Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Dec 2014 18:01:29 -0000 On 12/25/14 9:31 AM, Simon J. Gerraty wrote: > Alfred Perlstein wrote: >> Yes, the patch leaves it to the caller. The otherwise when you run "netstat --libxo json 1" you only see output every BUFSIZE bytes from netstat. > I was refering to the call to xo_flush in xo_emit. > The application best knows what are suitable points at which to flush > its output. > If you eliminate that call, it might be ok > I don't think my patch adds an xo_flush to xo_emit... ? Did I miss something? Yes, there is an explicit flush at the end of xo_message_hcv() but that should be harmless or possibly removed since the output channel for warnings should be unbuffered. -Alfred From owner-freebsd-arch@FreeBSD.ORG Fri Dec 26 01:57:45 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B81322DF; Fri, 26 Dec 2014 01:57:45 +0000 (UTC) Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2on0105.outbound.protection.outlook.com [207.46.100.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EEBF92AAE; Fri, 26 Dec 2014 01:57:44 +0000 (UTC) Received: from BLUPR05CA0069.namprd05.prod.outlook.com (10.141.20.39) by BN1PR05MB439.namprd05.prod.outlook.com (10.141.58.22) with Microsoft SMTP Server (TLS) id 15.1.49.12; Thu, 25 Dec 2014 22:23:56 +0000 Received: from BL2FFO11FD011.protection.gbl (2a01:111:f400:7c09::122) by BLUPR05CA0069.outlook.office365.com (2a01:111:e400:855::39) with Microsoft SMTP Server (TLS) id 15.1.49.12 via Frontend Transport; Thu, 25 Dec 2014 22:23:56 +0000 Received: from P-EMF02-SAC.jnpr.net (66.129.239.16) by BL2FFO11FD011.mail.protection.outlook.com (10.173.161.17) with Microsoft SMTP Server (TLS) id 15.1.49.13 via Frontend Transport; Thu, 25 Dec 2014 22:23:56 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF02-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Thu, 25 Dec 2014 14:23:55 -0800 Received: from chaos.jnpr.net (chaos.jnpr.net [172.21.16.28]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id sBPMNsW20429; Thu, 25 Dec 2014 14:23:54 -0800 (PST) (envelope-from sjg@juniper.net) Received: from chaos (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id 0AC01580A3; Thu, 25 Dec 2014 14:23:54 -0800 (PST) To: Alfred Perlstein Subject: Re: Libxo bugs and fixes. In-Reply-To: <549C50FB.6050908@mu.org> References: <201408141640.s7EGe422096656@idle.juniper.net> <53ED57F2.5020808@mu.org> <20140815053604.9E40B580A2@chaos.jnpr.net> <53EDB0EF.6090902@mu.org> <20140815173830.93832580A2@chaos.jnpr.net> <53EEA74B.9070107@mu.org> <20140816045254.5F47E580A2@chaos.jnpr.net> <549BA675.9070107@freebsd.org> <29784.1419492726@chaos> <26E73A9E-6EA4-4698-885D-BD91906D32D2@mu.org> <18276.1419528679@chaos> <549C50FB.6050908@mu.org> Comments: In-reply-to: Alfred Perlstein message dated "Thu, 25 Dec 2014 10:01:31 -0800." From: "Simon J. Gerraty" X-Mailer: MH-E 8.0.3; nmh 1.3; GNU Emacs 22.3.1 Date: Thu, 25 Dec 2014 14:23:54 -0800 Message-ID: <23687.1419546234@chaos> MIME-Version: 1.0 Content-Type: text/plain X-EOPAttributedMessage: 0 Received-SPF: SoftFail (protection.outlook.com: domain of transitioning juniper.net discourages use of 66.129.239.16 as permitted sender) Authentication-Results: spf=softfail (sender IP is 66.129.239.16) smtp.mailfrom=sjg@juniper.net; X-Forefront-Antispam-Report: CIP:66.129.239.16; CTRY:US; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10019020)(6009001)(24454002)(189002)(199003)(4396001)(50226001)(48376002)(76176999)(57986006)(33716001)(76506005)(64706001)(20776003)(558084003)(46102003)(97736003)(47776003)(93886004)(21056001)(86362001)(107046002)(89996001)(2950100001)(87936001)(92566001)(50466002)(31966008)(50986999)(68736005)(62966003)(77156002)(19580395003)(19580405001)(105596002)(6806004)(117636001)(120916001)(77096005)(69596002)(84676001)(106466001)(81156004)(99396003)(110136001)(42262002)(62816006); DIR:OUT; SFP:1102; SCL:1; SRVR:BN1PR05MB439; H:P-EMF02-SAC.jnpr.net; FPR:; SPF:SoftFail; MLV:sfv; PTR:InfoDomainNonexistent; MX:1; A:1; LANG:en; X-Microsoft-Antispam: UriScan:; X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:;SRVR:BN1PR05MB439; X-Forefront-PRVS: 04362AC73B X-OriginatorOrg: juniper.net X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Dec 2014 22:23:56.4497 (UTC) X-MS-Exchange-CrossTenant-Id: bea78b3c-4cdb-4130-854a-1d193232e5f4 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=bea78b3c-4cdb-4130-854a-1d193232e5f4; Ip=[66.129.239.16] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN1PR05MB439 Cc: Marcel Moolenaar , Phil Shafer , John-Mark Gurney , Alfred Perlstein , "arch@freebsd.org" , Poul-Henning Kamp , freebsd-arch , Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Dec 2014 01:57:45 -0000 Alfred Perlstein wrote: > I don't think my patch adds an xo_flush to xo_emit... ? No, you added a test around it - I only looked at the RHS. From owner-freebsd-arch@FreeBSD.ORG Fri Dec 26 16:53:46 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6A80536D; Fri, 26 Dec 2014 16:53:46 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0933032CC; Fri, 26 Dec 2014 16:53:45 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id sBQGrbkk053247 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 26 Dec 2014 18:53:37 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua sBQGrbkk053247 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id sBQGrb58053246; Fri, 26 Dec 2014 18:53:37 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 26 Dec 2014 18:53:37 +0200 From: Konstantin Belousov To: threads@freebsd.org Subject: Fixing dlopen("libpthread.so") Message-ID: <20141226165337.GJ1754@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Dec 2014 16:53:46 -0000 [Long] It is somewhat well-known that our libthr.so cannot be loaded dynamically into the process. Or rather, it can be, but the consequences are catastrophic. We recommend to link any program which may load modules, explicitely with -lpthread; the known workaround is to do LD_PRELOAD=libthr.so.3 for binaries which were not. I implemented support for ld -z nodlopen some time ago, but attempt to mark libthr.so as non-loadable caused extreme roar. A common opinion is that the proper way to fix the problem is to merge the actual code from libthr into libc, leaving libthr as the filter to preserve the current ABI. Unfortunately, there are some non-trivial and undesirable consequences of doing this. First, all pthread mutexes (and other kind of locks) would become fully initialized and used even for single-threaded programs, at least I do not see a way to work around this. Right now, libc shims for pthread_mutex_init() and pthread_mutex_lock(3) are nop. After the merge, init needs to allocate memory and lock/unlock operations, although uncontested, will start costing one atomic each. In particular, malloc(3) and stdio(3) are affected. Another very delicate issue is introducing unwanted cancellation points into libc functions after libthr wrappers become mandatory. This is fixable, but requires lot of mundane work and probably a long time to find missed places (i.e. bugs). There are probably more problems, and this brings an obvious alternative: fix the issues which make dlopen("libthr.so") so destructive. One known show-stopper is the broken errno after the load. The libthr provides the interposer for the errno and all cancellable functions from libc. If any interposed symbols have been resolved before the libthr.so was loaded, or non-lazy binding mode is requested, the bindings cannot be undonde. In particular, references to __error(), which implements errno, are bound to return locate of the main thread errno variable. Similarly, code referencing cancellable functions still gets the uncancellable libc implementations of them. Another issue is the recursion between malloc(3) and mutex_init(). The statically initialized pthread_mutex_t needs some further initialization before first use. Jemalloc calls pthread_mutex_init(3) for internally-used mutexes, which is nop stub from libc until libthr is loaded. After the load, first use of any mutex by malloc(3) leads to the thr_mutex.c initialization code, which needs calloc(3). This immediately leads to hang due to recursion on some internal libthr umtx. Making the lock recursive does not solve the problem, which is the infinite mutual recursion between malloc and pthread_mutex_lock() for uninitialized malloc mutex. Yet another issue is the signal handlers. The libthr routes signal delivery through its internal signal handler, to avoid interrupting critical sections. Any signal handler installed prior to libthr is loaded misses the wrapper, potentially breaking cancellation and critical sections. Proposed patch does the following: - Remove libthr interposers of the libc functions, including __error(). Instead, functions calls are indirected through the interposing table, similar to how pthread stubs in libc are already done. Libc by default points either to syscall trampolines or to existing libc implementations. On libthr load, it rewrites the pointers to the cancellable implementations already in libthr. - Postpone the malloc(3) internal mutexes initialization until libthr is loaded. - Reinstall signal handlers with wrapper on libthr load. The signal handler reinstallation on libthr initialization is only needed when libthr.so is dlopened. Performing 128*2 sigaction(2) calls on the startup of the binary which is linked to libthr, and thus libthr is guaranteed to install proper sighandler wrappers, is huge waste. So, I perform the hand-over of signal handlers only for the dlopen-ed libthr, which now needs to detect loading at startup vs. dlopen. I was unable to distinguish the cases using existing facilities, so new private rtld interface is implemented, _rtld_is_dlopened(), to query the way library was brought into the process address space. Without some special measures, static binaries would pull in the whole set of the interposed syscalls due to references from the interposition table. To fix it, the references are made weak. Also, to not pull in the pthread stubs, the interposition table is separate from pthreads stubs indirection table. The patch is available at https://www.kib.kiev.ua/kib/libthr_dlopen.1.patch . Among other things, I tested it with the program illustrating the issues https://www.kib.kiev.ua/kib/threaded_errno.c . Note that you must use matching versions of rtld, libc and libthr. Using old ld-elf.so.1 or old libc.so.7 with new libthr.so.3 will break the system. Work was sponsored by The FreeBSD Foundation. From owner-freebsd-arch@FreeBSD.ORG Fri Dec 26 18:03:53 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 97D31B4E; Fri, 26 Dec 2014 18:03:53 +0000 (UTC) Received: from mail.netplex.net (mail.netplex.net [204.213.176.9]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.netplex.net", Issuer "RapidSSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6B1D1666E8; Fri, 26 Dec 2014 18:03:53 +0000 (UTC) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.9/8.14.9/NETPLEX) with ESMTP id sBQHseTK006519; Fri, 26 Dec 2014 12:54:41 -0500 X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.4.3 (mail.netplex.net [204.213.176.9]); Fri, 26 Dec 2014 12:54:41 -0500 (EST) Date: Fri, 26 Dec 2014 12:54:40 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net Reply-To: Daniel Eischen To: Konstantin Belousov Subject: Re: Fixing dlopen("libpthread.so") In-Reply-To: <20141226165337.GJ1754@kib.kiev.ua> Message-ID: References: <20141226165337.GJ1754@kib.kiev.ua> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: threads@freebsd.org, arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Dec 2014 18:03:53 -0000 On Fri, 26 Dec 2014, Konstantin Belousov wrote: > [Long] > > It is somewhat well-known that our libthr.so cannot be loaded > dynamically into the process. Or rather, it can be, but the > consequences are catastrophic. We recommend to link any program which > may load modules, explicitely with -lpthread; the known workaround is > to do LD_PRELOAD=libthr.so.3 for binaries which were not. I > implemented support for ld -z nodlopen some time ago, but attempt to > mark libthr.so as non-loadable caused extreme roar. > > A common opinion is that the proper way to fix the problem is > to merge the actual code from libthr into libc, leaving libthr as the > filter to preserve the current ABI. Unfortunately, there are some > non-trivial and undesirable consequences of doing this. > > First, all pthread mutexes (and other kind of locks) would become > fully initialized and used even for single-threaded programs, at least > I do not see a way to work around this. Right now, libc shims for > pthread_mutex_init() and pthread_mutex_lock(3) are nop. After the > merge, init needs to allocate memory and lock/unlock operations, > although uncontested, will start costing one atomic each. In > particular, malloc(3) and stdio(3) are affected. > > Another very delicate issue is introducing unwanted cancellation > points into libc functions after libthr wrappers become mandatory. > This is fixable, but requires lot of mundane work and probably a long > time to find missed places (i.e. bugs). > > There are probably more problems, and this brings an obvious > alternative: fix the issues which make dlopen("libthr.so") so > destructive. > > One known show-stopper is the broken errno after the load. The libthr > provides the interposer for the errno and all cancellable functions > from libc. If any interposed symbols have been resolved before the > libthr.so was loaded, or non-lazy binding mode is requested, the > bindings cannot be undonde. In particular, references to __error(), > which implements errno, are bound to return locate of the main thread > errno variable. Similarly, code referencing cancellable functions > still gets the uncancellable libc implementations of them. > > Another issue is the recursion between malloc(3) and mutex_init(). > The statically initialized pthread_mutex_t needs some further > initialization before first use. Jemalloc calls pthread_mutex_init(3) > for internally-used mutexes, which is nop stub from libc until libthr > is loaded. After the load, first use of any mutex by malloc(3) leads > to the thr_mutex.c initialization code, which needs calloc(3). This > immediately leads to hang due to recursion on some internal libthr > umtx. Making the lock recursive does not solve the problem, which is > the infinite mutual recursion between malloc and pthread_mutex_lock() > for uninitialized malloc mutex. > > Yet another issue is the signal handlers. The libthr routes signal > delivery through its internal signal handler, to avoid interrupting > critical sections. Any signal handler installed prior to libthr is > loaded misses the wrapper, potentially breaking cancellation and > critical sections. > > Proposed patch does the following: > > - Remove libthr interposers of the libc functions, including > __error(). Instead, functions calls are indirected through the > interposing table, similar to how pthread stubs in libc are already > done. Libc by default points either to syscall trampolines or to > existing libc implementations. On libthr load, it rewrites the > pointers to the cancellable implementations already in libthr. > > - Postpone the malloc(3) internal mutexes initialization until libthr > is loaded. > > - Reinstall signal handlers with wrapper on libthr load. > > The signal handler reinstallation on libthr initialization is only > needed when libthr.so is dlopened. Performing 128*2 sigaction(2) > calls on the startup of the binary which is linked to libthr, and thus > libthr is guaranteed to install proper sighandler wrappers, is huge > waste. So, I perform the hand-over of signal handlers only for the > dlopen-ed libthr, which now needs to detect loading at startup > vs. dlopen. I was unable to distinguish the cases using existing > facilities, so new private rtld interface is implemented, > _rtld_is_dlopened(), to query the way library was brought into the > process address space. > > Without some special measures, static binaries would pull in the whole > set of the interposed syscalls due to references from the > interposition table. To fix it, the references are made weak. Also, > to not pull in the pthread stubs, the interposition table is separate > from pthreads stubs indirection table. > > The patch is available at > https://www.kib.kiev.ua/kib/libthr_dlopen.1.patch . > Among other things, I tested it with the program illustrating the > issues https://www.kib.kiev.ua/kib/threaded_errno.c . > Note that you must use matching versions of rtld, libc and libthr. > Using old ld-elf.so.1 or old libc.so.7 with new libthr.so.3 will > break the system. > > Work was sponsored by The FreeBSD Foundation. I took a once-over look at the patch and it looks good. I never liked the intimacy between malloc and libpthread, but there's nothing we can currently do about that until mutexes become real objects instead of pointers. -- DE From owner-freebsd-arch@FreeBSD.ORG Sat Dec 27 03:58:07 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 85F9BA81; Sat, 27 Dec 2014 03:58:07 +0000 (UTC) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1bbn0104.outbound.protection.outlook.com [157.56.111.104]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DDD0D66570; Sat, 27 Dec 2014 03:58:05 +0000 (UTC) Received: from CO2PR05CA039.namprd05.prod.outlook.com (10.141.241.167) by DM2PR05MB448.namprd05.prod.outlook.com (10.141.104.152) with Microsoft SMTP Server (TLS) id 15.1.49.12; Sat, 27 Dec 2014 01:26:28 +0000 Received: from BN1AFFO11FD009.protection.gbl (2a01:111:f400:7c10::102) by CO2PR05CA039.outlook.office365.com (2a01:111:e400:1429::39) with Microsoft SMTP Server (TLS) id 15.1.49.12 via Frontend Transport; Sat, 27 Dec 2014 01:26:28 +0000 Received: from P-EMF02-SAC.jnpr.net (66.129.239.16) by BN1AFFO11FD009.mail.protection.outlook.com (10.58.52.69) with Microsoft SMTP Server (TLS) id 15.1.49.13 via Frontend Transport; Sat, 27 Dec 2014 01:26:27 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF02-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Fri, 26 Dec 2014 17:26:24 -0800 Received: from idle.juniper.net (idleski.juniper.net [172.25.4.26]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id sBR1QMW60332; Fri, 26 Dec 2014 17:26:22 -0800 (PST) (envelope-from phil@juniper.net) Received: from idle.juniper.net (localhost [127.0.0.1]) by idle.juniper.net (8.14.4/8.14.3) with ESMTP id sBR1Q6U9005556; Fri, 26 Dec 2014 20:26:07 -0500 (EST) (envelope-from phil@idle.juniper.net) Message-ID: <201412270126.sBR1Q6U9005556@idle.juniper.net> To: Alfred Perlstein Subject: Re: XML Output: libxo - provide single API to output TXT, XML, JSON and HTML In-Reply-To: <549B8223.4010602@freebsd.org> Date: Fri, 26 Dec 2014 20:26:06 -0500 From: Phil Shafer MIME-Version: 1.0 Content-Type: text/plain X-EOPAttributedMessage: 0 Received-SPF: SoftFail (protection.outlook.com: domain of transitioning juniper.net discourages use of 66.129.239.16 as permitted sender) Authentication-Results: spf=softfail (sender IP is 66.129.239.16) smtp.mailfrom=phil@juniper.net; X-Forefront-Antispam-Report: CIP:66.129.239.16; CTRY:US; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10019020)(6009001)(189002)(199003)(164054003)(76506005)(21056001)(87936001)(97736003)(53416004)(575784001)(99396003)(86362001)(110136001)(46102003)(68736005)(15975445007)(2950100001)(120916001)(84676001)(77156002)(62966003)(103666002)(69596002)(6806004)(107046002)(106466001)(64706001)(92566001)(48376002)(50466002)(20776003)(50986999)(81156004)(77096005)(105596002)(54356999)(19580395003)(31966008)(4396001)(47776003); DIR:OUT; SFP:1102; SCL:1; SRVR:DM2PR05MB448; H:P-EMF02-SAC.jnpr.net; FPR:; SPF:SoftFail; MLV:sfv; PTR:InfoDomainNonexistent; MX:1; A:1; LANG:en; X-Microsoft-Antispam: UriScan:; X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:;SRVR:DM2PR05MB448; X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004); SRVR:DM2PR05MB448; X-Forefront-PRVS: 0438F90F17 X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:;SRVR:DM2PR05MB448; X-OriginatorOrg: juniper.net X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Dec 2014 01:26:27.6343 (UTC) X-MS-Exchange-CrossTenant-Id: bea78b3c-4cdb-4130-854a-1d193232e5f4 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=bea78b3c-4cdb-4130-854a-1d193232e5f4; Ip=[66.129.239.16] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM2PR05MB448 Cc: "adrian@FreeBSD.org" , Marcel Moolenaar , "Simon J. Gerraty" , Poul-Henning Kamp , freebsd-arch , Konstantin Belousov , Garrett Cooper X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Dec 2014 03:58:07 -0000 Alfred Perlstein writes: >There is a fix for the wc(1) output problem here: >https://reviews.freebsd.org/D1378 I have a fix for xo_err in the develop branch: https://github.com/Juniper/libxo/commit/6b4a946c38ce1e0e161b87ef56513b9e44c819af I can roll a new release when I catch up on these emails. Sorry for the slow replies, but it's been oh-so nice having an unplugged vacation. Thanks, Phil From owner-freebsd-arch@FreeBSD.ORG Sat Dec 27 21:31:11 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1FADABC; Sat, 27 Dec 2014 21:31:11 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id 0A54DA9F; Sat, 27 Dec 2014 21:31:11 +0000 (UTC) Received: from [10.0.1.20] (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id DA12B341F858; Sat, 27 Dec 2014 13:31:09 -0800 (PST) Subject: Re: XML Output: libxo - provide single API to output TXT, XML, JSON and HTML Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Alfred Perlstein In-Reply-To: <201412270126.sBR1Q6U9005556@idle.juniper.net> Date: Sat, 27 Dec 2014 13:31:09 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <201412270126.sBR1Q6U9005556@idle.juniper.net> To: Phil Shafer X-Mailer: Apple Mail (2.1283) Cc: "adrian@FreeBSD.org" , Marcel Moolenaar , "Simon J. Gerraty" , Poul-Henning Kamp , freebsd-arch , Konstantin Belousov , Garrett Cooper X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Dec 2014 21:31:11 -0000 On Dec 26, 2014, at 5:26 PM, Phil Shafer wrote: > Alfred Perlstein writes: >> There is a fix for the wc(1) output problem here: >> https://reviews.freebsd.org/D1378 >=20 > I have a fix for xo_err in the develop branch: >=20 > = https://github.com/Juniper/libxo/commit/6b4a946c38ce1e0e161b87ef56513b9e44= c819af >=20 > I can roll a new release when I catch up on these emails. >=20 > Sorry for the slow replies, but it's been oh-so nice having an > unplugged vacation. >=20 > Thanks, > Phil >=20 Phil, Would like to expedite process here, I'm going to check in a few interim = fixes so that people can continue hackathon on libxo over the holiday = break. When you're back please feel free to revert/modify my changes. Right now my only plans are to add fixes for xo_warn as well as add = ability to explicitly flush the stream. -Alfred