From owner-freebsd-ports@FreeBSD.ORG  Tue Dec 28 20:31:37 2010
Return-Path: <owner-freebsd-ports@FreeBSD.ORG>
Delivered-To: freebsd-ports@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id B962E106566C;
	Tue, 28 Dec 2010 20:31:37 +0000 (UTC) (envelope-from rea@codelabs.ru)
Received: from 0.mx.codelabs.ru (0.mx.codelabs.ru [144.206.177.45])
	by mx1.freebsd.org (Postfix) with ESMTP id 4EDD08FC17;
	Tue, 28 Dec 2010 20:31:37 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=codelabs.ru;
	s=two; 
	h=Sender:In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date;
	bh=rFMJccwV6jRbOSoku+l6H5DrnJvkn+KTlMxH/5T0UPE=; 
	b=q4xCOE7XWWKn20XxQNLuMQoDxm+gcWaJ3DhGrvQygh8Mk1YiwcvjT822/MvEXgeu5xH3O7BqgHPtrcNiMRMQJ+GXhJpvFsluOFltKSb2JY4Dmz/93U+DHs2U8PZYoM5mIcD3HuaAJYLYbP+O4Bf25n0nXPJKinr3BQiPZVy4zm3YCOA9UvpGk7gNSvTReHFwA+AnY7lMwFsQQwKbPj5PFCJKuBaEIt+UZxFIBKnbfH3HpilakNQkKuu5SIcHPcBPd+kuSk5czwo3IS/i1NPzaX+kXojmCNafZhID3ssxzdzEUjuGH9BQ/bReqRymGdk7akVmBOdX45UcnqKY7vGATA==;
Received: from void.codelabs.ru (void.codelabs.ru [144.206.177.25])
	by 0.mx.codelabs.ru with esmtpsa (TLSv1:AES256-SHA:256)
	id 1PXgCR-000Nag-Mh; Tue, 28 Dec 2010 23:31:15 +0300
Date: Tue, 28 Dec 2010 23:31:13 +0300
From: Eygene Ryabinkin <rea@freebsd.org>
To: Doug Barton <dougb@FreeBSD.org>
Message-ID: <+pyrRd3Y7AElxrhYeeViR6NWT78@QsmfhJNucgI88DfvPJdT1/nyboE>
References: <4D15D275.6000308@gmail.com> <4D194421.9080304@FreeBSD.org>
	<SaPpQ++iANujx2z50Bs5SbHA8lc@QsmfhJNucgI88DfvPJdT1/nyboE>
	<4D1A3288.70604@FreeBSD.org>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="twz1s1Hj1O0rHoT0"
Content-Disposition: inline
In-Reply-To: <4D1A3288.70604@FreeBSD.org>
Sender: rea@codelabs.ru
Cc: jhell <jhell@DataIX.net>, Peter Pentchev <roam@ringlet.net>,
	freebsd-ports@freebsd.org
Subject: Re: portmaster: print /usr/ports/UPDATING on update
X-BeenThere: freebsd-ports@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Porting software to FreeBSD <freebsd-ports.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-ports>,
	<mailto:freebsd-ports-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-ports>
List-Post: <mailto:freebsd-ports@freebsd.org>
List-Help: <mailto:freebsd-ports-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-ports>,
	<mailto:freebsd-ports-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 28 Dec 2010 20:31:37 -0000


--twz1s1Hj1O0rHoT0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Doug, good day.

Tue, Dec 28, 2010 at 10:55:04AM -0800, Doug Barton wrote:
> Do either of you actually have any familiarity at all with how portaudit=
=20
> works, and/or how it is integrated into the ports infrastructure? Based=
=20
> on what you've written today my guess is "no."

I am sorry, but you're wrong here: I am familiar with portaudit and
its internals:
  http://www.freebsd.org/cgi/query-pr.cgi?pr=3D126853

But the portaudit has very precise semantics: "if you have the port X
with versions in <range>, then it is vulnerable, here is the link to
the advisory".  UPDATING entries are slightly more complex beasts,
see below.

> When I wrote, "we need a tool with striking similarities to
> portaudit" without providing the details I was assuming that people
> are already familiar with it, how it works, etc. If you're not, then
> you really need to be before you can respond intelligently to my
> post.

Prerequisites are met from my side.

> If you take the time to become familiar with portaudit and
> subsequently need me to expand on my thoughts I'm happy to do that
> of course.

Let me explain my worries again.

What I meant is the following: one can use XML/JSON/SomeOtherMarkup
for the UPDATING.ng.  But why bother if the current UPDATING format
can be slightly extended and semantics can be attached to it without
using the new fancy languages that will need some specific ports to be
installed in order to process them?  I read your argument about
reusing the VuXML machinery; it is addressed below.

You write that
> From the user side, we're not really losing anything by not having
> "human readable" output readily available, since 99% of users will
> just want to be able to know what entries are relevant to their
> installation anyway.
but do you know how many users rely on the current UPDATING format?

Personally, I -- don't, so I can't make such assertions with
confidence.  Surely, we can have a tool that will output all entries
in the current UPDATING format -- it will save everyone who relies on
the current state of this file.

The XML in VuXML (that is used to create the portaudit database and
entries at http://www.vuxml.org/freebsd/) is mainly needed because
VuXML entries use HTML markup, so it is just easier to allow the whole
<body> tag to contain XHTML, because it can contain links, references,
lists and other markup.  So, the nested structure of XML fits here
rather good.  But, if you will suppress the complex <body> structure,
you can reformat the VuXML into the pseudo-RFC822 format and write
a simple validator for it.  XML has XSLT that can be used for transform
into HTML, so it is another reason why VuXML should be XML, but again,
mainly the body content plays its role here.  The <affects> structure
can be expressed via the RFC822-like headers -- there is no problem
to parse and validate it.

Current UPDATING is a lot simpler: it already has the pseudo-RFC822
form and it is perfectly human-readable.  What we need is the good
AFFECTS structure with clear semantics and validator for it.  I am not
saying that we can't XML here, I am just pointing out that it can be
done without XML and, thus, we don't need the dependencies on the
XML/DTD/XSLT stuff at all.

So, in my opinion, if I'll weight the pros (existing tools, standard
validation) and cons (throws a lot of tags to the reader, unsure how
to keep the current form of UPDATING's distribution) of the XML _for
UPDATING_, then I'd say that XML is redundant here.  But that's only
my opinion and it isn't neccessarily the right one; but if someone has
the other view -- it should be explained.


Thinking about portaudit, UPDATING in this form will be more-or-less
equal to the auditfile.  And currently, UPDATING has all important
properties of an auditfile, but one: AFFECTS have no clear syntax and
semantics (and UPDATING has slightly other format, but it is
consistent throughout the whole file, so it really doesn't matter).


About reusing the current VuXML machinery for UPDATING:

 - there will be just another schema for UPDATING, because VuXML is
   created for security vulnerabilities;

 - auditfile format isn't well-suited for the UPDATING, just
   because auditfile delegates the entry themselves to the
   Web server hosting the HTML'ized entries, but UPDATING should
   have the entry bodies available locally;

 - semantics of the UPDATING entries is different, so existing
   portaudit machinery should be rewritten: we can either create
   the complex tool to handle VuXML and UPDATING or to have two
   distinct tools that, perhaps, will share some code via the
   libpkg.


And here comes the next question: what syntax of AFFECTS we will need
and what semantics we will apply here.  There are at least two
cases:
 a) port X starting from version N requires user to make some actions
    before or after its installation;
 b) there were some infrastructural changes that touch big parts
    of the ports tree (or the whole tree).

Type-"a" entries should have port name and version; user should see
such entries if he upgrades the port X from version < N to the version
>=3D N.

Entries of type "b" tend to say the following: "if you had ports A, B,
C, ... installed before the date Z, then you should do so-and-so to
upgrade them without troubles".  So, user should see such entries if
either A, B, C, ... are upgraded and old version of ports being
upgraded were built before the date Z.

There can also be differences in when user should see the UPDATING
entries: before or after port building/installation.  Some entries can
say "in order to successfully build port A, you should to <smth>";
others can say "you can remove this-and-that after you had upgraded
port B".  So, the tool that will present the relevant updating
entries should have at least the following modes:

 - show me all UPDATING entries that will affect my upgrade
   (of the whole ports tree, for starters);

 - show me UPDATING entries I should read before upgrading port X;

 - show me UPDATING entries I should read after upgrading port X.


I think that the syntax/semantics problem is more-or-less orthogonal
to the matters of the UPDATING source language and tools that are used
to create/maintain it, and, to some extent, this problem is much more
vital to the business of making UPDATING entries to be
user-presentable.

Since I am trying to push the pkg_audit tool (that will intersect the
currently-installed ports and auditfile to give the caller that set of
VuXML entries that are applicable to his ports) to the base, I am
more-or-less familiar of a business of writing such a tool, so I am
taking the responsibility to write it once (and if) we will agree on
how to move UPDATING to the next stage of its life.  And I'll try
to make the architectural stage of these changes to be alive too.

But in order to move this activity any further, I'll need for a
constructive feedback.  I think that I'll try to summarize the current
thoughts at the FreeBSD Wiki, will post the link once I'll do that.

Sorry for a long letter ;))
--=20
Eygene Ryabinkin                                        ,,,^..^,,,
[ Life's unfair - but root password helps!           | codelabs.ru ]
[ 82FE 06BC D497 C0DE 49EC  4FF0 16AF 9EAE 8152 ECFB | freebsd.org ]

--twz1s1Hj1O0rHoT0
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.16 (FreeBSD)

iF4EAREIAAYFAk0aSREACgkQFq+eroFS7Pv+igD+PVDUa4lboZa9NT1y3U4Uug90
1Sfe5iN0HfQ+iiFv5CkA/257RBrNtICsGnhiQakhSXpf1c32VuLoFpDJc4aYHPaq
=O5D9
-----END PGP SIGNATURE-----

--twz1s1Hj1O0rHoT0--