Date: Sun, 23 Jun 2013 12:30:27 -0700 From: Peter Wemm <peter@wemm.org> To: Warner Losh <imp@bsdimp.com> Cc: src-committers@freebsd.org, Andre Oppermann <andre@freebsd.org>, John Baldwin <jhb@freebsd.org>, svn-src-all@freebsd.org, David Chisnall <theraven@freebsd.org>, Garance A Drosehn <gad@freebsd.org>, svn-src-head@freebsd.org, Tijl Coosemans <tijl@freebsd.org> Subject: Re: svn commit: r251886 - in head: contrib/apr contrib/apr-util contrib/serf contrib/sqlite3 contrib/subversion share/mk usr.bin usr.bin/svn usr.bin/svn/lib usr.bin/svn/lib/libapr usr.bin/svn/lib/libap... Message-ID: <CAGE5yCpJ2B-eNoRjABKjkJzc_hOD9Brws7ARezDyeYqcj1NgDA@mail.gmail.com> In-Reply-To: <3E89DDCB-38FA-4E7C-8F03-461516DD1871@bsdimp.com> References: <201306180253.r5I2rj45053959@svn.freebsd.org> <11DA3D8A-AD20-4DE1-B807-D09814F61947@bsdimp.com> <51C1C7BD.9060201@FreeBSD.org> <201306191113.29703.jhb@freebsd.org> <A5DB9B17-9622-4F65-A902-4984CDD82DC3@FreeBSD.org> <8D00BE2B-FD8E-4E7B-B818-1C4B7F6BB6A5@bsdimp.com> <F96228E4-16C3-4238-B54B-7E7C0C08B95B@FreeBSD.org> <68D70A89-22F2-412C-BAF4-72BEFE21EB2F@bsdimp.com> <51C5EF15.10305@FreeBSD.org> <51C660D9.8080804@FreeBSD.org> <51C6E89A.6060407@FreeBSD.org> <3E89DDCB-38FA-4E7C-8F03-461516DD1871@bsdimp.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Jun 23, 2013 at 10:12 AM, Warner Losh <imp@bsdimp.com> wrote: > On Jun 23, 2013, at 6:22 AM, Tijl Coosemans wrote: > > On 2013-06-23 04:43, Garance A Drosehn wrote: > >> On 6/22/13 2:38 PM, Tijl Coosemans wrote: > >>> On 2013-06-20 21:34, Warner Losh wrote: > >>>> > >>>> I think insisting on a definitive statement on svn lite's mission > >>>> statement is a way to obstruct progress. Sometimes you just need to > >>>> things because they feel right, not because they have been through a > >>>> 20-step approval process... > >>> > >>> For what it's worth, my reservations have always been because it > >>> didn't feel right. I never asked for an approval process. > >>> > >>> I do think there should be a tool in base that can fetch source > >>> updates and it would be nice if it could roll back changes and > >>> even nicer if it could do bisects. But svn itself is not the > >>> right tool for that. > >>> > >>> For instance, will you allow that svn is updated from version x.y > >>> to x.(y+1) in a stable branch? If yes, then users might have to run > >>> run "svn upgrade" which is a one way process, so how does importing > >>> svn allow you to roll back changes or do bisects then? If no, then > >>> who's volunteering to backport security fixes? > >> > >> What was added to the base system was 'svnlite', not 'svn' from > >> the official subversion project. The distinction is that > >> 'svnlite' is a version meant only for access to the official > >> FreeBSD repositories. 'svnlite' in the base system would only > >> be upgraded when it is needed to match the FreeBSD respository. > >> And if you need to run 'svn upgrade' to access the FreeBSD > >> repository, then it doesn't make much difference if you have > >> to do that with 'svnlite' or via the official 'svn' port. > >> > >> I'm not sure that my comments provide an answer to what you're > >> concerned about, but anyone who wants the latest version of > >> 'svn' to match their own repositories is still going to need > >> to install the port. We're not going to blindly update > >> 'svnlite' every time a new version of 'svn' is released. > >> 'svnlite' is going to be updated on *FreeBSD*'s schedule, > >> not on the schedule of the subversion project. > >> > >> It is true that we're going to have to be careful when it does > >> come time to switch to some new repo-format for the FreeBSD > >> repository. We might find ourselves in some kind of chicken- > >> and-egg situation at that point. And when we do make a > >> significant upgrade to the FreeBSD repository, then we're > >> going to have to upgrade 'svnlite' across multiple FreeBSD > >> branches at the same time, including all -stable branches. > >> But when that issue comes up it'll come up on our schedule, > >> because we'll control both 'svnlite' and the FreeBSD repo. > > > > It is precisely the other way around. Once you do a FreeBSD release > > (releng branch) that release will be stuck with the same version of > > svnlite for years. You will end up with multiple releases with > > multiple versions of svnlite that you cannot change. You have zero > > control. > > Then they will never have to do svn update, since their checked out tree will never be obsolete in relationship to the version that's installed. > > > A port on the other hand is the same for all branches and releases > > of FreeBSD. It is a single point where you have total control over > > the version of svn for all users. Conceptually, a port corresponds > > to the fact all branches and releases share the same subversion > > repo. > > Except that you still need to do svn update on trees that are checked out from old repos. > > I'm having a real hard time seeing this as an issue based on my experience with the svn port since we made the switch. Then again, I've been talking to the svn repo over http, which is independent of the version of the repo on the other end... > > Warner I've been resisting the urge to dive in. I will say right up front that a drive-by commit like this is Not The Way We Are Supposed To Do Things(TM). I know that I have ruffled some people's feathers, and for that I'm sorry. I've also got a lot of thank-you messages. I'll try and explain myself. [I know this is long. Please don't start replying until you've read it to the end.] The bottom line as to why I did this.. we are an open *source* operating system. After actually working fully from source on the freebsd.org cluster, it has become rather apparent that convenience of working with a pure-source environment has regressed considerably since 2008 when we switched from cvs. As an example of inconvenient, take this clean 24 core system with 24 GB ram. Suppose I have a problem that I want to do a bisection search to see when it first appeared (right there, freebsd-update is excluded). # rm -rf /usr/local/* /var/db/pkg/* /var/db/ports/* then a config-recursive, taking default options to match packages: # cd /usr/ports/devel/subversion; time make config-recursive (immediately hitting enter) 41.693u 4.778s 0:37.12 125.1% 30872+507k 185+13771io 360pf+0w 37 seconds just to configure the build options. # cd /usr/ports/devel/subversion; time make install [...] Installing subversion-1.8.0_1... done 902.035u 213.337s 13:53.08 133.8% 21955+423k 1282+248411io 18705pf+0w 14 minutes, 30 seconds on a reasonably fast system, not counting acquiring/extracting/updating the ports tree in the first place. (portsnap takes over 20 hours just to verify its checksums on my firewall, so please don't tell me portsnap is the solution for everything!) At the end the footprint is: # pkg info apr-1.4.6.1.4.1_3 Apache Portability Library autoconf-2.69 Automatically configure source code on many Un*x platforms autoconf-wrapper-20101119 Wrapper script for GNU autoconf automake-1.12.6 GNU Standards-compliant Makefile generator automake-wrapper-20101119 Wrapper script for GNU automake db42-4.2.52_5 The Berkeley DB package, revision 4.2 dialog4ports-0.1.5 Console Interface to configure ports expat-2.0.1_2 XML 1.0 parser written in C gdbm-1.9.1 GNU database manager gettext-0.18.1.1_1 GNU gettext package gmake-3.82_1 GNU version of 'make' utility help2man-1.43.2 Automatically generating simple manual pages from program output libiconv-1.14_1 A character set conversion library libtool-2.4.2 Generic shared library support script m4-1.4.16_1,1 GNU m4 p5-Locale-gettext-1.05_3 Message handling functions perl-5.14.4 Practical Extraction and Report Language pkg-1.0.14 New generation package manager pkgconf-0.9.2_1 Utility to help to configure compiler and linker flags python27-2.7.5_1 Interpreted object-oriented programming language serf-1.2.1 Serf HTTP client library sqlite3-3.7.17_1 SQL database engine in a C library subversion-1.8.0_1 Version control system highlights: db4.2(!!wtf?), gdbm, perl, python, make, m4 du -sh says: 238M /usr/local The same machine (WITHOUT_CLANG) builds world *and* its kernel in: 3570.850u 845.997s 13:29.91 545.3% 6978+1075k 28182+1449464io 21379pf+0w Same again but with clang enabled (and it's the default compiler. with maximum working -j): 8276.034u 746.466s 31:24.78 478.7% 29824+610k 25348+1683059io 36894pf+0w And svnlite, *with* clang to get the worst case: /usr/src/usr.bin/svn# time make -s -j24 depend all 117.943u 15.330s 0:42.74 311.8% 32804+522k 2+37666io 1231pf+0w du -sh /usr/bin/svnlite* says 18M In summary, on the same machine, no third party binaries: ?? acquire ports tree (talk to my soekris firewall for worst-case examples) ?? deal with perl directory layout changes (actually pretty quick if you just rm -rf /usr/local and start over) 14 minutes 30 seconds to build from svn ports vs 13 minutes 30 seconds to buildworld without clang 31 minutes 24 seconds to buildworld with clang vs 42 seconds to build svnlite (not actually lite, fully functional) Don't get me wrong, I love what pkgng brings to the table. But as a "source code" consumer, it makes my skin crawl each time somebody effectively says "oh, you shouldn't build from source, you should install binaries to solve the problem". Every time I hear that it feels like we're giving up. If I wanted binaries, I'd be running a binary-only OS like Windows, Linux or OSX. As for compatability and old releases: Your svn-1.4 / 1.5 binaries from years ago still work on the project's svn-1.8 infrastructure. svnlite doesn't change what "svn up" does on your $path. svnlite doesn't interfere with ports in ANY WAY. If your perl install broke and don't want to deal with it right now, you don't have to. svnlite doesn't interfere with your 1.5 or 1.6 checkouts unless you type it by name or compile it as "svn" with -DWITH_SVN=yes What it brings: svnlite restores the 'at-your-fingertips' convenience that we kind of lost when we switched from cvs in 2008. You can install an image in a VM or whatever and instantly participate in development. It brings WITH_SVN=yes in make.conf so you can get finger memory compatibility if you wish. I still want svnup to be finished and imported. Just like when we had cvs and csup, they're targeted at different people. If a few extra seconds of buildworld time bother you, there *is* a knob to turn it off. But if we need to talk about that then we also need to talk about multiple compilers, multiple installers, multiple firewalls, multiple command line editors, multiple nfs servers, multiple nfs clients, obsolete dns stacks, etc etc. The bottom line is that this is as close to "free" as it can get. It adds options for people. It does not limit or restrict people's options It does not interfere with people's existing svn checkouts (unless they want to, options!) And I am not willing to back down on it... But I do apologize for the hypocrisy in committing a drive-by and the upset that it caused a few people. I don't know about everyone else but I've now spent more time talking about this than I'll spend waiting for it to compile in the rest of my life using FreeBSD as a source-code-based OS. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAGE5yCpJ2B-eNoRjABKjkJzc_hOD9Brws7ARezDyeYqcj1NgDA>