Date: Mon, 01 Apr 2013 01:34:50 -0700 From: "Constantine A. Murenin" <cnst++@FreeBSD.org> To: FreeBSD-current@FreeBSD.org, FreeBSD-hackers@FreeBSD.org Subject: BXR.SU, Super User's BSD Cross Reference w/ OpenGrok, publicly private beta Message-ID: <515946AA.2060703@FreeBSD.org>
next in thread | raw e-mail | index | archive | help
Dear FreeBSD-{current,hackers}@,
It is my great pleasure to announce the immediate availability of a
publicly private IPv6-only beta test of BXR.SU -- Super User's BSD Cross
Reference.
BXR.SU is based on an OpenGrok fork, but it's more than just OpenGrok.
We've fixed a number of annoyances, eliminated features that just never
worked right from the outright, and provided integration with tools like
CVSweb (including awesome mirrors like allbsd.org), FreeBSD's ViewVC
(SVN), as well as Gitweb from git.freebsd.your.org, plus a tad of other
improvements, including a complete rewrite of an mdoc parser. Last, but
definitely not least, is an extensive set of nginx rewrite rules that
makes it a breeze to use BXR.SU as a deterministic URL compactor for
referencing BSD source code.
What's up with the publicly private beta test?
We're launching today in a publicly private beta. Participation in the
beta is invitation-only; everyone with IPv6 is invited.
We're cooperating with ISPs around the world, and in order to be able to
access BXR.SU during this beta phase, you must have a special token,
also known as a publicly routable IPv6-address, with proper
IPv6-connectivity and upstream peering. If you don't have IPv6 yet, but
want to participate in this beta test ASAP, then ask your ISP for IPv6
ASAP! Else, if your ISP is not part of our beta rollout, you could try
something like tunnelbroker.net from he.net.
What's the release schedule?
BXR.SU is available through IPv6 today, 2013-04-01. It is currently an
IPv6-only site, with an IPv6-only glue, too.
As an IPv6-only site, we hereby declare that 2013-04-04 is an IPv4 day.
On April 4, we will temporarily enable IPv4 connectivity, for one day,
to test the water. (We've heard that IPv4 has some connectivity issues
related to NAT, double-NAT, carrier-grade NAT and NAT64, and some small
percentage of users (but significant in absolute terms) might not be
able to access the site if an A record is published, due to the
plentiful of misconfigurations out there; so, we want to take things
slow, and ensure our users don't suffer from any inferior connectivity.)
If things do go well (we expect IPv6/IPv4-related improvements as time
goes by), we will permanently publish an A record for BXR.SU on 2013-04-14.
IPv4 glue records will be published shortly thereafter, on 2013-04-24
(we don't do this today, because we're afraid that the nameservers of
some ISPs are not configured correctly, and our IPv6 users won't be able
to access our site otherwise, so, we think it's a good idea to take
things slow and in steps).
But why another OpenGrok?
Over the years, there have been a number of OpenGrok installations that
have made it possible to study and grok BSD code, for which we are very
thankful to their maintainers. However, as a general rule, none of them
have been inclusive of all BSD flavours, all of them have had rather
long and hard-to-remember URLs, which also didn't really look permanent
at all, and, unfortunately, many of them no longer exist today, or some
new uber-inclusive services like code.metager.de have recently
flourished, with an astounding 8 second (yes, eight second) delay for
satisfying any single search query (hot queries are returned in as
little as just under 4 seconds by metager, yet everything is nonetheless
buffered, so, you get no rendering at all for those whole 4 or 8
seconds). So, we thought this had to change.
So, what's the deal?
It's simple.
Say, someone doesn't know who PHK is. You can point them to:
http://bxr.su/s?q=phk
Want to see if DragonFly keeps queue.h in sync? Take a look at:
http://bxr.su/d/sys/sys/queue.h
Want to look at FreeBSD's queue.h, to manually compare? Just change the
"d" from "/d/" (or select and replace the whole world "DragonFly" from
"/DragonFly/") to "f", and you're in FreeBSD:
http://bxr.su/f/sys/sys/queue.h
Too many /sys/sys/? We've still got you covered, thanks to nginx:
http://bxr.su/o/queue.h
Anyone uses TAILQ_SWAP? Is that a new thing? Check it out:
http://bxr.su/search?q=TAILQ_SWAP
Any mentions of "OpenBSD" or "NetBSD" in FreeBSD and DragonFly?
http://bxr.su/f,d/s?q=OpenBSD+OR+NetBSD
Who's this guy writing this email anyway? Is he BXR'able?
http://bxr.su/s?q=%22Constantine%20A.%20Murenin%22%20OR%20cnst
Etc.
Just how fast is BXR.SU?
We expect that most search requests should be fulfilled (search page
results generated) in well under 100ms. In my tests, and according to
OpenGrok metrics at the bottom of each search page, most search pages
are generated in about 30 to 50ms, so, it does seem like there's some
room to spare. In addition, of course we use nginx, so, once generated
at 40ms, a page should be available immediately in no time should a
subsequent identical request come along within a couple of seconds or so.
How does it compare with fxr.watson.org?
+ we're based on OpenGrok, instead of LXR
+ we also index userland of all BSDs, not just the kernels like the fxr
+ we do daily updates and re-index of all 4 BSDs
- we only track -CURRENT of each BSD
+ the -current that we track is actually current
Is this a replacement to nxr.netbsd.org?
Not necessarily. We've noted that the /gnu/ and /contrib/ directories
of various BSDs have been constantly giving out lots of false positives
on all kinds of search queries, and they have been excluded, for both
the accuracy and disc space utilisation. This only affects userland,
and mostly excludes stuff that noone really cares about anyways (some
exceptions were made, and this is not really in stone anyways).
Otherwise, BXR.SU is much faster than the nxr -- nxr search is
unbuffered, but takes several seconds on cold runs, and about half a
second on hot runs.
How does it compare with code.metager.de?
+ we're over 20000% faster (0.04s vs. 8s). 'nuff said.
I welcome comments, questions and suggestions. I hope that this site
will be useful to the BSD community, and will join other great and
invaluable BSD-related sites that we all use and depend on.
In summary, what's unique about bxr.su?
* Supports all 4 BSDs
* Daily updates
* Short URLs, deterministic URL shortener for BSD source code
* Kernel + non-gnu userland
* Very fast -- most search pages generated in under 50ms
* IPv6-only for now, will be dual-stacked very soon
Enjoy!
Constantine.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?515946AA.2060703>