From owner-freebsd-ports@FreeBSD.ORG  Thu Jul 22 10:17:35 2010
Return-Path: <owner-freebsd-ports@FreeBSD.ORG>
Delivered-To: ports@FreeBSD.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id CD426106567A
	for <ports@FreeBSD.org>; Thu, 22 Jul 2010 10:17:35 +0000 (UTC)
	(envelope-from ehaupt@FreeBSD.org)
Received: from mx.critical.ch (cl-8.zrh-02.ch.sixxs.net
	[IPv6:2001:1620:f00:7::2])
	by mx1.freebsd.org (Postfix) with ESMTP id 649738FC08
	for <ports@FreeBSD.org>; Thu, 22 Jul 2010 10:17:35 +0000 (UTC)
Received: from beaver.home.critical.ch (80-218-148-45.dclient.hispeed.ch
	[80.218.148.45]) (authenticated bits=0)
	by mx.critical.ch (8.14.3/8.14.3/critical-1.0) with ESMTP id
	o6MAHX5D025211
	for <ports@FreeBSD.org>; Thu, 22 Jul 2010 12:17:34 +0200 (CEST)
	(envelope-from ehaupt@FreeBSD.org)
Date: Thu, 22 Jul 2010 12:17:31 +0200
From: Emanuel Haupt <ehaupt@FreeBSD.org>
To: ports@FreeBSD.org
Message-Id: <20100722121731.34b47a2a.ehaupt@FreeBSD.org>
X-Mailer: Sylpheed 3.0.3 (GTK+ 2.20.1; amd64-portbld-freebsd8.1)
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Cc: 
Subject: MASTER_SITE defined by google search result
X-BeenThere: freebsd-ports@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Porting software to FreeBSD <freebsd-ports.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-ports>,
	<mailto:freebsd-ports-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-ports>
List-Post: <mailto:freebsd-ports@freebsd.org>
List-Help: <mailto:freebsd-ports-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-ports>,
	<mailto:freebsd-ports-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Jul 2010 10:17:35 -0000

Usually when I find that a port can't fetch the according distfile I
simply google for the distfile with the following search string (e.g.
for rsync):

	intitle:"index of" rsync-3.0.7.tar.gz

This approach almost always delivers plenty of results. I then manually
download one of the results into my DISTDIR and restart the normal
build process which then automatically takes care of the integrity
(checksum, size) of the distfile(s).

I thought, why not automate this and I came up with a simple perl
script (requires www/p5-Google-Search):

http://people.freebsd.org/~ehaupt/distfile-search/distfile-search

Adding the following make magic to /etc/make.conf

# distfile search
.if defined(DISTFILE_SEARCH_BACKUP)
pre-fetch-script:
MASTER_SITE_BACKUP+=    `/root/bin/distfile-search ${ALLFILES}`
.elif defined(DISTFILE_SEARCH_OVERRIDE)
pre-fetch-script:
MASTER_SITE_OVERRIDE=   `/root/bin/distfile-search ${ALLFILES}`
.endif

will let you choose whether you want to overwrite the pre defined
MASTER_SITES:

root@foo: /usr/ports/x11/xorg/ $ make -DDISTFILE_SEARCH_OVERRIDE fetch

or add the search result as backup MASTER_SITES:

root@foo: /usr/ports/x11/xorg/ $ make -DDISTFILE_SEARCH_BACKUP fetch

In the later example you even might want to define
DISTFILE_SEARCH_BACKUP globally.

As a proof of concept I was able to download all distfiles for x11/xorg
with this method (make -DDISTFILE_SEARCH_OVERRIDE fetch-recursive).

Please let me know what you think.

Emanuel