From owner-freebsd-emulation@FreeBSD.ORG  Mon Feb 28 09:15:44 2011
Return-Path: <owner-freebsd-emulation@FreeBSD.ORG>
Delivered-To: emulation@FreeBSD.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 1239E1065674;
	Mon, 28 Feb 2011 09:15:44 +0000 (UTC)
	(envelope-from ehaupt@FreeBSD.org)
Received: from mx.critical.ch (cl-8.zrh-02.ch.sixxs.net
	[IPv6:2001:1620:f00:7::2])
	by mx1.freebsd.org (Postfix) with ESMTP id 84D908FC12;
	Mon, 28 Feb 2011 09:15:43 +0000 (UTC)
Received: from wiggles.bwns.ch (localhost [IPv6:::1])
	by mx.critical.ch (8.14.4/8.14.4/critical-1.0) with SMTP id
	p1S9FfB8075780; Mon, 28 Feb 2011 10:15:42 +0100 (CET)
	(envelope-from ehaupt@FreeBSD.org)
Date: Mon, 28 Feb 2011 10:15:41 +0100
From: Emanuel Haupt <ehaupt@FreeBSD.org>
To: Alexander Leidinger <Alexander@leidinger.net>
Message-Id: <20110228101541.1926b275.ehaupt@FreeBSD.org>
In-Reply-To: <20110227145125.12961olps6d8bapw@webmail.leidinger.net>
References: <20110226125814.3b0a46e5.ehaupt@FreeBSD.org>
	<20110227145125.12961olps6d8bapw@webmail.leidinger.net>
X-Mailer: Sylpheed 3.1.0 (GTK+ 2.22.1; amd64-portbld-freebsd8.2)
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Cc: emulation@FreeBSD.org
Subject: Re: MASTER_SITE quality of emulation@, freebsd-emulation@ ports
X-BeenThere: freebsd-emulation@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Development of Emulators of other operating systems
	<freebsd-emulation.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-emulation>, 
	<mailto:freebsd-emulation-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-emulation>
List-Post: <mailto:freebsd-emulation@freebsd.org>
List-Help: <mailto:freebsd-emulation-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-emulation>, 
	<mailto:freebsd-emulation-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 28 Feb 2011 09:15:44 -0000

Alexander Leidinger <Alexander@leidinger.net> wrote:
> Quoting Emanuel Haupt <ehaupt@FreeBSD.org> (from Sat, 26 Feb 2011  
> 12:58:14 +0100):
> 
> > The distilator [1] shows that roughly 97% by a total of109969 all
> > referenced distfiles fail to download. Resulting in countless
> > timeouts.
> >
> > Excerpt from the most recent run:
> >
> > Maintainer                      ok      bad     % bad
> > emulation@FreeBSD.org           1005    45696   97.85%
> > freebsd-emulation@FreeBSD.org   1691    61577   97.33%
> 
> > Ideas?
> 
> Would it help to change the order of the master site list? I assume  
> that there are some which have everything.

Unfortunately that wouldn't be much help to people who use MASTER_SORT.
I believe the best way would be to cleanup Mk/bsd.sites.mk. In
particular MASTER_SITE_FEDORA_LINUX.

ports-mgmt/distilator can help identifying those sites. For instance:

$ distilator /usr/ports/emulators/linux_base-f10/

Result:
http://people.freebsd.org/~ehaupt/misc/linux_base-f10-distilator.log.gz

A quick check would suggests to only keep:
$ zcat linux_base-f10-distilator.log.gz  | grep ^200 | awk '{print $3}' \
	| perl -e 'use URI; while(<>) { chomp(); $u=URI->new($_); print $u->host . "\n"; }' \
	| sort | uniq

archives.fedoraproject.org
ftp.quicknet.nl
ftp.rhd.ru
ftp.udl.es
mirror.steadfast.net

...but reducing MASTER_SITE_FEDORA_LINUX to those hosts shows that not even those
mirrors are consistent. Hence my prior suggestion to get a complete set of all distfiles
needed by ports using MASTER_SITE_FEDORA_LINUX and mirror those on reliable mirrors.

I can assume that task if that's fine with emulation@ :-)

Emanuel