From owner-freebsd-current@FreeBSD.ORG  Sun Dec 10 01:49:25 2006
Return-Path: <owner-freebsd-current@FreeBSD.ORG>
X-Original-To: freebsd-current@freebsd.org
Delivered-To: freebsd-current@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id B980616A403;
	Sun, 10 Dec 2006 01:49:25 +0000 (UTC)
	(envelope-from mcdouga9@daemon.egr.msu.edu)
Received: from daemon.egr.msu.edu (daemon.egr.msu.edu [35.9.44.65])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 32BA743C9D;
	Sun, 10 Dec 2006 01:48:18 +0000 (GMT)
	(envelope-from mcdouga9@daemon.egr.msu.edu)
Received: by daemon.egr.msu.edu (Postfix, from userid 21281)
	id 050CF1CE6F; Sat,  9 Dec 2006 20:49:25 -0500 (EST)
Date: Sat, 9 Dec 2006 20:49:24 -0500
From: Adam McDougall <mcdouga9@egr.msu.edu>
To: Colin Percival <cperciva@freebsd.org>
Message-ID: <20061210014924.GU81923@egr.msu.edu>
References: <20061210010823.GS81923@egr.msu.edu> <457B621E.3020100@freebsd.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <457B621E.3020100@freebsd.org>
User-Agent: Mutt/1.5.13 (2006-08-11)
Cc: freebsd-current@freebsd.org
Subject: Re: Fwd: Re: pf: BAD state happens often with portsnap fetch update
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
	<freebsd-current.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
	<mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>,
	<mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 10 Dec 2006 01:49:25 -0000

On Sat, Dec 09, 2006 at 05:25:50PM -0800, Colin Percival wrote:

  Adam McDougall wrote:
  > # portsnap fetch update
  > [...]
  > Fetching 2688 new ports or files... /usr/sbin/portsnap: cannot open 
  > 3f115cb168a8e51fd0d19798f005ab7a251a1de6a5b9eda60cd327b60aa48799.gz: No such file or 
  > directory
  > snapshot is corrupt.
  > 
  > 2597 should have been fetched, but there was a stall at 30.. and after about a minute,
  > it continued on to 410...... and gave up apparently.  For all my servers without
  > direct internet access, I have to run portsnap several times until it succeeds.
  
  You have four options:
  (a) Lower pf's tcp.closed timeout,
  (b) Increase the high port range,
  (c) Fix squid so that it groks HTTP/1.1 properly, or
  (d) Stop using squid.
  
  The problem here is that your proxy is closing portsnap's HTTP connection after
  each file is downloaded.
  
I just tested tcp.closed with 3 seconds, down from 15 earlier but both were 
unsuccessful.  I will look at the other options as well, but do you have any explanation 
for why portsnap would use wildly randomish local ports that overlap too quickly
when fetch does not?  Is that a kernel controlled behavior that I can adjust?

65535-49152=16383 (unless I am looking at the wrong sysctl for the default values)
and I would think there are many less connections involved for 2597 fetches than
16363.  If 16363 isn't enough ports for 2597 fetches then it seems like a crapshoot to 
try doubling or tripling the range.