From owner-freebsd-questions@FreeBSD.ORG Sun Jul 19 06:11:57 2009 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85DF3106564A for ; Sun, 19 Jul 2009 06:11:57 +0000 (UTC) (envelope-from jjah@cloud.ccsf.cc.ca.us) Received: from cloud.ccsf.cc.ca.us (cloud.ccsf.cc.ca.us [147.144.1.212]) by mx1.freebsd.org (Postfix) with ESMTP id 6BF098FC0A for ; Sun, 19 Jul 2009 06:11:57 +0000 (UTC) (envelope-from jjah@cloud.ccsf.cc.ca.us) Received: from cloud.ccsf.cc.ca.us (localhost.ccsf.cc.ca.us [127.0.0.1]) by cloud.ccsf.cc.ca.us (8.14.2/8.14.2) with ESMTP id n6J6BwtW011128; Sat, 18 Jul 2009 23:11:58 -0700 (PDT) (envelope-from jjah@cloud.ccsf.cc.ca.us) Received: from localhost (jjah@localhost) by cloud.ccsf.cc.ca.us (8.14.2/8.14.2/Submit) with ESMTP id n6J6BwuS011125; Sat, 18 Jul 2009 23:11:58 -0700 (PDT) (envelope-from jjah@cloud.ccsf.cc.ca.us) Date: Sat, 18 Jul 2009 23:11:58 -0700 (PDT) From: "Joe R. Jah" To: Andrew Brampton In-Reply-To: Message-ID: <20090718230601.F10250@cloud.ccsf.cc.ca.us> References: <20090717144049.C35992@cloud.ccsf.cc.ca.us> <20090718093237.Y19472@cloud.ccsf.cc.ca.us> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="0-306400255-1247983918=:10250" Cc: freebsd-questions@freebsd.org Subject: Re: OT: wget bug X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Jul 2009 06:11:57 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --0-306400255-1247983918=:10250 Content-Type: TEXT/PLAIN; charset=UTF-8 Content-Transfer-Encoding: QUOTED-PRINTABLE On Sat, 18 Jul 2009, Andrew Brampton wrote: > Date: Sat, 18 Jul 2009 18:09:54 +0100 > From: Andrew Brampton > To: Joe R. Jah > Cc: freebsd-questions@freebsd.org > Subject: Re: OT: wget bug > > 2009/7/18 Joe R. Jah : > > Thank you Andrew. =C2=A0Yes the server is truly returning 401. =C2=A0I = have already > > reconfigured wget to download everything regardless of their timestamp, > > but it's a waste of bandwidth, because most of the site is unchanged. > > > > Do you know of any workaround in wget, or an alternative tool to ONLY > > download newer files by http? > > > > Joe, > There are two ways to check if the file has been changed. One, read > the time the file was last changed, or two, read the file and compare > it to a old copy. Wget was obviously trying to do option 1 but this is > denied by the remote server. You most likely could get it to do option > 2, however by doing so you are wasting bandwidth downloading unchanged > files just to check if they had been changed. > > If you have control over the remote webserver, then the simplest way > to solve this problem is to configure the webserver not to return 401 > when wget sends the If-Modified-Since HTTP header. A better solution, > again assuming you have control of the remote server, is to use > "rsync" as it is designed for this kind of task. > > If you don't have control over the remote server, then you are stuck > with your current solution. > > Andrew Thank you Andrew. Regards, Joe --=20 _/ _/_/_/ _/ ____________ __o _/ _/ _/ _/ ______________ _-\<,_ _/ _/ _/_/_/ _/ _/ ......(_)/ (_) _/_/ oe _/ _/. _/_/ ah jjah@cloud.ccsf.cc.ca.us --0-306400255-1247983918=:10250--