Date: Wed, 17 Oct 2007 17:41:42 GMT From: Fabian Keil <fk@fabiankeil.de> To: freebsd-gnats-submit@FreeBSD.org Subject: bin/117277: [PATCH] fetch's resume mode doesn't verify that it actually got partial content Message-ID: <200710171741.l9HHfgdE070161@www.freebsd.org> Resent-Message-ID: <200710171750.l9HHo1UW092762@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 117277 >Category: bin >Synopsis: [PATCH] fetch's resume mode doesn't verify that it actually got partial content >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Wed Oct 17 17:50:01 UTC 2007 >Closed-Date: >Last-Modified: >Originator: Fabian Keil >Release: RELENG_7 >Organization: >Environment: FreeBSD TP51.local 7.0-PRERELEASE FreeBSD 7.0-PRERELEASE #1: Mon Oct 15 18:31:16 CEST 2007 fk@TP51.local:/usr/obj/usr/src/sys/THINKPAD i386 >Description: When resuming a file over HTTP, fetch(1) treats complete retransmits like partial content and appends the whole response to the already existing part that was fetched earlier. The problem shows if the server doesn't support Range requests or if the proxy strips out the Range header to prevent range mismatches (which could happen if the proxy modified the first response): fk@TP51 /tank/fetch-tests $fetch -r http://10.0.0.1/BrooksDavis.EuroBSDCon.2007.avi BrooksDavis.EuroBSDCon.2007.avi 30% of 92 MB 8440 kBps^C fetch: transfer interrupted fk@TP51 /tank/fetch-tests $fetch -r http://10.0.0.1/BrooksDavis.EuroBSDCon.2007.avi BrooksDavis.EuroBSDCon.2007.avi 100% of 92 MB 17 MBps fk@TP51 /tank/fetch-tests $ls -lh total 123649 -rw-r--r-- 1 fk wheel 121M Sep 25 21:01 BrooksDavis.EuroBSDCon.2007.avi >How-To-Repeat: Resume a file from a HTTP server that doesn't support Range requests or with a proxy that removes the Range header. >Fix: While fetch(3) doesn't pass the HTTP status code to fetch(1), the problem can be solved by checking the content offset. If the offset is zero, the response contains the whole file and the already existing part of it has to be overwritten. With the attached patch it works as expected: fk@TP51 /tank/fetch-tests $rm BrooksDavis.EuroBSDCon.2007.avi fk@TP51 /tank/fetch-tests $fetch -r http://10.0.0.1/BrooksDavis.EuroBSDCon.2007.avi BrooksDavis.EuroBSDCon.2007.avi 58% of 92 MB 15 MBps^C fetch: transfer interrupted fk@TP51 /tank/fetch-tests $fetch -r http://10.0.0.1/BrooksDavis.EuroBSDCon.2007.avi BrooksDavis.EuroBSDCon.2007.avi 100% of 92 MB 13 MBps fk@TP51 /tank/fetch-tests $ls -lh total 95081 -rw-r--r-- 1 fk wheel 93M Sep 25 21:01 BrooksDavis.EuroBSDCon.2007.avi Patch attached with submission follows: --- .zfs/snapshot/2007-10-15/usr.bin/fetch/fetch.c 2006-11-10 23:05:41.000000000 +0100 +++ usr.bin/fetch/fetch.c 2007-10-16 14:21:20.221581714 +0200 @@ -488,8 +488,11 @@ if (o_stdout) { /* output to stdout */ of = stdout; - } else if (r_flag && sb.st_size != -1) { - /* resume mode, local file exists */ + } else if (r_flag && sb.st_size != -1 && url->offset) { + /* + * resume mode, local file exists and we + * actually received partial content as requested + */ if (!F_flag && us.mtime && sb.st_mtime != us.mtime) { /* no match! have to refetch */ fclose(f); >Release-Note: >Audit-Trail: >Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200710171741.l9HHfgdE070161>