Date: Mon, 22 Aug 2016 21:40:12 +0000 From: bugzilla-noreply@freebsd.org To: freebsd-bugs@FreeBSD.org Subject: [Bug 212065] fetch -r fails on a complete file, even with -S 12345 (OS version independent) Message-ID: <bug-212065-8@https.bugs.freebsd.org/bugzilla/>
index | next in thread | raw e-mail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=212065 Bug ID: 212065 Summary: fetch -r fails on a complete file, even with -S 12345 (OS version independent) Product: Base System Version: 10.3-RELEASE Hardware: Any OS: Any Status: New Severity: Affects Some People Priority: --- Component: bin Assignee: freebsd-bugs@FreeBSD.org Reporter: mandree@FreeBSD.org When fetch(1) is used to resume (-r) on a completely downloaded file, it erroneously fails with exit status 1. It should instead detect that condition "file is complete and same size as on server" and exit with status 0 (= success). Adding to the -r option also a -S 12345 option, where -S is the same size as the file locally has, and as the server thinks its file is, does NOT help. Using "-m" is not an option if the file date isn't reported by the server. This bug has been reported on #bsdports (IRC) against 9.3 with portsnap, and can be verified without portsnap on 10.3-RELEASE-p7. Commented verbose fetch output on a hacked "portsnap fetch" downloading the initial snapshot as of today. First, see we have the file: $ ls -l *.tgz -rw-r--r-- 1 root wheel 74481909 Aug 22 23:03 80c30ef3c99969de68fad7d92a47c4e69cd6f1c8135504152d794c21305b1245.tgz Next, ask server for the size: $ fetch -s http://ec2-eu-west-1.portsnap.freebsd.org/s/80c30ef3c99969de68fad7d92a47c4e69cd6f1c8135504152d794c21305b1245.tgz ; echo $? 74481909 0 So, sizes match. File is complete. Now, try to resume the transfer, verbose mode: $ fetch -rvv http://ec2-eu-west-1.portsnap.freebsd.org/s/80c30ef3c99969de68fad7d92a47c4e69cd6f1c8135504152d794c21305b1245.tgz ; echo $? [...] ---> ec2-eu-west-1.portsnap.freebsd.org:80 looking up ec2-eu-west-1.portsnap.freebsd.org connecting to ec2-eu-west-1.portsnap.freebsd.org:80 requesting http://ec2-eu-west-1.portsnap.freebsd.org/s/80c30ef3c99969de68fad7d92a47c4e69cd6f1c8135504152d794c21305b1245.tgz >>> GET /s/80c30ef3c99969de68fad7d92a47c4e69cd6f1c8135504152d794c21305b1245.tgz HTTP/1.1 >>> Host: ec2-eu-west-1.portsnap.freebsd.org >>> Accept: */* >>> User-Agent: fetch libfetch/2.0 >>> Range: bytes=74481909- >>> Connection: close >>> <<< HTTP/1.1 416 Requested Range Not Satisfiable <<< Content-Type: text/html <<< Accept-Ranges: bytes <<< Content-Length: 389 <<< Connection: close content length: [389] <<< Date: Mon, 22 Aug 2016 21:07:00 GMT <<< Server: lighttpd/1.4.33 <<< fetch: http://ec2-eu-west-1.portsnap.freebsd.org/s/80c30ef3c99969de68fad7d92a47c4e69cd6f1c8135504152d794c21305b1245.tgz: Requested Range Not Satisfiable 1 OOPS! See that -S doesn't help, same size as above: $ fetch -S 74481909 -r http://ec2-eu-west-1.portsnap.freebsd.org/s/80c30ef3c99969de68fad7d92a47c4e69cd6f1c8135504152d794c21305b1245.tgz ; echo $? fetch: http://ec2-eu-west-1.portsnap.freebsd.org/s/80c30ef3c99969de68fad7d92a47c4e69cd6f1c8135504152d794c21305b1245.tgz: Requested Range Not Satisfiable 1 NOTE: for proper robustness, fetch might want to issue a HEAD request to obtain the file size unconditionally when resuming, so as to be able to tell the condition "file shrunk on server" apart from "file has been completely downloaded". Other tools: * curl 7.50.1 from ports fails in a similar way, and to add insult to injury, wrongly concludes that the server did not support byte ranges when in fact it does (but cannot start a download beyond the end of a file). * wget 1.18 also sees the 416 code from the server, but concludes "The file is already fully retrieved; nothing to do." and exits with status 0. * lftp 4.7.3 with "get -c" does the right thing and exits silently if the file is complete. -- You are receiving this mail because: You are the assignee for the bug.help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-212065-8>