From owner-freebsd-stable@FreeBSD.ORG Mon Oct 20 20:23:06 2014 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 34602ECF for ; Mon, 20 Oct 2014 20:23:06 +0000 (UTC) Received: from mail.oitsec.umn.edu (mail.oitsec.umn.edu [128.101.238.120]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mail.oitsec.umn.edu", Issuer "InCommon Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0E2D73E4 for ; Mon, 20 Oct 2014 20:23:05 +0000 (UTC) Received: from mail.oitsec.umn.edu (localhost [127.0.0.1]) by mail.oitsec.umn.edu (Postfix) with ESMTP id 02DFD5C80C for ; Mon, 20 Oct 2014 15:20:07 -0500 (CDT) X-Virus-Scanned: amavisd-new at oitsec.umn.edu Received: from mail.oitsec.umn.edu ([127.0.0.1]) by mail.oitsec.umn.edu (mail.oitsec.umn.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wTMjf8Trfxhj for ; Mon, 20 Oct 2014 15:20:06 -0500 (CDT) Received: from optimator.oitsec.umn.edu (optimator.oitsec.umn.edu [134.84.23.1]) (Authenticated sender: amesbury) by mail.oitsec.umn.edu (Postfix) with ESMTPSA id 76D225C824 for ; Mon, 20 Oct 2014 15:20:06 -0500 (CDT) From: Alan Amesbury Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Problem with libfetch, pkg, and proxying? Message-Id: <42CAA1B4-1DE8-4CA6-85A4-29773844B0E2@oitsec.umn.edu> Date: Mon, 20 Oct 2014 15:20:06 -0500 To: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 8.0 \(1990.1\)) X-Mailer: Apple Mail (2.1990.1) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Oct 2014 20:23:06 -0000 Given FreeBSD-9.1-RELEASE, 'pkg' installed from ports, and a pkg.conf = that points to a proxy, it appears 'pkg' is ignoring the proxy setting = for HTTPS URLs. The contents of /usr/local/etc/pkg.conf consists of: pkg_env { http_proxy: http://proxyhost.fqdn:3128/ } 'uname -srm' =3D "FreeBSD 9.1-RELEASE-p19 amd64". It's not running = GENERIC, but I don't think that's relevant. :-) Network traffic shows the host uses the proxy correctly for the initial = HTTP callout to the local package repository, but tries to connect = directly when it receives an HTTP redirect to HTTPS. This is borne out = in output from 'truss', which shows (with some data redacted): . . . 72869: connect(5,{ AF_INET [NAMESERVER]:53 },16) =3D 0 (0x0) 72869: sendto(5,"\M-)W\^A\0\0\^A\0\0\0\0\0\0\apro"...,44,0x0,NULL,0x0) =3D= 44 (0x2c) 72869: clock_gettime(0,{1413835372.386244672 }) =3D 0 (0x0) 72869: = kevent(4,{0x5,EVFILT_READ,EV_ADD|EV_ONESHOT,0,0x0,0x0},1,{0x5,EVFILT_READ,= EV_ONESHOT,0,0xcb,0x0},1,{5.000000000 }) =3D 1 (0x1) 72869: recvfrom(5,"\M-)W\M^A\M^@\0\^A\0\^A\0\^B\0"...,65536,0x0,{ = AF_INET 128.101.101.101:53 },0x7fffffff77dc) =3D 203 (0xcb) 72869: close(5) =3D 0 (0x0) 72869: close(4) =3D 0 (0x0) 72869: = kqueue(0x7e6bfa380,0x7e7496000,0x10000058,0x7e7486000,0x10000,0x1) =3D 4 = (0x4) 72869: socket(PF_INET,SOCK_DGRAM,0) =3D 5 (0x5) 72869: connect(5,{ AF_INET [NAMESERVER]:53 },16) =3D 0 (0x0) 72869: sendto(5,"\M-)X\^A\0\0\^A\0\0\0\0\0\0\apro"...,44,0x0,NULL,0x0) =3D= 44 (0x2c) 72869: clock_gettime(0,{1413835372.388397497 }) =3D 0 (0x0) 72869: = kevent(4,{0x5,EVFILT_READ,EV_ADD|EV_ONESHOT,0,0x0,0x0},1,{0x5,EVFILT_READ,= EV_ONESHOT,0,0x69,0x0},1,{5.000000000 }) =3D 1 (0x1) 72869: recvfrom(5,"\M-)X\M^A\M^@\0\^A\0\0\0\^A\0\0"...,65536,0x0,{ = AF_INET 128.101.101.101:53 },0x7fffffff77dc) =3D 105 (0x69) 72869: close(5) =3D 0 (0x0) 72869: close(4) =3D 0 (0x0) 72869: madvise(0x7e7496000,0x10000,0x5,0x95,0x7fffffff7830,0x62c1b0) =3D = 0 (0x0) 72869: madvise(0x7e7476000,0x10000,0x5,0x75,0x7fffffff7d10,0xffffffff) =3D= 0 (0x0) 72869: madvise(0x7e7486000,0x10000,0x5,0x85,0x7fffffff7d10,0x62c1b0) =3D = 0 (0x0) 72869: socket(PF_INET,SOCK_STREAM,6) =3D 4 (0x4) 72869: connect(4,{ AF_INET [PROXY]:3128 },16) =3D 0 (0x0) 72869: fcntl(4,F_SETFL,O_NONBLOCK) =3D 0 (0x0) 72869: fcntl(4,F_SETFD,FD_CLOEXEC) =3D 0 (0x0) 72869: setsockopt(0x4,0xffff,0x800,0x7fffffff9144,0x4,0x0) =3D 0 (0x0) 72869: setsockopt(0x4,0x6,0x4,0x7fffffff9458,0x4,0x0) =3D 0 (0x0) . . . 72869: connect(5,{ AF_INET [NAMESERVER]:53 },16) =3D 0 (0x0) 72869: sendto(5,"\M-)Y\^A\0\0\^A\0\0\0\0\0\0\thor"...,42,0x0,NULL,0x0) =3D= 42 (0x2a) 72869: clock_gettime(0,{1413835372.458693385 }) =3D 0 (0x0) 72869: = kevent(4,{0x5,EVFILT_READ,EV_ADD|EV_ONESHOT,0,0x0,0x0},1,{0x5,EVFILT_READ,= EV_ONESHOT,0,0xc9,0x0},1,{5.000000000 }) =3D 1 (0x1) 72869: recvfrom(5,"\M-)Y\M^A\M^@\0\^A\0\^A\0\^B\0"...,65536,0x0,{ = AF_INET 128.101.101.101:53 },0x7fffffff77dc) =3D 201 (0xc9) 72869: close(5) =3D 0 (0x0) 72869: close(4) =3D 0 (0x0) 72869: = kqueue(0x7e6bfa380,0x7e7496000,0x10000058,0x7e7486000,0x10000,0x1) =3D 4 = (0x4) 72869: socket(PF_INET,SOCK_DGRAM,0) =3D 5 (0x5) 72869: connect(5,{ AF_INET [NAMESERVER]:53 },16) =3D 0 (0x0) 72869: sendto(5,"\M-)Z\^A\0\0\^A\0\0\0\0\0\0\thor"...,42,0x0,NULL,0x0) =3D= 42 (0x2a) 72869: clock_gettime(0,{1413835372.461001593 }) =3D 0 (0x0) 72869: = kevent(4,{0x5,EVFILT_READ,EV_ADD|EV_ONESHOT,0,0x0,0x0},1,{0x5,EVFILT_READ,= EV_ONESHOT,0,0x67,0x0},1,{5.000000000 }) =3D 1 (0x1) 72869: recvfrom(5,"\M-)Z\M^A\M^@\0\^A\0\0\0\^A\0\0"...,65536,0x0,{ = AF_INET 128.101.101.101:53 },0x7fffffff77dc) =3D 103 (0x67) 72869: close(5) =3D 0 (0x0) 72869: close(4) =3D 0 (0x0) 72869: madvise(0x7e7496000,0x10000,0x5,0x95,0x7fffffff7830,0x62c1b0) =3D = 0 (0x0) 72869: madvise(0x7e7476000,0x10000,0x5,0x75,0x7fffffff7d10,0xffffffff) =3D= 0 (0x0) 72869: madvise(0x7e7486000,0x10000,0x5,0x85,0x7fffffff7d10,0x62c1b0) =3D = 0 (0x0) 72869: socket(PF_INET,SOCK_STREAM,6) =3D 4 (0x4) 72869: connect(4,{ AF_INET [NOT_PROXY]:443 },16) ERR#60 'Operation timed = out' . . . The connection timed out because connections to hosts other than the = proxy aren't allowed. However, my reading of fetch(3) and fetch(1) = suggests that the environment variable for http_proxy should cover HTTP = and HTTPS URLs. Tests using lynx were different; lynx apparently uses = ${PROTOCOL}_PROXY where ${PROTOCOL} is the URL type, and HTTP and HTTPS = are different. Is this behavior correct? I don't think it is. Regardless, is there a = way to get 'pkg' to use HTTPS URLs through a proxy? Thanks in advance for any help/insights you can provide! --=20 Alan Amesbury University Information Security=