From nobody Wed Feb 14 21:28:11 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TZrs92fNlz53SSx for ; Wed, 14 Feb 2024 21:28:13 +0000 (UTC) (envelope-from leres@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TZrs924Hzz40Hy for ; Wed, 14 Feb 2024 21:28:13 +0000 (UTC) (envelope-from leres@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707946093; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=mR1SpYmsarzna1ZICfghcwRQCYkhVrxJpHDUOk9n3so=; b=QLRu2faicWKwE3P1jqU2ez7nWclKt2fo8OkClsayhqm7HUCpoidKG3XUxJO43YDtDi9u/T 6DyqMq0V31T98EUeZXviVxFtbebuqE2L13QgSxp7pxdYt4sTTxXTPOen7Vvy6Wj+6B8KgV emeXZg3Zlz5WZ+fu+VeTRv6o/rjOobC5GEiLBGW+JIZahsKIPA+kfjsb49cRvcLOpbjnG7 mfvroNCzne29HliupC4sTnuz2kQPYiYS3nemL70y8ZrAimEiOvdbqnaN7w6eVA1vOImdTI CWKZj71UZ5xlKbAKlYkCndwD6k8xbWxLRleXth3ZnSQmUGBcDQpDm3XhRRyEpg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1707946093; a=rsa-sha256; cv=none; b=TtUfJR6Vxi9IJei3W+PQfLFH+4vCuNit6LA9+NGnbf/lfdIHwKTQNbbez7hWcesTS3MDp/ 2DO7FIu3AVuOU+PH+GxgNKK8J8s7pDlgF8L7nOcHSKia8OP/opIWvZh1ZVK/fhTP0/fnqn 5mSNo8QfR4Cn4baeDMfgeWQpmrHUsDRkK/XjG486oI5/8UgOwzcLQ0ELFEdTFC883sylE1 nZT5sQs2wbUn2ct7U8aICWWEt8j/+KjfPn8l0fgDURhpsBw3d2XXt+w/YIRu5qNvnl8sCZ IJGh1T0D5Zk0w7UnsopDoSTER0MMSR93qd2cdhrz9PSM8Z5qsHISJYR0aFrwFg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707946093; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=mR1SpYmsarzna1ZICfghcwRQCYkhVrxJpHDUOk9n3so=; b=FF0+5gTsIz5vBmd3nR/AFKs5rE5P+mVTzaDXYM1oIRIvtbeAGTuVIZnslQc9HwY5tRR/02 4I9lBj1sqKvKUZA0I8GvhyQqot7FdlRRZuVqqCBzi96XccIFR9aK0jUyNvWSoNIWGU8tXy fzloMp9wSBT1Xwl+a8mLICd1AfwtW/1mILW9BV1tBFfW9y3hv0N+jmFlS+xIaA1srwQone CzK3/rU9KV/huha0HVuFck37jBMVSLV2a5jSAC/MLi0RDsWSpJPVr5FyztQ2WNEUhwfhVI v4hr9so289hLdbgiJaAEuhlqbcNykwZR+7Y9HBUCepjBb/BHam0MiaLEo6MsEQ== Received: from [IPV6:fd:1965::2] (unknown [IPv6:2600:1700:ab1b:6800:2e0:edff:fece:8f27]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: leres) by smtp.freebsd.org (Postfix) with ESMTPSA id 4TZrs900m7zWLG for ; Wed, 14 Feb 2024 21:28:12 +0000 (UTC) (envelope-from leres@freebsd.org) Message-ID: Date: Wed, 14 Feb 2024 13:28:11 -0800 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US From: Craig Leres To: freebsd-hackers@freebsd.org Subject: How to use an apache proxy+cache with pkg? Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Although I have gig fiber at home, when I do pkg updates I get at most 10 mbps from my custom build server (likely thanks to purposely bad at&t peering connections). I tried recently to configure apache on my home network to proxy and cache but haven't been able to make it work yet -- seems like pkg tries *really* hard to defeat caching. First I was hitting the CacheMaxFileSize default of 1000000 (which I would describe as ridiculously low). Then I figured out that pkg was appending '?' to make the downloads look like queries so I turned CacheIgnoreQueryString on. Next I saw that pkg was setting an If-Modified-Since to Jan 1970, CacheIgnoreCacheControl solved that. But it still doesn't cache and it feels like I'm missing some obvious knob (maybe in pkg.conf?) My config works for fetch and curl but not pkg. I think I saw some ngix configs to do this but I'm already using apache for other stuff and would prefer to not have to install ngix... Is anybody doing this with apache? For extra credit, I'd like to do the same for freebsd-update blobs. Craig ProxyRequests Off ProxyPreserveHost Off ProxyPass / http://pkg.example.com/ ProxyPassReverse / http://pkg.example.com/ ProxyHTMLURLMap http://pkg.example.com/ http://pkg.local.example.com/ CacheEnable disk / CacheRoot /var/cache/freebsd-pkg #CacheIgnoreNoLastMod On CacheDetailHeader On #CacheIgnoreHeaders Set-Cookie # Default is ridiculously small (1000000 or 0.95MB) CacheMaxFileSize 200000000 CacheMinFileSize 1024 CacheIgnoreQueryString On CacheIgnoreCacheControl On #LogLevel proxy:debug #LogLevel cache:debug #LogLevel cache_disk:debug