From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 18 18:47:13 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A6D75664 for ; Wed, 18 Feb 2015 18:47:13 +0000 (UTC) Received: from mail-qc0-f169.google.com (mail-qc0-f169.google.com [209.85.216.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 636ED3FF for ; Wed, 18 Feb 2015 18:47:12 +0000 (UTC) Received: by mail-qc0-f169.google.com with SMTP id m20so2453354qcx.0 for ; Wed, 18 Feb 2015 10:47:06 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:content-type:content-transfer-encoding; bh=k1Wp6E+omwljC3EP9Qoyigh2BtjDY5hhsjSudFbEG/0=; b=CfD6+sduweVdidCz0yS772ma0RsvOe79SdMosNbgqNpK/6WzipNo4QCtS5/vNF0zCq Edkmi4cyDi5jd/IVditkmklCcHPG2pO/MS9Sqxwx1B/Bm7yZndMF5hB8KZxN4cHYD/bG I2SeRLy9TiLQCaGoSORc/PhLK2Qlztb8cOGC/peyzH/zpTGR+TixDzV6IauHpJogYdFk SKtbsLryORQdiptiC69ZOHC2Ibyn6evEiLhgjr2phMcR7WEwuzvkQ0UC0vyPU2EAuoqx pe0bZ1bAeja/TNsJVU7IA3MYrWFqCS3eyohEoLVFtqTX5WkLSIM7m28NRn076PrGJwj3 8rew== X-Gm-Message-State: ALoCoQkFPEnBNeLJaM/+HF+LICmIVLCmVCf260Yi385wo0trRhE4k4Jl/FeZ0a6+18fqIOEG+AGg X-Received: by 10.140.231.204 with SMTP id b195mr2574730qhc.98.1424282077783; Wed, 18 Feb 2015 09:54:37 -0800 (PST) Received: from blindness.ops.about.com ([209.66.78.50]) by mx.google.com with ESMTPSA id x124sm8119435qha.2.2015.02.18.09.54.35 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Feb 2015 09:54:36 -0800 (PST) Message-ID: <54E4D1D9.2080006@longcount.org> Date: Wed, 18 Feb 2015 12:54:33 -0500 From: Mark Saad User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: local pkgng repo Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Feb 2015 18:47:13 -0000 All I am working on setting up a local repo for pkgng. I want to have a local copy of the quarterly packages, so I can archive them as needed and to run the repo inside of a private network. I am using FreeBSD 9.3-RELEASE amd64 whit pkg v 1.4.10 Here is what I did 1. Mirror the contents of the public mirror to my mirror curlmirror.pl -s 100000000000 -o 2014Q4 http://pkg.freebsd.org/freebsd:9:x86:64/quarterly 2. Setup thttpd to serve this directory thttpd -p 8080 -r -d /mnt/pkgng/ -l /tmp/thttpd-pkgng.log 3. Setup a client to use this new repo [root@roseta ~]# cat /usr/local/etc/pkg/repos/About.conf About: { url: "http://nyprdcpkgng1:8080/${ABI}/quarterly", mirror_type: "http", signature_type: "fingerprints", fingerprints: "/usr/share/keys/pkg", enabled: yes } 4. Disabled the master repo on the client [root@roseta ~]# cat /usr/local/etc/pkg/repos/FreeBSD.conf FreeBSD: { enabled: no } 5. Tested that I can browse to the http server via a real client. XXX>XXX>XXX>XXX - - [17/Feb/2015:21:05:10 +0000] "GET /freebsd%3a9%3ax86%3a64/ HTTP/1.1" 200 25000 "http://nyprdcpkgng1/" "Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Firefox/31.0" XXX>XXX>XXX>XXX- - [17/Feb/2015:21:05:14 +0000] "GET /freebsd%3a9%3ax86%3a64/2014Q4/ HTTP/1.1" 200 1652 "http://nyprdcpkgng1/freebsd%3a9%3ax86%3a64/" "Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Firefox/31.0 6. Then test the client [root@roseta ~]# pkg update Updating About repository catalogue... pkg: http://nyprdcpkgng1:8080/FreeBSD:9:amd64/quarterly/meta.txz: Not Found pkg: repository About has no meta file, using default settings pkg: http://nyprdcpkgng1:8080/FreeBSD:9:amd64/quarterly/packagesite.txz: Not Found pkg: Unable to update repository About Updating FreeBSD repository catalogue... FreeBSD repository is up-to-date. All repositories are up-to-date. 7. Check the logs on the pkgng mirror to see what happened. XXX>XXX>XXX>XXX - - [18/Feb/2015:17:13:06 +0000] "GET /FreeBSD:9:amd64/quarterly/meta.txz HTTP/1.1" 404 0 "" "pkg/1.4.10" XXX>XXX>XXX>XXX - - [18/Feb/2015:17:13:06 +0000] "GET /FreeBSD:9:amd64/quarterly/packagesite.txz HTTP/1.1" 404 0 "" "pkg/1.4.10" XXX>XXX>XXX>XXX - - [18/Feb/2015:17:21:15 +0000] "GET /FreeBSD:9:amd64/quarterly/meta.txz HTTP/1.1" 404 0 "" "pkg/1.4.10" XXX>XXX>XXX>XXX - - [18/Feb/2015:17:21:15 +0000] "GET /FreeBSD:9:amd64/quarterly/packagesite.txz HTTP/1.1" 404 0 Where do I stand --------------------- So in step 5 I noted that the request is for freebsd%3a9%3ax86%3a64 and not freebsd:9:x86:64 ok not sure why its html encoded but it still works secondly in step 7 the client appears to translate the ${ABI} in my config to FreeBSD:9:amd64 and not freebsd:9:x86:64 which is strange. What I tried . ------------------- So in the sake of trying sill things that should not matter I ln'd freebsd:9:x86:64 to FreeBSD:9:amd64 Then I went to my client and tested it [root@roseta ~]# pkg update Updating About repository catalogue... Fetching meta.txz: 100% 968 B 1.0kB/s 00:01 Fetching packagesite.txz: 100% 5 MiB 5.3MB/s 00:01 Processing entries: 100% About repository update completed. 23741 packages processed Updating FreeBSD repository catalogue... FreeBSD repository is up-to-date. [root@roseta ~]# That looks like it was ok let me try to install a package [root@roseta ~]# pkg instal figlet Updating About repository catalogue... About repository is up-to-date. Updating FreeBSD repository catalogue... FreeBSD repository is up-to-date. All repositories are up-to-date. The following 1 packages will be affected (of 0 checked): Installed packages to be REINSTALLED: figlet-2.2.5 [About] (options changed) The process will require 550 B more space. 115 KiB to be downloaded. Proceed with this action? [y/N]: y pkg: http://nyprdcpkgng1:8080/FreeBSD:9:amd64/quarterly/All/figlet-2.2.5.txz: Not Found so I tried to get the file manually from a browser and I checked the logs and noted this URL from the pkgng client XXX>XXX>XXX>XXX - - [18/Feb/2015:17:42:28 +0000] "GET /FreeBSD:9:amd64/quarterly/All/figlet_2.2.5.txz HT TP/1.1" 200 118512 "" "Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Firefox/31.0" URL from browsing to the package from / XXX>XXX>XXX>XXX - - [18/Feb/2015:17:42:58 +0000] "GET /FreeBSD:9:amd64/quarterly/All/figlet-2.2.5.txz HT TP/1.1" 404 0 "" "Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Firefox/31.0" The issue here is in the name of the package "figlet_2.2.5.txz vs figlet-2.2.5.txz" the char after the t is wrong but no idea why. Its like this all over the place. I am going to redownload the repo and see if I get the same odd - vs _ issue . Secondly does anyone have any comments on this ? Is there any major foot shooters I hit ? -- Mark saad | nonesuch@longcount.org