From owner-freebsd-cloud@freebsd.org Wed Jan 31 18:25:43 2018 Return-Path: Delivered-To: freebsd-cloud@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A454FEDB09C for ; Wed, 31 Jan 2018 18:25:43 +0000 (UTC) (envelope-from raf@rafal.net) Received: from fbo-2.mxes.net (mxout-22.mxes.net [216.86.168.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C2F3725EB; Wed, 31 Jan 2018 18:25:42 +0000 (UTC) (envelope-from raf@rafal.net) Received: from smtp-out-1.mxes.net (smtp-out-1.mxes.net [67.222.241.250]) by fbi-2.mxes.net (Postfix) with ESMTP id 807E327588; Wed, 31 Jan 2018 13:18:19 -0500 (EST) Received: from [192.168.40.37] (86-40-118-125-dynamic.agg2.bri.bbh-prp.eircom.net [86.40.118.125]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.mxes.net (Postfix) with ESMTPSA id 55A182753E; Wed, 31 Jan 2018 13:18:13 -0500 (EST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: AMI building AMI From: Rafal Lukawiecki In-Reply-To: Date: Wed, 31 Jan 2018 18:18:11 +0000 Cc: freebsd-cloud@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <38781F81-4051-4E6F-80B7-2207043C6489@rafal.net> References: <085D225B-2E99-4540-B693-223087B14A27@rafal.net> <0100016099578ea7-02da1d5c-997d-47be-9cd8-dc6b75bbeda6-000000@email.amazonses.com> <8BCB67B8-DE56-491A-8E6A-95AFB83F71D6@rafal.net> <0100016099671dff-32d986bf-3979-40a3-accf-4d17c2020403-000000@email.amazonses.com> <8538BFB9-B82C-4520-ADB3-E25040EA1B30@rafal.net> <01000160a0283ad5-12b330d3-3f4b-43ea-9cdc-2524f5f42f4a-000000@email.amazonses.com> <004116c5-0938-5570-f119-fdc21e91b2ad@freebsd.org> <559A9E86-6E97-4A8D-96CC-95FC9180A483@rafal.net> <50f75b01-2ea3-f7fd-24a6-65a4d64ae16d@freebsd.org> <77A688A9-6272-4FC6-89DF-4F9231B7B3CE@rafal.net> To: Colin Percival , Julian Elischer X-Mailer: Apple Mail (2.3445.5.20) X-BeenThere: freebsd-cloud@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "FreeBSD on cloud platforms \(EC2, GCE, Azure, etc.\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jan 2018 18:25:44 -0000 Dear Colin, Julian, I am afraid the script is not working reliably when it is launched by = Colin=E2=80=99s very useful configinit from EC2 user-data. If I run the = same script from the shell prompt, it works. The difference is that pkg = -r /mnt update -f fails a good few times, each time with these errors: /bin/sh: indexinfo: not found pkg: POST-INSTALL script failed Is indexinfo something that I could manually add? I am sorry I am not = more familiar with the inner workings of pkg=E2=80=A6 Many thanks indeed, Rafal=20 =20 > On 2 Jan 2018, at 21:17, Rafal Lukawiecki wrote: >=20 > Colin, Everyone, >=20 > Here is the final script that has managed to run, unattended! On a = c4.x4large it completes the task in 11 minutes, which is impressive, = considering the long list of packages (126). Following that, I am able = to encrypt the AMI while copying it across regions using: >=20 > aws ec2 copy-image --source-region us-east-1 --region eu-west-1 = --source-image-id ami-x --encrypted --name =E2=80=9CMy FreeBSD" >=20 > If you would like to update your blog post with this, now fully tested = script, perhaps others would benefit. Please note credit is due to = Julian, who got me unstuck regarding pkg -c and resolv.conf, many = thanks, Julian. >=20 > #!/bin/sh > set -x > export ASSUME_ALWAYS_YES=3DYES > PKGS_TO_INSTALL=3D=E2=80=9Clist of packages to install" > pkg -r /mnt fetch -d $PKGS_TO_INSTALL > pkg -r /mnt update -f > mv /mnt/etc/resolv.conf /mnt/etc/resolv.conf.orig > cp /etc/resolv.conf /mnt/etc/resolv.conf > yes n | pkg -c /mnt -o ASSUME_ALWAYS_YES=3DYES install = $PKGS_TO_INSTALL > mv /mnt/etc/resolv.conf.orig /mnt/etc/resolv.conf > svnlite --non-interactive --trust-server-cert-failures=3Dunknown-ca co = https://svn.freebsd.org/base/releng/11.1/ /usr/src/ > cd /usr/src > cat > /tmp/kernel.patch < *** patches to apply *** > EOT > patch < /tmp/kernel.patch > make DESTDIR=3D/mnt kernel -j16 > mkami "FreeBSD 11.1 Custom AMI=E2=80=9D =E2=80=9CMy custom AMI = description" > shutdown -p now >=20 > Rafal > -- > Rafal Lukawiecki > Data Scientist=20 > Project Botticelli Ltd