From owner-freebsd-hackers@freebsd.org Fri Feb 8 21:02:35 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7EAEC14C6257 for ; Fri, 8 Feb 2019 21:02:34 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "m5p.com", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EE0006D04F for ; Fri, 8 Feb 2019 21:02:25 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from [10.100.0.31] (haymarket.m5p.com [10.100.0.31]) (authenticated bits=0) by mailhost.m5p.com (8.15.2/8.15.2) with ESMTPSA id x18L25sk026271 (version=TLSv1.2 cipher=AES128-SHA bits=128 verify=NO) for ; Fri, 8 Feb 2019 16:02:15 -0500 (EST) (envelope-from george+freebsd@m5p.com) Subject: Re: Non-interactive multivolume restore To: freebsd-hackers@freebsd.org References: <5fc87495-8fdc-c1f3-0b95-8609347794f0@m5p.com> <32330.1549288285@critter.freebsd.dk> <5e7f90d8-d4a6-9ae3-3e92-2abc3103c683@m5p.com> From: George Mitchell Openpgp: preference=signencrypt Autocrypt: addr=george+freebsd@m5p.com; prefer-encrypt=mutual; keydata= mQINBFgnLnwBEADAJDiBKQX77LFRz9wZW8mz3KvaQol2nIremcws0F1mz/zgFlk6uhQVtwnL wb4XL5LdFwcNE1+QZzPLcbYWoWQlz0lBw1bMuKAgr0S6V2e0+I0DqhKeslVFctcTwtvT6pnK VLZXO/7ZGAaLzG4K5vSPzgoevU+YI/pxNsVCH2UO/c3jQW63uEt25mIZbCF1Pu4jgp4RhIgF ujn877r/j6OwBwjzRUu3E6ADp+U825d+5YCuQMEH0wIPnn9GTpXvfdKdbwOIl2akqXqs4cnk iATWfK3r6D4mvDEj1OPHlTvJYcfic7aOIiAwmx1C1v78GjXOdOOA0SGffNix3C2/8oZUO1+V Aet4MKpUKkduWSvULhIkHNZ5Nu8SIJOqge8pmtHxuNXAMfMrAjMdjPwwBFLsYg3Xa2E2oJwg ehTauwd/EDJFcVCyDCyCAYOi/BH/+XQyxzgDlY9N9qj9tHqhVPI6XK7t8UVffGiZUq4rHp5J RdOToqiTNC6eCJBczhMIW+DuFvWU9e6W708T1dz0Accn6Lrgk4eRIn3GFPBG+TxnpjAqHsbW 607dcnD3YKAqY4e+khczL4EObhe7dC1v2fmZiAC6Ds3WHR11IfqoUgCkIwJ590Ej+ElygJFF XxI82wtEz9hkeLLvItpyEJNVjppViRW+Dgl/U7ypHB3qDgYjgwARAQABtChHZW9yZ2UgTWl0 Y2hlbGwgPGdlb3JnZStmcmVlYnNkQG01cC5jb20+iQJUBBMBCAA+FiEENdM4ZHktsJW5kKZX wRES3m+p4fkFAlhZcR0CGyMFCQlmAYAFCwkIBwIGFQgJCgsCBBYCAwECHgECF4AACgkQwRES 3m+p4flqmw/9Emr/ydTG2n9o/IX1yVCNcHVFenVrcOY0L+DGQYZRO/XpLvsGYcuSIQId1w7h l4HZKI89ri2fF2ks6upMqBajLf8s7a8PnYrbw5bPaoOFyNTjv57GLZVsYw95kmMUpK6siuAA fXvHfKUpC/sThbwSv/1CLryVG74+5vdI8j7cQeDM436FThxlVfHKrILIiL34D4WThFB3hV/Y 2A+mQwXmdLcuQXXeAazqsFJL8sgEKSC7GMcExDkVpGc5Rh2hu97a4Sa7qWX9G/YdZOrcDacJ XxfvePn3m3WfRtXN/r0lUfiVXiqkFfbvqSaZQ0I4UvZXNGd/gH4jKHtX9RTH9G96UZeHNoMo tPw9U0fx8Ceh72nUL9qzqnmok/ryWm+6gt4Q1eRP7QAosOa1g/RgUdS1Z9IuCmbXMDp5kbNw L4ZoDMF5U3mmh8/IOKkhGopNLbNv0mwUgC59pnCptiOVx8DyckXWC4L2r6PKbWGrcGIzsUER 9smfL10gpp5H5agjwwPZI6/kzJ0R5nBzQWAlwqI73YAy6JI0HTD6lvxW7yWm2fGjEfmyaBOU 8OLUin7auoFSn+QmD5yNCUn5Ls77qHARkT1ZGocAnQkvZBGTwXpvyJixygXsm+vSUFDYBOSn cR54vdXOEMqrJk6SGau5YI9V7EhQVveE1BUp8ofWf2oo4RG5Ag0EWCcufAEQALuTOxmqMFE+ ieev/rcL3wVJrcuKS+pBbKCY9IIL0OwVf98HQJJcgdOsdDhruVd19nJNlwZ3Fc34wLw7y2GO 9WrpZiYKnI4n9urhLE5r1ydBInlI/1UKZWgM3/dPjJtcXMsC3vnqR9DmOxW4/SbqJDjP3XzO FleT4yip3AaNhPGwEPTZrubVp5hp/JojaZn690TLRwOFXg8NcjpOEs0Bq9M+OLpmsF0flrgs yDfS7y+SQ17R4Iq9T7RxZvZVAh510yGGIZIETYO/4Dh417VVm+gaksOVh2egetpUUvYYc0Ub KaP+5F/WGNrmRb1F6SKypvLlKkYAHCsUUSzsAGl9gbQhEEpuOMbUKp1979HoRMkW+8046kIo 8BZ6ph8izG/g4dZOaEqKGEhqdhYIB7UwMtFFuPtSs5Nl6JrZYni/nzFtTmtVCgcj9PNqrzqt fNFYhNznD6St6wxp3TOm9D3TQF0dzwBM1jZpb8WvmK3k6oy8hbpjiBzxn3kyRA9Vzy+PdbN/ G5a1k0rpZu/ivpBuLCDVGljUmgQigXg6xkk5UxBoHp7MPvG9prZ5jqdEa2r1KgnGjaq+VJsu Uqrw10dVgeG1NulDU1+sQl+/mwtflbkimhjDDjxsVgfrv7uvV/9be+gGm1KATuqdgCboSb1s QAo5ARfwFfChrnh+fTfPpPKHABEBAAGJAiQEGAEIAA8FAlgnLnwCGwwFCQlmAYAACgkQwRES 3m+p4fno9w/4m+swztkzxSWdutjgSv2mw+PdrKWVGFAUD2HoY1Qpi5LNLE6s9pP3qzwpQYwK viOufVJYWZ540ss6BImZBGJwyHouacqrpZjpRo5+ftj07rY1SNd8QjcHDggPfpgJ1D4Il3Xi vRg5/gzkXnRu8dXeVvMP1Ndk/F5wcoLZlQwFtPfu2xyRYIsveXMoyypAvAFSaAGXU0hRzuDJ fGI3LFvpI9UXU2C4MMzjfyZyD2NJEDKOACTo85QQzxgheTDQaDocXW00wknXFMwEItiXp8dO 2zEml/3Kj4efDfjqGpjNefjK0cnj02Byt7y6GozWXyIylrXu0SN9qWRzUVZH3+q+ijA4q3Gm 9uWzLdpjN4QWAiiaEvMhLPohp9DdLsy3kAWWrA3+pAfHSTZXrobMMbSeBkE9E4/WxdKl0nM7 TNslAWcxkTd/7Ly9cxwT8wFdHuQB1hgCmIQxDNXHL1N1ANTeUYum1w9nUg6e1M0UWu+nk3Cw qL7oL2KZe13mQnU/CFwlhbf+i//j3SXrQLlIVQv9Fn805bxIcVo9yqUZyoiV7EUpvOsxDCZh ej3mNYF5nRCf6trEJQVk0aLC26zJAYExykdUlRqc4I13XPhlt+aFSMMkoL/thYO6e9oNFK6Q aJEKXomzxxqpceJVmPH6zvqJbOboAdE/mOD0PoS1M6saIQ== Message-ID: <1f2f42bb-adf9-db72-25ba-b8a7a89e5b32@m5p.com> Date: Fri, 8 Feb 2019 16:01:58 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="jZC2b6vu4Wdh5A5DhnzpHlcNmBQrwpEEv" X-Spam-Status: No, score=-1.0 required=10.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mattapan.m5p.com X-Rspamd-Queue-Id: EE0006D04F X-Spamd-Bar: -------- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of george@m5p.com designates 74.104.188.4 as permitted sender) smtp.mailfrom=george@m5p.com X-Spamd-Result: default: False [-8.40 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a]; TO_MATCH_ENVRCPT_ALL(0.00)[]; HAS_ATTACHMENT(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; DMARC_NA(0.00)[m5p.com]; MX_GOOD(-0.01)[mailhost.m5p.com]; NEURAL_HAM_SHORT(-0.92)[-0.924,0]; IP_SCORE(-3.07)[ip: (-9.69), ipnet: 74.104.0.0/16(-4.85), asn: 701(-0.72), country: US(-0.07)]; SIGNED_PGP(-2.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+]; ASN(0.00)[asn:701, ipnet:74.104.0.0/16, country:US]; TAGGED_FROM(0.00)[freebsd]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Feb 2019 21:02:35 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --jZC2b6vu4Wdh5A5DhnzpHlcNmBQrwpEEv Content-Type: multipart/mixed; boundary="yI1m4BRQJHnDcESx7aRVS8qtRzFcHJ9aK"; protected-headers="v1" From: George Mitchell To: freebsd-hackers@freebsd.org Message-ID: <1f2f42bb-adf9-db72-25ba-b8a7a89e5b32@m5p.com> Subject: Re: Non-interactive multivolume restore References: <5fc87495-8fdc-c1f3-0b95-8609347794f0@m5p.com> <32330.1549288285@critter.freebsd.dk> <5e7f90d8-d4a6-9ae3-3e92-2abc3103c683@m5p.com> In-Reply-To: --yI1m4BRQJHnDcESx7aRVS8qtRzFcHJ9aK Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable Apparently people don't often have to restore multivolume dumps. (I rarely do it myself except that I'm in the middle of compiling an index of a plethora of ancient dumps before purging them.) Since it requires a lot of interaction (as least as far as I have been able to determine), I've writtien this silly python program to help out. It relies on misc/py-pexpect, which happily compiles in a py36 version that doesn't run, so this program specifies python2.7: --------------------------------------------------------------------- #!/usr/bin/env python2.7 from __future__ import print_function import pexpect import sys import os if len(sys.argv )< 2 or (len(sys.argv) =3D=3D 2 and sys.argv[1] =3D=3D '-= h'): print(''' Non-interactively restore from a multivolume dump. You can use a glob to specify the dump volume if the glob expands to the files in the correct order. Creates symbolic links to your dumps in $TMPDIR (or /tmp by default) and runs 'restore' with the -P option. Generates responses as needed to keep the restore program running. ''') print('You did not specify any dump volumes.') sys.exit(1) tmpdir =3D os.getenv('TMPDIR') or '/tmp' for i, volume in enumerate(sys.argv[1:]): try: os.remove(tmpdir + '/dumpvolume' + str(i + 1)) except: pass os.symlink(sys.argv[i + 1], tmpdir + '/dumpvolume' + str(i + 1)) child =3D pexpect.spawn("/sbin/restore -x -P 'cat " + tmpdir + "/dumpvolume$RESTORE_VOLUME'") child.logfile =3D sys.stdout while True: resp =3D child.expect(['Specify next volume #: ', 'default: ', "set owner/mode for '.'. .yn. ", pexpect.EOF], timeout=3D600) if resp =3D=3D 0: child.sendline('1') elif resp =3D=3D 1: child.sendline('') elif resp =3D=3D 2: child.sendline('y') else: break for i in len(sys.argv[1:]): os.delete(tmpdir + '/dumpvolume' + str(i + 1)) --yI1m4BRQJHnDcESx7aRVS8qtRzFcHJ9aK-- --jZC2b6vu4Wdh5A5DhnzpHlcNmBQrwpEEv Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEENdM4ZHktsJW5kKZXwRES3m+p4fkFAlxd7k0ACgkQwRES3m+p 4fmHCA/8CNWH5KHXhkK8XQUuYwuxWM4Fc8tJpTdwQ61/oWM7QoD5pxOVGdSSTMSZ zWjEn2LEysbmP+u+wGzsG+xZPKDjzrbBDaF+QC68uVEUaIa+oYZkz+FEMS40M5fL szzcZOkL4l0ZHk4tIxzSEJqLyh4qG6FRM1llmEQ1SZATYh22DICt20J53p9xeTG0 3veTw+iomj/0dK5sH1M56MDoC32bt42DpmN+MAH4efVzG9NGumFjvZ4gc4SwTGVW iKNu2jed8jjOH3/OogxCW8B+h8ChdybLc+dK/LKp2bIDO4cTN/h+i9ORCfUmHx+h gY+f4IfYQGEwXltSU+UC0u+C6SRjBmTE0RARSk06Dw6G6cmTBpEkEhYzrHp9PSld EJF6le9E2Ixi4I0PpqWtwLiKkv1vuZFOb18ky0cK+BXBvF+0DZohBh7koZMUswlu 1KuZ3iGrYWWbmssG9MVfA+knQsaa/yFZtyxo1pTwQ69wlzOeeWnSmbTQ2zmuYlY0 OcjFEyLwq8SF/OxfkCGkNXf+TgfsOMb5KOQOP15y09OxYNwevm8nhdkPdPwLiL3z DJzRS80Aup2+z+cCBq5pl/rOozRdJpNi43M5eRcTIGGV9XxADcysCJ/rOoe0knWU 0nDvdLUh01b7QsB6l3zR3W+qg6bp/+4bNqKNS+BBIzwpz114bBc= =VHSh -----END PGP SIGNATURE----- --jZC2b6vu4Wdh5A5DhnzpHlcNmBQrwpEEv--