Date: Tue, 30 Aug 2022 21:08:43 +0100 From: Vincent Hoffman <vince@unsane.co.uk> To: ports@freebsd.org Subject: Re: Recover from a corrupt pkg DB Message-ID: <10ec58a0-7c48-a27b-1775-883657f7f812@unsane.co.uk> In-Reply-To: <0BDF281F-B485-4EE6-B0A5-698F5B134C99@gushi.org> References: <CAN6yY1vGf=LdXQ4DZjJRxho2t%2BNcGd0jt2-T7nhm84YjtCKN5g@mail.gmail.com> <20220830172613.bl4q3ynadyktstgw@shelly.nomadlogic.org> <0BDF281F-B485-4EE6-B0A5-698F5B134C99@gushi.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format. --------------PckQvrnmewDHb0l0VAEp4Zk0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 30/08/2022 18:45, Dan Mahoney (Ports) wrote: > >> On Aug 30, 2022, at 10:26 AM, Pete Wright<pete@nomadlogic.org> wrote: >> >> On Tue, Aug 30, 2022 at 09:36:52AM -0700, Kevin Oberman wrote: >>> I had a crash while building ffmpeg on a new system. The build had not >>> begun, so I don't think ffmpeg is really an issue. portmaster(8) was >>> walking the dependency tree when the system panicked and, after a reboot >>> and full fsck, I get the error: >>> pkg: sqlite error while executing PRAGMA user_version; in file >>> pkgdb.c:2357: database disk image is malformed > So, I don’t know what tools are available in sqlite3 to dump a malformed db file and reimport it. I think that’s probably beyond the scope of this, but as I’ve in the past hit pkg issues, I’ve always wanted to know if there was an online description of the schema somewhere. (Obviously, you could peruse the schema on a second non-corrupt system, but that doesn’t describe how it’s used). https://wiki.freebsd.org/pkgng/CharterAndRoadMap#Database_Schema Not a description of it but an explaination of why its not really available online, that said https://github.com/freebsd/pkg/blob/master/libpkg/pkgdb.c contains it as part of pkgdb_init Vince > >>> Is there any way to rebuild the DB or, if not, how do I remove it and start >>> from scratch? > I mean, this is a *very* brute force approach, but if you know precisely what packages you had installed, you can always just reinstall pkg and pkg install -f those packages. > >> I ran into a similar issue a while back, I discovered I was able to >> recover by using one of the automated backups generated at: >> /var/backups/pkg.sql.xz* >> >> I can't remember the details, but I think it involved replacing >> repo-FreeBSD.sqlite in /var/db/pkg/. > I’ve dealt with this on another OS that did things far, far worse (RPM, in the days before Yum). It always stymied me that there was no online tool that could look at all the files you had in /usr/local/*, look at checksums of historical packages, and glean a “hey, it looks like these are the packages you had installed, but *these files* are the ones that don’t match any checksum on record.” > > -Dan > > > > --------------PckQvrnmewDHb0l0VAEp4Zk0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> </head> <body> <p><br> </p> <div class="moz-cite-prefix">On 30/08/2022 18:45, Dan Mahoney (Ports) wrote:<br> </div> <blockquote type="cite" cite="mid:0BDF281F-B485-4EE6-B0A5-698F5B134C99@gushi.org"> <pre class="moz-quote-pre" wrap=""> </pre> <blockquote type="cite"> <pre class="moz-quote-pre" wrap="">On Aug 30, 2022, at 10:26 AM, Pete Wright <a class="moz-txt-link-rfc2396E" href="mailto:pete@nomadlogic.org"><pete@nomadlogic.org></a> wrote: On Tue, Aug 30, 2022 at 09:36:52AM -0700, Kevin Oberman wrote: </pre> <blockquote type="cite"> <pre class="moz-quote-pre" wrap="">I had a crash while building ffmpeg on a new system. The build had not begun, so I don't think ffmpeg is really an issue. portmaster(8) was walking the dependency tree when the system panicked and, after a reboot and full fsck, I get the error: pkg: sqlite error while executing PRAGMA user_version; in file pkgdb.c:2357: database disk image is malformed </pre> </blockquote> </blockquote> <pre class="moz-quote-pre" wrap=""> So, I don’t know what tools are available in sqlite3 to dump a malformed db file and reimport it. I think that’s probably beyond the scope of this, but as I’ve in the past hit pkg issues, I’ve always wanted to know if there was an online description of the schema somewhere. (Obviously, you could peruse the schema on a second non-corrupt system, but that doesn’t describe how it’s used).</pre> </blockquote> <p><a class="moz-txt-link-freetext" href="https://wiki.freebsd.org/pkgng/CharterAndRoadMap#Database_Schema">https://wiki.freebsd.org/pkgng/CharterAndRoadMap#Database_Schema</a></p>; <p>Not a description of it but an explaination of why its not really available online, that said <br> </p> <p><a class="moz-txt-link-freetext" href="https://github.com/freebsd/pkg/blob/master/libpkg/pkgdb.c">https://github.com/freebsd/pkg/blob/master/libpkg/pkgdb.c</a>; <br> </p> <p>contains it as part of <span style="color: rgb(130, 80, 223); font-family: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">pkgdb_init</span></p> <p><span style="color: rgb(130, 80, 223); font-family: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;"> </span></p> <p><span style="color: rgb(130, 80, 223); font-family: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">Vince </span></p> <blockquote type="cite" cite="mid:0BDF281F-B485-4EE6-B0A5-698F5B134C99@gushi.org"> <pre class="moz-quote-pre" wrap=""> </pre> <blockquote type="cite"> <blockquote type="cite"> <pre class="moz-quote-pre" wrap=""> Is there any way to rebuild the DB or, if not, how do I remove it and start from scratch? </pre> </blockquote> </blockquote> <pre class="moz-quote-pre" wrap=""> I mean, this is a *very* brute force approach, but if you know precisely what packages you had installed, you can always just reinstall pkg and pkg install -f those packages. </pre> <blockquote type="cite"> <pre class="moz-quote-pre" wrap=""> I ran into a similar issue a while back, I discovered I was able to recover by using one of the automated backups generated at: /var/backups/pkg.sql.xz* I can't remember the details, but I think it involved replacing repo-FreeBSD.sqlite in /var/db/pkg/. </pre> </blockquote> <pre class="moz-quote-pre" wrap=""> I’ve dealt with this on another OS that did things far, far worse (RPM, in the days before Yum). It always stymied me that there was no online tool that could look at all the files you had in /usr/local/*, look at checksums of historical packages, and glean a “hey, it looks like these are the packages you had installed, but *these files* are the ones that don’t match any checksum on record.” -Dan </pre> </blockquote> </body> </html> --------------PckQvrnmewDHb0l0VAEp4Zk0--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?10ec58a0-7c48-a27b-1775-883657f7f812>