Date: Sun, 28 May 2023 03:07:08 -0400 From: Paul Procacci <pprocacci@gmail.com> To: Aryeh Friedman <aryeh.friedman@gmail.com> Cc: Odhiambo Washington <odhiambo@gmail.com>, FreeBSD Mailing List <freebsd-questions@freebsd.org> Subject: Re: What happened to my /tmp/mysql.sock ?!!??! Message-ID: <CAFbbPuixgmQTHyTyoczcPCR9V1aJ=V4fAcNaqsgB6yLzQipA2Q@mail.gmail.com> In-Reply-To: <CAGBxaX=oeaFri%2Bcz_9nKPpJT5xMfxudwUVXgeBD15eWSDM_GdQ@mail.gmail.com> References: <CAGBxaXkGMdp7sev9=UnE1sMQA9ni6-sBKfdE78-T28VPM2Hd%2BA@mail.gmail.com> <CAAdA2WNY9MH1fFrfF=25%2BquxeUvLR1B71QpUo9v-fyi4egv25g@mail.gmail.com> <CAGBxaXmKoYJS-giea7v5F39hZ5G06XnhMuQA0YNARQHJ4md=gw@mail.gmail.com> <CAAdA2WO7VNPLwqX06a-eoSpqxROezysy-vDT7zKcW1QYkM1xPw@mail.gmail.com> <CAGBxaXnDVHZ0%2BUS3j0deDXOEVQFznMV7GNHi6jaD%2B6RVws4uYg@mail.gmail.com> <CAAdA2WNLUEG8CYSkNXLZ8Re=F2sJyqGooD3mqBf8fDo%2By-sGug@mail.gmail.com> <CAGBxaX=UV=dXKO=qn7=KS=dip3rUG3=LT=qVRu2eKg%2BTsckaRA@mail.gmail.com> <CAFbbPugb3YkqXWsPGR%2B_2F1eHA_vwcKKz6=Fy5fgDhUjo0P%2B-A@mail.gmail.com> <CAGBxaXn9spaKrBxJ5Pq7LjFeotjAy0ZVZ3=N3jyQZAMcVKbPkw@mail.gmail.com> <CAFbbPuh9vHaUyxznq4gqdPQQDdyhTprS54Yu=P6rwFX0f7crdQ@mail.gmail.com> <CAGBxaXm%2BzwtN707Lp4CP0Ee-UuVchOEyJfnUDkbeb9R9u5UGKw@mail.gmail.com> <CAFbbPuj2pxXuVRZTih9Xb=5h6hmQaDK-9z8m1RL8pBuczCzgxA@mail.gmail.com> <CAGBxaX=oeaFri%2Bcz_9nKPpJT5xMfxudwUVXgeBD15eWSDM_GdQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--00000000000003a02605fcbb9fca Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sun, May 28, 2023 at 1:22=E2=80=AFAM Aryeh Friedman <aryeh.friedman@gmai= l.com> wrote: > On Sun, May 28, 2023 at 12:55=E2=80=AFAM Paul Procacci <pprocacci@gmail.c= om> > wrote: > > > > > > > > On Sat, May 27, 2023 at 10:33=E2=80=AFPM Aryeh Friedman < > aryeh.friedman@gmail.com> wrote: > >> > >> On Sat, May 27, 2023 at 10:17=E2=80=AFPM Paul Procacci <pprocacci@gmai= l.com> > wrote: > >> > > >> > > >> > > >> > On Sat, May 27, 2023 at 8:14=E2=80=AFPM Aryeh Friedman < > aryeh.friedman@gmail.com> wrote: > >> >> > >> >> On Sat, May 27, 2023 at 1:21=E2=80=AFPM Paul Procacci <pprocacci@gm= ail.com> > wrote: > >> >> > > >> >> > > >> >> > > >> >> > On Sat, May 27, 2023 at 1:13=E2=80=AFPM Aryeh Friedman < > aryeh.friedman@gmail.com> wrote: > >> >> >> > >> >> >> On Sat, May 27, 2023 at 1:09=E2=80=AFPM Odhiambo Washington < > odhiambo@gmail.com> wrote: > >> >> >> > 2023-05-27T12:40:12.835631Z 0 [ERROR] Fatal error: Can't open > and lock privilege tables: Table 'mysql.user' doesn't exist > >> >> >> > 2023-05-27T12:40:12.835637Z 0 [ERROR] Fatal error: Failed to > initialize ACL/grant/time zones structures or failed to remove temporary > table files. > >> >> >> > 2023-05-27T12:40:12.835752Z 0 [ERROR] Aborting > >> >> >> > </quote> > >> >> >> > > >> >> >> > Please completely uninstall Mysql and reinstall afresh and it > will start. > >> >> >> > >> >> >> This *IS* a completely new install !?!?!? Literally installed 5 > mins > >> >> >> before I posted on a complete fresh VM > >> >> >> > > >> >> >> > Even better, install MariaDB :-) > >> >> >> > >> >> >> Since I need to debug a mysql DB on a production server not an > option > >> >> >> -- > >> >> >> Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org > >> >> >> > >> >> > > >> >> > I didn't catch the version of mysql you were attempting to run bu= t > one of two commands will solve your problem. > >> >> > The first is potentially mysql_install_db. > >> >> > > >> >> > It's possible this command doesn't exist due to it being part of > mysqld in later versions of mysql. If that's the case then the following > should initialize the databases. > >> >> > > >> >> > mysqld --initialize > >> >> Already tried both this seems to be related to a series of bugs all > up > >> >> and down the mysql food chain in the last few weeks: > >> >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D270807 > >> >> > >> >> > >> >> > > >> >> > Once one of the two are completed, then you can try the things > that Mr. Washington was suggesting and repost an new error messages. > >> >> > > >> >> > ~Paul > >> >> > > >> >> > -- > >> >> > __________________ > >> >> > > >> >> > :(){ :|:& };: > >> >> > >> >> > >> >> > >> >> -- > >> >> Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org > >> > > >> > > >> > Some things excluded from the OPS original post (or I had to search > for referenced in the provided logs) > >> > The mysql version is 5.7.41. > >> > He's using latest. Not quarterly. > >> > > >> > Series of commands that I've done to create an mysql 5.7.41 > environment: > >> > zfs send tank/jails/templates/13.1-RELEASE-stripped@initial | zfs > recv tank/jails/hosts/mysql-test > >> > zfs set mountpoint=3D/jails/hosts/mysql-test tank/jails/hosts/mysql-= test > >> > pkg -r /jails/hosts/mysql-test install -y mysql57-server-5.7.41 > >> > /etc/rc.d/jail onestart mysql-test > >> > > >> > *To follow along the jail id that got created has a numeric > identifier of 17. > >> > From there: > >> > > >> > jexec -u root 17 /usr/local/etc/rc.d/mysql-server onestart > >> > # pgrep -j 17 -l mysqld > >> > 58398 mysqld > >> > > >> > I cannot replicate the OPs problem. > >> > > >> > Note the installation from the port: > >> > > >> > "=3D=3D=3D> NOTICE: > >> > > >> > This port is deprecated; you may wish to reconsider installing it:" > >> > >> It is the newest version of mysql that will not barf on the DB I am > >> attempting debug which a 5.1 with some really insane errors in it like > >> foreign-keys that are constraints but point to non-existent records. > >> And this is on a VM so need for fancy jailing. I am also on 13.2. > >> > >> > > >> > ~Paul > >> > -- > >> > __________________ > >> > > >> > :(){ :|:& };: > >> > >> > >> > >> -- > >> Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org > > > > > > > > You need to start giving us more to go on. Of your previous 5 posts, > you've just now decided to tell us it's running in a VM, yet even that > information is incomplete. > > > > Which vm is it running under? Which architecture is that VM running? > i386? adm64? powerpc? etc? > > bhyve (with petitecloud 0.2.6/1a1 running as a front end.... I can > handle any issues there if need to but I can't think of any) on a 12.4 > AMD64 host (ryzen 5 w/ 32 GB and 2 1 TB SSD's) > > > What kind of resources are allocated to the VM? Ram/cpu/disk > > 4 cores, 8 GB of RAM, 300 GB disk combined (mounted at /, > /var/db/mysql and /backups with nfs mounted /home) > > > You also mentioned 13.2, but didn't mention the branch or revision > you're running. Release? Stable? Current? > > release > > > > Given your response I may or may not be able to reproduce your > environment instead of trying to guess what it is. > > > > ~Paul > > -- > > __________________ > > > > :(){ :|:& };: > > > > -- > Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org > Thanks for providing that information. This is extremely helpful and I wish you led with these details. # /usr/local/etc/rc.d/mysql-server onestart 2023-05-28 02:52:49 [WARNING] mysql_install_db is deprecated. Please consider switching to mysqld --initialize 2023-05-28 02:52:49 [ERROR] The data directory '/var/db/mysql' already exist and is not empty. /usr/local/etc/rc.d/mysql-server: WARNING: failed precmd routine for mysql I got the above errors by editing the rc script and removing everything that was spitting useful information to /dev/null And sure enough: # ls -l /var/db/mysql total 640420 drwxrwxr-x 2 root operator 512 May 28 02:31 .snap -r-------- 1 root wheel 655589376 May 28 02:31 .sujournal This is obviously happening because /var/db/mysql is its own mount point and has journing enabled on a UFS FS. I suspect yours is the same. So, reboot into single user mode and turn off journaling on that FS. tunefs -j disable /var/db/myql >From there go back into multiuser and continue... cd /var/db/mysql chflags noschg .sujournal rm -rf .sujournal .snap chown mysql:mysql . /usr/local/etc/rc.d/mysql-server onestart Success! pgrep -l mysqld # pgrep -l mysqld 2024 mysqld So, the above was successful to me. I'm unsure if you're running into the same problem. Regardless, modifying the rc scripts and removing the stdout and stderr redirection to /dev/null goes a long way. ~Paul --=20 __________________ :(){ :|:& };: --00000000000003a02605fcbb9fca Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div><div dir=3D"ltr"><br></div><br><div class=3D"gmail_qu= ote"><div dir=3D"ltr" class=3D"gmail_attr">On Sun, May 28, 2023 at 1:22=E2= =80=AFAM Aryeh Friedman <<a href=3D"mailto:aryeh.friedman@gmail.com" tar= get=3D"_blank">aryeh.friedman@gmail.com</a>> wrote:<br></div><blockquote= class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px so= lid rgb(204,204,204);padding-left:1ex">On Sun, May 28, 2023 at 12:55=E2=80= =AFAM Paul Procacci <<a href=3D"mailto:pprocacci@gmail.com" target=3D"_b= lank">pprocacci@gmail.com</a>> wrote:<br> ><br> ><br> ><br> > On Sat, May 27, 2023 at 10:33=E2=80=AFPM Aryeh Friedman <<a href=3D= "mailto:aryeh.friedman@gmail.com" target=3D"_blank">aryeh.friedman@gmail.co= m</a>> wrote:<br> >><br> >> On Sat, May 27, 2023 at 10:17=E2=80=AFPM Paul Procacci <<a href= =3D"mailto:pprocacci@gmail.com" target=3D"_blank">pprocacci@gmail.com</a>&g= t; wrote:<br> >> ><br> >> ><br> >> ><br> >> > On Sat, May 27, 2023 at 8:14=E2=80=AFPM Aryeh Friedman <<a= href=3D"mailto:aryeh.friedman@gmail.com" target=3D"_blank">aryeh.friedman@= gmail.com</a>> wrote:<br> >> >><br> >> >> On Sat, May 27, 2023 at 1:21=E2=80=AFPM Paul Procacci <= ;<a href=3D"mailto:pprocacci@gmail.com" target=3D"_blank">pprocacci@gmail.c= om</a>> wrote:<br> >> >> ><br> >> >> ><br> >> >> ><br> >> >> > On Sat, May 27, 2023 at 1:13=E2=80=AFPM Aryeh Friedm= an <<a href=3D"mailto:aryeh.friedman@gmail.com" target=3D"_blank">aryeh.= friedman@gmail.com</a>> wrote:<br> >> >> >><br> >> >> >> On Sat, May 27, 2023 at 1:09=E2=80=AFPM Odhiambo= Washington <<a href=3D"mailto:odhiambo@gmail.com" target=3D"_blank">odh= iambo@gmail.com</a>> wrote:<br> >> >> >> > 2023-05-27T12:40:12.835631Z 0 [ERROR] Fatal= error: Can't open and lock privilege tables: Table 'mysql.user'= ; doesn't exist<br> >> >> >> > 2023-05-27T12:40:12.835637Z 0 [ERROR] Fatal= error: Failed to initialize ACL/grant/time zones structures or failed to r= emove temporary table files.<br> >> >> >> > 2023-05-27T12:40:12.835752Z 0 [ERROR] Abort= ing<br> >> >> >> > </quote><br> >> >> >> ><br> >> >> >> > Please completely uninstall Mysql and reins= tall afresh and it will start.<br> >> >> >><br> >> >> >> This *IS* a completely new install !?!?!?=C2=A0 = Literally installed 5 mins<br> >> >> >> before I posted on a complete fresh VM<br> >> >> >> ><br> >> >> >> > Even better, install MariaDB :-)<br> >> >> >><br> >> >> >> Since I need to debug a mysql DB on a production= server not an option<br> >> >> >> --<br> >> >> >> Aryeh M. Friedman, Lead Developer, <a href=3D"ht= tp://www.PetiteCloud.org" rel=3D"noreferrer" target=3D"_blank">http://www.P= etiteCloud.org</a><br> >> >> >><br> >> >> ><br> >> >> > I didn't catch the version of mysql you were att= empting to run but one of two commands will solve your problem.<br> >> >> > The first is potentially mysql_install_db.<br> >> >> ><br> >> >> > It's possible this command doesn't exist due= to it being part of mysqld in later versions of mysql.=C2=A0 If that's= the case then the following should initialize the databases.<br> >> >> ><br> >> >> > mysqld --initialize<br> >> >> Already tried both this seems to be related to a series o= f bugs all up<br> >> >> and down the mysql food chain in the last few weeks:<br> >> >> <a href=3D"https://bugs.freebsd.org/bugzilla/show_bug.cgi= ?id=3D270807" rel=3D"noreferrer" target=3D"_blank">https://bugs.freebsd.org= /bugzilla/show_bug.cgi?id=3D270807</a><br> >> >><br> >> >><br> >> >> ><br> >> >> > Once one of the two are completed, then you can try = the things that Mr. Washington was suggesting and repost an new error messa= ges.<br> >> >> ><br> >> >> > ~Paul<br> >> >> ><br> >> >> > --<br> >> >> > __________________<br> >> >> ><br> >> >> > :(){ :|:& };:<br> >> >><br> >> >><br> >> >><br> >> >> --<br> >> >> Aryeh M. Friedman, Lead Developer, <a href=3D"http://www.= PetiteCloud.org" rel=3D"noreferrer" target=3D"_blank">http://www.PetiteClou= d.org</a><br> >> ><br> >> ><br> >> > Some things excluded from the OPS original post (or I had to = search for referenced in the provided logs)<br> >> > The mysql version is 5.7.41.<br> >> > He's using latest.=C2=A0 Not quarterly.<br> >> ><br> >> > Series of commands that I've done to create an mysql 5.7.= 41 environment:<br> >> > zfs send tank/jails/templates/13.1-RELEASE-stripped@initial |= zfs recv tank/jails/hosts/mysql-test<br> >> > zfs set mountpoint=3D/jails/hosts/mysql-test tank/jails/hosts= /mysql-test<br> >> > pkg -r /jails/hosts/mysql-test install -y mysql57-server-5.7.= 41<br> >> > /etc/rc.d/jail onestart mysql-test<br> >> ><br> >> > *To follow along the jail id that got created has a numeric i= dentifier of 17.<br> >> > From there:<br> >> ><br> >> > jexec -u root 17 /usr/local/etc/rc.d/mysql-server onestart<br= > >> > # pgrep -j 17 -l mysqld<br> >> > 58398 mysqld<br> >> ><br> >> > I cannot replicate the OPs problem.<br> >> ><br> >> > Note the installation from the port:<br> >> ><br> >> > "=3D=3D=3D>=C2=A0 =C2=A0NOTICE:<br> >> ><br> >> > This port is deprecated; you may wish to reconsider installin= g it:"<br> >><br> >> It is the newest version of mysql that will not barf on the DB I a= m<br> >> attempting debug which a 5.1 with some really insane errors in it = like<br> >> foreign-keys that are constraints but point to non-existent record= s.<br> >> And this is on a VM so need for fancy jailing.=C2=A0 =C2=A0I am al= so on 13.2.<br> >><br> >> ><br> >> > ~Paul<br> >> > --<br> >> > __________________<br> >> ><br> >> > :(){ :|:& };:<br> >><br> >><br> >><br> >> --<br> >> Aryeh M. Friedman, Lead Developer, <a href=3D"http://www.PetiteClo= ud.org" rel=3D"noreferrer" target=3D"_blank">http://www.PetiteCloud.org</a>= <br> ><br> ><br> ><br> > You need to start giving us more to go on.=C2=A0 Of your previous 5 po= sts, you've just now decided to tell us it's running in a VM, yet e= ven that information is incomplete.<br> ><br> > Which vm is it running under?=C2=A0 Which architecture is that VM runn= ing?=C2=A0 i386?=C2=A0 adm64? powerpc?=C2=A0 etc?<br> <br> bhyve (with petitecloud 0.2.6/1a1 running as a front end.... I can<br> handle any issues there if need to but I can't think of any) on a 12.4<= br> AMD64 host (ryzen 5 w/ 32 GB and 2 1 TB SSD's)<br> <br> > What kind of resources are allocated to the VM?=C2=A0 Ram/cpu/disk<br> <br> 4 cores, 8 GB of RAM, 300 GB disk combined (mounted at /,<br> /var/db/mysql and /backups with nfs mounted /home)<br> <br> > You also mentioned 13.2, but didn't mention the branch or revision= you're running.=C2=A0 Release?=C2=A0 Stable?=C2=A0 Current?<br> <br> release<br> ><br> > Given your response I may or may not be able to reproduce your environ= ment instead of trying to guess what it is.<br> ><br> > ~Paul<br> > --<br> > __________________<br> ><br> > :(){ :|:& };:<br> <br> <br> <br> -- <br> Aryeh M. Friedman, Lead Developer, <a href=3D"http://www.PetiteCloud.org" r= el=3D"noreferrer" target=3D"_blank">http://www.PetiteCloud.org</a><br>; </blockquote></div><br clear=3D"all"></div><div><br></div>Thanks for provid= ing that information.=C2=A0 This is extremely helpful and I wish you led wi= th these details.<br><br># /usr/local/etc/rc.d/mysql-server onestart<br>202= 3-05-28 02:52:49 [WARNING] mysql_install_db is deprecated. Please consider = switching to mysqld --initialize<br>2023-05-28 02:52:49 [ERROR] =C2=A0 The = data directory '/var/db/mysql' already exist and is not empty.<br>/= usr/local/etc/rc.d/mysql-server: WARNING: failed precmd routine for mysql<b= r><div><br></div><div>I got the above errors by editing the rc script and r= emoving everything that was spitting useful information to /dev/null</div><= div><br></div><div>And sure enough:</div><div><br></div># ls -l /var/db/mys= ql<br>total 640420<br>drwxrwxr-x =C2=A02 root =C2=A0operator =C2=A0 =C2=A0 = =C2=A0 =C2=A0512 May 28 02:31 .snap<br>-r-------- =C2=A01 root =C2=A0wheel = =C2=A0 =C2=A0 655589376 May 28 02:31 .sujournal<br><br><div>This is obvious= ly happening because /var/db/mysql is its own mount point and has journing = enabled on a UFS FS.=C2=A0 I suspect yours is the same.</div><div>So, reboo= t into single user mode and turn off journaling on that FS.</div><div>=C2= =A0tunefs -j disable /var/db/myql<br></div><div><br></div><div>From there g= o back into multiuser and continue...</div><div><br></div><div>cd /var/db/m= ysql<br></div><div>chflags noschg .sujournal</div><div>rm -rf .sujournal .s= nap</div><div>chown mysql:mysql .</div><div>/usr/local/etc/rc.d/mysql-serve= r onestart<br></div><div><br></div><div>Success!<br></div><div> pgrep -l mysqld <br># pgrep -l mysqld<br>2024 mysqld<br><br></div><div>So, the above was su= ccessful to me.=C2=A0 I'm unsure if you're running into the same pr= oblem.=C2=A0 Regardless, modifying the rc scripts and removing the stdout a= nd stderr redirection to /dev/null goes a long way.</div><div><br></div><di= v><span class=3D"gmail_signature_prefix">~Paul</span></div><div><span class= =3D"gmail_signature_prefix"><br></span></div><div><span class=3D"gmail_sign= ature_prefix">-- </span></div><div><div><div><div dir=3D"ltr" class=3D"gmai= l_signature">__________________<br><br>:(){ :|:& };:</div></div></div><= /div></div> --00000000000003a02605fcbb9fca--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFbbPuixgmQTHyTyoczcPCR9V1aJ=V4fAcNaqsgB6yLzQipA2Q>