Date: Sun, 12 Apr 2020 13:30:17 +0200 From: Peter Eriksson <pen@lysator.liu.se> To: freebsd-fs <freebsd-fs@freebsd.org> Subject: Re: ZFS server has gone crazy slow Message-ID: <92901912-F3AA-4C44-BF6E-3F4E0F996BFF@lysator.liu.se> In-Reply-To: <708dabd7-838c-2b34-87dc-2151536fa8bc@wp.pl> References: <2182C27C-A5D3-41BF-9CE9-7C6883E43074@distal.com> <20200411174831.GA54397@fuz.su> <6190573D-BCA7-44F9-86BD-0DCBB1F69D1D@distal.com> <6fd7a561-462e-242d-5057-51c52d716d68@wp.pl> <7AA1EA07-6041-464A-A39A-158ACD1DC11C@distal.com> <FE84C045-89B1-4772-AF1F-35F78B9877D8@lysator.liu.se> <708dabd7-838c-2b34-87dc-2151536fa8bc@wp.pl>
next in thread | previous in thread | raw e-mail | index | archive | help
Sure You can find my stuff at my Github page: https://github.com/ptrrkssn/freebsd-stuff = <https://github.com/ptrrkssn/freebsd-stuff>; The things needed are the /etc/rc.d scripts (zfs, nfsd, samba_server, = cold) scripts in =E2=80=9Crc.d=E2=80=9D and the =E2=80=9Czfs-mountall=E2=80= =9D in =E2=80=9Cscripts=E2=80=9D that goes into /sbin. What they basically do are use a helper function =E2=80=9Cwaitfor=E2=80=9D= to wait for a =E2=80=9C${name}.done=E2=80=9D file to appear in /var/run = and then just=20 (waitfor zfs ; run_rc_command =E2=80=9C$1=E2=80=9D) & Instead of just the =E2=80=9Crun_rc_command =E2=80=9C$1=E2=80=9D. And = then we have the scripts create those =E2=80=9C${name}.done=E2=80=9D = files when they have started their daemons. Not foolproof (should check that the daemon really is successfully = running and more before signalling =E2=80=9Cdone=E2=80=9D). scripts/zfs-mountall=20 Mounts all filesystems under a specific DATASET sequentially. Basically = what a =E2=80=9Czfs mount -r=E2=80=9D would have done if that option had = existed. rc.d/zfs Executes /sbin/zfs-mountall to make sure all filesystems in the = =E2=80=98zroot=E2=80=99 (or whatever dataset / is mounted from) is = mounted first. This is needed to make sure all important filesystems are = mounted (like various /var filesystems etc). Then basically does: ( rm -f /var/run/zfs*.done zfs mount -va >/var/log/zfs/mount.log && touch = /var/run/zfs-mount.done zfs share -a >/var/log/zfs/share.log && touch /var/run/zfs.done ) & rc.d/mountd, rc.d/nfsd, rc.d/ctld & rc.d/samba_server all contains a = function: waitfor() { FIRST=3Dyes while [ ! -f "/var/run/$1.done" ]; do if [ $FIRST =3D yes ]; then echo "${name}: Waiting for '$1' to finish..." FIRST=3Dno fi sleep 1 done } It probably should go into /etc/rc.subr or somewhere else but=E2=80=A6 = =E2=80=9Cwaitfor=E2=80=9D basically is loop and waits for the = =E2=80=9C${name}.done=E2=80=9D file to appear in /var/run. rc.d/mountd then ends with this instead of just: =E2=80=98 = run_rc_command =E2=80=9C$1=E2=80=9D =E2=80=98 case "$1" in faststart) rm -f "/var/run/${name}.done" (waitfor zfs ; run_rc_command "$1" ; touch = "/var/run/${name}.done") & ;; *) run_rc_command "$1" ;; esac rc.d/nfsd ends with: case "$1" in faststart) (waitfor "mountd" ; run_rc_command "$1") & ;; *) run_rc_command "$1" ;; esac (And similar for cold) And rc.d/samba_server has a bit more modifications (we only want = =E2=80=9Csmbd=E2=80=9D to delay it=E2=80=99s startup, not winbindd or = numbed) so does this in the middle of the script. Also just waits for = the =E2=80=9Czfs mount -a=E2=80=9D to be finished, doesn=E2=80=99t have = to wait for =E2=80=9Czfs share -a=E2=80=9D (NFS)): # Daemon should be enabled and running = =20 if ( [ -n "${rcvar}" ] && checkyesno "${rcvar}" ) || [ -n = "$force_run" ]; then if [ "${rc_arg}" =3D "start" -a "${name}" =3D "smbd" ]; then (waitfor "zfs-mount" ; run_rc_command = "${_rc_prefix}${rc_arg}" ${rc_extra_args}) & else run_rc_command "${_rc_prefix}${rc_arg}" ${rc_extra_args} # If any of the commands failed, take it as a global = result =20 result=3D$((${result} || $?)) fi fi =20 - Peter > On 12 Apr 2020, at 09:17, Ireneusz Pluta/wp.pl <ipluta@wp.pl> wrote: >=20 > W dniu 2020-04-11 o 23:24, Peter Eriksson pisze: >> (We have modified our rc.d startup scripts so we do the =E2=80=9Czfs = mount -a=E2=80=9D part in the background so the rest of the system = doesn=E2=80=99t have to wait for it (so we can get a ssh login prompt = and login and check the progress of the filesystem mounts. (And then = =E2=80=9Cnfs exports=E2=80=9D, nfsd and samba also waits in the = background for that to finish before starting up. A bit of a hack but it = works (a real parallel server manager with dependencies would have been = better > would you share these modifications? >=20 >> :-)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?92901912-F3AA-4C44-BF6E-3F4E0F996BFF>