Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 21 Aug 2017 18:14:53 +0100
From:      Martin Simmons <martin@lispworks.com>
To:        Chris Ross <cross+freebsd@distal.com>
Cc:        freebsd-fs@freebsd.org
Subject:   Re: zfs recv went idle after 20+ hours
Message-ID:  <201708211714.v7LHErBa010629@higson.cam.lispworks.com>
In-Reply-To: <4BCBA5DE-6AC7-4B0A-BF33-CFFE5A54B2DF@distal.com> (message from Chris Ross on Fri, 18 Aug 2017 23:30:49 -0400)
References:  <4BCBA5DE-6AC7-4B0A-BF33-CFFE5A54B2DF@distal.com>

next in thread | previous in thread | raw e-mail | index | archive | help
>>>>> On Fri, 18 Aug 2017 23:30:49 -0400, Chris Ross said:
> 
>   From earlier posts on this list, many know I was dealing with an issue on a zfs pool.  I eventually got it running in an off way, and backed it up (via zfs send to a file in UFS).  I then destroyed and recreated the pool as I wanted it.  For the last 23 hours or so, I’ve been running a “zfs recv” from the large send output (~5.2T).  It was cooking along, more slowly than I’d expected, but moving.
> 
>   I looked in on it a short while ago, and the pv I’m using on the input file was reporting 0.0 B/s, 77% complete.  Looking at the iostat I had running, I could see the raidz disks still seeing activity, but less than a minute later that stopped.  Now, all disks on the system seem idle.  The “zfs recv” process is idle according to ps (STAT I+), and a ktrace on the zfs process shows no activity either.  The window in which I was running the commands look as follows:
> 
> cross@hyrule[~](511): zpool list
> NAME    SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
> tank   18.8T   949K  18.8T         -     0%     0%  1.00x  ONLINE  -
> zroot  50.5G  34.7G  15.8G         -    55%    68%  1.00x  ONLINE  -
> cross@hyrule[~](512): pv /mnt/tank@20170815| sudo zfs recv -Fvu tank
> receiving full stream of tank@20170815 into tank@20170815
> received 46.3KB stream in 1 seconds (46.3KB/sec)
> receiving full stream of tank/deluge@20170815 into tank/deluge@20170815
> received 2.87TB stream in 45452 seconds (66.3MB/sec)          ] 55% ETA 10:18:14
> receiving full stream of tank/timecapsule@20170815 into tank/timecapsule@20170815
> 3.99TiB 22:54:18 [0.00 B/s] [========================>         ] 76% ETA 7:04:49
> 
>   The time-of-day and ETA value keep increasing, but other than that, it’s just sitting.  zfs list shows:
> 
> hyrule# zfs list -t all | grep tank
> tank                  3.89T  8.28T  73.3K  none
> tank@20170815             0      -  73.3K  -
> tank/deluge           2.86T  8.28T  2.86T  /data/deluge
> tank/deluge@20170815      0      -  2.86T  -
> tank/timecapsule      1024G  8.28T  1024G  /data/timecapsule
> 
>   I’m going to leave it this way for a while, but it’s already been about 10 minutes since it started, so I don’t expect that it will free itself.  Does anyone have a suggestion, or something I should look at?  Did I do something wrong?

Check that top and/or ps show nothing else using CPU.

You could use procstat -kk to see what each unexpectedly idle process is
waiting for in the kernel.

__Martin



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201708211714.v7LHErBa010629>