Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 18 Jun 2018 06:47:30 +0200
From:      Michael Schuster <michaelsprivate@gmail.com>
To:        tech-lists@zyxst.net
Cc:        freebsd-stable@freebsd.org,  freeBSD Mailing List <freebsd-questions@freebsd.org>
Subject:   Re: lightly loaded system eats swap space
Message-ID:  <CADqw_g%2BJPzTOEcNFywV7j3VPZU2SW-4pNNu1BgiDOBpPHCkO_w@mail.gmail.com>
In-Reply-To: <c8277497-ffcf-0503-490b-96d1b4605af7@zyxst.net>
References:  <c8277497-ffcf-0503-490b-96d1b4605af7@zyxst.net>

next in thread | previous in thread | raw e-mail | index | archive | help
I'd suggest - if you haven't done so yet - you familiarize yourself with
DTrace and write a probe that fires when swap_pager_getswapspace() fails,
to print execname and both kernel and userland stacks (or aggregations
thereof). That should give you a starting idea of what's going on.

HTH
Michael

On Mon, Jun 18, 2018 at 12:20 AM tech-lists <tech-lists@zyxst.net> wrote:

> Hello list,
>
> context is (server)
> freebsd-11-stable r333874, ZFS raidz1-0 (3x4TB disks), 128GB RAM,
> E5-2630 @2.3GHz, generic kernel.
>
> There's one bhyve guest on this server (using 4x cpu and 16GB RAM, also
> freebsd-11-stable)
>
> There have been no special options for zfs configuration on the server,
> apart from several datasets having the compressed property set (lz4).
>
> The server runs nothing else really apart from sshd and it uses ntpd to
> sync local time.
>
> How come such a lightly loaded server with plenty of resources is eating
> up swap? If I run two bhyve instances, i.e. two of the same size as
> indicated above, so 32GB used for the bhyves, I'll get out-of-swapspace
> errors in the daily logs:
>
> +swap_pager_getswapspace(24): failed
> +swap_pager_getswapspace(24): failed
> +swap_pager_getswapspace(24): failed
>
> Here's top, with one bhyve instance running:
>
> last pid: 49494;  load averages:  0.12,  0.13,  0.88
>
>
>                               up 29+11:36:06  22:52:45
> 54 processes:  1 running, 53 sleeping
> CPU:  0.4% user,  0.0% nice,  0.4% system,  0.3% interrupt, 98.9% idle
> Mem: 8664K Active, 52M Inact, 4797M Laundry, 116G Wired, 1391M Buf,
> 4123M Free
> ARC: 108G Total, 1653M MFU, 105G MRU, 32K Anon, 382M Header, 632M Other
>       103G Compressed, 104G Uncompressed, 1.00:1 Ratio
> Swap: 4096M Total, 3502M Used, 594M Free, 85% Inuse
>
>    PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU
> COMMAND
> 49491 root          1   4    0 16444K 12024K select  9   0:12   6.49% ssh
> 32868 root         12  20    0  9241M  4038M kqread  2  23.2H   1.30% bhyve
> 49490 root          1  20    0 10812K  6192K sbwait  5   0:02   0.88% sftp
>
>  From the looks of it, a huge amount of ram is wired. Why is that, and
> how would I debug it?
>
> A server of similar spec which is running freebsd-current with seven
> bhyve instances doesn't have this issue:
>
> last pid: 41904;  load averages:  0.26,  0.19,  0.15
>
>
>                               up 17+01:06:11  23:14:13
> 27 processes:  1 running, 26 sleeping
> CPU:  0.1% user,  0.0% nice,  0.3% system,  0.0% interrupt, 99.6% idle
> Mem: 17G Active, 6951M Inact, 41G Laundry, 59G Wired, 1573M Buf, 1315M Free
> ARC: 53G Total, 700M MFU, 52G MRU, 512K Anon, 182M Header, 958K Other
>       53G Compressed, 69G Uncompressed, 1.30:1 Ratio, 122M Overhead
> Swap: 35G Total, 2163M Used, 33G Free, 6% Inuse
>
> thanks,
> --
> J.
> _______________________________________________
> freebsd-questions@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "
> freebsd-questions-unsubscribe@freebsd.org"
>


-- 
Michael Schuster
http://recursiveramblings.wordpress.com/
recursion, n: see 'recursion'



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CADqw_g%2BJPzTOEcNFywV7j3VPZU2SW-4pNNu1BgiDOBpPHCkO_w>