Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 06 Nov 2018 18:26:14 +0100
From:      "Sijmen J. Mulder" <ik@sjmulder.nl>
To:        freebsd-questions@freebsd.org
Subject:   NFS server stalls on low memory
Message-ID:  <1541525174.2175467.1567690832.1D278398@webmail.messagingengine.com>

next in thread | raw e-mail | index | archive | help
Hi,

When I write a large amount of data (>RAM) to a slow disk attached to my
FreeBSD 11.2-p3 NFS server all I/O to the disk (even local) is stalled
for minutes at a time until the paging daemon makes a pass. 

When writing such a large file locally, memory pressure also occurs but
the paging daemon keeps scanning and freeing continuously.

vmstat -w1 output of both such an NFS transfer and a local write below.

What's going on here?

Sijmen


=== 2 GB NFS WRITE ===

r b w  avm   fre   flt  re  pi  po    fr   sr vt0 vt1   in    sy    cs us sy id
0 0 0 185M  693M     0   0   0   0     0    6   0   0   18   113    81  0  0 100
0 0 0 185M  693M     0   0   0   0     0    6   0   0   17   113    81  0  1 99
0 0 0 185M  790M     0   0   0   0 30216    6   0 159 5267   113 14874  0 36 64
1 0 0 185M  767M     0   0   0   0     0    6   0 184 6224   119 17730  0 34 66
1 0 0 185M  745M     0   0   0   0     0    6   0 181 6325   113 17927  0 42 58
1 0 0 185M  724M     0   0   0   0     0    6   0 159 6329   113 17685  0 36 64
0 0 0 185M  709M     0   0   0   0     0    6   0 122 4656   113 12930  0 21 79
1 0 0 185M  697M     0   0   0   0    24    6   0 185 4019   113 11055  0 20 80
1 0 0 185M  675M     0   0   0   0     0    6   0 172 6628   119 18596  0 37 63
1 0 0 185M  653M     0   0   0   0     0    6   0 177 6121   116 17800  0 38 62
1 0 0 185M  633M     0   0   0   0     0    6   0 150 4850   113 14384  0 34 66
0 0 0 185M  611M     0   0   0   0     0    6   0 177 6764   113 18943  0 38 62
0 0 0 185M  599M     0   0   0   0     8    6   0 173 3407   114  9902  0 22 78
1 0 0 185M  583M     0   0   0   0    24    6   0 143 4190   119 12275  0 15 85
1 0 0 185M  565M     0   0   0   0     0    6   0 134 4825   113 13942  0 27 73
1 0 0 185M  545M     0   0   0   0     0    6   5 165 6065   113 17006  0 34 66
1 0 0 185M  531M     0   0   0   0     0    6   0 108 4604   113 12578  1 25 74
1 0 0 185M  516M     0   0   0   0   168    6   0 203 4851   113 13527  0 23 77
0 0 0 185M  496M     0   0   0   0     0    6   0 156 5053   119 14876  0 33 67
1 0 0 185M  479M     0   0   0   0     0    6   0 137 4925   113 14130  0 30 70
1 0 0 185M  465M     0   0   0   0     0    6   0 110 3980   113 11438  0 21 79
1 0 0 185M  451M     0   0   0   0     8    6   1 169 3690   113 10736  0 26 74
1 0 0 185M  432M     0   0   0   0     0    6   0 148 4999   113 14657  0 31 69
1 0 0 185M  415M     0   0   0   0     0    6   0 133 4549   119 13131  0 22 78
1 0 0 185M  395M     0   0   0   0     0    6   0 165 5195   113 15377  0 29 71
procs  memory       page                    disks     faults         cpu
r b w  avm   fre   flt  re  pi  po    fr   sr vt0 vt1   in    sy    cs us sy id
1 0 0 185M  379M     0   0   0   0     0   12   0 170 4413   115 12742  0 23 77
1 0 0 185M  363M     0   0   0   0    24    6   0 163 4913   113 13802  0 27 73
1 0 0 185M  343M     0   0   0   0     0    6   0 163 5704   113 16277  0 30 70
1 0 0 185M  320M     0   0   0   0     0    6   0 177 7410   119 20264  0 34 66
1 0 0 185M  306M     0   0   0   0     0    6   0 179 4448   113 12544  0 23 77
1 0 0 185M  286M     0   0   0   0     8    6   0 161 5482   113 15733  0 29 71
0 0 0 185M  267M     0   0   0   0     0    6   0 139 6092   113 16693  0 40 60
1 0 0 185M  246M     0   0   0   0     0    6   0 176 6475   119 18258  0 38 62
1 0 0 185M  232M     0   0   0   0    24    6   0 176 4447   113 12411  0 24 76
0 0 0 185M  216M     0   0   0   0     0    6   0 125 4895   113 13750  0 29 71
1 0 0 185M  199M     0   0   0   0     0    6   0 133 4075   113 12250  0 30 70
0 0 0 185M  180M     0   0   0   0     0    6   0 139 4639   113 13913  0 26 74
1 0 0 185M  167M     0   0   0   0     0    6   0 108 3017   119  9059  0 24 76
1 0 0 185M  162M     0   0   0   0     0    6   0  47 1244   113  3856  0 16 84
1 0 0 185M  150M     0   0   0   0    24    6   0 176 3042   116  8751  0 24 76
1 0 0 185M  132M     0   0   0   0     0    6   0 150 5259   113 15073  0 37 63
0 0 0 185M  110M     0   0   0   0     0    6   0 166 5979   113 17094  0 40 60
1 0 0 185M   96M     0   0   0   0     0    6   0 112 3923   120 11285  0 24 76
2 0 0 185M   84M     0   0   0   0     8    6   0 153 3730   113 10478  0 26 74
1 0 0 185M   64M     0   0   0   0     0    6   9 160 5786   113 16498  1 39 60
1 0 0 185M   47M     0   0   0   0     0    6   0 135 5089   113 14363  0 31 69
1 0 0 185M   27M     0   0   0   0     0    6   0 153 5863   114 16437  0 30 70
0 0 0 185M   12M     0   0   0   0     0    6   0 166 4388   119 12533  0 21 79
1 0 0 185M  8.6M     0 171   0   0  3577 3581   0 172 4973   113 14227  0 31 69
1 0 0 185M   18M     0   0   0   0  7095 7106   0 142 5010   113 14379  0 38 62
1 0 0 185M  9.9M     0   0   0   0  3552 3557   0 176 7739   113 20894  0 35 65
0 0 0 185M  7.7M     0   0   0   0  3549 3553   0 165 4882   113 13633  0 32 68
1 0 0 194M   14M   693   0  27   0  7194 7108  45 171 6492   764 17751  1 40 60
1 0 0 194M  8.8M    81   0   0   0  3546 3552   0 149 5718   127 15990  0 38 62
1 0 0 185M   18M    52   0   0   0  7635 7108   2 165 6453   175 17924  0 33 67
0 0 0 185M   17M     0   0   0   0  3552 3557   0 157 4050   113 11695  0 29 71
1 0 0 185M   12M     0   0   0   0  3576 3559   0 186 5395   113 15326  0 29 71
1 0 0 185M   20M     0   0   0   0  7097 7043   0 155 5046   119 14892  0 32 68
1 0 0 185M   13M     0   0   0   0  3555 3552   1 166 6733   123 18475  0 38 62
0 0 0 185M   13M     0   0   0   0  3549 3554   0 167 4390   113 12292  0 26 74
1 0 0 185M   10M     0   0   0   0  3560 3559   0 140 4830   113 13601  0 18 82
1 0 0 185M  8.3M     0   0   0   0  3552 3558   0 122 4864   113 13598  0 24 76
0 0 0 185M  7.0M     0   0   0   0  3552 3559   0 122 6095   119 15927  0 23 77
1 0 0 185M   16M     0   0   0   0  7104 7116   0 151 6270   113 17106  1 32 67
procs  memory       page                    disks     faults         cpu
r b w  avm   fre   flt  re  pi  po    fr   sr vt0 vt1   in    sy    cs us sy id
1 0 0 185M  8.6M     0   0   0   0     8    6   0 131 2085   115  5999  0 13 87
0 0 0 185M   15M     0   0   0   0  7120 7111   0 175 5725   113 16732  0 33 67
1 0 0 185M  8.6M     0   0   0   0  3552 3557   0 159 5678   113 16201  0 30 70
1 0 0 185M   17M     0   0   0   0  7104 7085   0 154 5627   119 15754  0 22 78
1 0 0 185M   18M     0   0   0   0  3569 3550   0 189 3638   113 10382  0 23 77
1 0 0 185M   10M     0   0   0   0  3548 3555   0 175 5039   116 15075  0 43 57
1 0 0 185M   20M     0   0   0   0  7104 7115   0 140 4805   113 13753  0 31 69
1 0 0 185M   14M     0   0   0   0  3552 3557   0 161 7209   113 19426  0 38 62
0 0 0 185M   14M     0   0   0   0  3552 3557   0 176 5632   120 14626  0 22 78
1 0 0 185M  9.2M     0   0   0   0  3560 3560   0 159 6185   113 17012  0 32 68
0 0 0 185M  8.2M     0   0   0   0  3552 3557   0 112 4000   113 11623  0 32 68
0 0 0 185M   19M     0   0   0   0  7104 7117   0 133 5116   113 14320  0 22 78
0 0 0 185M   18M     0   0   0   0  3552 3557   0 115 3820   113 11399  1 27 72
0 0 0 185M   10M     0   0   0   0     0    6   0 136 2084   119  5948  0 12 88
1 0 0 185M   19M     0   0   0   0  7128 6990   0 175 6408   113 17448  0 29 71
0 0 0 185M   15M     0   0   0   0  3552 3557   0 136 5663   113 15647  0 32 68
1 0 0 185M   12M     0   0   0   0  3552 3559  13 141 6497   113 17290  0 33 67
1 0 0 185M  8.4M     0   0   0   0  3549 3553   0 132 5125   113 14359  0 32 68
1 0 0 185M  8.3M     0   0   0   0  3556 3554   0 190 5281   119 13959  1 27 73
1 0 0 185M  8.4M     0   0   0   0  3546 3553   1 109 4840   113 13053  0 29 71
1 0 0 185M   18M     0   0   0   0  7104 7117   0 146 5152   113 14566  0 30 70
1 0 0 185M   14M     0   0   0   0  3552 3557   0 147 5826   113 16111  0 30 70
0 0 0 185M   15M     0   0   0   0  3552 3557   0 163 4477   113 12164  0 30 70
1 0 0 185M   12M     0   0   0   0  3584 3560   0 174 5637   120 15512  0 28 72
0 0 0 185M   19M     0   0   0   0  7104 7031   0 167 6171   113 17254  0 34 66
0 0 0 185M   15M     0   0   0   0     0    6   0  34 1208   113  3526  0  7 93
1 0 0 185M   15M     0   0   0   0     0    6   0 129  227   113   761  0  3 97
0 0 0 185M   15M     0   0   0   0     0    6   0 200  328   113  1095  0  3 97
0 0 0 185M   15M     0   0   0   0     0    6   0 196  317   119  1078  0  5 95
1 0 0 185M   15M     0   0   0   0     0    6   0 205  330   113  1099  0  2 98
1 0 0 185M   15M     0   0   0   0     0    6   0 201  328   113  1098  0  2 98
1 0 0 185M   15M     0   0   0   0     0    6   0 198  323   113  1083  0  3 97
1 0 0 185M   15M     0   0   0   0     0    6   0 167  278   113   934  0  6 94
1 0 0 185M   15M     0   0   0   0     0    6   0 177  287   119   963  0  3 97
0 0 0 185M   15M     0   0   0   0     0    6   0 122  200   116   697  0  5 95
1 0 0 185M   15M     0   0   0   0     0    6   0 182  295   113   986  0  5 95
1 0 0 185M   15M     0   0   0   0     0    6   0 200  323   113  1085  0  4 96
1 0 0 185M   15M     0   0   0   0     0    6   0 200  326   114  1089  0  3 97
1 0 0 185M   15M     0   0   0   0     0    6   0 200  321   119  1078  0  2 98
procs  memory       page                    disks     faults         cpu
r b w  avm   fre   flt  re  pi  po    fr   sr vt0 vt1   in    sy    cs us sy id
2 0 0 185M   15M     0   0   0   0     0    6   0 199  319   115  1080  0  5 95
1 0 0 185M   15M     0   0   0   0     0    6   0 144  245   113   825  0  4 96
1 0 0 185M   15M     0   0   0   0     0    6   0 150  255   114   852  0  2 98
1 0 0 185M   15M     0   0   0   0     0    6   0 200  324   113  1089  0  5 95
1 0 0 185M   15M     0   0   0   0     0    6   0 201  322   119  1087  0  5 95
1 0 0 185M   15M     0   0   0   0     0    6   0 200  323   113  1089  0  8 92
1 0 0 185M   15M     0   0   0   0     0    6   0 199  325   113  1086  0  1 99
1 0 0 185M   15M     0   0   0   0     0    6   0 201  320   113  1085  0  2 98
0 0 0 185M   14M     0   0   0   0     0    6   0 124  227   113   781  0  3 97
1 0 0 185M   14M     0   0   0   0     0    6   0 207  346   119  1150  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 184  307   113  1019  0  1 99
0 0 0 185M   14M     0   0   0   0     0    6   0 196  325   113  1079  0  5 95
1 0 0 194M   13M   693   0   0   0    89    6   0 202  332   757  1117  0  4 96
1 0 0 194M   12M    83   0   0   0     8    6   0 201  330   137  1112  0  3 97
1 0 0 185M   14M   417   0   4   0   825    6   9 200  347   547  1159  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 196  314   113  1058  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 179  300   113  1000  0  3 97
1 0 0 185M   14M     0   0   0   0     0    6   0 141  241   113   812  0  3 97
0 0 0 185M   14M     0   0   0   0     0    6   0 209  354   119  1173  0  7 93
1 0 0 185M   14M     0   0   0   0     0    6   0 204  334   113  1116  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 200  318   113  1074  0  4 96
1 0 0 185M   14M     0   0   0   0     0    6   0 199  316   113  1079  0  4 96
1 0 0 185M   14M     0   0   0   0     0    6   0 200  320   113  1082  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 200  324   119  1099  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 200  323   113  1087  0  5 95
0 0 0 185M   14M     0   0   0   0     0    6   0 192  317   116  1074  0  5 95
0 0 0 185M   14M     0   0   0   0     0    6   0 197  322   113  1088  0  4 96
1 0 0 185M   14M     0   0   0   0     0    6   0 127  205   113   709  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 198  313   120  1067  0  4 96
1 0 0 185M   14M     0   0   0   0     0    6   0 203  332   113  1117  0  5 95
1 0 0 185M   14M     0   0   0   0     0    6   0 200  325   113  1092  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 200  319   113  1080  0  3 97
1 0 0 185M   14M     0   0   0   0     0    6   0 200  321   113  1087  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 192  308   119  1048  0  3 97
1 0 0 185M   14M     0   0   0   0     0    6   0 204  338   113  1120  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   1 204  325   113  1104  0  6 94
1 0 0 185M   14M     0   0   0   0     0    6   0 186  317   113  1047  0  7 93
1 0 0 185M   14M     0   0   0   0     0    6   0 214  346   113  1166  0  3 97
1 0 0 185M   14M     0   0   0   0     0    6   0 199  329   119  1103  0  3 97
procs  memory       page                    disks     faults         cpu
r b w  avm   fre   flt  re  pi  po    fr   sr vt0 vt1   in    sy    cs us sy id
1 0 0 185M   14M     0   0   0   0     0    6   0 200  325   115  1093  0  3 97
1 0 0 185M   14M     0   0   0   0     0    6   0 201  339   113  1118  0  6 94
1 0 0 185M   14M     0   0   0   0     0    6  15 200  328   113  1086  0  5 95
1 0 0 185M   14M     0   0   0   0     0    6   0 200  323   113  1091  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 200  326   119  1094  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   1 197  325   113  1089  0  4 96
1 0 0 185M   14M     0   0   0   0     0    6   1 203  326   113  1101  0  5 95
1 0 0 185M   14M     0   0   0   0     0   12   0 200  330   113  1112  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 200  321   113  1075  0  5 95
1 0 0 185M   14M     0   0   0   0     0    6   0 198  326   119  1097  0  5 95
1 0 0 185M   14M     0   0   0   0     0    6   0 201  327   113  1101  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 201  320   113  1085  0  4 96
1 0 0 185M   14M     0   0   0   0     0    6   0 200  329   113  1103  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 150  242   113   828  0  3 97
1 0 0 185M   14M     0   0   0   0     8    6   0 203  328   119  1099  0  5 95
1 0 0 185M   14M     0   0   0   0     0    6   0 198  311   116  1059  0  4 96
1 0 0 185M   14M     0   0   0   0     0    6   0 198  322   113  1088  0  4 96
0 0 0 185M   14M     0   0   0   0     0    6   0 146  249   113   859  1  4 96
1 0 0 185M   14M     0   0   0   0     0    6   0 183  304   114  1004  0  3 97
1 0 0 185M   14M     0   0   0   0     0    6   0 131  223   119   764  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 207  341   113  1139  0  3 97
2 0 0 185M   14M     0   0   0   0     0    6   0 200  319   113  1082  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 200  329   114  1097  0  6 94
1 0 0 185M   14M     0   0   0   0     0    6   0 200  324   113  1081  0  3 97
1 0 0 185M   14M     0   0   0   0     0    6   0 198  328   119  1092  0  3 97
1 0 0 185M   14M     0   0   0   0     0    6   0 200  326   113  1095  0  5 95
1 0 0 185M   14M     0   0   0   0     0    6   0 188  313   113  1059  0  3 97
1 0 0 185M   14M     0   0   0   0     0    6   0 151  268   113   896  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 219  355   113  1191  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 189  311   119  1044  1  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 212  339   113  1132  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 194  329   113  1092  0  5 95
2 0 0 194M   13M   669   0   0   0    89    6   0 185  300   755  1015  2  5 94
1 0 0 194M   12M   108   0   0   0     0    6   0 216  361   131  1213  1  5 95
1 0 0 185M   14M    60   0   0   0   544    6   0 194  328   190  1099  0  6 94
1 0 0 185M   14M     0   0   0   0     0    6   0 205  341   113  1131  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 200  324   113  1082  0  3 97
1 0 0 185M   14M     0   0   0   0     0    6   0 200  327   113  1102  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 200  323   113  1079  0  3 97
procs  memory       page                    disks     faults         cpu
r b w  avm   fre   flt  re  pi  po    fr   sr vt0 vt1   in    sy    cs us sy id
1 0 0 185M   14M     0   0   0   0     0    6   0 135  245   121   816  0  4 96
1 0 0 185M   14M     0   0   0   0     0    6   0 206  337   113  1120  0  4 96
1 0 0 185M   14M     0   0   0   0     0    6   0 200  340   113  1125  1  3 96
1 0 0 185M   14M     0   0   0   0     0    6   0 190  321   113  1066  0  3 97
1 0 0 185M   14M     0   0   0   0     0    6   0 209  340   113  1146  0  5 95
0 0 0 185M   14M     0   0   0   0     0    6   0 153  257   119   869  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 194  324   116  1082  0  1 99
1 0 0 185M   14M     0   0   0   0     0    6   0 194  313   113  1051  0  2 98
0 0 0 185M   14M     0   0   0   0     0    6   0 191  312   113  1054  0  5 95
1 0 0 185M   14M     0   0   0   0     0    6   0 205  344   114  1129  0  5 95
1 0 0 185M   14M     0   0   0   0     0    6   0 200  326   119  1098  0  0 100
1 0 0 185M   14M     0   0   0   0     0    6   0 194  320   113  1084  0  3 97
1 0 0 185M   14M     0   0   0   0     0    6   0 129  219   113   738  0  3 97
1 0 0 185M   14M     0   0   0   0     0    6   0 214  382   113  1279  0  2 98
2 0 0 185M   14M     0   0   0   0     0    6   0 195  321   119  1066  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 205  332   113  1114  0  4 96
1 0 0 185M   14M     0   0   0   0     0    6   0 142  228   113   790  0  4 96
0 0 0 185M   14M     0   0   0   0     0   12   0 162  282   113   957  0  1 99
1 0 0 185M   14M     0   0   0   0     0    6   0 145  242   113   818  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 195  332   119  1102  0  4 96
1 0 0 185M   14M     0   0   0   0     0    6   0 206  326   113  1108  0  3 97
0 0 0 185M   14M     0   0   0   0     0    6   0 201  351   113  1147  0  4 96
1 0 0 185M   14M     0   0   0   0     0    6  13 150  254   113   847  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 195  333   113  1106  0  3 97
1 0 0 185M   14M     0   0   0   0     0    6   0 149  247   119   833  0  3 97
0 0 0 185M   14M     0   0   0   0     0    6   1 201  327   113  1118  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 200  330   113  1108  1  2 97
1 0 0 185M   14M     0   0   0   0     0    6   0 200  328   113  1102  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 196  308   113  1045  0  3 97
1 0 0 185M   14M     0   0   0   0     0    6   0 202  320   119  1087  0  3 97
1 0 0 185M   14M     0   0   0   0     0    6   0 202  331   113  1105  0  5 95
1 0 0 185M   14M     0   0   0   0     0    6   0 144  240   113   816  0  1 99
1 0 0 185M   14M     0   0   0   0     0    6   0 204  331   113  1100  0  4 96
1 0 0 185M   14M     0   0   0   0     0    6   0 207  332   113  1117  0  4 96
1 0 0 185M   14M     0   0   0   0     0    6   0 201  335   119  1104  1  3 96
0 0 0 185M   14M     0   0   0   0     0    6   0 137  230   116   782  0  3 97
1 0 0 185M   14M     0   0   0   0     0    6   0 205  334   113  1120  0  7 93
1 0 0 185M   14M     0   0   0   0     0    6   0 189  311   113  1046  0  1 99
1 0 0 185M   14M     0   0   0   0     0    6   0 213  343   114  1159  0  1 99
procs  memory       page                    disks     faults         cpu
r b w  avm   fre   flt  re  pi  po    fr   sr vt0 vt1   in    sy    cs us sy id
1 0 0 185M   14M     0   0   0   0     0    6   0 196  313   121  1069  0  5 95
1 0 0 185M   14M     0   0   0   0     0    6   0 149  242   113   832  0  2 98
0 0 0 185M   14M     0   0   0   0     0    6   0 204  329   113  1232  0  5 95
1 0 0 185M   14M     0   0   0   0     0    6   0 202  336   114  1230  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 199  322   113  1195  0  1 99
1 0 0 185M   14M     0   0   0   0     0    6   0 196  324   119  1196  0  1 99
1 0 0 185M   14M     0   0   0   0     0    6   0 203  336   113  1239  0  2 98
0 0 0 185M   14M     0   0   0   0     0    6   0 181  301   113  1127  0  3 97
1 0 0 185M   14M     0   0   0   0     0    6   0 168  273   113  1021  0  4 96
1 0 0 185M   14M     0   0   0   0     0    6   0 200  326   113  1203  0  2 98
1 0 0 185M   14M     0   0   0   0     0    6   0 200  327   119  1209  0  3 97
1 0 0 185M   14M     0   0   0   0     0    6   0 198  337   113  1223  0  0 100
1 0 0 185M  805M     0   0   0   0 204764    6   0 134 2962   113  8116  0 23 77
0 0 0 185M  781M     0   0   0   0     0    6   0 189 5550   113 16270  0 35 65
1 0 0 185M  764M     0   0   0   0     0    6   0 122 5920   113 15700  0 20 80
0 0 0 185M  751M     0   0   0   0     0    6   0  94 4130   129 11380  0 15 85
1 0 0 185M  733M     0   0   0   0     0    6   0 150 6376   113 16980  0 27 73
1 0 0 185M  715M     0   0   0   0     0    6   0 146 4550   113 13495  0 36 64
1 0 0 185M  703M     0   0   0   0   128    6   0 201 3290   113  9471  0 29 71
1 0 0 185M  685M     0   0   0   0     0    6   0 143 5023   113 14457  0 34 66
0 0 0 185M  668M     0   0   0   0     0    6   0 136 4356   119 12937  0 26 74
1 0 0 185M  651M     0   0   0   0     0    6   0 135 4311   113 12766  0 34 66
0 0 0 185M  638M     0   0   0   0     8    6   0 166 3800   113 10717  0 27 73
0 0 0 185M  617M     0   0   0   0    24    6   0 169 6790   113 18618  0 30 70
1 0 0 185M  598M     0   0   0   0     0    6   0 155 5562   113 15897  0 22 78
1 0 0 185M  579M     0   0   0   0     0    6   0 153 5652   119 15987  0 27 73
1 0 0 194M  565M   669   0   0   0    89    6   0  96 3612   758 10157  2 22 76
1 0 0 194M  548M   102   0   0   0     8    6   0 189 5379   129 14843  2 31 67
0 0 0 185M  530M    56   0   0   0   542    6   0 156 6011   176 16873  0 36 64
1 0 0 185M  512M     0   0   0   0     0    6   0 143 5328   114 14996  0 26 74
1 0 0 185M  496M     0   0   0   0     0    6   0 130 5059   119 14133  0 19 81
1 0 0 185M  490M     0   0   0   0    24    6   0 146 2028   113  5674  0 17 83
1 0 0 185M  467M     0   0   0   0     0    6   0 178 8175   113 21991  0 38 62
1 0 0 185M  447M     0   0   0   0     0    6   0 156 6588   113 18076  0 25 75
1 0 0 185M  425M     0   0   0   0     0    6   0 172 6434   113 18132  0 24 76
0 0 0 185M  409M     0   0   0   0    24    6   0 178 4633   119 13306  0 33 67
0 0 0 185M  388M     0   0   0   0     0    6   0 168 7495   113 20156  0 37 63
1 0 0 185M  367M     0   0   0   0     0    6   0 158 6177   113 17393  0 34 66
1 0 0 185M  345M     0   0   0   0     0    6   0 172 5444   113 16209  0 40 60
procs  memory       page                    disks     faults         cpu
r b w  avm   fre   flt  re  pi  po    fr   sr vt0 vt1   in    sy    cs us sy id
2 0 0 185M  327M     0   0   0   0     8   12   0 199 5553   115 15367  0 32 68
1 0 0 185M  309M     0   0   0   0    96    6   0 161 4358   119 13069  0 32 68
1 0 0 185M  295M     0   0   0   0     0    6   0 106 3227   113  9676  0 23 77
2 0 0 185M  273M     0   0   0   0     0    6   0 172 6435   113 18212  0 40 60
1 0 0 185M  259M     0   0   0   0    24    6   0 178 4398   113 12316  0 33 67
2 0 0 185M  241M     0   0   0   0     0    6   0 149 5096   113 14722  0 34 66
1 0 0 185M  220M     0   0   0   0     0    6   0 157 5780   119 16484  0 33 67
0 0 0 185M  202M     0   0   0   0     0    6   0 150 5558   113 15582  0 26 74
0 0 0 185M  189M     0   0   0   0     8    6   0 183 3583   113 10375  0 26 74
0 0 0 185M  171M     0   0   0   0     0    6   0 151 6028   113 16607  0 37 63
2 0 0 185M  152M     0   0   0   0     0    6   0 151 6194   119 17016  0 27 73
1 0 0 185M  131M     0   0   0   0     0    6   0 161 6855   113 18762  0 29 71
1 0 0 185M  120M     0   0   0   0    24    6   0 180 3529   113  9845  0 20 80
0 0 0 185M   97M     0   0   0   0     0    6   0 180 7655   113 20837  0 36 64
1 0 0 185M   77M     0   0   0   0     0    6   0 158 5571   113 15924  0 34 66
1 0 0 185M   69M     0   0   0   0     0    6   0  69 2201   119  6459  0 18 82
1 0 0 185M   55M     0   0   0   0     0    6  13 114 4474   116 12420  0 26 74
1 0 0 185M   37M     0   0   0   0     8    6   0 193 5481   113 15189  0 26 74
1 0 0 185M   21M     0   0   0   0     0    6   0 133 4471   113 13024  0 23 77
0 0 0 185M   16M     0  30   0   0  3549 3495   0 146 5632   113 15786  0 37 63
1 0 0 185M  9.9M     0   0   0   0  3547 3551   1 161 5531   120 15815  0 34 66
1 0 0 185M   18M     0   0   0   0  3572 3555   0 124 1629   113  4765  0 11 89
1 0 0 185M   13M     0   0   0   0  3549 3556   0 153 4967   113 14580  0 23 77
1 0 0 185M   11M     0   0   0   0  3552 3558   0 128 4782   113 13421  0 26 74
1 0 0 185M   20M     0   0   0   0  7104 7115   0 143 5743   114 15997  0 31 69
1 0 0 185M   16M     0   0   0   0  3552 3557   0 141 6007   119 16400  0 38 62
1 0 0 185M   15M     0   0   0   0  3576 3559   0 196 4808   113 13195  0 31 69
1 0 0 185M  8.3M     0   0   0   0  3552 3557   0 166 6533   113 18116  0 34 66
1 0 0 185M   19M     0  33   0   0  7104 7137   0 142 5264   113 14784  0 32 68
0 0 0 185M   11M     0   0   0   0  3554 3550   0 168 7698   113 20555  0 32 68
1 0 0 185M   11M     0   0   0   0  3558 3557   0 195 4451   119 12421  0 32 68
0 0 0 185M   20M     0   0   0   0  7104 7114   0 150 4873   113 14266  0 25 75
1 0 0 185M   13M     0   0   0   0  3552 3557   0 159 4882   113 14671  0 29 71
1 0 0 185M  7.4M     0   0   0   0  3552 3557   0 153 5292   113 15430  0 35 65
1 0 0 185M  7.3M     0   0   0   0  3576 3560   0 207 4325   113 12065  0 20 80
0 0 0 185M   15M     0   0   0   0  7097 7106   1 160 5880   129 16663  0 47 53
1 0 0 185M  6.9M     0   0   0   0  3552 3557   0 168 7255   113 19773  0 37 63
1 0 0 185M   17M     0   0   0   0  7104 7114   0 142 6576   113 17433  0 31 69
1 0 0 185M  9.8M     0   0   0   0     8    6   0 132 2343   113  6568  0 15 85
procs  memory       page                    disks     faults         cpu
r b w  avm   fre   flt  re  pi  po    fr   sr vt0 vt1   in    sy    cs us sy id
2 0 0 185M   17M     0   0   0   0  7099 6998   0 164 6669   115 18336  0 36 64
1 0 0 185M   10M     0   0   0   0  3548 3553   0 165 6011   119 16928  0 31 69
1 0 0 185M   17M     0   0   0   0  7097 7105   0 166 5958   113 16904  0 40 60
1 0 0 185M   19M     0   0   0   0  3573 3556   0 181 2481   113  7869  0 15 85
0 0 0 185M   14M     0   0   0   0  3552 3558   0 151 3931   113 12355  0 34 66
1 0 0 185M  9.7M     0   0   0   0  3552 3557   0 137 3997   113 12233  0 29 71
0 0 0 185M   17M     0   0   0   0  7104 7115   0 165 4781   119 14492  0 29 71
0 0 0 185M  8.7M     0   0   0   0     0    6   0 143 2046   116  6195  0 14 86
0 0 0 185M   20M     0   0   0   0  7112 7111   0 129 5860   113 15714  0 31 69
0 0 0 185M   16M     0   0   0   0  3552 3468   0 147 6886   113 18143  0 30 70
0 0 0 185M   13M     0   0   0   0  3551 3557   0 133 4892   114 13815  0 30 70
0 0 0 185M  7.6M     0   0   0   0  3549 3554   0 155 5831   119 16350  0 36 64
1 0 0 185M  8.6M     0   0   0   0  3572 3553   0 183 2667   113  8421  0 21 79
1 0 0 185M   15M     0   0   0   0  7099 7111   0 166 5136   113 15355  0 32 68
1 0 0 185M  9.3M     0   0   0   0  3552 3557   0 158 4326   113 13462  0 30 70
1 0 0 185M   18M     0   0   0   0  7104 7114   0 157 4033   113 12737  0 37 63
1 0 0 185M   15M     0   0   0   0  3576 3558   0 187 3438   119 10723  0 27 73
1 0 0 185M   14M     0   0   0   0  3552 3559   0 126 4898   113 13664  0 26 74
1 0 0 194M  7.7M   670   0  27   0  3641 3557  46 139 4937   759 14211  0 33 67
1 0 0 194M   13M   106   0   0   0  3547 3552   0  63 2334   130  6662  2 14 84
1 0 0 185M  9.8M    52   0   0   0  4087 3536   2 151 5950   174 16635  1 27 72
1 0 0 185M  7.7M     0   0   0   0  3555 3554   0 202 4330   119 12538  0 17 83
1 0 0 185M   16M     0   0   0   0  7104 7114   0 157 5661   113 16158  0 32 68
0 0 0 185M   16M     0   0   0   0  3552 3557   8 107 4416   113 12025  0 25 75
1 0 0 185M   18M     0   0   0   0  3552 3561   0  99 3806   113 10588  0 17 83
1 0 0 185M   16M     0   0   0   0  3576 3557   0 187 5255   113 14359  0 26 74
0 0 0 185M   11M     0   0   0   0  3552 3558   0 155 5314   119 15344  0 34 66
1 0 0 185M  8.4M     0   0   0   0  3552 3557   0 124 4360   113 12596  0 27 73
1 0 0 185M   16M     0   0   0   0  7104 7115   0 160 5914   113 16646  0 27 73
0 0 0 185M   15M     0   0   0   0  3552 3542   0 136 3938   113 11562  0 30 70
1 0 0 185M   20M     0   0   0   0  3560 3559   0 126 2496   113  7294  0 16 84
1 0 0 185M  9.5M     0   0   0   0     0    7   0  94 2996   119  8450  0 16 84
1 0 0 185M   17M     0   0   0   0  7104 7112   0 158 6337   113 17511  0 28 72
1 0 0 185M   19M     0   0   0   0  3552 3559   0  88 3948   113 10910  0 22 78
0 0 0 185M   20M     0   0   0   0  3552 3559   0 114 4124   113 11433  0 20 80
1 0 0 185M   16M     0   0   0   0     0    6   0  31  729   113  2397  0  9 91
0 0 0 185M   11M     0   0   0   0     0    6   0  81 1023   119  3301  0 11 89
1 0 0 185M   10M     0   0   0   0  3576 3556   0 137 3385   116 10420  0 22 78
1 0 0 185M   19M     0   0   0   0  7104 7114   0 143 4690   113 13858  0 24 76
procs  memory       page                    disks     faults         cpu
r b w  avm   fre   flt  re  pi  po    fr   sr vt0 vt1   in    sy    cs us sy id
1 0 0 185M   18M     0   0   0   0  3552 3529   0 127 5219   115 14271  0 30 70
1 0 0 185M   14M     0   0   0   0  3552 3557   0 138 4711   114 13613  0 30 70
0 0 0 185M   17M     0   0   0   0  3557 3557   0 175 3369   119  9408  0 24 76
0 0 0 185M   13M     0   0   0   0  3552 3557   0 145 5127   113 14588  0 33 67
1 0 0 185M  9.4M     0   0   0   0  3552 3557   0 139 4925   113 14105  0 32 68
1 0 0 185M  7.7M     0   0   0   0  3576 3557   0 132 5706   114 15291  0 27 73
0 0 0 185M   17M     0   0   0   0  7104 7115   0 147 5817   113 16179  0 30 70
1 0 0 185M   20M     0   0   0   0  3576 3560   0 158 3231   119  9033  0 23 77
1 0 0 185M   15M     0   0   0   0  3552 3557  13 154 6170   113 17155  0 29 71
1 0 0 185M  9.7M     0   0   0   0  3552 3558   0 152 5908   113 16417  0 33 67
1 0 0 185M   17M     0   0   0   0  7102 7073   0 160 6372   113 17626  0 27 73
1 0 0 185M   20M     0   0   0   0  3569 3552   1 178 3637   113 10082  0 18 82
0 0 0 185M  8.9M     0   0   0   0     0    6   0  77 3271   119  9234  0 15 85
1 0 0 185M  8.0M     0   0   0   0  3552 3555   0 129 4843   113 13300  1 18 81
0 0 0 185M   16M     0   0   0   0  7104 7114   0 156 6118   113 17043  0 30 70
1 0 0 185M   17M     0   0   0   0  3552 3557   0 180 3850   113 10979  0 19 81
1 0 0 185M   17M     0   0   0   0     0    6   1 200  321   123  1087  0  3 97
0 0 0 185M   17M     0   0   0   0     0    6   0 198  332   119  1122  0  4 96


=== 2 GB LOCAL WRITE ===

procs  memory       page                    disks     faults         cpu
r b w  avm   fre   flt  re  pi  po    fr   sr vt0 vt1   in    sy    cs us sy id
1 0 0 163M  801M     4   0   0   0     0    5   0   0   19   233   106  1  0 99
0 0 0 163M  801M     0   0   0   0     0    5   0   0   13   137    79  0  0 100
0 0 0 169M  778M   367   0   0   0    34    5   0 164  135   253   343  1  2 97
0 0 0 169M  752M     0   0   0   0     0    5   0 197  175   165   420  0  2 98
0 0 0 169M  726M     0   0   0   0     0    5   0 196  176   163   416  0  2 98
0 0 0 169M  700M     0   0   0   0     0    5   0 208  186   165   439  0  5 95
0 0 0 169M  674M     0   0   0   0     0    5   0 201  195   165   470  1  4 95
0 0 0 169M  648M     0   0   0   0     0    5   0 203  195   169   470  0  8 92
0 0 0 169M  623M     0   0   0   0     0    5   0 200  216   170   515  0  5 95
0 0 0 169M  597M     0   0   0   0     0    5   0 200  204   163   486  0  5 95
1 0 0 169M  572M     0   0   0   0     0    5   0 200  220   163   519  0  5 95
0 0 0 169M  546M     0   0   0   0     0    5   0 204  213   166   520  0  2 98
0 0 0 169M  521M     0   0   0   0     0    5   0 200  177   169   432  0  5 95
0 0 0 169M  495M     0   0   0   0     0    5   2 207  193   163   481  0  3 97
0 0 0 169M  470M     0   0   0   0     0    5   0 190  159   161   400  0  2 98
0 0 0 179M  446M   677   0  26   0    97    5  41 166  192   800   503  1  3 96
0 0 0 179M  419M   106   0   0   0     0    6   0 208  178   183   452  0  5 95
0 0 0 169M  395M    52   0   0   0   541    6   2 202  192   230   482  0  3 97
0 0 0 169M  376M     0   0   0   0   136    5   9 147  139   151   359  1  9 91
0 0 0 169M  349M     0   0   0   0     0    5   0 215  176   167   430  0  2 98
0 0 0 169M  323M     0   0   0   0     0    5   0 203  196   165   469  0  4 96
0 0 0 169M  299M     0   0   0   0     0    5   0 189  185   167   452  0  3 97
0 0 0 169M  276M     0   0   0   0     0    5   0 179  140   161   362  0  3 97
0 0 0 169M  249M     0   0   0   0     0    5   0 207  172   165   431  0  5 95
0 0 0 169M  223M     0   0   0   0     0    5   0 199  172   165   429  0  3 97
0 0 0 169M  198M     0   0   0   0     0    5   0 203  135   165   351  0  5 95
0 0 0 169M  173M     0   0   0   0     0    5   0 198  163   169   410  0  5 95
0 0 0 169M  147M     0   0   0   0     0    5   0 190  153   163   380  0  4 96
0 0 0 169M  124M     0   0   0   0     0    5   0 185  168   159   410  0  4 96
0 0 0 169M  100M     0   0   0   0    72    5   0 201  201   163   483  0  3 97
0 0 0 169M   76M     0   0   0   0   136    5   0 214  176   159   457  0 10 90
0 0 0 169M   51M     0   0   0   0     8    5   0 200  190   169   457  0  5 95
1 0 0 169M   26M     0   0   0   0     0    5   0 201  192   163   459  0  5 95
0 0 0 169M   14M     0   4   0   0  3614   81   0 200  171   165   411  0  7 93
0 0 0 169M   17M     0   0   0   0  7251 7259   0 205  193   163   480  1  5 95
1 0 0 169M   19M     0   0   0   0  7177 7184   0 211  190   165   476  0  5 95
0 0 0 169M  8.0M     0   0   0   0  3596 3600   0 200  203   169   518  0  2 98
1 0 0 169M   11M     0   0   0   0  7245 7253   0 197  190   166   461  0  2 98
0 0 0 169M   13M     0   0   0   0  7245 7253   0 205  162   165   426  0  8 92
procs  memory       page                    disks     faults         cpu
r b w  avm   fre   flt  re  pi  po    fr   sr vt0 vt1   in    sy    cs us sy id
1 0 0 169M   16M     0   0   0   0  7293 7301   0 202  189   165   457  1  4 95
1 0 0 169M   19M     0   0   0   0  7225 7233   0 196  175   163   432  0  6 94
0 0 0 169M   12M     0   0   0   0  3553 3557   1 161  123   162   321  0  2 98
0 0 0 169M   15M     0   0   0   0  7289 7298   0 202  164   163   410  0  5 95
0 0 0 169M   18M     0  32   0   0  7301 7347   9 201  185   165   468  0  5 95
1 0 0 169M   21M     0   0   0   0  7197 7205   0 198  197   162   476  0  5 95
0 0 0 169M  8.5M     0   0   0   0  3644 3648   0 202  196   167   478  0  3 97
0 0 0 169M   12M     0   0   0   0  7357 7237  24 200  188   167   459  0  4 96
0 0 0 169M   14M     0   0   0   0  7122 7130   1 198  186   165   455  0  2 98
1 0 0 169M  7.7M     0   0   0   0  3589 3592   0 166  108   153   292  0  7 93
1 0 0 169M   12M     0   0   0   0  7309 7317   0 188  120   161   321  0  2 98
0 0 0 169M   14M     0   0   0   0  7293 7301   0 192  150   165   388  0  4 96
0 0 0 169M   21M     0   0   0   0  7197 7206   0 171  131   161   337  0  2 98
1 0 0 169M   10M     0   0   0   0  3548 3552   0 194  154   161   382  0  1 99
0 0 0 169M   11M     0   0   0   0  7293 7301   0 205  178   169   437  1  3 96
0 0 0 169M   14M     0   0   0   0  7273 7281   0 202  195   163   480  0  2 98
0 0 0 169M   16M     0   0   0   0  7230 7238   1 202  194   175   478  0  4 96
0 0 0 169M   19M     0   0   0   0  7197 7205   0 199  199   169   478  0  3 97
0 0 0 169M   21M     0   0   0   0  7225 7232   0 206  186   165   465  0  6 94
0 0 0 169M   12M     0   0   0   0  3764 3616   0 194  167   159   410  0  5 95
1 0 0 169M   19M     0   0   0   0  7277 7286   0 175  121   157   321  0  2 98
0 0 0 169M   21M     0   0   0   0  7189 7197   0 199  159   163   403  0  5 95
0 0 0 169M  9.7M     0   0   0   0  3653 3657   0 199  161   169   399  0  5 95
0 0 0 169M   11M     0   0   0   0  7229 7237   0 202  195   167   483  0  4 96
0 0 0 169M   13M     0   0   0   0  7229 7237   0 201  142   165   375  0  3 97
0 0 0 169M   20M     0   0   0   0  7241 7249   0 168  108   155   295  0  3 97
0 0 0 169M  8.9M     0   0   0   0  3616 3620   0 197  196   163   474  0  5 95
0 0 0 169M   18M     0   0   0   0  7225 7232   0 150  122   159   326  0  3 97
0 0 0 169M   18M     0   0   0   0  7229 7238   0 213  167   170   427  0  5 95
1 0 0 169M  7.0M     0   0   0   0  3549 3553   0 197  153   163   384  0  3 97
0 0 0 169M  7.9M     0   0   0   0  7309 7316   0 209  163   167   417  0  3 97
0 0 0 169M   11M     0   0   0   0  7229 7237   0 200  188   164   461  0  3 97
0 0 0 179M   10M   774   0  27   0  7207 7125  49 200  238   828   625  1  5 95
0 0 0 169M   15M    54   0   0   0  7780 7249   3 199  188   226   477  0  3 97
0 0 0 169M   16M     0   0   0   0  7220 7227   0 211  179   167   451  0  5 95
1 0 0 169M   19M     0   0   0   0  7213 7221   0 203  191   163   480  0  6 94
0 0 0 169M  7.5M     0   0   0   0  3692 3696   0 199  193   165   470  0  2 98
0 0 0 169M   10M     0   0   0   0  7197 7205   0 201  177   169   435  0  5 95
0 0 0 169M   13M     0   0   0   0  7229 7237   0 199  185   163   463  1  3 96
procs  memory       page                    disks     faults         cpu
r b w  avm   fre   flt  re  pi  po    fr   sr vt0 vt1   in    sy    cs us sy id
1 0 0 169M   15M     0   0   0   0  7261 7269   0 202  190   167   465  0  2 98
1 0 0 169M   18M     0   0   0   0  7209 7217   0 199  184   163   445  0  2 98
0 0 0 169M   21M     0   0   0   0  7197 7205   0 198  187   163   453  0  5 95
1 0 0 169M  9.7M     0   0   0   0  3644 3648   0 201  196   169   482  0  3 97
0 0 0 163M   13M    20   0   0   0  3901 3553   0 106   98   181   269  0  2 98
0 0 0 163M   13M     0   0   0   0    72    5   0   9   15   113    77  0  1 99
0 0 0 163M   13M     0   0   0   0     0   10   0   0   16   113    79  0  1 99
0 0 0 163M   13M     0   0   0   0     0    5   0   0   21   113    91  0  0 100



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1541525174.2175467.1567690832.1D278398>