Date: Thu, 3 Apr 2014 17:00:01 GMT From: Karl Denninger <karl@denninger.net> To: freebsd-fs@FreeBSD.org Subject: Re: kern/187594: [zfs] [patch] ZFS ARC behavior problem and fix Message-ID: <201404031700.s33H01Hk087810@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
The following reply was made to PR kern/187594; it has been noted by GNATS.
From: Karl Denninger <karl@denninger.net>
To: bug-followup@FreeBSD.org
Cc:
Subject: Re: kern/187594: [zfs] [patch] ZFS ARC behavior problem and fix
Date: Thu, 03 Apr 2014 11:57:50 -0500
This is a cryptographically signed message in MIME format.
--------------ms040709030506010201090909
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: quoted-printable
After more than a week of operation without any changes on a very busy=20
production server this is what the status looks like at this particular=20
moment in time (caught it being pretty quiet at the moment... slow day):
[karl@NewFS ~]$ uptime
11:56AM up 10 days, 20:37, 1 user, load averages: 0.80, 0.59, 0.58
[karl@NewFS ~]$ uname -v
FreeBSD 10.0-STABLE #22 r263665:263671M: Sun Mar 23 15:00:48 CDT 2014
karl@NewFS.denninger.net:/usr/obj/usr/src/sys/KSD-SMP
1 users Load 0.50 0.57 0.58 Apr 3 11:52
Mem:KB REAL VIRTUAL VN PAGER SWAP P=
AGER
Tot Share Tot Share Free in out in =
out
Act 4503936 32680 9319616 54908 701712 count
All 17598k 42312 10162228 293268 pages
Proc: Interrup=
ts
r p d s w Csw Trp Sys Int Sof Flt ioflt 2635 to=
tal
2 245 3 9936 4302 12k 990 442 2878 1161 cow 11 ua=
rt0 4
1460 zfod 53 uh=
ci0 16
0.6%Sys 0.1%Intr 1.5%User 0.0%Nice 97.8%Idle ozfod pc=
m0 17
| | | | | | | | | | %ozfod ehc=
i0 uhci
> daefr uhc=
i1 21
dtbuf 1779 prcfr 532 uh=
ci3 ehci
Namei Name-cache Dir-cache 485888 desvn 3862 totfr 44 twa=
0 30
Calls hits % hits % 145761 numvn react 989 cp=
u0:timer
18611 18549 100 121467 frevn pdwak 69 mp=
s0 256
909 pdpgs 24 em=
0:rx 0
Disks da0 da1 da2 da3 da4 da5 da6 intrn 32 em0=
:tx 0
KB/t 10.30 10.39 0.00 0.00 22.61 24.69 24.39 19017980 wire em0=
:link
tps 21 21 0 0 10 16 16 2197580 act 118 em1=
:rx 0
MB/s 0.22 0.22 0.00 0.00 0.22 0.39 0.39 2544544 inact 107 em1=
:tx 0
%busy 19 19 0 0 0 1 1 3276 cache em1=
:link
698064 free ah=
ci0:ch0
buf 32 cp=
u1:timer
24 cp=
u10:time
50 cp=
u6:timer
26 cp=
u12:time
37 cp=
u7:timer
45 cp=
u14:time
41 cp=
u4:timer
35 cp=
u15:time
25 cp=
u5:timer
45 cp=
u9:timer
45 cp=
u2:timer
102 cp=
u11:time
63 cp=
u3:timer
41 cp=
u13:time
45 cp=
u8:timer
[karl@NewFS ~]$ zpool list
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
media 2.72T 2.12T 616G 77% 1.00x ONLINE -
zroot 234G 18.8G 215G 8% 1.36x ONLINE -
zstore 3.63T 2.50T 1.13T 68% 1.00x ONLINE -
[karl@NewFS ~]$ zfs-stats -A
------------------------------------------------------------------------
ZFS Subsystem Report Thu Apr 3 11:53:42 2014
------------------------------------------------------------------------
ARC Summary: (HEALTHY)
Memory Throttle Count: 0
ARC Misc:
Deleted: 27.84m
Recycle Misses: 1.12m
Mutex Misses: 2.65k
Evict Skips: 39.26m
ARC Size: 59.13% 13.20 GiB
Target Size: (Adaptive) 59.14% 13.20 GiB
Min Size (Hard Limit): 12.50% 2.79 GiB
Max Size (High Water): 8:1 22.33 GiB
ARC Size Breakdown:
Recently Used Cache Size: 81.41% 10.75 GiB
Frequently Used Cache Size: 18.59% 2.46 GiB
ARC Hash Breakdown:
Elements Max: 2.69m
Elements Current: 63.22% 1.70m
Collisions: 95.13m
Chain Max: 24
Chains: 413.62k
------------------------------------------------------------------------
[karl@NewFS ~]$ zfs-stats -E
------------------------------------------------------------------------
ZFS Subsystem Report Thu Apr 3 11:53:59 2014
------------------------------------------------------------------------
ARC Efficiency: 1.28b
Cache Hit Ratio: 98.37% 1.26b
Cache Miss Ratio: 1.63% 20.80m
Actual Hit Ratio: 60.07% 766.91m
Data Demand Efficiency: 99.15% 435.02m
Data Prefetch Efficiency: 20.45% 17.49m
CACHE HITS BY CACHE LIST:
Anonymously Used: 38.72% 486.24m
Most Recently Used: 3.74% 46.94m
Most Frequently Used: 57.33% 719.97m
Most Recently Used Ghost: 0.06% 792.68k
Most Frequently Used Ghost: 0.16% 1.97m
CACHE HITS BY DATA TYPE:
Demand Data: 34.34% 431.32m
Prefetch Data: 0.28% 3.58m
Demand Metadata: 23.72% 297.92m
Prefetch Metadata: 41.65% 523.09m
CACHE MISSES BY DATA TYPE:
Demand Data: 17.75% 3.69m
Prefetch Data: 66.88% 13.91m
Demand Metadata: 5.78% 1.20m
Prefetch Metadata: 9.60% 2.00m
------------------------------------------------------------------------
Grinnin' big, in short.
I have no reason to make further changes to the code or defaults.
--=20
-- Karl
karl@denninger.net
--------------ms040709030506010201090909
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature
MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFTzCC
BUswggQzoAMCAQICAQgwDQYJKoZIhvcNAQEFBQAwgZ0xCzAJBgNVBAYTAlVTMRAwDgYDVQQI
EwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM
TEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkqhkiG9w0BCQEWIGN1c3Rv
bWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0MB4XDTEzMDgyNDE5MDM0NFoXDTE4MDgyMzE5
MDM0NFowWzELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExFzAVBgNVBAMTDkthcmwg
RGVubmluZ2VyMSEwHwYJKoZIhvcNAQkBFhJrYXJsQGRlbm5pbmdlci5uZXQwggIiMA0GCSqG
SIb3DQEBAQUAA4ICDwAwggIKAoICAQC5n2KBrBmG22nVntVdvgKCB9UcnapNThrW1L+dq6th
d9l4mj+qYMUpJ+8I0rTbY1dn21IXQBoBQmy8t1doKwmTdQ59F0FwZEPt/fGbRgBKVt3Quf6W
6n7kRk9MG6gdD7V9vPpFV41e+5MWYtqGWY3ScDP8SyYLjL/Xgr+5KFKkDfuubK8DeNqdLniV
jHo/vqmIgO+6NgzPGPgmbutzFQXlxUqjiNAAKzF2+Tkddi+WKABrcc/EqnBb0X8GdqcIamO5
SyVmuM+7Zdns7D9pcV16zMMQ8LfNFQCDvbCuuQKMDg2F22x5ekYXpwjqTyfjcHBkWC8vFNoY
5aFMdyiN/Kkz0/kduP2ekYOgkRqcShfLEcG9SQ4LQZgqjMpTjSOGzBr3tOvVn5LkSJSHW2Z8
Q0dxSkvFG2/lsOWFbwQeeZSaBi5vRZCYCOf5tRd1+E93FyQfpt4vsrXshIAk7IK7f0qXvxP4
GDli5PKIEubD2Bn+gp3vB/DkfKySh5NBHVB+OPCoXRUWBkQxme65wBO02OZZt0k8Iq0i4Rci
WV6z+lQHqDKtaVGgMsHn6PoeYhjf5Al5SP+U3imTjF2aCca1iDB5JOccX04MNljvifXgcbJN
nkMgrzmm1ZgJ1PLur/ADWPlnz45quOhHg1TfUCLfI/DzgG7Z6u+oy4siQuFr9QT0MQIDAQAB
o4HWMIHTMAkGA1UdEwQCMAAwEQYJYIZIAYb4QgEBBAQDAgWgMAsGA1UdDwQEAwIF4DAsBglg
hkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFHw4
+LnuALyLA5Cgy7T5ZAX1WzKPMB8GA1UdIwQYMBaAFF3U3hpBZq40HB5VM7B44/gmXiI0MDgG
CWCGSAGG+EIBAwQrFilodHRwczovL2N1ZGFzeXN0ZW1zLm5ldDoxMTQ0My9yZXZva2VkLmNy
bDANBgkqhkiG9w0BAQUFAAOCAQEAZ0L4tQbBd0hd4wuw/YVqEBDDXJ54q2AoqQAmsOlnoxLO
31ehM/LvrTIP4yK2u1VmXtUumQ4Ao15JFM+xmwqtEGsh70RRrfVBAGd7KOZ3GB39FP2TgN/c
L5fJKVxOqvEnW6cL9QtvUlcM3hXg8kDv60OB+LIcSE/P3/s+0tEpWPjxm3LHVE7JmPbZIcJ1
YMoZvHh0NSjY5D0HZlwtbDO7pDz9sZf1QEOgjH828fhtborkaHaUI46pmrMjiBnY6ujXMcWD
pxtikki0zY22nrxfTs5xDWGxyrc/cmucjxClJF6+OYVUSaZhiiHfa9Pr+41okLgsRB0AmNwE
f6ItY3TI8DGCBQowggUGAgEBMIGjMIGdMQswCQYDVQQGEwJVUzEQMA4GA1UECBMHRmxvcmlk
YTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExDMRwwGgYD
VQQDExNDdWRhIFN5c3RlbXMgTExDIENBMS8wLQYJKoZIhvcNAQkBFiBjdXN0b21lci1zZXJ2
aWNlQGN1ZGFzeXN0ZW1zLm5ldAIBCDAJBgUrDgMCGgUAoIICOzAYBgkqhkiG9w0BCQMxCwYJ
KoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNDA0MDMxNjU3NTBaMCMGCSqGSIb3DQEJBDEW
BBQtiQvKlj5Ru9Pv6YxPqFzRIx08NDBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjAL
BglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFA
MAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIG0BgkrBgEEAYI3EAQxgaYwgaMwgZ0xCzAJBgNV
BAYTAlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoT
EEN1ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkq
hkiG9w0BCQEWIGN1c3RvbWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0AgEIMIG2BgsqhkiG
9w0BCRACCzGBpqCBozCBnTELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExEjAQBgNV
BAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1zIExMQzEcMBoGA1UEAxMTQ3Vk
YSBTeXN0ZW1zIExMQyBDQTEvMC0GCSqGSIb3DQEJARYgY3VzdG9tZXItc2VydmljZUBjdWRh
c3lzdGVtcy5uZXQCAQgwDQYJKoZIhvcNAQEBBQAEggIASySS2MCYfM+HpbvhybMFLOf39MPG
9lKM9kaeWPUhLSzp5DqLZuL4cvQvel17Pr7SNed8aqtR+VoSQ3QyxDEdNRRh3jGA+Nk1GYdL
UFaOPwxrmie0+IaMxjay5SoZoaBnuDL15QoExVIQIwindXtuX2R6ze97agYUn91exV/6GLhg
51QuOancUKExT1zGOEMzLg2YxpYQOb6yicsjaqXOTLHcUGh+oZRRzNelEpUCSAotnkOa7ikx
juNPdyfyJPZO4Oapvn0TwWY03JBX2BbhCRU5wLU7U0PPpExH/wbH1EpjXMT5Xx5g15EERk7N
6E7nMROdEJmyK2N1pkD43paPX4oz5pjwiZZSOzr8HrV/pxzUitv2zOvonpmwYIpFlub9XGk1
MUn2NNCpttbhLRgMSFFa99gakenZEjq3mrW4chJyHGg10FXh+Mrxh8Dv/HRB1sVRzp3D8LPw
2nUw2MV9mhfZJGGz5QSkqudGwOkC7EMvHtdhEiyLWiHs6Ro9YWT65uGiz1uZcdhexGyjJjg8
pSkcLaG++Ty+LtwJknhdwgHlDdDsThE1Zf8YXf10BA96uykt53sLbwnW20yy5FpGr8dcgiWh
H4a84Div8YZ0OnR3+MyOXMU0+EBet1ojAarc2xMH6m+MnqIMl6nL9BhiEA51KE84odL6Nw8z
sNJgryAAAAAAAAA=
--------------ms040709030506010201090909--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201404031700.s33H01Hk087810>