Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 3 Apr 2014 17:00:01 GMT
From:      Karl Denninger <karl@denninger.net>
To:        freebsd-fs@FreeBSD.org
Subject:   Re: kern/187594: [zfs] [patch] ZFS ARC behavior problem and fix
Message-ID:  <201404031700.s33H01Hk087810@freefall.freebsd.org>

next in thread | raw e-mail | index | archive | help
The following reply was made to PR kern/187594; it has been noted by GNATS.

From: Karl Denninger <karl@denninger.net>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/187594: [zfs] [patch] ZFS ARC behavior problem and fix
Date: Thu, 03 Apr 2014 11:57:50 -0500

 This is a cryptographically signed message in MIME format.
 
 --------------ms040709030506010201090909
 Content-Type: text/plain; charset=ISO-8859-1; format=flowed
 Content-Transfer-Encoding: quoted-printable
 
 After more than a week of operation without any changes on a very busy=20
 production server this is what the status looks like at this particular=20
 moment in time (caught it being pretty quiet at the moment... slow day):
 
 [karl@NewFS ~]$ uptime
 11:56AM  up 10 days, 20:37, 1 user, load averages: 0.80, 0.59, 0.58
 
 [karl@NewFS ~]$ uname -v
 FreeBSD 10.0-STABLE #22 r263665:263671M: Sun Mar 23 15:00:48 CDT 2014
   karl@NewFS.denninger.net:/usr/obj/usr/src/sys/KSD-SMP
 
 
      1 users    Load  0.50  0.57  0.58                  Apr  3 11:52
 
 Mem:KB    REAL            VIRTUAL                       VN PAGER   SWAP P=
 AGER
          Tot   Share      Tot    Share    Free           in   out     in =
   out
 Act 4503936   32680  9319616    54908  701712  count
 All  17598k   42312 10162228   293268          pages
 Proc:                                                            Interrup=
 ts
    r   p   d   s   w   Csw  Trp  Sys  Int  Sof  Flt        ioflt  2635 to=
 tal
    2         245   3  9936 4302  12k  990  442 2878   1161 cow      11 ua=
 rt0 4
                                                       1460 zfod     53 uh=
 ci0 16
   0.6%Sys   0.1%Intr  1.5%User  0.0%Nice 97.8%Idle         ozfod       pc=
 m0 17
 |    |    |    |    |    |    |    |    |    |           %ozfod       ehc=
 i0 uhci
 >                                                         daefr       uhc=
 i1 21
                                             dtbuf     1779 prcfr   532 uh=
 ci3 ehci
 Namei     Name-cache   Dir-cache    485888 desvn     3862 totfr    44 twa=
 0 30
     Calls    hits   %    hits   %    145761 numvn          react   989 cp=
 u0:timer
     18611   18549 100                121467 frevn          pdwak    69 mp=
 s0 256
                                                        909 pdpgs    24 em=
 0:rx 0
 Disks   da0   da1   da2   da3   da4   da5   da6           intrn    32 em0=
 :tx 0
 KB/t  10.30 10.39  0.00  0.00 22.61 24.69 24.39  19017980 wire        em0=
 :link
 tps      21    21     0     0    10    16    16   2197580 act     118 em1=
 :rx 0
 MB/s   0.22  0.22  0.00  0.00  0.22  0.39  0.39   2544544 inact   107 em1=
 :tx 0
 %busy    19    19     0     0     0     1     1      3276 cache       em1=
 :link
                                                     698064 free        ah=
 ci0:ch0
                                                            buf      32 cp=
 u1:timer
                                                                     24 cp=
 u10:time
                                                                     50 cp=
 u6:timer
                                                                     26 cp=
 u12:time
                                                                     37 cp=
 u7:timer
                                                                     45 cp=
 u14:time
                                                                     41 cp=
 u4:timer
                                                                     35 cp=
 u15:time
                                                                     25 cp=
 u5:timer
                                                                     45 cp=
 u9:timer
                                                                     45 cp=
 u2:timer
                                                                    102 cp=
 u11:time
                                                                     63 cp=
 u3:timer
                                                                     41 cp=
 u13:time
                                                                     45 cp=
 u8:timer
 
 [karl@NewFS ~]$ zpool list
 NAME     SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
 media   2.72T  2.12T   616G    77%  1.00x  ONLINE  -
 zroot    234G  18.8G   215G     8%  1.36x  ONLINE  -
 zstore  3.63T  2.50T  1.13T    68%  1.00x  ONLINE  -
 
 [karl@NewFS ~]$ zfs-stats -A
 
 ------------------------------------------------------------------------
 ZFS Subsystem Report                            Thu Apr  3 11:53:42 2014
 ------------------------------------------------------------------------
 
 ARC Summary: (HEALTHY)
          Memory Throttle Count:                  0
 
 ARC Misc:
          Deleted:                                27.84m
          Recycle Misses:                         1.12m
          Mutex Misses:                           2.65k
          Evict Skips:                            39.26m
 
 ARC Size:                               59.13%  13.20   GiB
          Target Size: (Adaptive)         59.14%  13.20   GiB
          Min Size (Hard Limit):          12.50%  2.79    GiB
          Max Size (High Water):          8:1     22.33   GiB
 
 ARC Size Breakdown:
          Recently Used Cache Size:       81.41%  10.75   GiB
          Frequently Used Cache Size:     18.59%  2.46    GiB
 
 ARC Hash Breakdown:
          Elements Max:                           2.69m
          Elements Current:               63.22%  1.70m
          Collisions:                             95.13m
          Chain Max:                              24
          Chains:                                 413.62k
 
 ------------------------------------------------------------------------
 
 [karl@NewFS ~]$ zfs-stats -E
 
 ------------------------------------------------------------------------
 ZFS Subsystem Report                            Thu Apr  3 11:53:59 2014
 ------------------------------------------------------------------------
 
 ARC Efficiency:                                 1.28b
          Cache Hit Ratio:                98.37%  1.26b
          Cache Miss Ratio:               1.63%   20.80m
          Actual Hit Ratio:               60.07%  766.91m
 
          Data Demand Efficiency:         99.15%  435.02m
          Data Prefetch Efficiency:       20.45%  17.49m
 
          CACHE HITS BY CACHE LIST:
            Anonymously Used:             38.72%  486.24m
            Most Recently Used:           3.74%   46.94m
            Most Frequently Used:         57.33%  719.97m
            Most Recently Used Ghost:     0.06%   792.68k
            Most Frequently Used Ghost:   0.16%   1.97m
 
          CACHE HITS BY DATA TYPE:
            Demand Data:                  34.34%  431.32m
            Prefetch Data:                0.28%   3.58m
            Demand Metadata:              23.72%  297.92m
            Prefetch Metadata:            41.65%  523.09m
 
          CACHE MISSES BY DATA TYPE:
            Demand Data:                  17.75%  3.69m
            Prefetch Data:                66.88%  13.91m
            Demand Metadata:              5.78%   1.20m
            Prefetch Metadata:            9.60%   2.00m
 
 ------------------------------------------------------------------------
 
 Grinnin' big, in short.
 
 I have no reason to make further changes to the code or defaults.
 
 --=20
 -- Karl
 karl@denninger.net
 
 
 
 --------------ms040709030506010201090909
 Content-Type: application/pkcs7-signature; name="smime.p7s"
 Content-Transfer-Encoding: base64
 Content-Disposition: attachment; filename="smime.p7s"
 Content-Description: S/MIME Cryptographic Signature
 
 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFTzCC
 BUswggQzoAMCAQICAQgwDQYJKoZIhvcNAQEFBQAwgZ0xCzAJBgNVBAYTAlVTMRAwDgYDVQQI
 EwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM
 TEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkqhkiG9w0BCQEWIGN1c3Rv
 bWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0MB4XDTEzMDgyNDE5MDM0NFoXDTE4MDgyMzE5
 MDM0NFowWzELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExFzAVBgNVBAMTDkthcmwg
 RGVubmluZ2VyMSEwHwYJKoZIhvcNAQkBFhJrYXJsQGRlbm5pbmdlci5uZXQwggIiMA0GCSqG
 SIb3DQEBAQUAA4ICDwAwggIKAoICAQC5n2KBrBmG22nVntVdvgKCB9UcnapNThrW1L+dq6th
 d9l4mj+qYMUpJ+8I0rTbY1dn21IXQBoBQmy8t1doKwmTdQ59F0FwZEPt/fGbRgBKVt3Quf6W
 6n7kRk9MG6gdD7V9vPpFV41e+5MWYtqGWY3ScDP8SyYLjL/Xgr+5KFKkDfuubK8DeNqdLniV
 jHo/vqmIgO+6NgzPGPgmbutzFQXlxUqjiNAAKzF2+Tkddi+WKABrcc/EqnBb0X8GdqcIamO5
 SyVmuM+7Zdns7D9pcV16zMMQ8LfNFQCDvbCuuQKMDg2F22x5ekYXpwjqTyfjcHBkWC8vFNoY
 5aFMdyiN/Kkz0/kduP2ekYOgkRqcShfLEcG9SQ4LQZgqjMpTjSOGzBr3tOvVn5LkSJSHW2Z8
 Q0dxSkvFG2/lsOWFbwQeeZSaBi5vRZCYCOf5tRd1+E93FyQfpt4vsrXshIAk7IK7f0qXvxP4
 GDli5PKIEubD2Bn+gp3vB/DkfKySh5NBHVB+OPCoXRUWBkQxme65wBO02OZZt0k8Iq0i4Rci
 WV6z+lQHqDKtaVGgMsHn6PoeYhjf5Al5SP+U3imTjF2aCca1iDB5JOccX04MNljvifXgcbJN
 nkMgrzmm1ZgJ1PLur/ADWPlnz45quOhHg1TfUCLfI/DzgG7Z6u+oy4siQuFr9QT0MQIDAQAB
 o4HWMIHTMAkGA1UdEwQCMAAwEQYJYIZIAYb4QgEBBAQDAgWgMAsGA1UdDwQEAwIF4DAsBglg
 hkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFHw4
 +LnuALyLA5Cgy7T5ZAX1WzKPMB8GA1UdIwQYMBaAFF3U3hpBZq40HB5VM7B44/gmXiI0MDgG
 CWCGSAGG+EIBAwQrFilodHRwczovL2N1ZGFzeXN0ZW1zLm5ldDoxMTQ0My9yZXZva2VkLmNy
 bDANBgkqhkiG9w0BAQUFAAOCAQEAZ0L4tQbBd0hd4wuw/YVqEBDDXJ54q2AoqQAmsOlnoxLO
 31ehM/LvrTIP4yK2u1VmXtUumQ4Ao15JFM+xmwqtEGsh70RRrfVBAGd7KOZ3GB39FP2TgN/c
 L5fJKVxOqvEnW6cL9QtvUlcM3hXg8kDv60OB+LIcSE/P3/s+0tEpWPjxm3LHVE7JmPbZIcJ1
 YMoZvHh0NSjY5D0HZlwtbDO7pDz9sZf1QEOgjH828fhtborkaHaUI46pmrMjiBnY6ujXMcWD
 pxtikki0zY22nrxfTs5xDWGxyrc/cmucjxClJF6+OYVUSaZhiiHfa9Pr+41okLgsRB0AmNwE
 f6ItY3TI8DGCBQowggUGAgEBMIGjMIGdMQswCQYDVQQGEwJVUzEQMA4GA1UECBMHRmxvcmlk
 YTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExDMRwwGgYD
 VQQDExNDdWRhIFN5c3RlbXMgTExDIENBMS8wLQYJKoZIhvcNAQkBFiBjdXN0b21lci1zZXJ2
 aWNlQGN1ZGFzeXN0ZW1zLm5ldAIBCDAJBgUrDgMCGgUAoIICOzAYBgkqhkiG9w0BCQMxCwYJ
 KoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNDA0MDMxNjU3NTBaMCMGCSqGSIb3DQEJBDEW
 BBQtiQvKlj5Ru9Pv6YxPqFzRIx08NDBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjAL
 BglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFA
 MAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIG0BgkrBgEEAYI3EAQxgaYwgaMwgZ0xCzAJBgNV
 BAYTAlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoT
 EEN1ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkq
 hkiG9w0BCQEWIGN1c3RvbWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0AgEIMIG2BgsqhkiG
 9w0BCRACCzGBpqCBozCBnTELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExEjAQBgNV
 BAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1zIExMQzEcMBoGA1UEAxMTQ3Vk
 YSBTeXN0ZW1zIExMQyBDQTEvMC0GCSqGSIb3DQEJARYgY3VzdG9tZXItc2VydmljZUBjdWRh
 c3lzdGVtcy5uZXQCAQgwDQYJKoZIhvcNAQEBBQAEggIASySS2MCYfM+HpbvhybMFLOf39MPG
 9lKM9kaeWPUhLSzp5DqLZuL4cvQvel17Pr7SNed8aqtR+VoSQ3QyxDEdNRRh3jGA+Nk1GYdL
 UFaOPwxrmie0+IaMxjay5SoZoaBnuDL15QoExVIQIwindXtuX2R6ze97agYUn91exV/6GLhg
 51QuOancUKExT1zGOEMzLg2YxpYQOb6yicsjaqXOTLHcUGh+oZRRzNelEpUCSAotnkOa7ikx
 juNPdyfyJPZO4Oapvn0TwWY03JBX2BbhCRU5wLU7U0PPpExH/wbH1EpjXMT5Xx5g15EERk7N
 6E7nMROdEJmyK2N1pkD43paPX4oz5pjwiZZSOzr8HrV/pxzUitv2zOvonpmwYIpFlub9XGk1
 MUn2NNCpttbhLRgMSFFa99gakenZEjq3mrW4chJyHGg10FXh+Mrxh8Dv/HRB1sVRzp3D8LPw
 2nUw2MV9mhfZJGGz5QSkqudGwOkC7EMvHtdhEiyLWiHs6Ro9YWT65uGiz1uZcdhexGyjJjg8
 pSkcLaG++Ty+LtwJknhdwgHlDdDsThE1Zf8YXf10BA96uykt53sLbwnW20yy5FpGr8dcgiWh
 H4a84Div8YZ0OnR3+MyOXMU0+EBet1ojAarc2xMH6m+MnqIMl6nL9BhiEA51KE84odL6Nw8z
 sNJgryAAAAAAAAA=
 --------------ms040709030506010201090909--
 
 



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201404031700.s33H01Hk087810>