Date: Mon, 4 Jul 2016 21:32:49 -0500 From: Karl Denninger <karl@denninger.net> To: freebsd-hackers@freebsd.org Subject: Re: ZFS ARC and mmap/page cache coherency question Message-ID: <768b6169-70d9-5500-c455-563d8340972e@denninger.net> In-Reply-To: <34cf2d30-8884-95b6-f852-457d55710daf@freebsd.org> References: <20160630140625.3b4aece3@splash.akips.com> <CALXu0UfxRMnaamh%2Bpo5zp=iXdNUNuyj%2B7e_N1z8j46MtJmvyVA@mail.gmail.com> <20160703123004.74a7385a@splash.akips.com> <155afb8148f.c6f5294d33485.2952538647262141073@nextbsd.org> <45865ae6-18c9-ce9a-4a1e-6b2a8e44a8b2@denninger.net> <155b84da0aa.ad3af0e6139335.8627172617037605875@nextbsd.org> <7e00af5a-86cd-25f8-a4c6-2d946b507409@denninger.net> <34cf2d30-8884-95b6-f852-457d55710daf@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a cryptographically signed message in MIME format. --------------ms010208080905010008010306 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 7/4/2016 21:28, Allan Jude wrote: > On 2016-07-04 22:26, Karl Denninger wrote: >> >> On 7/4/2016 18:45, Matthew Macy wrote: >>> >>> ---- On Sun, 03 Jul 2016 08:43:19 -0700 Karl Denninger <karl@denning= er.net> wrote ----=20 >>> > =20 >>> > On 7/3/2016 02:45, Matthew Macy wrote:=20 >>> > > =20 >>> > > Cedric greatly overstates the intractability of reso= lving it. Nonetheless, since the initial import very little has been done= to improve integration, and I don't know of anyone who is up to the task= taking an interest in it. Consequently, mmap() performance is likely "do= omed" for the foreseeable future.-M---- =20 >>> > =20 >>> > Wellllll....=20 >>> > =20 >>> > I've done a fair bit of work here (see=20 >>> > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D187594) and th= e=20 >>> > political issues are at least as bad as the coding ones.=20 >>> > =20 >>> =20 >>> >>> Strictly speaking, the root of the problem is the ARC. Not ZFS per se= =2E Have you ever tried disabling MFU caching to see how much worse LRU o= nly is? I'm not really convinced the ARC's benefits justify its cost. >>> >>> -M >>> >> The ARC is very useful when it gets a hit as it avoid an I/O that woul= d >> otherwise take place. >> >> Where it sucks is when the system evicts working set to preserve ARC. = >> That's always wrong in that you're trading a speculative I/O (if the >> cache is hit later) for a *guaranteed* one (to page out) and maybe *tw= o* >> (to page back in.) >> > ZFS is better behaved in 11.x, there is a sysctl vfs.zfs.arc_free_targe= t > that makes sure the ARC is reined in when there is memory pressure, by > ensuring a minimum amount of actually free pages. > Oh, but..... Again, go read the PR I linked (and the current version of the patch against 10-STABLE.) The issues are far more intertwined than that.=20 Specifically, the dmu_tx cache decision (size of the write-back cache) is flat-out broken and inappropriate in essentially all cases, and the interaction of UMA and ARC is very destructive under a wide variety of workloads. The patch has hack-around for the dmu_tx problem and a reasonably-effective fix for the UMA issues. Actually fixing dmu_tx, however, is nowhere near that easy since it really needs to be computed per-zvol on an actual bytes moved per-unit-of-time basis. Note that one of the patches in the set I developed is indeed arc_free_target (indeed it was the first approach I took) -- but without addressing the other two issues it doesn't solve the problem. --=20 Karl Denninger karl@denninger.net <mailto:karl@denninger.net> /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms010208080905010008010306 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjA3MDUwMjMyNDlaME8GCSqGSIb3DQEJBDFCBEAV KMg4AHaoLMMe3So8i486K5oIjQpUdwmN9cYYgIYYLy9rftciot7s+0SuIHL4A7n80GJv/oWg 9uzBwLxM8hbqMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIAojhtlyEO eUqXIVLILPLpP1kqX5MZ+8QUJ+R/qE2CT/k59jwiQpxlfKinfuUoSJ/FIHjSlJX9Ky4x7bHg 9vulhAjnGvogIxmbdDLGGvbLhYb+hmWxTvugcLP8L3PrQ3Jya5hnmraZsU/ty84R40sO54Q1 ANySPdu0MtLhF8eMYPO2RmDk3I23WrSnQJpQlBwaBvO5hwPn5IZEVFJ2Cp3j0U02+O8Mvw2x MgiwWAE9uHfqVwRXwMVBfP+rnigZw03ocWf/GfLFvu7/Jz/Ce4KVuwx5Xt/jPIS2VRrLmL0F 1PTCH1MVgGsmDGn0EkwGIe6MyXbGoa1Ra/SoCAo9ROpGk/HlH1KPUlwGltJYtn8TzJBIHvWC nQ1kywEmPqU/8TXB4PgmcXqq6Wn9rR0rSi6cuwJzmswenV/UbD7pMhWHCOXK+23PQXxfv1vM rMmQKYrXSbyBDn8qfBMjenVwvAYt9S2wcz3JpDGCd+xRKV8AAXUGhRAe3uFmWTXn+qiAPF+k feMeDzeQ7UbCb2dM4JCuqtuef94qFAICgFzOg/FYcwQbcQu8blKvLDEDimhJEoUIk2nVgGG6 y+kxXbuGR3cNwu/mg4LoOx8DW9C8N9HNP9QSl4U4bG4VsGVSA+Vxd3eOw87TTGWOboavKUZM yhrT2cK0KifkNEBu5OyNwMEaOssAAAAAAAA= --------------ms010208080905010008010306--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?768b6169-70d9-5500-c455-563d8340972e>