From owner-freebsd-hackers@FreeBSD.ORG Mon May 13 15:37:18 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 23529AF1; Mon, 13 May 2013 15:37:18 +0000 (UTC) (envelope-from shiretu@gmail.com) Received: from mail-ea0-x22a.google.com (mail-ea0-x22a.google.com [IPv6:2a00:1450:4013:c01::22a]) by mx1.freebsd.org (Postfix) with ESMTP id 74763110; Mon, 13 May 2013 15:37:17 +0000 (UTC) Received: by mail-ea0-f170.google.com with SMTP id f15so2319364eak.29 for ; Mon, 13 May 2013 08:37:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:content-type:mime-version:subject:from:in-reply-to:date :cc:message-id:references:to:x-mailer; bh=9uRQsIQKkfBSQHvmyJNxI3q6nNKIQ+ry6tKFgk4aDQ4=; b=LJv1qtS4Rp6xDXukIcpbFnNn8krbkrFr+Y8W1+atgQ6/AO8BKIy+pR20ogy6/F/SoY Uf6PhCWTHkBHXs/kz3IDWpWkCyyMYoAcO7yjTViY7oAjyF36NmSaHbIA/mZJr5GwiJi/ vSyIdSDOYLy+pmIjT7kpeWOjdyIIz65IgmURH+rOQ62OfeWqUPNH3JKgON+xkRBDM8b1 sJiTAf8L64wgf6HPodiEMgrFii/RSoa3n/msckadQADbKXG6H9AIU3MdkyTNNReU0Dkn Siw3PWSdEfGhfy7jid3QMMLDXWKM3vk8s4WNiy4LnFL6T+nT7/3pSd1nCKoSlHIp3rZQ +Idw== X-Received: by 10.14.1.7 with SMTP id 7mr37131488eec.41.1368459436633; Mon, 13 May 2013 08:37:16 -0700 (PDT) Received: from [10.0.1.4] ([188.26.136.91]) by mx.google.com with ESMTPSA id s8sm23681226eeo.4.2013.05.13.08.37.14 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 13 May 2013 08:37:15 -0700 (PDT) Content-Type: multipart/signed; boundary="Apple-Mail=_C994B0A1-66E0-4EED-9E81-396CBD4C9EA8"; protocol="application/pkcs7-signature"; micalg=sha1 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: Managing userland data pointers in kqueue/kevent From: Eugen-Andrei Gavriloaie In-Reply-To: Date: Mon, 13 May 2013 18:37:13 +0300 Message-Id: <84DCA050-99D4-4B22-A031-35E0928709E0@gmail.com> References: To: Adrian Chadd X-Mailer: Apple Mail (2.1503) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 May 2013 15:37:18 -0000 --Apple-Mail=_C994B0A1-66E0-4EED-9E81-396CBD4C9EA8 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 Hi, Well, Paul already asked this question like 3-4 times now. Even = insisting on it. I will also ask it again: If user code is responsible of tracking down the data associated with = the signalled entity, what is the point of having user data? Is rendered completely useless=85 Not to mention, that your suggestion with FD index is a definite no-go. = The FD values are re-used. Especially in MT environments. Imagine one = kqueue call taking place in thread A and another one in thread B. Both = threads waiting for events. When A does his magic, because of internal business rules, it decides to = close FD number 123. It closes it and it connects somewhere else by = opening a new one. Surprise, we MAY get the value 123 again as a new = socket, we put it on our index, etc. Now, thread B comes in and it has = stale/old events for the old 123 FD. Somethings bad like EOF for the OLD = version of FD number 123 (the one we just closed anyway). Guess what=85 = thread B will deallocate the perfectly good thingy inside the index = associated with 123. And regarding the "thread happiness", that is not happiness at all IMHO=85= Best regards, Andrei ------ Eugen-Andrei Gavriloaie Web: http://www.rtmpd.com On May 13, 2013, at 6:25 PM, Adrian Chadd wrote: > ... or you could just track the per-descriptor / per-object stuff in > userland, and use the FD/signal as an index into the state you need. >=20 > adding thread happiness on top of that is trivial. >=20 > Done/done. >=20 >=20 >=20 >=20 > Adrian >=20 > On 13 May 2013 08:19, Eugen-Andrei Gavriloaie = wrote: >> Hello to all, >>=20 >> I'm trying to reply to this thread: >> = http://lists.freebsd.org/pipermail/freebsd-hackers/2010-November/033565.ht= ml >>=20 >> I also faced this very difficult task of tracking down the user data = registered into kq. >> I end up having some "Tokens" instances which I never deallocate but = always re-use them or create new ones if necessary. This tokens are used = as user data for kq. They are keeping the actual pointers inside them, = and the pointer itself has a reference to the Token. When the pointer = dies, I reset the guts of the token. When the time comes to use the = token, I have the guarantee is not the corpse of a token (never = deallocate them, remember?) and I can see that the actual pointer was = gone, everyone is happy. At the application shutdown, I cleanup the mess = (the tokens). However, I just want to say that Paul has a valid point = when he is wondering why EV_FREEWATCH was not provisioned/implemented. >>=20 >> The moment we throw multi-threading into equation, this becomes a = extremely hard thing to manage (close to impossible), including my = "proven-to-work" Token trick. It renders the user data pointer = completely useless because in the end we need to keep an association map = inside user space. That is forcing user space code to do a lookup before = use, instead of straightforward use. Not to mention the fact that we = need to perform a lock before searching it. That brings havoc on kernel = side on 1000+ active connections (a multi-threaded streaming server for = example). >>=20 >> I'm pretty sure this user data pointer is also breaking a well known = pointer management paradigm, but I just can't remember which. = Regardless, it has all the ingredients for memory leaks and/or, the = worst one, use of corpse pointers which are bound to crash the app. I = agree, C/C++ is not for the faint of heart, but with little or close to = no efforts, his EV_FREEWATCH can be put to very good use, and user space = code not only becomes less prone to mem issues, but also cleaner. >>=20 >> To summarise, +1 for the EV_FREEWATCH. I simply love the idea! Clean = and very very efficient. >>=20 >> Best regards, >> Andrei >>=20 >> ------ >> Eugen-Andrei Gavriloaie >> Web: http://www.rtmpd.com >>=20 --Apple-Mail=_C994B0A1-66E0-4EED-9E81-396CBD4C9EA8 Content-Disposition: attachment; filename=smime.p7s Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIO5jCCBJ0w ggOFoAMCAQICEDQ96SusJzT/j8s0lPvMcFQwDQYJKoZIhvcNAQEFBQAwbzELMAkGA1UEBhMCU0Ux FDASBgNVBAoTC0FkZFRydXN0IEFCMSYwJAYDVQQLEx1BZGRUcnVzdCBFeHRlcm5hbCBUVFAgTmV0 d29yazEiMCAGA1UEAxMZQWRkVHJ1c3QgRXh0ZXJuYWwgQ0EgUm9vdDAeFw0wNTA2MDcwODA5MTBa Fw0yMDA1MzAxMDQ4MzhaMIGuMQswCQYDVQQGEwJVUzELMAkGA1UECBMCVVQxFzAVBgNVBAcTDlNh bHQgTGFrZSBDaXR5MR4wHAYDVQQKExVUaGUgVVNFUlRSVVNUIE5ldHdvcmsxITAfBgNVBAsTGGh0 dHA6Ly93d3cudXNlcnRydXN0LmNvbTE2MDQGA1UEAxMtVVROLVVTRVJGaXJzdC1DbGllbnQgQXV0 aGVudGljYXRpb24gYW5kIEVtYWlsMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsjmF pPJ9q0E7YkY3rs3BYHW8OWX5ShpHornMSMxqmNVNNRm5pELlzkniii8efNIxB8dOtINknS4p1aJk xIW9hVE1eaROaJB7HHqkkqgX8pgV8pPMyaQylbsMTzC9mKALi+VuG6JG+ni8om+rWV6lL8/K2m2q L+usobNqqrcuZzWLeeEeaYji5kbNoKXqvgvOdjp6Dpvq/NonWz1zHyLmSGHGTPNpsaguG7bUMSAs vIKKjqQOpdeJQ/wWWq8dcdcRWdq6hw2v+vPhwvCkxWeM1tZUOt4KpLoDd7NlyP0e03RiqhjKaJMe oYV+9Udly/hNVyh00jT/MLbu9mIwFIws6wIDAQABo4H0MIHxMB8GA1UdIwQYMBaAFK29mHo0tCb3 +sQmVO8DveAky1QaMB0GA1UdDgQWBBSJgmd9xJ0mcABLtFBIfN49rgRufTAOBgNVHQ8BAf8EBAMC AQYwDwYDVR0TAQH/BAUwAwEB/zARBgNVHSAECjAIMAYGBFUdIAAwRAYDVR0fBD0wOzA5oDegNYYz aHR0cDovL2NybC51c2VydHJ1c3QuY29tL0FkZFRydXN0RXh0ZXJuYWxDQVJvb3QuY3JsMDUGCCsG AQUFBwEBBCkwJzAlBggrBgEFBQcwAYYZaHR0cDovL29jc3AudXNlcnRydXN0LmNvbTANBgkqhkiG 9w0BAQUFAAOCAQEAAbyc42MosPMxAcLfe91ioAGdIzEPnJJzU1HqH0z61p/Eyi9nfngzD3QWuZGH kfWKJvpkcADYHvkLBGJQh5OB1Nr1I9s0u4VWtHA0bniDNx6FHMURFZJfhxe9rGr98cLRzIlfsXzw PlHyNfN87GCYazor4O/fs32G67Ub9VvsonyYE9cAULnRLXPeA3h04QWFMV7LmrmdlMa5lDd1ctxE +2fo8PolHlKn2iXpR+CgxzygTrEKNvt3SJ/vl4r7tP7jlBSog7xcLT/SYHFg7sJxggzpiDbj2iC0 o6BsqpZLuICOdcpJB/Y7FLrf3AXZn9vgsuZNoHgm5+ctbn9fxh6IFTCCBRowggQCoAMCAQICEG0Z 6qcZT2ozIuYiMnqqcd4wDQYJKoZIhvcNAQEFBQAwga4xCzAJBgNVBAYTAlVTMQswCQYDVQQIEwJV VDEXMBUGA1UEBxMOU2FsdCBMYWtlIENpdHkxHjAcBgNVBAoTFVRoZSBVU0VSVFJVU1QgTmV0d29y azEhMB8GA1UECxMYaHR0cDovL3d3dy51c2VydHJ1c3QuY29tMTYwNAYDVQQDEy1VVE4tVVNFUkZp cnN0LUNsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQgRW1haWwwHhcNMTEwNDI4MDAwMDAwWhcNMjAw NTMwMTA0ODM4WjCBkzELMAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQ MA4GA1UEBxMHU2FsZm9yZDEaMBgGA1UEChMRQ09NT0RPIENBIExpbWl0ZWQxOTA3BgNVBAMTMENP TU9ETyBDbGllbnQgQXV0aGVudGljYXRpb24gYW5kIFNlY3VyZSBFbWFpbCBDQTCCASIwDQYJKoZI hvcNAQEBBQADggEPADCCAQoCggEBAJKEhFtLV5jUXi+LpOFAyKNTWF9mZfEyTvefMn1V0HhMVbdC lOD5J3EHxcZppLkyxPFAGpDMJ1Zifxe1cWmu5SAb5MtjXmDKokH2auGj/7jfH0htZUOMKi4rYzh3 37EXrMLaggLW1DJq1GdvIBOPXDX65VSAr9hxCh03CgJQU2yVHakQFLSZlVkSMf8JotJM3FLb3uJA AVtIaN3FSrTg7SQfOq9xXwfjrL8UO7AlcWg99A/WF1hGFYE8aIuLgw9teiFX5jSw2zJ+40rhpVJy ZCaRTqWSD//gsWD9Gm9oUZljjRqLpcxCm5t9ImPTqaD8zp6Q30QZ9FxbNboW86eb/8ECAwEAAaOC AUswggFHMB8GA1UdIwQYMBaAFImCZ33EnSZwAEu0UEh83j2uBG59MB0GA1UdDgQWBBR6E04AdFvG eGNkJ8Ev4qBbvHnFezAOBgNVHQ8BAf8EBAMCAQYwEgYDVR0TAQH/BAgwBgEB/wIBADARBgNVHSAE CjAIMAYGBFUdIAAwWAYDVR0fBFEwTzBNoEugSYZHaHR0cDovL2NybC51c2VydHJ1c3QuY29tL1VU Ti1VU0VSRmlyc3QtQ2xpZW50QXV0aGVudGljYXRpb25hbmRFbWFpbC5jcmwwdAYIKwYBBQUHAQEE aDBmMD0GCCsGAQUFBzAChjFodHRwOi8vY3J0LnVzZXJ0cnVzdC5jb20vVVROQWRkVHJ1c3RDbGll bnRfQ0EuY3J0MCUGCCsGAQUFBzABhhlodHRwOi8vb2NzcC51c2VydHJ1c3QuY29tMA0GCSqGSIb3 DQEBBQUAA4IBAQCF1r54V1VtM39EUv5C1QaoAQOAivsNsv1Kv/avQUn1G1rF0q0bc24+6SZ85kyY wTAo38v7QjyhJT4KddbQPTmGZtGhm7VNm2+vKGwdr+XqdFqo2rHA8XV6L566k3nK/uKRHlZ0sviN 0+BDchvtj/1gOSBH+4uvOmVIPJg9pSW/ve9g4EnlFsjrP0OD8ODuDcHTzTNfm9C9YGqzO/761Mk6 PB/tm/+bSTO+Qik5g+4zaS6CnUVNqGnagBsePdIaXXxHmaWbCG0SmYbWXVcHG6cwvktJRLiQfsrR eTjrtDP6oDpdJlieYVUYtCHVmdXgQ0BCML7qpeeU0rD+83X5f27nMIIFIzCCBAugAwIBAgIQY7lP +MJT4QzHh5dD4LWVRzANBgkqhkiG9w0BAQUFADCBkzELMAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdy ZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBxMHU2FsZm9yZDEaMBgGA1UEChMRQ09NT0RPIENBIExp bWl0ZWQxOTA3BgNVBAMTMENPTU9ETyBDbGllbnQgQXV0aGVudGljYXRpb24gYW5kIFNlY3VyZSBF bWFpbCBDQTAeFw0xMjA4MjEwMDAwMDBaFw0xMzA4MjEyMzU5NTlaMCIxIDAeBgkqhkiG9w0BCQEW EXNoaXJldHVAZ21haWwuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAziBH0E99 6jx0UrKapQcbxgWLgux1j9/Ec7K3QkkFtzCgxtF9ohpfsiAVOkYuCzVxoxoC0eNKO4qbZPYNbsN6 gtVZMrQnphgG3Wzmf28azn60TtCEQNoFXdW4On+cZgcvUxZS09DaPuTrHB5BuOmYG7b93ZjvJuov KCYAjYNrg0S9TKTQdRJAu95E2OuoVzin+QwIfEVwOA+PSRTPbNnjHvHUMg2G98CXFKIYPy9Pz8Kk rd54q7eSanjNiQK4Z9UYFjjNokx0nhymUAlnzWDAPj8jlh5i6gPrWsr9mYmQ6Y9wm2RHyf7QA6nM BIO43JNPklqjUT5LnB47rxVnLKnYmQIDAQABo4IB4TCCAd0wHwYDVR0jBBgwFoAUehNOAHRbxnhj ZCfBL+KgW7x5xXswHQYDVR0OBBYEFFoYqkq01lUdSiemIAjKHTuJiRj9MA4GA1UdDwEB/wQEAwIF oDAMBgNVHRMBAf8EAjAAMCAGA1UdJQQZMBcGCCsGAQUFBwMEBgsrBgEEAbIxAQMFAjARBglghkgB hvhCAQEEBAMCBSAwRgYDVR0gBD8wPTA7BgwrBgEEAbIxAQIBAQEwKzApBggrBgEFBQcCARYdaHR0 cHM6Ly9zZWN1cmUuY29tb2RvLm5ldC9DUFMwVwYDVR0fBFAwTjBMoEqgSIZGaHR0cDovL2NybC5j b21vZG9jYS5jb20vQ09NT0RPQ2xpZW50QXV0aGVudGljYXRpb25hbmRTZWN1cmVFbWFpbENBLmNy bDCBiAYIKwYBBQUHAQEEfDB6MFIGCCsGAQUFBzAChkZodHRwOi8vY3J0LmNvbW9kb2NhLmNvbS9D T01PRE9DbGllbnRBdXRoZW50aWNhdGlvbmFuZFNlY3VyZUVtYWlsQ0EuY3J0MCQGCCsGAQUFBzAB hhhodHRwOi8vb2NzcC5jb21vZG9jYS5jb20wHAYDVR0RBBUwE4ERc2hpcmV0dUBnbWFpbC5jb20w DQYJKoZIhvcNAQEFBQADggEBAHOQHTMzyiauVSuvCNgN2laY11tdDegdalfDK8UIFpdv08TE1viQ Q9gQfJpGNiO4wxpK8NeB8Qkh+AlL9ygq3jNwuAhk+TXMJ4Jr/I7LkTyJFnW45ADLZs/ufAbc+gdK lxdIxRJBOHmEpA51h0beSlfgHZcvvGshESzwLF290LuEdmLWncIWbhuO4vOJRK/7wJ/KXD8bZdx5 LGvA/QPPi5u95zFoQPRLSDl7lqJailBJZe6YFs7zBK2lm3u0w7exJw/rYeXFjMR0g5BPQCiihMq5 77k2T7uKmQqzapeCqLcd0sFNi94lzgSJxaQDyiDZYeYjWX0lSNVEG0z/fCIVJj4xggOrMIIDpwIB ATCBqDCBkzELMAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UE BxMHU2FsZm9yZDEaMBgGA1UEChMRQ09NT0RPIENBIExpbWl0ZWQxOTA3BgNVBAMTMENPTU9ETyBD bGllbnQgQXV0aGVudGljYXRpb24gYW5kIFNlY3VyZSBFbWFpbCBDQQIQY7lP+MJT4QzHh5dD4LWV RzAJBgUrDgMCGgUAoIIB1zAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEP Fw0xMzA1MTMxNTM3MTRaMCMGCSqGSIb3DQEJBDEWBBRlygkOdFbH71GuCSolNN2/AEc3wDCBuQYJ KwYBBAGCNxAEMYGrMIGoMIGTMQswCQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVz dGVyMRAwDgYDVQQHEwdTYWxmb3JkMRowGAYDVQQKExFDT01PRE8gQ0EgTGltaXRlZDE5MDcGA1UE AxMwQ09NT0RPIENsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBAhBjuU/4 wlPhDMeHl0PgtZVHMIG7BgsqhkiG9w0BCRACCzGBq6CBqDCBkzELMAkGA1UEBhMCR0IxGzAZBgNV BAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBxMHU2FsZm9yZDEaMBgGA1UEChMRQ09NT0RP IENBIExpbWl0ZWQxOTA3BgNVBAMTMENPTU9ETyBDbGllbnQgQXV0aGVudGljYXRpb24gYW5kIFNl Y3VyZSBFbWFpbCBDQQIQY7lP+MJT4QzHh5dD4LWVRzANBgkqhkiG9w0BAQEFAASCAQAt2uMcUqu8 ud3CnJfnlwKW+F28jHGLidDrUj4xagK2h02c8Z9ACSql2w80i05ef5yvgmEdzAgNrs/4App52FaF k+JFHzI2ImfrGGGHRUXHPdEUquSsPuul9xTUtn9OD/YJAv7O3eamaYcqgAxA70+l1gvkixtfri5k QBwYvax0jOn1qWJFS514KkXZZNp7SciFG5Yoy+mRB3lwgSJ1EVPqystTN5FBDlK5SlCeups1/Pns svACNarmK1UOHtpdYr4glcQh5uFhrC0LpGwyR5VLRikHtYI5Rn5aSgOlOpqxds/ZC6Dy0abmzpdO KEfOXkKuEga6Ls+0D3Z6mDQAkDxOAAAAAAAA --Apple-Mail=_C994B0A1-66E0-4EED-9E81-396CBD4C9EA8--