From owner-freebsd-hackers@FreeBSD.ORG Mon May 13 15:19:53 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 69313DF6 for ; Mon, 13 May 2013 15:19:53 +0000 (UTC) (envelope-from shiretu@gmail.com) Received: from mail-ee0-f47.google.com (mail-ee0-f47.google.com [74.125.83.47]) by mx1.freebsd.org (Postfix) with ESMTP id F3745EE0 for ; Mon, 13 May 2013 15:19:52 +0000 (UTC) Received: by mail-ee0-f47.google.com with SMTP id t10so1048611eei.6 for ; Mon, 13 May 2013 08:19:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:from:content-type:date:subject:to:message-id :mime-version:x-mailer; bh=jip9gP9EbwpLUtrFxxYwDihKGvICKPdmKd7Uq9+HQK0=; b=ERtwyiELxCfgYq9OcplF6ipgDCyVYmL1aM1WhSjNsQ5kI10z+ORyNnX3x0cGQ1Xn5p Vp5vwCpOStqC6OxQhyud0+bNx1mYi2mN5omXT9Xsm3nwz3qFyekt7EqHPb/f2Ld/6yFm nFMMa8f/4lbsLdjT1JEVb22e4v0SeITSFj6NCtmgMKW+CnN/iPnnvMQbdKzSR76215Yk RI3vw/J8ziOm0Nlj6R32NV+67zzvbEes5ESBftquZh2B2LpTSEdaeQM6+fhJ34zSj1ZZ 4IXeMYMcEBBAsFV4rxwNz48QdiIlNaCMS8xMnyziw0ocBuVhp+myQ2q3E7LezhlO17pq Sikg== X-Received: by 10.14.109.131 with SMTP id s3mr79432767eeg.26.1368458386315; Mon, 13 May 2013 08:19:46 -0700 (PDT) Received: from [10.0.1.4] ([188.26.136.91]) by mx.google.com with ESMTPSA id e50sm23564160eev.13.2013.05.13.08.19.44 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 13 May 2013 08:19:45 -0700 (PDT) From: Eugen-Andrei Gavriloaie Content-Type: multipart/signed; boundary="Apple-Mail=_DEAB8EC6-1EB6-4F18-8B38-E783ED902DC7"; protocol="application/pkcs7-signature"; micalg=sha1 Date: Mon, 13 May 2013 18:19:43 +0300 Subject: Managing userland data pointers in kqueue/kevent To: freebsd-hackers@freebsd.org Message-Id: Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) X-Mailer: Apple Mail (2.1503) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 May 2013 15:19:53 -0000 --Apple-Mail=_DEAB8EC6-1EB6-4F18-8B38-E783ED902DC7 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hello to all, I'm trying to reply to this thread: = http://lists.freebsd.org/pipermail/freebsd-hackers/2010-November/033565.ht= ml I also faced this very difficult task of tracking down the user data = registered into kq. I end up having some "Tokens" instances which I never deallocate but = always re-use them or create new ones if necessary. This tokens are used = as user data for kq. They are keeping the actual pointers inside them, = and the pointer itself has a reference to the Token. When the pointer = dies, I reset the guts of the token. When the time comes to use the = token, I have the guarantee is not the corpse of a token (never = deallocate them, remember?) and I can see that the actual pointer was = gone, everyone is happy. At the application shutdown, I cleanup the mess = (the tokens). However, I just want to say that Paul has a valid point = when he is wondering why EV_FREEWATCH was not provisioned/implemented.=20= The moment we throw multi-threading into equation, this becomes a = extremely hard thing to manage (close to impossible), including my = "proven-to-work" Token trick. It renders the user data pointer = completely useless because in the end we need to keep an association map = inside user space. That is forcing user space code to do a lookup before = use, instead of straightforward use. Not to mention the fact that we = need to perform a lock before searching it. That brings havoc on kernel = side on 1000+ active connections (a multi-threaded streaming server for = example).=20 I'm pretty sure this user data pointer is also breaking a well known = pointer management paradigm, but I just can't remember which. = Regardless, it has all the ingredients for memory leaks and/or, the = worst one, use of corpse pointers which are bound to crash the app. I = agree, C/C++ is not for the faint of heart, but with little or close to = no efforts, his EV_FREEWATCH can be put to very good use, and user space = code not only becomes less prone to mem issues, but also cleaner. To summarise, +1 for the EV_FREEWATCH. I simply love the idea! Clean and = very very efficient. Best regards, Andrei ------ Eugen-Andrei Gavriloaie Web: http://www.rtmpd.com --Apple-Mail=_DEAB8EC6-1EB6-4F18-8B38-E783ED902DC7 Content-Disposition: attachment; filename=smime.p7s Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIO5jCCBJ0w ggOFoAMCAQICEDQ96SusJzT/j8s0lPvMcFQwDQYJKoZIhvcNAQEFBQAwbzELMAkGA1UEBhMCU0Ux FDASBgNVBAoTC0FkZFRydXN0IEFCMSYwJAYDVQQLEx1BZGRUcnVzdCBFeHRlcm5hbCBUVFAgTmV0 d29yazEiMCAGA1UEAxMZQWRkVHJ1c3QgRXh0ZXJuYWwgQ0EgUm9vdDAeFw0wNTA2MDcwODA5MTBa Fw0yMDA1MzAxMDQ4MzhaMIGuMQswCQYDVQQGEwJVUzELMAkGA1UECBMCVVQxFzAVBgNVBAcTDlNh bHQgTGFrZSBDaXR5MR4wHAYDVQQKExVUaGUgVVNFUlRSVVNUIE5ldHdvcmsxITAfBgNVBAsTGGh0 dHA6Ly93d3cudXNlcnRydXN0LmNvbTE2MDQGA1UEAxMtVVROLVVTRVJGaXJzdC1DbGllbnQgQXV0 aGVudGljYXRpb24gYW5kIEVtYWlsMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsjmF pPJ9q0E7YkY3rs3BYHW8OWX5ShpHornMSMxqmNVNNRm5pELlzkniii8efNIxB8dOtINknS4p1aJk xIW9hVE1eaROaJB7HHqkkqgX8pgV8pPMyaQylbsMTzC9mKALi+VuG6JG+ni8om+rWV6lL8/K2m2q L+usobNqqrcuZzWLeeEeaYji5kbNoKXqvgvOdjp6Dpvq/NonWz1zHyLmSGHGTPNpsaguG7bUMSAs vIKKjqQOpdeJQ/wWWq8dcdcRWdq6hw2v+vPhwvCkxWeM1tZUOt4KpLoDd7NlyP0e03RiqhjKaJMe oYV+9Udly/hNVyh00jT/MLbu9mIwFIws6wIDAQABo4H0MIHxMB8GA1UdIwQYMBaAFK29mHo0tCb3 +sQmVO8DveAky1QaMB0GA1UdDgQWBBSJgmd9xJ0mcABLtFBIfN49rgRufTAOBgNVHQ8BAf8EBAMC AQYwDwYDVR0TAQH/BAUwAwEB/zARBgNVHSAECjAIMAYGBFUdIAAwRAYDVR0fBD0wOzA5oDegNYYz aHR0cDovL2NybC51c2VydHJ1c3QuY29tL0FkZFRydXN0RXh0ZXJuYWxDQVJvb3QuY3JsMDUGCCsG AQUFBwEBBCkwJzAlBggrBgEFBQcwAYYZaHR0cDovL29jc3AudXNlcnRydXN0LmNvbTANBgkqhkiG 9w0BAQUFAAOCAQEAAbyc42MosPMxAcLfe91ioAGdIzEPnJJzU1HqH0z61p/Eyi9nfngzD3QWuZGH kfWKJvpkcADYHvkLBGJQh5OB1Nr1I9s0u4VWtHA0bniDNx6FHMURFZJfhxe9rGr98cLRzIlfsXzw PlHyNfN87GCYazor4O/fs32G67Ub9VvsonyYE9cAULnRLXPeA3h04QWFMV7LmrmdlMa5lDd1ctxE +2fo8PolHlKn2iXpR+CgxzygTrEKNvt3SJ/vl4r7tP7jlBSog7xcLT/SYHFg7sJxggzpiDbj2iC0 o6BsqpZLuICOdcpJB/Y7FLrf3AXZn9vgsuZNoHgm5+ctbn9fxh6IFTCCBRowggQCoAMCAQICEG0Z 6qcZT2ozIuYiMnqqcd4wDQYJKoZIhvcNAQEFBQAwga4xCzAJBgNVBAYTAlVTMQswCQYDVQQIEwJV VDEXMBUGA1UEBxMOU2FsdCBMYWtlIENpdHkxHjAcBgNVBAoTFVRoZSBVU0VSVFJVU1QgTmV0d29y azEhMB8GA1UECxMYaHR0cDovL3d3dy51c2VydHJ1c3QuY29tMTYwNAYDVQQDEy1VVE4tVVNFUkZp cnN0LUNsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQgRW1haWwwHhcNMTEwNDI4MDAwMDAwWhcNMjAw NTMwMTA0ODM4WjCBkzELMAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQ MA4GA1UEBxMHU2FsZm9yZDEaMBgGA1UEChMRQ09NT0RPIENBIExpbWl0ZWQxOTA3BgNVBAMTMENP TU9ETyBDbGllbnQgQXV0aGVudGljYXRpb24gYW5kIFNlY3VyZSBFbWFpbCBDQTCCASIwDQYJKoZI hvcNAQEBBQADggEPADCCAQoCggEBAJKEhFtLV5jUXi+LpOFAyKNTWF9mZfEyTvefMn1V0HhMVbdC lOD5J3EHxcZppLkyxPFAGpDMJ1Zifxe1cWmu5SAb5MtjXmDKokH2auGj/7jfH0htZUOMKi4rYzh3 37EXrMLaggLW1DJq1GdvIBOPXDX65VSAr9hxCh03CgJQU2yVHakQFLSZlVkSMf8JotJM3FLb3uJA AVtIaN3FSrTg7SQfOq9xXwfjrL8UO7AlcWg99A/WF1hGFYE8aIuLgw9teiFX5jSw2zJ+40rhpVJy ZCaRTqWSD//gsWD9Gm9oUZljjRqLpcxCm5t9ImPTqaD8zp6Q30QZ9FxbNboW86eb/8ECAwEAAaOC AUswggFHMB8GA1UdIwQYMBaAFImCZ33EnSZwAEu0UEh83j2uBG59MB0GA1UdDgQWBBR6E04AdFvG eGNkJ8Ev4qBbvHnFezAOBgNVHQ8BAf8EBAMCAQYwEgYDVR0TAQH/BAgwBgEB/wIBADARBgNVHSAE CjAIMAYGBFUdIAAwWAYDVR0fBFEwTzBNoEugSYZHaHR0cDovL2NybC51c2VydHJ1c3QuY29tL1VU Ti1VU0VSRmlyc3QtQ2xpZW50QXV0aGVudGljYXRpb25hbmRFbWFpbC5jcmwwdAYIKwYBBQUHAQEE aDBmMD0GCCsGAQUFBzAChjFodHRwOi8vY3J0LnVzZXJ0cnVzdC5jb20vVVROQWRkVHJ1c3RDbGll bnRfQ0EuY3J0MCUGCCsGAQUFBzABhhlodHRwOi8vb2NzcC51c2VydHJ1c3QuY29tMA0GCSqGSIb3 DQEBBQUAA4IBAQCF1r54V1VtM39EUv5C1QaoAQOAivsNsv1Kv/avQUn1G1rF0q0bc24+6SZ85kyY wTAo38v7QjyhJT4KddbQPTmGZtGhm7VNm2+vKGwdr+XqdFqo2rHA8XV6L566k3nK/uKRHlZ0sviN 0+BDchvtj/1gOSBH+4uvOmVIPJg9pSW/ve9g4EnlFsjrP0OD8ODuDcHTzTNfm9C9YGqzO/761Mk6 PB/tm/+bSTO+Qik5g+4zaS6CnUVNqGnagBsePdIaXXxHmaWbCG0SmYbWXVcHG6cwvktJRLiQfsrR eTjrtDP6oDpdJlieYVUYtCHVmdXgQ0BCML7qpeeU0rD+83X5f27nMIIFIzCCBAugAwIBAgIQY7lP +MJT4QzHh5dD4LWVRzANBgkqhkiG9w0BAQUFADCBkzELMAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdy ZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBxMHU2FsZm9yZDEaMBgGA1UEChMRQ09NT0RPIENBIExp bWl0ZWQxOTA3BgNVBAMTMENPTU9ETyBDbGllbnQgQXV0aGVudGljYXRpb24gYW5kIFNlY3VyZSBF bWFpbCBDQTAeFw0xMjA4MjEwMDAwMDBaFw0xMzA4MjEyMzU5NTlaMCIxIDAeBgkqhkiG9w0BCQEW EXNoaXJldHVAZ21haWwuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAziBH0E99 6jx0UrKapQcbxgWLgux1j9/Ec7K3QkkFtzCgxtF9ohpfsiAVOkYuCzVxoxoC0eNKO4qbZPYNbsN6 gtVZMrQnphgG3Wzmf28azn60TtCEQNoFXdW4On+cZgcvUxZS09DaPuTrHB5BuOmYG7b93ZjvJuov KCYAjYNrg0S9TKTQdRJAu95E2OuoVzin+QwIfEVwOA+PSRTPbNnjHvHUMg2G98CXFKIYPy9Pz8Kk rd54q7eSanjNiQK4Z9UYFjjNokx0nhymUAlnzWDAPj8jlh5i6gPrWsr9mYmQ6Y9wm2RHyf7QA6nM BIO43JNPklqjUT5LnB47rxVnLKnYmQIDAQABo4IB4TCCAd0wHwYDVR0jBBgwFoAUehNOAHRbxnhj ZCfBL+KgW7x5xXswHQYDVR0OBBYEFFoYqkq01lUdSiemIAjKHTuJiRj9MA4GA1UdDwEB/wQEAwIF oDAMBgNVHRMBAf8EAjAAMCAGA1UdJQQZMBcGCCsGAQUFBwMEBgsrBgEEAbIxAQMFAjARBglghkgB hvhCAQEEBAMCBSAwRgYDVR0gBD8wPTA7BgwrBgEEAbIxAQIBAQEwKzApBggrBgEFBQcCARYdaHR0 cHM6Ly9zZWN1cmUuY29tb2RvLm5ldC9DUFMwVwYDVR0fBFAwTjBMoEqgSIZGaHR0cDovL2NybC5j b21vZG9jYS5jb20vQ09NT0RPQ2xpZW50QXV0aGVudGljYXRpb25hbmRTZWN1cmVFbWFpbENBLmNy bDCBiAYIKwYBBQUHAQEEfDB6MFIGCCsGAQUFBzAChkZodHRwOi8vY3J0LmNvbW9kb2NhLmNvbS9D T01PRE9DbGllbnRBdXRoZW50aWNhdGlvbmFuZFNlY3VyZUVtYWlsQ0EuY3J0MCQGCCsGAQUFBzAB hhhodHRwOi8vb2NzcC5jb21vZG9jYS5jb20wHAYDVR0RBBUwE4ERc2hpcmV0dUBnbWFpbC5jb20w DQYJKoZIhvcNAQEFBQADggEBAHOQHTMzyiauVSuvCNgN2laY11tdDegdalfDK8UIFpdv08TE1viQ Q9gQfJpGNiO4wxpK8NeB8Qkh+AlL9ygq3jNwuAhk+TXMJ4Jr/I7LkTyJFnW45ADLZs/ufAbc+gdK lxdIxRJBOHmEpA51h0beSlfgHZcvvGshESzwLF290LuEdmLWncIWbhuO4vOJRK/7wJ/KXD8bZdx5 LGvA/QPPi5u95zFoQPRLSDl7lqJailBJZe6YFs7zBK2lm3u0w7exJw/rYeXFjMR0g5BPQCiihMq5 77k2T7uKmQqzapeCqLcd0sFNi94lzgSJxaQDyiDZYeYjWX0lSNVEG0z/fCIVJj4xggOrMIIDpwIB ATCBqDCBkzELMAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UE BxMHU2FsZm9yZDEaMBgGA1UEChMRQ09NT0RPIENBIExpbWl0ZWQxOTA3BgNVBAMTMENPTU9ETyBD bGllbnQgQXV0aGVudGljYXRpb24gYW5kIFNlY3VyZSBFbWFpbCBDQQIQY7lP+MJT4QzHh5dD4LWV RzAJBgUrDgMCGgUAoIIB1zAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEP Fw0xMzA1MTMxNTE5NDRaMCMGCSqGSIb3DQEJBDEWBBRhiQeKFB9pcKDGPvguNmPQOfgTKzCBuQYJ KwYBBAGCNxAEMYGrMIGoMIGTMQswCQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVz dGVyMRAwDgYDVQQHEwdTYWxmb3JkMRowGAYDVQQKExFDT01PRE8gQ0EgTGltaXRlZDE5MDcGA1UE AxMwQ09NT0RPIENsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBAhBjuU/4 wlPhDMeHl0PgtZVHMIG7BgsqhkiG9w0BCRACCzGBq6CBqDCBkzELMAkGA1UEBhMCR0IxGzAZBgNV BAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBxMHU2FsZm9yZDEaMBgGA1UEChMRQ09NT0RP IENBIExpbWl0ZWQxOTA3BgNVBAMTMENPTU9ETyBDbGllbnQgQXV0aGVudGljYXRpb24gYW5kIFNl Y3VyZSBFbWFpbCBDQQIQY7lP+MJT4QzHh5dD4LWVRzANBgkqhkiG9w0BAQEFAASCAQCuNbVD8rRC kCMiQgc5D1T2wkyqUCMfxpa/xr2k8FRz8EpWq4vCGklqIozJJorph3nDG8iwmlC5jH10TpEAWPQ+ dNk7MSmuwX4Lf7E230T17B3HiQAgwBv/5uwTLnebVv2U2ShbaEmhaD6se9GkAgCZzmNO6kBVUTuD MwnzsDUAl+iyZ8r0rKknblA9bSM6hkvnxAwyc1vCaiodnH4LpBEfipmHfxiRQSiOc9De1FhFuf+Y aJst5mipFhj5PXVsiincjXx4xbMVGIMbn6tNxUOVoQjar6dH7sxq5zJ4Xk6a4mfGREPEBoLNJexs qKpctLIJ0s4PCA0qKJhTaxNdc8UVAAAAAAAA --Apple-Mail=_DEAB8EC6-1EB6-4F18-8B38-E783ED902DC7--