From owner-freebsd-hackers@FreeBSD.ORG Mon May 13 15:25:18 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 1631120E for ; Mon, 13 May 2013 15:25:18 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-we0-x22e.google.com (mail-we0-x22e.google.com [IPv6:2a00:1450:400c:c03::22e]) by mx1.freebsd.org (Postfix) with ESMTP id A571EF97 for ; Mon, 13 May 2013 15:25:17 +0000 (UTC) Received: by mail-we0-f174.google.com with SMTP id x53so6347414wes.5 for ; Mon, 13 May 2013 08:25:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=iaR8TxE59O/dVqdQUM0kUIdQt/CaeOeLj8dJF67ZLOo=; b=bnfsnxe3DFSi+22/BNDxu5dbGMsYCpnP9YVH1+tfDcPCFvsmFO/33zkodGQh11T8Mw MR/fCaUZILTS8LehJHRd8W19sgTjx13eBcXWgKcphmnNhYsKlIPzreHn8g0aLKMAU2qJ i+ExFiyusTe0rsgZN1pfJAG+33jj0RFJGcGCXB53xlcSyrxoAsaZ5s5W91zyVf7k7IfX p+Kl9eBhYNpZioYGb1UOwpuLtP9cB++I62tYIW2aIN0D9+7I8QJiuKQsM7G9R7oRYszX 1uDyxJ2y31fG18A/YgP9SQDjGbYlVvtmxZgEQPodK8K3nBkS4bB/D7yfZwBX8SusOoZR kc4A== MIME-Version: 1.0 X-Received: by 10.180.88.162 with SMTP id bh2mr19927105wib.3.1368458716714; Mon, 13 May 2013 08:25:16 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.217.58.138 with HTTP; Mon, 13 May 2013 08:25:16 -0700 (PDT) In-Reply-To: References: Date: Mon, 13 May 2013 08:25:16 -0700 X-Google-Sender-Auth: KoqulNAlwCyN8vdLl98Pfp3yinU Message-ID: Subject: Re: Managing userland data pointers in kqueue/kevent From: Adrian Chadd To: Eugen-Andrei Gavriloaie Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 May 2013 15:25:18 -0000 ... or you could just track the per-descriptor / per-object stuff in userland, and use the FD/signal as an index into the state you need. adding thread happiness on top of that is trivial. Done/done. Adrian On 13 May 2013 08:19, Eugen-Andrei Gavriloaie wrote: > Hello to all, > > I'm trying to reply to this thread: > http://lists.freebsd.org/pipermail/freebsd-hackers/2010-November/033565.h= tml > > I also faced this very difficult task of tracking down the user data regi= stered into kq. > I end up having some "Tokens" instances which I never deallocate but alwa= ys re-use them or create new ones if necessary. This tokens are used as use= r data for kq. They are keeping the actual pointers inside them, and the po= inter itself has a reference to the Token. When the pointer dies, I reset t= he guts of the token. When the time comes to use the token, I have the guar= antee is not the corpse of a token (never deallocate them, remember?) and I= can see that the actual pointer was gone, everyone is happy. At the applic= ation shutdown, I cleanup the mess (the tokens). However, I just want to sa= y that Paul has a valid point when he is wondering why EV_FREEWATCH was not= provisioned/implemented. > > The moment we throw multi-threading into equation, this becomes a extreme= ly hard thing to manage (close to impossible), including my "proven-to-work= " Token trick. It renders the user data pointer completely useless because= in the end we need to keep an association map inside user space. That is f= orcing user space code to do a lookup before use, instead of straightforwar= d use. Not to mention the fact that we need to perform a lock before search= ing it. That brings havoc on kernel side on 1000+ active connections (a mul= ti-threaded streaming server for example). > > I'm pretty sure this user data pointer is also breaking a well known poin= ter management paradigm, but I just can't remember which. Regardless, it ha= s all the ingredients for memory leaks and/or, the worst one, use of corpse= pointers which are bound to crash the app. I agree, C/C++ is not for the f= aint of heart, but with little or close to no efforts, his EV_FREEWATCH can= be put to very good use, and user space code not only becomes less prone t= o mem issues, but also cleaner. > > To summarise, +1 for the EV_FREEWATCH. I simply love the idea! Clean and = very very efficient. > > Best regards, > Andrei > > ------ > Eugen-Andrei Gavriloaie > Web: http://www.rtmpd.com >