Date: Tue, 29 May 2012 21:09:01 -0400 From: vasanth rao naik sabavat <vasanth.raonaik@gmail.com> To: "Robert N. M. Watson" <rwatson@freebsd.org> Cc: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org> Subject: Re: SMP: protocol control block protection for a multithreaded process (ex: udp). Message-ID: <CAAuizBhaNp-%2Bf7FD=e0Sp_15yuXYnAViBsH5%2B7DbDuDNoJxfMA@mail.gmail.com> In-Reply-To: <B6BB8E0D-F536-463E-B59C-A098038B8C1E@FreeBSD.org> References: <CAAuizBjhGUUH3D3XN1t7WMnOPTq0vZjnV1QXGrR99qBOD34rGQ@mail.gmail.com> <CAAuizBhn_QT4WCh1ZRyc%2BHBkOYGaGivsVGm4oLj-i9VY7a5wxw@mail.gmail.com> <alpine.BSF.2.00.1205292204590.15505@fledge.watson.org> <CAAuizBjpLHoWwQ_CYrY9H5xrJ8_e48S_hVyU8Fif_J2pEyiq6Q@mail.gmail.com> <B6BB8E0D-F536-463E-B59C-A098038B8C1E@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Robert, I am trying to understand the socket <--> protocol layer as part of our project. I was trying to understand why the sotoinpcb() is called before taking any locks. Also, I am trying to understand scenario of a multi-threaded process trying to do socket operations simultaneously on a multicore cpu. I have gone through the socket life cycle comments in the code and gave good understanding of the socket life cycle. Thank you for the reference. Thanks, Vasanth On Tue, May 29, 2012 at 7:14 PM, Robert N. M. Watson <rwatson@freebsd.org>wrote: > On 29 May 2012, at 18:06, vasanth rao naik sabavat < > vasanth.raonaik@gmail.com> wrote: > > My main concern is about the protocol control block "inp", a reference in > the socket structure. the udp_detach() free'es the inp but there is a > potential for other thread running udp_* functions to get hold of the > reference? Also, sofree() calls SOCK_UNLOCK() which potentially may allow > other thread of the same process to enter into the udp_* functions? I am > not sure if that is ever possible. > > > Hi Vasanth: > > Are you chasing a bug, or is this a more intellectual exercise in > understanding the stack? The scenario you are describing should be > prevented by the socket-layer reference model and so should not occur > unless a bug is manifesting. I am happy to answer questions about the > model, but if you are running into problems it might be easier if I gave > more directed advice about how to track problem. > > When a socket system call takes place on a UDP socket, several data > structures are involved, each with its own life cycle and reference model. > Most of the time, calls like send() use a per-system call counted reference > to the file descriptor to ensure valid access down the stack through the > socket and inpcb code -- this works because the file descriptor owns a > reference to the socket, and the socket owns a reference to the inpcb. > udp_detach is called only when the reference count on the socket reaches > zero, so the references down the stack held by invoking threads should > prevent udp_detach from being invoked while they are running. The right way > to view stability here is via the reference model, not the locking model: > locks come and go as data is processed and packets move around, but it is > the reference counts that prevent freeing of the sockt and hence inpcb. The > perspective is slightly different looking "up" from the input side, where > we sometimes do use locks to ensure stability, but that is rarely the case > in the down path. In UDP, unlike with TCP, socket close can only be > initiated from the down path, not the up path. > > If you haven't already read the large comment at the top of uipc_socket.c > on the socket life cycle, it would be a good idea to do so, as it lays out > some of the principles at play here. > > Robert > > Thanks, > Vasanth > > > > > > > > > > > > > > > > > > > > On Tue, May 29, 2012 at 5:06 PM, Robert Watson <rwatson@freebsd.org>wrote: > >> >> On Tue, 29 May 2012, vasanth rao naik sabavat wrote: >> >> Can somebody please reply to this email. >>> >>> basically, can udp_detach() and udp_send() execute simultaneously for a >>> process with multiple threads? if yes, then inp reference in udp_send() >>> will be stale if udp_detach() free's the inp? >>> >> >> You are confusing application-level close() with an actual close in the >> socket implementation. The socket will remain allocated as long as there >> are consumers using it, which is ensured through a reference count on the >> socket, regardless of close(). That isn't to say that there aren't bugs -- >> this stuff is pretty complex -- but the life cycle and synchronisation >> models around sockets should prevent the scenario you are describing from >> occurring. >> >> Robert >> >> >>> Thanks, >>> Vasanth >>> >>> >>> >>> On Tue, May 29, 2012 at 10:53 AM, vasanth rao naik sabavat < >>> vasanth.raonaik@gmail.com> wrote: >>> >>> Hi, >>>> >>>> In case of a Multicore cpu system running a multithreaded process. >>>> >>>> For protocol control blocks there is no protection provided in the >>>> FreeBSD >>>> 9. For example, udp_close() and udp_send() access the inp before taking >>>> the >>>> lock. Couldn't this cause the inp inconsistency on a multithreaded >>>> process >>>> running on multicore cpu system? >>>> >>>> Say, If the two threads of a process are concurrently executing socket >>>> send and socket close say on a udp connection (this can happen in case >>>> of >>>> poorly written user code.). >>>> udp_close() will access the inp on one cpu and udp_send() will access >>>> the >>>> inp on another cpu. it is possible that udp_close() gets the locks first >>>> and free's the inp before udp_send() has a chance to run? >>>> >>>> Am I missing anything? >>>> >>>> Thanks, >>>> Vasanth >>>> >>>> >>>> >>>> >>>> >>>> ______________________________**_________________ >>> freebsd-hackers@freebsd.org mailing list >>> http://lists.freebsd.org/**mailman/listinfo/freebsd-**hackers<http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>; >>> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@** >>> freebsd.org <freebsd-hackers-unsubscribe@freebsd.org>" >>> >>> >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAAuizBhaNp-%2Bf7FD=e0Sp_15yuXYnAViBsH5%2B7DbDuDNoJxfMA>