From owner-freebsd-smp Mon Jun 17 2: 4:41 2002 Delivered-To: freebsd-smp@freebsd.org Received: from rina.r.dl.itc.u-tokyo.ac.jp (rina.r.dl.itc.u-tokyo.ac.jp [133.11.199.247]) by hub.freebsd.org (Postfix) with ESMTP id 9CC4837B401; Mon, 17 Jun 2002 02:04:36 -0700 (PDT) Received: from rina.r.dl.itc.u-tokyo.ac.jp (localhost [127.0.0.1]) by rina.r.dl.itc.u-tokyo.ac.jp (8.12.3+3.5Wbeta/3.7W-rina.r-Nankai-Koya) with ESMTP id g5H94X3i076651 ; Mon, 17 Jun 2002 18:04:33 +0900 (JST) Message-Id: <200206170904.g5H94X3i076651@rina.r.dl.itc.u-tokyo.ac.jp> Date: Mon, 17 Jun 2002 18:04:33 +0900 From: Seigo Tanimura To: Jeffrey Hsu Cc: Seigo Tanimura , jhb@FreeBSD.org, smp@FreeBSD.org, yangjihui@yahoo.com Subject: Re: Sharing a single mutex between a socket and its PCB In-Reply-To: <0GXS004W453AEO@mta5.snfc21.pbi.net> References: <200206151545.g5FFipAY006726@silver.carrots.uucp.r.dl.itc.u-tokyo.ac.jp> <0GXS004W453AEO@mta5.snfc21.pbi.net> User-Agent: Wanderlust/2.8.1 (Something) SEMI/1.14.3 (Ushinoya) FLIM/1.14.3 (=?ISO-8859-1?Q?Unebigory=F2mae?=) APEL/10.3 MULE XEmacs/21.1 (patch 14) (Cuyahoga Valley) (i386--freebsd) Organization: Digital Library Research Division, Information Techinology Centre, The University of Tokyo MIME-Version: 1.0 (generated by SEMI 1.14.3 - "Ushinoya") Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Sat, 15 Jun 2002 20:45:19 -0700, Jeffrey Hsu said: >> Seigo Tanimura writes: >> As some socket operations (eg sosend(), soreceive(), ...) modify both >> a socket and its PCB at once, both of them should be locked by a >> single mutex. Since hsu has already locked down struct inpcb, I would >> like to protect a socket by the mutex of the PCB. >> In order for the socket subsystem to lock and unlock opaquely, two new >> usrreq methods will be added: >> - pru_lock() locks the PCB of a socket. >> - pru_unlock() unlocks the PCB of a socket. >> If the PCB has its own mutex, those methods simply lock and unlock the >> mutex. Otherwise, those methods lock and unlock the Giant lock. This >> is so that we can push down Giant for the socket subsystem later. >> Comments? hsu> Let's stick with the BSD/OS design, which is to have a separate socket buffer hsu> lock. It's better for concurrency this way. (BSD/OS also shows there's no hsu> need to have a separate socket lock. The socket buffer lock doubles as a hsu> socket lock.) It helps the lock order issue of a socket and PCB lock to export PCB's mutex. You never have to release a socket lock to acquire a PCB lock as BSD/OS does: tcp_usrreq(so) { sb = so->so_buf; SOCKBUF_UNLOCK(sb); /* XXX What if someone modified the socket? */ INP_LOCK(inp); SOCKBUF_LOCK(sb); tcp_do_usrreq(); INP_UNLOCK(inp); } Speaking of concurrency, it does not scale to run only one net swi thread on an MP machine. Maybe we can make a pool of net swi threads and run them in parallel. -- Seigo Tanimura To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jun 17 20:55:31 2002 Delivered-To: freebsd-smp@freebsd.org Received: from warp.ecn.net.au (warp.ecn.net.au [203.22.70.2]) by hub.freebsd.org (Postfix) with SMTP id ECA1837B437 for ; Mon, 17 Jun 2002 20:55:25 -0700 (PDT) Received: (qmail 23068 invoked from network); 18 Jun 2002 03:50:30 -0000 Received: from unknown (HELO ecn.net.au) (203.123.88.164) by warp.ecn.net.au with SMTP; 18 Jun 2002 03:50:30 -0000 Message-ID: <3D0EAEA5.7090707@ecn.net.au> Date: Tue, 18 Jun 2002 13:53:09 +1000 From: Wade User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.1a) Gecko/20020610 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-smp@freebsd.org Subject: Status Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org How about an updated status report? There was an effort in Jan to get these things going, yet that was the last update [Jan]. Non-coders tend to be very interested in these things, even if we don't do any cheerleading ;-) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 18 0:45:18 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mta6.snfc21.pbi.net (mta6.snfc21.pbi.net [206.13.28.240]) by hub.freebsd.org (Postfix) with ESMTP id A641E37B421 for ; Tue, 18 Jun 2002 00:45:06 -0700 (PDT) Received: from FreeBSD.org ([63.193.112.125]) by mta6.snfc21.pbi.net (iPlanet Messaging Server 5.1 (built May 7 2001)) with ESMTP id <0GXW00IUX5IPFL@mta6.snfc21.pbi.net> for smp@FreeBSD.org; Tue, 18 Jun 2002 00:44:49 -0700 (PDT) Date: Tue, 18 Jun 2002 00:45:15 -0700 From: Jeffrey Hsu Subject: Re: Sharing a single mutex between a socket and its PCB In-reply-to: Message from Seigo Tanimura "of Mon, 17 Jun 2002 18:04:33 +0900." <200206170904.g5H94X3i076651@rina.r.dl.itc.u-tokyo.ac.jp> To: Seigo Tanimura Cc: smp@FreeBSD.org, yangjihui@yahoo.com Message-id: <0GXW00IUY5IPFL@mta6.snfc21.pbi.net> MIME-version: 1.0 X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org > BSD/OS does: > tcp_usrreq(so) > { > sb = so->so_buf; > SOCKBUF_UNLOCK(sb); > /* XXX What if someone modified the socket? */ > INP_LOCK(inp); > SOCKBUF_LOCK(sb); > tcp_do_usrreq(); > INP_UNLOCK(inp); > } The XXX comment is yours. You're thinking locally instead of globally. SMP locking is tricky and fragile because you have to consider all the related code together, not individual accesses to structure fields in isolation. Find the two actual code paths in the source where a race condition arises due to the unlocking above. (Then convince yourself why that race condition doesn't really matter.) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message