From owner-freebsd-current@FreeBSD.ORG Thu Sep 18 17:28:22 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E42931065675 for ; Thu, 18 Sep 2008 17:28:22 +0000 (UTC) (envelope-from naylor.b.david@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.190]) by mx1.freebsd.org (Postfix) with ESMTP id 6AA728FC17 for ; Thu, 18 Sep 2008 17:28:22 +0000 (UTC) (envelope-from naylor.b.david@gmail.com) Received: by nf-out-0910.google.com with SMTP id h3so2884nfh.33 for ; Thu, 18 Sep 2008 10:28:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:organization:to:subject :date:user-agent:cc:references:in-reply-to:mime-version:content-type :content-transfer-encoding:message-id; bh=SoRWWgKlCJmLgU6/aRlixpyOW9S8sV1UWiB8silb3no=; b=jrnO2XftWvJIf8V9LYHq67LtpT9AOvFxhDsQUuIYF8JrjjEPUoKiU2wQLPR3G0PcPY 7fMnVULuIK/+AZWAL/55fQwsEFdple7BoPtMpvB7uB4KwcFJv+gVOUbIndNBP6H7fRIG v46+hzwDdVm3Z/9+PlR5ZE703FJFkkx8Tm4s4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:organization:to:subject:date:user-agent:cc:references :in-reply-to:mime-version:content-type:content-transfer-encoding :message-id; b=hGTiJwBHgrqoHoCpaLj5UZe6u5zPDhKB4s0YwtK2ttzehRtjCkQiCYbdUsfIuXp3pG ZX63rTaM4Suj+6LmjgZ6m3boLGw1phvgnIUywaToYFl3RThmy2TGfv+0XWXTOwaS3ZFJ +21vaP5YUptk4RV9sd4GEDVL4XOLVr2LnBXwo= Received: by 10.103.240.5 with SMTP id s5mr3133923mur.62.1221758900898; Thu, 18 Sep 2008 10:28:20 -0700 (PDT) Received: from ?0.0.0.0? ( [196.34.241.123]) by mx.google.com with ESMTPS id j10sm11192714mue.17.2008.09.18.10.28.14 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 18 Sep 2008 10:28:19 -0700 (PDT) From: David Naylor Organization: Private To: Daniel Eischen Date: Thu, 18 Sep 2008 19:27:47 +0200 User-Agent: KMail/1.9.7 References: <200809180631.47071.naylor.b.david@gmail.com> <200809181857.25872.naylor.b.david@gmail.com> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart12811148.oDATbWqooN"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200809181927.52384.naylor.b.david@gmail.com> X-Mailman-Approved-At: Thu, 18 Sep 2008 17:40:54 +0000 Cc: Tom Evans , freebsd-current@freebsd.org Subject: Re: FreeBSD deadlock (with fork?) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2008 17:28:23 -0000 --nextPart12811148.oDATbWqooN Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Thursday 18 September 2008 19:13:19 Daniel Eischen wrote: > On Thu, 18 Sep 2008, David Naylor wrote: > > On Thursday 18 September 2008 15:33:52 Tom Evans wrote: > >> On Thu, 2008-09-18 at 06:31 +0200, David Naylor wrote: > >>> Hi, > >>> > >>> I have a program that spawns a lot of subprocesses (with pipes open) > >>> from multiple threads. The problem is the program often deadlocks, b= ut > >>> not consistently. Sometimes the program can run over 5 times to > >>> competition without incidence and yet othertimes it locks within a few > >>> seconds. > >> > >> Do you create threads, which then fork(), or do you fork() and then > >> create threads? > > > > I have many threads that then fork. (aka forking threads). > > > >> I think the former will not work.. > > > > Is there any reason for this and is this a FreeBSD limitation or a > > general problem? > > No, the former will work. You can fork() from threads, only as long > as your forked processes only call async-signal-safe functions or > some form of exec(). For instance, you can't fork and then create > new threads from that child process. If you are not immediately > exec()'ing from the child process, then you have to be careful > and only use async-signal-safe functions. > > Remember that you have multiple threads in the parent, so the > state of libthr, libc, etc may be inconsistent in a child > process. If you have a look at the backtrace from my original post you will see that= =20 all my threads are getting locked. The program I am using is python and th= e=20 call that is forking is Popen(['some', 'program'], stdout=3DPIPE,=20 stderr=3DSTDOUT). As far as I know it just sets up the pipes and does an=20 execvp. (See=20 http://svn.python.org/view/*checkout*/python/tags/r252/Lib/subprocess.py?co= ntent-type=3Dtext%2Fplain&rev=3D60915=20 from line 981 to 1091). [[I tried appending close_fds=3DTrue to the argume= nt,=20 it appeared to make the program deadlock less often...]] Any ideas as to what might be causing this dead lock (and why all the threa= ds=20 are stopping in amd64 specific assembler)? --nextPart12811148.oDATbWqooN Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQBI0o+YUaaFgP9pFrIRArHCAJ4unxSPffSggossGz7l1Gz0LF8xugCfci3D c8wugKdG9K5F8CapMNITo7Y= =nEFp -----END PGP SIGNATURE----- --nextPart12811148.oDATbWqooN--