From owner-freebsd-hackers@FreeBSD.ORG Fri May 30 07:35:52 2003 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F0E0337B401; Fri, 30 May 2003 07:35:51 -0700 (PDT) Received: from whale.sunbay.crimea.ua (whale.sunbay.crimea.ua [212.110.138.65]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4BC9743FA3; Fri, 30 May 2003 07:35:46 -0700 (PDT) (envelope-from ru@whale.sunbay.crimea.ua) Received: from whale.sunbay.crimea.ua (ru@localhost [127.0.0.1]) h4UEZfEd045109 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 30 May 2003 17:35:41 +0300 (EEST) (envelope-from ru@whale.sunbay.crimea.ua) Received: (from ru@localhost) by whale.sunbay.crimea.ua (8.12.9/8.12.8/Submit) id h4UEZfqx045104; Fri, 30 May 2003 17:35:41 +0300 (EEST) (envelope-from ru) Date: Fri, 30 May 2003 17:35:41 +0300 From: Ruslan Ermilov To: Daniel Eischen , hackers@FreeBSD.org Message-ID: <20030530143541.GB42349@sunbay.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="T7mxYSe680VjQnyC" Content-Disposition: inline User-Agent: Mutt/1.5.4i cc: Yaroslav Levchenko cc: Sergey Starosek Subject: libc_r: threaded application could stuck in accept(2) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 May 2003 14:35:52 -0000 --T7mxYSe680VjQnyC Content-Type: multipart/mixed; boundary="z4+8/lEcDcG5Ke9S" Content-Disposition: inline --z4+8/lEcDcG5Ke9S Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi! We had a bug in our threaded application that would mistakenly close the descriptor 0, and this triggers a bug in libc_r which I will try to describe below. The bug (in libc_r only, libpthread^Wlibkse is unaffected) causes a threaded application to stuck in accept(2). libc_r makes every new descriptor non-blocking, and uses poll(2) and thread context switching to emulate a blocking behavior. The bug somehow causes the descriptor to be left in a blocking mode. Attached is the test case that demonstrates this bug (this is the same on 4.8-STABLE and 5.1-BETA). The utility runs two threads: the first thread prints the "alive" message every 3 seconds, and the second thread emulates what our unfortunate application did, i.e., open() a TCP socket, listen(), accept(), then close() it, then mistakenly close() descriptor 0. (Closing the descriptor 0 causes the next socket() call to return 0.) Some important notes: this bug is only applicable to descriptors 0 - 2 (stdio set), and might have something to do with the code in uthread_fd.c. If you remove two lines that free the descriptor 0 in the attached test case, the bug won't manifest itself. Attached also is the patch for the threaded close() function that avoids the bug by disallowing the close() call on a non-active descriptor. The patch should be committed, but I'm now more interested in what's going on inside libc_r that causes the descriptor 0 to be left in the blocking mode. IOW, I wonder if the attached patch is the real fix. ;) Ah yes, when you'll run the application, it will report which TCP port it listens to, you then connect to this port, the application closes the connection, and on the next loop the application gets stuck in accept(). Cheers, --=20 Ruslan Ermilov Sysadmin and DBA, ru@sunbay.com Sunbay Software AG, ru@FreeBSD.org FreeBSD committer. --z4+8/lEcDcG5Ke9S Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="bug.c" #include #include #include #include #include #include #include #include static void *thread1(void *); static void *thread2(void *); int main(void) { pthread_t thr1, thr2; int r; if ((r = pthread_create(&thr1, NULL, thread1, NULL)) != 0) errc(1, r, "pthread_create"); if ((r = pthread_create(&thr2, NULL, thread2, NULL)) != 0) errc(1, r, "pthread_create"); if ((r = pthread_join(thr1, NULL)) != 0) errc(1, r, "pthread_join"); if ((r = pthread_join(thr2, NULL)) != 0) errc(1, r, "pthread_join"); exit(0); } static void * thread1(void *arg __unused) { for (;;) { printf("thread 1: alive\n"); sleep(3); } } static void * thread2(void *arg __unused) { int s, s2; struct sockaddr_in addr; socklen_t addrlen; printf("thread 2: freeing descriptor 0 (closing)\n"); close(0); for (;;) { if ((s = socket(AF_INET, SOCK_STREAM, 0)) == -1) warn("thread 2: socket"); else { printf("thread 2: socket() returned %d\n", s); if (listen(s, 1) == -1) warn("thread 2: listen"); else { addrlen = sizeof(addr); if (getsockname(s, (struct sockaddr *)&addr, &addrlen) == -1) warn("thread 2: getsockname"); else printf("thread 2: listening on port %d\n", ntohs(addr.sin_port)); printf("thread 2: calling accept() on %d\n", s); s2 = accept(s, (struct sockaddr *)&addr, &addrlen); if (s2 == -1) warn("thread 2: accept"); else { printf("thread 2: accept() returned %d, closing\n", s2); close(s2); } } printf("thread 2: closing descriptor %d (1st)\n", s); close(s); printf("thread 2: closing descriptor %d (2nd)\n", s); close(s); } sleep(1); } } --z4+8/lEcDcG5Ke9S Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=p Content-Transfer-Encoding: quoted-printable Index: uthread_close.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/lib/libc_r/uthread/uthread_close.c,v retrieving revision 1.10.2.3 diff -u -p -r1.10.2.3 uthread_close.c --- uthread_close.c 22 Oct 2002 14:44:02 -0000 1.10.2.3 +++ uthread_close.c 30 May 2003 14:13:54 -0000 @@ -47,7 +47,8 @@ _close(int fd) struct stat sb; struct fd_table_entry *entry; =20 - if ((fd =3D=3D _thread_kern_pipe[0]) || (fd =3D=3D _thread_kern_pipe[1]))= { + if ((fd =3D=3D _thread_kern_pipe[0]) || (fd =3D=3D _thread_kern_pipe[1]) = || + (_thread_fd_table[fd] =3D=3D NULL)) { /* * Don't allow silly programs to close the kernel pipe. */ --z4+8/lEcDcG5Ke9S-- --T7mxYSe680VjQnyC Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (FreeBSD) iD8DBQE+12w9Ukv4P6juNwoRAny5AJ43WNZLkx4AHpUxjQyKG7xkMtVFZACfVM0X W/i97JIc22KogQ8JUGOHbi4= =hHar -----END PGP SIGNATURE----- --T7mxYSe680VjQnyC--