From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 29 10:44:16 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B7E47378 for ; Tue, 29 Oct 2013 10:44:16 +0000 (UTC) (envelope-from sebastian.huber@embedded-brains.de) Received: from mail.embedded-brains.de (host-82-135-62-35.customer.m-online.net [82.135.62.35]) by mx1.freebsd.org (Postfix) with ESMTP id 2BCB52B28 for ; Tue, 29 Oct 2013 10:44:15 +0000 (UTC) Received: by mail.embedded-brains.de (Postfix, from userid 65534) id 2E087652CF6; Tue, 29 Oct 2013 11:35:09 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on fidibusdmz X-Spam-Level: X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 Received: from [192.168.100.11] (unknown [192.168.100.11]) by mail.embedded-brains.de (Postfix) with ESMTP id 1D0DB65253A for ; Tue, 29 Oct 2013 11:35:09 +0100 (CET) Message-ID: <526F8F58.1090307@embedded-brains.de> Date: Tue, 29 Oct 2013 11:35:04 +0100 From: Sebastian Huber User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130329 Thunderbird/17.0.5 MIME-Version: 1.0 To: "freebsd-hackers@freebsd.org" Subject: soo_close() vs. filt_soread() Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Oct 2013 10:44:16 -0000 Hello, I port currently the FreeBSD network stack to a real-time operating system. The problem described below probably does not happen in a real FreeBSD kernel. I have the following test case: static void test_kqueue_close(test_context *ctx) { /* The cfd is some socket with a connected TCP stream */ int cfd = ctx->cfd; int kq; struct kevent change; struct kevent event; const struct timespec *timeout = NULL; int rv; ssize_t n; puts("test kqueue close"); assert(ctx->cfd >= 0); kq = kqueue(); assert(kq >= 0); EV_SET(&change, cfd, EVFILT_READ, EV_ADD | EV_ENABLE, 0, 0, TEST_UDATA); rv = kevent(kq, &change, 1, NULL, 0, timeout); assert(rv == 0); set_non_blocking(cfd, 1); do { errno = 0; n = read(cfd, &ctx->buf[0], sizeof(ctx->buf)); if (n == -1) { assert(errno = EAGAIN); } } while (n > 0); /* This tells some background entity that we want to close cfd once kevent blocks */ send_events(ctx, EVENT_CLOSE); assert(ctx->cfd >= 0); rv = kevent(kq, NULL, 0, &event, 1, timeout); assert(rv == 1); assert(event.ident == cfd); assert(event.filter == EVFILT_READ); assert(event.flags == 0); assert(event.fflags == 0); assert(event.data == 0); assert(event.udata == TEST_UDATA); assert(ctx->cfd == -1); rv = close(kq); assert(rv == 0); } This test registers a read event and then blocks for this event. Once the current thread blocked, someone will delete the corresponding socket. I have now a NULL pointer access. The socket close looks like this: /* * API socket close on file pointer. We call soclose() to close the socket * (including initiating closing protocols). soclose() will sorele() the * file reference but the actual socket will not go away until the socket's * ref count hits 0. */ /* ARGSUSED */ int soo_close(struct file *fp, struct thread *td) { int error = 0; struct socket *so; so = fp->f_data; fp->f_ops = &badfileops; fp->f_data = NULL; if (so) error = soclose(so); return (error); } Please note that fp->f_data is set to NULL. The close operation will end up in: /*ARGSUSED*/ static int filt_soread(struct knote *kn, long hint) { struct socket *so; so = kn->kn_fp->f_data; SOCKBUF_LOCK_ASSERT(&so->so_rcv); kn->kn_data = so->so_rcv.sb_cc - so->so_rcv.sb_ctl; if (so->so_rcv.sb_state & SBS_CANTRCVMORE) { kn->kn_flags |= EV_EOF; kn->kn_fflags = so->so_error; return (1); } else if (so->so_error) /* temporary udp error */ return (1); else if (kn->kn_sfflags & NOTE_LOWAT) return (kn->kn_data >= kn->kn_sdata); else return (so->so_rcv.sb_cc >= so->so_rcv.sb_lowat); } Here so == NULL, due to soo_close(). Breakpoint 11, filt_soread (kn=0x46ff74, hint=0) at freebsd/sys/kern/uipc_socket.c:3147 3147 so = kn->kn_fp->f_data; (gdb) p so $4 = (struct socket *) 0x0 (gdb) p kn->kn_ptr.p_fp $5 = (struct file *) 0x32cec8 (gdb) where #0 filt_soread (kn=0x46ff74, hint=0) at freebsd/sys/kern/uipc_socket.c:3147 #1 0x0010506e in knote (list=0x3f1190, hint=0, lockflags=1) at freebsd/sys/kern/kern_event.c:1957 #2 0x00191a60 in sowakeup (so=0x3f1134, sb=0x3f1188) at freebsd/sys/kern/uipc_sockbuf.c:191 #3 0x00127afe in soisdisconnecting (so=0x3f1134) at freebsd/sys/kern/uipc_socket.c:3341 #4 0x00153cca in tcp_disconnect (tp=0x3f75ac) at freebsd/sys/netinet/tcp_usrreq.c:1508 #5 0x00152b30 in tcp_usr_disconnect (so=0x3f1134) at freebsd/sys/netinet/tcp_usrreq.c:556 #6 0x00124754 in sodisconnect (so=0x3f1134) at freebsd/sys/kern/uipc_socket.c:816 #7 0x00124384 in soclose (so=0x3f1134) at freebsd/sys/kern/uipc_socket.c:664 #8 0x00128886 in soo_close (fp=0x32cec8, td=0x0) at freebsd/sys/kern/sys_socket.c:452 Is this an illegal kevent() use case? Are there some means that prevent this sequence in a real FreeBSD kernel? -- Sebastian Huber, embedded brains GmbH Address : Dornierstr. 4, D-82178 Puchheim, Germany Phone : +49 89 189 47 41-16 Fax : +49 89 189 47 41-09 E-Mail : sebastian.huber@embedded-brains.de PGP : Public key available on request. Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.