From owner-freebsd-arch@FreeBSD.ORG Sat Feb 18 22:27:12 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E971B1065674; Sat, 18 Feb 2012 22:27:12 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 102088FC1B; Sat, 18 Feb 2012 22:27:11 +0000 (UTC) Received: by bkcjg1 with SMTP id jg1so5198440bkc.13 for ; Sat, 18 Feb 2012 14:27:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:to:cc:subject:references:x-comment-to:sender:date:in-reply-to :message-id:user-agent:mime-version:content-type; bh=iKUbzntieTJP5C7Bckj6iWNyS1dJa1+wmJl7VO+EVTQ=; b=qnRfJbyIzKFMhqG/urYMj7haALimskhfWJ59Q1vvu7GPFsXxplK0aIXtptQUvTVOUK vrS6uSMx35O2OBURPFn8tHzlfyqmekUMqIdGtcr1aau8wbI85zPrIdwg4NLz4XDV101L 5FiKC8Y5qygD9M4S1fGjYVv/y6mx9MDv8FU/I= Received: by 10.204.151.207 with SMTP id d15mr6691133bkw.27.1329604030728; Sat, 18 Feb 2012 14:27:10 -0800 (PST) Received: from localhost ([95.69.173.122]) by mx.google.com with ESMTPS id y9sm30159678bkw.5.2012.02.18.14.27.08 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 18 Feb 2012 14:27:09 -0800 (PST) From: Mikolaj Golub To: Konstantin Belousov References: <86sjjobzmn.fsf@kopusha.home.net> <86fwfnti5t.fsf@kopusha.home.net> <20120112215106.GC31224@deviant.kiev.zoral.com.ua> <86hazntwmu.fsf@kopusha.home.net> <20120123031238.GL31224@deviant.kiev.zoral.com.ua> <86zkcfu9ac.fsf@kopusha.home.net> <20120218215003.GM3283@deviant.kiev.zoral.com.ua> X-Comment-To: Konstantin Belousov Sender: Mikolaj Golub Date: Sun, 19 Feb 2012 00:27:06 +0200 In-Reply-To: <20120218215003.GM3283@deviant.kiev.zoral.com.ua> (Konstantin Belousov's message of "Sat, 18 Feb 2012 23:50:03 +0200") Message-ID: <86vcn3u69x.fsf@kopusha.home.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: "Robert N. M. Watson" , freebsd-arch@freebsd.org Subject: Re: unix domain sockets on nullfs(5) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Feb 2012 22:27:13 -0000 On Sat, 18 Feb 2012 23:50:03 +0200 Konstantin Belousov wrote: KB> On Sat, Feb 18, 2012 at 11:22:03PM +0200, Mikolaj Golub wrote: >> After collecting all suggestions and additional testing I have got this patch >> set: >> >> http://people.freebsd.org/~trociny/unp_prepare_reclaim.1.patch KB> Including unpcb.h into vfs_subr.c looks too extreme. Put the prototype KB> into vnode.h, possibly renaming the function to vfs_unp_reclaim. >> http://people.freebsd.org/~trociny/unp_connect.LOCKSHARED.1.patch >> http://people.freebsd.org/~trociny/VOP_UNP.3.patch KB> I has a painting suggestion there, call the vops VOP_UNP_DETACH etc, KB> otherwise it takes too much reading to understand that it is not undetach. Thanks, will do both. >> >> It has survived some bind/connect/force umount stress testing revealing only >> some issues that are also observed without patching. KB> What are the issues ? I triggered this assert: #9 0x80a678cc in panic (fmt=0x80fcbba0 "sofree: so_comp populated") at /home/golub/freebsd/git/freebsd/sys/kern/kern_shutdown.c:633 #10 0x80ada1da in sofree (so=0x8c71b820) at /home/golub/freebsd/git/freebsd/sys/kern/uipc_socket.c:638 #11 0x80adbab6 in soclose (so=0x8c71b820) at /home/golub/freebsd/git/freebsd/sys/kern/uipc_socket.c:741 #12 0x80abe3e9 in soo_close (fp=0x89fb5cb0, td=0x8ad9db80) at /home/golub/freebsd/git/freebsd/sys/kern/sys_socket.c:294 #13 0x80a26c13 in _fdrop (fp=0x89fb5cb0, td=0x8ad9db80) at file.h:310 #14 0x80a29640 in closef (fp=0x89fb5cb0, td=0x8ad9db80) at /home/golub/freebsd/git/freebsd/sys/kern/kern_descrip.c:2246 #15 0x80a29a09 in kern_close (td=0x8ad9db80, fd=3) at /home/golub/freebsd/git/freebsd/sys/kern/kern_descrip.c:1232 #16 0x80a29baa in sys_close (td=0x8ad9db80, uap=0xdef62cec) at /home/golub/freebsd/git/freebsd/sys/kern/kern_descrip.c:1178 #17 0x80deca20 in syscall (frame=0xdef62d28) at subr_syscall.c:135 #18 0x80dd5ce1 in Xint0x80_syscall () at /home/golub/freebsd/git/freebsd/sys/i386/i386/exception.s:266 (kgdb) fr 10 #10 0x80ada1da in sofree (so=0x8c71b820) at /home/golub/freebsd/git/freebsd/sys/kern/uipc_socket.c:638 638 KASSERT((TAILQ_EMPTY(&so->so_incomp)), ("sofree: so_comp populated")); (kgdb) l 633 (so->so_qstate & SQ_INCOMP) == 0, 634 ("sofree: so_head == NULL, but still SQ_COMP(%d) or SQ_INCOMP(%d)", 635 so->so_qstate & SQ_COMP, so->so_qstate & SQ_INCOMP)); 636 if (so->so_options & SO_ACCEPTCONN) { 637 KASSERT((TAILQ_EMPTY(&so->so_comp)), ("sofree: so_comp populated")); 638 KASSERT((TAILQ_EMPTY(&so->so_incomp)), ("sofree: so_comp populated")); 639 } 640 SOCK_UNLOCK(so); 641 ACCEPT_UNLOCK(); 642 (BTW, the panic message should be: "sofree: so_incomp populated".) The test was to run bind/listen/accept/umount -f/close (with some variations) loop in one thread and connect/close in several others. I was able to trigger it for nullfs and was not able for ufs. I have not looked close at it yet, just checked that the same panic is observed on a kernel without my modifications. Another issue is a socket leak observed when running the above test: after the test stale sockets remain: kopusha:~% sockstat|grep test ? ? ? ? stream /mnt/upper/test.sock ? ? ? ? stream /mnt/lower/test.sock ? ? ? ? stream /mnt/upper/test.sock I am going to investigate this more some day. -- Mikolaj Golub