From owner-freebsd-fs@FreeBSD.ORG Mon Dec 31 16:53:27 2012 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BF5DD378; Mon, 31 Dec 2012 16:53:27 +0000 (UTC) (envelope-from ronald-freebsd8@klop.yi.org) Received: from cpsmtpb-ews09.kpnxchange.com (cpsmtpb-ews09.kpnxchange.com [213.75.39.14]) by mx1.freebsd.org (Postfix) with ESMTP id 0E1708FC12; Mon, 31 Dec 2012 16:53:26 +0000 (UTC) Received: from cpsps-ews23.kpnxchange.com ([10.94.84.189]) by cpsmtpb-ews09.kpnxchange.com with Microsoft SMTPSVC(7.5.7601.17514); Mon, 31 Dec 2012 17:49:49 +0100 Received: from CPSMTPM-TLF104.kpnxchange.com ([195.121.3.7]) by cpsps-ews23.kpnxchange.com with Microsoft SMTPSVC(7.5.7601.17514); Mon, 31 Dec 2012 17:49:49 +0100 Received: from sjakie.klop.ws ([212.182.167.131]) by CPSMTPM-TLF104.kpnxchange.com with Microsoft SMTPSVC(7.5.7601.17514); Mon, 31 Dec 2012 17:50:50 +0100 Received: from 212-182-167-131.ip.telfort.nl (localhost [127.0.0.1]) by sjakie.klop.ws (Postfix) with ESMTP id F09C37442; Mon, 31 Dec 2012 17:50:49 +0100 (CET) Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes To: "Mateusz Guzik" , "Konstantin Belousov" Subject: Re: Nandfs use of MNT_VNODE_FOREACH References: <20121227230223.GN82219@kib.kiev.ua> <20121231162145.GA29588@dft-labs.eu> <20121231162540.GB29588@dft-labs.eu> <20121231163039.GN82219@kib.kiev.ua> Date: Mon, 31 Dec 2012 17:50:48 +0100 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: "Ronald Klop" Message-ID: In-Reply-To: <20121231163039.GN82219@kib.kiev.ua> User-Agent: Opera Mail/12.12 (FreeBSD) X-OriginalArrivalTime: 31 Dec 2012 16:50:51.0007 (UTC) FILETIME=[FD9378F0:01CDE776] X-RcptDomain: freebsd.org Cc: gber@freebsd.org, cognet@freebsd.org, fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Dec 2012 16:53:27 -0000 On Mon, 31 Dec 2012 17:30:39 +0100, Konstantin Belousov wrote: > On Mon, Dec 31, 2012 at 05:25:40PM +0100, Mateusz Guzik wrote: >> On Mon, Dec 31, 2012 at 05:21:45PM +0100, Mateusz Guzik wrote: >> > Can you adjust this function so that 'if (mp->mnt_syncer == vp)' >> > performs VI_UNLOCK as well? >> > >> > Something like: >> > if (mp->mnt_syncer == vp || VOP_ISLOCKED(vp)) { >> > VI_UNLOCK(mp); >> > continue; >> > } >> > >> > I will have time to dig into this next week. >> >> Err.. I meant VI_UNLOCK(vp). > > This is definitely a bug, thank you for noticing. Updated patch below. > > diff --git a/sys/fs/nandfs/nandfs_segment.c > b/sys/fs/nandfs/nandfs_segment.c > index 836bead..7433e77 100644 > --- a/sys/fs/nandfs/nandfs_segment.c > +++ b/sys/fs/nandfs/nandfs_segment.c > @@ -478,39 +478,19 @@ nandfs_iterate_dirty_vnodes(struct mount *mp, > struct nandfs_seginfo *seginfo) > struct nandfs_node *nandfs_node; > struct vnode *vp, *mvp; > struct thread *td; > - int error, lockreq, update; > + int error, update; > td = curthread; > - lockreq = LK_EXCLUSIVE | LK_INTERLOCK | LK_RETRY; > - MNT_ILOCK(mp); > - > - MNT_VNODE_FOREACH(vp, mp, mvp) { > + MNT_VNODE_FOREACH_ACTIVE(vp, mp, mvp) { > update = 0; > - if (mp->mnt_syncer == vp) > - continue; > - if (VOP_ISLOCKED(vp)) > - continue; > - > - VI_LOCK(vp); > - MNT_IUNLOCK(mp); > - if (vp->v_iflag & VI_DOOMED) { > + if (mp->mnt_syncer == vp || VOP_ISLOCKED(vp)) { > VI_UNLOCK(vp); > - MNT_ILOCK(mp); > - continue; > - } > - > - if ((error = vget(vp, lockreq, td)) != 0) { > - MNT_ILOCK(mp); > continue; > } > - > - if (vp->v_iflag & VI_DOOMED) { > - vput(vp); > - MNT_ILOCK(mp); > + if (vget(vp, LK_EXCLUSIVE | LK_INTERLOCK | LK_NOWAIT, td) != 0) > continue; > - } > nandfs_node = VTON(vp); > if (nandfs_node->nn_flags & IN_MODIFIED) { > @@ -532,12 +512,8 @@ nandfs_iterate_dirty_vnodes(struct mount *mp, > struct nandfs_seginfo *seginfo) > if (update) > nandfs_node_update(nandfs_node); > - > - MNT_ILOCK(mp); > } > - MNT_IUNLOCK(mp); > - > return (0); > } > This boots fine. I added some options from the kernel debugging handbook. Some LORs appear. I will copy-paste them here. Trying to mount root from nandfs:/dev/gnand0s.root []... WARNING: NANDFS is considered to be a highly experimental feature in FreeBSD. (probe0:umass-sim0:0:0:0): Down reving Protocol Version from 2 to 0? pass0 at umass-sim0 bus 0 scbus0 target 0 lun 0 pass0: Removable Direct Access SCSI-0 device pass0: Serial Number AA00000000002542 pass0: 40.000MB/s transfers da0 at umass-sim0 bus 0 scbus0 target 0 lun 0 da0: Removable Direct Access SCSI-0 device da0: Serial Number AA00000000002542 da0: 40.000MB/s transfers da0: 3894MB (7975296 512 byte sectors: 255H 63S/T 496C) GEOM: new disk da0 lock order reversal: 1st 0xcc334d40 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:1898 2nd 0xc3ef4a30 snandfs (snandfs) @ /usr/src/sys/fs/nandfs/nandfs_subr.c:444 3rd 0xcc334860 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2631 KDB: stack backtrace: db_trace_self() at db_trace_self+0x10 scp=0xc0bb462c rlv=0xc09334d8 (X_db_sym_numargs+0x1bc) rsp=0xd35ae3ec rfp=0xd35ae508 X_db_sym_numargs() at X_db_sym_numargs+0x198 scp=0xc09334b4 rlv=0xc0a6ed5c (kdb_backtrace+0x3c) rsp=0xd35ae50c rfp=0xd35ae51c r4=0xc0cc7838 kdb_backtrace() at kdb_backtrace+0x10 scp=0xc0a6ed30 rlv=0xc0a853e8 (witness_display_spinlock+0x80) rsp=0xd35ae520 rfp=0xd35ae534 r4=0x00000001 witness_display_spinlock() at witness_display_spinlock+0x60 scp=0xc0a853c8 rlv=0xc0a861c4 (witness_checkorder+0x8b4) rsp=0xd35ae538 rfp=0xd35ae588 r5=0xc3ef4a30 r4=0xc0c27690 witness_checkorder() at witness_checkorder+0x10 scp=0xc0a85920 rlv=0xc0a253bc (__lockmgr_args+0x828) rsp=0xd35ae58c rfp=0xd35ae5e8 r10=0x00000000 r9=0xc3ef4ad0 r8=0x00081900 r7=0xc3685000 r6=0xcc334860 r5=0x00000100 r4=0x00080000 __lockmgr_args() at __lockmgr_args+0x10 scp=0xc0a24ba4 rlv=0xc0ab9ff0 (getblk+0x128) rsp=0xd35ae5ec rfp=0xd35ae630 r10=0x00000000 r9=0x00000000 r8=0x000002a4 r7=0xc3ef4ad0 r6=0xcc334800 r5=0x00000000 r4=0x00000000 getblk() at getblk+0x10 scp=0xc0ab9ed8 rlv=0xc0aba918 (breadn_flags+0x30) rsp=0xd35ae634 rfp=0xd35ae658 r10=0xd35ae6e4 r9=0x0001500a r8=0xc3fe8000 r7=0x00000000 r6=0xc3ef49d8 r5=0x00000000 r4=0x00000000 breadn_flags() at breadn_flags+0x10 scp=0xc0aba8f8 rlv=0xc098ea58 (nandfs_bread+0x80) rsp=0xd35ae65c rfp=0xd35ae6a0 r6=0x000002a4 r5=0xc3fe8000 r4=0x00000000 nandfs_bread() at nandfs_bread+0x10 scp=0xc098e9e8 rlv=0xc098efb0 (nandfs_vtop+0xfc) rsp=0xd35ae6a4 rfp=0xd35ae710 r10=0x00000000 r8=0xc3fe8000 r7=0x00000000 r6=0xd35ae72c r5=0xc3fe7db0 r4=0x00000000 nandfs_vtop() at nandfs_vtop+0x10 scp=0xc098eec4 rlv=0xc099551c (nandfs_itimes+0x304) rsp=0xd35ae714 rfp=0xd35ae764 r10=0x00001000 r9=0xc3fc5000 r8=0xc3fe7db0 r7=0x00000000 r6=0x00000000 r5=0xcc334ce0 r4=0x00001000 nandfs_itimes() at nandfs_itimes+0x228 scp=0xc0995440 rlv=0xc0be0284 (VOP_STRATEGY_APV+0xbc) rsp=0xd35ae768 rfp=0xd35ae780 r10=0xd35ae9ec r9=0x00000000 r8=0xc36e25c0 r7=0x00000000 r6=0xd35ae788 r5=0xc0c91a00 r4=0xc0c42434 VOP_STRATEGY_APV() at VOP_STRATEGY_APV+0x10 scp=0xc0be01d8 rlv=0xc0ab47a8 (bufstrategy+0x68) rsp=0xd35ae784 rfp=0xd35ae7a8 r6=0xc3ef4870 r5=0xcc334ce0 r4=0xc3ef4870 bufstrategy() at bufstrategy+0x10 scp=0xc0ab4750 rlv=0xc0abaa34 (breadn_flags+0x14c) rsp=0xd35ae7ac rfp=0xd35ae7d0 r5=0x00000000 r4=0xcc334ce0 breadn_flags() at breadn_flags+0x10 scp=0xc0aba8f8 rlv=0xc098ea58 (nandfs_bread+0x80) rsp=0xd35ae7d4 rfp=0xd35ae818 r6=0x00000000 r5=0xc3fe7db0 r4=0x00000000 nandfs_bread() at nandfs_bread+0x10 scp=0xc098e9e8 rlv=0xc0993afc (nandfs_wakeup_wait_sync+0x2e0c) rsp=0xd35ae81c rfp=0xd35aea9c r10=0xc3fe7db0 r8=0xc36e25c0 r7=0x00000000 r6=0xd35ae9bc r5=0x00000000 r4=0xc3fda360 nandfs_wakeup_wait_sync() at nandfs_wakeup_wait_sync+0xc9c scp=0xc099198c rlv=0xc0ac8390 (vfs_donmount+0x9c8) rsp=0xd35aeaa0 rfp=0xd35aec80 r10=0xd35aec84 r9=0x00000000 r8=0xc3fe0bc0 r7=0x00000000 r6=0x00004001 r5=0xd35aebbc r4=0x00000000 vfs_donmount() at vfs_donmount+0x10 scp=0xc0ac79d8 rlv=0xc0ac8ec4 (kernel_mount+0x90) rsp=0xd35aec84 rfp=0xd35aecc4 r10=0xc3fe5a00 r9=0x00000000 r8=0x00000000 r7=0x00004000 r6=0x00000000 r5=0xc3fda400 r4=0xffffffff kernel_mount() at kernel_mount+0x10 scp=0xc0ac8e44 rlv=0xc0ac99a0 (root_mount_hold+0x684) rsp=0xd35aecc8 rfp=0xd35aecfc r8=0xc36e2427 r7=0xc36e2420 r6=0xc3fda400 r5=0xd35aecd0 r4=0xffffffff root_mount_hold() at root_mount_hold+0x3a0 scp=0xc0ac96bc rlv=0xc0aca734 (vfs_mountroot+0xca0) rsp=0xd35aed00 rfp=0xd35aee2c r10=0xc36e23c0 r9=0xc3683000 r8=0xd35aeeac r7=0xc3fe1000 r6=0xc0c29240 r5=0x00000000 r4=0x00000000 vfs_mountroot() at vfs_mountroot+0x10 scp=0xc0ac9aa4 rlv=0xc09f8754 (exec_shell_imgact+0x7ac) rsp=0xd35aee30 rfp=0xd35aee84 r10=0x00000000 r9=0xc3683000 r8=0xd35aeeac r7=0x00000000 r6=0xc09f86fc r5=0xc0c1167c r4=0xc0cb9448 exec_shell_imgact() at exec_shell_imgact+0x764 scp=0xc09f870c rlv=0xc0a14fd4 (fork_exit+0x98) rsp=0xd35aee88 rfp=0xd35aeea8 r10=0x00000000 r9=0x00000000 r8=0xd35aeeac r7=0x00000000 r6=0xc09f86fc r5=0xc3683000 r4=0xc3685000 fork_exit() at fork_exit+0x10 scp=0xc0a14f4c rlv=0xc0bc4ad4 (fork_trampoline+0x14) rsp=0xd35aeeac rfp=0x00000000 r8=0x00000000 r7=0x00000000 r6=0x00000000 r5=0x00000000 r4=0xc09f86fc after vfs_mountroot() start_init: trying /sbin/init sys_execve returned 0 (da0:umass-sim0:0:0:0): PREVENT ALLOW MEDIUM REMOVAL. CDB: 1e 0 0 0 1 0 (da0:umass-sim0:0:0:0): CAM status: Auto-Sense Retrieval Failed (da0:umass-sim0:0:0:0): Error 5, Unretryable error Dec 31 16:42:47 init: init started Setting hostuuid: 64f53bc5-bfde-11d3-902f-005043016d4c. Setting hostid: 0x2afd1481. No suitable dump device was found. Entropy harvesting: interrupts ethernet point_to_pointsha256: /kernel: No such file or directory kickstart. Starting file system checks: /dev/da0s2: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0s2: clean, 894115 free (2531 frags, 111448 blocks, 0.3% fragmentation) /dev/da0s1: FILESYSTEM CLEAN; SKIPPING CHECKS lock order reversal: 1st 0xc3ef4a30 snandfs (snandfs) @ /usr/src/sys/fs/nandfs/nandfs_segment.c:1199 2nd 0xc3ef4b98 devfs (devfs) @ /usr/src/sys/fs/nandfs/nandfs_segment.c:793 KDB: stack backtrace: db_trace_self() at db_trace_self+0x10 scp=0xc0bb462c rlv=0xc09334d8 (X_db_sym_numargs+0x1bc) rsp=0xd5bf8b1c rfp=0xd5bf8c38 X_db_sym_numargs() at X_db_sym_numargs+0x198 scp=0xc09334b4 rlv=0xc0a6ed5c (kdb_backtrace+0x3c) rsp=0xd5bf8c3c rfp=0xd5bf8c4c r4=0xc0cc7838 kdb_backtrace() at kdb_backtrace+0x10 scp=0xc0a6ed30 rlv=0xc0a853e8 (witness_display_spinlock+0x80) rsp=0xd5bf8c50 rfp=0xd5bf8c64 r4=0x00000001 witness_display_spinlock() at witness_display_spinlock+0x60 scp=0xc0a853c8 rlv=0xc0a861c4 (witness_checkorder+0x8b4) rsp=0xd5bf8c68 rfp=0xd5bf8cb8 r5=0xc3ef4a30 r4=0xc0c14ad8 witness_checkorder() at witness_checkorder+0x10 scp=0xc0a85920 rlv=0xc0a253bc (__lockmgr_args+0x828) rsp=0xd5bf8cbc rfp=0xd5bf8d18 r10=0xc36e2680 r9=0xc3ef4c04 r8=0x00080000 r7=0xc400dc00 r6=0xc3ef4b98 r5=0x00000000 r4=0x00080000 __lockmgr_args() at __lockmgr_args+0x10 scp=0xc0a24ba4 rlv=0xc0abfc30 (vop_stdlock+0x48) rsp=0xd5bf8d1c rfp=0xd5bf8d38 r10=0xc36e2680 r9=0xc4011880 r8=0xc3fc5000 r7=0x00000001 r6=0x00000000 r5=0xd5bf8d68 r4=0xc0ca4dfc vop_stdlock() at vop_stdlock+0x10 scp=0xc0abfbf8 rlv=0xc0be0500 (VOP_LOCK1_APV+0xa0) rsp=0xd5bf8d3c rfp=0xd5bf8d50 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x10 scp=0xc0be0470 rlv=0xc098b824 (nandfs_node_create+0x1994) rsp=0xd5bf8d54 rfp=0xd5bf8da4 r5=0xc4011880 r4=0xd5bf8e14 nandfs_node_create() at nandfs_node_create+0x194c scp=0xc098b7dc rlv=0xc098bb90 (nandfs_node_create+0x1d00) rsp=0xd5bf8da8 rfp=0xd5bf8dcc r10=0xc36e2680 r9=0xc3fc5000 r8=0xc3fc5000 r7=0x00000001 r6=0x00000000 r5=0xc4011880 r4=0xd5bf8e14 nandfs_node_create() at nandfs_node_create+0x1c48 scp=0xc098bad8 rlv=0xc098c354 (nandfs_segment_constructor+0x740) rsp=0xd5bf8dd0 rfp=0xd5bf8e64 r10=0xc36e2680 r8=0xc3fc5000 r7=0xc36e2680 r6=0x00000000 r5=0x0003c9ca r4=0xd5bf8e14 nandfs_segment_constructor() at nandfs_segment_constructor+0x10 scp=0xc098bc24 rlv=0xc0990bc4 (nandfs_get_seg_stat+0x518) rsp=0xd5bf8e68 rfp=0xd5bf8e84 r10=0x00000104 r9=0x00000000 r8=0xd5bf8eac r7=0xc36e25c0 r6=0xc3fc5000 r5=0xc36e25c0 r4=0x00000000 nandfs_get_seg_stat() at nandfs_get_seg_stat+0x49c scp=0xc0990b48 rlv=0xc0a14fd4 (fork_exit+0x98) rsp=0xd5bf8e88 rfp=0xd5bf8ea8 r6=0xc0990b38 r5=0xc0cb74c4 r4=0xc400dc00 fork_exit() at fork_exit+0x10 scp=0xc0a14f4c rlv=0xc0bc4ad4 (fork_trampoline+0x14) rsp=0xd5bf8eac rfp=0x00000000 r8=0xc0cb77b0 r7=0x00000000 r6=0x00000000 r5=0xc36e25c0 r4=0xc0990b38 Mounting local file systems:. Writing entropy file:. Setting hostname: sh10.klop.ws. Starting Network: lo0 mge0. lo0: flags=8049 metric 0 mtu 16384 options=600003 inet 127.0.0.1 netmask 0xff000000 mge0: flags=8843 metric mge0: link state changed to UP 0 mtu 1500 options=8000b ether 00:50:43:01:6d:4c media: Ethernet autoselect (100baseTX ) status: active Starting devd. Starting dhclient. DHCPDISCOVER on mge0 to 255.255.255.255 port 67 interval 6 DHCPOFFER from 192.168.1.254 DHCPREQUEST on mge0 to 255.255.255.255 port 67 DHCPACK from 192.168.1.254 bound to 192.168.1.35 -- renewal in 43200 seconds. Creating and/or trimming log files. Starting syslogd. realpath: /dev/dumpdev: No such file or directory /etc/rc: WARNING: Dump device does not exist. Savecore not run. ELF ldconfig path: /lib /usr/lib /usr/lib/compat Clearing /tmp (X related). Updating motd:. Starting ntpd. Performing sanity check on sshd configuration. Starting sshd. Starting cron. Mon Dec 31 16:43:48 UTC 2012 FreeBSD/arm (sh10.klop.ws) (ttyu0) login: Regards, Ronald.