Date: Mon, 31 Dec 2012 17:50:48 +0100 From: "Ronald Klop" <ronald-freebsd8@klop.yi.org> To: "Mateusz Guzik" <mjguzik@gmail.com>, "Konstantin Belousov" <kostikbel@gmail.com> Cc: gber@freebsd.org, cognet@freebsd.org, fs@freebsd.org Subject: Re: Nandfs use of MNT_VNODE_FOREACH Message-ID: <op.wp6w2zg98527sy@212-182-167-131.ip.telfort.nl> In-Reply-To: <20121231163039.GN82219@kib.kiev.ua> References: <20121227230223.GN82219@kib.kiev.ua> <op.wp6d7vdo8527sy@212-182-167-131.ip.telfort.nl> <20121231162145.GA29588@dft-labs.eu> <20121231162540.GB29588@dft-labs.eu> <20121231163039.GN82219@kib.kiev.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, 31 Dec 2012 17:30:39 +0100, Konstantin Belousov
<kostikbel@gmail.com> wrote:
> On Mon, Dec 31, 2012 at 05:25:40PM +0100, Mateusz Guzik wrote:
>> On Mon, Dec 31, 2012 at 05:21:45PM +0100, Mateusz Guzik wrote:
>> > Can you adjust this function so that 'if (mp->mnt_syncer == vp)'
>> > performs VI_UNLOCK as well?
>> >
>> > Something like:
>> > if (mp->mnt_syncer == vp || VOP_ISLOCKED(vp)) {
>> > VI_UNLOCK(mp);
>> > continue;
>> > }
>> >
>> > I will have time to dig into this next week.
>>
>> Err.. I meant VI_UNLOCK(vp).
>
> This is definitely a bug, thank you for noticing. Updated patch below.
>
> diff --git a/sys/fs/nandfs/nandfs_segment.c
> b/sys/fs/nandfs/nandfs_segment.c
> index 836bead..7433e77 100644
> --- a/sys/fs/nandfs/nandfs_segment.c
> +++ b/sys/fs/nandfs/nandfs_segment.c
> @@ -478,39 +478,19 @@ nandfs_iterate_dirty_vnodes(struct mount *mp,
> struct nandfs_seginfo *seginfo)
> struct nandfs_node *nandfs_node;
> struct vnode *vp, *mvp;
> struct thread *td;
> - int error, lockreq, update;
> + int error, update;
> td = curthread;
> - lockreq = LK_EXCLUSIVE | LK_INTERLOCK | LK_RETRY;
> - MNT_ILOCK(mp);
> -
> - MNT_VNODE_FOREACH(vp, mp, mvp) {
> + MNT_VNODE_FOREACH_ACTIVE(vp, mp, mvp) {
> update = 0;
> - if (mp->mnt_syncer == vp)
> - continue;
> - if (VOP_ISLOCKED(vp))
> - continue;
> -
> - VI_LOCK(vp);
> - MNT_IUNLOCK(mp);
> - if (vp->v_iflag & VI_DOOMED) {
> + if (mp->mnt_syncer == vp || VOP_ISLOCKED(vp)) {
> VI_UNLOCK(vp);
> - MNT_ILOCK(mp);
> - continue;
> - }
> -
> - if ((error = vget(vp, lockreq, td)) != 0) {
> - MNT_ILOCK(mp);
> continue;
> }
> -
> - if (vp->v_iflag & VI_DOOMED) {
> - vput(vp);
> - MNT_ILOCK(mp);
> + if (vget(vp, LK_EXCLUSIVE | LK_INTERLOCK | LK_NOWAIT, td) != 0)
> continue;
> - }
> nandfs_node = VTON(vp);
> if (nandfs_node->nn_flags & IN_MODIFIED) {
> @@ -532,12 +512,8 @@ nandfs_iterate_dirty_vnodes(struct mount *mp,
> struct nandfs_seginfo *seginfo)
> if (update)
> nandfs_node_update(nandfs_node);
> -
> - MNT_ILOCK(mp);
> }
> - MNT_IUNLOCK(mp);
> -
> return (0);
> }
>
This boots fine. I added some options from the kernel debugging handbook.
Some LORs appear. I will copy-paste them here.
Trying to mount root from nandfs:/dev/gnand0s.root []...
WARNING: NANDFS is considered to be a highly experimental feature in
FreeBSD.
(probe0:umass-sim0:0:0:0): Down reving Protocol Version from 2 to 0?
pass0 at umass-sim0 bus 0 scbus0 target 0 lun 0
pass0: <USB 2.0 USB Flash Drive 1100> Removable Direct Access SCSI-0 device
pass0: Serial Number AA00000000002542
pass0: 40.000MB/s transfers
da0 at umass-sim0 bus 0 scbus0 target 0 lun 0
da0: <USB 2.0 USB Flash Drive 1100> Removable Direct Access SCSI-0 device
da0: Serial Number AA00000000002542
da0: 40.000MB/s transfers
da0: 3894MB (7975296 512 byte sectors: 255H 63S/T 496C)
GEOM: new disk da0
lock order reversal:
1st 0xcc334d40 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:1898
2nd 0xc3ef4a30 snandfs (snandfs) @
/usr/src/sys/fs/nandfs/nandfs_subr.c:444
3rd 0xcc334860 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2631
KDB: stack backtrace:
db_trace_self() at db_trace_self+0x10
scp=0xc0bb462c rlv=0xc09334d8 (X_db_sym_numargs+0x1bc)
rsp=0xd35ae3ec rfp=0xd35ae508
X_db_sym_numargs() at X_db_sym_numargs+0x198
scp=0xc09334b4 rlv=0xc0a6ed5c (kdb_backtrace+0x3c)
rsp=0xd35ae50c rfp=0xd35ae51c
r4=0xc0cc7838
kdb_backtrace() at kdb_backtrace+0x10
scp=0xc0a6ed30 rlv=0xc0a853e8 (witness_display_spinlock+0x80)
rsp=0xd35ae520 rfp=0xd35ae534
r4=0x00000001
witness_display_spinlock() at witness_display_spinlock+0x60
scp=0xc0a853c8 rlv=0xc0a861c4 (witness_checkorder+0x8b4)
rsp=0xd35ae538 rfp=0xd35ae588
r5=0xc3ef4a30 r4=0xc0c27690
witness_checkorder() at witness_checkorder+0x10
scp=0xc0a85920 rlv=0xc0a253bc (__lockmgr_args+0x828)
rsp=0xd35ae58c rfp=0xd35ae5e8
r10=0x00000000 r9=0xc3ef4ad0
r8=0x00081900 r7=0xc3685000 r6=0xcc334860 r5=0x00000100
r4=0x00080000
__lockmgr_args() at __lockmgr_args+0x10
scp=0xc0a24ba4 rlv=0xc0ab9ff0 (getblk+0x128)
rsp=0xd35ae5ec rfp=0xd35ae630
r10=0x00000000 r9=0x00000000
r8=0x000002a4 r7=0xc3ef4ad0 r6=0xcc334800 r5=0x00000000
r4=0x00000000
getblk() at getblk+0x10
scp=0xc0ab9ed8 rlv=0xc0aba918 (breadn_flags+0x30)
rsp=0xd35ae634 rfp=0xd35ae658
r10=0xd35ae6e4 r9=0x0001500a
r8=0xc3fe8000 r7=0x00000000 r6=0xc3ef49d8 r5=0x00000000
r4=0x00000000
breadn_flags() at breadn_flags+0x10
scp=0xc0aba8f8 rlv=0xc098ea58 (nandfs_bread+0x80)
rsp=0xd35ae65c rfp=0xd35ae6a0
r6=0x000002a4 r5=0xc3fe8000
r4=0x00000000
nandfs_bread() at nandfs_bread+0x10
scp=0xc098e9e8 rlv=0xc098efb0 (nandfs_vtop+0xfc)
rsp=0xd35ae6a4 rfp=0xd35ae710
r10=0x00000000 r8=0xc3fe8000
r7=0x00000000 r6=0xd35ae72c r5=0xc3fe7db0 r4=0x00000000
nandfs_vtop() at nandfs_vtop+0x10
scp=0xc098eec4 rlv=0xc099551c (nandfs_itimes+0x304)
rsp=0xd35ae714 rfp=0xd35ae764
r10=0x00001000 r9=0xc3fc5000
r8=0xc3fe7db0 r7=0x00000000 r6=0x00000000 r5=0xcc334ce0
r4=0x00001000
nandfs_itimes() at nandfs_itimes+0x228
scp=0xc0995440 rlv=0xc0be0284 (VOP_STRATEGY_APV+0xbc)
rsp=0xd35ae768 rfp=0xd35ae780
r10=0xd35ae9ec r9=0x00000000
r8=0xc36e25c0 r7=0x00000000 r6=0xd35ae788 r5=0xc0c91a00
r4=0xc0c42434
VOP_STRATEGY_APV() at VOP_STRATEGY_APV+0x10
scp=0xc0be01d8 rlv=0xc0ab47a8 (bufstrategy+0x68)
rsp=0xd35ae784 rfp=0xd35ae7a8
r6=0xc3ef4870 r5=0xcc334ce0
r4=0xc3ef4870
bufstrategy() at bufstrategy+0x10
scp=0xc0ab4750 rlv=0xc0abaa34 (breadn_flags+0x14c)
rsp=0xd35ae7ac rfp=0xd35ae7d0
r5=0x00000000 r4=0xcc334ce0
breadn_flags() at breadn_flags+0x10
scp=0xc0aba8f8 rlv=0xc098ea58 (nandfs_bread+0x80)
rsp=0xd35ae7d4 rfp=0xd35ae818
r6=0x00000000 r5=0xc3fe7db0
r4=0x00000000
nandfs_bread() at nandfs_bread+0x10
scp=0xc098e9e8 rlv=0xc0993afc (nandfs_wakeup_wait_sync+0x2e0c)
rsp=0xd35ae81c rfp=0xd35aea9c
r10=0xc3fe7db0 r8=0xc36e25c0
r7=0x00000000 r6=0xd35ae9bc r5=0x00000000 r4=0xc3fda360
nandfs_wakeup_wait_sync() at nandfs_wakeup_wait_sync+0xc9c
scp=0xc099198c rlv=0xc0ac8390 (vfs_donmount+0x9c8)
rsp=0xd35aeaa0 rfp=0xd35aec80
r10=0xd35aec84 r9=0x00000000
r8=0xc3fe0bc0 r7=0x00000000 r6=0x00004001 r5=0xd35aebbc
r4=0x00000000
vfs_donmount() at vfs_donmount+0x10
scp=0xc0ac79d8 rlv=0xc0ac8ec4 (kernel_mount+0x90)
rsp=0xd35aec84 rfp=0xd35aecc4
r10=0xc3fe5a00 r9=0x00000000
r8=0x00000000 r7=0x00004000 r6=0x00000000 r5=0xc3fda400
r4=0xffffffff
kernel_mount() at kernel_mount+0x10
scp=0xc0ac8e44 rlv=0xc0ac99a0 (root_mount_hold+0x684)
rsp=0xd35aecc8 rfp=0xd35aecfc
r8=0xc36e2427 r7=0xc36e2420
r6=0xc3fda400 r5=0xd35aecd0 r4=0xffffffff
root_mount_hold() at root_mount_hold+0x3a0
scp=0xc0ac96bc rlv=0xc0aca734 (vfs_mountroot+0xca0)
rsp=0xd35aed00 rfp=0xd35aee2c
r10=0xc36e23c0 r9=0xc3683000
r8=0xd35aeeac r7=0xc3fe1000 r6=0xc0c29240 r5=0x00000000
r4=0x00000000
vfs_mountroot() at vfs_mountroot+0x10
scp=0xc0ac9aa4 rlv=0xc09f8754 (exec_shell_imgact+0x7ac)
rsp=0xd35aee30 rfp=0xd35aee84
r10=0x00000000 r9=0xc3683000
r8=0xd35aeeac r7=0x00000000 r6=0xc09f86fc r5=0xc0c1167c
r4=0xc0cb9448
exec_shell_imgact() at exec_shell_imgact+0x764
scp=0xc09f870c rlv=0xc0a14fd4 (fork_exit+0x98)
rsp=0xd35aee88 rfp=0xd35aeea8
r10=0x00000000 r9=0x00000000
r8=0xd35aeeac r7=0x00000000 r6=0xc09f86fc r5=0xc3683000
r4=0xc3685000
fork_exit() at fork_exit+0x10
scp=0xc0a14f4c rlv=0xc0bc4ad4 (fork_trampoline+0x14)
rsp=0xd35aeeac rfp=0x00000000
r8=0x00000000 r7=0x00000000
r6=0x00000000 r5=0x00000000 r4=0xc09f86fc
after vfs_mountroot()
start_init: trying /sbin/init
sys_execve returned 0
(da0:umass-sim0:0:0:0): PREVENT ALLOW MEDIUM REMOVAL. CDB: 1e 0 0 0 1 0
(da0:umass-sim0:0:0:0): CAM status: Auto-Sense Retrieval Failed
(da0:umass-sim0:0:0:0): Error 5, Unretryable error
Dec 31 16:42:47 init: init started
Setting hostuuid: 64f53bc5-bfde-11d3-902f-005043016d4c.
Setting hostid: 0x2afd1481.
No suitable dump device was found.
Entropy harvesting: interrupts ethernet point_to_pointsha256: /kernel: No
such file or directory
kickstart.
Starting file system checks:
/dev/da0s2: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/da0s2: clean, 894115 free (2531 frags, 111448 blocks, 0.3%
fragmentation)
/dev/da0s1: FILESYSTEM CLEAN; SKIPPING CHECKS
lock order reversal:
1st 0xc3ef4a30 snandfs (snandfs) @
/usr/src/sys/fs/nandfs/nandfs_segment.c:1199
2nd 0xc3ef4b98 devfs (devfs) @ /usr/src/sys/fs/nandfs/nandfs_segment.c:793
KDB: stack backtrace:
db_trace_self() at db_trace_self+0x10
scp=0xc0bb462c rlv=0xc09334d8 (X_db_sym_numargs+0x1bc)
rsp=0xd5bf8b1c rfp=0xd5bf8c38
X_db_sym_numargs() at X_db_sym_numargs+0x198
scp=0xc09334b4 rlv=0xc0a6ed5c (kdb_backtrace+0x3c)
rsp=0xd5bf8c3c rfp=0xd5bf8c4c
r4=0xc0cc7838
kdb_backtrace() at kdb_backtrace+0x10
scp=0xc0a6ed30 rlv=0xc0a853e8 (witness_display_spinlock+0x80)
rsp=0xd5bf8c50 rfp=0xd5bf8c64
r4=0x00000001
witness_display_spinlock() at witness_display_spinlock+0x60
scp=0xc0a853c8 rlv=0xc0a861c4 (witness_checkorder+0x8b4)
rsp=0xd5bf8c68 rfp=0xd5bf8cb8
r5=0xc3ef4a30 r4=0xc0c14ad8
witness_checkorder() at witness_checkorder+0x10
scp=0xc0a85920 rlv=0xc0a253bc (__lockmgr_args+0x828)
rsp=0xd5bf8cbc rfp=0xd5bf8d18
r10=0xc36e2680 r9=0xc3ef4c04
r8=0x00080000 r7=0xc400dc00 r6=0xc3ef4b98 r5=0x00000000
r4=0x00080000
__lockmgr_args() at __lockmgr_args+0x10
scp=0xc0a24ba4 rlv=0xc0abfc30 (vop_stdlock+0x48)
rsp=0xd5bf8d1c rfp=0xd5bf8d38
r10=0xc36e2680 r9=0xc4011880
r8=0xc3fc5000 r7=0x00000001 r6=0x00000000 r5=0xd5bf8d68
r4=0xc0ca4dfc
vop_stdlock() at vop_stdlock+0x10
scp=0xc0abfbf8 rlv=0xc0be0500 (VOP_LOCK1_APV+0xa0)
rsp=0xd5bf8d3c rfp=0xd5bf8d50
VOP_LOCK1_APV() at VOP_LOCK1_APV+0x10
scp=0xc0be0470 rlv=0xc098b824 (nandfs_node_create+0x1994)
rsp=0xd5bf8d54 rfp=0xd5bf8da4
r5=0xc4011880 r4=0xd5bf8e14
nandfs_node_create() at nandfs_node_create+0x194c
scp=0xc098b7dc rlv=0xc098bb90 (nandfs_node_create+0x1d00)
rsp=0xd5bf8da8 rfp=0xd5bf8dcc
r10=0xc36e2680 r9=0xc3fc5000
r8=0xc3fc5000 r7=0x00000001 r6=0x00000000 r5=0xc4011880
r4=0xd5bf8e14
nandfs_node_create() at nandfs_node_create+0x1c48
scp=0xc098bad8 rlv=0xc098c354 (nandfs_segment_constructor+0x740)
rsp=0xd5bf8dd0 rfp=0xd5bf8e64
r10=0xc36e2680 r8=0xc3fc5000
r7=0xc36e2680 r6=0x00000000 r5=0x0003c9ca r4=0xd5bf8e14
nandfs_segment_constructor() at nandfs_segment_constructor+0x10
scp=0xc098bc24 rlv=0xc0990bc4 (nandfs_get_seg_stat+0x518)
rsp=0xd5bf8e68 rfp=0xd5bf8e84
r10=0x00000104 r9=0x00000000
r8=0xd5bf8eac r7=0xc36e25c0 r6=0xc3fc5000 r5=0xc36e25c0
r4=0x00000000
nandfs_get_seg_stat() at nandfs_get_seg_stat+0x49c
scp=0xc0990b48 rlv=0xc0a14fd4 (fork_exit+0x98)
rsp=0xd5bf8e88 rfp=0xd5bf8ea8
r6=0xc0990b38 r5=0xc0cb74c4
r4=0xc400dc00
fork_exit() at fork_exit+0x10
scp=0xc0a14f4c rlv=0xc0bc4ad4 (fork_trampoline+0x14)
rsp=0xd5bf8eac rfp=0x00000000
r8=0xc0cb77b0 r7=0x00000000
r6=0x00000000 r5=0xc36e25c0 r4=0xc0990b38
Mounting local file systems:.
Writing entropy file:.
Setting hostname: sh10.klop.ws.
Starting Network: lo0 mge0.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet 127.0.0.1 netmask 0xff000000
mge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric mge0: link
state changed to UP
0 mtu 1500
options=8000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE>
ether 00:50:43:01:6d:4c
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
Starting devd.
Starting dhclient.
DHCPDISCOVER on mge0 to 255.255.255.255 port 67 interval 6
DHCPOFFER from 192.168.1.254
DHCPREQUEST on mge0 to 255.255.255.255 port 67
DHCPACK from 192.168.1.254
bound to 192.168.1.35 -- renewal in 43200 seconds.
Creating and/or trimming log files.
Starting syslogd.
realpath: /dev/dumpdev: No such file or directory
/etc/rc: WARNING: Dump device does not exist. Savecore not run.
ELF ldconfig path: /lib /usr/lib /usr/lib/compat
Clearing /tmp (X related).
Updating motd:.
Starting ntpd.
Performing sanity check on sshd configuration.
Starting sshd.
Starting cron.
Mon Dec 31 16:43:48 UTC 2012
FreeBSD/arm (sh10.klop.ws) (ttyu0)
login:
Regards,
Ronald.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?op.wp6w2zg98527sy>