Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 28 Feb 2010 23:58:24 -0600
From:      "James R. Van Artsdalen" <james-freebsd-fs2@jrv.org>
To:        freebsd-fs <freebsd-fs@freebsd.org>
Subject:   [zfs] attach by name/uuid still attaches wrong device
Message-ID:  <4B8B5780.2050601@jrv.org>

next in thread | raw e-mail | index | archive | help
FreeBSD bigtex.housenet.jrv 9.0-CURRENT FreeBSD 9.0-CURRENT #2 r200727M:
Tue Dec 22 23:25:56 CST 2009    
james@bigtex.housenet.jrv:/usr/obj/usr/src/sys/BIGTEX  amd64

It appears the zfs/vdev_geom.c can still attach to the wrong device in
some cases.  Note in the zpool status output how ada10 appears in two
different vdevs.

What happened is that a disk failed completely (scbus3 target 3) and is
no longer detected by the driver.  At boot time:

1. ZFS fails to attach by path and UUID, since what was at ada11 is now
at ada10 and has a different  UUID.
2. ZFS fails to attach by UUID since that UUID is on a dead drive and
can no longer be found anywhere.
3. ZFS then attaches by path blindly, even though that drive is in a
different part of the pool and has a different UUID.

I don't think it's possible to do this right in vdev_geom.c: there's no
way to guess what is intended without a hint from higher ZFS layers as
to which drives should be found and which are new.

The best fixes I can think of are to expose drives by serial number in
GEOM, or perhaps as a fall-back expose names that are geographic
locations, i.e., "/dev/scbus0/target3/lun0".

# zpool status   
  pool: bigtex
 state: DEGRADED
status: One or more devices could not be used because the label is
missing or
        invalid.  Sufficient replicas exist for the pool to continue
        functioning in a degraded state.
action: Replace the device using 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-4J
 scrub: none requested
config:

        NAME                                            STATE     READ
WRITE CKSUM
        bigtex                                          DEGRADED    
0     0     0
          mirror                                        ONLINE      
0     0     0
            ada6                                        ONLINE      
0     0     0
            ada13                                       ONLINE      
0     0     0
          mirror                                        ONLINE      
0     0     0
            ada4                                        ONLINE      
0     0     0
            ada11                                       ONLINE      
0     0     0
          mirror                                        ONLINE      
0     0     0
            gptid/dbb5f9fd-5e40-11de-bef4-001aa01b0286  ONLINE      
0     0     0
            ada2p7                                      ONLINE      
0     0     0
          mirror                                        ONLINE      
0     0     0
            ada7                                        ONLINE      
0     0     0
            ada14                                       ONLINE      
0     0     0
          mirror                                        ONLINE      
0     0     0
            ada3                                        ONLINE      
0     0     0
            ada10                                       ONLINE      
0     0     0
          mirror                                        ONLINE      
0     0     0
            ada5                                        ONLINE      
0     0     0
            ada12                                       ONLINE      
0     0     0
          mirror                                        ONLINE      
0     0     0
            ada9                                        ONLINE      
0     0     0
            ada15                                       ONLINE      
0     0     0
          mirror                                        DEGRADED    
0     0     0
            ada10                                       FAULTED     10 
754K     0  corrupted data
            ada16                                       ONLINE      
0     0     0

errors: No known data errors

# camcontrol devlist
<WDC WD15EADS-00R6B0 01.00A01>     at scbus0 target 0 lun 0 (ada2,pass6)
<WDC WD20EADS-00R6B0 01.00A01>     at scbus0 target 1 lun 0 (ada3,pass7)
<WDC WD20EADS-00R6B0 01.00A01>     at scbus0 target 2 lun 0 (ada4,pass8)
<WDC WD20EADS-00R6B0 01.00A01>     at scbus0 target 3 lun 0 (ada5,pass9)
<Port Multiplier 37261095 1706>    at scbus0 target 15 lun 0 (pass0,pmp0)
<WDC WD20EADS-00R6B0 01.00A01>     at scbus3 target 0 lun 0 (ada6,pass10)
<WDC WD20EADS-00R6B0 01.00A01>     at scbus3 target 1 lun 0 (ada7,pass11)
<WDC WD20EADS-00R6B0 01.00A01>     at scbus3 target 2 lun 0 (ada9,pass13)
<Port Multiplier 37261095 1706>    at scbus3 target 15 lun 0 (pass1,pmp1)
<ST31500343AS SD35>                at scbus4 target 0 lun 0 (ada8,pass12)
<ST32000542AS CC32>                at scbus4 target 1 lun 0 (ada10,pass14)
<ST32000542AS CC32>                at scbus4 target 2 lun 0 (ada11,pass15)
<ST32000542AS CC32>                at scbus4 target 3 lun 0 (ada12,pass16)
<Port Multiplier 37261095 1706>    at scbus4 target 15 lun 0 (pass2,pmp2)
<ST32000542AS CC32>                at scbus7 target 0 lun 0 (ada13,pass17)
<ST32000542AS CC32>                at scbus7 target 1 lun 0 (ada14,pass18)
<ST32000542AS CC32>                at scbus7 target 2 lun 0 (ada15,pass19)
<ST32000542AS CC32>                at scbus7 target 3 lun 0 (ada16,pass20)
<Port Multiplier 37261095 1706>    at scbus7 target 15 lun 0 (pass3,pmp3)
<ST31500341AS CC1G>                at scbus8 target 0 lun 0 (pass4,ada0)
<ST31500341AS CC1G>                at scbus11 target 0 lun 0 (pass5,ada1)

# grep ada10 /var/run/dmesg.boot
vdev_geom_read_guid:301[1]: Reading guid from ada10...
vdev_geom_read_guid:339[1]: guid for ada10 is 12768899409278570370
vdev_geom_open_by_path:466[1]: Found provider by name /dev/ada10.
vdev_geom_attach:112[1]: Attaching to ada10.
vdev_geom_attach:138[1]: Found consumer for ada10.
vdev_geom_attach:157[1]: Used existing consumer for ada10.
vdev_geom_read_guid:301[1]: Reading guid from ada10...
vdev_geom_read_guid:339[1]: guid for ada10 is 12768899409278570370
vdev_geom_detach:173[1]: Closing access to ada10.
vdev_geom_open_by_path:477[1]: guid mismatch for provider /dev/ada10:
3665972767133355802 != 12768899409278570370.
vdev_geom_read_guid:301[1]: Reading guid from ada10...
vdev_geom_read_guid:339[1]: guid for ada10 is 12768899409278570370
vdev_geom_open_by_path:466[1]: Found provider by name /dev/ada10.
vdev_geom_attach:112[1]: Attaching to ada10.
vdev_geom_attach:138[1]: Found consumer for ada10.
vdev_geom_attach:157[1]: Used existing consumer for ada10.
vdev_geom_detach:173[1]: Closing access to ada10.
vdev_geom_detach:173[1]: Closing access to ada10.
vdev_geom_detach:177[1]: Destroyed consumer to ada10.
vdev_geom_read_guid:301[1]: Reading guid from ada10...
vdev_geom_read_guid:339[1]: guid for ada10 is 12768899409278570370
vdev_geom_attach:112[1]: Attaching to ada10.
vdev_geom_attach:153[1]: Created consumer for ada10.
vdev_geom_open_by_guid:446[1]: Attach by guid [12768899409278570370]
succeeded, provider /dev/ada10.
vdev_geom_read_guid:301[1]: Reading guid from ada10...
vdev_geom_read_guid:339[1]: guid for ada10 is 12768899409278570370
vdev_geom_open_by_path:466[1]: Found provider by name /dev/ada10.
vdev_geom_attach:112[1]: Attaching to ada10.
vdev_geom_attach:138[1]: Found consumer for ada10.
vdev_geom_attach:157[1]: Used existing consumer for ada10.
vdev_geom_read_guid:301[1]: Reading guid from ada10...
vdev_geom_read_guid:339[1]: guid for ada10 is 12768899409278570370
vdev_geom_detach:173[1]: Closing access to ada10.
vdev_geom_open_by_path:477[1]: guid mismatch for provider /dev/ada10:
3665972767133355802 != 12768899409278570370.
vdev_geom_read_guid:301[1]: Reading guid from ada10...
vdev_geom_read_guid:339[1]: guid for ada10 is 12768899409278570370
vdev_geom_open_by_path:466[1]: Found provider by name /dev/ada10.
vdev_geom_attach:112[1]: Attaching to ada10.
vdev_geom_attach:138[1]: Found consumer for ada10.
vdev_geom_attach:157[1]: Used existing consumer for ada10.
vdev_geom_detach:173[1]: Closing access to ada10.
#




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4B8B5780.2050601>