Date: Tue, 08 Oct 2002 21:44:53 -0700 From: Jack Twilley <jmt@twilley.org> To: freebsd-questions@freebsd.org Subject: Corrupt data with vinum in 4.6.2-RELEASE Message-ID: <86lm58w796.fsf@duchess.twilley.org>
next in thread | raw e-mail | index | archive | help
--=-=-= Content-Transfer-Encoding: quoted-printable I had a bad experience this weekend with vinum, SCSI and new disks, and I'm not sure which part's at fault. I installed ten new 4G disks, built two striped plexes of five disks each, and mirrored them. Two drives on the second plex generated pages of SCSI errors on the console when I was populating the new volume from my backups. Many of the files were corrupted in the restore. The vinum statistics reported no errors for the drive. I stopped the second plex, rebooted, and did another restore. The restore went flawlessly. I started the second plex, and the same two drives generated pages of SCSI errors on the console. Still no errors reported by vinum. The files were corrupted. What's going on here? There were many of these: Oct 7 01:24:28 duchess /kernel: (da11:ahc0:0:15:0): WRITE(10). CDB: 2a 0 0= 67 1 41 0 0 80 0=20 Oct 7 01:24:28 duchess /kernel: (da11:ahc0:0:15:0): ABORTED COMMAND asc:1b= ,0 Oct 7 01:24:28 duchess /kernel: (da11:ahc0:0:15:0): Synchronous data trans= fer error And then quite a few of these: Oct 7 01:36:32 duchess /kernel: (da4:ahc0:0:8:0): SCB 0x61 - timed out Oct 7 01:36:49 duchess /kernel: ahc0: Dumping Card State in Data-out phase= , at SEQADDR 0x7c Oct 7 01:36:49 duchess /kernel: ACCUM =3D 0x0, SINDEX =3D 0x8, DINDEX =3D = 0x8f, ARG_2 =3D 0xe Oct 7 01:36:49 duchess /kernel: HCNT =3D 0xe0 SCBPTR =3D 0x1 Oct 7 01:36:49 duchess /kernel: SCSISEQ =3D 0x12, SBLKCTL =3D 0x2 Oct 7 01:36:49 duchess /kernel: DFCNTRL =3D 0x7c, DFSTATUS =3D 0x6 Oct 7 01:36:49 duchess /kernel: LASTPHASE =3D 0x0, SCSISIGI =3D 0x4, SXFRC= TL0 =3D 0xa0 Oct 7 01:36:49 duchess /kernel: SSTAT0 =3D 0x0, SSTAT1 =3D 0x2 Oct 7 01:36:49 duchess /kernel: STACK =3D=3D 0x99, 0x190, 0x160, 0x0 Oct 7 01:36:49 duchess /kernel: SCB count =3D 254 Oct 7 01:36:49 duchess /kernel: Kernel NEXTQSCB =3D 138 Oct 7 01:36:49 duchess /kernel: Card NEXTQSCB =3D 47 Oct 7 01:36:50 duchess /kernel: QINFIFO entries: 47 25 89 74 62 116 94 137= 95 78 31 123 159 63 165 214 43 223 69 84 14 103 16 59 57 183=20 Oct 7 01:36:50 duchess /kernel: Waiting Queue entries:=20 Oct 7 01:36:50 duchess /kernel: Disconnected Queue entries: 14:97=20 Oct 7 01:36:50 duchess /kernel: QOUTFIFO entries:=20 Oct 7 01:36:50 duchess /kernel: Sequencer Free SCB List: 11 8 5 6 7 9 13 2= 0 10 4 12 15 3=20 Oct 7 01:36:50 duchess /kernel: Sequencer SCB Info: 0(c 0x68, s 0xb7, l 0,= t 0xff) 1(c 0x68, s 0xd7, l 0, t 0xbf) 2(c 0x68, s 0x57, l 0, t 0xff) 3(c = 0x68, s 0xc7, l 0, t 0xff) 4(c 0x68, s 0xc7, l 0, t 0xff) 5(c 0x68, s 0xd7,= l 0, t 0xff) 6(c 0x68, s 0x87, l 0, t 0xff) 7(c 0x68, s 0xd7, l 0, t 0xff)= 8(c 0x68, s 0xd7, l 0, t 0xff) 9(c 0x68, s 0xb7, l 0, t 0xff) 10(c 0x68, s= 0x57, l 0, t 0xff) 11(c 0x68, s 0xc7, l 0, t 0xff) 12(c 0x68, s 0x47, l 0,= t 0xff) 13(c 0x68, s 0x87, l 0, t 0xff) 14(c 0x6c, s 0x87, l 0, t 0x61) 15= (c 0x68, s 0xc7, l 0, t 0xff)=20 Oct 7 01:36:50 duchess /kernel: Pending list: 183(c 0x68, s 0xc7, l 0), 57= (c 0x6a, s 0xc7, l 0), 59(c 0x6a, s 0x57, l 0), 16(c 0x68, s 0xa7, l 0), 10= 3(c 0x68, s 0xa7, l 0), 14(c 0x68, s 0xa7, l 0), 84(c 0x68, s 0xa7, l 0), 6= 9(c 0x68, s 0xa7, l 0), 223(c 0x68, s 0x97, l 0), 43(c 0x68, s 0x97, l 0), = 214(c 0x68, s 0x97, l 0), 165(c 0x68, s 0x97, l 0), 63(c 0x68, s 0x97, l 0)= , 159(c 0x68, s 0x97, l 0), 123(c 0x68, s 0x97, l 0), 31(c 0x68, s 0xd7, l = 0), 78(c 0x68, s 0x97, l 0), 95(c 0x68, s 0x87, l 0), 137(c 0x68, s 0xd7, l= 0), 94(c 0x68, s 0x87, l 0), 116(c 0x68, s 0xd7, l 0), 62(c 0x68, s 0x87, = l 0), 74(c 0x68, s 0xd7, l 0), 89(c 0x68, s 0x87, l 0), 25(c 0x68, s 0xd7, = l 0), 47(c 0x68, s 0x87, l 0), 191(c 0x68, s 0xd7, l 0), 97(c 0x68, s 0x87,= l 0) Oct 7 01:36:50 duchess /kernel: Kernel Free SCB list: 49 225 13 109 150 42= 217 32 10 227 101 29 142 90 232 22 105 229 44 193 173 133 20 82 195 23 224= 112 242 65 202 160 244 172 117 185 236 124 68 241 189 33 28 228 88 75 104 = 164 41 118 76 246 233 144 206 85 53 58 179 234 81 162 121 15 192 201 226 19= 6 70 0 240 80 122 249 238 140 213 230 21 176 66 208 50 72 8 126 211 198 30 = 204 188 219 134 108 207 143 1 98 222 11 149 231 209 139 24 107 93 171 19 13= 0 127 79 120 197 39 2 71 106 187 113 132 131 145 186 125 216 34 77 200 141 = 155 167 114 252 35 83 237 221 203 218 161 56 158 157 147 18 251 55 17 169 1= 90 243 210 115 253 38 180 48 26 248 220 152 153 99 168 110 154 129 151 92 1= 82 136 235 239 64 54 27 245 60 177 194 52 119 9 156 7 163 96 247 215 212 73= 111 102 12 51 86 40 91 37 4 46 166 87 199 100 146 184 178 45 174 67 3 135 = 5 205 148 181 61 6 128 170 250 36 175=20 Oct 7 01:36:50 duchess /kernel: sg[0] - Addr 0x249f000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[1] - Addr 0x6460000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[2] - Addr 0x41a1000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[3] - Addr 0x45c2000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[4] - Addr 0x6583000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[5] - Addr 0x3c04000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[6] - Addr 0x2c25000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[7] - Addr 0x2da6000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[8] - Addr 0x5487000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[9] - Addr 0x71a8000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[10] - Addr 0x6fc9000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[11] - Addr 0x6f8a000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[12] - Addr 0x100b000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[13] - Addr 0xf6c000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[14] - Addr 0x416d000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[15] - Addr 0x6e8e000 : Length 4096 Oct 7 01:36:50 duchess /kernel: (da4:ahc0:0:8:0): Other SCB Timeout Oct 7 01:36:50 duchess /kernel: (da9:ahc0:0:13:0): SCB 0xbf - timed out Oct 7 01:36:50 duchess /kernel: ahc0: Dumping Card State in Data-out phase= , at SEQADDR 0x7d Oct 7 01:36:50 duchess /kernel: ACCUM =3D 0x0, SINDEX =3D 0x8, DINDEX =3D = 0x8f, ARG_2 =3D 0xe Oct 7 01:36:50 duchess /kernel: HCNT =3D 0xe0 SCBPTR =3D 0x1 Oct 7 01:36:50 duchess /kernel: SCSISEQ =3D 0x12, SBLKCTL =3D 0x2 Oct 7 01:36:50 duchess /kernel: DFCNTRL =3D 0x7c, DFSTATUS =3D 0x6 Oct 7 01:36:50 duchess /kernel: LASTPHASE =3D 0x0, SCSISIGI =3D 0x4, SXFRC= TL0 =3D 0xa0 Oct 7 01:36:50 duchess /kernel: SSTAT0 =3D 0x0, SSTAT1 =3D 0x2 Oct 7 01:36:50 duchess /kernel: STACK =3D=3D 0x99, 0x190, 0x160, 0x0 Oct 7 01:36:50 duchess /kernel: SCB count =3D 254 Oct 7 01:36:50 duchess /kernel: Kernel NEXTQSCB =3D 138 Oct 7 01:36:50 duchess /kernel: Card NEXTQSCB =3D 47 Oct 7 01:36:50 duchess /kernel: QINFIFO entries: 47 25 89 74 62 116 94 137= 95 78 31 123 159 63 165 214 43 223 69 84 14 103 16 59 57 183=20 Oct 7 01:36:50 duchess /kernel: Waiting Queue entries:=20 Oct 7 01:36:50 duchess /kernel: Disconnected Queue entries: 14:97=20 Oct 7 01:36:50 duchess /kernel: QOUTFIFO entries:=20 Oct 7 01:36:50 duchess /kernel: Sequencer Free SCB List: 11 8 5 6 7 9 13 2= 0 10 4 12 15 3=20 Oct 7 01:36:50 duchess /kernel: Sequencer SCB Info: 0(c 0x68, s 0xb7, l 0,= t 0xff) 1(c 0x68, s 0xd7, l 0, t 0xbf) 2(c 0x68, s 0x57, l 0, t 0xff) 3(c = 0x68, s 0xc7, l 0, t 0xff) 4(c 0x68, s 0xc7, l 0, t 0xff) 5(c 0x68, s 0xd7,= l 0, t 0xff) 6(c 0x68, s 0x87, l 0, t 0xff) 7(c 0x68, s 0xd7, l 0, t 0xff)= 8(c 0x68, s 0xd7, l 0, t 0xff) 9(c 0x68, s 0xb7, l 0, t 0xff) 10(c 0x68, s= 0x57, l 0, t 0xff) 11(c 0x68, s 0xc7, l 0, t 0xff) 12(c 0x68, s 0x47, l 0,= t 0xff) 13(c 0x68, s 0x87, l 0, t 0xff) 14(c 0x6c, s 0x87, l 0, t 0x61) 15= (c 0x68, s 0xc7, l 0, t 0xff)=20 Oct 7 01:36:50 duchess /kernel: Pending list: 183(c 0x68, s 0xc7, l 0), 57= (c 0x6a, s 0xc7, l 0), 59(c 0x6a, s 0x57, l 0), 16(c 0x68, s 0xa7, l 0), 10= 3(c 0x68, s 0xa7, l 0), 14(c 0x68, s 0xa7, l 0), 84(c 0x68, s 0xa7, l 0), 6= 9(c 0x68, s 0xa7, l 0), 223(c 0x68, s 0x97, l 0), 43(c 0x68, s 0x97, l 0), = 214(c 0x68, s 0x97, l 0), 165(c 0x68, s 0x97, l 0), 63(c 0x68, s 0x97, l 0)= , 159(c 0x68, s 0x97, l 0), 123(c 0x68, s 0x97, l 0), 31(c 0x68, s 0xd7, l = 0), 78(c 0x68, s 0x97, l 0), 95(c 0x68, s 0x87, l 0), 137(c 0x68, s 0xd7, l= 0), 94(c 0x68, s 0x87, l 0), 116(c 0x68, s 0xd7, l 0), 62(c 0x68, s 0x87, = l 0), 74(c 0x68, s 0xd7, l 0), 89(c 0x68, s 0x87, l 0), 25(c 0x68, s 0xd7, = l 0), 47(c 0x68, s 0x87, l 0), 191(c 0x68, s 0xd7, l 0), 97(c 0x68, s 0x87,= l 0) Oct 7 01:36:50 duchess /kernel: Kernel Free SCB list: 49 225 13 109 150 42= 217 32 10 227 101 29 142 90 232 22 105 229 44 193 173 133 20 82 195 23 224= 112 242 65 202 160 244 172 117 185 236 124 68 241 189 33 28 228 88 75 104 = 164 41 118 76 246 233 144 206 85 53 58 179 234 81 162 121 15 192 201 226 19= 6 70 0 240 80 122 249 238 140 213 230 21 176 66 208 50 72 8 126 211 198 30 = 204 188 219 134 108 207 143 1 98 222 11 149 231 209 139 24 107 93 171 19 13= 0 127 79 120 197 39 2 71 106 187 113 132 131 145 186 125 216 34 77 200 141 = 155 167 114 252 35 83 237 221 203 218 161 56 158 157 147 18 251 55 17 169 1= 90 243 210 115 253 38 180 48 26 248 220 152 153 99 168 110 154 129 151 92 1= 82 136 235 239 64 54 27 245 60 177 194 52 119 9 156 7 163 96 247 215 212 73= 111 102 12 51 86 40 91 37 4 46 166 87 199 100 146 184 178 45 174 67 3 135 = 5 205 148 181 61 6 128 170 250 36 175=20 Oct 7 01:36:50 duchess /kernel: sg[0] - Addr 0x249f000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[1] - Addr 0x6460000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[2] - Addr 0x41a1000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[3] - Addr 0x45c2000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[4] - Addr 0x6583000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[5] - Addr 0x3c04000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[6] - Addr 0x2c25000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[7] - Addr 0x2da6000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[8] - Addr 0x5487000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[9] - Addr 0x71a8000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[10] - Addr 0x6fc9000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[11] - Addr 0x6f8a000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[12] - Addr 0x100b000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[13] - Addr 0xf6c000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[14] - Addr 0x416d000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[15] - Addr 0x6e8e000 : Length 4096 Oct 7 01:36:50 duchess /kernel: (da9:ahc0:0:13:0): BDR message in message = buffer Oct 7 01:36:50 duchess /kernel: (da9:ahc0:0:13:0): SCB 0xbf - timed out Oct 7 01:36:50 duchess /kernel: ahc0: Dumping Card State in Data-out phase= , at SEQADDR 0x7c Oct 7 01:36:50 duchess /kernel: ACCUM =3D 0x0, SINDEX =3D 0x8, DINDEX =3D = 0x8f, ARG_2 =3D 0xe Oct 7 01:36:50 duchess /kernel: HCNT =3D 0xe0 SCBPTR =3D 0x1 Oct 7 01:36:50 duchess /kernel: SCSISEQ =3D 0x12, SBLKCTL =3D 0x2 Oct 7 01:36:50 duchess /kernel: DFCNTRL =3D 0x7c, DFSTATUS =3D 0x6 Oct 7 01:36:50 duchess /kernel: LASTPHASE =3D 0x0, SCSISIGI =3D 0x14, SXFR= CTL0 =3D 0xa0 Oct 7 01:36:50 duchess /kernel: SSTAT0 =3D 0x0, SSTAT1 =3D 0x2 Oct 7 01:36:50 duchess /kernel: STACK =3D=3D 0x99, 0x190, 0x160, 0x0 Oct 7 01:36:50 duchess /kernel: SCB count =3D 254 Oct 7 01:36:50 duchess /kernel: Kernel NEXTQSCB =3D 138 Oct 7 01:36:50 duchess /kernel: Card NEXTQSCB =3D 47 Oct 7 01:36:50 duchess /kernel: QINFIFO entries: 47 25 89 74 62 116 94 137= 95 78 31 123 159 63 165 214 43 223 69 84 14 103 16 59 57 183=20 Oct 7 01:36:50 duchess /kernel: Waiting Queue entries:=20 Oct 7 01:36:50 duchess /kernel: Disconnected Queue entries: 14:97=20 Oct 7 01:36:50 duchess /kernel: QOUTFIFO entries:=20 Oct 7 01:36:50 duchess /kernel: Sequencer Free SCB List: 11 8 5 6 7 9 13 2= 0 10 4 12 15 3=20 Oct 7 01:36:50 duchess /kernel: Sequencer SCB Info: 0(c 0x68, s 0xb7, l 0,= t 0xff) 1(c 0x68, s 0xd7, l 0, t 0xbf) 2(c 0x68, s 0x57, l 0, t 0xff) 3(c = 0x68, s 0xc7, l 0, t 0xff) 4(c 0x68, s 0xc7, l 0, t 0xff) 5(c 0x68, s 0xd7,= l 0, t 0xff) 6(c 0x68, s 0x87, l 0, t 0xff) 7(c 0x68, s 0xd7, l 0, t 0xff)= 8(c 0x68, s 0xd7, l 0, t 0xff) 9(c 0x68, s 0xb7, l 0, t 0xff) 10(c 0x68, s= 0x57, l 0, t 0xff) 11(c 0x68, s 0xc7, l 0, t 0xff) 12(c 0x68, s 0x47, l 0,= t 0xff) 13(c 0x68, s 0x87, l 0, t 0xff) 14(c 0x6c, s 0x87, l 0, t 0x61) 15= (c 0x68, s 0xc7, l 0, t 0xff)=20 Oct 7 01:36:50 duchess /kernel: Pending list: 183(c 0x68, s 0xc7, l 0), 57= (c 0x6a, s 0xc7, l 0), 59(c 0x6a, s 0x57, l 0), 16(c 0x68, s 0xa7, l 0), 10= 3(c 0x68, s 0xa7, l 0), 14(c 0x68, s 0xa7, l 0), 84(c 0x68, s 0xa7, l 0), 6= 9(c 0x68, s 0xa7, l 0), 223(c 0x68, s 0x97, l 0), 43(c 0x68, s 0x97, l 0), = 214(c 0x68, s 0x97, l 0), 165(c 0x68, s 0x97, l 0), 63(c 0x68, s 0x97, l 0)= , 159(c 0x68, s 0x97, l 0), 123(c 0x68, s 0x97, l 0), 31(c 0x68, s 0xd7, l = 0), 78(c 0x68, s 0x97, l 0), 95(c 0x68, s 0x87, l 0), 137(c 0x68, s 0xd7, l= 0), 94(c 0x68, s 0x87, l 0), 116(c 0x68, s 0xd7, l 0), 62(c 0x68, s 0x87, = l 0), 74(c 0x68, s 0xd7, l 0), 89(c 0x68, s 0x87, l 0), 25(c 0x68, s 0xd7, = l 0), 47(c 0x68, s 0x87, l 0), 191(c 0x68, s 0xd7, l 0), 97(c 0x68, s 0x87,= l 0) Oct 7 01:36:50 duchess /kernel: Kernel Free SCB list: 49 225 13 109 150 42= 217 32 10 227 101 29 142 90 232 22 105 229 44 193 173 133 20 82 195 23 224= 112 242 65 202 160 244 172 117 185 236 124 68 241 189 33 28 228 88 75 104 = 164 41 118 76 246 233 144 206 85 53 58 179 234 81 162 121 15 192 201 226 19= 6 70 0 240 80 122 249 238 140 213 230 21 176 66 208 50 72 8 126 211 198 30 = 204 188 219 134 108 207 143 1 98 222 11 149 231 209 139 24 107 93 171 19 13= 0 127 79 120 197 39 2 71 106 187 113 132 131 145 186 125 216 34 77 200 141 = 155 167 114 252 35 83 237 221 203 218 161 56 158 157 147 18 251 55 17 169 1= 90 243 210 115 253 38 180 48 26 248 220 152 153 99 168 110 154 129 151 92 1= 82 136 235 239 64 54 27 245 60 177 194 52 119 9 156 7 163 96 247 215 212 73= 111 102 12 51 86 40 91 37 4 46 166 87 199 100 146 184 178 45 174 67 3 135 = 5 205 148 181 61 6 128 170 250 36 175=20 Oct 7 01:36:50 duchess /kernel: sg[0] - Addr 0x249f000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[1] - Addr 0x6460000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[2] - Addr 0x41a1000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[3] - Addr 0x45c2000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[4] - Addr 0x6583000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[5] - Addr 0x3c04000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[6] - Addr 0x2c25000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[7] - Addr 0x2da6000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[8] - Addr 0x5487000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[9] - Addr 0x71a8000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[10] - Addr 0x6fc9000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[11] - Addr 0x6f8a000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[12] - Addr 0x100b000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[13] - Addr 0xf6c000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[14] - Addr 0x416d000 : Length 4096 Oct 7 01:36:50 duchess /kernel: sg[15] - Addr 0x6e8e000 : Length 4096 Oct 7 01:36:50 duchess /kernel: (da9:ahc0:0:13:0): no longer in timeout, s= tatus =3D 34b I am not on the mailing list, so please cc me on any replies. If there's anything else I can do to help, please let me know. Jack. =2D-=20 Jack Twilley jmt at twilley dot org http colon slash slash www dot twilley dot org slash tilde jmt slash --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.0 (FreeBSD) iD8DBQA9o7RLGPFSfAB/ezgRArSbAKDsJU5ML+LuBVtbmCrZYKnUMqelYgCeJIYX vhTkNrXf+QflgIkeDnHjrbU= =BTrh -----END PGP SIGNATURE----- --=-=-=-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?86lm58w796.fsf>