Date: Mon, 29 Apr 2013 11:44:00 +0100 From: "Steven Hartland" <killing@multiplay.co.uk> To: "Ajit Jain" <ajit.jain@cloudbyte.com> Cc: freebsd-fs <freebsd-fs@freebsd.org> Subject: Re: seeing data corruption with zfs trim functionality Message-ID: <3BFB7F45D8AF46879238F99E6CAEA348@multiplay.co.uk> References: <CAA71u6Y5dKZ9O0rqxCpx-9t7DYgTnPZSoNy-iHOnmzrOUYp%2Bvw@mail.gmail.com> <60316751643743738AB83DABC6A5934B@multiplay.co.uk> <CAA71u6ZehJq7LE-m3RMCGgZQPT4U%2BO_m6ezfT-a5ig%2Bd%2Bc8uzQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format.
------=_NextPart_000_0940_01CE44CE.D6F204E0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Ooo a SAS SSD.
Can you:-
1. Test with a SATA ssd after applying the attached patches.
2. Rerun from a current kernel and see if you still see corruption, this will eliminate the possibility of any missing patches.
----- Original Message -----=20
From: Ajit Jain=20
To: Steven Hartland=20
Cc: freebsd-fs=20
Sent: Monday, April 29, 2013 11:20 AM
Subject: Re: seeing data corruption with zfs trim functionality
Hi Steven,
Freebsd Version: 9
SSD: Seagate SSD, complete smartctl output is attached with the mail.
Not sure if I could provide the SSD information that you were looking for.
If not, could you please tell me command (if any) to get the information.
LSI card: =20
mpslsi0@pci0:2:0:0: class=3D0x010700 card=3D0x30801000 chip=3D0x00721000 rev=3D0x03 hdr=3D0x00
vendor =3D 'LSI Logic / Symbios Logic'
device =3D 'SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]'
class =3D mass storage
subclass =3D SAS
Complete pciconf -lv output is attached with mail.
thanks
ajit
On Mon, Apr 29, 2013 at 1:52 PM, Steven Hartland <killing@multiplay.co.uk> wrote:
----- Original Message ----- From: "Ajit Jain" <ajit.jain@cloudbyte.com>
I am running zfs with trim functionality (ported from head). Seeing data
corruption when running iotest* with multiple threads (never saw data
corruption with single thread).
The patches merged to add trim support are as follows:
1. 240868 (zfs trim patch)
2. 230053 and 245252 (block device driver trim support)
3. 239655 (fix an issue in patch 230053)
I am "NOT" seeing data corruption in the following cases:
1. Running iotest with single thread (Trim is enabled at entire io stack).
2. Trim is enabled at zfs layer but disable at driver layer i.e. delete
method is set to NONE (even with multiple threads).
Since patch 240868 alone was not working as I pulled in additional zfs trim
patches 244155, 244187, 244188, 248572 (however I am not using separate
L2arc device), 248573, 248574, 248575 and 248576. Still I am seeing the
same issue.
Issue: After some time running with multiple thread write system call
return sometimes with EIO or 122 (checksum error) error code.
I looked at GEOM code a bit I think it already has the trim (DELETE)
command support. Still I am doubtful if I have pulled in all required
patches in the entire I/O stack.
I am using a LSI SAS HBA card to connect to the SSD, firmware seems to
claim the support for trim.
*iotest: non standard freebsd FreeBSD utility, which creates files and does
I/O on the files and can be invoked in single/multithread mode to do the
I/O.
What version are you porting the changes to?
What SSD are you using?
What LSI controller are you using?
Regards
Steve
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it.=20
In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337
or return the E.mail to postmaster@multiplay.co.uk.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it.=20
In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337
or return the E.mail to postmaster@multiplay.co.uk.
------=_NextPart_000_0940_01CE44CE.D6F204E0
Content-Type: application/octet-stream;
name="cam-scsi_da-reprobe.patch"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
filename="cam-scsi_da-reprobe.patch"
Fix probe in progress check in dareprobe=0A=
--- sys/cam/scsi/scsi_da.c.orig2 2013-04-27 23:53:11.000000000 +0000=0A=
+++ sys/cam/scsi/scsi_da.c 2013-04-28 16:05:48.187091065 +0000=0A=
@@ -3208,7 +3244,7 @@=0A=
softc =3D (struct da_softc *)periph->softc;=0A=
=0A=
/* Probe in progress; don't interfere. */=0A=
- if ((softc->flags & DA_FLAG_PROBED) =3D=3D 0)=0A=
+ if (softc->state !=3D DA_STATE_NORMAL)=0A=
return;=0A=
=0A=
status =3D cam_periph_acquire(periph);=0A=
------=_NextPart_000_0940_01CE44CE.D6F204E0
Content-Type: application/octet-stream;
name="cam-scsi_da-enable-trim.patch"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
filename="cam-scsi_da-enable-trim.patch"
Enable ATA TRIM support choice by autodetection and correct method names =
after=0A=
increasing the priority of ATA TRIM=0A=
Index: sys/cam/scsi/scsi_da.c=0A=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A=
--- sys/cam/scsi/scsi_da.c (revision 249941)=0A=
+++ sys/cam/scsi/scsi_da.c (working copy)=0A=
@@ -133,14 +133,14 @@=0A=
DA_DELETE_WS16,=0A=
DA_DELETE_WS10,=0A=
DA_DELETE_ZERO,=0A=
- DA_DELETE_MIN =3D DA_DELETE_UNMAP,=0A=
+ DA_DELETE_MIN =3D DA_DELETE_ATA_TRIM,=0A=
DA_DELETE_MAX =3D DA_DELETE_ZERO=0A=
} da_delete_methods;=0A=
=0A=
static const char *da_delete_method_names[] =3D=0A=
- { "NONE", "DISABLE", "UNMAP", "ATA_TRIM", "WS16", "WS10", "ZERO" };=0A=
+ { "NONE", "DISABLE", "ATA_TRIM", "UNMAP", "WS16", "WS10", "ZERO" };=0A=
static const char *da_delete_method_desc[] =3D=0A=
- { "NONE", "DISABLED", "UNMAP", "ATA TRIM", "WRITE SAME(16) with =
UNMAP",=0A=
+ { "NONE", "DISABLED", "ATA TRIM", "UNMAP", "WRITE SAME(16) with =
UNMAP",=0A=
"WRITE SAME(10) with UNMAP", "ZERO" };=0A=
=0A=
/* Offsets into our private area for storing information */=0A=
------=_NextPart_000_0940_01CE44CE.D6F204E0
Content-Type: application/octet-stream;
name="cam-scsi_da-probe-order.patch"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
filename="cam-scsi_da-probe-order.patch"
Update probe flow so that devices with lbp can also disable disksort.=0A=
=0A=
Ensure that delete_available is reset so re-probes after a media change,=0A=
to one with different delete characteristics, will result in the correct=0A=
methods being flagged as available.=0A=
=0A=
Make all ccb state changes use a consistent flow:=0A=
* free()=0A=
* xpt_release_ccb()=0A=
* softc->state =3D <new state>=0A=
* xpt_schedule()=0A=
--- sys/cam/scsi/scsi_da.c.orig3 2013-04-28 17:26:38.778757822 +0000=0A=
+++ sys/cam/scsi/scsi_da.c 2013-04-28 18:24:20.917876197 +0000=0A=
@@ -2398,7 +2398,7 @@=0A=
=0A=
if (!scsi_vpd_supported_page(periph, SVPD_BLOCK_LIMITS)) {=0A=
/* Not supported skip to next probe */=0A=
- softc->state =3D DA_STATE_PROBE_ATA;=0A=
+ softc->state =3D DA_STATE_PROBE_BDC;=0A=
goto skipstate;=0A=
}=0A=
=0A=
@@ -2745,9 +2745,9 @@=0A=
* with the short version of the command.=0A=
*/=0A=
if (maxsector =3D=3D 0xffffffff) {=0A=
- softc->state =3D DA_STATE_PROBE_RC16;=0A=
free(rdcap, M_SCSIDA);=0A=
xpt_release_ccb(done_ccb);=0A=
+ softc->state =3D DA_STATE_PROBE_RC16;=0A=
xpt_schedule(periph, priority);=0A=
return;=0A=
}=0A=
@@ -2849,9 +2849,9 @@=0A=
(error_code =3D=3D SSD_CURRENT_ERROR) &&=0A=
(sense_key =3D=3D SSD_KEY_ILLEGAL_REQUEST)))) {=0A=
softc->flags &=3D ~DA_FLAG_CAN_RC16;=0A=
- softc->state =3D DA_STATE_PROBE_RC;=0A=
free(rdcap, M_SCSIDA);=0A=
xpt_release_ccb(done_ccb);=0A=
+ softc->state =3D DA_STATE_PROBE_RC;=0A=
xpt_schedule(periph, priority);=0A=
return;=0A=
} else=0A=
@@ -2908,36 +2908,39 @@=0A=
&softc->sysctl_task);=0A=
xpt_announce_periph(periph, announce_buf);=0A=
=0A=
- if (lbp) {=0A=
- /*=0A=
- * Based on older SBC-3 spec revisions=0A=
- * any of the UNMAP methods "may" be=0A=
- * available via LBP given this flag so=0A=
- * we flag all of them as availble and=0A=
- * then remove those which further=0A=
- * probes confirm aren't available=0A=
- * later.=0A=
- *=0A=
- * We could also check readcap(16) p_type=0A=
- * flag to exclude one or more invalid=0A=
- * write same (X) types here=0A=
- */=0A=
- dadeleteflag(softc, DA_DELETE_WS16, 1);=0A=
- dadeleteflag(softc, DA_DELETE_WS10, 1);=0A=
- dadeleteflag(softc, DA_DELETE_ZERO, 1);=0A=
- dadeleteflag(softc, DA_DELETE_UNMAP, 1);=0A=
-=0A=
- softc->state =3D DA_STATE_PROBE_LBP;=0A=
- xpt_release_ccb(done_ccb);=0A=
- xpt_schedule(periph, priority);=0A=
- return;=0A=
- }=0A=
} else {=0A=
xpt_print(periph->path, "fatal error, "=0A=
"could not acquire reference count\n");=0A=
}=0A=
}=0A=
=0A=
+ /* Ensure re-probe doesn't see old delete. */=0A=
+ softc->delete_available =3D 0;=0A=
+ if (lbp) {=0A=
+ /*=0A=
+ * Based on older SBC-3 spec revisions=0A=
+ * any of the UNMAP methods "may" be=0A=
+ * available via LBP given this flag so=0A=
+ * we flag all of them as availble and=0A=
+ * then remove those which further=0A=
+ * probes confirm aren't available=0A=
+ * later.=0A=
+ *=0A=
+ * We could also check readcap(16) p_type=0A=
+ * flag to exclude one or more invalid=0A=
+ * write same (X) types here=0A=
+ */=0A=
+ dadeleteflag(softc, DA_DELETE_WS16, 1);=0A=
+ dadeleteflag(softc, DA_DELETE_WS10, 1);=0A=
+ dadeleteflag(softc, DA_DELETE_ZERO, 1);=0A=
+ dadeleteflag(softc, DA_DELETE_UNMAP, 1);=0A=
+=0A=
+ xpt_release_ccb(done_ccb);=0A=
+ softc->state =3D DA_STATE_PROBE_LBP;=0A=
+ xpt_schedule(periph, priority);=0A=
+ return;=0A=
+ }=0A=
+=0A=
xpt_release_ccb(done_ccb);=0A=
softc->state =3D DA_STATE_PROBE_BDC;=0A=
xpt_schedule(periph, priority);=0A=
@@ -2965,8 +2968,8 @@=0A=
=0A=
if (lbp->flags & SVPD_LBP_UNMAP) {=0A=
free(lbp, M_SCSIDA);=0A=
- softc->state =3D DA_STATE_PROBE_BLK_LIMITS;=0A=
xpt_release_ccb(done_ccb);=0A=
+ softc->state =3D DA_STATE_PROBE_BLK_LIMITS;=0A=
xpt_schedule(periph, priority);=0A=
return;=0A=
}=0A=
@@ -2995,7 +2998,7 @@=0A=
=0A=
free(lbp, M_SCSIDA);=0A=
xpt_release_ccb(done_ccb);=0A=
- softc->state =3D DA_STATE_PROBE_ATA;=0A=
+ softc->state =3D DA_STATE_PROBE_BDC;=0A=
xpt_schedule(periph, priority);=0A=
return;=0A=
}=0A=
@@ -3058,7 +3061,7 @@=0A=
=0A=
free(block_limits, M_SCSIDA);=0A=
xpt_release_ccb(done_ccb);=0A=
- softc->state =3D DA_STATE_PROBE_ATA;=0A=
+ softc->state =3D DA_STATE_PROBE_BDC;=0A=
xpt_schedule(periph, priority);=0A=
return;=0A=
}=0A=
@@ -3095,8 +3098,8 @@=0A=
}=0A=
=0A=
free(bdc, M_SCSIDA);=0A=
- softc->state =3D DA_STATE_PROBE_ATA;=0A=
xpt_release_ccb(done_ccb);=0A=
+ softc->state =3D DA_STATE_PROBE_ATA;=0A=
xpt_schedule(periph, priority);=0A=
return;=0A=
}=0A=
------=_NextPart_000_0940_01CE44CE.D6F204E0--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3BFB7F45D8AF46879238F99E6CAEA348>