Date: Mon, 29 Apr 2013 12:08:39 +0100 From: "Steven Hartland" <killing@multiplay.co.uk> To: "Jeremy Chadwick" <jdc@koitsu.org> Cc: freebsd-fs@freebsd.org, Alexander Motin <mav@freebsd.org> Subject: Re: seeing data corruption with zfs trim functionality Message-ID: <6240B204CEB04158968B0C7AAAA98248@multiplay.co.uk> References: <CAA71u6Y5dKZ9O0rqxCpx-9t7DYgTnPZSoNy-iHOnmzrOUYp%2Bvw@mail.gmail.com> <60316751643743738AB83DABC6A5934B@multiplay.co.uk> <20130429105143.GA1492@icarus.home.lan> <20130429105916.GA1584@icarus.home.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
----- Original Message ----- From: "Jeremy Chadwick" <jdc@koitsu.org> To: "Steven Hartland" <killing@multiplay.co.uk> Cc: <freebsd-fs@freebsd.org>; "Alexander Motin" <mav@freebsd.org> Sent: Monday, April 29, 2013 11:59 AM Subject: Re: seeing data corruption with zfs trim functionality > On Mon, Apr 29, 2013 at 03:51:43AM -0700, Jeremy Chadwick wrote: >> On Mon, Apr 29, 2013 at 09:22:06AM +0100, Steven Hartland wrote: >> > ----- Original Message ----- From: "Ajit Jain" >> > <ajit.jain@cloudbyte.com> >> > >> > >I am running zfs with trim functionality (ported from head). Seeing data >> > >corruption when running iotest* with multiple threads (never saw data >> > >corruption with single thread). >> > > >> > >The patches merged to add trim support are as follows: >> > >1. 240868 (zfs trim patch) >> > >2. 230053 and 245252 (block device driver trim support) >> > >3. 239655 (fix an issue in patch 230053) >> > > >> > >I am "NOT" seeing data corruption in the following cases: >> > >1. Running iotest with single thread (Trim is enabled at entire io stack). >> > >2. Trim is enabled at zfs layer but disable at driver layer i.e. delete >> > >method is set to NONE (even with multiple threads). >> > > >> > > >> > >Since patch 240868 alone was not working as I pulled in additional zfs trim >> > >patches 244155, 244187, 244188, 248572 (however I am not using separate >> > >L2arc device), 248573, 248574, 248575 and 248576. Still I am seeing the >> > >same issue. >> > > >> > >Issue: After some time running with multiple thread write system call >> > >return sometimes with EIO or 122 (checksum error) error code. >> > > >> > >I looked at GEOM code a bit I think it already has the trim (DELETE) >> > >command support. Still I am doubtful if I have pulled in all required >> > >patches in the entire I/O stack. >> > > >> > >I am using a LSI SAS HBA card to connect to the SSD, firmware seems to >> > >claim the support for trim. >> > > >> > >*iotest: non standard freebsd FreeBSD utility, which creates files and does >> > >I/O on the files and can be invoked in single/multithread mode to do the >> > >I/O. >> > >> > What version are you porting the changes to? >> > >> > What SSD are you using? >> > >> > What LSI controller are you using? >> >> I'd also like to see "zpool status" (for every pool that involves this >> SSD) and "gpart show" against the disk itself. > > Also, the controller involved is an mps(4) controller, which to the > underlying subsystem is SCSI. > > TRIM (as it's called; the actual name per ATA standard is DATA SET > MANAGEMENT) is purely an ATA specification thing. > > The SCSI equivalent is called UNMAP, or alternately WRITE SAME. > > (This is not the case here, but just mentioning it: even in the cases of > SCSI controllers that have SATA disks attached, the OS ends up > submitting UNMAP/WRITE SAME, which the controller has to convert into > the relevant ATA DATA SET MANAGEMENT command. If the controller > firmware screws this up there not much we can do about it) > > References for FreeBSD: > > http://lists.freebsd.org/pipermail/freebsd-current/2011-December/030714.html > > PLEASE READ THE LAST PARAGRAPH OF THAT POST. > > This brings into question whether or not relevant subsystems (ranging > from mps(4) to GEOM(4) to CAM(4)) actually have working UNMAP/WRITE SAME > or not, or if the controller itself is doing something stupid with them. > > I'm CC'ing mav@ for what should be obvious reasons. ZFS "TRIM" just uses BIO_DELETE, which is translated to the relevant supported delete_method. For SATA disks this can be UNMAP, TRIM etc even when connected to SCSI controller; for SAS disks this can UNMAP, WS16 etc. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?6240B204CEB04158968B0C7AAAA98248>