From owner-freebsd-fs@freebsd.org Tue Dec 1 14:39:01 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6B8C9A3EA25 for ; Tue, 1 Dec 2015 14:39:01 +0000 (UTC) (envelope-from csforgeron@gmail.com) Received: from mail-io0-x230.google.com (mail-io0-x230.google.com [IPv6:2607:f8b0:4001:c06::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 342EE152B; Tue, 1 Dec 2015 14:39:01 +0000 (UTC) (envelope-from csforgeron@gmail.com) Received: by iouu10 with SMTP id u10so11327193iou.0; Tue, 01 Dec 2015 06:39:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=GrelPFeqFc2JyjbYRU31ul80xKukKWzUd126BSxFHxA=; b=TS9DHnb/pncRH8ZBhjewt9hrT1FDSpmCJnKYlLIZzHzb5w2iBnwrsjM/wpp/z7kKLR 1C2az0w9nNPIrL4Gr/Wte2tGJiONnryhCVNjoL97Sel0sjYARCcMULdNyOdeCR3hJr2o PXK2afOArWTLATD7jFhDRR1njIw5197ktyrFNRrmnGNn7IxVgBkKIHUlInT4Ti/Zxzm2 PgyL1WbZhv4cHXwK5Iqdwuop40n5Z3nGN84h3RfrJwSTtv0INTED/91GrtU+OeHx1dxt jWnlIFTvZvaSxhmZFLI9c8HVQf9GqQTBUBlb6s+VV+aAw7v9GnV9dYYIMQFgTKbsfPuZ ACew== MIME-Version: 1.0 X-Received: by 10.107.137.226 with SMTP id t95mr61959341ioi.188.1448980740704; Tue, 01 Dec 2015 06:39:00 -0800 (PST) Received: by 10.36.159.67 with HTTP; Tue, 1 Dec 2015 06:39:00 -0800 (PST) In-Reply-To: <54F88DEA.2070301@hotplug.ru> References: <54F88DEA.2070301@hotplug.ru> Date: Tue, 1 Dec 2015 10:39:00 -0400 Message-ID: Subject: Re: CAM Target over FC and UNMAP problem From: Christopher Forgeron To: Emil Muratov Cc: FreeBSD Filesystems , Alexander Motin Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Dec 2015 14:39:01 -0000 Did this ever progress further? I'm load testing 10.2 zvol / UNMAP, and having similar lockup issues. On Thu, Mar 5, 2015 at 1:10 PM, Emil Muratov wrote: > > I've got an issue with CTL UNMAP and zvol backends. > Seems that UNMAP from the initiator passed to the underlying disks > (without trim support) causes IO blocking to the whole pool. Not sure > where to address this problem. > > My setup: > - plain SATA 7.2 krpm drives attached to Adaptec aacraid SAS controller > - zfs raidz pool over plain drives, no partitioning > - zvol created with volmode=dev > - Qlogic ISP 2532 FC HBA in target mode > - FreeBSD 10.1-STABLE #1 r279593 > > Create a new LUN with a zvol backend > > ctladm realsync off > ctladm port -o on -p 5 > ctladm create -b block -o file=/dev/zvol/wd/tst1 -o unmap=on -l 0 -d > wd.tst1 -S tst1 > > Both target an initiator hosts connected to the FC fabric. Initiator is > Win2012 server, actually it is a VM with RDM LUN to the guest OS. > Formating, reading and writing large amounts of data (file copy/IOmeter) > - so far so good. > But as soon as I've tried to delete large files all IO to the LUN > blocks, initiator system just iowaits. gstat on target shows that > underlying disk load bumped to 100%, queue up to 10, but no iowrites > actually in progress, only decent amount of ioreads. After a minute or > so IO unblocks for a second or two than blocks again and so on again > until all UNMAPs are done, it could take up to 5 minutes to delete 10Gb > file. I can see that 'logicalused' property of a zvol shows that the > deleted space was actually released. System log is filled with CTL msgs: > > > kernel: (ctl2:isp1:0:0:3): ctlfestart: aborted command 0x12aaf4 discarded > kernel: (2:5:3/3): WRITE(10). CDB: 2a 00 2f d4 74 b8 00 00 08 00 > kernel: (2:5:3/3): Tag: 0x12ab24, type 1 > kernel: (2:5:3/3): ctl_process_done: 96 seconds > kernel: (ctl2:isp1:0:0:3): ctlfestart: aborted command 0x12afa4 discarded > kernel: (ctl2:isp1:0:0:3): ctlfestart: aborted command 0x12afd4 discarded > kernel: ctlfedone: got XPT_IMMEDIATE_NOTIFY status 0x36 tag 0xffffffff > seq 0x121104 > kernel: (ctl2:isp1:0:0:3): ctlfe_done: returning task I/O tag 0xffffffff > seq 0x1210d4 > > > I've tried to tackle some sysctls, but no success so far. > > vfs.zfs.vdev.bio_flush_disable: 1 > vfs.zfs.vdev.bio_delete_disable: 1 > vfs.zfs.trim.enabled=0 > > > Disabling UNMAP in CTL (-o unmap=off) resolves the issue completely but > than there is no space reclamation for zvol. > > Any hints would be appreciated. > > > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >