From owner-freebsd-scsi@FreeBSD.ORG Mon Jan 2 11:07:11 2012 Return-Path: Delivered-To: freebsd-scsi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7A8051065672 for ; Mon, 2 Jan 2012 11:07:11 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 5EDC68FC1D for ; Mon, 2 Jan 2012 11:07:11 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q02B7BjJ005235 for ; Mon, 2 Jan 2012 11:07:11 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q02B7AOM005233 for freebsd-scsi@FreeBSD.org; Mon, 2 Jan 2012 11:07:10 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 2 Jan 2012 11:07:10 GMT Message-Id: <201201021107.q02B7AOM005233@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-scsi@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-scsi@FreeBSD.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jan 2012 11:07:11 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/163713 scsi [aic7xxx] [patch] Add Adaptec29329LPE to aic79xx_pci.c f kern/163130 scsi [mpt] cannot dumpon to mpt connected disk o kern/162256 scsi [mpt] QUEUE FULL EVENT and 'mpt_cam_event: 0x0' o kern/161809 scsi [cam] [patch] set kern.cam.boot_delay via build option o kern/159412 scsi [ciss] 7.3 RELEASE: ciss0 ADAPTER HEARTBEAT FAILED err o kern/157770 scsi [iscsi] [panic] iscsi_initiator panic o kern/154432 scsi [xpt] run_interrupt_driven_hooks: still waiting after o kern/153514 scsi [cam] [panic] CAM related panic o kern/153361 scsi [ciss] Smart Array 5300 boot/detect drive problem o kern/152250 scsi [ciss] [patch] Kernel panic when hw.ciss.expose_hidden o kern/151564 scsi [ciss] ciss(4) should increase CISS_MAX_LOGICAL to 10 o docs/151336 scsi Missing documentation of scsi_ and ata_ functions in c s kern/149927 scsi [cam] hard drive not stopped before removing power dur o kern/148083 scsi [aac] Strange device reporting o kern/147704 scsi [mpt] sys/dev/mpt: new chip revision, partially unsupp o kern/146287 scsi [ciss] ciss(4) cannot see more than one SmartArray con o kern/145768 scsi [mpt] can't perform I/O on SAS based SAN disk in freeb o kern/144648 scsi [aac] Strange values of speed and bus width in dmesg o kern/144301 scsi [ciss] [hang] HP proliant server locks when using ciss o kern/142351 scsi [mpt] LSILogic driver performance problems o kern/134488 scsi [mpt] MPT SCSI driver probes max. 8 LUNs per device o kern/132250 scsi [ciss] ciss driver does not support more then 15 drive o kern/132206 scsi [mpt] system panics on boot when mirroring and 2nd dri o kern/130621 scsi [mpt] tranfer rate is inscrutable slow when use lsi213 o kern/129602 scsi [ahd] ahd(4) gets confused and wedges SCSI bus o kern/128452 scsi [sa] [panic] Accessing SCSI tape drive randomly crashe o kern/128245 scsi [scsi] "inquiry data fails comparison at DV1 step" [re o kern/127927 scsi [isp] isp(4) target driver crashes kernel when set up o kern/127717 scsi [ata] [patch] [request] - support write cache toggling o kern/123674 scsi [ahc] ahc driver dumping o kern/123520 scsi [ahd] unable to boot from net while using ahd o sparc/121676 scsi [iscsi] iscontrol do not connect iscsi-target on sparc o kern/120487 scsi [sg] scsi_sg incompatible with scanners o kern/120247 scsi [mpt] FreeBSD 6.3 and LSI Logic 1030 = only 3.300MB/s o kern/114597 scsi [sym] System hangs at SCSI bus reset with dual HBAs o kern/110847 scsi [ahd] Tyan U320 onboard problem with more than 3 disks o kern/99954 scsi [ahc] reading from DVD failes on 6.x [regression] o kern/92798 scsi [ahc] SCSI problem with timeouts o kern/90282 scsi [sym] SCSI bus resets cause loss of ch device o kern/76178 scsi [ahd] Problem with ahd and large SCSI Raid system o kern/74627 scsi [ahc] [hang] Adaptec 2940U2W Can't boot 5.3 s kern/61165 scsi [panic] kernel page fault after calling cam_send_ccb o kern/60641 scsi [sym] Sporadic SCSI bus resets with 53C810 under load o kern/60598 scsi wire down of scsi devices conflicts with config s kern/57398 scsi [mly] Current fails to install on mly(4) based RAID di o bin/57088 scsi [cam] [patch] for a possible fd leak in libcam.c o kern/52638 scsi [panic] SCSI U320 on SMP server won't run faster than o kern/44587 scsi dev/dpt/dpt.h is missing defines required for DPT_HAND o kern/39388 scsi ncr/sym drivers fail with 53c810 and more than 256MB m o kern/35234 scsi World access to /dev/pass? (for scanner) requires acce 50 problems total. From owner-freebsd-scsi@FreeBSD.ORG Mon Jan 2 14:40:49 2012 Return-Path: Delivered-To: scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8D507106564A for ; Mon, 2 Jan 2012 14:40:49 +0000 (UTC) (envelope-from anonymous@s257.xrea.com) Received: from s257.xrea.com (s257.xrea.com [210.172.108.236]) by mx1.freebsd.org (Postfix) with SMTP id 2340A8FC14 for ; Mon, 2 Jan 2012 14:40:48 +0000 (UTC) Received: (qmail 11280 invoked by uid 1000); 2 Jan 2012 23:28:12 +0900 Date: 2 Jan 2012 23:28:12 +0900 Message-ID: <20120102142812.11279.qmail@s257.xrea.com> To: scsi@freebsd.org X-Pidtime: 8660 20120102232812 From: Halifax Online Banking Content-Transfer-Encoding: 8bit MIME-Version: 1.0 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Final Verification! X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jan 2012 14:40:49 -0000 halifax logo Compliment Of The Season Dear Halifax Customer! We Recently Notice That your Online Banking was Accessed From Another Location in year 2011. Therefore We Temporarily suspend your Online banking Access. which will help us fight against online fraud and threat. You are mandated to verify your online banking profile to enjoy the benefit of our new security programmed. You are mandated to verify your online banking profile to enjoy the benefit of our new security programmed. Kindly click on our website below to Restore your profile. [1]www.halifax.co.uk Sincerely, Halifax Bank Plc All rights reserved. Copyright Άν 2012 Halifax Bank Plc References 1. https://www.ditgolfudstyr.dk/golf/wp-content/themes/headlines_enhanced/functions/images/colorpicker/include/login.jsp.php From owner-freebsd-scsi@FreeBSD.ORG Wed Jan 4 15:50:12 2012 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 298A61065675 for ; Wed, 4 Jan 2012 15:50:12 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-ee0-f54.google.com (mail-ee0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id B600B8FC0C for ; Wed, 4 Jan 2012 15:50:10 +0000 (UTC) Received: by eekc50 with SMTP id c50so19930319eek.13 for ; Wed, 04 Jan 2012 07:50:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=M1ObeM28l8KIysX64Qupi1u8OfqJO4fNxyUtaEZk8k0=; b=SWw8sDQfUyHyUVy0MRAf07iH+8ds7tkcZhoB9DqKufZ5IOcKExm3X88Q5qkFkyoixM /2kY+ThyGugsieIGfhGnxhYWsydQ+MWGc05icvzLjzv3W4i8NzUxe0H6WTrvyII2M2/R jsk5NmQ4UXO87nyy0p6dbAEq8xC3QhizUDrTw= Received: by 10.213.108.132 with SMTP id f4mr15158297ebp.35.1325692209904; Wed, 04 Jan 2012 07:50:09 -0800 (PST) Received: from mavbook2.mavhome.dp.ua (pc.mavhome.dp.ua. [212.86.226.226]) by mx.google.com with ESMTPS id 76sm221061816eeh.0.2012.01.04.07.50.07 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 04 Jan 2012 07:50:08 -0800 (PST) Sender: Alexander Motin Message-ID: <4F04752E.8010805@FreeBSD.org> Date: Wed, 04 Jan 2012 17:50:06 +0200 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:9.0) Gecko/20111227 Thunderbird/9.0 MIME-Version: 1.0 To: freebsd-scsi@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Thoughts about CAM SWI X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jan 2012 15:50:12 -0000 Hi. Many times was risen question about extra context switch in CAM from interrupt thread to the CAM SWI on command completion. I've tried to analyze the ways how can it be avoided. The main problem I see there is a problems with reenterability of CAM itself, peripheral drivers and SIMs. In general case it looks unsafe to handle command completion directly from the xpt_action() call, as caller may not expect state changes at this point. It also looks unsafe to handle command completion directly from xpt_done() in interrupt thread, as SIM may not expect new requests getting in at this point, for example, if some error recovery is in progress, or it needs state to be consistent to handle other completed (possibly with errors) requests. The most complicated case I see is if the xpt_done() called from inside of sim_action() method of SIM. In that case direct command completion may recursively cause problems for all sides. I've tried to find places where this call loop can be broken. First place I was thinking about was the xpt_action() call. It looks possible to turn the code upside down to handle completion directly, but to not submit new requests to the controller immediately if we are called from the SIM context (another question is how to identify it). The problem is that there are set of non-queued SIM requests that may affect it's state, for example, XPT_SET_TRAN_SETTINGS or XPT_RESET_BUS. Positive side it that we may hope to avoid SIM modification. Second idea is to allow SIMs to be more reenterable. It can be done in two ways simultaneously on the SIM author's choice: - adding another version of xpt_done(), like xpt_done_direct() that would allowed immediate command completion if SIM is sure it can handle reentrancy at this point. - adding some functions, like xpt_batch_start() and xpt_batch_done() instructing CAM to queue xpt_done() calls as usual between them, but to not run SWI, instead handle them directly on the xpt_batch_done() call, that supposed to be called at point where SIM state is consistent and permits reenterability. This approach is very simple from the CAM point of view, but needs small SIMs modifications, while keeping full compatibility with unmodified. I've made the patch implementing the last way for all ATA SIMs: http://people.freebsd.org/~mav/cam_batch.patch Results can be illustrated by simple synthetic test: dd if=/dev/ada0 of=/dev/null bs=512 count=500000 , where ada0 is Intel SATA SSD: x before + after +------------------------------------------------------------+ | x + + | |xxxx xxx +++ + + +| | |_A__| |_MA___| | +------------------------------------------------------------+ N Min Max Median Avg Stddev x 8 16186671 16270294 16222511 16227889 28506.033 + 8 16802958 16929107 16832671 16843965 43561.511 Difference at 95.0% confidence 616076 +/- 39480.5 3.7964% +/- 0.243288% (Student's t, pooled s = 36811.7) Total number of context switches in system reduced from 315K to 260K. Removed context switches gave 3.8% speedup. May be that is not much, and affects only specific situations of the sequential very high request rate workaloads, but it is almost free. -- Alexander Motin From owner-freebsd-scsi@FreeBSD.ORG Wed Jan 4 16:34:42 2012 Return-Path: Delivered-To: freebsd-scsi@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 08E66106564A; Wed, 4 Jan 2012 16:34:42 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D27318FC13; Wed, 4 Jan 2012 16:34:41 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q04GYfxi037135; Wed, 4 Jan 2012 16:34:41 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q04GYfRe037131; Wed, 4 Jan 2012 16:34:41 GMT (envelope-from linimon) Date: Wed, 4 Jan 2012 16:34:41 GMT Message-Id: <201201041634.q04GYfRe037131@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-amd64@FreeBSD.org, freebsd-scsi@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/163812: [mpt] problem with mpt driver for lsi controlled connected to FreeBSD thru VT-d technology X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jan 2012 16:34:42 -0000 Old Synopsis: problem with mpt driver for lsi controlled connected to FreeBSD thru VT-d technology New Synopsis: [mpt] problem with mpt driver for lsi controlled connected to FreeBSD thru VT-d technology Responsible-Changed-From-To: freebsd-amd64->freebsd-scsi Responsible-Changed-By: linimon Responsible-Changed-When: Wed Jan 4 16:33:35 UTC 2012 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=163812 From owner-freebsd-scsi@FreeBSD.ORG Thu Jan 5 04:53:12 2012 Return-Path: Delivered-To: scsi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 44F35106564A; Thu, 5 Jan 2012 04:53:12 +0000 (UTC) (envelope-from ken@kdm.org) Received: from nargothrond.kdm.org (nargothrond.kdm.org [70.56.43.81]) by mx1.freebsd.org (Postfix) with ESMTP id DB6838FC08; Thu, 5 Jan 2012 04:53:11 +0000 (UTC) Received: from nargothrond.kdm.org (localhost [127.0.0.1]) by nargothrond.kdm.org (8.14.2/8.14.2) with ESMTP id q054rBMk040397; Wed, 4 Jan 2012 21:53:11 -0700 (MST) (envelope-from ken@nargothrond.kdm.org) Received: (from ken@localhost) by nargothrond.kdm.org (8.14.2/8.14.2/Submit) id q054rBeJ040396; Wed, 4 Jan 2012 21:53:11 -0700 (MST) (envelope-from ken) Date: Wed, 4 Jan 2012 21:53:11 -0700 From: "Kenneth D. Merry" To: current@FreeBSD.org, scsi@FreeBSD.org Message-ID: <20120105045311.GA40378@nargothrond.kdm.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2i Cc: Subject: CAM Target Layer available X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jan 2012 04:53:12 -0000 The CAM Target Layer (CTL) is now available for testing. I am planning to commit it to to head next week, barring any major objections. CTL is a disk and processor device emulation subsystem originally written for Copan Systems under Linux starting in 2003. It has been shipping in Copan (now SGI) products since 2005. It was ported to FreeBSD in 2008, and thanks to an agreement between SGI (who acquired Copan's assets in 2010) and Spectra Logic in 2010, CTL is available under a BSD-style license. The intent behind the agreement was that Spectra would work to get CTL into the FreeBSD tree. The patches are against FreeBSD/head as of SVN change 229516 and are located here: http://people.freebsd.org/~ken/ctl/ctl_diffs.20120104.4.txt.gz The code is not "perfect" (few pieces of software are), but is in good shape from a functional standpoint. My intent is to get it out there for other folks to use, and perhaps help with improvements. There are a few other CAM changes included with these diffs, some of which will be committed separately from CTL, some concurrently. This is a quick summary: - Fix a panic in the da(4) driver when a drive disappears on boot. - Fix locking in the CAM EDT traversal code. - Add an optional sysctl/tunable (disabled by default) to suppress "duplicate" devices. This most frequently shows up with dual ported SAS drives. - Add some very basic error injection into the da(4) driver. - Bump the length field in the SCSI INQUIRY CDB to 2 bytes to line up with more recent SCSI specs. CTL Features: ============ - Disk and processor device emulation. - Tagged queueing - SCSI task attribute support (ordered, head of queue, simple tags) - SCSI implicit command ordering support. (e.g. if a read follows a mode select, the read will be blocked until the mode select completes.) - Full task management support (abort, LUN reset, target reset, etc.) - Support for multiple ports - Support for multiple simultaneous initiators - Support for multiple simultaneous backing stores - Persistent reservation support - Mode sense/select support - Error injection support - High Availability support (1) - All I/O handled in-kernel, no userland context switch overhead. (1) HA Support is just an API stub, and needs much more to be fully functional. See the to-do list below. Configuring and Running CTL: =========================== - After applying the CTL patchset to your tree, build world and install it on your target system. - Add 'device ctl' to your kernel configuration file. - If you're running with a 8Gb or 4Gb Qlogic FC board, add 'options ISP_TARGET_MODE' to your kernel config file. 'device ispfw' or loading the ispfw module is also recommended. - Rebuild and install a new kernel. - Reboot with the new kernel. - To add a LUN with the RAM disk backend: ctladm create -b ramdisk -s 10485760000000000000 ctladm port -o on - You should now see the CTL disk LUN through camcontrol devlist: scbus6 on ctl2cam0 bus 0: at scbus6 target 1 lun 0 (da24,pass32) <> at scbus6 target -1 lun -1 () This is visible through the CTL CAM SIM. This allows using CTL without any physical hardware. You should be able to issue any normal SCSI commands to the device via the pass(4)/da(4) devices. If any target-capable HBAs are in the system (e.g. isp(4)), and have target mode enabled, you should now also be able to see the CTL LUNs via that target interface. Note that all CTL LUNs are presented to all frontends. There is no LUN masking, or separate, per-port configuration. - Note that the ramdisk backend is a "fake" ramdisk. That is, it is backed by a small amount of RAM that is used for all I/O requests. This is useful for performance testing, but not for any data integrity tests. - To add a LUN with the block/file backend: truncate -s +1T myfile ctladm create -b block -o file=myfile ctladm port -o on - You can also see a list of LUNs and their backends like this: # ctladm devlist LUN Backend Size (Blocks) BS Serial Number Device ID 0 block 2147483648 512 MYSERIAL 0 MYDEVID 0 1 block 2147483648 512 MYSERIAL 1 MYDEVID 1 2 block 2147483648 512 MYSERIAL 2 MYDEVID 2 3 block 2147483648 512 MYSERIAL 3 MYDEVID 3 4 block 2147483648 512 MYSERIAL 4 MYDEVID 4 5 block 2147483648 512 MYSERIAL 5 MYDEVID 5 6 block 2147483648 512 MYSERIAL 6 MYDEVID 6 7 block 2147483648 512 MYSERIAL 7 MYDEVID 7 8 block 2147483648 512 MYSERIAL 8 MYDEVID 8 9 block 2147483648 512 MYSERIAL 9 MYDEVID 9 10 block 2147483648 512 MYSERIAL 10 MYDEVID 10 11 block 2147483648 512 MYSERIAL 11 MYDEVID 11 - You can see the LUN type and backing store for block/file backend LUNs like this: # ctladm devlist -v LUN Backend Size (Blocks) BS Serial Number Device ID 0 block 2147483648 512 MYSERIAL 0 MYDEVID 0 lun_type=0 num_threads=14 file=testdisk0 1 block 2147483648 512 MYSERIAL 1 MYDEVID 1 lun_type=0 num_threads=14 file=testdisk1 2 block 2147483648 512 MYSERIAL 2 MYDEVID 2 lun_type=0 num_threads=14 file=testdisk2 3 block 2147483648 512 MYSERIAL 3 MYDEVID 3 lun_type=0 num_threads=14 file=testdisk3 4 block 2147483648 512 MYSERIAL 4 MYDEVID 4 lun_type=0 num_threads=14 file=testdisk4 5 block 2147483648 512 MYSERIAL 5 MYDEVID 5 lun_type=0 num_threads=14 file=testdisk5 6 block 2147483648 512 MYSERIAL 6 MYDEVID 6 lun_type=0 num_threads=14 file=testdisk6 7 block 2147483648 512 MYSERIAL 7 MYDEVID 7 lun_type=0 num_threads=14 file=testdisk7 8 block 2147483648 512 MYSERIAL 8 MYDEVID 8 lun_type=0 num_threads=14 file=testdisk8 9 block 2147483648 512 MYSERIAL 9 MYDEVID 9 lun_type=0 num_threads=14 file=testdisk9 10 ramdisk 0 0 MYSERIAL 0 MYDEVID 0 lun_type=3 11 ramdisk 204800000000000 512 MYSERIAL 1 MYDEVID 1 lun_type=0 - To see system throughput, use ctlstat(8): # ctlstat -t System Read System Write System Total ms KB/t tps MB/s ms KB/t tps MB/s ms KB/t tps MB/s 1.71 50.64 0 0.00 1.24 512.00 0 0.03 2.05 245.20 0 0.03 1.0% 0.00 0.00 0 0.00 1.12 512.00 564 282.00 1.12 512.00 564 282.00 8.4% 0.00 0.00 0 0.00 1.27 512.00 536 268.00 1.27 512.00 536 268.00 10.0% 0.00 0.00 0 0.00 1.27 512.00 535 267.50 1.27 512.00 535 267.50 7.6% 0.00 0.00 0 0.00 1.12 512.00 520 260.00 1.12 512.00 520 260.00 10.9% 0.00 0.00 0 0.00 1.02 512.00 538 269.00 1.02 512.00 538 269.00 10.9% 0.00 0.00 0 0.00 1.10 512.00 557 278.50 1.10 512.00 557 278.50 9.6% 0.00 0.00 0 0.00 1.12 512.00 561 280.50 1.12 512.00 561 280.50 10.4% 0.00 0.00 0 0.00 1.14 512.00 502 251.00 1.14 512.00 502 251.00 6.5% 0.00 0.00 0 0.00 1.31 512.00 527 263.50 1.31 512.00 527 263.50 10.5% 0.00 0.00 0 0.00 1.07 512.00 560 280.00 1.07 512.00 560 280.00 10.3% CTL To Do List: ============== - Use devstat(9) for CTL's statistics collection. CTL uses a home-grown statistics collection system that is similar to devstat(9). ctlstat should be retired in favor of iostat, etc., once aggregation modes are available in iostat to match the behavior of ctlstat -t and dump modes are available to match the behavior of ctlstat -d/ctlstat -J. - ZFS ARC backend for CTL. Since ZFS copies all I/O into the ARC (Adaptive Replacement Cache), running the block/file backend on top of a ZFS-backed zdev or file will involve an extra set of copies. The optimal solution for backing targets served by CTL with ZFS would be to allocate buffers out of the ARC directly, and DMA to/from them directly. That would eliminate an extra data buffer allocation and copy. - Switch CTL over to using CAM CCBs instead of its own union ctl_io. This will likely require a significant amount of work, but will eliminate another data structure in the stack, more memory allocations, etc. This will also require changes to the CAM CCB structure to support CTL. - Full-featured High Availability support. The HA API that is in ctl_ha.h is essentially a renamed version of Copan's HA API. There is no substance to it, but it remains in CTL to show what needs to be done to implement active/active HA from a CTL standpoint. The things that would need to be done include: - A kernel level software API for message passing as well as DMA between at least two nodes. - Hardware support and drivers for inter-node communication. This could be as simples as ethernet hardware and drivers. - A "supervisor", or startup framework to control and coordinate HA startup, failover (going from active/active to single mode), and failback (going from single mode to active/active). - HA support in other components of the stack. The goal behind HA is that one node can fail and another node can seamlessly take over handling I/O requests. This requires support from pretty much every component in the storage stack, from top to bottom. CTL is one piece of it, but you also need support in the RAID stack/filesystem/backing store. You also need full configuration mirroring, and all peer nodes need to be able to talk to the underlying storage hardware. Thanks, Ken -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-scsi@FreeBSD.ORG Thu Jan 5 04:51:03 2012 Return-Path: Delivered-To: scsi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4FAC5106564A; Thu, 5 Jan 2012 04:51:03 +0000 (UTC) (envelope-from ken@kdm.org) Received: from nargothrond.kdm.org (nargothrond.kdm.org [70.56.43.81]) by mx1.freebsd.org (Postfix) with ESMTP id CB0058FC16; Thu, 5 Jan 2012 04:50:59 +0000 (UTC) Received: from nargothrond.kdm.org (localhost [127.0.0.1]) by nargothrond.kdm.org (8.14.2/8.14.2) with ESMTP id q054dYnQ039136; Wed, 4 Jan 2012 21:39:34 -0700 (MST) (envelope-from ken@nargothrond.kdm.org) Received: (from ken@localhost) by nargothrond.kdm.org (8.14.2/8.14.2/Submit) id q054dYRP039135; Wed, 4 Jan 2012 21:39:34 -0700 (MST) (envelope-from ken) Date: Wed, 4 Jan 2012 21:39:34 -0700 From: "Kenneth D. Merry" To: current@FreeBSD.org, scsi@FreeBSD.org Message-ID: <20120105043934.GA37322@nargothrond.kdm.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="yrj/dFKFPuw6o+aM" Content-Disposition: inline User-Agent: Mutt/1.4.2i X-Mailman-Approved-At: Thu, 05 Jan 2012 05:00:58 +0000 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: CAM Target Layer available X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jan 2012 04:51:03 -0000 --yrj/dFKFPuw6o+aM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline The CAM Target Layer (CTL) is now available for testing. I am planning to commit it to to head next week, barring any major objections. CTL is a disk and processor device emulation subsystem originally written for Copan Systems under Linux starting in 2003. It has been shipping in Copan (now SGI) products since 2005. It was ported to FreeBSD in 2008, and thanks to an agreement between SGI (who acquired Copan's assets in 2010) and Spectra Logic in 2010, CTL is available under a BSD-style license. The intent behind the agreement was that Spectra would work to get CTL into the FreeBSD tree. The attached patches are against FreeBSD/head as of SVN change 229516. They are also located here: http://people.freebsd.org/~ken/ctl/ctl_diffs.20120104.4.txt.gz The code is not "perfect" (few pieces of software are), but is in good shape from a functional standpoint. My intent is to get it out there for other folks to use, and perhaps help with improvements. There are a few other CAM changes included with these diffs, some of which will be committed separately from CTL, some concurrently. This is a quick summary: - Fix a panic in the da(4) driver when a drive disappears on boot. - Fix locking in the CAM EDT traversal code. - Add an optional sysctl/tunable (disabled by default) to suppress "duplicate" devices. This most frequently shows up with dual ported SAS drives. - Add some very basic error injection into the da(4) driver. - Bump the length field in the SCSI INQUIRY CDB to 2 bytes to line up with more recent SCSI specs. CTL Features: ============ - Disk and processor device emulation. - Tagged queueing - SCSI task attribute support (ordered, head of queue, simple tags) - SCSI implicit command ordering support. (e.g. if a read follows a mode select, the read will be blocked until the mode select completes.) - Full task management support (abort, LUN reset, target reset, etc.) - Support for multiple ports - Support for multiple simultaneous initiators - Support for multiple simultaneous backing stores - Persistent reservation support - Mode sense/select support - Error injection support - High Availability support (1) - All I/O handled in-kernel, no userland context switch overhead. (1) HA Support is just an API stub, and needs much more to be fully functional. See the to-do list below. Configuring and Running CTL: =========================== - After applying the CTL patchset to your tree, build world and install it on your target system. - Add 'device ctl' to your kernel configuration file. - If you're running with a 8Gb or 4Gb Qlogic FC board, add 'options ISP_TARGET_MODE' to your kernel config file. 'device ispfw' or loading the ispfw module is also recommended. - Rebuild and install a new kernel. - Reboot with the new kernel. - To add a LUN with the RAM disk backend: ctladm create -b ramdisk -s 10485760000000000000 ctladm port -o on - You should now see the CTL disk LUN through camcontrol devlist: scbus6 on ctl2cam0 bus 0: at scbus6 target 1 lun 0 (da24,pass32) <> at scbus6 target -1 lun -1 () This is visible through the CTL CAM SIM. This allows using CTL without any physical hardware. You should be able to issue any normal SCSI commands to the device via the pass(4)/da(4) devices. If any target-capable HBAs are in the system (e.g. isp(4)), and have target mode enabled, you should now also be able to see the CTL LUNs via that target interface. Note that all CTL LUNs are presented to all frontends. There is no LUN masking, or separate, per-port configuration. - Note that the ramdisk backend is a "fake" ramdisk. That is, it is backed by a small amount of RAM that is used for all I/O requests. This is useful for performance testing, but not for any data integrity tests. - To add a LUN with the block/file backend: truncate -s +1T myfile ctladm create -b block -o file=myfile ctladm port -o on - You can also see a list of LUNs and their backends like this: # ctladm devlist LUN Backend Size (Blocks) BS Serial Number Device ID 0 block 2147483648 512 MYSERIAL 0 MYDEVID 0 1 block 2147483648 512 MYSERIAL 1 MYDEVID 1 2 block 2147483648 512 MYSERIAL 2 MYDEVID 2 3 block 2147483648 512 MYSERIAL 3 MYDEVID 3 4 block 2147483648 512 MYSERIAL 4 MYDEVID 4 5 block 2147483648 512 MYSERIAL 5 MYDEVID 5 6 block 2147483648 512 MYSERIAL 6 MYDEVID 6 7 block 2147483648 512 MYSERIAL 7 MYDEVID 7 8 block 2147483648 512 MYSERIAL 8 MYDEVID 8 9 block 2147483648 512 MYSERIAL 9 MYDEVID 9 10 block 2147483648 512 MYSERIAL 10 MYDEVID 10 11 block 2147483648 512 MYSERIAL 11 MYDEVID 11 - You can see the LUN type and backing store for block/file backend LUNs like this: # ctladm devlist -v LUN Backend Size (Blocks) BS Serial Number Device ID 0 block 2147483648 512 MYSERIAL 0 MYDEVID 0 lun_type=0 num_threads=14 file=testdisk0 1 block 2147483648 512 MYSERIAL 1 MYDEVID 1 lun_type=0 num_threads=14 file=testdisk1 2 block 2147483648 512 MYSERIAL 2 MYDEVID 2 lun_type=0 num_threads=14 file=testdisk2 3 block 2147483648 512 MYSERIAL 3 MYDEVID 3 lun_type=0 num_threads=14 file=testdisk3 4 block 2147483648 512 MYSERIAL 4 MYDEVID 4 lun_type=0 num_threads=14 file=testdisk4 5 block 2147483648 512 MYSERIAL 5 MYDEVID 5 lun_type=0 num_threads=14 file=testdisk5 6 block 2147483648 512 MYSERIAL 6 MYDEVID 6 lun_type=0 num_threads=14 file=testdisk6 7 block 2147483648 512 MYSERIAL 7 MYDEVID 7 lun_type=0 num_threads=14 file=testdisk7 8 block 2147483648 512 MYSERIAL 8 MYDEVID 8 lun_type=0 num_threads=14 file=testdisk8 9 block 2147483648 512 MYSERIAL 9 MYDEVID 9 lun_type=0 num_threads=14 file=testdisk9 10 ramdisk 0 0 MYSERIAL 0 MYDEVID 0 lun_type=3 11 ramdisk 204800000000000 512 MYSERIAL 1 MYDEVID 1 lun_type=0 - To see system throughput, use ctlstat(8): # ctlstat -t System Read System Write System Total ms KB/t tps MB/s ms KB/t tps MB/s ms KB/t tps MB/s 1.71 50.64 0 0.00 1.24 512.00 0 0.03 2.05 245.20 0 0.03 1.0% 0.00 0.00 0 0.00 1.12 512.00 564 282.00 1.12 512.00 564 282.00 8.4% 0.00 0.00 0 0.00 1.27 512.00 536 268.00 1.27 512.00 536 268.00 10.0% 0.00 0.00 0 0.00 1.27 512.00 535 267.50 1.27 512.00 535 267.50 7.6% 0.00 0.00 0 0.00 1.12 512.00 520 260.00 1.12 512.00 520 260.00 10.9% 0.00 0.00 0 0.00 1.02 512.00 538 269.00 1.02 512.00 538 269.00 10.9% 0.00 0.00 0 0.00 1.10 512.00 557 278.50 1.10 512.00 557 278.50 9.6% 0.00 0.00 0 0.00 1.12 512.00 561 280.50 1.12 512.00 561 280.50 10.4% 0.00 0.00 0 0.00 1.14 512.00 502 251.00 1.14 512.00 502 251.00 6.5% 0.00 0.00 0 0.00 1.31 512.00 527 263.50 1.31 512.00 527 263.50 10.5% 0.00 0.00 0 0.00 1.07 512.00 560 280.00 1.07 512.00 560 280.00 10.3% CTL To Do List: ============== - Use devstat(9) for CTL's statistics collection. CTL uses a home-grown statistics collection system that is similar to devstat(9). ctlstat should be retired in favor of iostat, etc., once aggregation modes are available in iostat to match the behavior of ctlstat -t and dump modes are available to match the behavior of ctlstat -d/ctlstat -J. - ZFS ARC backend for CTL. Since ZFS copies all I/O into the ARC (Adaptive Replacement Cache), running the block/file backend on top of a ZFS-backed zdev or file will involve an extra set of copies. The optimal solution for backing targets served by CTL with ZFS would be to allocate buffers out of the ARC directly, and DMA to/from them directly. That would eliminate an extra data buffer allocation and copy. - Switch CTL over to using CAM CCBs instead of its own union ctl_io. This will likely require a significant amount of work, but will eliminate another data structure in the stack, more memory allocations, etc. This will also require changes to the CAM CCB structure to support CTL. - Full-featured High Availability support. The HA API that is in ctl_ha.h is essentially a renamed version of Copan's HA API. There is no substance to it, but it remains in CTL to show what needs to be done to implement active/active HA from a CTL standpoint. The things that would need to be done include: - A kernel level software API for message passing as well as DMA between at least two nodes. - Hardware support and drivers for inter-node communication. This could be as simples as ethernet hardware and drivers. - A "supervisor", or startup framework to control and coordinate HA startup, failover (going from active/active to single mode), and failback (going from single mode to active/active). - HA support in other components of the stack. The goal behind HA is that one node can fail and another node can seamlessly take over handling I/O requests. This requires support from pretty much every component in the storage stack, from top to bottom. CTL is one piece of it, but you also need support in the RAID stack/filesystem/backing store. You also need full configuration mirroring, and all peer nodes need to be able to talk to the underlying storage hardware. Thanks, Ken -- Kenneth Merry ken@FreeBSD.ORG --yrj/dFKFPuw6o+aM-- From owner-freebsd-scsi@FreeBSD.ORG Thu Jan 5 09:28:13 2012 Return-Path: Delivered-To: scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7F22A106566C; Thu, 5 Jan 2012 09:28:13 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id 0C8D48FC14; Thu, 5 Jan 2012 09:28:12 +0000 (UTC) Received: from julian-mac.elischer.org (c-67-180-24-15.hsd1.ca.comcast.net [67.180.24.15]) (authenticated bits=0) by vps1.elischer.org (8.14.4/8.14.4) with ESMTP id q05938Qf099748 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 5 Jan 2012 01:03:10 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <4F05677D.7070602@freebsd.org> Date: Thu, 05 Jan 2012 01:03:57 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.25) Gecko/20111213 Thunderbird/3.1.17 MIME-Version: 1.0 To: "Kenneth D. Merry" References: <20120105043934.GA37322@nargothrond.kdm.org> In-Reply-To: <20120105043934.GA37322@nargothrond.kdm.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: current@freebsd.org, scsi@freebsd.org Subject: Re: CAM Target Layer available X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jan 2012 09:28:13 -0000 On 1/4/12 8:39 PM, Kenneth D. Merry wrote: > The CAM Target Layer (CTL) is now available for testing. I am planning to > commit it to to head next week, barring any major objections. > > CTL is a disk and processor device emulation subsystem originally written > for Copan Systems under Linux starting in 2003. It has been shipping in > Copan (now SGI) products since 2005. > > It was ported to FreeBSD in 2008, and thanks to an agreement between SGI > (who acquired Copan's assets in 2010) and Spectra Logic in 2010, CTL is > available under a BSD-style license. The intent behind the agreement was > that Spectra would work to get CTL into the FreeBSD tree. [...] > - Note that the ramdisk backend is a "fake" ramdisk. That is, it is > backed by a small amount of RAM that is used for all I/O requests. This > is useful for performance testing, but not for any data integrity tests. so, If I read this correctly, I should be able to export a fusion-io flash card with ctladm create -b block -o file=/dev/fio0 ?? > - To add a LUN with the block/file backend: > > truncate -s +1T myfile > ctladm create -b block -o file=myfile > ctladm port -o on > From owner-freebsd-scsi@FreeBSD.ORG Thu Jan 5 14:24:36 2012 Return-Path: Delivered-To: scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 24DCD106566C; Thu, 5 Jan 2012 14:24:36 +0000 (UTC) (envelope-from ken@kdm.org) Received: from nargothrond.kdm.org (nargothrond.kdm.org [70.56.43.81]) by mx1.freebsd.org (Postfix) with ESMTP id E0A798FC1A; Thu, 5 Jan 2012 14:24:35 +0000 (UTC) Received: from nargothrond.kdm.org (localhost [127.0.0.1]) by nargothrond.kdm.org (8.14.2/8.14.2) with ESMTP id q05EOZ5m075787; Thu, 5 Jan 2012 07:24:35 -0700 (MST) (envelope-from ken@nargothrond.kdm.org) Received: (from ken@localhost) by nargothrond.kdm.org (8.14.2/8.14.2/Submit) id q05EOZ4L075786; Thu, 5 Jan 2012 07:24:35 -0700 (MST) (envelope-from ken) Date: Thu, 5 Jan 2012 07:24:35 -0700 From: "Kenneth D. Merry" To: Julian Elischer Message-ID: <20120105142435.GA75178@nargothrond.kdm.org> References: <20120105043934.GA37322@nargothrond.kdm.org> <4F05677D.7070602@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4F05677D.7070602@freebsd.org> User-Agent: Mutt/1.4.2i Cc: current@freebsd.org, scsi@freebsd.org Subject: Re: CAM Target Layer available X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jan 2012 14:24:36 -0000 On Thu, Jan 05, 2012 at 01:03:57 -0800, Julian Elischer wrote: > On 1/4/12 8:39 PM, Kenneth D. Merry wrote: > >The CAM Target Layer (CTL) is now available for testing. I am planning to > >commit it to to head next week, barring any major objections. > > > >CTL is a disk and processor device emulation subsystem originally written > >for Copan Systems under Linux starting in 2003. It has been shipping in > >Copan (now SGI) products since 2005. > > > >It was ported to FreeBSD in 2008, and thanks to an agreement between SGI > >(who acquired Copan's assets in 2010) and Spectra Logic in 2010, CTL is > >available under a BSD-style license. The intent behind the agreement was > >that Spectra would work to get CTL into the FreeBSD tree. > [...] > > > - Note that the ramdisk backend is a "fake" ramdisk. That is, it is > > backed by a small amount of RAM that is used for all I/O requests. > > This > > is useful for performance testing, but not for any data integrity > > tests. > > so, If I read this correctly, I should be able to export a fusion-io > flash card with > ctladm create -b block -o file=/dev/fio0 ?? Yes, that would work. Although last time I tried using a block device, as opposed to a file, I ran into a GEOM panic that looked like it should be pretty easy to fix. If you run into that, you can get around it by disabling sending sync cache to the back end: ctladm realsync off > > - To add a LUN with the block/file backend: > > > > truncate -s +1T myfile > > ctladm create -b block -o file=myfile > > ctladm port -o on > > Ken -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-scsi@FreeBSD.ORG Thu Jan 5 15:22:40 2012 Return-Path: Delivered-To: scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4FD3C106564A for ; Thu, 5 Jan 2012 15:22:40 +0000 (UTC) (envelope-from Kashyap.Desai@lsi.com) Received: from na3sys009aog125.obsmtp.com (na3sys009aog125.obsmtp.com [74.125.149.153]) by mx1.freebsd.org (Postfix) with ESMTP id D49F28FC12 for ; Thu, 5 Jan 2012 15:22:39 +0000 (UTC) Received: from paledge01.lsi.com ([192.19.193.42]) (using TLSv1) by na3sys009aob125.postini.com ([74.125.148.12]) with SMTP ID DSNKTwXAP4vWpuDn0cYVQdbsU5RLFKsNeCuB@postini.com; Thu, 05 Jan 2012 07:22:39 PST Received: from PALHUB01.lsi.com (128.94.213.114) by PALEDGE01.lsi.com (192.19.193.42) with Microsoft SMTP Server (TLS) id 8.3.137.0; Thu, 5 Jan 2012 10:16:06 -0500 Received: from inbexch01.lsi.com (135.36.98.37) by PALHUB01.lsi.com (128.94.213.114) with Microsoft SMTP Server (TLS) id 8.3.106.1; Thu, 5 Jan 2012 10:11:42 -0500 Received: from inbmail01.lsi.com ([135.36.98.64]) by inbexch01.lsi.com ([135.36.98.37]) with mapi; Thu, 5 Jan 2012 20:41:39 +0530 From: "Desai, Kashyap" To: "scsi@freebsd.org" , "Kenneth D. Merry" Date: Thu, 5 Jan 2012 20:41:38 +0530 Thread-Topic: kldunload hangs on FreeBSD with camcontrol debug on Thread-Index: AczLvFLEJn+sZ/RgRbiyo3blVV3MEw== Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "gibbs@FreeBSD.org" , "McConnell, Stephen" Subject: kldunload hangs on FreeBSD with camcontrol debug on X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jan 2012 15:22:40 -0000 While doing some sanity test on "mps" driver, I found kldunload hangs in be= low condition. Case-1 1. Connect LSI 2008 controller(any SAS2 card). Load there mps driver and de= tect the controller. 2. unload driver/load driver in loop. There will not be any problem. Case-2=20 Run above Case-1 with below settings. "camcontorl debug -IT " A moment I enable CAM debug level -I and -T, kldunload always hangs on "sim= ->refcount" Here is relavent output for reference. Jan 5 06:42:08 dhcp-135-24-192-146 kernel: mpslsi30: mps_detach_sas:754 re= f count 2 Jan 5 06:42:08 dhcp-135-24-192-146 kernel: mpslsi30: path ID 6 Jan 5 06:42:08 dhcp-135-24-192-146 kernel: (noperiph:mpslsi30:0:-1:-1): xp= t_compile_path Jan 5 06:42:08 dhcp-135-24-192-146 kernel: (noperiph:mpslsi30:0:-1:-1): xp= t_async Jan 5 06:42:08 dhcp-135-24-192-146 kernel: (noperiph:mpslsi30:0:-1:-1): xp= t_async Jan 5 06:42:08 dhcp-135-24-192-146 kernel: (noperiph:mpslsi30:0:-1:-1): xp= t_release_path Jan 5 06:42:08 dhcp-135-24-192-146 kernel: mpslsi30: mps_detach_sas:757 re= f count 2 I have added few prints in detach routine (mps_detach_sas) as below. if (sassc->sim !=3D NULL) { mps_dprint(sc, MPS_INFO, "%s:%d ref count %d\n", __func__,__LINE__,= sassc->sim->refcount); mps_dprint(sc, MPS_INFO, "path ID %d\n",cam_sim_path(sassc->sim)); xpt_bus_deregister(cam_sim_path(sassc->sim)); mps_dprint(sc, MPS_INFO, "%s:%d ref count %d\n", __func__,__LINE__,= sassc->sim->refcount); cam_sim_free(sassc->sim, FALSE); } Before entering into " xpt_bus_deregister" sim->refcount is 2 and which nev= er goes to 0. And eventually Driver unloading hangs at Cam_sim_free() at below location.... error =3D msleep(sim, sim->mtx, PRIBIO, "simfree", 0); Any idea about this issue ? Regards, Kashyap