From owner-freebsd-scsi@FreeBSD.ORG Thu Dec 30 16:48:38 2004 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EAB3416A4CE for ; Thu, 30 Dec 2004 16:48:37 +0000 (GMT) Received: from smtp2.enst.fr (revol1.enst.fr [137.194.32.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2ABE243D3F for ; Thu, 30 Dec 2004 16:48:37 +0000 (GMT) (envelope-from beyssac@bofh.enst.fr) Received: from localhost (localhost.enst.fr [127.0.0.1]) by smtp2.enst.fr (Postfix) with ESMTP id C30C5164993 for ; Thu, 30 Dec 2004 17:48:33 +0100 (CET) Received: from smtp2.enst.fr ([127.0.0.1]) by localhost (revol1.enst.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 45944-02 for ; Thu, 30 Dec 2004 17:48:22 +0100 (CET) Received: from bofh.enst.fr (bofh.enst.fr [137.194.32.191]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "bofh.enst.fr", Issuer "ENST CA" (not verified)) by smtp2.enst.fr (Postfix) with ESMTP id BCCD8164950 for ; Thu, 30 Dec 2004 17:48:22 +0100 (CET) Received: from bofh.enst.fr (localhost [127.0.0.1]) by bofh.enst.fr (8.13.1/8.13.1) with ESMTP id iBUGmMEU030221 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 30 Dec 2004 17:48:22 +0100 (CET) (envelope-from beyssac@bofh.enst.fr) Received: (from beyssac@localhost) by bofh.enst.fr (8.13.1/8.13.1/Submit) id iBUGmM6C030220 for freebsd-scsi@freebsd.org; Thu, 30 Dec 2004 17:48:22 +0100 (CET) (envelope-from beyssac) Date: Thu, 30 Dec 2004 17:48:22 +0100 From: Pierre Beyssac To: freebsd-scsi@freebsd.org Message-ID: <20041230164822.GA29784@bofh.enst.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.1i X-message-flag: Warning! Use of Microsoft Outlook makes your system susceptible to worms and viruses X-Virus-Scanned: amavisd-new at enst.fr Subject: unable to access disk block number > 2^32 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Dec 2004 16:48:38 -0000 Hello, I've acquired a 5-Terabyte SCSI RAID disk that I seem to be unable to use satisfactorily under FreeBSD. It's most likely a FreeBSD problem because a Linux 2.6/Knoppix CD on this machine is able to access the disk nicely. To make a long story short, any access to a disk block number over 2^32 returns 0 bytes without any error logged anywhere or returned to userland. Apparently the read request is correctly routed through by the kernel to the CAM layer, I checked this using a printf in /sys/cam/scsi/scsi_all.c in the READ_16 request code. Any ideas where I should look then? An obvious candidate would be the Symbios driver but I don't see how it could possibly process a READ_16 request differently from a READ_12 request. Note: I'm looking for a fix, not for a workaround such as defining several LUNs under 2TB each and gconcat'ing them. sym0: <1010-66> port 0x8000-0x80ff mem 0xfc200000-0xfc201fff,0xfc202000-0xfc2023ff irq 48 at device 1.0 on pci3 sym0: Symbios NVRAM, ID 7, Fast-80, LVD, parity checking sym0: open drain IRQ line driver, using on-chip SRAM sym0: using LOAD/STORE-based firmware. sym0: handling phase mismatch from SCRIPTS. sym0: [GIANT-LOCKED] da3 at sym0 bus 0 target 9 lun 0 da3: Fixed Direct Access SCSI-3 device da3: 160.000MB/s transfers (80.000MHz, offset 62, 16bit), Tagged Queueing Enabled da3: 5338102MB (10932432896 512 byte sectors: 255H 63S/T 680512C) # dd if=/dev/da3 skip=9000000000 count=100 of=/dev/null 0+0 records in 0+0 records out 0 bytes transferred in 0.000898 secs (0 bytes/sec) # dd if=/dev/da3 skip=5000000000 count=100 of=/dev/null 0+0 records in 0+0 records out 0 bytes transferred in 0.000620 secs (0 bytes/sec) # dd if=/dev/da3 skip=2000000000 count=100 of=/dev/null 100+0 records in 100+0 records out 51200 bytes transferred in 0.044276 secs (1156383 bytes/sec) The only thing that appears in the logs is the following, which seems mostly harmless (it appears even on blocks < 2^32 and does not prevent reading/writing to them). (da3:sym0:0:9:0): phase change 2-7 16@607ad968 resid=10. (da3:sym0:0:9:0): phase change 2-7 16@607ad968 resid=10. (da3:sym0:0:9:0): phase change 2-7 16@0035b368 resid=10. (da3:sym0:0:9:0): phase change 2-7 16@0035b368 resid=10. (da3:sym0:0:9:0): phase change 2-7 16@60c2fb68 resid=10. (da3:sym0:0:9:0): phase change 2-7 16@60c2fb68 resid=10. (da3:sym0:0:9:0): phase change 2-7 16@002eb968 resid=10. (da3:sym0:0:9:0): phase change 2-7 16@002eb968 resid=10. (da3:sym0:0:9:0): phase change 2-7 16@002eb968 resid=10. Under Linux the disk works like a charm (but only Reiserfs is able to newfs it): root@ttyp1[knoppix]# dd if=/dev/sdd skip=9000000000 of=/dev/null count=1000 1000+0 records in 1000+0 records out 512000 bytes transferred in 0.006852 seconds (74721797 bytes/sec) root@ttyp1[tmp]# mount -t reiserfs /dev/sdd /mnt root@ttyp1[tmp]# df -k Filesystem 1K-blocks Used Available Use% Mounted on /dev/root 3471 948 2523 28% / /dev/scd0 715764 715764 0 100% /cdrom /dev/cloop 1947338 1947338 0 100% /KNOPPIX /ramdisk 1644620 2396 1642224 1% /ramdisk /dev/sdd 5466049628 32840 5466016788 1% /mnt root@ttyp1[tmp]# -- A: Yes. Pierre Beyssac pb@enst.fr >Q: Are you sure? >>A: Because it reverses the logical flow of conversation. >>>Q: Why is top posting annoying in email?