From owner-freebsd-hackers@freebsd.org Sat Jul 11 17:53:18 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AC44A99822E for ; Sat, 11 Jul 2015 17:53:18 +0000 (UTC) (envelope-from dieterbsd@gmail.com) Received: from mail-ig0-x235.google.com (mail-ig0-x235.google.com [IPv6:2607:f8b0:4001:c05::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7D07912F2 for ; Sat, 11 Jul 2015 17:53:18 +0000 (UTC) (envelope-from dieterbsd@gmail.com) Received: by igrv9 with SMTP id v9so33009527igr.1 for ; Sat, 11 Jul 2015 10:53:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=6859PY2vJs9S4ZpLRyq8HCWMmwanhZz6RY25fgL5Lvw=; b=F6Bdfyu38HiyaTdu1wNrpuJRvjmLZoZFxbHJVEy4prA9zpnPPUW1Tr+ygp+CIS2z86 E701Edb6a0maKdgUHmjJrs8eoX0x2uJfX/2lTaJKnqezSxooLUomJBYm2kuorlQuKqC0 ysk82ZLKk/NaZpXA6nSeqBE00yCd8hC+A7iPO8KVcruYAxJoTr6jurZqJBk3kKYaQhpG 2KQUdcElYW6MupHR8yiRfjLulTfv12+xpgungyd+gdD4rc9rAvPcpKVxU8XtRSP+DaBJ dyGMr665aGhpXpnPSt1Tl1VfJZLvMO15+vGbNIaTkWPlFJbjuZwYmK78Z1jTBKNrretx 5KUQ== MIME-Version: 1.0 X-Received: by 10.107.168.83 with SMTP id r80mr797159ioe.20.1436637197787; Sat, 11 Jul 2015 10:53:17 -0700 (PDT) Received: by 10.64.2.132 with HTTP; Sat, 11 Jul 2015 10:53:17 -0700 (PDT) Date: Sat, 11 Jul 2015 10:53:17 -0700 Message-ID: Subject: Re: format/newfs larger external consumer drives From: Dieter BSD To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Jul 2015 17:53:18 -0000 > When building a filesystem (FFS) on these 1/2/3/4TB external USB > drives, any suggestions? E.g., changing block sizes? Cut into > multiple slices to make fsck's job easier (as well as *recovery*)? If the average filesize will be large, use large block/frag sizes. I use 64 KiB / 8 KiB. And reduce the number of inodes. I reduce inodes as much as newfs allows and there are still way too many. These 2 changes waste less space to filesystem overhead, and fsck runs a lot faster. If the disk has 4 KiB sectors, have the frag size be at least 4 KiB, and have partitions start at a multiple of 4 KiB. I recommend soft updates. When I started using su I magically stopped losing files. Have newfs label the filesystem(s). Then use /dev/ufs/mylabel in /etc/fstab and the filesystems will get mounted on the right places regardless of the disk being da0 or da1 or whatever. Swap can be labeled with glabel(8). Multiple slices? Consider the size of whatever media you'll be using for backups. Consider making a read-only partition, as it will not require fscking after a crash. Consider gpt. If you decide on one big filesystem you don't need any partitioning, just newfs the raw drive. USB specific stuff: There is an off by 1 sector problem, which will bite you if you switch a drive between using the sata-usb bridge and connecting the drive directly to a sata controller. I had to do a fair bit of hacking on the kernel to deal with this. (No, I don't have a nice clean patch to offer, sorry. Besides, I just plastered over the problem rather than hunting down and fixing the root cause, which is what really needs to be done.) I don't recall if it is off by one physical sector, or one logical sector. On a 4 KiB drive it *might* mess up having things be on 4K multiples. Word is that the disk manufacturers have changed the interface between the bridge and the drive on recent external drives, making it difficult-to-impossible to use the drive connected directly to a sata controller. :-( Some people believe that drives are binned, and the less-wonderful drives go into the externals. This could explain externals being less expensive, despite having more stuff (enclosure, bridge, wall-wart, cables). Given how high the failure rate of internal drives is, I hate to think how bad the externals must be. I have yet to see a [ps]ata-to-usb bridge that allows turning the disk's write cache off. Having the write cache turned off is kinda important. If anyone knows of a bridge that does allow turning the write cache off I'd love to hear about it, > Of course, anything external means the user could insert/remove it at > will -- which complicates how the system will deal with the resource. If the drive disappears with filesystem(s) mounted. the kernel might very well panic. There was a discussion of this problem recently. I thought that FUSE was suggested as a possible solution, but I can't find the discussion. This problem is not limited to users disconnecting usb drives without unmounting them. The problem happens all by itself with internal drives, as the drive, port multiplier, controller, or device driver decides to go out to lunch, and the kernel panics. This happens *far* too often, and *kills* reliability. We really need a solution for this.