From owner-freebsd-questions@freebsd.org Mon Mar 16 18:38:07 2020 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C8B1326C6B5 for ; Mon, 16 Mar 2020 18:38:07 +0000 (UTC) (envelope-from freebsd@dreamchaser.org) Received: from nightmare.dreamchaser.org (ns.dreamchaser.org [66.109.141.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "dreamchaser.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48h4ms4rpFz3JTq for ; Mon, 16 Mar 2020 18:38:05 +0000 (UTC) (envelope-from freebsd@dreamchaser.org) Received: from breakaway.dreamchaser.org (breakaway [192.168.151.122]) by nightmare.dreamchaser.org (8.15.2/8.15.2) with ESMTP id 02GIbqoo015429; Mon, 16 Mar 2020 12:37:52 -0600 (MDT) (envelope-from freebsd@dreamchaser.org) Reply-To: freebsd@dreamchaser.org Subject: Re: SD card formatting To: Polytropon Cc: FreeBSD Mailing List References: <20200316000029.a6ba47c9.freebsd@edvax.de> <90f83226-b557-98c4-749b-81d9dbd795c3@dreamchaser.org> <20200316090538.27b1ecb1.freebsd@edvax.de> From: Gary Aitken Message-ID: <3b552229-fe07-941b-2000-56036f15c0a8@dreamchaser.org> Date: Mon, 16 Mar 2020 12:36:25 -0600 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.6.0 MIME-Version: 1.0 In-Reply-To: <20200316090538.27b1ecb1.freebsd@edvax.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (nightmare.dreamchaser.org [192.168.151.101]); Mon, 16 Mar 2020 12:37:53 -0600 (MDT) X-Rspamd-Queue-Id: 48h4ms4rpFz3JTq X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd@dreamchaser.org designates 66.109.141.57 as permitted sender) smtp.mailfrom=freebsd@dreamchaser.org X-Spamd-Result: default: False [-5.51 / 15.00]; ARC_NA(0.00)[]; HAS_REPLYTO(0.00)[freebsd@dreamchaser.org]; NEURAL_HAM_MEDIUM(-0.99)[-0.988,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[dreamchaser.org]; REPLYTO_ADDR_EQ_FROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(-3.22)[ip: (-8.45), ipnet: 66.109.128.0/19(-4.23), asn: 21947(-3.38), country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:21947, ipnet:66.109.128.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2020 18:38:07 -0000 On 3/16/20 2:05 AM, Polytropon wrote: > On Sun, 15 Mar 2020 19:06:22 -0600, Gary Aitken wrote: >> On 3/15/20 5:00 PM, Polytropon wrote: >>> On Sun, 15 Mar 2020 15:39:33 -0600, Gary Aitken wrote: >>>> 11.3-RELEASE-p6 GENERIC amd64 >>>> >>>> I'm having trouble reading SD cards formatted in my camera (Olympus >>>> EM1-MkII) or on a Win 7 system. >>> >>> What filesystem is in use? For cards, it's typically FAT. >>> >>> >>> >>>> When attempting to mount, I get the following: >>>> >>>> $ mount -t msdosfs /dev/da0s1 /mnt/memstick mount_msdosfs: >>>> /dev/da0s1: Invalid argument $ mount -t ntfs /dev/da0s1 >>>> /mnt/memstick mount: /dev/da0s1: Operation not supported by device >>>> >>>> Cards used without formatting *usually* seem to work. >>> >>> So those are preformatted (sometimes sold this way, sometimes >>> initialized when first used in a camera). >>> >>> >>> >>>> If I look at the cards which don't mount using gpart, I see: >>>> >>>> Card formatted in camera: >>>> $ gpart show -p /dev/da0 >>>> => 63 120944577 da0 MBR (58G) >>>> 63 32705 - free - (16M) >>>> 32768 120911872 da0s1 ntfs [active] (58G) >>> ^^^^ >>> >>> It seems that the card has been accidentally formatted with NTFS. In >>> most cases, cameras cannot use that. If you want to mount it on >>> FreeBSD, use ntfs-3g. >> >> As stated above, this is a card formatted _in the camera_. So the >> camera can read it fine. The problem is fbsd can't mount it. > >>From the program output presented, it seems that you're > trying to mount a NTFS partition using the FAT mount option, > which will not work. Try ntfs-3g (needs to be installed; > mount_ntfs is no longer part of the OS). Ah, that's a help, thanks. Ugh: # ntfs-3g /dev/da0s1 /mnt/memstick NTFS signature is missing. # ntfs-3g.probe --readonly /dev/da0s1 NTFS signature is missing. # file -r - >> Personally, I tend to leave the formatting to the camera which I want >>> to use the card in; the camera "knows best" what it can uderstand. >>> :-) >> >> In this case, the camera is not the problem; cards formatted in the camera >> are usable by the camera. But they aren't mountable on fbsd. So I can't >> transfer the files to fbsd, either from the camera or by inserting the >> card in an fbsd usb adapter. > > Depending on the camera, a USB connection can show up as > a mass storage device (leads to /dev/da) or a > MTP interface (leads to /dev/ugen). In many cases, > the "personality" can be seleted by the menu in the camera. It shows up as mass storage, /dev/da0 in this case >> Since cards formatted by the camera are not mountable on fbsd, I was >> hoping to find a way to format the card on fbsd that would be >> readable both by the camera and by fbsd. > > The camera seems to prefer NTFS, so when you (correctly) > format a card as FAT, the camera refuses to read it. However, > that's the first occassion I hear about a camera that seems > to use NTFS natively... not that this is impossible, it's > just something I never heared about it. :-) > > > >>> There is a simple reason: Several variants of FAT exist, and some >>> cameras often only support one out of that bag. If you guess (and >>> use) the wrong one, the camera will report an error. So, as I >>> suggested, let the camera do the right thing. >>> >>>> If I again format the card in the camera, I can't mount it. >>> >>> Have you checked _how_ the camera formats it? Check with gpart (or >>> fdisk, it still works!). There's also the following totally unelegant >>> way: >>> >>> # file -r - < /dev/da0 >>> >>> and / or >>> >>> # file -r - < /dev/da0s1 >>> >>> You should see a FAT summary. >> >> For a card formatted in the camera, which the camera can read but fbsd >> cannot mount: >> >> # file -r - < /dev/da0 >> /dev/stdin: DOS/MBR boot sector; partition 1 : ID=0x7, active, >> start-CHS (0x2,10,9), end-CHS (0x3ff,254,63), startsector 32768, >> 120911872 sectors >> # file -r - < /dev/da0s1 >> /dev/stdin: DOS/MBR boot sector >> # mount_msdosfs /dev/da0s1 /mnt/memstick >> mount_msdosfs: /dev/da0s1: Invalid argument > > According to the gpart output, it's not FAT (msdosfs), > but NTFS. > > > >> For a card never reformatted (i.e. formatted by the manufacturer, and >> used in the camera); this card is usable by the camera and mountable >> by fbsd: >> >> # file -r - < /dev/da0 >> /dev/stdin: DOS/MBR boot sector; partition 1 : ID=0xc, active, >> start-CHS (0x0,130,3), end-CHS (0x3ff,254,63), startsector 8192, >> 31496192 sectors >> # file -r - < /dev/da0s1 >> /dev/stdin: , code offset 0x0+3, OEM-ID " ", sectors/cluster 64, >> reserved sectors 504, Media descriptor 0xf8, sectors/track 63, >> heads 255, hidden sectors 8192, sectors 31496192 (volumes > 32 MB), >> FAT (32 bit), sectors/FAT 3844, serial number 0x0, unlabeled >> # mount_msdosfs /dev/da0s1 /mnt/memstick >> # ls /mnt/memstick >> DCIM System Volume Information > > This doesn't actually look like a "mint condition" card. A > directory named "DCIM" is typically created by a camera. > However, _this_ card is FAT-formatted, and therefore can be > mounted with msdosfs as correctly shown above. The directory > name "System Volume Information" suggests it has been in a > "Windows" PC before. COnclusion: Card works both in camera > an in "Windows" PC. Correct, it's not an unused card. It's been put in the camera, recorded on, and then put in a windows system to transfer images. > So the camera seems (!) to initialize cards as NTFS (strange, > but surely possible), but also understands FAT. That seems to be the case. Since ntfs-3g doesn't seem to like the camera-generated NTFS filesystem, it seems my best option is to try to format with the FAT it understands. > All you need to do now is to replicate _that_ formatting. > The output shows "FAT (32 bit)", and the partitioning also > looks reasonable. You should be able to create this either > with gpart + newfs_msdosfs (should be quite standard today), > or the traditional way with fdisk + newfs_msdosfs (archaic, > but should also still work). Tried the following: # gpart create -s MBR /dev/da0 da0 created # gpart show -l da0 => 63 120944577 da0 MBR (58G) 63 120944577 - free - (58G) # gpart add -t fat32 -a 4K /dev/da0 da0s1 added # gpart show -l /dev/da0 => 63 120944577 da0 MBR (58G) 63 1 - free - (512B) 64 120944576 1 (null) (58G) # newfs_msdos -F 32 -A /dev/da0s1 /dev/da0s1: 120885440 sectors in 1888835 FAT32 clusters (32768 bytes/cluster) BytesPerSec=512 SecPerClust=64 ResSectors=46 FATs=2 Media=0xf0 SecPerTrack=63 Heads=255 HiddenSecs=0 HugeSectors=120944576 FATsecs=14761 RootCluster=2 FSInfo=1 Backup=2 # mount -t msdosfs /dev/da0s1 /mnt/memstick mount_msdosfs: /dev/da0s1: Invalid argument Obviously I'm missing something? >>>> I need to reformat these cards so I can use them in the camera and >>>> on freebsd, and apparently neither the camera nor win7 does that, >>>> so... What's the right way to lay out, format, and create a file >>>> system on an SD card so it is usable by cameras and windoze? >>> >>> Typically using the _camera_ to do this job is the right thing. It's >>> often nice to have the card in "mint condition" (i. e., nothing on >>> it, no partitions, no filesystems). Using dd to overwrite the first >>> MB should be enough. On a "Windows" PC, the card should then >>> correctly be recognized as FAT. And on FreeBSD, you can easily mount >>> it with the -t msdos type. Just make sure that you use /dev/da0 or >>> /dev/da0s1 (some cameras are tricky and initialize the card as >>> /dev/da0 instead of /dev/da0s1, I never understood why). >> >> The problem is, on fbsd I _can't_ mount it. >> >> # dd if=/dev/zero of=/dev/da0 bs=1K count=1K >> 1024+0 records in >> 1024+0 records out >> 1048576 bytes transferred in 0.507045 secs (2068012 bytes/sec) >> >> put card in camera and format it >> >> # gpart show -r /dev/da0 >> => 63 120944577 da0 MBR (58G) >> 63 32705 - free - (16M) >> 32768 120911872 1 7 [active] (58G) >> # mount -t msdosfs /dev/da0s1 /mnt/memstick >> mount_msdosfs: /dev/da0s1: Invalid argument >> >> The card is readable by windows, however. > > It would be nice to see if _this_ card now is FAT or NTFS. > If it's NTFS, "Windows" will of course be able to use it. > On FreeBSD; you need to use nrfs_3g to mount it, though. :-) As shown earlier, ntfs-3g won't mount it, even though it's type 7 (ntfs) Thanks for your perseverance on helping explain all this... Gary