Last minute camera shopping
The week before going on a holiday, I realised my old Canon Ixus 40 camera would no longer live up to my expectations. So I quickly checked dpreview.com for the latest and greatest compact camera’s and decided I wanted the Canon PowerShot SX240 HS. 15 Minutes later I found myself in the Media Markt with the camera in my hand (unfortunately only the silver colour was available, I had hoped to buy the black one).
I need a bigger memory card
I then realised that my existing 512MB and 1GB SD cards would no longer be large enough for the 12Mpixel photos of this new camera. A few meters down the corridor was a huge wall covered in memory cards. I decided to buy a Samsung 8GB SDHC card, which would be able to hold up to 1500 photos or an hour of HD video. When I returned home, I formatted the card in-camera, and played around with the photo and video functions. Three days later my holiday would start and I would have a good (enough) camera to capture some fine moments!
Fast-forward about two weeks, half-way into my holiday. I’ve happily been taking photos and videos with my new camera. While I’m reviewing the last five photos, the camera suddenly displays a “broken picture” image on the screen. I take another photo, review it and it’s immediately “broken” again. I’m approximately on two-thirds of the card’s capacity and it is corrupting my photos. I do what I’ve learned from experience with corrupt memory cards: turn off the camera immediately, remove the SD card, and to be sure, “lock” the card (making it read-only) using the physical switch on the SD card. At this point, I just hope for the best that only those last two photos were corrupted.
Back at home a week-and-a-half later, it turns out I’m extremely lucky! Indeed only those last two pictures were corrupted!
Preventive detection of bad memory cards
How could I have detected on beforehand the memory card was corrupt? And how would I be able to determine the corrupt block(s) now? I just started google-ing for “verify integrity sd card”, expecting to find hundreds of results. Not so much really, in fact, I couldn’t find any fully automated test programs at all!
Windows based “h2testw”
Until I stumbled upon “h2testw“, written by Harald Bögeholz. It’s Windows only, but fortunately I have a VMWare Fusion hosted Windows 7 virtual machine. This software will write to all available free blocks on a volume and will then verify (read back) whether the files are still in the right shape. It does this byte-by-byte, so not a single bit will go untested (well, actually, the very last Megabyte might go untested, see the Readme).
Testing SD card using h2testw on Windows on VMWare Fusion
In VMWare Fusion, I connected my MacBook Pro’s “Apple Internal Memory Card Reader” to the virtual machine (this will temporarily disconnect the device from Mac OS X, so eject any card you have in the reader before attaching it to the virtual machine). Next, I inserted the suspicious memory card. Since h2testw will only test empty (free) space on a selected volume, I “quick” formatted the sd card before starting the verification. To my surprise, the software found no errors! I formatted again, reran the verification and again, no errors! Was my camera faulty?
I read a bit more about SD cards and it seems they can re-assign bad blocks (just like a regular hard drive), but only after you’ve tried to write to them. Had I fully written to the SD card immediately after buying it, the bad blocks would probably have been re-assigned.
Testing SD card using F3 on Mac OS X
So, I’d found a solution for Windows, but what about the Mac? Google-ing for “h2testw mac” pointed me (eventually) to the software “F3” (either meaning “Fight Flash Fraud” or “Fight Fake Flash” according to its author Michel Machado). A page with a manual for F3 can be found here, while the source of F3 is on Github. F3 is an open-source implementation of the same algorithm h2testw employs to determine the integrity of a memory card (or any volume you mount on your Mac for that matter).
Compiling the source of F3
[Edit 30 March 2018]
Save yourself the trouble of downloading and compiling source code, use
brew downloaded from https://brew.sh, after which you can simply
brew install f3. After you’ve done that,
f3write will be symlinked in
/usr/local/bin (and probably in your PATH) and readily available from the Terminal.
I first downloaded the source of the latest version of F3 from Github as a Zip file. Next, I double-clicked the Zip file in the Finder to extract the files. A folder “AltraMayor-f3-c24c098” was created.
I opened Terminal and entered:
The source code still needs to be compiled into a program. I think you’ll need XCode (since you need gcc to compile), but I’m not sure since I’ve had XCode installed since I don’t know when… Following the instructions on Github, the only thing I need to enter now is:
Zero-ing out the memory card
Before we run the tests, let’s completely erase the memory card. Open “Disk Utility” on your Mac with the SD card inserted. Select the disk (not the underlying volume), mine is called “8,07 APPLE SD Card Reader Media”. Go to the second tab “Erase”. Make sure the “Format” is “MS-DOS (FAT)”, the name doesn’t really matter (I left it at “UNTITLED”). Click on the “Security Options…” button, select “Zero Out Data” and click on “OK”. It now says “Erase option: Zero Out Data selected.” above the “Security Options…” and “Erase…” buttons. Next, click on the “Erase…” button and click “Erase” to confirm. Using Activity Monitor, I could see Disk Utility was writing zeroes to my memory card at 15MB/s. It took about 8 minutes to totally fill my 8GB memory card.
This is a good test to start with. If zero-ing the card yields errors, it certainly is corrupt.
Testing SD card using F3 on Mac OS X
There are a couple of new files in the folder now, most notably f3write and f3read, which are the actual programs we’ll need to continue testing the SD card.
First, let’s find out what the actual “volume” of the SD card is. In Terminal, enter:
cd /Volumes/ ls
Those are two separate commands, each line followed by hitting “enter/return”! This is what I got as a result:
$ cd /Volumes/ $ ls Macintosh HD UNTITLED
So I’ve got two mounted volumes “Macintosh HD” and “UNTITLED”. My best guess is I’m interested in the latter 🙂
Let’s go back to the F3 program folder:
And execute the f3write program first:
The F3 write program now first starts to write to all free/empty space and will report its progress. This is also a good moment to check the write speed of the card. Mine delivered a minimum rate of about 10MB/s. According to Wikipedia, this is “Class 10”. That’s also what is printed on my SD card, so lucky me!
Next, we’ll have to check whether the data is OK. To this avail, we need to run the “f3read” program. In Terminal, I entered:
You can monitor its progress for each xxxx.fff file the program finds (these .fff files were written by f3write). The manual page linked above explains the different columns (ok/corrupted/changed/overwritten).
So, that’s it! I finally have a program for Mac OS X to check the integrity of a SD card! From now on, the first thing I will do after buying a memory card is validating whether it isn’t corrupt, if only to re-assign potential bad blocks…
61 thoughts on “Verify the integrity of a flash/SD card on a Mac”
@JJ speed depends on the speed of the card and/or the card reader. 80MB/s (megabyte per second) is 640Mb/s (megabit per second) which is faster than USB2 theoretical speed (480Mb/s), so you need a pretty fast USB3 card reader to obtain such (advertised) read speeds.
Thanks for replying 🙂 I ran the test on my 2010 MacBook so I'll have to figure out how fast it can read. Thanks again.
What an utterly miserable sequence of events.
How does anyone expect to 'fight' fake flash cards with such a horrific user experience? Almost no-one will bother.
Is a DMG really that much effort?
Installing homebrew on mac is not terribly difficult and unlocks a whole new world of awesome. If you want a .app then go write one 😉
Instead of formatting to FAT, I would check the existing format and format to whatever it is. For large cards, FAT will not work. You need FAT32.
For big (micro)sdcards, you will usually use EXFAT rather than FAT.
Also, on MacOS Catalina (can't vouch for previous versions of the OS), Disk utility's erase advanced security settings feature a slider. The equivalent setting to zeroing the card is the second "fastest" option from the left, which does a first pass of random data then a second pass of zeros.
Finally, f3 is really a "high level" test (it writes files to the filesystem on the card). What would really be better is a utility which would write and read at the memory cell/block level without any potential intermediate caching messing with error detection and also able to blacklist bad memory blocks (SSD have a map of the memory cells and are able to track individual age and blacklist dead ones, but are SDcards able to too?)
I was really pleased to discover this but it soon turns into code gobbledy gook so useless to me
@Alan he gives step by step instructions. Use your brain instead of giving up so easily.
You can install f3 with homebrew!
F3XSwift downloadable version here (OSX);
The last post on this page by robgemdreau links to the F3 site directly to a GUI version of F3 that runs on OSX. I’m running it right now to test a very cheap card that is actually performing above specs for the 4K rating (Gladstone pack of 10, found on Amazon). Source page: https://forums.macrumors.com/threads/hi-everyone-is-there-a-way-to-test-a-fake-sd-card-on-macbook-pro.2226549/
Sorry is Gigastone, not Gladstone