Verify the integrity of a flash/SD card on a Mac

August 21st, 2012 by

Last minute camera shopping

The week before going on a holiday, I realised my old Canon Ixus 40 camera would no longer live up to my expectations. So I quickly checked dpreview.com for the latest and greatest compact camera’s and decided I wanted the Canon PowerShot SX240 HS. 15 Minutes later I found myself in the Media Markt with the camera in my hand (unfortunately only the silver colour was available, I had hoped to buy the black one).

I need a bigger memory card

I then realised that my existing 512MB and 1GB SD cards would no longer be large enough for the 12Mpixel photos of this new camera. A few meters down the corridor was a huge wall covered in memory cards. I decided to buy a Samsung 8GB SDHC card, which would be able to hold up to 1500 photos or an hour of HD video. When I returned home, I formatted the card in-camera, and played around with the photo and video functions. Three days later my holiday would start and I would have a good (enough) camera to capture some fine moments!

Corrupted photos

Fast-forward about two weeks, half-way into my holiday. I’ve happily been taking photos and videos with my new camera. While I’m reviewing the last five photos, the camera suddenly displays a “broken picture” image on the screen. I take another photo, review it and it’s immediately “broken” again. I’m approximately on two-thirds of the card’s capacity and it is corrupting my photos. I do what I’ve learned from experience with corrupt memory cards: turn off the camera immediately, remove the SD card, and to be sure, “lock” the card (making it read-only) using the physical switch on the SD card. At this point, I just hope for the best that only those last two photos were corrupted.

Back at home a week-and-a-half later, it turns out I’m extremely lucky! Indeed only those last two pictures were corrupted!

Preventive detection of bad memory cards

How could I have detected on beforehand the memory card was corrupt? And how would I be able to determine the corrupt block(s) now? I just started google-ing for “verify integrity sd card”, expecting to find hundreds of results. Not so much really, in fact, I couldn’t find any fully automated test programs at all!

Windows based “h2testw”

Until I stumbled upon “h2testw“, written by Harald Bögeholz. It’s Windows only, but fortunately I have a VMWare Fusion hosted Windows 7 virtual machine. This software will write to all available free blocks on a volume and will then verify (read back) whether the files are still in the right shape. It does this byte-by-byte, so not a single bit will go untested (well, actually, the very last Megabyte might go untested, see the Readme).

Testing SD card using h2testw on Windows on VMWare Fusion

In VMWare Fusion, I connected my MacBook Pro’s “Apple Internal Memory Card Reader” to the virtual machine (this will temporarily disconnect the device from Mac OS X, so eject any card you have in the reader before attaching it to the virtual machine). Next, I inserted the suspicious memory card. Since h2testw will only test empty (free) space on a selected volume, I “quick” formatted the sd card before starting the verification. To my surprise, the software found no errors! I formatted again, reran the verification and again, no errors! Was my camera faulty?

I read a bit more about SD cards and it seems they can re-assign bad blocks (just like a regular hard drive), but only after you’ve tried to write to them. Had I fully written to the SD card immediately after buying it, the bad blocks would probably have been re-assigned.

Testing SD card using F3 on Mac OS X

So, I’d found a solution for Windows, but what about the Mac? Google-ing for “h2testw mac” pointed me (eventually) to the software “F3″ (either meaning “Fight Flash Fraud” or “Fight Fake Flash” according to its author Michel Machado). A page with a manual for F3 can be found here, while the source of F3 is on Github. F3 is an open-source implementation of the same algorithm h2testw employs to determine the integrity of a memory card (or any volume you mount on your Mac for that matter).

Compiling the source of F3

I first downloaded the source of the latest version of F3 from Github as a Zip file. Next, I double-clicked the Zip file in the Finder to extract the files. A folder “AltraMayor-f3-c24c098″ was created.
I opened Terminal and entered:

cd ~/Downloads/AltraMayor-f3-c24c098

The source code still needs to be compiled into a program. I think you’ll need XCode (since you need gcc to compile), but I’m not sure since I’ve had XCode installed since I don’t know when… Following the instructions on Github, the only thing I need to enter now is:

make mac

Zero-ing out the memory card

Before we run the tests, let’s completely erase the memory card. Open “Disk Utility” on your Mac with the SD card inserted. Select the disk (not the underlying volume), mine is called “8,07 APPLE SD Card Reader Media”. Go to the second tab “Erase”. Make sure the “Format” is “MS-DOS (FAT)”, the name doesn’t really matter (I left it at “UNTITLED”). Click on the “Security Options…” button, select “Zero Out Data” and click on “OK”. It now says “Erase option: Zero Out Data selected.” above the “Security Options…” and “Erase…” buttons. Next, click on the “Erase…” button and click “Erase” to confirm. Using Activity Monitor, I could see Disk Utility was writing zeroes to my memory card at 15MB/s. It took about 8 minutes to totally fill my 8GB memory card.
This is a good test to start with. If zero-ing the card yields errors, it certainly is corrupt.

Testing SD card using F3 on Mac OS X

There are a couple of new files in the folder now, most notably f3write and f3read, which are the actual programs we’ll need to continue testing the SD card.
First, let’s find out what the actual “volume” of the SD card is. In Terminal, enter:

cd /Volumes/
ls

Those are two separate commands, each line followed by hitting “enter/return”! This is what I got as a result:

$ cd /Volumes/
$ ls
Macintosh HD	UNTITLED

So I’ve got two mounted volumes “Macintosh HD” and “UNTITLED”. My best guess is I’m interested in the latter :)
Let’s go back to the F3 program folder:

cd ~/Downloads/AltraMayor-f3-c24c098/

And execute the f3write program first:

./f3write /Volumes/UNTITLED/

The F3 write program now first starts to write to all free/empty space and will report its progress. This is also a good moment to check the write speed of the card. Mine delivered a minimum rate of about 10MB/s. According to Wikipedia, this is “Class 10″. That’s also what is printed on my SD card, so lucky me!

Next, we’ll have to check whether the data is OK. To this avail, we need to run the “f3read” program. In Terminal, I entered:

./f3read /Volumes/UNTITLED/

You can monitor its progress for each xxxx.fff file the program finds (these .fff files were written by f3write). The manual page linked above explains the different columns (ok/corrupted/changed/overwritten).

Finishing up

So, that’s it! I finally have a program for Mac OS X to check the integrity of a SD card! From now on, the first thing I will do after buying a memory card is validating whether it isn’t corrupt, if only to re-assign potential bad blocks…

17 Responses to “Verify the integrity of a flash/SD card on a Mac”

  1. RC Fisher Says:

    I was able to change directories but I couldn’t compile the app and it wouldn’t run either. I get the following

    I checked and I also have X11 installed

    Ideas

  2. RC Fisher Says:

    Well I’ll try again

    robert-c-fishers-macbook-pro-7:f3v2_1 rcfisher$ make mac
    -bash: make: command not found

  3. Jonathan Says:

    X11 (Unix windowing app stuff) is not the same as Xcode (Apple’s compiler and development environment), although they’re both optional installations. Xcode is included with the install discs for older releases of OSX, and can be downloaded from https://developer.apple.com/xcode/ for newer ones.

    (Note: the headline version on the Apple site right now requires OSX 10.8. Older OS releases will require older Xcode versions.)

    F3 is now covered by the Homebrew project ( mxcl.github.com/homebrew/ ), which will make things a lot easier. You’ll still need to install Xcode as well as Homebrew itself, but then you just have to say “brew install f3″ to have it download and build the programs automagically.

  4. JV Says:

    It turns out that the “gcc” compiler required is no longer included in the Xcode install and also it’s a huge download. To avoid that, you can go to:

    https://developer.apple.com/downloads

    You’ll need to establish a free developer account. (This can be attached to your exisiting Apple ID, I believe).

    Then in the downloads searchbox, enter “command line tools.” Select the version of the command line tools for your Mac OS version (either Lion or Mountain Lion), download it, and then install the package. Then you will be able to execute the “make mac” command.

  5. GEARTALK: THE STORY OF THE NOT SO FAKE SD-CARD | ISMONO Blog Says:

    [...] the written files for corruption. On mac you have to do some Terminal command magic, so go over here for some detailed instructions. So in this test my cards could be filled up completely and all [...]

  6. Alan Says:

    Does f3 hold any advantage over the standard *nix “badblocks” , which can be found in “e2fsprogs” on Mac Homebrew for example? That tool can perform read-write checks as well.

  7. BroX Says:

    Is there a GUI of F3? Thanks.

  8. Thijs Kuipers Says:

    @BroX: I don’t think a GUI for F3 exists.

    When the “make” command doesn’t work (on your Mac), you’ll probably need the “Command Line Tools for XCode” available from https://developer.apple.com/downloads/ for your OS version (Lion, Mountain Lion, Mavericks).

  9. Jan van Haarst Says:

    I found a GUI for F3 (sort of) : https://github.com/flagsoft/sdspeed
    This points to http://www.flagsoft.com/cmswp/products/sdspeed-sd-card-memory-speed-test/

  10. BroX Says:

    Thanks, Jan van Haarst. I have installed the latest sdspeed 1.1 beta, yet it does not see the broken USB pendrive.

    To cut a long story short, the pendrive is
    LaCie XtremKey USB 3.0
    http://www.lacie.com/us/products/product.htm?id=10612

    It broke when automatically unmounted while copying a VMware Fusion disk image.

    Since I cannot repair it (Disk Utility, DiskWarrior, TechTool Pro, DiskTools Pro & SpeedTools Utilities Pro, and Drive Genius on Mac with OS X 10.9.1 Mavericks), I just want to format it, yet I cannot with any tool tested so far.

    Any idea how to repair or format it?
    Thanks.

  11. eyekey Says:

    Thank you for the detailed information. My PNY StoreEDGE seems to be built pretty bad. Got my Parallels File corrupted. I ordered a second one, hopefully this one will work, we´ll see.

    Is it really necessary to be Fat? Why would this not work with ExFat or OSX Journaled?

  12. newtothis Says:

    Hello there,

    I am testing my 64GB microSDXC card with this program. After filling my card with files for about three days, this is what happened next:

    Removing old file 1.h2w …
    Removing old file 2.h2w …
    Removing old file 3.h2w …
    Removing old file 4.h2w …
    Removing old file 5.h2w …
    Removing old file 6.h2w …
    Removing old file 7.h2w …
    Removing old file 8.h2w …
    Removing old file 9.h2w …
    Removing old file 10.h2w …
    Removing old file 11.h2w …
    Removing old file 12.h2w …
    Removing old file 13.h2w …
    Removing old file 14.h2w …
    Removing old file 15.h2w …
    Removing old file 16.h2w …
    Removing old file 17.h2w …
    Removing old file 18.h2w …
    Removing old file 19.h2w …
    Removing old file 20.h2w …
    Removing old file 21.h2w …
    Removing old file 22.h2w …
    Removing old file 23.h2w …
    Removing old file 24.h2w …
    Removing old file 25.h2w …
    Removing old file 26.h2w …
    Removing old file 27.h2w …
    Removing old file 28.h2w …
    Removing old file 29.h2w …
    Removing old file 30.h2w …
    Removing old file 31.h2w …
    Removing old file 32.h2w …
    Removing old file 33.h2w …
    Removing old file 34.h2w …
    Removing old file 35.h2w …
    Removing old file 36.h2w …
    Removing old file 37.h2w …
    Removing old file 38.h2w …
    Removing old file 39.h2w …
    Removing old file 40.h2w …
    Removing old file 41.h2w …
    Removing old file 42.h2w …
    Removing old file 43.h2w …
    Removing old file 44.h2w …
    Removing old file 45.h2w …
    Removing old file 46.h2w …
    Removing old file 47.h2w …
    Removing old file 48.h2w …
    Removing old file 49.h2w …
    Removing old file 50.h2w …
    Removing old file 51.h2w …
    Removing old file 52.h2w …
    Removing old file 53.h2w …
    Removing old file 54.h2w …
    Removing old file 55.h2w …
    Removing old file 56.h2w …
    Removing old file 57.h2w …
    Removing old file 58.h2w …
    Removing old file 59.h2w …
    Removing old file 60.h2w …
    Removing old file 61.h2w …
    Removing old file 62.h2w …
    Removing old file 63.h2w …
    Free space: 62.00 GB
    Creating file 1.h2w … OK!
    Creating file 2.h2w … OK!
    Creating file 3.h2w … OK!
    Creating file 4.h2w … OK!
    Creating file 5.h2w … OK!
    Creating file 6.h2w … OK!
    Creating file 7.h2w … OK!
    Creating file 8.h2w … OK!
    Creating file 9.h2w … OK!
    Creating file 10.h2w … OK!
    Creating file 11.h2w … OK!
    Creating file 12.h2w … OK!
    Creating file 13.h2w … OK!
    Creating file 14.h2w … OK!
    Creating file 15.h2w … OK!
    Creating file 16.h2w … 24.93% — 2.78 MB/s — 22:00:24

    and then it automatically began the whole process again. Is this suppose to happen?

  13. DAN IOAN MILU Says:

    there is other possiblity for this, for endusers? if i have to choose to do everithing you did or throwing away the card, i shurely throw away the card…
    simple as that: download – install – have a damn test – repairing the card id necessary – end

  14. me Says:

    Has anyone compiled a usable binary yet, or is there another Mac utility out there that works GUI?
    Or is this the only thing that is working at this time?
    If there’s nothing else, anyone know of a GUI program that will run under Ubuntu?
    Thanks

  15. Jon Says:

    I was able to get a usable binary on OS X using the homebrew package manager and running “brew install f3″. It installed the binary under /usr/local/Cellar/f3/2.0/bin. The homebrew package manager is available at http://brew.sh/. f3 2.0 might be older but it still works nicely.

  16. Jon Says:

    Small correction – I just updated my copy of homebrew and it now builds f3 3.0!

  17. Antonio Says:

    Hi. Thank you for the info. But could you please post a compiled version for the mac. It will be very appreciated. Thank you.

Leave a Reply