data recovery basics
March 6, 1997

want to recover a dead floppy disk? Try this..

Scenario: your drive spins up OK but you can't get to the data. You get messages like Invalid Media Type, Invalid Drive Specification, Invalid Partition Table, Sector Not Found, masses of lost clusters, and/or cross-linked files.

If this is you, and you have Norton Utilities' Disk Editor 9.0, you may be able to recover your data. Notably, every attempt by Norton Disk Doctor to fix the problems below did NOT work. NDD is good for minor corruption.. Disk Editor is for the hardcore stuff.

Whenever data recovery is concerned, it is important to minimise writes to the drive containing the data to be recovered. Writing to the disk might nuke the stuff you want to recover. So don't copy the recovery programs to the disk - run them from floppy. Don't boot Windows, don't create undo files on the drive, and don't create "temporary test" directories in which to experiment during the recovery. Always run from a floppy, and only write to the drive when you have to.

If things are really bad, minimise the combinations of problems you have to deal with. Do this by killing off other running programs (including unneccesary device drivers), pull out any unnecessary hardware devices, reset the BIOS to defaults and refresh BIOS drive autodetection, check the remaining hardware, scan for viruses, and generally minimise the chances of getting messed around by some exotic combination of problems.

If you are using new hardware, you probably have a configuration problem. If you have older hardware, ensure it's not a hardware fault by swapping, removing or reconfiguring each component. You can check the hard drive subsystem by refreshing the BIOS autodetection and running a non-destructive surface test. If the autodetection does not work, you have a hardware problem. If the surface test finds problems, the hardware may be sick, but you may still be able to recover data. After you have attempted data recovery, try formatting the drive: if it has a large number of bad sectors, it may have failed and should not be trusted, while if it has a few or no bad sectors, something nuked the drive, but it survived. If the hardware is known good, the important bits on the drive are probably damaged.

Next, isolate the problem.

If you can boot from the drive and you can see an intact root directory, your master boot record (MBR) and partition table are OK, and your file allocation tables (FATs) are functional, if not OK. There's no disk problem.. run SCANDISK and DEFRAG and relax.

If you can't boot from the drive but can see an intact root directory on the drive once you boot from a floppy, your partition table is OK but your MBR is probably damaged. You can repair this with Norton Disk Doctor.

If you can't boot from the drive, yet you can see an intact root directory, and the MBR is fine, the copy of the operating system on the drive may be corrupted. To repair this, first ensure there are no other disk problems (run Disk Doctor, or ScanDisk), copy the operating system onto the drive using the SYS C: command, and reboot.

If you can't boot from the drive, and see a jumbled root directory, your FAT is probably damaged. See below.

If you can't boot from the drive, can see a drive letter, but cannot see the root directory ("Invalid Media Type"), you have a damaged MBR. Fix this with Norton Disk Doctor.

If you can't boot from the drive, can see a drive letter, but cannot see the root directory (" Sector Not Found"), you have a damaged FAT. See below.

If you can't boot from the drive, and cannot see a drive letter ("Invalid Drive Specification"), you probably have a damaged MBR and/or damaged partition table. This means you will not be able to see the drive as a logical device in Disk Editor, and will not be able to recover data using the method below as a conseqence.

turning your damaged drive into a logical device

A logical device is a drive to which DOS has assigned a drive letter. Logical devices are important to the data recovery method below, as it requires the use of clusters, small chunks of data created by DOS upon every logical device. If you cannot get Disk Editor to see your drive as a logical device, you won't be able to use the recovery process below. To be a logical device, a drive must have a functional MBR and a functional partition table. DOS only assigns logical devices on boot, so you'll need to reboot each time to see whether your changes have had the desired effect (which is to see a drive letter, probably "C:").

A damaged MBR that is not repaired by Norton Disk Doctor probably needs a low-level format (LLF). This process is not recommended for IDE drives. LLF's can nuke all data on the entire drive, so ensure you do a "bounded initialisation" that only LLF's sector 0. This is head 0, cylinder 0 only. Try a few different LLF routines - start with the one in the controller's BIOS, if it exists. Use LLF as a data-recovery procedure sparingly - it does erase data, after all.

If you cannot see your drive and your MBR is OK (assume that your MBR is OK until the partition table is known to be OK, and you still can't see your drive) then you probably have a damaged partition table. Put Disk Editor into Physical mode by running it with the /M parameter, or selecting your drive as a Physical device off the Drive.. dialog under the Object menu. Then, jump to the Partition by selecting it from the Object menu. Check that you're looking at Side 1, Cylinder 0, Sector 0 - anything else isn't the partition table. Also check you're in View as Partition display mode - press F6 (I think) to change to this (you could also use the View.. menu).

If your partition table is filled with crap, or all zeros, rebuild it. You may need to "zero out" garbage entries. Check the help for the appropriate type - usually BIGDOS is the one. "boot" is the same as "active partition" in FDISK. If you're not sure of the starting and ending parameters, use the Advanced Recovery Mode (on the Tools.. menu) to tell you. Remember these are the parameters that DOS sees, not the physical geometry of the drive. Most disk subsystems translate their geometry into something DOS can handle, so don't use the specs off the top of the drive. The Advanced Recovery Mode knows all this and displays the translated specs. Edit them into the partition table. To get the relative sectors, select the partition (press Ctrl-B), then use Recalculate Partition from the Tools.. menu.

To summarise..

When the MBR and partition table are functional, a drive letter will be visible.

Exporting your data.. wholesale

If you have a FAT problem, there will be jumbled directories, lots of lost clusters and cross-linked files, "Sector Not Found" messages, among other things. A corrupted FAT means DOS will have difficulty finding things on the drive (such as files and directories), and deciding which file owns what data. If your FAT is fine you should be able to see and use your data. Run Disk Doctor or ScanDisk to ensure everything is OK, and relax.

If your FAT is not fine, you might be able to see directories, but not change to them, or see your file listings, but DOS reports 100% free space. If your FAT is not fine, don't think about rebuilding it.. it's too hard. Find the data you want to keep, save it, and format the drive.

If you have a FAT problem, assuming you can see your drive as a logical device, you can search for the directories you're interested in, and copy the clusters associated with the files in those directories. This technique bypasses DOS and the FAT and reads the data right off the drive.

Find the subdirectories containing the files you want to recover. Disk Editor can do this by searching for subdirectories - use Find Object.. on the Tools menu until you find the ones you are interested in. If the search fails, try keyword searching for the directory name, or some of the filenames you are interested in. Ensure you are in View as Directory display mode (Press F4 before searching). Also ensure you are searching the entire drive - select it from the Object Menu. You might want to keep a note of the cluster number of the directories you locate (just in case).

Disk Editor has a neato feature that exports directory listings to file. This is very useful where the FAT is nuked but the data is OK, because when combined with a cluster size, the directory information can be used to figure out where the files on your drive start and end. These start and end numbers can be fed back into Disk Editor, which can export a range of clusters to disk. So, once the directory is visible to Disk Editor, files can be recovered.

Critically, this approach does not work if the files are fragmented. This means the probability of recovering large files is lowered, however it's worth a shot. A fragmented file will be partially recovered; the first part of the file will be OK, but any fragmented parts will be lost. Half a file may or may not be useful. Note that the recovered file will be approximately the same size as the size in the directory listing, however after the first occurence of fragmentation, the remainder of the recovered file will be parts of other files. For this reason, start-and-end cluster offsets may overlap - if another file appears to start part-way through the file you're recovering, the recovered file is fragmented, and will not be fully recovered.

Disk Editor does not calculate the start and end cluster offsets by itself. It generates the report from which the offsets can be calculated. These calcs are done by a program I wrote called DIRREAD, which reads the Disk Editor report created when the user prints the object to file, using Print Object.. off the Tools menu, when the user is in the directory containing the files which need to be recovered. So, find the subdirectories you're interested in, print them to file, and run DIRREAD over them.

DIRREAD has limitations: max filesize is 9999999999 bytes, and a maximum of cluster offset of 9999999 used by any file being recovered.

DIRREAD needs to know the size of the clusters on the drive which contains the files being recovered. This figure is in bytes, and can be uncovered by using Disk Editor's Drive Info.. dialog on the Info menu. Multiply the bytes per sector reading by the sectors per cluster reading to get bytes per cluster. These figures are contained in the logical characteristics box.

DIRREAD outputs a file called DIRREAD.OUT, which contains a directory listing that includes starting and ending cluster offsets for each file. Print this out and go back to Disk Editor and tell it to export to file a range of clusters corresponding to the files being recovered. Do this by pressing Alt-C to select a range of clusters, and then using Alt-W to write the selection to disk. Even though you are writing clusters, save them as a file. These files are your data files. Don't bother recovering your program files.. reinstall instead.

You'd better hope your application programs don't mind a bit of junk at the end of their data files; there is usually some floating around in the "overhang" area of most clusters. This overhang data will be exported with your file - it's recommended that after recovery, the file be examined, and resaved, if still readable. This overhang is the reason that the recovered files are slightly larger than the size reported by the directory listing. The recovered file sizes are all multiples of the cluster size, which explains why you might see magic numbers like 32768 (32k - eight 4k clusters).

Repeat: this approach will not work with files that are fragmented. If the drive is relatively new, used to have lots of space free, contained files that were not changed much, or was defragmented recently, fragmentation will be reduced, and the likelyhood of successful recovery increased. Conversely, on older drives with no space that have never been defragmented and contain large files that are updated often, fragmentation is likely, and the chances of recovery lowered. This is a good argument to defrag your drives relatively often.

But then, it's worth a shot. When data is valuable enough to spend time like this, anything is better than nothing. I have recovered over 100Mb of data this way.. it does work.

Incidentally, Disk Editor will occasionally scan the disk for corruption, perhaps while you're in the middle of the recovery process. If this happens, it will probably report errors. If it does, ignore them. It's right, all that stuff really is wrong.. but we're going past the FAT and direct to the contiguous clusters that store your data. Right after you've got your data off, the drive deserves a good formatting. So FAT errors just aren't a problem.

To summarise: