The Big Book

RECOVR

RSX-11/ODS-1 Corrupted Disk

and Lost File Recovery

Machine Intelligence and Industrial Magic

MIIM Press

Eau Claire, Wisconsin USA

This document is not in the public domain. Neither this document nor the file containing it are shareware or freeware. It may not be translated, reformatted, or redistributed without written permission of the copyright owner. Placement of this document or a file containing it on any computer network, including the “Internet”, is specifically prohibited and will result in prosecution for copyright violation.

The software described in this manual is provided by Machine Intelligence and Industrial Magic (hereafter referred to as "MIIM") under a nonexclusive single site license agreement. The ownership of this software is not transferred to the licensee and remains with MIIM. Copyright © 1997, Bruce R. Mitchell. All rights, including reproduction by any means whatsoever, are reserved and remain with MIIM.

MIIM makes no warranties, either express or implied, with respect to this software, its quality, performance, merchantability or fitness for any particular purpose. This software is licensed "AS IS." The entire risk as to its quality and performance is with the licensee. Should this software prove defective following its licensing, the licensee assumes the entire cost of all necessary servicing, repair or corrections and any incidental or consequential damages.

In no event shall MIIM be liable for any damages resulting from the use of this software, including direct, indirect, incidental or consequential damages, regardless of the legal theory asserted including negligence and strict liability, even if MIIM has been advised of the possibility of such damages.

This software is not licensed for use in commercial disk recovery operations. Any such use is a violation of the licensing agreement and is cause for license revocation.

The foregoing limitations of remedies and liabilities may not be altered.

This document is typeset in Times New Roman

and was produced using Microsoft™ Word for Windows™.

Table of Contents

Chapter 1 -- Introduction...... 9

Scope...... 9

To the Novice User...... 9

Overview...... 10

Chapter 2 -- Common Disk Corruptions...... 11

Scope...... 11

Types of Corruptions...... 11

Physical Corruptions...... 11

Bad Blocks...... 11

Head Crashes...... 12

Head Misalignment...... 12

Logical Corruptions...... 12

Badblocking and Formatting...... 12

Directory File Deletion...... 13

User File Deletion...... 13

Home Block Corruption...... 13

Initialization...... 13

Random Corruption...... 13

Chapter 3 -- The Distribution Kit...... 15

Scope...... 15

Distribution Media...... 15

Kit Contents...... 15

Installation...... 15

Magnetic Tape Installation...... 15

Floppy Disk Installation...... 16

Chapter 4 -- Preparing for Recovery...... 19

Scope...... 19

Preparing the Host System...... 19

Corrupt Medium Setup...... 19

Recovery Medium Setup...... 18

Chapter 5 -- Recovery Options...... 19

Scope...... 19

Option Setup...... 19

Long Explanations...... 19

Dry Run...... 20

Crashed Disk...... 20

Output Disk...... 21

Logging File Location...... 21

Logging File Spooling...... 22

Bad Block Search...... 22

Bad Block Read Retry...... 23

Pause/Restart...... 23

System File Recovery...... 24

Directory File Listing...... 24

Zero Block File Recovery...... 25

Multiheader File Recovery...... 25

Multiheader File Only Recovery...... 25

Selective Recovery...... 26

Deleted File Recovery...... 26

Deleted File Only Recovery...... 27

Virtual Disk Search...... 27

Multiple Allocation Processing...... 27

In-File Bad Block Replacement...... 28

Contiguous Recovery...... 28

Individual Recovery Verification...... 29

Fatal Error Processing...... 29

Verbose Recovery...... 29

Option Reprise...... 30

Chapter 6 -- Running a Recovery...... 31

Scope...... 31

Starting Recovery...... 31

Recovery Run Phases...... 31

Running Time...... 32

Chapter 7 -- After a Recovery...... 33

Scope...... 33

Post-Recovery Takedown...... 33

Corrupt Medium Takedown...... 33

Post-Recovery Odds and Ends...... 33

Multiheader Files...... 33

EDT Generated Files...... 34

System Files...... 34

Virtual and Reinitialized Disks...... 34

Files Containing Bad Blocks...... 34

Chapter 8 -- Common Recovery Problems...... 35

Scope...... 35

Install-and-Run Failure...... 35

Initialization Failure...... 35

Owning and Directory UFD Differences...... 35

Named Directories...... 36

Decimal File Version Numbers...... 36

Files Generated by EDT...... 36

Suspicious Files...... 37

Chapter 9 -- Corruption Prevention...... 39

Scope...... 39

Backups...... 39

DSC, or How to Shoot Yourself in the Foot...... 40

VFY (Verify)...... 40

Disk Drive Maintenance...... 40

Media Maintenance...... 41

Disk Shadowing...... 42

Chapter 10 -- User Support Services...... 43

Scope...... 43

Software Assistance...... 43

Physical Damage Assistance...... 43

Software Maintenance...... 43

Software Updates...... 44

Error Reporting...... 44

Suggestions...... 44

Appendix A -- Sample Recovery Setup...... 45

Scope...... 45

Sample Recovery Run...... 45

Appendix B -- Sample Recovery Log File...... 51

Scope...... 51

Sample Log File...... 51

Appendix C -- Theory of Disk Recovery...... 65

Scope...... 65

The Files-11 On-Disk Structure...... 65

The Index File...... 65

The Retrieval Pointer...... 66

Multiheader Files...... 66

Theory of Disk Recovery...... 66

Appendix D -- An Introduction to Files-11...... 69

Scope...... 69

The Files-11 On-Disk Structure...... 69

What Is Files-11?...... 69

The Virgin Physical Disk...... 70

Formatting...... 70

Bad Blocks...... 71

The BAD Utility...... 71

The Initializable Disk...... 71

The Boot Block...... 72

The Home Block...... 72

Deeper Into the Intializable Disk...... 72

The Files-11 Logical Structure...... 73

The File Concept...... 73

Disk Directories...... 73

Fragmentation, and What to Do About It...... 74

Free Block Housekeeping...... 74

Index File Bitmap...... 75

Bad, Boot and Home Block Preservation...... 75

Directory Files...... 75

File IDs...... 76

Further Into Files-11...... 77

Files-11 on the Disk...... 77

Mounting the Disk...... 79

Files-11 Failure Points...... 79

Head Crashes...... 80

Badblocking and Formatting...... 80

Initialization...... 80

Home Block Damage...... 80

Bad Blocks...... 81

Directory Deletion...... 81

File Deletion...... 81

Random Corruptions...... 82

Boot Block Damage...... 82

Appendix E -- Bibliography...... 85

Scope...... 85

Bibliography...... 85

Chapter 1

Introduction

Caution!

Please read this entire manual before doing a recovery. It is possible to run a recovery without reading this manual, but westrongly advise against it. Patience will prevent further troubles ... from which there may be no recovery.

1.1 Scope

This chapter is a brief overview of disk recovery and of the rest of this document.

1.2 To the Novice User

READ ME FIRST!

If this is your first experience with corrupt disk recovery, you're probably confused, angry, and afraid that your data is lost for good. Your site's production may be stopped until the problem is corrected. Your job may even be on the line.

It may sound stupid, but this is the time when you must keep a cool head. Don't become overexcited about the trouble and run around like the proverbial chicken. Properly handled, the situation cannot get worse; only better.

Take your time, don't rush into anything, and do things correctly. Improperly handled, things can get much worse.

If you feel frustrated, scream a little and pound the desk. After you have calmed down, continue reading.

You have in your hands documentation for the finest, fastest, most sophisticated disk corruption recovery tool ever made available to RSX users. Everything that you are going through, we have been through too. This is the big gun. RECOVR gets your files back, if it's possible to get them back. It recovers what's left, if the files have been corrupted. And if it can't be done, it tells you so.

We think that's pretty good, considering that there are a lot of pitfalls in this disk recovery business. Hang on just a while longer and soon you'll be recovering files just like all us old pros -- hopefully, not too often.

1.3 Overview

RECOVR is an end-user utility for the RSX family of operating systems. Its field of application is in recovering from common disk corruptions - accidental initialization, lost header files, deleted files, corrupt index files, bad home blocks and the like.

This is the documentation for the RECOVR disk recovery software. It explains how to copy the task from the distribution kit, how to set up and run a recovery, and how to guard against further corruptions.

If installation is to be done from scratch, with a new distribution kit, please proceed with the following section. It describes installation from the distribution kits.

If the RECOVR distribution kit is already installed, please proceed with section 5, which explains how to set up for a recovery.

Examples of a typical setup and recovery are in Appendices A and B. These examples are often useful to the novice user.

For the technically sophisticated user, a general description of the theory behind corrupt ODS-1 disk recovery is found in Appendix C. A bibliography of published literature on Files-11 disk structure and recovery is found in Appendix D.

If anything in this manual is confusing, or if you feel something should be added, please let us know. We're trying to make this manual as readable and useful as possible.

In closing, though it is admittedly cribbing from the greeting card industry, we wish to offer our best wishes for a speedy and complete recovery.

Chapter 2

Common Disk Corruptions

2.1 Scope

This chapter discusses several common disk corruptions, and what can be done about them using RECOVR.

2.2 Corruptions

There are many types of disk corruption, but all of them can be broadly classified as either physical or logical. Physical corruptions involve the mechanical structure of the disk platters; logical corruptions involve the file structure written on the drive.

Each type of corruption can be treated, within limits, by RECOVR. Let us now proceed with a general discussion of each type of corruption.

2.3 Physical Corruptions

Physical corruptions occur because of a physical problem with the actual disk medium. This type of corruption generally involves bad blocks on the disk. An example of physical corruption is the common head crash, during which a normally flying read/write head lands on the disk surface and damages the surface.

There are three types of physical corruptions; bad blocks, head crashes, and head misalignment. RECOVR recovers disks from physical corruptions to the extent possible. Depending on the corruption, anywhere from nothing to the entire disk is recoverable.

Each type of physical corruption is described below.

2.3.1 Bad Blocks

Bad blocks appear unexpectedly and randomly on media as marginally usable sectors deteriorate. This is an unavoidable problem with all magnetic media, even so-called "error free" media. All disks eventually develop bad blocks.

The appearance of a bad block on a disk causes two different types of problems, depending on whether the bad block landed inside or outside of the index file.

If the bad block appears in the index file, the header for a file may be lost. This depends on where the bad block lands -- on an unused header block or on an active header block. If the bad block lands on an inactive header block, there is no harm done.

If the bad block lands on an active primary header, the entire file to which that block corresponds is also lost, even though the individual blocks of the file may be intact. Once the header is lost, there is no way to reassemble the file short of hand search and inspection of the disk. If an active secondary header is lost, that segment of the file is lost. In this case, RECOVR fails to recover files with bad headers.

If the bad block appears elsewhere on the disk, within a file, the file is recoverable with the exception of the bad block. Depending on the disk controller, even the data in the bad block may be partially usable. In this case, RECOVR recovers a file with an embedded bad block.

2.3.2 Head Crashes

Physical head crashes on a disk are the most severe recovery problem, and are difficult to deal with. We strongly recommend that an expert in this field be consulted before doing anything.

If the crash is not severe, it may be possible to recover a majority of the disk. If only one head fails, it may be possible to remove it and recover the disk with the exception of the bad surface. If multiple heads fail, it is probably not possible to recover anything.

In this case, RECOVR should be used as an additional tool by an experienced physical disk disaster recovery team, and recovers anywhere from nothing to the entire disk.

2.3.3 Head Misalignment

Head misalignment is a problem more properly dealt with by the RSX operating system, disk drivers, drive controllers, and a good disk drive technician. We strongly recommend that a hardware expert, conversant with the specific disk drive model in question, be consulted before doing anything.

This problem usually shows up immediately after long neglect followed by a sudden complete realignment of the heads on a pack disk drive of the DEC RM or RP series (CDC 976x series drives). Expert class technical assistance of a better than usual drive service person is required to get the drive's 10, 12 or 16 heads back to the previous misaligned positions so that data can be recovered.

Fortunately, this problem does not occur with any frequency on modern 5.25" or 3.5" sealed unit disk drives.

In this case, RECOVR should be used as an additional tool by the disk expert, and recovers anywhere from nothing to the entire disk.

2.4 Logical Corruptions

Logical corruptions generally do not involve bad blocks on the disk, although controllers running wild over a disk can certainly cause bad blocks and logical corruptions. Logical corruptions occur because of a break in the Files-11 structure written on the medium. An example of this type of corruption is accidental deletion of a file.

There are several types of logical corruptions. RECOVR recovers disks from all logical corruptions to the extent possible. Depending on the corruption, anywhere from nothing to the entire disk is recoverable.

Each type of logical corruption is described below.

2.4.1 Badblocking and Formatting

Accidentally running BAD or FMT against a disk is potentially the worst possible logical corruption. Both utilities completely destroy all data on the disk. Depending on how quickly the accident is noticed, corruption ranges from minimal to complete.

If the utility is aborted immediately, only the boot block, home block and some file contents are lost. If the utility does not reach the index file, files not located in the overwritten area are recoverable. If it reaches the index file, some files may be recovered but many are lost. If the utility runs to completion, the entire disk is lost with no possibility of recovery.

In this case, RECOVR recovers anywhere from nothing to the entire disk, depending on how far the utility went into the disk.

2.4.2 Directory File Deletion

Accidentally deleting a directory file sounds horrible, but is actually not a serious problem. Although the directory file is deleted, the files and their headers still exist completely intact on the disk. Two approaches are possible here: (1) Recover the directory file, or (2) Create a new directory and recover the individual files.

In either case, RECOVR recovers the lost files.

2.4.3 User File Deletion

Accidentally deleting a file or files is a straightforward recovery, if the deletion is noted immediately and all activity on the disk in question is stopped (preferably by immediately hitting the HALT switch on the CPU.)

Provided that the system is stopped and / or the disk is spun down immediately after the deletion, RECOVR recovers the deleted files.

2.4.4 Home Block Corruption

If a home block is overwritten, the disk is unmountable but all files and file headers on the disk are still intact. This is an irritating but relatively minor problem.

In this case, RECOVR recovers all files on the disk.

2.4.5 Initialization

Accidentally initializing a disk is generally a very serious corruption. When a disk is initialized, the boot block, home block, and system files are permanently lost. In addition, anywhere from a small part to all of the index file may be lost, depending on the location of the index file and its physical configuration on the disk.

In this case, RECOVR recovers anywhere from nothing to the majority of user files on the disk.

2.4.6 Random Corruption

Random corruption of a disk may be caused by a defective controller, a privileged task running wild, running an XXDP+ exerciser against a disk, or any of a number of other problems.

In random corruptions, recoverability of the disk depends on where the corruptions occurred and how densely the disk is populated. Corrupt blocks do corrupt the files in which they are found to some extent. Whether a file recovered after such a problem is of any use depends on the application.

In this case, RECOVR recovers all files with uncorrupt file headers, though some recovered files are likely to contain blocks of "garbage."

Chapter 3

The Distribution Kit

3.1 Scope

This chapter discusses the distribution kit. The medium, the contents of the medium, and installation of the distribution kit are covered.

3.2 Distribution Media

RECOVR is provided on standard RSX distribution media as part of the licensing fee. The normal distribution media are:

800/1600 BPI, 9 channel, 600 foot magnetic tape in DOS FLX format
Single density (RX01 format), 8 inch floppy disk in Files-11 format
Single density (RX50 format), 5.25" floppy disk in Files-11 format

Special distribution media are available on request. If you have ordered your software on a special distribution medium, it is formatted according to your specifications.

3.3 Kit Contents

In UFD [1,54] on your distribution kit is a single runnable task image. The filename of this task image is RECOVR.TSK. It is a privileged (/PR:0), no external header (/-XH), non-checkpointable (/-CP/-AL), high priority (PRI=100) task. The task image requires from 350 to 450 contiguous blocks of disk; the size of the task varies according to internal options, buffering levels and hardware options available on the target CPU.

3.4 Installation

There are two different installation procedures depending on the distribution medium. Follow the installation procedure which is correct for your medium.

3.4.1 Magnetic Tape Installation

Installation of the magnetic tape distribution kit is somewhat complex due to the need for backward compatibility with older versions of RSX. Use the following procedure: