The data is misaligned with the sector boundaries on those tracks. This thermal effect also can work in reverse: If the drive is formatted and written to while it is extremely hot, it may not read properly while cold because of dimensional changes in the platters. This problem occurs with drives that have the "Monday-morning blues," in which they spin but cannot read data properly when they are first powered on, especially after being off for an extended period (over a weekend, for example). If you leave the power to the system on for some time so that the drive can warm up, the system then may boot and run normally.

If this happens, the next step is to back up the drive completely and initiate a new LLF at the proper operating temperature (described next). This procedure enables the drive to work normally again until temperature-induced mistracking becomes great enough to cause the problem again.

Knowing that temperature fluctuations can cause mistracking, you should understand the reasons for the following basic rules of disk use:

• Leave the system's power on for at least 30 minutes before performing an LLF on its hard disk. This step ensures that the platters are at a normal operating temperature and have stabilized dimensionally.
  
  
• If possible, allow a system some time to warm up after power-on before storing any data on the hard disk. This procedure is not required for voice coil drives.

If you have a cheap stepper motor drive that consistently exhibits temperature-related mistracking problems, you may want to consider running the drive constantly. Doing so would extend its trouble-free life span significantly because the temperature and dimensions of the platters would stay relatively constant.

These kinds of temperature-fluctuation problems are more of a problem with drives that have open-loop stepper motor actuators (which offer no thermal compensation) than with the closed-loop voice coil actuators (which follow temperature-induced track migration and compensate completely, resulting in no tracking errors even with large changes in platter dimensions).

Modern voice coil actuator drives do not exhibit these dimensional instabilities due to thermal expansion and contraction of the platters because they have a track-following servo mechanism. As the tracks move, the positioner automatically compensates. Many of these drives undergo a noticeable thermal compensation sequence every five minutes or so for the first 30 minutes after being powered on, and usually every 30 minutes after that. During these thermal-compensation routines, you hear the heads move back and forth as they measure and compensate for platter-dimension changes.

Drive Operating Position

Another consideration before formatting a drive is ensuring that the drive is formatted in the operating position it will have when it is installed in the system. Gravity can place on the head actuator different loads that can cause mistracking if the drive changes between a vertical and a horizontal position. This effect is minimized or even eliminated in most voice coil drives, but this procedure cannot hurt.

Additionally, drives that are not properly shock-mounted should be formatted only when they are installed in the system because the installation screws exert twisting forces on the drive's Head Disk Assembly (HDA), which can cause mistracking. If you format the drive with the mounting screws installed tightly, it may not read with the screws out, and vice versa. Be careful not to overtighten the mounting screws, because doing so can stress the HDA. This usually is not a problem if the drive's HDA is isolated from the frame by rubber bushings.

In summary, for a proper LLF, the drive should be

• At a normal operating temperature
  
  
• In a normal operating position
  
  
• Mounted in the host system (if the drive HDA is not shock-mounted or isolated from the drive frame by rubber bushings)

Because many different makes and models of controllers differ in the way that they write data to a drive, especially with respect to the encoding scheme, it is best to format the drive using the same make and model of controller as the controller that will be used in the host system. Some brands of controllers work exactly alike, however, so this is not an absolute requirement even if the interface is the same. This problem does not occur with IDE or SCSI drives, of course, because the controller is built into the drive. Usually, if the controller establishes the drive type by using its own on-board ROM rather than the system setup program, it will be incompatible with other controllers.

Low-Level Format

Of these procedures, the LLF is most important to ensure trouble-free operation of the drive. This format is the most critical of the operations and must be done correctly for the drive to work properly. The LLF includes several subprocedures:

• Scanning for existing defect mapping
  
  
• Selecting the interleave
  
  
• Formatting and marking (or remarking) manufacturer defects
  
  
• Running a surface analysis

On all new systems, these subprocedures are performed automatically by the system's LLF program and require no user intervention, and you need not continue in this section. On older systems, you must take the initiative, and should read on.

To perform the drive defect mapping, to select an interleave, and to complete a surface analysis of the drive, you need information about the drive, the controller, and possibly the system. This information usually is provided in separate manuals or documents for each item; therefore, be sure that you get the complete documentation for your drive and controller products when you purchase them. The specific information required depends on the type of system, controller, and LLF program that you are using.

Defect Mapping

Before formatting the disk, you need to know whether the drive has defects that have to be mapped out. Older drives came with a list of defects discovered by the manufacturer during the drive's final quality control testing. These defects were marked so that they are not used later to store programs or data.

Defect mapping was one of the most critical aspects of low-level formatting, now just a historical curiosity as low-level formatting is now done almost exclusively by the manufacturer. If you would like to understand the defect-mapping procedures, you first must understand what happens when a defect is mapped on a drive.

The manufacturer's defect list usually indicated defects by cylinder and track. When this information is entered, an LLF program marked these tracks with invalid checksum figures in the header of each of the sectors, ensuring that nothing can read or write to these locations. When DOS performed a high-level format of the disk, the DOS FORMAT program could not read these locations, and it marked the involved clusters in the File Allocation Table (FAT) so that they never will be used.

The list of defects that the manufacturer gives you probably is more extensive than what a program could determine from your system, because the manufacturer's test equipment is far more sensitive than a regular disk controller. The FORMAT program did not find the defects automatically; you probably had to enter them manually. The exceptions are new systems, in which the defect list is encoded in a special area of the drive that normal software cannot access. The LLF program (included in the motherboard BIOS program that comes with the systems) reads this special map, thereby eliminating the need to enter these locations manually.

---

TIP: If your motherboard BIOS does not have an LLF utility built in, you are probably better off not low-level formatting an IDE drive.

---

All new drives are low-level formatted by the manufacturer. If you bought a system with a drive already installed by the manufacturer or dealer, an LLF probably was done for you. Most manufacturers no longer recommend you LLF any IDE type drive.

Although an actual defect is technically different from a marked defect, these defects should correspond to one another if the drive is formatted properly. For example, I can enter the location of a good track into the LLF program as a defective track. The LLF program then corrupts the checksum values for the sectors on that track, rendering them unreadable and unwriteable. When the DOS FORMAT program encounters that track, it finds the track unreadable and marks the clusters occupying that track as being bad. After that, as the drive is used, DOS ensures that no data ever is written to that track. The drive stays in that condition until you redo the LLF of that track, indicate that the track is not to be marked defective, and redo the high-level format that no longer will find the track unreadable and therefore permit those clusters to be used. In general, unless an area is marked as defective in the LLF, it will not be found as defective by the high-level format, and DOS will subsequently use it for data storage.

Defect mapping becomes a problem when someone formats a hard disk and fails to enter the manufacturer's defect list, which contains actual defect locations, so that the LLF can establish these tracks or sectors as marked defects. Letting a defect go unmarked will cost you data when the area is used to store a file that you subsequently cannot retrieve.

Unfortunately, the LLF program does not automatically find and mark any areas on a disk that are defective. The manufacturer's defect list is produced by very sensitive test equipment that tests the drive at an analog level. Most manufacturers indicate areas as being defective even if they are just marginal. The problem is that a marginal area today may be totally unreadable in the future. You should avoid any area suspected as being defective by entering the location during the LLF so that the area is marked; then DOS is forced to avoid the area.

Currently Marked Defects Scan

Most LLF programs have the capability to perform a scan for previously marked defects on a drive. Some programs call this operation a defect scan; IBM calls it Read Verify in the IBM Advanced Diagnostics. This type of operation is nondestructive and reports by cylinder and head position all track locations marked bad. Do not mistake this for a true scan for defective tracks on a disk, which is a destructive operation normally called a surface analysis (discussed in the section "Surface Analysis" later in this chapter). If a drive was low-level formatted previously, you should scan the disk for previously marked defects before running a fresh LLF for several reasons:

• To ensure that the previous LLF correctly marked all manufacturer-listed defects. Compare the report of the defect scan with the manufacturer's list, and note any discrepancies. Any defects on the manufacturer's list that were not found by the defect scan were not marked properly.
  
  
• To look for tracks that are marked as defective but are not on the manufacturer's list. These tracks may have been added by a previously run surface-analysis program (in which case they should be retained), or they may result from the previous formatter's typographical errors in marking the manufacturer's defect. One of my drive manufacturer's lists showed Cylinder 514 Head 14 as defective. A defect scan, however, showed that track as good but Cylinder 814, Head 14 as bad. Because the latter location was not on the manufacturer's list, and because typing 5 instead of 8 would be an easy mistake to make, I concluded that a typographical error was the cause and then reformatted the drive, marking Cylinder 514, Head 14 as bad and enabling Cylinder, 814 Head 14 to be formatted as a good track, thus "unmarking" it.

If you run a surface analysis and encounter defects in addition to those on the manufacturer's list, you can do one of two things:

• If the drive is under warranty, consider returning it.
  
  
• If the drive is out of warranty, grab a pen and write on the defect-list sticker, adding the bad tracks discovered by the surface-analysis program. (The low-level formatter built into the system BIOS performs a surface analysis immediately after the LLF; if it discovers additional defects, it automatically adds them to the defect list recorded on the drive.) Adding new defects to the sticker in this manner means that these areas are not forgotten when the drive is subsequently reformatted.

Manufacturer's Defect List

The manufacturer tests a new hard disk by using sophisticated analog test instruments that perform an extensive analysis of the surface of the platters. This kind of testing can indicate the functionality of an area of the disk with great accuracy, precisely measuring information such as the signal-to-noise ratio and recording accuracy.

Some manufacturers have more demanding standards than others about what they consider to be defects. Many people are bothered by the fact that when they purchase a new drive, it comes with a list of defective locations; some even demand that the seller install a defect-free drive. The seller can satisfy this request by substituting a drive made by a company with less-stringent quality control, but the drive will be of poorer quality. The manufacturer that produces drives with more listed defects usually has a higher-quality product because the number of listed defects depends on the level of quality control. What constitutes a defect depends on who is interpreting the test results.

To mark the manufacturer defects listed for the drive, consult the documentation for your LLF program. For most drives, the manufacturer's defect list shows the defects by cylinder and head; other lists locate the defect down to the bit on the track that is bad, starting with the index location.

---

CAUTION: Make sure that all manufacturer's defects have been entered before proceeding with the LLF.

---

Some systems automatically mark the manufacturer's defects, using a special defect file recorded on the drive by the manufacturer. For such a system, you need a special LLF program that knows how to find and read this file. Automatic defect-map entry is standard for IBM PS/2 systems and for most ESDI and all SCSI systems. Consult the drive or controller vendor for the proper LLF program and defect-handling procedures for your drive.

---

NOTE: Data recovery utilities such as ScanDisk (DOS/Windows) or Norton Utilities cannot mark the sectors or tracks at the physical format level. The bad cluster marks that they make are stored only in the FAT and are easily erased during the next high-level format operation. You should also use an LLF utility designed for your drive (contact the manufacturer), which will properly mark bad sectors and assign spares at the physical disk level.

---

Surface Analysis

A defect scan is a scan for marked defects; a surface analysis is a scan for actual defects. A surface analysis ignores tracks already marked defective by an LLF and tests the unmarked tracks. The surface-analysis program writes 512 bytes to each sector on the good tracks, reads the sectors back, and compares the data read to what was written. If the data does not verify, the program (like an LLF) marks the track bad by corrupting the checksum values for each sector on that track. A proper surface analysis is like an LLF program in that it should bypass the DOS and the BIOS so that it can turn off controller retry operations and also see when ECC (Error Correction Code) is invoked to correct soft errors.

Surface-analysis programs are destructive: They write over every sector except those that already are marked as bad. You should run a surface-analysis program immediately after running an LLF to determine whether defects have appeared in addition to the manu-facturer's defects entered during the LLF. A defect scan after the LLF and the surface analysis shows the cumulative tracks that were marked bad by both programs.

If you have lost the manufacturer's defect list, you can use the surface-analysis program to indicate which tracks are bad, but this program never can duplicate the accuracy or sensitivity of the original manufacturer testing.

Although it is recommended if you have been experiencing any problems with a drive, on new drives I normally do not run a surface analysis after low-level formatting for several reasons:

• Compared with formatting, surface analysis takes a long time. Most surface-analysis programs take two to five times longer than an LLF. An LLF of a 120M drive takes about 15 minutes; a surface analysis of the same drive takes an hour or more. Moreover, if you increase the accuracy of the surface analysis by allowing multiple passes or multiple patterns, the surface analysis takes even longer.
  
  
• With high-quality drives, I never find defects beyond those that the manufacturer specified. In fact, the surface-analysis programs do not find all the manufacturer's defects if I do not enter them manually. Because the high-quality (voice coil) drives that I use have been tested by the manufacturer to a greater degree than a program can perform on my system, I simply mark all the defects from the manufacturer's list in the LLF. If I were using a low-quality (stepper motor) drive or installing a used and out-of-warranty drive, I would consider performing a surface analysis after the LLF.

Why Low-Level Format?

Even though it generally is not necessary (or even recommended) to LLF IDE or SCSI drives, there are a few good reasons to consider an LLF. One reason is that an LLF will wipe out all the data on a drive, ensuring that other people will not be able to read or recover that data. This procedure is useful if you are selling a system and do not want your data to be readable by the purchaser. Another reason for wiping all the data from a drive is to remove corrupted or non-DOS operating-system partitions and even virus infections. The best reason is for defect management. As you may have noticed, most ATA-IDE drives appear to have no "bytes in bad sectors" under CHKDSK or any other software.

Any defects that were present on the drive after manufacturing were reallocated by the factory LLF. Essentially, any known bad sectors are replaced by spare sectors stored in different parts of the drive. If any new defects occur, such as from a minor head/platter contact or drive mishandling, a proper IDE-aware LLF program can map the new bad sectors to other spares, hiding them and restoring the drive to what appears to be defect-free status.

Because the IDE (ATA) specification is an extension of the IBM/WD ST-506/412 controller interface, the specification includes several new CCB commands that were not part of the original INT 13h/CCB support. Some of these new CCB commands are vendor-specific and are unique to each IDE drive manufacturer. Some manufacturers use these special CCB commands for tasks such as rewriting the sector headers to flag bad sectors, which in essence means LLF. When using these commands, the drive controller can rewrite the sector headers and data areas and then carefully step over any servo information (if the drive uses an embedded servo).

IDE drives can be low-level formatted, although some drives require special vendor- specific commands to activate certain LLF features and defect-management options. Seagate, Western Digital, Maxtor, IBM, and others make specific LLF and spare-sector defect-management software specific to their respective IDE drives. Conner drives are unique in that to actually LLF them, you need a special hardware device that attaches to a diagnostic port connector on the Conner IDE drive. A company called TCE (they are in the vendor list in Appendix A) sells such a device for $99. Coincidentally, this device is called The Conner. It includes software and the special adapter device that permits true low-level formatting (including rewriting all sectors and sector headers, as well as completely managing spare sector defects) at the factory level.

Other companies have developed LLF software that recognizes the particular IDE drive and uses the correct vendor-specific commands for the LLF and defect mapping. The best of these programs is Ontrack's Disk Manager. A general-purpose diagnostic program that also supports IDE-drive formatting is the MicroScope package by Micro 2000.

Intelligent IDE drives must be in nontranslating, or native, mode to LLF them. Zoned Recording drives can perform only a partial LLF, in which the defect map is updated and new defective sectors can be marked or spared, but the sector headers usually are rewritten only partially, and only for the purpose of defect mapping. In any case, you are writing to some of the sector headers in one form, and physical (sector-level) defect mapping and sector sparing can be performed. This procedure is, by any standard definition, an LLF.

On an embedded servo drive, all the servo data for a track is recorded at the same time by a specialized (usually laser-guided) servowriter. This servo information is used to update the head position continuously during drive function, so that the drive automatically compensates for thermal effects. As a result, all the individual servo bursts are in line on the track. Because the servo controls head position, there is no appreciable head-to-sector drift, as there could be on a nonservo drive.

This is why even though it is possible to LLF embedded servo drives, it rarely is necessary. The only purpose for performing an LLF on an embedded servo drive is to perform additional physical- (sector-) level defect mapping or sector sparing for the purpose of managing defects that occur after manufacture. Because no drift occurs, when a sector is found to contain a flaw, it should remain permanently marked bad; a physical flaw cannot be repaired by reformatting.

Most IDE drives have three to four spare sectors for each physical cylinder of the drive. These hundreds of spare sectors are more than enough to accommodate the original defects and any subsequent defects. If more sectors are required, the drive likely has serious physical problems that cannot be fixed by software.

Software for Low-Level Formatting

You often can choose among several types of LLF programs, but no single LLF program works on all drives or all systems. Because LLF programs must operate very closely with the controller, they often are specific to a controller or controller type. Therefore, ask the controller manufacturer for the formatting software it recommends.

If the controller manufacturer supplies an LLF program (usually in the controller's ROM), use that program, because it is the one most specifically designed for your system and controller. The manufacturer's program can take advantage of special defect-mapping features, for example. A different format program not only might fail to use a manufacturer-written defect map, but also might overwrite and destroy it.

For a general-purpose ST-506/412, ESDI, or IDE LLF program, I recommend the Disk Manager program by Ontrack. For the ST-506/412 interface only, I recommend the IBM Advanced Diagnostics or the HDtest program by Jim Bracking, a user-supported product found on many electronic bulletin boards, including CompuServe. (These companies are listed in the vendor list at the back of this book.) For SCSI systems and systems on which the other recommended programs do not work, you normally use will the format program supplied with the SCSI host adapter.

How Low-Level Format Software Works

There are several ways that a program can LLF a drive. The simplest way is to call the BIOS by using INT 13h functions such as the INT 13h, function 05h (Format Track) command. The BIOS then converts this command to what is called a CCB (Command Control Block) command: A block of bytes sent from the proper I/O ports directly to the disk controller. In this example, the BIOS would take INT 13h, 05h and convert it to a CCB 50h (Format Track) command, which would be sent through the Command Register Port (I/O address 1F7h for ST-506/412 or IDE). When the controller receives the CCB Format Track command, it may actually format the track or may simply fill the data areas of each sector on the track with a predetermined pattern.

The best way to LLF a drive is to bypass the ROM BIOS and send the CCB commands directly to the controller. Probably the greatest benefit in sending commands directly to the drive controller is being able to correctly flag defective sectors via the CCB Format Track command, including the capability to perform sector sparing. This is why IDE drives that are properly low-level formatted never show any bad sectors.

By using the CCB commands, you also gain the ability to read the Command Status and Error registers (which enable you to detect things such as ECC corrected data, which is masked by DOS INT 13h). You also can detect whether a sector was marked bad by the manufacturer or during a previous LLF and can maintain those marks in any subsequent Format Track commands, thereby preserving the defect list. I do not recommend unmarking a sector (returning it to "good" status), especially if the manufacturer previously marked it as bad.

When you use CCB commands, you can read and write sector(s) with automatic retries as well as ECC turned off. This capability is essential for any good surface analysis or LLF program, and this is why I recommend programs that use the CCB hardware interface rather than the DOS INT 13h interface.

Ontrack Disk Manager

For AT-type systems and other systems with controllers that do not have an autoconfigure routine, the Disk Manager program from Ontrack is excellent. It probably is the most sophisticated hard disk format tool available and has many capabilities that make it a desirable addition to your toolbox.

Disk Manager is a true register-level format program that goes around the BIOS and manipulates the disk controller directly. This direct controller access gives it powerful capabilities that simply are not possible in programs that work through the BIOS.

Some of these advanced features include the capability to set head- and cylinder-skew factors. Disk Manager also can detect intermittent (soft) errors much better than most other programs can, because it can turn off the automatic retries that most controllers perform. The program also can tell when ECC has been used to correct data, indicating that an error occurred, as well as directly manipulate the bytes that are used for ECC. Disk Manager has been written to handle most IDE drives and uses vendor-specific commands to unlock the capability to perform a true LLF on IDE drives.

All these capabilities make Disk Manager one of the most powerful and capable LLF programs available. Ontrack also offers an excellent package of hard disk diagnostic and data-recovery utilities called DOS Utils. Anybody who has to support, maintain, troubleshoot, repair, or upgrade PCs needs a powerful disk formatter such as Disk Manager.

HDtest

HDtest is an excellent BIOS-level format program that will function on virtually any drive that has an INT 13h ROM BIOS interface, which includes most drives. HDtest does not have some of the capabilities of true register-level format programs, but it can be used when the additional capabilities of a register-level program are not required. For example, you can use this program to do a quick wipe of all the data on a drive, no matter what the interface or controller type is. HDtest also is good for BIOS-level read and write testing, and has proved to be especially useful in verifying the functions of disk interface BIOS code.

HDtest, by Jim Bracking, is a user-supported software program. This program is distributed through electronic bulletin boards and public-domain software libraries. You also can obtain the program from the Public Software Library, listed in Appendix A. It costs $35, but you can try it for free.

HDtest has an easy-to-use interface and pull-down menu system. The program offers all functions that normally are associated with a standard LLF program, as well as some extras:

• Normal formatting
  
  
• Defect mapping
  
  
• Surface analysis
  
  
• Interleave test
  
  
• Nondestructive low-level reformat
  
  
• Hard disk tests (duplicate of the IBM Advanced Diagnostics hard disk tests), including tests for drive seek, head selection, and error detection and correction, as well as a read/write/verify of the diagnostics cylinder. This program also can run low-level ROM BIOS commands to the controller.

HDtest includes most of what you would want in a generic LLF program and hard disk diagnostics utility. Its real limitation is that it works only through the BIOS and cannot perform functions that a true register-level format program can. In some cases, the program cannot format a drive that a register-level program could format. Only register-level programs can perform defect mapping in most IDE and SCSI environments.

SCSI Low-Level Format Software

If you are using a SCSI drive, you must use the LLF program provided by the manufacturer of the SCSI host adapter. The design of these devices varies enough that a register-level program can work only if it is tailored to the individual controller. Fortunately, all SCSI host adapters include such format software, either in the host adapter's BIOS or in a separate disk-based program.

The interface to the SCSI drive is through the host adapter. SCSI is a standard, but there are no true standards for what a host adapter is supposed to look like. This means that any formatting or configuration software will be specific to a particular host adapter. For example, IBM supplies formatting and defect-management software that works with the IBM PS/2 SCSI host adapters directly on the PS/2 Reference disk. That software performs everything that needs to be done to a SCSI hard disk connected to an IBM host adapter. IBM has defined a standard interface to its adapter through an INT 13h and INT 4Bh BIOS interface in a ROM installed on the card. The IBM adapters also include a special ABIOS (Advanced BIOS) interface that runs in the processor's protected mode of operation (for use under protected-mode operating systems such as OS/2).

Other SCSI host adapters often include the complete setup, configuration, and formatting software in the host adapter's on-board ROM BIOS. Most of these adapters also include an INT 13h interface in the BIOS. The best example is the Adaptec 1540/1542C adapters, which include software in ROM that completely configures the card and all attached SCSI devices.

---

NOTE: Notice that SCSI format and configuration software is keyed to the host adapter and is not specific in any way to the particular SCSI hard disk drive that you are using.

---

IDE Low-Level Format Software

IDE drive manufacturers have defined extensions to the standard WD1002/1003 AT interface, which was further standardized for IDE drives as the ATA (AT Attachment) interface. The ATA specification provides for vendor-unique commands, which are manufacturer proprietary extensions to the standard. To prevent improper low-level formatting, many of these IDE drives have special codes that must be sent to the drive to unlock the format routines. These codes vary among manufacturers. If possible, you should obtain LLF and defect-management software from the drive manufacturer; this software usually is specific to that manufacturer's products.

The custom nature of the ATA interface drives is the source of some myths about IDE. Many people say, for example, that you cannot perform an LLF on an IDE drive, and that if you do, you will wreck the drive. This statement is untrue! What can happen is that in some drives, you may be able to set new head and sector skew factors that are not as optimal for the drive as the ones that the manufacturer set, and you also may be able to overwrite the defect-map information. This situation is not good, but you still can use the drive with no problems provided that you perform a proper surface analysis.

Most ATA IDE drives are protected from any alteration to the skew factors or defect map erasure because they are in a translated mode. Zoned Recording drives always are in translation mode and are fully protected. Most ATA drives have a custom command set that must be used in the format process; the standard format commands defined by the ATA specification usually do not work, especially with intelligent or Zoned Recording IDE drives. Without the proper manufacturer-specific format commands, you will not be able to perform the defect management by the manufacturer-specified method, in which bad sectors often can be spared.

Currently, the following manufacturers offer specific LLF and defect-management software for their own IDE drives:

• Seagate
  
  
• Maxtor
  
  
• Western Digital
  
  
• IBM

These utilities are available for downloading on the various BBSes run by these companies. The numbers appear in Appendix A.

Conner Peripherals drives are unique in that they cannot be low-level formatted through the standard interface; they must be formatted by a device that attaches to a special diagnostics and setup port on the drive. You see this device as a 12-pin connector on Conner drives. A company called TCE sells an inexpensive device that attaches your PC to this port through a serial port in your system, and includes special software that can perform sophisticated test, formatting, and surface-analysis operations. The product is called The Conner. (TCE is listed in Appendix A.)

For other drives, I recommend Disk Manager by Ontrack, as well as the MicroScope program by Micro 2000. These programs can format most IDE drives because they know the manufacturer-specific IDE format commands and routines. They also can perform defect-mapping and surface-analysis procedures.

Nondestructive Formatters

General-purpose, BIOS-level, nondestructive formatters such as Calibrate and SpinRite are not recommended in most situations for which a real LLF is required. These programs have several limitations and problems that limit their effectiveness; in some cases, they can even cause problems with the way defects are handled on a drive. These programs attempt to perform a track-by-track LLF by using BIOS functions, while backing up and restoring the track data as they go. These programs do not actually perform a complete LLF, because they do not even try to LLF the first track (Cylinder 0, Head 0) due to problems with some controller types that store hidden information on the first track.

These programs also do not perform defect mapping in the way that standard LLF programs do, and they can even remove the carefully applied sector header defect marks during a proper LLF. This situation potentially allows data to be stored in sectors that originally were marked defective and may actually void the manufacturer's warranty on some drives. Another problem is that these programs work only on drives that are already formatted and can format only drives that are formattable through BIOS functions.

A true LLF program bypasses the system BIOS and send commands directly to the disk controller hardware. For this reason, many LLF programs are specific to the disk controller hardware for which they are designed. It is virtually impossible to have a single format program that will run on all different types of controllers. Many hard drives have been incorrectly diagnosed as being defective because the wrong format program was used and the program did not operate properly.

Drive Partitioning

Partitioning a hard disk is the act of defining areas of the disk for an operating system to use as a volume. To DOS, a volume is an area of a disk denoted as a drive letter; for example, drive C is volume C, drive D is volume D, and so on. Some people think that you have to partition a disk only if you are going to divide it into more than one volume. This is a misunderstanding; a disk must be partitioned even if it will be the single volume C.

When a disk is partitioned, a master partition boot sector is written at cylinder 0, head 0, sector 1--the first sector on the hard disk. This sector contains data that describes the partitions by their starting and ending cylinder, head, and sector locations. The partition table also indicates to the ROM BIOS which of the partitions is bootable and, therefore, where to look for an operating system to load. A single hard disk can have 1 to 24 partitions. This number includes all the hard drives installed in the system, which means that you can have as many as 24 separate hard disks with one partition each, a single hard disk with 24 partitions, or a combination of disks and partitions such that the total number of partitions is no more than 24. If you have more than 24 drives or partitions, DOS does not recognize them, although other operating systems may. What limits DOS is that a letter is used to name a volume, and the Roman alphabet ends with Z--the 24th volume, when you begin with C.

FDISK

The DOS FDISK program is the accepted standard for partitioning hard disks. Partitioning prepares the boot sector of the disk in such a way that the DOS FORMAT program can operate correctly; it also enables different operating systems to coexist on a single hard disk.

If a disk is set up with two or more partitions, FDISK shows only two total DOS partitions: the primary partition and the extended partition. The extended partition then is divided into logical DOS volumes, which are partitions themselves. FDISK gives a false impression of how the partitioning is done. FDISK reports that a disk divided as C, D, E, and F is set up as two partitions, with a primary partition having a volume designator of C and a single extended partition containing logical DOS volumes D, E, and F. But in the real structure of the disk, each logical DOS volume is a separate partition with an extended partition boot sector describing it. Each drive volume constitutes a separate partition on the disk, and the partitions point to one another in a daisy-chain arrangement.

The minimum size for a partition in any version of DOS is one cylinder; however, FDISK in DOS 4 and later versions allocates partitions in megabytes, meaning that the minimum-size partition is 1M. DOS 4.x and later versions permit individual partitions or volumes to be as large as 2G, whereas versions of DOS earlier than 4.0 have a maximum partition size of 32M.

The current DOS (version 7 underlying Windows 95 OSR2) supports partition sizes of up to 2T using FAT 32.

FDISK Undocumented Functions

FDISK is a very powerful program, and in DOS 5 and later versions, it gained some additional capabilities. Unfortunately, these capabilities were never documented in the DOS manual and remain undocumented even in DOS 7. The most important undocumented parameter in FDISK is the /MBR (Master Boot Record) parameter, which causes FDISK to rewrite the Master Boot Sector code area, leaving the partition tables intact.

---

CAUTION: Beware: It will overwrite the partition tables if the two signature bytes at the end of the sector (55AAh) are damaged. This situation is highly unlikely, however. In fact, if these signature bytes were damaged, you would know; the system would not boot and would act as though there were no partitions at all.

---

The /MBR parameter seems to be tailor-made for eliminating boot-sector virus programs that infect the Master Partition Boot Sector (Cylinder 0, Head 0, Sector 1) of a hard disk. To use this feature, you simply enter

FDISK /MBR

FDISK then rewrites the boot sector code, leaving the partition tables intact. This should not cause any problems on a normally functioning system, but just in case, I recommend backing up the partition table information to floppy disk before trying it. You can do this with the following command:

MIRROR /PARTN

This procedure uses the MIRROR command to store partition-table information in a file called PARTNSAV.FIL, which should be stored on a floppy disk for safekeeping. To restore the complete partition-table information, including all the master and extended partition boot sectors, you would use the UNFORMAT command as follows:

UNFORMAT /PARTN

This procedure causes the UNFORMAT command to ask for the floppy disk containing the PARTNSAV.FIL file and then to restore that file to the hard disk.

Note that if you are using Windows 95, the MIRROR and UNFORMAT programs have been eliminated, and you will have to purchase Norton Utilities instead.

FDISK also has three other undocumented parameters: /PRI:, /EXT:, and /LOG:. These parameters can be used to have FDISK create master and extended partitions, as well as logical DOS volumes in the extended partition, directly from the command line rather than through the FDISK menus. This feature was designed so that you can run FDISK in a batch file to partition drives automatically. Some system vendors probably use these parameters (if they know about them, that is!) when setting up systems on the production line. Other than that, these parameters have little use for a normal user, and in fact may be dangerous!

Other Partitioning Software

Since DOS 4, there has been little need for aftermarket disk partitioning utilities, except in special cases. If a system is having problems that cause you to consider using a partitioning utility, I recommend that you upgrade to a newer version of DOS instead. Using nonstandard partitioning programs to partition your disk jeopardizes the data in the partitions and makes recovery of data lost in these partitions extremely difficult.

The reason why disk partitioning utilities other than FDISK even existed is that the maximum partition size in older DOS versions was restricted: 16M for DOS 2.x and 32M for DOS 3.x. These limits are bothersome for people whose hard disks are much larger than 32M, because they must divide the hard disk into many partitions to use all of the disk. Versions of DOS before 3.3 cannot even create more than a single DOS-accessible partition on a hard disk. If you have a 120M hard disk and are using DOS 3.2 or an earlier version, you can access only 32M of that disk as a C partition.

To overcome this limitation, several software companies created enhanced partitioning programs which you can use rather than FDISK. These programs create multiple partitions and partitions larger than 32M on a disk that DOS can recognize. These partitioning programs include a high-level format program, because the FORMAT program in DOS 3.3 and earlier versions can format partitions only up to 32M.

Disk Manager by Ontrack is probably the best-known of the partitioning utilities. These programs include LLF capabilities, so you can use one of them as a single tool to set up a hard disk. The programs even include disk driver software that provides the capability to override the physical type selections in the system ROM BIOS, enabling a system to use all of a disk, even though the drive-type table in the system ROM BIOS does not have an entry that matches the hard disk.

Many drive vendors and integrators gave away these nonstandard partitioning and formatting programs, which makes some purchasers of such products feel that they must use those drivers to operate the drive. In most cases, however, better alternatives are available; nonstandard disk partitioning and formatting can cause more problems than it solves.

For example, Seagate shipped Ontrack Disk Manager with its drives larger than 32M. One purpose of the program is to perform low-level formatting of the drive, which Disk Manager does well, and I recommend it highly for this function. If possible, however, you should avoid the partitioning and high-level formatting functions and stick with FDISK and FORMAT.

When you use a program other than standard FDISK and FORMAT to partition and high-level (DOS) format a drive, the drive is set up in a nonstandard way, different from pure DOS. This difference can cause trouble with utilities--including disk cache programs, disk test and interleave check programs, and data recovery or retrieval programs--written to function with a standard DOS disk structure. In many situations that a standard format would avoid, a nonstandard disk format can cause data loss and also can make data recovery impossible.

---

CAUTION: You should use only standard DOS FDISK and FORMAT to partition or high-level format your hard disks. If you use aftermarket partitioning software to create a nonstandard disk system, some programs that bypass DOS for disk access will not understand the disk properly and may write in the wrong place. Windows is an example of a program that bypasses DOS when you turn on the Use 32-bit Disk Access option in the Control Panel.

---

It is especially dangerous to use these partitioning programs to override your ROM BIOS disk-table settings. Consider the following disaster scenario.

Suppose that you have a Seagate ST-4096 hard disk, which has 1,024 cylinders and nine heads, and requires that your controller never perform a data-write modification called write precompensation to cylinders of the disk. Some drives require this precom-pensation on the inner cylinders to compensate for the peak shifting that takes place because of the higher density of data on the (smaller) inner cylinders. The ST-4096 internally compensates for this effect and therefore needs no precompensation from the controller.

Now suppose that you install this drive in an IBM AT that does not have a ROM BIOS drive table that matches the drive. The best matching type you can select is type 18, which enables you to use only 977 cylinders and seven heads--56.77M of what should be a 76.5M hard disk. If your IBM AT is one of the older ones with a ROM BIOS dated 01/10/84, the situation is worse, because its drive-table ends with type 14. In that case, you would have to select type 12 as the best match, giving you access to 855 cylinders and seven heads, or only 49.68M of a 76.5M drive.

ROM BIOS drive tables are listed in Appendix A. Most IBM-compatibles have a more complete drive-type table and would have an exact table match for this drive, allowing you to use the full 76.5M with no problems. In most compatibles with a Phoenix ROM BIOS, for example, you would select type 35, which would support the drive entirely.

Now suppose that you are not content with using only 50M or 57M of this 76.5M drive. You invoke the SuperPartition aftermarket partitioning program that came with the drive and use it to LLF the drive. Then you use the aftermarket program to override the type 18 or type 12 settings in the drive table. The program instructs you to set up a very small C partition (of only 1M) and then partitions the remaining 75.5M of the disk as D. This partitioning overrides the DOS 3.3 32M partition limitation. (If you had an IBM-compatible system that did not require the drive-type override, you still would need to use the aftermarket partitioner to create partitions larger than the DOS 3.3 standard 32M.) Following that, you use the partitioner to high-level format the C and D partitions, because the DOS high-level format in DOS 3.3 works only on volumes of 32M or less.

Most aftermarket partitioners create a special driver file that they install in the CONFIG. SYS file through the DEVICE command. After the system boots from the C partition and loads the device driver, the 75.5M D partition is completely accessible.

Along comes an innocent user of the system who always boots from her own DOS floppy disk. After booting from the floppy, she tries to log into the D partition. No matter what version of DOS this user boots from on the floppy disk, the D partition seems to have vanished. An attempt to log into that partition results in an Invalid drive specification error message. No standard version of DOS can recognize that specially created D partition if the device driver is not loaded.

An attempt by this user to recover data on this drive with a utility program such as Norton or PC Tools results in failure, because these programs interpret the drive as having 977 cylinders and seven heads (type 18) or 855 cylinders and seven heads (type 12). In fact, when these programs attempt to correct what seems to be partition-table damage, data will be corrupted in the vanished D partition.

Thinking that there may be a physical problem with the disk, the innocent user boots and runs the Advanced Diagnostics software to test the hard disk. Because Advanced Diagnostics incorporates its own special boot code and does not use standard DOS, it does not examine partitioning but goes to the ROM BIOS drive-type table to determine the capacity of the hard disk. It sees the unit as having only 977 or 855 cylinders (indicated by the type 18 or 12 settings), as well as only seven heads. The user then runs the Advanced Diagnostics hard disk tests, which use the last cylinder of the disk as a test cylinder for diagnostic read and write tests. This cylinder subsequently is overwritten by the diagnostics tests, which the drive passes because there is no physical problem with the drive.

This innocent user has just wiped out the D-drive data that happened to be on cylinder 976 in the type-18 setup or on cylinder 854 in the type-12 setup. Had the drive been partitioned by FDISK, the last cylinder indicated by the ROM BIOS drive table would have been left out of any partitions, being reserved so that diagnostics tests could be performed on the drive without damaging data.

Beyond the kind of disaster scenario just described, other potential problems can be caused by nonstandard disk partitioning and formatting, such as the following:

• Problems in using the 32-bit Disk Access feature provided by Windows, which bypasses the BIOS for faster disk access in 386 Enhanced Mode.
  
  
• Data loss by using OS/2, UNIX, XENIX, Novell Advanced NetWare, or other non-DOS operating systems that do not recognize the disk or the nonstandard partitions.
  
  
• Difficulty in upgrading a system from one DOS version to another.
  
  
• Difficulty in installing a different operating system, such as OS/2, on the hard disk.
  
  
• Data loss by using an LLF utility to run an interleave test; the test area for the interleave test is the diagnostics cylinder, which contains data on disks formatted with Disk Manager.
  
  
• Data loss by accidentally deleting or overwriting the driver file and causing the D partition to disappear after the next boot.
  
  
• Data-recovery difficulty or failure because nonstandard partitions do not follow the rules and guidelines set by Microsoft and IBM, and no documentation on their structure exists. The sizes and locations of the FATs and root directory are not standard, and the detailed reference charts in this book (which are valid for an FDISK-created partition) are inaccurate for nonstandard partitions.

I could continue, but I think you get the idea. If these utility programs are used only for low-level formatting, they do not cause problems; it is the drive-type override, partitioning, and high-level format operations that cause difficulty. If you consider data integrity to be important, and you want to be able to perform data recovery, follow these disk support and partitioning rules:

• Every hard disk must be properly supported by system ROM BIOS, with no software overrides. If the system does not have a drive table that supports the full capacity of the drive, accept the table's limit, upgrade to a new ROM BIOS (preferably with a user-definable drive-type setting), or use a disk controller with an on-board ROM BIOS for drive support.
  
  
• Use only FDISK to partition a hard disk. If you want partitions larger than 32M, use DOS 4 or a later version.

High-Level (Operating-System) Format

The final step in the software preparation of a hard disk is the DOS high-level format. The primary function of the high-level format is to create a FAT and a directory system on the disk so that DOS can manage files. Usually, you perform the high-level format with the standard DOS FORMAT program, using the following syntax:

FORMAT C: /S /V

This step high-level formats drive C (or volume C, in a multivolume drive), places the hidden operating-system files in the first part of this partition, and prompts for the entry of a volume label to be stored on the disk at completion.

The high-level format program performs the following functions and procedures:

1. Scans the disk (read only) for tracks and sectors marked as bad during the LLF, and notes these tracks as being unreadable.

2. Returns the drive heads to the first cylinder of the partition, and at that cylinder, head 1, sector 1, writes a DOS volume boot sector.

3. Writes a FAT at head 1, sector 2. Immediately after this FAT, it writes a second copy of the FAT. These FATs essentially are blank except for bad-cluster marks noting areas of the disk that were found to be unreadable during the marked-defect scan.

4. Writes a blank root directory.

5. If the /S parameter is specified, copies the system files (IBMBIO.COM and IBMDOS.COM or IO.SYS and MSDOS.SYS, depending on which DOS you run) and COMMAND.COM files to the disk (in that order).

6. If the /V parameter is specified, prompts the user for a volume label, which is written as the fourth file entry in the root directory.

Now DOS can use the disk for storing and retrieving files, and the disk is a bootable disk.

---

NOTE: The Format command can be run through the Windows Explorer within Windows 95 even on hard disks, as long as no files are open. You cannot format the drive where Windows 95 resides.

---

During the first phase of the high-level format, a marked defect scan is performed. Defects marked by the LLF operation show up during this scan as being unreadable tracks or sectors. When the high-level format encounters one of these areas, it automatically performs up to five retries to read these tracks or sectors. If the unreadable area was marked by the LLF, the read fails on all attempts.

After five retries, the DOS FORMAT program gives up on this track or sector and moves to the next. Areas that remain unreadable after the initial read and the five retries are noted in the FAT as being bad clusters. DOS 3.3 and earlier versions can mark only entire tracks bad in the FAT, even if only one sector was marked in the LLF. DOS 4 and later versions individually check each cluster on the track and recover clusters that do not involve the LLF marked-bad sectors. Because most LLF programs mark all the sectors on a track as bad, rather than the individual sector that contains the defect, the result of using DOS 3.3 or 4 is the same: all clusters involving sectors on that track are marked in the FAT as bad.

---

NOTE: Some LLF programs mark only the individual sector that is bad on a track, rather than the entire track. This is true of the IBM PS/2 low-level formatters on the IBM PS/2 Advanced Diagnostics or Reference disk. In this case, high-level formatting with DOS 4 or later versions results in fewer lost bytes in bad sectors, because only the clusters that contain the marked bad sectors are marked bad in the FAT. DOS 4 and later versions display the Attempting to recover allocation unit x message (in which x is the number of the cluster) in an attempt to determine whether a single cluster or all the clusters on the track should be marked bad in the FAT.

---

If the controller and LLF program together support sector and track sparing, the high-level format finds the entire disk defect-free, because all the defective sectors have been exchanged for spare good ones.

If a disk has been low-level formatted correctly, the number of bytes in bad sectors is the same before and after the high-level format. If the number does change after you repeat a high-level format (reporting fewer bytes or none), the LLF was not done correctly. The manufacturer's defects were not marked correctly; or Norton, Mace, PC Tools, or a similar utility was used to mark defective clusters on the disk. The utilities cannot mark the sectors or tracks at the LLF level; the bad-cluster marks that they make are stored only in the FAT and erased during the next high-level format operation. Defect marks made in the LLF consistently show up as bad bytes in the high-level format, no matter how many times you run the format.

Only an LLF or a surface-analysis tool can correctly mark defects on a disk; anything else makes only temporary bad-cluster marks in the FAT. This kind of marking may be acceptable temporarily, but when additional bad areas are found on a disk, you should run a new LLF of the disk and either mark the area manually or run a surface analysis to place a more permanent mark on the disk.

Hard Disk Drive Troubleshooting and Repair

If a hard disk drive has a problem inside its sealed HDA, repairing the drive usually is not feasible. If the failure is in the logic board, you can replace that assembly with a new or rebuilt assembly easily and at a much lower cost than replacing the entire drive.

Most hard disk problems really are not hardware problems; instead, they are soft problems that can be solved by a new LLF and defect-mapping session. Soft problems are characterized by a drive that sounds normal but produces various read and write errors.

Hard problems are mechanical, such as when the drive sounds as though it contains loose marbles. Constant scraping and grinding noises from the drive, with no reading or writing capability, also qualify as hard errors. In these cases, it is unlikely that an LLF will put the drive back into service. If a hardware problem is indicated, first replace the logic-board assembly. You can make this repair yourself and, if successful, you can recover the data from the drive.

If replacing the logic assembly does not solve the problem, contact the manufacturer or a specialized repair shop that has clean-room facilities for hard disk repair. (See Appendix B for a list of drive manufacturers and companies that specialize in hard disk drive repair.)

17xx, 104xx, and 210xxxx Hardware Error Codes

When a failure occurs in the hard disk subsystem at power-on, the Power-On Self Test (POST) finds the problem and reports it with an error message. The 17xx, 104xx, and 210xxxx errors during the POST or while running the Advanced Diagnostics indicate problems with hard disks, controllers, or cables. The 17xx codes apply to ST-506/412 interface drives and controllers; 104xx errors apply to ESDI drives and controllers; and 210xxxx errors apply to SCSI drives and host adapters.

Table 16.11 shows a breakdown of these error messages and their meanings.

Table 16.11  Hard Disk and Controller Diagnostic Error Codes

ST-506/412 Drive and Controller Error Codes

Error	Description
1701	Fixed disk general POST error
1702	Drive/controller time-out error
1703	Drive seek error
1704	Controller failed
1705	Drive sector not found error
1706	Write fault error
1707	Drive track 0 error
1708	Head select error
1709	Error Correction Code (ECC) error
1710	Sector buffer overrun
1711	Bad address mark
1712	Internal controller diagnostics failure
1713	Data compare error
1714	Drive not ready
1715	Track 0 indicator failure
1716	Diagnostics cylinder errors
1717	Surface read errors
1718	Hard drive type error
1720	Bad diagnostics cylinder
1726	Data compare error
1730	Controller error
1731	Controller error
1732	Controller error
1733	BIOS undefined error return
1735	Bad command error
1736	Data corrected error
1737	Bad track error
1738	Bad sector error
1739	Bad initialization error
1740	Bad sense error
1750	Drive verify failure
1751	Drive read failure
1752	Drive write failure
1753	Drive random read test failure
1754	Drive seek test failure
1755	Controller failure
1756	Controller Error Correction Code (ECC) test failure
1757	Controller head select failure
1780	Seek failure; drive 0
1781	Seek failure; drive 1
1782	Controller test failure
1790	Diagnostic cylinder read error; drive 0
1791	Diagnostic cylinder read error; drive 1
SCSI Drive and Host Adapter Error Codes
096xxxx	SCSI adapter with cache (32-bit) errors
112xxxx	SCSI adapter (16-bit without cache) errors
113xxxx	System board SCSI adapter (16-bit) errors
210xxxx	SCSI fixed disk errors

The first x in xxxx is the SCSI ID number.
The second x in xxxx is the logical unit number (usually 0).
The third x in xxxx is the host adapter slot number.
The fourth x in xxxx is a letter code indicating drive capacity.

Most of the time, a seek failure indicates that the drive is not responding to the controller. This failure usually is caused by one of the following problems:

• Incorrect drive-select jumper setting
  
  
• Loose, damaged, or backward control cable
  
  
• Loose or bad power cable
  
  
• Stiction between drive heads and platters
  
  
• Bad power supply

If a diagnostics cylinder read error occurs, the most likely problems are these:

• Incorrect drive-type setting
  
  
• Loose, damaged, or backward data cable
  
  
• Temperature-induced mistracking

The methods for correcting most of these problems are obvious. If the drive-select jumper setting is incorrect, for example, correct it. If a cable is loose, tighten it. If the power supply is bad, replace it. You get the idea.

If the problem is temperature-related, the drive usually will read data acceptably at the same temperature at which it was written. Let the drive warm up for a while and then attempt to reboot it, or let the drive cool and reread the disk if the drive has overheated.

---

© Copyright, Macmillan Computer Publishing. All rights reserved.