DAMNSPOT USER MANUAL Version 2.1 HISTORY Version 1.0 A few years ago I wrote a bad blocks program in SBasic on my new Kaypro II. The resulting DAMNSPOT (DS) worked well on my machine, but was riddled with machine and operating system dependencies. It is given here as Version 1.0 although I don't know that I ever bestowed the dignity of a version number on it. Version 2.0 This was a complete rewrite in Turbo Pascal and aimed at general CP/M compatibility. The most limiting restriction was that large disks were almost automatically excluded because the entire directory was stored in memory - even the unused entries. Another problem with Version 2.0 arose when testing it on a Kaypro IV with mixed disks - i.e. a Kaypro IV formatted disk (double-sided) in one drive and a Kaypro II formatted disk (single-sided) in the other. It turns out that Kaypro uses the same area of memory for the disk parameter block of both disks, copying the appropriate values in depending on which disk is selected. Since DS was referencing disk parameters via the pointer returned by a one-time Bios Select Disk call, the values referenced were always those for the currently selected disk. Version 2.1 The major change in Version 2.1 is that only live directory entries are stored in memory. This gives a limited capability on hard disk systems. Limited because there is still only so much memory and hard disks may be permitted thousands of entries. Limited also because some of the options, such as moving the disk to a new drive or copying the disk's contents require a compatible drive. How many out there have two hard disks? Damnspot should allow a bit over 450 entries on a drive that uses CP/M groups 4K in size. This limit could be moved upwards somewhat by making Damnspot much terser and recompiling it with a higher memory limit, target machine permitting. Space could also be freed by overlaying Damnspot but then it would not allow free use of all drives. The current version as distributed requires that memory be free up to D000. Other minor changes have been made. Disk parameters are scooped up and stored where they will not be subject to Kaypro's (and possibly other manufacturers') vanishing act. Some code has been simplified and some unnecessary input eliminated.INTRODUCTION DS looks for bad sectors on a floppy disk and tries to offer the user a range of options to deal with them as well as the information necessary to choose between them. What you do with a bad sector may well depend upon which area of the disk it lies in - system, directory, user - and on whether or not it is part of a valuable file. If your disk has just been formatted, or at least contains no files of importance, FINDBAD will do the job simply and conveniently. If the disk contains valuable files, DS will give you more options, information, and safeguards. To be fair, being new it probably also offers more bugs. DS reads all live directory entries into memory and keeps track of the status of every block. If it later finds bad blocks, the memory image of the directory is modified to a form that would better suit the disk. Also, it may from time to time find it necessary to write some information to unused blocks on the disk under test. These changes have no immediate effect on the disk. It will not be altered (except possibly for unused blocks) unless the user gives update permission after the scan is complete. The disk is processed in three stages: the directory, the system area (optional), and the user file area. As bad blocks are encountered the user is given options to deal with them. When the disk has been completely scanned, the user is asked whether or not to update the directory on disk, thereby putting any changes into effect. If bad blocks have been found in files it may be preferred that DS do nothing until other tries have been made to recover the data. NOTE TO THE IMPATIENT I am advised by a (tactful) friend that I write terribly turgid documentation. The truth is that I write it for me and I already know how to use the thing. If you want to jump right in, at your own risk, mount some unimportant disks with, say, DS.COM and DAMNSPOT.HLP on drive A and a disk to check in drive B. Type: A:DS and follow the cues. Control-C will give you an abort option, Control-S will scroll long displays, and Q for Quit will normally take you back to an earlier menu. A non-typical, but illustrative, terminal session is contained in DSRUN.WS which you might print out beforehand. Note that DSRUN.PIC was produced using Version 2.0 so your displays may differ slightly. LIMITATIONS 1. DS was written in Turbo Pascal (is there a nicer Pascal) with an eye to general CP/M compatibility but has only been tested on a Kaypro II and Kaypro IV. To make it work on other machines may well require recompilation and even then there are no guarantees. At any rate, no fancy screen capabilities have been used and all disk I/O is either through BDOS or BIOS. 2. Turbo allocates variable storage in high memory and the compiler has to be told how much memory is available. The distribution version of DS.COM assumes that all memory up to D000 is available. If your machine has less available, the program will tell you that it has insufficient memory and abort as soon as you try to run it. If you have this problem, it may be that your lack of memory is caused by things like Smartkey hogging up high memory. If so, do a machine reset and try DS again. 3. DS may well not work on hard disks because they have large directories and a large number of blocks. Since DS requires 32 bytes of memory for every live directory entry, 3 bytes for every block, and enough memory to hold a disk block (potentially as large as 16K) problems may arise. If they do, Damnspot will fail gracefully (I hope). 4. DS does its disk I/O through your machine's BIOS. Most machines will not report a read or write error unless a number of consecutive tries fail. Hence DS can easily miss bad sectors that are only flaky. On my Kaypro II flaky blocks are easily heard because the machine begins G R I N D I N G for several seconds. Every so many retries the disk head is homed and a reseek done to the desired track/sector. Only after several such homes will the error be reported. A 'good' block may well take fifty tries to read. DS will not detect such a problem should the read eventually work, but you can observe the block numbers displayed on the screen as reads and writes are being attempted. Blocks on which the program dwells are candidates to be forced to bad block status (which DS allows). 5. DS is big. Sorry about that. TERMINOLOGY Herewith a bit about tracks, sectors and blocks. These terms are variously used and there is little point in arguing whether one usage is right or wrong. The important point is to have some idea of whether or not the Track and Sector numbers used by one program bear any relationship to those used by another. DS displays track and sector numbers in decimal. Block numbers are always shown in hexadecimal because DU2 (a version of DUU) employs hexadecimal block numbers and is the only public domain utility that I know of that lets you go directly to any block on a disk and read and write information. P.S. DU2 calls them groups rather than blocks. Track The disk is divided into circular tracks numbered starting at zero. CP/M normally reserves one or more tracks for itself starting at track zero. The directory starts at the beginning of the first unreserved track. Several logical disks can be placed onto the same physical disk by telling CP/M that the higher disks have a large number of reserved tracks - large enough to accommodate the lower disks. Sector This term is terribly ambiguous having at least four different usages. In DS sectors are logical CP/M sectors. Each track is regarded by CP/M as divided into 128-byte sectors. These (logical) sectors start at zero on each track. This schema has nothing to do with the way data is physically laid out on the disk. Your disk system probably reads and writes in larger chunks than 128 bytes, and the physical sectors are probably in non-consecutive numeric order (skewed). It is CP/M's job, specifically that of the BIOS, to make the disk look like the model described to user programs. Two programs of great usefulness in detecting and dealing with bad blocks are DU2 and your Format/Verify utility. The way that they use sector numbers is described briefly below. DU2 (or DUU) This displays both a 'logical' and a 'physical' sector number. For some reason or other my copy of DU2 counts logical sector numbers beginning at one rather than zero on each track. So, to convert a DS sector number to a DU2 logical sector number, just add one. Format/Verify Utility I am assuming that you have such a thing. If probably is totally hardware oriented and makes no attempt to speak CP/M. My Kaypro has ten true physical sectors per track, each 512 bytes long. My F/V utility reports bad sectors as numbers in [0,9], whereas both CP/M physical and logical sector numbers are in the range [0,39]. Some knowledge of your particular machine is needed to convert between the two systems.Block CP/M allocates disk space to files in blocks, each of which consists of a number of (logically) contiguous sectors. This is also frequently referred to as a CP/M group, which term has the advantage of being unambiguous. It is not used here because I do not wish to redo all the documentation and source code. The block size is a compromise between fast disk access (big blocks better) and the desire to avoid wasting disk space (small blocks better). Each directory entry in CP/M governs a maximum of eight or sixteen blocks, so big files need several entries. My Kaypro II works with 1K blocks, double sided Kaypro's use 2K blocks, and some systems with large hard disks may use blocks as large as 16K. Blocks have no meaning in the system and directory areas of the disk but apply rather in the user area. The terms bad block and bad sector are used more or less interchangeably here. Note that what actually goes bad is a physical sector, which is usually intermediate in size between a CP/M logical sector and a CP/M block. BAD BLOCK/SECTOR CONSIDERATIONS Since some bad sectors are badder than others, several cases have to be considered. Bad Sector in Directory A bad sector in the directory is fatal as far as DS is concerned. If an error is encountered processing of the disk will be immediately discontinued. Processing will also be discontinued if the number of live entries is greater than the amount of memory available for them. If the directory is bad, no intelligent processing is possible and your Format/Verify program is a much better tool for dumb bad sector scanning. If the directory area is bad, the disk will eventually require reformatting, which of course will erase any information on it. The best hope for recovering information is to try the same disk on a different drive or a different machine. There are several public-domain programs around that may help you to recover files even when part of the directory is bad. The best that I have found is DU2 but it does require a fair degree of knowledge to use it effectively. Even with the necessary expertise, recovering information from a disk with a corrupt directory is no fun. It is usually quicker (when possible) to regenerate the information by other means. Bad Sector in System Area Checking the system area is optional with DS. This is so for two reasons: 1. Some machines format the system tracks differently than the rest of the disk. In this case DS may well not function at all in the system area because its only view of the disk is that provided by CP/M, which only concerns itself with the directory and user area. 2. Some machines place two or more logical disks on a physical disk by declaring the higher ones as having a large number of reserved tracks (reserved for the lower logical disks). DS will simply interpret this as a very large system area to scan. A bad sector in the system area of the disk is only bad if you want the disk to be bootable; it saves a lot of trouble if all disks are. DS tells you about any problem and gives you a chance to dispense with checking the rest of the system area. [UNUSED].BAD and BLOCKS.BAD Blocks in files named [UNUSED].BAD or BLOCKS.BAD are automatically considered to be bad and are not checked further. DS places new bad blocks into [UNUSED].BAD but recognizes BLOCKS.BAD also in order to remain compatible with an earlier version. If you want blocks in these files to be rechecked you must first erase the current file. This is a little tricky since they may be Read Only and also System type so that you may not even know they exist (they will not appear in a DIR listing). Use STAT or any of several other utilities to remove the Read Only status and then erase it. E.g. STAT *.BAD $R/W ERA *.BAD DS will always create [UNUSED].BAD as User 0, Read Only and System type. I don't know what various versions of FINDBAD do. In deleting any [UNUSED].BAD or BLOCKS.BAD file bear in mind the reason that DS does not check its blocks. Flaky blocks can slip by DS or FINDBAD and once in the bad blocks file it seems a pity to give them a chance of escaping. The only satisfactory way of restoring bad blocks to life is by reformatting the disk, and even then you have to wonder. Bad Block in User Area but not in File DS will add any bad block that does not belong to a file to the [UNUSED].BAD file, creating that file if necessary. The change does not actually take effect until you give update authorization on conclusion of the scan. If the disk directory or memory is full and more room is needed for another [UNUSED].BAD entry, you will be informed of the problem and the block will not be covered. Bad Block in a File This is not good. Naturally you don't want to throw this into the [UNUSED].BAD trash bin without some say in the matter. It might be an unimportant file or it might be your only copy of your electronic little black book. (That's silly! If you had one what would you be doing checking disks for bad blocks?). DS will tell you the situation and the name of the corrupt file. It will then give you the following options: Quit Disk If the file is important this may be the best thing to do, although Retry deserves a chance. If Retry fails, quit DS and try to PIP the file, a few times if necessary. You might get lucky and wind up with a readable backup. Even better may be to first change the disk to another drive before the PIP attempt. Even a tiny difference in the disk head sensitivity or alignment can make the difference needed. Retry DS will try again to read the block. If it succeeds, DS will look for an unused block, save the recovered information there, and confirm the save by trying to read back the data from the new block. If that block is in turn bad, DS will look for another and so on. If the block is successfully saved, the memory image of the directory is modified to hook the new block into the file and stuff the trouble block into [UNUSED].BAD. DS does not alter the directory on disk at this time - only its memory image. Hence the file will not truly be fixed unless you give directory update approval after the whole disk has been scanned. Substitute a Readable Block In this case DS fakes a successful read of the problem block, substituting a distinctive pattern of Ascii characters. It then proceeds as with the Retry option above, attempting to hook a readable block into your file and banish the problem block to limbo. Of course the new block has garbage in it so you are unlikely to be wildly happy. However, what remains of your file will be editable (assuming it was originally a text file) and the clobbered parts will be clearly visible. Ignore If you tell DS to ignore the bad block it does just that, although it will remember it, remind you of its existence in the final summary, and make no attempt to use it as a replacement for other bad blocks.GENERAL OPERATION OF DS DS is built in a loop so that several disks can be processed without leaving the program. Furthermore, DS resides in memory so once it is loaded the DS disk can be removed and all drives used for other purposes. On the other hand, DS keeps its help messages in a file called DAMNSPOT.HLP. If this file is not on- line, the help facility will be unavailable. DS is menu driven so after a little experience the help file should be unnecessary. Disk Drives DS tries to do some smarts to prevent catastrophes involving sending output to the wrong disk or to an incompatible disk. To this end it examines the characteristics of all your disk drives - not just those you mention. The price paid is that every disk drive must have a disk in it or the program will hang. If one of your drives is in at the shop, DS won't work. A caution to people who have machines that automatically distinguish between different formats of disks. The Kaypro IV does this but only on the first select of a drive following a Warm Boot or Reset. DS does a disk reset and re-examines disk parameters each time a new disk is specified but that will not detect a change in disk format. In short, do not mix and match during one DS session. Prompts Many of the prompts in DS are of the general form: Would you like to see my etchings < Y:es N:o *H:elp > : The choices in the example are selected by pressing either Y, N or H - either upper or lower case. The asterisk indicates a default response obtainable by simply pressing the Return key. Aborting and Scrolling At most points in the program, pressing Control-C will cause a prompt asking if the user wishes to leave the program. It will also mention if changes are pending - i.e. whether the disk would be changed if operation were to continue to the Directory Update stage. If the user confirms the abort, DS will terminate in a Warm Boot. To quit a particular disk, rather than the whole program, a few Q:uits, backing up through menus, is normally available. Output to the screen can in most entries be controlled with the Control-S stop scroll feature. This applies only to parts of the program that generate a lot of screen output; it does not work on a character by character basis as in standard CP/M. Help Whenever Help is offered as a choice, it will only be useful if the file DAMNSPOT.HLP is on-line. Incidentally, the structure of the Help file is very plain and can be tailored to your tastes. The only things that must not be changed are the %TITLE lines and the closing & character. A word of advice. DS has quite a few options, which means quite a few questions that need to be answered. When learning to use it, keep a copy of DAMNSPOT.HLP somewhere on-line. If it is on any drive, DS should find it. If it is not available, any request for help will simply yield a message saying that the help file is absent. GETTING STARTED DS first asks for the drive to be scanned and does a disk reset so disks may be changed if desired. It also asks whether you want to scan the System area. DS then gets disk parameters from CP/M for all drives and attempts to seize enough memory to accommodate an array of status information for each block, a buffer as large as one block on the disk, and enough room to hold a copy of the test disk's original directory checksum vector. The number of extra bytes of memory required is approximately: directory_sectors_to_check + 3 * number_blocks + blocksize The rest of memory (up to a compile-time specified limit) is then allocated as storage for live directory entries, including any new entries needed for [UNUSED].BAD. DS will announce the maximum number of directory entries permitted. For floppy disks there should be no problem but for large hard disks DS may well refuse (gracefully) to continue. Next DS figures out where the system, directory and user areas are, based on the CP/M info. It then proceeds in the following order: 1. Scan the directory, building a memory image of all live entries. If a read error is encountered, or if the number of entries is too large, processing of the disk will terminate with an appropriate error message. 2. Scan the system area if requested. If errors are encountered the user can quit the system area scanning or continue it. In any event, DS will subsequently proceed to the user area. 3. Scan the user area. Whenever a bad block is detected, inform the user of the fact, together with the name of any file within which the bad block lies, and give him some options (Retry, Ignore, Substitute). Although DS cannot automatically detect a flaky block, it does display the block numbers and containing file names as it processes them. If the machine seems to dwell on a particular block, it can later be forced into the bad blocks file by the user (see below). 4. Allow the user to force specific blocks into the bad blocks file, warning of any consequences to files. This facility is very useful. One can often tell that a block is on the verge of bad by the fact that several attempts are needed to read it. I can hear the drive misbehaving and see the block number lingering on the display. But if BIOS eventually reads it (mine tries many times with several disk homes) then DS won't know about it. 5. Display a summary of the disk's status on the screen. 6. This step is bypassed initially if there are changes pending. Ask the user if he wishes to quit or proceed to the directory options menu (step 7 below). 7. Control comes directly here from step 5 initially if changes are pending. This menu allows the user to Update the directory, Quit back to step 6, and several other options designed primarily to try to recover something in the (PANIC!) situation where a write error clobbers the directory as you are updating it. This is seemingly a low probability event, but one does spend more time with a bad blocks program when strange things are happening with the drives. Other facilities based on the (possibly changed) directory image in memory, are: a. Send the directory information to a disk file in Ascii or Binary. b. Send the directory information to the terminal or printer in Ascii. c. Try to rewrite the directory, with or without first moving the disk to a new drive. If the write error has clobbered the sector's formatting information this cannot work. On the other hand, it might. d. Copy the directory info, from memory, and the block info, from disk, to another drive. A NOTE ON DU2 This is a fantastic program. I think the true name is DUU (Disk Utility Universal or something like that) but my copy, acquired from a friend, was named DU2. The important thing about DU2, as opposed to programs like Disk Doctor or Edfile, is that it lets you do disk I/O anywhere on the disk, regardless of whether the area you are working in belongs to a file or not. There doubtless are exceptions to that in the case of machines that format the system area differently than the rest of the disk, but most don't. It couples this ability with a wide range of things you can do and about five screens full of help information. DU2 can be confusing despite that, being primarily a specialist's tool. Given full rein to do anything on the disk anywhere you want means that you have to supply the smarts - where you want to be and what you want to do there. I intend here to point out only two of DU2's capabilities that might be relevant to recovering information from a bad disk. The precise syntax will be yours to discover. Trying to Read a Bad Block Suppose you know or suspect that block A7 is bad. With DU2 you can go directly there (DU2 calls them groups rather than block) by typing GA7. Then the command G. It still remains to patch up the directory entry entry for the bad file. That is a matter of locating the entry in the directory (the F and M commands may help) and changing the block number A7 to B5 using the CH command followed by a W command to write the changed info back to disk. When you are dealing with blocks (groups) you usually don't have to worry about track and sector numbers. However, if you do wish to relate them between DS and DU2 remember that DU2's logical sector number is always one bigger, if a later version has not corrected this. The very first sector on the disk is Track 0, Sector 0 to DS and Track 0, Sector 1 to DU2. Recovering a Directory Clobbered by DS When worst comes to worst, as it must (Borel-Cantelli) you will tell DS to update a disk directory and it mention the word CATASTROPHE on your screen. What do you do? I assume that repeated tries to update the directory all fail and that recovering the files on the disk is very important because, otherwise, what follows may not be worth it. The first thing to do is to use the Send option in DS to create both a binary and Ascii image of the directory on disk. Be careful to specify a drive other than the one holding the trouble disk. At the very least, get a paper copy of the directory information using the S:end and P:rinter options. These files may not be much use to you directly but they could be essential to any wizard that you manage to con into helping recover your files. Next, use the C:opy facility in DS to copy blocks from the trouble disk and the directory from memory to another unimportant disk. Note that any files on it will be clobbered. In the best of circumstances this copy will be enough. In other circumstances it may be somewhat wanting. Note any messages displayed on your screen during the copy. Now if that doesn't solve the problem, DU2 may help. Please bear in mind that the following is conceptual; I do not have a disk with an unreadable directory to try it on so it may not work. It all depends on whether DU2 lets you get at a disk whose directory cannot be read. 1. While still in DS, use the directory options menu to save the directory information to a binary disk file on a good disk. 2. Use your Format/Verify utility to format just the directory track(s) of your disk. If you don't know which track(s) the directory occupies, use the G0 option in DU2 to find out. It will display the corresponding track and sector number, which is the start point of the directory. 3. Now use DU2 to scoop the PANIC.BIN file into memory and deposit it onto the freshly formatted directory track. You will need to use the L command to switch back and forth between the two disks, the G command to position yourself, and the G commands to read and write blocks. The M or F commands will tell you where PANIC.BIN resides; for example purposes assume that it occupies blocks 34 and 35 on drive A whereas the trouble disk is in drive B. Your side of the dialogue might look something like this: DU2 Run DU2 LA Log in the A drive G34 Go to the first group of PANIC.BIN G Deposit the first group G1 The directory covers two blocks in this example >G Deposit the second group X Exit DU2 Y Confirm warm boot. The above is pretty bare bones. It should be noticed that if the directory shares a track with user blocks, those blocks will be erased during the reformatting. It might be possible to pick them out and hide them away elsewhere with DU2 but then again, DU2 may just refuse to log in the disk. I don't know. At any rate, if you know which blocks are in jeopardy (block 4 on a Kaypro II) you can get DS or DU2 to tell you what is in it. All of this may sound a little esoteric, but if the worst happens to you, have DS generate a PANIC.ASC and a PANIC.BIN. An expert may find them very useful. P.S. In a case like this, half an expert on your machine is worth ten experts on other machines. A NOTE TO KAYPRO II OWNERS The FORMAT program supplied by Kaypro (CP/M does not create these) describes bad sectors by a track number (0 - 39) and a sector number (0 - 9). The reason for the restricted range of sector numbers is that FORMAT thinks in terms of the true physical sector size on the disk, which is 512 bytes. Fortunately the numbers can be converted quite easily to the logical sector numbers used by DS and DU2. The idea is that Physical sector 0 contains logical sectors 0,1,2, and 3; Physical sector 1 contains Logical sectors 4,5,6 and 7; and so on. An example should make this clear. You format and verify a disk. FORMAT tells you that there is a bad sector at Track 37, Sector 6. Since the verification routine only gives a sector one chance to be read, rather than thirty-odd, this is a more sensitive way of finding flaky sectors. The first Logical sector in Physical sector 6 is 24 (4 * 6). Also, since DU2 kicks its sector numbers up one, the DU2 sector number is 25; the track number is the same (37). So, tell DU2 to go to Track 37, Sector 25 with the commands: T37 S25 and DU2 will tell you what Group you are in. On my machine it says: Group = B7:00 which means that you are in group B7. The :nn is the relative 128-byte logical record within the group - i.e. here you are at the very start of group B7. Since a Kaypro II physical sector (512 bytes) takes up half a group (1024 bytes) you know that the bad physical sector is contained within group B7. You could then use DS to lock this block out of action.