Best Practices: 8mm Exabyte Drive

Bandit LOAF

Long Live the Confederation!
The collection includes between ten and thirty Exatapes (8mm, 112m). These are likely to contain Origin's earliest surviving archives, material from the early 1990s (Wing Commander, Wing Commander II, Ultima VII, etc.)

Our team is in the process of putting together two desktop PCs which include SCSI adapters and compatable ExaByte drives. We believe that the tapes are archives of Novell servers, *possibly* created with a backup utility called MaynStream.

What do we do? My current theory is that we can use the Linux 'dd' command to simply copy the contents of each tape into a file for later study and recovery - the goal being to get the data into our hands, given the limited amount of time and resources available at Electronic Arts. Am I correct in believing that if worse comes to worse we can simply put the copied bits back onto another tape which we can recover at our leisure?

Questions:

* Can we do this in Windows? I have never used Linux and adding a Linux partition, drivers, etc. worries me to no end. I am much more comfortable in Windows or DOS. Can we use one of the utilities listed here to do exactly this, but in Windows XP?: http://www.geocities.com/lc_castro/winnt.htm (either rcopy or the tape drive specific tool).

* If not, how do we set up a Linux partition, find drivers, etc. - and then how do we use them and transfer data back to our central archive.

* How do we maximize the speed, both technically and procedurally, given ~8 hours a day x 5 days at the EA Mythic offices?
 
Here's an urgent question: how long will it take to read an entire 112m tape? I need to know how many machines we will need working on this to get it done within the five day time limit (assume there are 40 tapes).
 
Well, both rcopy and tapecopy are stated as being about to copy from tape devices, but it doesn't say which are supported, so I am not sure. It would be good if you could access a few dumby ones to see if and how it works. I would guess, though, that it should work as long as you can hook up the drives.

For your second question, I did a little initial online research and found this: http://tldp.org/HOWTO/Partition/intro.html#AEN57

This howto deals with several strategies on creating a dual-boot of Windows and Linux:
http://www.tldp.org/HOWTO/Install-Strategies/

This howto deals with backing up of a Linux drive/'machine' to one that is MSDOS:
http://www.ibiblio.org/pub/Linux/docs/HOWTO/other-formats/html_single/Backup-With-MSDOS.html

Probably it would be best to have both ready just in case the rcopy or tapecopy are unable to access all of the tape devices.

I will try and do some more in-depth online research to help you guys out with this stuff, but my knowledge is sorely limited. I think I can get into understanding some of this stuff, though, with a little time and effort spent. (and sufficient wiki articles to get some basics)
 
Back in the late 90's Exabyte use to offer a service where they would pull data off their tapes for you. I don't know if they still offer this now that they were bought out by Tandberg Data. Their site is vague about this but I would call them and see. Unfortunately I never kept their business card and they haven't be in since you posted this. If they do I'll ask them.

If they still offer the service it will be worth whatever they charge to have it done professionally.

If you are going to do it yourself make sure you clean the tape drive first. You should be able to find tape cleaners for it still.
 
I second Shades2585's last comment. In fact I would suggest that you make sure that you have a small supply of cleaning tapes and any cleaning solutions that the drive manufacturer recommends on hand for each type of drive you plan to use before you get to the archive site.

Most any type of magnetic media is going to produce oxides and collect contaminants as time passes, even if the media is just sitting on a shelf. In fact, the oxide accumulation may very well be greater if the media is unused since the oxides don't get transferred to the read/write heads in a drive as they would during regular use. The oxides and contaminants prevent read/write operations from completing successfully, and they can cause tapes to break or jam and this in turn can cause tape drives to fail.

You may need to perform cleaning operations more frequently than the manufacturer recommends, and it may even be necessary to perform cleaning operations after every single tape cartridge is copied. I worked as an operator in a facility that processes data from orbiting satellites, and we had standing orders to clean drives after every open reel or tape cartridge was processed on older systems. In some cases, the physical degradation of the media and degradation of the magnetic signal was so bad that you got a different result every single time you processed a particular reel or cartridge, no matter how many times or how well you cleaned the tape drive.

Concerning LOAF's first question about whether the focus should be on capturing binary images of files and backups, I would say that LOAF hit the nail on the head: get the files and backups on new and reliable media, and you can worry about making sense out of the data at a later date. To be even more specific, I would (if time and equipment allows) focus on getting the files and backups copied to hard drives and/or solid state media (like USB drives) and defer any attempt to store the data on something like high-capacity optical media or tape media (CD-ROMs/DVD-ROMs/Blu-ray/etc.) until you get back from the expedition to the archive site. Getting data onto the latter types of media WITHOUT ERROR is too time consuming a task for this type of expedition.

I would also suggest, if time allows, that the expedition make a point of performing verification operations every time something is copied. Generally speaking, this will DOUBLE the time required to copy a particular tape cartridge, so the expedition will have to make a judgment call once they reach the archive site as to whether this is feasible based on the number of tapes to be copied, the time required to copy a single tape with and without verification, and the time available at the archive. I haven't used the 112m tapes myself, so I can't answer that particular question. The reason I make this suggestion is that too many people who use backup software don't grasp the fact that no medium or backup process is 100% error-free. Consequently, they aren't willing to spend the necessary time to perform a verification/validation of the backup, and they can't understand why their restore processes fail due to a corrupt backup. I can't make specific recommendations on the exact commands or procedures to perform the verification operations at this point because they will depend on what OS and what specific software and commands are used to extract the data from the archive's tapes, but the people doing research for the project should keep an eye out for such commands/procedures in addition to the copy commands/procedures.

This leads to LOAF's question about whether the copies can be done using Windows. Based on my experience, it is probably possible to perform the copies and verification operations on a wide number of platforms including Windows, The real question is what specific commercial or freeware software is available at this date. Most drive manufacturers are going to include some basic utilities for particular OSes along with drivers on floppies or CD-ROMs (or perhaps DVD-ROMs) when the drive is sold, and that is what the expedition should look into first. If someone is loaning a drive to the expedition, be sure to ask if they can provide copies or originals of ANY software that came with the drive if at all possible. Freeware like the one suggest by LOAF may very well work, but you really have no way of knowing if it can be used reliably without testing it, so it would pay to lay your hands on at least one tape cartridge that you can use for testing the hardware configuration and any software you are considering. In addition to compatibility issues, there is always the possibility that someone may be trying to spread malware by releasing a freeware product, so make sure that you perform virus/malware scans on any freeware before you bring it anywhere near your systems that you are taking to the archive site. I would also, as a matter of principle, give preference to freeware where the source is included along with instructions on how to build the binaries from scratch. Finally, it may very well be possible to find new or used third-party software for use with the Exabyte drives, although if Windows 98 SE is the target system, I suspect that newer software releases may very not have support for that OS while WinNT/Win2K/WinXP/Vista are much more likely to be supported.

As far as the use of Linux distributions goes, I do have a few comments. First, be aware that a number of recent Linux distributions include a "Live" version that allows you to boot from a CD-ROM or DVD-ROM without installing ANYTHING on an existing Windows, Linux, or sometimes even a Macintosh system (depending on the hardware). These "Live" versions use existing swap partitions on Linux systems and free space in Windows partitions for swap files. These "Live" versions necessarily take longer to boot since they have to do comprehensive hardware detection, but you wind up with a working Linux distribution with little or no difference from a regular Linux installation, and you can perform a regular Linux installation using the "Live" CD-ROM or DVD-ROM as well. The major drawback is that you probably will need at least 512 MB of memory to get a usable system with a "Live" distro, and more memory is to be preferred, so your Win98 boxes may not be good candidates for these distributions if they are short on memory. Looking at my spreadsheet, I have the following Live distributions:

SuSE Linux 9.2 LiveCD Gnome
SuSE Linux 9.2 LiveCD KDE
SuSE LiveDVD 9.3
openSUSE 10.3 Live GNOME
openSUSE 10.3 Live KDE
Fedora 7 KDE Live
Fedora 7 Live
Fedora 8 Live KDE
Fedora 8 Live GNOME

(GNOME and KDE are two different desktop environments that give a different flavor to the Linux user interface, by the way.)

Wingnuts could possibly help the project by supplying the expedition team with copies of as many different versions as possible or by doing live installs and determining if their particular distributions have support for the specific tape drives and interface cards that the expedition plans to use. I would be glad to burn copies of any of the above distributions for the expedition's use.

The latest and greatest Linux distributions may not be the best for the expedition's use if the machines it plans to use are somewhat older pieces of hardware with comparatively less memory. In that case, you should be aware that many older and most newer distributions include one or more boot managers such as LILO or Grub that can be used to set up a dual or multiple boot environment at the time that Linux is installed on an existing Windows machine. Some distributions such as Caldera Open Linux include Windows-based partition management software such as Partition Magic in addition to LILO and/or Grub. Third-party partition management software such as Partition Commander can also be used to allow dual or multiple booting on an existing Windows machine. I would suggest that if you do plan to perform any actual Linux installations that you make a point of creating (and verifying) an external backup of any existing Windows partitions prior to the Linux installation in order to easily get back to your starting point and also to allow you to do the same at the archive site in case of hardware failure (hard drive), etc. I would also suggest that you reserve some free space in an extended partition on the Windows machines on which you might install Linux for the Linux OS and a swap partition. If it turns out that you don't install Linux on these machines, it is fairly easy to create FAT32 partitions with this unallocated space which can be used by Windows in addition to existing partitions. Check the documentation on the Linux distributions you are considering for the correct amount of space to reserve (this can vary greatly depending on the type and amount of software you choose to install in the Linux partition, and it can run from a few hundred megabytes to several gigabytes).

Most Linux distributions don't have any problem reading and writing to FAT32 partitions, so you can either make a point of saving tape cartridge data to FAT32 partitions and using Windows to perform your data transfers from dual boot machines to other machines or you can probably use FTP transfers for this purpose. Most of the Linux distributions that I have used automatically set up Linux to function as an FTP server in addition to providing one or more FTP clients. Versions of Windows since Windows 95 have a command line FTP client built in, and there are plenty of decent freeware GUI FTP clients that have been available in the last decade. Most Linux distributions also have the ability to run a SAMBA client/server that makes the machine look like a Windows network share in a Windows network which will allow you to transfer files as well. So there are quite a few ways of getting files from a Win/Linux box to other machines.

One of the most helpful things the expedition team can do is post the configurations of the boxes it plans to take to the archive site including model numbers for relevant hardware and whether or not original (or updated) drivers and other utility software is available from the manufacturer or the source of the component if it is on loan.
 
Linux DD backups.

Get some fat hard drives like 1tb or whatever size the tapes are. Install any copy of linux, you don't even need the gui part. All you need to do is get the tape connected up and have a driver to read it. A /dev/sda /dev/sdb etc device node should be created for the drive automatically as soon as it's plugged in. If you then run dd if=/dev/sda of=/file.iso as root. That will rip the entire contents of that drive including any filesystem on it to the /file.iso file You can then perform your data recovery using that file. You can also transfer that file to another tape using a reverse method so dd if=/file.iso of=/dev/sd(tape drive). Since this is obscure old tech, a copy of solaris might also be advisable to take along since it might support some of the weird old hardware linux doesn't. Fat32 is going to be a bad choice if you want to archive data because it has a filesize limit of 4gb per file. If you are unable to access inividual files and need to backup an entire tape etc you'll need at LEAST ntfs to store the image of the tape as it'll be over 4gb in size. you mentioned several gb sized tapes? anything over 4gb you need a modern filesystem for.
 
This is all pretty overwhelming.

The tape drives are Exabyte 8505s, the adapters are Adaptec 2940s.

Do we need special drivers for Linux? We plugged the first one into a Win98 machine and it worked right out of the box (Windows Backup restores an unrelated test tape correctly - unfortunately, that setup is at Joe's museum and he hasn't tested that freeware yet).

Linux sounds like a good solution, but I don't have any experience with it... and, as you guessed, the specs for the tape drive PCs are pretty low. I switched to buying laptops some years ago and am putting these together out of scrap (the lesser of the two is a Pentium III/650 with 128 megs of RAM).

On the other hand, it means that I have no problem blanking whatever's on them right now - I just don't know how. Do we have any local Linux gurus who might be willing to come set something up (either in advance or working on site - both are welcome.)

Can anyone think of a cost-effective way to put more desktops into service here? We're running low on cash and time.

(Here's another concept that Popsiclepete wasn't fond of - what about a PCMCIA SCSI card? We could plug several external Exabyte drives into several higher powered laptops that way... at least, in my head.)
 
Tandberg Data seems to have decent support for the discontinued Exabyte gear. Here is the support page for the Exabyte 8505 drive:

http://www.exabyte.com/support/products/get_products_support.cfm?prod_id=109

This page lets you download a copy of the product specifications, and the document seems to answer LOAF's question about the length of time required to copy the entire contents of a 112m 8mm tape:

http://www.exabyte.com/support/products/manual.cfm?id=302

Table 1-3 on PDF page 19 shows a maximum sustained data transfer rate of 500 kbytes/sec with a maximum of 5.0 GB per tape in uncompressed format on a 112m tape while the maximum sustained data transfer rate of 1 Mbytes/sec with a maximum of 10.0 GB per tape in compressed format on a 112m tape. If the tape was completely full, and your interface is transferring at the maximum possible rate, it would take 10,000 seconds or 2.78 hours to transfer the contents of a tape in either circumstance (compressed or uncompressed), so that is your upper bound on the time required for one tape.

This page has the technical specs for the Adaptec AHA-2940 card (click the Technical Specifications tab):

http://www.adaptec.com/en-US/products/host_tech/scsi_drives/AHA-2940AU/index.htm

The maximum data transfer rate for this SCSI adapter is 20MB/sec, so you shouldn't have any trouble sustaining the maximum transfer rate of the Exabyte drive if the SCSI host adapter is configured properly.

The Software/OS compatibility link on the Tandberg site shows applications designed to work with the 8505 Exabyte drive on the following OSes: AIX, HP-UX, Linux, Mac OS, SCO Unixware, IRIX, Solaris, Tru 64, Windows, and NetWare, so Linux drivers apparently exist.

The 5 and 10 GB maximum tape capacities given above also mean that Darkmage's comments about the 4 GB limit have to be taken into consideration. In other words, even an uncompressed tape puts you at risk of exceeding Win98's file size limit.
 
The 5 and 10 GB maximum tape capacities given above also mean that Darkmage's comments about the 4 GB limit have to be taken into consideration. In other words, even an uncompressed tape puts you at risk of exceeding Win98's file size limit.

Long shot, but I don't suppose Windows 98 understands NTFS-formatted hard drives plugged into it via a USB tray, does it?
 
windows 98 has trouble with NTFS in general. is there any reason you can't format the drive to fat 16 or fat 32?
 
Fat32 wont store a 5 or 10Gb image. I seriously doubt that anything backed up that long ago will have files anywhere near that large, but since we don't know the format of the tapes, we should be prepared to rip the raw data off the tape and interpret it later. There is freeware which claims to allow windows 98 to use NTFS, but it would need to be stress tested beforehand.

You could pick up a 1Tb network drive - with network drives, windows does not care about the filesystem. Then again, the restore software may want to write directly to the local drive, and having to construct a LAN on site is one more thing that can go wrong.
 
Table 1-3 on PDF page 19 shows a maximum sustained data transfer rate of 500 kbytes/sec with a maximum of 5.0 GB per tape in uncompressed format on a 112m tape while the maximum sustained data transfer rate of 1 Mbytes/sec with a maximum of 10.0 GB per tape in compressed format on a 112m tape. If the tape was completely full, and your interface is transferring at the maximum possible rate, it would take 10,000 seconds or 2.78 hours to transfer the contents of a tape in either circumstance (compressed or uncompressed), so that is your upper bound on the time required for one tape.

Okay, here's a question: what if the tape *isn't* full? Will we save time there, or will we still have to go through the entire tape? It's unlikely that these tapes each have five gigs of data on them, since they're dated project archives rather than company-wide backups (ie, everything from Ultima VII as of May 22nd).

(We face the same issue with the VHSes - 50 VHS tapes is *theoretically* as many as 400 hours of video... but in practice a majority of them are things like four minute news segments or twenty minute store window loops.)

Long shot, but I don't suppose Windows 98 understands NTFS-formatted hard drives plugged into it via a USB tray, does it?

I don't think we need Win98 - I'll stick XP on my SyQuest/tape computer and I'll pray that Joe can manage the same on his tape-only one.

nearly three hours per...uh oh...and approx. how many were there?

My (high, possibly to the point of twice as many as in reality) estimate based on the first survey was 40 tapes.
 
Okay, here's a question: what if the tape *isn't* full? Will we save time there, or will we still have to go through the entire tape? It's unlikely that these tapes each have five gigs of data on them, since they're dated project archives rather than company-wide backups (ie, everything from Ultima VII as of May 22nd).

The drive puts an End-of-Data (EOD) marker at the end of the last piece of data on the tape, so it's a pretty good bet that any software or OS commands will not read past that marker.

(Here's another concept that Popsiclepete wasn't fond of - what about a PCMCIA SCSI card? We could plug several external Exabyte drives into several higher powered laptops that way... at least, in my head.)

I'm with Popsiclepete. I wouldn't want my laptop to undergo that kind of thermal torture for hours on end. Still, if you don't have the desktop boxes on hand....
 
For essentially the same thing as 'dd' under windows, find a copy of the 'rawrite.exe' utility. I know that Slackware Linux's install CDs include them. A simpler syntax than dd, and serves many of the same purposes. You can probably find it at ftp://ftp.slackware.com
 
My DOS days are sadly behind me, but if this is the same rawrite then didn't it just image floppy disks?
 
I sifted through the Windows 98 Resource Kit to see if there was any information on tape drives or Exabyte in particular. This is what I found:

From Chapter 10 Disks and File Systems (p. 479):

Backup does not detect your tape drive.
Microsoft Backup does not work with all tape drives. If you install Backup and get a message that your tape drive has not been detected, click the Help button and connect to Seagate’s Web site using the provided URL to see a list of supported tape drives.

Tape drive detection.
Plug and Play software does not detect tape drives. This is a function of the drivers provided with the Windows 98 Backup program, or the software that came with the drive. As a result, the tape drive may appear as an unknown device in Device Manager. If your backup software does not detect your tape drive, contact the company that wrote the software.

Tape drive compatibility.
If your tape drive does not appear in the list of supported drives located on Seagate’s Web site (click the Help button and connect to Seagate’s Web site using the provided URL), contact your tape drive manufacturer for information about backup software that you can use with Windows 98.

From Chapter 24 Device Management (p. 1160):

A SCSI or IDE tape drive or scanner does not show up in Device Manager.
Windows 98 does not assign drive letters to tape drives and scanners, because they have no drive to assign a letter to; that is, they have no official hardware class designation. Therefore, they might appear as Unknown Devices in Device Manager. (If you run Microsoft Backup, however, they will be moved to a new device class and will not appear as Unknown Devices.) After you start Windows 98, it asks if you have a driver for these devices. If you have Windows 98 drivers, click Yes, and then type the path to where the drivers are located. To use existing real-mode drivers, click No. Windows 98 will continue to recognize and support these devices although they are listed as Unknown Devices.


So there is no real tape drive support in Win98, and apparently Seagate wrote the Microsoft Backup software, and Microsoft deflects all support requests to them. This does not inspire confidence, and it makes me wonder what kind of tape drive support is available for legacy tape drives in Win2K/WinXP/Vista.

I also came across the following page on the Tandberg Data site that gives instructions for integrating a tape drive in a Linux system:

http://www.exabyte.com/support/online/kb/display.cfm?id=215

I would guess that this information was generated about 8 or more years ago. It's good to see that the 8505 drive doesn't seem to have any real idiosyncrasies compared to some of the other models.
 
I can confirm that Windows 98 does recognize the drive and that Microsoft Backup can restore data from them... but in all likelyhood not in the fashion that we want.
 
Back
Top