Preserving Oral History Recordings

Like many national, regional and research libraries, the National Library of Australia (NLA) is actively facing a wide range of challenges associated with using digital technology to improve access to its collections. In at least one area this is not discretionary: the Oral History collections are intrinsically affected by technological change and without moving to digital, access to the collections will be lost as analogue audio technology loses market support. The NLA is implementing a strategy to bridge the difficult transition between fully analogue and fully digital environments.

The Oral History collections cover a wide range of material collected (with varying levels of activity) over the past 50 years. In recent years the focus has been on biographical interviews with prominent Australians, social history projects and folklore collecting. Currently holding about 30,000 hours of original recordings on reel to reel analogue tape, the collection is important and well housed, but inadequately preserved. Only about a quarter of the material had been copied to acceptable preservation standards by the end of 1995, with similarly low levels of transcription and cataloguing. If preservation is ultimately about accessibility, we had (and have) very large challenges even without an enforced change in the technology base of the collection. This responsibility is managed by our Sound Preservation and Technical Services unit (SPATS) - part of our Preservation Services section - using a combination of inhouse facilities and expertise, and outsourcing where it is cost effective to do so. With the collection growing at around 1,000 - 1,500 hours per year, just keeping up with each year’s new acquisitions has stretched resources to the limit.

The investment required to move to digital technology has acted as a catalyst on a range of issues that will improve the way we manage these collections.

This paper does not attempt to present all the background and details of these changes, described more fully in three existing papers on the NLA Web site [1], [2], [3]. While covering some of the same ground, this paper tries to take a more evaluative approach, looking at the objectives and roles which the changes are meant to support; the principles we have used and how they have been affected by digitisation; why the changes were made; comparisons with what is happening in other comparable institutions; costs and benefits; what it has to teach us about handling other digital info; and some implications and possibilities.

Roles

SPATS has two major and overlapping roles:

1. preserving the audio component of the collections generated or acquired by the Oral History program, and
2. providing technical support to the Oral History section.

These roles address an overall objective of supporting the Oral History program’s collecting, access and archiving intentions.

SPATS’ preservation responsibility is to ensure that acquired audio information remains accessible for as long as it is judged by the Library to be needed and in a form judged to be the most appropriate. This is addressed mainly by:

• copying selected recordings to stable carriers and providing access copies
• storing recordings in protective environments
• researching and keeping up to date on causes of deterioration and preservation methods
• repairing and restoring damaged recordings
• ensuring standards are used that will facilitate long term preservation
• training and advising Oral History staff, collectors and interviewers on achieving appropriate standards of recording and handling of sound recordings.

In addition, SPATS performs a technical support role, ensuring that the acquisition and access intentions are most efficiently and effectively supported with appropriate equipment and expertise. This is addressed mainly by:

• recording studio interviews to an appropriate standard for inclusion in the collections
• purchasing and maintaining appropriate equipment
• managing the availability of equipment and facilities for studio and field recording
• providing access copies
• providing editing, pre-mastering, and if necessary recording, services for publication projects
• providing soundscapes for exhibitions.

All of these functions have been impacted on by digitisation.

Changes in technology and archiving procedures

Tapes in the collection are streamed into categories for action depending on their technical requirements, ranging from recordings made by NLA on studio equipment, through field recordings that may or may not be recorded to standards, to acquired collections with some problems, to material that is unplayable.

The intended outcome for all categories is basically the same, namely to produce a preservation copy, a working copy, and a service copy for access purposes. (This is similar to the approach taken with microfilm produced in the Preservation Reformatting Unit. In both areas, the approach is intended to protect the Library’s considerable investment in producing the copy; with sound archiving, however, there is the added imperative that one of the copies is the original - if it is lost, the information is lost as well as the investment.)

This approach has been a standard one in sound archives world wide. It aims to achieve the following:

• to preserve the recorded information as accurately as possible on a reasonably long lasting carrier that can be stored securely;
• to provide a back-up copy in case of unexpected damage to the preservation copy. (This is a standard approach with other kinds of electronic information as well, of course.) For efficiency’s sake, we use the back-up as a working copy which may be filtered and used to produce publication and access copies if required;
• to provide an inexpensive, easy to use access copy that can be duplicated almost automatically for sale or distribution.

Until 1992, analogue technology was used for recording and preservation copying. From 1992 we have increasingly used DAT (digital audio tape) for recording, especially in the field. This produces a high quality but short lived digital master that needs recopying or migrating for archival uses. We have copied DAT to analogue tape for preservation.

The CD-R (CD-Recordable)-based digital technology we have adopted since 1996 retains most of these principles, because they are useful principles for archiving such vulnerable material. We now produce:

• a CD-R, as both a working copy and as the copy from which we will make later digital transfers (so in some ways it is our preservation copy);
• a safety copy on analogue reel to reel tape;
• a service copy cassette (either analogue, or DAT if that has been produced in the original recording).

The recordings are processed in two ways using this technology:

• analogue originals are converted to digital on the hard disk of a Sonic Solutions Digital Audio Workstation (DAW), and an analogue reel and cassette are made simultaneously. The digital copy is processed with the necessary indexing codes added. A CD is then burnt in the background while the DAW is used to record the next analogue tape;
• DAT recordings are processed via a stand alone CD recorder with automatic index coding. The same digital signal is converted to analogue and a reel and cassette are made simultaneously.

We are still making analogue back-up copies to give us a different medium from the preservation copy: experience with degrading sound carriers has shown us the risks of putting all one’s preservation eggs in the one basket. This is probably an interim measure while we satisfy ourselves that CD-R is reliable and we learn more about how it deteriorates. We expect to drop the analogue safety copy from the process eventually and produce two digital copies, although we are not sure when it will be necessary, and safe, to take that step. We are also not sure whether both digital copies will be on CD-R.

We are still making analogue service copy cassettes because they are cheaper than CDs to produce, and because we have the equipment to do it quickly and easily. At some stage in the future when equipment needs to be replaced or the demand for CDs outstrips the demand for cassettes, we expect that access will be given from digital sources, either on CDs, or via inhouse or wider networks.

The basic change in principles here is very small: we are recording to the best standard we can afford, and ending up with 3 copies with similar functions in both analogue and digital environments. However, the inputs and the outcomes (and potential outcomes) have changed greatly. On the input side different equipment, procedures, and skills have had to be acquired and developed, while changed outcomes and possibilities include the ease of copying, indexing, searching, automating retrieval, error correction, networked access, mass storage, editing, manipulation, and convergence with other data, along with different migration intervals. This explains why we are fairly comfortable with the changes: we can see the principles that still apply, allowing us to explore the areas that genuinely do need to change.

Three major areas of investigation still haven’t been resolved. One concerns the CD testing regimes we need to put in place so that we can keep track of errors and deterioration that are not immediately apparent. We are involved in international discussion of the most cost effective ways of building testing into archiving systems.

The second research focus, which we have decided is down the track a little way for us, is a future move to a digital mass storage system. This will again require careful thought and investment. The third area of investigation is the access options that digitisation offers us. This is being actively pursued right now, and is discussed later.

Why were these changes made?

The point of this process is to maintain accessibility in the face of technological change both in the short term and in a future of ongoing change. The technology was also chosen, after an extensive and rigorous investigation of options, to offer some improvements in our services and to be cost effective.

The anticipated loss of support for analogue equipment and tape among suppliers and technicians meant we had to go digital at some fairly early stage or lose access to the collection. The longer we stayed with an analogue-based technology the larger the backlog of material eventually having to be copied to digital at real time.

We chose CD-R because it is a widely used, successful technology likely to be supported for a relatively long time. Some of the alternatives offered better performance but with very poor chances of being supported. CD-R is also expected to have a reasonably long carrier life (likely to outlive the technology itself), and can be used flexibly either as a bridging strategy to a mass system, or as part of a mass system.

Technologically, the 16 bit linear Audio CD format is appropriate to the standard of material we hold. The sonic and dynamic range of the CD is roughly equivalent to that of a DAT, our present field recording medium, and is unlikely to be exceeded in any meaningful way by that of a standard analogue reel recording.

It is unlikely that we will have the resources to repeat this analogue to digital transfer. By using this format we may be sacrificing some detail that could be picked up by other digital formats, which could have been used later to remove minor speed variations inherent in all analogue recordings. Given the penalties for not beginning the transfer of a large collection, this is a compromise we are willing to make.

At this stage, our digital strategy is based on migration. The Library is also actively exploring archiving and preservation of other digital information such as online publications and in-collection items like floppy disks and CD-ROMs, where we are also assuming migration will be the only viable preservation path. For most of this material the oft-praised perfect cloning of digital data seems an ironic joke: fancy formatting and legal constraints both mean that we maybe able to copy the bits but not the pieces.

For our digital sound recordings we expect no such difficulties: we have much greater control over the formats and standards that we use and the rights to reformat the information, so we are reasonably assured that migration will work.

Approaches being used elsewhere

There are many local and state organisations involved in oral history and folklore recording programs in Australia, but in most cases they have limited archiving aims. The number of institutions performing a national sound archiving role in this country is small. The following comments reflect what we believe is happening in an almost random selection of overseas and local institutions with comparable collections and responsibilities to NLA: all are either using digital technology or planning to do so, and most are using CD- Rs; a number are producing both CD-R and analogue copies.

• Berlin Phonogrammarchiv. A collection of ethnomusic/folklore with related oral and social testimonies supporting the material. Have been writing CDs for nearly two years.
• Sudvestfunk (German Public Radio Archive). A very large collection of unique analogue reel recordings and also a collection of published material. CDs have been written for around three years. Now carrying out a major pilot project using mass data storage and online access using a purpose built network. CDs are still used for some distribution but will probably be phased out.
• National Sound Archive, British Library. A very large collection of ethnomusic, commercial, historic, oral history and folklore recordings with a variety of unevenly implemented standards. Have made digital recordings for many years, using a number of now-superseded media. Recordings now made on CD-R as well as other more experimental digital formats. Also make analogue reel copies. Prepare the CD using a SADiE hard disk editing system.
• Discoteca di Stato (State Sound Archives of Italy). Large collection of oral history, folklore and ethnomusical materials, as well as a collection of published materials. Using Sonic Solutions (like NLA) and make two CDs, one analogue reel and a cassette in accordance with current International Association of Sound and Audio-Visual Archives (IASAVA) Technical Committee recommendations [4].
• The Ulster Folk and Transport Museum. Large collection of folklore material from the eight counties as well as material from the BBC in Northern Ireland. All unique original recordings. Write CD, analogue reel and cassette using stand-alone CD writers.
• US Library of Congress. Are still making two analogue copies, though experimenting with many digital systems including CD-R. As their collection is probably the largest and most varied in the world, there are difficulties in making one system meet all their archiving needs.
• The Australian Institute of Aboriginal and Torres Strait Islander Studies has decided to provide CDs for access copies. They also write multimedia CDs containing image and audio for Aboriginal communities. They are presently actively preparing a program to transfer their entire sound collection to a digital format, probably CD-R.
• National Film and Sound Archive of Australia. Producing CDs, DAT, and analogue reels for preservation copies. Currently investigating the use of digital mass storage systems.

With the exception of Sudvestfunk, all have developed their systems from a previous two analogue reel preservation system.

What costs have been involved in the transfer to digital technology:

1: in absolute terms?

We have yet to evaluate the project, detailing all the costs, but we know we have spent approximately $A130,000 on equipment specifically for this, covering our network of digital audio workstations, CD writers, and digital adaptation of existing equipment wherever possible. We know we still need to upgrade some other existing equipment and to install CD testers. We also know we have spent about $A40,000 on digital studio and field recording equipment since 1990 - most of it purchased before we entered this project.

In terms of future costs, we know we will need to maintain the analogue equipment we are still using for the transfer to digital data; we will also need to invest resources to support the increased accessibility that becomes possible through networks, publications and exhibitions. Much more significant will be the costs associated with future technological change: at some stage the CD-based equipment will need to be replaced. It is likely that a mass storage system will be feasible within the next 10 years, although it is impossible to say when it will be, how much it will cost, and to what extent it can be integrated with other data storage in the Library or with other institutions.

The switch from analogue to digital has always been planned to happen over a number of years to take advantage of relatively reliable technology when it is available, and to spread the cost of the conversion.

2: in comparison with staying with analogue technology?

Savings that can be quantified and are available immediately are as follows:

• recording media used in the studio - one CD-R replaces one analogue reel in the case of speech, or two analogue reels in the case of music. On the basis of current costs this represents a saving per hour preserved of 22% and 25% respectively.
• recording media used in the field - a single DAT cassette replaces two analogue reels, resulting in a saving of about 50% per hour recorded.
• studio equipment - if the DAWs had not been purchased the Library would have needed to replace two analogue studio recorders at a cost approaching that of the digital equipment. The preferred studio replay machine, Studer A820, is now only available as a custom made piece of equipment.
• field equipment - the program uses a range of DAT recorders according to the particular recording situation. The cost of the DAT recorders is only about 25-30% of the cost of equivalent analogue field recorders.
• storage space - eight CDs can be stored in the same space as one analogue reel. If the shelving was reorganised to optimise CD storage, a space saving of 56% could be realised. Shelving them side by side in existing shelving still realises a space saving of 25%.

Other savings are less immediate and harder to quantify:

• the time taken to record and process digital and analogue are similar, so there are not significant short term efficiencies in processing. However, when the recordings have to be migrated to a new system, the savings will be very significant as the digital-to-digital transfer can be largely automated and done at faster than real time speeds.
• some of the benefits listed later imply longer term, as yet unquantified, savings.

Other benefits

The major benefits already discussed can be summarised as maintaining preservation effectiveness, providing some ongoing productivity gains, and preventing the further build up of a backlog of real time analogue duplicating needing to be done in the future.

Some of the other benefits that can be expected from our digital system are these:

• it gives us more choice about the timing and options for moving to a mass storage system;
• the system has been designed with migration in mind: software upgrades are included as part of system maintenance; the system is modular, minimising replacement costs and allowing a flexible response to technological changes; upgrading to the next system can be done in steps, and the core Sonic Solutions system can be retained for other carriers when CD-R has been superseded;
• the system is internally networked, allowing staff and equipment to be used more flexibly;
• the ability to check data quality - eventually automatically - may help us to take more of a ‘just in time’ rather than a ‘just in case’ approach to preservation;
• the standard of work by contractors becomes easier to monitor;
• the ability to include metadata on disk should allow us to automate more of the procedures for migration and for retrieval, with possible linking to the catalogue record;
• it should be possible to link audio to transcripts or summaries, making access to the material much more flexible than at present; this may also reduce the number of full transcripts needed if summaries can be successfully linked;
• the system provides the basic foundation for networked access to the collection, initially in-house and later externally;
• digital data is also easier to prepare for public use - it is easier to locate, duplicate, and clean up on demand;
• we can also expect to develop skills in handling digital media and systems that will be useful in evaluating larger mass systems for the Library;
• finally, there is also some benefit for interviewers in digital media and equipment that allow longer recording times, lower weight machines and higher quality audio, while for the Library there are lower battery and freight costs for field equipment and supplies.

Implications

We have already indicated that the project is yet to undergo a formal evaluation process, but there has been much informal explanation, discussion and evaluation of its implications. This is part of the normal accountability process, but people have also begun to recognise the possibilities for improving the way the collections are managed and used. Our formal evaluation will look at questions like what we think of the technology, what it is delivering, what users think, problems encountered and overcome, costs and benefits, along with crucial questions of what our objectives were, whether they were achieved cost effectively, whether they were still worth achieving, whether we could have done it better any other way, what value it has provided to the Library, its users, and the community, what we have learned from it, and problem areas still needing attention.

Recent discussions in the Library have highlighted a number of issues and comments that will be relevant:

1. what we think of the technology. The technology was specifically selected not to be a dead end; time will tell if we were wrong, but it is still looking good in terms of delivering what we want of it. It looks flexible enough to be managed in ways that suit the Library, and open enough to migrate well. Longer term the technology needs to provide more efficient data storage, more feasible networking, and automated migration, but these will almost certainly evolve.
2. what users think of the outcomes. Because we are still providing access via analogue cassettes, most end users have not had direct contact with the digital output. At this stage we are assuming they will appreciate increased rates of access and improved retrievability when we can provide them, although we are really only beginning to explore questions of how users will want to use digital sound. Specialised users such as broadcasters, publishers and exhibition curators have responded positively to our ability to fit in with their largely digital systems.
3. problems encountered. There have been few teething problems with the technology itself. Technological change often comes with human problems and challenges involved in making and justifying decisions, winning organisational and staff support, clarifying expectations and developing expertise. We have experienced some aspects of all of these, though perhaps their tractability has been more remarkable than their existence. At a purely practical level, the use of a hard disk to manage recorded data has marginally added to conflicting demands on our facilities, as we need to process the data before we can handle the next studio interview. There have also been design problems to overcome in that we are operating in both digital and analogue domains. We have also had to convince our contractors to install compatible systems for work that we want to outsource.
4. what is the project delivering? The system has been working well for some months, producing the quantity outputs, and we believe the quality outputs, that we aimed for, although we cannot be certain of that until we have full CD testing arrangements in place. It is delivering some of the benefits we looked for, though many of them remain in the future. It hasn’t immediately increased our capacity to collect and manage material, but it should longer term. The system itself does not solve workflow problems. We still need to make procedures as streamlined and effective as possible. At the same time as we are changing technology a lot of attention is being given to improving the information that is available for managing the collection, rationalising procedures, prioritising material for attention, and identifying material that should not be in the collection or should not be digitised.

Apart from the sound files, the main deliverables so far seem to be the possibilities that the system is opening up, with implications that reach throughout the Oral History program and beyond. Because it has helped us to focus on how we can better manage the collections and provide access to them, it has influenced us to look at a raft of issues including:

• what goes into the collections;
• what preparation work and what post-interview work is required from interviewers;
• what is needed for effective cataloguing;
• what material should be transcribed;
• whether we can link the catalogue record (which has been digital for a long time), the transcript (which is already in a digital form or could be easily scanned) and sound;
• what gets preserved and how;
• how much we try to digitise and how quickly;
• who looks after the digital sound files;
• how completely and seamlessly digital audio can be integrated with other Library digital collections;
• managing accountability;
• staff skills and succession planning.

We have already started on many of these. Working groups have been set up to look at redefining the collection development policy, sorting out priorities for preservation and other collection management strategies, and future access options. For the latter we are exploring ways of linking indexing points in the sound file with indexing points in either the transcript or a detailed summary. We are also looking at the technology and investment that would be required for networked access in the Library building and beyond. We are collaborating with Australia’s CSIRO to adapt its FRANK software, developed for moving images, for use with audio files [5].

There are other options for cooperation that are also being explored, sharing research and information, working together on standards and guidelines, sharing equipment and even data storage facilities. Collaboration and convergence raise a number of dilemmas including a tension between technological perspectives (which see sense in putting all data together), and a client service perspective (which seeks to make it possible for users themselves to bring data together easily, while retaining the context value added by institutions and professionals who organise their material in particular ways, maintaining levels of choice for the client).

It costs a lot of money to create and acquire the Library’s Oral History collections and to maintain access, and it will continue to be an expensive process. All electronic information collections that are vulnerable to carrier deterioration and technology obsolescence are expensive to maintain for ongoing access. The digital systems that have been installed seem to offer good benefits for the costs, many of which are unavoidable if the collection is to continue to be available. It is important that the Library’s investment continues to be managed in an accountable manner, making the best use of the resources involved, while defining and taking advantage of the right possibilities.

References

1. Colin Webb, 1996. Stairways to digital heaven? Preserving Oral History Recordings at the National Library of Australia, Canberra, National Library of Australia,
   http://www.nla.gov.au/nla/staffpaper/cwebb2.html
2. Kevin Bradley, 1995. Preservation of the National Library of Australia’s Oral History Collection. Text and images prepared for a display at the 2nd National Preservation Office Conference: Multimedia Preservation - Capturing the Rainbow, in Brisbane, Queensland, 28-30 November 1995, Canberra, National Library of Australia.
   http://www.nla.gov.au./nla/staffpaper/npokb.html
3. Colin Webb, 1995. Transfer from Analogue to Digital: Selecting a System. Paper presented at the 2nd National Preservation Office Conference: Multimedia Preservation - Capturing the Rainbow, in Brisbane, Queensland, 28 - 30 November 1995. Canberra, National Library of Australia Available from:
   http://www.nla.gov.au/nla/staffpaper/npocw.html
4. ‘The Technical Committee members discussed what advice they could give to collections facing the need to make preservation transfers of decaying audio documents. CD-Recordable is increasingly being used by collections for making access and preservation copies of audio documents. In the absence of a digital medium with a better proven format stability, it has been widely accepted as a digital target format. The consensus was that CD-R offered, with some reservations, the best digital solution at the moment. CD-R is not an ideal solution but a “least-worst” solution. The making of a parallel analogue preservation copy of a ¹⁄₄ inch polyester tape should also be considered. The cost of the tape is low compared to the additional security gained:
   ‘Any archive using CD-R disks for preservation copies is strongly urged to monitor the condition of the collection by checking sample recordings at regular intervals using a suitable CD tester.’ IASAVA Technical Committee, Annual Committee, Annual Conference, Perugia, August/September 1996, Minutes of working meeting, p.5
5. CSIRO Division of Information Technology DIMMIS homepage. A Web site describing the Distributed Interactive Multimedia Information Service (DIMMIS) Project, including the FRANK project.
   http://www.syd.dit.csiro.au/projects/dimmis/

Author Details