Like many national, regional and research libraries, the National Library of Australia (NLA) is actively facing a wide range of challenges associated with using digital technology to improve access to its collections. In at least one area this is not discretionary: the Oral History collections are intrinsically affected by technological change and without moving to digital, access to the collections will be lost as analogue audio technology loses market support. The NLA is implementing a strategy to bridge the difficult transition between fully analogue and fully digital environments.
The Oral History collections cover a wide range of material collected (with varying levels of activity) over the past 50 years. In recent years the focus has been on biographical interviews with prominent Australians, social history projects and folklore collecting. Currently holding about 30,000 hours of original recordings on reel to reel analogue tape, the collection is important and well housed, but inadequately preserved. Only about a quarter of the material had been copied to acceptable preservation standards by the end of 1995, with similarly low levels of transcription and cataloguing. If preservation is ultimately about accessibility, we had (and have) very large challenges even without an enforced change in the technology base of the collection. This responsibility is managed by our Sound Preservation and Technical Services unit (SPATS) - part of our Preservation Services section - using a combination of inhouse facilities and expertise, and outsourcing where it is cost effective to do so. With the collection growing at around 1,000 - 1,500 hours per year, just keeping up with each year's new acquisitions has stretched resources to the limit.
The investment required to move to digital technology has acted as a catalyst on a range of issues that will improve the way we manage these collections.
This paper does not attempt to present all the background and details of these changes, described more fully in three existing papers on the NLA Web site , , . While covering some of the same ground, this paper tries to take a more evaluative approach, looking at the objectives and roles which the changes are meant to support; the principles we have used and how they have been affected by digitisation; why the changes were made; comparisons with what is happening in other comparable institutions; costs and benefits; what it has to teach us about handling other digital info; and some implications and possibilities.
SPATS has two major and overlapping roles:
These roles address an overall objective of supporting the Oral History program's collecting, access and archiving intentions.
SPATS' preservation responsibility is to ensure that acquired audio information remains accessible for as long as it is judged by the Library to be needed and in a form judged to be the most appropriate. This is addressed mainly by:
In addition, SPATS performs a technical support role, ensuring that the acquisition and access intentions are most efficiently and effectively supported with appropriate equipment and expertise. This is addressed mainly by:
All of these functions have been impacted on by digitisation.
Tapes in the collection are streamed into categories for action depending on their technical requirements, ranging from recordings made by NLA on studio equipment, through field recordings that may or may not be recorded to standards, to acquired collections with some problems, to material that is unplayable.
The intended outcome for all categories is basically the same, namely to produce a preservation copy, a working copy, and a service copy for access purposes. (This is similar to the approach taken with microfilm produced in the Preservation Reformatting Unit. In both areas, the approach is intended to protect the Library's considerable investment in producing the copy; with sound archiving, however, there is the added imperative that one of the copies is the original - if it is lost, the information is lost as well as the investment.)
This approach has been a standard one in sound archives world wide. It aims to achieve the following:
Until 1992, analogue technology was used for recording and preservation copying. From 1992 we have increasingly used DAT (digital audio tape) for recording, especially in the field. This produces a high quality but short lived digital master that needs recopying or migrating for archival uses. We have copied DAT to analogue tape for preservation.
The CD-R (CD-Recordable)-based digital technology we have adopted since 1996 retains most of these principles, because they are useful principles for archiving such vulnerable material. We now produce:
The recordings are processed in two ways using this technology:
We are still making analogue back-up copies to give us a different medium from the preservation copy: experience with degrading sound carriers has shown us the risks of putting all one's preservation eggs in the one basket. This is probably an interim measure while we satisfy ourselves that CD-R is reliable and we learn more about how it deteriorates. We expect to drop the analogue safety copy from the process eventually and produce two digital copies, although we are not sure when it will be necessary, and safe, to take that step. We are also not sure whether both digital copies will be on CD-R.
We are still making analogue service copy cassettes because they are cheaper than CDs to produce, and because we have the equipment to do it quickly and easily. At some stage in the future when equipment needs to be replaced or the demand for CDs outstrips the demand for cassettes, we expect that access will be given from digital sources, either on CDs, or via inhouse or wider networks.
The basic change in principles here is very small: we are recording to the best standard we can afford, and ending up with 3 copies with similar functions in both analogue and digital environments. However, the inputs and the outcomes (and potential outcomes) have changed greatly. On the input side different equipment, procedures, and skills have had to be acquired and developed, while changed outcomes and possibilities include the ease of copying, indexing, searching, automating retrieval, error correction, networked access, mass storage, editing, manipulation, and convergence with other data, along with different migration intervals. This explains why we are fairly comfortable with the changes: we can see the principles that still apply, allowing us to explore the areas that genuinely do need to change.
Three major areas of investigation still haven't been resolved. One concerns the CD testing regimes we need to put in place so that we can keep track of errors and deterioration that are not immediately apparent. We are involved in international discussion of the most cost effective ways of building testing into archiving systems.
The second research focus, which we have decided is down the track a little way for us, is a future move to a digital mass storage system. This will again require careful thought and investment. The third area of investigation is the access options that digitisation offers us. This is being actively pursued right now, and is discussed later.
The point of this process is to maintain accessibility in the face of technological change both in the short term and in a future of ongoing change. The technology was also chosen, after an extensive and rigorous investigation of options, to offer some improvements in our services and to be cost effective.
The anticipated loss of support for analogue equipment and tape among suppliers and technicians meant we had to go digital at some fairly early stage or lose access to the collection. The longer we stayed with an analogue-based technology the larger the backlog of material eventually having to be copied to digital at real time.
We chose CD-R because it is a widely used, successful technology likely to be supported for a relatively long time. Some of the alternatives offered better performance but with very poor chances of being supported. CD-R is also expected to have a reasonably long carrier life (likely to outlive the technology itself), and can be used flexibly either as a bridging strategy to a mass system, or as part of a mass system.
Technologically, the 16 bit linear Audio CD format is appropriate to the standard of material we hold. The sonic and dynamic range of the CD is roughly equivalent to that of a DAT, our present field recording medium, and is unlikely to be exceeded in any meaningful way by that of a standard analogue reel recording.
It is unlikely that we will have the resources to repeat this analogue to digital transfer. By using this format we may be sacrificing some detail that could be picked up by other digital formats, which could have been used later to remove minor speed variations inherent in all analogue recordings. Given the penalties for not beginning the transfer of a large collection, this is a compromise we are willing to make.
At this stage, our digital strategy is based on migration. The Library is also actively exploring archiving and preservation of other digital information such as online publications and in-collection items like floppy disks and CD-ROMs, where we are also assuming migration will be the only viable preservation path. For most of this material the oft-praised perfect cloning of digital data seems an ironic joke: fancy formatting and legal constraints both mean that we maybe able to copy the bits but not the pieces.
For our digital sound recordings we expect no such difficulties: we have much greater control over the formats and standards that we use and the rights to reformat the information, so we are reasonably assured that migration will work.
There are many local and state organisations involved in oral history and folklore recording programs in Australia, but in most cases they have limited archiving aims. The number of institutions performing a national sound archiving role in this country is small. The following comments reflect what we believe is happening in an almost random selection of overseas and local institutions with comparable collections and responsibilities to NLA: all are either using digital technology or planning to do so, and most are using CD- Rs; a number are producing both CD-R and analogue copies.
With the exception of Sudvestfunk, all have developed their systems from a previous two analogue reel preservation system.
1: in absolute terms?
We have yet to evaluate the project, detailing all the costs, but we know we have spent approximately $A130,000 on equipment specifically for this, covering our network of digital audio workstations, CD writers, and digital adaptation of existing equipment wherever possible. We know we still need to upgrade some other existing equipment and to install CD testers. We also know we have spent about $A40,000 on digital studio and field recording equipment since 1990 - most of it purchased before we entered this project.
In terms of future costs, we know we will need to maintain the analogue equipment we are still using for the transfer to digital data; we will also need to invest resources to support the increased accessibility that becomes possible through networks, publications and exhibitions. Much more significant will be the costs associated with future technological change: at some stage the CD-based equipment will need to be replaced. It is likely that a mass storage system will be feasible within the next 10 years, although it is impossible to say when it will be, how much it will cost, and to what extent it can be integrated with other data storage in the Library or with other institutions.
The switch from analogue to digital has always been planned to happen over a number of years to take advantage of relatively reliable technology when it is available, and to spread the cost of the conversion.
2: in comparison with staying with analogue technology?
Savings that can be quantified and are available immediately are as follows:
Other savings are less immediate and harder to quantify:
The major benefits already discussed can be summarised as maintaining preservation effectiveness, providing some ongoing productivity gains, and preventing the further build up of a backlog of real time analogue duplicating needing to be done in the future.
Some of the other benefits that can be expected from our digital system are these:
We have already indicated that the project is yet to undergo a formal evaluation process, but there has been much informal explanation, discussion and evaluation of its implications. This is part of the normal accountability process, but people have also begun to recognise the possibilities for improving the way the collections are managed and used. Our formal evaluation will look at questions like what we think of the technology, what it is delivering, what users think, problems encountered and overcome, costs and benefits, along with crucial questions of what our objectives were, whether they were achieved cost effectively, whether they were still worth achieving, whether we could have done it better any other way, what value it has provided to the Library, its users, and the community, what we have learned from it, and problem areas still needing attention.
Recent discussions in the Library have highlighted a number of issues and comments that will be relevant:
Apart from the sound files, the main deliverables so far seem to be the possibilities that the system is opening up, with implications that reach throughout the Oral History program and beyond. Because it has helped us to focus on how we can better manage the collections and provide access to them, it has influenced us to look at a raft of issues including:
We have already started on many of these. Working groups have been set up to look at redefining the collection development policy, sorting out priorities for preservation and other collection management strategies, and future access options. For the latter we are exploring ways of linking indexing points in the sound file with indexing points in either the transcript or a detailed summary. We are also looking at the technology and investment that would be required for networked access in the Library building and beyond. We are collaborating with Australia's CSIRO to adapt its FRANK software, developed for moving images, for use with audio files .
There are other options for cooperation that are also being explored, sharing research and information, working together on standards and guidelines, sharing equipment and even data storage facilities. Collaboration and convergence raise a number of dilemmas including a tension between technological perspectives (which see sense in putting all data together), and a client service perspective (which seeks to make it possible for users themselves to bring data together easily, while retaining the context value added by institutions and professionals who organise their material in particular ways, maintaining levels of choice for the client).
It costs a lot of money to create and acquire the Library's Oral History collections and to maintain access, and it will continue to be an expensive process. All electronic information collections that are vulnerable to carrier deterioration and technology obsolescence are expensive to maintain for ongoing access. The digital systems that have been installed seem to offer good benefits for the costs, many of which are unavoidable if the collection is to continue to be available. It is important that the Library's investment continues to be managed in an accountable manner, making the best use of the resources involved, while defining and taking advantage of the right possibilities.