Web Magazine for Information Professionals

Scientific, Industrial, and Cultural Heritage: A Shared Approach

Lorcan Dempsey presents a research framework for libraries, archives and museums prepared for the European Commission.

The Information Society Technologies programme within the EU's Framework Programme Five supports access to, and preservation of, digital cultural content. This document describes some common concerns of libraries, archival institutions and museums as they work together to address the issues the Programme raises. This accounts for three major emphases in the document. First, discussion is very much about what brings these organisations together, rather than about what separates them. Second, it describes an area within which a research agenda can be identified; its purpose is not to propose a programme of work or actions, rather a framework within such a programme might be developed. Finally, although the main focus is on access to resources, this is placed in an overall life-cycle context.

Broad Goals

This document is based on the assumption that libraries, archives and museums have shared research interests. We can identify several broad goals which underpin these, and which encourage collaborative activity between libraries, museums and archival institutions. These include:


A feature of change is that we have no settled vocabulary. Some terms may have partial or sectoral associations, which are not commonly shared. This might be within particular curatorial traditions (library, museum or archival), particular disciplines, or particular national or language contexts. We adopt the following terms, and explain them here to ensure shared understanding while acknowledging their imprecision.
  • Archive. For conciseness, we typically use archive in place of archival repository or archival institution.
  • Cultural. This document is about the scientific, industrial and cultural heritage. In general we use the word 'cultural' in a broad sense to cover all of these.
  • Memory institution. We have no term in routine use which includes libraries, archives and museums. Again, for conciseness, we sometimes use cultural institutions and memory institutions in this inclusive sense.
  • Network service. A resource is made network accessible through a network service. So, for example, a catalogue may be made accessible through a telnet service, an http service, and a Z39.50 service. A broker or mediator is a service which provides consistent access to other network services, typically to heterogenous or homogenous services from variously located service providers.
  • Resource. This is a cultural entity of interest. It may be a database, an artifact, a document, a newsgroup, a mailing list, a learning environment, an image, a map, a geographic information system, and so on. Many resources typically reside in collections where a collection comprises similar or related resources. Such collections are also, of course, resources, and collections may contain other collections. We talk about the contents of libraries, museums and archival institutions as collections.

The Vision

Archives, libraries and museums are memory institutions: they organise the European cultural and intellectual record. Their collections contain the memory of peoples, communities, institutions and individuals, the scientific and cultural heritage, and the products throughout time of our imagination, craft and learning. They join us to our ancestors and are our legacy to future generations. They are used by the child, the scholar, and the citizen, by the business person, the tourist and the learner. These in turn are creating the heritage of the future. Memory institutions contribute directly and indirectly to prosperity through support for learning, commerce, tourism, and personal fulfilment.

They are an important part of the civic fabric, woven into people's working and imaginative lives and into the public identity of communities, cities and nations. They are social assembly places, physical knowledge exchanges, whose use and civic presence acknowledge their social significance, and the public value accorded to them. They form a widely dispersed physical network of hospitable places, open to all. They will continue to fulfil these roles, as people wish to enjoy the physical experiences they and the use of their collections offer.

However, we are now seeing the creation of new places which offer a new type of experience, a global digital space based on the Internet and other digital networks. Memory institutions are actively connecting their collections to these emerging knowledge networks. They are creating innovative network services based on digital surrogates of their current collections in rich interactive digital environments. They are focusing their traditional curatorial values on the challenges of the rapidly changing and growing digital resource, and developing relevant practices to support its use and management over time.

The archive, library and museum communities are addressing these issues within their own curatorial traditions and organisational contexts, and within specific national or other administrative frameworks. They are exploring how to provide learning, research and cultural opportunities, and how to identify and grow new communities of users. They are developing strategies to manage the physical, the digitised, and the born-digital as complementary parts of a unified resource. They are developing strategies for the initial investment and managed intervention that is required to preserve the value of digital resources. They are ensuring that 'born-digital' documents and artifacts become integrated into the cultural record, by being organised and documented so that they will be accessible, and become a part of the memory of future generations.

At the same time, they recognise their convergent interests in a shared network space. This convergence is driven by the desire to release the value of their collections into this space in ways that support creative use by as many users as possible. They recognise their users' desire to refer to intellectual and cultural materials flexibly and transparently, without concern for institutional or national boundaries. To support this, they recognise the need for services which provide unified routes into their deep collective resources, which protect the value of their resources in an environment of easy access and reuse, and which ensure the authenticity and integrity of resources. They wish to enhance and personalise their offerings through the collection of data about use and users, while preserving privacy. These aims pose shared technical challenges, but also highlight the benefits of concerted attention to business and policy issues. There is advantage in working together to develop business models which recognise the long-term ownership costs of digital media while preserving the public interest in equitable access; to establish and promote best practice for content creators and others which reduce the long-term costs of data ownership and enhance its use; and to explore what it means to develop cultural institutions in a digital environment.

Finally, they are unified in the belief that without the rich cultural resources memory institutions offer, emerging network places will be impoverished, as will the lives of the people who assemble there.

The Challenge

The digital medium is radically new. Although there is continuity of purpose and value within cultural institutions, these exist alongside a fundamental examination of roles and practices. The costs of developing necessary roles and sustainable practices will be high, as will the social and organisational costs of change and institution building. However the costs of not doing so will be higher, as the cultural and intellectual legacy to future generations is entrusted to a house of cards built on a million web sites.

The Challenge of Serving the Active User

The focus of service delivery is becoming the active user in a shared network space. The user wants resources bundled in terms of their own interests and needs, not determined by the constraints of media, the capabilities of the supplier, or by arbitrary historical practices. The growth in the variety, volume and volatility of digital resources means that effective use depends not merely on pointing people to resources, but on supporting selection, aggregation and use. It may mean providing interpretive environments in which resources are situated in relation to wider contexts. It may mean supporting reuse and repackaging of materials.

Human attention is more valuable than computing resource, and it should not be wasted in unnecessary tasks. Access and use may be situational: the 'information landscape' will be adapted to the needs of users or groups of users, rather than to the constraints of particular media or systems. These factors shift the emphasis of automation from inward-looking collection management to outward-looking services to the user.

The Challenge of Living with the Radically New

What is a 'document', or a 'publication', or an 'exhibition'? We often cannot 'see' a digital resource, we cannot sense its scale or scope, or its internal organisation. Nor can we always think in terms of traditional analogues: for example, we cannot simply 'print out' meteorological data, which may occupy gigabytes on disc. Data and programs may be integrated in complex applications, difficult to disentangle. Occupying a network space shifts the emphasis from standalone services constructed for human visitors alone, to services designed also to be visited by automated services which provide aggregation, filtering, selection or other services to human users. Users may interact with resources through digital library services, learning environments, games or exhibitions. The issues involved in providing services in such distributed, multi-layered environments are poorly understood. Everywhere we are living with new ways of doing things.

A major feature of the new is that fluidity replaces fixity as a dominant characteristic of resource creation and use. Fluid because data flows: it can be shared, reused, analysed; can be adapted, reconfigured, copied, and newly combined in ways which were not possible before. A resource dissolves into multiple individually addressable resources, or can be aggregated in multiple combinations. Resources can carry information about themselves, can communicate to automate processes or deliver new services, and can yield up use or status data which can drive decisions and inform behaviour. The creation and use of flows in a digital medium offer unprecedented flexibility, enhancing and augmenting services. However, with fluidity also comes the challenge of managing a fugitive and fragile resource:

The Challenge of Planning for the Radically Unpredictable

Not only is change rapid, it is unpredictable. The transforming influence of the Web was not predicted several years ago, nor was the rapid takeup of e-mail. However, this is not a technological issue alone. As networking becomes pervasive of more parts of our lives, the complex interdependencies of technology development, service provision, business models, and user behaviour make innovation, reconfiguration and unpredictability integral to practice.

As service, technology and business opportunities co-evolve, new service provider configurations or divisions of responsibility will emerge. These may include third party resource discovery, authentication, or ratings services; long-term archiving facilities; or a new class of 'broker' or 'mediator' service which provides a single point of access across distributed collections. User behaviour will be shaped by and will shape opportunity and development.

This unpredictability emphasises the need for approaches which do not lock providers into inflexible or unresponsive offerings, and which support movement of data and services through changing environments. Without such approaches investment will be wasted and data will potentially be lost or difficult to use.

The Challenge of Institution Building

Over the last years, emphasis has shifted from technology development to serving users and managing content. However, procedures are still preliminary or provisional, awaiting agreements or technical developments which will provide routine and predictable services. They have often not become part of 'business as usual'.

This is part of a wider institutional flux. Institutions are relatively persistent embodiments of values and practices organised around particular goals, and we only beginning to sense how institutions will be built and modified in digital spaces. What is the institutional context of the library, of the museum, or of the archive, as they evolve and as the expectations and practices of their users evolve. And, importantly, as the institutions of learning and culture, of trade, of civic engagement, and of entertainment also evolve, altering patterns of relatedness and inclusion.

Development will depend on agreed business models, on an understanding of the role of public services in a digital environment, on emerging agreement about roles and responsibilities. These in turn will depend on more mature information infrastructure, including the ability to conduct commerce, to identify and authenticate users and content, to develop personalised services, to guarantee persistence and predictability. Institutions secure stable services.
creation, management and use cycle (chart)

The Way Forward

A life-cycle approach to the creation, management and use of resources emphasises the interdependence of choices in the life of cultural content. A choice made at any stage may ramify throughout the life of a resource, facilitating or impeding its appropriate flow through different custodial and use environments, and its ability to be an agile component of information and learning environments.

We discuss a research agenda based around the life-cycle presented in the accompanying figure. There has been a historical separation between the 'supply-side' interests of memory institutions, traditionally focused on the management of collections, and the interests of users and creators of cultural content. However, a significant feature of the digital environment is that memory institutions become centrally interested in supporting the creation and use of cultural content in more direct ways, as institutions and their users share the same digital space. For example, archival institutions wish to influence the format of the records transferred to them, so as to reduce the costs of management and ownership. Similarly, libraries, museums and archives are increasingly interested in serving up resources in such a way that their use is encouraged as active components of exhibitions, learning environments, and so on. Services which manipulate or analyse various types of data may be provided alongside the data itself. And of course, memory institutions are increasingly creating digital surrogates of items in their collections, or repackaging digital items in new offerings.

In what follows, different stages of the life-cycle are considered separately, with a concluding piece with gathers some organisational concerns. We consider the following life cycle stages, acknowledging that these are not definitive or exhaustive: collection development, collection management, access (including discovery and retrieval), use, and creation.

It is important to note that this document concerns itself with the life of cultural content in relation to the interests of memory institutions and their users, not in a wider context. A different perspective would yield different stages and emphases. Special attention is given to questions of access to cultural resources and network services, as increasingly, all operations will be carried out in the environment it describes where actors are supported by variously assembled network services to carry out their tasks.

Accessing Resources and Services

Memory institutions provide and use many network services to disclose and deliver their content. They are individually valuable, however they do not yet seamlessly work together or rely on each other for services. They do not communicate easily or share content. For example, it is not yet common to interact with more than one catalogue at a time. At the same time, as there is more development in a network environment additional network services are required, authentication for example.

This variety is potentially confusing and adds effort for the user or developer who has to discover what is available, cope with many different interfaces, negotiate different authentication systems, work out terms and conditions, manage different results, and move data between services. In the current environment, the collection of information about a particular topic - Darwin and finches, for example, as in the accompanying sidebar - is inhibitively labour-intensive. One has to know quite a bit, and do a lot of work, to get effective results. Some of this work is mechanical - it would benefit from being automated. The focus has been on the automation of individual tasks rather than on the automation of end-to-end processes, the sharing of content, or other examples of communication between tasks.

In a shared network space, a new way of working is required, one which recognises that network services do not stand alone as the sole focus of a user's attention but need to be part of a fabric of opportunity. Services need to be aggregated in support of business objectives and user needs. They need to be able to pass data between them - between a search service, a requesting service, and an accounting service, for example. This requirement is driving an interest in 'content infrastructure', which works towards 'plug and play' infrastructure for discovery, use and exploitation of content in managed environments. Monolithic applications are being broken down and reconfigured as systems of communicating services.

What services will be deployed by libraries, archives, and museums? Some they will provide themselves; some will be third-party services used by them to augment their own. They will also have to consider how these services are combined in helpful ways. We consider some network services here in very general terms in the next section, and issues arising from their combination in a following section.

Some Network Services

This is an indicative list, whose purpose is to illustrate the variety of activity; it is not exhaustive and adopts an inclusive view of 'service'. The services discussed are provided in various ways and are at different levels of granularity.

First and foremost, libraries, archives and museums disclose and deliver cultural content.

Some areas of development are especially important as they can lever significant expertise and knowledge within the library, museum and archival traditions to provide new types of service:

Some services will become 'infrastructural' in the sense that they become services which are predictably and consistently available in a shared way across providers, rather than being developed on a per-provider basis. General third party services might include:

Some services might begin to be shared within the memory domain, perhaps using generally available infrastructural services (such as directory services for example). Examples might include:

Memory institutions will develop particular services to enhance access to their resources, individually or collectively:

Some basic services may be used to support other activities, for example:

Combining Network Services

Increasingly, such services will be built from other communicating services. Services may share some basic services and infrastructure within agreed frameworks for communication.

For example, resource discovery services may report on the availability of services, may use location services to identify instances of resources (mirror sites for example), may be combined with user profile or ratings services to refine selections, and so on. A service which mediates access to the holdings of several memory organisations might provide support for discovery and selection of services, manage service requests, translate formats, aggregate services, consolidate results, manage authentication and financial transactions, and so on.

Such 'mediator' or 'broker' services, which provide consistent access over other services and which allow them to communicate, will become more important. These may be implemented in various ways, for example as bespoke applications or as networks of reusable components (e.g. software agents). Communication will benefit from agreed APIs (application programming interfaces) and data exchange formats. Increasingly, applications will be built within a distributed object framework.

In order for a client or broker service to access a network service it must know: the location of that service and details of how to communicate with it. We call such details a 'service profile'. For example, in a particular case, such a profile might include the access protocol (which may be the simple web protocol HTTP, a search and retrieve protocol, a directory protocol or other form of protocol depending on the service being accessed); the request format (which may be defined by a query language); and the schema(s) relevant to the service (for example the metadata format in use).

Interworking across services will not be achieved by enforcing uniformity. It is not desirable, nor would it be possible, to suggest that network services converge on a single service profile, although agreement over profiles for classes of service would facilitate interworking, as would a mechanism for sharing service profile descriptions. A variety of service profiles will be in use across cultural domains, reflecting the variety of services they make available and the different curatorial traditions and professional practices in operation.

There are significant research and development challenges in the development of distributed content infrastructure. Some areas which require attentions are:

A project team wants to find out more about Darwin's finches: they want some information about how they contributed to his theory of evolution. They are doing a part-time course, and are exploring the leads provided in the course pack.
  • They decide to browse some of the resources mentioned.
  • They follow a link to Down House, where he lived, and explore the rooms where he lived.
  • They browse his Notebooks, going to relevant pages.
  • Then they get more focused. They do a distribution map for finches in the Galapagoes Islands. They think there should be more, and do a comparison with other island groupings in the Pacific.
  • They do a general search on the Galapagos, and get back a number of suggestions. They follow some through and collect some photographs.
  • Some of these are in the Natural History Museum in London - this leads them to see if it has some of the finches. It does, and it also has a n online exhibition on the 'Voyages of Discovery' which has a copy of the first edition of Darwin's Origin of Species, together with specimens - beetles, fish, fossils and the actual finches brought back by him from the Galapagos Islands. They order pictures of the finches from the Museum's Picture Library, and noticie on the 'fuel guage' that they have spent more than they intended.
  • There is a video to accompany the exhibition, but it is expensive to download. They do a query and it is available at their college.
  • Then they switch to search for some articles or other material. They decide to focus on undergraduate and general material, screening out more specialist stuff. They request some articles, and view an encyclopeida entry. There is still quite a lot of material, so they limit it to materials available to them without payment as part of their course, and immediately available to them electronically or available in print locally.
This is a simple scenario, but if it is to work seamlessly would require quite a bit of 'content infrastructure'. The team has potentially used many different services. For example:

  • To avoid multiple challenges some form of distributed authentication would need to be in use.
  • To support this level of trading of content, rights management, e-commerce, billing, and other services would have to be deployed.
  • To support sensible navigation or selection of resources, collections and services need to be described, and indexed for searching.
  • Data needs to be shared between services to support forwarding of queries.
  • Searching or requesting across different services would require some mediating services.
  • Service for manipulating files, converting formats, and so on might be needed to deal with a variety of content.
  • And so on...

Use

Memory institutions will support use in ways which reflect new opportunity, and the changing behaviour and needs of active users. There is clearly a close relationship between resource use and adjoining operations in the chain: resource access and resource creation. Areas for investigation include:

Creation

Memory organisations create digital cultural content themselves. They can influence the creation activity of others, whose outputs they collect or are transferred into their holding. They can provide resource creation services to their users. In each case, they have a shared interest in promoting future use and in minimising the costs of accessioning and management, by working to develop good practice and the tools to support it.

Developing Collections

Libraries, archival institutions and museums have developed their collections in line with specific missions and according to different curatorial traditions. A national library may benefit from legal deposit arrangements, for example. Component collections may be built up to meet some particular interests, or may be transferred into memory institutions as complete entities. Collections may be unified by theme, by medium, by ownership, by provenance, by administration. Some large cultural organisations may have contained a combination of library, archival and museum collections, managed by staff from appropriate professional domains. Memory institutions exert different levels of control over the materials they accession. How can we expect the nature and composition of collections to change in the digital environment?


The Science Museum is in the process of creating a series of narrative-based digital documents, 'Exhiblets'. These are intended to perform a number of functions, including acting as online resources for some of the thousands of collections-related enquiries the Museum receives each year. They are valuable learning resources and provide audience-friendly access to the vast range of collections held by the Museum.

Exhiblets draw on information held in various forms, managed by a number of domains across the Museum and beyond. Comprising information drawn from the Museum collections, the Museum's Library, its Archive and existing publications, Exhiblets depend on information being made accessible from these domains at item and collection level, and place this content in a narrative context. The narrative provides a means of leading the user into the subject area and adds meaning to the structured information, making links between resources from the three domains, and building a story around the associations which can be made.

For example, an Exhiblet on the Portsmouth Blockmaking Machinery tells the story of the evolution of the first ever suite of machinery used for mass production. Descriptions and images of the objects themselves, held at the Science Museum and also at other museums across the country, are placed in the context of the early stages of the industrial revolution. Reference is made to the holdings of the Science Museum Library, providing a technological and historical background. Letters between the inventors and manufacturers are drawn from archival sources and biographical information tells the story of how designers and manufacturers worked together to produce the machinery.

The potential for this type of format is great, providing as it does a jumping-off point for a user to follow the trails which lead off from this narrative. Implementations of these Exhiblets will also provide users with the opportunity to provide their own knowledge which the Museum may be able to incorporate into the resource store for use in developing other resources.

Curators as 'knowledge providers' work with researcher-editors to make the production process as simple as possible. However there are a number of issues which have yet to be resolved if this type of resource is to fulfill its potential.

Management

Libraries, archival institutions and museums deploy long-standing procedures and professional practices in the management of their collections. These have been adapted as they move to 'hybrid' collections, collections which contain physical materials and newer digital material. New procedures and professional practice are preliminary, are often exploratory or experimental, and may be confined to limited, highly labour-intensive operations. Routine approaches will benefit from down-cycle technical and business agreement, but practice is still ahead of standards. This will be increasingly unsustainable as memory institutions have to manage large-scale digital repositories, accession large volumes of heterogeneous digital materials, and organise them for immediate use and long-term access. Such digital materials may be internally complex, multiply linked, and created in different computing environments. Fine-grained rights management frameworks will need to be deployed. Effective management of the radically new, and the radically unpredictable, poses many challenges.

Organisational Frameworks

We have suggested that technology, service and business contexts are coevolving to shape opportunities and obligations in a new, shared network space. This has some implications. There are greater opportunities for scale economies as the level of infrastructure rises, in the form of technology or utility services. A distributed service environment requires levels of agreement about how services and organisations relate to each other, as services are recognised as part of a fabric of provision rather than as standalone offerings. New roles and divisions of responsibilities are emerging. And finally, new practices must be institutionalised to secure their stability.

Acknowledgements

We are grateful to Pat Manson, DGXIII, for her support during the preparation of this document. We are also grateful to the several people who have commented on the document in draft. Prepared for DGXIII in the programme area: COLLABORATION BETWEEN ARCHIVES, LIBRARIES AND MUSEUMS UNDER KEY ACTION 3 (MULTIMEDIA CONTENT AND TOOLS) OF THE INFORMATION SOCIETY TECHNOLOGIES PROGRAMME UNDER FP5 (November 1999)

Author Details

 

Lorcan Dempsey
Director, UK Office for Library and Information Networking (UKOLN)
University of Bath, Bath BA2 7AY, UK
email: l.dempsey@ukoln.ac.uk
web site at: http://www.ukoln.ac.uk