Web Magazine for Information Professionals

What Happens If I Click on This?': Experiences of the Archives Hub

Jane Stevenson describes the results of usability testing for the Archives Hub Web site.

For online services, the importance of developing user-friendly and accessible Web sites is of paramount importance. This article is about user testing recently carried out by the Archives Hub [1], an online service run by Mimas [2], which is a national data centre that provides access to a whole range of online services for research, learning and teaching.

The Archives Hub is a JISC-funded service that provides a gateway to search descriptions of archives for education and research, enabling researchers to discover unique archival sources held in repositories across the country. One of our challenges is to create a site that primarily meets the needs of academic researchers but also works well for a whole range of users, from experienced researchers to family historians who may not be so familiar with using online services. But describing archive collections in an online environment in a way that is both immediate and informative presents substantial challenges. An archive usually consists of a whole range of records, accumulated over the course of an individual’s lifetime or during the day-to-day running of a business or organisation [3]. Records are often interrelated in complex ways, forming hierarchies that reflect the creation and use of records over time. But there can be a tendency to assume that archival structure and hierarchy are self-evident. Conveying this structure to a user, especially one who is not an experienced user of archives, is not easy to do. It is therefore important to understand how users react to the presentation of information on the Archives Hub, and whether the layout, design and terminology help to clarify to users what they are seeing. In addition, it is all too easy to design a Web site and build in functionality based upon perceived ideas about users, rather than engaging users directly.

This usability testing was carried out prior to a planned redesign of the Archives Hub Web site, and therefore we wanted to get valuable feedback that would help to inform any changes that we might make. It was intended to be fairly small-scale and low-budget, and focused on our core academic users. Rather than engage external consultants, we particularly wanted to have direct contact with users, or potential users, in order to gain a better understanding of their perspective and how they viewed the Archives Hub. We could bring to the process our detailed knowledge of the service and experiences that we have had with users over time, and we could also carefully control the focus of the sessions.

Creating a Questionnaire

We decided to create a questionnaire that we would use to interview each participant. This would bring some level of consistency and help to provide focus. At the same time, we intended to ask fairly open-ended questions to encourage participants to give us their own thoughts in their own words. We felt that this would yield reasonably structured data, but that it would essentially be qualitative than quantitative. At several points during the sessions, we asked participants about their impressions of the page display, and we tended to ask follow-up questions designed to allow them a free and discursive response. We also asked a number of more specific questions, particularly about terminology used and the layout and content of the actual descriptions.

A questionnaire does introduce a certain level of artificiality, not least because we asked all participants to look at the same descriptions, which we deliberately picked as being fairly typical of the average Hub description. This meant they would not be using the service in a way that reflected their research interests and therefore it might make the service seem less relevant to them. However, if we asked them to enter a search term randomly they may come to atypical descriptions which might lead off into unforeseen tangents and it might be difficult to stick to the questions that we wanted to ask.

The Sessions

We had ten participants involved in the user testing sessions. Five were academics from various disciplines, one was a research associate, one was a first-year student, one had just completed her degree and two were PhD students. This article provides the text of some of their answers pretty much verbatim, though with some editing for the sake of clarity.

The first question that we asked participants was about their understanding of the term ‘archives’. This is the sort of question that may easily be overlooked, but it is clearly central to the use of the Hub and it yielded some interesting responses. The Archives Hub Web site essentially assumes that people understand this term: our tag line describes the site as providing ‘a national gateway to descriptions of archives’.

Question: What do you understand by the term ‘archives’?

There was a general feeling that archives are usually old or historical stuff, and reference was also made to an archive as a store or repository. Some responses were determined by individual experiences: the participant who was involved in family history research talked about census records and marriage records, and the participant who had been working on an oral history project relating to people living in institutions referred to the life and workings of institutions. Overall, there was a general understanding that archives may consist of a broad range of types of sources and that they are usually historical rather than current. But it was clear that some participants did not entirely appreciate the distinction between published and unpublished sources, between archives, monographs and articles. The term ‘archive’ is increasingly being used in diverse ways, not least with the rise of digital information and issues surrounding the long-term access and preservation of a whole range of digital materials. An e-prints repository will often be described as a repository of archives, so it is little wonder that people are increasingly thinking of journal articles as archives. Therefore, the onus must be on us to clarify what archives are in the context of the Archives Hub service.

When we asked about using archives for their research, some participants continued to group published sources together with archive sources. Seven of the participants did use a range of primary source materials to varying degrees: one had a particularly detailed knowledge of older archive sources, having worked with Anglo-Saxon charters, one had experience of using archives in Russia, and another participant was familiar with family history sources.

There was not a great deal of common ground in the way that people search for archive sources, and it very much depends upon their experience, general knowledge of archives and type of research. A number of our participants referred to making direct contact with relevant people or organisations as a preferred option and two of our participants referred to phoning round appropriate bodies; for example, contacting theatre companies (for performance history research). One participant referred to building up bookmarked favourites through following Web links, references in journal articles and generally serendipitous searching. For one researcher, whose work focuses on local history, the main approach is to contact the appropriate local record office. Some participants did refer to online sources: the participant with an interest in family history talked about Web sites such as ancestry.com, and a participant working on the literature and history of the 1640s and 1650s referred to The National Archives [4] and British Library Catalogue [5] as well as various dictionaries and guides. We did ask whether participants were confident using online services generally: five were positively confident, three were fairly confident and two were not really confident. Not surprisingly, those who were most confident had used the greatest variety of online resources.

Most participants did not mention any kind of cross-searching networks - only one mentioned the Archives Hub as a service that they had previously used.

First Impressions

One of our key aims was to get immediate reactions to the Archives Hub home page. Whilst it is true that people increasingly access services like the Hub from different locations, the home page is still ‘the most desirable real estate’ [6] of the site. It should clearly convey the identity and mission of the site as well as what it has to offer, where to start and what the search options are. It should also create a good impression visually, so that it engages people’s attention. There is a difficult balance to strike here. The site needs to sell itself well enough to entice visitors into trying out a search, but too much content can clutter the page and people will not read, and may be put off by, substantial amounts of text. The Hub includes a ‘Collections of the Month’ feature, highlighting collections by featuring a new subject each month. The two subjects covered during the time our sessions took place were Stanley Kubrick, the film director, and British Women’s Emigration.

screenshot (80KB) : Figure 1 : Screenshot of the Archives Hub homepage (October 2008)

Figure 1: Screenshot of the Archives Hub homepage (October 2008)

Question: How would you describe what this site is, and what it is for, from looking at the home page?

The answers given to this question were quite diverse: a good illustration of how people look at the same Web site in different ways. Half of the participants did seem clear about what the site is for, although there was some feeling that understanding of what the site is can only really come from trying out some searches. Two of the participants were misled by the ‘Collections of the Month’ feature, which illustrates the natural tendency people have to scan a page quickly and make assumptions. It was interesting that several participants did not read the tag line; ‘Providing access to archives in UK universities and colleges’, even when specifically asked what the site is for (whilst a couple of participants seemed suspicious of what they thought was such an obvious question!). We were left wondering how we can effectively convey what the site is for when people so often do not read the text, not even one short line that defines the content.

We went on to ask participants ‘what do you think you will find if you carry out a search?’ There was a general feeling that there would be a broad range of material, although a few participants still focused on the Collections of the Month feature. Two participants seemed to be thinking in terms of typical University administration records (minutes of meetings, conference records, staff records), whilst another was not confident that universities and colleges would actually hold archives. On the whole, participants were uncertain what they would find, but there was no indication that this deterred them from using the site. The impression was that they would simply try a search and find out.

The Archives Hub is primarily for the education and research community, but at the same time, we want to attract a broad range of users, and we wanted to find out from the participants whether they thought that the site was of general interest or whether it looked quite specialised.

Question: Do you have any thoughts about who you think the site is aimed at?

Four participants thought the site is aimed at academics/researchers. Most did refer to academics, students and then possibly broader users as well. The answers given generally do reflect our intention that the site has a general appeal, but with an academic and researcher bias.

We ended this general section by asking them if they had any further thoughts about the site.

Question: Can you tell me anything more about your initial impressions of the site?

These reactions suggest that the Archives Hub home page needs to be more explicit about what people will find when they search, and they also show that reaction to design is always a very individual thing - we had a few participants who praised the simplicity and the colours used on the Web site and a few who wanted more complexity and a more vibrant design.

Summary of Findings

I do not have the space here to provide a detailed analysis of the whole questionnaire [7], so instead I will sum-up by listing a number of observations and recommendations that we took from the study.

Provide a Site that Meets Expectations

When we asked participants about the search options available, including the advanced search, there was a strong consensus that the options were what they expected: ‘A good list of options here. It’s the kind of thing we’re used to’. The sessions certainly confirmed our assumption that a prominent and simple quick search box is an essential feature of the home page. Seven out of the ten participants said that the first thing they would do would be to use the quick search, two thought they might use the browse option and one did actually say that he would use the ‘About the Hub’ option to find out more about the service.

The collection description also generally met expectations, or participants said that they did not really have any expectations against which to measure it. Overall, there were a number of references to things on the site being as they would expect, and therefore being reasonably familiar and intuitive to use.

Clarify the Content

We asked several questions about the content available via the Archives Hub, and each question drew different responses, clearly showing a degree of uncertainty. When we asked what would be displayed when participants ran a search, the answers given showed that people did not really know what to expect. Maybe asking a question like this is a little artificial in that people do not typically think that carefully about what they will see beforehand, but the fact that participants referred to articles, pictures, newspapers, museum archives, exhibitions and William Wilberforce (the search was for anti-slavery) does suggest that the Hub needs to be more explicit about the type of content users will find.

Clarify What is Excluded as Well as What Is Included

When looking at the hit list for a search, one participant questioned how comprehensive the list was: ‘Is this everything to do with this subject?’ A couple of other participants assumed the list was a comprehensive list of archive sources. This, of course, is always a difficult issue because we want to promote the Hub as an effective service for locating archive sources, particularly those held in university and college archives, but that does not mean that we describe all of the archives on a particular subject. The challenge is to find a way to describe the content accurately, and ensure that users are aware that there may be other valuable sources relating to their research area.

Use Clear Terminology

We were very keen to focus on terminology in this study, as we were aware that some terms are archival, with headings often taken from the International Standard for Archive Repositories, ISAD(G) [8], which provides guidance on standard elements of description, although it does not recommend headings to use for user interfaces.

Amongst those terms questioned were ‘unpublished handlist’, ‘access points’, ‘related units of description’, ‘item-level’, ‘level of description’ and ‘fonds’. ‘Item-level’ is a very commonly used term to distinguish individual items within a collection, as opposed to a whole collection or a series of items within a collection. Five participants said it did not mean anything to them, two said it was to do with searching for something specific, one said it was to do with only matching a keyword as opposed to every word in text and one thought it would provide some kind of fuller explanation. The participant who had used the Hub before did understand the term in the archival sense. It is not surprising that there was a lack of understanding, although how to refer to items within an archival hierarchy in a way that the majority of people can understand is certainly a challenge.

A term that was almost universally unknown was ‘fonds’, which is archival terminology for a collection of material accumulated by one individual, family or organisation. In a user study by Duff and Stoyanova [9], several respondents thought that ‘fonds’ was part of the name of the record creator, as the collection they were assessing was labelled ‘Margaret Laurence Fonds’. Archivists have tended to carry the term from the professional sphere, where it is useful, into the public sphere, where it simply causes confusion.

The term ‘access points’ also caused some confusion: ‘ What do the access points lead to…maybe to another source or to separate items or general searches?’, ‘ I’m not sure about the meaning of Access Points – are they links to other subjects?’

screenshot (73KB) : Figure 2 : Figure 2: Screenshots showing the top of a search results page, the top of a collection description and the Access Points at the end of the description.

Figure 2: Screenshots showing the top of a search results page, the top of a collection description and the Access Points at the end of the description.

Access Points are, in fact, index terms such as names and subjects that become hypertext links within descriptions, and they lead to a browsing list where users can find other descriptions that use the same index terms. There was a general impression that Access Points would lead to more resources of some kind relating to these subjects: some participants felt that the links might go to external Web sites, others thought that they might relate to the current collection. Whilst it does seem worth rethinking this terminology, when asked whether they would click on the links, all participants said that they would. So the impression was that as long as it was clear that they were links to other related resources of some kind, people would be happy just to click and see what they got. In fact, whilst it would certainly be preferable to simplify some terminology, we did find that participants tended to accept that there was some language that was not clear to them and as long as it didn’t impede searching then they simply ignored it, or assumed its meaning from the context. The only problem with this is that they might therefore make false assumptions that would lead them to misinterpret what they were seeing.

You Cannot Please All of the People All of the Time

Inevitably, there were differences of opinion during the sessions, in particular about the design of the site, which some people praised and others criticised. Participants disagreed on some of the content of the descriptions, such as how useful the biographical information was and whether it would be better to have it further down the order, with the scope and content nearer the top. Two participants were happy that the description was not too long, so that it was reasonably easy to scan through, one thought that maybe there was too much information all at once, whilst one wanted as much information as possible. In general, the feeling seemed to be that the description was about right in terms of length and content. This particular description was about average in terms of what is on the Hub - many of the descriptions are much longer.

People Are Not the Same: Except over the Back Button

Running this questionnaire gave us a chance to find out, at least as far as you can with a small sample of people, how people commonly use the ‘Back’ button to navigate through Web sites such as the Hub. Breaking the Back button was one of Jakob Nielsen’s top 10 Web design mistakes back in 1999 [10]. In fact, all participants without exception said that they would use the Back button when asked how they would return to their results set, so the rule certainly still held for our sample of users.

Are Features a Help or Hindrance?

This particular point is illustrated by the use of relevance ranking to order the results lists on the Hub. We found that participants focused on this when they were looking at the results of a subject search for railways, and were generally puzzled by it:

screenshot (49KB) : Figure 3 : Screenshot of a subject search on the Archives Hub, showing relevance ranking

Figure 3: Screenshot of a subject search on the Archives Hub, showing relevance ranking

Clearly, the relevance rankings confused several of the participants; yet at the same time, many of them liked the idea of ranking results in order of importance. The algorithm responsible for ranking results on the Hub looks at the frequency that a word occurs in different descriptions and then looks at other subject terms within that description. Whilst this may produce accurate relevancy rankings in a strictly technical way, it doesn’t necessarily reflect human perceptions of relevance between subjects.

Users Learn by Using the Site Rather Than Reading the Text

Our findings reinforced this generally accepted user characteristic. Participants often paused when looking at a new page, questioned some aspects of it, and then quite quickly worked many things out for themselves. They asked questions about what this or that is for, or what this term or that term means, but often they answered their own questions after a short pause for reflection:

The browse list probably caused the most confusion, particularly linking to it from the access points: ‘ I’m not sure why some stuff is there - it doesn’t seem relevant to anti-slavery as a subject.’; ‘I was expecting to be taken to another Web site…so I think these are keywords? I’m not sure to be honest…’. Clearly some of the participants expected the display to be related to the subject that they had clicked on (anti-slavery) and were thrown by the other terms that were being listed. However, even when their expectations were not met, participants seemed to re-orientate themselves quickly to what they were seeing, and accepted the list to be some kind of keyword or category list for material about different subjects.

After looking at a collection description, we asked participants about a longer and more complex description that included a table of contents. The impression was that even in the short time they had been looking at the site they felt more confident about what they were looking at, and were more inclined to analyse the description, offering suggestions for improvement:

The overall impression was that participants were gradually orientating themselves and working out what they could see in front of them. They were thinking aloud and adjusting their view of what they were seeing as they absorbed the information. It may be that academic users are more likely to exhibit this behaviour, although a study by Wendy Schier also noted a similar finding with novice users of archive Web sites, who, she noted, tend to go through a fairly rapid process of self-education; once they understand the structure the process becomes fairly straightforward [11].

Last Impressions

The last section of the questionnaire asked participants to sum up what they felt about the site, and here some participants explicitly stated that even if some aspects of the site were not clear, this would not be a barrier to use, and it is fine for there to be some level of learning process involved.

Question: Having looked at the site, what is your overall impression of it?

We asked whether anything could be added to the site to encourage participants to use it, and four of them said that they would be happy to use it as it is. One referred to links to things like the Dictionary of National Biography, one referred to the ability to save searches and one summed up the difficulties of not being aware of the content coverage:

Only one person, in response to this question, referred to links to digital resources (digital surrogates of archives). When we asked whether participants would have expected links to digital content, five of them said that they wouldn’t expect this, which is an interesting result because we do tend to always assume that users expect instant access to digital content. The impression we got from talking to the participants of this study was that researchers, particularly the more experienced researchers, understand that this is not easy to achieve. However, one participant did come up with a compromise:

This is, in fact, something we are planning to introduce into the Hub over the next year.

Conclusions

Whilst numerous studies point to the fact that users will give up on Web sites if they do not find what they are looking for relatively quickly, our study rather emphasised the self-learning behaviour of the participants, and many of them suggested that they would be keen to browse around the site and just see what they could find. It may be that when seeking primary source material to support research, users are prepared to spend longer, as the search is part of the research process and researchers will want to ensure that they locate as many relevant sources as possible. It may also be the case that a user may spend 20 minutes happily browsing through the Hub’s collections even though they have not found what they were looking for, and often users will be performing open-ended searches anyway, where they do not have a specific aim in mind. Certainly, people have very different perspectives and it is not easy to accurately predict what people are going to think and how they are going to behave; but we found a real tendency for people to want to just click and find out what they got, without thinking much about it beforehand. We found that our participants wanted to learn about the site through exploration rather than reading text. They were happy to explore and learn as they went along, as long as they found the content worth exploring.

References

  1. Archives Hub http://www.archiveshub.ac.uk/
  2. Mimas National Data Centre http://www.mimas.ac.uk
  3. Wikipedia: Archives (accessed September 2008) http://en.wikipedia.org/wiki/Archive
  4. The National Archives http://www.nationalarchives.gov.uk
  5. The British Library Integrated Catalogue, http://catalogue.bl.uk/F/?func=file&file_name=login-bl-list
  6. Don’t Make Me Think, Steve Krug, Macmillan Computer, August 2005, ISBN-13: 9780321344755
  7. The full text can be requested by emailing me: jane.stevenson@manchester.ac.uk
  8. International Council on Archives, International Standard Archival Description (General) http://www.ica.org/en/node/30000
  9. W. Duff and P. Stoyanova, Transforming the Crazy Quilt: Archival Displays from the Users’ Point of View, Archivaria 45.
  10. Jakob Nielsen, The Top Ten Web Design Mistakes of 1999, Jakob Nielsen’s Alertbox, 30 May, 1999 http://www.useit.com/alertbox/990530.html
  11. Wendy Schier, Report on a Qualitative Exploratory Study of Novice User Experience with Online Finding Aids, Journal of Archival Organization, 2005, 3⁄4.

Author Details

Jane Stevenson
Archives Hub Co-ordinator
Mimas
University of Manchester

Email: jane.stevenson@manchester.ac.uk
Web site: http://www.archiveshub.ac.uk/

Return to top

Article Title: “‘What Happens If I Click on This?’: Experiences of the Archives Hub”
Author: Jane Stevenson
Publication Date: 30-October-2008 Publication: Ariadne Issue 57
Originating URL: http://www.ariadne.ac.uk/issue57/stevenson/