OCLC WorldCat Local usability study

JISC LMS Project: usability report

A single iteration usability evaluation of York St John University’s implementation of OCLC WorldCat Local

Date of report	4th October, 2010
Date of evaluation	29th and 30th June, 2010

This document describes a broad brushstrokes usability evaluation of WorldCat Local that was conducted in order to understand how returning students would react to the change in library catalogue.

Findings indicate that users generally responded positively to the change, but will need to be encouraged to revise their general search strategy to adapt to WorldCat Local’s strengths. Users found simple searches for known items straightforward, but speculative searches caused more of a problem as users often found it difficult to identify the type of item they were looking at. In particular, users struggled greatly with the task concerned with journals, and many seemed unaware of the difference between journals and articles.

Some of the difficulties users got into can be explained by flaws in the WorldCat Local interface or local configuration, but others might be more usefully addressed through the information skills strategy and our underlying cataloguing processes.

Table of contents

Methodology
Findings and recommendations
Introductory questions
Task 1
Task 2
Task 3
Task 4
Task 5
Concluding questions

Methodology

What we evaluated, and why

The purpose of a library catalogue is to provide a searchable index to a library’s collections. The core function of such software is to reveal the extent of those collections to the user. A library catalogue should enable a user to easily explore a collection and, once a desired search result is chosen, it should clearly deliver the information the user requires in order to be able to locate the corresponding physical or electronic item for use.

Findings from a previous study indicated that our existing library catalogue, SirsiDynix’s iBistro, did not provide an intuitive user experience while performing these core functions. Most significantly, the interface failed to give sufficient context or feedback to the user. Informed choices are difficult to make, and the user must either guess the best course of action or spend a non-trivial amount of time learning the quirks of the interface in order to succeed (for example, librarians who have been using the interface for some years have few problems with it; new users using it for the first time are often left guessing). In particular, an ambiguity of terminology and unpredictability in patterns of interface behaviour mean that the user often cannot accurately evaluate either their position in the interface or the meaning of the results they had been served with.

This usability evaluation explored the suitability of OCLC’s WorldCat Local as a replacement resource discovery interface for iBistro. Our intention was to try to improve our understanding of how returning students will approach using WorldCat Local given what they have already learned of iBistro, and then to use that improved understanding to inform the development of information skills sessions for new and returning students. We also wanted to base any configuration changes made to WorldCat Local on evidence rather than instinct.

What happened during the usability evaluation

The usability evaluation of York St John University’s implementation of WorldCat Local (WCL) was conducted by Graham St John on 29th and 30th June, 2010.
During the evaluation, six participants were asked to spend between 30 minutes and one hour with the site. During this time, participants:

• answered some general questions about the site
• performed real-world tasks on the site while thinking aloud
• rated tasks on completion according to perceived difficulty and success

Who participated

Six participants were recruited through the student volunteering service from existing York St John students. Participants were offered a £10.00 Amazon gift voucher and an OCLC goody bag in return for taking part in the evaluation. All participants were aware that they were being asked to evaluate a library catalogue, and all had used our existing catalogue previously. We were interested in observing how patterns of learnt behaviour might affect returning students’ adoption of WorldCat Local.

Participant	Course	Year of study (in June 2010)
1	Business management	3
2	Medicine	1
3	English and theatre	1
4	Management	MA
5	Primary education	2
6	Communication and IT	2

Table 1: participants’ courses and years of study

What the participants did

During the evaluation, participants were asked to complete five tasks using WorldCat Local. The tasks were presented in the same order for each participant, but none of the tasks depended on another for success.

The following tasks were devised after scrutiny of the results of previous usability studies carried out on this and other library catalogues, and from input from library staff (particularly the Academic Liaison Librarians).

#	Task
1	You want to know if the library has any copies of The lion, the witch and the wardrobe available to borrow. How would you use WorldCat Local to find this out, and how would you find them on the shelves?
2	You need to write a literature review on recent books on emotional intelligence. How would you use WorldCat Local to start researching on that topic?
3	You are looking for an article from the 1999 53(4) issue of the American Journal of Occupational Therapy. How would you use WorldCat Local to see if we have access to this issue, either in print or electronically?
4	You would like to access the electronic version of a book about occupational therapy for children by Jenny Ziviani. How would you use WorldCat Local to do that?
5	You think that some of your books might be overdue. How would you use WorldCat Local to check their return dates, and renew them if possible?

Table 2: tasks undertaken by participants

The data we collected

Participants were asked to think aloud while working their way through the five tasks. We captured video of the computer screen and the accompanying audio of the session. The observer was allowed to prompt the participant to explain the rationale behind a particular choice or action, but was not allowed to otherwise intercede unless it became obvious that the participant had entirely misunderstood the task they were working on.

Although we were primarily concerned with collecting the rich sense data that resulted, in the form of video, audio and observer’s notes, from these sessions, we also gathered more quantitative data such as task timings, success rates (both perceived and actual) and perceived difficulty ratings.

How we analysed the data

Analysis of the quantitative data gathered consisted of simple comparisons to draw patterns. Any conclusions drawn from this information must, necessarily, be qualified by the small number of participants.

A rather more complex method of data analysis was employed when dealing with the rich sense data resulting from the audio and video recordings. We began by watching the video footage through completely in order to ascertain task timings. In this phase we also noted down brief comments at points that might be considered a critical incident. A second pass of the footage allowed us to identify those moments more clearly and begin to associate them with the headings in the table of critical incidents (see below). The small number of participants is less of an issue with these results, as the sample size is measured in the number of critical incidents observed.

Theme/Category	Description
Layout
Clear layout	The layout of the interface, from a design perspective, guides the user into making a good choice
Unclear layout	The layout means the user has to guess what to do
Ambiguous terminology	The words used to describe elements in the page – such as links – are ambiguous or unclear, meaning that the user cannot guess at their function
Context
Clear context	The interface offers a clear explanation of what it does, guiding the user into making a good choice
Unclear context	The interface offers ambiguous explanation of what it does, meaning the user has to guess what to do
No context	The interface offers no explanation of what it does, meaning the user is at a complete loss as to what to do
Not sufficiently specific	Existing content is general or vague, meaning the user has to guess what to do
Feedback
Clear feedback	The interface gives very clear feedback, meaning the user understands clearly what to do
Unclear feedback	The interface offers ambiguous feedback, meaning the user has to guess what to do
No feedback	The interface offers no feedback, meaning the user is at a complete loss as to what to do
Expected results	The interface behaves as the user expects, meaning the user understands clearly what to do
Unexpected results	The interface does not behave as the user expects, meaning the user either has to guess or is at a complete loss as to what to do
Information overload	The user cannot assimilate the amount of information the interface is delivering, and misses something important
Guessing
Epiphanic moment	The user enjoys a moment of intuition that results in understanding
Lucky guess	The user makes a guess that happens to yield the hoped for result
Educated guess	The user makes a guess based on previous experience that yields the hoped for result
False epiphany	The user enjoys a moment of intuition that does not result in understanding
Lost
Floundering	The user is guessing without insight at what to do
Dead end	The user has come to an impasse and cannot think of anything more to do
User behaviour
False assumption	The user assumes something about the interface that is actually not true
Other issues
Broken site	The interface suffers a technical problem that is beyond the power of the user to resolve or negotiate

Table 3: critical incidents

Where we evaluated

Usability audio and video were captured using an UltraHD Flip videocamera. Sessions took place in a private interview room containing a desk, a PC and two chairs. We provided water. The observer, sitting in the room with the participant in order to facilitate the evaluation, took notes using pen and paper.

URL tested	http://yorksj.worldcat.org
Computer details	Intel Pentium 4; 2GB RAM
Browser used	Microsoft Internet Explorer 8
Screen resolution	1024×768
Operating system	Windows 7 Professional
Connection	University wired network

Table 4: computer settings for evaluation sessions

Findings and recommendations: introductory questions

At the beginning of each session we started on the initial search screen and asked the following question:

• do you have any comments about this page?

Figure 1: http://yorksj.worldcat.org

Participant	Comment
1	“It looks good.” – also noticed the languages options
2	“It looks quite easy.”
3	“It’s a bit nicer than the old one, I’ve got to say.”
4	Participant 4 stated that the interface looked user-friendly, but also made some suggestions for improvements to the page layout – positioning of elements such as the logo, search box and languages options.
5	“Not vastly different really, is it? Just looks a bit more professional.” – also wanted to know where the eJournal Finder link had gone

Table 5: initial comments from the participants

Findings and recommendations: the tasks

Task 1: You want to know if the library has any copies of The lion, the witch and the wardrobe available to borrow. How would you use WorldCat Local to find this out, and how would you find them on the shelves?

Rankings and timings

Participant	1	2	3	4	5	6
Difficulty	1	1	1	2	1	2
Outcome (actual)	S	S	S	F	S	S
Outcome (perceived)	S	S	S	S	S	S
Task timing	02:08	01:11	01:00	n/a	00:51	03:03

Summary

• Five out of the six participants managed to succeed at this task; the sixth thought he had succeeded but he had not
• Two participants were unfamiliar with the title and had to be corrected by the observer after initially starting to search for ‘The Lion’, ‘The Witch’ and ‘The Wardrobe’ as three separate titles – this may have had implications for how they used (or did not use) the facets further on in the task, as they were searching for an unknown item whereas the other four participants were searching for a known item
• Of the six participants, five used the search box presented to them at http://yorksj.worldcat.org and one selected ‘Search for Library items’ from one of the drop-down menus at the top of the screen
• Participant 4 consistently over-estimated his ability to find accurate information, and more importantly, to interpret it correctly – this led to a disparity between his perception of success and the actual outcome observed

Critical incidents

The search

This task was characterised by most participants clearly understanding how to begin their search (clear layout), whether that was through the simple keyword search box initially presented to them or through alternate routes. All participants eventually elected to search for some variation of ‘the lion, the witch and the wardrobe’ as a simple search.

The results

The presentation of the brief search results seemed familiar to all participants, although some had trouble differentiating between the different types of material returned in the results – articles, books and DVDs (unclear layout). Participants 4 and 6 particularly did not seem to notice that the results were a mixture of item types (false assumption); Participants 1, 2, 3 and 5 did notice these distinctions, with Participants 2 and 5 making useful choices based on this information (clear layout). Participants 2 and 3 made use of the facets – one narrowing by the book format, and one by the author C. S. Lewis – to narrow down their search results (clear layout) whereas the other participants scanned the bibliographic details and other information presented in the brief search results to make their choice: all but one found the appropriate book with this method.

Finding the item’s physical location

Two participants had slight problems with locating the holdings information for the item, once they had found a search result they were happy with. Participant 1 minimised the holdings information almost before looking at them and took a few moments to maximise them again and find and interpret the correct information (educated guess). Participant 4 was misled by the manner in which the holdings information for his result was displayed. Although he had not selected the correct item, he could still interpret the holdings information. Unfortunately, it displayed a withdrawn item as available (coloured in green), leading Participant 4 to assume that the item was available to borrow (unclear feedback). All other participants located and interpreted the holdings information for their item without issue, and were confident that they would be able to find it in the library (clear layout).

Significant incidents

• Participant 1 – floundering – minimised the holdings information
• Participant 4 – false assumption – looked for another search box to start with
• Participant 4 – unclear feedback – withdrawn items showing as available
• Participant 6 – false assumption – briefly thinks that the OCLC# might be the call number

Recommendations

Information skills

In terms of information skills, it appears that it will be important to emphasise the existence of facets and how they work as none of the participants made the best use of this facility in this task – indeed, some seemed not to notice them at all. This will be particularly true for returning students who are used to not being able to further refine the jumble of results presented by iBistro.

WorldCat Local interface and configuration

The interface elements that reveal material type aren’t, on the whole, obvious enough, as almost everyone ignored them. This could be partly because the participants were not expecting articles to show up in search results because they don’t in iBistro, but other media (such as DVDs) seemed to surprise some of them as well. It may be within our control to darken the font colour (which is a very pale grey at present), but other changes – such as larger icons for material type – would have to be requested as enhancements from OCLC.

One participant commented that the presence of the item’s call number on the brief search results page would be useful – this is beyond the scope of our configuration options to implement, but might be requested as an enhancement from OCLC.

At the time of evaluation, some local configuration options had not been fully explored. The situation Participant 4 found himself in, with withdrawn items showing as available, has since been amended. Participant 4’s experience, however, emphasises the importance of providing accurate data to WorldCat Local as even small inaccuracies may mislead the user.

Task 2: You need to write a literature review on recent books on emotional intelligence. How would you use WorldCat Local to start researching on that topic?

Rankings and timings

Participant	1	2	3	4	5	6
Difficulty	1	2	1	2	1	1
Outcome (actual)	S	S	S	F	S	F
Outcome (perceived)	S	S	S	S	S	S
Task timing	01:08	n/a	00:57	n/a	02:00	n/a

Summary

• Participant 4’s results have not been included in the discussion as he misunderstood the task so thoroughly that a comparison of his experience with the others’ would be meaningless
• Some participants did not understand what a literature review is, meaning it took them a little time to understand the task
• Although no one was particularly confident performing a speculative search, one participant was particularly uncomfortable with the process

Critical incidents

The search

Participants 2, 3 and 6 elected to perform a simple keyword search using the search terms ’emotional intelligence’. The other two participants chose an advanced search (clear layout), Participant 1 performing a title search and Participant 5 a keyword search, both for ’emotional intelligence’. It is unclear whether either participant really understood the difference between keyword and title searches, but in this instance the returned results did not make this distinction a problem.

The results

All participants except Participant 6 (floundering) found some way to limit the results by date. Participants 1 and 5, who used the advanced search, made use of the date limiter option to limit before searching (clear layout), whereas Participants 2 and 3 used the facets to limit by date after the search had been performed (clear layout). Either method would allow the user to complete the task, although the method using the advanced search could be more efficient as it allows for a range of dates to be specified. Participants 2 and 6 were not aware that their search results included articles (the task asks for books), although Participant 6 later realised that this was the case (epiphanic moment). Unfortunately this led to the false assumption that journals and articles are the same thing, again possibly because iBistro did not include articles in search results and the participant was not expecting articles to be present.

Significant incidents

• Participant 1 – unclear layout – ignores the facets and returns to the advanced search to amend results
• Participant 5 – unexpected results – the York St John logo led back to the library homepage, not to a new catalogue search
• Participant 6 – epiphanic moment – “So the journals and the books are together?” “It’s all mixed up.”
• Participant 6 – false epiphany – now thinks that articles and journals are the same thing

Recommendations

Information skills

Users do not seem to be having a problem using WorldCat Local to search for things, although (somewhat understandably) they aren’t making entirely informed use of the advanced options for searching. However, problems are occurring in the interpretation of the search results – some users are ignoring the facets entirely. The results from this task echo those of the first task: indicating the usefulness of the facets to the users will be important, particularly if they have used iBistro in the past.

Rather more alarmingly, some users seem to be unaware of the difference between books, journals and articles. It is probable that this uncertainty is caused by a variety of issues: elements in the interface may not be obvious enough; past experience using iBistro has taught users to not expect certain kinds of item to appear (specifically journal articles); and a genuine lack of awareness on the part of the students that finding different kinds of items requires different discovery strategies. Of course, it is an aim of WorldCat Local (and, to some extent, our aim in adopting WorldCat Local as our library catalogue) to smooth out these differences. Nevertheless, due to the information available in the underlying catalogue records, users will still need to be able to be taught to differentiate between the different types of material available in the catalogue in order to make best use of the resources provided by the library.

WorldCat Local interface and configuration

At the time of evaluation, the York St John logo pointed to the library webpages, which caused some confusion. This has since been amended.

Task 3: You are looking for an article from the 1999 53(4) issue of the American Journal of Occupational Therapy. How would you use WorldCat Local to see if we have access to this issue, either in print or electronically?

Rankings and timings

Participant	1	2	3	4	5	6
Difficulty	2	2	3	4	3-4	5
Outcome (actual)	F	S	F	F	F	F
Outcome (perceived)	S	S	S	S	F	F
Task timing	n/a	02:35	n/a	n/a	n/a	n/a

Summary

• Participant 6 has been mostly excluded from the discussion because limited video footage was captured of her experience of this task
• This was the most complex task the participants were asked to perform, and it shows in both the success rates and the time it took participants to navigate the systems involved
• Only one participant managed to correctly ascertain the extent of our print and electronic holdings for this title
• Participants showed a limited understanding of the relationship between journals and articles
• The task was phrased to represent a question a user might come to the catalogue to answer, and the results illustrate how the underlying data feeding the catalogue is not necessarily designed to answer this sort of question

Critical incidents

The search

Participants chose a variety of approaches to answering the question posed by Task 3, some getting to the point where they should be able to locate the required information through the record for the journal title and some following an article record. Participants 1 and 6 began with an advanced search, but Participant 1 decided on a simple keyword search after 6 minutes (floundering) and Participant 6 did the same after 2 minutes (false assumption). Except Participant 1, all participants eventually entered a simple keyword search for ‘American Journal of Occupational Therapy’ (educated guess). Participant 4 then qualified that search by adding ‘1999’ (false assumption, unexpected results) as a keyword.

The results

None of the participants seemed terribly comfortable interpreting the results of their search. Three of the six used facets to narrow down their results; Participant 5 thought about using the facets but did not identify one that answered his question (unclear context). Of the three participants who did choose facets, two chose to narrow by date to 1999 and one chose to narrow by format to journal. Four of the participants (2, 3, 4 and 5) selected their record by looking at the bibliographic information on the brief results screen; most looked for 1999 and 53(4). Out of those four participants, Participants 2 and 4 choose to look at an article and Participants 3 and 5 choose the journal title.

Establishing access

Participants 3, 4 and 5 all had difficulty interpreting the correct item’s call number as it is a bound volume and contains no part numbers. All three found the correct item, but discounted it because it does not match the pattern they are expecting (false assumption/unexpected results). Participant 2 found the link for electronic access to the item, but was uncertain whether or not to choose the 856 link or the ‘Check for Electronic Resources’ button (unclear layout). Of those participants who found their way to the eJournal Finder, which should reveal electronic holdings, Participants 3 and 4 misinterpreted the information in front of them and came to the wrong conclusion that we do have electronic access (false assumption/unclear layout).

Significant incidents

• Participant 1 – floundering – spends around 6 minutes trying to get an answer out of the advanced search
• Participant 4 – false assumption – expects the correct result to be ranked in the first few answers without really understanding how relevance is calculated
• Participant 4 – false assumption – expands scope of search to Libraries Worldwide
• Participant 5 – floundering – he isn’t confident, but he isn’t reading the information on the screen either

Recommendations

Information skills

Few of the participants seemed to understand how journals are organised, or the relationship between journals and articles. They were also largely unaware of the limitations in the information stored in the underlying catalogue (ie, the year 1999 will probably not appear in the journal title record). How much the confusion about journals and articles is down to information skills, how much is down to how libraries generally manage eResources and cataloguing practices, and how much is down to the limitations of the interface is unclear. It is recommended that further work be carried out in this area in order to better understand the extent and nature of the problems faced by users.

WorldCat Local interface and configuration

The participants were attempting to use WorldCat Local to answer the task in a logical manner, but the interface does not make it easy for the user to ascertain the extent of our print and electronic holdings. It would be helpful to make summary information about local print and electronic holdings more obvious – this may be beyond the scope of our configuration options, but some kind of enhancement request could be made of OCLC.

It would also be helpful to remove the dual options for electronic access for eJournals and articles – eJournals could rely on the 856 field to pass the user through to the resource; articles can be passed by WorldCat Local to the eJournal Finder, which should then let the user know whether or not the library has access to that particular article. Some work to amend this situation has already been carried out.

Task 4: You would like to access the electronic version of a book about occupational therapy for children by Jenny Ziviani. How would you use WorldCat Local to do that?

Rankings and timings

Participant	1	2	3	4	5	6
Difficulty	1	2	1	3	2	1
Outcome (actual)	S	F	S	S	S	S
Outcome (perceived)	S	S	S	S	S	S
Task timing	04:34	n/a	00:58	16:17	04:10	?

Summary

• Participant 6 has been excluded from the discussion because no video footage was captured of her experience of this task
• The use of the phrase ‘Internet Resource’ to demarcate an eBook was generally considered unhelpful
• Most users clicked on the link from the 856 field in favour of the ‘Check for Electronic Resources’ button, but the existence of that button was confusing

Critical incidents

The search

Some participants found the initial search difficult because they misinterpreted the question as asking for a specific title. Once these participants had been corrected, most took the option of searching for ‘Jenny Ziviani’, either as an author in the advanced search or just as keywords in the simple search, or searching for ‘occupational therapy children’ as a keyword search (clear layout).

The results

There was some confusion when article results were presented in the search results with Jenny Ziviani as an author, although most participants discounted them either by accident through choosing to filter results via the ‘Book’ or ‘Internet Resource’ facet, or by design through noticing that they were not explicitly about occupational therapy for children. Again, participants seemed to have difficulty identifying these items as articles (unclear layout) and discounting them for that reason. Participant 1 made use of the ‘View all editions and formats’ link (lucky guess), whereas Participant 5 ignored it entirely (unclear layout). Participant 1 also identified the eBook by its icon (educated guess) whereas the other participants seemed to rely on the presence of the 856 link to alert them to the availability of an eBook.

Establishing access

Once participants had found the correct record, four of the five found the 856 link quickly and navigated through to the eBook (clear layout). Participant 2, however, chose to click on the ‘Check for Electronic Resources’ button (unclear layout) and found herself in a loop in eJournal Finder and iBistro. Although she did eventually find her way to the eBook, it was through iBistro not WorldCat Local.

Significant incidents

• Participant 2 – floundering – chopping and changing limiting options until she stumbles on internet resource
• Participant 2 – unclear layout – chooses ‘Check for Electronic Resources’ instead of the 856 link, which leaves her in a loop between eJournal Finder and iBistro
• Participant 3 – false assumption – starts to limit to ‘computer file’
• Participant 5 – floundering – spends a long time looking for a way to limit to eBook

Recommendations

Information skills

Users seem to understand rather better the relationship between books and eBooks than they do the relationship between print and electronic journals. This is perhaps because it is a more simplistic relationship – one-to-one rather than some-to-some. As such, continuing to educate users as to the existence of eBooks, and to show them how to identify them within WorldCat Local, seems to be a sensible strategy. Educating users to perform a broad search first and then filter results by format (in this case) might also be helpful.

WorldCat Local interface and configuration

A few of the interface elements, some editable by us and others not, had the potential to cause confusion to users searching for eBooks. The ‘Check for Electronic Resources’ button passes users to the eJournal Finder, which does not contain information about our eBook holdings. We have since removed this option for eBooks.

It is also probable that eBooks would often become buried behind ‘View all editions and formats’ in the simple search results as a side-effect of WorldCat Local’s implementation of FRBR (Functional Requirements of Bibliographic Records). We have since made some cataloguing choices that should alleviate this problem by linking print and electronic records so a user will see the 856 link whether or not they view the print or electronic versions of the title.

The use of ‘Internet Resource’ as the label for eBooks is unhelpful as an internet resource could be many things. It is beyond the scope of our local configuration options to change this, but it would certainly be something worth suggesting to OCLC as an enhancement.

Task 5: You think that some of your books might be overdue. How would you use WorldCat Local to check their return dates, and renew them if possible?

Rankings and timings

Participant	1	2	3	4	5	6
Difficulty	1	1	1	3	1	1
Outcome (actual)	S	S	S	F	S	S
Outcome (perceived)	S	S	S	S	S	S
Task timing	instant	instant	instant	n/a	instant	?

Summary

• Participant 6 has been excluded from the discussion because no video footage was captured of her experience of this task
• On performing this task, it became apparent that the WorldCat Local was not passing users through to iBistro in order that they could log in to their account
• Of the six participants, only one did not complete the task and only one (Participant 4) was confused by the multiple sign in options offered by WorldCat Local. Significantly, this participant was not aware that it has always been possible to log in to your library account to check loans, fines and renew items

Critical incidents

Five of the six participants found the correct link quickly, and had no hesitation in asserting that it was the correct link – despite the fact that the process to pass the user through to the underlying library account information was not working (broken site). Participant 4 rejected the correct link, and chose the ‘Sign in’ option at the bottom of the initial search screen, causing him to attempt to log in to WorldCat Local with his University username and password (false assumption).

Significant incidents

• Participant 4 – false assumption – hovered over the correct option, but rejected it, electing to sign in to WorldCat Local by the link at the bottom of the page instead

Recommendations

WorldCat Local interface and configuration

It may be sensible to add further options to the drop-down menu, such as ‘Check what you’ve borrowed’. Although such links would lead to the same place as ‘Renew your library books’, they do so in answer to a different question from the user.

Obviously a single place to sign on would be far less confusing to the user, but in most cases it seems that the link text for renewals disambiguates nicely. It is also not clear how a single place to sign on might be achieved, considering the process is essentially to fake a sense of connectedness between two disparate pieces of software.

Since the initial evaluation, we have fixed the link between WorldCat Local and the library accounts held within iBistro.

Findings and recommendations: concluding questions

• overall impressions of WCL
• any of the tasks you’d like to comment further on?
• any closing comments about WCL?

Participant	Comment
1	Participant 1 found WorldCat Local easier to use than the old catalogue. “I would recommend this.”
2	Participant 2 would prefer advanced search options on the initial search screen
3	“It’s like searching Amazon.” Easier to use, the facets are good. Would like the blurbs back (enrichment)
4	Participant 4 would prefer to be able to limit before searching.
5	Participant 5 liked the facets and that the search box appeared on all pages.
6	Participant 6 found the layout easy and simple to use, but preferred iBistro’s up-front search limiters.

Table 6: concluding comments from the participants

From these concluding comments, and the patterns of behaviour observed as the participants completed the five tasks, it is clear that library users with previous experience of iBistro will have to be encouraged to re-think their general search strategy. WorldCat Local encourages the user to throw keywords at it, and then to use the facets and other limiters to narrow down results until the desired item is found.