Usability update

August 12, 2010 at 6:32 pm | Posted in usability | 4 Comments
Tags: JISC, jiscLMS, update, usability

Formal usability assessments of York St John’s WorldCat Local implementation took place, as scheduled, on 29th and 30th June. These assessments took the form of six sessions lasting an hour and a half each. Participants, recruited from existing YSJ students, were asked to undertake five pre-defined tasks and to think aloud as they used the interface. Sessions were guided and video recorded by a facilitator.

We are now firmly in the data analysis phase of the usability project. This is usually the most time-consuming part of any usability assessment, due to the large amounts of rich sense data generated from usability assessments – in our case, six videos averaging an hour in length. We have gathered those bits of quantitative data that we are interested in – task timings, user rankings of tasks (in terms of difficulty) and task success rates. Next comes the job of viewing the videos, applying the Critical Incident Technique to the results and the process of drawing conclusions from that point. We hope to have the second phase of data analysis completed by mid-September.

Like this:

Be the first to like this post.

4 Comments »

RSS feed for comments on this post. TrackBack URI

Hello team Yocalcat! I’ve joined the JISC LMS project to carry out a ‘synthesis liason’ role. I just thought I’d say hello and let you know that yours is the first project I’ve taken an in-depth look at … possibly influenced by the fact that I live in York and have a background in usability testing

It looks like you’re putting a lot of thought into the the documentation you can produce to allow other institutions to learn from your experience which is great. I’ll be doing what I can to spread the word about what you’re doing too.

I’ve been thinking about the issue that you’re facing with being such a small team (in terms of the analysis of usability data and with the additional steps that are being introduced in your cataloguing process) … I wonder whether there’s a way of building in some additional engagement with students by getting them involved as a part of their information literacy learning (and help with your project workload at the same time)? I’ll keep my eye out and let you know if I come across anyone institutions/projects who are making use of students in this way.

Comment by Helen Harrop— August 13, 2010 #

Reply
- Hello Helen – having students help with the data analysis is a really good idea, and I wish I’d thought of it! Unfortunately I think we’ve left it a bit late to involve them, what with it being summer and all. Next time… See you next week in Glasgow?
  
  Comment by ljshipley— September 2, 2010 #
  
  Reply
As I will be absent from the JISC meeting next week in order to attend ECDL in full, I thought I’d post a comment to say hello

I look forward to reading the full report. I’ll be particularly interested in the quantitative data captured, including task time and success rate. This is something we decided not to capture due to the difficulty in determining these aspects with accuracy and real meaning for our project. We’re currently half way through our testing and hoping to share the findings soon.

Comment by Lorraine— August 30, 2010 #

Reply
- Hey Lorraine Thank you for the comment. The quantitative data is very minimal, and to be absolutely honest I’m not sure how meaningfulit is either – I share your project’s concerns about accuracy, and I’ve always found more value (ie, more useful conclusions and moments that make me go ‘oh yeah – there’s another design assumption that makes no sense at all…’) in the qualitative side of usability.
  
  Comment by ljshipley— September 2, 2010 #
  
  Reply

Yocalcat's JISC LMS Blog