CCPL Usability Analysis
Date: Jan. - Mar. 2017
Course: User Experience & Evaluation
Role: UX Researcher
Methods: Heuristic Evaluation, Cognitive Walkthrough, Competitive UX Evaluation, Remote Unmoderated Usability Testing
Background: The Charleston County Public Library (CCPL) is the public library system for Charleston County, South Carolina. I was tasked with picking an organization whose website could benefit from an expert usability analysis. I chose the CCPL website because it clearly had not been updated in a long time and I believed my analysis and recommendations could benefit them in a potential re-design, as well as benefit all the patrons of the CCPL system who use the website on a regular basis.
Goal: Discover as many usability issues with the CCPL website as possible through a variety of research methods. For each issue uncovered, I also wanted to recommend any and all immediately actionable improvements to be made to improve usability for the website. My final goal was to present my findings to the director's board of the CCPL and help them with a re-design of their website. Since I completed this project, the CCPL completely re-did their website and implemented many of my suggestions.
Heuristic Evaluation slide deck
Cognitive Walkthrough slide deck
Competitive/Comparative Evaluation slide deck
Final video presentation on usability testing findings
To do a full evaluation of the CCPL website, I employed a four-prong approach utilizing different methods to discover as many usability issues as possible by triangulation.
Expert Heuristic Evaluations
Four UX designers, myself included, were enlisted to conduct a heuristic evaluation of the CCPL website to determine violations against Nielsen's 10 Heuristics. We all collaborated on a Google spreadsheet where we could fill out any and all violations of the heuristics that we saw fit.
The heuristic evaluations uncovered 42 usability issues on the CCPL website of varying severity. The most severe issues included extremely unclear navigation, page headings, and section labeling. Several crucial parts of the website that users might want to find, such as finding out information on how to become a member of CCPL or events happening at the library branches, were hidden behind a confusing and clunky navigation scheme. For each issue I found, I made a correlate actionable recommendation. These recommendations will help the CCPL website immensely in fixing its usability flaws.
View my slide deck for a more thorough look at my process.
The second step in my process was to conduct a cognitive walkthrough where I picked one crucial task on the CCPL website that the majority of users will utilize to analyze step-by-step. I decided to analyze logging into the system and placing an item on hold for checkout. I then wrote out each step necessary in the process and asked the questions [look up cognitive walkthrough questions] and analyzed the results.
During the log-in process, I forgot the 4-digit PIN number that I was required to make when I signed up for my CCPL membership and I discovered a crucial usability issue. If a user forgets their PIN, there is no way for them to retrieve the PIN or reset it. The system does not prompt the user with any information on how to retrieve their PIN, either. Thus, I actually was not even able to complete the entire task for my cognitive walkthrough because of this breakdown.
Employing this cognitive walkthrough proved to be extremely beneficial to my process as it led me to uncover this critical breakdown that is impossible for users of the CCPL website to work around. I made the recommendation to implement a system in which users could answer security questions when they sign up for their CCPL membership like many other websites do. Thus, if users forget their PIN, they can answer security questions and the system can send them their PIN via e-mail.
View my slide deck for a more thorough look at my process.
Since a county library system does not necessarily "compete" with other library systems since it is the only one available in the county, I chose to do a hybrid competitive and comparative evaluation. For the comparative evaluation, I compared the CCPL website with the websites of the two other largest counties in the state of South Carolina. For the competitive evaluation, I chose to compare the CCPL site with an online repository of eBooks since a site like that could drive potential CCPL users away from utilizing their services.
After creating a competitive/comparative matrix, I found that the other counties' websites were updated to modern standards with clear navigation, headings, easy-to-use calendar and search features, as well as prominent displays of all the services the libraries have to offer its patrons. Studying the other libraries' websites really helped me vision what a re-design of the CCPL website could be. For the online eBook repository, I mainly focused on the services it could provide and less on the actual website itself. I found that, because of copyright laws, only books written before 1923 could be downloaded. Thus, the advantage goes to the CCPL library for being able to provide 1) media of all times and 2) media from all time.
View my slide deck for a my thorough look at my process.
Remote Unmoderated Usability Testing
After conducting my own investigations into the CCPL website, I wanted to see how potential users actually navigate through the site. I tend to prefer moderated user testing but due to a time constraint of only two weeks and a goal of attaining almost 100 participants, I chose to do remote unmoderated user testing. I used the UserTesting.com and Loop11.com services to recruit participants.
UserTesting.com records participants’ screens as they complete the tasks you set out and they are encouraged to talk through their thought processes. I recruited three participants through this service and received three videos of usability tests to analyze. I laid out 4 tasks for the participants of UserTesting.com and Loop11. I asked the users to really describe what they were thinking as they were completing the tasks so I could get some good quotes for the CCPL board about their website.
Loop11.com does not record the participants’ screens or voices; it just collects user metrics such as page clicks and navigation schemes, heat maps, time spent on certain pages, and success rates and failure rates of tasks. Since I could not hear the users’ thought processes, I ensured the tasks would be simpler ones, and also ones where I would benefit from knowing that user metric data. Particularly, I wanted to know the success and fail rates of certain navigation schemes within the website by employing tasks where users had to get from point A to point B. Seeing the high failure rates of users trying to find certain information or pages was particularly beneficial for my recommendations in create a clearer navigation scheme throughout the website.
Based on the user testing, I compiled recommendations based on highest to lowest priority depending on what tasks the users failed at the most. Implementing a better navigation scheme and headings was the issue that seemed to be the most severe, then other issues such as aesthetics could be the least severe.
View my final video presentation on the results of the usability testing and the recommendations I make with prioritization going to the most severe usability offenders.