Usability Study:
Colorado Unemployment Website
This study was conducted to evaluate the overall usability of the Colorado unemployment website. There was a heavy focus on finding out whether users are able to successfully get to where they need to on the website to accomplish their goals.
Designed and Conducted by Mckenna Brewer & Chance Wills
Applying for unemployment is a process that no one wants to go through, but over a million Americans currently receive unemployment benefits every month. Each state has its own unique process of applying for unemployment, and Colorado is no different. Any citizen of Colorado looking to apply for unemployment must use the dedicated unemployment website created by the Colorado Department of Labor and Employment. All issues surrounding unemployment must be solved through the use of this website, as there is no dedicated in-person unemployment office citizens can just walk into. The site takes users through the steps of applying for unemployment benefits, provides users with assistance managing their claims, and offers job placement resources. At first glance, the website seems pretty standard for a government run site; a little confusing in places but generally usable. However, the more you explore the site, the more visual and navigation errors begin to appear.
Specific problems include poor navigation, inconsistent visual design, and a variety of redundancies that lead to an overwhelming amount of information for users.
IMPORTANT CONTEXT
In the United States, the most common person applying for unemployment is:
A high school graduate who may or may not have some college experience or an associates degree
Was last employed in natural resources, construction, or maintenance occupations
Is between the ages of 25 to 54
Colorado has the slowest rate of claim processing and first payments in the entire country.
When considering the background and circumstances of the users of this site, the website's seemingly minor usability issues begin to seem a lot more severe.
1. What is the relationship between technological ability and a user's ability to overcome small usability issues?
2. How successfully do users actually begin the unemployment application process upon their first visit to the website?
3. How well (and easily) can users navigate through the website?
4. How easily can users find the answers to common questions about unemployment?
5. Are users satisfied with the overall experience?
RESEARCH QUESTIONS
Based on the above goal, this study aimed to answer the following research questions:
PROBLEM: The Colorado Unemployment website is difficult to use and creates a barrier to access for many users with poor technology skills.
GOAL: Test the severity of the site's problems, determine whether or not these problems inhibit usability, and identify specific design solutions that could improve usability.
DISCOVER PHASE
DEFINE PHASE
METHODOLOGY
SESSIONS:
This usability study was conducted using a summative testing method in which data was collected by an evaluator and through participant questionnaires. The tests conducted were designed to adhere to a within-subjects experimental design. The testing sessions were designed to last approximately 30 minutes. There were six tests conducted in total, one pilot test and five official tests. The pilot test and one usability test were conducted in person, while the remaining four usability tests were conducted remotely over Zoom.
Participants were asked to complete four tasks on the Colorado Unemployment website. Their performance was evaluated using these metrics:
Successful Task Completion
Completion Time
Following of The Happy Path
Error Rate
TASKS:
Participants completed four tasks centered around key interactions that users might use on the site. Scenarios were created to help guide the participants through the tasks.
DATA COLLECTION:
Both attitudinal and behavioral data was collected during the test. Metrics were collected by the test evaluator on an evaluator log sheet, and by pre, during, and post task questionnaires.
PARTICIPANTS:
In the test plan for this study, we initially identified the desire to test users from a range of age, technical ability, and employment experience. Unfortunately, we were unable to recruit such a broad sample due to researcher limitations. One of the researchers had to rush their recruitment and testing timeline in anticipation of Hurricane Debby and possible power/wifi outages. This led to them having to interview from a smaller personal network that would be available before the storm. Due to this adjustment, four out of five participants were within the age range of 20-24. We were able to recruit one participant to represent the 51-65 age range (it is worth noting that our pilot test participant also came from this range). This range biases our findings to primarily users in their early 20s.
RESULTS & FINDINGS
The results of this study revealed many issues plaguing the Colorado Unemployment site. Users had to spend a great deal of time navigating the site to find where to go, and when they did choose a direction, it was often the wrong one.
NAVIGATION & NAMING ISSUES
Of the issues plaguing the Colorado Unemployment site, navigation is the most disruptive. Users frequently struggled and became frustrated trying to find the pages and information they needed on the site.
Users struggled to find the happy path for 3 of the 4 tasks
Despite the availability of quick links users overwhelmingly favored the top nav bar, FAQs, and the virtual assistant
Inconsistent language through interactions
Undefined/uncommon abbreviations/jargon used in critical interactions
Users were frequently prompted to log-in to “MyUI+” throughout the process
POOR INFORMATION ARCHITECTURE & VISUAL HIERARCHY
Many of the navigation issues participants faced can be attributed to the information architecture of the site. The site consistently fails to present information in a clear and coherent manner that users would expect to see on a site like this.
Quick links and other navigation shortcuts are not immediately visible upon opening the page
Important interactions were overshadowed by less important sections
Dense, difficult-to-skim text overwhelmed and confused users
DESIGN PHASE
OUR RECOMMENDATIONS
1. Reduce the amount of page content
Improve overall information architecture/visual hierarchy to make skimming/reading easier
2. Top-level navigation needs to change
Explicitly name important sections, change UI naming convention to provide clarity, and create a dedicated section for all types of claim holds.
3. Implement a consistent design system
4. Fix website responsiveness issues
5. Cut down on virtual assistant prompts to log in to MyUI+
FINAL THOUGHTS
The process of creating this usability study and conducting it was a valuable learning experience for us both. While we generally feel that by the end of the study there were very few issues to be resolved, there are still improvements to be made. The number one issue we ran into during the tests were participants not knowing how to communicate that they felt they completed their tasks. Oftentimes, there would be an awkward pause while the participant sort of sat there doing nothing, and as evaluators we had to ask if they were finished. Going forward, we would include instructions in the script that let participants know what it means to be done with their tasks and that they need to communicate once they complete said task.
It is of the belief of the team that if the Colorado unemployment website was to undergo changes and follow the recommendations of this study, its users would likely feel a lot less stressed and on edge while using the site. Stress and frustrations will never be entirely eliminated of course, due to the fact that being unemployed as an adult is generally not a positive experience. However, these changes would make it so unemployed citizens could feel some relief after being able to quickly and successfully apply for benefits.