top of page
USABILITY TESTING
The Product
8 Billion Minds 8 Billion Minds is a platform for live learning that facilitates knowledge exchange worldwide, completely free of charge. Users can share, teach, or learn about a wide variety of subjects in a casual and informal setting.
The Task
A beta site was created and was in need of testing with real users, this is where I come in. No testing was conducted prior and was needed to validate current design decisions.
The Goal
I wanted to assess the learnability for new users interacting with the 8 Billion Minds beta site for the first time on a desktop device. I wanted to observe and measure if users understood the site's basic functions and navigation to access lessons and teachers. Finally, I wanted to identify any pain points and provide recommendations to address the issues.
Timeline:
11 Weeks on part time schedule
Team:
Sr UX Designer, PM, Founder & UX Designer (myself)
Key Goal:
Capture user feedback on current user experience of beta site
The Process
Before beginning the testing, it was important to create a testing plan to stay organized and focused. The following usability testing process was created:
Create a Test Plan
Facilitate the Test
Analyze & Synthesize
Create Report
-
Conduct test
-
Observe & record user behaviour
-
Noteate pain points
-
Identify goals & user tasks
-
Create a test script
-
Recruit users
-
Review video & assess user behaviour
-
Affinity map
-
Feature prioritization matrix
-
Provide user centered recommendations
-
Wireframes
Creating the Test Plan
Once the scope of work was defined and with the goal in mind, I defined my test objectives. Next, I wrote the test script and began recruiting participants.
Identify Testing Goals & Tasks
I set three main goals to accomplish in this usability test:
-
To observe how easy users could complete the onboarding and sign up
-
To determine if participants could quickly and easily understand the purpose of the site and the value it provides
-
To observe the usability of important key features like, search for a subject/teacher, request a session, send a message and find resources
These are the final tasks that users will need to complete during the testing session:
-
Complete the onboarding steps 1-3.
-
Create an account.
-
Search for a subject that you are interested in learning more about.
-
Search for a teacher.
-
Send a message to a teacher.
-
Request a session.
-
Find your resources.
I recruited 5 participants
Age of participants: 20 - 60 years of age
Skills: Everyone had experience online learning as a student or as a teacher
Method: Remote moderated testing
Sandra
Jeremiah
Becky
-
Age range 40 - 50
-
8th grade science teacher
-
Likes learning from online platforms
-
Enjoys teaching
-
Age range 40 - 50
-
High school art teacher
-
Likes learning and sharing knowledge
-
Open to online learning
-
Age range 30 - 40
-
Nursing Student
-
Prefers learning in-person but would like online mentors
-
Open to learning not teaching
Stephanie
Eric
-
Age range 20 - 30
-
Vocational nurse
-
Current student
-
Open to online learning not teaching
-
Age range 50 - 60
-
Software engineer
-
Prefers learning in-person but would like online advice
-
Open to learning
Facilitating the Test
To get started, I explained the session agenda and made sure to put each participant at ease by initiating a conversation with general questions.
Once they were comfortable, we began the usability test. I directed them to the beta site and explained the tasks one by one, encouraged them to think out loud, and had them sign a video recording consent form.
I sat back and observed each user's reactions, identified pain points, and noted valuable feedback as they completed each task. I redirected the user to the task at hand when needed.
Remote testing was held from June 10 - 14, 2022, using Zoom. Participants were given details about the session, including the video link and consent form, through email prior to the testing date. The time allocated for each session was 20 - 35 minutes, which included a short briefing before the session, tasks, and a debriefing.
The Results
In total, we observed 35 attempts to perform tasks. Of those attempts, 10 were successful, 20 were partially successful, and 5 were failures. For this particular site, we assigned half a point for partial success.
Therefore, the success rate was calculated as (10 + (20 * 0.5)) / 35 = 57%.
57% of Success Rate.
Tasks
-
Complete onboarding steps
No. Participants Completed Successfully
0
No. Participants Completed with Difficulty
5
No. Participants Failed Completion
0
2. Create an account
1
4
0
3. Search a subject
2
3
0
4. Search a teacher
5. Send teacher a
message
2
4
3
1
0
0
6. Request a session
1
4
0
7. Find resources
0
0
5
Pain point areas
All users expressed frustration or confusion with the following:
-
Search for a subject: Some users questioned if their subject was available and were confused with the suggested list. They thought they were limited to the provided suggested list.
-
Completing onboarding steps 1-3: One user wasn't sure if the example greeting was an example or something someone had typed. All completed steps 1-3, but all expressed some sort of uncertainty about the form being used.
-
Creating an account: All users did not like having to figure out their time zone.
-
Searching for a teacher: After searching and coming to the matches screen, some users questioned if the available cards were teachers. It was not as obvious. They would all like to see more details on bios and experience.
-
Requesting a session: Two users skipped the duration feature, and all had to think about their time zone.
-
Sending a message to teacher: Most users were confused by the button stating "message send, close here" after they submitted their messages.
-
Finding resources: All users spent time clicking on the "about us" icon or info button, hoping to find help or resources. None found it.
-
Adding their time zones: All users expressed frustration with the current experience of figuring out their time zone.
Analyze & Synthesize
After reviewing my findings I carefully laid out the identified pain points in an affinity map to help me analyze and synthesize the data into categories and patterns. Below are my results.
Feature Prioritization 2 x 2 Matrix
To help us prioritize the issues, the founder, UX designer, and I used a feature prioritization 2x2 matrix. This helped us prioritize the categories of issues based on their importance to the business (x-axis) and the importance to the user (y-axis). We organized each issue in its corresponding area and labeled them in order of importance.
Recommendations
After identifying the top 3 issues in the feature prioritization 2x2 matrix, I developed user-centered recommendations to address each of them. These recommendations were based on insights gathered from the usability testing, as well as industry best practices and design principles. The recommendations were presented to the founder, UX designer, and development team in a clear and actionable format, including design mockups and prototypes, to help guide the implementation process.
1. Sign up - time zone feature
Pain point: 4 out of 5 users had trouble figuring out their time zone and questioned parts of the sign-up form. I noted other comments the users made and kept those in mind when creating an easier, faster, and more secure sign-up process.
Beta sign up form
Beta time zone list
Recommendation: Remove the time zone list and add a "Find my Location" feature. Use a more modern and familiar form to improve the user experience. Also, update the UI to complement the current brand.
2. Onboarding Steps 1, 2 and 3
Pain point: Although all users were able to complete all three steps of the onboarding, they expressed uncertainty about the form being used.
Our first solution was to change the order of steps, as starting with the introduction screen was found to be intimidating for some users.
Old Order:
-
Introduce yourself to the community
-
What subject do you want to learn?
-
What subjects can you teach?
The old order of the onboarding steps may have been intimidating for some users, as they were immediately asked to introduce themselves to the community before even being sure if they wanted to use the platform. As a solution, we decided to change the order of the steps to make it more user-friendly.
New order:
-
What subject do you want to learn?
-
What subjects can you teach?
-
Introduce yourself to the community
The new order proposed addresses the issue of users feeling intimidated by the introduction screen in the old order.
Since the goal of the site was to provide a live learning platform that fosters knowledge-trading around the world, and users felt uncomfortable introducing themselves, the new order mentioned above was implemented.
Original onboarding steps 1, 2 & 3
Our recommendations include updating the form to a more common format, redesigning the UI to align with the brand, removing the footer from onboarding screens, and updating the action buttons.
3. Search/Landing Page
Pain point: All users found the search box on the landing page, but some questioned if their subject was available and were confused with the suggested list that would appear as they input their subject. They thought they were limited to the list.
Insight: Allow the user to be the first user to create a subject category if their subject isn't found
Insight: Give the user options to get engaged and get excited to sign up with a popular subjects area
Original Search/Landing Page
Recommendation for Search/Landing Page
Action Buttons
Improved accessibility by adding contrasting colors for the action buttons.
Typography
Improved typography by reducing the number of lines in the title and subtitle, changing the font, and making better use of negative space for improved readability.
Search/Explore
Some users were not sure what to search for, so to help engage them I added an "Explore" button, which allows them to browse and discover new possibilities.
Next Steps
The recommendations and wireframes were created for all seven tasks in the usability tests. Once the prototype is complete with the iterations, the following steps will be taken to assess new changes.
Round 2 of Usability Testing
Another round of usability testing would prove how successful our updated iterations will be with our users.
ortizcreativeworks
bottom of page