Milestone 7 > User Testing
Due Date: 12/05/2007 | Owner: All
User Testing Methodology
Using our existing prototype, we tested 9 users on 8 tasks using think aloud as the primary form of getting feedback. We recorded their mouse movements and voice to allow us to review their completion of the tasks and understand better what parts of the interface were easy to understand and what caused problems.
The script used for the tasks
Imagine that you are interested in obtaining a position as product manager. You are using CourseNav to find some courses suited to your career objective. You are now on the homepage of CourseNav.
- Find a page giving you information about product management.
- Look through the list of recommended courses for a product manager and find the course that students like the best according to CourseNav.
- You are curious to learn more about this course. Particularly, what do other students like about it?
- If you took this class last semester and really enjoyed it, how would you use this site to tell other students about it?
- You are interested in courses related to BIT 512. How would you find out more about them?
- Describe the visualization - what information does it give you?
- Check if there are prerequisites for BIT 512 and learn more about them.
- Save the prerequisite course so that you can find it again later.
Quotes
The way MBAs select courses now is "the most awful thing in the entire world."
After testing, a student said, "This rocks, I'd really like to see this."
Finding #1: The "Career" Button
Some users were a little confused about the career button on the homepage of CourseNav. They were unsure if this was a directory for jobs(listing open positions) or a career guide(information about a career).
Recommendation
Add a tooltip to each major button when you hover, e.g.:
"Careers" - explore courses for your career path
"Courses" - prereqs, descriptions, students' comments and more!
Finding #2: Eye to the Left
Users tended to focus on the left navigation panel of Impact to begin their search for information. Once they knew they were on the CourseNav homepage, they were able to focus on the content at the center of the screen.
Recommendation
Ensure that CourseNav is present in the left navigation panel of Impact
Finding #3: I want to know when!
Several users said they wanted to know what times and days of the week a course was offered.
Recommendations
Display days and times for courses.
Finding #4: Planets!!!
"The visualization is neat!! Like a planet map!!" Several users gave positive comments about the visualization and thought this would add fun elements to CourseNav.
- Some users were not sure what the plus/minus buttons meant on the left-hand side toolbar. The "plus" button may carry the dual meanings of viewing more objects or getting closer to an object.
- Users were uncertain about the meanings of the size of a circle and the distance between circles.
Recommendations
- Use "single tree" and "three trees" metaphor to substitute the plus/minus button. This gives users a quick sense of what the buttons mean.
- Add a link "what these images mean" at the bottom of the visualization box and clearly define the visualization elements:
Size: a formula composed of the course evaluation ratings and the average bidding points
Distance: shows the concurrence of the two courses. The shorter the distance, the more the number of students who have taken both of the classes
Rolling mouse over a circle shows the ratings and average bidding points.
Finding #5: I want to rate it!
Some of the students wanted to rate the courses from CourseNav, but the ratings are actually static Course Evaluation scores that are from end of semester evaluation forms.
Recommendation
Change the term "Student Rating" to a more specific phrase "Course Evaluation Rating"
