College of Computing Website Overhaul Aug 2019 - Dec 2019
Introduction This project was a semester long project which consisted on preparing a set of design recommendations for the future College of Computing website based on the needs of the student population. We targeted both current and prospective students during our research phase. While the project was semester-long, involved a team of 5 people, and yielded a myriad of documentation (there are 4 reports available if interested), I will make an effort to condense the information into the most relevant to my involvement in the project. My key roles were on Data Analysis and User-based Evaluation.
Research Methods & Data Analysis We used a variety research methods to address different aspects of the redesign. We looked at Survey responses, Website Analytics, Heuristic Evaluation, Card Sorting, and Stakeholder Interviews. We began our work with the Survey which revealed interest rates, opinions, common issues, and, topic of navigation per device.
Below is a graph depicting the percentage of students interested in topics found on the website. The data was been normalized to account for differences in the sample size of each of our populations (i.e. Undergraduate, Master, PhD).
We also conducted a semi-closed Card Sorting activity with seven participants across all three student groups. We pulled all the tabs in the CoC homepage and created the cards with it. The main titles for the drop-downs were printed larger and the participants were told the smaller cards should fit into this larger categories. We call it a semi-closed card sort because we allowed participants to trash cards they found useless for the CoC website and to make up their own categories if this made more sense to them.
We compiled all the participants' responses and created the above table. We use a heat indicator (hue of blue) to quickly visualize on which categories participants agreed most. The yellow outline on some categories represents what the current tabs under each of the drop-downs are. In summary, if the color inside the yellow square is dark blue those categories have been accurately placed on the website.
On this activity we caught a glimpse of our population's mental model, most of it was matched with what the website had already in place, but there were some categories that were completely out of place. The activity also revealed many nomenclature issues with labels that didn't accurately portray the content of the site. This data was used to document our decision of building a site map centered around the specific user needs rather than broad topics.
Design Evaluation To evaluate our design we decided to construct two full paths aimed to target prospective and current students needs. The two paths we decided to construct were chosen from the research findings in the prior phases, mainly from the survey results and the qualitative feedback from the card sort activity.
A|B Testing We used a platform called Usertesting.com. It allowed us to set up a scenario and tasks for a variety of participants to follow. The platform allows you to set up different screening conditions to filter who can do your test according to some variables (i.e. salary, language, status, etc). For our testing we specified we wanted participants between 18 and 30 in order to find people who were students now, or had been not too long ago. We did this so all our participants were familiar with how a college website operates, if wouldn't help if our participants attended college when there was no internet.
We had 12 participants in total spanning across four groups of three participants each. We tested the navigation on both Desktop and Mobile version of the site, and with the legacy site and the new design. We set up 4 tasks for the participants to try accomplish. These came from the earlier results from the Survey.
We picked topics were students opinions were distributed in different forms. One where the opinions was negatively skewed, one positively skewed, and one split down the middle. Doing this activity allowed us to revalidate our earlier prior results on the research phase. There was an additional task we included related to international students. During a cognitive walkthrough session with an international student, she pointed out that there was no place for an international student to find information. We had seen none of this on the survey or earlier methods so we decided to include a task to look for international student relevant information in the evaluation.
The tasks we set up were geared at highlighting the current and largest problems with the site to see if our designs could make an improvement. To assess if this was true, we looked at time-on-task metrics and at Likert Scale information.
The time on task confirmed what we saw in the beginning of the project, tuition information was very hard to find. However this data should be taken with a grain of salt, the legacy site contains many more pages and content than the designed pathways. So it makes sense that the time on task for the legacy is longer because the participants can get lost through more pages. That's why we also looked at Likert scale information.
It was hard to define a clear advantage on using the designed site compared to the legacy. In some aspects it seemed better while in others it was worse. This graph clearly shows that there is still improvement on the positioning of information on the site.
Heuristic Evaluation Qualitative Data Our experts gave us valuable insights on what aspects of the site we could use, the feedback was categorized into Navigation, Content, and UI Elements.
Team Shihui Ruan Prabodh Sakhardande Jordan Hill Harshali Wadge Santiago Arconada Alvarez
All team members participated on the project as part of the Research Methods class from the MS HCI program at Georgia Tech.
