Identifying User Needs For Grading Data
for Edtech Product
Identifying User Needs For Grading Data
for Edtech Product
Identifying User Needs For Grading Data
for Edtech Product
Role: Senior UX Researcher
Company: TPT (Teachers Pay Teachers)
Stakeholders: Senior Leadership, UX Design, Product Management, Software Engineering, Product Marketing
Core Responsibilities: Survey Development & Synthesis, In-Depth Interview Moderation, Usability Test Moderation, Interview Synthesis & Recommendations
Methodology: Survey, Discovery Interviews, Usability Testing
Tools: Hotjar, Zoom, Figma, Miro, Dovetail, Google Slides
Project Overview
Easel by TPT is TPT’s flagship digital learning tool. In 2022, the product team was in the process of developing self-checking features – which were highly requested by both seller and teacher audiences – that would help teachers better evaluate their students’ understanding of different topics and skills.
Problem
One newer feature of Easel by TPT was that it could automatically grade assignments. The team needed to figure out a design solution that could clearly illustrate those grades as easy to read data reports. After completing a round of internal stakeholder research, it became clear that the team 1) needed to better understand teachers’ current workflows with data from digital learning tools before designing concepts for sharing self-checking results with teachers. Once the team had this foundational understanding, they then 2) needed to design and evaluate potential solutions to ensure they meet user needs.
Research Objectives
Understand when and why teachers use data from digital learning tools
Explore teachers’ end-to-end journey with data and reporting from digital learning tools
Discover table stakes for data reporting for Easel by TPT
Evaluate designs for data reporting in Easel by TPT
Methodology
Intercept Survey
In-Depth Interviews
Concept Tests
Intercept Survey
In order to understand teachers’ current workflows with data from digital learning tools, I recommended that the team first identify which tools teachers actually relied on for data and reporting. As a result, I developed and fielded an intercept survey using Hotjar to ask teachers which tools they used.
Role: Senior UX Researcher
Company: TPT (Teachers Pay Teachers)
Stakeholders: Senior Leadership, UX Design, Product Management, Software Engineering, Product Marketing
Core Responsibilities: Survey Development & Synthesis, In-Depth Interview Moderation, Usability Test Moderation, Interview Synthesis & Recommendations
Methodology: Survey, Discovery Interviews, Usability Testing
Tools: Hotjar, Zoom, Figma, Miro, Dovetail, Google Slides
Project Overview
Easel by TPT is TPT’s flagship digital learning tool. In 2022, the product team was in the process of developing self-checking features – which were highly requested by both seller and teacher audiences – that would help teachers better evaluate their students’ understanding of different topics and skills.
Problem
One newer feature of Easel by TPT was that it could automatically grade assignments. The team needed to figure out a design solution that could clearly illustrate those grades as easy to read data reports. After completing a round of internal stakeholder research, it became clear that the team 1) needed to better understand teachers’ current workflows with data from digital learning tools before designing concepts for sharing self-checking results with teachers. Once the team had this foundational understanding, they then 2) needed to design and evaluate potential solutions to ensure they meet user needs.
Research Objectives
Understand when and why teachers use data from digital learning tools
Explore teachers’ end-to-end journey with data and reporting from digital learning tools
Discover table stakes for data reporting for Easel by TPT
Evaluate designs for data reporting in Easel by TPT
Methodology
Intercept Survey
In-Depth Interviews
Concept Tests
Intercept Survey
In order to understand teachers’ current workflows with data from digital learning tools, I recommended that the team first identify which tools teachers actually relied on for data and reporting. As a result, I developed and fielded an intercept survey using Hotjar to ask teachers which tools they used.
Role: Senior UX Researcher
Company: TPT (Teachers Pay Teachers)
Stakeholders: Senior Leadership, UX Design, Product Management, Software Engineering, Product Marketing
Core Responsibilities: Survey Development & Synthesis, In-Depth Interview Moderation, Usability Test Moderation, Interview Synthesis & Recommendations
Methodology: Survey, Discovery Interviews, Usability Testing
Tools: Hotjar, Zoom, Figma, Miro, Dovetail, Google Slides
Project Overview
Easel by TPT is TPT’s flagship digital learning tool. In 2022, the product team was in the process of developing self-checking features – which were highly requested by both seller and teacher audiences – that would help teachers better evaluate their students’ understanding of different topics and skills.
Problem
One newer feature of Easel by TPT was that it could automatically grade assignments. The team needed to figure out a design solution that could clearly illustrate those grades as easy to read data reports. After completing a round of internal stakeholder research, it became clear that the team 1) needed to better understand teachers’ current workflows with data from digital learning tools before designing concepts for sharing self-checking results with teachers. Once the team had this foundational understanding, they then 2) needed to design and evaluate potential solutions to ensure they meet user needs.
Research Objectives
Understand when and why teachers use data from digital learning tools
Explore teachers’ end-to-end journey with data and reporting from digital learning tools
Discover table stakes for data reporting for Easel by TPT
Evaluate designs for data reporting in Easel by TPT
Methodology
Intercept Survey
In-Depth Interviews
Concept Tests
Intercept Survey
In order to understand teachers’ current workflows with data from digital learning tools, I recommended that the team first identify which tools teachers actually relied on for data and reporting. As a result, I developed and fielded an intercept survey using Hotjar to ask teachers which tools they used.
Key takeaways from insights report
Key takeaways from insights report
Snapshot from intercept survey results report
Snapshot from intercept survey results report
Snapshot from intercept survey results report
A framework for teachers' mental model for data
A framework for teachers' mental model for data
Presentation of concepts tested
Presentation of concepts tested
With this survey, I was able to determine the top four digital learning tools that over 300 teachers reported they used data from.
In-Depth Interviews
With that understanding of the tools teachers were using, I recruited participants – that had indicated they used those tools – for in-depth interviews. I then developed a discussion guide that focused on those teachers’ use of data from those tools, particularly when and why they used the data, how they used it, and what they liked/disliked about those experiences. Ultimately with these interviews, I wanted to be able to help the team determine table stakes requirements for data and reporting in Easel by TPT.
With this survey, I was able to determine the top four digital learning tools that over 300 teachers reported they used data from.
In-Depth Interviews
With that understanding of the tools teachers were using, I recruited participants – that had indicated they used those tools – for in-depth interviews. I then developed a discussion guide that focused on those teachers’ use of data from those tools, particularly when and why they used the data, how they used it, and what they liked/disliked about those experiences. Ultimately with these interviews, I wanted to be able to help the team determine table stakes requirements for data and reporting in Easel by TPT.
With this survey, I was able to determine the top four digital learning tools that over 300 teachers reported they used data from.
In-Depth Interviews
With that understanding of the tools teachers were using, I recruited participants – that had indicated they used those tools – for in-depth interviews. I then developed a discussion guide that focused on those teachers’ use of data from those tools, particularly when and why they used the data, how they used it, and what they liked/disliked about those experiences. Ultimately with these interviews, I wanted to be able to help the team determine table stakes requirements for data and reporting in Easel by TPT.
Key takeaways from insights report
Key takeaways from insights report
A framework for teachers’ mental model for data
A framework for teachers’ mental model for data
Example of key takeaways
Example of key takeaways
Example of more detailed findings
Example of more detailed findings
Stakeholders debriefed interviews
Stakeholders debriefed interviews
I conducted hour-long in-depth interviews with eight teachers and synthesized learnings into a final report that was presented to product, design, and engineering stakeholders. The report included key recommendations for designing the data and reporting experience in Easel by TPT.
Concept Tests
Based on learnings from the exploratory in-depth interviews, the team came up with multiple concepts for displaying and reporting data to teachers. I worked with the designer and product manager to prioritize which concepts to test based on users’ needs and current workflows with data. I then created a research plan for concept testing, developed a discussion guide, and supported recruitment of research participants.
I moderated concept tests with six teachers to 1) learn about their impressions of data in Easel by TPT from various mocks and 2) identify what resonated, excited, and/or caused questions or concerns.
After completing the tests, I synthesized insights into a final report that I presented to product, design, product marketing, and engineering along with recommendations for the final product designs.
I conducted hour-long in-depth interviews with eight teachers and synthesized learnings into a final report that was presented to product, design, and engineering stakeholders. The report included key recommendations for designing the data and reporting experience in Easel by TPT.
Concept Tests
Based on learnings from the exploratory in-depth interviews, the team came up with multiple concepts for displaying and reporting data to teachers. I worked with the designer and product manager to prioritize which concepts to test based on users’ needs and current workflows with data. I then created a research plan for concept testing, developed a discussion guide, and supported recruitment of research participants.
I moderated concept tests with six teachers to 1) learn about their impressions of data in Easel by TPT from various mocks and 2) identify what resonated, excited, and/or caused questions or concerns.
After completing the tests, I synthesized insights into a final report that I presented to product, design, product marketing, and engineering along with recommendations for the final product designs.
I conducted hour-long in-depth interviews with eight teachers and synthesized learnings into a final report that was presented to product, design, and engineering stakeholders. The report included key recommendations for designing the data and reporting experience in Easel by TPT.
Concept Tests
Based on learnings from the exploratory in-depth interviews, the team came up with multiple concepts for displaying and reporting data to teachers. I worked with the designer and product manager to prioritize which concepts to test based on users’ needs and current workflows with data. I then created a research plan for concept testing, developed a discussion guide, and supported recruitment of research participants.
I moderated concept tests with six teachers to 1) learn about their impressions of data in Easel by TPT from various mocks and 2) identify what resonated, excited, and/or caused questions or concerns.
After completing the tests, I synthesized insights into a final report that I presented to product, design, product marketing, and engineering along with recommendations for the final product designs.
Presentation of concepts tested
Presentation of concepts tested
Example of key takeaways
Example of key takeaways
Example of more detailed findings
Example of more detailed findings
Shared updates via Slack
Shared updates via Slack
Led workshops with stakeholders
Led workshops with stakeholders
Additional Deliverables
Throughout the study, I shared one-off findings and user quotes to keep stakeholders engaged in the research study in addition to insights reports and workshops.
Additional Deliverables
Throughout the study, I shared one-off findings and user quotes to keep stakeholders engaged in the research study in addition to insights reports and workshops.
Additional Deliverables
Throughout the study, I shared one-off findings and user quotes to keep stakeholders engaged in the research study in addition to insights reports and workshops.
Stakeholders debriefed interviews in Miro
Stakeholders debriefed interviews in Miro
Shared updates via Slack
Shared updates via Slack
Led workshops with stakeholders
Led workshops with stakeholders
Conclusion
Based on this research, the team iterated on designs and launched data and reporting views for self-checking assignments within Easel by TPT, which is now used by over 3M teachers and used by 7.5M students.
Conclusion
Based on this research, the team iterated on designs and launched data and reporting views for self-checking assignments within Easel by TPT, which is now used by over 3M teachers and used by 7.5M students.
Conclusion
Based on this research, the team iterated on designs and launched data and reporting views for self-checking assignments within Easel by TPT, which is now used by over 3M teachers and used by 7.5M students.
