User testing coaching day

I recently ran a user testing coaching day at work with my teammates. The goal was to teach colleagues how to set up and run usability tests. This generated user testing findings that we could implement in our work, whilst learning and practicing a new skill. I aimed to provide my teammates with the opportunity to practice user testing in a safe and unpressured, yet real-world, environment in order to give them the confidence to plan and run their own user tests.

My colleagues mainly come from a design background. So equipping them with the skills and resources needed to think and perform from a UX perspective, is important for their personal development within UX and meeting the wider aims of our organisation. From a personal perspective, I took the opportunity to improve my skills in coaching others and organising and managing a collaborative team event.

The user testing coaching day

To prepare for the day, I outlined the research aims and created a scenario, persona and prototype. I created these materials beforehand because I wanted the focus of the day to be on creating the research plan and running the tests, rather than wireframing and prototyping. I pre-arranged 8 user tests with a separate set of colleagues within the business, so we had real people to run our tests on.

Pre-prepared material presented on frames on Miro

To begin the day I presented the days material on Miro. I used this platform as everyone was already familiar with it and the platform encourages collaboration, enabling everyone to work together as a team.

I outlined the structure, aims and desired outputs of the day, split the team in two and commenced the user testing coaching day. Each team spent the morning finalising their prototype, defining the test scenario and tasks and running a pilot test. The teams also made sure their technology worked and decided how to record and analyse the tests.

In the afternoon each team conducted 4 user tests, analysed the results, prepared then presented their findings to the other team.

What did the team learn about running user tests?

At the end of the day, I asked the team to reflect on what they had learned. Outlined below are the top 5 learnings.

Reflections from the day presented on post-it notes on Miro

1. Why and how to gain and maintain consent
Not only is it a legal requirement, but it is of ethical research importance to inform participants about the purpose of the research, their rights and how their data will be used and stored.

2. The importance of running pilot tests
Pilot tests give you the chance to resolve issues and problems with, for example, the test script, prototype and recording or communication technologies, before going out into the real world.

3. Explain how complete the functionality of the prototype is
Unless you explain the functional ability of the prototype, participants tend to expect a fully working application which can create confusion when the prototype does not work the way they expect it to.

4. A single tester is required for good controls
Every researcher has their own individual style and variations of this across a single study can influence the results.

5. Have a clear understanding of how to analyse results
If multiple people are analysing the tests, there may be different definitions that result in a large variation between results. For example, when analysing task completion rates define among the team when a task starts and ends or what a successful or failed task looks like.

What did I learn about running a caoching day?

1. Preparation and planning is equally as important as giving the attendees the freedom and flexibility to do what they want.
2. Be emotionally intelligent — The facilitator is responsible for recognising the needs of others and making sure everyone is comfortable and understands their role.
3. Time should be made at the end of the day to allow those involved to reflect on what they have learned.
4. Producing guidance and resources based on the attendees learning’s encourages them to reuse their new-found knowledge and skills in the future.

Conclusion

My aim was to give my colleagues the opportunity to learn and practice their user testing skills in an unpressured, yet real-world environment in order to give them the confidence to plan and run their own tests. One colleague told me,

“I’ve only had a bit of user testing experience at uni, so doing it today was a real learning curve for me!”

This colleague quoted above went on to plan, run and analyse her own usability test. She used the findings to support a design decision with quantitative evidence when presenting it to the development team. This demonstrates that the aim of the day was met because she had the skills and confidence to conduct her own test.

The most important take away for me was how to be emotionally intelligent and recognise when individuals needed help and guidance, or alternative solutions, to ensure they felt comfortable throughout the day. Feedback from my colleague illustrates that I displayed this behaviour throughout the day, and it is something I will actively undertake in the future when running collaborative sessions.

“…You not only explained everything clearly, but gave support and guidance when you could. I found your approach to be very tentative and inclusive…”

Overall the user testing coaching day was a success and equipped the team with the skills and knowledge needed to run many more user tests in the future.