As Tom mentioned, we hosted a “bug bash” on Wednesday night and called it PizzaBox, naturally. We invited 20 or so friends to help us test Filtrbox and find bugs, usability issues, etc. Not only was it fun and productive from a QA perspective, it was really educational for us to see so many people using the product at once. The event was a resounding success, and we will definitely do one again.

The recipe we used was simple:

• Invited friends, friends of friends, and colleagues (this provided a good mix of “user types”)
• Relaxed, casual atmosphere (too much structure and its no fun and you are controlling the results)
• Loosely guided testing (we handed out sheets with key areas to test and how to submit feedback)
• Testers brought their own laptops (diverse environments and screen sizes)
• Beer and Pizza (lots of both…)

Why this worked well was simple - everyone who showed up WANTED to be there, and they were happy to play with something new and be critical of it. By having everyone on their own laptops we got to gather test data from 20+ different system configurations, not a standard build like you get in a lab.

We gathered all of the feedback by

1. entering tickets into our bug system in realtime
2. having people email notes and screenshots
3. collecting data “over the shoulder” and seeing their experiences first hand

All in all, we collected as much or more usability feedback as actual product-defect type bugs. Seeing how people “think” and how they interpret the Filtrbox dashboard was really helpful for us. We have a number of areas we want to improve to enhance the overall intuitiveness and workflow within the dashboard.

Thanks to everyone who came out! Filtrbox loves you

This entry was posted on Friday, March 28th, 2008 at 5:57 pm and is filed under random. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.