How We Prioritize Tests With Power-Ups At Trello

Published by Nate Selvidge on May 9, 2017

In my last post I talked about the ICE prioritization framework and how it can be used to figure out what tests to run. In this post I’ll go over how we use Trello and a custom Power-Up to keep track of our ICE scores and what we want to test next.

The Idea Board

Idea Board

You can organize your idea board any way you like, but an easy way to group your ideas is by the pirate metric that they correspond to, which makes it easy to see what metrics you will be targeting with the test. The general flow on the board is that new ideas come into the “Incoming” list, then the team sits down once per week to score the ideas in the incoming list using the ICE framework. Once an idea has been scored, it can be moved to the appropriate list for the metric that it is targeting.

Idea Cards

Idea Card

Each card should represent a single testable idea, and should include any supporting evidence that explains why the proposer originally came up with the idea. The test idea doesn’t have to be totally fleshed out to add it to the board since that might make people hesitant to put ideas on there that are worth discussing, but they should be detailed enough that the implementation is obvious.

The ICE Scoring Power-Up

In order to keep track of ICE scores and keep our lists sorted by the ICE score for each idea, we created a custom Trello Power-Up to manage everything. You can view the full source code of the Power-Up and give it a try here! We created this custom Power-Up by remixing the Custom Trello Power-Up Glitch project and customizing it to fit our needs.

The ICE Scoring Power-Up lets you score ideas with an individual score, or by having multiple team members vote and then averaging their scores. Once you’ve got scores on your ideas you can then click the “Sort all lists” button to sort your lists by the ICE score for each idea.

ICE Score Voting

At Trello when we give ideas ICE scores we play a planning poker-style game where we have a brief discussion about the idea before having every team member vote on their best estimation of the ICE score without knowing what anyone else’s score is. The Ice Scoring Power-Up keeps track of all the scores, and hides them from people who haven’t voted. The ICE Score on the card ends up being an average of all of the scores.

In the Power-Ups menu you can set the option for whether or not scores should be voted on, or set directly by setting the “Enable Voting on Scores” option. The “Hide Scores Until I Vote” option will prevent you from seeing the score on cards that you haven’t voted on.

Power-Up Menu

An Example of ICE Scoring

To see it in action, let’s try scoring the idea from above. To score an idea, go to the card details and click the “Add ICE Score” button.

Empty ICE Score

Add your scores in the appropriate boxes then click “Save” and you should see any other votes that have been added along with your own, and an average of all votes on the card.

Idea Card with a Vote

Sorting Your Lists

Now that you’ve got a score on your idea, add it to the appropriate category list.

Card in appropriate list

Click the “Sort All Lists” button and authenticate if necessary.

Need to Authenticate

Authentication Window

Once you’ve authenticated, then you should see your lists update in the correct order.

Updated List

Getting the Power-Up

You can get the Power-Up by following the instructions in the README. Once you’ve remixed the Power-Up you can customize it however you like to fit the needs of your team.