Test Groups have arrived at New/Mode! This allows you to pit multiple variants of the same tool against each other to determine which version is the most effective. This happens in 3 steps.
Step 1: Determine which tool you'd like to set up a Test Group with. Go through the Test Group Setup article to get the test group up and running.
Step 2: Our awesome algorithm will randomly display the different variants of your tool and figure out which variant is more successful in converting action viewers to action do-ers!
Step 3: As more submissions come in, the most successful version of the tool will be crowned winner, and be displayed more often than the others. The winner may change throughout the campaign, and the algorithm will always show the Active Winner more often than the other variants.
Here's some best practices when setting up your Test Groups, to ensure you're keeping your tests as effective as possible:
- Start off with a hypothesis, and base the variants you create from those. If you've been engaging your supporters, you may already have some idea of what their preferences are. Go off of those intuitions and come up with a hypothesis to test, then the ideas for what variants to create to test that hypothesis will come easily!
- Only test one thing at a time! To figure out what details of a form affect your supporters, everything between different variants needs to stay the same, except the detail you're testing. So if you're wondering which color palette has higher conversions, the best is to change the palette and keep the wording, format, and everything else the same!
- Don't be afraid to challenge your assumptions about your supporters! When you've been working with the same supporters for a long time, it's easy to make assumptions about what works and what doesn't. Test Groups provide a way to challenge those assumptions, so go ahead and add variants that are different from what your supporters are used to-- they may surprise you!
- Avoid making changes once your Test Group goes live. Changes made to tools once the Test Group has gone live may affect which variant is the Active Winner, so making changes to those tools once the group has gone live is like starting a new Test Group halfway through a test.
- Re-test once your campaign is done. If your campaign has wrapped up and you've got an Active Winner from your variants, it's always worth trying to recreate the differences between the variants you tested to see if your conclusions from one Test Group carry into different campaigns.
- Tag your variants consistently. This is for your own records, since tags are not supporter-facing in any way, but tags are a useful way to keep track of which variant is which without having to change the title of the tool.
That's it! Test Groups are a simple way to help you run crazy effective campaigns and develop the best version of your tools.