Summer of Metrics: Experiment

To wrap up the Summer of Metrics, lets get everything you've learned all packed up and shipped. Where did I leave the tape?

Last time I shared how optimizing your store by improving one metric can butterfly-effect and change a bunch of other metrics. That then requires more change which causes more changes to spread. Keeping everything going in the right direction can feel like juggling sharks.

This can lead to people deciding to just not bother and keep plugging along and hoping their metrics turn out okay.

That's a huge missed opportunity, which is sad.

An alternative is to do what scientists do. Try to change one variable (metric) at a time while holding all other variables (metrics) constant.

That means if you decide to optimize your Average Order Value to increase it, you'd work to keep your Repeat Purchase Rate and Average Customer Purchase Latency constant. You won't be able to keep them perfectly constant but you'd avoid changing other things that would impact them (e.g. welcome campaign, reminder emails). Instead leave those alone until you've finished with your Average Order Value experiment.

Once you're done or decide to pause your AOV optimization, then decide if you need to optimize the metrics that AOV impacted.

Your goal is to try to minimize how much the other metrics change while you're optimizing Average Order Value and then go fix-up any impacted metrics.

You'll end up jumping from metric to metric over-time as changes to one change others. As long as they are all improving overall, you're heading in the right direction. If you know this process going in, you won't feel like your constantly chasing problems. You know the Repeat Purchase Rate got impacted and you have a plan to deal with it once the Average Order Value is where you want it.

(One trap to watch out for is going back and forth between two metrics. e.g. AOV weakens RPR, so you repair RPR only to harm AOV, which you then repair and hurt RPR... forever stuck in a loop)

Since you'll want a good amount of time dedicated to each metric you can even schedule them around your annual calendar. Perhaps you focus on AOV during the holiday season, RPR in spring, Average LTV in summer, and Average Latency in fall. If you do this, make sure to compare this year against last year, otherwise seasonality will throw your numbers off. The Insights System in Repeat Customer Insights can automatically compare year-over-year for you.

Above all you'll want to follow the main metrics that matter: revenue and profit. All these other metrics should be improving those over the long-term.

With this, the Summer of Metrics has concluded. As a lot was covered in a short two week, you can always come back to these in my archive. Time to gear up for autumn. Plenty of time for some experiments before Black Friday.

Eric Davis