A Vision for Unlocking the Value of Historical Analytics

Analytics tools are limited by the fact that they don’t include (1) a visual history of the apps we’re developing and (2) a record of the team intentions behind launches.

Analytics tools are useful for understanding how the live version of an app is performing. If you wonder how many people are clicking a button, you can see the number of clicks and easily interpret the results.

But analytics tools are not good at painting a detailed picture of how previous versions of the app performed. If you look back at the behavior of your users a year ago according to your analytics, it can be difficult to interpret the recorded actions that are no longer possible in the app. A year ago, clicks on “right nav menu” may have been a common action. This doesn’t carry much meaning if your site hasn’t had a right nav during your tenure at the company.

The inability to visualize previous versions of your app limits the value of old usage data, and this is a missed opportunity.

Let’s say your company sets a goal to increase video streams per user. Your website has had videos for a couple years, but the company has never intentionally tried to increase streams. In your analytics, you see that the number of video streams elevated on your homepage 14 months ago and then dropped down two months later, before you joined. The company has redesigned the homepage twice since then. The previous product manager added some annotations about the redesigns, but you can’t see what changed or why.

Wouldn’t it be great if you could see, directly in your analytics tool, the version of the site that existed when the video streams spiked? You could incorporate aspects of the old design into your new design for testing. Because you can’t see it, it could take several iterations to discover a version as successful as the previous one, if you ever find it.

Analytics tools also do little to prevent you from retrying failed ideas from the past.

Let’s say you’re trying to increase the conversion rate on your checkout page. Working for a company that’s been around for a while, you’re clearly not the first person with this goal. Wouldn’t it be great if you could browse all the previous attempts to see what was tried and why? A tool like Optimizely would provide some history of experiments, but it’s not designed to expose learning in a form that can be plumbed for future inspiration.

I’m writing this post to convey a future iteration of my project, Double-Loop. The current version of Double-Loop provides part of the puzzle: it enables teams to efficiently record their history of launches and learning. Combining analytics with Double-Loop will open new possibilities of mining value from past analytics.

Here’s an early mockup, developed with my designer friend Dennis Crothers, of how this might look:

While analytics integration isn’t live yet, get a head start building your Double-Loop launch timeline! I believe you’ll find value in co-authoring your company history with teammates. Future team members will love you for it.