How We Increased Productivity 83%, Improved Remote Collaboration, and Prevented Burnout by…

Thousands of happy customers validated our software– but dogfooding Velocity validated our values.

A couple of years ago, our engineering leaders were anecdote-driven, like just about everyone else in the industry. We knew how much progress an engineer had made on a new feature because we asked. We knew why we missed a deadline for launching a new feature because we discussed it in our retros.

But we couldn’t answer simple questions like:

• Is our efficiency increasing or decreasing over time?
• What work slows us down and where is the biggest opportunity to improve?
• Which developers on my team are stuck, right now?

Even on a small development team, the insights that we had into our own productivity were limited.

Last year, we launched Velocity, an engineering analytics tool designed to help teams big and small with exactly the issue that we, ourselves, experienced. In the process of iterating and improving the platform, our Engineering Manager, Ale Paredes dogfooded our app. She used each feature from Velocity as soon as it was built and has experienced the benefits of data-driven engineering first-hand.

Armed with Velocity, she addressed burnout, improved remote collaboration, and helped restructure the engineering team. The result was an 83% increase in productivity, and a healthier, more engaged dev team.

Here’s what dogfooding Velocity has done for Code Climate.

We Ditched Flat and Near Doubled Our Efficiency

Flat structure doesn’t scale. There comes a point along every engineering team’s growth where the same set up that once built camaraderie and a shared purpose now fuels disengagement and a lack of focus.

Ale first started noticing that disengagement during daily standups. Some people were zoning out; others were on their phones. Almost no discussion was happening. It was 20 minutes of engineers taking turns speaking, one by one. When she brought this up in her 1:1s that week, she found that the team was in unanimous agreement: meetings were inefficient and taking up too much time.

Ale checked her hypothesis against the metrics by looking at Pushes per Day over the last 6 months.

A custom Velocity report Ale built to see how Pushes per Day has been affected as the team scaled.

Velocity gave her confirmation that the team’s efficiency had, in fact, been steadily decreasing over the past four months.

Ale took this insight back to the leadership team, and together, they decided it was time to ditch flat. Software development is inherently collaborative, and if engineers have too many co-workers to support, they can’t feel invested in one another’s work.

We decided to break up the ten-person team into groups of three or four engineers. We hypothesized that breaking up teams by Bezo’s 2-pizza rule would help individual engineers focus on less tracks of work. As a result, meetings would suck up less time and be more engaging, as smaller teams only hash out problems relevant to the few people involved.

When Ale implemented this change, the feedback was overwhelmingly positive. Engineers were doing more of what they liked (coding), and less of what they hated (meetings). Over the course of several months, the metrics corroborated what the team was feeling: we started moving much faster.

A custom Velocity report Ale built to see how Pushes per Day has increased since the re-org.

We went from an average of 12 pushes/day in November to an average of 21 pushes/day in January, an 83% increase.

We Coached Remote Team Members for Better Collaboration

Agile software development relies on strong communication, and, in many cases, frequent face time, so managing team members that are an ocean and three time zones away is nontrivial. It’s hard to perceive frustration through Slack. It’s hard to understand workload through Zoom.

Ale made a concerted effort to check in with remote team members and find out what’s going well and where they might be in need of coaching. But not everyone is an open book– many of her remote team members hesitated to bring up troublesome pull requests or bottlenecks during these meetings.

To find a starting point for these conversations, Ale started looking through a handful of contributor metrics on a weekly basis. She would pay particular attention to overall throughput (# of PRs merged), rework (how often developers work on their own recently merged code), and cycle time (how long it takes code to go from a developer’s laptop to being a merged PR).

By putting together a chart in Velocity, she started noticing a pattern. Remote developers had a significantly higher cycle time than HQ team members.

A custom Velocity report Ale built to see Median Cycle Time by team member.

Before a 1:1 with Legolas Greenleaf, Ale dug deeper into Velocity to see where, exactly, in the PR process contributors were getting stuck.

A Velocity chart that shows how contributors are performing on metrics compared to the industry standard.

Ale discovered that Legolas was spending a lot of his time working on code on his laptop before opening up a Pull Request and getting his work looked over.

The next time they sat down, Ale discussed the trend she noticed. The remote team member was able to speak more specifically about what has been slowing him down and what challenges he was facing. In their conversation, she guided Legolas towards learning how he could’ve better approached some of the last few PRs.

Together, they discovered the heart of the problem was that his PRs were too big. Splitting up projects into smaller pieces was a work habit that the HQ team had heard a ton, but the distance between Ale and the remote workers meant they weren’t aligned in terms of best practices. The data brought Ale’s attention to the problem, and gave her something objective to work off for her coaching sessions.

We Prevented Burnout in our Engineers

Meeting deadlines and building great products isn’t the only measure of a good team. We also want engineers to be engaged and not overworked so that our pace is sustainable. Too often, however, burnout is impossible to sniff out until it’s too late.

Last December, the team was on a tear. We were shipping features every other day, we had redesigned most of the pages in the application, and everyone was stoked about all the cool new stuff we were building. Everything seemed great.

During her weekly check-ins, however, Ale started noticing a trend.

Engineers were excited about their work, but everyone seemed to be feeling a lot of pressure to get features shipped– even though we were ahead of schedule. One senior engineer mentioned that he barely had time for code reviews. Other team members started neglecting pair programming in favor of spending more time getting stuff out the door.

Ale used Velocity to take a look at what the team had on their plate:

Velocity’s Activity Log showed leadership how much individuals were working.

She noticed that some developers had so much activity, it barely fit in their row. In a few cases, we were even pushing on weekends, which is highly unusual for our organization.

Looking through all the PRs and reviews the engineering team was churning out, Ale became suspicious that people were working overtime. She put together a report using Velocity’s Reports Builder to see how many pushes were made after 8pm each week:

Ale built a custom Velocity report to see who was working late hours.

During the last two weeks, the team had been working late hours more than they had in the previous three months combined. Ale brought this data to the leadership team and was able to successfully advocate scaling back work and giving the engineering team a well-deserved day off.

Now, Ale checks these dashboards regularly and sends out a monthly survey to check in on how work is distributed and how team members are feeling.

Measure and Mobilize

Metrics are not everything. Metrics didn’t help Ale notice that engineers were disengaged during meetings, or that remote team members weren’t voicing problems. Instead, metrics helped her check her intuition with more transparency into what’s going on. They also enabled Ale to verify that the changes she made generated an improvement in our productivity.

This gut-check was invaluable in helping us move faster and communicate better, both inside the engineering team and with the organization as a whole.

If you want to learn more on how Velocity can let you see deeper into your team’s work habits (and help you move at least 20% faster), sign up for a demo here.

Originally published at codeclimate.com.