logo

Measuring True Engineering Velocity With Time To First Review
May 28, 2024

I think we can all agree that peer review is a fantastic addition to any process when it comes to individuals collaborating on a code base.

With that being said; one of the most critical metrics that often feels overlooked in the hustle of daily tasks is the time in which it takes for a the first review to arrive. I like to think of this metric as being Time To First Review.

This isn't just a number on a performance review; it's a fundamental aspect the development process that can dramatically influence project outcomes as well as an indicator into team health. From my experience, a shorter Time To First Review not only speeds up the development cycle but also significantly boosts the team's morale by fostering a culture of immediate feedback and continuous improvement.

It helps engineers feel like there’s value in the work that they’re producing and helps to keep us motivated while reducing the frustration that often comes from waiting for feedback.

This level of responsiveness often reflects a healthy team dynamic where members are committed to supporting each other and enforces their commitment to pushing the project forward collectively.

Challenging Norms

Let’s be clear- Time To First Review isn’t related to how long it takes for the code review to meet the approval conditions, it’s more important than that- in fact:

I would even be so bold to say it’s more important for the individual responses to come quickly than it is for the whole process to happen rapidly.

A high Time To First Review time indicates that there is some kind of dysfunction- hidden or otherwise. It may signal that the team is understaffed or overworked, perhaps there’s a uneven distribution of work amongst the team. Such situations can lead to burnout and reduce trust amongst the team, affecting overall work health.

In my personal experience I find it’s the most accurate reflection of our responsiveness and agility. When a engineer submits a pull request and it languishes without review, it's not just the project that stalls but also the developer's enthusiasm and momentum. It's crucial to maintain a quick feedback loop. This quick loop means engineers can iterate on their code rapidly, incorporating feedback almost in real-time, which invariably enhances the quality of the software we're developing.

As with anything relating to software engineering at scale- automation plays a huge role here. Tools that automatically check for coding standards, syntax errors, and even run preliminary tests. These tools can perform the initial "review" within minutes of submission. While they don't replace a human reviewer, they ensure that any submission is at least compliant with our basic quality standards before we introduce the cost of real life human being to spend their time trying to point out obvious flaws.

Beyond tools and perhaps more importantly how we manage our workflow also impacts Time To First Review. This might involve re-evaluating our pull request templates to ensure they provide all necessary information for a review or adjusting our notification settings so that reviewers are alerted as soon as a pull request is made. We also emphasise the importance of reviews in our daily stand ups and encourage team members to prioritise them.

If I were to attempt to appear smarter than I really am I would condense it into a basic formula

$$ T=n×rq×c​ $$

$$ T $$

  • be the Time to First Review (TTFR).
  • nnn be the number of available reviewers.
  • qqq be the queue length of pending reviews.
  • ccc be the complexity of the code submitted for review.
  • ebe the average review speed per reviewer.

In this equation:

  • The term q×c represents the total "review load," combining the number of items waiting (queue length) and the complexity of each item.

    q×cq \times c

  • The denominator n×r represents the total review capacity, which is the number of reviewers multiplied by how quickly each can review (assuming simplicity in uniform speed and capability).

    n×rn \times r

This equation suggests that the limiting factors are the pool of total reviews for the team, greater complexity of the code in the review and decreases with more reviewers or faster review capabilities.

It's a simplified model but can be a good starting point for analysing and improving the review process in a development team. If you're looking for a more detailed or specific model, we can adjust the parameters or add more factors to reflect additional complexities.

Lastly, resource allocation is critical, especially during peak times. We analyse our development cycles to predict when we'll likely see a surge in review needs and plan accordingly, sometimes even reallocating team members temporarily to ensure that reviews are conducted swiftly and efficiently.

In all, reducing TTFR isn't just about speeding up a metric. It's about building a responsive, engaged, and efficient team that can handle the fast-paced nature of software development today. As we continue to refine our approach, the focus always remains on maintaining a balance between speed and quality, ensuring that every line of code not only moves quickly from conception to deployment without sacrificing quality.