Distribution Charts & Code Bypassing Pull Requests

Hot on the announcement of 4 new charts earlier this week we bring 2 new updates today.

We got valuable feedback from our users that while their Cycle Time across each stage is useful to quickly spot where the bottlenecks in their Software Delivery Pipeline are, it often prompts the question: How do I know if my Average Cycle Time isn’t dominated by outliers?

In the coming months you’ll see more features being released that help you as an engineering leader to identify outliers (we released the default option to remove stalled PRs back in June). Which is why we’re introducing Distribution Charts.

Distribution Charts

Now your Cycle Time in each stage (WIP, Review, Merge and Release) and your Lead Time, no longer just show their trend over time but also provide a distribution.

Screen Shot 2020-08-07 at 2

On the y-axis you find the # of Pull Requests and on the x-axis you find your PRs bucketed based on the time spent in each stage. This is done on a logarithmic scale so you can quickly spot your outliers.

Let me give you a great example where this is very apparent. The following company uses CI/CD across their repositories and almost always releases in less ±1 minute, but it’s Average Cycle Time in their Release stage is 24 hours. With the distribution chart you can now see that while 97% of their PRs release in ±1 minute, a small number of PRs end up being significant outliers:

Screen Shot 2020-08-07 at 2
Screen Shot 2020-08-07 at 2

As an engineering leader, it’s the outliers here you want to focus on and understand what happened there, and to either choose to ignore it or to adopt your practices or tooling to avoid these.

Code Bypassing Pull Requests

Athenian’s users are diligent about using Pull Requests. However at times there are a hidden issues in our Software Delivery Pipeline that stops engineers from using PRs and instead commit directly. A common example is that locally all the tests passed and the engineer finds himself waiting for a long running CI check when they open a PR, so instead they choose to commit directly.

As engineering leaders we want to discover “Why are we committing directly without a Pull Request?, this new table in the Work In Progress section shows you the repositories where this is most common and allows you to inspect the offending commits.

Screen Shot 2020-08-07 at 7

About the author

Eiso Kant

Eiso Kant is the Founder and CEO of Athenian where he gets to work on helping software development teams reach their potential. Eiso is a big believer of using data to support engineering teams in improving their processes and removing bottlenecks. Never forgetting that software development is about people and culture first, the tools we use just give us extra superpowers.