What is Impact?

Tags: Flow

Impact is a way to measure the ‘bigness’ of code changes that are happening, in a way that goes beyond simplistic measurements like lines of code.

Who can use this?



Impact attempts to answer the question: “Roughly how much cognitive load did the engineer carry when implementing these changes?”

To illustrate, consider the following example:

One engineer makes a contribution of 100 new lines of code to a single file.

Compare that to another engineer’s contribution, which touches three files, at multiple insertion points, where they add 16 lines, while removing 24 lines.

The significance of each contribution can’t be boiled down to just the amount of code being checked in. Even without knowing specifics, it’s likely that the second set of changes were more difficult to implement, given that they involved several spot-edits to old code.

On the left we have someone adding 100 lines of new code to a single file. On the right is an example that represents only 24 lines of new code written, but there’s a bit more going on here:

  • this change required modifying previous work
  • the edits happened in four different locations
  • three different files were affected

Even without knowing the severity of changes or comparing to historical changes, it’s probably safe to assume that the second contribution involved a higher cognitive load and therefore carries a higher impact score.

Although Change set #1 is technically more code, the added complexity of the work happening in Change set #2 could arguably make that change set at least as much work, and possibly more—even in the simplistic representation above, it’s clear that there’s more than just lines of code at play.

Impact is a measure of work size that takes this into account. Impact considers the following:

  1. The amount of code in the change
  2. What percentage of the work is edits to old code
  3. The surface area of the change, like the number of edit locations
  4. The number of files affected
  5. The severity of changes when old code is modified
  6. How this change compares to others from the project history

We roll all of this up to create a score that more accurately lets us understand how “big” a particular piece of work is. The example above is based on data from a Flow repository, and when we evaluate the impact of these two changes we got the following results:

  • Change set #1: 24 impact score
  • Change set #2: 36 impact score

The more involved nature of Change set #2 yielded a higher impact score, even though only about a quarter of the amount of raw code was written.

Tip: Impact and Commit complexity can seem very similar. The two metrics serve two different purposes and are not interchangeable. While Impact considers the cognitive load of the work being done, Commit complexity looks at the riskiness of a commit.

back to top

If you need help, please contact Pluralsight Support.