Skip to main content

Retention

OTIF widget: Retention with four indicators

Retention measures what percentage of added lines of code survived, that is, was not overwritten or deleted, after a given number of days. In the interface, the widget is called "OTIF - Retention", from on time in full, a measure of delivering work in full. It answers the question of whether the changes made stick, or are quickly corrected or reverted.

Indicators​

Four variants of the same measurement, differing only in the number of days after which the durability of the change is checked:

  • R1: after 1 day.
  • R7: after 7 days.
  • R30: after 30 days. The only one visible in tables by default.
  • R90: after 90 days.

Calculation method​

For each commit, the added lines are compared with what remains in the code after the given number of days have passed. The mechanism recognizes the typical operations that tidy up Git history (merge, squash, rebase, cherry-pick: different ways of combining and rewriting commit history that do not, by themselves, change the final code) and checks the appropriate version of the code depending on whether the commit in question ultimately landed on the main branch, so that Retention is not penalized for this kind of standard history cleanup.

The result for a project, team, or participant is an average weighted by the number of added lines: a commit adding 500 lines has a greater effect on the result than a commit adding 5.

Observation window​

Assessing durability requires waiting out the full number of days, so each indicator is calculated on commits from the selected date range shifted back by exactly that many days. With a range of the last 14 days, R1 describes commits from 1 to 15 days ago, and R90 describes commits from 90 to 104 days ago. Each of the four numbers therefore has a full observation window behind it, and a commit too fresh to assess for a given indicator does not enter it at all.

This has one important consequence when reading it: the four indicators describe four different sets of commits, not the same code viewed at successive moments. R30 being higher than R7 is therefore not an error, just information that code written a month ago turned out to be more durable than that from the last week.

Interpreting the Retention level

High Retention (close to 100%) means that a given person's or team's changes mostly stay in the code. Low Retention can be a natural part of the work and does not, by itself, say anything about its quality: it can point to intensive iteration, active code review resulting in corrections, or work on a part of the code that keeps changing.

  • Low R1 and R7 are a picture of code corrected right after being written: bugfixes, review comments, polishing a freshly added piece. With an established pattern of work, R30 and R90 are then usually low as well, because a line once overwritten does not come back.
  • R30 and R90 markedly lower than R1 and R7 point to changes coming from outside the development process: rebuilds after a change in business requirements, through which code that previously worked gets rewritten.
  • High R30 and R90 with a low R1 says that the work from previous weeks has held up, and the last few days were a period of intensive corrections.

Because the aggregation is weighted by lines, a few large commits with low durability can dominate the result more strongly than the mere number of commits would suggest. With an unusual result, it is worth checking whether it is not driven by a single large change, for example an import or a mass migration.

Levels​

Retention is calculated for a participant, then aggregated (using the same method: the line-weighted average described above) up to the project, the team, and separately as a participant's combined retention across all their projects. Because this aggregation is weighted by lines rather than a plain average of the individual participants' results, the result for a project or team is dominated by the people who added the most code in the given period: a participant with a small line contribution, even with an extremely low or high Retention of their own, has little effect on the result of the whole group.

See also​