Security Metrics / BSides Charm

Why Vulnerability
MTTR Alone
Misleads
Add MOVA to Measure Real Risk
Caleb Kinney

MTTR just doubled

Mean time to remediate

Last month

15d

→

This month

31d

Dashboards call this failure. The better question is: what actually changed?

What happened to backlog age? What happened to the oldest findings?

This actually happened

MTTR increased after we made it a KPI.

  • MTTR became a KPI
  • Teams started closing older findings
  • MTTR went up
  • The first question in the review was: “What changed?”
  • The dashboard said: ‘we got worse’

MTTR rose because we finally closed the oldest findings.

Paradox

The MTTR Paradox

MTTR ↑

while

MOVA ↓

MTTR can rise while backlog age falls.

Flow vs. backlog age

Flow

MTTR

How fast are we closing work?

Backlog age

MOVA

How old is the open backlog today?

MTTR shows flow.

MOVA shows backlog age.

Averages hide the tail

Most findings

95

at 7 days

Old tail

5

at 400 days

The average looks fine.

The oldest findings are not moving.

MTTR measures closed work

MTTR

mttr = (
    vulns.filter(pl.col("resolved_at").is_not_null())
    .with_columns(
        (pl.col("resolved_at") - pl.col("created_at")).dt.total_days().alias("age_days")
    )
    .select(pl.col("age_days").mean().alias("mttr_days"))
)

Closed work. Time to remediate. Age at closure.

MOVA measures backlog age

MOVA

mova = (
    vulns.filter(pl.col("resolved_at").is_null())
    .with_columns(
        (pl.datetime_now() - pl.col("created_at")).dt.total_days().alias("age_days")
    )
    .select(pl.col("age_days").mean().alias("mova_days"))
)

Backlog age today. What is still open.

You already have the data

  • MTTR and MOVA use the same data
  • A CSV export is often enough
  • Backlog age comes from open findings
  • Older findings usually create more exposure over time

SLA views tell you if you met a deadline, not the backlog age you still carry.

A simple simulation

Hold constant

Setup

Start with one backlog.

Keep arrivals and capacity fixed.

Change one thing

Prioritization

Newest-First vs. Oldest-First.

Only the order of closure changes.

Same team. Same capacity. Different prioritization.

MTTR rewards recent closures

Line chart of MTTR over time under two prioritization strategies. Newest-First is highlighted and finishes with the lower MTTR because it closes recent findings first.

On MTTR, Newest-First looks like the winner.

MOVA shows backlog age

Line chart of MOVA over time under two prioritization strategies. Oldest-First is highlighted and finishes with the lower backlog age because it removes older findings first.

On backlog age, Oldest-First is the healthier system.

The 180+ tail tells the truth

Line chart of open vulnerabilities older than 180 days under two prioritization strategies. Oldest-First is highlighted and reduces the tail, while Newest-First leaves the 180+ tail stranded.

Newest-First leaves the aging tail stranded.

End state: the tail tells the story

Oldest-First

0

findings older than 180 days

Newest-First

146

findings older than 180 days

Under Newest-First, the oldest backlog barely moves.

Same data, wrong winner

Strategy comparison
Newest-First wins on MTTR. Oldest-First wins on backlog age.
MTTR Backlog Age Backlog Interpretation
MTTR MOVA 180+ tail Open count Interpretation
Oldest-First 92.6 36.5 0 151 Recommended: best backlog outcome
Newest-First 16.1 726.4 146 151 MTTR-only winner

Read only MTTR and you pick the wrong winner.

This is not anti-MTTR

  • MTTR is still useful
  • It shows age at closure
  • It reveals process friction

The problem is using MTTR alone.

When the dashboard shows only MTTR

Newest-First

16.1 days

MTTR

Oldest-First

92.6 days

MTTR

If this is the only KPI, the system rewards the strategy that closes recent work fastest.

When the dashboard shows the full system

Newest-First

MTTR 16.1 days

MOVA 726.4 days

Open count 151

180+ days 146

Oldest-First

MTTR 92.6 days

MOVA 36.5 days

Open count 151

180+ days 0

MTTR still matters. It cannot stand alone.

Read the signals together

  • MTTR up + MOVA down = older findings are getting cleared
  • MTTR down + MOVA up = recent closures look better while backlog age worsens
  • MTTR up can mean effort shifted to older findings
  • 180+ up = the aging tail is getting stranded

What to report

  • Put MTTR and MOVA on the same monthly view
  • Add open count and a threshold such as 180+ days
  • Review MTTR only with backlog age and the 180+ tail visible
  • Label the MTTR time window before calling a regression

Add data science to your toolbox

Most teams already have the data. A CSV export is often enough.

Define the metric in code before it reaches a dashboard.

Define the metric
Define the logic in code before it ever reaches a dashboard
Polars
Inspect the raw data
Inspect raw records early to catch edge cases before they spread
Positron
Visualize the trend
Visualize flow, backlog age, and the tail clearly over time
Plotnine
Summarize the backlog
Summarize count, age, and tail so risk is clear at a glance
Great Tables
Explore the metric
Turn the analysis into a simple app people can inspect and use
Shiny for Python
Publish the analysis
Publish code, charts, and narrative together in one consistent artifact
Quarto

Start with an export. Define the metric in code before you visualize it.

A simple policy

Policy

Track MTTR + MOVA

Prioritize older findings first.

Guardrail

MTTR alone

Can reward the wrong queue.

Do not let low MTTR outrank backlog age.

Read the system, not just the metric

MTTR shows flow.

MOVA shows backlog age.

You need both.

typeerror.com/mova