Explore
Pick a validation layer and a configuration
Configurations frozen before the months were collected, run once. The official test: nothing here was chosen after seeing the result.
Changing the layer reloads the configuration list; the fold filter applies to the case study below.
Readable metrics
Activity-recency clocks + Contributor dynamics (Logistic Regression)
oot_champion · 3 folds, 12,318 test rows, 127 advisory outcomes; pooled metrics combine every fold's predictions.
Per-fold detail (3 folds)
| Test window | Labels as-of | Positives / rows | ROC-AUC | Avg precision | AP lift |
|---|---|---|---|---|---|
| 2025-07 – 2025-08 | 2025-08 | 40 / 4106 | 0.589 | 0.0293 | 3.01× |
| 2025-09 – 2025-10 | 2025-10 | 31 / 4106 | 0.632 | 0.0380 | 5.03× |
| 2025-11 – 2025-12 | 2025-12 | 56 / 4106 | 0.758 | 0.0705 | 5.17× |
Feature drivers
What moved the score, and how consistently
Logistic-regression coefficients on standardized features, averaged over the 3 folds; the bar is the mean magnitude and the last column says how many folds agreed on the sign. A driver that flips sign between folds is not a stable signal. Features are hover-documented.
| Feature | Magnitude | Mean coefficient | Direction |
|---|---|---|---|
ghdyn_active_actors_12m | +3.21 | raises risk in 3 of 3 folds | |
ghdyn_new_actors_12m | -3.10 | lowers risk in 3 of 3 folds | |
ghclock_days_since_human_push | -0.70 | lowers risk in 3 of 3 folds | |
ghclock_days_since_any_push | +0.48 | raises risk in 3 of 3 folds | |
ghclock_days_since_tag_create | -0.22 | lowers risk in 3 of 3 folds | |
ghdyn_active_actors_3m | +0.21 | raises risk in 3 of 3 folds | |
ghclock_days_since_pr_opened | -0.20 | lowers risk in 3 of 3 folds | |
ghdyn_has_actors | -0.14 | lowers risk in 3 of 3 folds | |
ghdyn_top_actor_share_12m | +0.14 | raises risk in 3 of 3 folds | |
ghclock_days_since_release | +0.13 | raises risk in 3 of 3 folds | |
ghdyn_departed_actors_12m | -0.12 | lowers risk in 3 of 3 folds | |
ghclock_days_since_pr_review | +0.11 | raises risk in 3 of 3 folds | |
ghdyn_single_actor_12m | -0.09 | lowers risk in 3 of 3 folds | |
ghclock_days_since_pr_merged | +0.06 | raises risk in 3 of 3 folds | |
ghdyn_turnover_rate_12m | +0.05 | raises risk in 3 of 3 folds | |
ghclock_has_events | -0.05 | lowers risk in 3 of 3 folds | |
ghdyn_events_12m | -0.01 | lowers risk in 2 of 3 folds | |
ghclock_days_since_issue_opened | -0.01 | lowers risk in 3 of 3 folds |
What the holdout ranking looks like
Advisories caught vs plugins reviewed, out-of-time
Reviewing the top 20% of plugins by score would have caught 52% of the advisories that followed across the holdout folds (by fold: 42%, 52%, 61%), against a base rate of 1.0%. The thick line is the pooled ranking over every holdout fold; thin lines are the individual folds (12,318 plugin-months, 127 advisory outcomes). The dashed diagonal is random order. Right: the ROC curve of each fold for the first configuration.
Out-of-time outcomes
Top-25 per holdout fold vs. what followed
Activity-recency clocks + Contributor dynamics · Logistic Regression · oot_champion. Each plugin's best month inside the fold is shown; training labels were rebuilt from advisories known before the fold's forecast date, and the folds were run once.
Across the holdout folds shown, 5 of the 75 top-25 slots were followed by an advisory within 180 days. That is the honest precision at 25 for this configuration: a ranking signal with real lift over the base rate, not an operational triage list.
Holdout fold 2025-07 (1 of 25 confirmed)
| # | Plugin | Scored | Score | Confirmed | Severity | Advisory date | Lead time | Advisory |
|---|---|---|---|---|---|---|---|---|
| 11 | git-client | 2025-07 | 79.2% | ✓ | Medium (5.0) | 2025-12-10 | 162 days | SECURITY-3614 |
The other 24 of the top 25
| # | Plugin | Scored | Score | Confirmed | Severity | Advisory date | Lead time | Advisory |
|---|---|---|---|---|---|---|---|---|
| 1 | ec2 | 2025-08 | 88.7% | – | no advisory in the 180-day window | |||
| 2 | pipeline-graph-view | 2025-08 | 87.0% | – | no advisory in the 180-day window | |||
| 3 | github-branch-source | 2025-08 | 85.9% | – | no advisory in the 180-day window | |||
| 4 | cloudbees-folder | 2025-07 | 85.6% | – | no advisory in the 180-day window | |||
| 5 | kubernetes | 2025-08 | 84.8% | – | no advisory in the 180-day window | |||
| 6 | cloudbees-bitbucket-branch-source | 2025-07 | 83.8% | – | no advisory in the 180-day window | |||
| 7 | google-compute-engine | 2025-07 | 82.3% | – | no advisory in the 180-day window | |||
| 8 | git | 2025-07 | 80.9% | – | no advisory in the 180-day window | |||
| 9 | datadog | 2025-08 | 80.4% | – | no advisory in the 180-day window | |||
| 10 | junit | 2025-07 | 79.7% | – | no advisory in the 180-day window | |||
| 12 | oic-auth | 2025-08 | 78.5% | – | no advisory in the 180-day window | |||
| 13 | credentials | 2025-07 | 78.3% | – | no advisory in the 180-day window | |||
| 14 | configuration-as-code | 2025-07 | 78.2% | – | no advisory in the 180-day window | |||
| 15 | workflow-cps | 2025-08 | 77.8% | – | no advisory in the 180-day window | |||
| 16 | branch-api | 2025-07 | 77.3% | – | no advisory in the 180-day window | |||
| 17 | active-directory | 2025-07 | 76.8% | – | no advisory in the 180-day window | |||
| 18 | workflow-job | 2025-07 | 76.7% | – | no advisory in the 180-day window | |||
| 19 | pipeline-model-api | 2025-07 | 76.6% | – | no advisory in the 180-day window | |||
| 20 | pipeline-model-declarative-agent | 2025-07 | 76.6% | – | no advisory in the 180-day window | |||
| 21 | pipeline-model-definition | 2025-07 | 76.6% | – | no advisory in the 180-day window | |||
| 22 | pipeline-model-extensions | 2025-07 | 76.6% | – | no advisory in the 180-day window | |||
| 23 | pipeline-stage-tags-metadata | 2025-07 | 76.6% | – | no advisory in the 180-day window | |||
| 24 | lockable-resources | 2025-08 | 76.3% | – | no advisory in the 180-day window | |||
| 25 | badge | 2025-07 | 75.8% | – | no advisory in the 180-day window | |||
Holdout fold 2025-09 (2 of 25 confirmed)
| # | Plugin | Scored | Score | Confirmed | Severity | Advisory date | Lead time | Advisory |
|---|---|---|---|---|---|---|---|---|
| 2 | github-branch-source | 2025-10 | 89.3% | ✓ | Medium (4.3) | 2026-04-29 | 210 days | SECURITY-3702 |
| 16 | git-client | 2025-09 | 75.5% | ✓ | Medium (5.0) | 2025-12-10 | 100 days | SECURITY-3614 |
The other 23 of the top 25
| # | Plugin | Scored | Score | Confirmed | Severity | Advisory date | Lead time | Advisory |
|---|---|---|---|---|---|---|---|---|
| 1 | ec2 | 2025-10 | 91.5% | – | no advisory in the 180-day window | |||
| 3 | pipeline-graph-view | 2025-10 | 87.6% | – | no advisory in the 180-day window | |||
| 4 | cloudbees-folder | 2025-09 | 84.7% | – | no advisory in the 180-day window | |||
| 5 | cloudbees-bitbucket-branch-source | 2025-09 | 83.9% | – | no advisory in the 180-day window | |||
| 6 | google-compute-engine | 2025-09 | 82.5% | – | no advisory in the 180-day window | |||
| 7 | datadog | 2025-09 | 81.7% | – | no advisory in the 180-day window | |||
| 8 | kubernetes | 2025-09 | 80.3% | – | no advisory in the 180-day window | |||
| 9 | workflow-cps | 2025-09 | 78.9% | – | no advisory in the 180-day window | |||
| 10 | ldap | 2025-09 | 78.3% | – | no advisory in the 180-day window | |||
| 11 | active-directory | 2025-09 | 77.6% | – | no advisory in the 180-day window | |||
| 12 | git | 2025-09 | 77.3% | – | no advisory in the 180-day window | |||
| 13 | oic-auth | 2025-10 | 77.0% | – | no advisory in the 180-day window | |||
| 14 | junit | 2025-09 | 76.7% | – | no advisory in the 180-day window | |||
| 15 | azure-vm-agents | 2025-10 | 76.3% | – | no advisory in the 180-day window | |||
| 17 | branch-api | 2025-10 | 75.5% | – | no advisory in the 180-day window | |||
| 18 | support-core | 2025-10 | 74.9% | – | no advisory in the 180-day window | |||
| 19 | pipeline-model-api | 2025-09 | 74.6% | – | no advisory in the 180-day window | |||
| 20 | pipeline-model-declarative-agent | 2025-09 | 74.6% | – | no advisory in the 180-day window | |||
| 21 | pipeline-model-definition | 2025-09 | 74.6% | – | no advisory in the 180-day window | |||
| 22 | pipeline-model-extensions | 2025-09 | 74.6% | – | no advisory in the 180-day window | |||
| 23 | pipeline-stage-tags-metadata | 2025-09 | 74.6% | – | no advisory in the 180-day window | |||
| 24 | lockable-resources | 2025-10 | 74.1% | – | no advisory in the 180-day window | |||
| 25 | credentials | 2025-09 | 73.9% | – | no advisory in the 180-day window | |||
Holdout fold 2025-11 (2 of 25 confirmed)
| # | Plugin | Scored | Score | Confirmed | Severity | Advisory date | Lead time | Advisory |
|---|---|---|---|---|---|---|---|---|
| 2 | github-branch-source | 2025-11 | 87.1% | ✓ | Medium (4.3) | 2026-04-29 | 179 days | SECURITY-3702 |
| 6 | workflow-cps | 2025-12 | 80.6% | ✓ | Medium (4.3) | 2026-06-24 | 205 days | SECURITY-3677 |
The other 23 of the top 25
| # | Plugin | Scored | Score | Confirmed | Severity | Advisory date | Lead time | Advisory |
|---|---|---|---|---|---|---|---|---|
| 1 | ec2 | 2025-11 | 91.5% | – | no advisory in the 180-day window | |||
| 3 | pipeline-graph-view | 2025-12 | 84.7% | – | no advisory in the 180-day window | |||
| 4 | cloudbees-bitbucket-branch-source | 2025-11 | 82.2% | – | no advisory in the 180-day window | |||
| 5 | cloudbees-folder | 2025-11 | 81.3% | – | no advisory in the 180-day window | |||
| 7 | junit | 2025-11 | 79.4% | – | no advisory in the 180-day window | |||
| 8 | kubernetes | 2025-12 | 79.2% | – | no advisory in the 180-day window | |||
| 9 | google-compute-engine | 2025-11 | 76.1% | – | no advisory in the 180-day window | |||
| 10 | datadog | 2025-11 | 75.9% | – | no advisory in the 180-day window | |||
| 11 | pipeline-model-api | 2025-12 | 75.8% | – | no advisory in the 180-day window | |||
| 12 | pipeline-model-declarative-agent | 2025-12 | 75.8% | – | no advisory in the 180-day window | |||
| 13 | pipeline-model-definition | 2025-12 | 75.8% | – | no advisory in the 180-day window | |||
| 14 | pipeline-model-extensions | 2025-12 | 75.8% | – | no advisory in the 180-day window | |||
| 15 | pipeline-stage-tags-metadata | 2025-12 | 75.8% | – | no advisory in the 180-day window | |||
| 16 | credentials | 2025-11 | 75.1% | – | no advisory in the 180-day window | |||
| 17 | branch-api | 2025-11 | 74.2% | – | no advisory in the 180-day window | |||
| 18 | pipeline-maven | 2025-12 | 74.1% | – | no advisory in the 180-day window | |||
| 19 | pipeline-maven-api | 2025-12 | 74.1% | – | no advisory in the 180-day window | |||
| 20 | pipeline-maven-database | 2025-12 | 74.1% | – | no advisory in the 180-day window | |||
| 21 | gradle | 2025-12 | 74.1% | – | no advisory in the 180-day window | |||
| 22 | azure-vm-agents | 2025-12 | 73.6% | – | no advisory in the 180-day window | |||
| 23 | support-core | 2025-12 | 73.5% | – | no advisory in the 180-day window | |||
| 24 | scm-api | 2025-12 | 72.4% | – | no advisory in the 180-day window | |||
| 25 | oic-auth | 2025-11 | 71.8% | – | no advisory in the 180-day window | |||
Advisory details come from the local Jenkins advisory dataset; a confirmed row without details is one whose stored label was positive. Lead time = days from the scored month to publication. Plugin names link to their score page, which shows the plugin's rank at every forecast date.