security-tracker-stats-dashboard

Capability: substrate:analytics

Harness: agnostic

Generate a self-contained HTML dashboard of <tracker> repository statistics — issue-lifecycle bands (untriaged / triaged / PR-merged / fixed-released / closed-other), opened-vs-untriaged backlog, cumulative opened/closed, and mean-time-to-triage / first-response / PR-open / PR-merge / advisory-announced.

All charts are line / area (no bars) with connectgaps: true. Plotly loaded via CDN — the output HTML is self-contained but viewing it requires network access for the chart library.

The tool is read-only on GitHub — it does not create or modify issues, comments, labels, or PRs. It only fetches data via gh and renders an HTML file.

The companion agentic skill at .claude/skills/security-tracker-stats-dashboard/SKILL.md wraps this tool and surfaces it through Claude Code’s slash-command interface; both routes (script-only and skill-driven) run the same fetch + render pipeline.

Layout

tools/security-tracker-stats-dashboard/
├── README.md             (this file)
├── default-config.yaml   (config schema + adopter-overridable defaults)
├── render.py             (renders cached data to HTML; reads config)
├── fetch_issues.py       (gh issue list, incl. body + closedByPRs -> issues.json)
├── fetch_roster.py       (gh api collaborators -> roster.txt)
├── fetch_bodies.py       (body + closedByPRs from issues.json -> issue_extra.json)
├── fetch_events.py       (per-issue label history -> events/<N>.json)
├── fetch_prs.py          (per-PR metadata from <upstream> -> prs.json)
└── run.sh                (orchestrator)

Invocation

bash <framework>/tools/security-tracker-stats-dashboard/run.sh [<output-path>]

Env knobs (all optional):

Resume behaviour

Each fetch script resumes from cache, so re-running after a partial failure (rate limit, transient HTTP error) only re-fetches what is missing. Delete the cache dir to force a fresh full fetch.

Fetches are parallelised (ThreadPoolExecutor, ~10 workers). A fresh run is ~5–10 minutes on a 250-issue tracker; incremental re-renders (cache warm) are ~30 seconds.

Configuration

render.py loads configuration in this order, highest priority last:

1. default-config.yaml (in this directory).
2. $TRACKER_STATS_CONFIG overlay YAML, when set (typically <adopter-repo>/.apache-magpie-overrides/security-tracker-stats.yaml). Deep-merged with the default. Every list is REPLACED entirely (not concatenated) — milestones, categories, bot_prefixes, keywords, and any other list-valued key. Overlaying a single entry requires re-stating the whole list, otherwise the defaults are silently dropped.
3. Env-var quick overrides for the most common knobs: TRACKER_STATS_BUCKETS, TRACKER_STATS_START, TRACKER_STATS_UPSTREAM_REPO.

See default-config.yaml for the full schema with inline documentation of every predicate key.

Categories (lifecycle bands)

Mutually-exclusive states per tracker at each bucket-end snapshot, evaluated top-to-bottom with first-match-wins. Multiple rules can share a name to express disjoint branches of the same final category — the default set uses this for the open / closed fork on fixed_released. The set of distinct names defines the stack order in the lifecycle chart (overridable via the stack_order: config key).

Supported predicate keys:

Snapshot reconstruction replays each tracker’s event stream (labeled / unlabeled / closed / reopened) chronologically from {labels: [], state: OPEN} at createdAt, evaluated at the bucket-end timestamp (Mar 31 / Jun 30 / Sep 30 / Dec 31 at 23:59:59 UTC for quarterly; calendar-month last day for monthly).

Time-to-triage signal

First tracker comment whose author is on the roster (from fetch_roster.py) AND whose body matches any triage.keywords[] regex (case-insensitive). Falls back to the first non-bot roster comment when no keyword matches (useful for older trackers that predate the team’s triage-comment convention). The triage.bot_prefixes[] list skips automated rollup / sync / import comments.

Milestones (vertical annotations)

milestones[] produces a vertical dashed line + top-label annotation on every time-axis chart. Each entry needs date: YYYY-MM-DD (mapped onto the bucket axis) and label. Set milestones: [] in an overlay to remove them entirely.

Current-bucket projection (projection)

projection:
  enabled: true
  min_elapsed_fraction: 0.1

The last bucket on the axis is always cut short by “now”, so every count in it reads low against complete buckets — regenerate the dashboard on the 8th of a month and September looks like a collapse in reports rather than a month that is 23 % over. The projection extrapolates that final bucket to where it is on course to end.

Two kinds of series need two different extrapolations:

Only the movement inside the bucket is scaled for a level series — scaling the level itself would multiply years of accumulated history by four. A falling level (a backlog being worked down) projects further down, floored at zero. A rate projection is never below what has already happened.

Projections are drawn on every chart that carries a projectable series: the lifecycle bands, Reported vs. opened vs. untriaged, the cumulative chart, and the rejections chart. Each is a dotted two-point segment from the last complete bucket’s actual value to the projected end-of-bucket value, so a forecast can never be mistaken for a measurement. On the stacked lifecycle chart each band’s forecast is drawn at its projected position in the stack, with the band’s own projected count in the hover (legend entries suppressed — five extra rows would double the legend). Since the bands partition every tracker, their projected values sum to the projected cumulative opened, which makes an easy sanity check.

The mean-time charts are deliberately not projected. A mean over the items seen so far is already an estimate of the bucket’s mean, not a partial accumulation — scaling it by elapsed time would be meaningless.

An HTML header banner carries the operational headline (reported, opened, and the projected untriaged backlog); stdout lists every projected series:

Current-bucket projection (2026-09, 23% elapsed, now -> month-end):
  opened                      9 -> 38
  rejected                    3 -> 13
  reported                   12 -> 51
  cum_opened                355 -> 384
  cum_closed                313 -> 323
  band:open_untriaged        14 -> 7
  band:open_triaged          16 -> 39

min_elapsed_fraction suppresses the projection early in a bucket, where the extrapolation is noise: two days into a month a single report projects to fifteen. Below the threshold the banner and traces are omitted and stdout says why. The projection is also skipped when the axis holds a single bucket — the level series have no baseline to project from. enabled: false switches the whole stat off.

Rejected-without-tracker ledger (rejections_ledger_label)

rejections_ledger_label: rejections-ledger   # null to disable

The security-issue-import skill sometimes rejects a report with a canned reply without creating a tracker — the disposition lives only on the mail thread, so it is invisible to every other stat on this dashboard. To make those rejections countable the team records each one as a comment on a single dedicated ledger issue in the <tracker> repo: one open issue, labelled with rejections_ledger_label (default rejections-ledger) and not carrying the security-marker label.

fetch_issues.py already pulls every issue’s labels + comments, so the ledger and its comments arrive in issues.json with no extra fetch. render.py:

• identifies any issue whose labels include rejections_ledger_label as a ledger issue;
• excludes the ledger issue from all tracker classification — it never appears in the lifecycle bands, open/closed KPIs, triage medians, or any per-bucket count;
• parses the ledger’s comments for the rejection markers below and renders a rejected (no tracker) count.

Each per-rejection comment carries a machine-parseable block:

<!-- rejection v1 -->
date: YYYY-MM-DD
reporter: <email/name>
canned: <canned-response-slug>
thread: <url-or-threadid>
summary: <one line>

A one-time historical backfill is recorded as a single comment (count only, no per-entry dates):

<!-- rejection-backfill v1 count: N -->

Dated rejections are bucketed by their date: into the same monthly / quarterly axis as the trackers and rendered as a rejected (no tracker) area series (c_rejected). The undated backfill count: N is kept as a separate historical (pre-ledger) headline number rather than smeared across buckets, so the per-bucket series only ever shows real dated rejections. Both the stdout summary and an HTML header banner print <dated> + <historical> = <total>.

The parse is defensive: comments without a marker are ignored, malformed date: lines are skipped, and an absent ledger issue (or rejections_ledger_label: null) simply omits the stat / shows 0.

When upstream_repo is null

The c_prc / c_prm / c_rel PR-driven mean-time charts are omitted, the fetch_prs.py stage is a silent no-op, and the pr_merged_by_snapshot predicate is always false (so the open_pr_merged snapshot back-fill rule is disabled). The remaining charts still render.

Prerequisites

• Runtime: Python 3.11+ (python3). Run each stage as python3 fetch_events.py, etc. Optional: pyyaml — when missing, render.py uses a bundled minimal YAML subset parser sufficient for default-config.yaml and typical overlays; set TRACKER_STATS_PY=uv-yaml to pin clean PyYAML invocations via uv run --with pyyaml.
• CLIs: gh (authenticated with read access to <tracker>, and to <upstream> when PR metadata is enabled); jq (used by fetch scripts via gh --jq).
• Credentials / auth: gh auth status must show a logged-in account with read access to <tracker>. <upstream> access is required only when pr_repo is configured.
• Network: api.github.com (REST) for event and PR fetch; Plotly CDN (cdn.plot.ly) when viewing the generated HTML dashboards in a browser.

Failure modes