Bundle Quality Scorecards
Deterministic, mathematically reproducible quality scoring (0–100) and letter grades (A–F) for agent memory bundles. Protect your AI coding agents against hallucinated patterns, broken knowledge graphs, and unmaintained legacy code.
The Three Pillars of Memory Quality
Every OKF bundle is scored across three non-overlapping quality pillars computed purely offline and deterministically without external telemetries.
Epistemic Maintenance
Freshness horizons, active concept lifecycles, and zero-staleness guarantees.
Architectural decisions degrade when underlying frameworks change. OKF concepts define explicit review horizons via stale_after: YYYY-MM-DD.
100 × (1 - stale_count / concept_count)
- 100% Score: All concepts are within their active freshness horizon or are durable invariants.
- Deductions: Linear degradation for every concept past its
stale_afterdate. - Protection: Prevents AI agents from applying obsolete API configurations or deprecated security rules.
Human Stewardship
Human-in-the-loop attestation and maintainer accountability.
While AI agents can generate hundreds of decisions in seconds, production safety requires human attestation. Concepts declare human review via verified.by: human/<handle>.
100 × (human_verified / concept_count)
- 100% Score: All decisions and invariants verified or authored by designated human domain experts.
- Transparent Attestation: Official Core bundles are verified by maintainers (e.g.
human/sknr). - Defense: Mitigates synthetic agent-hallucination loops in autonomous workflows.
Conformance Integrity
Strict OKF v0.2 DAG validation, zero broken links, and zero orphans.
Agents navigate project memory by traversing markdown links and indexes. Any broken link creates reasoning blind spots; orphaned concepts waste context windows.
max(0, 100 - (broken×50 + orphans×30))
- Zero Tolerance: Strict DAG validation according to OKF v0.2 §5 & §7.
- -50% Penalty: Applied for every broken relative markdown link.
- -30% Penalty: Applied for every orphaned or unindexed concept.
Letter Grades & Autonomous Trust Levels
The composite quality score maps directly to standardized letter grades (A through F), defining safe agent automation boundaries.
| Grade | Score | Trust Tier | Typical Characteristics | Recommended Agent Policy |
|---|---|---|---|---|
| Grade A 90 – 100 | 90 – 100 | Enterprise & Mission-Critical | Characteristics 100% conformant graph, zero broken links, 100% verified by human maintainers, zero stale concepts. | Agent Policy Safe for autonomous agent execution with write and tool permissions. |
| Grade B 80 – 89 | 80 – 89 | Production Ready | Characteristics Conformant graph, majority human-verified (>60%), fresh concepts, minor formatting warnings. | Agent Policy Standard development workflows; human approval required for live deployments. |
| Grade C 70 – 79 | 70 – 79 | Community & Experimental | Characteristics Valid DAG without broken links, automated AI generation with partial human oversight (20–50%). | Agent Policy Prototyping, exploratory tasks, and local sandbox development. |
| Grade D 60 – 69 | 60 – 69 | Caution / Needs Review | Characteristics Pure unverified bot output, approaching staleness horizon, or weak index connectivity. | Agent Policy Read-only reference; agents must request explicit user sign-off before adoption. |
| Grade F < 60 | < 60 | Failing / Untrusted | Characteristics Broken links, orphaned concepts, expired unmaintained rules, or schema violations. | Agent Policy Blocked. Quality gating strictly rejects installation. |
Quality Gating in CLI & CI/CD Pipelines
Enforce quality policies programmatically using the zero-dependency okf Go toolchain.
1. Inspect Local Quality Scorecard
Run full epistemic, stewardship, and conformance evaluation on your local bundle:
2. Quality Gating During Pull
Block unmaintained or unverified bundles before they touch your project memory:
3. Verify Dependencies on Restore
Guarantee all vendored bundles in knowledge/vendor meet organizational thresholds:
4. Machine-Readable CI Checks
Integrate scorecard gates directly into GitHub Actions or GitLab CI runners:
How Authors Reach Grade A (100)
Follow this 4-step checklist to achieve Grade A certification for your decentralized community bundle.