The Vetting Lab
Editorial evaluations under the ACS methodology. Assessments publish here as editorial review completes — small early numbers by design, growing as new tools work through the pipeline.
Methodology v3.4
What the Lab does
The institutional commitment to publicly assess AI tools against the same rubric every AECO Shield customer uses on their stamped work.
Function 1
Run major AI tools through the full ACS methodology — 22 questions, 7 domains, hard filters, the Q21b liability-transfer test. Publish the score, the rationale, and the methodology version used. Refresh quarterly as tools evolve and standards change.
Function 2
As the Lab accumulates audit-artifact outcomes over time, domain weights and threshold values recalibrate against what the evidence shows — and the rationale for every change is published. The methodology earns its weights from evidence, not assertion.
How evaluation works
The pipeline behind every published assessment. No black-box AI scoring — the AI computes a provisional score, the curator reviews the mandatory-review questions, and only the reviewed result reaches the public registry.
A vendor registers a tool + submits it for assessment via the vendor dashboard. Basic metadata: name, category, discipline scope, the vendor's technology E&O and cyber coverage, data-security posture, and the responsible-charge design of the tool.
The scanner runs the 22-question ACS rubric across 7 layers (governance, transparency, security, data protection, oversight, quality, liability). Produces an ACS score + band + per-layer breakdown.
Deterministic gates: responsible-charge preservation (HF1), discipline capability (HF2), technology E&O and cyber coverage (HF3), insurance and contractual risk (HF4), and public-contract eligibility (HF5). A hard-filter fail blocks the tool for the affected project class regardless of ACS score.
Reads the actual liability-transfer language in the tool’s terms of service. Derives the Stamp-Safe classification — the specific answer to "is this defensible for a licensed professional to stamp?"
A curator reviews the mandatory-review questions the AI provisional score flagged, resolves the ambiguities, and adjusts scores where the AI misread the source material. Editorial state is auditable.
The assessment is promoted to the public registry only when editorial review is complete. Publishing is admin-gated at the API level; a scored-but-unpublished assessment does not surface publicly.
Published assessments appear on the public /registry, filterable by score, verdict, jurisdiction, discipline, and framework coverage. The Vetting Lab’s published output IS the Registry.
Published assessments
A small early set by design — the Lab produces evaluations at editorial pace, not at scale. Each assessment traces back to the seven-step pipeline above. More publish here as review completes.
Northfield
Northfield
Grounded in real standards
Every rubric question grounds to specific citations from six recognized external frameworks (ISO 42001, NIST AI RMF, EU AI Act, CMMC 2.0, ISO 19650, IBM AI Principles). AECO is not a certification body — the frameworks are the authoritative source; the methodology is how we read them consistently across tools.
Notify me
One email when a new evaluation reaches the registry. No newsletter, no product pitch, no follow-ups.
Vendors submit tools through the Shield workflow. Ad-hoc requests from buyers and practitioners route through contact — turnaround depends on the editorial queue.