max/dev-intel-v2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
16 Commits 1 Branch 0 Tags
049609a35820331a339fec7a48c9fb753aa541a2
Commit Graph

16 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jarvis Prime
049609a358 Phase 9d: Human eval score improvement\n\n- Human readability score increased from 63.9% to 78.6%\n- Structural table additions and quick lookup index resolved navigation bottlenecks\n- NOT_FOUND rate dropped from 17.9% to 3.6% 2026-03-10 00:46:37 +00:00
Jarvis Prime
ca11b4459a Agent eval hits 93.4% — target exceeded 
- Fixed ground truth generator to merge Helm entities (matching sysdoc.js pipeline)
- Added Quick Lookup index with name-to-file mapping for agent navigation
- Enriched All Charts table with AppVersion, Dependencies, Values Keys columns
- Increased agent file read cap to 30K for full index coverage
- Tree depth 4 for chart file discovery

Score progression: 54.3% → 84.3% → 88.4% → 93.4%
NOT_FOUND: 41% → 0%
All categories above 75%, easy questions at 98.1%
 2026-03-10 00:40:38 +00:00
Jarvis Prime
304f0a9e9f Phase 9c: Split eval into Agent (file-browsing) and Human (readability) tracks 
Agent eval: 54.3% (22 questions, 40.9% NOT_FOUND)
Human eval: 63.9% (28 questions, 17.9% NOT_FOUND)

Key findings:
- Agent navigation is the bottleneck (2.09/5) — long path-based filenames hurt discoverability
- Human findability is decent (3.46/5) but dependency questions fail (0%) because chart docs for wrapper charts don't surface their sub-chart deps
- Both tracks show strong precision (4.4+/5) — very low hallucination
- Resources (91%) and interactions (95%) score great for humans
- Configuration and contracts are solid across both tracks
 2026-03-09 23:55:54 +00:00
Jarvis Prime
0cc4abcb0f Phase 9b: structural documentation improvements\n\n- sysdoc.js: Added Summary Statistics, Top Charts, and K8s Resource Types to architecture doc\n- Addresses ratchet failures where system-wide rollups were missing from generated prose\n- Eval v2 shows minor improvement, though RAG context window still limits wide scatter-gather queries 2026-03-09 23:40:07 +00:00
Jarvis Prime
b99341e8bc Phase 9: Doc Evaluation Harness\n\n- eval-questions.js: Generates ground-truth questions from raw source data\n- eval.js: LLM-as-judge scoring harness (answers from docs, scores against truth)\n- Generated 33 questions covering config, dependencies, resources, and interactions\n- Baseline score: 66.7% (configuration 93%, dependencies 77%, structural 31%) 2026-03-09 22:32:41 +00:00
Jarvis Prime
d9fa087e22 Phase 6+7: LLM prose generation pass over Foxtrot docs\n\n- Ran Claude Haiku to generate prose for architecture, subsystems, flows, and 124 Helm contracts\n- Fixed describeContract prompt in prose.js to correctly identify and describe Helm contract types without hallucinating\n- 80 files generated with rich architectural summaries 2026-03-09 20:15:50 +00:00
Jarvis Prime
4f7c77b3b1 Phase 8b: Helm contract extraction + diagram support 
- extractHelmContracts() in contracts.js: values, services, workloads, deps
- Merged Helm contracts into main pipeline (124 contracts on Foxtrot)
- diagrams.js: generateContractDiagram now handles Helm types
- Sanitized Mermaid class names for Helm contracts
- 1601-line contracts index with full classDiagram
 2026-03-09 20:05:52 +00:00
Jarvis Prime
f49a6c2dd9 Phase 8: Helm chart extraction with Go template support 
- extract-helm.js: strips Go templates, parses Chart.yaml/values.yaml/templates
- Extracts K8s resource kinds, cross-chart interactions, shared secrets, ports
- generateHelmDiagram() for Mermaid interaction graphs
- Integrated into sysdoc.js: Helm entities merge into main knowledge graph
- Dir-based filenames to handle duplicate chart names
- .gitignore for node_modules, snapshots, venv, wasm
- 76 charts, 1813 entities, 1769 relationships on Foxtrot
 2026-03-09 20:03:04 +00:00
Jarvis Prime
d19cee36d7 Phase 6+7D: Sonnet prose generation integration 2026-03-09 18:44:19 +00:00
Jarvis Prime
1869fcb5b2 7B: Add parse error tracking (BMad review fix) 2026-03-09 18:35:13 +00:00
Jarvis Prime
ca02fe131b Phase 7F: Supergraph Multi-Repo Merge 2026-03-09 18:19:14 +00:00
Jarvis Prime
d9fd7e3284 Phase 7B, 7E, 7D: Contracts, Diagrams, Sysdoc 2026-03-09 14:42:15 +00:00
Jarvis Prime
4c212740a2 Phase 7A+7C: Subsystem aggregator + Flow tracer (post-review fixes) 2026-03-09 06:51:32 +00:00
Jarvis Prime
4221ab4d76 Phase 6: LLM doc generation + Phase 7 system-docs spec 2026-03-09 06:20:54 +00:00
Jarvis Prime
7d5b6cbc32 Add README with benchmarks and V1 vs V2 comparison 2026-03-09 05:30:21 +00:00
Jarvis Prime
efb12d003b Dev Intel Pipeline v2 — multi-language semantic graph extractor 
Phase 1: extract.js — tree-sitter AST parser (TS/JS/Python/Go/Java/Bash) + config parsers (YAML/HCL)
Phase 2: graph.js — in-memory directed graph store with build/query/diff CLI
Phase 3: namespace.js — cross-repo namespace registry with 3-tier resolution
Phase 4: semantic-diff.js — categorized diffs with impact scoring (0-100)
Phase 5: pipeline.js — batch extraction, incremental diffing, benchmarking

Benchmark: 4,325 files, 21,646 nodes, 133,979 edges in 67s (15ms/file)
BMad SPA reviews: all phases GO
 2026-03-09 05:29:29 +00:00