feat: repo-agnostic refactor (BMad spec-test-build loop)
- NEW: repo-profiler.js — deterministic archetype detection (Infra, Frontend, Backend, etc.) - NEW: extract-dynamic.js — generic extractor replacing hardcoded Foxtrot patterns - NEW: eval-generator.js — dynamic ground-truth question generation from any repo graph - NEW: specs/bmad-agnostic-refactor-spec.md — full BMad spec with acceptance criteria - REFACTORED: prose.js — two-pass LLM synthesis with rich context (shared secrets, ports, service refs) - REFACTORED: sysdoc.js — wired repo-profiler + extract-dynamic, --legacy escape hatch - REFACTORED: wiggum-v2.sh — uses eval-generator before benchmarks - FIXED: graph.js — _edgeSet rebuilt on loadSnapshot() (edge dedup was broken) - FIXED: graph.js — recursive sortKeys() for deep equality in diffing - FIXED: prose.js — robust JSON array extraction from LLM output - FIXED: ratchet.js — syntax validation (node --check) before saving LLM mutations - FIXED: extract-dynamic.js — centralized state services regex, added console.warn for silent failures - TESTS: test-eval-generator, test-repo-profiler, test-synthesis-quality + mock fixtures Eval: 81.5% on Foxtrot (fully repo-agnostic, no hardcoded reference pages) BMad reviews: Architect B+, Dev Lead B-, TEA B-
This commit is contained in:
49
wiggum-v2-bmad.log
Normal file
49
wiggum-v2-bmad.log
Normal file
@@ -0,0 +1,49 @@
|
||||
🔁 Ralph Wiggum Loop (V2) — max 1 iterations, target 77%
|
||||
Benchmark: Confluence Gold Standard (/home/node/.openclaw/workspace/projects/dev-intel-v2/eval-confluence-ref-questions.json)
|
||||
|
||||
=== Iteration 1/1 ===
|
||||
📝 Running V2 pipeline...
|
||||
State services: 9
|
||||
Auto-detected 0 entry points
|
||||
Flow traces: 0 valid out of 0 attempted
|
||||
Impact analysis: 0 nodes with downstream dependents
|
||||
Generating architecture overview...
|
||||
Generating prose for subsystem: account-common...
|
||||
Generating prose for subsystem: app-common...
|
||||
Generating prose for subsystem: app-tools...
|
||||
Generating prose for subsystem: compute-common...
|
||||
Generating prose for subsystem: compute-tools...
|
||||
Generating prose for subsystem: control-core...
|
||||
Generating prose for subsystem: ipam-core...
|
||||
Generating prose for subsystem: ipam-tools...
|
||||
Generating prose for subsystem: network-common...
|
||||
Generating prose for subsystem: network-core...
|
||||
Generating prose for subsystem: runtime...
|
||||
Generating prose for subsystem: root...
|
||||
Generating prose for 124 contracts...
|
||||
Agent KB: 12 subsystems, 76 charts
|
||||
Synthesizing dynamic reference pages via LLM...
|
||||
Dynamic reference pages and index synthesized.
|
||||
Generated docs in ./foxtrot-docs
|
||||
- 12 subsystems
|
||||
- 124 contracts
|
||||
- 0 flows
|
||||
📊 Running agent file-browsing eval against Confluence questions...
|
||||
Using model: claude-haiku-4.5
|
||||
Agent Eval: 32 machine-audience questions
|
||||
[1/32] arch-layered-order... 25% (A:1 C:1 P:2 N:1) files:2
|
||||
[2/32] arch-hub-spoke-ownership... 70% (A:4 C:2 P:4 N:4) files:2 [NOT_FOUND]
|
||||
[3/32] arch-aws-regions... 50% (A:2 C:5 P:1 N:2) files:2
|
||||
[4/32] arch-gcp-shared-vpc-host... 45% (A:2 C:1 P:4 N:2) files:2 [NOT_FOUND]
|
||||
[5/32] arch-cidr-employee-access... 25% (A:0 C:0 P:5 N:0) files:2 [NOT_FOUND]
|
||||
[6/32] arch-production-cidr... 25% (A:0 C:0 P:5 N:0) files:2 [NOT_FOUND]
|
||||
[7/32] dep-runtime-common-horizontal... 20% (A:0 C:0 P:3 N:1) files:2 [NOT_FOUND]
|
||||
[8/32] dep-vertical-layers... 45% (A:2 C:1 P:3 N:3) files:2
|
||||
[9/32] dep-create-account-repos... 25% (A:0 C:0 P:5 N:0) files:2 [NOT_FOUND]
|
||||
[10/32] dep-create-cluster-repos... 25% (A:0 C:0 P:5 N:0) files:2 [NOT_FOUND]
|
||||
[11/32] dep-compute-common-deps... 15% (A:1 C:0 P:1 N:1) files:2
|
||||
[12/32] ops-argocd-deployment-flow... 25% (A:0 C:0 P:5 N:0) files:2 [NOT_FOUND]
|
||||
[13/32] ops-ebf-release-pattern... 25% (A:0 C:0 P:5 N:0) files:2 [NOT_FOUND]
|
||||
[14/32] ops-rollback-procedure... 25% (A:0 C:0 P:5 N:0) files:2 [NOT_FOUND]
|
||||
[15/32] ops-branch-cluster-mapping... 25% (A:0 C:0 P:5 N:0) files:2 [NOT_FOUND]
|
||||
[16/32] ops-jenkins-jobs...
|
||||
Reference in New Issue
Block a user