feat: repo-agnostic refactor (BMad spec-test-build loop)
- NEW: repo-profiler.js — deterministic archetype detection (Infra, Frontend, Backend, etc.) - NEW: extract-dynamic.js — generic extractor replacing hardcoded Foxtrot patterns - NEW: eval-generator.js — dynamic ground-truth question generation from any repo graph - NEW: specs/bmad-agnostic-refactor-spec.md — full BMad spec with acceptance criteria - REFACTORED: prose.js — two-pass LLM synthesis with rich context (shared secrets, ports, service refs) - REFACTORED: sysdoc.js — wired repo-profiler + extract-dynamic, --legacy escape hatch - REFACTORED: wiggum-v2.sh — uses eval-generator before benchmarks - FIXED: graph.js — _edgeSet rebuilt on loadSnapshot() (edge dedup was broken) - FIXED: graph.js — recursive sortKeys() for deep equality in diffing - FIXED: prose.js — robust JSON array extraction from LLM output - FIXED: ratchet.js — syntax validation (node --check) before saving LLM mutations - FIXED: extract-dynamic.js — centralized state services regex, added console.warn for silent failures - TESTS: test-eval-generator, test-repo-profiler, test-synthesis-quality + mock fixtures Eval: 81.5% on Foxtrot (fully repo-agnostic, no hardcoded reference pages) BMad reviews: Architect B+, Dev Lead B-, TEA B-
This commit is contained in:
30
specs/agnostic-synthesis-plan.md
Normal file
30
specs/agnostic-synthesis-plan.md
Normal file
@@ -0,0 +1,30 @@
|
||||
# Implementation Plan: Repo-Agnostic Synthesis
|
||||
|
||||
## 1. Data Collection & Formatting
|
||||
We already have the data in `sysdoc.js`:
|
||||
- `deepData`: Output of `extract-deep.js` (addons, tfConfigs, scriptParams, helmValues, stateServices).
|
||||
- `helmGraph`: The Helm dependencies and charts.
|
||||
- `patterns`: System patterns, layers, sync waves.
|
||||
- `subs`: The extracted subsystems.
|
||||
- *Action*: In `sysdoc.js`, format this raw data into a large stringified JSON or Markdown list to serve as context for the synthesis LLM.
|
||||
|
||||
## 2. LLM Synthesis Module (`prose.js`)
|
||||
Create a new exported function: `synthesizeReferencePages(extractedContext, outDir, llmOpts)`.
|
||||
This function will make parallel or sequential LLM calls to generate specific reference topics based on the extracted context.
|
||||
|
||||
**Prompts:**
|
||||
* `network-architecture.md`: Focus on CIDR allocations, VPCs, network routing, NAT, bastions found in the `tfConfigs` and `helmValues`.
|
||||
* `operations.md`: Focus on CI/CD pipelines, Jenkins jobs, branch mappings, timeout parameters, and deployment flows found in `scriptParams` and repo patterns.
|
||||
* `configuration.md`: Focus on config merge orders, region code logic, identifiers, naming conventions, and default values found in `helmValues` and `tfConfigs`.
|
||||
* `dependencies.md`: Focus on vertical layer dependencies, Helm chart consumption (e.g., runtime consuming common), and PR cross-repo dependencies.
|
||||
* `index.md`: An LLM call that takes summaries of the 4 generated pages and produces a keyword-rich routing table.
|
||||
|
||||
## 3. Pipeline Update (`sysdoc.js`)
|
||||
At the end of the `generateDocs` function, just before writing the final files or after writing the basic ones, check if `opts.prose` is true. If so, call `await proseMod.synthesizeReferencePages(extractedContext, referenceDir, llmOpts)`.
|
||||
|
||||
## 4. Cleanup
|
||||
- `rm generate-reference-pages.js`
|
||||
- Edit `wiggum-v2.sh` to remove the call to `generate-reference-pages.js`.
|
||||
|
||||
## 5. Execution
|
||||
Run `wiggum-v2.sh` to generate the docs dynamically, then trigger the agent evaluation. The score should remain high without us cheating.
|
||||
Reference in New Issue
Block a user