59 lines
3.2 KiB
Plaintext
59 lines
3.2 KiB
Plaintext
|
🔁 Ralph Wiggum Loop (V2) — max 1 iterations, target 77%
|
||
|
|
Benchmark: Confluence Gold Standard (/home/node/.openclaw/workspace/projects/dev-intel-v2/eval-confluence-ref-questions.json)
|
||
|
|
|
||
|
|
=== Iteration 1/1 ===
|
||
|
|
📝 Running V2 pipeline...
|
||
|
|
State services: 9
|
||
|
|
Auto-detected 0 entry points
|
||
|
|
Flow traces: 0 valid out of 0 attempted
|
||
|
|
Impact analysis: 0 nodes with downstream dependents
|
||
|
|
Generating architecture overview...
|
||
|
|
Generating prose for subsystem: account-common...
|
||
|
|
Generating prose for subsystem: app-common...
|
||
|
|
Generating prose for subsystem: app-tools...
|
||
|
|
Generating prose for subsystem: compute-common...
|
||
|
|
Generating prose for subsystem: compute-tools...
|
||
|
|
Generating prose for subsystem: control-core...
|
||
|
|
Generating prose for subsystem: ipam-core...
|
||
|
|
Generating prose for subsystem: ipam-tools...
|
||
|
|
Generating prose for subsystem: network-common...
|
||
|
|
Generating prose for subsystem: network-core...
|
||
|
|
Generating prose for subsystem: runtime...
|
||
|
|
Generating prose for subsystem: root...
|
||
|
|
Generating prose for 124 contracts...
|
||
|
|
Agent KB: 12 subsystems, 76 charts
|
||
|
|
Synthesizing dynamic reference pages via LLM...
|
||
|
|
Dynamic reference pages and index synthesized.
|
||
|
|
Generated docs in ./foxtrot-docs
|
||
|
|
- 12 subsystems
|
||
|
|
- 124 contracts
|
||
|
|
- 0 flows
|
||
|
|
📊 Running agent file-browsing eval against Confluence questions...
|
||
|
|
Using model: claude-haiku-4.5
|
||
|
|
Agent Eval: 32 machine-audience questions
|
||
|
|
[1/32] arch-layered-order... 30% (A:1 C:2 P:1 N:2) files:5
|
||
|
|
[2/32] arch-hub-spoke-ownership... 85% (A:5 C:4 P:4 N:4) files:5
|
||
|
|
[3/32] arch-aws-regions... 50% (A:2 C:5 P:1 N:2) files:5
|
||
|
|
[4/32] arch-gcp-shared-vpc-host... 25% (A:0 C:0 P:5 N:0) files:5 [NOT_FOUND]
|
||
|
|
[5/32] arch-cidr-employee-access... 25% (A:0 C:0 P:5 N:0) files:5 [NOT_FOUND]
|
||
|
|
[6/32] arch-production-cidr... 25% (A:0 C:0 P:5 N:0) files:5 [NOT_FOUND]
|
||
|
|
[7/32] dep-runtime-common-horizontal... 65% (A:4 C:2 P:4 N:3) files:5
|
||
|
|
[8/32] dep-vertical-layers... 35% (A:1 C:2 P:2 N:2) files:5
|
||
|
|
[9/32] dep-create-account-repos... 25% (A:0 C:0 P:5 N:0) files:5 [NOT_FOUND]
|
||
|
|
[10/32] dep-create-cluster-repos... 25% (A:0 C:0 P:5 N:0) files:5 [NOT_FOUND]
|
||
|
|
[11/32] dep-compute-common-deps... 30% (A:2 C:1 P:2 N:1) files:5
|
||
|
|
[12/32] ops-argocd-deployment-flow... 25% (A:0 C:0 P:5 N:0) files:5 [NOT_FOUND]
|
||
|
|
[13/32] ops-ebf-release-pattern... 25% (A:0 C:0 P:5 N:0) files:5 [NOT_FOUND]
|
||
|
|
[14/32] ops-rollback-procedure... 25% (A:0 C:0 P:5 N:0) files:5 [NOT_FOUND]
|
||
|
|
[15/32] ops-branch-cluster-mapping... 25% (A:0 C:0 P:5 N:0) files:5 [NOT_FOUND]
|
||
|
|
[16/32] ops-jenkins-jobs... 25% (A:0 C:0 P:5 N:0) files:5 [NOT_FOUND]
|
||
|
|
[17/32] ops-create-cluster-timeout... 35% (A:2 C:2 P:1 N:2) files:5
|
||
|
|
[18/32] config-cloud-resource-naming... 55% (A:3 C:2 P:4 N:2) files:5
|
||
|
|
[19/32] config-region-code-algorithm... 25% (A:0 C:0 P:5 N:0) files:5 [NOT_FOUND]
|
||
|
|
[20/32] config-app-config-merge-order... 10% (A:0 C:0 P:1 N:1) files:5
|
||
|
|
[21/32] config-account-creation-product-id... 25% (A:0 C:0 P:5 N:0) files:5 [NOT_FOUND]
|
||
|
|
[22/32] config-ipam-rds-backup... 80% (A:5 C:5 P:4 N:2) files:5
|
||
|
|
[23/32] config-dev-artifact-naming... 25% (A:0 C:0 P:5 N:0) files:5 [NOT_FOUND]
|
||
|
|
[24/32] services-tech-stack-orchestration... 25% (A:0 C:0 P:5 N:0) files:5 [NOT_FOUND]
|
||
|
|
[25/32] services-state-management... 25% (A:0 C:0 P:5 N:0) files:5 [NOT_FOUND]
|
||
|
|
[26/32] services-eks-addon-versions...
|