M8 — Orchestration, error handling, validation & eval #18

david · 2026-07-25T03:38:10Z

david commented

2026-07-25 03:38:10 +00:00

Summary

M8 wires the previously-built M1–M7 stages into one autonomous run and enforces the error-handling policy end to end:

Jira fetch → implement → review → remediate → MR → write-back

Closes #9.

Changes

src/orchestrator/ (new module):
- runPipeline — sequential orchestrator; every stage/effect is an injectable seam so the whole flow is unit-testable without git, an LLM, or the network.
- classifyError — maps thrown errors + the executing stage onto an error.json report (uses the retry helper's annotated attempt count).
- buildFinalSummary — MR URL, remaining findings, degraded stages, and the Jira write-back outcome.
- ErrorReport / stage / kind types.
error.json artifact — validateErrorReport + writeErrorReport/readErrorReport, mirroring the existing artifact pattern.
runCli is now async — runs the pipeline after setup, returns exit 0 (success) / 1 (pipeline failure) / 2 (usage), and emits the final summary to stdout + run.log.
Gated full-pipeline E2E (runPipeline.e2e.test.ts, PILOOP_E2E=1) — asserts on artifacts/exit conditions (never exact LLM text); skips when PILOOP_E2E_JIRA_KEY / PILOOP_E2E_REPO are unset.
Docs — README exit codes, error-handling policy, error.json schema, E2E env vars; M8 checkboxes ticked in IMPLEMENTATION_PLAN.md.

Error-handling policy (enforced)

Fail-fast on invalid setup/config and Jira 401/403/404 (no agent session starts).
Bounded retries on transient calls (handled inside the Jira client / retry helper).
Per-stage degrade → aggregated degradedStages.
Abort + persist artifacts + error.json (kind: agent-failure) on hard agent failure.
Non-fatal write-back after MR creation (surfaced in the summary).
session.dispose() in finally is owned by runStage (per the M2 contract).

Testing

All tests pass — 60 files, 416 tests (baseline 374 → +42)
npm run build (tsc --noEmit) clean
npm run lint clean
Gated E2E passes (PILOOP_E2E=1): 1 pass, 1 skipped (no sandbox)
Light eval checkpoint against 2–3 live tickets — operator-run (needs live creds; gated E2E is the harness)

Checklist

Self-reviewed the diff
Code follows project conventions (injectable seams, co-located tests, Result-style outcomes)
Descriptive naming
Documentation updated (README + IMPLEMENTATION_PLAN)

Note: 5 pre-existing Prettier warnings in unrelated committed files were left untouched.

## Summary M8 wires the previously-built M1–M7 stages into one autonomous run and enforces the error-handling policy end to end: ``` Jira fetch → implement → review → remediate → MR → write-back ``` Closes #9. ## Changes - **`src/orchestrator/`** (new module): - `runPipeline` — sequential orchestrator; every stage/effect is an injectable seam so the whole flow is unit-testable without git, an LLM, or the network. - `classifyError` — maps thrown errors + the executing stage onto an `error.json` report (uses the retry helper's annotated attempt count). - `buildFinalSummary` — MR URL, remaining findings, degraded stages, and the Jira write-back outcome. - `ErrorReport` / stage / kind types. - **`error.json` artifact** — `validateErrorReport` + `writeErrorReport`/`readErrorReport`, mirroring the existing artifact pattern. - **`runCli` is now async** — runs the pipeline after setup, returns exit `0` (success) / `1` (pipeline failure) / `2` (usage), and emits the final summary to stdout + `run.log`. - **Gated full-pipeline E2E** (`runPipeline.e2e.test.ts`, `PILOOP_E2E=1`) — asserts on artifacts/exit conditions (never exact LLM text); skips when `PILOOP_E2E_JIRA_KEY` / `PILOOP_E2E_REPO` are unset. - **Docs** — README exit codes, error-handling policy, `error.json` schema, E2E env vars; M8 checkboxes ticked in `IMPLEMENTATION_PLAN.md`. ## Error-handling policy (enforced) - Fail-fast on invalid setup/config and Jira `401/403/404` (no agent session starts). - Bounded retries on transient calls (handled inside the Jira client / retry helper). - Per-stage degrade → aggregated `degradedStages`. - Abort + persist artifacts + `error.json` (`kind: agent-failure`) on hard agent failure. - Non-fatal write-back after MR creation (surfaced in the summary). - `session.dispose()` in `finally` is owned by `runStage` (per the M2 contract). ## Testing - [x] All tests pass — 60 files, **416 tests** (baseline 374 → +42) - [x] `npm run build` (tsc --noEmit) clean - [x] `npm run lint` clean - [x] Gated E2E passes (`PILOOP_E2E=1`): 1 pass, 1 skipped (no sandbox) - [ ] Light eval checkpoint against 2–3 live tickets — operator-run (needs live creds; gated E2E is the harness) ## Checklist - [x] Self-reviewed the diff - [x] Code follows project conventions (injectable seams, co-located tests, Result-style outcomes) - [x] Descriptive naming - [x] Documentation updated (README + IMPLEMENTATION_PLAN) Note: 5 pre-existing Prettier warnings in unrelated committed files were left untouched.

david added 1 commit

2026-07-25 03:38:11 +00:00

M8 — Orchestration, error handling, validation & eval 6b6662f42b

Wire M1–M7 stages into one autonomous run:
Jira fetch → implement → review → remediate → MR → write-back.

- Add src/orchestrator: runPipeline (all effects injectable), classifyError,
  buildFinalSummary, and ErrorReport types.
- Enforce the error-handling policy: fail-fast setup/fetch, per-stage degrade
  aggregation, abort+persist on agent failure, non-fatal write-back.
- Add error.json artifact (ErrorReport type + validator + read/write IO).
- Make runCli async; run the pipeline after setup; exit 0/1/2; emit the final
  summary to stdout + run.log.
- Add a gated full-pipeline E2E (PILOOP_E2E=1) asserting on artifacts/exit
  conditions; skips without a provisioned sandbox.
- Document exit codes, error-handling policy, error.json schema, and E2E env
  in README; tick M8 boxes in IMPLEMENTATION_PLAN.

Full suite: 60 files, 416 tests pass. Build + lint clean.

david added 1 commit

2026-07-25 03:57:38 +00:00

M8 review fixes: import ordering, logger dedup, summary sink, explicit Jira-creds invariant 712fee8e50

- orchestrator/types: move import type block above ErrorStage/ErrorKind exports (Task 001)
- runCli: build base logger once; derive setup logger; route final summary through shared sinks instead of raw appendFileSync (Tasks 002, 003)
- jira: extract requireJiraCreds() as single fail-fast source of truth for creds; use it in fetchJiraContext and defaultWriteClient instead of implicit ?? '' fallbacks (Task 004)
- add requireJiraCreds unit tests

david merged commit 0df99d8304 into main

2026-07-25 04:06:30 +00:00

david referenced this pull request from a commit

2026-07-25 04:06:30 +00:00

M8 — Orchestration, error handling, validation & eval (#18)

david deleted branch issue-9/stage-m8-orchestration-and-error-handling

2026-07-25 04:06:30 +00:00

Sign in to join this conversation.

No reviewers