Gated resume E2E for a fatal comment-post failure (PILOOP_E2E=1) #310

New issue

david · 2026-09-05T18:36:13Z

david commented

2026-09-05 18:36:13 +00:00

Summary

Prove end-to-end (gated, PILOOP_E2E=1) that a fatal comment-post failure resumes to exactly one posted comment: the first run fails with a write-back-attributed error and zero issue comments; the resumed run skips the implement agent and posts exactly once (ADR-021 §9).

Background

Depends on: #306, #308

The fault-injectable postNoCodeChangeComment seam (from the short-circuit wiring) makes the failure injectable; the resume mapping (valid no-code-change.json = implement-complete) makes the resumed run skip implement. The existing gated E2E infra lives under e2e/ (PILOOP_E2E=1 npm run e2e). Assert only artifacts and platform state — never exact LLM text.

Implementation Details

Add an E2E fixture to e2e/ (follow existing fixture conventions: sandbox repo + throwaway issue):
1. Run the pipeline against a sandbox issue whose implement stage produces no-code-change.json (seed the artifact or use a scripted implement outcome as existing fixtures do).
2. Inject a failure at the comment step (fault-inject the comment client seam) → assert the run fails with the write-back-attributed error and that the issue received zero comments.
3. Re-run (resume) → assert the implement agent is not re-invoked (implement skipped), the comment posts exactly once (marker written), the run completes ok, and the issue has exactly one comment.

Acceptance Criteria

The fixture's first phase fails with stage write-back attribution and zero comments on the issue.
The resume phase skips implement, posts exactly one comment, writes the marker artifact, and completes ok.
PILOOP_E2E=1 suite passes on a sandbox repo.
No gated test asserts on exact LLM text — only artifacts and platform state.

Test Plan

PILOOP_E2E=1 npm run e2e on a sandbox repo (matching what existing fixtures target). A real one-off no-code-change run to eyeball the posted comment text may be done here or in the final docs/validation step.

## Summary Prove end-to-end (gated, `PILOOP_E2E=1`) that a fatal comment-post failure resumes to exactly one posted comment: the first run fails with a write-back-attributed error and **zero** issue comments; the resumed run skips the implement agent and posts **exactly once** (ADR-021 §9). ## Background **Depends on:** #306, #308 The fault-injectable `postNoCodeChangeComment` seam (from the short-circuit wiring) makes the failure injectable; the resume mapping (valid `no-code-change.json` = implement-complete) makes the resumed run skip implement. The existing gated E2E infra lives under `e2e/` (`PILOOP_E2E=1 npm run e2e`). Assert only artifacts and platform state — never exact LLM text. ## Implementation Details - Add an E2E fixture to `e2e/` (follow existing fixture conventions: sandbox repo + throwaway issue): 1. Run the pipeline against a sandbox issue whose implement stage produces `no-code-change.json` (seed the artifact or use a scripted implement outcome as existing fixtures do). 2. Inject a failure at the comment step (fault-inject the comment client seam) → assert the run fails with the **write-back-attributed** error and that the issue received **zero** comments. 3. Re-run (resume) → assert the implement agent is **not** re-invoked (implement skipped), the comment posts exactly once (marker written), the run completes `ok`, and the issue has exactly **one** comment. ## Acceptance Criteria - [ ] The fixture's first phase fails with stage `write-back` attribution and zero comments on the issue. - [ ] The resume phase skips implement, posts exactly one comment, writes the marker artifact, and completes `ok`. - [ ] `PILOOP_E2E=1` suite passes on a sandbox repo. - [ ] No gated test asserts on exact LLM text — only artifacts and platform state. ## Test Plan - `PILOOP_E2E=1 npm run e2e` on a sandbox repo (matching what existing fixtures target). A real one-off no-code-change run to eyeball the posted comment text may be done here or in the final docs/validation step.

david added this to the No-code-change comment: verification & docs milestone

2026-09-05 18:36:13 +00:00

david added the

chore

label

2026-09-05 18:36:13 +00:00

david referenced this issue

2026-09-05 18:36:22 +00:00

Doc updates and validation for the no-code-change comment feature #311

Sign in to join this conversation.