Validate screenshot reconstruction against the live instance and record fidelity honestly #213

New issue

Closed

opened 2026-09-14 23:14:37 +00:00 by david · 1 comment

david commented

2026-09-14 23:14:37 +00:00

Owner

Summary

Rehearse the documented image → design path end to end on a real screenshot and record the result honestly: which regions reconstructed well, which did not, whether the output is genuinely editable and library-linked, and whether the workflow instructions were sufficient to follow without improvisation.

Background

Depends on: #212

This is the only creation path whose quality depends on the agent's judgement rather than on payload correctness, so it cannot be covered by unit tests — it has to be exercised and reported on. Two questions are being answered:

Does the path produce usable output? An editable, library-linked reconstruction that an agent can iterate on — not a picture, not path soup.
Are the skill's instructions adequate? If the agent had to invent steps the skill does not describe, that is a defect in the skill and must be fixed here, because the next agent will face the same gap.

Approximate fidelity is explicitly accepted. What is not acceptable is an unreported gap: the validation must state plainly which parts came out well and which did not, so the skill can warn a user in advance.

Use a screenshot of a screen whose real design system is known, so the reconstruction can be compared against both the image and the intended tokens. Prefer a screen from the reference project (Tailwind @theme palette with a DESIGN.md) over a synthetic mockup.

Documentation Required

A separate process downloads these into the listed folders before this issue is implemented. Check the folders for the actual reference material before starting.

docs/reference/pi-coding-agent/

https://pi.dev/docs/latest/skills — the skill authoring rules, needed when fixing gaps found in skills/penpot/SKILL.md.
https://pi.dev/docs/latest/extensions — the tool family and result shapes the workflow depends on (so a workflow step referencing a non-existent tool is caught here).

docs/reference/penpot-api/

https://help.penpot.app/user-guide/design-systems/layout/ — what the Layout panel must show for a reconstruction to count as structurally faithful (frames with real layout, not absolute scatter).
https://help.penpot.app/user-guide/design-systems/assets/ — what a linked swatch/text style looks like, for the "library-linked, not literal colours" check.
https://help.penpot.app/user-guide/design-systems/components/ — what a real instance looks like in the properties panel, for the "reused components are instances" check.

docs/reference/tailwindcss/

https://tailwindcss.com/docs/theme — the token names a reconstruction should reuse, and the scale (--spacing-*, --text-*) that maps onto layout gaps and typography assets.
https://tailwindcss.com/docs/adding-custom-styles — the idiom set behind the screenshots being reconstructed (rounded/shadow/gradient recipes).

Implementation Details

Pick the screenshot and, before reading it, note the design system it belongs to (the relevant DESIGN.md and @theme palette) plus the component inventory available from the imported library.
Follow the skill literally. Open skills/penpot/SKILL.md and execute the image → design section step by step, using only the tools it names. Keep a note of every moment where the instructions were ambiguous, missing a step, or wrong — that list is as important as the reconstruction.
Reconstruct the screen into a scratch file:
- read the image with pi's read;
- describe the structure first (regions, repeated components, text levels, spacing rhythm);
- reconstruct in passes (shell → regions → contents → text/styles), committing per pass;
- use penpot_instance_component where a library component matches a repeated element, and composed primitives elsewhere.
Assess objectively. In the Penpot UI:
- is the output a shape tree a human can edit (named layers, text objects, frames with layout), or a flattened picture?
- are colours linked to library assets rather than literal hex?
- are repeated elements instances of library components where expected?
- which regions match the screenshot well, and which are wrong (proportions, spacing, missing elements, invented elements)?
Record honestly in extensions/penpot/findings.md under "Image → design":
- the source screenshot and the design system it belongs to;
- what was reconstructed faithfully and what was not, region by region — no rounding up;
- the ratio of library-linked styling to literal styling, and of component instances to hand-built groups;
- every gap found in the skill's instructions and the fix applied;
- a short verdict on whether the path is usable for a first-pass reconstruction, with the caveats a user must be told.

Fix any defect found: skill wording, missing tool guidance, or extension behaviour discovered along the way. Do not merely note a gap in the skill — edit it.

Acceptance Criteria

A real screenshot is reconstructed into a scratch file by following skills/penpot/SKILL.md literally, using only the tools it names.
The output is an editable shape tree: named layers, real text objects, and frames (not a single flattened image or path soup).
Colours come from library assets wherever the design system has a matching token; any literal colours used are justified in the record.
Repeated elements that match library components are real component instances in the UI inspector.
The reconstruction commits in passes (one commit per pass), each advancing the file's revision exactly once.
extensions/penpot/findings.md records a region-by-region assessment — including what did not come out well — plus the library-linked/instance ratios and an overall verdict with caveats.
Every ambiguity or gap found in the skill is fixed in skills/penpot/SKILL.md in this step, not merely listed.
The scratch file's state leaves later milestones free to keep validating (do not delete the scratch project).

Test Plan

Manual, against the live instance:

# in pi
/reload
# read <screenshot path>
# penpot_authorize_target(project_id=<scratch project>)
# penpot_import_library(...) + penpot_link_library(...)   # if the library is not already available
# ... follow skills/penpot/SKILL.md's image -> design section ...
# penpot_commit(file_id=<scratch file>)   # once per pass

Then in the Penpot UI:

Inspect the Layers panel: are layers named and is the structure region-based?
Select a text layer and type into it — proving it is real text.
Check a colour swatch: linked (library) or local?
Check a repeated element: component instance or a plain group?
Compare the canvas against the screenshot and write down the differences, including elements that were missed or invented.

## Summary Rehearse the documented image → design path end to end on a real screenshot and record the result honestly: which regions reconstructed well, which did not, whether the output is genuinely editable and library-linked, and whether the workflow instructions were sufficient to follow without improvisation. ## Background **Depends on:** #212 This is the only creation path whose quality depends on the agent's judgement rather than on payload correctness, so it cannot be covered by unit tests — it has to be exercised and reported on. Two questions are being answered: 1. **Does the path produce usable output?** An editable, library-linked reconstruction that an agent can iterate on — not a picture, not path soup. 2. **Are the skill's instructions adequate?** If the agent had to invent steps the skill does not describe, that is a defect in the skill and must be fixed here, because the next agent will face the same gap. Approximate fidelity is explicitly accepted. What is not acceptable is an unreported gap: the validation must state plainly which parts came out well and which did not, so the skill can warn a user in advance. Use a screenshot of a screen whose real design system is known, so the reconstruction can be compared against both the image and the intended tokens. Prefer a screen from the reference project (Tailwind `@theme` palette with a `DESIGN.md`) over a synthetic mockup. ## Documentation Required A separate process downloads these into the listed folders before this issue is implemented. Check the folders for the actual reference material before starting. **`docs/reference/pi-coding-agent/`** - https://pi.dev/docs/latest/skills — the skill authoring rules, needed when fixing gaps found in `skills/penpot/SKILL.md`. - https://pi.dev/docs/latest/extensions — the tool family and result shapes the workflow depends on (so a workflow step referencing a non-existent tool is caught here). **`docs/reference/penpot-api/`** - https://help.penpot.app/user-guide/design-systems/layout/ — what the Layout panel must show for a reconstruction to count as structurally faithful (frames with real layout, not absolute scatter). - https://help.penpot.app/user-guide/design-systems/assets/ — what a linked swatch/text style looks like, for the "library-linked, not literal colours" check. - https://help.penpot.app/user-guide/design-systems/components/ — what a real instance looks like in the properties panel, for the "reused components are instances" check. **`docs/reference/tailwindcss/`** - https://tailwindcss.com/docs/theme — the token names a reconstruction should reuse, and the scale (`--spacing-*`, `--text-*`) that maps onto layout gaps and typography assets. - https://tailwindcss.com/docs/adding-custom-styles — the idiom set behind the screenshots being reconstructed (rounded/shadow/gradient recipes). ## Implementation Details 1. **Pick the screenshot** and, before reading it, note the design system it belongs to (the relevant `DESIGN.md` and `@theme` palette) plus the component inventory available from the imported library. 2. **Follow the skill literally.** Open `skills/penpot/SKILL.md` and execute the image → design section step by step, using only the tools it names. Keep a note of every moment where the instructions were ambiguous, missing a step, or wrong — that list is as important as the reconstruction. 3. **Reconstruct** the screen into a scratch file: - read the image with pi's `read`; - describe the structure first (regions, repeated components, text levels, spacing rhythm); - reconstruct in passes (shell → regions → contents → text/styles), committing per pass; - use `penpot_instance_component` where a library component matches a repeated element, and composed primitives elsewhere. 4. **Assess objectively.** In the Penpot UI: - is the output a shape tree a human can edit (named layers, text objects, frames with layout), or a flattened picture? - are colours **linked** to library assets rather than literal hex? - are repeated elements **instances** of library components where expected? - which regions match the screenshot well, and which are wrong (proportions, spacing, missing elements, invented elements)? 5. **Record honestly** in `extensions/penpot/findings.md` under "Image → design": - the source screenshot and the design system it belongs to; - what was reconstructed faithfully and what was not, region by region — no rounding up; - the ratio of library-linked styling to literal styling, and of component instances to hand-built groups; - every gap found in the skill's instructions and the fix applied; - a short verdict on whether the path is usable for a first-pass reconstruction, with the caveats a user must be told. Fix any defect found: skill wording, missing tool guidance, or extension behaviour discovered along the way. Do not merely note a gap in the skill — edit it. ## Acceptance Criteria - [ ] A real screenshot is reconstructed into a scratch file by following `skills/penpot/SKILL.md` literally, using only the tools it names. - [ ] The output is an editable shape tree: named layers, real text objects, and frames (not a single flattened image or path soup). - [ ] Colours come from **library assets** wherever the design system has a matching token; any literal colours used are justified in the record. - [ ] Repeated elements that match library components are real **component instances** in the UI inspector. - [ ] The reconstruction commits in passes (one commit per pass), each advancing the file's revision exactly once. - [ ] `extensions/penpot/findings.md` records a region-by-region assessment — including what did **not** come out well — plus the library-linked/instance ratios and an overall verdict with caveats. - [ ] Every ambiguity or gap found in the skill is fixed in `skills/penpot/SKILL.md` in this step, not merely listed. - [ ] The scratch file's state leaves later milestones free to keep validating (do not delete the scratch project). ## Test Plan Manual, against the live instance: ```bash # in pi /reload # read <screenshot path> # penpot_authorize_target(project_id=<scratch project>) # penpot_import_library(...) + penpot_link_library(...) # if the library is not already available # ... follow skills/penpot/SKILL.md's image -> design section ... # penpot_commit(file_id=<scratch file>) # once per pass ``` Then in the Penpot UI: 1. Inspect the Layers panel: are layers named and is the structure region-based? 2. Select a text layer and type into it — proving it is real text. 3. Check a colour swatch: linked (`library`) or local? 4. Check a repeated element: component instance or a plain group? 5. Compare the canvas against the screenshot and write down the differences, including elements that were missed or invented.

david added this to the Penpot: M7 - Image to design workflow milestone

2026-09-14 23:14:37 +00:00

david added the

feature

label

2026-09-14 23:14:37 +00:00

david referenced this issue

2026-09-14 23:15:15 +00:00

Write the penpot SKILL.md core workflow, token conventions and Tailwind translation table #215

david referenced this issue

2026-09-14 23:15:29 +00:00

Add the penpot skill and extension to the repo root README #216