Add the image-to-design workflow to skills/penpot/SKILL.md #212

New issue

Closed

opened 2026-09-14 23:14:25 +00:00 by david · 1 comment

david commented

2026-09-14 23:14:25 +00:00

Owner

Summary

Create skills/penpot/SKILL.md with valid pi skill frontmatter and the image → design creation path: read a screenshot or mockup with pi's read tool, reconstruct its structure semantically, and emit Penpot primitives — with an optional rsvg-convert visual self-critique step. Vision-only: no tracing dependency, and approximate fidelity is stated as the accepted outcome.

Background

Depends on: #201

Image → design is deliberately not a tracing pipeline. PNG→SVG tracing was rejected as the primary mechanism because it produces anonymous path soup with no layer names, no text objects and no asset references — which contradicts the whole point of the feature ("editable, design-system-linked output"). Instead the agent looks at the image, infers structure (what is a card, what is a heading, what is a button), and composes with the same primitives a written spec would use.

The skill is the right home for this: the extension cannot know that a rounded rectangle with a label is a button; that is judgement. What the extension guarantees is that whatever the agent decides is emitted as valid, library-linked shapes.

pi reads images as attachments through the read tool, so the workflow is: read the screenshot, describe the structure to yourself, map each region to primitives, commit, then look at the result and iterate.

This step creates the skill with its frontmatter and this one path; the remaining skill content (token conventions, composition rules, the Tailwind translation table, the other two paths, the vendored-docs pointer) is written in the following milestone. Do not leave placeholder sections for them.

Documentation Required

A separate process downloads these into the listed folders before this issue is implemented. Check the folders for the actual reference material before starting.

docs/reference/pi-coding-agent/

https://pi.dev/docs/latest/skills — skill authoring: the SKILL.md frontmatter (name, description), where skills live, how they are discovered, and what makes a description trigger correctly. The new file must pass this validation.
https://pi.dev/docs/latest/extensions — the tool family this skill drives (tool names, parameters and result shapes), so the workflow references real capabilities.
https://pi.dev/docs/latest/ — the read tool's image handling, confirming that an image path is sent to the model as an attachment.

docs/reference/penpot-api/

https://help.penpot.app/user-guide/design-systems/layout/ — how a designer composes with layout frames, so the reconstruction guidance matches Penpot's model.
https://help.penpot.app/user-guide/design-systems/assets/ — linked swatches and text styles, so the guidance pushes reconstructed regions onto library assets rather than literal colours.
https://help.penpot.app/user-guide/boards/ — boards/frames, for the "one region = one frame" advice.

docs/reference/tailwindcss/

https://tailwindcss.com/docs/theme — the @theme token block, since screenshots of a coded UI are usually screenshots of a Tailwind design system; the reconstruction should reuse those names.
https://tailwindcss.com/docs/adding-custom-styles — custom utilities/idioms that appear in the target UI and need translating.

docs/reference/librsvg/

https://github.com/GNOME/librsvg/blob/main/rsvg-convert.rst — the rsvg-convert(1) manual for the optional self-critique step (render agent-authored SVG to PNG so the agent can look at its own output).

Implementation Details

Create skills/penpot/SKILL.md with:

Frontmatter matching the repo's other skills:

---
name: penpot
description: <one sentence on what the skill does> — plus explicit "use when" triggers: creating designs in Penpot from a DESIGN.md, a prompt, an SVG or a screenshot; bootstrapping a design system's colours/typography/components; composing screens with the penpot_* extension tools. Mention the extension dependency by name.
---

The description must be specific enough to trigger on design-creation requests and must name the penpot extension, mirroring skills/victorialogs-debugging/SKILL.md's style.

A short intro stating what the skill does, that it requires the penpot extension plus PENPOT_URL/PENPOT_TOKEN, and that the extension owns payload correctness while the skill owns judgement.
The image → design path section, covering:
- Read the image with pi's read (path to the screenshot) and describe the structure before writing anything: shell/chrome, regions, repeated components, text levels, spacing rhythm.
- Reconstruct semantically, not visually. Identify named regions (header, nav, card list, footer) and map each to a frame; map repeated elements to a library component instance where one exists, and to composed primitives where it does not. Explicitly forbid per-pixel tracing and "draw what you see" reasoning.
- Assign text levels from the image's visual hierarchy to the design system's typography assets, not to guessed point sizes.
- Map colours to the library by role (surface, primary, muted, accent), not by eyedropper hex, and let the agent say when the image's colour is not in the library.
- Spacing guidance: infer a scale (4/8/12/16/24/32) and use layout gaps/padding rather than dumping absolute coordinates wherever a layout frame can express the intent.
- Work in passes: shell → regions → contents → text/styles, committing per screen (one commit = one revision) rather than one mega-commit.
- State fidelity honestly: this is an approximate reconstruction, not a pixel-accurate import, and the agent must say which areas it did not reconstruct faithfully.
- Reference-layer option: dropping the source screenshot as a locked layer for comparison is optional (not committed behaviour) — if used, it must be clearly named and locked.
The optional self-critique step: for agent-authored SVG (not for Penpot output), render to PNG with rsvg-convert and read it back to compare against intent. State plainly that pi reads SVG as text, so the PNG render is the only way to see an SVG.
A "what this path is not" note: no tracing (potrace/vtracer/Inkscape), no HTML/CSS input, no animation.

Keep it operational: short imperative instructions with the concrete tool calls named (penpot_authorize_target, penpot_add_frame, penpot_add_text, penpot_instance_component, penpot_commit), not abstract advice.

Acceptance Criteria

skills/penpot/SKILL.md exists with valid frontmatter (name: penpot, a specific description) and is discovered by pi (/skills lists it).
The description names the extension and includes explicit "use when" triggers covering DESIGN.md, prompt, SVG and screenshot inputs.
The image → design section describes a read → describe → map-to-primitives → commit → iterate loop using the real tool names.
The section explicitly rejects per-pixel tracing and explains why (editable output, layer names, real text, asset links).
Guidance requires mapping colours and text levels onto library assets by role, and requires stating when an image's colour/type is absent from the library.
Guidance requires committing per screen and stating fidelity gaps honestly.
The optional rsvg-convert self-critique step is documented as for agent-authored SVG only, with the reason (pi reads SVG as text).
No placeholder sections for content owned by the following milestone.

Test Plan

Load the skill in pi:

# in pi
/skills        # expect `penpot` listed with its description
/reload        # no skill-validation errors

Frontmatter sanity check:
```
head -10 skills/penpot/SKILL.md
```
End-to-end rehearsal of the documented path on a real screenshot is the next issue's job; here, confirm the instructions name only tools that actually exist in the extension.

## Summary Create `skills/penpot/SKILL.md` with valid pi skill frontmatter and the **image → design** creation path: read a screenshot or mockup with pi's `read` tool, reconstruct its structure semantically, and emit Penpot primitives — with an optional `rsvg-convert` visual self-critique step. Vision-only: no tracing dependency, and approximate fidelity is stated as the accepted outcome. ## Background **Depends on:** #201 Image → design is deliberately **not** a tracing pipeline. PNG→SVG tracing was rejected as the primary mechanism because it produces anonymous path soup with no layer names, no text objects and no asset references — which contradicts the whole point of the feature ("editable, design-system-linked output"). Instead the agent looks at the image, infers structure (what is a card, what is a heading, what is a button), and composes with the same primitives a written spec would use. The skill is the right home for this: the extension cannot know that a rounded rectangle with a label is a button; that is judgement. What the extension guarantees is that whatever the agent decides is emitted as valid, library-linked shapes. pi reads images as attachments through the `read` tool, so the workflow is: `read` the screenshot, describe the structure to yourself, map each region to primitives, commit, then look at the result and iterate. This step creates the skill with its frontmatter and this one path; the remaining skill content (token conventions, composition rules, the Tailwind translation table, the other two paths, the vendored-docs pointer) is written in the following milestone. Do not leave placeholder sections for them. ## Documentation Required A separate process downloads these into the listed folders before this issue is implemented. Check the folders for the actual reference material before starting. **`docs/reference/pi-coding-agent/`** - https://pi.dev/docs/latest/skills — skill authoring: the `SKILL.md` frontmatter (`name`, `description`), where skills live, how they are discovered, and what makes a description trigger correctly. The new file must pass this validation. - https://pi.dev/docs/latest/extensions — the tool family this skill drives (tool names, parameters and result shapes), so the workflow references real capabilities. - https://pi.dev/docs/latest/ — the `read` tool's image handling, confirming that an image path is sent to the model as an attachment. **`docs/reference/penpot-api/`** - https://help.penpot.app/user-guide/design-systems/layout/ — how a designer composes with layout frames, so the reconstruction guidance matches Penpot's model. - https://help.penpot.app/user-guide/design-systems/assets/ — linked swatches and text styles, so the guidance pushes reconstructed regions onto library assets rather than literal colours. - https://help.penpot.app/user-guide/boards/ — boards/frames, for the "one region = one frame" advice. **`docs/reference/tailwindcss/`** - https://tailwindcss.com/docs/theme — the `@theme` token block, since screenshots of a coded UI are usually screenshots of a Tailwind design system; the reconstruction should reuse those names. - https://tailwindcss.com/docs/adding-custom-styles — custom utilities/idioms that appear in the target UI and need translating. **`docs/reference/librsvg/`** - https://github.com/GNOME/librsvg/blob/main/rsvg-convert.rst — the `rsvg-convert(1)` manual for the optional self-critique step (render agent-authored SVG to PNG so the agent can look at its own output). ## Implementation Details Create `skills/penpot/SKILL.md` with: 1. **Frontmatter** matching the repo's other skills: ```yaml --- name: penpot description: <one sentence on what the skill does> — plus explicit "use when" triggers: creating designs in Penpot from a DESIGN.md, a prompt, an SVG or a screenshot; bootstrapping a design system's colours/typography/components; composing screens with the penpot_* extension tools. Mention the extension dependency by name. --- ``` The description must be specific enough to trigger on design-creation requests and must name the `penpot` extension, mirroring `skills/victorialogs-debugging/SKILL.md`'s style. 2. **A short intro** stating what the skill does, that it requires the `penpot` extension plus `PENPOT_URL`/`PENPOT_TOKEN`, and that the extension owns payload correctness while the skill owns judgement. 3. **The image → design path** section, covering: - **Read the image** with pi's `read` (path to the screenshot) and describe the structure before writing anything: shell/chrome, regions, repeated components, text levels, spacing rhythm. - **Reconstruct semantically, not visually.** Identify named regions (header, nav, card list, footer) and map each to a frame; map repeated elements to a **library component instance** where one exists, and to composed primitives where it does not. Explicitly forbid per-pixel tracing and "draw what you see" reasoning. - **Assign text levels** from the image's visual hierarchy to the design system's typography assets, not to guessed point sizes. - **Map colours** to the library by role (surface, primary, muted, accent), not by eyedropper hex, and let the agent say when the image's colour is not in the library. - **Spacing** guidance: infer a scale (4/8/12/16/24/32) and use layout gaps/padding rather than dumping absolute coordinates wherever a layout frame can express the intent. - **Work in passes**: shell → regions → contents → text/styles, committing per screen (one commit = one revision) rather than one mega-commit. - **State fidelity honestly**: this is an approximate reconstruction, not a pixel-accurate import, and the agent must say which areas it did not reconstruct faithfully. - **Reference-layer option**: dropping the source screenshot as a locked layer for comparison is optional (not committed behaviour) — if used, it must be clearly named and locked. 4. **The optional self-critique step**: for agent-authored SVG (not for Penpot output), render to PNG with `rsvg-convert` and `read` it back to compare against intent. State plainly that pi reads SVG as text, so the PNG render is the only way to *see* an SVG. 5. **A "what this path is not" note**: no tracing (potrace/vtracer/Inkscape), no HTML/CSS input, no animation. Keep it operational: short imperative instructions with the concrete tool calls named (`penpot_authorize_target`, `penpot_add_frame`, `penpot_add_text`, `penpot_instance_component`, `penpot_commit`), not abstract advice. ## Acceptance Criteria - [ ] `skills/penpot/SKILL.md` exists with valid frontmatter (`name: penpot`, a specific `description`) and is discovered by pi (`/skills` lists it). - [ ] The description names the extension and includes explicit "use when" triggers covering DESIGN.md, prompt, SVG and screenshot inputs. - [ ] The image → design section describes a read → describe → map-to-primitives → commit → iterate loop using the real tool names. - [ ] The section explicitly rejects per-pixel tracing and explains why (editable output, layer names, real text, asset links). - [ ] Guidance requires mapping colours and text levels onto **library assets** by role, and requires stating when an image's colour/type is absent from the library. - [ ] Guidance requires committing per screen and stating fidelity gaps honestly. - [ ] The optional `rsvg-convert` self-critique step is documented as for agent-authored SVG only, with the reason (pi reads SVG as text). - [ ] No placeholder sections for content owned by the following milestone. ## Test Plan 1. Load the skill in pi: ```bash # in pi /skills # expect `penpot` listed with its description /reload # no skill-validation errors ``` 2. Frontmatter sanity check: ```bash head -10 skills/penpot/SKILL.md ``` 3. End-to-end rehearsal of the documented path on a real screenshot is the next issue's job; here, confirm the instructions name only tools that actually exist in the extension.

david added this to the Penpot: M7 - Image to design workflow milestone

2026-09-14 23:14:25 +00:00

david added the

feature

label

2026-09-14 23:14:25 +00:00

david referenced this issue

2026-09-14 23:14:37 +00:00

Validate screenshot reconstruction against the live instance and record fidelity honestly #213

david referenced this issue from a pull request that will close it,

2026-09-15 07:15:57 +00:00

issue-212: Add the image-to-design workflow to skills/penpot/SKILL.md #245