Validate read primitives against the live Penpot instance #193

New issue

Closed

opened 2026-09-14 23:10:05 +00:00 by david · 1 comment

david commented

2026-09-14 23:10:05 +00:00

Owner

Summary

Validate the read primitives (penpot_whoami, penpot_list_projects, penpot_get_file, penpot_list_library) against the live Penpot instance, confirm their output matches the Penpot UI, and record the findings — including any drift between the real payloads and what the earlier steps assumed.

Background

Depends on: #192

The read side is where the payload schema is pinned. Documented and design-doc descriptions of Penpot's file shape can drift from the version actually running (this repo targets Penpot 2.17), and a wrong read schema silently corrupts every later write. Milestones are worked through in order and none advances on an unvalidated predecessor, so this step is a gate rather than a formality.

Required environment: PENPOT_URL and PENPOT_TOKEN for a live instance plus a scratch project (the one used during design validation is named pi-penpot-scratch -- safe to delete). Never run this against a project holding real designs; expect to create and delete files in the scratch project.

Documentation Required

A separate process downloads these into the listed folders before this issue is implemented. Check the folders for the actual reference material before starting.

docs/reference/penpot-api/

<PENPOT_URL>/api/main/doc/openapi.json — the instance's own spec, to confirm the command set and parameters actually served (this is the ground truth for the running version).
https://help.penpot.app/user-guide/design-systems/libraries/ — to know what the Assets panel is expected to show, so tool output can be compared against it.
https://help.penpot.app/user-guide/ — UI reference for locating a project's file list and a file's pages, for the comparison steps below.

docs/reference/pi-coding-agent/

https://pi.dev/docs/latest/extensions — /reload behaviour for extensions under development, and how to tell a tool registered successfully from one that failed to load.

Implementation Details

Exercise each tool against the live instance and compare against the UI. Where a tool's output disagrees with the instance, fix the tool in this step (a small, focused change) and re-run.

Auth. Call penpot_whoami; confirm the name/email belong to the token owner. Then repeat with a deliberately invalid token and confirm the error is readable.
Teams and projects. Call penpot_list_projects; confirm the team(s), their enabled features, the caller's permissions and the project names/ids match the Penpot UI sidebar. Then call it with a single team_id and confirm the narrower listing.
File read. Call penpot_get_file on a scratch file; confirm page names and ids match the UI's page list, that revn/vern are present, and that the reported object count for the default page (which should contain the Root Frame at 00000000-0000-0000-0000-000000000000) is plausible.
Revision tracking. Note the reported revn, make a trivial change in the UI (e.g. rename the page), then re-read and confirm revn advanced. This pins the semantics the commit step will rely on.
Library read. Create two colours and one typography by hand in the UI, then call penpot_list_library and confirm names and ids match the Assets panel exactly. Add a second colour with a duplicate name and confirm the ambiguity is reported with both ids.

Record the outcome in extensions/penpot/findings.md (create the file if it does not exist; the repo's extensions/forgejo/findings.md shows the house style):

the exact command/tool call used for each check,
the observed result,
every place the real payload differed from what the implementation or the design document assumed, and what was changed in response,
anything that could not be verified and is therefore still an open risk for the write-side milestones.

Do not delete the scratch project in this step — later milestones reuse it.

Acceptance Criteria

penpot_whoami, penpot_list_projects, penpot_get_file and penpot_list_library all return correct results against the live instance.
Each tool's output has been compared against the corresponding Penpot UI view and matches (teams/projects, page list, Assets panel).
penpot_get_file reports a revn that advances after a UI edit to the same file.
An invalid token and a bogus file id each produce readable errors (no raw Transit, no stack traces).
A duplicate asset name is reported as ambiguous with both ids.
extensions/penpot/findings.md records every check, its observed result, the version validated (2.17), and every schema difference found and corrected.
Any schema drift found has been fixed in the tool code (not merely noted).
The scratch project still exists and holds the two colours, one typography and the duplicate-name colour used for the checks.

Test Plan

Manual, against the live instance — record the output of each step in extensions/penpot/findings.md:

# in pi
/reload
# then call: penpot_whoami -> penpot_list_projects -> penpot_get_file -> penpot_list_library

Compare penpot_list_projects output against the Penpot UI sidebar for the same account.
Open the scratch file in the UI and compare the page list and Assets panel against penpot_get_file / penpot_list_library output.
Rename a page in the UI, re-run penpot_get_file, and confirm revn advanced.
With PENPOT_TOKEN temporarily broken, confirm every tool returns an actionable error rather than an exception.

## Summary Validate the read primitives (`penpot_whoami`, `penpot_list_projects`, `penpot_get_file`, `penpot_list_library`) against the live Penpot instance, confirm their output matches the Penpot UI, and record the findings — including any drift between the real payloads and what the earlier steps assumed. ## Background **Depends on:** #192 The read side is where the payload schema is pinned. Documented and design-doc descriptions of Penpot's file shape can drift from the version actually running (this repo targets **Penpot 2.17**), and a wrong read schema silently corrupts every later write. Milestones are worked through in order and none advances on an unvalidated predecessor, so this step is a gate rather than a formality. Required environment: `PENPOT_URL` and `PENPOT_TOKEN` for a live instance plus a **scratch project** (the one used during design validation is named `pi-penpot-scratch -- safe to delete`). Never run this against a project holding real designs; expect to create and delete files in the scratch project. ## Documentation Required A separate process downloads these into the listed folders before this issue is implemented. Check the folders for the actual reference material before starting. **`docs/reference/penpot-api/`** - `<PENPOT_URL>/api/main/doc/openapi.json` — the instance's own spec, to confirm the command set and parameters actually served (this is the ground truth for the running version). - https://help.penpot.app/user-guide/design-systems/libraries/ — to know what the Assets panel is expected to show, so tool output can be compared against it. - https://help.penpot.app/user-guide/ — UI reference for locating a project's file list and a file's pages, for the comparison steps below. **`docs/reference/pi-coding-agent/`** - https://pi.dev/docs/latest/extensions — `/reload` behaviour for extensions under development, and how to tell a tool registered successfully from one that failed to load. ## Implementation Details Exercise each tool against the live instance and compare against the UI. Where a tool's output disagrees with the instance, fix the tool in this step (a small, focused change) and re-run. 1. **Auth.** Call `penpot_whoami`; confirm the name/email belong to the token owner. Then repeat with a deliberately invalid token and confirm the error is readable. 2. **Teams and projects.** Call `penpot_list_projects`; confirm the team(s), their enabled features, the caller's permissions and the project names/ids match the Penpot UI sidebar. Then call it with a single `team_id` and confirm the narrower listing. 3. **File read.** Call `penpot_get_file` on a scratch file; confirm page names and ids match the UI's page list, that `revn`/`vern` are present, and that the reported object count for the default page (which should contain the `Root Frame` at `00000000-0000-0000-0000-000000000000`) is plausible. 4. **Revision tracking.** Note the reported `revn`, make a trivial change in the UI (e.g. rename the page), then re-read and confirm `revn` advanced. This pins the semantics the commit step will rely on. 5. **Library read.** Create two colours and one typography by hand in the UI, then call `penpot_list_library` and confirm names and ids match the Assets panel exactly. Add a second colour with a duplicate name and confirm the ambiguity is reported with both ids. Record the outcome in `extensions/penpot/findings.md` (create the file if it does not exist; the repo's `extensions/forgejo/findings.md` shows the house style): - the exact command/tool call used for each check, - the observed result, - **every place the real payload differed from what the implementation or the design document assumed**, and what was changed in response, - anything that could not be verified and is therefore still an open risk for the write-side milestones. Do not delete the scratch project in this step — later milestones reuse it. ## Acceptance Criteria - [ ] `penpot_whoami`, `penpot_list_projects`, `penpot_get_file` and `penpot_list_library` all return correct results against the live instance. - [ ] Each tool's output has been compared against the corresponding Penpot UI view and matches (teams/projects, page list, Assets panel). - [ ] `penpot_get_file` reports a `revn` that advances after a UI edit to the same file. - [ ] An invalid token and a bogus file id each produce readable errors (no raw Transit, no stack traces). - [ ] A duplicate asset name is reported as ambiguous with both ids. - [ ] `extensions/penpot/findings.md` records every check, its observed result, the version validated (`2.17`), and every schema difference found and corrected. - [ ] Any schema drift found has been fixed in the tool code (not merely noted). - [ ] The scratch project still exists and holds the two colours, one typography and the duplicate-name colour used for the checks. ## Test Plan Manual, against the live instance — record the output of each step in `extensions/penpot/findings.md`: ```bash # in pi /reload # then call: penpot_whoami -> penpot_list_projects -> penpot_get_file -> penpot_list_library ``` 1. Compare `penpot_list_projects` output against the Penpot UI sidebar for the same account. 2. Open the scratch file in the UI and compare the page list and Assets panel against `penpot_get_file` / `penpot_list_library` output. 3. Rename a page in the UI, re-run `penpot_get_file`, and confirm `revn` advanced. 4. With `PENPOT_TOKEN` temporarily broken, confirm every tool returns an actionable error rather than an exception.

david added this to the Penpot: M1 - Read primitives milestone

2026-09-14 23:10:05 +00:00

david added the

feature

label

2026-09-14 23:10:05 +00:00

david referenced this issue

2026-09-14 23:10:21 +00:00

Implement the staged changeset store and the colour/typography primitives with unit tests #194

david referenced this issue from a pull request that will close it,

2026-09-15 00:25:04 +00:00

issue-193: Validate read primitives against the live Penpot instance #226