Implement and run fetch-penpot-docs.mjs and commit the vendored Penpot reference #214

New issue

Closed

opened 2026-09-14 23:14:55 +00:00 by david · 1 comment

david commented

2026-09-14 23:14:55 +00:00

Owner

Summary

Implement skills/penpot/scripts/fetch-penpot-docs.mjs: a one-command, version-pinned refresh of the skill's vendored Penpot reference — the instance's OpenAPI spec plus the relevant common/src schemas — with deterministic output. Run it and commit the result under skills/penpot/docs/.

Background

Depends on: #188

Penpot's RPC API is internal and unversioned and drifts between releases; the validated target is Penpot 2.17. Drift is real and already observed: export-binfile takes includeLibraries/embedAssets booleans on 2.17 while later versions document a type parameter. Vendoring heavily is the right call precisely because the version is controlled — refresh becomes a routine step of a Penpot upgrade instead of a re-learning exercise.

Two sources, both needed:

The instance's OpenAPI spec at <PENPOT_URL>/api/main/doc/openapi.json — ~1 MB, generated from Penpot's Clojure source, and the single highest-value artifact: it contains the command list, the change-type union, the shape variants and the layout enums. Note the URL carefully: /api/main/doc/openapi (without .json) serves a Swagger HTML page, so a wrong URL yields HTML that looks like a corrupted download.
The relevant common/src/app/common/types schemas from the Penpot repo at the tag matching the instance — the source of truth for field-level detail the spec only sketches: file, page, pages_list, shape, shape_tree, path, text, typography, typographies_list, color, component, components_list, fills, stroke, grid, token, tokens_lib, variant, library, objects_map.

Determinism matters because the "re-run produces the committed bytes" check is how the vendored copy is kept honest: a diffable refresh means a Penpot upgrade produces a reviewable PR rather than an opaque blob dump.

Documentation Required

A separate process downloads these into the listed folders before this issue is implemented. Check the folders for the actual reference material before starting.

docs/reference/penpot-api/

<PENPOT_URL>/api/main/doc/openapi.json — the spec this script downloads; open it once by hand to confirm the URL returns JSON (not the Swagger UI page) and to note the top-level structure that will be written to disk.
https://help.penpot.app/technical-guide/integration/ — auth and base URL shape, including the requirement that PENPOT_URL has no /api suffix.
https://help.penpot.app/technical-guide/configuration/ — how to determine the running Penpot version (so the pinned schema tag can be matched to the instance rather than guessed).

Penpot source (pinned tag)

https://github.com/penpot/penpot/tree/2.17.2/common/src/app/common/types — the directory the schema files are read from; confirm the tag exists before pinning it, and use the tag matching the running instance.
https://raw.githubusercontent.com/penpot/penpot/2.17.2/common/src/app/common/types/file.cljc — example of a raw.githubusercontent.com URL shape the script should construct for each schema file.

docs/reference/nodejs/

https://nodejs.org/api/fs.html — writing files and creating directories (mkdirSync/writeFileSync), plus reading a file back for the idempotency check.
https://nodejs.org/api/globals.html#fetch — the built-in fetch used for both sources (no dependency).
https://nodejs.org/api/process.html — exit codes and process.env, so the script fails loudly in a pipeline.
https://nodejs.org/api/test.html — the pattern the repo's scripts/local-install.test.mjs follows, if a test is added for the script's pure helpers.

Pi package conventions

https://pi.dev/docs/latest/packages — how a skill's docs/ directory is packaged, so the vendored files ship with the skill.

Implementation Details

Create skills/penpot/scripts/fetch-penpot-docs.mjs (plain ESM .mjs, matching scripts/local-install.mjs's style; no dependencies):

Configuration
- Read PENPOT_URL (normalise a trailing / or /api, reusing the same rule as extensions/penpot/src/env.ts — duplicate the small helper rather than importing TS from a .mjs script, or import it if Node's type stripping makes that clean).
- PENPOT_TOKEN optional: send it when present (so a private instance works), and warn if absent.
- PENPOT_VERSION env override for the schema tag, defaulting to the pinned version recorded in the script (start at 2.17.2, the newest 2.17 tag). Verify the tag exists with a HEAD/GET before downloading, and fail with a clear message naming the tag if not.
- Optional --version <tag> / --out <dir> CLI flags.
Fetch, in order, into skills/penpot/docs/
- openapi.json — from the instance. If the response is HTML or the content type is not JSON, fail with the URL and a hint about the .json suffix (this is the documented trap).
- schema/<file>.cljc — one file per schema in the list above, from raw.githubusercontent.com/penpot/penpot/<tag>/common/src/app/common/types/<file> at the pinned tag.
- VERSION — a small text file recording: the Penpot version/tag, the instance host, the UTC date of the fetch, and the OpenAPI spec's own version/info fields if present.
Determinism
- Write files with a stable ordering and stable formatting; no timestamps inside openapi.json (write the parsed-and-re-serialised JSON with sorted keys if the raw bytes prove unstable, and document the choice). The VERSION file is the only place a date appears, and it must be excluded from the byte-comparison check.
- Print a summary: files written, byte sizes, the spec's command count, and the tag used.
- Refuse to write if a fetch fails partway (no half-vendored docs directory): fetch everything first, then write.
Run it and commit the output, then add a short pointer index in skills/penpot/SKILL.md — a "Reference" section listing docs/openapi.json, docs/schema/*.cljc and docs/VERSION with one line on what each is for. (The rest of SKILL.md is written in the next step; add only this section and keep the file valid.)
Document the upgrade path: one short paragraph in the script's header comment and in skills/penpot/docs/README.md — how to bump the pin on a Penpot upgrade (find the running version, update the pin, re-run, review the diff).

Acceptance Criteria

skills/penpot/scripts/fetch-penpot-docs.mjs runs with no third-party dependency and writes skills/penpot/docs/openapi.json, skills/penpot/docs/schema/*.cljc and skills/penpot/docs/VERSION.
A non-JSON response from the OpenAPI URL fails with a message naming the URL and the .json-suffix trap.
An unknown/invalid version tag fails with a message naming the tag, before writing anything.
A failed fetch leaves the existing vendored docs untouched (fetch-then-write, no partial writes).
Re-running produces byte-identical files for everything except docs/VERSION (proven by deleting docs/, re-running, and diffing).
skills/penpot/docs/README.md documents what the vendored files are and how to refresh them on a Penpot upgrade.
skills/penpot/SKILL.md has a "Reference" section pointing at the vendored files, and the file remains valid skill frontmatter-wise.
The fetched spec's command count and the pinned tag are printed by the script and recorded in docs/VERSION.

Test Plan

# 1. Fetch
node skills/penpot/scripts/fetch-penpot-docs.mjs
ls -R skills/penpot/docs | head -40
node -e "const s=require('./skills/penpot/docs/openapi.json'); console.log(Object.keys(s.paths ?? {}).length)"

# 2. Reproducibility
rm -rf skills/penpot/docs
node skills/penpot/scripts/fetch-penpot-docs.mjs
git status --porcelain skills/penpot/docs   # expect no diff vs the committed copy

# 3. Failure modes
node skills/penpot/scripts/fetch-penpot-docs.mjs --version 0.0.0-nope   # expect a clear error, no writes
PENPOT_URL=https://example.invalid node skills/penpot/scripts/fetch-penpot-docs.mjs   # expect a clear transport error, no partial writes

Then confirm in pi that /skills still lists penpot and that the vendored files are present in the installed package.

## Summary Implement `skills/penpot/scripts/fetch-penpot-docs.mjs`: a one-command, version-pinned refresh of the skill's vendored Penpot reference — the instance's OpenAPI spec plus the relevant `common/src` schemas — with **deterministic** output. Run it and commit the result under `skills/penpot/docs/`. ## Background **Depends on:** #188 Penpot's RPC API is internal and **unversioned** and drifts between releases; the validated target is **Penpot 2.17**. Drift is real and already observed: `export-binfile` takes `includeLibraries`/`embedAssets` booleans on 2.17 while later versions document a `type` parameter. Vendoring heavily is the right call precisely because the version is controlled — refresh becomes a routine step of a Penpot upgrade instead of a re-learning exercise. Two sources, both needed: 1. **The instance's OpenAPI spec** at `<PENPOT_URL>/api/main/doc/openapi.json` — ~1 MB, generated from Penpot's Clojure source, and the single highest-value artifact: it contains the command list, the change-type union, the shape variants and the layout enums. Note the URL carefully: `/api/main/doc/openapi` (without `.json`) serves a Swagger **HTML page**, so a wrong URL yields HTML that looks like a corrupted download. 2. **The relevant `common/src/app/common/types` schemas** from the Penpot repo at the tag matching the instance — the source of truth for field-level detail the spec only sketches: `file`, `page`, `pages_list`, `shape`, `shape_tree`, `path`, `text`, `typography`, `typographies_list`, `color`, `component`, `components_list`, `fills`, `stroke`, `grid`, `token`, `tokens_lib`, `variant`, `library`, `objects_map`. Determinism matters because the "re-run produces the committed bytes" check is how the vendored copy is kept honest: a diffable refresh means a Penpot upgrade produces a reviewable PR rather than an opaque blob dump. ## Documentation Required A separate process downloads these into the listed folders before this issue is implemented. Check the folders for the actual reference material before starting. **`docs/reference/penpot-api/`** - `<PENPOT_URL>/api/main/doc/openapi.json` — the spec this script downloads; open it once by hand to confirm the URL returns JSON (not the Swagger UI page) and to note the top-level structure that will be written to disk. - https://help.penpot.app/technical-guide/integration/ — auth and base URL shape, including the requirement that `PENPOT_URL` has no `/api` suffix. - https://help.penpot.app/technical-guide/configuration/ — how to determine the running Penpot version (so the pinned schema tag can be matched to the instance rather than guessed). **Penpot source (pinned tag)** - https://github.com/penpot/penpot/tree/2.17.2/common/src/app/common/types — the directory the schema files are read from; confirm the tag exists before pinning it, and use the tag matching the running instance. - https://raw.githubusercontent.com/penpot/penpot/2.17.2/common/src/app/common/types/file.cljc — example of a `raw.githubusercontent.com` URL shape the script should construct for each schema file. **`docs/reference/nodejs/`** - https://nodejs.org/api/fs.html — writing files and creating directories (`mkdirSync`/`writeFileSync`), plus reading a file back for the idempotency check. - https://nodejs.org/api/globals.html#fetch — the built-in `fetch` used for both sources (no dependency). - https://nodejs.org/api/process.html — exit codes and `process.env`, so the script fails loudly in a pipeline. - https://nodejs.org/api/test.html — the pattern the repo's `scripts/local-install.test.mjs` follows, if a test is added for the script's pure helpers. **Pi package conventions** - https://pi.dev/docs/latest/packages — how a skill's `docs/` directory is packaged, so the vendored files ship with the skill. ## Implementation Details Create `skills/penpot/scripts/fetch-penpot-docs.mjs` (plain ESM `.mjs`, matching `scripts/local-install.mjs`'s style; no dependencies): 1. **Configuration** - Read `PENPOT_URL` (normalise a trailing `/` or `/api`, reusing the same rule as `extensions/penpot/src/env.ts` — duplicate the small helper rather than importing TS from a `.mjs` script, or import it if Node's type stripping makes that clean). - `PENPOT_TOKEN` optional: send it when present (so a private instance works), and warn if absent. - `PENPOT_VERSION` env override for the schema tag, defaulting to the pinned version recorded in the script (start at `2.17.2`, the newest 2.17 tag). Verify the tag exists with a HEAD/GET before downloading, and fail with a clear message naming the tag if not. - Optional `--version <tag>` / `--out <dir>` CLI flags. 2. **Fetch, in order, into `skills/penpot/docs/`** - `openapi.json` — from the instance. If the response is HTML or the content type is not JSON, fail with the URL and a hint about the `.json` suffix (this is the documented trap). - `schema/<file>.cljc` — one file per schema in the list above, from `raw.githubusercontent.com/penpot/penpot/<tag>/common/src/app/common/types/<file>` at the pinned tag. - `VERSION` — a small text file recording: the Penpot version/tag, the instance host, the UTC date of the fetch, and the OpenAPI spec's own version/info fields if present. 3. **Determinism** - Write files with a stable ordering and stable formatting; no timestamps inside `openapi.json` (write the parsed-and-re-serialised JSON with sorted keys if the raw bytes prove unstable, and document the choice). The `VERSION` file is the only place a date appears, and it must be excluded from the byte-comparison check. - Print a summary: files written, byte sizes, the spec's command count, and the tag used. - Refuse to write if a fetch fails partway (no half-vendored docs directory): fetch everything first, then write. 4. **Run it and commit** the output, then add a short **pointer index** in `skills/penpot/SKILL.md` — a "Reference" section listing `docs/openapi.json`, `docs/schema/*.cljc` and `docs/VERSION` with one line on what each is for. (The rest of `SKILL.md` is written in the next step; add only this section and keep the file valid.) 5. **Document the upgrade path**: one short paragraph in the script's header comment and in `skills/penpot/docs/README.md` — how to bump the pin on a Penpot upgrade (find the running version, update the pin, re-run, review the diff). ## Acceptance Criteria - [ ] `skills/penpot/scripts/fetch-penpot-docs.mjs` runs with no third-party dependency and writes `skills/penpot/docs/openapi.json`, `skills/penpot/docs/schema/*.cljc` and `skills/penpot/docs/VERSION`. - [ ] A non-JSON response from the OpenAPI URL fails with a message naming the URL and the `.json`-suffix trap. - [ ] An unknown/invalid version tag fails with a message naming the tag, before writing anything. - [ ] A failed fetch leaves the existing vendored docs untouched (fetch-then-write, no partial writes). - [ ] Re-running produces byte-identical files for everything except `docs/VERSION` (proven by deleting `docs/`, re-running, and diffing). - [ ] `skills/penpot/docs/README.md` documents what the vendored files are and how to refresh them on a Penpot upgrade. - [ ] `skills/penpot/SKILL.md` has a "Reference" section pointing at the vendored files, and the file remains valid skill frontmatter-wise. - [ ] The fetched spec's command count and the pinned tag are printed by the script and recorded in `docs/VERSION`. ## Test Plan ```bash # 1. Fetch node skills/penpot/scripts/fetch-penpot-docs.mjs ls -R skills/penpot/docs | head -40 node -e "const s=require('./skills/penpot/docs/openapi.json'); console.log(Object.keys(s.paths ?? {}).length)" # 2. Reproducibility rm -rf skills/penpot/docs node skills/penpot/scripts/fetch-penpot-docs.mjs git status --porcelain skills/penpot/docs # expect no diff vs the committed copy # 3. Failure modes node skills/penpot/scripts/fetch-penpot-docs.mjs --version 0.0.0-nope # expect a clear error, no writes PENPOT_URL=https://example.invalid node skills/penpot/scripts/fetch-penpot-docs.mjs # expect a clear transport error, no partial writes ``` Then confirm in pi that `/skills` still lists `penpot` and that the vendored files are present in the installed package.

david added this to the Penpot: M8 - Skill content and vendored docs milestone

2026-09-14 23:14:55 +00:00

david added the

feature

label

2026-09-14 23:14:55 +00:00

david referenced this issue

2026-09-14 23:15:15 +00:00

Write the penpot SKILL.md core workflow, token conventions and Tailwind translation table #215

david referenced this issue from a pull request that will close it,

2026-09-15 07:46:06 +00:00

issue-214: Implement and run fetch-penpot-docs.mjs and commit the vendored Penpot reference #247