david/pi-extensions-and-skills

Watch

Fork 0

Implement extensions/vision/src/images.ts (read, validate, data URLs) #255

New issue

Closed

opened 2026-09-17 03:33:06 +00:00 by david · 1 comment

david commented

2026-09-17 03:33:06 +00:00

Owner

Summary

Create extensions/vision/src/images.ts: resolve local image paths against the working directory, read each file, validate it against DeepSeek's documented limits (format, size, dimensions, count, total body size), and return a ValidatedImage[] carrying the metadata and the base64 data: URL the API will receive. Every failure throws a ToolError with category image.

Background

Depends on: #253

This is the local-input boundary of the vision extension: nothing leaves the machine until every path in the call has passed. The limits are DeepSeek's published numbers, which is why they live in a single injectable ValidationLimits object — tests exercise the failure paths with tiny synthetic limits instead of allocating 32 MiB buffers.

The module is also where a path becomes bytes: prompt, image order, and the later request body all depend on the returned array preserving the caller's path order.

image-size (v2, pure JS, ESM) supplies dimensions and the detected type; the format is detected from the file's content, so it is trusted over the file extension.

Documentation Required

A separate process downloads these into the listed folders before this issue is implemented. Check the folder for the actual reference material before starting.

docs/reference/image-size/

https://www.npmjs.com/package/image-size — v2 API: imageSizeFromFile(path) (async) / imageSize(buffer), the returned { width, height, type, orientation?, images? } shape, and disableTypes.
https://github.com/image-size/image-size#readme — same README on GitHub: the type values per format and the file-read concurrency limit (100). Note: the GitHub repo is archived and read-only; the npm package is still published, which is what this extension depends on.

docs/reference/deepseek-api/

https://api-docs.deepseek.com/guides/vision/ — the Limits table (formats JPEG/PNG/GIF/WebP; 48 MiB request body; 32 MiB per base64 image; 600 images per request; 8192 px per side, dropping to 4096 px at ≥ 15 images) and the "format is detected from actual file content" statement.

docs/reference/nodejs/

https://nodejs.org/api/fs.html#fspromisesstatpath-options — fs.promises.stat for regular-file/size checks.
https://nodejs.org/api/fs.html#fspromisesreadfilepath-options — fs.promises.readFile returning a Buffer.

docs/reference/mdn/

https://developer.mozilla.org/en-US/docs/Web/URI/Reference/Schemes/data — data:<mediatype>;base64,<data> URL construction.

Implementation Details

Add the dependency first (this step is the one that needs it): npm install image-size@^2 and confirm package.json dependencies gains "image-size": "^2".

Public surface:

export interface ValidationLimits {
  maxImageBytes: number;    // 32 * 1024 * 1024
  maxImages: number;        // 600
  maxBodyBytes: number;     // 48 * 1024 * 1024
  maxDimension: number;     // 8192
  maxDimensionMany: number; // 4096, applied when images.length >= 15
  manyImagesThreshold: number; // 15
}

export interface ValidatedImage {
  path: string;       // absolute, as resolved
  mime: string;       // image/png | image/jpeg | image/gif | image/webp
  width: number;
  height: number;
  bytes: number;
  dataUrl: string;    // data:<mime>;base64,<...>
}

export const DEFAULT_LIMITS: ValidationLimits; // the numbers above

export function loadAndValidateImages(
  paths: string[],
  cwd: string,
  limits?: ValidationLimits, // defaults to DEFAULT_LIMITS
): Promise<ValidatedImage[]>;

Rules (each failure is a ToolError with category === "image", message naming the offending path/limit/fix):

Check	Limit	Failure category
Path resolves to an existing regular file	—	`image`
File is readable	—	`image`
File is non-empty	—	`image`
Detected format is JPEG/PNG/GIF/WebP	—	`image`
Per-image byte size	32 MiB	`image`
Width and height	8192 px per side	`image`
Width and height when `images.length >= 15`	4096 px per side	`image`
Image count	600	`image`
Total encoded body size (sum of data URLs + prompt + overhead)	48 MiB	`image`

Implementation notes:

Resolve each path with path.resolve(cwd, p) so relative paths work from ctx.cwd; preserve the caller's order in the result.
Check count before reading files, so a 601-path call fails fast without IO.
Use imageSizeFromFile (or imageSize(buffer) on the already-read buffer — either is acceptable, but read the buffer anyway because the data URL needs it; prefer imageSize(buffer) to avoid a double read). Map type → MIME: png→image/png, jpg/jpeg→image/jpeg, gif→image/gif, webp→image/webp; any other type is rejected as unsupported.
An unparseable buffer (no type back, or image-size throws) must be surfaced as a ToolError("image") naming the file and stating it is not a supported image — never leak the raw image-size exception.
Dimensions are checked against maxDimension, and against the stricter maxDimensionMany when paths.length >= limits.manyImagesThreshold.
The body-size check sums dataUrl.length for every image plus the prompt/overhead allowance; the signature above has no prompt argument, so use the encoded data URLs plus a fixed overhead constant and document the assumption in a comment (the tool layer passes the real prompt separately; keep the check conservative).
dataUrl is data:${mime};base64,${buffer.toString("base64")}.
Keep bytes as the on-disk byte length (stat.size / buffer.length), not the base64 length.

Test file extensions/vision/src/images.test.ts (TDD):

Write tiny real fixtures for PNG/JPEG/GIF/WebP into a temp dir (a 1×1 pixel fixture per format is enough; Buffer.from(base64, "base64") of a known-good minimal file is the cheapest way).
Assert MIME derivation, cwd-relative and absolute paths, order preservation, and correct width/height/bytes/dataUrl prefix.
One test per failure row in the table, using injected tiny ValidationLimits (e.g. maxImageBytes: 10, maxImages: 2, maxDimension: 4, manyImagesThreshold: 2, maxDimensionMany: 2) so no large buffers are allocated. Each asserts instanceof ToolError and the expected category.
Cover: missing file, a directory passed as a path, empty file, non-image bytes (e.g. a text file), an unsupported real image format (e.g. a tiny BMP/TIFF if one is easy to fixture — otherwise a header of a format image-size recognises but the MIME map does not, and document it), over maxImageBytes, over maxImages, over maxDimension, over maxDimensionMany at the threshold, over maxBodyBytes.

Acceptance Criteria

package.json gains "image-size": "^2" and npm install resolves it.
loadAndValidateImages returns one ValidatedImage per input path, in order, with absolute path, mapped mime, dimensions, byte size, and a data:<mime>;base64,... URL.
Format is taken from image-size's detected type, not from the file extension.
Every row in the rules table has a test, and every failure throws ToolError with category === "image" and a message naming the file and the limit.
An unparseable/non-image buffer is reported as an image ToolError, never a raw image-size exception.
Limits are injectable: tests use tiny synthetic limits and allocate no large buffers.
node --test extensions/vision/src/images.test.ts passes from the repo root.

Test Plan

cd /Users/david/Projects/pi-extensions-and-skills
node --test extensions/vision/src/images.test.ts

Expected: all tests pass with no network access; temp fixtures are created under the OS temp dir and cleaned up.

Manual smoke check with a real image:

node --experimental-strip-types -e "import('./extensions/vision/src/images.ts').then(async m => { const r = await m.loadAndValidateImages(['README.md'], process.cwd()); console.log(r); }).catch(e => console.log(e.name, e.category, e.message))"

Expected: a ToolError with image category (README.md is not an image), not a crash.

## Summary Create `extensions/vision/src/images.ts`: resolve local image paths against the working directory, read each file, validate it against DeepSeek's documented limits (format, size, dimensions, count, total body size), and return a `ValidatedImage[]` carrying the metadata and the base64 `data:` URL the API will receive. Every failure throws a `ToolError` with category `image`. ## Background **Depends on:** #253 This is the local-input boundary of the `vision` extension: nothing leaves the machine until every path in the call has passed. The limits are DeepSeek's published numbers, which is why they live in a single injectable `ValidationLimits` object — tests exercise the failure paths with tiny synthetic limits instead of allocating 32 MiB buffers. The module is also where a path becomes bytes: `prompt`, image order, and the later request body all depend on the returned array preserving the caller's path order. `image-size` (v2, pure JS, ESM) supplies dimensions and the detected `type`; the format is detected from the file's content, so it is trusted over the file extension. ## Documentation Required A separate process downloads these into the listed folders before this issue is implemented. Check the folder for the actual reference material before starting. **`docs/reference/image-size/`** - https://www.npmjs.com/package/image-size — v2 API: `imageSizeFromFile(path)` (async) / `imageSize(buffer)`, the returned `{ width, height, type, orientation?, images? }` shape, and `disableTypes`. - https://github.com/image-size/image-size#readme — same README on GitHub: the `type` values per format and the file-read concurrency limit (100). Note: the GitHub repo is archived and read-only; the npm package is still published, which is what this extension depends on. **`docs/reference/deepseek-api/`** - https://api-docs.deepseek.com/guides/vision/ — the Limits table (formats JPEG/PNG/GIF/WebP; 48 MiB request body; 32 MiB per base64 image; 600 images per request; 8192 px per side, dropping to 4096 px at ≥ 15 images) and the "format is detected from actual file content" statement. **`docs/reference/nodejs/`** - https://nodejs.org/api/fs.html#fspromisesstatpath-options — `fs.promises.stat` for regular-file/size checks. - https://nodejs.org/api/fs.html#fspromisesreadfilepath-options — `fs.promises.readFile` returning a `Buffer`. **`docs/reference/mdn/`** - https://developer.mozilla.org/en-US/docs/Web/URI/Reference/Schemes/data — `data:<mediatype>;base64,<data>` URL construction. ## Implementation Details Add the dependency first (this step is the one that needs it): `npm install image-size@^2` and confirm `package.json` `dependencies` gains `"image-size": "^2"`. Public surface: ```ts export interface ValidationLimits { maxImageBytes: number; // 32 * 1024 * 1024 maxImages: number; // 600 maxBodyBytes: number; // 48 * 1024 * 1024 maxDimension: number; // 8192 maxDimensionMany: number; // 4096, applied when images.length >= 15 manyImagesThreshold: number; // 15 } export interface ValidatedImage { path: string; // absolute, as resolved mime: string; // image/png | image/jpeg | image/gif | image/webp width: number; height: number; bytes: number; dataUrl: string; // data:<mime>;base64,<...> } export const DEFAULT_LIMITS: ValidationLimits; // the numbers above export function loadAndValidateImages( paths: string[], cwd: string, limits?: ValidationLimits, // defaults to DEFAULT_LIMITS ): Promise<ValidatedImage[]>; ``` Rules (each failure is a `ToolError` with `category === "image"`, message naming the offending path/limit/fix): | Check | Limit | Failure category | |---|---|---| | Path resolves to an existing regular file | — | `image` | | File is readable | — | `image` | | File is non-empty | — | `image` | | Detected format is JPEG/PNG/GIF/WebP | — | `image` | | Per-image byte size | 32 MiB | `image` | | Width and height | 8192 px per side | `image` | | Width and height when `images.length >= 15` | 4096 px per side | `image` | | Image count | 600 | `image` | | Total encoded body size (sum of data URLs + prompt + overhead) | 48 MiB | `image` | Implementation notes: - Resolve each path with `path.resolve(cwd, p)` so relative paths work from `ctx.cwd`; preserve the caller's order in the result. - Check count **before** reading files, so a 601-path call fails fast without IO. - Use `imageSizeFromFile` (or `imageSize(buffer)` on the already-read buffer — either is acceptable, but read the buffer anyway because the data URL needs it; prefer `imageSize(buffer)` to avoid a double read). Map `type` → MIME: `png→image/png`, `jpg`/`jpeg`→`image/jpeg`, `gif→image/gif`, `webp→image/webp`; any other `type` is rejected as unsupported. - An unparseable buffer (no `type` back, or `image-size` throws) must be surfaced as a `ToolError("image")` naming the file and stating it is not a supported image — never leak the raw `image-size` exception. - Dimensions are checked against `maxDimension`, and against the stricter `maxDimensionMany` when `paths.length >= limits.manyImagesThreshold`. - The body-size check sums `dataUrl.length` for every image plus the prompt/overhead allowance; the signature above has no prompt argument, so use the encoded data URLs plus a fixed overhead constant and document the assumption in a comment (the tool layer passes the real prompt separately; keep the check conservative). - `dataUrl` is `data:${mime};base64,${buffer.toString("base64")}`. - Keep `bytes` as the on-disk byte length (`stat.size` / `buffer.length`), not the base64 length. Test file `extensions/vision/src/images.test.ts` (TDD): - Write tiny **real** fixtures for PNG/JPEG/GIF/WebP into a temp dir (a 1×1 pixel fixture per format is enough; `Buffer.from(base64, "base64")` of a known-good minimal file is the cheapest way). - Assert MIME derivation, `cwd`-relative and absolute paths, order preservation, and correct `width`/`height`/`bytes`/`dataUrl` prefix. - One test per failure row in the table, using injected tiny `ValidationLimits` (e.g. `maxImageBytes: 10`, `maxImages: 2`, `maxDimension: 4`, `manyImagesThreshold: 2`, `maxDimensionMany: 2`) so no large buffers are allocated. Each asserts `instanceof ToolError` and the expected `category`. - Cover: missing file, a directory passed as a path, empty file, non-image bytes (e.g. a text file), an unsupported real image format (e.g. a tiny BMP/TIFF if one is easy to fixture — otherwise a header of a format `image-size` recognises but the MIME map does not, and document it), over `maxImageBytes`, over `maxImages`, over `maxDimension`, over `maxDimensionMany` at the threshold, over `maxBodyBytes`. ## Acceptance Criteria - [ ] `package.json` gains `"image-size": "^2"` and `npm install` resolves it. - [ ] `loadAndValidateImages` returns one `ValidatedImage` per input path, in order, with absolute `path`, mapped `mime`, dimensions, byte size, and a `data:<mime>;base64,...` URL. - [ ] Format is taken from `image-size`'s detected `type`, not from the file extension. - [ ] Every row in the rules table has a test, and every failure throws `ToolError` with `category === "image"` and a message naming the file and the limit. - [ ] An unparseable/non-image buffer is reported as an `image` `ToolError`, never a raw `image-size` exception. - [ ] Limits are injectable: tests use tiny synthetic limits and allocate no large buffers. - [ ] `node --test extensions/vision/src/images.test.ts` passes from the repo root. ## Test Plan ```bash cd /Users/david/Projects/pi-extensions-and-skills node --test extensions/vision/src/images.test.ts ``` Expected: all tests pass with no network access; temp fixtures are created under the OS temp dir and cleaned up. Manual smoke check with a real image: ```bash node --experimental-strip-types -e "import('./extensions/vision/src/images.ts').then(async m => { const r = await m.loadAndValidateImages(['README.md'], process.cwd()); console.log(r); }).catch(e => console.log(e.name, e.category, e.message))" ``` Expected: a `ToolError` with `image` category (README.md is not an image), not a crash.

david added this to the Vision: M2 - Image validation milestone

2026-09-17 03:33:06 +00:00

david added the

feature

label

2026-09-17 03:33:06 +00:00

david self-assigned this

2026-09-17 03:33:06 +00:00

david referenced this issue

2026-09-17 03:33:38 +00:00

Implement extensions/vision/src/client.ts (request build, fetch, parse, error mapping) #257

david referenced this issue