← Documentation index Architecture guide › scratchpad

Metnos

Observation scratchpad

What the module provides and what the runtime actually uses.

runtime/scratchpad.py implements temporary storage designed to keep oversized results out of the model context. It could retain the complete result while leaving only a short summary and identifier in the plan. The API exists, but the current engine does not use it to archive observations automatically and does not expose scratchpad_read in its ordinary catalog. It is therefore an available component, not an active product feature.

Current behaviour
Available API
Storage and lifetime
The summary produced by the module
Sliced reads
What integration requires
Required verification
Security boundaries

1. Current behaviour

The idea is simple. If an executor returned a thousand lines of text, for example, the scratchpad could retain them and show the model only “one thousand lines available, identifier X.” The model could then read just the beginning, the end, or the interval it needs. This keeps unnecessary data from filling the context.

In the active engine, step results remain in the turn's execution history. A later step reuses them through from_step or a ${stepN.field} reference; the runtime resolves the actual value without asking the model to copy it. Final synthesis projects and limits the text fields shown to the model, while full data remain in the step result.

The code also uses the word scratchpad more loosely for the step history held during a turn or its snapshot when a suspended dialog resumes. That history is active, but it is not the SQLite database described on this page.

At the beginning of a turn, agent_runtime opens the scratchpad database and removes expired rows. On the current path, however, it does not call Scratchpad.put. The historical scratchpad_threshold parameter does not control engine execution either. It is therefore incorrect to say that every observation above 4 KB is automatically offloaded to the database.

2. Available API

Method or value	Role
`Scratchpad.open(path)`	Open or create the SQLite database and schema.
`put(turn_id, step_num, executor_name, observation, ttl_seconds)`	Store an observation and return a synthetic representation.
`get(id)`	Return the complete row for an identifier.
`read(id, mode, n, start, end)`	Return all content, its beginning, its end, or a selected interval.
`list_for_turn(turn_id)`	List metadata and summaries for a turn.
`gc(now)`	Delete rows whose expiry has passed.
`stats()`	Count stored rows and bytes.
`SCRATCHPAD_READ_TOOL`	Describe the possible builtin tool; the constant alone does not make it visible to the current planner.

3. Storage and lifetime

The default location is PATH_USER_DATA/scratchpad.db. Its table stores:

id · turn_id · step_num · executor_name · content_kind
content · size_bytes · summary · created_at · expires_at

put defaults to a one-hour lifetime. Deletion is not an independent background process: it happens only when a caller runs gc. An expired row may therefore remain on disk until the next cleanup.

4. The summary produced by the module

When put is called, long text is stored in full and the summary shows its beginning and end. For binary data, the module stores the bytes and reports their size and a SHA-256 prefix. For a structured result, it looks for fields such as entries, matches, or results and reports count and schema without placing the items in the summary.

The summary can retain useful simple values such as counts, truncation state, size, message, and error. The ref_hint field explains how to reuse a step result or request a content slice.

5. Sliced reads

Mode	Result
`full`	Complete content.
`head`	First `n` characters or bytes; the tool declaration defaults to 2,000.
`tail`	Last `n` characters or bytes.
`range`	Interval from inclusive `start` to exclusive `end`.

Text is returned as UTF-8 with replacement for invalid bytes. Binary content is Base64-encoded. Metadata report full size, returned size, kind, and read mode.

6. What integration requires

A correct engine integration requires at least:

one decision point comparing observation size with a canonical setting;
a call to put before building model-visible context;
exposure of scratchpad_read only when the current user and turn have accessible rows;
an ownership check on every get, read, and list_for_turn call;
limits on full reads, which currently have no cap of their own, so a large row cannot simply re-enter the context it was meant to protect;
cleanup, observability, and resume tests after a suspended dialog.

7. Required verification

Automatic engine-to-database integration is not active. Enabling it requires dedicated tests for text, binary data, structured results, ranges, expiry, concurrency, and turn resume.

The most important test is multi-user isolation: an identifier obtained by one user must never read another user's row, even when passed directly to the builtin API.

8. Security boundaries

The current schema has no owner. It stores turn_id, but no owner_user_id, actor, or channel. get and read accept only an identifier and perform no authorisation.
An opaque identifier is not a credential. Difficulty of guessing it does not replace ownership checks.
The database is shared by the installation. It must not be described as per-user isolated until schema and callers enforce that separation.
The module does not encrypt content. At-rest protection depends on data-directory permissions and the hosting environment.
Expiry is logical. Data are removed only when gc runs.

For these reasons, the module can be studied and tested as a component, but must not be connected to a multi-user surface before ownership, authorisation, and isolation tests are added.