Executors that use a model

An intelligent executor uses a model to resolve one or more uncertain steps while keeping the ordinary executor contract. The planner gives it a precise task and receives the expected result without having to know its internal route.

Core idea. The model may propose how to proceed within a bounded set of possibilities. This does not grant it new permissions or let it decide by itself what counts as success.

Stable contractArguments, authority, and result shape do not change when the model is involved.

Bounded choiceThe model proposes only values or actions that the adapter knows how to validate.

Verified outcomeThe executor's postcondition, not a model's claim, determines whether an attempt succeeded.

The architectural boundary

The executor's manifest and code define its purpose, arguments, capabilities, consent, limits, and result shape. When part of the route needs a model, the adapter prepares a bounded observation, validates the proposal, and decides which concrete operation to run. The model does not invoke executors directly or alter its own contract.

In a manifest, intelligence = "agentic" declares this mode so that the registry and interface can represent it correctly. The declaration is validated, but it does not prove that an implementation is safe: proposal limits, validators, and postconditions must still be checked in code and tests.

The shared loop limits attempts, elapsed time, history, and observation size. The adapter retains control of the action.

What the shared runtime provides today

Metnos provides both synchronous and asynchronous loops. On each attempt, the loop may refresh the observation, request a proposal, validate it, execute it, and check its postcondition. The internal outcome is completed, inconclusive, or exhausted. A deterministic-first variant is also available: when the model fallback fails to produce a valid improvement, the initial deterministic result is retained.

Responsibility	Shared runtime	Executor adapter
Limits	Caps attempts, elapsed time, history, and observations	May choose tighter limits and must bound each individual operation
Proposal	Requests it and manages the loop	Decides what the model may see and propose
Validation	Never executes a rejected proposal	Defines the concrete admission rules
Success	Returns the loop outcome	Defines and checks the postcondition
Authority	Does not expand it	Operates within the manifest, policy, consent, and sandbox

The loop's elapsed-time limit does not automatically cancel an operation that has already started. It stops further attempts. The adapter must give each operation its own timeout, especially when that operation can change data.

Where it is used

Controlled selection

During web navigation, the model may choose a control identifier from candidates prepared by the broker. It cannot produce CSS or XPath selectors.

During image search, it may reorder known identifiers when metadata provides no useful lexical signal.

Controlled fallback

OCR tries Tesseract first. Only when the text is insufficient may it ask the vision model for a transcription; a weak result does not replace the original.

Structured extraction and some consultation paths apply the same principle under their own contracts.

A model is useful when the purpose is narrow, the route may vary, and the end state is verifiable. It is not a good way to hide open-ended research, cross-domain strategy, or a task without a crisp completion criterion inside an executor. Those decisions belong to the planner.

Metnos first looks for a sign-in form. If it cannot find one, it follows relevant account links and menus. It uses signals from the page; when these are insufficient, the local model chooses among controls actually observed. It does not invent links or use routes written for individual websites.

The search explores up to four steps per route, with a shared limit of 32 actions. At a dead end, it returns to a verified branch and tries an alternative. A changed route or an ambiguous control stops that route. Pausing for consent does not reset the attempts.

Once a form is found, the browser component enters credentials without giving them to the model. It checks the authorized website and field values before submission. An ambiguous form or an unexpected value prevents submission. In two-step forms, it distinguishes the email address from the password; a sign-in link elsewhere on the page does not turn a newsletter form into a login form.

Saving, listing, using, and deleting credentials share one encrypted store. Listings show metadata only. Values do not appear in results or logs, and changing the instance data directory does not select a different store.

Collecting data across pages

A request such as “collect all invoices from 2026” requires Metnos to distinguish the objects sought, the filters, and the fields to extract. Finding the first list does not finish the job: Metnos also looks for next pages, archives, other periods, and details needed for the requested fields. Visible controls are examined in groups, so a long page does not hide later ones.

The search explores up to three levels. Expansions of the same list, such as the next page, remain at the same level; a detail view enriches an item already found. Forward navigation, scrolling, and returns share 256 actions and five minutes of active work. Waiting for the user does not consume that time or reset the attempts. Each local model decision has 20 seconds, included in the overall time limit.

On returning to a page, Metnos finds previously verified controls without repeating unnecessary clicks. Identical buttons are distinguished by the visible text of their own row. Indistinguishable rows remain ambiguous; hidden text and editable values do not distinguish them. A control that cannot be identified reliably is not chosen by its position.

Before clicking, Metnos checks again that the control is the one observed, on the same page and in the same context. If it has changed, a previously visited route may be prepared again once, keeping consent requirements and limits. This recovery does not repeat a click already dispatched or preparation that has already closed overlays or handled consent.

The final check observes the control without rewriting identifiers in the page. Approval is consumed when the click is invoked: an error or timeout does not establish the absence of effects and does not authorize another click. A fresh observation can clarify the state without resending the command.

Observed navigation can link a row to its detail view. Only an unambiguous link lets Metnos merge the two views, preferring detail fields while retaining information found only in the list. Matching dates or amounts do not prove that two items are the same. Ambiguous or truncated views remain separate, and the original observations are retained.

When a cloud model is used

If configured, the cloud model in the Frontier tier chooses routes to explore; data recognition remains local. It receives the request, sign-in state, and control names with their section titles. It does not receive the full page content, extracted fields, or credentials. Control names can still contain personal data. The response states this and the step displays a red $; a request already sent remains counted even if the available time expires.

Without Frontier configured, the local model makes the choice. If Frontier is configured but its decision fails or times out, the collection remains incomplete: it does not automatically switch to another model. Administrators can disable this route selection with METNOS_SITES_FRONTIER_ROUTES=0.

Filters, dates, and spreadsheets

“All” keeps its meaning of a complete collection even when the plan shortens the request. Period, category, and other constraints must refer unambiguously to the same action and apply together to every item. Filters apply to collected data: an intermediate page’s date is not enough to exclude a route. A summary or another invoice does not establish that an item meets the requirements.

Dates retain the precision of their source: “08/2026” means a month, “July–August 2026” an interval, and “15/09/2026” a day. English month names are also recognized. The year must appear in the source; numeric dates retain day/month order regardless of interface language. Unreadable dates are reported with the number of excluded rows and the affected field, including in the spreadsheet result.

A spreadsheet reuses an extracted field when a heading differs only in capitalization or surrounding spaces. It first looks for an exact name, then a unique match allowing those differences, then a unique match ignoring punctuation and accents. It does not resolve ambiguity by choosing the first field. Issue dates and due dates remain distinct.

Spreadsheet headings are labels, separate from extracted keys. They do not add fields to an already declared schema; an explicit heading-to-source binding can request another field. Without a schema, headings can define it. A temporal filter already declaring its field and period does not receive a second automatic filter without a field.

Field discovery covers every row. A unique field that is present but empty can produce an empty column; a collision remains ambiguous even when it first appears in the last rows.

Values supplied as data never become formulas. In XLSX files, new text is stored as text, preserving existing formulas when rows are appended. In CSV files and Google Sheets writes, active text receives a protective leading apostrophe: in CSV this is part of the value and may be visible. Receipts report the number of cells protected this way. Devices need the updated package: an old package does not gain this protection without an update.

For a failed browser action, internal diagnostics distinguish preparation, command dispatch, subsequent work, cleanup, and recording. They retain the exception type and signals from a closed set, without returning messages, selectors, or page data. Signals may also match quoted text: they do not establish a cause and do not change outcomes, retries, or limits.

Complete collection is still being tested. Isolated IT/EN filter tests pass; completeness and absence of duplicates still need confirmation on the real website. Reaching a time, action, depth, or reading limit produces a partial result without inventing the total number of objects.

If an obstacle interrupts the search, data already observed can still be delivered, including in a spreadsheet, with an incomplete-collection warning and its cause. Failed sign-in, a refusal with no data, or pending consent stops the sequence. An ambiguous control does not justify raising the limits.

CAPTCHAs, codes, and resuming

Website boundaries and consent requirements remain in force throughout navigation. Cookies follow a separate procedure that rejects optional choices and rechecks new notices before submission. OTP codes are requested from the user unless automation has been explicitly authorized; they are not treated as CAPTCHAs.

For supported Cloudflare CAPTCHAs, Metnos tries the local playwright-captcha library once, in the same session and for no more than ten seconds. The attempt stays within sign-in limits, is checked on the page, and uses no paid services. If the check remains open or is unsupported, such as reCAPTCHA, Metnos asks for your help.

The panel displays the already open session page with sensitive fields obscured. You can click, scroll, refresh the image, and select Resume. Metnos checks the individual CAPTCHA and continues the original request. If credentials were already submitted, it checks that attempt without submitting them again. The panel expires after ten minutes; cancelling it closes the session. Checks requiring keyboard or audio input are not supported by this panel.

Resuming creates a new identifier linked to the suspended turn and preserves the user, conversation, results, and attachments. After form submission, the page shows progress and retrieves the result even after a reload or a lost response, without sending the values again. If it cannot confirm the result, it says so and does not automatically repeat the operation.

When it cannot finish

Since release 122: when a web control is obstructed, the managed browser automatically saves a static page copy and the obstruction measurements. No console script is required. The copy omits private fields and active code; it stays local, protected like screenshots. Capture is skipped during pending consent or authentication steps and does not certify the website workflow.

If no proposal is admitted, the loop is inconclusive. If its attempts or elapsed-time budget runs out, it is exhausted. The adapter then maps that outcome to its executor's public contract: it may keep the deterministic result, return a typed error, or request user intervention. The shared loop does not invent a result or promise persistent resumption by itself.

Transport and intelligence are independent. A local or remote executor may be direct or intelligent. Transport determines how it is reached; the manifest, policy, and code determine what it may do.