Best practices

The request shape, the budget and the error handling each change what comes back. This page is the short list of choices that matter, with a link to the page that specifies each one.

The short version

Call it after you have assembled the context and before you send it to the model. Never on the system prompt, the user turn or tool schemas.
Send assembled packets and tool output as text with budget.autochunk. Send records when you already hold units you want kept whole and ids you need back.
Size record text against 100,000 tokens; the request is refused at 128,000, counted over everything you send.
Set max_context_tokens from what your model call can accept and leave it there. Pin operating_point. Supply request_id and record ids.
A stand-down is a 200 with fallback_used: true. Send what you submitted. Do not retry it, do not alarm on it.
Fail open on every error path in your own code, and run shadow mode on real traffic before you change a prompt.

Where the call sits

Needlepath selects from the context you were about to send. It sits on the hop between your context assembly and your model call: retrieval, graph traversal, tool execution and memory lookups have run; the packet exists; the model has not read it yet.

retrieve / traverse / run tools  ->  assemble the packet  ->  SELECT  ->  model call

Two consequences:

Usually one call per model call. An agent loop makes the call on every turn that carries new tool output or retrieved context; a pipeline makes it once per decision step. The published adapters also sit at a tool-result or retrieval seam where that is the natural place (integrations). The request is stateless; nothing is kept between calls.
Only candidate context goes in. The system prompt, the current user turn, tool and function schemas, and anything that must reach the model stay out of the request. Concatenate them around rendered_context yourself. They do not benefit from selection, and nothing in the request pins a record today (why).

prompt = system_prompt + "\n\n" + rendered_context + "\n\n" + user_turn

The call does the most on a packet that is large relative to the step: a long tool result, a retrieval that returned many passages, an assembled record set for one decision. On a packet that is already small, a stand-down is a normal answer.

Choose the payload shape

The endpoint accepts two shapes. They are not interchangeable, and the choice is the first thing to get right.

	`text`	`records[]`
What you send	One string. Needlepath splits it on blank lines into records, each with a content-hash id.	Units you have already split, each with your `id`, `kind`, `title`, `source`.
Add `budget.autochunk`	Yes, for anything longer than a few paragraphs. Each record can then be selected within, not only kept or dropped whole.	Same, for long records.
Use it for	Assembled packets, tool output, retrieved passages, anything that arrives as prose or structured text.	Units you need back by your own id, units you want kept as your own records (kept or dropped whole without `autochunk`, excerpted within it), records where `kind`, `title` and `source` matter.
What you get back	`selected[].record_id` is a content hash; join on it only if you kept the split.	`selected[].record_id` is your id; join on it against your own copy.
Ceilings to watch	1,000 normalized units on the current operating points (`400`, `reason: "semantic_normalization_limit"`); 384 autochunk candidates (`413`).	1,000 records (`413`); 384 autochunk candidates (`413`).

Do not wrap an assembled packet as one record, or as a handful of large ones. A packet sent as a single record, or as a few very large ones, commonly comes back as a stand-down; the same content sent as text with autochunk is more often selected within. If you build records yourself, split along the boundaries that already exist in your system (one tool result, one passage, one document section) and send autochunk for anything long (records and tasks).

Records are still the right shape when the fields they carry matter. kind is read; title and source come back on the selection; a stable id is what lets you resolve a selection to your own objects with byte fidelity. text records have none of that: content and a hash. Choose by what you need back, not by what is easiest to serialise.

Size the request

The ceiling is checked on the whole request as counted in Limits. Titles, the task prompt and recent_prompts count alongside record text.

	Figure	Where it comes from
Hard ceiling	128,000 tokens	Refused with `413`, `limit_name: "input_tokens"`. The public offering table serves it as `max_tokens_per_request`.
Sizing figure	100,000 tokens of record text	Leaves room for the other counted fields. Served as `plan_tokens_per_request`.
Records	1,000 per request	`413`, `limit_name: "records"`.
Body	6 MB	Platform cap, enforced before the service sees the request. Guard it client-side.

When a packet is over the ceiling, pre-filter with your retriever so that fewer candidates arrive, or split it along its natural boundaries into several requests. If you merge the selections from several requests, size each request’s budget so the merged result still fits your downstream window: the budget and the record cap apply per request, not to the merge. Do not shrink a packet by concatenating records into fewer, larger ones; that trades a 413 for a stand-down. Three fields people reach for to get under the ceiling, and why not to:

attributes, keywords, tags are not counted, and nothing parked there can be selected or returned. They are hints with a published size cap. Content goes in text.
recent_prompts is counted and does not affect selection. Send a handful or none, never the transcript.
title is counted. Keep it a short label.

Set the budget from the downstream window

max_context_tokens is the token budget the selection has to fit inside. Set it from what your model call can accept after the system prompt and the user turn are added, and leave it there.

"budget": {
  "max_context_tokens": 4000,
  "operating_point": "np-2026-10-r6",
  "autochunk": true
}

Do not sweep the budget looking for a quality optimum. Selections at different budgets are different computations; a larger budget is not a superset of a smaller one.
Use mode: "adaptive" when difficulty varies across requests and you would otherwise overspend on the common case or under-serve the hard one. Each rung is a full pass, so the cost is latency. Watch attempted_budget_tokens (adaptive budget).
max_records caps the selection, not the request. It is the right field when your downstream step can take at most N items.

Pin the operating point and keep what it returns

Always send budget.operating_point. An unpinned request inherits the server default, and the default moves when a new label is minted. Pin it in your client’s constructor, not per call, and read the registry at /v1/operating-points when you want to move. Keep policy_version from every response that carries one. It names the frozen configuration that produced the selection, and it is how you notice that a request ran on a label you did not choose. Keep your own request_id on every call, built from something deterministic on your side (session, step, attempt), so a response can be matched to the decision it served. The same request under the same operating point returns the same selection, and record order is part of the request. Send records in a stable order.

Handle a stand-down

Needlepath does not force a selection on every call. When it cannot find evidence that a smaller context covers the task, it hands your context back and says so. On the current operating points that is a 200 with fallback_used: true, every record returned, and usage.outcome reading pass_through; on older operating points it is an empty selection with outcome: "stood_down" (both shapes).

{
  "outcome": "engaged",
  "fallback_used": true,
  "records_selected": 3,
  "records_available": 3,
  "tokens_saved": 0,
  "usage": { "outcome": "pass_through" }
}

What to do with it:

Send what you submitted. Use your own copy of the packet, or rendered_context, which returns it essentially intact.
Do not retry with a smaller budget. A stand-down is an answer, not a budget error.
Do not alarm on it, and do not read tokens_saved: 0 as a failure. On some workload shapes it is the common outcome. Track the engage rate as a property of the workload, in shadow mode first.
Alarm on selection_error. That field is set when the selection pass itself raised; the status is still 200 and the body still carries a usable full-context result. It is the one case among the four that produce fallback_used: true which is an outage rather than an answer (reading a stand-down).

Read outcome, usage.outcome, selection_error and format_metrics.engine_fallback_reason if you need to tell the cases apart. fallback_used alone does not.

Read the response

rendered_context is the model-facing block and is always returned. For most callers it is the whole answer.
selected[] is for joining back. Set return_per_record: false if you do not need it; the body gets materially smaller.
Join on record_id against your own copy when you need exact bytes. An excerpt is built from your content and never rewritten, but it may be non-contiguous, whitespace-normalised, or truncated with a ...[truncated] marker. Send render_format: "plain" if you re-parse excerpts as JSON.
score is comparable within one response only. Rank with it; do not threshold on an absolute value or compare across requests.
reason is open-ended and outcome is an open enum. Show them in a debugging view; do not branch on exact strings, and do not fail closed on a value you have not seen.
records_expanded is present only when you sent autochunk. A 200 on a request that set it means the operating point accepted and ran it; an operating point that does not support it refuses with 400.

Log enough to replay

Keep the request as you sent it, and these fields from the response, and a selection can be re-requested later and explained without re-running anything:

Field	Why
`request_id` (yours)	Ties the selection to the decision it served.
`policy_version`	The exact configuration that ran.
`outcome`, `usage.outcome`, `fallback_used`, `selection_error`	What happened, in enough detail to separate an answer from an outage.
`records_available`, `records_selected`, `records_expanded`	What was considered, what was kept, whether autochunk ran.
`tokens_before`, `tokens_after`, `usage.input_tokens`	The engine’s accounting and the counted figure on the receipt, which differ by design.
`selected[].record_id`	Which of your objects reached the model. Pair with the record order you sent.
`engine_latency_ms`	Server-side selection time, separable from your round trip.

The request as sent, including the operating point and the record order, is what makes the re-request possible; the fields above are what let you state which records reached the model and which did not.

Fail open in your own code

These are the rules our own adapters follow, and they are the right rules for yours (integrations):

A timeout, a non-2xx or an unreadable body means send the original context unchanged and log a metadata-only warning. On a 200, follow fallback_used; do not infer anything from the length of selected[], which is also empty when you set return_per_record: false.
Never send a credential, a prompt or record text to your own logs on that path.
Pin the operating point in the constructor.
Guard the 6 MB body limit before sending.

Anti-patterns

Seen	Do this instead
An entire assembled packet sent as one record, or as three or four large ones	`text` with `autochunk`, or records split along natural boundaries.
Record text sized to 128,000	Size record text against 100,000; the ceiling counts titles and prompts too.
The whole transcript in `recent_prompts`	A handful or none. It is counted and does not affect selection.
Content moved into `attributes` to get under the ceiling	Content in `text`; hints in `attributes`. Nothing parked there can be selected.
Relying on `required_record_ids` or `importance: 1.0` to guarantee a record	Keep must-send material out of the request and concatenate it around `rendered_context`.
Retrying a stand-down at a smaller budget	Send what you submitted.
Sweeping `max_context_tokens` for a quality optimum	Set it from the downstream window and leave it.
Branching on `reason`, `detail` or an exact `outcome` string	Branch on status code and `error`; treat `outcome` as open.
No `id` on records, no `request_id` on the call	Supply both from the first call; they are how a selection maps back to anything.
Changing prompts on day one	Shadow mode on real traffic first.

Get started

Release notes

How selection works

Running it in production

Billing

Integrations

The short version

Where the call sits

Choose the payload shape

Size the request

Set the budget from the downstream window

Pin the operating point and keep what it returns

Handle a stand-down

Read the response

Log enough to replay

Fail open in your own code

Anti-patterns

​The short version

​Where the call sits

​Choose the payload shape

​Size the request

​Set the budget from the downstream window

​Pin the operating point and keep what it returns

​Handle a stand-down

​Read the response

​Log enough to replay

​Fail open in your own code

​Anti-patterns

The short version

Where the call sits

Choose the payload shape

Size the request

Set the budget from the downstream window

Pin the operating point and keep what it returns

Handle a stand-down

Read the response

Log enough to replay

Fail open in your own code

Anti-patterns