Skip to main content

​
What an operating point is

An operating point is an opaque, versioned label that names a frozen engine configuration server-side. Everything that tunes selection (thresholds, weightings, tier sizes, gate parameters) is resolved from the label. None of it appears in your request, and none of it crosses the wire. That is a contract property rather than an omission. It is what lets us retune without breaking you, and it is what makes a result citable.

​
Three rules

1

Immutable

A published label never changes meaning. Retuning mints a new label (np-2026-07-r2, np-2026-08-r3, np-2026-08-r4, …); it never edits an existing one.
2

Echoed

Every response returns policy_version, the resolved frozen version behind the label that actually ran. Record it alongside your results and a result file becomes self-describing.
3

Pinned by you

Always send budget.operating_point. Read the registry to see what exists.

​
Always pin it

If you omit operating_point, you inherit the server’s current default, and that default moves when a new label is minted. Nothing about your code changes, and your behaviour does.
An unpinned caller is the single most common way results end up attributed to a configuration nobody chose. If you compare two runs, log two numbers, or publish anything, pin the label.

​
The registry

default is the label applied when you omit one. It is reported so you can see what you would have inherited, not so you can rely on it.

​
The trade-off, stated plainly

We would rather you read this here than infer it later. Read the middle row precisely. It promises that the settings come back, which is not the same promise as byte-identical output forever. The next section is that distinction in full, and it is the one worth your time.

​
What a label pins, and what it cannot

This is the subtlety that catches people, and it matters if you are comparing against a number you recorded months ago. An operating point pins a configuration: a fixed set of named settings, written into a server-side registry under that label and never edited afterwards. Ask for np-2026-07-r2 today, next quarter, or next year and the same settings are loaded. It does not pin the engine build: the code that reads those settings and does the selection. Every operating point is served by the same deployed engine, and the engine ships together with the registry, so publishing a new build moves the code under every label at once. There is no per-label engine, and no way to request an older one.
policy_version alone does not identify a result. Two responses can both say np-2026-07-r2 and have been produced by different engine builds. If you are recording results you intend to compare later, record build_id from /v1/health alongside policy_version. It is the only wire-observable handle on the layer a label does not freeze.

​
So what does pinning actually buy you

Stated as plainly as we can put it:
  • It buys you configuration stability. Your results will not move because somebody retuned a threshold, because a new label was minted, or because a default changed underneath you. That is the failure mode pinning exists to remove, and it removes it completely.
  • It does not buy you byte-identical output across engine deploys. A build can change a score, a reason string, or an excerpt boundary while the configuration behind the label is untouched.
The practical rule: compare like with like by pinning the label and recording the build you measured on. If a comparison matters, re-measure your baseline rather than reaching back for an old number under an old label. Re-running a historical label on the current engine tells you what that configuration does now, which is usually the question you actually have.
This applies to older labels too. np-2026-07-r1 stays in the registry so that work citing it keeps resolving, and re-requesting it applies the settings it was minted with. It does not reconstruct the engine those settings were first measured against.

​
Choosing a label

Pin a label explicitly. A new integration can start on the newest published label, which is what the SDK examples do; an integration already running an older label should evaluate before moving, and the next page shows how. Older labels stay in the registry so that work citing them stays resolvable; they are not “stable” versions to prefer, and they are not tuned differently from the current one unless the release notes say so. When a new label is minted, the migration is a one-line change and there is no deadline attached to it: the old label keeps resolving.

Choosing an operating point

What each currently published label does differently on the wire, how to evaluate one against the other on your own traffic, and how to roll back.

​
Models

Needlepath never calls a model. It has no model key, makes no inference request, and has no opinion about which provider you use. It reads your records and returns a subset of them. That is the whole surface. Three consequences worth stating:
The token counts in a request and response come from a deterministic heuristic, not from any provider’s tokenizer. That is deliberate: it makes the numbers reproducible with no model dependency and no per-provider drift.It also means the counts will not match your provider’s: the heuristic is an approximation of the request’s token count, not a measurement of it.Two different jobs need two different numbers, so keep them apart. Before you send, estimate with the heuristic: max_context_tokens and the per-request ceiling are denominated in it, so a provider’s tokenizer will not predict either of them. After the response, read usage.input_tokens: that is the count the request was actually metered and billed on, and it is authoritative over anything you computed yourself.Reach for your provider’s own count only when you need their exact number for their billing, and expect it to differ from ours.
There is no downstream inference call in the selection path, so there is no third party to disclose on it. See Retention and trust.
Set it from what your downstream call can actually accept, minus whatever else you are putting in the prompt. Needlepath does not know your model’s window and will not infer one.

​
A note on max_context_tokens

Set it from your real downstream constraint and leave it there.
Set it from your downstream constraint, not by sweeping. Selections at different budgets are different computations: a larger allowance is not guaranteed to contain, or to beat, a smaller one. So do not sweep the budget looking for a quality optimum, and do not put it behind a user-facing “quality vs cost” dial.
If your budget is genuinely uncertain, the right tool is adaptive mode, which lets the engine escalate its own budget along a ladder you define, rather than you guessing a single number.