Integrations

No SDK or framework package has shipped yet. Everything below the first section describes what is being built. What exists today is the HTTP contract — which is what the packages will wrap, so an integration you write now is not throwaway work.

Writing it yourself today

An integration is smaller than it looks. It is three things: map your messages to records, call the endpoint, and fail open.

KIND_BY_ROLE = {
    "user": "user_input",
    "assistant": "llm_response",
    "tool": "tool_result",
    "system": "external_data",
}


def to_records(messages):
    return [
        {
            "id": m.get("id") or f"msg-{i}",
            "kind": KIND_BY_ROLE.get(m["role"], "external_data"),
            "text": m["content"],
            "step_id": m.get("step_id"),
        }
        for i, m in enumerate(messages)
    ]


def select(messages, prompt, *, budget=8000, keep=()):
    """Returns the message subset to send, or the original list on any failure.

    `keep` is the set of messages that must survive — the system prompt, the
    current user turn. They are held back from selection entirely rather than
    sent with a pin, because nothing in the request pins a record today.
    """
    keep_set = set(keep)
    candidates = [m for m in messages if id(m) not in {id(k) for k in keep_set}]
    if not candidates:
        return messages

    try:
        response = requests.post(
            f"https://{HOST}/v1/context/select",
            headers={"Authorization": f"Bearer {API_KEY}",
                     "content-type": "application/json"},
            json={
                "request_id": new_request_id(),
                "records": to_records(candidates),
                "task": {"prompt": prompt},
                "budget": {"max_context_tokens": budget,
                           "operating_point": "np-2026-07-r2"},
                "return_per_record": True,
            },
            timeout=10,
        )
        response.raise_for_status()
        result = response.json()
    except Exception:
        return messages                       # fail open

    kept = {s["record_id"] for s in result.get("selected", [])}
    if not kept:
        return messages                       # empty selection is pass-through
    survivors = [m for i, m in enumerate(candidates)
                 if (m.get("id") or f"msg-{i}") in kept]
    # Original order, with the held-back messages back in place.
    return [m for m in messages if m in keep_set or m in survivors]

Three details in there are the whole art of it:

Hold back what must survive. required_record_ids is the field designed to pin a record and it does not pin today. Keeping a message out of records[] is the only guarantee, and it is the right shape anyway — a system prompt does not benefit from being selected.
Supply an id on every record. Without one you get a server-generated id back that joins to nothing on your side, and two identical calls return different ids.
Two pass-through paths — an exception, and an empty selection. Both return the original list. An empty selection returned as a success is a failure mode, not a valid answer.

Keep the message order. The example filters the original list rather than rebuilding from selected[], so ordering is preserved for free. selected[] is in selection order, not conversation order, and reconstructing from it directly will scramble a conversation.

What is coming

needlepath / @needlepath/sdk

Framework-free core clients for Python and TypeScript. One seam for transport and auth, so localhost, hosted and any future deployment are the same client. Shadow mode as a one-line switch.

LangChain and LangGraph

One AgentMiddleware reaching both. wrap_tool_call is the best-fit hook in the ecosystem for our sweet spot: it hands over a tool call and accepts a rewritten tool message, which is exactly a tool_result record, structurally typed, before it enters the message list.LangGraph-native users get a documented recipe in the same package — not a second package.

LiteLLM

A guardrail for the LiteLLM proxy. Mutation is proxy-only; the SDK callbacks are observability-only, so this integration is deliberately scoped to the proxy rather than pretending to work in both.

LlamaIndex

A node postprocessor, following the host’s naming convention. Worth knowing what it is and is not: a retrieval-time seam that never sees tool outputs, so it exercises the case Needlepath is least differentiated on.

The LangGraph recipe, in advance

For a LangGraph graph, the seam that composes with a non-destructive selection service is a pre-model hook that changes what the model sees without mutating graph state — the llm_input_messages pattern. Selection is advisory per call, so a later node still has the full state to work from. Deleting messages from state with RemoveMessage also works and is the right tool if you genuinely want them gone, but it is destructive: a stand-down cannot undo it, and a subsequent turn cannot recover a record the engine would have selected next time.

Rules every adapter follows

These are binding on our packages and are the right rules for yours.

Fail open on every error path

Timeout, non-2xx, empty selection — pass the original messages through unmodified and emit a metadata-only warning. At a framework boundary this matters more than anywhere else, because the caller has no way to inspect what happened.

Pin the operating point in the constructor

Not per call, and never left to the server default. Behaviour must not shift under an application that has not changed.

Guard the 6 MB body limit client-side

It is a platform cap enforced before the service is reached, so the error it produces will not be a Needlepath error shape. Check before sending.

Depend only on public, documented framework symbols

And cap the framework at a major version. Reaching into a framework’s internals buys a feature now and an outage on its next minor release.

Choosing between us and what your framework already ships

Several frameworks now include some form of context trimming in-tree and free — LangChain, for example, ships middleware that clears tool results once a token threshold is crossed. A developer evaluating Needlepath usually already has that installed, so the honest comparison is not “can something prune tool results”. It is three narrower questions:

Question	In-tree trimming	Needlepath
What happens to the content?	Typically cleared or replaced with a placeholder	Selected, not rewritten — a kept record’s excerpt is built from its own content, never paraphrased into new prose (with a stated exception)
What triggers it?	A token threshold	A judgement about whether trimming would pay on this request
What happens when trimming would not help?	It trims anyway — the threshold is the only input	It stands down and returns your context intact
What do you learn?	That something was removed	Which records, with a score and a reason, plus the token arithmetic

If your problem is “the context window overflows and I need it to stop”, a threshold-based trimmer in your framework is a genuinely reasonable answer and it costs nothing. Reach for Needlepath when which records survive is the part that matters.

Try it on your own traffic first

Run selection alongside your existing prompts, record what would have been dropped, and change nothing until you have looked.

Get started

How selection works

Running it in production

Billing

Integrations

Writing it yourself today

What is coming

needlepath / @needlepath/sdk

LangChain and LangGraph

LiteLLM

LlamaIndex

The LangGraph recipe, in advance

Rules every adapter follows

Choosing between us and what your framework already ships

Try it on your own traffic first

​Writing it yourself today

​What is coming

needlepath / @needlepath/sdk

LangChain and LangGraph

LiteLLM

LlamaIndex

​The LangGraph recipe, in advance

​Rules every adapter follows

​Choosing between us and what your framework already ships

Try it on your own traffic first

Writing it yourself today

What is coming

The LangGraph recipe, in advance

Rules every adapter follows

Choosing between us and what your framework already ships