How tool discovery keeps 25,000 tools out of your context window

A tool definition is not free. A realistic one, counting name, description, JSON Schema for parameters and a couple of examples, runs 80 to 150 tokens. Multiply by a catalog of 25,000 and you are looking at something in the region of two and a half million tokens of schema.

You cannot send that. The interesting question is what you send instead.

Why naive selection fails

The obvious answers all break somewhere:

• Send everything. Impossible at this scale, and expensive well before it becomes impossible.
• Let the user pick. Works until the task needs a tool the user did not anticipate, which is most interesting tasks.
• Keyword match on the prompt. "Close the loop with the customer" contains no word matching zendesk.ticket.update.
• Send one connector at a time. Real tasks cross systems. The interesting ones always do.

The requirement is a working set: small enough to be cheap, complete enough that the agent is not blocked mid-task.

Retrieval over a tool catalog

Tool selection is a retrieval problem with unusual properties. The corpus is small by search standards but highly structured, the query is a task description rather than a keyword, and the cost of a miss is high: a missing tool does not degrade the answer, it stops the work.

The layers that matter:

Semantic matching

Every tool is embedded from its name, description, parameter names and the connector it belongs to. The task description is embedded the same way. This is what connects "refund that order" to stripe.refund.create without either string sharing a word.

Scope filtering first

Retrieval runs after the key's allowlist, never before. A key without Stripe access should never see a Stripe tool ranked, considered, or mentioned in an error. Filtering first also shrinks the candidate set, which makes everything downstream faster.

Co-occurrence expansion

Tools travel in packs. An agent that calls ticket.search usually calls ticket.update shortly after. Expanding the working set along observed call sequences prevents the failure where an agent retrieves a read tool, does the read, and then discovers it cannot act.

Progressive disclosure

The first turn gets a compact set. If the agent's next move needs something outside it, a second retrieval widens the window mid-conversation. Most tasks never need this; the ones that do would otherwise have failed.

25,000 tools in catalog
   ↓  scope filter (key allowlist)
   ~400 candidates
   ↓  semantic rank against task
   ~40 candidates
   ↓  co-occurrence expansion + dedupe
   ~18 tools exposed   ≈ 2,000 tokens

What this buys

Roughly three orders of magnitude off the schema budget, and a second-order effect that matters more: smaller tool sets make better agents. A model choosing between eighteen relevant tools picks correctly far more often than one choosing between four hundred, where near-duplicates across connectors invite exactly the wrong call.

The counterintuitive result is that the constraint improves the outcome. You are not compromising by showing fewer tools. You are removing the noise that was causing the mistakes.