Routing: One Agent per Tool Group

A small model picks better tools when it sees fewer of them. In this part I add a router to the hand-built graph: one cheap model call classifies the request, and then only that group's tools reach the agent. It works, but it fixes routing and nothing else, and I'll show the two places it didn't help.

Prerequisites: Part 2 (why fewer tools help) and Part 3 (the agent -> tools loop and State).

The idea

Part 2 cut 39 tools to a curated read-only set. That helped, but the model still saw every group at once: file tools, IP tools and web tools. The next step was to split the set by MCP server and decide, per request, which group to show.

The curated set from agent.py is already grouped:

agent.py · TOOL_GROUPS

TOOL_GROUPS = {
    "filesystem": {
        "list_dir", "tree", "read_file", "head", "tail", "stat", "file_info", "find", "grep",
        "search_files", "diff",
    },
    "geoip": {"validate_ip", "lookup_ip", "query_hosts", "get_stats"},
    "webcrawler": {"crawl_website", "list_crawls", "get_crawl_results", "get_page_content"},
}  # fmt: skip

That is 11, 4 and 4 tools per route. (Some tools need a database; I left them out of the curated set.) Any stdio MCP server can take the place of these three; the router only needs a category word per group.

The graph

The last arrow stands for four: tools returns to whichever agent called it. This diagram leaves out the approval step (Part 6), which sits between each agent and tools.

The classifier

The router is one extra model call with a short prompt and no tools.

agent_graph.py · ROUTER_PROMPT

ROUTER_PROMPT = """Classify the user's request into exactly one category. \
Answer with only the category word.

- filesystem: files, folders, directories, file contents, searching text or code on disk
- geoip: IP addresses, IP geolocation, hosts
- webcrawler: web pages, websites, URLs, crawling, screenshots
- general: anything else, including follow-up questions about the conversation"""

agent_graph.py · classify

async def classify(model: ChatOpenAI, message: BaseMessage) -> str:
    """Pick a route for one user message. Falls back to 'general' on unparseable output."""
    reply = await model.ainvoke([SystemMessage(ROUTER_PROMPT), message])
    text = str(reply.content).lower()
    return next((r for r in TOOL_GROUPS if r in text), "general")

Details that matter:

• It classifies only the newest user message, not the history. That keeps the call short and the prompt unconfused by earlier turns. The "general" description includes follow-up questions, though as the run below shows the 7B model doesn't always honour that.
• It doesn't parse strictly. It lowercases the reply and returns the first route name found in it, scanning TOOL_GROUPS in order. A reply like "The category is geoip." still works.
• The fallback is general. If the reply contains none of the three names (an empty reply, "I'm not sure"), the request goes to general, which can use any tool. A bad classification degrades to the old behaviour instead of failing.
• The classifier calls the same model, without bind_tools, so it can't call a tool by mistake.

The router node and state["route"]

agent_graph.py · router

    async def router(state: State) -> dict:
        """Node: route on the newest user message only."""
        return {"route": await classify(model, state["messages"][-1])}

It writes route into state. route has no reducer, so it is overwritten on each turn. The ⇢ route: lines in the transcript below are printed when display.run() sees this update.

One agent node per route

A node is a function, so I generate them with a factory. Each one is bound to only its route's tools and gets its own hint appended to the shared system prompt.

agent_graph.py · ROUTE_HINTS

ROUTE_HINTS = {
    "general": "This is a general question or a follow-up. Do not call tools unless the "
    "answer truly needs files, IPs or web pages. Otherwise answer from the conversation "
    "or your own knowledge, and say so if you are unsure. This overrides the rule above "
    "about basing answers only on tool output.",
    "filesystem": "Focus: local files. Use list_dir to look around, read_file to read, "
    "grep or find to search. Directories have type 'directory'; files have type 'file'. "
    "Write tools (write_file, patch_file, make_dir, touch, copy, move, delete) pause for "
    "the user's approval; use them only when the user asks you to change files.",
    "geoip": "Focus: IP addresses. validate_ip checks the format and whether an IP is "
    "public; lookup_ip, query_hosts and get_stats use the database.",
    "webcrawler": "Focus: web pages. Use get_page_content for a single page; use "
    "crawl_website only when asked to crawl several pages.",
}

agent_graph.py · make_agent

    def make_agent(route: str):
        """Build an agent node bound to one route's tools ('general' gets all of them)."""
        names = GROUPS.get(route)
        group = [t for t in tools if names is None or t.name in names]
        model_with_tools = model.bind_tools(group)
        prompt = f"{SYSTEM_PROMPT}\n\n{ROUTE_HINTS.get(route, '')}".strip()

        async def agent(state: State) -> dict:
            # The system prompt is added on every call but not stored in state; old tool
            # results are shortened for the model only (state keeps them in full).
            messages = [SystemMessage(prompt), *compact(state["messages"])]
            return {"messages": [await model_with_tools.ainvoke(messages)]}

        return agent

GROUPS.get("general") is None, so general gets every tool and the hint that overrides the "answer only from tool output" rule. (In the final file the filesystem group also holds the write tools, behind approval; that is why a run says "Loaded 26 tools": the 19 read-only tools plus 7 write tools.)

Conditional edges

add_conditional_edges(source, path_function, path_map) takes three things:

1. the node the edge leaves,
2. a function that gets the state and returns a key,
3. a dict mapping each possible key to a node name.

agent_graph.py · wiring (nodes and approval cut)

    def agent_for_route(state: State) -> str:
        return f"agent_{state['route']}"

    routes = [*GROUPS, "general"]
    agent_nodes = {f"agent_{r}": f"agent_{r}" for r in routes}

    # ...
    builder.add_edge(START, "router")
    builder.add_conditional_edges("router", agent_for_route, agent_nodes)
    # Tool results go back to the same agent that asked for them.
    builder.add_conditional_edges("tools", agent_for_route, agent_nodes)

The path map is optional in LangGraph, but giving it lets the graph know every possible destination (it matters for drawing the graph and validating it). Here each key maps to the node of the same name.

Why is tools a conditional edge and not add_edge("tools", "agent")? Because there is no single agent any more. tools is shared by all four, so after it runs the graph needs to know who to return to, and the answer is whatever the router put in state["route"]. If tool results went to the wrong agent, it would see tools it didn't ask for and a different hint. The router's choice lasts the whole turn, and gets overwritten only when the next user message arrives.

A real run

One run, five prompts in one thread (so the second prompt is a follow-up). Model output varies; the home path is replaced with ~ and long lines are as the project's display trims them.

Loaded 26 tools. Thread 'doc-part4': new conversation.

you>   ⇢ route: filesystem
  → list_dir(path='~/projects')
  ← { "entries": [ { "name": ".DS_Store", "path": "~/projects/.DS_Store", "type": "file", "size": 6148, "modified": 1776964055.5414355 }, { "name" … (930 chars)

The files and directories in ~/projects are:

- .DS_Store (file, size: 6148 bytes, modified: 1776964055.5414355)
- FileSystem (directory, modified: 1771544574.37199)
- GeoIP (directory, modified: 1770222527.677097)
- LangGraph-Starter (directory, modified: 1794365488.590585)
- Orchestrator (directory, modified: 1770912879.995739)
- Webcrawler (directory, modified: 1794359215.5960104)

you>   ⇢ route: filesystem

There are 5 directories listed in the contents of ~/projects.

you>   ⇢ route: geoip
  → validate_ip(ip_address='8.8.8.8')
  ← { "ip_address": "8.8.8.8", "valid_format": true, "is_public": true, "classification": "public" }

Yes, 8.8.8.8 is a valid public IP address.

you>   ⇢ route: webcrawler
  → get_page_content(url='https://example.com')
  ← ## Example Domain **URL:** https://example.com/ **Type:** generic | **Words:** 131 | **Reading time:** 0.5min **Language:** en ### Content This domain is for us … (1039 chars)

The title of the page at <https://example.com> is "Example Domain".

you>   ⇢ route: general

Based on the information available, LangGraph-Starter is a directory in the project but no specific details about "LangGraph" itself are provided. LangGraph might be a component or project within the projects directory structure, but its exact nature is not described in the given context.

All five prompts reached a sensible route. The follow-up ("How many folders were in that list?") went to filesystem rather than general, which is arguably what my own general description says it shouldn't do, but it was harmless: the agent answered with no tool call, from the earlier list_dir result in memory. The router chooses which tools and hint the agent gets; it does not decide whether a tool gets used.

What the router did not fix

The fifth prompt in the run above is the other half of the story.

A fresh-thread run of that prompt after the hint was added:

Loaded 26 tools. Thread 'doc-part4b': new conversation.

you>   ⇢ route: general

LangGraph is a graph database and knowledge graph construction toolkit designed for representing and querying semantic relationships between language elements.

That is wrong: LangGraph is a library for building stateful agent graphs, the thing this series is about. The honest lesson is that the router fixed routing, not knowledge. A 7B model that has never seen a library will describe it fluently anyway. The hint made it willing to answer; it didn't make it right. (In the five-prompt run, the same question in a thread that already held directory listings produced the vague kind of answer instead.) If the answer matters, the model needs a tool (such as the web crawler on the docs) or a bigger model, and you should not trust a general-route answer about anything recent or niche.

Recap

• A router node classifies the newest user message and stores it in state["route"]; classify() falls back to general if the reply names no route.
• make_agent(route) builds one agent node per route, bound to only that route's tools and a route hint.
• Conditional edges (add_conditional_edges(source, fn, path_map)) send the router to the right agent, and tools back to the agent that asked.
• Routing fixed tool choice for my test prompts; it did not fix counting or a model that doesn't know a library.

Next: Part 5 adds a small eval, so I can tell when a prompt change breaks routing.