This starter is about 500 lines of Python across a handful of files. It is meant to teach, so it leaves out most of what a real deployment needs. This part lists what is missing, what I would do next with this exact project, and what I would do differently.
Prerequisites: Part 1 through Part 9.
What the starter leaves out on purpose
I also built a larger orchestrator on LangGraph separately. Comparing the two shows what a starter skips:
- A real checkpointer. The starter uses SQLite in a local file, which is fine for one user on one machine. A shared service needs a database-backed checkpointer.
- Background workers. Everything here runs in the foreground of one terminal. Long jobs need a queue and workers.
- A web UI. The terminal is the only interface.
- Multiple specialized agents. The starter has one agent per tool group, each with a short prompt. A larger system gives agents their own prompts, tools and responsibilities.
- Authentication. The approval step is a usability safeguard, not a security boundary (Part 6). Real access control belongs in the MCP server and in whatever sits in front of the agent.
What I would do next with this project
Richer evals
The 21-case router eval (Part 5) checks one thing: the route. Next:
- Add prompts that mix categories ("where is this IP, and what is in this folder?"), which a single route cannot serve.
- Check tool choice: did the agent call the expected tool, not just land in the right group?
- Check answers: for questions with known answers, assert on the final text.
A check_url tool
From Part 9: an httpx request that returns the status code and elapsed time. Small, no browser, and it answers "is it up?" honestly.
Token-aware trimming
Part 7 trims by turns and characters (KEEP_TURNS = 2, KEEP_CHARS = 200). That is crude. Counting tokens against the model's context window would keep more when there is room and cut more when there isn't.
Streaming tokens
The terminal streams node updates (stream_mode="updates"), so you see each step as it finishes but not the answer as it is written. Streaming message tokens would feel faster, at the cost of more display code.
Per-tool approval
Approval today covers the whole batch of write calls the model proposes at once. Letting me approve or deny each call separately is a better fit when the model proposes several.
A better model
Many problems in the field log are 7B problems. A larger local model, or a hosted one, would fix some of them (miscounting, vague answers). It would also change how much the routing and prompt workarounds are worth, so re-run the eval after switching.
Public MCP servers
Nothing in the graph depends on my three servers. Any stdio MCP server can be swapped in. Try a public one and see how the curated tool list and route groups have to change.
Packaging
Right now you run scripts from the project folder. A proper entry point, a config file for server commands and a pinned lockfile would let someone else install and run it.
What I learned
These are observations from one 7B model on one machine, not general rules.
- Fewer tools beat better prompts. Cutting 39 tools to a curated 19, then to a small group per route, did more than any prompt edit.
- Build the loop by hand once. Seeing state, nodes and edges directly made the prebuilt agent understandable, and I only needed the prebuilt one after that.
- Tests before cleverness. The eval is small and imperfect, but it caught routing regressions I would have missed.
- The model is not the only failure. The ChromeDriver bug was in a tool, and the model made it worse by explaining it wrongly.
What I would do differently
I would write the eval earlier, before the first routing change instead of after. I would give every tool a deliberate failure message and test those, since the model reads them as evidence. And I would add check_url-style purpose-built tools sooner, rather than reuse a heavy one for a light question.
Closing
The point of the starter was to understand each moving part, and a small local model made every weakness visible. If you build your own, keep a field log. It is the most useful document I produced.
By Lex Gaines.