Python, Rust, and Go at the boundaries of an agent system

Start with the request that will not finish

A multi-agent demonstration can make every component look independent and responsive. The more informative example is a request whose retrieval step stalls while a tool call has already changed something. The user disconnects, a retry arrives, and the orchestrator cannot tell whether to continue. That scenario exposes the system’s real contracts: deadlines, cancellation, ownership, idempotency, and evidence of side effects.

I would use those contracts to decide where Python, Rust, and Go belong. Python may offer the shortest path through a model ecosystem. Rust may be useful for a constrained native component. Go may suit a service whose operational model fits its concurrency tools. None of those choices establishes correctness by itself. This article proposes a way to evaluate a mixed-language architecture without assuming that adding languages makes it more scalable.

Separate orchestration from execution authority

The orchestrator decides which bounded work to request and how to combine the results. A tool executor validates and performs a specific operation under an explicit authorization context. These can begin as modules in one service. A process or language boundary becomes useful when it provides isolation, independent scaling, or an operational advantage large enough to justify serialization and deployment complexity.

An MCP tool interface is still an interface to a capability. Its presence does not remove the need to validate arguments, authorize the caller, classify side effects, or handle uncertain outcomes. I would give every tool a versioned input and output schema, a resource budget, an error taxonomy, and a declared replay policy. A read-only lookup and a command that changes an external system must not share an undifferentiated retry mechanism.

The request envelope should carry an operation identifier, correlation identifier, remaining deadline or an agreed deadline representation, and references to trusted server-side identity. The model may propose arguments, but the executor reconstructs authority from trusted context. Transport metadata should not become an opportunity for a generated argument to choose another tenant or impersonate an operator.

Make cancellation a contract across the boundary

Python asyncio documents task cancellation and structured concurrency mechanisms, including the importance of propagating cancellation correctly. Tokio’s shutdown guidance separates detecting shutdown, notifying work, and waiting for completion. Go’s pipeline guidance similarly shows why producers need a way to stop when downstream consumers are finished. These are related ideas, but an RPC boundary does not automatically translate one runtime’s cancellation into another’s.

I would propagate the request deadline to every downstream operation, reserving time for cleanup and response assembly. A worker should stop accepting new work when cancellation arrives and release resources it owns. For a remote operation that may already have committed, cancellation means that the caller stopped waiting; it does not prove that the side effect was undone. That distinction needs to appear in the response contract.

A practical state model distinguishes not_started, running, succeeded, failed, canceled_before_effect, and outcome_unknown. The last state matters after a network timeout. Recovery should query the operation’s durable status or follow the tool’s documented reconciliation procedure. Blindly retrying a payment, deployment, or destructive command because the client stopped waiting can duplicate the very action the system was meant to control.

Bound the queue before optimizing the worker

Backpressure is a statement about how much unfinished work the system is willing to own. I would bound admission, active tool calls, and queued payload bytes separately. A queue limit measured only in item count may still allow a few very large inputs to exhaust memory. The rejection behavior should tell the caller whether to retry later, reduce the request, or treat the work as permanently invalid.

The Go example below illustrates a small admission primitive for one process. It refuses to wait indefinitely for a worker slot and returns a release function for successful admission. It is not a distributed rate limiter, a fairness mechanism, or a complete HTTP server. A production service would also account for payload size, per-tenant budgets, shutdown, and the lifetime of the actual work.

I would apply the same resource model in Python or Rust. Changing the language does not repair an unbounded queue. Record time spent waiting separately from time spent executing, and use those measurements to distinguish overloaded admission from a slow dependency. Otherwise scaling a worker pool can increase pressure on an already saturated database or model endpoint.

1package admission
2 
3import (
4    "context"
5    "errors"
6)
7 
8var ErrBusy = errors.New("worker capacity exhausted")
9 
10type Gate struct { slots chan struct{} }
11 
12func New(limit int) *Gate {
13    if limit < 1 { panic("limit must be positive") }
14    return &Gate{slots: make(chan struct{}, limit)}
15}
16 
17func (g *Gate) Acquire(ctx context.Context) (func(), error) {
18    if err := ctx.Err(); err != nil { return nil, err }
19    select {
20    case g.slots <- struct{}{}:
21        // The caller must invoke release exactly once.
22        return func() { <-g.slots }, nil
23    default:
24        return nil, ErrBusy
25    }
26}

Measure the serialization tax

A Python-to-Rust boundary can be an in-process extension, a subprocess protocol, or a network service. Those choices have different failure and data-movement properties. I would measure copying, encoding, process startup, transport, and result validation in addition to the native operation itself. For a small task, the boundary can dominate. For a large batch, amortizing it may make the same operation worthwhile.

Schema evolution deserves equal attention. A tool result should distinguish missing, null, empty, and unknown values where those states have different meanings. Large integers, binary data, timestamps, and decimal values need an agreed representation across runtimes. A successful JSON parse is not sufficient evidence that a financial amount or a deadline retained its intended meaning.

I would begin with contract fixtures shared across implementations. Include maximum-size inputs, invalid encodings, extra fields, older schema versions, and partial responses. If a Rust implementation replaces a Python component, run both against the same fixtures and compare semantic outputs. Only then compare performance. A faster result with subtly different error behavior may not be a compatible replacement.

Choose one experiment that could reject the rewrite

The initial benchmark should isolate a real bottleneck. For example, a bounded document-normalization operation could be evaluated as an existing Python implementation, a vectorized library path, and a custom Rust component. A Go gateway could be evaluated separately for admission and connection handling. Testing a new gateway, a new parser, a new batching strategy, and a new model provider together would obscure the cause of any improvement.

I would record request size distributions, concurrency, warmup policy, resource limits, dependency behavior, and failure injection. Report tail latency and completion rate alongside throughput, then inspect the traces behind the tail. A benchmark that simply discards timed-out requests can make an overloaded service appear fast. Failed and canceled work still consumed resources and affected users.

The acceptance rule should include an operational threshold agreed before running the experiment. How much improvement would justify another deployment artifact, on-call surface, and release process? The rule may favor an in-process optimization over a service split, or favor a simpler implementation even when it is marginally slower. The point is to make the tradeoff visible rather than using a language preference as the evaluation criterion.

Keep the operation understandable after failure

A completed request should produce a traceable record of its stages, tool versions, authorization decisions, and durable side effects. A failed request should be equally understandable. I would test a worker crash before execution, a crash after execution but before acknowledgement, a client disconnect, and a shutdown while the queue is full. These cases establish whether the system can reconcile its own actions.

The architectural outcome may still involve Python, Rust, and Go, but each would have a defensible responsibility. It may also remain mostly Python with a small native library and better request contracts. I am interested in the system that can explain its behavior under load and interruption. The programming language is part of that explanation; it should not be asked to stand in for it.