MedInsight: Following a Request from Go to Gemma 2

Start with a request, not a cluster

Imagine sending a set of medical observations to an application and asking for an analysis. You want two things: a quick acknowledgment that the request arrived, and a useful answer when the model finishes thinking. Those happen on very different timescales.

MedInsight brings together Go services for receiving and retrieving data, Python services for model inference, and Google Kubernetes Engine (GKE) for running the services. The model layer uses Gemma 2. The interesting part is the boundary between the fast request path and the slower analysis path.

Let's follow one illustrative request through that design. You don't need to know Kubernetes first. By the end, the boxes in the architecture should have a reason to exist.

The central idea: accepting work and completing work are different promises. A queue lets the application make the first promise without waiting for the second.

What happens after you click Analyze?

Here's the same journey with the moving parts visible. Indigo receives work, amber carries the handoff, teal runs inference, and slate represents stored state. The arrows show a conceptual data flow rather than every deployment resource.

Notice the dotted connection. Publishing an answer and making it retrievable are separate responsibilities. The result lookup is a necessary application contract; the diagram doesn't prescribe a particular database schema or polling implementation.

Why Go at the entrance and Python at the model?

“Polyglot” just means the system uses more than one programming language. Here, each language sits where its tooling is useful.

Go is a useful fit for network services that spend their time receiving requests, talking to storage, and coordinating concurrent work. Python has the model ecosystem: loading Gemma 2, preparing tensors, and calling inference libraries fits naturally there.

That doesn't mean Python cannot serve an API or Go is automatically faster for every task. The decision is about separating workloads with different dependencies and resource needs.

The most important shared artifact is therefore the message contract: a job identifier, a versioned payload, and a clear definition of success and failure. Changing a field in Go must not silently break a Python consumer.

The queue buys time. It also creates obligations.

Suppose the model worker is busy. Without the asynchronous handoff, an incoming request may sit waiting on inference. With a queue, the application can accept the job and show a pending state while a worker becomes available.

But a queue doesn't remove the wait. It moves the wait somewhere you can manage it. If jobs arrive faster than workers finish them, the backlog still grows. Queue age becomes a useful signal because it describes how long a user may wait before analysis even begins.

Now suppose a worker finishes the analysis and crashes before acknowledging the message. A retry can deliver the same job again. The result path needs idempotency: processing the same job twice should not create two conflicting final results.

These are requirements for making the architecture dependable, not properties that appear simply by adding Pub/Sub.

Where does Kubernetes enter the story?

So far we've described application behavior. GKE addresses a different question: where do those services run, and how are they kept running?

A pod is Kubernetes' unit for running containers. A deployment manages replicas of a service. Separating the Go and Python services into deployments lets their resource allocations differ: request handlers need capacity for network traffic, while model workers may need substantially more memory and GPU resources.

Think of the queue as the handoff between two teams and GKE as the system assigning each team its workspace. Hiring more people at the front desk doesn't clear the analysis team's backlog.

The scaling signals should reflect that distinction. API traffic helps describe the entrance workload; backlog, job duration, and worker utilization help describe the inference workload. Scaling only on API request count can miss a model bottleneck entirely.

A service boundary is also a data boundary

Medical observations make data handling part of the architecture. Before a job crosses into model inference, the system needs an explicit decision about which fields the worker is allowed to receive. Logs, retries, and error reports belong in that decision too.

A helper named sanitize_data or a prompt asking the model to protect privacy is not evidence that sensitive information has been removed. Nor does separating Go from Python establish that a system is ready for clinical use. Those claims need validation beyond an architecture diagram.

For this walkthrough, the useful engineering question is concrete: can you trace every place a request's data may travel, including its failure path? If the answer is unclear, another microservice won't fix it.

What would you test first?

Start at the seams. Send a malformed job and check that it is rejected. Deliver the same job twice and check that it produces one logical outcome. Stop a worker between writing a result and acknowledging its message. Make inference fail and check that the client eventually sees a useful failure state.

Those tests explain more about the architecture than a coverage percentage alone. Similarly, API throughput and model completion time measure different stages; a fast intake path doesn't establish a fast end-to-end analysis.

The companion article on testing MedInsight takes these failure cases further.

What to take into your next project

The lesson I want this design to communicate is simple: split a system where the work changes character. Receiving a request, waiting for capacity, running inference, and serving a result each have different failure modes.

If your application has a quick entrance and an expensive background task, first define what “accepted” and “finished” mean. Then decide whether a queue and separate workers earn their complexity. The language choices and deployment platform come after that boundary is clear.