Your documents, your cluster: deploying a full RAG stack with Docker and Helm

A RAG system answers questions about your internal documents. Read that sentence again before picking infrastructure: whatever you deploy will hold your contracts, your wiki, your unreleased plans — as plaintext chunks and as embeddings. Part 4, the final part of my RAG tutorial series, packages everything from parts 1–3 for production — and its most important property is that the fully self-hosted variant, where no document leaves your cluster, is one helm install away, not a special build.

Same profiles, laptop to cluster

The application code is unchanged from part 3 — deliberately. What part 4 adds is packaging: a multi-stage Dockerfile for the Spring Boot backend, an Angular-build-into-nginx image for the frontend, and a Helm chart.

The design decision I'd defend hardest: the chart doesn't invent a new configuration language. backend.springProfiles: "pgvector,ollama" is the same string you pass to mvn spring-boot:run locally. The profile mechanics from part 2 — provider × store, everything else derived — carry from laptop to cluster untranslated. Every mapping layer between "how I run it locally" and "how it runs in prod" is a place where the two quietly diverge; the chart simply doesn't have one.

Three install examples ship with the chart: in-memory + Azure (smallest footprint), pgvector + OpenAI (persistent store), and the one that matters for the data-sovereignty conversation:

helm install rag ./helm/rag-tutorial \
  --set backend.springProfiles="pgvector\,ollama" \
  --set postgres.enabled=true \
  --set postgres.password="$(openssl rand -hex 16)" \
  --set ollama.enabled=true

Local Mistral, own Postgres with pgvector, no API key anywhere in the values. When postgres.enabled or ollama.enabled is true, the backend automatically gets POSTGRES_URL and OLLAMA_BASE_URL pointed at the in-cluster services — enabling a dependency and wiring it up is one flag, not two steps that can disagree.

Config and secrets: split by sensitivity, skip when empty

The chart splits backend environment into a ConfigMap (endpoints, model names, Confluence base URL) and a Secret (API keys, the Teams HMAC secret, Slack signing secret and bot token), both mounted via envFrom, both defaulting to empty, real values passed at install time via --set — never committed.

One template detail earns its comment: entries with empty values are skipped, not rendered. That's not cosmetics. The part-3 connectors activate on property presence — an empty RAG_SLACK_SIGNING_SECRET="" in the environment would count as "set" and boot a Slack connector with no secret. A template that renders empty strings would silently defeat the feature flags. Deployment machinery has to understand the application's activation semantics, or it will fight them.

Second detail: the deployments carry checksums of the ConfigMap and Secret as pod annotations, so helm upgrade with changed config rolls the pods. Without it, config changes land in the cluster but not in the running processes — the classic "I changed it, why is it still broken" hour.

nginx quietly solves the webhook problem

The frontend image serves the Angular dashboard and reverse-proxies three path prefixes to the backend: /api, /teams, /slack. That third pair is the payoff: during local development of part 3, Teams and Slack could only reach the backend through an ngrok tunnel, because both platforms insist on public HTTPS endpoints. Deployed behind one host with an Ingress, the webhook URLs are just there — stable, public, on the same domain as the dashboard. A deployment concern (one public entry point) dissolved an integration concern (tunnels) as a side effect.

Verified, not assumed

The habit that caught real bugs throughout this series applies to packaging too: both images build from a clean context, the chart passes helm lint, and helm template renders both the default and the everything-enabled configuration. The startup crash I wrote about in part 2 — Azure autoconfiguration activating in the offline profile — was found by booting the artifact with no credentials, which is precisely what a fresh cluster does to your pod. Your CI should treat "renders and boots credential-less" as a test, not a hope.

The takeaway — and the series

If your RAG system exists because your documents are sensitive, self-hosting can't be an afterthought — make the offline variant a first-class install target, and keep local and cluster configuration the same words. That's the standard worth holding any internal-knowledge system to, whether you build or buy.

The series, complete: part 1 — the smallest working RAG loop; part 2 — swappable LLMs and stores via profiles; part 3 — Confluence, Teams and Slack as off-by-default connectors; part 4 — this post. Each part is a self-contained, runnable project with English and German walkthroughs: github.com/halviclabs/rag-tutorials.

And if what you actually want is agents that take a ticket all the way to tested, reviewed, deployed code — that's the same engineering discipline at larger scale, and it's what we build at anvil-coder.