Production RAG Architecture: Permissions, Versions, and Observability

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard: RAG — Production RAG Architecture.

The gap between a RAG demo and production is mostly boundary conditions: who can see a document, how indexes update, how answers cite evidence, and what happens when an upstream is unavailable.

Core mental model

Separate ingestion, indexing, query serving, and evaluation planes. Online traffic reads only the active index while new versions build in the background and switch atomically.

Key mechanics

Enforce permissions during retrieval. Cache keys should include tenant, a permission digest, query-normalization version, and index version. Deletions must reach every replica.

Python example

from dataclasses import dataclass

@dataclass
class QueryContext:
 tenant_id: str
 allowed_collections: set[str]
 index_version: str

def cache_key(query: str, ctx: QueryContext) -> str:
 scopes = ",".join(sorted(ctx.allowed_collections))
 return f"rag:v2:{ctx.tenant_id}:{ctx.index_version}:{scopes}:{normalize(query)}"

Course focus

This article turns the whiteboard into explicit engineering boundaries: define inputs and outputs first, then decide how state, failures, and observability work. The examples use Python and focus on durable design principles rather than a particular provider version; verify APIs against the documentation for your installed dependencies.

Engineering practice

Keep business rules in the application layer instead of hiding them in untestable prompts or route handlers.
Add timeouts, bounded retries, and idempotency keys to external calls; retries are not a complete error strategy.
Record a request id, latency, input version, model/index version, and outcome without logging sensitive raw content.
Use a small fixed regression set first, then monitor quality and cost with sampled production traffic.

Common mistakes

Drawing only the happy path and omitting timeouts, empty results, rate limits, and rollback paths.
Letting one function parse input, call providers, build prompts, and persist data.
Replacing typed contracts with string conventions that can only be verified by manual integration.

Production checklist

Inputs, outputs, and error responses have explicit schemas
External dependencies have timeouts, bounded retries, rate limits, and fallbacks
Logs, metrics, and traces can be correlated to one request
Critical paths have unit tests, integration tests, and offline evaluation samples
Secrets, user content, and provider responses follow least-privilege and privacy rules

Practice

Implement the smallest loop shown on the whiteboard. Inject a timeout, an empty result, and a malformed payload, then check whether the system remains stable and diagnosable. Add one metric that proves your optimization improved quality or latency.

Hands-on exercise

Design active and candidate indexes. Simulate a failed publish and a document deletion, verify traffic never receives unauthorized stale data, and persist evidence ids for every answer.

Conclusion

An AI feature becomes maintainable when every arrow on the whiteboard maps to an input, an output, and a failure strategy.

Production Spring Boot Architecture

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard note: the goal is not to memorize boxes, but to connect each arrow to a request, a contract, and an operational signal.

Start with a real scenario

Use this situation as the mental model: A flash-sale service protects the database with cache, bounded writes, idempotency, and asynchronous inventory events.. A useful backend diagram answers four questions: where does input enter, who owns state, which boundary can fail, and how do we know the system is healthy? The flow is Gateway → Service → Cache / DB / MQ → Observability.

Deconstructing the architecture

Contract: define input, output, and error shape before adding implementation detail.
Mechanism: Nginx / Gateway → auth + rate limit; Service → idempotency + transaction. This is the part that determines latency, throughput, and testability.
State and resources: Redis → cache-aside with TTL; MySQL → source of truth. Explain creation, ownership, cleanup, and recovery.
Failure path: RocketMQ → async side effects; combine it with Metrics + Logs + Traces → feedback loop when choosing a fallback, retry, or rollback.

A small implementation boundary

The following sketch is intentionally small. It shows where a production implementation should place validation and ownership rather than pretending that a happy path is enough.

request → bounded resource → validated boundary → observable result

Keep external systems behind an adapter, repository, client, or gateway. The domain layer should depend on a stable contract so tests can use fakes and incidents can be isolated to one boundary.

Engineering decisions for the scenario

Protect dependencies first. Pools, queues, semaphores, caches, and worker counts need explicit limits. An unbounded queue only hides overload.
Preserve correctness. Use an idempotency key, transaction boundary, lock, version check, or schema validation where retries or concurrency can duplicate work.
Optimize with evidence. Use traces, slow-query data, GC pauses, queue depth, or p99 latency before choosing a cache, index, batch size, or concurrency setting.
Make recovery testable. Timeout, retry, rate limit, circuit breaking, rollback, and alerting should be exercised in a drill.

Common mistakes

Drawing only the happy path while omitting timeout, empty data, rejection, rollback, or replica lag.
Treating framework defaults as business contracts and discovering their limits after an upgrade.
Scaling machines before measuring pool exhaustion, lock contention, slow SQL, allocation, or event-loop blocking.
Treating logs as the only observability tool; metrics, traces, sampling, and redaction are also required.

Production checklist

Input, output, and error responses have explicit schemas
Dependencies have timeout, retry budgets, rate limits, and fallback behavior
Pools and queues expose capacity, waiting, and rejected-work metrics
Important state has idempotency, transaction, or recovery semantics
Logs, metrics, and traces share a request or correlation ID
Regression tests, load tests, and failure drills use production-like data

Summary

You understand this whiteboard when you can start from one request, explain every arrow’s contract, state how each risk is handled, and name the signal that would tell you to change the design.

Production | 1byteone - Java Backend / AI Application Developer

Production RAG Architecture: Permissions, Versions, and Observability

Core mental model

Key mechanics

Python example

Course focus

Engineering practice

Common mistakes

Production checklist

Practice

Hands-on exercise

Conclusion

Production Spring Boot Architecture

Start with a real scenario

Deconstructing the architecture

A small implementation boundary

Engineering decisions for the scenario

Common mistakes

Production checklist

Summary