Technical Blog | 1byteone - Java Backend / AI Application Developer

AI Engineering with Python

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard note: the goal is not to memorize boxes, but to connect each arrow to a request, a contract, and an operational signal.

Start with a real scenario

Use this situation as the mental model: A support copilot answers only from authorized policy passages and returns a review state when evidence is weak.. A useful backend diagram answers four questions: where does input enter, who owns state, which boundary can fail, and how do we know the system is healthy? The flow is Application → Prompt / RAG / Tool → Model → Validated Result.

Deconstructing the architecture

Contract: define input, output, and error shape before adding implementation detail.
Mechanism: FastAPI → typed request / response; Retriever → evidence + ACL filter. This is the part that determines latency, throughput, and testability.
State and resources: Model call → timeout + retry budget; Tool → allowlist + argument validation. Explain creation, ownership, cleanup, and recovery.
Failure path: Structured output → schema parse; combine it with Trace → prompt version + cost + quality when choosing a fallback, retry, or rollback.

A small implementation boundary

The following sketch is intentionally small. It shows where a production implementation should place validation and ownership rather than pretending that a happy path is enough.

request → bounded resource → validated boundary → observable result

Keep external systems behind an adapter, repository, client, or gateway. The domain layer should depend on a stable contract so tests can use fakes and incidents can be isolated to one boundary.

Engineering decisions for the scenario

Protect dependencies first. Pools, queues, semaphores, caches, and worker counts need explicit limits. An unbounded queue only hides overload.
Preserve correctness. Use an idempotency key, transaction boundary, lock, version check, or schema validation where retries or concurrency can duplicate work.
Optimize with evidence. Use traces, slow-query data, GC pauses, queue depth, or p99 latency before choosing a cache, index, batch size, or concurrency setting.
Make recovery testable. Timeout, retry, rate limit, circuit breaking, rollback, and alerting should be exercised in a drill.

Common mistakes

Drawing only the happy path while omitting timeout, empty data, rejection, rollback, or replica lag.
Treating framework defaults as business contracts and discovering their limits after an upgrade.
Scaling machines before measuring pool exhaustion, lock contention, slow SQL, allocation, or event-loop blocking.
Treating logs as the only observability tool; metrics, traces, sampling, and redaction are also required.

Production checklist

Input, output, and error responses have explicit schemas
Dependencies have timeout, retry budgets, rate limits, and fallback behavior
Pools and queues expose capacity, waiting, and rejected-work metrics
Important state has idempotency, transaction, or recovery semantics
Logs, metrics, and traces share a request or correlation ID
Regression tests, load tests, and failure drills use production-like data

Summary

You understand this whiteboard when you can start from one request, explain every arrow’s contract, state how each risk is handled, and name the signal that would tell you to change the design.

FastAPI AI Streaming APIs with Server-Sent Events

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard: FastAPI — FastAPI AI Streaming APIs with Server-Sent Events.

Streaming improves time-to-first-byte and perceived responsiveness; it does not make model computation faster. Define event framing, completion signals, error events, and cleanup on disconnect.

Core mental model

The server emits data events and the client parses event boundaries. Do not concatenate arbitrary unescaped text into SSE; prefer JSON events with a stable type field.

Key mechanics

Use try/finally in the generator to release upstream resources. Include a request id in each event. Convert model failures into an error event and close with an explicit done signal.

Python example

import json
from fastapi.responses import StreamingResponse

async def events(prompt: str):
 try:
 async for token in provider.stream(prompt):
 yield f"data: {json.dumps({'type': 'token', 'text': token})}\n\n"
 yield "data: {\"type\":\"done\"}\n\n"
 except Exception:
 yield "data: {\"type\":\"error\",\"code\":\"upstream_failed\"}\n\n"

@app.post("/v1/chat/stream")
async def stream(req: ChatRequest):
 return StreamingResponse(events(req.message), media_type="text/event-stream")

Course focus

This article turns the whiteboard into explicit engineering boundaries: define inputs and outputs first, then decide how state, failures, and observability work. The examples use Python and focus on durable design principles rather than a particular provider version; verify APIs against the documentation for your installed dependencies.

Engineering practice

Keep business rules in the application layer instead of hiding them in untestable prompts or route handlers.
Add timeouts, bounded retries, and idempotency keys to external calls; retries are not a complete error strategy.
Record a request id, latency, input version, model/index version, and outcome without logging sensitive raw content.
Use a small fixed regression set first, then monitor quality and cost with sampled production traffic.

Common mistakes

Drawing only the happy path and omitting timeouts, empty results, rate limits, and rollback paths.
Letting one function parse input, call providers, build prompts, and persist data.
Replacing typed contracts with string conventions that can only be verified by manual integration.

Production checklist

Inputs, outputs, and error responses have explicit schemas
External dependencies have timeouts, bounded retries, rate limits, and fallbacks
Logs, metrics, and traces can be correlated to one request
Critical paths have unit tests, integration tests, and offline evaluation samples
Secrets, user content, and provider responses follow least-privilege and privacy rules

Practice

Implement the smallest loop shown on the whiteboard. Inject a timeout, an empty result, and a malformed payload, then check whether the system remains stable and diagnosable. Add one metric that proves your optimization improved quality or latency.

Hands-on exercise

Write a browser SSE client and test normal completion, upstream failure, refresh, and network disconnect. Track TTFT, full latency, and cancellation rate.

Conclusion

An AI feature becomes maintainable when every arrow on the whiteboard maps to an input, an output, and a failure strategy.

FastAPI Async and Dependency Injection: Keep the Event Loop Healthy

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard: FastAPI — FastAPI Async and Dependency Injection.

async def does not make blocking code faster. File I/O, synchronous SDKs, and CPU-heavy work can stall the event loop and slow every request.

Core mental model

Async routes are appropriate for async network I/O. Isolate synchronous dependencies explicitly, and move CPU-heavy work to a queue or process pool. Dependency injection should make lifetimes visible and replacements easy.

Key mechanics

Create shared clients in lifespan and reuse their connections. Use a semaphore to bound concurrent provider calls so your service does not overwhelm an upstream.

Python example

import asyncio
from contextlib import asynccontextmanager
from fastapi import FastAPI

limit = asyncio.Semaphore(20)

async def call_provider(client, payload):
 async with limit:
 return await client.post("/responses", json=payload, timeout=15)

@asynccontextmanager
async def lifespan(app: FastAPI):
 app.state.client = make_async_client()
 yield
 await app.state.client.aclose()

app = FastAPI(lifespan=lifespan)

Course focus

Engineering practice

Keep business rules in the application layer instead of hiding them in untestable prompts or route handlers.
Add timeouts, bounded retries, and idempotency keys to external calls; retries are not a complete error strategy.
Record a request id, latency, input version, model/index version, and outcome without logging sensitive raw content.
Use a small fixed regression set first, then monitor quality and cost with sampled production traffic.

Common mistakes

Drawing only the happy path and omitting timeouts, empty results, rate limits, and rollback paths.
Letting one function parse input, call providers, build prompts, and persist data.
Replacing typed contracts with string conventions that can only be verified by manual integration.

Production checklist

Inputs, outputs, and error responses have explicit schemas
External dependencies have timeouts, bounded retries, rate limits, and fallbacks
Logs, metrics, and traces can be correlated to one request
Critical paths have unit tests, integration tests, and offline evaluation samples
Secrets, user content, and provider responses follow least-privilege and privacy rules

Practice

Hands-on exercise

Benchmark 50 concurrent requests with a synchronous fake client and an async fake client. Add a semaphore and compare upstream error rate and mean latency.

Conclusion

An AI feature becomes maintainable when every arrow on the whiteboard maps to an input, an output, and a failure strategy.

FastAPI Project Architecture: Layering Routes and Domain Services

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard: FastAPI — FastAPI Project Architecture.

FastAPI makes it easy to start in one file—and easy to end up with an unmaintainable global script. A durable structure lets routes handle HTTP, services handle use cases, and repositories handle persistence.

Core mental model

A useful direction is router → service → repository/provider, with dependencies injected through function parameters. Domain services should not depend on Request or HTTP status codes, so they can be reused by jobs and tests.

Key mechanics

Load and validate configuration at startup. Create and close pools in the lifespan context. Map exceptions centrally to a stable error schema.

Python example

from fastapi import APIRouter, Depends
from pydantic import BaseModel

router = APIRouter(prefix="/v1/chat", tags=["chat"])

class ChatRequest(BaseModel):
 message: str

class ChatResponse(BaseModel):
 answer: str

@router.post("", response_model=ChatResponse)
async def chat(req: ChatRequest, service = Depends(get_chat_service)):
 return ChatResponse(answer=await service.answer(req.message))

Course focus

Engineering practice

Keep business rules in the application layer instead of hiding them in untestable prompts or route handlers.
Add timeouts, bounded retries, and idempotency keys to external calls; retries are not a complete error strategy.
Record a request id, latency, input version, model/index version, and outcome without logging sensitive raw content.
Use a small fixed regression set first, then monitor quality and cost with sampled production traffic.

Common mistakes

Drawing only the happy path and omitting timeouts, empty results, rate limits, and rollback paths.
Letting one function parse input, call providers, build prompts, and persist data.
Replacing typed contracts with string conventions that can only be verified by manual integration.

Production checklist

Inputs, outputs, and error responses have explicit schemas
External dependencies have timeouts, bounded retries, rate limits, and fallbacks
Logs, metrics, and traces can be correlated to one request
Critical paths have unit tests, integration tests, and offline evaluation samples
Secrets, user content, and provider responses follow least-privilege and privacy rules

Practice

Hands-on exercise

Split a model-calling route into api, services, providers, and schemas. Inject a fake provider into the service and use TestClient to verify error mapping.

Conclusion

An AI feature becomes maintainable when every arrow on the whiteboard maps to an input, an output, and a failure strategy.

FastAPI Request and Response Contracts with Pydantic

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard: FastAPI — FastAPI Request and Response Contracts with Pydantic.

AI endpoints receive untrusted client input and often relay unstable model or third-party output. Pydantic schemas create the first boundary by turning data that looks right into data that has been validated.

Core mental model

Request models validate inputs, domain models enforce business invariants, and response models define the public surface. Do not return ORM objects or raw provider JSON directly.

Key mechanics

Bound string lengths, use Literal for enumerations, and model nested structures. Parse model output before returning it; route failures through an observable 502/422 branch.

Python example

from typing import Literal
from pydantic import BaseModel, Field

class ChatRequest(BaseModel):
 message: str = Field(min_length=1, max_length=4000)
 mode: Literal["answer", "summarize"] = "answer"

class ChatResponse(BaseModel):
 request_id: str
 answer: str
 citations: list[str] = Field(default_factory=list)

Course focus

Engineering practice

Keep business rules in the application layer instead of hiding them in untestable prompts or route handlers.
Add timeouts, bounded retries, and idempotency keys to external calls; retries are not a complete error strategy.
Record a request id, latency, input version, model/index version, and outcome without logging sensitive raw content.
Use a small fixed regression set first, then monitor quality and cost with sampled production traffic.

Common mistakes

Drawing only the happy path and omitting timeouts, empty results, rate limits, and rollback paths.
Letting one function parse input, call providers, build prompts, and persist data.
Replacing typed contracts with string conventions that can only be verified by manual integration.

Production checklist

Inputs, outputs, and error responses have explicit schemas
External dependencies have timeouts, bounded retries, rate limits, and fallbacks
Logs, metrics, and traces can be correlated to one request
Critical paths have unit tests, integration tests, and offline evaluation samples
Secrets, user content, and provider responses follow least-privilege and privacy rules

Practice

Hands-on exercise

Add request_id, length limits, a mode enum, and citations to a Q&A endpoint. Submit an empty string, oversized text, unknown mode, and missing fields and verify stable client errors.

Conclusion

An AI feature becomes maintainable when every arrow on the whiteboard maps to an input, an output, and a failure strategy.

Java Collections & HashMap Internals

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard note: the goal is not to memorize boxes, but to connect each arrow to a request, a contract, and an operational signal.

Start with a real scenario

Use this situation as the mental model: An order API deduplicates request IDs before writing to MySQL.. A useful backend diagram answers four questions: where does input enter, who owns state, which boundary can fail, and how do we know the system is healthy? The flow is Collection hierarchy + hash spreading + collision handling.

Deconstructing the architecture

Contract: define input, output, and error shape before adding implementation detail.
Mechanism: key → hash() → hash spreading; index = (n - 1) & hash. This is the part that determines latency, throughput, and testability.
State and resources: Bucket: Node → Node → Node; Collision → Linked List / Tree Bin. Explain creation, ownership, cleanup, and recovery.
Failure path: Capacity × Load Factor → Resize; combine it with Average lookup O(1) when distribution is healthy when choosing a fallback, retry, or rollback.

A small implementation boundary

The following sketch is intentionally small. It shows where a production implementation should place validation and ownership rather than pretending that a happy path is enough.

request → bounded resource → validated boundary → observable result

Keep external systems behind an adapter, repository, client, or gateway. The domain layer should depend on a stable contract so tests can use fakes and incidents can be isolated to one boundary.

Engineering decisions for the scenario

Protect dependencies first. Pools, queues, semaphores, caches, and worker counts need explicit limits. An unbounded queue only hides overload.
Preserve correctness. Use an idempotency key, transaction boundary, lock, version check, or schema validation where retries or concurrency can duplicate work.
Optimize with evidence. Use traces, slow-query data, GC pauses, queue depth, or p99 latency before choosing a cache, index, batch size, or concurrency setting.
Make recovery testable. Timeout, retry, rate limit, circuit breaking, rollback, and alerting should be exercised in a drill.

Common mistakes

Drawing only the happy path while omitting timeout, empty data, rejection, rollback, or replica lag.
Treating framework defaults as business contracts and discovering their limits after an upgrade.
Scaling machines before measuring pool exhaustion, lock contention, slow SQL, allocation, or event-loop blocking.
Treating logs as the only observability tool; metrics, traces, sampling, and redaction are also required.

Production checklist

Input, output, and error responses have explicit schemas
Dependencies have timeout, retry budgets, rate limits, and fallback behavior
Pools and queues expose capacity, waiting, and rejected-work metrics
Important state has idempotency, transaction, or recovery semantics
Logs, metrics, and traces share a request or correlation ID
Regression tests, load tests, and failure drills use production-like data

Summary

You understand this whiteboard when you can start from one request, explain every arrow’s contract, state how each risk is handled, and name the signal that would tell you to change the design.

Java Concurrency Model

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard note: the goal is not to memorize boxes, but to connect each arrow to a request, a contract, and an operational signal.

Start with a real scenario

Use this situation as the mental model: A notification service absorbs bursts with a bounded pool and a visible rejection policy.. A useful backend diagram answers four questions: where does input enter, who owns state, which boundary can fail, and how do we know the system is healthy? The flow is Shared state → synchronization → safe result.

Deconstructing the architecture

Contract: define input, output, and error shape before adding implementation detail.
Mechanism: Core Pool Size + Maximum Pool Size; Work Queue → backpressure boundary. This is the part that determines latency, throughput, and testability.
State and resources: Keep Alive Time → idle worker policy; Rejected Handler → explicit overload path. Explain creation, ownership, cleanup, and recovery.
Failure path: CAS: expected → compare → swap; combine it with volatile / lock → memory visibility when choosing a fallback, retry, or rollback.

A small implementation boundary

The following sketch is intentionally small. It shows where a production implementation should place validation and ownership rather than pretending that a happy path is enough.

request → bounded resource → validated boundary → observable result

Keep external systems behind an adapter, repository, client, or gateway. The domain layer should depend on a stable contract so tests can use fakes and incidents can be isolated to one boundary.

Engineering decisions for the scenario

Protect dependencies first. Pools, queues, semaphores, caches, and worker counts need explicit limits. An unbounded queue only hides overload.
Preserve correctness. Use an idempotency key, transaction boundary, lock, version check, or schema validation where retries or concurrency can duplicate work.
Optimize with evidence. Use traces, slow-query data, GC pauses, queue depth, or p99 latency before choosing a cache, index, batch size, or concurrency setting.
Make recovery testable. Timeout, retry, rate limit, circuit breaking, rollback, and alerting should be exercised in a drill.

Common mistakes

Drawing only the happy path while omitting timeout, empty data, rejection, rollback, or replica lag.
Treating framework defaults as business contracts and discovering their limits after an upgrade.
Scaling machines before measuring pool exhaustion, lock contention, slow SQL, allocation, or event-loop blocking.
Treating logs as the only observability tool; metrics, traces, sampling, and redaction are also required.

Production checklist

Input, output, and error responses have explicit schemas
Dependencies have timeout, retry budgets, rate limits, and fallback behavior
Pools and queues expose capacity, waiting, and rejected-work metrics
Important state has idempotency, transaction, or recovery semantics
Logs, metrics, and traces share a request or correlation ID
Regression tests, load tests, and failure drills use production-like data

Summary

You understand this whiteboard when you can start from one request, explain every arrow’s contract, state how each risk is handled, and name the signal that would tell you to change the design.

Java Core Architecture

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard note: the goal is not to memorize boxes, but to connect each arrow to a request, a contract, and an operational signal.

Start with a real scenario

Use this situation as the mental model: A payment service starts slowly, then becomes faster as JIT compiles its hot methods.. A useful backend diagram answers four questions: where does input enter, who owns state, which boundary can fail, and how do we know the system is healthy? The flow is Source → Bytecode → JVM → Machine Code.

Deconstructing the architecture

Contract: define input, output, and error shape before adding implementation detail.
Mechanism: Class Loader → load / link / initialize; Heap → objects + GC managed memory. This is the part that determines latency, throughput, and testability.
State and resources: Stack → frames + local variables; Method Area → class metadata. Explain creation, ownership, cleanup, and recovery.
Failure path: PC Register + Native Method Stack; combine it with Execution Engine → Interpreter + JIT when choosing a fallback, retry, or rollback.

A small implementation boundary

The following sketch is intentionally small. It shows where a production implementation should place validation and ownership rather than pretending that a happy path is enough.

request → bounded resource → validated boundary → observable result

Keep external systems behind an adapter, repository, client, or gateway. The domain layer should depend on a stable contract so tests can use fakes and incidents can be isolated to one boundary.

Engineering decisions for the scenario

Protect dependencies first. Pools, queues, semaphores, caches, and worker counts need explicit limits. An unbounded queue only hides overload.
Preserve correctness. Use an idempotency key, transaction boundary, lock, version check, or schema validation where retries or concurrency can duplicate work.
Optimize with evidence. Use traces, slow-query data, GC pauses, queue depth, or p99 latency before choosing a cache, index, batch size, or concurrency setting.
Make recovery testable. Timeout, retry, rate limit, circuit breaking, rollback, and alerting should be exercised in a drill.

Common mistakes

Drawing only the happy path while omitting timeout, empty data, rejection, rollback, or replica lag.
Treating framework defaults as business contracts and discovering their limits after an upgrade.
Scaling machines before measuring pool exhaustion, lock contention, slow SQL, allocation, or event-loop blocking.
Treating logs as the only observability tool; metrics, traces, sampling, and redaction are also required.

Production checklist

Input, output, and error responses have explicit schemas
Dependencies have timeout, retry budgets, rate limits, and fallback behavior
Pools and queues expose capacity, waiting, and rejected-work metrics
Important state has idempotency, transaction, or recovery semantics
Logs, metrics, and traces share a request or correlation ID
Regression tests, load tests, and failure drills use production-like data

Summary

You understand this whiteboard when you can start from one request, explain every arrow’s contract, state how each risk is handled, and name the signal that would tell you to change the design.

JVM Memory & Garbage Collection

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard note: the goal is not to memorize boxes, but to connect each arrow to a request, a contract, and an operational signal.

Start with a real scenario

Use this situation as the mental model: A recommendation API protects p99 latency by measuring allocation bursts rather than guessing GC flags.. A useful backend diagram answers four questions: where does input enter, who owns state, which boundary can fail, and how do we know the system is healthy? The flow is Allocation → promotion → reachability → collection.

Deconstructing the architecture

Contract: define input, output, and error shape before adding implementation detail.
Mechanism: GC Root → Reachability Analysis; Young Gen: Eden + Survivor From / To. This is the part that determines latency, throughput, and testability.
State and resources: Minor GC → copy live objects; Promotion → long-lived objects enter Old Gen. Explain creation, ownership, cleanup, and recovery.
Failure path: Mark → Sweep / Compact; combine it with G1 / ZGC → choose for latency goals when choosing a fallback, retry, or rollback.

A small implementation boundary

The following sketch is intentionally small. It shows where a production implementation should place validation and ownership rather than pretending that a happy path is enough.

request → bounded resource → validated boundary → observable result

Keep external systems behind an adapter, repository, client, or gateway. The domain layer should depend on a stable contract so tests can use fakes and incidents can be isolated to one boundary.

Engineering decisions for the scenario

Protect dependencies first. Pools, queues, semaphores, caches, and worker counts need explicit limits. An unbounded queue only hides overload.
Preserve correctness. Use an idempotency key, transaction boundary, lock, version check, or schema validation where retries or concurrency can duplicate work.
Optimize with evidence. Use traces, slow-query data, GC pauses, queue depth, or p99 latency before choosing a cache, index, batch size, or concurrency setting.
Make recovery testable. Timeout, retry, rate limit, circuit breaking, rollback, and alerting should be exercised in a drill.

Common mistakes

Drawing only the happy path while omitting timeout, empty data, rejection, rollback, or replica lag.
Treating framework defaults as business contracts and discovering their limits after an upgrade.
Scaling machines before measuring pool exhaustion, lock contention, slow SQL, allocation, or event-loop blocking.
Treating logs as the only observability tool; metrics, traces, sampling, and redaction are also required.

Production checklist

Input, output, and error responses have explicit schemas
Dependencies have timeout, retry budgets, rate limits, and fallback behavior
Pools and queues expose capacity, waiting, and rejected-work metrics
Important state has idempotency, transaction, or recovery semantics
Logs, metrics, and traces share a request or correlation ID
Regression tests, load tests, and failure drills use production-like data

Summary

You understand this whiteboard when you can start from one request, explain every arrow’s contract, state how each risk is handled, and name the signal that would tell you to change the design.

LangChain Core Architecture: Runnables, Models, and Composable Chains

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard: LangChain — LangChain Core Architecture.

LangChain is more than a wrapper around a model. Its useful abstraction is a composable runtime: once a prompt, model, and parser are treated as Runnable stages, invocation, batching, streaming, and tracing follow the same shape.

Core mental model

A typical path is input dict → ChatPromptTemplate → ChatModel → StrOutputParser. Each node owns a contract; the chain only composes them, making model swaps and isolated tests straightforward.

Key mechanics

The | operator describes data flow. Use invoke for one request, batch for bulk work, and stream for low time-to-first-token. Production code should validate at boundaries and enforce timeouts instead of swallowing exceptions.

Python example

from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI

prompt = ChatPromptTemplate.from_messages([
 ("system", "You are a concise technical assistant."),
 ("human", "Explain in three points: {question}"),
])
model = ChatOpenAI(model="gpt-4o-mini", temperature=0)
chain = prompt | model | StrOutputParser()
answer = chain.invoke({"question": "What is idempotency?"})

Course focus

Engineering practice

Keep business rules in the application layer instead of hiding them in untestable prompts or route handlers.
Add timeouts, bounded retries, and idempotency keys to external calls; retries are not a complete error strategy.
Record a request id, latency, input version, model/index version, and outcome without logging sensitive raw content.
Use a small fixed regression set first, then monitor quality and cost with sampled production traffic.

Common mistakes

Drawing only the happy path and omitting timeouts, empty results, rate limits, and rollback paths.
Letting one function parse input, call providers, build prompts, and persist data.
Replacing typed contracts with string conventions that can only be verified by manual integration.

Production checklist

Inputs, outputs, and error responses have explicit schemas
External dependencies have timeouts, bounded retries, rate limits, and fallbacks
Logs, metrics, and traces can be correlated to one request
Critical paths have unit tests, integration tests, and offline evaluation samples
Secrets, user content, and provider responses follow least-privilege and privacy rules

Practice

Hands-on exercise

Split the chain into replaceable render_prompt, call_model, and parse_output stages, and test the parser without a network call. When the model changes, tests should assert the final contract rather than provider-specific response fields.

Conclusion

An AI feature becomes maintainable when every arrow on the whiteboard maps to an input, an output, and a failure strategy.

LangChain Memory: Designing Controlled Multi-Turn State

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard: LangChain — LangChain Memory.

The challenge of multi-turn chat is not putting every message back into the prompt. It is deciding which facts to retain, for how long, and who can delete them. Memory is explicit data, not an implicit model capability.

Core mental model

Short-term memory serves the current conversation, summaries compress earlier turns, and long-term memory stores consented preferences. All three need a conversation id, tenant isolation, and retention policy.

Key mechanics

Budget tokens before deciding whether to truncate or summarize. Filter PII and deduplicate before writing long-term memory. When reading memory, include source and timestamp so stale facts do not override newer ones.

Python example

from dataclasses import dataclass
from datetime import datetime

@dataclass
class MemoryItem:
 conversation_id: str
 text: str
 source: str
 created_at: datetime

def build_context(history: list[MemoryItem], max_chars: int = 6000) -> str:
 selected, size = [], 0
 for item in reversed(history):
 if size + len(item.text) > max_chars: break
 selected.append(f"[{item.source}] {item.text}")
 size += len(item.text)
 return "\n".join(reversed(selected))

Course focus

Engineering practice

Keep business rules in the application layer instead of hiding them in untestable prompts or route handlers.
Add timeouts, bounded retries, and idempotency keys to external calls; retries are not a complete error strategy.
Record a request id, latency, input version, model/index version, and outcome without logging sensitive raw content.
Use a small fixed regression set first, then monitor quality and cost with sampled production traffic.

Common mistakes

Drawing only the happy path and omitting timeouts, empty results, rate limits, and rollback paths.
Letting one function parse input, call providers, build prompts, and persist data.
Replacing typed contracts with string conventions that can only be verified by manual integration.

Production checklist

Inputs, outputs, and error responses have explicit schemas
External dependencies have timeouts, bounded retries, rate limits, and fallbacks
Logs, metrics, and traces can be correlated to one request
Critical paths have unit tests, integration tests, and offline evaluation samples
Secrets, user content, and provider responses follow least-privilege and privacy rules

Practice

Hands-on exercise

Implement three context layers—recent turns, a conversation summary, and user preferences—and record which message ids each summary covers. When deleting a conversation, verify caches, vector stores, and audit copies are all removed.

Conclusion

An AI feature becomes maintainable when every arrow on the whiteboard maps to an input, an output, and a failure strategy.

LangChain Prompt Chains: Treat Prompts as Maintainable Programs

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard: LangChain — LangChain Prompt Chains.

A prompt is not an untracked block of text buried in code. It is a program component that needs versioning, input validation, and regression tests. This article builds a reliable template, structured input, model, and parser flow.

Core mental model

A stable Prompt Chain separates system rules, the human task, and conversation context. Variable names should carry semantics; avoid letting a generic text value change meaning across stages.

Key mechanics

Render and inspect the prompt before calling the model. Budget long context and delimit user text; never concatenate user-controlled content into system instructions.

Python example

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

prompt = ChatPromptTemplate.from_messages([
 ("system", "Answer only from context; say when it is insufficient."),
 ("human", "<context>\n{context}\n</context>\nQuestion: {question}"),
])
chain = prompt | model | StrOutputParser()
print(chain.invoke({"context": "An idempotent request does not create extra side effects when repeated.", "question": "What problem does idempotency solve?"}))

Course focus

Engineering practice

Keep business rules in the application layer instead of hiding them in untestable prompts or route handlers.
Add timeouts, bounded retries, and idempotency keys to external calls; retries are not a complete error strategy.
Record a request id, latency, input version, model/index version, and outcome without logging sensitive raw content.
Use a small fixed regression set first, then monitor quality and cost with sampled production traffic.

Common mistakes

Drawing only the happy path and omitting timeouts, empty results, rate limits, and rollback paths.
Letting one function parse input, call providers, build prompts, and persist data.
Replacing typed contracts with string conventions that can only be verified by manual integration.

Production checklist

Inputs, outputs, and error responses have explicit schemas
External dependencies have timeouts, bounded retries, rate limits, and fallbacks
Logs, metrics, and traces can be correlated to one request
Critical paths have unit tests, integration tests, and offline evaluation samples
Secrets, user content, and provider responses follow least-privilege and privacy rules

Practice

Hands-on exercise

Add a prompt_version, keep 20 fixed questions as a regression set, and compare factuality, abstention rate, and token usage before and after each prompt change.

Conclusion

An AI feature becomes maintainable when every arrow on the whiteboard maps to an input, an output, and a failure strategy.

LangChain Tool-Calling Agents: Separate Decisions from Execution

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard: LangChain — LangChain Tool-Calling Agents.

An agent should not give a model direct access to your system. The model proposes a structured tool intent; the application validates and executes it. The central boundary is: the model decides, the application executes.

Core mental model

The loop is Think → Call → Observe, while the host application controls iteration limits, tool allowlists, parameter validation, and permissions. Good descriptions improve selection but are not security policies.

Key mechanics

Tools should return machine-readable success or failure results; do not expose raw stack traces to the model or user. Require explicit confirmation for writes and emit an audit event for every call.

Python example

from langchain_core.tools import tool

@tool
def get_weather(city: str) -> dict:
 """Return current weather for an allowed city."""
 if city not in {"Shanghai", "Beijing"}:
 return {"ok": False, "error": "city_not_allowed"}
 return {"ok": True, "city": city, "celsius": 26}

llm_with_tools = model.bind_tools([get_weather])
message = llm_with_tools.invoke("Is Shanghai good for a run today?")
if message.tool_calls:
 result = get_weather.invoke(message.tool_calls[0]["args"])

Course focus

Engineering practice

Keep business rules in the application layer instead of hiding them in untestable prompts or route handlers.
Add timeouts, bounded retries, and idempotency keys to external calls; retries are not a complete error strategy.
Record a request id, latency, input version, model/index version, and outcome without logging sensitive raw content.
Use a small fixed regression set first, then monitor quality and cost with sampled production traffic.

Common mistakes

Drawing only the happy path and omitting timeouts, empty results, rate limits, and rollback paths.
Letting one function parse input, call providers, build prompts, and persist data.
Replacing typed contracts with string conventions that can only be verified by manual integration.

Production checklist

Inputs, outputs, and error responses have explicit schemas
External dependencies have timeouts, bounded retries, rate limits, and fallbacks
Logs, metrics, and traces can be correlated to one request
Critical paths have unit tests, integration tests, and offline evaluation samples
Secrets, user content, and provider responses follow least-privilege and privacy rules

Practice

Hands-on exercise

Add one read-only search tool and one side-effecting refund tool. Auto-run the first, return requires_confirmation for the second, and test that the agent cannot bypass confirmation.

Conclusion

An AI feature becomes maintainable when every arrow on the whiteboard maps to an input, an output, and a failure strategy.

MySQL Architecture

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard note: the goal is not to memorize boxes, but to connect each arrow to a request, a contract, and an operational signal.

Start with a real scenario

Use this situation as the mental model: A read API uses a pool and EXPLAIN to ensure its query reaches an index instead of scanning the table.. A useful backend diagram answers four questions: where does input enter, who owns state, which boundary can fail, and how do we know the system is healthy? The flow is Connection → Parser → Optimizer → Executor → Storage Engine.

Deconstructing the architecture

Contract: define input, output, and error shape before adding implementation detail.
Mechanism: Connection Layer → sessions + auth; SQL Layer → parse / optimize / execute. This is the part that determines latency, throughput, and testability.
State and resources: Cost model chooses access path; InnoDB → Buffer Pool + B+Tree. Explain creation, ownership, cleanup, and recovery.
Failure path: Redo Log → crash recovery durability; combine it with Undo Log → rollback + consistent reads when choosing a fallback, retry, or rollback.

A small implementation boundary

The following sketch is intentionally small. It shows where a production implementation should place validation and ownership rather than pretending that a happy path is enough.

request → bounded resource → validated boundary → observable result

Keep external systems behind an adapter, repository, client, or gateway. The domain layer should depend on a stable contract so tests can use fakes and incidents can be isolated to one boundary.

Engineering decisions for the scenario

Protect dependencies first. Pools, queues, semaphores, caches, and worker counts need explicit limits. An unbounded queue only hides overload.
Preserve correctness. Use an idempotency key, transaction boundary, lock, version check, or schema validation where retries or concurrency can duplicate work.
Optimize with evidence. Use traces, slow-query data, GC pauses, queue depth, or p99 latency before choosing a cache, index, batch size, or concurrency setting.
Make recovery testable. Timeout, retry, rate limit, circuit breaking, rollback, and alerting should be exercised in a drill.

Common mistakes

Drawing only the happy path while omitting timeout, empty data, rejection, rollback, or replica lag.
Treating framework defaults as business contracts and discovering their limits after an upgrade.
Scaling machines before measuring pool exhaustion, lock contention, slow SQL, allocation, or event-loop blocking.
Treating logs as the only observability tool; metrics, traces, sampling, and redaction are also required.

Production checklist

Input, output, and error responses have explicit schemas
Dependencies have timeout, retry budgets, rate limits, and fallback behavior
Pools and queues expose capacity, waiting, and rejected-work metrics
Important state has idempotency, transaction, or recovery semantics
Logs, metrics, and traces share a request or correlation ID
Regression tests, load tests, and failure drills use production-like data

Summary

You understand this whiteboard when you can start from one request, explain every arrow’s contract, state how each risk is handled, and name the signal that would tell you to change the design.

MySQL Index & B+Tree

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard note: the goal is not to memorize boxes, but to connect each arrow to a request, a contract, and an operational signal.

Start with a real scenario

Use this situation as the mental model: A user-search endpoint compares its EXPLAIN plan before and after adding a composite index.. A useful backend diagram answers four questions: where does input enter, who owns state, which boundary can fail, and how do we know the system is healthy? The flow is SQL → Index → B+Tree search → Leaf → Row.

Deconstructing the architecture

Contract: define input, output, and error shape before adding implementation detail.
Mechanism: Root → Internal Node → Leaf Node; Leaf: key → primary key / row pointer. This is the part that determines latency, throughput, and testability.
State and resources: Clustered Index → primary key + data; Secondary Index → key + primary key. Explain creation, ownership, cleanup, and recovery.
Failure path: Range scan keeps ordered leaf traversal; combine it with Full Table Scan × when selectivity is poor when choosing a fallback, retry, or rollback.

A small implementation boundary

The following sketch is intentionally small. It shows where a production implementation should place validation and ownership rather than pretending that a happy path is enough.

request → bounded resource → validated boundary → observable result

Keep external systems behind an adapter, repository, client, or gateway. The domain layer should depend on a stable contract so tests can use fakes and incidents can be isolated to one boundary.

Engineering decisions for the scenario

Protect dependencies first. Pools, queues, semaphores, caches, and worker counts need explicit limits. An unbounded queue only hides overload.
Preserve correctness. Use an idempotency key, transaction boundary, lock, version check, or schema validation where retries or concurrency can duplicate work.
Optimize with evidence. Use traces, slow-query data, GC pauses, queue depth, or p99 latency before choosing a cache, index, batch size, or concurrency setting.
Make recovery testable. Timeout, retry, rate limit, circuit breaking, rollback, and alerting should be exercised in a drill.

Common mistakes

Drawing only the happy path while omitting timeout, empty data, rejection, rollback, or replica lag.
Treating framework defaults as business contracts and discovering their limits after an upgrade.
Scaling machines before measuring pool exhaustion, lock contention, slow SQL, allocation, or event-loop blocking.
Treating logs as the only observability tool; metrics, traces, sampling, and redaction are also required.

Production checklist

Input, output, and error responses have explicit schemas
Dependencies have timeout, retry budgets, rate limits, and fallback behavior
Pools and queues expose capacity, waiting, and rejected-work metrics
Important state has idempotency, transaction, or recovery semantics
Logs, metrics, and traces share a request or correlation ID
Regression tests, load tests, and failure drills use production-like data

Summary

You understand this whiteboard when you can start from one request, explain every arrow’s contract, state how each risk is handled, and name the signal that would tell you to change the design.

MySQL Transaction & MVCC

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard note: the goal is not to memorize boxes, but to connect each arrow to a request, a contract, and an operational signal.

Start with a real scenario

Use this situation as the mental model: Inventory deduction uses a short transaction and row locking; a deadlock retry is idempotent.. A useful backend diagram answers four questions: where does input enter, who owns state, which boundary can fail, and how do we know the system is healthy? The flow is BEGIN → SQL → Undo Log / Read View → COMMIT.

Deconstructing the architecture

Contract: define input, output, and error shape before adding implementation detail.
Mechanism: Atomicity → all or rollback; Consistency → constraints + invariants. This is the part that determines latency, throughput, and testability.
State and resources: Isolation → visibility policy; Durability → redo log + flush. Explain creation, ownership, cleanup, and recovery.
Failure path: Undo Log → older row versions; combine it with Read View → choose visible snapshot when choosing a fallback, retry, or rollback.

A small implementation boundary

The following sketch is intentionally small. It shows where a production implementation should place validation and ownership rather than pretending that a happy path is enough.

request → bounded resource → validated boundary → observable result

Keep external systems behind an adapter, repository, client, or gateway. The domain layer should depend on a stable contract so tests can use fakes and incidents can be isolated to one boundary.

Engineering decisions for the scenario

Protect dependencies first. Pools, queues, semaphores, caches, and worker counts need explicit limits. An unbounded queue only hides overload.
Preserve correctness. Use an idempotency key, transaction boundary, lock, version check, or schema validation where retries or concurrency can duplicate work.
Optimize with evidence. Use traces, slow-query data, GC pauses, queue depth, or p99 latency before choosing a cache, index, batch size, or concurrency setting.
Make recovery testable. Timeout, retry, rate limit, circuit breaking, rollback, and alerting should be exercised in a drill.

Common mistakes

Drawing only the happy path while omitting timeout, empty data, rejection, rollback, or replica lag.
Treating framework defaults as business contracts and discovering their limits after an upgrade.
Scaling machines before measuring pool exhaustion, lock contention, slow SQL, allocation, or event-loop blocking.
Treating logs as the only observability tool; metrics, traces, sampling, and redaction are also required.

Production checklist

Input, output, and error responses have explicit schemas
Dependencies have timeout, retry budgets, rate limits, and fallback behavior
Pools and queues expose capacity, waiting, and rejected-work metrics
Important state has idempotency, transaction, or recovery semantics
Logs, metrics, and traces share a request or correlation ID
Regression tests, load tests, and failure drills use production-like data

Summary

You understand this whiteboard when you can start from one request, explain every arrow’s contract, state how each risk is handled, and name the signal that would tell you to change the design.

OpenAI API Fundamentals: Requests, Messages, Parameters, and Reliability

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard: OpenAI — OpenAI API Fundamentals.

The core of model API integration is not memorizing an endpoint. Treat the network call as an unreliable dependency and design request contracts, timeouts, error classes, and usage tracking from day one.

Core mental model

A request contains system/developer rules, user input, and optional tools. The model output is a candidate result; the application must still validate, filter, and persist it.

Key mechanics

Separate 4xx input/auth errors, 429 rate limits, 5xx upstream failures, and local timeouts. Retry only transient classes with jittered exponential backoff and a total budget.

Python example

import asyncio, random

async def call_with_retry(client, request, attempts=3):
 for n in range(attempts):
 try:
 return await client.responses.create(**request, timeout=20)
 except (TimeoutError, RateLimitError):
 if n == attempts - 1: raise
 await asyncio.sleep((2 ** n) + random.random())

Course focus

Engineering practice

Keep business rules in the application layer instead of hiding them in untestable prompts or route handlers.
Add timeouts, bounded retries, and idempotency keys to external calls; retries are not a complete error strategy.
Record a request id, latency, input version, model/index version, and outcome without logging sensitive raw content.
Use a small fixed regression set first, then monitor quality and cost with sampled production traffic.

Common mistakes

Drawing only the happy path and omitting timeouts, empty results, rate limits, and rollback paths.
Letting one function parse input, call providers, build prompts, and persist data.
Replacing typed contracts with string conventions that can only be verified by manual integration.

Production checklist

Inputs, outputs, and error responses have explicit schemas
External dependencies have timeouts, bounded retries, rate limits, and fallbacks
Logs, metrics, and traces can be correlated to one request
Critical paths have unit tests, integration tests, and offline evaluation samples
Secrets, user content, and provider responses follow least-privilege and privacy rules

Practice

Hands-on exercise

Add request id, latency, input/output tokens, and error class logging. Use a fake client that emits 429 and 400 to verify only 429 is retried.

Conclusion

An AI feature becomes maintainable when every arrow on the whiteboard maps to an input, an output, and a failure strategy.

OpenAI Multimodal AI: Working with Text, Images, and Documents

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard: OpenAI — OpenAI Multimodal AI.

The hard part of multimodal applications is input governance, not uploading a file. Each media type needs type checks, size limits, safety screening, authorization, and traceable preprocessing.

Core mental model

The flow is Input → Normalize → Multimodal Model → Structured Output → Application. Images may need resizing, PDFs may need OCR/layout parsing, and audio may need transcription with timestamps.

Key mechanics

Decouple file storage from model requests: create a short-lived object reference, scan it, and send only the minimum resolution or pages needed. Store modality, preprocessing version, and evidence regions.

Python example

from pydantic import BaseModel, Field

class ImageInsight(BaseModel):
 objects: list[str] = Field(default_factory=list, max_length=20)
 summary: str = Field(max_length=1000)
 confidence: float = Field(ge=0, le=1)

def validate_upload(content_type: str, size: int) -> None:
 if content_type not in {"image/png", "image/jpeg", "application/pdf"}:
 raise ValueError("unsupported_media_type")
 if size > 10 * 1024 * 1024:
 raise ValueError("file_too_large")

Course focus

Engineering practice

Keep business rules in the application layer instead of hiding them in untestable prompts or route handlers.
Add timeouts, bounded retries, and idempotency keys to external calls; retries are not a complete error strategy.
Record a request id, latency, input version, model/index version, and outcome without logging sensitive raw content.
Use a small fixed regression set first, then monitor quality and cost with sampled production traffic.

Common mistakes

Drawing only the happy path and omitting timeouts, empty results, rate limits, and rollback paths.
Letting one function parse input, call providers, build prompts, and persist data.
Replacing typed contracts with string conventions that can only be verified by manual integration.

Production checklist

Inputs, outputs, and error responses have explicit schemas
External dependencies have timeouts, bounded retries, rate limits, and fallbacks
Logs, metrics, and traces can be correlated to one request
Critical paths have unit tests, integration tests, and offline evaluation samples
Secrets, user content, and provider responses follow least-privilege and privacy rules

Practice

Hands-on exercise

Build an invoice-image-to-JSON experiment. Test a clear image, rotated image, low-resolution image, and oversized PDF; record field accuracy, latency, and cost.

Conclusion

An AI feature becomes maintainable when every arrow on the whiteboard maps to an input, an output, and a failure strategy.

OpenAI Structured Outputs: Make Model Results Verifiable

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard: OpenAI — OpenAI Structured Outputs.

Free text is useful for display; structured output is useful for programs. Structured Outputs do not make a model infallible—they make success, refusal, and failure distinguishable to code.

Core mental model

A schema constrains types, required fields, and enums but cannot guarantee factual correctness. Business validation must still check ranges, permissions, and citation existence.

Key mechanics

Separate transport, schema, and domain validation. When the model refuses or returns an incomplete result, do not silently fill defaults and disguise uncertainty as fact.

Python example

from pydantic import BaseModel, Field

class Incident(BaseModel):
 severity: str = Field(pattern="^(low|medium|high)$")
 summary: str = Field(min_length=1, max_length=500)
 actions: list[str] = Field(min_length=1, max_length=5)

response = client.responses.parse(model="gpt-4o-mini", input="Turn this incident log into a structured summary", text_format=Incident)
if response.output_parsed is None: raise ValueError("model_refusal_or_parse_failure")

Course focus

Engineering practice

Keep business rules in the application layer instead of hiding them in untestable prompts or route handlers.
Add timeouts, bounded retries, and idempotency keys to external calls; retries are not a complete error strategy.
Record a request id, latency, input version, model/index version, and outcome without logging sensitive raw content.
Use a small fixed regression set first, then monitor quality and cost with sampled production traffic.

Common mistakes

Drawing only the happy path and omitting timeouts, empty results, rate limits, and rollback paths.
Letting one function parse input, call providers, build prompts, and persist data.
Replacing typed contracts with string conventions that can only be verified by manual integration.

Production checklist

Inputs, outputs, and error responses have explicit schemas
External dependencies have timeouts, bounded retries, rate limits, and fallbacks
Logs, metrics, and traces can be correlated to one request
Critical paths have unit tests, integration tests, and offline evaluation samples
Secrets, user content, and provider responses follow least-privilege and privacy rules

Practice

Hands-on exercise

Add a schema version and backward-compatible fields. Prepare normal, refusal, missing-field, and out-of-range fixtures and verify the API never returns half-valid JSON as success.

Conclusion

An AI feature becomes maintainable when every arrow on the whiteboard maps to an input, an output, and a failure strategy.

OpenAI Tool Calling: Schemas, Execution Loops, and Safety Boundaries

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard: OpenAI — OpenAI Tool Calling.

Tool Calling lets a model emit a function name and arguments; it does not execute the function. The application must check allowlists, arguments, and authorization before returning a tool result.

Core mental model

The loop is request → tool_call → validate → execute → tool_result → request. Bound every round, and never let a final answer bypass business authorization.

Key mechanics

A schema describes how to call; a policy decides whether the call is allowed. Read operations may be automatic, while writes need confirmation, idempotency, and audit logs.

Python example

def execute_tool(call, user):
 if call.name not in TOOL_ALLOWLIST:
 return {"ok": False, "error": "tool_not_allowed"}
 args = validate_args(call.name, call.arguments)
 authorize(user, call.name, args)
 if is_side_effecting(call.name) and not user.confirmed:
 return {"ok": False, "error": "confirmation_required"}
 return TOOL_REGISTRY[call.name](**args)

Course focus

Engineering practice

Keep business rules in the application layer instead of hiding them in untestable prompts or route handlers.
Add timeouts, bounded retries, and idempotency keys to external calls; retries are not a complete error strategy.
Record a request id, latency, input version, model/index version, and outcome without logging sensitive raw content.
Use a small fixed regression set first, then monitor quality and cost with sampled production traffic.

Common mistakes

Drawing only the happy path and omitting timeouts, empty results, rate limits, and rollback paths.
Letting one function parse input, call providers, build prompts, and persist data.
Replacing typed contracts with string conventions that can only be verified by manual integration.

Production checklist

Inputs, outputs, and error responses have explicit schemas
External dependencies have timeouts, bounded retries, rate limits, and fallbacks
Logs, metrics, and traces can be correlated to one request
Critical paths have unit tests, integration tests, and offline evaluation samples
Secrets, user content, and provider responses follow least-privilege and privacy rules

Practice

Hands-on exercise

Implement weather lookup and refund request tools. Let weather run automatically, but require confirmation, an idempotency key, and an audit log for refunds.

Conclusion

An AI feature becomes maintainable when every arrow on the whiteboard maps to an input, an output, and a failure strategy.

Optimizing RAG Retrievers: Top-K, Hybrid Search, and Reranking

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard: RAG — Optimizing RAG Retrievers.

The goal of retrieval optimization is not to retrieve as much as possible. It is to rank supporting evidence first within a context and latency budget.

Core mental model

Keyword search excels at exact terms while vector search captures semantics. Hybrid retrieval merges candidates, then a reranker scores the query-document pair.

Key mechanics

Tune chunk size, overlap, K, thresholds, and reranking on an offline set. Higher recall can add noise; evaluate nDCG, answer support rate, and end-to-end latency together.

Python example

def hybrid_retrieve(query: str, k: int = 20):
 dense = vector_store.search(query, k=k)
 lexical = bm25.search(query, k=k)
 candidates = deduplicate(dense + lexical)
 ranked = reranker.rank(query, candidates)
 return [item for item in ranked if item.score >= 0.35][:5]

Course focus

Engineering practice

Keep business rules in the application layer instead of hiding them in untestable prompts or route handlers.
Add timeouts, bounded retries, and idempotency keys to external calls; retries are not a complete error strategy.
Record a request id, latency, input version, model/index version, and outcome without logging sensitive raw content.
Use a small fixed regression set first, then monitor quality and cost with sampled production traffic.

Common mistakes

Drawing only the happy path and omitting timeouts, empty results, rate limits, and rollback paths.
Letting one function parse input, call providers, build prompts, and persist data.
Replacing typed contracts with string conventions that can only be verified by manual integration.

Production checklist

Inputs, outputs, and error responses have explicit schemas
External dependencies have timeouts, bounded retries, rate limits, and fallbacks
Logs, metrics, and traces can be correlated to one request
Critical paths have unit tests, integration tests, and offline evaluation samples
Secrets, user content, and provider responses follow least-privilege and privacy rules

Practice

Hands-on exercise

Build queries with abbreviations, numbers, and paraphrases. Compare vector-only, BM25-only, hybrid, and reranked variants with Recall@K and latency curves.

Conclusion

An AI feature becomes maintainable when every arrow on the whiteboard maps to an input, an output, and a failure strategy.

Production MySQL High-Concurrency Architecture

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard note: the goal is not to memorize boxes, but to connect each arrow to a request, a contract, and an operational signal.

Start with a real scenario

Use this situation as the mental model: A feed service handles a hot tenant by caching reads, routing writes to the master, and testing restore before launch.. A useful backend diagram answers four questions: where does input enter, who owns state, which boundary can fail, and how do we know the system is healthy? The flow is Client → Pool → Cache / Master / Replicas → Backup.

Deconstructing the architecture

Contract: define input, output, and error shape before adding implementation detail.
Mechanism: Cache-aside → read cache, write source; Master → writes + binlog. This is the part that determines latency, throughput, and testability.
State and resources: Replicas → read scaling with lag awareness; Sharding → user_0 … user_N. Explain creation, ownership, cleanup, and recovery.
Failure path: Slow Query → EXPLAIN → index tuning; combine it with Backup → restore drill, not just backup files when choosing a fallback, retry, or rollback.

A small implementation boundary

The following sketch is intentionally small. It shows where a production implementation should place validation and ownership rather than pretending that a happy path is enough.

request → bounded resource → validated boundary → observable result

Keep external systems behind an adapter, repository, client, or gateway. The domain layer should depend on a stable contract so tests can use fakes and incidents can be isolated to one boundary.

Engineering decisions for the scenario

Protect dependencies first. Pools, queues, semaphores, caches, and worker counts need explicit limits. An unbounded queue only hides overload.
Preserve correctness. Use an idempotency key, transaction boundary, lock, version check, or schema validation where retries or concurrency can duplicate work.
Optimize with evidence. Use traces, slow-query data, GC pauses, queue depth, or p99 latency before choosing a cache, index, batch size, or concurrency setting.
Make recovery testable. Timeout, retry, rate limit, circuit breaking, rollback, and alerting should be exercised in a drill.

Common mistakes

Drawing only the happy path while omitting timeout, empty data, rejection, rollback, or replica lag.
Treating framework defaults as business contracts and discovering their limits after an upgrade.
Scaling machines before measuring pool exhaustion, lock contention, slow SQL, allocation, or event-loop blocking.
Treating logs as the only observability tool; metrics, traces, sampling, and redaction are also required.

Production checklist

Input, output, and error responses have explicit schemas
Dependencies have timeout, retry budgets, rate limits, and fallback behavior
Pools and queues expose capacity, waiting, and rejected-work metrics
Important state has idempotency, transaction, or recovery semantics
Logs, metrics, and traces share a request or correlation ID
Regression tests, load tests, and failure drills use production-like data

Summary

You understand this whiteboard when you can start from one request, explain every arrow’s contract, state how each risk is handled, and name the signal that would tell you to change the design.

Production RAG Architecture: Permissions, Versions, and Observability

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard: RAG — Production RAG Architecture.

The gap between a RAG demo and production is mostly boundary conditions: who can see a document, how indexes update, how answers cite evidence, and what happens when an upstream is unavailable.

Core mental model

Separate ingestion, indexing, query serving, and evaluation planes. Online traffic reads only the active index while new versions build in the background and switch atomically.

Key mechanics

Enforce permissions during retrieval. Cache keys should include tenant, a permission digest, query-normalization version, and index version. Deletions must reach every replica.

Python example

from dataclasses import dataclass

@dataclass
class QueryContext:
 tenant_id: str
 allowed_collections: set[str]
 index_version: str

def cache_key(query: str, ctx: QueryContext) -> str:
 scopes = ",".join(sorted(ctx.allowed_collections))
 return f"rag:v2:{ctx.tenant_id}:{ctx.index_version}:{scopes}:{normalize(query)}"

Course focus

Engineering practice

Keep business rules in the application layer instead of hiding them in untestable prompts or route handlers.
Add timeouts, bounded retries, and idempotency keys to external calls; retries are not a complete error strategy.
Record a request id, latency, input version, model/index version, and outcome without logging sensitive raw content.
Use a small fixed regression set first, then monitor quality and cost with sampled production traffic.

Common mistakes

Drawing only the happy path and omitting timeouts, empty results, rate limits, and rollback paths.
Letting one function parse input, call providers, build prompts, and persist data.
Replacing typed contracts with string conventions that can only be verified by manual integration.

Production checklist

Inputs, outputs, and error responses have explicit schemas
External dependencies have timeouts, bounded retries, rate limits, and fallbacks
Logs, metrics, and traces can be correlated to one request
Critical paths have unit tests, integration tests, and offline evaluation samples
Secrets, user content, and provider responses follow least-privilege and privacy rules

Practice

Hands-on exercise

Design active and candidate indexes. Simulate a failed publish and a document deletion, verify traffic never receives unauthorized stale data, and persist evidence ids for every answer.

Conclusion

An AI feature becomes maintainable when every arrow on the whiteboard maps to an input, an output, and a failure strategy.

Production Spring Boot Architecture

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard note: the goal is not to memorize boxes, but to connect each arrow to a request, a contract, and an operational signal.

Start with a real scenario

Use this situation as the mental model: A flash-sale service protects the database with cache, bounded writes, idempotency, and asynchronous inventory events.. A useful backend diagram answers four questions: where does input enter, who owns state, which boundary can fail, and how do we know the system is healthy? The flow is Gateway → Service → Cache / DB / MQ → Observability.

Deconstructing the architecture

Contract: define input, output, and error shape before adding implementation detail.
Mechanism: Nginx / Gateway → auth + rate limit; Service → idempotency + transaction. This is the part that determines latency, throughput, and testability.
State and resources: Redis → cache-aside with TTL; MySQL → source of truth. Explain creation, ownership, cleanup, and recovery.
Failure path: RocketMQ → async side effects; combine it with Metrics + Logs + Traces → feedback loop when choosing a fallback, retry, or rollback.

A small implementation boundary

The following sketch is intentionally small. It shows where a production implementation should place validation and ownership rather than pretending that a happy path is enough.

request → bounded resource → validated boundary → observable result

Keep external systems behind an adapter, repository, client, or gateway. The domain layer should depend on a stable contract so tests can use fakes and incidents can be isolated to one boundary.

Engineering decisions for the scenario

Protect dependencies first. Pools, queues, semaphores, caches, and worker counts need explicit limits. An unbounded queue only hides overload.
Preserve correctness. Use an idempotency key, transaction boundary, lock, version check, or schema validation where retries or concurrency can duplicate work.
Optimize with evidence. Use traces, slow-query data, GC pauses, queue depth, or p99 latency before choosing a cache, index, batch size, or concurrency setting.
Make recovery testable. Timeout, retry, rate limit, circuit breaking, rollback, and alerting should be exercised in a drill.

Common mistakes

Drawing only the happy path while omitting timeout, empty data, rejection, rollback, or replica lag.
Treating framework defaults as business contracts and discovering their limits after an upgrade.
Scaling machines before measuring pool exhaustion, lock contention, slow SQL, allocation, or event-loop blocking.
Treating logs as the only observability tool; metrics, traces, sampling, and redaction are also required.

Production checklist

Input, output, and error responses have explicit schemas
Dependencies have timeout, retry budgets, rate limits, and fallback behavior
Pools and queues expose capacity, waiting, and rejected-work metrics
Important state has idempotency, transaction, or recovery semantics
Logs, metrics, and traces share a request or correlation ID
Regression tests, load tests, and failure drills use production-like data

Summary

You understand this whiteboard when you can start from one request, explain every arrow’s contract, state how each risk is handled, and name the signal that would tell you to change the design.

Python asyncio & Concurrency

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard note: the goal is not to memorize boxes, but to connect each arrow to a request, a contract, and an operational signal.

Start with a real scenario

Use this situation as the mental model: A profile page fans out to several services while a semaphore protects the slowest dependency.. A useful backend diagram answers four questions: where does input enter, who owns state, which boundary can fail, and how do we know the system is healthy? The flow is Coroutine → Event Loop → Await I/O → Gather Results.

Deconstructing the architecture

Contract: define input, output, and error shape before adding implementation detail.
Mechanism: async def → coroutine object; create_task → scheduled work. This is the part that determines latency, throughput, and testability.
State and resources: await → yield while I/O is pending; gather / TaskGroup → coordinate tasks. Explain creation, ownership, cleanup, and recovery.
Failure path: Semaphore → bound downstream concurrency; combine it with Cancellation + timeout → cleanup path when choosing a fallback, retry, or rollback.

A small implementation boundary

The following sketch is intentionally small. It shows where a production implementation should place validation and ownership rather than pretending that a happy path is enough.

request → bounded resource → validated boundary → observable result

Keep external systems behind an adapter, repository, client, or gateway. The domain layer should depend on a stable contract so tests can use fakes and incidents can be isolated to one boundary.

Engineering decisions for the scenario

Protect dependencies first. Pools, queues, semaphores, caches, and worker counts need explicit limits. An unbounded queue only hides overload.
Preserve correctness. Use an idempotency key, transaction boundary, lock, version check, or schema validation where retries or concurrency can duplicate work.
Optimize with evidence. Use traces, slow-query data, GC pauses, queue depth, or p99 latency before choosing a cache, index, batch size, or concurrency setting.
Make recovery testable. Timeout, retry, rate limit, circuit breaking, rollback, and alerting should be exercised in a drill.

Common mistakes

Drawing only the happy path while omitting timeout, empty data, rejection, rollback, or replica lag.
Treating framework defaults as business contracts and discovering their limits after an upgrade.
Scaling machines before measuring pool exhaustion, lock contention, slow SQL, allocation, or event-loop blocking.
Treating logs as the only observability tool; metrics, traces, sampling, and redaction are also required.

Production checklist

Input, output, and error responses have explicit schemas
Dependencies have timeout, retry budgets, rate limits, and fallback behavior
Pools and queues expose capacity, waiting, and rejected-work metrics
Important state has idempotency, transaction, or recovery semantics
Logs, metrics, and traces share a request or correlation ID
Regression tests, load tests, and failure drills use production-like data

Summary

You understand this whiteboard when you can start from one request, explain every arrow’s contract, state how each risk is handled, and name the signal that would tell you to change the design.

Python Data Structures & Decorators

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard note: the goal is not to memorize boxes, but to connect each arrow to a request, a contract, and an operational signal.

Start with a real scenario

Use this situation as the mental model: An ETL job streams a large CSV, deduplicates IDs with a set, and wraps retries without hiding failures.. A useful backend diagram answers four questions: where does input enter, who owns state, which boundary can fail, and how do we know the system is healthy? The flow is Choose the right container; wrap behavior explicitly.

Deconstructing the architecture

Contract: define input, output, and error shape before adding implementation detail.
Mechanism: list → ordered mutable sequence; tuple → immutable record-like value. This is the part that determines latency, throughput, and testability.
State and resources: dict → keyed lookup + explicit schema; set → membership / deduplication. Explain creation, ownership, cleanup, and recovery.
Failure path: Decorator → wrapper preserves contract; combine it with Generator → lazy iteration + bounded memory when choosing a fallback, retry, or rollback.

A small implementation boundary

The following sketch is intentionally small. It shows where a production implementation should place validation and ownership rather than pretending that a happy path is enough.

request → bounded resource → validated boundary → observable result

Keep external systems behind an adapter, repository, client, or gateway. The domain layer should depend on a stable contract so tests can use fakes and incidents can be isolated to one boundary.

Engineering decisions for the scenario

Protect dependencies first. Pools, queues, semaphores, caches, and worker counts need explicit limits. An unbounded queue only hides overload.
Preserve correctness. Use an idempotency key, transaction boundary, lock, version check, or schema validation where retries or concurrency can duplicate work.
Optimize with evidence. Use traces, slow-query data, GC pauses, queue depth, or p99 latency before choosing a cache, index, batch size, or concurrency setting.
Make recovery testable. Timeout, retry, rate limit, circuit breaking, rollback, and alerting should be exercised in a drill.

Common mistakes

Drawing only the happy path while omitting timeout, empty data, rejection, rollback, or replica lag.
Treating framework defaults as business contracts and discovering their limits after an upgrade.
Scaling machines before measuring pool exhaustion, lock contention, slow SQL, allocation, or event-loop blocking.
Treating logs as the only observability tool; metrics, traces, sampling, and redaction are also required.

Production checklist

Input, output, and error responses have explicit schemas
Dependencies have timeout, retry budgets, rate limits, and fallback behavior
Pools and queues expose capacity, waiting, and rejected-work metrics
Important state has idempotency, transaction, or recovery semantics
Logs, metrics, and traces share a request or correlation ID
Regression tests, load tests, and failure drills use production-like data

Summary

You understand this whiteboard when you can start from one request, explain every arrow’s contract, state how each risk is handled, and name the signal that would tell you to change the design.

Python Runtime & Basics

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard note: the goal is not to memorize boxes, but to connect each arrow to a request, a contract, and an operational signal.

Start with a real scenario

Use this situation as the mental model: A data pipeline validates records at its boundary and uses profiling to find the real CPU hotspot.. A useful backend diagram answers four questions: where does input enter, who owns state, which boundary can fail, and how do we know the system is healthy? The flow is Python Code → Parser → AST → Bytecode → Interpreter.

Deconstructing the architecture

Contract: define input, output, and error shape before adding implementation detail.
Mechanism: Source → tokens → AST; AST → bytecode instructions. This is the part that determines latency, throughput, and testability.
State and resources: Interpreter loop executes bytecode; int / str / list / dict are objects. Explain creation, ownership, cleanup, and recovery.
Failure path: Dynamic typing resolves at runtime; combine it with Reference counting + cyclic GC reclaim objects when choosing a fallback, retry, or rollback.

A small implementation boundary

The following sketch is intentionally small. It shows where a production implementation should place validation and ownership rather than pretending that a happy path is enough.

request → bounded resource → validated boundary → observable result

Keep external systems behind an adapter, repository, client, or gateway. The domain layer should depend on a stable contract so tests can use fakes and incidents can be isolated to one boundary.

Engineering decisions for the scenario

Protect dependencies first. Pools, queues, semaphores, caches, and worker counts need explicit limits. An unbounded queue only hides overload.
Preserve correctness. Use an idempotency key, transaction boundary, lock, version check, or schema validation where retries or concurrency can duplicate work.
Optimize with evidence. Use traces, slow-query data, GC pauses, queue depth, or p99 latency before choosing a cache, index, batch size, or concurrency setting.
Make recovery testable. Timeout, retry, rate limit, circuit breaking, rollback, and alerting should be exercised in a drill.

Common mistakes

Drawing only the happy path while omitting timeout, empty data, rejection, rollback, or replica lag.
Treating framework defaults as business contracts and discovering their limits after an upgrade.
Scaling machines before measuring pool exhaustion, lock contention, slow SQL, allocation, or event-loop blocking.
Treating logs as the only observability tool; metrics, traces, sampling, and redaction are also required.

Production checklist

Input, output, and error responses have explicit schemas
Dependencies have timeout, retry budgets, rate limits, and fallback behavior
Pools and queues expose capacity, waiting, and rejected-work metrics
Important state has idempotency, transaction, or recovery semantics
Logs, metrics, and traces share a request or correlation ID
Regression tests, load tests, and failure drills use production-like data

Summary

You understand this whiteboard when you can start from one request, explain every arrow’s contract, state how each risk is handled, and name the signal that would tell you to change the design.

RAG Embeddings and Vector Databases: The Engineering Behind Similarity

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard: RAG — RAG Embeddings and Vector Databases.

A vector database does not understand your business automatically. It efficiently finds neighbors in a chosen vector space. The engineering questions are embedding versions, distance metrics, filters, and consistency.

Core mental model

Documents and queries need compatible embedding models and preprocessing. Index records should include embedding_model, dimensions, and corpus_version for rebuilds and rollback.

Key mechanics

Apply metadata filters before or as part of vector search so different tenants, languages, and permission scopes never share the candidate set.

Python example

from uuid import uuid4

record = {
 "id": str(uuid4()),
 "text": "A refund request needs an order id and a reason.",
 "vector": embed("A refund request needs an order id and a reason."),
 "metadata": {"tenant_id": "acme", "language": "en", "embedding_model": "text-embedding-3-small", "corpus_version": "2026-08-21"},
}
# Similarity is never an authorization check.

Course focus

Engineering practice

Keep business rules in the application layer instead of hiding them in untestable prompts or route handlers.
Add timeouts, bounded retries, and idempotency keys to external calls; retries are not a complete error strategy.
Record a request id, latency, input version, model/index version, and outcome without logging sensitive raw content.
Use a small fixed regression set first, then monitor quality and cost with sampled production traffic.

Common mistakes

Drawing only the happy path and omitting timeouts, empty results, rate limits, and rollback paths.
Letting one function parse input, call providers, build prompts, and persist data.
Replacing typed contracts with string conventions that can only be verified by manual integration.

Production checklist

Inputs, outputs, and error responses have explicit schemas
External dependencies have timeouts, bounded retries, rate limits, and fallbacks
Logs, metrics, and traces can be correlated to one request
Critical paths have unit tests, integration tests, and offline evaluation samples
Secrets, user content, and provider responses follow least-privilege and privacy rules

Practice

Hands-on exercise

Build two embedding versions from the same corpus. Record dimensions, index size, Top-K overlap, and Recall@K before deciding whether to switch.

Conclusion

An AI feature becomes maintainable when every arrow on the whiteboard maps to an input, an output, and a failure strategy.

Spring Boot Application Architecture

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard note: the goal is not to memorize boxes, but to connect each arrow to a request, a contract, and an operational signal.

Start with a real scenario

Use this situation as the mental model: An order endpoint keeps HTTP, business rules, and SQL separate so each layer can be tested independently.. A useful backend diagram answers four questions: where does input enter, who owns state, which boundary can fail, and how do we know the system is healthy? The flow is Client → Controller → Service → Repository → Database.

Deconstructing the architecture

Contract: define input, output, and error shape before adding implementation detail.
Mechanism: BeanDefinition → BeanFactory → ApplicationContext; @Controller → transport boundary. This is the part that determines latency, throughput, and testability.
State and resources: @Service → domain orchestration; @Repository → persistence boundary. Explain creation, ownership, cleanup, and recovery.
Failure path: Auto Configuration + Starter; combine it with Embedded Server + externalized config when choosing a fallback, retry, or rollback.

A small implementation boundary

The following sketch is intentionally small. It shows where a production implementation should place validation and ownership rather than pretending that a happy path is enough.

request → bounded resource → validated boundary → observable result

Keep external systems behind an adapter, repository, client, or gateway. The domain layer should depend on a stable contract so tests can use fakes and incidents can be isolated to one boundary.

Engineering decisions for the scenario

Protect dependencies first. Pools, queues, semaphores, caches, and worker counts need explicit limits. An unbounded queue only hides overload.
Preserve correctness. Use an idempotency key, transaction boundary, lock, version check, or schema validation where retries or concurrency can duplicate work.
Optimize with evidence. Use traces, slow-query data, GC pauses, queue depth, or p99 latency before choosing a cache, index, batch size, or concurrency setting.
Make recovery testable. Timeout, retry, rate limit, circuit breaking, rollback, and alerting should be exercised in a drill.

Common mistakes

Drawing only the happy path while omitting timeout, empty data, rejection, rollback, or replica lag.
Treating framework defaults as business contracts and discovering their limits after an upgrade.
Scaling machines before measuring pool exhaustion, lock contention, slow SQL, allocation, or event-loop blocking.
Treating logs as the only observability tool; metrics, traces, sampling, and redaction are also required.

Production checklist

Input, output, and error responses have explicit schemas
Dependencies have timeout, retry budgets, rate limits, and fallback behavior
Pools and queues expose capacity, waiting, and rejected-work metrics
Important state has idempotency, transaction, or recovery semantics
Logs, metrics, and traces share a request or correlation ID
Regression tests, load tests, and failure drills use production-like data

Summary

You understand this whiteboard when you can start from one request, explain every arrow’s contract, state how each risk is handled, and name the signal that would tell you to change the design.

Spring Boot IoC & Dependency Injection

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard note: the goal is not to memorize boxes, but to connect each arrow to a request, a contract, and an operational signal.

Start with a real scenario

Use this situation as the mental model: The same payment use case runs with a sandbox gateway in tests and a real gateway in production.. A useful backend diagram answers four questions: where does input enter, who owns state, which boundary can fail, and how do we know the system is healthy? The flow is Container owns construction; application owns behavior.

Deconstructing the architecture

Contract: define input, output, and error shape before adding implementation detail.
Mechanism: Scan / Register → BeanDefinition; Instantiate → constructor injection. This is the part that determines latency, throughput, and testability.
State and resources: Populate → dependencies are resolved; Initialize → @PostConstruct / proxy. Explain creation, ownership, cleanup, and recovery.
Failure path: Ready → request can use the Bean; combine it with Destroy → cleanup resources when choosing a fallback, retry, or rollback.

A small implementation boundary

The following sketch is intentionally small. It shows where a production implementation should place validation and ownership rather than pretending that a happy path is enough.

request → bounded resource → validated boundary → observable result

Keep external systems behind an adapter, repository, client, or gateway. The domain layer should depend on a stable contract so tests can use fakes and incidents can be isolated to one boundary.

Engineering decisions for the scenario

Protect dependencies first. Pools, queues, semaphores, caches, and worker counts need explicit limits. An unbounded queue only hides overload.
Preserve correctness. Use an idempotency key, transaction boundary, lock, version check, or schema validation where retries or concurrency can duplicate work.
Optimize with evidence. Use traces, slow-query data, GC pauses, queue depth, or p99 latency before choosing a cache, index, batch size, or concurrency setting.
Make recovery testable. Timeout, retry, rate limit, circuit breaking, rollback, and alerting should be exercised in a drill.

Common mistakes

Drawing only the happy path while omitting timeout, empty data, rejection, rollback, or replica lag.
Treating framework defaults as business contracts and discovering their limits after an upgrade.
Scaling machines before measuring pool exhaustion, lock contention, slow SQL, allocation, or event-loop blocking.
Treating logs as the only observability tool; metrics, traces, sampling, and redaction are also required.

Production checklist

Input, output, and error responses have explicit schemas
Dependencies have timeout, retry budgets, rate limits, and fallback behavior
Pools and queues expose capacity, waiting, and rejected-work metrics
Important state has idempotency, transaction, or recovery semantics
Logs, metrics, and traces share a request or correlation ID
Regression tests, load tests, and failure drills use production-like data

Summary

You understand this whiteboard when you can start from one request, explain every arrow’s contract, state how each risk is handled, and name the signal that would tell you to change the design.

Spring Boot Request Lifecycle

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard note: the goal is not to memorize boxes, but to connect each arrow to a request, a contract, and an operational signal.

Start with a real scenario

Use this situation as the mental model: A public API distinguishes authentication errors, validation errors, and downstream timeouts.. A useful backend diagram answers four questions: where does input enter, who owns state, which boundary can fail, and how do we know the system is healthy? The flow is Filter → DispatcherServlet → Controller → Response.

Deconstructing the architecture

Contract: define input, output, and error shape before adding implementation detail.
Mechanism: Filter → trace id / auth / CORS; Handler Mapping → select endpoint. This is the part that determines latency, throughput, and testability.
State and resources: Argument Resolver → bind + validate DTO; Interceptor → timing / policy checks. Explain creation, ownership, cleanup, and recovery.
Failure path: Exception Handler → stable error schema; combine it with Message Converter → JSON response when choosing a fallback, retry, or rollback.

A small implementation boundary

The following sketch is intentionally small. It shows where a production implementation should place validation and ownership rather than pretending that a happy path is enough.

request → bounded resource → validated boundary → observable result

Keep external systems behind an adapter, repository, client, or gateway. The domain layer should depend on a stable contract so tests can use fakes and incidents can be isolated to one boundary.

Engineering decisions for the scenario

Protect dependencies first. Pools, queues, semaphores, caches, and worker counts need explicit limits. An unbounded queue only hides overload.
Preserve correctness. Use an idempotency key, transaction boundary, lock, version check, or schema validation where retries or concurrency can duplicate work.
Optimize with evidence. Use traces, slow-query data, GC pauses, queue depth, or p99 latency before choosing a cache, index, batch size, or concurrency setting.
Make recovery testable. Timeout, retry, rate limit, circuit breaking, rollback, and alerting should be exercised in a drill.

Common mistakes

Drawing only the happy path while omitting timeout, empty data, rejection, rollback, or replica lag.
Treating framework defaults as business contracts and discovering their limits after an upgrade.
Scaling machines before measuring pool exhaustion, lock contention, slow SQL, allocation, or event-loop blocking.
Treating logs as the only observability tool; metrics, traces, sampling, and redaction are also required.

Production checklist

Input, output, and error responses have explicit schemas
Dependencies have timeout, retry budgets, rate limits, and fallback behavior
Pools and queues expose capacity, waiting, and rejected-work metrics
Important state has idempotency, transaction, or recovery semantics
Logs, metrics, and traces share a request or correlation ID
Regression tests, load tests, and failure drills use production-like data

Summary

You understand this whiteboard when you can start from one request, explain every arrow’s contract, state how each risk is handled, and name the signal that would tell you to change the design.

The RAG Basics: From Documents to Cited Answers

Fri, 21 Aug 2026 00:00:00 +0000

Whiteboard: RAG — The RAG Basics.

RAG is not simply pasting a few passages into a prompt. It is a knowledge pipeline whose quality depends on parsing, chunking, recall, prompting, and answer validation.

Core mental model

The offline path is Load → Clean → Chunk → Embed → Index; the online path is Query → Retrieve → Rerank/Filter → Generate → Cite. Version the boundary between them.

Key mechanics

Store source, page, section, and a content hash with every chunk. Pass only context above a relevance threshold and require traceable citations.

Python example

from dataclasses import dataclass

@dataclass
class Chunk:
 text: str
 source: str
 page: int
 chunk_id: str

def make_context(chunks: list[Chunk]) -> str:
 return "\n\n".join(f"[{c.chunk_id}] {c.text} (source={c.source}, page={c.page})" for c in chunks)

Course focus

Engineering practice

Keep business rules in the application layer instead of hiding them in untestable prompts or route handlers.
Add timeouts, bounded retries, and idempotency keys to external calls; retries are not a complete error strategy.
Record a request id, latency, input version, model/index version, and outcome without logging sensitive raw content.
Use a small fixed regression set first, then monitor quality and cost with sampled production traffic.

Common mistakes

Drawing only the happy path and omitting timeouts, empty results, rate limits, and rollback paths.
Letting one function parse input, call providers, build prompts, and persist data.
Replacing typed contracts with string conventions that can only be verified by manual integration.

Production checklist

Inputs, outputs, and error responses have explicit schemas
External dependencies have timeouts, bounded retries, rate limits, and fallbacks
Logs, metrics, and traces can be correlated to one request
Critical paths have unit tests, integration tests, and offline evaluation samples
Secrets, user content, and provider responses follow least-privilege and privacy rules

Practice

Hands-on exercise

Create 30 questions with reference answers and sources. Measure retrieval hit rate, citation correctness, and abstention accuracy—not just whether the answer sounds plausible.

Conclusion

An AI feature becomes maintainable when every arrow on the whiteboard maps to an input, an output, and a failure strategy.

Three-Layer Hallucination Prevention in RAG Systems: A Practical Guide

Wed, 05 Aug 2026 00:00:00 +0000

Hallucination is one of the biggest challenges in deploying LLMs for production use. This guide presents a practical three-layer approach to prevent hallucinations in RAG systems.

Above: Three-layer defense architecture for RAG hallucination prevention — Retrieval Filtering → Prompt Engineering → Output Validation

Understanding Hallucination

Hallucination occurs when LLMs generate plausible-sounding but factually incorrect information. In RAG systems, this typically happens when:

Retrieved context is irrelevant or insufficient
LLMs prioritize their training data over provided context
Models “fill in gaps” with fabricated information

Common Scenarios

Product Information - LLMs inventing prices, specifications, or availability
Medical/Agricultural Advice - Generating incorrect dosages or treatments
Legal/Financial - Creating non-existent regulations or policies

Layer 1: Retrieval Filtering

Ensure only relevant information reaches the LLM:

from langchain.retrievers import ContextualCompressionRetriever
from langchain.retrievers.document_compressors import LLMChainExtractor

class RetrievalFilter:
 def __init__(self, vector_store, llm):
 self.vector_store = vector_store
 self.llm = llm

 def filter_documents(self, query: str, threshold: float = 0.7):
 """Filter retrieved documents by relevance score."""
 # Get initial results
 results = self.vector_store.similarity_search_with_score(query, k=10)

 # Filter by score threshold
 filtered = [
 (doc, score)
 for doc, score in results
 if score >= threshold
 ]

 # Rerank using LLM
 reranked = self.rerank_with_llm(query, filtered)

 return reranked

 def rerank_with_llm(self, query: str, documents):
 """Use LLM to rerank documents by relevance."""
 prompt = f"""Rank these documents by relevance to the query: {query}

Documents:
{[doc.page_content for doc, _ in documents]}

Return the most relevant document indices (comma-separated):"""

 response = self.llm.predict(prompt)
 # Parse and return top documents
 # Implementation details...

Filtering Strategies

Score Thresholding - Remove low-relevance documents
Diversity Filtering - Ensure varied perspectives
Source Verification - Prioritize authoritative sources
Freshness Filtering - Prefer recent information

Layer 2: Prompt Engineering

Design prompts that constrain LLM behavior:

HALLUCINATION_PREVENTION_PROMPT = """You are a helpful assistant. Answer the question based ONLY on the provided context.

Rules:
1. If the context doesn't contain enough information, say "I don't have enough information to answer this question."
2. Never make up information not present in the context.
3. Always cite the source document when possible.
4. If multiple sources conflict, present both perspectives.
5. For numerical data, quote exact figures from the context.

Context:
{context}

Question: {input}

Answer:"""

Prompt Templates by Domain

E-Commerce Products:

ECOMMERC_PROMPT = """You are a product expert. Answer questions about products using ONLY the provided product information.

Rules:
1. Never invent prices, specifications, or availability
2. If product info is incomplete, say "I don't have complete information about this product"
3. Always mention the product name and key features from the data
4. For comparisons, stick strictly to the provided data

Product Information:
{context}

Question: {input}
"""

Agricultural Knowledge:

AGRICULTURAL_PROMPT = """You are an agricultural expert. Answer farming questions using ONLY the provided knowledge base.

Rules:
1. Never invent pesticide dosages or treatment methods
2. If the knowledge base doesn't cover the specific crop/disease, recommend consulting a local expert
3. Always cite the source manual when possible
4. For safety-critical advice, include appropriate warnings

Knowledge Base:
{context}

Question: {input}
"""

Layer 3: Output Validation

Verify LLM responses against source data:

import re
from typing import Dict, List, Tuple

class OutputValidator:
 def __init__(self, llm):
 self.llm = llm

 def validate_response(self, response: str, context: str) -> Dict:
 """Validate LLM response against source context."""
 validation_results = {
 "is_valid": True,
 "issues": [],
 "corrected_response": response
 }

 # Check 1: Numerical claims
 numerical_issues = self.check_numerical_claims(response, context)
 if numerical_issues:
 validation_results["issues"].extend(numerical_issues)

 # Check 2: Entity verification
 entity_issues = self.verify_entities(response, context)
 if entity_issues:
 validation_results["issues"].extend(entity_issues)

 # Check 3: Source citation
 citation_issues = self.check_citations(response, context)
 if citation_issues:
 validation_results["issues"].extend(citation_issues)

 # If issues found, generate corrected response
 if validation_results["issues"]:
 validation_results["corrected_response"] = self.generate_corrected_response(
 response, validation_results["issues"], context
 )
 validation_results["is_valid"] = False

 return validation_results

 def check_numerical_claims(self, response: str, context: str) -> List[str]:
 """Verify numerical values in response exist in context."""
 issues = []

 # Extract numbers from response
 numbers = re.findall(r'\b\d+(?:\.\d+)?(?:%|元|kg|ml)?\b', response)

 for number in numbers:
 if number not in context:
 issues.append(f"Numerical value '{number}' not found in source context")

 return issues

 def verify_entities(self, response: str, context: str) -> List[str]:
 """Verify product/entity names exist in context."""
 issues = []

 # Simple entity extraction (can be enhanced with NER)
 entities = re.findall(r'[A-Z][a-z]+(?:\s[A-Z][a-z]+)*', response)

 for entity in entities:
 if entity.lower() not in context.lower():
 issues.append(f"Entity '{entity}' not found in source context")

 return issues

Implementation Example

Complete three-layer prevention system:

from langchain.chains import LLMChain
from langchain.prompts import ChatPromptTemplate

class HallucinationPreventionSystem:
 def __init__(self, vector_store, llm):
 self.vector_store = vector_store
 self.llm = llm
 self.retrieval_filter = RetrievalFilter(vector_store, llm)
 self.output_validator = OutputValidator(llm)

 def query(self, user_input: str) -> Dict:
 """Process query with three-layer hallucination prevention."""

 # Layer 1: Filter retrieval
 filtered_docs = self.retrieval_filter.filter_documents(user_input)
 context = "\n\n".join([doc.page_content for doc in filtered_docs])

 # Layer 2: Generate response with constrained prompt
 prompt = ChatPromptTemplate.from_template(HALLUCINATION_PREVENTION_PROMPT)
 chain = LLMChain(llm=self.llm, prompt=prompt)
 response = chain.invoke({"context": context, "input": user_input})

 # Layer 3: Validate output
 validation = self.output_validator.validate_response(
 response["text"], context
 )

 return {
 "response": validation["corrected_response"],
 "is_valid": validation["is_valid"],
 "issues": validation["issues"],
 "sources": [doc.metadata.get("source") for doc in filtered_docs]
 }

Measuring Effectiveness

Track key metrics to evaluate your prevention system:

class HallucinationMetrics:
 def __init__(self):
 self.metrics = {
 "total_queries": 0,
 "hallucinations_detected": 0,
 "false_positives": 0,
 "retrieval_accuracy": 0
 }

 def evaluate_response(self, response: str, context: str, ground_truth: str):
 """Evaluate response quality and hallucination presence."""
 self.metrics["total_queries"] += 1

 # Check for hallucinations
 if self.detect_hallucination(response, context):
 self.metrics["hallucinations_detected"] += 1

 # Compare with ground truth
 accuracy = self.calculate_accuracy(response, ground_truth)
 self.metrics["retrieval_accuracy"] += accuracy

 def detect_hallucination(self, response: str, context: str) -> bool:
 """Simple hallucination detection."""
 # Check if response contains information not in context
 response_words = set(response.lower().split())
 context_words = set(context.lower().split())

 # Words in response but not in context
 novel_words = response_words - context_words

 # If too many novel words, likely hallucination
 return len(novel_words) > len(response_words) * 0.3

Metrics Dashboard

Metric	Description	Target
Hallucination Rate	% of responses with hallucinations	< 5%
Retrieval Accuracy	% of relevant documents retrieved	> 90%
Response Quality	User satisfaction score	> 4.5/5
Latency	Response time including validation	< 2s

Conclusion

Preventing hallucinations in RAG systems requires a multi-layered approach:

Retrieval Filtering - Ensure only relevant information reaches the LLM
Prompt Engineering - Constrain LLM behavior with clear instructions
Output Validation - Verify responses against source data

The complete implementation is available on .

Questions? Reach out on or email me at !

Building AI Agents with LangChain: Tool Integration and Multi-Turn Conversations

Fri, 10 Jul 2026 00:00:00 +0000

AI agents are revolutionizing how we build intelligent applications. Unlike simple chatbots, agents can use tools, maintain context, and make decisions. This guide covers building production-ready agents with LangChain.

Above: LangChain Agent architecture — User Input → Reasoning Loop → Tool Calling → Multi-turn Conversation

What are AI Agents?

Agents combine LLMs with tools and reasoning capabilities:

User Input → Agent (Reasoning) → Tool Selection → Tool Execution → Response

Key characteristics:

Tool Usage - Can interact with external systems
Reasoning - Decides which tools to use and when
Memory - Maintains context across interactions
Autonomy - Can chain multiple actions together

LangChain Agent Framework

LangChain provides a powerful framework for building agents:

from langchain.agents import create_openai_tools_agent
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

# Initialize LLM
llm = ChatOpenAI(model="gpt-4", temperature=0)

# Define prompt
prompt = ChatPromptTemplate.from_messages([
 ("system", "You are a helpful assistant with access to tools."),
 ("human", "{input}"),
 ("placeholder", "{agent_scratchpad}")
])

# Create agent
agent = create_openai_tools_agent(llm, tools, prompt)

Custom Tool Development

Build tools specific to your use case:

from langchain_core.tools import tool
from typing import Optional

@tool
def search_knowledge_base(query: str) -> str:
 """Search the private knowledge base for information."""
 # Implementation here
 results = vector_store.similarity_search(query, k=3)
 return "\n".join([doc.page_content for doc in results])

@tool
def get_product_info(product_id: str) -> str:
 """Get detailed product information by ID."""
 # Database query here
 product = db.get_product(product_id)
 return f"Product: {product.name}, Price: {product.price}"

# Register tools
tools = [search_knowledge_base, get_product_info]

Tool Best Practices

Clear Descriptions - Help the agent understand when to use each tool
Error Handling - Gracefully handle tool failures
Input Validation - Validate inputs before processing
Rate Limiting - Prevent abuse of external APIs

Memory Management

Implement conversation memory for context persistence:

from langchain.memory import ConversationBufferMemory
from langchain.chains import ConversationChain

# Initialize memory
memory = ConversationBufferMemory(return_messages=True)

# Create conversation chain
conversation = ConversationChain(
 llm=llm,
 memory=memory,
 verbose=True
)

# Use in agent
agent = create_openai_tools_agent(
 llm, tools, prompt,
 memory=memory
)

Memory Types

Type	Use Case	Trade-offs
Buffer	Simple conversations	Limited context
Summary	Long conversations	Information loss
Vector	Semantic search	Storage overhead
Entity	Entity tracking	Complexity

Multi-turn Conversations

Handle complex conversation flows:

from langchain.agents import AgentExecutor

# Create agent executor
agent_executor = AgentExecutor(
 agent=agent,
 tools=tools,
 verbose=True,
 max_iterations=10, # Prevent infinite loops
 handle_parsing_errors=True
)

# Conversation with context
def chat(user_input: str, conversation_id: str):
 # Load conversation history
 history = load_conversation_history(conversation_id)

 # Run agent
 result = agent_executor.invoke({
 "input": user_input,
 "chat_history": history
 })

 # Save to history
 save_conversation_history(conversation_id, user_input, result["output"])

 return result["output"]

Production Deployment

1. FastAPI Integration

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel

app = FastAPI()

class ChatRequest(BaseModel):
 message: str
 conversation_id: str

class ChatResponse(BaseModel):
 response: str
 conversation_id: str

@app.post("/chat", response_model=ChatResponse)
async def chat_endpoint(request: ChatRequest):
 try:
 response = chat(request.message, request.conversation_id)
 return ChatResponse(
 response=response,
 conversation_id=request.conversation_id
 )
 except Exception as e:
 raise HTTPException(status_code=500, detail=str(e))

2. Streaming Responses

from fastapi.responses import StreamingResponse

@app.post("/chat/stream")
async def chat_stream(request: ChatRequest):
 async def generate():
 async for event in agent_executor.astream_events(
 {"input": request.message},
 version="v1"
 ):
 if event["event"] == "on_chat_model_stream":
 content = event["data"]["chunk"].content
 if content:
 yield f"data: {content}\n\n"
 yield "data: [DONE]\n\n"

 return StreamingResponse(generate(), media_type="text/event-stream")

3. Error Handling

@app.middleware("http")
async def error_handler(request, call_next):
 try:
 response = await call_next(request)
 return response
 except Exception as e:
 logger.error(f"Agent error: {e}")
 return JSONResponse(
 status_code=500,
 content={"error": "Internal server error"}
 )

Real-World Example: Agricultural Q&A Agent

Here’s how I built an agricultural knowledge base agent:

from langchain.agents import create_openai_tools_agent
from langchain.tools import Tool

# Define agricultural tools
tools = [
 Tool(
 name="SearchKnowledgeBase",
 func=search_agricultural_knowledge,
 description="Search private agricultural manuals for crop diseases, treatments, and best practices"
 ),
 Tool(
 name="GeneralAgriculturalKnowledge",
 func=get_general_knowledge,
 description="Get general agricultural information when private knowledge base doesn't have the answer"
 )
]

# System prompt emphasizing private data priority
system_prompt = """You are an agricultural expert assistant.

IMPORTANT: Always search the private knowledge base FIRST. Only use general knowledge if the private search returns no relevant results.

When answering:
1. Cite the source document when possible
2. Be specific about crop names, disease symptoms, and treatments
3. If unsure, say "I recommend consulting a local agricultural expert"
"""

Conclusion

Building AI agents requires:

Tool Design - Clear, well-documented tools
Memory Management - Appropriate memory strategy for your use case
Error Handling - Graceful failure management
User Experience - Streaming and clear feedback

The complete agricultural agent code is available on .

Questions? Reach out on or email me at !

Building a RAG Pipeline with LangChain: From Document Processing to Vector Search

Mon, 15 Jun 2026 00:00:00 +0000

Retrieval-Augmented Generation (RAG) has become the standard approach for building AI applications that need to access specific knowledge bases. This guide walks through building a complete RAG pipeline from scratch using LangChain.

Above: Production RAG Pipeline — Document Ingestion → Embedding → Retrieval → Augmented Generation → Output

What is RAG?

RAG combines the power of large language models with external knowledge retrieval:

User Query → Retrieval → Context + Query → LLM → Answer

This approach solves the core problem of LLMs: they can only answer based on their training data. RAG allows them to access up-to-date, domain-specific, or private information.

Document Processing

The first step is preparing your documents for retrieval:

from langchain.document_loaders import PyPDFLoader
from langchain.text_splitter import TokenTextSplitter

# Load documents
loader = PyPDFLoader("knowledge_base.pdf")
documents = loader.load()

# Split into chunks
text_splitter = TokenTextSplitter(
 chunk_size=500,
 chunk_overlap=50,
 encoding_name="cl100k_base" # OpenAI tokenizer
)

chunks = text_splitter.split_documents(documents)

Key Considerations

Chunk Size - Balance between context and retrieval accuracy
Overlap - Prevent important information from being split across chunks
Metadata - Preserve source information for attribution

Vector Storage

Store embeddings for efficient similarity search:

from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import Redis

# Initialize embeddings
embeddings = OpenAIEmbeddings()

# Create vector store
vector_store = Redis.from_documents(
 documents=chunks,
 embedding=embeddings,
 redis_url="redis://localhost:6379",
 index_name="knowledge_base"
)

Choosing a Vector Store

Store	Performance	Features	Best For
Redis	High	Real-time updates	Production
Pinecone	High	Managed service	Enterprise
FAISS	Medium	Local, free	Development
Chroma	Medium	Simple setup	Prototyping

Retrieval Strategies

Implement smart retrieval with LangChain:

from langchain.retrievers import ContextualCompressionRetriever
from langchain.retrievers.document_compressors import LLMChainExtractor

# Basic similarity search
retriever = vector_store.as_retriever(
 search_type="similarity",
 search_kwargs={"k": 5}
)

# Advanced: Contextual compression
compressor = LLMChainExtractor.from_llm(llm)
compression_retriever = ContextualCompressionRetriever(
 base_compressor=compressor,
 base_retriever=retriever
)

Hallucination Prevention

Critical for production RAG systems:

from langchain.prompts import ChatPromptTemplate

system_prompt = """You are an agricultural expert. Answer questions based ONLY on the provided context. If the context doesn't contain enough information, say "I don't have enough information to answer this question."

Context: {context}
"""

prompt = ChatPromptTemplate.from_messages([
 ("system", system_prompt),
 ("human", "{input}")
])

Three-Layer Prevention

Retrieval Filtering - Only pass high-relevance documents
Prompt Engineering - Force model to cite sources
Output Validation - Verify responses against source data

Streaming Interface

Implement real-time responses with FastAPI:

from fastapi import FastAPI
from fastapi.responses import StreamingResponse
import asyncio

app = FastAPI()

@app.post("/chat")
async def chat(query: str):
 async def generate():
 async for chunk in chain.astream({"input": query}):
 yield f"data: {chunk.content}\n\n"
 await asyncio.sleep(0.01) # Simulate typing
 yield "data: [DONE]\n\n"

 return StreamingResponse(
 generate(),
 media_type="text/event-stream"
 )

Production Considerations

1. Error Handling

try:
 response = await chain.ainvoke({"input": query})
except Exception as e:
 logger.error(f"RAG error: {e}")
 return {"error": "Failed to generate response"}

2. Caching

from langchain.cache import RedisCache

cache = RedisCache(redis_url="redis://localhost:6379")
llm = OpenAI(cache=cache)

3. Monitoring

Track key metrics:

Retrieval accuracy
Response latency
User satisfaction
Hallucination rate

Conclusion

Building a production RAG system requires attention to:

Document Quality - Good preprocessing is crucial
Retrieval Strategy - Balance precision and recall
Hallucination Prevention - Multi-layered approach
User Experience - Streaming and error handling

The complete code is available on .

Questions? Reach out on or email me at !