venkatesh
THE NOTEBOOK · ARCHIVE

Everything I've written, in one place.

Production bugs, system patterns, and lessons I keep coming back to. Full version blogs here.

41 entries · 6 tags · since jan 2026
№055
careersystem-design · 3 min · oct 08, 2026

Interviewer: Build a Task Scheduler

A queue, then a PriorityQueue, then a mistake: remove() on a heap is O(n).

read →
№054
prod-bugbackend · 3 min · sep 30, 2026

2ms in Staging, 4s in Production

A harmless-looking loop hid 200 database trips, and every fix came with a trade-off.

read →
3 Spring Annotations I Can Never Forget
prod-bugbackend · 2 min · sep 28, 2026

3 Spring Annotations I Can Never Forget

@Transactional, @Scheduled, and @Lazy each broke in production because I didn't understand how they work.

read →
Our Consumer Read the Order 12ms Before It Existed
prod-bugbackend · 3 min · sep 23, 2026

Our Consumer Read the Order 12ms Before It Existed

The event was published inside the transaction, so the consumer sometimes beat the commit.

read →
Prod Went Down. Everyone Says Check the Logs. In Which Order?
prod-bugbackend · 2 min · sep 21, 2026

Prod Went Down. Everyone Says Check the Logs. In Which Order?

Logs don't tell you the error. They show you the path to the bug.

read →
30 iPhones in Stock, 44 Orders Confirmed
system-designbackend · 3 min · sep 18, 2026

30 iPhones in Stock, 44 Orders Confirmed

Every step of the checkout code looked correct. Together, under concurrency, they oversold the stock.

read →
What Happens to Your Database When Redis Goes Down?
system-designbackend · 3 min · sep 16, 2026

What Happens to Your Database When Redis Goes Down?

The cache made us fast. It also made the database forget what real traffic looks like.

read →
I Asked AI to Build My DSA Practice and Revision Path
aicareer · 2 min · sep 13, 2026

I Asked AI to Build My DSA Practice and Revision Path

A personalized schedule turns a static problem sheet into a practice, tracking, and revision loop.

read →
My Production Checklist for AI-Generated Code
aibackend · 3 min · sep 10, 2026

My Production Checklist for AI-Generated Code

Functionally correct code can still hide unsafe assumptions about scale, failures, retries, and data.

read →
Friday-Evening Debugging With Codex
aiopinion · 1 min · sep 04, 2026

Friday-Evening Debugging With Codex

When gathering context takes forever, Monday becomes the most reliable debugging strategy.

read →
AI Cut Read Latency and Made Two Write APIs Slower
prod-bugaibackend · 3 min · sep 02, 2026

AI Cut Read Latency and Made Two Write APIs Slower

Three new indexes improved one query by 70%, but the table's total workload paid the bill.

read →
Retries Turned a Four-Minute Outage Into Four Hours
prod-bugsystem-designbackend · 4 min · sep 01, 2026

Retries Turned a Four-Minute Outage Into Four Hours

Fixed delays synchronized clients and multiplied load on the exact service trying to recover.

read →
The Software Engineer Interview-Prep Loop
careeropinion · 1 min · aug 29, 2026

The Software Engineer Interview-Prep Loop

Monday motivation, Wednesday's hard problem, weekend guilt, and another fresh start.

read →
Why Offset Pagination Repeated Page-One Records
prod-bugbackend · 4 min · aug 27, 2026

Why Offset Pagination Repeated Page-One Records

New rows shifted the window, while deep offsets forced the database to walk and discard thousands of rows.

read →
The Interview Was Just: Find the Bugs
careerbackend · 3 min · aug 24, 2026

The Interview Was Just: Find the Bugs

An unfamiliar repository and a 45-minute clock test reading, prioritization, and engineering judgment.

read →
Loading Is Slow; Thinking Is AI-Native
aiopinion · 1 min · aug 20, 2026

Loading Is Slow; Thinking Is AI-Native

A five-second panel is apparently a branding opportunity when the real fix takes time.

read →
a payment succeeded, then failed: handling out-of-order payment callbacks
prod-bugbackend · 5 min · aug 19, 2026

a payment succeeded, then failed: handling out-of-order payment callbacks

Two callbacks landed 200ms apart. Whichever arrived last won — sometimes the wrong one.

read →
AI Moved My Design Review Until After Implementation
aiopinion · 2 min · aug 17, 2026

AI Moved My Design Review Until After Implementation

The questions still matter; I just started letting the agent make assumptions before I reviewed them.

read →
Never Put Lombok @Data on a JPA Entity
prod-bugbackend · 4 min · aug 04, 2026

Never Put Lombok @Data on a JPA Entity

A generated hashCode changed after persistence and made the same object disappear inside a HashSet.

read →
A Cache Is More Than the Happy Path
system-designbackend · 3 min · aug 03, 2026

A Cache Is More Than the Happy Path

The real design question is what happens when cached data is empty, stale, or unavailable.

read →
TCP vs UDP: Safe Delivery or Lower Latency?
system-designbackend · 2 min · aug 01, 2026

TCP vs UDP: Safe Delivery or Lower Latency?

A memorable comparison of coordination, retries, delivery guarantees, and speed.

read →
A Perfectly Optimized Objective With Imperfect Constraints
aiopinion · 3 min · jul 24, 2026

A Perfectly Optimized Objective With Imperfect Constraints

An AI sandbox escape illustrates a familiar engineering lesson: objectives need guardrails.

read →
Why @Transactional Failed During Self-Invocation
prod-bugbackend · 3 min · jul 22, 2026

Why @Transactional Failed During Self-Invocation

The annotation described the transaction, but a bypassed Spring proxy never applied it.

read →
Polling vs Long Polling vs SSE vs WebSockets
system-designbackend · 3 min · jul 17, 2026

Polling vs Long Polling vs SSE vs WebSockets

A practical way to choose the simplest real-time protocol your feature actually needs.

read →
One @Transactional Annotation Doubled Our API Latency
prod-bugbackend · 3 min · jul 16, 2026

One @Transactional Annotation Doubled Our API Latency

The database was fast; we were holding its connections while waiting on the network.

read →
Why Java Strings Are Objects, Not Character Arrays
backendopinion · 2 min · jul 13, 2026

Why Java Strings Are Objects, Not Character Arrays

The extra ceremony around Java strings buys immutability, pooling, cached hashes, and thread safety.

read →
One Redis Key Expired. 100 Requests Hit Our DB.
prod-bugsystem-design · 4 min · jun 08, 2026

One Redis Key Expired. 100 Requests Hit Our DB.

Cache stampedes are invisible until they're not.

read →
Circular Dependency in Spring
system-designprod-bug · 3 min · jun 01, 2026

Circular Dependency in Spring

Circular dependency isn't a Spring problem. It's a design problem.

read →
Debugging AI's Code
aiopinion · 2 min · may 25, 2026

Debugging AI's Code

Only one of us sleeps after deployment

read →
The ORDER BY Trap
prod-bugsystem-design · 2 min · may 25, 2026

The ORDER BY Trap

Looked free. Cost 8 seconds.

read →
Hundreds of Payments Broken in 1ms
prod-bugsystem-design · 3 min · may 25, 2026

Hundreds of Payments Broken in 1ms

Two webhooks. Same row. One vanished.

read →
The System Design Pyramid
system-designcareer · 2 min · may 25, 2026

The System Design Pyramid

You don't need 25 systems. You need 8, done right.

read →
Redis Keys vs Scan — Which to Use?
prod-bugsystem-design · 2 min · may 25, 2026

Redis Keys vs Scan — Which to Use?

One command froze production for 2 seconds.

read →
When Round Robin Drowned Our Agents
prod-bugsystem-design · 2 min · may 25, 2026

When Round Robin Drowned Our Agents

Fair on paper. Chaos in production.

read →
'@Transactional Was There. Rollback Wasn''t.'
prod-bugsystem-design · 3 min · may 25, 2026

'@Transactional Was There. Rollback Wasn''t.'

Self-invocation is the sneakiest Spring trap.

read →
The DB Migration That Took Down Payments
prod-bugsystem-design · 2 min · may 25, 2026

The DB Migration That Took Down Payments

Code deploys gradually. DB changes are instant. Never mix them.

read →
How Redis Betrayed Us With Locks
prod-bugsystem-design · 2 min · may 25, 2026

How Redis Betrayed Us With Locks

Lock expired mid-job. Users got duplicate notifications.

read →
User Reuploaded. Still Saw the Old Image.
prod-bugsystem-design · 2 min · may 25, 2026

User Reuploaded. Still Saw the Old Image.

Webhook order is unpredictable. Version numbers aren't.

read →
10,000 DB Queries Per API Call
prod-bugsystem-design · 2 min · may 01, 2026

10,000 DB Queries Per API Call

N+1 hiding in plain sight.

read →
Circuit Breakers Alone Won't Save You
system-designprod-bug · 4 min · feb 10, 2026

Circuit Breakers Alone Won't Save You

Timeouts + breakers + smart ramp-up — three layers, one pattern.

read →
Half Our Events Just Vanished
prod-bugaisystem-design · 3 min · jan 20, 2026

Half Our Events Just Vanished

AI writes the code. Architecture decisions are still on you.

read →
copied!