Conversation Memory And Session Storage

Covers Q5, Q8, Q9, Q16, Q22, Q28, Q35, Q36, Q39.

What The Interviewer Is Testing

• Whether you understand why session state exists and what must be persisted.
• Whether you can reason about DynamoDB limits instead of naming it casually.
• Whether you know when summarization beats truncation and when Redis helps.

Deep Dive

Session Item Shape

Expected session attributes usually include:

• session_id
• customer_id
• turns
• page_context
• created_at
• updated_at
• ttl
• turn_count
• last_intent
• optional rolling summary

Why TTL Exists

TTL is a product decision tied to session usefulness, cost, and privacy posture. In this design, 24 hours makes sense because shopping assistance is short-lived and stale state is actively harmful.

Summarization Versus Truncation

Summarization is better when older turns still encode preferences, dislikes, abandoned paths, or disambiguation history. Truncation is cheaper, but it destroys semantic continuity.

DynamoDB Constraints That Matter

• 400 KB item-size limit.
• Partition-key distribution and GSI hotspot risk.
• Read and write amplification when the turns list grows.

When Redis Helps

Redis is useful as a low-latency cache for active sessions, but it is not a free replacement for durable history. The best answer is usually hybrid:

• Read hot sessions from Redis.
• Persist durable truth in DynamoDB.
• Expire cache aggressively.

Branching Conversations

Flat turn lists are fine for MVP. If users can jump back and branch from earlier turns, move toward parent-child turn references or store a summary plus branch pointer metadata.

Strong Answer Pattern

• "The memory layer preserves context needed for better routing and better prompts."
• "DynamoDB is the durable source of truth; Redis is a latency optimization."
• "Summarization is a token-budget tool, not just a storage optimization."

Scenario 1: Session Item Is Near 400 KB

Primary Prompt

One session has unusually long messages and the DynamoDB item is approaching 350 KB. What do you do now, and what do you change permanently?

Follow-Up 1

Why is waiting until 399 KB a bad plan?

Follow-Up 2

Would you store each turn as a separate item instead of a list? What do you gain and lose?

Follow-Up 3

How do you preserve enough context for the LLM if you stop storing full history in one item?

Strong Answer Markers

• Adds proactive summarization before hard limits.
• Mentions API-level message length constraints.
• Knows the trade-off between simple reads and scalable turn storage.
• Preserves a rolling summary plus recent turns for prompt assembly.

Scenario 2: Hot Partition Or Hot GSI

Primary Prompt

The session table is fine, but a customer_id GSI is getting uneven write pressure for heavy users. How do you reason about that?

Follow-Up 1

Why is a single session_id usually not the real hot-partition risk?

Follow-Up 2

Would you shard the GSI or accept eventual consistency and lower query frequency?

Follow-Up 3

What monitoring would confirm you fixed the issue?

Strong Answer Markers

• Distinguishes base-table access from GSI hotspot behavior.
• Mentions adaptive capacity but does not treat it as magic.
• Considers write sharding or changed access patterns.
• Talks about consumed capacity, throttle rate, and latency.

Scenario 3: Product Wants Conversation Branching

Primary Prompt

Users want to say, "Go back to the earlier recommendation path." How would you add branching without wrecking the current design?

Follow-Up 1

What is the simplest MVP solution that avoids a full tree rewrite?

Follow-Up 2

If you do build full branching, what key would each turn need?

Follow-Up 3

How would prompt assembly know which branch is active?

Strong Answer Markers

• Starts with a low-complexity reconstruction approach.
• Moves to parent_turn_id only if the product really needs persistent branching.
• Keeps active-branch prompt assembly explicit.

Red Flags

• Saying DynamoDB has no practical size constraints.
• Treating Redis as durable truth.
• Recommending truncation without discussing loss of preference context.
• Designing full tree history before proving user need.

Two-Minute Whiteboard Version

Draw three boxes:

1. Durable session store.
2. Optional hot-session cache.
3. Prompt builder consuming summary plus recent turns.