Tech Stack, Libraries, and Engineering Innovation
How I selected, evaluated, and championed high-performance tooling for MangaAssist, Amazon JP's AI-powered manga shopping assistant.
This folder documents the engineering philosophy behind MangaAssist's technology choices: the libraries we adopted, the tradeoffs we navigated, and the culture of continuous innovation I drove as the senior engineer on the team.
Document Index
| # | Document | Focus |
|---|---|---|
| 1 | 01-detailed-tech-stack.md | Complete technology stack with rationale for every layer |
| 2 | 02-open-source-libraries.md | Open-source high-performance libraries and how they transformed the chatbot |
| 3 | 03-mlflow-llm-observability.md | MLflow tracing for LLM observability, setup, impact, and lessons learned |
| 4 | 04-innovation-and-tradeoffs.md | Innovation approach: how I continuously scouted tools and rapidly evaluated tradeoffs |
| 5 | vllm-deep-dive/README.md | Scenario pack, low-level design notes, and interview prep for the vLLM decision |
Related Deep Dive
For the project-wide MLflow walkthrough, see ../MLflow/README.md. That folder expands the observability note into scenario deep dives, Bedrock and AWS service integrations, and a low-level implementation guide.
For the vLLM inference story, see ./vllm-deep-dive/README.md. That pack expands the vLLM discussion into production scenarios, low-level implementation choices, and interview-ready answers.
Why This Matters
Building a production LLM chatbot at Amazon scale is not just about writing code. It is about making hundreds of technology decisions under uncertainty. For every component, I asked:
- Does a high-quality open-source option exist?
- What are the quantified tradeoffs?
- How does it fit our constraints?
- What is the migration path if it fails?
This systematic approach saved us money in inference costs and kept us ahead of the rapidly evolving GenAI tooling landscape.
Interview Walkthrough Arc
When discussing tech stack decisions in a system design interview:
Start: "Here's the tech stack I selected and why"
-> Deep-dive: "For inference, I championed vLLM because..."
-> Tradeoff: "I evaluated vLLM vs TGI vs Triton - here are the numbers"
-> Impact: "This reduced our GPU costs and P99 latency"
-> Observability: "I instrumented everything with MLflow Tracing to prove it"
-> Culture: "I built a tradeoff framework the team now uses for all decisions"