MangaAssist Interview Pack - Very Hard With Hints

Level: Very Hard
How to use: Good answers here should connect product behavior, architecture, safety, operations, and governance in one narrative.

Incident Map

graph TD
    A[Traffic Spike] --> B[More requests]
    B --> C[Higher downstream fan-out]
    C --> D[Higher LLM load]
    D --> E[Latency increase]
    E --> F[Timeouts]
    F --> G[More retries]
    G --> H[Dependency stress]
    H --> I[Circuit breakers open]
    I --> J[Fallback paths]
    J --> K[Lower experience quality]
    K --> L[Escalation rate rises]

Interview Questions With Hints

Principal Engineer

Imagine recommendation, promotions, and reviews are all degraded at the same time during a major manga release. How should the orchestrator degrade the customer experience without making the system feel broken?

Hint: Keep the assistant responsive, remove unavailable features explicitly, fall back to popular titles or search links, and avoid pretending the missing systems succeeded.

How would you explain the difference between a graceful fallback and a misleading fallback in this project?

Hint: Graceful means transparent reduction in capability; misleading means fabricated confidence or stale facts.

If guardrail block rate suddenly doubles after a prompt change, how would you isolate whether the issue is prompt quality, policy strictness, or data quality?

Hint: Compare versions, inspect blocked samples, segment by intent, and separate input-side from output-side causes.

Applied Scientist

How would you build a feedback loop that improves intent routing, retrieval quality, and recommendation explanation quality without retraining blindly on noisy user feedback?

Hint: Treat feedback as weak signal, combine it with human review and outcome metrics, and retrain only after curation.

What evaluation set would you create for this chatbot so that you can detect regressions in RAG relevance, hallucination control, and user resolution?

Hint: Build a balanced golden set across discovery, FAQ, post-purchase, edge cases, abuse attempts, and escalation scenarios.

Security Engineer

How would you respond if an audit found that raw prompts containing sensitive user text were retained longer than the policy intended?

Hint: Contain, assess impact, correct retention paths, clean historical data where required, and tighten controls and audits.

What are the most realistic abuse patterns for a shopping chatbot like this, and which ones are more dangerous than they first appear?

Hint: Prompt injection, scraping, social engineering, spam, and adversarial escalation patterns. Data exfiltration and trust erosion matter more than nuisance spam.

SRE

Design a canary release strategy for a new LLM or prompt version so that quality regressions are caught before they affect most users.

Hint: Shadow or low-percent rollout, compare quality and latency metrics, and have automated rollback thresholds.

What dashboards and alerts would you put in front of the on-call engineer so they can distinguish between model issues, dependency issues, and traffic issues quickly?

Hint: Use separate panels for dependency health, model latency/errors, intent mix, guardrail behavior, and business impact.

Distinguished Engineer

Which single architectural assumption in MangaAssist would you challenge first if this system had to expand from one niche storefront to many retail domains?

Hint: Challenge single-domain assumptions in taxonomy, knowledge ownership, integration contracts, and orchestration logic.

Recovery Sequence

sequenceDiagram
    participant User
    participant Edge as Gateway
    participant Orch as Orchestrator
    participant Dep as Downstream Services
    participant Safe as Guardrails/Fallback
    participant Agent as Human Queue

    User->>Edge: Send message
    Edge->>Orch: Request
    Orch->>Dep: Parallel lookups
    Dep-->>Orch: Partial failures
    Orch->>Safe: Check fallback policy
    alt recoverable
        Safe-->>User: Reduced but safe answer
    else not recoverable
        Safe->>Agent: Escalate with context
        Agent-->>User: Handoff path
    end