Capstone Decision Scenarios - MangaAssist

This capstone document shows how to choose among fine-tuning techniques for MangaAssist. It is a decision playbook, not another model recipe.

Core Question

When MangaAssist quality drops, ask:

Is the failure caused by routing, retrieval, ranking, generation, grounding,
alignment, serving constraints, or stale data?

The answer determines which fine-tuning topic is appropriate.

Scenario 1 - User Gets The Wrong Workflow

Example:

"I need a refund for a damaged volume."

Wrong behavior:

• routed to faq or product_question.

Use:

1. intent classifier error analysis,
2. calibration and margin policy,
3. business-weighted error scoring,
4. multi-intent detection if both refund and damaged item apply,
5. sentiment/escalation detector if frustration is present.

Do not start with LoRA. The response model cannot fix a wrong route reliably.

Scenario 2 - User Gets Bad Manga Results

Example:

"Give me quiet healing manga like Yokohama Kaidashi Kikou."

Wrong behavior:

• returns generic best sellers.

Use:

1. embedding adapter,
2. hard-negative mining,
3. cross-encoder reranker,
4. data curation for tone and theme labels,
5. interpretability on embedding neighborhoods.

Scenario 3 - Answer Uses Wrong Facts

Example:

"Is this box set complete?"

Wrong behavior:

• model invents availability or volume count.

Use:

1. retrieval quality check,
2. RAFT if context is retrieved but ignored,
3. data freshness checks,
4. DPO only after factual grounding is stable.

Do not teach volatile catalog facts into model weights unless the use case truly needs it.

Scenario 4 - Answer Feels Off-Brand

Wrong behavior:

• too formal,
• too verbose,
• not spoiler-safe,
• ignores user constraints.

Use:

1. prompt or prefix tuning for light behavior control,
2. LoRA for deeper domain behavior,
3. DPO preference alignment,
4. human evaluation with a MangaAssist rubric.

Scenario 5 - Model Is Good But Too Expensive

Use:

1. quantization-aware training,
2. knowledge distillation,
3. smaller reranker candidate count,
4. MoE or adapter routing only if specialization reduces total cost.

Cost should be evaluated with quality gates, not raw dollars alone.

Technique Selection Table

Final Promotion Checklist

[ ] failure source identified
[ ] dataset matches failure
[ ] baseline and candidate compared
[ ] critical slices reviewed
[ ] latency and cost measured
[ ] business-weighted risk checked
[ ] shadow deployment completed
[ ] rollback path documented

Final Decision

The best MangaAssist fine-tuning strategy is usually a sequence, not a single trick: route correctly, retrieve correctly, ground the answer, align the response, then optimize cost. This keeps the chatbot useful without turning every problem into model-weight surgery.