Continual Learning Scenarios - MangaAssist
Continual learning keeps MangaAssist current as manga trends, catalog inventory, policies, and user language change. The challenge is improving on new data without forgetting old behavior that still matters.
When This Topic Matters
Use continual learning when:
- new manga releases change search and recommendation patterns,
- seasonal promotions alter traffic mix,
- users adopt new slang,
- policy updates change support answers,
- a retrained model regresses on old intents.
Scenario 1 - Monthly Intent Refresh
The intent classifier already handles 10 known intents. Each month, collect:
- low-confidence requests,
- corrected routing labels,
- support outcome labels,
- rejected OOD clusters,
- sampled high-confidence predictions for regression checks.
Training strategy:
| Component | Purpose |
|---|---|
| new labeled data | learn current behavior |
| rehearsal buffer | preserve older intent patterns |
| golden set | block regressions on critical workflows |
| drift report | decide whether retraining is needed |
Promotion gate:
| Metric | Gate |
|---|---|
| new-month accuracy | improves by >= 1 point |
| old golden-set accuracy | no drop over 0.3 points |
| rare-class recall | no critical regression |
| business-weighted harm | improves or stays within budget |
Scenario 2 - New Release Drift
Example event:
A major anime adaptation launches. Users suddenly search for a newly popular title,
its spin-offs, and related merch.
Risk:
- product discovery traffic spikes,
- embedding retrieval overweights old best sellers,
- recommendation model misses new comparisons.
Continual update:
- add recent clicks and purchases,
- add editorial pairs for the new title,
- refresh retrieval adapter monthly,
- keep an old-title validation set to avoid popularity collapse.
Scenario 3 - Policy Change Without Forgetting
If return policy changes, support answers need current behavior, but the model must still handle older orders correctly.
Do not solve this only with fine-tuning. Use retrieval-grounded policy context first. Continual fine-tuning can teach answer structure, but the exact policy should come from current retrieved documents.
Failure Modes
| Failure | Detection | Fix |
|---|---|---|
| catastrophic forgetting | old golden-set drop | rehearsal buffer and lower LR |
| trend overfitting | model over-routes to new title | balanced sampling |
| stale support behavior | old policy appears in answer | RAFT with current policy docs |
| unnecessary retraining | cost rises without quality gain | require drift plus accuracy evidence |
Production Log
{
"event": "continual_learning_decision",
"window": "2026-04",
"kl_divergence": 0.031,
"sampled_accuracy": 0.898,
"golden_set_accuracy": 0.921,
"decision": "retrain_with_rehearsal"
}
Final Decision
For MangaAssist, continual learning should be evidence-driven. Retrain when drift creates measurable harm, then protect old behavior with rehearsal data, golden sets, and business-weighted gates.