7. Third-Party & Supply Chain Risk Management

Why This Matters for MangaAssist

MangaAssist looks like a single chatbot, but the production response depends on a long chain of upstream suppliers:

• AWS managed services such as Bedrock, OpenSearch, DynamoDB, S3, Lambda, API Gateway, WAF, and CloudWatch
• Open-source Python libraries such as boto3, langchain, fastapi, pydantic, and transformers
• Build infrastructure such as CI runners, base images, package registries, signing systems, and deployment templates
• Prompt templates, guardrail policies, model IDs, feature flags, and AppConfig versions
• External data sources such as product catalog feeds, order APIs, policy documents, and review content

That means a supply chain problem does not need to start in our repository. A bad package release, silent model behavior change, compromised data feed, or weak build policy can still reach customers if we do not treat every upstream input as untrusted until verified.

For an LLM system, the supply chain is wider than code. It includes:

• Code supply chain: libraries, base runtimes, Lambda layers, shared utilities
• Model supply chain: foundation model version, tokenizer behavior, safety model behavior, moderation model behavior
• Prompt supply chain: system prompts, tool schemas, guardrail configs, allowlists, feature flags
• Data supply chain: RAG documents, catalog metadata, review feeds, order data, policy content
• Deployment supply chain: IaC modules, build containers, artifact stores, signing keys, rollout automation

If any one of those changes unexpectedly, the chatbot can become unsafe even when the business logic is unchanged.

Threat Model at a Glance

The key design principle is simple: every upstream artifact must be both approved and observable. If we cannot answer "where did this artifact come from, who approved it, what version is running, and how do we roll it back?" then the supply chain is not under control.

High-Level Design (HLD)

Supply Chain Security Architecture

flowchart TB
    subgraph Intake["1. Supplier Intake Plane"]
        Req["New dependency / vendor request"]
        Review["Security review<br/>owner, license, data access, blast radius"]
        Approve{"Approved?"}
        Catalog["Approved dependency catalog"]
        Reject["Reject or exception denied"]
        Req --> Review --> Approve
        Approve -->|Yes| Catalog
        Approve -->|No| Reject
    end

    subgraph Build["2. Build and Provenance Plane"]
        Repo["Source repo<br/>code + prompts + IaC"]
        Mirror["Private package mirror<br/>approved packages only"]
        CI["CI build runner"]
        Scan["SCA + license scan + secret scan<br/>IaC scan + unit/integration tests"]
        SBOM["SBOM + dependency diff<br/>provenance attestation"]
        Sign["Artifact signing"]
        Artifact["Signed deployment artifact"]

        Repo --> CI
        Catalog --> CI
        Mirror --> CI
        CI --> Scan --> SBOM --> Sign --> Artifact
    end

    subgraph Deploy["3. Deployment Policy Plane"]
        Policy["Promotion policy<br/>signature, SBOM, CVE, eval, approvals"]
        Canary["Canary rollout<br/>security probes + golden tests"]
        Prod["Production alias"]
        Artifact --> Policy --> Canary --> Prod
    end

    subgraph Runtime["4. Runtime Enforcement Plane"]
        Orch["Chatbot Orchestrator"]
        Config["Signed AppConfig<br/>prompt + model + guardrail versions"]
        Bedrock["AWS Bedrock"]
        Search["OpenSearch"]
        DDB["DynamoDB"]
        APIs["Catalog / Order / Policy APIs"]
        Logs["Audit logs + metrics + alerts"]

        Prod --> Orch
        Config --> Orch
        Orch --> Bedrock
        Orch --> Search
        Orch --> DDB
        Orch --> APIs
        Orch --> Logs
    end

HLD Explanation

The HLD is split into four control planes because supply chain risk does not live in one place:

• The supplier intake plane decides what is allowed into the ecosystem in the first place.
• The build and provenance plane turns reviewed source into a traceable artifact with evidence attached.
• The deployment policy plane ensures only trusted and tested artifacts reach customers.
• The runtime enforcement plane limits blast radius when a third-party dependency still fails or behaves badly.

This separation matters because each control plane answers a different question:

• Intake: "Should we trust this dependency at all?"
• Build: "What exactly did we build, and from which inputs?"
• Deploy: "Has this artifact met all release policy requirements?"
• Runtime: "If the supplier fails, how much damage can it cause before we detect and contain it?"

Trust Boundaries and Ownership

Shared responsibility is real, but it is not a defense. AWS owns service infrastructure. We still own:

• what we send to Bedrock
• which model IDs we allow
• how we validate retrieval data
• which libraries we install
• how we gate builds and deployments
• how we respond when a vendor fails

Data Flow: From Commit to Customer Response

sequenceDiagram
    participant Eng as Engineer
    participant Git as Source Repo
    participant CI as CI Runner
    participant Mirror as Private Mirror
    participant Scan as Security Scanners
    participant Sign as Signer
    participant Policy as Deploy Policy
    participant RT as Runtime
    participant Vendors as Third-Party Services
    participant Obs as Audit and Monitoring

    Eng->>Git: Commit code / prompt / dependency change
    Git->>CI: Trigger build
    CI->>Mirror: Resolve pinned dependencies
    Mirror-->>CI: Approved versions + hashes
    CI->>Scan: Run SCA, license, IaC, secret, test suite
    Scan-->>CI: Findings + dependency diff
    CI->>Sign: Generate SBOM, provenance, artifact signature
    Sign-->>CI: Signed artifact
    CI->>Policy: Submit artifact + evidence
    Policy-->>CI: Allow or block promotion
    Policy->>RT: Deploy canary then production
    RT->>Vendors: Call Bedrock, OpenSearch, APIs
    RT->>Obs: Emit artifact version, model ID, prompt version, index version
    Vendors-->>Obs: Health, latency, error signals

What Evidence Is Produced in This Flow

Every release should leave a paper trail:

• Lockfile digest
• Dependency diff from previous release
• SBOM
• Build provenance attestation
• Artifact signature
• Security scan results
• Evaluation suite result
• Deployment ticket or approval record
• Runtime metadata tying customer traffic to artifact version and config version

If one of those artifacts is missing, we intentionally fail closed and do not promote the release.

Low-Level Design (LLD)

LLD 1: Dependency Intake and Approval Workflow

New third-party packages do not enter production through an ad hoc pip install. They enter through an approval path with explicit ownership.

stateDiagram-v2
    [*] --> Requested
    Requested --> Review
    Review --> LicenseCheck
    LicenseCheck --> SecurityCheck
    SecurityCheck --> BlastRadiusCheck
    BlastRadiusCheck --> Decision
    Decision --> Approved
    Decision --> Rejected
    Approved --> ActiveUse
    ActiveUse --> PeriodicReview
    PeriodicReview --> ActiveUse
    PeriodicReview --> Retired

Required Intake Metadata

Example Dependency Record

dependency:
  name: langchain
  source: pypi
  exact_version: 0.1.12
  owner: chatbot-platform
  business_use: orchestration wrappers for retrieval and prompt execution
  data_touched:
    - prompt_templates
    - retrieved_context
    - model_output
  network_required: false
  license: MIT
  criticality: high
  fallback_plan: custom orchestrator wrapper can replace high-risk chain logic
  patch_sla: 24h_for_critical

LLD 2: Build-Time Dependency Resolution

The most important preventive rule is: production builds never install directly from the public internet.

flowchart LR
    Req["requirements.in"] --> Lock["Compile lockfile<br/>exact versions + hashes"]
    Lock --> Mirror["Install from private mirror only"]
    Mirror --> Test["Unit + integration tests"]
    Test --> Scan["SCA + license + secret + IaC scan"]
    Scan --> SBOM["Generate SBOM + dependency diff"]
    SBOM --> Sign["Sign artifact + provenance"]
    Sign --> Publish["Publish immutable artifact"]

Implementation Details

• Direct dependencies are declared in requirements.in.
• CI compiles an exact lockfile with hashes.
• CI installs from a private mirror or artifact proxy that contains only approved packages.
• The build fails if:
• a package is missing a pinned hash
• a new transitive dependency appears without approval
• a critical or high CVE is open beyond policy
• the SBOM cannot be generated
• the build provenance is missing

Example Build Commands

pip-compile --generate-hashes requirements.in -o requirements.lock
pip install --require-hashes --no-deps -r requirements.lock --index-url $PRIVATE_MIRROR_URL
pip-audit -r requirements.lock --strict
cyclonedx-py requirements -i requirements.lock -o sbom.json

Example Lockfile Pattern

langchain==0.1.12 \
  --hash=sha256:<approved-hash-1> \
  --hash=sha256:<approved-hash-2>

urllib3==2.0.7 \
  --hash=sha256:<approved-hash-3> \
  --hash=sha256:<approved-hash-4>

The combination of exact version plus hash is important. Version pinning alone prevents accidental upgrades, but it does not prove the artifact bytes are the ones we approved.

LLD 3: Deployment Promotion Policy

Even a successful build is not enough. Promotion to production is blocked unless the artifact also has the required security evidence.

flowchart TD
    Artifact["Candidate artifact"] --> Sig{"Signature present?"}
    Sig -->|No| Block1["BLOCK deployment"]
    Sig -->|Yes| SBOM{"SBOM present?"}
    SBOM -->|No| Block2["BLOCK deployment"]
    SBOM -->|Yes| CVE{"Critical / High CVE open?"}
    CVE -->|Yes| Block3["BLOCK deployment<br/>or require emergency exception"]
    CVE -->|No| Eval{"Security and regression eval passed?"}
    Eval -->|No| Block4["BLOCK deployment"]
    Eval -->|Yes| Canary["Canary rollout"]
    Canary --> Healthy{"Healthy?"}
    Healthy -->|No| Rollback["Rollback to previous signed version"]
    Healthy -->|Yes| Promote["Promote to production alias"]

Promotion Policy Fields

{
  "require_signature": true,
  "require_sbom": true,
  "require_dependency_diff": true,
  "max_open_critical_cves": 0,
  "max_open_high_cves": 0,
  "require_private_mirror_install": true,
  "require_security_eval_suite": true,
  "require_model_allowlist_match": true
}

The policy intentionally fails closed. Manual overrides are rare, time-boxed, and recorded. If engineering can silently bypass the gate, the gate is theater.

LLD 4: Runtime Dependency Isolation

Once the artifact is deployed, we assume a supplier can still fail. Runtime controls are there to reduce blast radius.

flowchart LR
    subgraph Runtime["Production Runtime"]
        Orch["Lambda Orchestrator"]
        VPCE["VPC endpoints / private service access"]
        Config["Signed config snapshot"]
        Logs["Audit logs"]
    end

    Orch --> VPCE
    VPCE --> Bedrock["Bedrock"]
    VPCE --> Search["OpenSearch"]
    VPCE --> DDB["DynamoDB"]
    VPCE --> S3["S3"]

    Config --> Orch
    Orch --> Logs

    Orch -. blocked .-> PublicPyPI["Public package registries"]
    Orch -. blocked .-> RandomAPI["Unapproved public APIs"]

Runtime Rules

Example Runtime Metadata Logged Per Request

{
  "artifact_version": "chatbot-2026-03-24.4",
  "git_sha": "a1b2c3d4",
  "lockfile_digest": "sha256:...",
  "model_id": "approved-primary-model",
  "prompt_version": "prompt-v18",
  "kb_index_version": "kb-2026-03-24",
  "region": "us-east-1"
}

LLD 5: Model, Prompt, and Knowledge-Base Supply Chain

This is the LLM-specific part that most teams miss. A model swap, prompt update, or retrieval corpus change is a supply chain event and must be treated like a code deployment.

flowchart TB
    subgraph PromptSupply["Prompt and Config Supply Chain"]
        PromptRepo["Prompt repo / guardrail config"]
        Eval["Offline eval suite<br/>security + quality + regression"]
        AppCfg["Signed AppConfig version"]
        PromptRepo --> Eval --> AppCfg
    end

    subgraph DataSupply["Knowledge and Data Supply Chain"]
        Sources["Catalog / policy docs / reviews"]
        Quarantine["Quarantine bucket"]
        Validate["AV scan + schema validation<br/>PII scan + source provenance"]
        Index["Versioned vector index"]
        Sources --> Quarantine --> Validate --> Index
    end

    subgraph RuntimeUse["Runtime"]
        Orch["Chatbot Orchestrator"]
        Model["Approved model allowlist"]
    end

    AppCfg --> Orch
    Index --> Orch
    Model --> Orch

Concrete Rules for LLM-Specific Artifacts

• Prompt changes require code review, evaluation, and versioned rollout exactly like code.
• Model IDs are pinned to an approved allowlist. Runtime blocks unapproved model IDs.
• Retrieval indexes are versioned. Re-indexing is a deployment event, not a background side effect.
• Documents are ingested through a quarantine step before they are chunked or embedded.
• Guardrail thresholds live in versioned config, not hardcoded runtime edits.

Example Startup Verification Logic

APPROVED_MODELS = {"approved-primary-model", "approved-failover-model"}

def startup_preflight(config: dict, build_meta: dict) -> None:
    if not build_meta.get("signed"):
        raise RuntimeError("Unsigned artifact cannot start")

    if config["model_id"] not in APPROVED_MODELS:
        raise RuntimeError("Unapproved model selected")

    if not config.get("prompt_version"):
        raise RuntimeError("Prompt version missing")

    if not config.get("kb_index_version"):
        raise RuntimeError("Knowledge index version missing")

This preflight is intentionally simple. Its purpose is not to do deep cryptography inside application code. Its purpose is to make unsafe configuration impossible to ignore.

Control Matrix

Detailed Implementation Patterns

Pattern 1: Private Mirror Instead of Direct Internet Install

We do not let CI or production fetch Python packages directly from public registries. Instead:

• A separate intake process imports approved packages into the private mirror.
• The mirror records package name, version, and hash.
• Builds can only resolve dependencies from the mirror.
• Emergency patches are applied by updating the mirrored approved version, not by bypassing the mirror.

This is the single biggest control against surprise dependency changes.

Pattern 2: Dependency Diff as a First-Class Release Artifact

Most teams generate an SBOM but never read it. We generate both:

• full SBOM for audit and incident response
• human-readable dependency diff between the previous and current release

That diff answers questions like:

• Did this release add a new transitive package?
• Did a package jump multiple versions?
• Did a dependency move from direct to transitive ownership?
• Did a package start requiring new permissions or network behavior?

Pattern 3: Config Is Supply Chain, Not Just "Settings"

For MangaAssist, the following are treated as release artifacts:

• system prompt text
• guardrail thresholds
• allowlists for competitor mentions and domain terms
• model routing policy
• retrieval index version
• fallback region configuration

If those values change, customer behavior changes. So they go through the same review and evidence pipeline as code.

Pattern 4: Fail-Closed Release Gates, Fail-Open Customer Fallbacks

This distinction matters:

• Release pipeline: fail closed. Missing signature, missing SBOM, or risky dependency change blocks release.
• Customer experience: fail open to safe fallback. If Bedrock degrades or OpenSearch times out, respond with a bounded fallback rather than a broken or unsafe answer.

Security controls should stop unsafe software from shipping, but once users are live, resilience controls should degrade safely instead of crashing noisily.

Scenarios I Handled

Scenario 1: Framework Upgrade Introduced Unreviewed Transitive Packages

Context: A developer proposed a langchain upgrade to improve prompt orchestration. The direct package change looked small, but the lockfile diff introduced seven new transitive packages.

sequenceDiagram
    participant Dev as Developer
    participant Repo as Source Repo
    participant CI as CI Pipeline
    participant Mirror as Private Mirror
    participant Policy as Dependency Policy
    participant Sec as Security Team

    Dev->>Repo: PR bumps langchain version
    Repo->>CI: Build triggered
    CI->>Mirror: Resolve exact dependencies
    Mirror-->>CI: 7 new transitive packages
    CI->>Policy: Compare against approved catalog
    Policy-->>CI: Block - unapproved packages detected
    CI-->>Sec: Alert with dependency diff
    Sec-->>Dev: Require review or alternate implementation

Detection: The build failed before tests even started because the dependency policy checks package count changes and rejects any new package not already present in the approved catalog.

Why this mattered: A major risk in modern Python ecosystems is not only a known CVE. It is sudden growth in the dependency tree. Each new package expands:

• the maintainer trust set
• the potential CVE surface
• the code executed at import time
• the future patch burden

Investigation:

1. We reviewed the transitive packages individually.
2. Two were low-risk utility packages.
3. Three were poorly maintained and had weak release hygiene.
4. Two were community extensions that were not necessary for our usage.

Decision: We rejected the framework upgrade and kept the existing version. We then extracted the one useful behavior we needed into our own wrapper instead of inheriting a much larger dependency tree.

Implementation lesson: In LLM applications, orchestration frameworks can expand quickly. Treat framework upgrades as architecture changes, not library bumps.

Scenario 2: Critical CVE in a Transitive Dependency Required Emergency Patch

Context: Nightly scanning flagged a high-severity issue in urllib3, which was pulled indirectly by the AWS SDK path. Even though the exploit path was low-likelihood for our traffic pattern, the policy required remediation because the dependency was present in production.

flowchart LR
    Feed["CVE feed update"] --> SBOM["Match against release SBOMs"]
    SBOM --> Affected{"Running release affected?"}
    Affected -->|No| Close["No action"]
    Affected -->|Yes| Patch["Create emergency patch branch"]
    Patch --> Build["Rebuild from patched lockfile"]
    Build --> Test["Run security + regression tests"]
    Test --> Canary["Canary deploy"]
    Canary --> Healthy{"Healthy?"}
    Healthy -->|No| Rollback["Rollback and reassess"]
    Healthy -->|Yes| Prod["Promote to production"]

Detection: The nightly SBOM-to-CVE matcher identified that three active releases still referenced the vulnerable version.

Response flow:

1. Opened a security incident.
2. Confirmed affected services by SBOM, not by guesswork.
3. Updated the lockfile to the patched version.
4. Rebuilt the artifact from clean CI with the same source commit except for the lockfile change.
5. Re-ran regression suites focused on: - Bedrock invocation - DynamoDB session handling - OpenSearch retrieval - outbound AWS SDK calls
6. Canary deployed for 30 minutes.
7. Promoted globally after error and latency metrics stayed flat.

Why SBOM mattered: Without an SBOM, we would have spent hours asking basic questions:

• Are we affected?
• Which release is affected?
• Which service pulled the dependency in?
• Did the patched version already reach canary?

With SBOMs, the incident started with facts instead of assumptions.

Implementation lesson: Supply chain readiness is mostly about response speed. A team that can answer "which release contains this package?" within minutes will consistently outperform a team that starts by searching repositories manually.

Scenario 3: Bedrock Model Behavior Drift After an Upstream Change

Context: Response quality suddenly shifted. The model started producing longer answers, more speculative bundle suggestions, and a slightly higher rate of price-related fallback blocks. No application code or prompt change had been deployed that day.

sequenceDiagram
    participant Eval as Scheduled Eval Runner
    participant Model as Bedrock Primary Model
    participant Metrics as Drift Dashboard
    participant AppCfg as AppConfig Router
    participant Runtime as Orchestrator

    Eval->>Model: Run golden prompts + adversarial probes
    Model-->>Eval: New response distribution
    Eval->>Metrics: Compare with baseline
    Metrics-->>AppCfg: Drift threshold exceeded
    AppCfg-->>Runtime: Route new sessions to approved failover model
    Runtime-->>Metrics: Emit recovery metrics

Detection:

• Daily eval suite reported a 6% drop in price-grounding accuracy.
• Response length P95 increased significantly.
• Guardrail stage 2 block rate rose above normal baseline.

Why this is a supply chain issue: We did not change our code, but the upstream model dependency effectively changed the runtime behavior of the product. For an LLM system, that is equivalent to an upstream library changing semantics.

Response:

1. Froze new prompt and config promotions.
2. Switched new sessions to the approved failover model via AppConfig.
3. Increased sampling for online review.
4. Re-ran the full evaluation suite against both primary and failover models.

Permanent control added:

• Model routing policy now requires a daily live-endpoint evaluation run.
• Production dashboards track response length, refusal rate, price-block rate, and topic drift by model ID.
• Canary traffic is segmented by model ID so regressions are attributable quickly.

Implementation lesson: Model version pinning helps, but it is not enough on its own. You also need behavioral canaries, because the customer experiences behavior, not just an ID string.

Scenario 4: Bedrock Regional Degradation Exposed a Failover Gap

Context: The primary region experienced elevated Bedrock timeout rates. The rest of the application stack was healthy, but generation latency degraded sharply.

flowchart LR
    User["User request"] --> Router["Region router"]
    Router -->|"Primary healthy"| East["us-east-1<br/>primary Bedrock path"]
    Router -->|"Primary degraded"| West["us-west-2<br/>warm standby path"]
    East --> Health["Health monitor"]
    West --> Health
    Health --> Router
    Router --> Cache["Cached response / safe fallback if both unhealthy"]

Detection: CloudWatch alarms triggered on Bedrock error rate and latency thresholds. The critical point is that this was not an application bug. It was a third-party dependency availability problem.

Response:

1. Retried transient failures with capped exponential backoff.
2. Routed new sessions to the warm standby region after the health threshold was exceeded.
3. Served cached answers and deterministic fallbacks for simple intents during the transition window.

Tradeoff: Warm standby costs more than a single-region setup, but it significantly reduces the blast radius of vendor-side regional incidents.

Implementation lesson: Availability is part of supply chain risk. A dependency can be honest and secure and still fail in a way that harms customers.

Architecture Decisions and Tradeoffs

What I Would Measure

Supply chain controls are only real if they are measurable.

Follow-Up Questions and Deep-Dive Answers

Follow-Up Question 1: What is your single most important supply chain control?

Deep-dive answer: Preventing direct installs from public registries during build and runtime is the highest-value control. If CI can fetch whatever is latest from the internet, then version pinning, review, and incident response all become weaker. A private mirror forces every package through an intake path, which means we can record ownership, exact hashes, approval status, and patch history. It also makes builds reproducible because the dependency bytes are stable, not just the version strings.

Follow-Up Question 2: How do you secure transitive dependencies that engineers never imported directly?

Deep-dive answer: I do not rely on developer awareness for transitive dependencies. I rely on lockfiles, SBOMs, and dependency diffs. The lockfile tells me exactly what is installed. The SBOM lets me answer audit and CVE questions across releases. The dependency diff tells me what changed in the tree when a direct dependency moved. That combination makes transitive risk visible. Without it, teams only notice transitive dependencies after a CVE lands, which is too late.

Follow-Up Question 3: Why do you treat prompts and model IDs as supply chain artifacts?

Deep-dive answer: Because they change runtime behavior as much as code does. A prompt edit can create a new information leak path. A model swap can change safety behavior, tone, or grounding. A guardrail threshold change can move the system from safe to permissive in one config push. If those artifacts are not versioned, reviewed, tested, and logged, then the application has an invisible release channel outside the normal SDLC. That is exactly the kind of hidden change path that causes incidents.

Follow-Up Question 4: How do you prove that the running production artifact came from reviewed source?

Deep-dive answer: I want a chain of evidence from source commit to runtime metadata. That means:

• source commit SHA attached to the build
• signed build provenance
• SBOM tied to the build output
• immutable artifact version or digest
• deployment record showing who promoted it
• runtime logs that include artifact version, model ID, prompt version, and index version

When an incident happens, I should be able to point at one request and answer exactly which artifact and config produced it.

Follow-Up Question 5: What do you do when a zero-day lands and no patch exists yet?

Deep-dive answer: First, determine exploitability against our actual architecture. Then add compensating controls immediately. Depending on the dependency, that can mean disabling a feature path, reducing IAM permissions, tightening network egress, forcing traffic to a safer code path, or temporarily routing to fallback behavior. The important point is not to confuse "no patch available" with "nothing we can do." Supply chain resilience is about containment as much as patching.

Follow-Up Question 6: Why not auto-merge all dependency updates to stay current?

Deep-dive answer: Because freshness and safety are not the same thing. In a conventional web app, a patch upgrade may be mostly a regression risk. In an LLM application, framework changes can alter prompt construction, token budgeting, tool invocation, or memory behavior in ways that directly affect guardrails. I support automation for visibility and proposal generation, but promotion still needs policy gates and risk-based review. The right model is automated detection plus controlled adoption, not blind auto-merge.

Follow-Up Question 7: How do you handle vendor lock-in without pretending you are multi-cloud on day one?

Deep-dive answer: I accept practical lock-in where it buys speed, but I isolate the highest-risk seams. For MangaAssist, that means interface-driven model invocation, structured logs, portable data models, versioned prompt/config artifacts, and explicit failover logic. I do not try to make every component cloud-neutral from day one. I focus on the components where switching cost or outage risk would materially hurt the business, especially the foundation model path and critical data dependencies.

Follow-Up Question 8: What evidence would you show an auditor or incident responder?

Deep-dive answer: I would show the approved dependency catalog, the lockfile for the affected release, the SBOM, build provenance, the artifact signature, deployment approvals, runtime metadata, alert history, and rollback record. That evidence proves not only what was running, but also whether our controls actually executed. Auditors do not want general statements like "we scan dependencies." They want release-specific proof.

Key Lessons

1. The supply chain for an LLM application is wider than package management. Models, prompts, guardrail configs, and retrieval corpora are all release artifacts.
2. Determinism is a security feature. Exact versions, hashes, signatures, and immutable artifacts make both rollback and forensics faster.
3. Behavioral drift matters as much as CVEs. A model that changes behavior without a code change is still a supply chain event.
4. Evidence has to exist before the incident. You cannot generate trustworthy SBOMs, signatures, or provenance after the fact.
5. Resilience is part of supply chain risk management. Vendor outages and regional degradation can hurt users even without compromise.

Cross-References

• Prompt-level hardening: 01-prompt-injection-defense.md
• PII controls at the response layer: 02-pii-protection-data-privacy.md
• Post-generation guardrails: 03-guardrails-pipeline-deep-dive.md
• Incident handling and containment: 05-incident-response-forensics.md
• RAG poisoning and model-specific risks: 06-ml-specific-threats.md
• System architecture overview: ../04-architecture-hld.md
• Scalability and reliability: ../11-scalability-reliability.md