---
name: Dynamic LoRA Router
description: Serve one open-source base model as many specialists — an intent classifier hot-swaps task-specific LoRA adapters (PRD authoring, code refactoring, HR, and more) at inference time, with cached adapters, a confidence threshold, and a base-model fallback, so a single local model gets specialized depth per task without misrouting or over-claiming general expertise.
audience: AI engineer · product / design ops · platform lead · PM
---

# Dynamic LoRA Router

## What this is
A method for making one base model behave like a bench of specialists: a lightweight router reads the request's intent and swaps in the matching LoRA adapter (e.g., a PRD-writing adapter, a code-refactor adapter, an HR adapter) before generation — keeping adapters hot in memory so the switch is fast and the base weights stay shared.

## What this is NOT
- **Not a claim of frontier-level general intelligence.** A LoRA adds *task-shaped depth* to a base model; it does not make a small open model an all-knowing expert. The framing is "specialized for this task," not "a genius at everything."
- **Not misrouting-proof.** The router is a classifier that can be wrong; the method requires a confidence threshold, a sensible default, and a graceful fallback to the base model — a bad route degrades, it doesn't fail silently.
- **Not a substitute for the domain's own guardrails.** An HR or legal-flavoured adapter still routes real decisions to the relevant human/skill; the adapter shapes *style and task competence*, not authority.

## When to use
Serving many task specializations from one local model to save memory and cost; adding a new capability as a LoRA instead of a new model; routing mixed workloads (PRDs, refactors, support) through a single endpoint; keeping specialization on-prem for data residency.

## Operating principle
One base, many lenses — chosen on evidence, with a fallback. The router picks the adapter from a classified intent, but a low-confidence route falls back to the base model rather than forcing a wrong specialist, and the adapter is treated as a competence boost, not a licence to overclaim.

## Capabilities
- **Intent routing** — Goal: the right adapter per request. Method: a fast intent classifier (embedding/keyword/small-model) maps the request to an adapter or the base, with a confidence threshold and a default; ambiguous or novel intents fall back rather than guess. Output: a routing policy with thresholds + fallback. Quality bar: below-threshold intent goes to the base model, not a misfit specialist; routing decisions are logged for tuning.
- **Adapter serving & hot-swap** — Goal: fast, memory-efficient specialization. Method: shared base weights with LoRA adapters loaded/cached in VRAM, swapped per request (batched-multi-adapter serving where the runtime supports it), sized to the hardware. Output: an adapter-serving setup + a "why this fits the VRAM" note. Quality bar: switch latency is measured on the target hardware and reported as a number, not asserted as "instant"; adapters that don't fit degrade to sequential load, not OOM.
- **Adapter lifecycle & evaluation** — Goal: specialists that actually help. Method: train/version each adapter on task data, evaluate against the base model on that task (does it beat base?), version and roll back, watch for regressions. Output: an adapter registry + per-task eval cards. Quality bar: an adapter ships only if it beats the base model on its task; each carries an eval, not a vibe.

## A worked example
"One local model that writes PRDs, refactors code, and drafts HR notes." → The router classifies each request and hot-swaps the matching cached adapter; a request it can't confidently place ("summarize this contract") scores below threshold and falls back to the base model rather than forcing the HR adapter. Switch latency is measured (e.g., reported in ms on the given GPU, not claimed), each adapter has an eval showing it beats base on its task, and the HR adapter still routes an actual employment decision to a human. One model, many competent lenses — honestly bounded.

## Guardrails & escalation
Consequential decisions in a specialized domain (HR, legal, financial) → the domain's human/skill, never the adapter alone. Latency/throughput claims → measured on the real hardware, labelled. Adapter that doesn't beat base → not shipped. Serving at scale / multi-tenant isolation → ML-serving engineering.

## References
LoRA (Hu et al.); multi-adapter serving (e.g., S-LoRA / Punica-style batching) as a pattern reference; intent-classification and routing practice; per-task evaluation against a base-model baseline. Verify switch latency and quality gains on your own model, adapters, and hardware.