Fine-tuning for the wrong reason
Teams treat fine-tuning as a way to inject knowledge — when the symptom was actually a prompt or retrieval problem. RAG would have done the job in a fraction of the time, at none of the maintenance cost.
Expertise / Custom & Domain-Specific Models
Smaller, faster, yours. Built only when it earns the cost.
Most enterprises don’t need a fine-tuned model — but the ones that do are reaching for the wrong tool, the wrong technique, and a lifecycle they can’t sustain. We test the escalation ladder before climbing it, build small sovereign models when the math actually works, and engineer the revalidation discipline that keeps them relevant as base models evolve.
Built by senior teams from the University of Edinburgh, Amazon, Stanford & the Alan Turing Institute
The gap
Why most “custom model” projects underdeliver.
Fine-tuning is for form, not facts — but most teams reach for it as a knowledge-injection mechanism, build the wrong thing, and watch the fine-tune silently degrade the next time the base model updates. By 2027, Gartner projects enterprise use of small task-specific models will run 3× that of LLMs. The hard question isn’t how to fine-tune. It’s when — and most teams answer it too fast.
No revalidation discipline
The base model gets an update; the adapter degrades silently. Without quarterly revalidation tied to a structured eval set, the model rots in production and nobody sees it until users do.
Privacy and sovereignty solved for the demo, not for audit
A private endpoint is not a sovereign deployment. The full lifecycle — training data, eval set, weights, inference, monitoring — has to land on infrastructure you control, with an audit trail a regulator will accept.
Over-tuning to small evals
A 200-example eval set looks great in the report. Production data exposes the brittleness within weeks. Without domain-specific evals and regression tests sized to the real distribution, you ship a model that benchmarks well and fails quietly.
How we build
The right escalation. Not the default escalation.
Six steps for deciding when to customize and how to do it without creating maintenance debt your team can’t carry.
- 01 · Test the escalation ladder. Start with prompt engineering and RAG. Most “we need fine-tuning” turns out to be a prompt or retrieval problem in disguise — and the lower rungs ship in days, not quarters.
- 02 · Decide where customization earns its keep. Behavior, structured output, refusal patterns, tone — yes. Knowledge injection — no. Latency or unit cost at frontier-model scale — sometimes. We make the call against your data, not a vendor’s pitch deck.
- 03 · Choose the right technique. LoRA, QLoRA, or DoRA for most cases. Full fine-tune rarely. DPO, ORPO, or KTO for alignment — PPO-RLHF has been displaced in production by 2026.
- 04 · Generate or curate training data. Synthetic data from teacher models where volume matters. Expert curation where domain language matters. Both, audited.
- 05 · Pick the right base. Small language models (1B–10B) for narrow tasks. Open-weight where sovereignty matters. A frontier API only where it’s genuinely the right tool — not the default one.
- 06 · Engineer the lifecycle. Quarterly revalidation, base-model drift monitoring, structured evals tied to the deployment release, and a deprecation playbook from day one. Customization is a commitment, and we treat it like one.
Most teams escalate too fast. We test the ladder before climbing it — and engineer the lifecycle to keep the model relevant as base models evolve.
What we ship
Custom model patterns we ship.
Three production patterns — each engineered for a different shape of customization work.
01
Domain-specific SLMs
Fine-tuned 7B-class models on your task — cheaper, faster, sovereign. A fine-tuned 7B legal SLM hits 94% on contract tasks against 87% for the frontier alternative, at 10–30× lower cost. We pick the base, build the data pipeline, run the alignment, and engineer the eval set.
02
Distilled task models
Teacher–student distillation from a frontier model into a small, deployable model that holds the same accuracy on your task. The right pattern when you need frontier behavior at edge-deployable cost and latency.
03
On-prem & sovereign deployments
Full-stack custom-model lifecycle on infrastructure you control — for EU AI Act obligations, sector regulators, and sovereignty mandates. Training, weights, evals, inference, monitoring — all inside your perimeter, with the audit trail to prove it.
The production layer
What we engineer around the model.
The model is one piece. These are the layers that make it trustworthy after the launch deck closes.
01
Training data pipeline
Synthetic generation from teacher models, expert curation, augmentation, deduplication, and contamination checks. The eval set and the training set are designed together — not as an afterthought.
02
Evaluation harness
Domain-specific evals sized to the real distribution. Regression tests on every release. Drift monitoring against the base model and against your production data. Without it, you don’t know when the model is wrong before users do.
03
Lifecycle & revalidation
Quarterly revalidation cycles, base-model drift detection, structured release gates, and a deprecation playbook. A fine-tune is a living artifact, and the lifecycle is what keeps it earning its cost.
Where it earns its keep
Where a custom model pays for itself.
The strongest cases share a shape: narrow, repeatable, high-stakes work where a small model on infrastructure you control beats a frontier API on cost, latency, or sovereignty — and earns its keep.
✓
Contract & policy analysis
Domain SLMs outperform frontier models on narrow legal tasks — and run on infrastructure firms can defend in a regulator meeting.
✓
Clinical-note structuring
Privacy and sovereignty drive on-prem deployment. The model never leaves the hospital network, and the evidence trail is built in.
✓
Code generation for proprietary stacks
Custom models trained on your internal code outperform generic copilots on the languages, frameworks, and patterns that matter inside your codebase.
✓
Claims adjudication & support routing
Narrow, repeatable tasks where small task-specific models ship cheaper, run faster, and degrade more predictably than a frontier API.
✓
Sovereign-cloud copilots
Regulated sectors needing EU-only or jurisdiction-specific deployment — finance, defense, public sector — where the residency of the weights matters as much as the residency of the data.
✓
Latency-critical inference
Real-time CX, trading, and operations workloads where frontier-API latency is unacceptable and a small model on dedicated hardware is the only path that works.
FAQ
Questions we get asked before the project starts.
Do we actually need a custom model, or is this a prompt/RAG problem?
Usually a prompt or RAG problem. Roughly four out of five “we need fine-tuning” conversations resolve on the lower rungs of the ladder — better prompts, better retrieval, better grounding. We test the ladder against your data before recommending you climb it, because customization is a commitment with a lifecycle attached.
When does an SLM outperform a frontier model?
When the task is narrow and you have examples. A fine-tuned 7B legal SLM hits 94% on contract tasks against 87% for the frontier alternative, at roughly an order of magnitude lower cost. Narrow tasks, latency budgets, and sovereignty requirements are the three signals that an SLM is the right pattern.
Should we use LoRA, full fine-tuning, distillation, or DPO?
Almost always LoRA, QLoRA, or DoRA — full fine-tunes are rare in 2026 production. Distillation is the right answer when you need frontier behavior at edge-deployable cost. DPO, ORPO, or KTO for alignment work — PPO-RLHF has been displaced in production. The technique falls out of the use case, not the other way around.
How do we deploy on-prem or in a sovereign environment?
The whole lifecycle moves, not just the inference endpoint. Training data, evals, weights, inference, and monitoring all sit on infrastructure you control — typically VPC, dedicated hardware, or sovereign cloud (Together AI, Fireworks, Modal, Databricks Mosaic, or your own GPUs). The audit trail is engineered alongside it, because a regulator will ask.
How often do we need to revalidate?
Quarterly, at minimum. Base models update, your data drifts, and the eval set you launched with stops representing the real distribution. We build the revalidation cadence into the project so it’s a scheduled engineering activity, not a fire drill triggered by a user complaint.
Do we own the model you build with us?
Yes. The weights, the training data pipeline, the eval harness, the lifecycle runbook — all yours. We design every engagement so your team can revalidate, retrain, and redeploy independently. Enablement is built into the project, not sold as an upsell.
Where to start
Where to start.
Bring us the use case driving the “do we need a custom model?” question.
We test the escalation ladder against your data — prompt, RAG, PEFT, distill — benchmark each rung for accuracy, cost, and latency, and deliver a recommendation with the trade-offs spelled out. If a custom model is the right call, you leave with the deployment architecture, lifecycle plan, and technique choice. If it isn’t, you save the quarter.
What you get: a rung-by-rung benchmark on your data with accuracy and cost numbers; a recommendation on technique (LoRA/QLoRA/DoRA, distillation, full fine-tune, or none); a target deployment architecture matched to your sovereignty and latency constraints; a lifecycle plan with revalidation cadence and base-model drift monitoring; and one workshop with your engineering and data leads. Led by a senior consultant — fixed scope, fixed fee.
Book a Custom Model Review →Start the conversation
Ready to find out whether a custom model is the right answer?
A 30-minute conversation with a senior consultant. Bring the use case driving the question — a workflow where frontier API cost is unsustainable, a domain where a generic model doesn’t speak the language, a regulator asking where the weights live. We’ll tell you whether customization is the right rung, what the trade-offs look like, and what a Custom Model Review would surface.
Book a Custom Model Review →