Frontier Models Are Becoming the Teacher, Not the Runtime

An evidence index of 13 companies moving high-volume AI work to smaller models, open weights, routing and deterministic systems.

Executive answer

Enterprises are not abandoning frontier models. They are changing where frontier intelligence sits in the system.

In the first wave of generative AI, one powerful model often did everything. It interpreted the request, searched for context, generated the answer, checked the answer and sometimes decided what to do next. That was a sensible way to validate demand quickly. It is usually an expensive way to process predictable production traffic.

The production evidence now points to a different architecture. A frontier model becomes the teacher, evaluator, prototype engine or fallback. Smaller language models, open-weight models, specialized speech systems, narrow classifiers and deterministic rules carry the repeated work. A router decides which path each request needs.

The cases in this index cover 13 companies and several kinds of workloads:

• Ito moved a high-volume software QA agent from GPT-5.3-codex to MiniMax M3 and reported a 55 percent lower median model cost per run.
• Checkr selected a fine-tuned Llama 3 8B model over GPT-4 for difficult background-check classifications and estimated an $800 monthly SLM cost versus$7,000 to $12,000 for the GPT alternatives.
• Dropbox adapted an OpenAI o3 relevance judge to an open-weight model and generated 10 to 100 times more labels at the same cost.
• Pinterest reported similar performance from adapted open models at less than 10 percent of proprietary-model cost for relevant visual and multimodal work.
• Replit reported the same output quality at 65 percent lower cost using intelligent model routing.
• Meta designed a production funnel in which deterministic rules handle about 85 percent of traffic and the LLM handles the ambiguous 15 percent.
• Siemens switched global-search agents to a better-fitting model and reported 70 percent lower cost, 300 percent faster search and 11 percent higher satisfaction.
• CarbonTrail reported an open-model emissions pipeline that cost 88 percent less than a comparable GPT-4 plus embeddings architecture.
• Sully.ai reported more than 90 percent lower inference cost after moving clinical-agent workloads from closed to open models.
• Decagon reported nearly 6 times lower cost per voice turn from a multi-model stack built around fine-tuned open models.
• Boosted.ai reported 90 percent lower model cost after replacing a nearly $1 million annual general-purpose model path with a smaller finance-tuned model.
• StudyFetch reported roughly 10 times lower cost on its largest inference workload after moving from managed transcription to specialized speech AI.
• Forethought reported up to 66 percent lower multi-model serving cost and about 80 percent lower cloud cost for small customer-specific classifiers.

These percentages do not use the same denominator, and most have not been independently audited. They should not be averaged. They do, however, show a repeatable operating pattern across search, finance, healthcare, speech, software engineering, customer support, emissions analysis, privacy classification and visual discovery.

The pattern is simple:

Use the least expensive system that passes the quality, latency, security and governance gates for the specific request.

That sentence is the practical meaning of an enterprise AI model-routing strategy. It is also the commercial reason to build models around enterprise workflows instead of forcing every workflow through the same general-purpose API.

The Frontier Model Downshift Index

How to read the index

The word migration can hide several different decisions. Ito and Sully.ai document real production moves from named or closed model services. Checkr tested GPT-4 and selected a smaller model for production. CarbonTrail compares its architecture with a GPT-4 alternative. Meta replaced many model calls with rules. Replit routes requests among models. Dropbox still retains o3 where the highest judge quality is valuable.

Those are related patterns, not identical events. The common decision is to stop treating maximum general capability as the default requirement for every unit of work.

Why the change is happening now

Production volume exposes the frontier-model tax

At pilot volume, model price is often a rounding error. At production volume, a tiny inefficiency multiplies across every request, retry, retrieved document, reasoning token, guardrail call and judge call.

The basic token invoice is:

monthly model cost = calls x (input tokens x input rate + output tokens x output rate)

But enterprises do not buy tokens. They buy accepted business outcomes. The better denominator is:

cost per accepted outcome = all model, retrieval, infrastructure, review, retry and operations cost / accepted outcomes

An outcome might be an accepted transcript hour, a correctly classified record, an issue resolved without escalation, a search session that finds the right document, or an agent action completed without rollback.

This is why a cheaper token can still produce an expensive workflow. Ito's open model used 2.51 times more input tokens and 2.18 times more output and reasoning tokens on a matched task sample. It still cut median cost because each token was much cheaper. Its step failures also rose. A token-efficiency leaderboard would have made the wrong decision. Production invoices and accepted outcomes made the right one.

The AI FinOps approach should therefore track at least:

• cost per accepted outcome;
• first-pass acceptance rate;
• retries per successful task;
• human review and correction minutes;
• cache-read share and cache-hit rate;
• p50, p95 and p99 latency;
• escalation rate to a larger model;
• infrastructure utilization and idle capacity;
• severe-error rate, not only average accuracy;
• business value per completed task.

Model quality has become workload-specific

A frontier model is optimized to perform broadly across many tasks. An enterprise workload is usually narrower. Checkr needed a classifier that could distinguish among a constrained set of background-check categories. StudyFetch needed accurate lecture transcription. Meta needed a policy-aware asset classifier. Dropbox needed a relevance score from 1 to 5 in valid JSON.

Once the task, data and acceptance metric are stable, specialization becomes possible. The smaller model does not need to know everything. It needs to perform one bounded function with predictable quality.

Pinterest CEO Bill Ready states the idea directly: “Compact, fit-for-purpose models can meaningfully out-perform general purpose LLMs on specific tasks.” Pinterest Engineering then describes a mixed model strategy in which proprietary services help teams prototype while internally hosted systems absorb scaled, differentiated work.

Enterprises now have evaluation data

Early pilots rarely have a representative evaluation set. Production systems create one through user corrections, rejected outputs, escalations, resolved cases and audit findings. That evidence changes the build-versus-buy calculation.

Checkr had years of labeled classification data. Dropbox had human relevance scores and explanations. Meta built reviewed truth sets and per-class quality gates. Ito metered every agent step and could compare its production eras. The enterprise data asset was not simply a document corpus. It was a record of what a correct outcome looks like.

That is why the evaluation set becomes an AI moat. A company can use a frontier model to create candidate labels, review and improve those labels, then train or adapt a smaller system to reproduce the stable behavior. The frontier model helps manufacture the runtime that will eventually handle most requests.

Open models improved, but closed models did not disappear

The market evidence is mixed, and the counterevidence matters. Investor surveys from Menlo Ventures and the a16z survey of 100 enterprise CIOs show broad use of multiple models and providers. They also show that proprietary models remain central to enterprise adoption. Open weights have not simply replaced closed APIs across the market.

There are good reasons:

• frontier APIs offer strong zero-shot capability;
• managed services transfer infrastructure work to the provider;
• a smaller model may require data preparation and evaluation that the buyer does not have;
• low-volume or unpredictable workloads may be cheaper through an API;
• security teams may trust an established commercial service more than a new serving chain;
• a rapidly falling API price can erase part of a self-hosting business case;
• model switching creates regression and integration risk.

The result is not an open-versus-closed winner. It is a model portfolio governed by an enterprise-owned evaluation and routing layer.

Five production patterns behind the downshift

Pattern 1: Fine-tune a smaller model for a narrow decision

Checkr's difficult background-check classifications are a clear example. In a Computerworld interview, Checkr reported that GPT-4 reached 82 percent accuracy on its difficult records and GPT-4 with RAG reached 79 percent. The fine-tuned Llama 3 8B model reached 85 percent on that subset and 97 percent overall.

The estimated monthly economics were equally clear: about $12,000 for direct GPT-4,$7,000 with RAG and $800 for the SLM. The production path ran on one A100 and could share the base model across task-specific adapters.

Boosted.ai applied the same principle to financial analysis. According to AWS, its general-purpose model path cost nearly $1 million annually. The company sought the smallest model that could meet its finance requirements and reported a 90 percent reduction at similar task quality.

This pattern fits classification, extraction, structured generation, scoring and repeated transformations. It is less suitable when the request distribution is open-ended, changes rapidly, or requires broad reasoning that the smaller model cannot reproduce.

Pattern 2: Use frontier models as teachers and judges

Dropbox's relevance judge demonstrates a different split. The most powerful judge used OpenAI o3, but it was expensive for orders of magnitude more query-document pairs. Dropbox adapted the task to gpt-oss-120b using a fixed dataset, fixed objective and DSPy optimization.

The optimized open-model prompt cut normalized mean squared error from 8.83 to 4.86 relative to the starting open prompt. Adaptation time fell from one to two weeks to one to two days. The cheaper path enabled 10 to 100 times more labels at the same cost. Dropbox still retained o3 for the highest-quality judge path.

This is the most realistic architecture for many enterprises:

1. A strong model labels difficult examples or evaluates candidate outputs.
2. Human reviewers resolve disagreements and high-risk cases.
3. A smaller model learns the stable task.
4. The smaller model handles routine production volume.
5. The strong model receives only low-confidence or novel cases.
6. Production failures return to the evaluation and training set.

The enterprise is not betting against frontier progress. It is converting expensive intelligence into reusable organizational capability.

Pattern 3: Route each request to the right model

Replit reports that Intelligent Model Routing preserved output quality while cutting cost 65 percent against its earlier Max Mode. The router chooses among models as the task evolves, while enterprise administrators define the approved set.

Academic results support the mechanism. RouteLLM reports more than 2 times lower cost in selected benchmark settings by routing between stronger and weaker models without broad quality loss. Benchmarks are not a production guarantee, but they show why routing works: many requests do not need the strongest model, while a small minority determines most visible quality risk.

A practical router can use:

• task type;
• input length and modality;
• expected answer format;
• risk category;
• customer or data residency policy;
• model confidence;
• latency budget;
• current provider health;
• predicted cost;
• recent evaluation performance.

The router must also produce an audit trace. A regulated enterprise should be able to explain which model handled a request, why it was selected, what data it received and whether a fallback occurred.

Pattern 4: Replace repeatable model behavior with deterministic execution

Meta's asset-classification system goes further. Versioned rules handle about 85 percent of production traffic in single-digit milliseconds. An LLM handles the 15 percent that remains novel or ambiguous. Meta reports that the LLM path is about 400 times the compute cost and takes seconds.

The system does not freeze. It samples served decisions, compares them with reviewed truth, distills stable patterns into rules and promotes reviewed rules into the live funnel. Both paths emit the same output schema.

This pattern applies when a model discovers rules that can later be stated explicitly:

• document-routing policies;
• data-classification decisions;
• eligibility checks;
• standard compliance mappings;
• known exception signatures;
• workflow validation;
• fixed response templates.

The cheapest correct model call is sometimes no model call. The important control is to send novel and ambiguous inputs back to a model or human instead of forcing deterministic logic beyond its verified coverage.

Pattern 5: Specialize the full inference stack

StudyFetch's cost reduction did not come from replacing one text model with another. It moved lecture transcription from a managed service to a specialized speech stack using NVIDIA Riva, Parakeet, NIM containers and L40S GPUs. NVIDIA reports roughly 10 times lower cost on the company's largest inference workload.

Decagon combines fine-tuned open models, custom speculative draft models, larger verifiers, prompt caching and optimized serving to meet real-time voice constraints. Together AI reports nearly 6 times lower cost per turn versus closed alternatives such as GPT-5 mini and p95 model latency below 400 milliseconds.

Forethought improved economics through multi-model endpoints and serverless placement. The model, serving engine, accelerator, caching policy, batching, scaling behavior and demand shape all affect cost. The enterprise AI cost-reduction playbook therefore treats model choice as one lever among many.

The 13 company cases in context

Ito: the most transparent migration in the index

Ito's case is valuable because the company reported what got worse. The open model used more turns and tokens, median steps slowed from 23 to 29 minutes, and the terminal failure rate increased from 2.4 to 4.5 percent. Yet median model cost per pipeline run fell 55 percent and weekly spend fell about 28 percent while volume increased about 8 percent.

That is a real production tradeoff. Six extra minutes were tolerable for background QA, but they might be unacceptable in an interactive coding loop. Read the full Ito evidence brief.

Checkr: enterprise data changed the model ranking

General benchmarks could not tell Checkr which model would classify its difficult records. Years of labeled examples could. The smaller model won on the specific minority classes that mattered, while constrained labels and fallbacks made hallucinations manageable. Read the Checkr SLM case study.

Dropbox: adaptation speed is part of model portability

Switching models is not only an inference-cost decision. Prompts often fail to transfer. Dropbox used a measurable optimization loop to compress weeks of manual adaptation into days. That capability turns model portability from a contract clause into an engineering practice. Read the Dropbox open-model case study.

Pinterest: build, adapt and buy at the same time

Pinterest still uses proprietary AI. It also reports strong economics from fine-tuned open and internal models where its own visual, graph and behavioral data produces differentiated quality. Read the Pinterest open-source AI strategy.

Replit: hide model complexity behind policy-aware routing

Users should not need to follow weekly benchmark changes. Administrators should still control the approved model set. Replit's router separates those concerns and reports a 65 percent cost improvement at the same tested output quality. Read the Replit routing case study.

Meta: reserve probabilistic reasoning for uncertainty

Meta's classifier makes rule coverage a production metric. As validated rules cover more stable cases, LLM calls and latency fall. The architecture uses the model where ambiguity justifies it. Read the Meta deterministic-first AI case study.

Siemens: fit the model to each agent

The Siemens search system validates, classifies, searches, summarizes and applies guardrails across 15 to 20 sites. Siemens selected the best-fitting model for each agent and moved its search workload to Nova 2 Lite. Read the Siemens model-selection case study.

CarbonTrail: compare complete architectures

CarbonTrail's Llama, Bedrock and Inferentia stack is reported against a comparable GPT-4 plus embeddings design. The public result combines model and infrastructure choices. That is precisely why enterprise buyers should compare production architectures rather than API price lists. Read the CarbonTrail cost case study.

Sully.ai: closed to open in a latency-sensitive clinical workflow

Sully.ai reports both cost and latency gains after moving to open models. Note generation fell from roughly 70 to 25 seconds at the median, while decision support fell from about 20 to 5 seconds. Read the Sully.ai open-model migration case study.

Decagon: voice agents require a model system

Voice latency is cumulative. Recognition, reasoning, tool calls, guardrails and speech generation all compete for the same conversational pause. Decagon uses multiple specialized models and speculative decoding instead of one closed model for every stage. Read the Decagon voice AI case study.

Boosted.ai: buy more analysis with the same budget

The goal was not only to lower the bill. Boosted.ai wanted five to ten times more analysis and near-real-time updates instead of overnight processing. A smaller finance-tuned model changed which product experience was affordable. Read the Boosted.ai smaller-model case study.

StudyFetch: specialized speech changed unit economics

StudyFetch's six-figure monthly transcription line item shows how quickly per-minute costs compound. The detailed evidence distinguishes a 10 times reduction in one workload from a 10 times reduction in the whole AI budget. Read the StudyFetch specialized speech brief or the full StudyFetch cost reconstruction.

Forethought: many small models need different serving economics

Customer-specific classifiers create many model variants, often with bursty demand. Forethought used shared endpoints and serverless inference to avoid dedicating a full stack to every customer model. Read the Forethought task-specific model case study.

What the evidence proves, and what it does not

It proves that frontier-only is not the default economic optimum

Across these workloads, companies found meaningful savings by specializing, routing, opening or eliminating model calls. The pattern appears in both AI-native companies and large enterprises.

It proves that enterprise data can reverse public model rankings

Checkr, Dropbox, Pinterest and Meta all relied on internal labels, context or rules. A broad benchmark did not define production quality. The company's own evaluation set did.

It proves that architecture matters as much as parameter count

Ito's cache economics, Decagon's speculative decoding, Forethought's endpoint sharing and StudyFetch's optimized speech serving show that the same model can have very different economics depending on the system around it.

It does not prove that open weights are always cheaper

Low utilization, infrastructure staffing, data preparation, review and reliability work can eliminate the apparent price advantage. Ito found that the same open model behaved differently across providers and gateway paths. Self-hosting introduces responsibility for security patches, scaling, observability and incidents.

It does not prove that every company completed a frontier migration

Some cases are comparisons, selections or routing changes. CarbonTrail compared against GPT-4. Siemens did not name its earlier model. Dropbox retained o3 in a high-quality path. Honest classification makes the evidence more credible.

It does not prove that the reported savings transfer

Vendor customer stories are useful but commercially interested. Traffic, contracts, hardware availability, quality thresholds and human-review costs differ. Every percentage in this index is a hypothesis generator for a local benchmark, not a guaranteed discount.

A procurement playbook for enterprise teams

1. Break the workflow into model-sized tasks

Do not evaluate “customer service AI” or “enterprise search” as one task. Separate intent classification, retrieval, reranking, summarization, tool selection, structured extraction, response generation, safety review and speech.

2. Build a representative evaluation set

Include common cases, minority classes, adversarial inputs, long context, multilingual traffic, malformed documents and high-consequence failures. Lock the test set before comparing systems.

3. Establish the frontier quality ceiling

Use a strong model to learn what quality is possible. Record failures and human disagreements. A frontier model is an excellent benchmark even if it is not the final runtime.

4. Test the smallest credible alternative

Compare prompting, retrieval, fine-tuning, distillation, constrained generation and deterministic rules. Move upward in model capability only when the smaller path misses a material gate.

5. Compare fully loaded cost

Include model usage, retrieval, storage, accelerators, engineering, monitoring, security, support, retries and human review. Use cost per accepted outcome at real demand, not list price per million tokens.

6. Deploy in shadow mode

Run the candidate against real traffic without changing visible output. Compare by cohort and failure severity. A global average can hide a dangerous drop in one language, document type or customer segment.

7. Route with a rollback path

Start with low-risk traffic. Escalate low-confidence cases. Keep a same-day rollback or provider fallback. Ito's staged repository-by-repository cutover is a strong example.

8. Re-evaluate continuously

Model quality and prices move quickly. A system optimized once can become uneconomic months later. Store task, model, version, latency, tokens, cost, confidence, acceptance and outcome for every production decision.

When a bespoke model is commercially justified

A bespoke model or model system is most attractive when five conditions overlap:

1. The task repeats at meaningful volume.
2. The output can be evaluated against a stable acceptance gate.
3. The enterprise owns examples, corrections or domain knowledge that improve specialization.
4. Cost, latency, privacy or deployment control is strategically important.
5. The workflow is valuable enough to maintain monitoring and iteration.

It is less attractive for an exploratory feature with little traffic, no labeled data, open-ended requirements and no owner. In that case, a frontier API can be the faster and cheaper choice.

The business case should compare four options: remain on the current provider, switch managed providers, adapt an open model on managed infrastructure, or build a bespoke model system. The right answer can combine all four across different stages of the workflow. That is the conclusion of the enterprise build-versus-buy framework and the bespoke AI business case.

Research limits

This index prioritizes first-party engineering posts, public company interviews, research papers and named customer reports. Several numerical cases come from infrastructure vendors describing their customers. Those sources are close to the implementation but have a commercial interest in the result.

We therefore apply five rules:

• identify the source type;
• preserve the reported denominator;
• distinguish a comparison from a completed migration;
• disclose technical or quality regressions;
• avoid converting percentage savings into an undisclosed cash amount.

The index was researched through September 6, 2026. Model names, prices and deployment options will change. The architecture principle is more durable than any model leaderboard.

Primary evidence sources

The company results in this index are grounded in the following public reports. First-party engineering posts are generally stronger evidence than vendor-supported customer stories, so the source type is preserved in the index rather than hidden.

1. Ito production telemetry for its move to open weights
2. Computerworld's interview with Checkr's machine-learning engineer
3. Dropbox Engineering on adapting its relevance judge
4. Pinterest Engineering on re-prioritizing open-source AI
5. Replit on Intelligent Model Routing
6. Meta Engineering on deterministic-first asset classification
7. AWS customer case on Siemens global search
8. AWS customer case on CarbonTrail emissions intelligence
9. Baseten customer case on Sully.ai's open-model migration
10. Together AI customer case on Decagon voice inference
11. AWS startup case on Boosted.ai's smaller finance model
12. NVIDIA customer case on StudyFetch transcription
13. AWS customer case on Forethought model serving

Building a Production-Ready System

The companies in this index did not create savings by choosing a smaller model from a dropdown. They defined the task, built evaluation data, adapted the model, designed the serving path, added routing and fallbacks, and measured the result in production.

That is the work Conscious Engines is built to do for enterprises.

We design bespoke model systems around the operating constraint: specialized speech-to-text, text-to-speech, voice agents, task-specific language models, enterprise RAG, evaluation, routing and private deployment. A frontier model can remain in the architecture where its capability pays for itself. Routine traffic moves to the system that clears the enterprise's quality gate at the lowest fully loaded cost.

The deliverable is not a cheaper chatbot. It is an owned production capability with measurable unit economics, model portability and a controlled path for improvement.