Forethought Cut Model-Serving Cost Up to 80% by Matching Infrastructure to the Workload

How shared endpoints and serverless inference changed the economics of many customer-specific support models.

The production problem

Forethought powers more than 30 million customer-support interactions per year. Its platform often trains multiple personalized models for each customer. One model may autocomplete text, while another classifies ticket priority or supports a different workflow.

That architecture creates a serving problem. Giving every customer-specific model its own always-on endpoint can waste GPUs and increase operational work. Keeping bursty, small classifiers running continuously is also inefficient.

Forethought moved inference from Amazon EKS to SageMaker multi-model endpoints and serverless inference. According to the AWS customer case, it reported:

Many bespoke models can share one base

Task-specific models do not always require task-specific infrastructure. A shared endpoint can load or serve multiple model variants on common hardware. Low-rank adapters can also share a base model while changing task behavior.

The economic model becomes:

cost per customer model = shared capacity + model-specific load and storage + requests

instead of:

cost per customer model = one dedicated always-on endpoint

This is especially useful when each customer has valuable private examples but no single model generates enough sustained traffic to fill a GPU.

Serverless fit the small, bursty classifiers

Forethought used serverless inference for models such as customer-specific ticket-priority classifiers. These workloads can be intermittent and small. Paying for an idle server can exceed the cost of the actual inference.

Serverless infrastructure shifts the balance toward paying around execution, with tradeoffs in cold starts, supported model size and platform control. The correct placement depends on arrival rate and latency budget:

• steady, high-volume model: dedicated or provisioned capacity;
• many variants with moderate shared demand: multi-model endpoint;
• intermittent small classifier: serverless;
• strict local or sovereignty requirement: customer-controlled deployment.

The model and serving choice must be made together.

Operational capacity is part of cost

Forethought had a three-person cloud infrastructure team. Managing many Kubernetes clusters and model deployments consumed time that could be used elsewhere. The managed serving move reduced memory and availability work while supporting growth. Engineering director Jad Chamoun described the outcome as the ability to “support customers at a lower cost per customer.”

Fully loaded economics should include:

model cost + infrastructure + idle capacity + platform engineering + incident work + review

A self-hosted open model may have a lower compute rate but a higher operating burden. A managed endpoint may be cheaper when team capacity is the binding constraint. This is why enterprise build versus buy is a workload decision, not an ideology.

What the case proves

Forethought shows that bespoke or task-specific models do not automatically create infrastructure sprawl. A shared base, multi-model endpoint or serverless path can preserve customer specialization while improving utilization.

It also shows that smaller models can belong in a broader generative AI product. SupportGPT uses large language models and retrieval, while narrow classifiers handle tasks such as ticket priority. Each model receives the job that fits it.

Evidence limits

This is an AWS-supported customer story and the reported results are not independently audited. The 66 percent and 80 percent figures refer to different serving patterns and cost pools, so they should not be combined. The case does not name a frontier model that Forethought replaced.

It belongs in the Frontier Model Downshift Index because it shows production specialization and workload-aware serving, not because it documents a full closed-to-open migration.

Enterprises can connect it with Checkr's shared adapter pattern, Meta's narrow classifier, Replit's model routing, the AI FinOps framework and enterprise cost-reduction playbook.

Building a Production-Ready System

Conscious Engines builds and operates portfolios of customer-specific classifiers, language models, RAG components and voice systems without treating every model as an isolated infrastructure project.

We select shared, serverless, provisioned or private deployment based on traffic and controls, then track cost per accepted business outcome. The enterprise receives specialization where it creates value and shared infrastructure where it reduces waste.