A production privacy-classification case that shows how models can discover stable behavior and deterministic systems can serve it.
The architecture
Meta documented a production system for classifying data assets under privacy-aware policies. The system does not ask an LLM to classify every asset. It assembles a structured evidence brief and sends the request through two paths:
- A deterministic path applies known, versioned and reviewed rules.
- An LLM path handles novel, ambiguous or out-of-coverage cases.
According to Meta Engineering, deterministic rules resolve roughly 85 percent of production traffic in single-digit milliseconds. The LLM handles roughly 15 percent, takes seconds and uses about 400 times the compute.
That is not a small-language-model case. It is a stronger economic idea: stable model behavior can sometimes become software.
How the decision funnel works
Every request uses a small, explicit contract. The system returns a category, confidence, decision trace, matched rule and version metadata. Both the rule and model paths emit the same schema, so downstream enforcement does not need to know how the decision was produced.
Meta pre-ranks context into an evidence brief instead of sending dozens of raw fields to a model. The brief can contain supporting, contradictory and suppressed signals with weights. Sensitive or circular fields are masked from both the model and the learned rule process.
The operating loop then looks like this:
context -> reviewed rule if covered -> LLM if ambiguous -> sampled review -> independent evaluation -> distilled rule -> shadow test -> promotion
Rule coverage becomes an operating metric. If coverage rises while recall and severe-error controls hold, the system uses fewer expensive calls and becomes easier to replay and audit.
Reported Production Results
~85%
Traffic resolved by rules
Reviewed deterministic rules handled the majority of production requests.
~15%
Traffic sent to the LLM
The model handled novel, ambiguous, or out-of-coverage cases.
~400x
LLM compute multiple
Reported compute difference between the LLM path and deterministic rules.
Single-digit ms
Rule-path latency
Reported latency for production requests covered by deterministic rules.
The LLM is used where uncertainty has value
Meta's design does not treat the model as a failure. The LLM is valuable during cold start, for rare classes, conflicting evidence and new patterns. It can reason over context that a fixed rule set does not yet cover.
The model is simply not the default enforcement engine. Meta writes: “Rules are better for routine enforcement.”
This distinction is useful across enterprise functions:
- known invoice layouts can use extraction rules, with unusual documents sent to a model;
- standard policy mappings can be deterministic, with new clauses escalated;
- common service requests can use workflows, with ambiguous intent sent to an agent;
- known data classifications can use rules, with rare combinations sent to an LLM;
- recurring voice-agent steps can be fixed state transitions, with open-ended questions escalated.
Quality controls make distillation governable
A careless model-to-rules process can encode bias, leakage or brittle correlations. Meta separates learning from evaluation so the classifier does not grade its own work. It uses human-reviewed reference labels, per-class recall, macro F1, Matthews correlation coefficient, calibration and shadow deployment.
The system also versions context, prompt and rules. That allows a decision to be reproduced during an audit. Deterministic execution is not automatically safer; it becomes safer when the rule provenance, evaluation and approval path are explicit.
This is why production AI evaluation must track minority classes and high-consequence errors. An average accuracy number could remain stable even if a privacy-sensitive class degraded.
The economic calculation
Suppose the LLM path costs 400 units and the rule path costs 1 unit of compute. If all 100 requests use the LLM, the index cost is 40,000 units. If 85 use rules and 15 use the LLM, it becomes:
85 x 1 + 15 x 400 = 6,085 units
That illustration is about 85 percent lower compute than the all-LLM design. Meta does not publish a dollar-saving percentage, and the real system also pays for context assembly, offline evaluation, review and rule management. The calculation shows why coverage is financially powerful.
What enterprises should copy
Begin with a stable decision contract. Measure rule coverage, model fallback rate, human review, severe errors and decision latency. Only promote a learned rule after independent validation. Preserve a fast rollback and send out-of-distribution inputs back to the model or a human.
The system belongs beside Checkr's constrained classifier, Replit's model router and Dropbox's teacher-runtime split in the Frontier Model Downshift Index. Each reserves expensive reasoning for the part of the distribution that needs it.
For a broader cost model, see the enterprise AI cost-reduction playbook.
Building a Production-Ready System
Conscious Engines designs compound AI systems, not only model endpoints. We identify which decisions can be deterministic, which need a task-specific model, which require retrieval and which must escalate to a stronger model or human.
That layered design reduces cost and latency while giving enterprise teams a traceable, governable decision path.