We’re putting $5.5M behind research and development of smaller, specialized AI models for real-world deployment — Read our manifesto →
    All posts

    [Case Study] When a Model Update Invents Transcript Text: Remote Legal's ASR Migration

    A legal deposition platform changed speech providers in one week after phantom text appeared, then rebuilt reliability and reduced nonproduction transcription cost by 85%.

    Conscious Engines

    Industry: Legal technology
    Organization: Remote Legal
    Use case: Live deposition transcription
    Evidence basis: AWS customer case using company-reported results
    Disclosure: This is an independent analysis by Conscious Engines. The principal source is a vendor customer story.

    1. Outcome at a Glance

    In April 2023, Remote Legal says an update from its previous speech provider began inserting phantom text, words that had not been spoken, into legal transcriptions. The company migrated to Amazon Transcribe in one week and reported that the new system virtually eliminated those errors.

    Key Outcomes

    1 week

    Migration time

    Rapid provider change reported in the cited case.

    98%

    Deposition success rate

    Defined as depositions that start and finish successfully.

    85%

    Cost reduction

    Applies to test and demo environments versus production configuration.

    MeasureReported resultImportant qualifier
    Migration time1 weekRapid provider change
    Deposition success rate98%Defined as depositions that start and finish successfully
    Failed or “busted” sessionsFewer than 5 per monthCompany reported
    Cost reduction85%Applies to test and demo environments versus production configuration
    Managed-service benefitAbout half a cloud engineer's salaryCompany estimate

    The cost claim is frequently easy to misread. The reported 85% reduction was for test and demonstration environments, where on-demand services replaced an always-on production-style setup. It was not presented as an 85% reduction in total production transcription cost.

    2. The Operational Problem

    Legal deposition transcription has a different failure threshold from meeting notes. A phrase that was never spoken can alter the record, confuse counsel, and damage trust. Names, exhibits, citations, interruptions, accents, and remote audio conditions add complexity.

    The incident also exposes a wider enterprise risk: model behavior can change without the customer's workflow changing. A provider may update acoustic models, language models, decoding, or post-processing. Average benchmark quality can improve while one customer's critical error class gets worse.

    Remote Legal's platform combined video, document handling, and transcription. A failure in any one part could disrupt the deposition. Reliability therefore had to be measured at the session level, not only with word error rate.

    This case is a warning against treating AI as a stable API. Production models are dependencies that require regression monitoring, version awareness, fallback, and portability.

    3. What Was Built

    The migrated stack used Amazon Transcribe within Remote Legal's deposition platform. The system supported the three core elements of a remote proceeding: video, documents, and transcription.

    System at a Glance

    Audio capture

    Receive participant audio and preserve session timing.

    Streaming ASR

    Produce live text with low latency.

    Legal vocabulary

    Improve recognition of names, terms, and recurring phrases.

    Session orchestration

    Keep video, documents, and transcript synchronized.

    Reliability monitoring

    Detect stream failure, latency, and anomalous text.

    Human workflow

    Support reporters, counsel, and correction processes.

    LayerFunction
    Audio captureReceive participant audio and preserve session timing
    Streaming ASRProduce live text with low latency
    Legal vocabularyImprove recognition of names, terms, and recurring phrases
    Session orchestrationKeep video, documents, and transcript synchronized
    Reliability monitoringDetect stream failure, latency, and anomalous text
    Human workflowSupport reporters, counsel, and correction processes

    A robust legal ASR architecture should include custom vocabulary, speaker diarization, confidence data, time alignment, and an immutable audio source. The final record must be correctable against the audio.

    For “phantom text,” ordinary word error rate is not enough. The evaluation set should separately count insertions, especially long hallucinated spans, and weight them by legal significance. Canary sessions and shadow transcription can detect a regression before a provider update reaches all customers.

    4. How It Reached Production

    The one-week migration indicates that Remote Legal had a separable transcription interface and could replace a critical provider without redesigning the entire product.

    Design for model portability. Normalize transcript events, timestamps, speakers, confidence, and errors behind an internal interface. Do not spread provider-specific formats through the application.

    Maintain a legal audio regression set. Include accents, interruptions, poor microphones, exhibit references, case names, numbers, and silent periods. Track insertion errors separately.

    Use live health checks. Detect stalled streams, repeated output, impossible speech during silence, latency spikes, and disconnects. Route to a fallback model or human process when thresholds are crossed.

    Test provider updates. If version pinning is unavailable, continuous sampling and alerting become more important. Contracts should address change notice, incident support, data handling, and exit.

    Separate environment economics. Test and demo workloads are intermittent, so on-demand infrastructure can reduce waste. Production may require reserved or provisioned capacity for predictable reliability.

    Remote Legal's story is valuable because it centers failure, not only success. A model update introduced unacceptable behavior, and architecture made a one-week migration possible.

    Production legal speech infrastructure combines domain-adapted ASR, provider-neutral interfaces, custom vocabulary, live anomaly detection, immutable audio alignment, fallback routing, and client-owned evaluation sets. A bespoke model or fine-tuned open model can provide more control where cloud updates and data sensitivity are unacceptable.

    The operating metric should be successful, legally usable session completion, supported by insertion rate, critical-term accuracy, latency, and cost per audio hour. Low average error is not sufficient if the system invents text.

    Sources

    • AWS, Remote Legal case study
    • The source defines the 98% success rate operationally and limits the 85% cost reduction to test and demo environments. Those qualifiers should remain attached to any reuse of the figures.