Ternary mixture-of- experts model deployed on the Apple Watch Neural Engine

We built a small sparse language model for local semantic tasks. The full model stores 203.2 million parameters, activates 70.5 million per token, and uses ternary weights in its large projection matrices. The current foundation was trained for 14,761,230,336 tokens.

Quick Demo

For deployment, we selected one specialist path, removed the inactive experts and router, and exported a static Core ML model. The resulting Watch deployment is about 70 MB. It is bundled in a watchOS app and runs locally on Apple Watch.

This article describes that path: model, training, specialist, Core ML, and Watch.

The model

The model is a 12-layer Mixture-of-Experts with a hidden width of 640. Each layer contains four feed-forward experts. A learned router selects one expert per token during base training.

Only one expert is active in each layer. This gives the model 203.2M stored parameters without executing the whole model for every token. The active computation consists of 26.2M shared parameters and one 44.2M expert path.

Six layers use grouped-query attention. The alternating six use short LFM2-style convolutions with a cache length of three. The attention layers provide periodic global context. The convolution layers provide local mixing with less KV-cache state.

The operator mix and weight encoding were selected after earlier measurements of language-model architectures on the Apple Neural Engine. The resulting model uses half attention, heavily grouped KV heads, and ternary projection weights.

Native ternary training

The large projection and expert matrices use three effective values: -scale, 0, and +scale. Activations use per-token int8 fake quantization. Embeddings, normalization, routers, and the small convolution kernels remain at higher precision.

This quantized computation is present during training. The optimizer maintains high-precision master weights, but every forward pass uses the ternary projections. The model is therefore trained under the same numerical constraint used by the deployment artifact.

This differs from post-training quantization. A dense model is not trained first and compressed later. Weight encoding is part of the model definition from initialization.

Training the foundation

The current deployed lineage was trained from random initialization for exactly 14,761,230,336 tokens, or 14.76B.

The public-language mixture included FineWeb, FineWeb-Edu, OpenWebText, English Wikipedia, UltraChat, SNLI, MultiNLI, XSum, and Cosmopedia. It was intended to teach reusable language representations, paraphrase, grounding, conversation, summarization, and light logical structure. Application schemas and specialist tool syntax were added later.

Long training runs are now common among small-model releases, and several public models go much further. Liquid AI reports 19T pretraining tokens for LFM2.5-230M, a dense model close to ours in stored parameter count. Google reports 6T tokens for Gemma 3 270M. Hugging Face trained the 135M and 360M SmolLM models on 600B tokens each and found that its 125M experiments continued to improve beyond the Chinchilla-optimal point.

These runs are not direct comparisons: their architectures, data mixtures, objectives, and token-accounting methods differ. They show that a small parameter count does not imply a small pretraining budget. Our 14.76B-token run is modest beside those releases, while still longer than older parameter-to-token rules would suggest.

Checkpoint selection used more than validation loss. The evaluation suite included language benchmarks, grounded extraction, qualitative generation, and specialist regressions. Some continuations reduced loss while making grounded behavior worse. Those checkpoints were not used for deployment.

The foundation is not treated as a complete chat model. It supplies language and token mixing for bounded specialists.

Training a specialist

A specialist occupies one complete expert path across all 12 layers. That path contains 44,236,800 parameters.

In the isolated training procedure, the shared trunk, router, output head, and other experts are frozen. The selected expert is the only optimizer parameter group. Hashes are computed before and after training to verify that the selected expert changed and every non-selected parameter remained byte-identical.

This produces an expert pack tied to a specific foundation hash, tokenizer hash, corpus manifest, and expert index. The pack can be evaluated and stored without replacing the shared foundation.

The grounded Watch deployment uses expert 2. It also includes a 1.67M-parameter LoRA refinement for its answer protocol. The full expert carries the broader capability; the smaller refinement adjusts the final application behavior.

Fixed-expert export

The full training checkpoint is not copied to the Watch.

At export, expert 2 is selected in every layer. The learned token router and the other three experts are removed. The ternary matrices are packed, compact tied embeddings are retained, and the LoRA weights become fixed linear residuals.

The resulting Core ML package has two functions:

• prefill processes the prompt in a batch;
• decode generates one token using external attention and convolution caches.

The Watch deployment is about 70 MB. It contains the shared trunk, one expert path, the protocol refinement, and both inference functions. It does not load experts or adapters dynamically at runtime.

Training and deployment therefore use different forms of the same model. Training uses four experts and learned routing. Deployment uses one static graph.

Running on Apple Watch

The model is bundled directly in a watchOS target. Prompts and model state remain on the Watch rather than being sent to the paired iPhone or a server.

The signed Watch app is about 70 MB. Its TestFlight build passed App Store processing and remained below the 75 MB uncompressed watchOS app limit.

Both prefill and decode load through Core ML with .cpuAndNeuralEngine. Supported operations can run on the Watch Neural Engine, with CPU fallback where required.

A physical Watch run produces roughly 30 end-to-end generated tokens per second. This measurement includes prompt processing, model prediction, host-side cache updates, argmax, token decoding, callbacks, and UI streaming. It is an application measurement rather than a model-only decode benchmark.

Result

The project produced three related artifacts:

1. a 203.2M-parameter sparse ternary training model;
2. a 44.2M-parameter specialist path attached to its shared foundation;
3. a roughly 70 MB static Core ML deployment in a Watch app.

The model was trained for 14.76B tokens, specialized without replacing its foundation, and reduced to one fixed inference graph. Apple Watch is the current small-device deployment of that model.

References

• LFM2.5-230M: Built to Run Anywhere - Liquid AI
• Gemma 3 model card - Google DeepMind
• SmolLM: blazingly fast and remarkably powerful - Hugging Face
• The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits - Ma et al.