Conscious Engines
Story of Attention - Part 1: Before the Transformer
From n-gram counting and RNNs to the first attention mechanisms.
guide
Read more →Research and Development
Experiments in model architecture, training, evaluation, and efficient inference
Papers
2 publishedThe Joint Effect of Quantization and Sampling Temperature on LLM Safety Alignment
How weight quantization and sampling temperature jointly affect LLM safety alignment — a factorial analysis of the interaction.
arXiv
LLM Inference at the Edge
Sustained-load benchmarks across mobile, NPU, and GPU platforms reveal how LLM inference performance really degrades under back-to-back requests.
arXiv
Featured
Conscious Engines
Building a tasks app that puts all iOS 27 on-device AI features to work
How we used an ensemble of on-device models to build consumer AI that works offline, costs nothing to run, and never leaves the phone
apple silicon
Read more ↗Conscious Engines
The Need for Speed: Why LLMs Are Slow and What Speculation Promises
Your GPU sits nearly idle during token generation — the memory-bandwidth bottleneck behind LLM latency, and how speculation exploits it.
Inference
Read more →Conscious Engines
Kinetic-4B: A 4-Billion Parameter Model That Outperforms Claude Haiku at Tool Calling
A fine-tuned 4B open-source model that beats frontier models at tool calling, with p95 latency under two seconds on a single GPU.
Agentic AI
Read more →All research
Building a tasks app that puts all iOS 27 on-device AI features to workexternal ↗
Conscious Engines
How we used an ensemble of on-device models to build consumer AI that works offline, costs nothing to run, and never leaves the phone
apple silicon
Running Bonsai 27B as an agent on my phone
Conscious Engines
Running a 27B model (1-bit) on my phone and letting it perform agentic tasks
apple silicon
How we trained a ternary mixture-of- experts model from scratch and deployed it on the Apple Watch Neural Engine
Conscious Engines
A 203M-parameter language model, trained for 14.76B tokens and deployed as one fixed specialist.
apple silicon
Squeezing a 26B diffusion LLM onto a Mac
Conscious Engines
Optimizing DiffusionGemma (26B-A4B-IT-4Bit) on an Apple M5 Pro, what worked, what didn't.
apple silicon
Does the 27B Bonsai actually deliver 27B?
Conscious Engines
A ternary-quantized 27B model that scores like an 8B on tool-calling - but deletes active records when nobody double-checks the honest answer.
apple silicon
Speculative Decoding, Formally: The Algorithm, the Proof, and the Metrics That Matter
Conscious Engines
The draft-then-verify algorithm behind speculative decoding, its rejection-sampling proof of losslessness, and the metrics that matter.
Inference
The Need for Speed: Why LLMs Are Slow and What Speculation Promises
Conscious Engines
Your GPU sits nearly idle during token generation — the memory-bandwidth bottleneck behind LLM latency, and how speculation exploits it.
Inference
A Field Guide to Speculative Decoding Methods
Conscious Engines
Every major speculative decoding method compared: Medusa, lookahead, SpecInfer, Sequoia, self-speculation, and more, mapped in one guide.
Inference
The EAGLE Family: Speculating in Feature Space
Conscious Engines
Three generations of EAGLE speculative decoding, from feature-level drafting to state-of-the-art speedups — compared side by side.
Inference
Parallel Drafting with Block Diffusion: DFlash and DDTree
Conscious Engines
DFlash and DDTree fill an entire token block per drafting pass, pushing speculative decoding speedups past 7x over sequential drafters.
Inference
Diffusion vs Autoregression: Why Language Models May Not Need to Think Left to Right
Conscious Engines
Why diffusion language models, which denoise a full sequence instead of writing left to right, can outperform autoregressive models on some tasks.
dLLM
Putting It to Work: Serving Speculative Decoding with vLLM and SGLang
Conscious Engines
A practical guide to enabling EAGLE, n-gram, and Medusa speculative decoding in vLLM and SGLang, with real production speedup numbers.
Inference