Research and Development
Experiments in model architecture, training, evaluation, and efficient inference
All research
Diffusion vs Autoregression: Why Language Models May Not Need to Think Left to Right
Conscious Engines
Why diffusion language models, which denoise a full sequence instead of writing left to right, can outperform autoregressive models on some tasks.
dLLM
Broad Review of DLM architectures
Conscious Engines
A field guide to diffusion language model architectures — how LLaDA, Dream, Block Diffusion, and MoE variants like LLaDA 2.0 approach denoising.
dLLM
Why Diffusion LLM Quantization Is Harder Than It Looks
Conscious Engines
Diffusion LLMs share transformer modules with AR models but not inference physics — why GPTQ, AWQ, and QuaRot fail, and what fixes them.
dLLMquantization