How diffusion models make an image from noise
Image generators do not paint. They start from static and repeatedly subtract what a network predicts is noise, until what remains is a picture.
Exploring the intersection of advanced technology and human purpose.
Image generators do not paint. They start from static and repeatedly subtract what a network predicts is noise, until what remains is a picture.
A language model has no memory, no plan and no idea what it is about to say. It predicts one token, then reads its own output and predicts the next. Attention is what makes that work.
RAG turns a recall problem into a reading problem. Most of the engineering is not in the model — it is in deciding what to put in front of it.
A sparse model holds a trillion parameters and uses a small fraction of them per token. That trade is what made the current generation of frontier models affordable to serve.
Neural network weights are stored at far higher precision than they need. Dropping from 16 bits to 4 is mostly free — until, quite suddenly, it is not.
A base model predicts text. Everything you recognise as an assistant — the helpfulness, the refusals, the tone — was added afterwards by training on human preference.
No neurons fire. Nothing is understood. A network learns by being wrong in a measurable way and adjusting millions of numbers slightly in the direction that was less wrong.
Loss falls as a predictable power law in compute, parameters and data. Working out which of the three to spend on changed the industry more than any architecture did.
A convolutional network learns one small filter and applies it everywhere. That single constraint is what makes vision tractable — and it is still the right tool for a great many problems.
Quantum computers do not try every answer at once. That explanation is wrong, and the real mechanism — interference — is both stranger and more useful to understand.
Road networks, utility grids, molecules and social ties are not grids or sequences. Forcing them into one loses the structure that mattered.
A resident photographs a pothole. Thirty seconds later it is classified, routed and tracked. Here is what happens in between.
A model can learn from data it never receives. That is genuinely useful, and the guarantee is narrower than the phrase suggests.
Press releases count physical qubits. The figure that decides when quantum computing becomes useful is how many of them it takes to make one that works.
One size fits none. Why a civic platform needs purpose-built architecture rather than a general model with a clever prompt.