I’ve heard it a hundred times before, and I bet you have too: “the way AI systems are trained means that they only produce the statistically average answer to any question.” And yet, anyone who’s recently used an LLM would tell you that this obviously isn’t true.
Over the last few years I’ve shared many reasons why that assumption is incorrect. I’ve tried to explain how post-training works, why agents are more than just LLMs, and how these systems develop emergent capabilities, but I found a far better explanation in chapter one of Ethan Mollick’s excellent book Co-Existence. Instead of writing a blog post that summarizes his argument, I’d rather share his excerpt (emphasis mine) — and suggest that you buy Ethan’s book!
A common misconception about AIs is that they give the “average answer” to a question they are asked. This intuition makes sense if you understand that AIs are trained on all the written data AI companies could get their hands on. But at the same time, this is obviously not true; AI systems routinely outperform the humans whose data they were trained on. Pick a message from the internet at random and it is likely badly written, but AI writes quite well. Pick a random person to answer a math problem and they will likely get it wrong, but AI gets it right. A group of Harvard researchers argue that this is due to a phenomenon called transcendence.
The core insight is that AI isn’t trained to imitate any single person. It’s trained on data from millions of people, each with their own expertise, biases, and blind spots. Under the right conditions, a model trained to mimic this diverse crowd can outperform every individual member of that crowd. Not because it has developed some mysterious new capability, but because it has learned to combine human knowledge in ways no single human could.
One way this works is that the AI automatically takes advantage of the so-called wisdom of the crowd. When many people attempt the same task, they make different mistakes. If those errors are uncorrelated—meaning one person’s slipup doesn’t predict another’s—then the crowd’s average judgment tends to be more accurate than any one individual’s. A model trained on a thousand chess players, each making their own individual blunders, can learn to play better than any of them by implicitly “voting” among the patterns it has learned. The Harvard team found that even when every individual expert got facts wrong 80 percent of the time, a model trained on a hundred such experts achieved over 80 percent accuracy simply by learning which answers showed up most consistently.
But AI has another trick. Sometimes the issue isn’t the wisdom of the crowd but the wisdom of the right expert. A lawyer knows law, a marine biologist knows otters, and neither knows much about the other’s field. A model trained on both marine biology and law can handle legal questions by drawing on patterns from the legal data and otter questions by drawing on patterns from the marine biology data. It doesn’t need a single expert who knows everything. It just needs many experts with different specializations, and a way to tell which expertise applies to which question.
The key condition the researchers found is that experts must comment more often on topics within their domain than outside it. When that is true, the model learns not just facts but which types of experts to trust about which topics. And even more impressively, sometimes a model can answer questions that no individual expert could answer by combining knowledge from multiple domains. A lot of knowledge is siloed, but many problems span expertise. A model trained on these separate experts can connect the dots, representing their knowledge in a shared internal space and composing an answer. The results can feel surprising because we intuitively think of AI as imitating a person, when really it’s imitating a population.
So there you have it, some good reasons why AI systems produce output that’s above average — or even downright superhuman results.