Where large language models came from, what actually changed, and why the honest answer to the question “is this working?” is more interesting than either the hype or the backlash.
If you feel unsure about all of this, you are reading the situation correctly
Three years ago most people had never typed a prompt. Today a lot of developers open a model before they open a search engine, and many of them are quietly unsure whether that is progress or a very well marketed detour.
That doubt is not noise. It shows up in the data. In the 2025 Stack Overflow Developer Survey, 84 per cent of developers use or plan to use AI tools and 47 per cent use them daily. In the same survey, favourable sentiment fell from 77 per cent in 2023 to about 60 per cent in 2025, only a third of respondents trust the accuracy of what the tools produce, and the most common complaint, from 66 per cent of them, is that the answer is almost right but not quite.
The mood is not irrational. It is the predictable result of using something powerful and unreliable every day. Adoption went up, trust went down, and holding both of those at once is the correct posture right now.
A word on where I am standing, because it shapes what follows. Before Capitole Lab I worked as an AI Innovation Expert (fancy word for Lead Data Scientist) at a large insurance company, and I watched the transition from traditional machine learning to LLM and agent-based pipelines from the inside. It was an honour to be somewhere where adoption moved that fast, because it meant we were doing early, and sometimes clumsily, what has since become the norm: AI governance, token limits and budgets, model selection treated as a real engineering decision rather than a preference, and encapsulating the whole thing inside a single cloud provider for control and compliance. It is also why the second half of this article spends more time on operations than on models.
Where they came from, because it was not overnight
The moment that felt like magic in November 2022 was built on work that was already five years old. In June 2017 a team at Google published “Attention Is All You Need”, and its contribution was to throw out recurrence. Until then models read text one token at a time, in order, and the signal between distant words decayed as it travelled. Self attention let every token look at every other token at once, in parallel, which removed the bottleneck that made training slow and made long range relationships cheap to represent.
That unlocked scale, and scale is the story of everything after. GPT-1 arrived in 2018 with 117 million parameters, GPT-2 in 2019 with 1.5 billion, and GPT-3 in 2020 with 175 billion, roughly 1,500 times the first model in about two years. It also came with a property nobody had ordered: describe a task in the prompt and get useful behaviour without retraining anything.
What happened on 30 November 2022 was not a scientific breakthrough. It was a product decision: wrap a capable model in a chat box and let anyone try it. Search interest for AI had been flat for years. Within weeks it was not.
What actually changed under the hood
It is tempting to dismiss the last three years as marketing. The capability curves say otherwise, as long as you read them carefully. On MMLU, a broad knowledge exam, GPT-3 scored around 44 per cent, barely above guessing. By 2024 the frontier was in the high eighties and the benchmark stopped being interesting, so the field moved to GPQA Diamond, PhD level science questions where GPT-4 started at 39 per cent and a domain expert scores about 65 per cent. Frontier models now report low to mid nineties. Then came SWE-bench Verified, which asks a model to resolve real GitHub issues unaided: 49 per cent in late 2024, and eighties to nineties at the frontier in 2026.
Two caveats. Evaluation conditions vary between labs, so these numbers are not perfectly comparable, and the highest scores are usually self reported at launch. The trend is solid. The last decimal is not. The other changes get fewer headlines and matter more for real work.
- Memory. Context windows went from 2,048 tokens in GPT-3 to a million or more as standard, roughly 500 times more in six years. A million tokens is about 75,000 lines of code, or an hour of video. That is the difference between asking about a file and asking about a system.
- Senses. Text only became vision in 2023, then voice in 2024, then native image and video generation. A model that reads a screenshot, hears a question and answers out loud in a quarter of a second is a different interaction model, not a faster autocomplete.
- Autonomy. The shift still in progress, and the one where the language runs ahead of the reality.
The agent story has a clear lineage: the ReAct paper in 2022, function calling in the API in mid 2023, models able to operate a computer screen in late 2024, and the Model Context Protocol in November 2024, which gave tools and data a standard way to talk to models and was adopted by rival labs within months. Coding agents went from demo to a billion dollar run rate business in about a year.
Here the honesty matters again. Around 31 per cent of developers reported actually using agents in 2025, and one enterprise study found that only 16 per cent of deployed agentic systems were genuinely autonomous. The rest were fixed sequences of steps with a model inside. Agents are real, they are early, and the same word covers both.
A field note from our own lab
I want to ground this in something we are building, because the acceleration of the last 18 months is easier to feel than to plot.
At Capitole Lab we are working on a use case for a car engine manufacturer that compares elements across industrial drawings, the kind of dense technical documents where a small difference between two revisions matters a great deal. We had scoped the first version as weeks of work. With the newest frontier tier, the Fable generation, our pipeline detected the differences after a single prompt, and the whole thing came together in a day.
It ran in about 40 minutes and cost around 40 euros of inference. Nobody should ship that. A production version has to be optimised on both axes, time and money, and that optimisation is real engineering work rather than a better prompt. As a proof of concept, though, it told us something worth knowing: a task we had estimated in weeks became a day, and the part we assumed would be hardest, actually understanding what is in the drawing, was not where the effort went.
That is the shape of the last year and a half for us. Not a smooth curve on a chart, but a series of moments where something we had written off as too expensive or too fiddly quietly became possible. It is promising, to say the least, and it is also why the cost question deserves its own section.
What it costs, and who is paying for it
Two statements are both true, and people usually pick one to argue with. For a fixed level of capability, inference became dramatically cheaper: the Stanford AI Index puts GPT-3.5 level performance at 20 dollars per million tokens in late 2022 and 7 cents by late 2024, roughly 280 times cheaper in eighteen months. At the same time the top of the market became more expensive. GPT-4.5 launched in early 2025 at 75 dollars per million input tokens and 150 per million output, so expensive that it was pulled from the API within months. The frontier tier today sits around 10 and 50 dollars.
So the floor sinks while the ceiling climbs. The intelligence you paid a premium for last year is nearly free now, and the best available intelligence costs more than it ever has. Which of those facts governs your budget depends on whether your product needs the frontier or just needs something that works. The 40 euro run I mentioned sat deliberately at the expensive end, because at proof of concept stage we were buying capability rather than efficiency. Moving a pipeline like that down towards the floor, with smaller models where they are enough, caching, and fewer wasted passes, is a large part of what productionising actually means.
There is also a bill that arrives outside the API invoice. Memory makers shifted capacity towards high bandwidth memory for AI servers, and consumer DDR5 went from about 4.70 dollars per gigabyte in 2023 to 15.66 by early 2026, up roughly 230 per cent, with DRAM rising about 90 per cent in a single quarter. NVIDIA went from around 337 billion dollars of market value in October 2022 to the first 4 trillion dollar company in July 2025, and hyperscaler capital expenditure guidance for 2026 sits between 630 and 725 billion dollars.
If you recently priced a memory upgrade and wondered what happened, this is what happened. The abstraction has a physical footprint, and it is now big enough to move consumer hardware prices. It is also a reminder that this is a business, and somebody has to earn a return on that spending whether we like it or not.
The uncomfortable part: most projects still fail
This is the section I would want to read if I were sitting on a stalled pilot. The numbers are consistent across independent sources and they are not kind. NTT Data reports that 70 to 85 per cent of generative AI deployment efforts fail. A 2025 RAND meta analysis found around 80 per cent of enterprise AI projects fail to deliver their promised value, split between projects abandoned before production, projects that reach production without delivering, and projects that run but never recover their cost. The MIT NANDA study found about 95 per cent of pilots showed no measurable return. S&P Global found the share of companies scrapping most of their initiatives rose from 17 to 42 per cent in a single year.
Read that as reassurance rather than as an indictment. If your proof of concept impressed the room and then quietly died, you did not fail unusually. You landed on the base rate. The reasons cited are almost never model quality: weak or ungoverned data, no clear business case, organisational maturity, and cost that outruns measurable value. The hard part sits where it always sat, in the gap between a working demo and a working process.
You can also see the speed of the change somewhere developers feel it personally. Stack Overflow received around 200,000 new questions a month at its 2014 peak, and over 108,000 in the month ChatGPT launched. By December 2025 it was under 4,000. The corpus that helped train these models is drying up because the models made asking unnecessary.
What the small group that succeeds does differently
The MIT study is more useful for its winners than for its losers. The teams that crossed over did not have a better model or a friendlier regulator. They had a narrower question. The projects that tend to work look like this.
- Call summarisation, routing and support triage. Clear workflow, obvious time saving, low ambiguity.
- Document automation for contracts and forms. Repetitive, measurable, easy for a human to review.
- Repetitive engineering work: tests, documentation, migrations. The developer verifies the output and cycle time is measurable.
- Back office finance, operations and administration. A real cost baseline, and often external spend to displace.
- Sales follow up and lead qualification, when it lives inside the CRM rather than beside it.
- Vertical or agentic tools with memory, which learn from feedback and fit one process properly.
The pattern underneath matters more than the list. Successful work is narrow, embedded in a workflow rather than offered as a tool, and measured against a number somebody already cares about. Buying or partnering tends to beat building, and line managers who own the process tend to outperform central innovation labs. McKinsey found that the organisations reporting real value are around three times more likely to have redesigned the workflow itself, instead of inserting a model into the old one.
From wow to working model
Here is the part that gets the least applause and does the most work. Roughly nine in ten large organisations have a language model somewhere in production, while only about one in twenty pilots shows a measurable return. That gap is operational, not intellectual: prompt versions nobody tracks, retrieval indexes nobody owns, evaluations living in a spreadsheet, token spend nobody forecasts, and no audit trail when a regulator or a customer asks what the system did and why.
The discipline for this already exists. It is called MLOps, and the mistake is treating language models as a separate universe with a parallel stack. Prompts and retrieval indexes are versioned artifacts, like features and models. Evaluations and LLM as judge scoring are quality gates, like offline metrics. Traces, drift and token cost belong on one observability plane. One registry, one approval gate, one lineage graph, one policy engine, covering classic models and generative systems together. Regulation is turning that from good practice into an obligation: EU AI Act prohibitions applied from February 2025 and general purpose model obligations from August 2025, with penalties reaching 35 million euros or 7 per cent of global turnover.
Where this goes, honestly
I am wary of confident predictions in a field that redraws itself every six months, so here is only what I would defend. The floor will keep falling, which moves the interesting question from whether the model can do it to whether the workflow is worth automating. The ceiling will keep rising and staying expensive, because every new tier debuts at a premium, and most products will never need it. Agents will become genuinely useful in narrow, well instrumented places long before they are trustworthy for open ended work, so treating them as junior colleagues whose output you review is not pessimism. It is where the evidence sits.
And governance stops being a compliance chore and becomes the thing that decides who captures value. When everyone reaches the same models at similar prices, the differentiator is your data, your workflows, and your ability to run the thing reliably.
What I do not know: whether current architectures keep scaling into the reliability that removes the human from the loop, when multi step autonomy becomes dependable unsupervised, and how the capital expenditure cycle resolves for everyone downstream of it, including your hardware budget.
If you are uncertain, you are in good company, and uncertainty is not an argument for waiting or for going all in. It is an argument for scope. Pick one workflow that somebody already complains about. Instrument it before you change it. Put a model inside it, keep a human in the loop, and check in six months whether the number moved. That is unglamorous, and it is exactly how the wow turns into work.
