GPT-3 and the new baseline for language models
What the release of GPT-3 means for search, support, text analytics and product UX - a view for founders and directors.
In late May 2020, OpenAI published a paper describing GPT-3 - a language model with parameters an order of magnitude larger than its predecessors. The model demonstrated that at sufficient scale, language systems begin handling tasks they were never explicitly trained for. Summarisation, translation, code writing, question answering - all without separate fine-tuning for each task.
I have been watching the development of language models for several years. GPT-3 is not the first large model and certainly not the last. But it sets a new reference point that is worth understanding at the business level, not just the technical one.
What changed compared to previous models
Before, getting a language model to perform well on a specific task required fine-tuning it on examples of that task. If you want a sentiment classifier for reviews - you need labelled reviews. If you want a named entity extraction model - you need entity annotations.
GPT-3 demonstrates something fundamentally different: describing the task in plain language and including a few examples in the request - and the model begins handling the task without retraining. This is what practitioners call few-shot learning. Not perfect quality, but sufficient for many practical scenarios.
This does not mean specialised models have become unnecessary. But it changes the conversation about when it makes sense to start an experiment.
Where this changes product possibilities
Search and document work. One bottleneck in corporate systems is search across documents, knowledge bases, internal guidelines. Classical search works on keywords. Language models understand the meaning of a query and can find relevant answers even when the exact words do not appear in the document. For companies with large volumes of internal documentation, this changes the scenario.
Customer support. Not a replacement for agents, but a different level of automated request handling. Classification, prioritisation, automatic responses to common questions become more precise when the model understands natural language rather than just keywords.
Unstructured text analytics. Customer reviews, support requests, public data - a large volume of text that most companies cannot systematically analyse. Language models make this class of tasks more accessible.
Interface as a query. The idea that a user can interact with a system through a plain text request rather than menus and forms is not new. But GPT-3's comprehension quality brings it closer to a practically applicable level for certain scenarios.
Where expectations should be tempered
GPT-3 is a language model, not a knowledge system. It generates plausible text but does not verify it against facts. For tasks where accuracy is critical - legal texts, medical data, financial calculations - this is a significant limitation.
The model is very large. The API is available to researchers via limited beta access; there is still a distance to product scenarios at industrial scale. This is not today's story for most businesses - it is the story of the next few years.
Working with personal data through external APIs raises regulatory questions that need to be resolved before, not after.
The practical takeaway
GPT-3 does not change what needs to be done today with data and infrastructure. But it raises expectations about what will be possible in the coming years. For companies that work with large volumes of text - in support, in analytics, in knowledge management - it is worth starting to think about where language processing can create real value while the technology moves toward accessibility.
Not as a hype case, but as a concrete question: what text in our business is currently processed manually because there is no good enough tool?