GPT-2 and language models: what the signal means for business right now
After GPT-2, the conversation about text generation shifted. I look at what actually changes for companies today and what is still in the lab.
In February 2019, OpenAI published results for GPT-2 - a language model that generates coherent text at a level the public had not seen before. The reaction was uneasy: the lab was criticised for deliberately holding back the full version on safety grounds. They eventually released it. The conversation about the capabilities and risks of text generation moved out of research communities and into the mainstream.
I have been watching this space for several years, and right now I see two distinct conversations that people keep conflating. The first is about disinformation and synthetic content at scale. The second is about what is practically applicable inside a company. These are different topics, and for an owner or manager the second one matters more.
What changed technically
Language models existed before, but GPT-2 marked a qualitative shift in one specific place: the coherence of generated text over longer stretches. Earlier models handled short next-word predictions well but lost the thread after a few paragraphs. This model holds context longer.
The practical consequence is that tasks requiring a person to "write connected text from a template" become candidates for partial automation. Summarisation, drafting from structured data, responses to routine requests.
An important limitation that often gets missed: the model knows nothing about your business. It generates plausible-sounding text based on language statistics. That is fundamentally different from "knowing something." Without feeding the right context into the prompt, the output will look fluent but be factually hollow or wrong.
Where this is applicable today
There are a few areas where the technology is already mature enough for a careful pilot.
Classification and routing of incoming messages. If you have a flow of requests - from customers, partners, internal tickets - a model can help identify the type of request and direct it to the right person. This is comprehension, not generation, and language models are more reliable here.
Drafts from structured data. If you have template texts that shift slightly depending on numerical parameters - reports, notifications, standard letters - a model can produce the draft. A person edits rather than writes from scratch.
Search across internal documents. Not keyword search but semantic search. This direction is maturing and already produces results in specific tasks.
Where you should not expect miracles yet
Anything requiring factual precision - financial documents, legal texts, technical specifications. The model cannot distinguish truth from plausibility, and without human verification that is a real risk.
Any customer-facing communication where a mistake is reputationally expensive. A draft inside the company is fine. A public response without review is not.
Tasks where there is not enough example material for fine-tuning. A general language model without adaptation to your domain often produces texts that sound right but do not match your context and terminology.
Signals worth watching for
I am not suggesting anything urgent needs to change. But if you want to understand how relevant this is for your company, a few questions:
- Do you have a routine flow of text requests that a person currently processes following template logic?
- Do you have documents or knowledge bases where employees keep searching for the same information?
- Do you have standard texts that get "rewritten" each time following the same structure?
If even one answer is yes, it makes sense to start understanding this now, before a competitor does. Not because it is urgent, but because the work takes time: collecting examples, understanding where quality becomes acceptable, building a review process.
The technology is moving faster than operating practice in most companies can usually keep up with.