Text analysis becomes practical: what it means for business
Natural language processing tools have reached the point where they can be used without a research lab. What to do with that.
A few years ago, analysing text at business scale was a task for research labs and large technology companies. The tools existed, but they required specialised knowledge and significant computing resources.
In 2014 the picture is different. Natural language processing libraries have become considerably more accessible. Tools have appeared that analysts without a linguistics background can work with. Cloud computing removed the performance barrier. For business, this means: tasks that previously required a research specialist can now be approached by teams with ordinary technical resources.
What text analysis tools can do right now
Without exaggeration: a few classes of problems are handled reliably enough.
Text classification. Automatic sorting of enquiries, tickets, or reviews into categories - works well given enough training examples. Reducing the time spent on initial triage of incoming message volume is achievable.
Information extraction. From unstructured text you can pull out specific entities: organisation names, dates, amounts, addresses. Useful where there is a stream of documents or messages from which specific data needs to be extracted.
Sentiment analysis. Determining "positive / negative / neutral" in reviews, enquiries, comments - a task that can be solved with acceptable accuracy for most business cases.
Semantic search. Finding similar documents or grouping texts by meaning without predefined categories - the shift that word2vec made practical for applied search.
Where it does not work
Subtle language nuance, sarcasm, domain-specific professional jargon, short texts without context - all of these reduce quality. Systems trained on one domain transfer poorly to another without retraining.
This is a genuine limitation worth checking against your specific data before committing.
Three practical scenarios for business
First: automatic routing of incoming requests. If you have a support function where enquiries are read manually and assigned to agents - automatic classification can handle the initial triage. Not replace the agent, but remove the mechanical work from them.
Second: review and mention monitoring. Automatic collection and initial sentiment assessment of reviews about the company or products - at large volumes this saves analytical time.
Third: structuring incoming documents. If your business processes a stream of unstructured documents - applications, letters, reports - and part of the work is extracting specific data from them, automating that step is realistic.
How to assess task readiness
Before investing resources, a few things worth checking.
Do you have enough texts for training or testing? If not, results will be unreliable, and you need to accumulate data first.
Can you state the task clearly: what exactly should the system classify or extract? Vague tasks produce vague results.
Are you prepared for the system to be wrong in some fraction of cases, and is there a review and feedback mechanism? Without that, quality will not improve over time.
If the answers are positive, the task is likely solvable with reasonable effort. If not, start by clarifying the baseline conditions.