Social Sentiment at Scale: A Pipeline That Survives Real Text
Sentiment models look easy until they meet sarcasm, spam and emoji. Most of the accuracy came from what we removed before classification, not from the classifier.
- Client
- Brand monitoring team
- Industry
- Marketing analytics
80–90%
Classification accuracy
10,000s
Posts per run
3-class
Positive / neutral / negative
Repeatable
End-to-end pipeline
The problem
The team needed to track sentiment across social channels at a volume no one could read manually. Off-the-shelf sentiment scoring had been tried and produced numbers that moved for reasons nobody could explain — usually because spam, bot posts and irrelevant mentions were being scored alongside genuine ones.
A monitoring signal that cannot be trusted is worse than no signal, because decisions get made on it anyway.
What we built
A preprocessing stage that does most of the real work: normalising casing and punctuation, handling emoji and hashtags as signal rather than noise, stripping boilerplate, and filtering spam-like and bot-shaped posts before anything is classified.
Feature engineering over the cleaned text, then a three-class classifier — positive, neutral, negative — with the neutral class treated as a real category rather than a dumping ground, since most social mentions are genuinely neutral and forcing them to a pole is what makes sentiment dashboards oscillate.
The whole thing runs as a repeatable pipeline that processes tens of thousands of posts per run, so tracking over time compares like with like.
Results
Classification accuracy sits in the 80–90% range depending on class balance in the batch — the honest way to state it, since a batch skewed toward one class scores differently from a balanced one.
The larger practical gain was noise removal. Filtering irrelevant and spam-like content before classification meant the reported trend reflected actual audience sentiment rather than bot volume, and the numbers started moving for reasons the team could explain.
What we'd flag
Sarcasm and domain slang remain the hard cases, and no general-purpose sentiment model handles them reliably. Where a category matters commercially, a small labelled set from your own channels beats any generic model.
We would also caution against reading small movements as signal. Within the accuracy band, a couple of points of drift is noise, and the pipeline is built for trend direction rather than precision to the decimal.
