Experiment 02
ChatGPT Review Analysis
A star rating shows how much users like an app, not why. The review text holds the why, but nobody can read 197,000 reviews by hand. So I taught Python to read them, and checked where it was wrong.
- Period
- Built Oct 2026; reviews Jul 2023 to Aug 2024
- Stack
- Python 3.12, pandas, TextBlob, NLTK, seaborn, scikit-learn
- Data
- 196,727 raw reviews, 193,148 after cleaning
- Context
- Crio.Do DA-201, session 5
What do users feel about ChatGPT, how strongly, and about what?
- What is the general sentiment? (polarity: positive, neutral, negative)
- How strong are the opinions? (subjectivity: 0 factual to 1 opinionated)
- Which features do people praise or criticise most?
The data: Google Play reviews of the ChatGPT Android app, from 25 July 2023 to 23 August 2024.
Clean, score, validate, then look for themes
- Prepare. Standardise column names, drop 6 empty reviews and 3,573 duplicate review IDs. Duplicate text stays: two people who wrote "good" are two votes.
- Score. TextBlob polarity and subjectivity. Positive above +0.05, negative below −0.05. Chosen because it returns both numbers, is fast and is explainable.
- Validate. Compare sentiment with star ratings, study the mismatches, and check non-English reviews separately.
- Mine. Top words and two-word phrases for 4–5★ and 1–2★ reviews, keeping negations so "not working" stays distinct.
- Aspects. Share of praise and criticism mentioning errors, login, pricing, accuracy and helpfulness.
People like it. They are annoyed by reliability and access, not by the AI
- 193,148reviews after cleaning
- 4.51 / 5average rating
- 76 · 21 · 4% positive, neutral, negative
- 0.25rating vs polarity (Spearman)
Praise is about usefulness: helpful, best ever, helps a lot, user friendly. Criticism is mostly about reliability and access:
Stars and text disagree more than you would expect: the correlation is only 0.25, and about 1 in 8 low-star reviews use clearly positive wording (probable mis-taps, mixed reviews, typos such as "Nott good").
A lexicon-based model has blind spots
- TextBlob misses sarcasm, slang and context, and the median review is only 4 words. About 20% of reviews score exactly 0.
- It is English-only. Reviews with emoji or other languages land in "neutral" about twice as often (about 35% against 18%), which inflates the neutral share.
- Aspect detection uses simple keyword rules, so the percentages are a lower bound.
- One platform and about 13 months of data.
Ideas for further work: transformer-based multilingual sentiment, language detection, topic modelling on 1–2★ reviews, and linking review spikes to release dates.
Contact
Got a messy dataset or a science question?
I read everything and answer when I can. The quickest way is email, and LinkedIn works too.