DSDD
Making sense of feedback
Finding patterns in customer complaints about Scotiabank.
At Scotiabank Data Science Discovery Days, we worked through customer app reviews to find recurring complaints. A low rating tells you someone had a bad experience. The text tells you whether it was a failed login, a missing notification, or a transfer that would not go through.
Finding the topics
We cleaned the text and used BERTopic to cluster related reviews. Inspecting the topic keywords helped us interpret those clusters: “Factor”, “2 Step”, and “Verification” all pointed to two-step verification, so we grouped them under 2SV.
That inspection also caught a cluster full of “room”, “hotel”, and “stay”. The data had travel-related feedback mixed in. Looking inside the clusters exposed that before we treated every topic as a banking problem.
One review, several issues
A review can describe several failures at once. We explored one-shot and few-shot prompting: give the LLM examples of review-to-tag mappings, then ask it to assign issue labels. Missing notifications and broken verification belong to two tags, even when they share a sentence.
One shared issue label
“Barely works. No notifications now 2SV doesn't work”
We used those issue groups to connect complaints to product changes: more reliable biometric login, clearer balance information, and real-time support. The tags kept the recommendations tied to specific problems in the reviews.
I worked with Harsha Puranik and Karthik Krishna as Team Tango. We placed third and won $1,000.
