Forum Sentiment Analysis: The mood of the ‘nation’ in 1 graph

by 28

The Singletrack Forum is really quite a thing to study. In the last 14 years its accrued more than 10 million posts generated by more than 120k registered users (In 2021 we averaged 1.5 million visitors per month). In that time every topic and reply has been stored in our database from both the Chat and Bike forums. As has always been the case the Chat forum edges out the Bike forum in terms of popularity which means overall our forum tends to reflect every thing from the first economic crash in 2008 to the latest global crisis in eastern Europe.

More recently we’ve been monitoring what we like to call the ‘mood’ of the forum using some natural language machine learning tools created by Google. What this essentially boils down to is looking at every post in the forum and assigning it a score based on it’s perceived sentiment. Sentiment just means is the post generally positive or negative. The score that comes back is in the range of +1 to -1 with 0 representing a neutral sentiment.

There’s lots of clever stuff going on with the analysis like ignoring any post that’s too short and the like. It’s all statistics and it’s obviously down to the individual how that is interpreted.

Here is the result of the last 18 months of forum activity all melted down and converted in to a sentiment analysis. Each data point on this graph represents the average sentiment score for the forum posts made in that week, of which there are typically thousands.

The data is anonymous so there’s no telling who is responsible for anything on this chart and I’m posting it here essentially for fun, interest, curiosity.

We’d love to here your theories on what’s happening and if you want to check out more of the technology behind this then this is good place to start for the coders out there.

And just for fun you can actually try it out for yourself here.

forum sentiment analysis machine learning

The graph below represents the daily sentiment score of the forum over the last 90 days. You can clearly see that the forum was most happy on the 25th December.

machine learning forum sentiment analysis
Thanks for popping by - why not stay a while?IT'S FREE

Sign up as a Singletrack Member and you can leave comments on stories, use the classified ads, and post in our forums.

Join us, join in, it’s free, and fun.


Viewing 28 posts - 1 through 28 (of 28 total)
  • Forum Sentiment Analysis: The mood of the ‘nation’ in 1 graph
  • Premier Icon ScotRoutes
    Full Member

    LOL. But only happy on two days out of the last 90 😉

    Premier Icon teaandbiscuit
    Free Member

    This is great! Can you filter it further down or over a longer time? Would be interesting to see sentiment change as a result of covid, or if threads of the week have an impact. Are Fridays happier than other days, or Mondays if we’re sharing pics of weekend rides? Do your upbeat articles get more hits if launched on happier days or if people are miserable is that the time for an upbeat article?

    Interesting!

    Premier Icon kaiser
    Free Member

    Was it 2008 then it all started .. thought I was on here before that but perhaps wrong.

    Premier Icon ampthill
    Full Member

    How do I point the api at my posts?

    Premier Icon scuttler
    Full Member

    Can anyone pinpoint when Perchy stopped regularly posting (no cheating).

    Premier Icon weeksy
    Full Member

    I think in the coming weeks you may need to change the graph so it can go down further !

    Premier Icon nickc
    Full Member

    Have you able to compare different forum to see if here is wildly different to say Mumsnet or Pistonheads?

    Premier Icon Martinhuutch
    Full Member

    I’ll bet it’s really good at detecting sarcasm.

    Premier Icon Mark Alker
    Full Member

    Lots of cool questions.

    We can’t compare against other forums like mumsnet because we can’t connect to their database.

    We can’t go back further than 18 months as that’s when we first connected up the Google API. We can drill down further into the data though and I’m getting to grips with the analysis tools.

    Premier Icon Martinhuutch
    Full Member

    Are you going to purge the most miserable members? I feel I need to ask.

    Premier Icon footflaps
    Full Member

    Are you going to purge the most miserable members? I feel I need to ask.

    We need a good old witch hunt first to identify them, then we need a few weeks of persecution and maybe then purge or possibly just drown them as witches etc.

    Premier Icon thegeneralist
    Full Member

    I’ll bet it’s really good at detecting sarcasm

    ISWYDT 🙂

    Premier Icon imnotverygood
    Full Member

    Happy happy Happy happy Happy happy Happy happy Happy happy Happy happy Happy happy Happy happy
    Happy happy Happy happy Happy happy Happy happy Happy happy Happy happy Happy happy Happy happyHappy happy Happy happy Happy happy Happy happy Happy happy Happy happy Happy happy Happy happy Happy happy Happy happy Happy happy Happy happy Happy happy Happy happy Happy happy Happy happy

    That should help screw up the analysis

    Premier Icon Mark Alker
    Full Member

    Sentiment analysis

    Not by much

    Premier Icon fatbrad
    Free Member

    So there’s no analysis of Badger’s tubeless woes?

    Premier Icon Mark Alker
    Full Member

    You could always cut and paste older content into the demo box linked above if you like.

    It’s actually a relevant point to note that all the stored data we have is from the pandemic times so it could be that the forum was much happier before.

    Premier Icon hungry-monkey
    Free Member

    What happened on the 24th Feb I wonder..!

    Premier Icon hungry-monkey
    Free Member

    Ah – that’ll be the day Putin invaded Ukraine!

    Premier Icon LAT
    Full Member

    could this be used to check and correct the tone of emails so people at work don’t irritate each other?

    Premier Icon footflaps
    Full Member

    could this be used to check and correct the tone of emails so people at work don’t irritate each other?

    Those plug-ins already exist for Outlook and Teams…

    Premier Icon bikesandboots
    Full Member

    Not sure if I’d want irritating people hiding behind a tone checker. Better to just see the true person.

    Premier Icon MonkeySpacePilot
    Full Member

    Are you going to purge the most miserable members? I feel I need to ask.

    How can you know true happiness, without first suffocating in misery.

    Premier Icon LAT
    Full Member

    Those plug-ins already exist for Outlook and Teams…

    wow! it’s a long time since i was in that kind of employment. i’ve not even seen teams!

    Premier Icon matt_outandabout
    Full Member

    could this be used to check and correct the tone of emails so people at work don’t irritate each other?

    Does it work on zoom calls too?

    Premier Icon Tasha
    Full Member

    What is the y-axis metric? Is zero relative to a ‘internet’ baseline, or based on the dataset you put in i.e. the forum’s own ‘zero’?

    Premier Icon Mark Alker
    Full Member

    The calibration is done by Google and their Natural Language AI project. If you follow the link to the Google demo page you can test it yourself. On an individual basis with a single test it’s probably easy to confuse it, especially if you enter something sarcastic, but the reason it kind of works on our site is simply down to the huge amount of data it is being fed. It’s hard to overload the system with sarcasm when there are thousands of posts a day 🙂

    Premier Icon twisty
    Full Member

    How does the mood of Singletrack compare to other forums though?
    Just realised I’m at least the 3rd person to ask that – hope that doesn’t upset anybody 🙂

    Premier Icon footflaps
    Full Member

    How does the mood of Singletrack compare to other forums though?
    Just realised I’m at least the 3rd person to ask that – hope that doesn’t upset anybody 🙂

    Well the ‘average’ forum should be close to the Internet basline. However, averages don’t describe individuals so there will be individual forums more / less happy than the baseline and also more / less than STW.

Viewing 28 posts - 1 through 28 (of 28 total)

You must be logged in to reply to this topic.