Edison Research has released the first edition of Edison Podcast Metrics UK, its attempt to provide a chart that benchmarks the reach of all of the UK’s podcasts.
There are a number of ‘podcast rankers’ around the world – these usually bring together a number of the main networks who share their download data with a third party who publish the data. The challenge with these rankers is that not everyone participates. In the Australian ranker, for example, Spotify and the ABC don’t (currently) take part. So whilst it gives a lot of data, it’s not complete.
In the UK we don’t have a centralised ranker. RAJAR have looked at it, as have some other people, but they haven’t been able to get all the participants to agree.
In the US, as well as the Triton ranker, Edison Research created their own chart, but with some different methodology. Instead of taking the download data from the networks, they just did a big survey that asked listeners what podcasts they’re listening to. This way you don’t need to get networks to buy in, or provide data. The results are the ranker includes any podcast made by any network. Here’s their recent US data.
But surely the download-based rankers are more accurate? Well, perhaps not. They include data from every episode downloaded but it can’t tell whether anyones listened to that file. I’m sure we all have a load of shows that have been downloaded to our podcast app but not listened to. Measuring downloads also means you can’t tell if multiple people have tuned in – a family listening in a car for example.
With survey-based research, like the Edison one, you’re asking people what they can remember they’ve listened to. Whilst comprehensive – it will catalogue any show mentioned – the main challenge is the sample size. Can you reach enough people to provide a comprehensive overview?
The current release of the UK data says they interviewed 2,273 people from April 2023 to June 2023, a three month period, so I imagine circa 200 people a week.
As well as the main chart they’ve released, subscribers will also be able to access broader demographic breakdowns and network-reach numbers.
I’ve always been pretty fascinated by this sort of exercise. My corporate radio career involved lots of research – both quantitative and qualitative – and I’ve been thinking about podcast data for quite a long time.
Indeed, when me and Matt Hill ran the British Podcast Awards, we had a go at creating our own UK podcast ranker at the beginning of 2021.
We worked with research company Kantar and spoke to around 9,000 people to find 3,000 podcast listeners over 12 weeks. Back then our Top 5 was:
- The Joe Rogan Experience
- BBC Friday Night Comedy
- Happy Hour Podcast (Jaack Mate)
- That Peter Crouch Podcast (BBC)
- Shagged Married Annoyed
It was a fascinating exercise and quite a complicated one. We also spent quite a lot of money on it. Our attempt was to see whether we could create perhaps a weekly or monthly chart. As you can maybe tell, it wasn’t something that we ended up launching!
The key challenge was the sample size and the long-tail nature of podcasting. Now, the shows up the top of the list definitely passed the smell test. Did I, as a podcast expert, think that looked like the biggest shows? Sure. Even better, the demographic data looked pretty resilient too. Our problem was that as you went down the list the data started to look a little more wobbly. There were only a few shows that had hundreds of our respondents listening. Our top show – Joe Rogan – only had a reach of 5% of the audience. Pretty quickly we were into just tens of people saying they tune in. Overall, we felt it would be hard to provide something comprehensive to the sector, so we went back to the drawing board.
Now I have a lot of time for Edison Research. They’re easily one of the best audio research companies in the world, and nice people as well. They will have been much better than us at doing the research. So lets look at what they’ve come up with. Their ‘top 25’ is as follows.
This definitely passes the ‘smell test’. I could absolutely believe they were the top 25 shows in the UK.
The challenge is that there are charts that we do all see in podcasting – that’s the Apple and Spotify ones – where even though we know that the algorithm is designed to reflect changes in popularity, rather than total downloads, it does skew our perception slightly.
The fact the Edison research measures recall – shows that listeners can remember what they’ve tuned into – and also means that it will be skewed towards the more popular, memorable shows. If I asked you to name the podcasts you follow in your app, how many could you remember? Five’s probably do-able. Could you list 10? If you read this newsletter then you probably follow many more.
Update: Megan from Edison dropped me a note about this:
When respondents are asked to list the podcasts they’ve listened to in the past week, they are also asked to consider all the places they consume them (e.g. apps, services, social media, websites, YouTube). They are then given extensive instructions on how to access their listening histories to ensure the list is as comprehensive as possible. It is a multi-step process to collect responses from study participants.
There’s also some interesting things around names. Friday Night Comedy doesn’t appear in the list – but then do people refer to the shows on it, like Dead Ringers or The News Quiz? The BBC shows they include are roughly in the order of the BBC’s data for the same quarter – though Edison picks up shows like Moneybox and Infinite Monkey Cage that don’t seem to trouble the BBC’s own downloads-based chart.
This, of course, is one of the issues – the methodology is different so you wouldn’t expect the same results.
Similarly comparing some of the shows that are hosted by Acast – Off Menu, Shagged Married Annoyed, That Peter Crouch Podcast, The Therapy Crouch, Happy Place and Newlyweds – the order is pretty close when comparing the actual UK downloads Acast tells its advertisers. However it misses some that, based on downloads, you would expect to see – Guardian Football Weekly, The High Performance Podcast – that rank higher than some of those listed.
Now who’s to say these didn’t appear in places 26-30? Again, the methodology is different to measuring downloads.
Weekly reach is also very different to total downloads from a show. Today in Focus, from the Guardian, has as similar weekly reach to Newlyweds (though doesn’t make the list above) but, with five episodes a week, it does nearly five times the total downloads. The latter is clearly a ‘bigger’ show, but, once again, it depends how you measure it.
Finally, the chart data combines the recall of three months of respondents. This definitely means it will lean into shows that run all year round. There’s, in effect, 12 opportunities to capture people mentioning a show. If you’re only a six part series, you might be one of the most popular shows for a short period of time, but you may miss out on a chart placement.
Summary
So, is the Edison Research chart a comprehensive list of the top 25 shows in the country? No. Is it a pretty good approximation? Yes. But you need to remember what’s being measured.
AOB
- In this week’s Media Podcast, I catch up with Charlotte Tobitt, Editor of Press Gazette and Chris Lockery, Editor of Popbitch to talk about Russell Brand, Rupert Murdoch and more. Listen here. Watch here.
- I’m looking forward to speaking at Podnews Live this Wednesday where I’m going to be talking about what makes a Top 200 show. I’m also at the, now sold out, British Podcast Awards on Thursday. Do say hello if you see me at either.
- If you want more on the Edison Research study, Adam Bowie’s written about it too.