We’ve just wrapped up our first major data collection effort for the next edition of the Media Bias Chart, which will be interactive! Here are some highlights of what our dedicated team of analysts, coordinators, and developers accomplished, and what’s next:
Each analyst completed over four hours of training on Ad Fontes’ methodology for analyzing the content of individual news articles and TV shows. In these trainings, we covered how to account for veracity, expression, fairness, headlines, graphics, show guests, political positions, terminology, omission, and comparison.
Analysts took these factors into account, but also had discretion to weigh other factors on a case-by-case basis, which we believe is necessary for accuracy. After all, there are many reasons articles can be higher or lower in reliability, or more or less biased. Analysts gave each article and show a two-dimensional score—one on the y-axis for reliability and one on the x-axis for bias, and had the option to include comments on reasons for individual ratings.
We rated nearly 2,000 individual news articles from over 100 news sources. We also rated most of the daytime, evening, and weekend news shows on CNN, MSNBC, and Fox. We collected at least three ratings for each: one from a left-leaning analyst, one from a centrist analyst, and one from a right-leaning analyst. Each of the analysts completed self-rating spreadsheets on their political leanings for twenty different political issues prior to their participation in the project. Each of them analyzed articles from each of the 100+ sources; the exercise of rating articles from that many sources gave them a perspective that most news consumers never experience.
The rankings of each article and show will be displayed on the interactive chart in addition to the ranking of the overall source, so you can see how individual article and shows are used to determine overall source rankings
To wrap up this initial ratings period, we had a call with our analysts and team to debrief and collect feedback about how we can improve things moving forward, on everything from our training and methodology to our logistics, software, and more.
That is the exciting thing, though—because we conducted this first major ratings project, we will be able to move forward. We now have the logistical and technology infrastructure to expand this project to continuously rate more and more sources, building up our database and improving this resource over time.
We were able to execute this ratings project due to the very hard work of our small team. A special thanks goes to Robert Rapplean, our database architect and developer of the ratings interface our analysts used to submit rankings, Romano Nickerson, our front-end data visualization developer, and Ria Otero, our operations guru who handles day-to-day logistics.
Our dedicated coordinators and analysts did the very hard grunt work of selecting articles, handling logistics, and of course, performing content analysis. We have several other team members and advisors who have dedicated time to this project that are important in too many ways to list here (but I’ll go ahead and list Wally Dean, Maxwell Stearns, and Keven Ellison, who have been instrumental since the very early days). You can see the bios of our team and advisors on our site now, and we’ll be adding those of our analysts on our new site here shortly. The thing they all have in common is that they believe we can make society better by making our news better, and I’m so grateful for them.
For the next few weeks, we’ll be working hard on putting out the interactive edition of the chart on our updated site. You’ll be able to explore our source rankings and thousands of data points. We’ll always have our main chart and its primary functions free and available for people to check as a reference on our site, because part of our mission is to make news consumers smarter. To do that, the majority of our source rankings have to be free for all to access.
For those of you with a deeper curiosity for data and a need for more features, we’ll have quite a few interactive tools available by subscription. You’ll be able to customize your own licensed versions of the chart and will have access to data and visualizations beyond the free version. Some of you who were crowd-funders of this project back in November have already secured your subscription access!
Subscribers will not only get additional valuable insights about the media landscape; they will also help fund the continuation of this work. We plan to increase the number of sources rated on the interactive version of the chart to 200 in three months and 400 in six months, and we have plans to grow beyond that to rate substantially all US news sources.
Thanks to all of you who have been watching and cheering on the success of this project. We have seen how this concept of mapping the media landscape has helped people bridge political divides and create meaningful consensus, so we are motivated to do the best job we can. Your online shares, messages, and comments mean the world to me personally and to our Ad Fontes team.
I hope you are spending this Memorial Day weekend with those you love. Let us honor those who gave all they had to give to this country by striving toward that more perfect union.