The Media Bias Chart Content Ratings Research Project

Ad Fontes Media is proud to announce its first extensive multi-analyst content ratings research project!

Since the first Media Bias Chart was launched, people have been asking for more data and transparency about the news source ratings. Within the next two weeks, we are kicking off a project to provide exactly that. During this project, we will be performing ratings of nearly 2,000 individual articles and TV news shows, which will be conducted by 20 dedicated, trained analysts having political viewpoints from across the spectrum. Once the research project is completed, we will publish the results via an interactive version of the Media Bias Chart, which will show where each article and TV show was ranked by our analysts.

Each of the nearly 2000 individual articles and TV news shows will be rated by four analysts with different political views. This means each analyst will be analyzing about 370 articles and about 17 TV shows, and each analyst will be rating at least four articles from each of the 100 news sources on the chart. As a result, we will have nearly 8,000 individual ratings.

The multi-person ranking per article is designed to minimize the impact of any one person’s political bias on the published ranking, and the breadth of coverage by each analyst over all of the sources is designed to enhance each analyst’s familiarity with sources across the spectrum.

The content ratings research project will run over a course of nine weeks, with three study periods of three weeks each. The articles and shows that will be rated will cover a minimum of 15 articles per source, but for some larger sources we will rate up to 90 articles.

 The sample sets of articles and shows will be pulled from sites three times throughout the project, and each time, all articles will be pulled on the same day, meaning that they will be from the same news cycle. The purpose of pulling all articles from the same day is so that analysts will be able to incorporate evaluations of bias by omission and bias by topic selection.

The type of rating we will ask each analyst to provide is an overall coordinate ranking on the chart (i.e., “40, -12”). These rankings will be based on the methodology shown and described in Ad Fontes Media’s grading rubrics (which you may have seen on our website). The ranking methodology is rigorous and rule-based. There are many specific factors we take into account for both quality and bias because there are many measurable indicators of each. Therefore, the ratings are not simply subjective opinion polling, but rather methodical content analysis. Overall source rankings will be composite weighted rankings of the individual article and story scores.

This project is happening because of the support of all of you in our News Nerd community. Everyone who has shared the Media Bias Chart, engaged in conversations, taught media literacy, and crowdfunded the project itself have been critical to its progress. It’s also happening because of the fabulous individuals on our team who have stepped up to do the hard work of content analysis. They are an exceptional group.

We look forward to updating you on the progress of this content ratings research project as we go along!

By |2019-02-16T22:21:13+00:00February 16th, 2019|Uncategorized|4 Comments

About the Author:

4
Leave a Reply

avatar
3 Comment threads
1 Thread replies
1 Followers
 
Most reacted comment
Hottest comment thread
4 Comment authors
Jim hurlVanessaachiappanzaAlethea Kelly Recent comment authors

This site uses Akismet to reduce spam. Learn how your comment data is processed.

  Subscribe  
newest oldest most voted
Notify of
Alethea Kelly
Guest
Alethea Kelly

Hi Vanessa. What goes in to determing if a statement is true? Also, what use is a scale of 1-5 in this regard? If it’s mostly true, it’s partly false, meaning it is false. Or am I missing something? Thanks!

achiappanza
Guest

This is very interesting and cool. Do analysts go back and re-rate earlier articles if they decide their vision of the spectrum needs recalibration?

Jim hurl
Guest
Jim hurl

I noticed that ALL CENTER FACTUAL sources are still pushing the conspiracy theory of trump Russian collusion…traitorous…and the media sources that proved it wrong…are listed as not factual propaganda…the centerline should move right past all the conspiracy laden media…please correct graph as you are supporting propaganda…I bet you. won’t print this comment…as it is Negetive and points out your left leaning bias…that’s ok I shared this story and my comment on Fbook…Logical Conjecture…11419

We are working hard on the Media Bias Chart project, and will have new updates to it soon. Don’t miss the latest versions!

Sign up to receive email notifications whenever new content is published.