Ad Fontes Media is proud to announce its first extensive multi-analyst content ratings research project!
Since the first Media Bias Chart was launched, people have been asking for more data and transparency about the news source ratings. Within the next two weeks, we are kicking off a project to provide exactly that. During this project, we will be performing ratings of nearly 2,000 individual articles and TV news shows, which will be conducted by 20 dedicated, trained analysts having political viewpoints from across the spectrum. Once the research project is completed, we will publish the results via an interactive version of the Media Bias Chart, which will show where each article and TV show was ranked by our analysts.
Each of the nearly 2000 individual articles and TV news shows will be rated by four analysts with different political views. This means each analyst will be analyzing about 370 articles and about 17 TV shows, and each analyst will be rating at least four articles from each of the 100 news sources on the chart. As a result, we will have nearly 8,000 individual ratings.
The multi-person ranking per article is designed to minimize the impact of any one person’s political bias on the published ranking, and the breadth of coverage by each analyst over all of the sources is designed to enhance each analyst’s familiarity with sources across the spectrum.
The content ratings research project will run over a course of nine weeks, with three study periods of three weeks each. The articles and shows that will be rated will cover a minimum of 15 articles per source, but for some larger sources we will rate up to 90 articles.
The sample sets of articles and shows will be pulled from sites three times throughout the project, and each time, all articles will be pulled on the same day, meaning that they will be from the same news cycle. The purpose of pulling all articles from the same day is so that analysts will be able to incorporate evaluations of bias by omission and bias by topic selection.
The type of rating we will ask each analyst to provide is an overall coordinate ranking on the chart (i.e., “40, -12”). These rankings will be based on the methodology shown and described in Ad Fontes Media’s grading rubrics (which you may have seen on our website). The ranking methodology is rigorous and rule-based. There are many specific factors we take into account for both quality and bias because there are many measurable indicators of each. Therefore, the ratings are not simply subjective opinion polling, but rather methodical content analysis. Overall source rankings will be composite weighted rankings of the individual article and story scores.
This project is happening because of the support of all of you in our News Nerd community. Everyone who has shared the Media Bias Chart, engaged in conversations, taught media literacy, and crowdfunded the project itself have been critical to its progress. It’s also happening because of the fabulous individuals on our team who have stepped up to do the hard work of content analysis. They are an exceptional group.
We look forward to updating you on the progress of this content ratings research project as we go along!