Media Bias Chart 4.0: What’s New2018-10-03T17:34:34+00:00

Media Bias Chart 4.0: What’s New

In addition to the aesthetic changes, quite a lot is new, as I’ll explain here. But if your first thought is “whoa, it’s hard to see all those sources,” don’t worry! This version is searchable and enlargeable at adfontesmedia.com*, AND, soon, we’ll be launching an interactive web version of it where you can select additional sources to view and deselect some to see more clearly.

This soon-to-come interactive web version will be just the first iteration of the Media Bias Chart Web App.  You can help us build the features you want to see by supporting our crowdfunding effort, which is now live! Click here to chip in.

As you can see, in certain areas of the chart, we’ve reached the limit of sources we can add while still being able to see them. There are countless more sources out there, of course. We’d like to include them, but we need to move to the interactive version for that. If you want to see more sources, please see the aforementioned crowdfunding effort 🙂

*(I changed the name of the website to the name of the company I started to expand this work! Mediabiaschart.com and allgeneralizationsarefalse.com still work though!)

*(I changed the name of the website to the name of the company I started to expand this work! Mediabiaschart.com and allgeneralizationsarefalse.com still work though!)

What’s New:

  • No more circles and ellipses in the background. In previous versions, those were meant to express a range in which individual stories would typically fall for sources within them. But that was a bit confusing. In place of this, on the interactive version, you will be able to click on a single source and see a sub-chart for shows and/or articles ranked within that source.
  • Sub-charts! If you’ve ever wondered how the different shows on the TV network affect the overall ranking, take a look at these first ones for Fox News, MSNBC, CNN, and TYT (The Young Turks) Network. (link)
  • The green, yellow, orange, and red lines overlap and cover more areas. These overlaps show more intuitively that sources within the overlaps consistently comprise individual articles that fall across the categories.
  • The designations “left” and “right” replace “liberal” and “conservative” for a number of reasons, the most important of which is that many political concepts fall outside many people’s contemporary definitions of “liberal” and “conservative” (even though “liberal” and “conservative” can quite broadly defined in contemporary usage). “Left” and “right” are broader.
  • The lines are evenly spaced, which shifted the distribution of sources. While they still fall in somewhat of a bell-curve, they do so less than before.
  • The category “Complex Analysis” has been updated to “Complex Analysis OR Mix of Fact Reporting and Analysis.” This update more accurately reflects the existence of a large and important genre of news that combines fact reporting and analysis as necessary for context and ease of understanding. Not all of these kinds of sources provide what we could call “complex analysis,” though. In fact, many stories in this genre of news are quite short. In terms of overall quality, they are often just as important in the news eco-system as complex analysis stories.
  • There is alt-text in the image to make it accessible to the visually impaired.
  • There is an underlying coordinate system to the categories, which is on an arbitrary scale of 0 to 64 vertically and -42 to +42 horizontally. The numbers marking the division lines between the categories are visible. If you are having trouble finding a particular source, there’s a table next to it on the home page that has the coordinates of each source. You can sort it alphabetically or numerically.

New Sources:

-Business Insider

-Daily Signal

-Financial Times

-Forbes

-Fortune

-FreeSpeech TV

-Guacamoley

-IJR

-LA Times

-Marketwatch

-News and Guts

-OZY

-Quartz

-ShareBlue

-The Gateway Pundit

-The Skimm

-The Young Turks

-Talking Points Memo

-Think Progress

-Truthout

-Twitchy

-Vice News

-Washington Monthly

-WND

-Wonkette

Individual Source Changes:

In general, the refinement of my methodology resulted in some minor repositioning of many previously existing sources. Generally, the changes reflect a more representative sample size of the types of stories consistently presented by the respective news sources In a few cases, the methodology refinement has resulted in some major shifts.

BuzzFeed has moved higher in the quality metric for one main reason, which is that my ranking no longer disproportionately punishes BuzzFeed News for that one article about the Steele Dossier. (See here for the controversy surrounding the publication of that article) I, along with many other media observers, viewed that editorial decision by BuzzFeed to be a breach of journalism ethics. I still view it as such, even if the contents of the dossier ultimately prove to be partially or completely true, because it is a journalistic ethical standard to report things as “news” only if they have been verified to have happened, at the time they have been verified, and not earlier. This is why certain publications on the chart are ranked so low for quality, such as the National Enquirer (specifically because they publish things that are poorly sourced, in addition to their actual false headlines and articles) and the Palmer Report (because they publish speculation and leap to conclusions as if they are facts). In this ratings system, no one gets points for making predictions and publishing them as if they are news.

However, in the process of standardizing my methodology, I have had to approach controversial and outlier articles more thoughtfully, and in the context of the entire body of work of the news source. I noticed that commentators on my blog would refute the placement of one of the sources on the chart by pointing to one controversial or unrepresentative article from the source. Essentially, they would say “the NYT is in “fact reporting/skews liberal”?!?! Ha! Look at this one crappy, hyper-partisan opinion article they published!” I found those comments unfair, because one article does not make up the whole ranking. In fact, I have said over and over that an overall source ranking is made up of ratings of many individual articles and stories. Therefore, it was unfair of me to put BuzzFeed News down so low and to the left in view of that one article. Its current ranking is more reflective of its representative stories.

That leaves the question of how to deal with these controversial and outlier stories, because they do still affect the quality and bias measures of a source. Keep in mind that the placement of a source represents a weighted average of scores of articles. Further, the ranking of an article represents a weighted average of the sentences. Really low-quality/ highly biased sentences really drag down (and out) the score of an article, and really low-quality articles really drag down (and out) the score of a source. For more on this, see our updated methodology.

Controversial and outlier articles are often, rightfully, quite damaging to the reputation of a source in the minds of news consumers, especially when they are recent (e.g., the BuzzFeed scenario discussed above). To deal with these outliers, future versions of the chart will have both weekly and historical moving averages. That way, users can see if a source took a credibility hit in a particular week, but can also see what that source’s credibility track record has been over time.

Another source that moved quite a bit was NewsMax. Admittedly, my previous sample was not very representative of their typical stories. While they are certainly very, very conservative, especially in terms of topic focus, their stories typically comprise mostly fact reporting and analysis (and prominent reprints of AP/Reuters stories). My more complete sample of ratings resulted in a higher vertical placement. Similarly, Drudge Report, which is primarily an aggregator, was moved up and more toward the center for similar reasons. The majority of its content is from other sources, and the overall ranking mostly reflects the averaged rankings of the aggregated sources. The conservative bias reflected in its original content and in its top story/headline selection push the average downward and rightward.

 

Health Sources

Another change was that I removed low-quality “health information” sites such as David Wolfe and Natural News. I removed them for a couple of reasons. First, although they do produce some politically-oriented content (which is mostly garbage), most of their content is geared toward health, lifestyle, and the environment (most of this is garbage as well). This chart primarily deals with sources which either primarily cover politics OR contain a significant amount of political content as a percentage of their overall content. To be sure, many sources on the chart have a significant percentage of non-political, news-reporting content, sometimes in widely diverging areas. For example, Vanity Fair and Forbes, which focus on fashion and business news, respectively, have niches in different areas but regularly devote part of their coverage to political topics. Therefore they have a significant enough percentage of regularly relevant political stories that lend themselves to a placement on the chart. The junk health sites tended to have more randomly timed and inconsistent political content.

Second, it has become apparent that the variation of quality of scientific and health information is a problem that would likely be better addressed via a separate chart. I’ve actually had several requests from medical librarians and scientists about this very concept. The taxonomy would have to have some different measures vertically and horizontally. Perhaps in the future this can be a separate project. For now, though, I’ve removed the primarily health-focused garbage sites. In future versions, I may include individual rankings of their purely political stories.

The Future

As you can see from this update and the sub-charts, there are a lot more things the Media Bias Chart can provide. In particular, there’s a huge need for more sources, more features, and more transparent data.

We’ve gotten a lot of feedback so far about what you want to see—please keep that feedback coming. Check out what we have planned for the future and join us if you can!