Reading Charts

Jason Oberholtzer

The three big rules of research, as they were taught to me in high school, are pay attention to source bias, never trust the Internet (especially Wikipedia), and remember that statistics can be misleading. I suspect that most students are taught these tenets; keeping them in mind helps avoids sloppy or inaccurate research and helps the researcher explore more independent angles.

Unfortunately, the mantra of skeptical research can continue past its usefulness, as those of us accustomed to being rewarded for our ability to critically assess rapaciously search out bias and inaccuracy all around us. Everybody has an agenda and it is our job to expose them!

Under unrelenting skepticism, substantive information can get devalued as merely "somebody's take" on an issue, or  a "biased presentation," and fact can start to seem like opinion. This does us all a disservice. Sometimes, the numbers speak for themselves, and getting hung up their presentation serves only to distract us from the purpose of the data. Rather than scrutinizing the messenger and calling it a day, we should scrutinize the message, correcting for the messenger, and not shy away from fact-based conclusions.

Yesterday on I Love Charts, I posted a chart that compared the growth rate of tuition fees at 4-year public colleges and the growth rate of median American income.

I'll admit, the chart takes a bit of time to come to grips with, with its separate y-axis labels and color coordination. At a quick glance, the chart seems to be unfairly comparing a $4000 scale to a $48,000 scale. What do they have in common? Of course, this must be biased to overplay the growth in tuition. However, then you look at percentages, each side is measuring 260% growth, and I'm convinced this chart was the only way to accurately represent their point. Tuition growth is inordinately large unless somebody is willing to make the point that the education provided was massively undervalued in 1988 or offers a much greater return on investment at present. I suspect you will not find a lot of support for those positions.

However, some of my readers disagreed with the chart, perceiving bias, and offered their own corrections.

I'll reproduce the corrected charts, with my responses (in italics) below.



Although I haven't taken a stats class since high school (the underlying theme of this post is apparently high school lessons; shout out, Glastonbury High School!), I think that my explanations hold water. If not, this post will disappear, and I will deny ever having written it. Thank you, Internet.

What is interesting about the two corrected posts is that each creator, through their well-meaning vigilance against bias, created what they were trying to avoid: an actually biased chart and one that is essentially meaningless. Conversations about bias often devolve into competitions of competing bias, or recursive conversations about data presentation. What gets ignored are the facts. Sometimes the numbers speak for themselves. This is one of those cases. The only argument on the table is whether there is commensurate difference in education between now and 1988, either in its quality or in the financial rewards it promises down the line. If so, this data is reasonable. If not, it might be time for this education bubble to burst.