Who’s Afraid of Robot Journalists?
Suppose this column was written by a robot. Would you care? Should you?
The Associated Press recently announced that most of its coverage of corporate earnings reports will soon be “produced using automation technology.” It’s just the latest example of algorithm-driven tools that arrange straightforward English sentences into a news story, with little or no human intervention — the bot-ification of journalism.
It’s no surprise that the headlines about this sort of thing sound so jarring — “The AP’s newest business reporter is an algorithm” or “Associated Press Will Use Robots To Write Articles.” Journalists (me included!) have been obsessing for a couple of years about code-driven storytelling as a “potentially job-killing technology.”
But forget for a moment about the plight of journalists. (You probably already have, and you may not even trust us in the first place!) What does auto-journalism mean to the average reader? Can the robo-press produce material that’s useful, readable, relevant?
To find out, I spent a good amount of time reading and scrutinizing the output of computerized journalists. I realized two things. First, it’s already doing a better job than you probably think. And, second, while it’s not going to replace journalism as we know it (sorry, “lamestream media” haters!) it is going to add a lot to contemporary information flow — mostly for the better.
At the moment, bot-news seems to be dominated by just two major players.
One is the company the AP just hired: Automated Insights, which says its Wordsmith product “transforms Big Data into narrative reports by spotting patterns, correlations and key insights in the data and then describing them in plain English, just like a human would.”
The other is called Narrative Science. Its product is Quill: “an artificial intelligence platform” that mines data sets and “delivers meaning and insight in a form that makes natural sense to all of us, as narratives.” As it happens, this algorithm, or robot, or whatever, already writes earnings previews for Forbes.com. Like Automated Insights, it evolved out of sports coverage, starting out as a university project that generated recaps from box scores.
(Disclosure: Automated Insights’ technology is used in Yahoo’s fantasy sports coverage.)
I sampled the prose generated by both of these firms, and while I wish I could point to stylistic differences, they were pretty similar. And competent.
Certainly there are occasional oddities: “The majority of analysts (100%) rate Steelcase as a buy,” notes this Narrative Science write-up. “All two analysts rate Steelcase as a buy.” All two, eh?
And the game recaps generated on Automated Insights’ StatSheet site tend toward unwieldy sentences like this: “Brian Roberts finished 4-for-5 and David Robertson notched his 20th save of the year as New York stopped any hopes of a rally to top Minnesota, 6-5.”
But those are quibbles. And the one relevant study I’m aware of comparing reader reactions to stories written by bots vs. humans resulted in, basically, a draw. Subjects reportedly found the human-made article more “pleasant” to read — but considered the computer-generated piece more “trustworthy.”
See what you think: Take our quiz to test your skill at telling bot-prose from the human kind.
Most interestingly, even the reaction gaps in that study were marginal. ‟The lack of difference may be seen as an indicator that the software is doing a good job, or it may indicate that the journalist is doing a poor job,” one researcher commented. “Or perhaps both are doing a good (or poor) job?”