Bots Are Better Than Humans at Solving CAPTCHA Tests, According to New Study

Declan Gallagher
·2 min read

A recent study has found that robots have a higher success rate in solving so-called CAPTCHA tests than human beings.

CAPTCHA stands for “Completely Automated Public Turing test to tell Computers and Humans Apart.” The acronym refers to the inane puzzles in which you’re asked to identify all of the images containing a boat, or to point a fist in the same direction a tiger is facing.

The puzzles, commonly used in online tools and services, ostensibly served as safeguards against bots and hackers for decades. However, new research conducted by the University of California, Irvine has shown that robots are able to solve CAPTCHA tests almost 100 percent of the time. Alarmingly, humans have a success rate of between 50 and 84 percent. Furthermore, humans took as long as 15 seconds to solve the puzzles while bots completed them in under one second.

“Automated bots pose a significant challenge for, and danger to, many website operators and providers,” researchers noted in their paper, published by arXiv. “Given this long-standing and still-ongoing arms race, it is critical to investigate how long it takes legitimate users to solve modern CAPTCHAs, and how they are perceived by those users.”

UCI recruited 1,400 participants to solve 10 CAPTCHA tests on 120 of the world’s 200 most popular websites. They were measured on how quickly and accurately each puzzle was solved. Image recognition, distorted text, and puzzle sliders were among the quizzes administered. The human results were then compared to those of bots coded with the intent of solving CAPTCHA tests.

Google’s image-based reCAPTCHA came out of the study looking the best. Bots solved those puzzles with “only” 85 percent accuracy, and they took around the same time as humans to do so.

Scientists are now calling upon tech companies to refine their procedures and institute something safer (and less obnoxious) than CAPTCHAs. “We do know for sure that they are very much unloved. We didn't have to do a study to come to that conclusion,” lead researcher Gene Tsudik told New Scientist. “But people don't know whether that effort, that colossal global effort that is invested into solving CAPTCHAs every day, every year, every month, whether that effort is actually worthwhile.”