Qualcomm responds to benchmark cheating allegations — Snapdragon X Elite/Plus benchmarks claimed to be fraudulent (Updated)
The notoriously confrontational tech site SemiAccurate claims that Qualcomm is cheating on the benchmarks of its new Snapdragon X Elite and Plus laptop processors, and Qualcomm has now responded to those accusations. The Snapdragon X Plus, announced yesterday, joins the previously announced X Elite in Qualcomm's upcoming processor lineup for Windows notebooks. The chips stand out as Qualcomm's first potentially competitive laptop processor against Apple, Intel, and AMD, and the best chance for Windows on Arm to take off. However, SemiAccurate.com claims that Qualcomm has been far from forthcoming with its curated benchmark results that have been presented to the press.
SemiAccurate, claiming two "major" OEMs and a "deep source at Qualcomm" for their benchmark cheating claims, insists that Qualcomm benchmarks are impossible to recreate even by class-leading OEM developers.
A Qualcomm representative sent Tom's Hardware an official comment on the matter, saying succinctly, "We stand behind our performance claims and are excited for consumers to get their hands on Snapdragon X Elite and X Plus devices soon.”
Back at Qualcomm's Snapdragon Summit conference held last October, the public saw the first benchmarks of the forthcoming X Elite processors. These numbers, claimed to be clean by SemiAccurate's sources, were slower than expected, though the Qualcomm source claimed the software wasn't final. And those benchmarks were a complete black box — no one at the conference could see the settings used for testing, and vague charts showing performance against competitors Apple and Intel were unverifiable.
After OEMs got their hands on the X Series chips, SemiAccurate claims no OEM has been able to replicate Qualcomm's touted test numbers. In fact, the site reports OEMs first saw numbers "far sub-50%" of the numbers Qualcomm claimed. Qualcomm engineers reportedly told SemiAccurate that they blamed poor optimization from Windows running on Arm, and poor cooling. Even after some time had passed and cooling could be improved, however, testers still reportedly never came close to Qualcomm's curated benchmarks, with one anonymous source comparing the Snapdragon X Elite to Intel Celeron chips.
SemiAccurate makes no claims about specifics of how Qualcomm is allegedly cheating its benchmark numbers. If "tier 1" OEMs are reportedly unable to even come close to recreating the X Series benchmarks on their own, wrongdoing by Qualcomm, the site alleges, must be the only other option.
The site also claims that Qualcomm's product slides are presenting "tainted" numbers to the press and its OEM partners. At the Snapdragon Summit, the company avoided answering technical questions on its forthcoming X Series, promising more answers and deep technical briefings before launching its chips. It also apparently promised reviewers the opportunity and ability to independently benchmark the new chips before launch; neither of these promises came to be, though there's still time before the chips launch in mid-2024. (The press could run benchmarks on devices at hands-on events, but the software was all pre-installed in tightly controlled conditions.)
SemiAccurate says Qualcomm's stats, revealed today, on its new line of chips lacked the deep technical elements originally promised, with more time spent on potential camera specs enabled by the SoC processors than any real technical stats or deep analysis. We did get core counts, cache, and clock speeds, but nothing deeply technical outside of spec sheets.
If the claims prove true, it would certainly rock a laptop world that looked to the X Series as Windows' response to Apple's M Series processors, but for now, these claims are unsubstantiated. Naturally, we can't test the veracity of these huge claims until the silicon comes to market.
Microsoft is expected to make big announcements in this space at its Build developer conference in May, hopefully giving us a better idea of the shape of Windows on Arm.
Edit 04/25/2024 3:30am PT: Added comment from Qualcomm.
