Skip to content

Karios

Tech news

Support me 1

Gaming History Podcast (GR)

  • Gaming History – Επεισόδιο 32 – Atari Jaguar
Primary Menu
  • Home
  • Books
  • Shop
    • Browse Shop
    • Cart
    • Checkout
    • Orders
    • Account details
    • Lost password
  • Articles
    • All Articles
    • Games
    • DIY
    • Retro
  • Gaming History Podcast (GR)
  • Contact me
  • News

Testing suggests Google’s AI Overviews tells millions of lies per hour

admin Posted on 8 months ago 2 minutes read
Testing suggests Google's AI Overviews tells millions of lies per hour

​Is 90 percent accuracy good enough for a search robot? 

Looking up information on Google today means confronting AI Overviews, the Gemini-powered search robot that appears at the top of the results page. AI Overviews has had a rough time since its 2024 launch, attracting user ire over its scattershot accuracy, but it’s getting better and usually provides the right answer. That’s a low bar, though. A new analysis from The New York Times attempted to assess the accuracy of AI Overviews, finding it’s right 90 percent of the time. The flip side is that 1 in 10 AI answers is wrong, and for Google, that means hundreds of thousands of lies going out every minute of the day.

The Times conducted this analysis with the help of a startup called Oumi, which itself is deeply involved in developing AI models. The company used AI tools to probe AI Overviews with the SimpleQA evaluation, a common test to rank the factuality of generative models like Gemini. Released by OpenAI in 2024, SimpleQA is essentially a list of more than 4,000 questions with verifiable answers that can be fed into an AI.

Oumi began running its test last year when Gemini 2.5 was still the company’s best model. At the time, the benchmark showed an 85 percent accuracy rate. When the test was rerun following the Gemini 3 update, AI Overviews answered 91 percent of the questions correctly. If you extrapolate this miss rate out to all Google searches, AI Overviews is generating tens of millions of incorrect answers per day.

Read full article

Comments

 Ars Technica – All content Read More

About The Author

admin

See author's posts

Post navigation

Previous: Linux kernel maintainers are following through on removing Intel 486 support
Next: No big trucks for little roads: American OEMs say EU is blocking imports

Related

Vulcan failure caused by inadequate fuel supply to engines, ULA says
  • News

Vulcan failure caused by inadequate fuel supply to engines, ULA says

admin Posted on 19 hours ago
Waymo has stopped its robotaxis from driving on the freeway—here’s why
  • News

Waymo has stopped its robotaxis from driving on the freeway—here’s why

admin Posted on 19 hours ago
Tesla’s $30,000 Cybercab comes with a $299 monthly catch
  • News

Tesla’s $30,000 Cybercab comes with a $299 monthly catch

admin Posted on 22 hours ago

Listen on Apple Music

You may have missed

Vulcan failure caused by inadequate fuel supply to engines, ULA says
  • News

Vulcan failure caused by inadequate fuel supply to engines, ULA says

admin Posted on 19 hours ago
Waymo has stopped its robotaxis from driving on the freeway—here’s why
  • News

Waymo has stopped its robotaxis from driving on the freeway—here’s why

admin Posted on 19 hours ago
Tesla’s $30,000 Cybercab comes with a $299 monthly catch
  • News

Tesla’s $30,000 Cybercab comes with a $299 monthly catch

admin Posted on 22 hours ago
technews
  • News

Apple Tightens Mac Privacy Controls, but AI Agents May Keep Existing Access

admin Posted on 23 hours ago

If you are located in Europe, you can buy my new sci-fi fantasy book “Ground Zero” directly from me for a better price than Amazon.