Nimble finds over 99% noise in agentic AI retail web queries

Nimble published research on how agentic AI retrieves information from retail webpages, finding that most of what the agents process does not contribute to the final answer. The results point to both an accuracy risk and added processing cost when agents read full web pages to respond to short questions.

The study analyzed 250 live retail queries that covered product prices, ratings, availability, discounts, shipping policies, and product descriptions. Nimble said the dataset showed a signal-to-noise problem that affected how accuracy and cost scale in production AI systems.

Across the full set of queries, the average webpage contained 8,795 characters while the average answer was 31.7 characters long. Nimble reported that fewer than 0.4% of all characters processed were relevant to the question. For price queries, the company said typical pages exceeded 9,000 characters while correct dollar amounts were four to six characters, producing a 99.48% noise rate.

Uri Knorovich, Nimble’s CEO and co-founder, said, “These numbers aren't a rounding error; they indicate a structural problem,” and added, “The web was built for humans, not machines.” Knorovich also said, “The model isn't the bottleneck.” In the dataset, Nimble said LLM API pricing is approximately one token per four characters, and with an average noise rate of 97.9%, it estimated that an agent running 1,000 queries per day would process roughly 8.5 million characters of content that did not contribute to answers.

Nimble said its research supports a shift toward agents retrieving structured data directly rather than parsing answers from raw webpages, and it made the dataset and methodology available via a blog post.