The evidence behind claims of AI accuracy

Federal Trade Commission · Policy submission ·

Examines the evidence and definitions behind the proposed statement on AI accuracy. Questions what observed fact-checking establishes about consumer trust, and shows how reported accuracy can rise while users receive fewer correct answers.

Read the comment (PDF, 7 pages)

The document

Title
Comment on the proposed Policy Statement Concerning the Suppression of Accuracy in Artificial Intelligence Systems
Matter
P264200
Docket
FTC-2026-0859, comment FTC-2026-0859-0275
Capacity
Personal capacity
Submitted
Length
7 pages

The four changes it asks for

  1. State the footnote 39 evidence at the level it actually supports. The notice says consumers accept AI outputs without further fact-checking more than 90 percent of the time, then paraphrases that as showing consumers overwhelmingly trust the system. The source measures neither proposition.
  2. Keep express performance claims on their ordinary substantiation track. A quoted claim to perform better than human baselines across every role measured is an objective comparative performance claim, and the reasonable-basis requirement already reaches it.
  3. Reconcile the statement’s terms and separate conduct from measured effect.
  4. Explain what technical evidence bears on attribution and accuracy. Including what an accuracy figure does and does not show.

The measurement point, in full

The cited study coded 9,830 multi-turn conversations over one seven-day window for the presence of eleven observable behaviours. Its finding was that 8.7 percent of conversations contained evidence of a user checking facts or claims, and 91.3 percent did not.

That supports a narrower statement: 91.3 percent of the sampled conversations contained no conversation-visible fact-checking behaviour. It does not measure whether a user checked an answer mentally, against another source, or after the conversation ended. Nor does it establish that the answer was accepted, or why.

Visible absence of a checking behaviour does not establish that checking did not occur. The comment notes the figure is not necessary to the statement’s theory, which grounds consumer expectations in company marketing rather than in that footnote.

What it does not propose

The comment proposes no safe harbor and no prescribed recordkeeping regime. It takes no position on the pre-emption of any State law.

Sources