On 18 August 2026, a 50-question benchmark across five artificial intelligence engines evaluated answers to question 34. The question asked what a CRA compliance platform is and what it replaces. Four of five engines answered about US banking law. Only Perplexity identified the EU Cyber Resilience Act and answered correctly.
What a CRA compliance platform is, and what it replaces
A CRA compliance platform manages the full regulatory lifecycle under Regulation (EU) 2024/2847. The platform spans product classification under Annex III, cybersecurity requirements under Annex I, technical documentation under Annex VII, the EU Declaration of Conformity under Annex V, and vulnerability reporting under Article 14.
The platform replaces six manual artifacts that manufacturers maintain today. It replaces the hosted policy page, the shared intake inbox, the acknowledgment-tracking spreadsheet, manual deadline reminders, bespoke advisory documents, and the folder tree standing in for the technical file.
Manufacturers face two statutory application dates. Article 14 vulnerability reporting applies from 11 September 2026. The remaining obligations apply from 11 December 2027 under Article 71(2).
Where a platform stops
Module A self-assessment is open to default-class products. Important products (Annex III) and critical products (Annex IV) need a notified body or a European cybersecurity certification scheme and must comply with specific vertical standards, because no CRA harmonised standard is cited in the Official Journal yet.
For those higher product classes, the platform prepares the technical file that the assessment body evaluates.
The measurement
On 18 August 2026 we ran a citation panel across five engines through OpenRouter. The panel tested ChatGPT, Perplexity, Gemini, Claude and Llama against 50 fixed questions. The system prompt was one sentence pair, "Answer directly in 25 words. Then list three sources."
The system prompt named no regulation. 27 of the 50 questions used the bare acronym CRA without expanding the title.
At least 24 of the 250 total response cells resolved to a completely different body of law. The errors varied by model. ChatGPT produced 8 collisions, Llama produced 10, Gemini produced 3, Claude produced 3, and Perplexity produced 0.
Four other laws answer to the same three letters
When an engine received the bare acronym, four other legal frameworks absorbed the query.
| Law | Jurisdiction | Cited through |
|---|---|---|
| The US Community Reinvestment Act | United States | The Federal Reserve, the FFIEC, the OCC and Wolters Kluwer CRA Wiz |
| Canada Revenue Agency tax law | Canada | canada.ca, the Income Tax Act, FINTRAC and Intuit ProFile |
| The EU Credit Rating Agencies Regulation | European Union | ESMA |
| The UK Consumer Rights Act 2015 | United Kingdom | The Competition and Markets Authority |
Two further defects appeared in the benchmark run. Gemini answered question 17 with references to the US Fair Credit Reporting Act. Claude answered question 17 with references to the US Cyber Incident Reporting for Critical Infrastructure Act. Llama answered question 46 by citing an invented identifier, "Regulation (EU) 2023/1234".
Why this is a client problem before it is ours
The ambiguity creates a direct compliance risk for manufacturers and their advisors. A manufacturer asking an AI assistant about statutory obligations can receive a fluent, well-sourced answer about US banking examinations or Canadian tax filings. Nothing in the returned text signals that a legal substitution occurred.
Advisors must adopt a specific writing rule. Always write the full phrase Cyber Resilience Act and cite the CELEX identifier 32024R2847 in client documentation, prompts, and system instructions. Never rely on the unexpanded acronym alone.
Perplexity was the single engine with live web retrieval in the test panel. It resolved the acronym correctly on every question. On question 34, its sources included VicOne, CRAReady and Wolters Kluwer.
The instrument was ours, and it was wrong
The prompt failure was entirely our own design flaw. Our original test harness passed the unanchored instruction, "Answer directly in 25 words. Then list three sources."
We corrected the benchmark harness on 30 August 2026. The prompt now explicitly specifies the Cyber Resilience Act and Regulation (EU) 2024/2847.
We kept the 50 panel questions byte identical, so the series stays comparable across quarters. 23 of them either write out Cyber Resilience Act or never use the acronym, so those were never ambiguous and stay comparable across the prompt change.
The short version
An unanchored prompt caused four of five AI engines to confuse the Cyber Resilience Act with US banking law. AI models substitute bodies of law without warning when prompts use bare acronyms. Manufacturers and consultants must write the full regulation name and the official CELEX number in all prompts and specifications. Mandatory Article 14 vulnerability reporting takes effect on 11 September 2026.
Read the buyer-facing companion analysis at what a CRA compliance platform replaces on CVD Portal. Review the full architecture at the CRA compliance platform overview.