Skip to content
[ SIGNAL VERIFIED ] INDEPENDENT CYBERCRIME JOURNALISMCLASSIFIED // THE UNDERGROUND HAS A NEWSROOM / RSS
.COM
Latest News
ALL NEWS
The Digital Arsenal

Digital Arsenal: Audit AI Model Gateways for Hidden Prompt Retention and Third-Party Routing

A research-based seven-step guide to checking AI gateway routing, data retention, sub-processors, secret handling, test evidence and incident response.

The Digital Arsenal editorial artwork
PUBLISHED

IPCRIME Digital Arsenal | November 22, 2026 | Research-based guide, not firsthand benchmark testing. Model gateways and AI coding assistants can make useful tools available with a single API. But when a private prompt leaves your device, the service that accepts it may not be the only organization that handles it. OpenRouter’s November provider review, alongside notices by Together AI, Phala and Novita, shows why privacy auditing must include every processing hop, not only the provider displayed on a dashboard.

1. Map the actual route for a request

Write down the app sending the prompt, the gateway, the provider that hosts the model, any reseller or partner API, the region and the retention policy that applies to each stage. A request can move between providers or infrastructure as models become available. When a service offers a zero-retention setting, check whether that setting is valid for the exact endpoint and whether partners also agree to it.

OpenRouter says it removed 125 model endpoints from Zero Data Retention eligibility after a researcher investigation. That is evidence the eligible endpoint set can change. It does not show that 125 endpoints were themselves hacked. Require an up-to-date inventory rather than treating a historic screenshot of available models as a lasting guarantee.

2. Classify prompt content before transmission

  • Public content: published documentation, open-source examples and synthetic sample questions can often be handled with ordinary approved provider policies.
  • Internal content: unpublished code, operational alerts, confidential drafts and vendor diagrams deserve explicit organizational approval and clear retention expectations.
  • Restricted content: production API keys, passwords, private customer data, medical records and private investigative sources should not be pasted into prompts unless a controlled, authorized use case specifically requires and protects them.
  • Derived outputs: generated summaries can themselves reproduce sensitive information, so review where outputs are stored and shared.

3. Treat secret exposure as a credential problem

Together AI disclosed a stored prompt-sample download enabled by a credential committed to a public repository. Phala disclosed compromised infrastructure where some in-flight model requests could be observed. Novita reported unintended request forwarding to another provider. These are different mechanisms, but all illustrate why a prompt may carry secrets beyond the expected trust boundary. Remove production secrets from context by default, use placeholders during development and prefer short-lived, narrowly scoped service credentials.

If a real key has already been sent during a relevant incident window, rotate it at the service that issued it and review usage records for unexpected behavior. Rewriting a prompt or deleting your local conversation does not revoke a key that another party may have seen.

4. Ask precise questions about the retention guarantee

  • Is Zero Data Retention a routing filter, a provider contract, a technical block on logs or a combination?
  • Does it apply to prompts and model responses, failed requests, abuse logs and developer-support samples?
  • Can the gateway change providers automatically? If so, how are eligibility changes communicated to customers?
  • What happens when a provider forwards inference to a partner API? Is that downstream processor named in the published policy?
  • Are sensitive data and customer account credentials segregated from development repositories, build logs and AI coding agents?

5. Test using authorized synthetic data

A cautious team can send distinctive but non-sensitive, organization-owned test content through approved APIs to check observable routing and logs. Do not use real customer records, exposed credentials or private endpoints as test bait. Record the model, endpoint, configuration, provider policy version, date, expected route, observed route and any anomalies. Third-party research should be described as third-party research unless your team performed and documented its own test.

6. Put a response decision in writing

For a provider notice, determine whether your organization used the affected model during the stated window. Identify which employees or applications sent requests, whether any tokens or private information were included, and which downstream partners might be involved. Escalate only confirmed or plausible exposures based on evidence. Notify affected people according to applicable requirements and your legal and privacy review; do not declare a breach for every customer solely because a gateway withdrew an eligibility label.

7. Keep an exit route

Use narrow access permissions, audit logs and an approved alternate model or offline process for confidential tasks. Where policies change, pause sensitive traffic to the affected endpoint until routing and privacy controls are verified. Production teams should retain the ability to disable a provider without rewriting all their applications.

Sources, comparison method and limitations

OpenRouter — November 22 provider response; Dragos — AI inference canary research; Together AI — November 21 prompt-sample incident; Phala — November 10 incident report; Novita — November 22 routing notice. IPCRIME has not run a controlled provider benchmark. This is a documented evaluation and incident-response framework based on primary disclosures. Related: Digital Arsenal Comparisons and Breach Response Center.

Editorial note

IPCRIME distinguishes public records and documented findings from allegations, analysis and commentary. Report corrections through our contact page.