# An AI Key Leaked. Would We Notice?

The quieter calls before a surge in stolen AI usage caught my attention. A look at the provider-side evidence I would want, and the limits of an agent trace.

Published: 2026-11-02

The part of Wiz's recent report that caught my attention was the small amount of activity before the expensive part began.

A stolen credential does not have to announce itself with a dramatic spike. Someone can check that it works, leave it alone for a while, and return later. An agent application's own activity could look perfectly ordinary in the meantime.

That leaves me with a fairly practical question: if somebody used one of our AI credentials without going through our application, where would we notice?

## The first check was not the flood

In its October 30 investigation, Wiz describes two cases of enterprise credentials abused after developer-platform breaches. Initial validation happened within thirty minutes of the reported credential access. Low-volume checks followed. The large surges began four and seven days after that first validation, respectively. Wiz also observed model-access testing and quota enumeration. Those are vendor-reported observations, not a sequence I have reproduced. [Read the investigation](https://www.wiz.io/blog/black-october-for-ai-developer-platforms).

The distinction matters. This was not a thirty-minute journey from exposure to the entire bill. The early checks and later bulk use offer different opportunities for investigation.

The attribution also has limits. Wiz did not identify one vulnerability connecting all the platform incidents or establish that the initial attackers, credential resellers and high-volume users were the same operation. That leaves too much unresolved to call this one proven agent swarm.

Nor would I wait for an unauthorized invocation before treating a credible credential exposure as an incident. The calls are evidence of what happened next, not the threshold that makes an exposed secret worth dealing with.

## The application may never see the request

An agent trace is useful for following work through the agent. It is less useful when someone takes a credential and skips the agent entirely.

The visibility gap is simple. This is a conceptual path, not a reconstruction of the vendor breaches:

```text
Credential stored in a hosted development tool
                  |
             copied elsewhere
                  |
         direct provider request
                  |
     no corresponding application run
```

In [the evaluation-sandbox article](/blog/your-evaluation-sandbox-holds-production-authority/), I looked at credentials that remain useful after their original workload ends. Here I would start with the evidence of their use outside that workload.

For one integration, the starting record is manageable: provider account, responsible owner, expected calling identities, normal network origins and permitted models. The harder question is whether a provider record can distinguish that integration from others sharing the same underlying identity.

If it cannot, that is a visibility limitation to record. Putting the raw key into a log to make attribution easier would create a new exposure problem.

## Check which logging you actually have

Bedrock has several logging paths. The endpoint and operations in use determine which documentation and collection settings matter.

AWS currently documents `InvokeModel`, `InvokeModelWithResponseStream`, `Converse` and `ConverseStream` on `bedrock-runtime` as CloudTrail management events. Other operations, including agent invocations and some streaming or asynchronous operations, use data events with different configuration requirements. An account's selectors and downstream collection still need checking. [AWS CloudTrail documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/logging-using-cloudtrail.html).

Separate model-invocation logging is disabled by default. It can collect request and response content as well as metadata, and its documented endpoint coverage is not universal. I would decide whether that content is needed before enabling collection, with access and retention set accordingly. Hunting credential abuse should not casually produce another store of confidential prompts. [AWS invocation-logging documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-invocation-logging.html).

These are the evidence questions I would start with:

| Question | Evidence I would look for |
| --- | --- |
| Who made the request? | Provider identity and a mapping to the workload or credential owner |
| Where did it come from? | Source network compared with the integration's expected origins |
| What was requested? | Available operation, model or profile identifiers, and result information |
| Was there an authorized run? | An application or broker record that explains the provider activity |
| What can we not see? | Missing operations, unavailable identifiers, collection failures and delivery delay |

That last row belongs in the review. No matching application record could mean bypass, but it could also mean a missing log or a broken join.

## Look for a sequence, not a suspicious-looking IP

My starting hunt would be a credential or workload identity appearing from an unfamiliar origin, followed by attempts against models it rarely uses, repeated checks, or a change in request volume. The sequence is what interests me.

Those are hypotheses for tuning and testing, not ready-made detections. A residential address alone is not a verdict. A model-access failure may be an ordinary configuration mistake. Quota enumeration may come from legitimate platform tooling.

The useful context is whether the behavior fits this particular integration. A workload that has always called one approved model from a fixed environment gives us a different baseline from a developer tool with variable egress and broad model access.

Collection latency needs checking, too. A query that finds yesterday's activity may be useful for response without being an early-warning system. A useful review result includes how long the evidence took to arrive.

## Regional limits and budget alerts have different jobs

Wiz's two cases had different permissions for global and US cross-region inference profiles. That affected throttling, but the region-restricted case still had more than a million successful calls. Regional scope is not, by itself, a spending ceiling. [Wiz's case comparison](https://www.wiz.io/blog/black-october-for-ai-developer-platforms).

Model and geographic access can be chosen according to the workload's actual needs. Volume and cost controls need a separate assessment.

AWS Budgets is useful for cost visibility and notifications, but its information is not updated in real time. AWS warns that additional usage can accrue before a notification arrives. A budget alert is not an immediate hard cap. [AWS Budgets documentation](https://docs.aws.amazon.com/cost-management/latest/userguide/budgets-managing-costs.html).

A gateway's usage controls also need a boundary check. If someone can copy its upstream credential and call the provider directly, a limit enforced only inside the gateway will not govern those requests.

## The response note I would keep nearby

Before an incident, I would write down who can disable each integration's access and what legitimate work that will interrupt. That is easier to settle while nobody is watching a usage chart climb.

For a suspected exposure, my proposed response sequence would be:

1. Identify the affected credential, related identities and the systems that held them. Record the uncertainty in the exposure window.
2. Preserve available audit evidence without delaying necessary revocation or containment.
3. Disable the affected access through its actual provider-side mechanism. Investigate other stored credentials when the compromise scope warrants it.
4. Investigate accepted requests and resulting actions. Revocation does not undo completed work.
5. Issue replacement access from a known-clean administrative environment. Do not put it back into a service that is still suspected of compromise.
6. Confirm both that the old access fails and that legitimate work resumes with the intended permissions.

The exact steps depend on the credential type and incident. A provider API key, an IAM user's access key and a temporary session do not have interchangeable revocation procedures.

## A small rehearsal would answer more than a diagram

I would test this with an isolated, low-cost identity, harmless requests and an approved second client. First establish ordinary application activity. Then make a small authorized request outside that application and check which provider evidence arrives, how quickly it arrives, and whether a reviewer can explain the missing application run.

Finally, revoke the test access and verify the outcome. Record gaps rather than filling them in with what the architecture was supposed to do.

I have not run that exercise against the affected vendors or a Bedrock account for this article. It is the acceptance test I would propose before relying on the monitoring.

The companion piece, [Two Secret Scanners Saw Different Leaks](/blog/two-secret-scanners-saw-different-leaks/), starts earlier: whether we can find the exposure in the first place. Here I would begin by choosing one integration and finding out what its first unexpected provider request would actually look like to us.