AI API keys are becoming the new security perimeter
Why connecting to language models through an API creates a new class of risks and what to do about it now, before the keys have spread across the entire infrastructure.
When a company starts using GPT-4, Claude, or any other language model through an API, it usually thinks of this as connecting to a service. Get a key, embed it in the code, everything works. Security is a problem for later.
I see this pattern in many companies right now, riding the wave of LLM integrations. And right now is the moment to pause and think about the risks, before keys have spread across the entire infrastructure.
Why LLM API keys are not ordinary keys
A typical API key - say, for an SMS-sending service - allows sending SMS. The damage from a leak is bounded: money off the account, unwanted spam. The scale is clear.
A key to an LLM API is a different class of access. All requests to the model pass through it. If the model is connected to tools - knowledge-base search, code execution, file access - then a compromised key gives access to everything the model can do. And the cost of a compromised key is not just the money on the account; it includes potential data leaks from everything passed in the requests.
The second point: billing. LLM APIs are usually priced by tokens. A leaked key is a resource that can be used for someone else's purposes at your expense. A few days of uncontrolled use can cost more than an expected monthly budget.
Common mistakes I see
Key in the repository. Even a private one. Even in a .env file that "accidentally" ended up in git. This is a classic that keeps happening despite all warnings.
One key for everything. One key used in production, during development, in analytics scripts, and in "quick experiments". When something goes wrong - you cannot tell where the leak came from or what exactly was compromised.
Key with no limits. Many LLM providers allow restricting a key by spend, IP addresses, or operation types. Most people do not use this.
No spend monitoring. An unexpected invoice is discovered at the end of the month, not at the moment of the anomaly.
Minimum controls
This does not require complex infrastructure. Basic measures:
Different keys for different environments. Production, development, testing - separate keys. Each can be rotated and revoked independently.
Keys through a secrets manager, not manual environment variables. AWS Secrets Manager, HashiCorp Vault, Azure Key Vault - it does not matter which, the point is no hardcoding and no .env in the repository.
Spend limits on each key. Most providers support this. Set a reasonable limit and an alert when it is reached.
Call logging. At minimum: who, when, how many tokens. This gives a baseline for investigation if an incident occurs.
Regular rotation. Keys that have not changed in a year are keys that may have leaked without your knowledge.
A check question
If you asked every person on the team with LLM API access to list all the keys they use right now - would the result be a complete list? Or would some be "somewhere in an old script" or "in my local .env"?
If the answer is unclear - that is a signal. Not a catastrophe, but a signal. Running an inventory and tidying up now, while it is a small problem, is much cheaper than dealing with an incident six months from now.