This article walks the canonical ML secrets surface, the management patterns that scale, and the recurring failure modes that show up in industry incident retrospectives.
The ML secrets surface
ML workloads concentrate secrets at four points in the lifecycle.
Training-time data access. Training pipelines read from data sources that each have their own credential — database connection strings, object storage signed URLs, message-bus consumer credentials, third-party data API keys. A single training job may use a dozen distinct credentials. The credentials must reach the training environment, persist across job restarts, and be rotated without breaking long-running training runs. The naive pattern — embed the credentials in environment variables or in the training script — is the source of the majority of credential-leak incidents in industry retrospectives.
Model registry and artefact storage. Trained models are written to and read from model registries and artefact stores that authenticate writers and readers separately. Write credentials must be tightly held by the build pipeline; read credentials must be available to the inference service but should not be available to general engineering. The two credentials must be rotated independently because they have different exposure profiles.
Inference-time downstream calls. Inference services that call downstream APIs — to enrich a request with reference data, to log a result to a downstream system, to invoke a tool on behalf of a Large Language Model (LLM) — carry the credentials for each downstream system. The inference service is uniquely high-value to attackers because compromising it grants access to every downstream credential it carries. The blast radius is bounded only by the discipline of scoping each downstream credential to the minimum required.
Third-party model provider credentials. Inference services that call hosted model APIs (OpenAI, Anthropic, Google, Cohere, Mistral, the cloud providers’ managed model services) carry the API keys for those providers. The keys typically have broad permissions and metered cost; their loss enables both data exfiltration (the attacker calls the model with the operator’s quota) and abuse (the attacker exhausts the operator’s bill). NIST SP 800-218A https://csrc.nist.gov/pubs/sp/800/218/a/final names third-party API credential protection as a required Secure Software Development Framework practice.
A fifth surface — credentials accidentally embedded in the model itself through training-data leakage — exists for models trained on data that included credentials. The phenomenon is well documented for Large Language Models trained on code corpora that contained committed secrets; the model can be coerced to emit the credentials at inference time. The defense is upstream, at the data-curation stage; the inference-time defense is output filtering as discussed in Article 3.
The management patterns that work
Mature secrets management for ML workloads applies the same patterns mature secrets management applies anywhere — with attention to the operational characteristics of training and inference workloads.
Centralized secret store. Secrets live in a dedicated secret manager — HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, Google Secret Manager, or an equivalent — and are retrieved at runtime by workloads that authenticate to the secret store with platform-native identity. Secrets do not live in environment variables, configuration files, source repositories, container images, or Jupyter notebooks. The centralized pattern enables audit (every secret retrieval is logged), enables rotation (the new secret is published to one place and propagates), and enables revocation (the access path is broken at the source).
Workload identity, not embedded credentials. Training jobs, inference services, and orchestration components authenticate to the secret store using their platform-issued identity (a Kubernetes service account, an AWS IAM role, an Azure managed identity, a Google service account). The identity is bound to the workload by the platform; no long-lived credential is required to bootstrap. Workload identity is the single most important pattern in modern secrets management because it eliminates the chicken-and-egg problem of how to authenticate the authenticator.
Short-lived, dynamically issued credentials. Where possible, the secret store issues credentials that are valid for hours rather than years. Database connection credentials are issued on demand and expire after the training job completes; cloud API credentials are vended through Security Token Service-style assume-role patterns. Short-lived credentials limit the blast radius of any single leak and make rotation a non-event.
Least-privilege scoping. Every credential is scoped to the minimum permissions required for the workload that uses it. A training job that reads from one bucket has a credential that reads from one bucket — not a credential with broad data-warehouse access. An inference service that calls one downstream API has a credential limited to that API. Scoping is harder to maintain than to establish because permissions tend to grow as systems evolve; periodic least-privilege audits are required.
Rotation as routine. Secrets are rotated on a schedule (every credential type has a rotation period) and on event (every employee departure, every suspected exposure, every dependency update that touches credential handling). Rotation is automated through the secret store; manual rotation is a fragile process that breaks in production at the worst times.
Audit and detection. Every secret retrieval is logged into the SIEM (Article 13) with sufficient fidelity to support both compliance audit and security detection. Anomalous retrieval patterns — a credential retrieved by a workload that does not normally use it, a credential retrieved at an unusual time of day, a credential retrieved by a network position that does not match the workload’s expected location — trigger alerts.
ISO/IEC 42001:2023 Annex A.6 https://www.iso.org/standard/81230.html requires AI Management System operators to manage cryptographic and authentication material with controls that explicitly contemplate the patterns above. The Gartner AI TRiSM framework https://www.gartner.com/en/articles/gartner-top-strategic-technology-trends-for-2024 tracks the maturity of secret-management tooling specific to ML platforms, including the integration patterns between secret stores and ML training and serving infrastructure.
The failure modes that recur
Industry incident retrospectives — the public ones the affected organizations have written up, and the private ones the security community shares informally — show the same handful of failure modes recurring across organizations and across years.
Secrets in notebooks and notebooks in repositories. Data scientists develop in Jupyter notebooks that include credentials for convenience. The notebooks are committed to source repositories. The repositories are pushed to public hosts or to internal hosts with broad read access. The credentials are then available to anyone who can read the repository, and they remain available even after rotation because the historical commits retain them. The cure is automated secret-scanning in pre-commit hooks (TruffleHog, Gitleaks, the cloud-native equivalents), education, and the cultural shift to notebook patterns that retrieve secrets from the secret store at runtime rather than embedding them.
Long-lived credentials that survive their owner. A data scientist creates a credential to run an experiment. The credential persists in a configuration file. The data scientist leaves the company. The credential continues to authenticate, used by automation no one remembers building, until something changes upstream and the credential breaks loudly — or, worse, until an attacker discovers it. The cure is short-lived credentials, workload identity, and periodic credential audits that flag credentials with no recent legitimate use.
Over-scoped credentials. A credential is created with broad permissions for convenience and never narrowed. The cure is least-privilege at issuance, plus periodic audits that compare actual usage against granted permissions and recommend tightening.
Third-party API key abuse. A hosted-model-provider API key is exposed and an attacker uses it to incur cost on the operator’s bill, sometimes hundreds of thousands of dollars before detection. The cure is per-key rate limiting, cost alerting, and the use of provider-side IP allowlists where available.
The MITRE ATLAS knowledge base https://atlas.mitre.org/ documents the credential-related attack techniques relevant to ML workloads under Initial Access and Privilege Escalation tactics.
Maturity Indicators
Foundational. Secrets are embedded in environment variables, configuration files, or source code. There is no central secret store. Credentials are long-lived and broadly scoped. Rotation is manual and ad hoc. The team cannot enumerate which credentials a given workload uses.
Applied. A central secret store exists and at least production credentials are stored there. Secrets are not in source repositories (verified by automated scanning). Some credentials are short-lived. The team has a written secret-management policy.
Advanced. Workload identity is used everywhere; long-lived bootstrap credentials have been eliminated. Credentials are short-lived and dynamically issued where possible. Least-privilege scoping is enforced and periodically audited. Rotation is automated and routine. Every secret retrieval is logged into the SIEM. The threat model from Article 1 names credential compromise as a vector and the controls map back to it.
Strategic. Secrets management is a first-class governance surface. Anomalous credential usage is detected and triggers incident response (Article 14). Third-party API credentials are protected with cost alerting and rate limiting. Credential management is itself audited on a regular schedule by external specialists. Red-team exercises (Article 11) include credential-discovery attempts against the ML platform.
Practical Application
A team that today has secrets in environment variables and configuration files should make three changes this quarter. First, deploy or adopt a central secret store and migrate the highest-value credentials into it (third-party model API keys, model-registry write credentials, production database access). Second, run an automated secret-scanning sweep over every source repository and every notebook archive to find embedded credentials, rotate every credential found, and add a pre-commit hook that fails any subsequent commit containing a secret. Third, audit which credentials have been used in the last ninety days and revoke the ones that have not — the audit will surface a substantial fraction of the credential surface that the team did not know existed.
These three actions cut the most likely attack vectors, create the artefacts on which workload identity and short-lived credentials are subsequently built, and dramatically improve the team’s ability to respond to a future credential-compromise incident.
© FlowRidge.io — COMPEL AI Transformation Methodology. All rights reserved.