Secret Extraction Framework for Black-box LLMs
Practical Secrets Extraction against Black-box LLMs
This research exposes a critical vulnerability in widely deployed LLMs that could lead to credential leakage from memorized training data. Security teams and developers using API-based LLMs should audit their systems for potential secret exposure and implement additional safeguards. The findings particularly impact enterprises in ASEAN regions where LLM adoption is accelerating but security practices may lag behind.
Researchers developed a black-box secret extraction framework for commercial API-based LLMs under output-only access.
The framework uses Cross-Validated Secret Knowledge Distillation and Proxy-Guided Secret Extraction to recover credentials.
On controlled API-key benchmarks, it improved recovery effectiveness and real-key rates while reducing extraction latency.
The framework successfully recovered masked credentials from OpenAI and Claude Code systems in real-world evaluations.
Practical Secrets Extraction against Black-box LLMs
Large language models (LLMs) increasingly power autonomous coding agents such as Codex and Claude Code, yet their training corpora may contain confidential credentials exposed in public repositories or collected from private development artifacts, creating risks of memorization and subsequent leakage. Existing extraction audits, however, largely assume access to model weights or token probabilities. In this work, we present a black-box secret extraction framework for commercial, API-based LLMs under output-only access. It comprises (i) \emph{Cross-Validated Secret Knowledge Distillation}, which uses semantics-preserving prompt variants, response cross-validation, and provider-specific format filtering to distill secret-relevant behavior into a local white-box proxy; and (ii) \emph{Proxy-Guided Secret Extraction and Candidate Filtering}, which combines truncated top-$p$ sampling with local token entropy, $N$-gram frequency profiling, and provider-specific structural priors. On controlled API-key benchmarks, our framework improves recovery effectiveness and real-key rates over representative baselines while reducing extraction latency. A responsible real-world evaluation further recovers masked provider-specific credentials from three independently deployed black-box LLM systems spanning OpenAI and Claude Code, showing that memorized secrets can be exposed under output-only access.
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI regulation, compliance frameworks, and sovereign compute policy and AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include LLMs, Claude Code, OpenAI, API security.
Primary source verification: published directly by arXiv: OpenAI with first-party authority. Cross-source corroboration: 11 independent sources have covered this development (including Gergely Orosz, Tibor Blaho, IT之家), confirming industry-wide signal strength.