Clone What You Can't Steal: Black-Box LLM Replication via Logit Leakage and Distillation

The record for why logprobs on a public API are a model-extraction surface: reconstructs the output projection matrix from top-k logits collected in under 10,000 queries via SVD, then distils. A 6-layer student recovers 97.6% of the teacher's hidden-state geometry at +7.31% perplexity; a 4-layer variant gives 17.1% faster inference at 18.1% fewer parameters. The entire attack completes in under 24 GPU-hours.

paper

2025-08-31

2026-10-04