Litigant Uses Hidden Prompt Injections in Court Documents to Trick Judicial AI
A pro se litigant hid instructions inside legal filings hoping court-side AI summarizers would rule in his favor.
In an unusual legal dispute, a pro se litigant suspected the court was using automated AI tools to digest filings, so he embedded white-font prompt injections instructing any reviewing model to find in his favor and dismiss opposing arguments.
Prompt Injection in the Wild
While human clerks quickly noticed the formatting tricks and issued a stern warning, the incident highlights a growing real-world vulnerability as automated document processing spreads through legal, medical, and financial operations.
What This Means for You as a Builder
If you build document extraction pipelines or summarizers that ingest untrusted user uploads (like PDFs or scanned forms), you must treat file content as untrusted raw text. Always isolate user data from system instructions, strip out hidden formatting tricks, and never let extracted text directly override your workflow logic.