how it happened
- whoengineersused ChatGPT for work
- the gapconsumer AI serviceno agreement on data
- the resultcompany-wide bantemporary
what everyone assumed
A chatbot looks like a search box, and nobody ever leaked anything by searching. The mental model is question and answer, so what goes into the prompt feels as private as a thought. Under that model, pasting code into the box is just asking for help.
what happened
In the weeks after access opened, confidential material left through ordinary prompts: source code, internal notes, the daily substance of the work. Each paste was a working session; collectively they were transfers to an external service on consumer terms, with no agreement governing what happened next.
step by step
- engineers get access to a public chatbotOn consumer terms.
- source code and notes go into promptsEach paste is a transfer to an outside service.the assumption breaks here
- surfaced internally within weeks
- 2023AI tools restricted company-wide
how it surfaced
The incidents surfaced internally within weeks and publicly soon after. Samsung restricted AI tools company-wide, and the episode became the canonical example of the prompt as a data channel.
A prompt is an upload.
The interface hides it, but data pasted into a consumer AI tool has left the company as surely as an attachment mailed to a stranger.
picture it at a company like yours
The company here is made up. The other two paths are not: they happened, and each links to the story.
- in this storythe day it mattersConfidential material is now with a vendor on consumer terms.what it costs youYou ban AI tools company-wide in a hurry, and people use them on their phones instead.
- or it goes like thisit happenedDeepSeek exposed database, 2025howThe AI vendor leaves its own database open.what leaveseveryone's chats and the vendor's keyswho ends up with itanyone on the internet
- or it goes like thisit happenedChatGPT shared links, 2025howA shared chat link becomes public with one checkbox.what leavesclient names and personal detailswho ends up with itanyone using a search engine
sources
your turn
what you can do about it
this week
Check that a sanctioned AI tool on business terms is there before people paste.
People in your company pasted something into an AI tool today, because that is how the tools are used. The question is whether a sanctioned tool on enterprise terms was there to receive it, or whether the consumer tier got it instead. Which one happened was decided by what you provisioned, long before any policy text.
go deeper
if you're not sure where to start
Run the offboarding checklist on one leaver. It covers every account and token to close when someone leaves.
open the checklist →